awk Part III · Module 11 · Synthesis and Coverage Report Analysis
Functional Coverage
Functional coverage is the metric the verification plan is written against, and it is the only one in this module where the tool's own headline number is not a bin ratio. A covergroup's score is a weighted mean over its items, so a cross holding four fifths of the bins can carry a quarter of the score, and a coverpoint given a weight of zero vanishes from the figure while its uncovered bins remain. This chapter extracts the items from a CSV, computes both aggregations, and shows them differing by twenty-seven points on the same covergroup.
Advanced18 min readawkPart IIICoverageCovergroupsWeights
Part III · Module 11 · Chapter 11.7 · Functional Coverage
1. Why This Matters
Line, toggle and FSM coverage are all derived from the design automatically. Functional coverage is written by hand — someone declared a covergroup, chose its coverpoints, decided which crosses matter, and may have assigned weights. It is the metric that corresponds to a verification plan rather than to a netlist, which makes it the one quoted in sign-off meetings.
It is also the one where "what fraction is covered" has two defensible answers, because a covergroup is a collection of items with their own sizes, and the tool's reported number weights the items rather than the bins.
2. The Report Is Usually a CSV
Functional coverage is commonly exported rather than read as a formatted page, which makes -F, the whole parsing story:
covergroup,item,kind,bins_hit,bins_total,weight
cg_alu_ops,cp_opcode,coverpoint,14,16,1
cg_alu_ops,cp_operand_a,coverpoint,6,8,1
cg_alu_ops,cp_operand_b,coverpoint,5,8,1
cg_alu_ops,cr_opcode_x_opa,cross,31,128,1
cg_dma_xfer,cp_burst_len,coverpoint,4,4,1
cg_dma_xfer,cp_direction,coverpoint,2,2,1
cg_dma_xfer,cr_len_x_dir,cross,5,8,5
cg_io_cfg,cp_mode,coverpoint,3,4,1
cg_io_cfg,cp_speed,coverpoint,2,3,0awk -F, 'NR > 1 { printf " %-12s %-16s %-11s %4d of %4d = %6.2f%% weight %d\n", $1, $2, $3, $4, $5, 100*$4/$5, $6 }' funccov.csv cg_alu_ops cp_opcode coverpoint 14 of 16 = 87.50% weight 1
cg_alu_ops cp_operand_a coverpoint 6 of 8 = 75.00% weight 1
cg_alu_ops cp_operand_b coverpoint 5 of 8 = 62.50% weight 1
cg_alu_ops cr_opcode_x_opa cross 31 of 128 = 24.22% weight 1
cg_dma_xfer cp_burst_len coverpoint 4 of 4 = 100.00% weight 1
cg_dma_xfer cp_direction coverpoint 2 of 2 = 100.00% weight 1
cg_dma_xfer cr_len_x_dir cross 5 of 8 = 62.50% weight 5
cg_io_cfg cp_mode coverpoint 3 of 4 = 75.00% weight 1
cg_io_cfg cp_speed coverpoint 2 of 3 = 66.67% weight 0NR > 1 skips the header, and the numeric columns need no + 0 because a CSV field with no percent sign is a numeric-looking string — which is Chapter 10.5's rule holding where Chapter 11.4 section 4 found its exception. The weight column is the subject of section 5.
3. A Cross Holds Most of the Bins
A cross covers every combination of its coverpoints' bins, so its bin count is a product where a coverpoint's is a sum:
awk -F, 'NR > 1 { b[$1] += $5; n[$1]++; if ($3 == "cross") { x[$1] += $5; xn[$1]++ } if (!($1 in seen)) { seen[$1]=1; o[++k]=$1 } } END { for (i=1;i<=k;i++) { g=o[i]; printf " %-12s %d item(s), %d bins; %d cross(es) contribute %d bins = %.1f%%\n", g, n[g], b[g], xn[g]+0, x[g]+0, 100*(x[g]+0)/b[g] } }' funccov.csv cg_alu_ops 4 item(s), 160 bins; 1 cross(es) contribute 128 bins = 80.0%
cg_dma_xfer 3 item(s), 14 bins; 1 cross(es) contribute 8 bins = 57.1%
cg_io_cfg 2 item(s), 7 bins; 0 cross(es) contribute 0 bins = 0.0%One of cg_alu_ops's four items holds 80% of its bins. cr_opcode_x_opa crosses a 16-bin opcode with an 8-bin operand, giving 128 combinations — and it is the hardest item in the group to cover, at 24.22%, because filling it requires every opcode with every operand class.
That asymmetry is the whole reason the next two sections matter.
4. Two Aggregations, Twenty-Seven Points Apart
awk -F, 'NR > 1 { bh[$1] += $4; bt[$1] += $5; w[$1] += $6; ws[$1] += $6 * (100*$4/$5); if (!($1 in seen)) { seen[$1]=1; o[++k]=$1 } } END { for (i=1;i<=k;i++) { g=o[i]; printf " %-12s bins %3d of %3d = %6.2f%% weighted item avg = %6.2f%% diff %+6.2f\n", g, bh[g], bt[g], 100*bh[g]/bt[g], (w[g] ? ws[g]/w[g] : 0), (w[g] ? ws[g]/w[g] : 0) - 100*bh[g]/bt[g] } }' funccov.csv cg_alu_ops bins 56 of 160 = 35.00% weighted item avg = 62.30% diff +27.30
cg_dma_xfer bins 11 of 14 = 78.57% weighted item avg = 73.21% diff -5.36
cg_io_cfg bins 5 of 7 = 71.43% weighted item avg = 75.00% diff +3.57Three covergroups, three disagreements, and they go in both directions. cg_alu_ops is twenty-seven points higher by item average than by bin count; cg_dma_xfer is five points lower.
Decomposing the worst one:
awk -F, 'NR>1 && $1=="cg_alu_ops" { if ($3=="coverpoint") { ch+=$4; ct+=$5; cp+=100*$4/$5; cn++ } else { xh+=$4; xt+=$5; xp=100*$4/$5 } }
END { printf " %d coverpoints: %d of %d bins = %.2f%%, mean of their percentages = %.2f%%\n", cn, ch, ct, 100*ch/ct, cp/cn
printf " 1 cross: %d of %d bins = %.2f%%\n", xh, xt, xp
printf " by bin count: %d of %d = %.2f%%\n", ch+xh, ct+xt, 100*(ch+xh)/(ct+xt)
printf " by item mean: (%.2f*%d + %.2f)/%d = %.2f%%\n", cp/cn, cn, xp, cn+1, (cp+xp)/(cn+1)
printf " the cross holds %.1f%% of the bins and %.0f%% of the item weight\n", 100*xt/(ct+xt), 100.0/(cn+1) }' funccov.csv 3 coverpoints: 25 of 32 bins = 78.12%, mean of their percentages = 75.00%
1 cross: 31 of 128 bins = 24.22%
by bin count: 56 of 160 = 35.00%
by item mean: (75.00*3 + 24.22)/4 = 62.30%
the cross holds 80.0% of the bins and 25% of the item weight80% of the bins and 25% of the weight. The bin ratio is dragged down to 35.00% by the cross because the cross is most of the bins; the item mean holds up at 62.30% because the cross is one of four items.
5. The Weight Column
The second aggregation is not an alternative someone invented — it is how a SystemVerilog covergroup's coverage is defined. The covergroup's score is the weighted mean of its items' coverages, each item's own coverage being its bin ratio, with per-item weights taken from the weight option. That is a language convention rather than something measurable here, and the consequence is measurable:
awk -F, 'NR > 1 && $6 == 0 { printf " %-12s %-12s %d of %d = %.2f%% WEIGHT 0 -- excluded from the covergroup score\n", $1, $2, $4, $5, 100*$4/$5 }' funccov.csv
awk -F, 'NR > 1 && $1 == "cg_io_cfg" { w += $6; ws += $6 * (100*$4/$5); bh += $4; bt += $5 } END { printf " cg_io_cfg weighted avg over weight %d = %.2f%%, but its bins are %d of %d = %.2f%%\n", w, ws/w, bh, bt, 100*bh/bt }' funccov.csv cg_io_cfg cp_speed 2 of 3 = 66.67% WEIGHT 0 -- excluded from the covergroup score
cg_io_cfg weighted avg over weight 1 = 75.00%, but its bins are 5 of 7 = 71.43%cp_speed is 66.67% covered and contributes nothing to the score. cg_io_cfg reports 75.00%, which is cp_mode alone, while one of its two coverpoints has an uncovered bin that no reported figure reflects.
6. Common Mistakes
Computing a bin ratio and calling it the covergroup score. 35.00% against a reported 62.30% — section 4.
Dropping the weight column. A zero-weighted item disappears from the score — section 5.
Reporting a weighted score without the zero-weight count. 100% with exclusions is a different claim.
Treating a cross as one item like any other. It can hold 80% of the bins — section 3.
Comparing covergroups by bin count. cg_alu_ops has 160 bins and cg_io_cfg has 7; the sizes are a design choice, not a measure of thoroughness.
Forgetting NR > 1 on a CSV. The header row parses as data and 100*$4/$5 on it is a division by a non-numeric field.
Assuming -F, is sufficient. A quoted field containing a comma breaks it; check for quotes before trusting the split.
Averaging covergroup scores across groups. That is Chapter 11.8's subject and the same error one level up.
7. Exercises
Exercise 1 — Items per covergroup
awk -F, 'NR > 1 { n[$1]++; if (!($1 in seen)) { seen[$1]=1; o[++k]=$1 } } END { for (i=1;i<=k;i++) printf "%-12s %d item(s)\n", o[i], n[o[i]] }' funccov.csvcg_alu_ops 4 item(s)
cg_dma_xfer 3 item(s)
cg_io_cfg 2 item(s)Exercise 2 — Crosses only, with their share of the bins
awk -F, 'NR > 1 && $3 == "cross" { printf "%-12s %-16s %d of %d = %6.2f%% weight %d\n", $1, $2, $4, $5, 100*$4/$5, $6 }' funccov.csvcg_alu_ops cr_opcode_x_opa 31 of 128 = 24.22% weight 1
cg_dma_xfer cr_len_x_dir 5 of 8 = 62.50% weight 5Exercise 3 — Debug a covergroup score that does not match the bins
A dashboard reports cg_alu_ops at 35.00%; the simulator reports 62.30%.
awk -F, 'NR>1 && $1=="cg_alu_ops" { bh+=$4; bt+=$5; w+=$6; ws+=$6*(100*$4/$5) } END { printf "bin ratio %d of %d = %.2f%%\nweighted mean %.2f%% over weight %d\n", bh, bt, 100*bh/bt, ws/w, w }' funccov.csvbin ratio 56 of 160 = 35.00%
weighted mean 62.30% over weight 4Both are arithmetically correct and only one is the covergroup's coverage. The cause is the bin distribution:
awk -F, 'NR>1 && $1=="cg_alu_ops" { printf " %-16s %4d bins (%4.1f%% of the group), %6.2f%% covered, weight %d\n", $2, $5, 100*$5/160, 100*$4/$5, $6 }' funccov.csv cp_opcode 16 bins (10.0% of the group), 87.50% covered, weight 1
cp_operand_a 8 bins ( 5.0% of the group), 75.00% covered, weight 1
cp_operand_b 8 bins ( 5.0% of the group), 62.50% covered, weight 1
cr_opcode_x_opa 128 bins (80.0% of the group), 24.22% covered, weight 1The last row is the explanation. cr_opcode_x_opa is 80.0% of the group's bins at 24.22% covered, so it dominates the bin ratio — but it is one item of four, so it moves the weighted mean by only a quarter of its shortfall.
Exercise 4 — Debug a missing coverpoint
cg_io_cfg reports 75.00% and a reviewer cannot reconcile it with the item list.
awk -F, 'NR>1 && $1=="cg_io_cfg" { printf " %-12s %d of %d = %6.2f%% weight %d%s\n", $2, $4, $5, 100*$4/$5, $6, ($6 == 0 ? " <- excluded from the score" : "") }' funccov.csv
awk -F, 'NR>1 && $1=="cg_io_cfg" { w+=$6; ws+=$6*(100*$4/$5); z += ($6 == 0) } END { printf " score %.2f%% over weight %d, %d item(s) excluded\n", ws/w, w, z }' funccov.csv cp_mode 3 of 4 = 75.00% weight 1
cp_speed 2 of 3 = 66.67% weight 0 <- excluded from the score
score 75.00% over weight 1, 1 item(s) excludedThe 75.00% is cp_mode alone. cp_speed has an uncovered bin and a weight of zero, so no reported figure includes it.
Exercise 5 — Engineering application
The functional-coverage summary a sign-off review needs: the tool's weighted score, the bin ratio beside it, the zero-weighted items named, and crosses flagged by their share of the bins.
# funccov.awk — covergroup summary with both aggregations.
#
# Usage: awk -F, -f funccov.awk funccov.csv
# awk -F, -v gate=90 -f funccov.awk funccov.csv
# Exit: 0 every covergroup at or above the gate with nothing excluded
# 1 a covergroup is below the gate
# 2 an item has weight 0 (the score hides an uncovered bin)
BEGIN { if (gate == "") gate = 90 }
NR == 1 { next }
{
g = $1
if (!(g in seen)) { seen[g] = 1; order[++ng] = g }
n[g]++
bh[g] += $4; bt[g] += $5
w[g] += $6
ws[g] += $6 * (100 * $4 / $5)
if ($6 == 0) { zero[g] = zero[g] " " $2; nzero++ }
if ($3 == "cross") { xbins[g] += $5; nx[g]++ }
}
END {
printf "%-12s %5s %9s %9s %8s %s\n", "covergroup", "items", "score", "bin ratio", "cross%", "note"
for (i = 1; i <= ng; i++) {
g = order[i]
score = (w[g] ? ws[g] / w[g] : 0)
ratio = 100 * bh[g] / bt[g]
note = ""
if (zero[g] != "") note = "weight 0:" zero[g]
if (score < gate) { note = note (note == "" ? "" : "; ") "BELOW " gate "%"; nbelow++ }
if (note == "") note = "[ok]"
printf "%-12s %5d %8.2f%% %8.2f%% %7.1f%% %s\n", g, n[g], score, ratio,
100 * (xbins[g] + 0) / bt[g], note
}
printf "#\n# score = weighted mean of item coverages, which is what the tool reports\n"
printf "# bin ratio = hit bins over total bins, which it is not\n"
if (nzero) printf "# %d item(s) carry weight 0 and are absent from every score above\n", nzero
if (nzero) exit 2
if (nbelow) exit 1
}awk -F, -f funccov.awk funccov.csv
echo "status: $?"covergroup items score bin ratio cross% note
cg_alu_ops 4 62.30% 35.00% 80.0% BELOW 90%
cg_dma_xfer 3 73.21% 78.57% 57.1% BELOW 90%
cg_io_cfg 2 75.00% 71.43% 0.0% weight 0: cp_speed; BELOW 90%
#
# score = weighted mean of item coverages, which is what the tool reports
# bin ratio = hit bins over total bins, which it is not
# 1 item(s) carry weight 0 and are absent from every score above
status: 2Both aggregations in adjacent columns with a one-line note saying which is which, the cross share as its own column, and cp_speed named rather than silently dropped.
The exit status prefers 2 over 1: a covergroup below the gate is expected mid-project, whereas a zero-weighted item means the score itself is not measuring everything that was collected, and that is the condition a reviewer should see first.
Exercise 6 — Challenge: show that the score cannot be recovered from the bins
Given only bins_hit and bins_total per covergroup, show that the tool's score is not computable — and that the reverse also fails.
awk -F, 'NR>1 { bh[$1]+=$4; bt[$1]+=$5; w[$1]+=$6; ws[$1]+=$6*(100*$4/$5); if (!($1 in s)) { s[$1]=1; o[++k]=$1 } } END { for (i=1;i<=k;i++) { g=o[i]; printf " %-12s bins %3d of %3d -> ratio %6.2f%%, score %6.2f%%, score/ratio %.3f\n", g, bh[g], bt[g], 100*bh[g]/bt[g], ws[g]/w[g], (ws[g]/w[g])/(100*bh[g]/bt[g]) } }' funccov.csv cg_alu_ops bins 56 of 160 -> ratio 35.00%, score 62.30%, score/ratio 1.780
cg_dma_xfer bins 11 of 14 -> ratio 78.57%, score 73.21%, score/ratio 0.932
cg_io_cfg bins 5 of 7 -> ratio 71.43%, score 75.00%, score/ratio 1.050The ratio between the two figures is 1.780, 0.932 and 1.050 — no constant, and not even consistently on one side of 1. So there is no correction factor, and the score is not a function of the group's bin totals.
It cannot be, because the score depends on how the bins are partitioned into items and on the weights, and the group totals carry neither. Two covergroups with identical 56 of 160 bins can score anything from near 0% to near 100% depending on whether the uncovered bins sit in one heavily-weighted item or are spread across many lightly-weighted ones.
The reverse direction fails too:
awk -F, 'NR>1 { w[$1]+=$6; ws[$1]+=$6*(100*$4/$5); bt[$1]+=$5; if (!($1 in s)) { s[$1]=1; o[++k]=$1 } } END { for (i=1;i<=k;i++) { g=o[i]; printf " %-12s score %6.2f%% with %3d total bins\n", g, ws[g]/w[g], bt[g] } }' funccov.csv cg_alu_ops score 62.30% with 160 total bins
cg_dma_xfer score 73.21% with 14 total bins
cg_io_cfg score 75.00% with 7 total binsA score of 62.30% over 160 bins and 75.00% over 7 bins say nothing comparable about how much verification work remains — the first has 104 uncovered bins and the second has 2.
So the item-level rows are the only sufficient data, and a parser that reduces them to per-group figures has thrown away the ability to answer either question. Keep the rows; aggregate at the point of reporting, and say which aggregation you used. That is the conclusion Chapter 11.4 reached for line coverage, and the next chapter makes it the rule.
8. Summary
- Functional coverage is hand-written, so its items and weights are a verification-plan choice, not a property of the netlist.
- A covergroup's reported score is the weighted mean of its items' coverages, not its bin ratio — a language convention.
- The two disagree in both directions: +27.30, −5.36 and +3.57 across three groups — section 4.
- A cross's bin count is a product. One cross held 80.0% of a group's 160 bins and 25% of its score.
- A weight of 0 excludes an item from every reported score while its uncovered bins remain —
cp_speedat 66.67% inside a group reporting 75.00%. - Extract hit, total, weight and the item percentage. Report the score, the bin ratio and the count of zero-weighted items.
NR > 1on a CSV, and check for quoted commas before trusting-F,.- Group bin totals are not comparable — 160 bins against 7 is a design choice.
- The score is not a function of the bin totals. The ratio between them was 1.780, 0.932 and 0.933 — no correction factor exists.
- Keep the item rows. Aggregate at reporting time and name the aggregation.
Chapter 11.8 — Aggregating Coverage Metrics Correctly closes the module on the question every chapter here has deferred: given per-unit and per-item numbers across several metrics, what single figure may honestly be quoted, and what must be said alongside it.
Where this fits
Part of the awk curriculum.