awk Part III · Module 11 · Synthesis and Coverage Report Analysis
Toggle Coverage
Line coverage says a statement executed. Toggle coverage says a bit took both values — and the gap between them is one of the most useful diagnostics a coverage report offers, because it separates code that was never reached from code that ran on data that never varied. This chapter extracts toggle coverage per block, measures that gap across a design, and then goes one level down into a single unit to show what a block-level percentage conceals: at 61.90% this one has three signals that never changed at all, and its own output has twenty-four of thirty-two bits stuck.
Intermediate17 min readawkPart IIICoverageToggleDiagnostics
Part III · Module 11 · Chapter 11.5 · Toggle Coverage
1. Why This Matters
Toggle coverage asks, of every bit in the design, whether it was ever observed at 0 and ever observed at 1. That is a question about data, where line coverage asks a question about control flow, and the two fail for different reasons:
- A line at 0% was never reached. Some branch, mode or configuration was not exercised.
- A bit at 0% was reached and never changed. The code ran; the value was constant.
The second is the harder problem to find by reading a test list, and it is the one that hides behind a respectable-looking block percentage.
2. Toggle Coverage per Unit
From cov.rpt, listed in full in Chapter 11.4 section 2:
awk '/^Design Unit:/ { u = $3 } /^ Toggle Coverage/ { printf " %-10s %7.2f%% %s of %s bits\n", u, $3 + 0, substr($4,2), $6; u = "" }' cov.rpt alu_core 74.19% 46 of 62 bits
u_regfile 98.44% 126 of 128 bits
u_alu 61.90% 78 of 126 bits
u_ctrl 45.31% 29 of 64 bits
u_io 100.00% 16 of 16 bits
u_debug 0.00% 0 of 24 bitsSame shape as Chapter 11.4, same + 0 on the percentage for the reason section 4 of that chapter measured, same substr($4, 2) to drop the parenthesis.
3. Bits and Lines Are Not the Same Unit
awk '/^Design Unit: u_alu/ { inu=1 } inu && /^ Line Coverage/ { printf " u_alu lines: %s of %s\n", substr($4,2), $6 } inu && /^ Toggle Coverage/ { printf " u_alu bits: %s of %s\n", substr($4,2), $6; inu=0 }' cov.rpt u_alu lines: 75 of 85
u_alu bits: 78 of 126Eighty-five lines and a hundred and twenty-six bits. They are different populations of different things, and the consequences are worth stating plainly because they are all violations people commit:
- They cannot be summed. Chapter 11.4's Exercise 6 added them and got a thoroughly plausible 74.35% that measures nothing.
- They cannot be averaged together into "overall coverage" without a weighting decision, which is Chapter 11.8's subject.
- Their magnitudes are not comparable. 88.24% of lines and 61.90% of bits are not two measurements of the same thing at different levels.
What they can do is be compared as a pair, which is the next section.
4. The Gap Is the Diagnostic
awk '/^Design Unit:/ { u = $3 } /^ Line Coverage/ { l[u] = $3 + 0 } /^ Toggle Coverage/ { t[u] = $3 + 0; if (!(u in seen)) { seen[u]=1; o[++n]=u } }
END { printf " %-10s %8s %8s %8s\n", "unit", "line", "toggle", "gap"
for (i=1;i<=n;i++) { k=o[i]; printf " %-10s %7.2f%% %7.2f%% %+7.2f\n", k, l[k], t[k], t[k]-l[k] } }' cov.rpt unit line toggle gap
alu_core 96.55% 74.19% -22.36
u_regfile 100.00% 98.44% -1.56
u_alu 88.24% 61.90% -26.34
u_ctrl 72.00% 45.31% -26.69
u_io 100.00% 100.00% +0.00
u_debug 0.00% 0.00% +0.00Toggle sits at or below line coverage in every unit. That is the normal relationship and the size of the gap is what carries information:
u_regfile, −1.56. Lines all ran and bits nearly all moved. A register file written and read with varied data.u_alu, −26.34 andu_ctrl, −26.69. Most lines ran; well over a third of the bits never changed. The tests reach the code and then feed it a narrow set of values.u_io, +0.00 at 100% of both. Fully exercised.u_debug, +0.00 at 0% of both. Never reached at all, which is a different problem — there is no stimulus, not narrow stimulus.
The last two rows share a gap of zero and mean opposite things, which is why the gap is read alongside the levels and not instead of them.
5. The Block Percentage Hides Stuck Signals
A per-signal breakdown of the worst-gap unit:
Toggle Coverage Detail : u_alu
Signal Bits Toggled Percent
------------------------------------------------------
u_alu/operand_a 32 32 100.00%
u_alu/operand_b 32 30 93.75%
u_alu/result 32 8 25.00%
u_alu/opcode 5 4 80.00%
u_alu/flags 4 2 50.00%
u_alu/carry_chain 8 0 0.00%
u_alu/enable 1 1 100.00%
u_alu/mode_sel 2 0 0.00%
u_alu/overflow 1 1 100.00%
u_alu/spare 9 0 0.00%
------------------------------------------------------
TOTAL 126 78 61.90%The detail must agree with the summary, and checking that is the first thing to do with any drill-down:
awk '/^Design Unit: u_alu/ { inu = 1 } inu && /^ Toggle Coverage/ { printf " cov.rpt says u_alu toggle = %s (%s of %s bits)\n", $3, substr($4,2), $6; inu = 0 }' cov.rpt
awk 'NR>3 && !/^-|^TOTAL/ { b += $2; t += $3 } END { printf " toggle.rpt detail sums to %d of %d bits = %.2f%%\n", t, b, 100*t/b }' toggle.rpt cov.rpt says u_alu toggle = 61.90% (78 of 126 bits)
toggle.rpt detail sums to 78 of 126 bits = 61.90%Now what the 61.90% was covering for:
awk 'NR>3 && !/^-|^TOTAL/ && $3 == 0 { printf " %-22s %2d bit(s), 0 toggled\n", $1, $2; n++; bits += $2 } END { printf " %d signal(s), %d bits never toggled\n", n, bits }' toggle.rpt u_alu/carry_chain 8 bit(s), 0 toggled
u_alu/mode_sel 2 bit(s), 0 toggled
u_alu/spare 9 bit(s), 0 toggled
3 signal(s), 19 bits never toggledThree signals that never changed state at all, nineteen bits between them:
awk 'NR>3 && !/^-|^TOTAL/ { if ($3 == 0) { z += $2; zn++ }; b += $2; t += $3 }
END { printf " unit reads %.2f%% (%d of %d bits)\n", 100*t/b, t, b
printf " but %d signal(s) / %d bits (%.1f%% of the unit) never toggled at all\n", zn, z, 100*z/b
printf " and removing them would read %.2f%% on the remaining %d bits\n", 100*t/(b-z), b-z }' toggle.rpt unit reads 61.90% (78 of 126 bits)
but 3 signal(s) / 19 bits (15.1% of the unit) never toggled at all
and removing them would read 72.90% on the remaining 107 bitsA $3 == 0 test is a one-line scan with a specific meaning: a signal that never toggled is almost always either tied off, unreachable, or a leftover. u_alu/spare at nine bits is the shape of a signal that was never connected to anything.
6. Percentage and Bit Count Rank Differently
printf " by percent: "; awk 'NR>3 && !/^-|^TOTAL/ { printf "%s\t%.2f\t%d\n", $1, $4+0, $2-$3 }' toggle.rpt | sort -t"$(printf '\t')" -k2,2n -k1,1 | head -4 | awk -F'\t' '{ printf "[%s %.0f%%]", $1, $2 } END { print "" }'
printf " by untoggled: "; awk 'NR>3 && !/^-|^TOTAL/ { printf "%s\t%.2f\t%d\n", $1, $4+0, $2-$3 }' toggle.rpt | sort -t"$(printf '\t')" -k3,3rn -k1,1 | head -4 | awk -F'\t' '{ printf "[%s %d bits]", $1, $3 } END { print "" }' by percent: [u_alu/carry_chain 0%][u_alu/mode_sel 0%][u_alu/spare 0%][u_alu/result 25%]
by untoggled: [u_alu/result 24 bits][u_alu/spare 9 bits][u_alu/carry_chain 8 bits][u_alu/flags 2 bits]The two lists share only their membership, not their order, and the disagreement is the whole point:
u_alu/resultis first by bit count and fourth by percentage. It is the unit's output, 32 bits wide, with 24 of them never observed changing. Whatever this ALU computes, three quarters of its result bus has been constant for the entire regression.u_alu/mode_selis joint-first by percentage and does not make the top four by bit count. It is 2 bits at 0% — real, and two bits of work.
Both rankings are defensible and they prioritise differently, which is Chapter 11.2 section 7's lesson in a second setting: rank by the quantity you will act on. For "which signal is most likely to be hiding a bug", result at 24 stuck bits is the answer. For "which signals are completely unexercised", the 0% list is.
The -k2,2n -k1,1 and -k3,3rn -k1,1 key specifications are the tie discipline from Module 9 — carry_chain, mode_sel and spare are all at exactly 0.00%, so without the name as a tiebreaker their order is undefined.
7. Common Mistakes
Comparing a toggle percentage against a line percentage as magnitudes. Different populations — section 3.
Summing bits and lines. Chapter 11.4 Exercise 6's 74.35% measures nothing.
Reading the gap without the levels. u_io and u_debug both have a gap of zero and mean opposite things — section 4.
Interpreting a toggle gap on a unit with low line coverage. Most of it is unreached code.
Trusting a block percentage. u_alu at 61.90% has three signals at 0% — section 5.
Not cross-checking a drill-down against its summary. The detail must sum to the block figure.
Ranking signals only by percentage. The 24-bit stuck output lands fourth — section 6.
Sorting tied percentages without a tiebreaker. Three signals sit at exactly 0.00%.
Treating a 0% signal as a coverage problem. It is usually tied off, unreachable or a leftover — a design question first.
8. Exercises
Exercise 1 — Toggle coverage per unit
awk '/^Design Unit:/ { u = $3 } /^ Toggle Coverage/ { printf "%-10s %7.2f%% %3s of %3s bits\n", u, $3 + 0, substr($4,2), $6; u = "" }' cov.rptalu_core 74.19% 46 of 62 bits
u_regfile 98.44% 126 of 128 bits
u_alu 61.90% 78 of 126 bits
u_ctrl 45.31% 29 of 64 bits
u_io 100.00% 16 of 16 bits
u_debug 0.00% 0 of 24 bitsExercise 2 — Signals that never toggled
awk 'NR>3 && !/^-|^TOTAL/ && $3 + 0 == 0 { printf "%-22s %2d bit(s)\n", $1, $2 }' toggle.rptu_alu/carry_chain 8 bit(s)
u_alu/mode_sel 2 bit(s)
u_alu/spare 9 bit(s)Exercise 3 — Debug a block figure that looks acceptable
A unit reports 61.90% toggle coverage and a reviewer asks whether any signal is completely stuck.
awk 'NR>3 && !/^-|^TOTAL/ { n++; if ($3 + 0 == 0) z++ } END { printf "%d signal(s), %d entirely untoggled\n", n, z }' toggle.rpt
awk 'NR>3 && !/^-|^TOTAL/ && $3 + 0 == 0 { b += $2 } END { printf "%d bits in those signals\n", b }' toggle.rpt10 signal(s), 3 entirely untoggled
19 bits in those signalsThree of ten signals contribute nothing at all. The block percentage averages them against operand_a at 100%, which is what makes 61.90% look like a uniform shortfall rather than a mix of fully covered and fully dead.
Exercise 4 — Debug a priority list that misses the output bus
A triage list ranks signals by percentage and u_alu/result is fourth.
awk 'NR>3 && !/^-|^TOTAL/ { printf "%-22s %6.2f%% %2d untoggled of %2d\n", $1, $4+0, $2-$3, $2 }' toggle.rpt | sort -k2,2n | head -4u_alu/carry_chain 0.00% 8 untoggled of 8
u_alu/mode_sel 0.00% 2 untoggled of 2
u_alu/spare 0.00% 9 untoggled of 9
u_alu/result 25.00% 24 untoggled of 32result has more untoggled bits than the three 0% signals combined have bits, and ranks below all of them. Ranked by the count:
awk 'NR>3 && !/^-|^TOTAL/ { printf "%-22s %6.2f%% %2d untoggled of %2d\n", $1, $4+0, $2-$3, $2 }' toggle.rpt | sort -k3,3rn -k1,1 | head -3u_alu/result 25.00% 24 untoggled of 32
u_alu/spare 0.00% 9 untoggled of 9
u_alu/carry_chain 0.00% 8 untoggled of 8Exercise 5 — Engineering application
The toggle triage a verification review wants: the block figure cross-checked against the detail, stuck signals separated from partially-covered ones, and both rankings available.
# toggles.awk — toggle triage from a per-signal detail report.
#
# Usage: awk -f toggles.awk toggle.rpt
# awk -v expect=61.90 -f toggles.awk toggle.rpt
# Exit: 0 nothing entirely stuck
# 1 at least one signal never toggled
# 2 the detail does not match the block figure given in -v expect
BEGIN { TOL = 0.005 }
/^Signal|^-|^Toggle Coverage Detail/ { next }
$1 == "TOTAL" { rtot_b = $2; rtot_t = $3; next }
NF >= 4 {
n++
bits[$1] = $2 + 0
tog[$1] = $3 + 0
order[n] = $1
sumb += $2 + 0
sumt += $3 + 0
if ($3 + 0 == 0) { stuck[$1] = 1; nstuck++; stuckbits += $2 + 0 }
}
END {
if (sumb == 0) { print "ERROR: no signals found" > "/dev/stderr"; exit 2 }
pct = 100 * sumt / sumb
# the detail must reproduce the report's own TOTAL row
if (rtot_b != "" && (rtot_b != sumb || rtot_t != sumt)) {
printf "ERROR: detail sums to %d of %d but the TOTAL row says %d of %d\n", sumt, sumb, rtot_t, rtot_b > "/dev/stderr"
exit 2
}
# and, when given, the block figure from the summary report
if (expect != "" && !(pct - expect < TOL + 0.005 && expect - pct < TOL + 0.005)) {
printf "ERROR: detail gives %.2f%% but the block figure is %.2f%%\n", pct, expect > "/dev/stderr"
exit 2
}
printf "%-22s %5s %5s %8s %6s\n", "signal", "bits", "tog", "percent", "gap"
for (i = 1; i <= n; i++) {
s = order[i]
printf "%-22s %5d %5d %7.2f%% %6d%s\n", s, bits[s], tog[s], 100 * tog[s] / bits[s], bits[s] - tog[s],
(s in stuck ? " STUCK" : "")
}
printf "#\n# %d signal(s), %d of %d bits = %.2f%%\n", n, sumt, sumb, pct
printf "# %d signal(s) never toggled, %d bits (%.1f%% of the unit)\n", nstuck + 0, stuckbits + 0, 100 * stuckbits / sumb
if (stuckbits) printf "# excluding them, the rest reads %.2f%% on %d bits\n", 100 * sumt / (sumb - stuckbits), sumb - stuckbits
if (nstuck) exit 1
}awk -v expect=61.90 -f toggles.awk toggle.rpt
echo "status: $?"signal bits tog percent gap
u_alu/operand_a 32 32 100.00% 0
u_alu/operand_b 32 30 93.75% 2
u_alu/result 32 8 25.00% 24
u_alu/opcode 5 4 80.00% 1
u_alu/flags 4 2 50.00% 2
u_alu/carry_chain 8 0 0.00% 8 STUCK
u_alu/enable 1 1 100.00% 0
u_alu/mode_sel 2 0 0.00% 2 STUCK
u_alu/overflow 1 1 100.00% 0
u_alu/spare 9 0 0.00% 9 STUCK
#
# 10 signal(s), 78 of 126 bits = 61.90%
# 3 signal(s) never toggled, 19 bits (15.1% of the unit)
# excluding them, the rest reads 72.90% on 107 bits
status: 1The gap column is the untoggled bit count, so the table can be read for either priority without re-sorting, and the three STUCK markers say which rows are a design question rather than a stimulus question.
Two independent cross-checks run before anything is printed: the detail must reproduce its own TOTAL row, and — when -v expect is given from the summary report — it must reproduce the block figure too. Passing a wrong one is refused:
awk -v expect=70.00 -f toggles.awk toggle.rpt 2>&1 >/dev/null
echo "status: $?"ERROR: detail gives 61.90% but the block figure is 70.00%
status: 2That is the guard proven able to fire, which Chapter 11.2 Exercise 6 argued is the only way to know a guard works.
Exercise 6 — Challenge: two distributions, one headline
u_alu reads 61.90%. Build a second signal list with the same bits, the same toggled count and therefore the same block figure, but no stuck signal at all — then show which statistics distinguish them.
Toggle Coverage Detail : u_flat
Signal Bits Toggled Percent
------------------------------------------------------
u_flat/sig1 12 7 58.33%
u_flat/sig2 12 7 58.33%
u_flat/sig3 12 7 58.33%
u_flat/sig4 12 7 58.33%
u_flat/sig5 12 7 58.33%
u_flat/sig6 12 7 58.33%
u_flat/sig7 12 7 58.33%
u_flat/sig8 12 7 58.33%
u_flat/sig9 12 8 66.67%
u_flat/sig10 12 8 66.67%
u_flat/sig11 6 6 100.00%
------------------------------------------------------
TOTAL 126 78 61.90%Selecting rows by shape rather than by line number, since the two files have different headers:
SEL='NF >= 4 && $1 !~ /^(Signal|TOTAL|-)/ && $2 + 0 > 0'
for f in toggle.rpt flat.rpt; do
printf " %-12s " "$f"
awk "$SEL { n++; b += \$2; t += \$3; if (\$3 + 0 == 0) z++; p = 100*\$3/\$2; if (n == 1 || p < lo) lo = p } END { printf \"%d signal(s), %d of %d bits = %.2f%%, %d stuck, lowest signal %.2f%%\n\", n, t, b, 100*t/b, z+0, lo }" $f
done toggle.rpt 10 signal(s), 78 of 126 bits = 61.90%, 3 stuck, lowest signal 0.00%
flat.rpt 11 signal(s), 78 of 126 bits = 61.90%, 0 stuck, lowest signal 58.33%Byte-identical on bits, toggled bits and percentage. 78 of 126 = 61.90% for both, and the two reports describe completely different situations:
toggle.rpthas three signals that never moved and a 32-bit output at 25%. The work is to find out why three signals are dead, and that is a design question.flat.rpthas nothing below 58.33% and nothing stuck. The work is to broaden the stimulus a little across the board, and that is a testbench question.
The two statistics that separate them are the stuck count and the minimum per-signal percentage, and neither is derivable from the block figure — which is the general claim: a ratio over a population says nothing about the distribution within it.
Note the row selector. NR > 3 would have been wrong here, because the two files do not have the same number of header lines, and a selector keyed to line numbers silently includes a header row or drops a data row when the format shifts. Selecting on NF >= 4 plus a negative name match plus $2 + 0 > 0 keys on what a data row is, which survives both files — the same argument as Chapter 10.6 section 8, where a check had to count markers rather than trust positions.
9. Summary
- Line coverage is control flow; toggle coverage is data. A line at 0% was never reached; a bit at 0% never changed.
- Bits and lines are different populations — 85 lines against 126 bits in one unit. Never summed, never compared as magnitudes.
- Toggle sits at or below line coverage, and the gap is the diagnostic: −1.56 on a well-exercised unit, −26.69 on one whose stimulus is narrow.
- Read the gap with the levels. A gap of zero at 100% and at 0% mean opposite things.
- A gap only means something where line coverage is already high.
- A block percentage hides stuck signals.
u_aluat 61.90% has three signals at 0% — 19 bits, 15.1% of the unit. - Cross-check a drill-down against its summary, and against the detail's own
TOTALrow. - The two rankings disagree. The 24-bit stuck output ranks fourth by percentage; a 2-bit signal ranks first.
- A 0% signal is a design question first — tied off, unreachable, or a leftover.
- When a derived count disagrees with one you already verified, print the fields. Exercise 6's row selection, not its arithmetic, was the fault.
Chapter 11.6 — FSM Coverage takes the metric that is really two metrics: state coverage and transition coverage are different denominators printed side by side, and a state machine can reach every state while exercising barely half its arcs.
Where this fits
Part of the awk curriculum.