awk Part III · Module 11 · Synthesis and Coverage Report Analysis
FSM Coverage
An FSM coverage report prints two percentages per state machine and they answer different questions: whether every state was ever entered, and whether every legal transition between states was ever taken. The first is much easier to satisfy than the second, and this chapter measures by how much — three machines in one report all reach 100% of their states while their transition coverage reads 64%, 36% and 100%. It also covers the field-counting trap this report shape introduces, where the obvious denominator is a word.
Advanced16 min readawkPart IIICoverageFSMFields
Part III · Module 11 · Chapter 11.6 · FSM Coverage
1. Why This Matters
A state machine has states and it has transitions between them, and a coverage report measures both separately because they are different populations:
- State coverage asks whether each state was ever entered. A machine with six states has six items to cover.
- Transition coverage asks whether each legal arc between states was ever taken. That same machine might have fourteen.
Reaching every state requires only that some path visits each one. Taking every arc requires exercising every choice the machine can make from every state it can make it in, which is a much larger job — and the two numbers printed side by side invite exactly the wrong summary.
2. The Report
========================================
FSM Coverage Report : alu_core
========================================
FSM: u_ctrl/main_fsm
State Coverage 100.00% (6 of 6 states)
Transition Coverage 64.29% (9 of 14 transitions)
FSM: u_dma/desc_fsm
State Coverage 100.00% (5 of 5 states)
Transition Coverage 36.36% (4 of 11 transitions)
FSM: u_io/pad_fsm
State Coverage 100.00% (4 of 4 states)
Transition Coverage 100.00% (7 of 7 transitions)
FSM: u_alu/seq_fsm
State Coverage 62.50% (5 of 8 states)
Transition Coverage 50.00% (11 of 22 transitions)
FSM: u_debug/dbg_fsm
State Coverage 0.00% (0 of 3 states)
Transition Coverage 0.00% (0 of 5 transitions)
========================================
TOTAL State 76.92% (20 of 26 states)
TOTAL Transition 52.54% (31 of 59 transitions)
========================================Twenty-six states and fifty-nine transitions across five machines — and the FSM name is on its own line, so Chapter 10.2's carried state is needed again.
3. $NF Is a Word
The parenthesised counts end with the item name, which changes where the denominator is:
awk '/^ State Coverage/ { printf " NF=%d $3=[%s] $4=[%s] $5=[%s] $6=[%s] $NF=[%s]\n", NF, $3, $4, $5, $6, $NF }' fsm.rpt | head -2 NF=7 $3=[100.00%] $4=[(6] $5=[of] $6=[6] $NF=[states)]
NF=7 $3=[100.00%] $4=[(5] $5=[of] $6=[5] $NF=[states)]$NF is states) — the item name, not a number. And the same holds on the TOTAL rows:
awk '/^TOTAL/ { printf " wrong ($NF): %-11s %8s %s of %s\n", $2, $3, substr($4,2), $NF }' fsm.rpt
awk '/^TOTAL/ { printf " right ($(NF-1)): %-11s %8s %s of %s\n", $2, $3, substr($4,2), $(NF-1) }' fsm.rpt wrong ($NF): State 76.92% 20 of states)
wrong ($NF): Transition 52.54% 31 of transitions)
right ($(NF-1)): State 76.92% 20 of 26
right ($(NF-1)): Transition 52.54% 31 of 5920 of states) is what $NF gives. Every line that carries a count has the same shape, which is worth confirming rather than assuming:
awk '/Coverage|^TOTAL/ && /of/ { printf " %-28s NF=%d $(NF-2)=[%s] $(NF-1)=[%s] $NF=[%s]\n", substr($0,1,26), NF, $(NF-2), $(NF-1), $NF }' fsm.rpt | head -4 State Coverage 10 NF=7 $(NF-2)=[of] $(NF-1)=[6] $NF=[states)]
Transition Coverage 6 NF=7 $(NF-2)=[of] $(NF-1)=[14] $NF=[transitions)]
State Coverage 10 NF=7 $(NF-2)=[of] $(NF-1)=[5] $NF=[states)]
Transition Coverage 3 NF=7 $(NF-2)=[of] $(NF-1)=[11] $NF=[transitions)]NF=7 on all of them, with of at $(NF-2), the denominator at $(NF-1) and the item word at $NF. So one rule covers the whole report:
awk '/of/ && /Coverage|^TOTAL/ { d = $(NF-1); printf " %-14s %8s %s of %s\n", ($1 == "TOTAL" ? "TOTAL " $2 : $1), $3, substr($4,2), d }' fsm.rpt | tail -4 State 0.00% 0 of 3
Transition 0.00% 0 of 5
TOTAL State 76.92% 20 of 26
TOTAL Transition 52.54% 31 of 59The reason to prefer $(NF-1) over $6 — which also works on every line here — is that it is anchored to the thing that is reliably last. Chapter 11.3 section 2 counted from NF because the left of the line was ragged; this report is ragged on the left too, with a two-word metric name on the indented rows and a three-word one on the totals.
And + 0 would have hidden the mistake rather than exposing it:
awk '/^ State Coverage/ { printf " $NF=[%s] $NF+0=%s -> a denominator of %s\n", $NF, $NF+0, $NF+0; exit }' fsm.rpt $NF=[states)] $NF+0=0 -> a denominator of 0A denominator of zero, which is then fatal for the reason Chapter 11.1 section 8 measured. The word in the output is the more useful failure — it names the field that was read, where + 0 converts a visible error into a crash several lines later.
4. Every State, a Third of the Arcs
awk '/^FSM:/ { f = $2 } /^ State Coverage/ { s = $3 + 0 } /^ Transition Coverage/ { if (s == 100) printf " %-18s states 100%% transitions %6.2f%%\n", f, $3 + 0; f = "" }' fsm.rpt u_ctrl/main_fsm states 100% transitions 64.29%
u_dma/desc_fsm states 100% transitions 36.36%
u_io/pad_fsm states 100% transitions 100.00%Three machines, all at 100% state coverage, with transition coverage of 64.29%, 36.36% and 100.00%. u_dma/desc_fsm visits all five of its states while taking four of its eleven arcs.
Which makes the relationship precise: state coverage at 100% tells you almost nothing about transition coverage. The three values above span nearly the whole range.
The reason is structural. Visiting every state needs one walk that touches each node; a machine with n states can be fully state-covered by a single path of n steps. Taking every arc needs every outgoing choice from every state, and a machine with n states may have far more than n arcs — this report has 26 states and 59 transitions, so arcs outnumber states better than two to one.
5. The Two Cannot Be Combined
Averaging the percentages:
awk '/^FSM:/ { f = $2 } /^ State Coverage/ { s = $3 + 0 } /^ Transition Coverage/ { printf " %-18s mean of the two percentages = %6.2f%%\n", f, (s + $3 + 0) / 2; f = "" }' fsm.rpt u_ctrl/main_fsm mean of the two percentages = 82.15%
u_dma/desc_fsm mean of the two percentages = 68.18%
u_io/pad_fsm mean of the two percentages = 100.00%
u_alu/seq_fsm mean of the two percentages = 56.25%
u_debug/dbg_fsm mean of the two percentages = 0.00%u_dma/desc_fsm is "68.18% covered" — a number corresponding to no population at all. It is neither 68.18% of its states nor 68.18% of its transitions, and there is nothing it is 68.18% of.
Aggregating the counts is worse, because it produces something that looks rigorous:
awk '/^ State Coverage/ { sh += substr($4,2)+0; st += $6+0 } /^ Transition Coverage/ { th += substr($4,2)+0; tt += $6+0 } END { printf " states %d of %d = %.2f%%\n transitions %d of %d = %.2f%%\n states+transitions as one ratio: %d of %d = %.2f%% <- MEANINGLESS\n", sh, st, 100*sh/st, th, tt, 100*th/tt, sh+th, st+tt, 100*(sh+th)/(st+tt) }' fsm.rpt states 20 of 26 = 76.92%
transitions 31 of 59 = 52.54%
states+transitions as one ratio: 51 of 85 = 60.00% <- MEANINGLESS51 of 85 has the form of a proper ratio with real counts behind it, and the eighty-five items are twenty-six of one kind and fifty-nine of another. It weights transitions more than twice as heavily as states for no stated reason beyond there happening to be more of them — the same defect as Chapter 11.4 Exercise 6's 487 of 655, which mixed lines and bits into an entirely plausible 74.35%.
This is the third time in Module 11 that a correct-looking ratio has been built from two incompatible populations, and it is why Chapter 11.8 exists.
6. Unreached States Against Unexercised Transitions
The counts, which are what a verification plan can act on:
awk '/^FSM:/ { f = $2 } /^ State Coverage/ { ms = $6 - substr($4,2) } /^ Transition Coverage/ { mt = $6 - substr($4,2); printf " %-18s %d unreached state(s), %d unexercised transition(s)\n", f, ms, mt; f = "" }' fsm.rpt u_ctrl/main_fsm 0 unreached state(s), 5 unexercised transition(s)
u_dma/desc_fsm 0 unreached state(s), 7 unexercised transition(s)
u_io/pad_fsm 0 unreached state(s), 0 unexercised transition(s)
u_alu/seq_fsm 3 unreached state(s), 11 unexercised transition(s)
u_debug/dbg_fsm 3 unreached state(s), 5 unexercised transition(s)Reading the five rows as work:
u_io/pad_fsm— nothing to do.u_ctrl/main_fsm,u_dma/desc_fsm— states all reached, 5 and 7 arcs to provoke. Stimulus work on an already-reachable machine.u_alu/seq_fsm— 3 states never entered and 11 arcs untaken. The unreached states are the thing to fix first, because arcs into and out of them cannot be covered until they are.u_debug/dbg_fsm— nothing ran at all.
Note the dependency in the fourth row, because it decides the order of work: an unreached state makes every arc touching it uncoverable. Three unreached states out of eight can easily account for most of the eleven missing arcs, so the 50.00% transition figure is partly a symptom rather than a separate problem. Chapter 11.5 section 4 made the same point about toggle coverage on a unit with low line coverage — a second metric is hard to interpret while the first is incomplete.
7. Common Mistakes
$NF for the denominator. It is the item word — states) — and the TOTAL rows expose it as 20 of states) — section 3.
+ 0 on that word. It gives 0, and a zero denominator is fatal.
Assuming a field position that held on the first line shape. The TOTAL rows have a different field count.
Quoting state coverage as "FSM coverage". Three machines at 100% state have transitions from 36.36% to 100% — section 4.
Averaging the two percentages. 68.18% of nothing — section 5.
Aggregating states and transitions into one ratio. 51 of 85 weights arcs over states for no reason.
Reading transition coverage on a machine with unreached states. Arcs touching them cannot be covered — section 6.
Not carrying the FSM name. It is on a separate line from both metrics.
8. Exercises
Exercise 1 — Both metrics per machine
awk '/^FSM:/ { f = $2 } /^ State Coverage/ { s = $3 + 0; sh = substr($4,2)+0; st = $6+0 } /^ Transition Coverage/ { printf "%-18s states %6.2f%% (%2d of %2d) transitions %6.2f%% (%2d of %2d)\n", f, s, sh, st, $3+0, substr($4,2)+0, $6+0; f = "" }' fsm.rptu_ctrl/main_fsm states 100.00% ( 6 of 6) transitions 64.29% ( 9 of 14)
u_dma/desc_fsm states 100.00% ( 5 of 5) transitions 36.36% ( 4 of 11)
u_io/pad_fsm states 100.00% ( 4 of 4) transitions 100.00% ( 7 of 7)
u_alu/seq_fsm states 62.50% ( 5 of 8) transitions 50.00% (11 of 22)
u_debug/dbg_fsm states 0.00% ( 0 of 3) transitions 0.00% ( 0 of 5)Exercise 2 — Machines fully state-covered but not fully transition-covered
awk '/^FSM:/ { f = $2 } /^ State Coverage/ { s = $3 + 0 } /^ Transition Coverage/ { if (s == 100 && $3 + 0 < 100) printf "%-18s all states reached, %d of %d arcs taken\n", f, substr($4,2)+0, $6+0; f = "" }' fsm.rptu_ctrl/main_fsm all states reached, 9 of 14 arcs taken
u_dma/desc_fsm all states reached, 4 of 11 arcs takenExercise 3 — Debug a denominator that is a word
A script reports 20 of states) for total state coverage.
awk '/^TOTAL State/ { printf "NF=%d ", NF; for (i = 1; i <= NF; i++) printf "[%d:%s]", i, $i; print "" }' fsm.rptNF=7 [1:TOTAL][2:State][3:76.92%][4:(20][5:of][6:26][7:states)]$NF is $7, the item word. The denominator is $6, which is $(NF-1).
awk '/^TOTAL/ { printf "%-11s $(NF-1)=[%s] $NF=[%s]\n", $2, $(NF-1), $NF }' fsm.rpt
awk '/^ State Coverage/ { printf "%-11s $(NF-1)=[%s] $NF=[%s]\n", "metric", $(NF-1), $NF; exit }' fsm.rptState $(NF-1)=[26] $NF=[states)]
Transition $(NF-1)=[59] $NF=[transitions)]
metric $(NF-1)=[6] $NF=[states)]The same layout on both line shapes, which is what makes $(NF-1) usable as a single rule. Worth confirming rather than assuming — a report whose item word were absent on some rows would need the number located by pattern instead:
awk '/of/ && match($0, /of +[0-9]+/) { d = substr($0, RSTART + 3, RLENGTH - 3); gsub(/ /, "", d); printf " %-12s denominator %s\n", $2, d }' fsm.rpt | head -3 Coverage denominator 6
Coverage denominator 14
Coverage denominator 5match() on of +[0-9]+ finds the number after the word of wherever it sits, which does not depend on the field count at all — the technique from Chapter 9.4, where a greedy field-position extraction broke on a record with a different shape.
Exercise 4 — Debug a combined FSM percentage
A dashboard reports FSM coverage at 60.00%.
awk '/^ State Coverage/ { sh += substr($4,2)+0; st += $6+0 } /^ Transition Coverage/ { th += substr($4,2)+0; tt += $6+0 } END { printf "combined: %d of %d = %.2f%%\n", sh+th, st+tt, 100*(sh+th)/(st+tt); printf "states: %d of %d = %.2f%%\ntransitions: %d of %d = %.2f%%\n", sh, st, 100*sh/st, th, tt, 100*th/tt }' fsm.rptcombined: 51 of 85 = 60.00%
states: 20 of 26 = 76.92%
transitions: 31 of 59 = 52.54%The 60.00% is 26 states and 59 transitions pooled into one denominator, so transitions carry 69% of the weight by accident of count. Neither 76.92% nor 52.54% is near it.
Exercise 5 — Engineering application
The FSM section of a coverage review: both metrics per machine, the dependency flagged, the report's own totals reproduced, and separate exit conditions for the two kinds of gap.
# fsms.awk — FSM state and transition coverage, kept separate.
#
# Usage: awk -f fsms.awk fsm.rpt
# Exit: 0 every machine fully covered on both metrics
# 1 arcs missing but every state reached
# 2 at least one state never reached (fix these first)
# 3 the per-machine counts do not reproduce the TOTAL rows
/^FSM:/ { f = $2; n++; order[n] = f }
/^ State Coverage/ { sh[f] = substr($4,2) + 0; st[f] = $6 + 0 }
/^ Transition Coverage/ { th[f] = substr($4,2) + 0; tt[f] = $6 + 0 }
/^TOTAL State/ { rsh = substr($4,2) + 0; rst = $6 + 0 }
/^TOTAL Transition/ { rth = substr($4,2) + 0; rtt = $6 + 0 }
END {
printf "%-18s %18s %22s %s\n", "fsm", "states", "transitions", "note"
for (i = 1; i <= n; i++) {
m = order[i]
SH += sh[m]; ST += st[m]; TH += th[m]; TT += tt[m]
ms = st[m] - sh[m]; mt = tt[m] - th[m]
note = "[no gap]"
if (ms > 0 && th[m] == 0) { note = "nothing ran"; nostim++ }
else if (ms > 0) { note = ms " state(s) unreached -- arcs blocked"; unreached++ }
else if (mt > 0) { note = mt " arc(s) to provoke"; arcsonly++ }
printf "%-18s %6.2f%% (%2d of %2d) %9.2f%% (%2d of %2d) %s\n", m,
100 * sh[m] / st[m], sh[m], st[m], 100 * th[m] / tt[m], th[m], tt[m], note
}
if (SH != rsh || ST != rst || TH != rth || TT != rtt) {
printf "ERROR: per-machine %d/%d states and %d/%d arcs, TOTAL says %d/%d and %d/%d\n",
SH, ST, TH, TT, rsh, rst, rth, rtt > "/dev/stderr"
exit 3
}
printf "#\n# states %d of %d = %.2f%%\n# transitions %d of %d = %.2f%%\n", SH, ST, 100*SH/ST, TH, TT, 100*TH/TT
printf "# the two are NOT combined: %d states and %d arcs are different populations\n", ST, TT
if (unreached + nostim) exit 2
if (arcsonly) exit 1
}awk -f fsms.awk fsm.rpt
echo "status: $?"fsm states transitions note
u_ctrl/main_fsm 100.00% ( 6 of 6) 64.29% ( 9 of 14) 5 arc(s) to provoke
u_dma/desc_fsm 100.00% ( 5 of 5) 36.36% ( 4 of 11) 7 arc(s) to provoke
u_io/pad_fsm 100.00% ( 4 of 4) 100.00% ( 7 of 7) [no gap]
u_alu/seq_fsm 62.50% ( 5 of 8) 50.00% (11 of 22) 3 state(s) unreached -- arcs blocked
u_debug/dbg_fsm 0.00% ( 0 of 3) 0.00% ( 0 of 5) nothing ran
#
# states 20 of 26 = 76.92%
# transitions 31 of 59 = 52.54%
# the two are NOT combined: 26 states and 59 arcs are different populations
status: 2The note column encodes the dependency from section 6 as three distinct cases, and the exit status is graded by which of them occurred: 2 means a state was never reached, which must be fixed before its arcs can be; 1 means every state is reachable and only arcs are missing, which is stimulus work; 3 means the per-machine counts do not reproduce the TOTAL rows, so the parse is not trustworthy.
Status 2 rather than 1 is the useful behaviour here: two machines need only arc stimulus, but u_alu/seq_fsm has unreached states, and that is the more serious finding of the two.
Exercise 6 — Challenge: show that one FSM number cannot be reconstructed
Given only the per-machine state percentages, show that transition coverage cannot be recovered — and that the converse also fails.
awk '/^ State Coverage/ { printf "[%s]", $3 } END { print "" }' fsm.rpt
awk '/^ Transition Coverage/ { printf "[%s]", $3 } END { print "" }' fsm.rpt[100.00%][100.00%][100.00%][62.50%][0.00%]
[64.29%][36.36%][100.00%][50.00%][0.00%]The first three state values are identical and their transition values are 64.29%, 36.36% and 100.00%. No function of the state percentage can produce three different outputs from the same input, so transition coverage is not recoverable from state coverage even in principle.
The converse is weaker but also fails:
awk '/^FSM:/ { f = $2 } /^ State Coverage/ { s = $3 + 0 } /^ Transition Coverage/ { t = $3 + 0; printf " %-18s transitions %6.2f%% -> states %6.2f%%\n", f, t, s; f = "" }' fsm.rpt | sort -k3,3n u_debug/dbg_fsm transitions 0.00% -> states 0.00%
u_dma/desc_fsm transitions 36.36% -> states 100.00%
u_alu/seq_fsm transitions 50.00% -> states 62.50%
u_ctrl/main_fsm transitions 64.29% -> states 100.00%
u_io/pad_fsm transitions 100.00% -> states 100.00%Transition coverage of 36.36% goes with 100% state coverage and 50.00% goes with 62.50% — the mapping is not even monotonic. A machine with better arc coverage has worse state coverage, which happens because u_alu/seq_fsm has three unreachable-in-practice states dragging its state figure down while its reachable part is reasonably exercised.
So the two metrics are not redundant in either direction, and a report that prints one is not a compressed version of a report that prints both. Both numbers have to be extracted, and this is the measurement that says so — the same argument Chapter 11.5 section 3 made for lines against bits, now with the stronger evidence that identical values of one metric coexist with a 64-point spread in the other.
9. Summary
- State coverage and transition coverage are different populations — 26 states and 59 transitions here.
$NFis the item word on a metric line, and theTOTALrows have a different field count.20 of states)is the symptom — section 3.- Verify field positions on every line shape the report contains, not the first.
- Three machines at 100% state coverage have transitions at 64.29%, 36.36% and 100.00% — section 4.
- 100% state coverage is nearly free; one walk touching every node achieves it. Every arc needs every decision from every state.
- The uncovered arcs are the interesting ones — aborts, timeouts, error recovery.
- Averaging the two gives 68.18% of nothing; pooling the counts gives
51 of 85, weighting arcs 69% by accident of count — section 5. - An unreached state blocks every arc touching it, so fix states before arcs — section 6.
- Neither metric predicts the other, and the mapping is not even monotonic — Exercise 6.
Chapter 11.7 — Functional Coverage moves to the metric the verification plan is actually written against, where the items are covergroups, coverpoints and crosses, the tool applies weights you did not choose, and the report is often a CSV rather than a formatted page.
Where this fits
Part of the awk curriculum.