awk Part III · Module 11 · Synthesis and Coverage Report Analysis

Line Coverage

Area is a measurement. A coverage percentage is a ratio, and that changes what you can do with it: two ratios cannot be averaged, a ratio with a zero denominator is not zero, and a ratio printed with a percent sign is not a number as far as awk is concerned. This chapter extracts line coverage per design unit and then works through all three, ending on the comparison that matters most in practice — a unit at 100.00% that fails a 90% coverage gate, and the one character that causes it.

Intermediate17 min readawkPart IIICoveragePercentagesComparison

Part III · Module 11 · Chapter 11.4 · Line Coverage

1. Why This Matters

Coverage closure is a gate. Somewhere in every verification flow there is a script deciding whether a number clears a threshold, and the number comes out of a report like the one below.

The three chapters before this one parsed synthesis reports, where every value was a measured quantity on its own line. Coverage values are ratios, and three properties of ratios break the habits that worked on areas: they cannot be meaningfully averaged, a zero denominator does not mean zero, and the convention of printing them with a % makes them something awk will not compare as numbers.

2. The Report

Azvya Education Pvt. Ltd.VLSI Mentor

cov.rpt — synthetic teaching data. No coverage tool runs in this environment, so this report was generated such that every percentage equals its own counts and the TOTAL rows are the exact aggregates; section 7 verifies both. Chapters 11.5 and 11.6 build on it.

========================================
Coverage Report : alu_core
Tests merged    : 148
========================================

Design Unit: alu_core
  Line Coverage         96.55%      (28 of 29 lines)
  Toggle Coverage       74.19%      (46 of 62 bits)
  Branch Coverage       91.67%      (11 of 12 branches)

Design Unit: u_regfile
  Line Coverage        100.00%      (41 of 41 lines)
  Toggle Coverage       98.44%      (126 of 128 bits)
  Branch Coverage      100.00%      (6 of 6 branches)

Design Unit: u_alu
  Line Coverage         88.24%      (75 of 85 lines)
  Toggle Coverage       61.90%      (78 of 126 bits)
  Branch Coverage       76.47%      (13 of 17 branches)

Design Unit: u_ctrl
  Line Coverage         72.00%      (36 of 50 lines)
  Toggle Coverage       45.31%      (29 of 64 bits)
  Branch Coverage       64.29%      (9 of 14 branches)

Design Unit: u_io
  Line Coverage        100.00%      (12 of 12 lines)
  Toggle Coverage      100.00%      (16 of 16 bits)
  Branch Coverage            n/a      (0 of 0 branches)

Design Unit: u_debug
  Line Coverage          0.00%      (0 of 18 lines)
  Toggle Coverage        0.00%      (0 of 24 bits)
  Branch Coverage        0.00%      (0 of 4 branches)

========================================
TOTAL  Line   Coverage    81.70%     (192 of 235)
TOTAL  Toggle Coverage    70.24%     (295 of 420)
TOTAL  Branch Coverage    73.58%     (39 of 53)
========================================

Three things to notice before parsing anything: the unit name is on a separate line from its metrics, every percentage is followed by the counts that produced it, and u_io has n/a for branch coverage because it has (0 of 0 branches).

3. Carrying the Unit Name

The metric lines do not say which unit they belong to, so this is Chapter 10.2's pattern again:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '/^Design Unit:/ { u = $3 } /^  Line Coverage/ { printf "  %-10s %8s  %s of %s\n", u, $3, substr($4,2), $6; u = "" }' cov.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  alu_core     96.55%  28 of 29
  u_regfile   100.00%  41 of 41
  u_alu        88.24%  75 of 85
  u_ctrl       72.00%  36 of 50
  u_io        100.00%  12 of 12
  u_debug       0.00%  0 of 18

substr($4, 2) drops the opening parenthesis from (28. The u = "" after consuming is the clear-on-use rule from Chapter 10.2 section 7 — a metric line with no preceding Design Unit: would otherwise inherit the previous unit's name.

Patterns need anchoring here as much as in Chapter 11.1:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

for pat in '/Line Coverage/' '/^  Line Coverage/' '/Coverage/'; do
  printf "  %-22s " "$pat"; awk "$pat { n++ } END { print n+0 \" line(s)\" }" cov.rpt
done

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  /Line Coverage/        6 line(s)
  /^  Line Coverage/     6 line(s)
  /Coverage/             22 line(s)

/Coverage/ sweeps up all eighteen metric lines, the three totals and the report title. The two Line Coverage forms agree at six here — the TOTAL rows read TOTAL Line Coverage with the words separated, so they do not match. That is a property of this report's column alignment and not something to rely on; anchor anyway.

4. A Unit at 100% Fails a 90% Gate

This is the chapter's central result. A small file makes it unambiguous:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

printf 'u_alu 100.00%%\nu_ctrl 87.50%%\nu_io 9.00%%\nu_dma 100.00\n' > pf.txt
awk '{ printf "  %-8s field [%-8s]  $2 >= 90 -> %-5s   $2 + 0 >= 90 -> %-5s   $2+0 = %s\n", $1, $2, ($2 >= 90 ? "TRUE" : "false"), ($2 + 0 >= 90 ? "TRUE" : "false"), $2 + 0 }' pf.txt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  u_alu    field [100.00% ]  $2 >= 90 -> false   $2 + 0 >= 90 -> TRUE    $2+0 = 100
  u_ctrl   field [87.50%  ]  $2 >= 90 -> false   $2 + 0 >= 90 -> false   $2+0 = 87.5
  u_io     field [9.00%   ]  $2 >= 90 -> false   $2 + 0 >= 90 -> false   $2+0 = 9
  u_dma    field [100.00  ]  $2 >= 90 -> TRUE    $2 + 0 >= 90 -> TRUE    $2+0 = 100

u_alu is at 100.00% and $2 >= 90 is false. u_dma is also at 100 and passes. The only difference between the two rows is that one value carries a percent sign.

As gate lists:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

printf "  passes 90%%  without +0: "; awk '$2 >= 90 { printf "[%s]", $1 } END { print "" }' pf.txt
printf "  passes 90%%  with    +0: "; awk '$2 + 0 >= 90 { printf "[%s]", $1 } END { print "" }' pf.txt
printf "  FAILS  90%%  without +0: "; awk '$2 < 90 { printf "[%s]", $1 } END { print "" }' pf.txt
printf "  FAILS  90%%  with    +0: "; awk '$2 + 0 < 90 { printf "[%s]", $1 } END { print "" }' pf.txt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  passes 90%  without +0: [u_dma]
  passes 90%  with    +0: [u_alu][u_dma]
  FAILS  90%  without +0: [u_alu][u_ctrl][u_io]
  FAILS  90%  with    +0: [u_ctrl][u_io]

u_alu appears on the failure list at 100% coverage.

Chapter 10.5's rule has an exception, and this is it

Chapter 10.5 section 6 established the comparison rule and it is worth restating precisely, because this chapter is where it bites differently.

That rule said: two values that both came from input and both look numeric compare numerically, and only a string constant forces a lexical comparison. On that basis $3 < w was safe in Chapter 10.5 and the hazard was writing $3 < "-0.050".

The clause that matters here is and both look numeric. 100.00% does not. The percent sign makes it an ordinary string, so comparing it against 90 is lexical with no quoted constant anywhere in the expression — and lexically 1 precedes 9, so 100.00% sorts below 90.

Which means the failure is worst on the best data:

A unit at 100.00% compares as less than 90 and is reported failing.
A unit at 9.00% also compares as less than 90, and is correctly reported failing.
A unit at 87.50% is correctly reported failing.

So a coverage gate written this way passes nothing that has a percent sign, flags fully-covered units as failures, and still produces a plausible-looking list of units to go and work on. The one row that behaves is the one whose report omitted the %.

The fix is + 0, which parses the numeric prefix and stops at the first character it cannot use. It costs nothing and it is the same habit Chapter 10.5 recommended for a different reason — which is the argument for applying it unconditionally rather than reasoning about whether this particular field is a strnum.

+ 0 handles the whole range of forms a report might print:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk 'BEGIN { split("87.50% 100.00% 0.00% n/a -- 100% 7.5%", v, " ")
  for (i = 1; i <= 7; i++) printf "  [%-8s] +0 -> %-8s  >=90 -> %s\n", v[i], v[i]+0, (v[i]+0>=90?"yes":"no") }'

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  [87.50%  ] +0 -> 87.5      >=90 -> no
  [100.00% ] +0 -> 100       >=90 -> yes
  [0.00%   ] +0 -> 0         >=90 -> no
  [n/a     ] +0 -> 0         >=90 -> no
  [--      ] +0 -> 0         >=90 -> no
  [100%    ] +0 -> 100       >=90 -> yes
  [7.5%    ] +0 -> 7.5       >=90 -> no

Both 100% forms now agree, and 7.5% is correctly below the gate rather than lexically above it. The conversion has to appear in the comparison itself — + 0 applied only to a printed column leaves the test comparing the raw string, which is the defect section 4 opened with.

The three rows that convert to 0 are the subject of the next section, because only one of them means zero coverage.

5. `n/a` Is Not Zero

Rows three, four and five of that table all converted to 0, and only one of them means zero coverage. u_io in the report has Branch Coverage n/a (0 of 0 branches) — it contains no branches, so there is nothing to cover.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '/^  Branch Coverage/ { p = $3; printf "  [%-9s] +0 = [%s]\n", p, p + 0 }' cov.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  [91.67%   ] +0 = [91.67]
  [100.00%  ] +0 = [100]
  [76.47%   ] +0 = [76.47]
  [64.29%   ] +0 = [64.29]
  [n/a      ] +0 = [0]
  [0.00%    ] +0 = [0]

The fifth and sixth rows are indistinguishable after conversion, and they are different facts: u_io has no branches, u_debug has four and covered none.

Which gives branch coverage three different answers:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '/^  Branch Coverage/ { p = $3; s += p + 0; n++ } END { printf "  n/a as 0%%:    mean of %d units = %.2f%%\n", n, s/n }' cov.rpt
awk '/^  Branch Coverage/ { if ($3 == "n/a") { skipped++; next } p = $3; s += p + 0; n++ } END { printf "  n/a excluded: mean of %d units = %.2f%%  (%d skipped)\n", n, s/n, skipped }' cov.rpt
awk '/^TOTAL  Branch/ { printf "  report TOTAL:                   %s\n", $4 }' cov.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  n/a as 0%:    mean of 6 units = 55.41%
  n/a excluded: mean of 5 units = 66.49%  (1 skipped)
  report TOTAL:                   73.58%

55.41%, 66.49% and 73.58% from one report, and the first two are both wrong.

6. Mean of Percentages Is Not the Coverage

The third answer is right, and the report proves it:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '/^  Branch Coverage/ { c += substr($4,2) + 0; t += $6 + 0 } END { printf "  recomputed from counts: %d of %d = %.2f%%\n", c, t, 100*c/t }' cov.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  recomputed from counts: 39 of 53 = 73.58%

Aggregating the counts reproduces the report's own total exactly. Averaging the percentages does not, and the gap is not small:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '/^  Line Coverage/ { p = $3; sub(/%$/,"",p); s += p+0; n++; c += substr($4,2)+0; t += $6+0 }
  END { printf "  mean of %d per-unit percentages : %.2f%%\n", n, s/n; printf "  aggregate %d of %d lines     : %.2f%%\n", c, t, 100*c/t }' cov.rpt
awk '/^TOTAL  Line/ { printf "  report TOTAL                   : %s\n", $4 }' cov.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  mean of 6 per-unit percentages : 76.13%
  aggregate 192 of 235 lines     : 81.70%
  report TOTAL                   : 81.70%

76.13% against 81.70%, with the report agreeing on the latter. The mean treats u_io's twelve lines as worth the same as u_alu's eighty-five; the aggregate does not.

Extract the counts, not the percentage

The practical consequence for every coverage parser in this module:

A percentage is a derived number that has already thrown away what you need. 96.55% does not say whether it came from 28 of 29 lines or 2800 of 2900, and those two facts support completely different decisions. Once six such percentages are in hand there is no way to combine them correctly, because the weights are gone.

So the rule is to take (28 of 29) and leave 96.55% alone — and if a report gives only the percentage, say so in the output rather than averaging anyway. Chapter 11.8 is about this and goes further into the weights a tool applies; what is already visible here is that the counts are sufficient and the percentages are not.

Two secondary benefits of parsing the counts, both of which this chapter has used:

The counts let you check the report against itself. Every percentage in cov.rpt was verified against its own (covered of total) pair, which is how the fixture's three hand-written total rows were found to be wrong before publication.
The counts distinguish n/a from 0%. (0 of 0) and (0 of 4) are different; n/a and 0.00% both convert to the same 0.

A denominator of zero still needs guarding, for the reason Chapter 11.1 section 8 measured: division by zero is fatal and exits 2.

7. Verifying the Report Against Itself

Every percentage against its own counts:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '/^  (Line|Toggle|Branch) Coverage/ { p = $3
   if (p == "n/a") { printf "  %-8s n/a      (%s of %s) -> no coverable items\n", $1, substr($4,2), $6; next }
   sub(/%$/, "", p); n = substr($4, 2) + 0; m = $6 + 0
   if (m == 0) { printf "  %-8s %7s%% with a zero denominator -- cannot check\n", $1, p; next }
   r = 100 * n / m
   printf "  %-8s %7s%%  %4d of %4d -> recomputed %6.2f%%  %s\n", $1, p, n, m, r, (p+0 - r < 0.005 && r - (p+0) < 0.005 ? "ok" : "MISMATCH") }' cov.rpt | head -7

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  Line       96.55%    28 of   29 -> recomputed  96.55%  ok
  Toggle     74.19%    46 of   62 -> recomputed  74.19%  ok
  Branch     91.67%    11 of   12 -> recomputed  91.67%  ok
  Line      100.00%    41 of   41 -> recomputed 100.00%  ok
  Toggle     98.44%   126 of  128 -> recomputed  98.44%  ok
  Branch    100.00%     6 of    6 -> recomputed 100.00%  ok
  Line       88.24%    75 of   85 -> recomputed  88.24%  ok

The m == 0 guard comes before the division, not after — Chapter 11.1 section 8's division by zero is fatal, so a guard that runs afterwards never runs.

And the three totals:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '/^  Line Coverage/ { lc += substr($4,2)+0; lt += $6+0 } /^  Toggle Coverage/ { tc += substr($4,2)+0; tt += $6+0 } /^  Branch Coverage/ { bc += substr($4,2)+0; bt += $6+0 }
  END { printf "  Line   %d of %d = %.2f%%\n  Toggle %d of %d = %.2f%%\n  Branch %d of %d = %.2f%%\n", lc, lt, 100*lc/lt, tc, tt, 100*tc/tt, bc, bt, 100*bc/bt }' cov.rpt
awk '/^TOTAL/ { printf "  report: %-6s %s  %s %s %s\n", $2, $4, substr($5,2), $6, substr($7,1,length($7)-1) }' cov.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  Line   192 of 235 = 81.70%
  Toggle 295 of 420 = 70.24%
  Branch 39 of 53 = 73.58%
  report: Line   81.70%  192 of 235
  report: Toggle 70.24%  295 of 420
  report: Branch 73.58%  39 of 53

Every figure agrees. That is the check worth running on any real coverage report before using a single number from it.

8. Common Mistakes

Comparing a percentage field against a threshold without + 0. The % makes it a string, so 100.00% >= 90 is false — section 4.

Applying + 0 to the printed column but not to the test. Convert into a variable and compare the variable.

Treating n/a as 0%. It means no coverable items — section 5.

Averaging per-unit percentages. 76.13% against a true 81.70% — section 6.

Keeping the percentage and discarding the counts. The weights are then unrecoverable.

/Coverage/ as the metric pattern. It matches 22 lines including the report title — section 3.

Not carrying the unit name, or not clearing it. The name is on a separate line from the metrics.

Dividing before guarding a zero denominator. The division is fatal, so the guard must come first.

9. Exercises

Exercise 1 — Line coverage per unit

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '/^Design Unit:/ { u = $3 } /^  Line Coverage/ { printf "%-10s %7.2f%%  %s of %s\n", u, $3 + 0, substr($4,2), $6; u = "" }' cov.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

alu_core     96.55%  28 of 29
u_regfile   100.00%  41 of 41
u_alu        88.24%  75 of 85
u_ctrl       72.00%  36 of 50
u_io        100.00%  12 of 12
u_debug       0.00%  0 of 18

Exercise 2 — Units below a threshold

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk -v gate=90 '/^Design Unit:/ { u = $3 } /^  Line Coverage/ { p = $3 + 0; if (p < gate) printf "%-10s %7.2f%%  below %d%%\n", u, p, gate; u = "" }' cov.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

u_alu        88.24%  below 90%
u_ctrl       72.00%  below 90%
u_debug       0.00%  below 90%

Three units, and neither of the two at 100.00% is among them.

Exercise 3 — Debug a gate that fails a fully covered unit

A coverage gate reports u_alu failing at 100% line coverage.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

printf 'u_alu 100.00%%\nu_dma 100.00\n' > two.txt
awk '{ printf "%-8s [%s]  >= 90 without +0 -> %-5s   with +0 -> %s\n", $1, $2, ($2 >= 90 ? "TRUE" : "false"), ($2 + 0 >= 90 ? "TRUE" : "false") }' two.txt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

u_alu    [100.00%]  >= 90 without +0 -> false   with +0 -> TRUE
u_dma    [100.00]  >= 90 without +0 -> TRUE    with +0 -> TRUE

Both are at 100. The one with the percent sign is not a number, so its comparison is lexical and 1 sorts below 9.

Exercise 4 — Debug a coverage figure that is too low

A dashboard reports branch coverage at 55.41% where the report says 73.58%.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '/^  Branch Coverage/ { printf "[%s]", $3 } END { print "" }' cov.rpt
awk '/^  Branch Coverage/ { s += $3 + 0; n++ } END { printf "  mean of %d = %.2f%%\n", n, s/n }' cov.rpt
awk '/^  Branch Coverage/ { c += substr($4,2)+0; t += $6+0 } END { printf "  aggregate %d of %d = %.2f%%\n", c, t, 100*c/t }' cov.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

[91.67%][100.00%][76.47%][64.29%][n/a][0.00%]
  mean of 6 = 55.41%
  aggregate 39 of 53 = 73.58%

The dashboard counted the n/a unit as 0% and averaged the six percentages. Aggregating the counts instead gives 39 of 53, which is the report's own figure. And excluding the n/a row changes nothing, because (0 of 0) contributes zero to both the numerator and the denominator — the aggregate handles the undefined case for free, which averaging cannot.

Exercise 5 — Engineering application

The coverage gate a regression actually runs: counts aggregated, n/a distinguished, every percentage checked against its own counts, and a non-zero status when a unit is below the threshold.

Azvya Education Pvt. Ltd.VLSI Mentor

covgate.awk — synthetic teaching script.

# covgate.awk — per-unit coverage gate with aggregate totals.
#
# Usage:  awk -f covgate.awk cov.rpt
#         awk -v gate=90 -v metric=Line -f covgate.awk cov.rpt
# Exit:   0 all units at or above the gate
#         1 at least one unit below the gate
#         2 a percentage disagrees with its own counts

BEGIN { if (gate == "") gate = 90; if (metric == "") metric = "Line"; TOL = 0.005 }

/^Design Unit:/ { u = $3 }

$1 == metric && $2 == "Coverage" && /^  / {
    pct = $3
    n = substr($4, 2) + 0
    m = $6 + 0
    nu++

    if (pct == "n/a" || m == 0) {
        printf "%-10s %8s   %4d of %4d   no coverable items\n", u, "n/a", n, m
        skipped++
        u = ""
        next
    }

    p = pct + 0
    r = 100 * n / m
    if (!(p - r < TOL && r - p < TOL)) { printf "MISMATCH: %s reports %s but %d of %d is %.2f%%\n", u, pct, n, m, r > "/dev/stderr"; bad++ }

    cov += n; tot += m
    below = (p < gate)
    if (below) nbelow++
    printf "%-10s %8.2f%%  %4d of %4d   %s\n", u, p, n, m, (below ? "BELOW " gate "%" : "ok")
    u = ""
}

END {
    if (bad) exit 2
    if (tot == 0) { print "ERROR: no coverable items at all" > "/dev/stderr"; exit 2 }
    printf "#\n# %s coverage, aggregated: %d of %d = %.2f%%\n", metric, cov, tot, 100 * cov / tot
    printf "# %d unit(s), %d below %d%%, %d with nothing to cover\n", nu, nbelow + 0, gate, skipped + 0
    if (nbelow) exit 1
}

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk -f covgate.awk cov.rpt
echo "status: $?"

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

alu_core      96.55%    28 of   29   ok
u_regfile    100.00%    41 of   41   ok
u_alu         88.24%    75 of   85   BELOW 90%
u_ctrl        72.00%    36 of   50   BELOW 90%
u_io         100.00%    12 of   12   ok
u_debug        0.00%     0 of   18   BELOW 90%
#
# Line coverage, aggregated: 192 of 235 = 81.70%
# 6 unit(s), 3 below 90%, 0 with nothing to cover
status: 1

The aggregate 81.70% matches the report's own TOTAL, which is the check that the parse was right rather than merely plausible. Status 1 means at least one unit is below the gate — a result for a human to act on, not an error, and the same polarity as every gate script in Module 10.

On branch coverage, where the n/a unit appears:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk -v metric=Branch -v gate=75 -f covgate.awk cov.rpt
echo "status: $?"

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

alu_core      91.67%    11 of   12   ok
u_regfile    100.00%     6 of    6   ok
u_alu         76.47%    13 of   17   ok
u_ctrl        64.29%     9 of   14   BELOW 75%
u_io            n/a      0 of    0   no coverable items
u_debug        0.00%     0 of    4   BELOW 75%
#
# Branch coverage, aggregated: 39 of 53 = 73.58%
# 6 unit(s), 2 below 75%, 1 with nothing to cover
status: 1

u_io is reported as having nothing to cover rather than as a 0% failure, and it contributes 0 of 0 to the aggregate, which changes neither the numerator nor the denominator. The aggregate 73.58% again matches the report.

Exercise 6 — Challenge: show that the gate's own total is the only available check

covgate.awk compares its aggregate against nothing — it just prints it. Show that a plausible parsing error produces a plausible aggregate, and that only the report's TOTAL row distinguishes them.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '/^  Line Coverage/ { c += substr($4,2)+0; t += $6+0 } END { printf "correct:        %d of %d = %.2f%%\n", c, t, 100*c/t }' cov.rpt
awk '/^  Line Coverage/ { c += $4+0; t += $6+0 } END { printf "forgot substr:  %d of %d = %.2f%%\n", c, t, 100*c/t }' cov.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

correct:        192 of 235 = 81.70%
forgot substr:  0 of 235 = 0.00%

Forgetting substr gives 0.00%, because (28 converts to 0 — a visibly broken answer. Now a subtler slip, reading the denominator one field over:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '/^  Line Coverage/ { c += substr($4,2)+0; t += $5+0 } END { printf "wrong field:    %d of %d = %s\n", c, t, (t ? sprintf("%.2f%%", 100*c/t) : "undefined") }' cov.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

wrong field:    192 of 0 = undefined

Also visibly broken — $5 is the word of. The dangerous slip is one that still produces numbers:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '/^  (Line|Toggle) Coverage/ { c += substr($4,2)+0; t += $6+0 } END { printf "line+toggle:    %d of %d = %.2f%%\n", c, t, 100*c/t }' cov.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

line+toggle:    487 of 655 = 74.35%

74.35% is a completely plausible coverage figure and it is the line and toggle metrics added together — two different kinds of item in one ratio. Nothing about the number suggests it, and it sits between the true line figure of 81.70% and the true toggle figure of 70.24%, which is exactly where a reviewer's intuition would put a believable answer.

The report's own total is what refuses it:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '/^TOTAL  Line/ { d = $7; sub(/\)$/, "", d); printf "  report TOTAL Line: %s  (%s of %s)\n", $4, substr($5,2), d }' cov.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  report TOTAL Line: 81.70%  (192 of 235)

487 of 655 is not 192 of 235, and no amount of inspecting 74.35% would have told you. A coverage parser should always print its aggregate counts and compare them against the report's, because the percentage alone cannot be checked — the fourth time in two modules that the conclusion has been to validate against the input's own arithmetic rather than the output's plausibility.

10. Summary

A coverage value is a ratio, so it cannot be averaged, and a zero denominator is not zero.
The % sign makes the field non-numeric, so a comparison against a threshold is lexical even with no quoted constant — and 100.00% >= 90 is false.
This is a genuine exception to Chapter 10.5's rule, which held for fields that look numeric.
Convert into a variable, then compare the variable. + 0 in the printed column does not protect the test.
n/a converts to 0 and means no coverable items. (0 of 0) and (0 of 4) are different facts.
Branch coverage has three answers — 55.41% with n/a as zero, 66.49% excluding it, 73.58% aggregating the counts. Only the last matches the report.
Averaging per-unit line percentages gives 76.13% against a true 81.70%.
Extract the counts, not the percentage. The percentage has already discarded the weights, and the counts handle (0 of 0) for free.
Guard a zero denominator before dividing, because the division is fatal.
Print your aggregate counts and compare them against the report's TOTAL. Mixing line and toggle items gave a thoroughly plausible 74.35%.

Chapter 11.5 — Toggle Coverage stays on the same report and takes the metric whose meaning is most often misread: an untoggled bit and an unexecuted line are different failures, and a bus reported at 61.90% may have every bit of its data path stuck.

Where this fits

Part of the awk curriculum.

1. Why This Matters

2. The Report

3. Carrying the Unit Name

4. A Unit at 100% Fails a 90% Gate

5. n/a Is Not Zero

6. Mean of Percentages Is Not the Coverage

7. Verifying the Report Against Itself

8. Common Mistakes

9. Exercises

Exercise 1 — Line coverage per unit

Exercise 2 — Units below a threshold

Exercise 3 — Debug a gate that fails a fully covered unit

Exercise 4 — Debug a coverage figure that is too low

Exercise 5 — Engineering application

Exercise 6 — Challenge: show that the gate's own total is the only available check

10. Summary

Where this fits

5. `n/a` Is Not Zero