awk Part IV · Module 14 · AWK Debugging, Review, and Engineering Mastery

Handling Malformed Reports

Every parser in Part III assumed the report was well formed. Real reports are not: they arrive with CRLF endings from a Windows-hosted tool, with comment banners and page headers repeated every sixty lines, with a column that is present on most rows, truncated halfway through because the tool was killed, and carrying values that look numeric and are not. This chapter measures what each of those does to a working parser — and finds a NaN that compares equal to everything in this awk, passes a reconciliation check, and cannot be detected by the portable test for it.

Intermediate22 min readawkPart IVDebuggingRobustnessData Quality

Part IV · Module 14 · Chapter 14.4 · Handling Malformed Reports

1. Why This Matters

Chapters 14.1 through 14.3 debugged the program. This one debugs the input, and the distinction matters because the symptoms are indistinguishable: a parser that produces no rows, wrong rows or a quietly wrong total looks the same whether the fault is in the pattern or in the file.

The thread running through every section is the one Part IV keeps returning to. A malformed input almost never breaks the total. It breaks the denominator, or the selection, or one row — and the headline number stays plausible. Section 4 produces the clearest instance in the whole module: four different guards, four different record counts, and the identical sum.

2. CRLF Is Fatal on a String Field and Harmless on a Numeric One

A report from a Windows-hosted tool ends every line with \r\n. awk splits on \n, so the \r stays — attached to the last field.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

printf 'u_alu 1024.500 PASS\r\nu_ctrl 512.250 FAIL\r\n' > crlf.rpt
cat -e crlf.rpt | sed 's/^/  /'
awk '{ c = $3; gsub(/\r/, "<CR>", c); printf "  NR=%d NF=%d $3=[%s] len=%d\n", NR, NF, c, length($3) }' crlf.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  u_alu 1024.500 PASS^M$
  u_ctrl 512.250 FAIL^M$
  NR=1 NF=3 $3=[PASS<CR>] len=5
  NR=2 NF=3 $3=[FAIL<CR>] len=5

length($3) is 5 for a four-letter word. The field is PASS\r.

The gsub(/\r/, "<CR>", c) is not decoration. Printing $3 directly emits the carriage return into the output stream, where it returns the cursor to the start of the line and the rest of the text overwrites what was already there — so the field appears to be PASS, the brackets appear to close normally, and nothing looks wrong. Chapter 14.1 established the first half of this: a bracketed dump of the raw record looks normal, which is why the carriage return is the cause a careful engineer misses. The second half is that the bytes are not normal — capture that same output to a file and the ] sits on its own line. Substituting a visible token for the control character is the only way to print it honestly, and cat -e with its ^M is the equivalent for the raw file.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '$3 == "PASS" { n++ } END { printf "  $3 == \"PASS\"      : %d\n", n+0 }' crlf.rpt
awk '{ sub(/\r$/, ""); if ($3 == "PASS") n++ } END { printf "  after sub(/\\r$/) : %d\n", n+0 }' crlf.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  $3 == "PASS"      : 0
  after sub(/\r$/) : 1

Zero against one. The string comparison fails on every row, and a pass-counting script reports zero passes on a report where everything passed — which, as Chapter 12.6 noted, reads as catastrophic news rather than as a parse failure.

Now the same report with the number last:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

printf 'u_alu PASS 1024.500\r\nu_ctrl FAIL 512.250\r\n' > crlf2.rpt
awk '{ s += $3 } END { printf "  sum of $3 with CR   : %.3f\n", s }' crlf2.rpt
awk '{ sub(/\r$/, ""); s += $3 } END { printf "  sum of $3 cleaned   : %.3f\n", s }' crlf2.rpt
awk '$3 > 600 { n++ } END { printf "  count $3 > 600 CR   : %d\n", n+0 }' crlf2.rpt
awk '{ sub(/\r$/, "") } $3 > 600 { n++ } END { printf "  count $3 > 600 clean: %d\n", n+0 }' crlf2.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  sum of $3 with CR   : 1536.750
  sum of $3 cleaned   : 1536.750
  count $3 > 600 CR   : 1
  count $3 > 600 clean: 1

The CRLF hazard is string-only

Four results, and the two pairs agree exactly. The sum is identical with and without the carriage return, and so is the threshold count.

The reason is that awk's string-to-number conversion reads a numeric prefix and stops at the first character that cannot continue a number. "512.250\r" + 0 is 512.250, and a numeric comparison converts both sides, so $3 > 600 is evaluated on the number.

This is why a CRLF report can run through an arithmetic pipeline for months without anyone noticing, and then fail the day someone adds a verdict column or groups by a name. The failure is latent in exactly the parsers that do not compare strings — and Part III's parsers compare strings constantly: $1 == "TOTAL", a verdict word, a clock name used as an array key.

An array key is the worst case, because c["clk_io\r"] and c["clk_io"] are different keys that print identically. A grouping report then shows the same clock twice with no visible difference between the rows.

The fix is one rule at the top of the program, and it is cheap enough to be unconditional:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '{ sub(/\r$/, "") } { printf "  NR=%d last=[%s] len=%d\n", NR, $NF, length($NF) }' crlf.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  NR=1 last=[PASS] len=4
  NR=2 last=[FAIL] len=4

length is now 4. Note sub(/\r$/, "") acts on $0 and therefore re-splits the fields, which is what makes $NF clean rather than just $0.

3. Comments and Blank Lines: Four Guards, One Sum, Four Counts

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

cat > area.rpt <<'EOF'
# area report, rev 3
# generated by a synthetic flow

u_alu   1024.500  3094
u_ctrl   512.250  1180

// trailing note
u_io     256.125   402
EOF
awk '{ printf "  NR=%d NF=%d [%s]\n", NR, NF, $0 }' area.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  NR=1 NF=5 [# area report, rev 3]
  NR=2 NF=6 [# generated by a synthetic flow]
  NR=3 NF=0 []
  NR=4 NF=3 [u_alu   1024.500  3094]
  NR=5 NF=3 [u_ctrl   512.250  1180]
  NR=6 NF=0 []
  NR=7 NF=3 [// trailing note]
  NR=8 NF=3 [u_io     256.125   402]

Eight records, three of which are data. Note NR=7: // trailing note has NF=3, the same field count as a data row.

Four guards, increasingly careful:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '{ s += $2; n++ } END { printf "  no guard     : n=%d sum=%.3f mean=%.3f\n", n, s, s/n }' area.rpt
awk 'NF { s += $2; n++ } END { printf "  NF           : n=%d sum=%.3f mean=%.3f\n", n, s, s/n }' area.rpt
awk 'NF && $1 !~ /^(#|\/\/)/ { s += $2; n++ } END { printf "  NF + comment : n=%d sum=%.3f mean=%.3f\n", n, s, s/n }' area.rpt
awk 'NF == 3 && $2 + 0 == $2 { s += $2; n++ } END { printf "  shape        : n=%d sum=%.3f mean=%.3f\n", n, s, s/n }' area.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  no guard     : n=8 sum=1792.875 mean=224.109
  NF           : n=6 sum=1792.875 mean=298.812
  NF + comment : n=3 sum=1792.875 mean=597.625
  shape        : n=3 sum=1792.875 mean=597.625

The sum is right in all four and the mean is wrong in three

1792.875 is the true total: 1024.500 + 512.250 + 256.125. The completely unguarded program gets it exactly right.

It gets it right because every non-data line contributes zero. "#" + 0 is 0, "//" + 0 is 0, and an empty $2 on a blank line is 0. Adding zero to a sum is invisible.

The count is 8, 6, 3 and 3 — and the mean therefore spans 224.109 to 597.625, a factor of 2.67, on identical data with an identical and correct sum.

So a reviewer checking this parser against the report's own total would sign off on all four versions. The error only becomes visible in a per-module average, a "modules processed" line, a percentage, or any denominator at all. This is Chapter 11.8's lesson arriving from the opposite direction: there, the aggregate moved while the units did not; here the aggregate is immovable while the population underneath it changes by 2.67×.

The practical rule: always print the kept-record count beside any derived figure, and reconcile the count, not only the total. Chapter 12.6 graded exit statuses for this reason, and Chapter 13.5 stored numerators and denominators separately rather than percentages.

And the NF == 3 guard is not doing what it looks like it is doing:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk 'NF == 3 { printf "  NF==3 accepts NR=%d [%s]\n", NR, $0 }' area.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  NF==3 accepts NR=4 [u_alu   1024.500  3094]
  NF==3 accepts NR=5 [u_ctrl   512.250  1180]
  NF==3 accepts NR=7 [// trailing note]
  NF==3 accepts NR=8 [u_io     256.125   402]

Four records, one of them the // comment. The field count alone admits it; the part that excludes it is $2 + 0 == $2, because "trailing" + 0 is 0 and 0 == "trailing" is false. The field count is not the shape — the shape is the field count plus a type assertion on the fields that will be used as numbers.

4. A Field Count Is Not a Shape

Now a report where the row width is legitimately variable: the clock column is present only where the tool had one to print.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

cat > sparse.rpt <<'EOF'
u_alu   1024.500  3094  clk_core
u_ctrl   512.250  1180
u_io     256.125   402  clk_io
EOF
awk '{ printf "  NR=%d NF=%d $4=[%s]\n", NR, NF, $4 }' sparse.rpt
awk 'NF == 3 && $2 + 0 == $2 { s += $2; k++ } END { printf "  NF==3 guard  : kept=%d sum=%.3f\n", k+0, s }' sparse.rpt
awk 'NF >= 3 && $2 + 0 == $2 { s += $2; k++ } END { printf "  NF>=3 guard  : kept=%d sum=%.3f\n", k+0, s }' sparse.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  NR=1 NF=4 $4=[clk_core]
  NR=2 NF=3 $4=[]
  NR=3 NF=4 $4=[clk_io]
  NF==3 guard  : kept=1 sum=512.250
  NF>=3 guard  : kept=3 sum=1792.875

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

for w in 3 4; do
  printf "  want=%s -> " "$w"
  awk -v want="$w" 'NF == want && $2 + 0 == $2 { s += $2; k++ } END { printf "kept=%d sum=%.3f\n", k+0, s }' sparse.rpt
done
printf "  NF>=3   -> "
awk 'NF >= 3 && $2 + 0 == $2 { s += $2; k++ } END { printf "kept=%d sum=%.3f\n", k+0, s }' sparse.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  want=3 -> kept=1 sum=512.250
  want=4 -> kept=2 sum=1280.625
  NF>=3   -> kept=3 sum=1792.875

No exact width is correct. Three gives 512.250, four gives 1280.625, and only the minimum-width test gives the true 1792.875. The two wrong answers are 71.4% and 28.6% low, and both are plausible figures from a plausible-looking guard.

The NF == 3 guard kept exactly one row — the malformed one. It rejected both complete records and accepted the only record missing its clock, reporting 512.250 against a true 1792.875: a 71.4% undercount, from a guard written to improve robustness.

This is the characteristic failure of a width test on a report with an optional column. The guard was calibrated on whichever rows the author happened to look at, and it inverts the moment the other width is the normal one.

The absent field also does something quiet to a grouping:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '{ c[$4 == "" ? "<absent>" : $4]++ }
     END { n = split("<absent> clk_core clk_io", o, " ")
            for (i = 1; i <= n; i++) if (o[i] in c) printf "  group [%s] -> %d\n", o[i], c[o[i]] }' sparse.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  group [<absent>] -> 1
  group [clk_core] -> 1
  group [clk_io] -> 1

Written as c[$4]++ the third group would be keyed on the empty string, which prints as nothing at all — a report line reading group [] -> 1, or, with a %s and no brackets, a blank row. Naming the absent case is what makes it legible, and it is the same reason Chapter 14.2 bracketed every field it printed.

Note the iteration: a fixed name list rather than for (k in c), because for (k in c) has no defined order and Chapter 7.7 measured one program giving three different orders for three key prefixes.

5. Truncation: Two Line Counts That Disagree

A tool killed mid-write leaves a final record with no newline.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

printf 'u_alu   1024.500  3094\nu_ctrl   512.250  1180\nu_io     256.1' > trunc.rpt
awk '{ printf "  NR=%d NF=%d [%s]\n", NR, NF, $0 }' trunc.rpt
printf "  wc -l  : %s\n" "$(wc -l < trunc.rpt | tr -d ' ')"
printf "  awk NR : %s\n" "$(awk 'END { print NR }' trunc.rpt)"
awk 'NF == 3 && $2 + 0 == $2 { s += $2; k++ } END { printf "  guarded sum: kept=%d sum=%.3f\n", k+0, s }' trunc.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  NR=1 NF=3 [u_alu   1024.500  3094]
  NR=2 NF=3 [u_ctrl   512.250  1180]
  NR=3 NF=2 [u_io     256.1]
  wc -l  : 2
  awk NR : 3
  guarded sum: kept=2 sum=1536.750

wc -l counts newlines; awk counts records — and the disagreement is a symptom, not a verdict

wc -l says 2 and awk says 3. wc -l counts newline characters, and this file has two. awk reads the trailing bytes as a third record because end-of-file terminates a record just as a newline does.

The test is one line and needs no knowledge of the report's format:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  test -s FILE  &&  [ "$(wc -l < FILE)" = "$(awk 'END { print NR }' FILE)" ]

What it decides, exactly, is whether the last byte of the file is a newline — equivalently tail -c 1. That is evidence of truncation, and it is not the same thing as truncation. It fails in both directions, and both are cheap to produce:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

printf 'u_alu   1024.500  3094\nu_ctrl   512.250  1180\nu_io     256.125   402' > complete_no_nl.rpt
printf "  complete file, no final newline -> wc-l=%s NR=%s verdict=%s\n" \
  "$(wc -l < complete_no_nl.rpt | tr -d ' ')" "$(awk 'END { print NR }' complete_no_nl.rpt)" \
  "$([ "$(wc -l < complete_no_nl.rpt | tr -d ' ')" = "$(awk 'END { print NR }' complete_no_nl.rpt)" ] && echo complete || echo TRUNCATED)"
awk '{ n++ } $2 + 0 == $2 { s += $2 } END { printf "  ...yet %d whole record(s) and sum %.3f, which is correct\n", n, s }' complete_no_nl.rpt
printf 'u_alu   1024.500  3094\nu_ctrl   512.250  1180\n' > trunc_on_nl.rpt
printf "  truncated on a newline         -> wc-l=%s NR=%s verdict=%s\n" \
  "$(wc -l < trunc_on_nl.rpt | tr -d ' ')" "$(awk 'END { print NR }' trunc_on_nl.rpt)" \
  "$([ "$(wc -l < trunc_on_nl.rpt | tr -d ' ')" = "$(awk 'END { print NR }' trunc_on_nl.rpt)" ] && echo complete || echo TRUNCATED)"
awk '{ s += $2 } END { printf "  ...yet u_io is absent and the sum reads %.3f against a true 1792.875\n", s }' trunc_on_nl.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  complete file, no final newline -> wc-l=2 NR=3 verdict=TRUNCATED
  ...yet 3 whole record(s) and sum 1792.875, which is correct
  truncated on a newline         -> wc-l=2 NR=2 verdict=complete
  ...yet u_io is absent and the sum reads 1536.750 against a true 1792.875

A false alarm and a miss. A complete three-record file written without a trailing newline — which printf produces by default and several tools emit by design — is reported TRUNCATED while every record is whole and the sum is right. And a file whose writer died immediately after a newline is reported complete while a whole record is missing and the sum is wrong by 256.125, which is exactly the silent undercount this chapter is about.

So use it as what it is: a conservative reason to refuse to parse, because a file whose last record may be partial cannot be trusted, and refusing costs nothing. It is not a proof of truncation, and it cannot be the only check — the per-record accounting in section 9 and the reconciliation against the report's own total are what catch a truncation that landed on a newline.

Note what the guard did: kept=2 sum=1536.750, against the true 1792.875. The partial row was silently dropped, the program exited 0, and the only evidence is the kept count — which is the fourth time in this chapter that the kept count is the load-bearing number.

The partial row is also a reminder that truncation can be invisible at the field level. Had the tool been killed one character later, after 256.125 but before the cell count, the record would read u_io 256.125 — NF=2 again, but with a complete and correct area value. There is no way to tell from the fields whether a short row lost data; only the file-level check knows.

6. Values That Look Numeric and Are Not

The type assertion used throughout this chapter is $x + 0 == $x. It is worth knowing exactly what it accepts.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

printf '1024.500\n0\n0.000\n-0.000\n1e3\n12ps\n1,234\nn/a\n-\nNaN\n  12\ninf\n' > vals.txt
awk '{ printf "  [%-8s] +0=%-10s numeric?=%s\n", $1, $1 + 0, ($1 + 0 == $1 ? "yes" : "no") }' vals.txt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  [1024.500] +0=1024.5     numeric?=yes
  [0       ] +0=0          numeric?=yes
  [0.000   ] +0=0          numeric?=yes
  [-0.000  ] +0=0          numeric?=yes
  [1e3     ] +0=1000       numeric?=yes
  [12ps    ] +0=12         numeric?=no
  [1,234   ] +0=1          numeric?=no
  [n/a     ] +0=0          numeric?=no
  [-       ] +0=0          numeric?=no
  [NaN     ] +0=nan        numeric?=yes
  [12      ] +0=12         numeric?=yes
  [inf     ] +0=inf        numeric?=yes

The eleventh input was typed as 12 with two leading spaces and prints as 12 — the default field splitting discarded the leading blanks before $1 existed, so there was never anything for the numeric test to object to. Leading and trailing whitespace is the one malformation in this chapter that the default FS handles for free — and section 8 measures what naming an explicit separator costs you.

Read the +0 column first, because it is what an unguarded program uses:

12ps becomes 12. A unit suffix is silently truncated, not rejected.
1,234 becomes 1. A thousands separator truncates at the comma, so a four-digit value enters the arithmetic as a single digit.
n/a and - become 0, which Chapter 11.4 showed means no coverable items in one report and not measured in another, and zero is the right answer for only one of those.
-0.000 becomes 0, and Chapter 10.3 showed that in a timing report -0.000 is a violation the tool rounded.

Then read the numeric? column, which is what a guarded program uses. The guard catches 12ps, 1,234, n/a and -. It accepts NaN and inf — and those two are section 7.

The thousands separator is worth seeing as a total, because the damage is not proportional to anything:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

printf 'u_alu 1,234.500\nu_ctrl 987.250\n' > sep.rpt
awk '{ s += $2 } END { printf "  sum as read        : %.3f\n", s }' sep.rpt
awk '{ gsub(/,/, "", $2); s += $2 } END { printf "  sum after gsub     : %.3f\n", s }' sep.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  sum as read        : 988.250
  sum after gsub     : 2221.750

988.250 against 2221.750 — a 55.5% undercount, from one comma. And the wrong figure is entirely plausible: it is the right order of magnitude, with three decimal places, in a report whose other row is about that size.

The unit suffix is the opposite case, and it is the more dangerous one:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

printf 'u_alu 1024.500ps\nu_ctrl 512.250ps\n' > units.rpt
awk '{ s += $2 } END { printf "  sum with ps suffix  : %.3f\n", s }' units.rpt
cat > mixed_units.rpt <<'EOF'
u_alu   1.500ns
u_ctrl  512.250ps
u_io    256.125ps
EOF
awk '{ s += $2; n++ } END { printf "  mixed units, naive  : %.3f over %d rows\n", s, n }' mixed_units.rpt
awk '{ u = $2; sub(/^[-0-9.eE+]*/, "", u); c[u]++ }
     END { n = split("ps ns", o, " "); for (i = 1; i <= n; i++) if (o[i] in c) printf "  unit [%s] appears %d time(s)\n", o[i], c[o[i]] }' mixed_units.rpt
awk '{ v = $2 + 0; u = $2; sub(/^[-0-9.eE+]*/, "", u)
       if (u == "ns") v *= 1000; else if (u != "ps") { bad++; next }
       s += v; n++ }
     END { printf "  unit-aware          : %.3f ps over %d rows, %d rejected\n", s, n, bad+0 }' mixed_units.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  sum with ps suffix  : 1536.750
  mixed units, naive  : 769.875 over 3 rows
  unit [ps] appears 2 time(s)
  unit [ns] appears 1 time(s)
  unit-aware          : 2268.375 ps over 3 rows, 0 rejected

The first sum is correct: 1024.500ps + 512.250ps summed as numbers gives the right total in picoseconds, because the suffix is uniform and gets truncated uniformly. A parser can run for years like that.

Then one row arrives in nanoseconds and the naive sum reads 769.875 against a true 2268.375 — a factor of 2.95, because 1.500ns entered the sum as 1.5. The suffix was never parsed, so there was nothing to notice.

The lesson is that truncating a unit is not an error until the units stop agreeing, which makes it a latent defect of exactly the kind section 5 of Chapter 14.3 described for the dot. The remedy is to extract the suffix and count the distinct values — one unit means the sum is safe, two means it is not, and that check costs one array.

7. NaN Compares Equal to Everything

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk 'BEGIN {
  x = "NaN" + 0
  printf "  x            = %s\n", x
  printf "  x == 0       : %s\n", (x == 0 ? "TRUE" : "false")
  printf "  x == x       : %s\n", (x == x ? "TRUE" : "false")
  printf "  x != x       : %s\n", (x != x ? "TRUE" : "false")
  printf "  x - x        = %s\n", x - x
  printf "  (x - x) == 0 : %s\n", ((x - x) == 0 ? "TRUE" : "false")
  printf "  x < 0        : %s\n", (x < 0 ? "TRUE" : "false")
  printf "  x > 0        : %s\n", (x > 0 ? "TRUE" : "false")
}'

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  x            = nan
  x == 0       : TRUE
  x == x       : TRUE
  x != x       : false
  x - x        = nan
  (x - x) == 0 : TRUE
  x < 0        : false
  x > 0        : false

x != x does not detect NaN in this awk

Read the three comparison results together. x < 0 is false and x > 0 is false — correct IEEE behaviour, since every ordered comparison with NaN is false. But x == 0 is TRUE.

That combination is what a three-way comparison implemented as two tests produces: if neither < nor > holds, the values are concluded equal. So in this awk a NaN is equal to zero, equal to itself, equal to any number you compare it with — and x != x, the portable IEEE test for NaN that every other language relies on, is false.

The practical consequences are severe:

if (v != v) — the standard NaN check — never fires.
if (v == 0) fires for NaN, so a divide-by-zero guard treats a NaN denominator as a zero one. That is accidentally safe.
if (sum == total) fires for NaN against any total, which is not safe at all.

The one test that works is a range test, because it is built from the ordered comparisons that behave correctly: v > -1e308 && v < 1e308 is false for NaN, for inf and for -inf, since each of those comparisons is false.

The reconciliation case is the one worth memorising, because reconciling against the report's own total is the single most common sanity check in Part III:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

cat > recon.rpt <<'EOF'
u_alu   1024.500  3094
u_ctrl       NaN  1180
u_io     256.125   402
TOTAL   1792.875  4676
EOF
awk '$1 == "TOTAL" { t = $2; next } { s += $2 }
     END { printf "  own sum = %s  reported total = %s\n", s, t
            printf "  equality  : %s\n", (s == t ? "PASS" : "FAIL")
            printf "  tolerance : %s\n", ((s - t < 0.001 && t - s < 0.001) ? "PASS" : "FAIL") }' recon.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  own sum = nan  reported total = 1792.875
  equality  : PASS
  tolerance : FAIL

The equality check passes and the tolerance check fails, on a sum that is NaN.

The equality passes for the reason above: nan == 1792.875 is TRUE. The tolerance check fails because it is built from <, and nan - 1792.875 < 0.001 is false, so the conjunction is false.

This is a remarkable convergence with Chapter 13.4, which concluded that a tolerance is a policy, not a bug fix after measuring that == on two computed areas was exact on one fixture and failed on 58.3% of random value sets. The tolerance was adopted there to handle floating-point representation. It turns out to be the only form of the check that also survives a NaN — a second, independent reason for the same decision.

The guard, and what it costs:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

cat > wild.rpt <<'EOF'
u_alu   1024.500  3094
u_nan        NaN  1180
u_inf        inf   900
u_neg       -inf   120
u_io     256.125   402
EOF
awk '{ s += $2; n++ } END { printf "  naive             : n=%d sum=%s\n", n, s }' wild.rpt
awk 'NF == 3 && $2 + 0 == $2 { s += $2; n++ } END { printf "  numeric test only : n=%d sum=%s\n", n, s }' wild.rpt
awk 'NF == 3 && $2 + 0 == $2 { v = $2 + 0
       if (!(v > -1e308 && v < 1e308)) { bad++; next }
       s += v; n++ }
     END { printf "  range guard       : n=%d sum=%.3f rejected=%d\n", n+0, s, bad+0 }' wild.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  naive             : n=5 sum=nan
  numeric test only : n=5 sum=nan
  range guard       : n=2 sum=1280.625 rejected=3

The numeric test alone is no improvement at all — n=5, sum=nan, identical to no guard. One NaN anywhere in a column makes the entire total NaN, which is at least loud. Being loud is the only mercy here, and the previous example showed how easily an == check silences it.

The range test's threshold is at the representable limit, not at any domain value, so it rejects only what genuinely is not a finite number:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

printf 'u_big 1e307 10\nu_huge 1e309 10\n' > big.rpt
awk '{ v = $2 + 0; printf "  [%s] -> finite?=%s\n", $2, ((v > -1e308 && v < 1e308) ? "yes" : "NO") }' big.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  [1e307] -> finite?=yes
  [1e309] -> finite?=NO

1e307 is a perfectly good number and passes. 1e309 exceeds what a double can represent, becomes inf on conversion, and is rejected — which is correct, because the value awk holds is not the value the report contained.

8. A BOM, a Repeated Header, and a Tab

Three smaller cases, each with a distinct signature.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

printf '\357\273\277module area\nu_alu 1024.500\n' > bom.rpt
od -c bom.rpt | head -1 | sed 's/^/  /'
awk 'NR == 1 { printf "  NR=1 $1=[%s] len=%d\n", $1, length($1) }' bom.rpt
awk 'NR == 1 && $1 == "module" { print "  matched" } END { printf "  $1 == \"module\" matched? %s\n", (NR ? "see above" : "") }' bom.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  0000000  357 273 277   m   o   d   u   l   e       a   r   e   a  \n   u
  NR=1 $1=[module] len=9
  $1 == "module" matched? see above

No matched line was printed. $1 looks like module and length($1) is 9 — the three BOM bytes plus six letters — so $1 == "module" is false. A header-detection rule keyed on the first word of the first line silently fails, and the header is then processed as data.

The signature is a length that exceeds the visible text by exactly 3. sub(/^\357\273\277/, "") on NR == 1 removes it.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

cat > paged.rpt <<'EOF'
module    area    cells
u_alu   1024.500  3094
u_ctrl   512.250  1180
module    area    cells
u_io     256.125   402
EOF
awk '$1 == "module" { h++ } END { printf "  header rows seen : %d\n", h+0 }' paged.rpt
awk 'NR > 1 { s += $2; n++ } END { printf "  NR>1 guard       : n=%d sum=%.3f\n", n, s }' paged.rpt
awk 'NF == 3 && $2 + 0 == $2 { s += $2; n++ } END { printf "  shape guard      : n=%d sum=%.3f\n", n, s }' paged.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  header rows seen : 2
  NR>1 guard       : n=4 sum=1792.875
  shape guard      : n=3 sum=1792.875

Two headers, and NR > 1 catches only the first. A paginated report repeats its header every page, and the NR > 1 idiom — which is correct on a single-header file and appears in a great deal of working code — leaves the rest in the data.

The sums are again identical at 1792.875, because "area" + 0 is 0. Only the count differs: 4 against 3. Fifth instance in this chapter.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

printf 'u_alu\t1024.500\t3094\nu_ctrl   512.250  1180\n' > tabs.rpt
awk '{ printf "  default FS: NR=%d NF=%d $2=[%s]\n", NR, NF, $2 }' tabs.rpt
awk -F'\t' '{ printf "  -F tab    : NR=%d NF=%d $2=[%s]\n", NR, NF, $2 }' tabs.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  default FS: NR=1 NF=3 $2=[1024.500]
  default FS: NR=2 NF=3 $2=[512.250]
  -F tab    : NR=1 NF=3 $2=[1024.500]
  -F tab    : NR=2 NF=1 $2=[]

The default FS handles both rows and -F'\t' breaks one. Default field splitting treats any run of blanks — spaces and tabs — as one separator, so a report with inconsistent whitespace parses correctly by default. Naming the tab explicitly narrows the parser to exactly the rows that use it, and NF=1 with an empty $2 on the other.

That inverts the usual instinct. -F is right when the delimiter is significant — Chapter 12.4's delimited interface, where an empty field must survive — and wrong when it is merely whitespace. On a human-formatted report, the default is the robust choice, and a -F'\t' added "to be explicit" is a regression.

9. The Assembled Guard, and a Defect in My First Version

Everything above collects into one reusable rule set:

Azvya Education Pvt. Ltd.VLSI Mentor

clean.awk — synthetic teaching script.

# clean.awk -- accept a data row only if it has the expected shape and a
# finite number, and account for every row that is rejected.
#
# Rules are ordered cheapest-and-most-specific first, and each one exits via
# next, so a row is counted under exactly one reason.

{ sub(/\r$/, "") }                                   # section 2, unconditional

NF == 0                     { why["blank"]++;       rejected++; next }
$1 ~ /^(#|\/\/)/            { why["comment"]++;     rejected++; next }
NF != 3                     { why["field-count"]++; rejected++; next }
$2 + 0 != $2                { why["nonnumeric"]++;  rejected++; next }
{ v = $2 + 0 }
!(v > -1e308 && v < 1e308)  { why["nonfinite"]++;   rejected++; next }

{ s += v; kept++ }

END {
    printf "  kept=%d sum=%.3f rejected=%d", kept + 0, s, rejected + 0
    nk = split("blank comment field-count nonnumeric nonfinite", ord, " ")
    for (i = 1; i <= nk; i++) if (ord[i] in why) r = r " " ord[i] "=" why[ord[i]]
    printf "%s\n", (r == "" ? "" : "  (" substr(r, 2) ")")
}

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

for f in area.rpt paged.rpt trunc.rpt sparse.rpt wild.rpt; do
  printf "  %-12s -> " "$f"
  awk -f clean.awk "$f"
done

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  area.rpt     ->   kept=3 sum=1792.875 rejected=5  (blank=2 comment=3)
  paged.rpt    ->   kept=3 sum=1792.875 rejected=2  (nonnumeric=2)
  trunc.rpt    ->   kept=2 sum=1536.750 rejected=1  (field-count=1)
  sparse.rpt   ->   kept=1 sum=512.250 rejected=2  (field-count=2)
  wild.rpt     ->   kept=2 sum=1280.625 rejected=3  (nonfinite=3)

Five files, five different rejection profiles, and every rejected row accounted for by a named reason. Two rows of that table are worth reading as warnings rather than successes:

sparse.rpt reports kept=1. The guard is working exactly as written and the specification is wrong — section 4's optional column needs NF >= 3. The reason column says field-count=2, which is what makes the misconfiguration visible instead of mysterious.
trunc.rpt reports kept=2 and nothing in this output says the file was truncated. The reason field-count=1 is a hint, but the wc -l against NR check from section 5 is the only thing that states it.

My first version of this reported the wrong reason

The classifier above orders its tests NF == 0, then the comment pattern, then NF != 3, then the type, then the range. My first version collapsed the reasons into one ternary chain that tested $2 before NF:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  { rejected++; why[NF == 0 ? "blank" : ($2 + 0 == $2 ? "nonfinite" : "nonnumeric")]++ }

On trunc.rpt and sparse.rpt that reported nonfinite for every rejected row. Both files' rejected rows hold a perfectly good number — 256.1, 1024.500, 256.125 — and were rejected for their field count. The chain never tested NF != 0, so it fell through to the type test, which passed, and labelled the row as a bad value.

The kept, sum and rejected numbers were correct in every one of the five files. Only the explanation was wrong.

That is the defect shape to carry out of this chapter: a diagnostic can be wrong while its counts are right, and nothing downstream will notice, because the counts are what gets reconciled and the reason is what gets read by a human at 2 a.m. A reason code is a claim about causation and needs the same test-ordering discipline as the parser it is diagnosing — each test exiting via next so that exactly one reason can fire, ordered so that the most fundamental fact about the row is established first.

10. Common Mistakes

Ignoring CRLF because the arithmetic works. The sum and the threshold count were identical with the CR; only string comparisons and array keys broke — section 2.

Trusting a total. Four guards gave the identical correct sum and counts of 8, 6, 3 and 3 — a 2.67× spread on the mean.

Treating the field count as the shape. // trailing note has NF=3. The type assertion is what excludes it.

NF == 3 on a report with an optional column. It kept the one malformed row and rejected both complete ones — a 71.4% undercount.

Keying a group on a possibly-empty field. c[$4]++ files the absent case under the empty string, which prints as nothing.

Reading a wc -l / NR disagreement as proof of truncation. It proves only that the last byte is not a newline — a complete file can fail it and a newline-aligned truncation passes it.

Believing $x + 0 == $x means "is a number". It accepts NaN and inf.

Letting a unit suffix through. Harmless while the units agree; a 2.95× error the first time one row arrives in ns.

A thousands separator. 1,234.500 enters the arithmetic as 1, giving a plausible 55.5% undercount.

if (v != v) to detect NaN. It is false in this awk. Use a range test.

if (sum == total) as a reconciliation. A NaN sum reports PASS against any total. Use the tolerance form.

NR > 1 to skip a header. A paginated report has more than one.

Adding -F'\t' for explicitness. It broke the space-separated row that the default FS handled correctly.

A reason code derived from an unordered test chain. The counts stay right and the explanation goes wrong — section 9.

11. Exercises

Exercise 1 — Strip the carriage return and recount

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

printf 'u_alu 1024.500 PASS\r\nu_ctrl 512.250 FAIL\r\nu_io 256.125 PASS\r\n' > v.rpt
awk '$3 == "PASS" { n++ } END { printf "  without sub : %d pass\n", n+0 }' v.rpt
awk '{ sub(/\r$/, "") } $3 == "PASS" { n++ } END { printf "  with sub    : %d pass\n", n+0 }' v.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  without sub : 0 pass
  with sub    : 2 pass

Exercise 2 — Count the distinct units before trusting a sum

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '{ u = $2; sub(/^[-0-9.eE+]*/, "", u); if (!(u in seen)) { seen[u]; k++ } }
     END { printf "  distinct units: %d -> %s\n", k, (k == 1 ? "sum is safe" : "SUM IS NOT COMPARABLE") }' units.rpt
awk '{ u = $2; sub(/^[-0-9.eE+]*/, "", u); if (!(u in seen)) { seen[u]; k++ } }
     END { printf "  distinct units: %d -> %s\n", k, (k == 1 ? "sum is safe" : "SUM IS NOT COMPARABLE") }' mixed_units.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  distinct units: 1 -> sum is safe
  distinct units: 2 -> SUM IS NOT COMPARABLE

Note if (!(u in seen)) { seen[u]; k++ } rather than seen[u]++ with a later length(). Chapter 6.1 measured that reading an array element creates it, so a membership test must use in, and counting as you go avoids depending on length(), which is not POSIX.

Exercise 3 — Debug a reconciliation that passes

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '$1 == "TOTAL" { t = $2; next } { s += $2; n++ }
     END { printf "  rows=%d  sum=%s  total=%s  equality=%s\n", n, s, t, (s == t ? "PASS" : "FAIL") }' recon.rpt
awk '$1 == "TOTAL" { t = $2; next }
     { v = $2 + 0
       if (!(v > -1e308 && v < 1e308)) { bad++; next }
       s += v; n++ }
     END { printf "  rows=%d  sum=%.3f  total=%s  rejected=%d  verdict=%s\n", n, s, t, bad + 0,
                  (bad ? "CANNOT RECONCILE" : (t - s < 0.001 && s - t < 0.001 ? "PASS" : "FAIL")) }' recon.rpt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  rows=3  sum=nan  total=1792.875  equality=PASS
  rows=2  sum=1280.625  total=1792.875  rejected=1  verdict=CANNOT RECONCILE

Read the first line: three rows, a NaN sum, and equality=PASS against a total of 1792.875. The check that exists to catch a parse error reports that the parse agrees with the report.

The second refuses rather than reconciling. Note that it reports rows=2 — the row count is the usable rows, not the rows seen, which is the distinction section 3 argued for. CANNOT RECONCILE is the Chapter 12.6 verdict for a state in which nothing can be concluded, and in a script it exits 2 rather than 0 or 1.

Exercise 4 — Detect truncation without knowing the format

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

for f in area.rpt paged.rpt sparse.rpt trunc.rpt; do
  nl=$(wc -l < "$f" | tr -d ' ')
  nr=$(awk 'END { print NR }' "$f")
  printf "  %-12s wc-l=%-3s NR=%-3s %s\n" "$f" "$nl" "$nr" "$([ "$nl" = "$nr" ] && echo complete || echo TRUNCATED)"
done

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  area.rpt     wc-l=8   NR=8   complete
  paged.rpt    wc-l=5   NR=5   complete
  sparse.rpt   wc-l=3   NR=3   complete
  trunc.rpt    wc-l=2   NR=3   TRUNCATED

One line, no format knowledge, correct on all four — including sparse.rpt, whose rows have inconsistent widths and which is nevertheless a complete file. A width problem and a truncation problem are different faults and need different detectors.

Exercise 5 — Engineering application

Azvya Education Pvt. Ltd.VLSI Mentor

checkrpt.sh — synthetic teaching script.

#!/bin/sh
# checkrpt.sh — grade a report's parseability before anything parses it.
#
# Usage:  sh checkrpt.sh <report> [minimum-field-count]
#
# Exit:  0 every row usable
#        1 some rows rejected, the rest are usable
#        2 nothing can be concluded (missing, empty or truncated file)
#
# The file-level checks run FIRST and exit 2, because section 5 showed that
# truncation is invisible at the field level: a short row may hold perfectly
# good values and nothing about the fields says data was lost.
#
# `want` is a MINIMUM (NF < want), not an exact width. Section 4 measured
# that no exact width parses a report with an optional column: 3 kept one row
# of three and 4 kept two, where NF >= 3 keeps all three.

f=$1
want=${2:-3}

[ -n "$f" ]  || { echo "usage: checkrpt.sh <report> [fields]" >&2; exit 2; }
[ -r "$f" ]  || { echo "UNREADABLE: $f" >&2; exit 2; }
[ -s "$f" ]  || { echo "EMPTY: $f" >&2; exit 2; }

nl=$(wc -l < "$f" | tr -d ' ')
nr=$(awk 'END { print NR }' "$f")
if [ "$nl" != "$nr" ]; then
    echo "TRUNCATED: $f has $nl newline(s) and $nr record(s) -- no final newline" >&2
    exit 2
fi

awk -v want="$want" '
    { if ($0 ~ /\r/) cr++; sub(/\r$/, "") }

    NF == 0                     { why["blank"]++;          bad++; next }
    $1 ~ /^(#|\/\/)/            { why["comment"]++;        bad++; next }
    NF < want                   { why["too-few-fields"]++; bad++; next }
    $2 + 0 != $2                { why["nonnumeric"]++;     bad++; next }
    { v = $2 + 0 }
    !(v > -1e308 && v < 1e308)  { why["nonfinite"]++;      bad++; next }

    { u = $2; sub(/^[-0-9.eE+]*/, "", u); if (!(u in units)) { units[u]; nu++ }
      s += v; kept++ }

    END {
        printf "rows=%d kept=%d rejected=%d\n", NR, kept + 0, bad + 0
        nk = split("blank comment too-few-fields nonnumeric nonfinite", ord, " ")
        for (i = 1; i <= nk; i++) if (ord[i] in why) printf "  %-15s %d\n", ord[i], why[ord[i]]
        if (cr) printf "  %-15s %d (stripped)\n", "crlf", cr
        printf "sum=%.3f over %d distinct unit(s)%s\n", s, nu + 0,
               (nu > 1 ? " -- NOT COMPARABLE" : "")
        exit (bad ? 1 : 0)
    }
' "$f"

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

for f in area.rpt sparse.rpt wild.rpt trunc.rpt; do
  printf "== %s\n" "$f"
  sh checkrpt.sh "$f" > r.txt 2>&1; rc=$?
  sed 's/^/  /' r.txt
  printf "  rc=%s\n" "$rc"
done

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

== area.rpt
  rows=8 kept=3 rejected=5
    blank           2
    comment         3
  sum=1792.875 over 1 distinct unit(s)
  rc=1
== sparse.rpt
  rows=3 kept=3 rejected=0
  sum=1792.875 over 1 distinct unit(s)
  rc=0
== wild.rpt
  rows=5 kept=2 rejected=3
    nonfinite       3
  sum=1280.625 over 1 distinct unit(s)
  rc=1
== trunc.rpt
  TRUNCATED: trunc.rpt has 2 newline(s) and 3 record(s) -- no final newline
  rc=2

Four files, four distinct verdicts, and sparse.rpt now reports kept=3 sum=1792.875 — the correct total, because want is a minimum rather than an exact width.

Two details of how that was run matter as much as the script:

The status is captured from a redirect, not a pipe. sh checkrpt.sh "$f" > r.txt 2>&1; rc=$? then sed on the file. Written the other way round — sh checkrpt.sh "$f" | sed 's/^/ /'; rc=$? — every rc reads 0, because Chapter 12.1 section 7 measured that $? after a pipeline is the last command's status. Under /bin/sh there is no pipestatus, so a redirect is the portable remedy.
area.rpt exits 1, and that is correct but not actionable. Its five rejected rows are comment banners and blank lines — expected content in every report this flow produces. A caller that treats 1 as a failure rejects every commented report it will ever see.

The second point is a design conclusion, not a bug: the status grades parseability, not acceptability. Distinguishing a routine rejection from a suspicious one requires the reason breakdown, which is why the reasons are printed and named rather than summed into a single count — Chapter 13.5's argument for storing numerators and denominators instead of percentages, applied to diagnostics.

Exercise 6 — Challenge: show that the width argument is a policy, not a fix

Section 4 showed that no exact width parses sparse.rpt. Show what the want argument actually controls, and whether a caller can choose it correctly without knowing the report.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

for w in 3 4; do
  printf "== sparse.rpt want=%s\n" "$w"
  sh checkrpt.sh sparse.rpt "$w" > "o$w.txt" 2>&1; printf "  rc=%s\n" "$?"
  sed 's/^/  /' "o$w.txt"
done
echo "  --- diff ---"
diff o3.txt o4.txt > d.txt 2>&1 && echo "  identical" || sed 's/^/  /' d.txt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

== sparse.rpt want=3
  rc=0
  rows=3 kept=3 rejected=0
  sum=1792.875 over 1 distinct unit(s)
== sparse.rpt want=4
  rc=1
  rows=3 kept=2 rejected=1
    too-few-fields  1
  sum=1280.625 over 1 distinct unit(s)
  --- diff ---
  1,2c1,3
  < rows=3 kept=3 rejected=0
  < sum=1792.875 over 1 distinct unit(s)
  ---
  > rows=3 kept=2 rejected=1
  >   too-few-fields  1
  > sum=1280.625 over 1 distinct unit(s)

Three things are now established, and the exit codes carry two of them:

want controls the policy, not the correctness. At 3 every row is usable and the sum is the true 1792.875 with rc=0. At 4 the row without a clock is rejected, the sum drops to 1280.625, and rc=1 — and that is also a defensible run, if the caller's question is "what is the area of the modules whose clock the tool reported?" The script answers whichever question it was asked.
The caller cannot choose want from the data. Nothing in sparse.rpt says whether the missing clock makes a row unusable. The 4 run is not wrong; it is a different question with a different answer, and the only thing distinguishing them is a number supplied from outside.
The exit status is what separates them. Both runs print a plausible total. rc=0 against rc=1 is the only machine-readable signal that one run discarded a row, which is why the status is graded at all rather than being 0 for "it ran".

This is Chapter 13.4's conclusion in a new setting: a tolerance is a policy, not a bug fix. There it was a floating-point threshold that five values spanning fourteen orders of magnitude could not distinguish; here it is a field-count minimum. In both cases the parameter looks like a correctness knob, behaves like an editorial decision, and must be recorded in the output — which is what the too-few-fields 1 line and the exit status together do.

12. Summary

CRLF is string-only. $3 == "PASS" gave 0 while the sum and the threshold count were unaffected, because numeric conversion stops at the \r. { sub(/\r$/, "") } as rule one, unconditionally.
An array key carrying \r is a second, invisible key that prints identically to the clean one.
Four guards, one sum, four counts. 1792.875 every time, with n = 8, 6, 3, 3 and means spanning 2.67×. Every non-data line contributes zero.
Print the kept count beside every derived figure, and reconcile the count as well as the total.
The field count is not the shape. // trailing note has NF=3; the type assertion excludes it.
NF == 3 on a report with an optional column kept only the malformed row — 512.250 against 1792.875. Use NF >= 3 and name the absent case.
wc -l against awk END NR decides whether the last byte is a newline, which is evidence and not a verdict: a complete file with no trailing newline reads TRUNCATED, and a truncation that landed on a newline reads complete while its sum is 256.125 short. Use it to refuse, not to conclude.
$x + 0 == $x accepts NaN and inf and rejects 12ps, 1,234, n/a and -.
A thousands separator cost 55.5% and a mixed ps/ns column cost a factor of 2.95 — both silently, both plausibly.
Count the distinct unit suffixes. One means the sum is comparable; two means it is not.
In this awk NaN compares equal to everything. nan == 0 is TRUE, nan != nan is false, and nan < 0 and nan > 0 are both false — a three-way comparison built from two tests.
So if (v != v) cannot detect NaN, and if (sum == total) reports PASS against any total. The tolerance form fails correctly, which is Chapter 13.4's conclusion reached by a second route.
A range test is the working guard: v > -1e308 && v < 1e308. 1e307 passes, 1e309 becomes inf and is rejected.
A BOM makes $1 three bytes longer and $1 == "module" false; NR > 1 skips only the first of a paginated report's repeated headers.
The default FS handled a mixed tab/space report and -F'\t' broke it. Name the separator only when it is significant.
A diagnostic can be wrong while its counts are right — my first classifier tested $2 before NF and blamed the value for a field-count rejection in two of five files.
Order the tests and exit each via next, so exactly one reason fires and the most fundamental fact about the row is established first.

Chapter 14.5 — Performance and Maintainability turns from correctness to cost: what is actually slow in an awk program, which of the usual optimisations are measurable, and what makes a script survive the engineer who inherits it.

Where this fits

Part of the awk curriculum.