awk Part IV · Module 13 · Real-World AWK Engineering Projects

Project: UVM Regression Analyzer

Part IV builds the scripts a team keeps, and this is the first of them. It reads a regression's test list, finds and classifies each log, groups the failures by signature, and leaves an exit status a flow can branch on — and every design decision in it answers a failure measured somewhere in Part III. The result on this regression is the kind of thing the script exists to produce: nine tests resolve into six distinct states, and one failure signature turns out to be present in three of the four failing tests.

Advanced20 min readawkPart IVProjectRegressionUVM

Part IV · Module 13 · Chapter 13.1 · Project: UVM Regression Analyzer

1. What the Script Has to Do

A regression finishes and someone has to answer three questions before breakfast:

Which tests failed, and which did not run at all? Those are different problems with different owners.
How many distinct problems are there? Forty failing tests can be one bug.
Should the flow treat this run as valid? A run where a third of the logs are missing says nothing about the design.

Modules 9 to 12 established that each of those has a way to get it wrong that produces a plausible answer. This chapter builds the script that gets all three right at once, and the stages below are the order to build it in.

2. A Regression With the Real Cases

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

rm -rf reg && mkdir -p reg/logs
cat > reg/tests.txt <<'EOF'
alu_smoke
alu_random
alu_corner
fifo_basic
fifo_stress
dma_desc
dma_stress
cdc_sync
io_loopback
EOF
for t in alu_smoke fifo_basic; do
printf 'UVM_INFO @ 0: reporter [RNTST] Running test %s...\nUVM_INFO ./env/mon.sv(40) @ 10: top.mon [XACT] ok\n--- UVM Report Summary ---\nUVM_ERROR :    0\nUVM_FATAL :    0\n' "$t" > "reg/logs/$t.log"
done
for t in alu_random alu_corner; do
printf 'UVM_INFO @ 0: reporter [RNTST] Running test %s...\nUVM_ERROR ./env/sb.sv(148) @ 24: top.env.sb [MISMATCH] expected=32h5A got=32hFF\nUVM_ERROR ./env/sb.sv(148) @ 48: top.env.sb [MISMATCH] expected=32h11 got=32h00\n--- UVM Report Summary ---\nUVM_ERROR :    2\nUVM_FATAL :    0\n' "$t" > "reg/logs/$t.log"
done
printf 'UVM_INFO @ 0: reporter [RNTST] Running test fifo_stress...\nUVM_ERROR ./env/fifo_sb.sv(77) @ 12: top.env.fsb [OVERFLOW] depth exceeded\nUVM_ERROR ./env/sb.sv(148) @ 90: top.env.sb [MISMATCH] expected=32h01 got=32h02\n--- UVM Report Summary ---\nUVM_ERROR :    2\nUVM_FATAL :    0\n' > "reg/logs/fifo_stress.log"
printf 'UVM_INFO @ 0: reporter [RNTST] Running test dma_desc...\nUVM_ERROR ./env/dma_sb.sv(203) @ 31: top.env.dsb [MISMATCH] desc field wrong\n--- UVM Report Summary ---\nUVM_ERROR :    1\nUVM_FATAL :    0\n' > "reg/logs/dma_desc.log"
printf 'UVM_INFO @ 0: reporter [RNTST] Running test dma_stress...\nUVM_FATAL ./env/drv.sv(12) @ 8: top.env.drv [NOCLK] clock never started\n' > "reg/logs/dma_stress.log"
: > "reg/logs/io_loopback.log"
echo "  tests listed: $(wc -l < reg/tests.txt | tr -d ' ')   logs on disk: $(ls reg/logs | wc -l | tr -d ' ')"

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  tests listed: 9   logs on disk: 8

Nine tests, eight logs. cdc_sync has none. The composition is deliberate: two passes, four failures of which three share a signature, one UVM_FATAL, one zero-byte log, and one test whose log was never written. Every one of those is something a real regression produces weekly.

3. Stage One: Classify Every Test

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '{ want[++n] = $1 }
 END { for (i = 1; i <= n; i++) {
   t = want[i]; f = "reg/logs/" t ".log"; e = 0; fa = 0; dec = ""; nrec = 0
   r = (getline l < f)
   if (r < 0) { printf "  %-14s NO LOG\n", t; close(f); continue }
   while (r > 0) { nrec++
     if (l ~ /^UVM_ERROR (\.|\/|@)/) e++
     if (l ~ /^UVM_FATAL (\.|\/|@)/) fa++
     if (l ~ /^UVM_ERROR :/) { split(l, a, ":"); dec = a[2] + 0 }
     r = (getline l < f) }
   close(f)
   if (nrec == 0) st = "EMPTY"
   else if (fa) st = "FATAL"
   else if (dec == "") st = "NO SUMMARY"
   else if (dec + 0 != e) st = sprintf("INCONSISTENT (%d vs %d)", e, dec)
   else if (e) st = sprintf("FAIL (%d)", e)
   else st = "PASS"
   printf "  %-14s %s\n", t, st } }' reg/tests.txt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  alu_smoke      PASS
  alu_random     FAIL (2)
  alu_corner     FAIL (2)
  fifo_basic     PASS
  fifo_stress    FAIL (2)
  dma_desc       FAIL (1)
  dma_stress     FATAL
  cdc_sync       NO LOG
  io_loopback    EMPTY

Nine rows for nine tests, and six distinct states. Four decisions in that stage are load-bearing and none is obvious:

The loop walks reg/tests.txt, not reg/logs/*.log. A log-driven loop produces eight rows and no mention of cdc_sync — Chapter 9.5.
r < 0 is tested separately from nrec == 0. Chapter 12.6 measured that getline returns −1 for a missing file and 0 for an empty one, and that −1 is truthy — so a boolean test gets both wrong.
The pattern requires a path or an @ after the severity. /^UVM_ERROR/ alone also matches the report summary line UVM_ERROR : 2, which inflates every count — Chapter 9.3.
The parsed count is checked against the log's own summary. A disagreement means the pattern is wrong, which no arithmetic check can find — Chapter 9.2.

4. Stage Two: Choose the Grouping Key

Four failing tests, seven error records. How many problems is that? It depends entirely on the key:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

for key in id id_loc id_loc_hier; do
printf "  by %-12s " "$key"
awk -v k=$key '
 /^UVM_(ERROR|FATAL) (\.|\/)/ {
   loc = $2
   id = ""; for (i = 1; i <= NF; i++) if ($i ~ /^\[.*\]$/) { id = $i; gsub(/[\[\]]/, "", id); break }
   hier = ""; for (i = 1; i <= NF; i++) if ($i ~ /^top\./) { hier = $i; break }
   sig = (k == "id" ? id : (k == "id_loc" ? id " " loc : id " " loc " " hier))
   seen[sig] = 1 }
 END { n = 0; for (s in seen) n++; printf "%d distinct signature(s)\n", n }' reg/logs/*.log
done

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  by id           3 distinct signature(s)
  by id_loc       4 distinct signature(s)
  by id_loc_hier  4 distinct signature(s)

Three, four, four. Chapter 9.6 measured this on six records and got 2, 4 or 5; here the same effect is smaller and the reason is visible:

id alone gives 3 because it merges MISMATCH at ./env/sb.sv(148) with MISMATCH at ./env/dma_sb.sv(203) — two unrelated scoreboards reporting the same message id. That understates the work.
(id, location) gives 4, which separates them.
Adding the hierarchy also gives 4 here, because each location has exactly one component reporting from it. On a design with a module instantiated four times it would give 7 and invent three problems that do not exist.

So (id, location) is the default, and the choice has to be stated in the output — Chapter 11.8's rule that an aggregation which changes the answer belongs in the report.

Figure 1 — The analyzer's two independent reductions. Per-test classification answers which tests are usable; signature grouping answers how many problems there are. Both are needed, and neither is derivable from the other.

5. Stage Three: The Signature Table

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '
 FNR == 1 { t = FILENAME; sub(/.*\//, "", t); sub(/\.log$/, "", t) }
 /^UVM_(ERROR|FATAL) (\.|\/)/ {
   sev = $1; loc = $2
   id = ""; for (i = 1; i <= NF; i++) if ($i ~ /^\[.*\]$/) { id = $i; gsub(/[\[\]]/, "", id); break }
   sig = sev " " id " " loc
   rec[sig]++
   if (!((sig SUBSEP t) in pair)) { pair[sig, t] = 1; tests[sig]++; who[sig] = who[sig] " " t }
   if (!(sig in seen)) { seen[sig] = 1; order[++n] = sig } }
 END { printf "  %-42s %5s %5s  %s\n", "signature", "recs", "tests", "in"
        for (i = 1; i <= n; i++) { s = order[i]; printf "  %-42s %5d %5d %s\n", s, rec[s], tests[s], who[s] } }' reg/logs/*.log

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  signature                                   recs tests  in
  UVM_ERROR MISMATCH ./env/sb.sv(148)            5     3  alu_corner alu_random fifo_stress
  UVM_ERROR MISMATCH ./env/dma_sb.sv(203)        1     1  dma_desc
  UVM_FATAL NOCLK ./env/drv.sv(12)               1     1  dma_stress
  UVM_ERROR OVERFLOW ./env/fifo_sb.sv(77)        1     1  fifo_stress

One signature in three of the four failing tests

This row is what the script exists to produce:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

UVM_ERROR MISMATCH ./env/sb.sv(148)   5 records   3 tests

Five of the seven error records, in three of the four failing tests. One fix addresses three quarters of the failures, and a per-test table cannot show it — a per-test table shows four failing tests and invites four investigations.

Chapter 9.6 called a cross-test signature the cheapest halving of work available. Here it is quantified: the triage order is sb.sv(148) first, and the other three signatures are one test each.

Two cautions that keep this honest:

It is a hypothesis, not a conclusion. Three tests reporting from the same scoreboard line may be one bug or three bugs that happen to be checked in the same place. The script says where to look first, not what is wrong.
The record count and the test count measure different things. Five records across three tests partly reflects test length — Chapter 9.6 measured one bug producing 3 records on a short run and 40 on a long one. The test count is the stable figure and is the one to triage on; the record count is context.

The SUBSEP idiom is what makes the test count possible: pair[sig, t] is a two-dimensional key, so tests[sig] counts distinct tests rather than records. Without it, a signature appearing twice in one test would read as two tests.

6. The Complete Script

Azvya Education Pvt. Ltd.VLSI Mentor

regress.awk — synthetic teaching script.

# regress.awk — UVM regression analyzer.
#
# Usage:  awk -v logdir=reg/logs -f regress.awk reg/tests.txt
# Input:  one test name per line; each test's log is <logdir>/<name>.log
# Exit:   0 every test ran and passed
#         1 tests failed, and every log was usable
#         2 at least one log was unusable -- nothing can be concluded
#
# Design notes (each from a measured failure earlier in this track):
#   - Enumerates from the TEST LIST, never from the logs: a log-driven loop
#     cannot report the test whose log is absent.           (Module 9.5)
#   - Branches on all three getline outcomes: 1 record, 0 empty, -1 missing.
#     -1 is truthy and 0 is falsy, so a boolean test is backwards. (Module 12.6)
#   - Matches the RECORD SHAPE, not just the severity: the report summary line
#     also begins with UVM_ERROR.                            (Module 9.3)
#   - Cross-checks the parsed count against the log's own summary; a
#     disagreement is a PATTERN error, which arithmetic cannot find. (9.2)
#   - Groups failures by (id, location). `id` alone merges distinct sites;
#     adding the hierarchy splits one RTL bug across instances. (Module 9.6)
#   - Reports the per-signature TEST count, because a signature shared across
#     tests is the cheapest reduction in work available.      (Module 9.6)

BEGIN { if (logdir == "") logdir = "reg/logs" }

{ want[++ntest] = $1 }

function sigof(line,   i, nf, parts, id, loc) {
    nf = split(line, parts, " ")
    loc = parts[2]
    id = "?"
    for (i = 1; i <= nf; i++) if (parts[i] ~ /^\[.*\]$/) { id = parts[i]; gsub(/[\[\]]/, "", id); break }
    return id " " loc
}

END {
    if (ntest == 0) { print "ERROR: empty test list" > "/dev/stderr"; exit 2 }

    for (i = 1; i <= ntest; i++) {
        t = want[i]; f = logdir "/" t ".log"
        e = 0; fa = 0; dec = ""; nrec = 0

        r = (getline l < f)
        if (r < 0) { close(f); state[t] = "NO LOG"; unusable++; continue }
        while (r > 0) {
            nrec++
            if (l ~ /^UVM_ERROR (\.|\/)/) { e++; s = sigof(l); rec[s]++; if (!((s SUBSEP t) in pair)) { pair[s, t] = 1; tcount[s]++; who[s] = who[s] " " t } if (!(s in sseen)) { sseen[s] = 1; sord[++nsig] = s } }
            if (l ~ /^UVM_FATAL (\.|\/)/) { fa++; s = "FATAL " sigof(l); rec[s]++; if (!((s SUBSEP t) in pair)) { pair[s, t] = 1; tcount[s]++; who[s] = who[s] " " t } if (!(s in sseen)) { sseen[s] = 1; sord[++nsig] = s } }
            if (l ~ /^UVM_ERROR :/) { split(l, a, ":"); dec = a[2] + 0 }
            r = (getline l < f)
        }
        close(f)

        if (nrec == 0)                      { state[t] = "EMPTY";        unusable++ }
        else if (fa)                        { state[t] = "FATAL";        unusable++ }
        else if (dec == "")                 { state[t] = "NO SUMMARY";   unusable++ }
        else if (dec + 0 != e)              { state[t] = sprintf("INCONSISTENT records=%d summary=%d", e, dec); unusable++ }
        else if (e)                         { state[t] = sprintf("FAIL %d error(s)", e); failed++; ran++; errs += e }
        else                                { state[t] = "PASS";         passed++; ran++ }
    }

    print "TEST STATUS"
    for (i = 1; i <= ntest; i++) printf "  %-14s %s\n", want[i], state[want[i]]

    if (nsig) {
        print ""
        print "FAILURE SIGNATURES  (grouped by id and location)"
        printf "  %-40s %5s %5s  %s\n", "signature", "recs", "tests", "in"
        best = ""; bc = 0
        for (i = 1; i <= nsig; i++) { s = sord[i]
            printf "  %-40s %5d %5d %s\n", s, rec[s], tcount[s], who[s]
            if (tcount[s] > bc) { bc = tcount[s]; best = s } }
        if (bc > 1) printf "  -> %s appears in %d test(s); it is the single cheapest fix\n", best, bc
    }

    print ""
    printf "SUMMARY  tests=%d ran=%d passed=%d failed=%d errors=%d unusable=%d signatures=%d\n",
           ntest, ran + 0, passed + 0, failed + 0, errs + 0, unusable + 0, nsig + 0
    if (ran) printf "  pass rate: %d of %d that ran (%.1f%%), %d of %d attempted (%.1f%%)\n",
           passed + 0, ran, 100 * passed / ran, passed + 0, ntest, 100 * passed / ntest
    printf "  verdict: %s\n", (unusable ? "FLOW BROKEN" : (failed ? "TESTS FAILED" : "CLEAN"))

    if (unusable) exit 2
    if (failed) exit 1
}

Note the function sigof(line, i, nf, parts, id, loc) signature. The extra parameters after the gap are local variables — awk has no other way to declare one, and omitting them would leak i, parts and the rest into the global scope, which Chapter 6.5 measured as a bug that returns the correct answer and corrupts the caller's state.

7. Running It

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk -v logdir=reg/logs -f regress.awk reg/tests.txt
echo "status: $?"

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

TEST STATUS
  alu_smoke      PASS
  alu_random     FAIL 2 error(s)
  alu_corner     FAIL 2 error(s)
  fifo_basic     PASS
  fifo_stress    FAIL 2 error(s)
  dma_desc       FAIL 1 error(s)
  dma_stress     FATAL
  cdc_sync       NO LOG
  io_loopback    EMPTY

FAILURE SIGNATURES  (grouped by id and location)
  signature                                 recs tests  in
  MISMATCH ./env/sb.sv(148)                    5     3  alu_random alu_corner fifo_stress
  OVERFLOW ./env/fifo_sb.sv(77)                1     1  fifo_stress
  MISMATCH ./env/dma_sb.sv(203)                1     1  dma_desc
  FATAL NOCLK ./env/drv.sv(12)                 1     1  dma_stress
  -> MISMATCH ./env/sb.sv(148) appears in 3 test(s); it is the single cheapest fix

SUMMARY  tests=9 ran=6 passed=2 failed=4 errors=7 unusable=3 signatures=4
  pass rate: 2 of 6 that ran (33.3%), 2 of 9 attempted (22.2%)
  verdict: FLOW BROKEN
status: 2

Reading the summary line as a reviewer would:

ran=6 against tests=9 — three tests produced nothing usable, so the run is not a measurement of the design.
errors=7 and signatures=4 — seven records, four problems. The ratio is the point of the signature grouping.
33.3% and 22.2% — both printed, because the first excludes the three unusable tests and therefore improves as more tests fail to run (Chapter 9.5).
FLOW BROKEN and status: 2 — not TESTS FAILED, even though four tests failed, because the first thing to fix is the three logs.

8. Every Decision and Its Source

decision	alternative that fails	measured in
enumerate from the test list	a `logs/*.log` loop omits `cdc_sync`	9.5
branch on `getline`'s three values	`if (getline …)` calls missing readable, empty unreadable	12.6
require a path or `@` after the severity	`/^UVM_ERROR/` counts the summary line	9.3
cross-check against the log's summary	arithmetic cannot detect a pattern error	9.2
group by `(id, location)`	`id` merges sites, `+hierarchy` splits one bug	9.6
count distinct tests per signature	the record count partly measures test length	9.6
print both pass-rate denominators	the run-rate figure improves as the flow degrades	9.5, 11.8
grade the exit status 0/1/2	one non-zero sends a licence outage to RTL	9.7, 12.6
declare locals in the function signature	they leak and corrupt the caller	6.5
name the grouping in the output	an aggregation that changes the answer is part of it	11.8

Ten decisions, no new ones. That is the point of a project chapter: the script is not clever, it is the accumulation of answers to failures that each looked like a non-issue until measured.

9. Common Mistakes

Looping over the logs. The test with no log disappears — section 3.

Two states instead of six. Four of these six are flow problems.

Grouping by message id alone. It merged two unrelated scoreboards — section 4.

Adding the hierarchy to the key by default. It splits one RTL bug across instances.

Triaging on the record count. It partly measures test length; triage on the test count.

Treating a shared signature as a proven single bug. It is where to look first.

Omitting SUBSEP from the pair key. A signature twice in one test then reads as two tests.

One non-zero exit status. FLOW BROKEN and TESTS FAILED have different owners.

Undeclared function locals. They leak into the global scope — Chapter 6.5.

10. Exercises

Exercise 1 — Just the unusable tests

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '{ want[++n] = $1 }
 END { for (i = 1; i <= n; i++) { t = want[i]; f = "reg/logs/" t ".log"; fa = 0; nrec = 0
   r = (getline l < f)
   if (r < 0) { printf "  %-14s NO LOG\n", t; close(f); continue }
   while (r > 0) { nrec++; if (l ~ /^UVM_FATAL (\.|\/)/) fa++; r = (getline l < f) }
   close(f)
   if (nrec == 0) printf "  %-14s EMPTY\n", t
   else if (fa)   printf "  %-14s FATAL\n", t } }' reg/tests.txt

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  dma_stress     FATAL
  cdc_sync       NO LOG
  io_loopback    EMPTY

Three rows, three different flow problems, none of them about the design.

Note that this reads the logs, not the analyzer's report. Grepping the report for NO LOG|EMPTY|FATAL also matches the FATAL NOCLK … row in the signature table, and adding an indent prefix to lines that are already indented gives a second level of it. Chapter 12.4's rule — parse the source, not the formatted output — applies to your own script's output as much as to a tool's.

Exercise 2 — The triage order

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '
 FNR == 1 { t = FILENAME; sub(/.*\//, "", t); sub(/\.log$/, "", t) }
 /^UVM_(ERROR|FATAL) (\.|\/)/ {
   id = ""; for (i = 1; i <= NF; i++) if ($i ~ /^\[.*\]$/) { id = $i; gsub(/[\[\]]/, "", id); break }
   sig = id " " $2
   rec[sig]++
   if (!((sig SUBSEP t) in p)) { p[sig, t] = 1; tc[sig]++ }
   if (!(sig in s)) { s[sig] = 1; o[++n] = sig } }
 END { for (i = 1; i <= n; i++) printf "%d\t%d\t%s\n", tc[o[i]], rec[o[i]], o[i] }' reg/logs/*.log \
 | sort -t"$(printf '\t')" -k1,1rn -k2,2rn -k3,3 \
 | awk -F'\t' '{ printf "  %d test(s), %d record(s)  %s\n", $1, $2, $3 }'

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  3 test(s), 5 record(s)  MISMATCH ./env/sb.sv(148)
  1 test(s), 1 record(s)  MISMATCH ./env/dma_sb.sv(203)
  1 test(s), 1 record(s)  NOCLK ./env/drv.sv(12)
  1 test(s), 1 record(s)  OVERFLOW ./env/fifo_sb.sv(77)

Sorted by test count first and record count second, with the signature name as the final tiebreaker — three signatures sit at one test and one record each, so without it their order is undefined. Emitting tab-separated fields and sorting outside awk is Chapter 7.6's shape; sorting on the test count first is section 5's conclusion about which figure to triage on.

Exercise 3 — Debug a regression that reports eight tests

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

ls reg/logs/*.log | wc -l | awk '{ printf "  logs on disk: %d\n", $1 }'
wc -l < reg/tests.txt | awk '{ printf "  tests listed:  %d\n", $1 }'
for f in reg/logs/*.log; do basename "$f" .log; done | sort > seen.txt
sort reg/tests.txt > want.txt
comm -13 seen.txt want.txt | awk '{ printf "  listed but no log: %s\n", $0 }'

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  logs on disk: 8
  tests listed:  9
  listed but no log: cdc_sync

comm -13 names the test the log directory cannot know about: seen.txt is what exists and want.txt is what was expected, so lines unique to the second file are the missing tests. Both files are sorted, which comm requires.

Exercise 4 — Debug a signature count that halved

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

for key in id id_loc; do
printf "  by %-8s " "$key"
awk -v k=$key '/^UVM_ERROR (\.|\/)/ {
   id = ""; for (i = 1; i <= NF; i++) if ($i ~ /^\[.*\]$/) { id = $i; gsub(/[\[\]]/, "", id); break }
   seen[(k == "id" ? id : id " " $2)] = 1 }
 END { n = 0; for (s in seen) { n++; out[n] = s }
       printf "%d signature(s): ", n
       for (i = 1; i < n; i++) for (j = i+1; j <= n; j++) if (out[j] < out[i]) { x = out[i]; out[i] = out[j]; out[j] = x }
       for (i = 1; i <= n; i++) printf "[%s]", out[i]; print "" }' reg/logs/*.log
done

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  by id       2 signature(s): [MISMATCH][OVERFLOW]
  by id_loc   3 signature(s): [MISMATCH ./env/dma_sb.sv(203)][MISMATCH ./env/sb.sv(148)][OVERFLOW ./env/fifo_sb.sv(77)]

Two against three: MISMATCH alone covers two unrelated scoreboards, ./env/sb.sv(148) and ./env/dma_sb.sv(203). The signatures are collected into out[] and sorted before printing, because for (s in seen) has no defined order — Chapter 10.4 section 8 — and an unsorted listing here differs between runs. The script itself uses an insertion-ordered array for the same reason.

Exercise 5 — Engineering application

The script is the application. What remains is wiring it into a flow, which means the exit status and nothing else:

Azvya Education Pvt. Ltd.VLSI Mentor

nightly.sh — synthetic teaching script.

#!/bin/sh
# nightly.sh — run the analyzer and route the outcome.
#
# Note: awk is NOT at the end of a pipeline, so $? is awk's own status --
# Chapter 12.1 section 7 measured that a pipe replaces it with the last
# command's. The report goes to a file and is read afterwards.

awk -v logdir=reg/logs -f regress.awk reg/tests.txt > report.txt 2> errors.txt
status=$?

case $status in
  0) echo "CLEAN: every test ran and passed" ;;
  1) echo "TESTS FAILED: triage the top signature in report.txt" ;;
  2) echo "FLOW BROKEN: fix the unusable logs before reading any count" ;;
  *) echo "ANALYZER ITSELF FAILED (status $status)"; cat errors.txt ;;
esac
exit $status

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

sh nightly.sh
echo "status: $?"

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

FLOW BROKEN: fix the unusable logs before reading any count
status: 2

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

grep -c . report.txt | awk '{ printf "  report.txt: %d non-empty line(s)\n", $1 }'
wc -c < errors.txt | awk '{ printf "  errors.txt: %d byte(s)\n", $1 }'

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  report.txt: 20 non-empty line(s)
  errors.txt: 0 byte(s)

Twenty lines of report and nothing on stderr — the analyzer ran cleanly and the run did not. Those are different things, which is exactly why the fourth case arm exists.

Exercise 6 — Challenge: show that the summary line alone is not enough to triage

The summary says failed=4 signatures=4. Show that those two numbers, without the table, point at the wrong amount of work.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk -v logdir=reg/logs -f regress.awk reg/tests.txt 2>/dev/null | grep '^SUMMARY'

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

SUMMARY  tests=9 ran=6 passed=2 failed=4 errors=7 unusable=3 signatures=4

Four failures and four signatures reads as four independent problems — one per failing test. The table says otherwise:

The temptation is to re-parse the script's own table. Do not: $(NF-1) on the shared row lands on a test name rather than the test count, because that row has three names and the single-test rows have one. Chapter 12.4 made this point about tool reports and it applies to your own output — compute from the source instead:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '
 FNR == 1 { t = FILENAME; sub(/.*\//, "", t); sub(/\.log$/, "", t) }
 /^UVM_(ERROR|FATAL) (\.|\/)/ {
   id = ""; for (i = 1; i <= NF; i++) if ($i ~ /^\[.*\]$/) { id = $i; gsub(/[\[\]]/, "", id); break }
   sig = $1 " " id " " $2
   if (!((sig SUBSEP t) in p)) { p[sig, t] = 1; tc[sig]++ }
   if (!(sig in s)) { s[sig] = 1; o[++n] = sig } }
 END { for (i = 1; i <= n; i++) if (tc[o[i]] > 1) sh++; else sg++
       printf "  %d signature(s) in more than one test, %d in exactly one\n", sh+0, sg+0 }' reg/logs/*.log

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

  1 signature(s) in more than one test, 3 in exactly one

One shared and three single, and the shared one covers three of the four failing tests. So the four failing tests are at most two investigations: the shared scoreboard line, and fifo_stress's own OVERFLOW — because dma_desc and the alu_* pair are already accounted for.

Neither failed=4 nor signatures=4 carries that. The information is in the mapping from signatures to tests, which is a relation rather than a count, and no pair of totals can represent it.

That is the module's first statement of what a project script is for: the summary line exists to be stored and compared, and the table exists to be acted on. Chapter 11.8 argued a figure needs its denominator; this is the stronger form — some findings are not figures at all.

11. Summary

Nine tests resolve into six states, and only PASS and FAIL concern the design.
Enumerate from the test list. A log-driven loop omitted cdc_sync entirely.
Branch on getline's three return values, because −1 is truthy and 0 is falsy.
Require the record shape, or the report summary line inflates every count.
Cross-check against the log's own summary — it catches a pattern error, which arithmetic cannot.
Group by (id, location). id alone gave 3 signatures and merged two unrelated scoreboards; adding the hierarchy would split one bug across instances.
Count distinct tests per signature with a SUBSEP pair key. The record count partly measures test length.
One signature appeared in 3 of the 4 failing tests — five of seven records — making it the single cheapest fix, as a hypothesis.
Print both pass rates with denominators: 2 of 6 that ran, 2 of 9 attempted.
Grade the exit status. FLOW BROKEN outranks TESTS FAILED because the logs must be fixed before any count means anything.
Declare function locals in the signature, or they leak into the global scope.
The summary line is for storing; the signature table is for acting. failed=4 signatures=4 reads as four problems and the mapping says at most two.

Chapter 13.2 — Project: Timing Violation Extractor builds the second script: one row per violated path with its endpoint, clock and slack, and a worst-negative-slack figure that does not compare setup against hold.

Where this fits

Part of the awk curriculum.