Regression Log Processing

Part IV · Module 10 · Chapter 10.3 · Regression Log Processing

1. The Question Changes With the Scale

A regression produces one log per test. Nobody reads them. What gets read is the comparison: which tests changed state since the last run, and what is actually different about the ones that did.

That makes the goal a canonical form — each log reduced to the part that matters, with everything run-specific replaced by a placeholder, so that diff against last night shows only real changes. Everything in this chapter serves that.

cat -n regr/test1.log

     1	Run directory: /runs/20261012_1
     2	seed=481
     3	UVM_INFO /proj/tb/base_test.sv(88) @ 0: starting
     4	UVM_ERROR /proj/tb/scoreboard.sv(142) @ 981: data mismatch
     5	Summary
     6	  UVM_ERROR : 1
     7	  UVM_FATAL : 0
     8	trailing noise line

Three things differ between any two runs here for reasons nobody cares about: the run directory, the seed, and the simulation time. One thing is the actual result: the error and the summary counts.

2. The Shape of the Job

3. Step 3 on One File

Extracting the section up to the summary is a range, from Chapter 4.4:

sed -n '1,/Summary/p' regr/test1.log

Run directory: /runs/20261012_1
seed=481
UVM_INFO /proj/tb/base_test.sv(88) @ 0: starting
UVM_ERROR /proj/tb/scoreboard.sv(142) @ 981: data mismatch
Summary

Five lines; the summary counts and the trailing noise are dropped. Note the start address is the line number 1 rather than a pattern — Chapter 5.2's point that a number-anchored range cannot reopen, which is exactly what you want for "from the top, once".

4. Step 2 — the Failure That Only Exists Across Files

Now apply it to the regression. The obvious command:

sed -n '1,/Summary/p' regr/*.log

Run directory: /runs/20261012_1
seed=481
UVM_INFO /proj/tb/base_test.sv(88) @ 0: starting
UVM_ERROR /proj/tb/scoreboard.sv(142) @ 981: data mismatch
Summary

sed -n '1,/Summary/p' regr/*.log | wc -l

       5

Five lines from two files. test2.log contributed nothing at all, and exit status is 0.

Chapter 2.4 measured why: several file operands are one stream. The address 1 means line 1 of that stream, which belongs to test1.log, and the range closed at the first Summary — also in test1.log. There is no line 1 of the second file, so the range never reopened.

The output is a perfectly plausible extraction. At two files you can count; at two hundred, the result is one test's section presented as a regression summary.

The portable fix is to stop giving sed the glob and let the shell iterate:

for f in regr/*.log; do sed -n '1,/Summary/p' "$f"; done

Run directory: /runs/20261012_1
seed=481
UVM_INFO /proj/tb/base_test.sv(88) @ 0: starting
UVM_ERROR /proj/tb/scoreboard.sv(142) @ 981: data mismatch
Summary
Run directory: /runs/20261012_2
seed=482
UVM_INFO /proj/tb/base_test.sv(88) @ 0: starting
UVM_ERROR /proj/tb/scoreboard.sv(142) @ 982: data mismatch
Summary

for f in regr/*.log; do sed -n '1,/Summary/p' "$f"; done | wc -l

      10

Ten lines — both sections. Each invocation gets exactly one file, so 1 and $ mean what they look like.

5. Step 4 — Masking, and a Pattern That Half Works

The registered normaliser replaces the run directory and the seed:

sed -E 's|/runs/[0-9]{8}_[0-9]+/|/runs/<run>/|g; s/seed=[0-9]+/seed=<seed>/' regr/test1.log | head -3

Run directory: /runs/20261012_1
seed=<seed>
UVM_INFO /proj/tb/base_test.sv(88) @ 0: starting

Read that carefully: the seed was masked and the run directory was not. Line 2 is seed=<seed>; line 1 still says /runs/20261012_1.

A partial normalisation is the worst outcome of the three possible ones — it looks like it worked. Isolating which half fired takes two commands:

sed -E 's|/runs/[0-9]{8}_[0-9]+/|/runs/<run>/|g' regr/test1.log | grep -c '<run>'

0

sed -E 's/seed=[0-9]+/seed=<seed>/' regr/test1.log | grep -c '<seed>'

1

Zero and one. The seed substitution is fine; the run-directory one matches nothing.

The reason is the trailing slash in /runs/[0-9]{8}_[0-9]+/. The log line is:

grep -n '/runs/' regr/test1.log

1:Run directory: /runs/20261012_1

The path ends at _1 with nothing after it, so there is no trailing slash to match. Drop that requirement and both substitutions fire:

sed -E 's|/runs/[0-9]{8}_[0-9]+|/runs/<run>|g; s/seed=[0-9]+/seed=<seed>/' regr/test1.log | head -3

Run directory: /runs/<run>
seed=<seed>
UVM_INFO /proj/tb/base_test.sv(88) @ 0: starting

This is Chapter 9.1's trailing-slash lesson arriving in a different costume. There it left a filelist half-migrated; here it leaves a diff noisy, which is subtler — the pipeline runs, produces output, and simply fails to remove the noise it exists to remove.

The check is the same one as Chapter 10.2. Count before and after:

   grep -c '/runs/' regr/test1.log        before: 1
   … | grep -c '/runs/<run>'              after:  1

If the after-count is zero, the mask did nothing.

6. Step 5 — What the Normalisation Is Worth

Here is the payoff, measured. Two logs from two runs of the same test, compared raw:

diff -u regr/test1.log regr/test2.log

--- regr/test1.log	2026-10-12 21:58:26
+++ regr/test2.log	2026-10-12 21:58:26
@@ -1,7 +1,7 @@
-Run directory: /runs/20261012_1
-seed=481
+Run directory: /runs/20261012_2
+seed=482
 UVM_INFO /proj/tb/base_test.sv(88) @ 0: starting
-UVM_ERROR /proj/tb/scoreboard.sv(142) @ 981: data mismatch
+UVM_ERROR /proj/tb/scoreboard.sv(142) @ 982: data mismatch
 Summary
   UVM_ERROR : 1
   UVM_FATAL : 0

Three changed lines — and every one of them is noise. Different run directory, different seed, a timestamp one nanosecond apart. The two runs did the same thing and hit the same error.

Now the same two logs through the full pipeline:

for f in regr/test1.log regr/test2.log; do
    sed -n '1,/Summary/p' "$f" \
      | sed -E 's|/runs/[0-9]{8}_[0-9]+|/runs/<run>|g; s/seed=[0-9]+/seed=<seed>/; s/@ [0-9]+:/@ <time>:/' \
      > "$f.norm"
done
diff -u regr/test1.log.norm regr/test2.log.norm && echo 'IDENTICAL after normalisation'

IDENTICAL after normalisation

diff found nothing, exited 0, and the && fired. Two logs that differed on three lines are now byte-identical, because none of those three differences meant anything.

That is the whole argument. Without normalisation, a regression of two hundred tests produces two hundred noisy diffs and the real change is somewhere inside them. With it, the tests that are genuinely different are the only ones that appear.

7. Think It Through

1. sed -n '1,/Summary/p' logs/*.log on a 200-test regression. What do you get?

One test's section. The files are one stream, so the address 1 matches only the first line of the first file and the range closes at the first Summary it meets. Exit 0, output that looks like a valid extraction. Section 4 measured 5 lines where the loop gave 10.

2. Why not just use -s?

Because it is GNU-only and this track's reference sed rejects it with illegal option -- s, exit 1. That failure is at least loud, so a GNU-only script breaks visibly rather than silently — but the shell loop works everywhere and costs one line.

3. Your normalised diffs are still noisy. Where do you look?

At what you are masking versus what actually varies. Section 5 measured a run-directory pattern that required a trailing slash the log did not have, so it matched nothing while the command exited 0. Run grep -c with each mask pattern against a real log: a before-count of zero means that mask is doing nothing.

4. Which direction does the trailing-slash error fail in, and why is that worse than the opposite?

It fails toward not matching. A pattern that is too specific silently does nothing; a pattern that is too loose at least changes something you can see in the diff. Chapter 9.1 had the same character cause the opposite problem — there the unanchored version matched too much. The asymmetry is that over-matching shows up and under-matching does not.

5. Should the normalised logs replace the originals?

No. Normalisation is lossy by design — the masked seed and time are exactly what you need when a diff finally shows a real change. Write .norm copies as a derived artifact and leave the tool's output alone. This is one of the few sed pipelines where -i is simply the wrong tool.

6. A regression's logs live in 200 directories with generated names. How do you build the file list?

find runs -name '*.log' -print0 | xargs -0 -n1 echo to read it first, then feed the same find to the loop — Chapter 8.3's procedure. A glob is both unsafe with unusual names and liable to exceed the shell's argument limit at a few thousand entries, which a find pipeline does not.

8. Summary

• At regression scale the question is "what changed", not "what happened" — so the goal is a canonical form that diff can compare.
• A per-file operation needs per-file invocation. sed -n '1,/Summary/p' logs/*.log gave 5 lines from two files; the shell loop gave 10. The files are one stream, the range fires once, and exit status is 0.
• GNU's -s fixes it and does not exist here — illegal option -- s, exit 1. Loud, which makes it the safer of the two wrong answers.
• The portable form is for f in …; do sed … "$f"; done, and for a real regression build the list with find … -print0 rather than a glob.
• The registered run-directory mask required a trailing slash the log does not have, so it matched nothing while exiting 0 — Chapter 9.1's lesson, now costing a noisy diff instead of a broken filelist.
• Check every mask with grep -c before and after. A before-count of zero means the mask is a no-op.
• The payoff, measured: two logs with a 3-line raw diff — run directory, seed, timestamp — came out byte-identical after normalisation. Nothing that differed meant anything.
• Normalisation is lossy on purpose. Write .norm copies and keep the originals; -i has no place here.

That closes Module 10. The simulation flow is covered end to end: the scripts that drive the tools, the logs they produce, and the regression that compares them. Module 11 — Synthesis and STA Automation moves to the implementation side — library and constraint paths that change every release, corner names that migrate, and timing reports whose columns move every time a number gains a digit.