What a Single Run Leaves Behind

Part II · Module 28 · Chapter 28.2 · What a Single Run Leaves Behind

Chapter 28.1 established the categories. This chapter takes one run — a genuinely failing one, because that is the case where the artifacts matter — and goes through it file by file. The exercise is worth doing once in your own flow, because the sizes and the values turn out to be ordered oppositely, and that ordering is the entire basis of a retention policy.

1. The Inventory

A real failing run: an arbiter with one mutated case arm, compiled and simulated with dumping on, recorded the way Chapter 27.9 records.

$ grep -E 'ERROR|TEST' runs/fail_s1/run.log
TB  ERROR t=45 gnt=10 expected 01
TB  TEST FAILED  test=smoke checks=3 errors=1

$ find runs/fail_s1 -type f -exec stat -f '%z %N' {} \; | sort -rn \
>   | awk '{printf "  %8d  %s\n", $1, $2}'
     17022  runs/fail_s1/sim.vvp
      1465  runs/fail_s1/dump.vcd
       553  runs/fail_s1/env.txt
       256  runs/fail_s1/run.log
        95  runs/fail_s1/tool.txt
        70  runs/fail_s1/cmd.args
         7  runs/fail_s1/status.txt
         0  runs/fail_s1/compile.log
$ du -sk runs/fail_s1
44	runs/fail_s1

Eight files, 19,468 bytes of content, 44 KB on disk. The gap between those two numbers is block rounding and is Chapter 28.3's subject; here the point is the ordering.

And that is only the Icarus half. The nvc run in the same directory adds a library:

$ find runs/smoke_s1/work -type f | sort
runs/smoke_s1/work/WORK.ARB
runs/smoke_s1/work/WORK.ARB-RTL
runs/smoke_s1/work/WORK.TB_ARB
runs/smoke_s1/work/WORK.TB_ARB-SIM
runs/smoke_s1/work/WORK.TB_ARB.elab
runs/smoke_s1/work/_NVC_LIB
runs/smoke_s1/work/_index
$ du -sk runs/smoke_s1/work
28	runs/smoke_s1/work

Seven more files under names that are not documented anywhere a user reads, which is the practical argument from Chapter 27.3 restated as an inventory: you cannot write a retention rule per filename, because you do not know the filenames.

2. The Largest File Is the Cheapest to Lose

sim.vvp is 17,022 bytes — 39% of the directory — and Chapter 27.8 measured the command that produces it at 57 ms. The rebuild test from Chapter 28.1 passes immediately:

$ rm -f runs/fail_s1/sim.vvp
$ iverilog -o runs/fail_s1/sim.vvp -f filelists/sim.f
$ [ -s runs/fail_s1/sim.vvp ] && echo REBUILT
REBUILT

The same holds for the nvc library and for the dump. On a real design the proportions become extreme rather than merely lopsided: Chapter 27.6 measured a whole-run dump at 21,394,015 bytes for a four-signal design over 200,000 cycles. On a block with ten thousand signals and a million cycles, the dump is the only artifact whose size anybody discusses.

So the first rule of a retention policy writes itself, and it is the opposite of the intuitive one: the files worth deleting are the big ones, and they are worth deleting precisely because they are reproducible. Size and disposability are correlated, not opposed.

3. The Log Is Derived and Irreplaceable

run.log is 256 bytes and is the only thing in the directory that says what happened:

TB  test=smoke seed=1 cycles=40 verbosity=LOW
TB  ERROR t=45 gnt=10 expected 01
TB  TEST FAILED  test=smoke checks=3 errors=1

Technically it is derived — a tool wrote it, and re-running the command produces one. Practically it is irreplaceable, and the reason is worth stating precisely: re-running does not reproduce the run, it produces a new run. For a deterministic failure the new log will match; for a failure that depends on a seed you no longer have, or a tool version that has since changed, or a machine that is now configured differently, it will not. And the only way to find out is to still have the original to compare against.

There is a second reason, which Module 27 spent nine chapters establishing: the log is the verdict. The exit status was 0 on this failing run. If the log goes, the directory contains no record that anything was wrong.

4. The Smallest Files Are the Only Irreplaceable Ones

cmd.args is 70 bytes. tool.txt is 95. status.txt is 7. Together they are 0.9% of the directory and no tool can regenerate any of them, because they are not outputs of a computation — they are statements about an event. There is no command whose output is "the path vvp resolved to on the ninth of October".

This is where the chapter's two orderings meet. Sort the directory by size and you get sim.vvp first and status.txt last. Sort it by what a debug three weeks from now actually needs and the order very nearly reverses.

And the claim is testable rather than rhetorical. Chapter 28.5 runs the experiment in full; the result is worth previewing here because it is this chapter's thesis as a number:

archive of the record + the three sources it names:   4.0 KB, 10 entries
   ... then the entire run directory was deleted ...
reproduced elsewhere from the archive:
    TB  ERROR t=45 gnt=10 expected 01
    TB  TEST FAILED  test=smoke checks=3 errors=1
the original verdict, read back out of the archived log:
    TB  ERROR t=45 gnt=10 expected 01
    TB  TEST FAILED  test=smoke checks=3 errors=1

4 KB preserved everything a 60 MB archive would have, because the large artifacts were reproducible and the small ones were not.

5. Asking the Question Per File

The useful form of all this is a single question applied to every artifact, with only three possible answers:

#!/bin/bash
# inventory RUNDIR — every artifact, its size, and its retention class.
set -u
rundir=${1:?usage: inventory RUNDIR}
[ -d "$rundir" ] || { echo "inventory: no such directory: $rundir" >&2; exit 2; }

printf '%10s  %-9s %-28s %s\n' BYTES CLASS FILE WHY
find "$rundir" -type f | sort | while IFS= read -r f; do
  b=$(stat -f '%z' "$f")
  case "${f##*/}" in
    (cmd.args|cmd.sh|tool.txt|env.txt|context.txt|status.txt|config.txt)
        cls=KEEP;  why="describes the event; nothing can regenerate it" ;;
    (*.log|stages.txt)
        cls=KEEP;  why="the verdict and the evidence live here" ;;
    (*.vcd|*.fst|*.fsdb|*.shm)
        cls=DROP;  why="reproducible by re-running with dumping on" ;;
    (*.vvp|*.so|*.o|*.a|simv|*.daidir)
        cls=DROP;  why="rebuildable from source in seconds" ;;
    (*) cls=ASK;   why="unclassified -- add a rule before any cleanup" ;;
  esac
  printf '%10s  %-9s %-28s %s\n' "$b" "$cls" "${f#$rundir/}" "$why"
done

$ inventory runs/fail_s1
     BYTES  CLASS     FILE                         WHY
        70  KEEP      cmd.args                     describes the event; nothing can regenerate it
         0  KEEP      compile.log                  the verdict and the evidence live here
      1465  DROP      dump.vcd                     reproducible by re-running with dumping on
       553  KEEP      env.txt                      describes the event; nothing can regenerate it
       256  KEEP      run.log                      the verdict and the evidence live here
     17022  DROP      sim.vvp                      rebuildable from source in seconds
         7  KEEP      status.txt                   describes the event; nothing can regenerate it
        95  KEEP      tool.txt                     describes the event; nothing can regenerate it

Three design points, each of which is a decision rather than a detail.

ASK exists, and it is the whole safety mechanism. It is Chapter 28.1's UNCLASSIFIED under another name. Run this against a commercial flow and the ASK rows will be the majority on the first attempt — csrc/, simv.daidir/, ucli.key, vc_hdrs.h, .vlogansetup.args — and each one is a rule somebody has to write. A classifier with no ASK has to guess, and the guesses that go wrong go wrong permanently.

The patterns match the basename, not the path. ${f##*/} strips the directory, so a run.log keeps its classification wherever a tool decides to put it. Matching on the full path is how a rule stops working when a tool changes its layout.

Nothing here deletes anything. It prints a classification, and the deletion is Chapter 28.4's job, under its own guards. Keeping classification and deletion in separate programs means the classification can be reviewed, diffed and argued about before anything is irreversible — which is the same separation Chapter 27.2's dry-run mode exists for.

6. What Changes on a Real Design

Three things scale and one does not, which is the whole reason this inventory is worth doing on a tiny fixture first.

The dump scales with signals × cycles, so it moves from kilobytes to gigabytes and dominates everything. Chapter 27.6's windowing measurement — 927× smaller — is the lever.

The compiled artifact scales with design size and becomes hundreds of megabytes for a full chip. Still reproducible, still the cheapest thing in the directory to lose, now worth real money in disk.

The log scales with verbosity and run length, and a high-verbosity UVM run of a long test produces a log large enough to matter. This is the one category where the scaling creates genuine tension, because the log is both large and irreplaceable — and the resolution is the one Chapter 27.5 already set out: run at low verbosity by default and raise it only on a rerun of something that already failed.

The record does not scale at all. cmd.args, tool.txt, env.txt and status.txt are the same few hundred bytes for a four-signal arbiter and for a full chip. They are a rounding error in any retention budget and the only files in the directory with no alternative source.

So the policy that falls out is short enough to state in one line, and the rest of Module 28 is its implementation: keep the record and the logs forever, keep the dumps as long as the disk allows, and delete the rebuildable artifacts as soon as the run is no longer current.

Practice

1. Inventory one run directory from your own flow, sorted by size, and count the files you cannot name the purpose of.
2. Find your tool's library or intermediate directory and list its filenames. Decide whether you could have written a glob for them.
3. Count the zero-byte files and determine, for each, whether the stage that wrote it succeeded.
4. Delete the largest artifact and rebuild it. Time the rebuild.
5. Then try to rebuild the log and explain precisely why a rerun is not the same thing.
6. Check the exit status of your most recent failing run, and decide what would remain if the log were deleted.
7. Sum the sizes of the record files and express them as a percentage of the directory.
8. Write inventory with its three classes, and run it on a run directory.
9. Run it against a commercial flow's output and count the ASK rows. Write a rule for the largest one.
10. Change the patterns to match full paths instead of basenames, move a log, and watch the classification break.
11. Estimate each category's size on your real design and decide which of the four is the one your retention policy is actually about.

Check Your Understanding

1. Eight files held 19,468 bytes of content and occupied 44 KB. What accounts for the difference?
2. Why can a retention rule not be written per filename?
3. Which file was the largest, and how long does it take to rebuild?
4. Why are size and disposability correlated rather than opposed?
5. In what sense is a log derived, and in what sense is it irreplaceable?
6. Why is "re-running produces the same log" false in general?
7. What would a find runs -size 0 -delete rule destroy, and what distinction would it make unrecoverable?
8. Why can no tool regenerate tool.txt?
9. What did the 4 KB archive demonstrate?
10. Why does inventory match the basename rather than the path?
11. Why are classification and deletion separate programs?
12. Of the four categories, which does not scale with design size?

7. Remember

• Inventory a run once, in your own flow. Eight files, 19.5 KB of content, 44 KB on disk, plus a seven-file library under names nobody documents.
• Size and value run in opposite directions. The 17 KB executable rebuilds in 57 ms; the 70-byte record cannot be rebuilt at all.
• The big artifacts are the deletable ones, because being computed is what makes them reproducible.
• A log is derived and irreplaceable: a rerun is a different run, and the log — not the exit status — carries the verdict.
• Do not delete zero-byte logs. They come from successful stages, and removing them makes a missing log ambiguous.
• The record does not scale. It is a rounding error in any budget and the only thing with no alternative source.
• Keep an ASK class. On a commercial flow it will be the majority on the first attempt, and every row is a rule somebody must write.
• Match basenames, not paths, so a rule survives a tool changing its layout.
• Classify in one program and delete in another, so the classification can be reviewed before anything is irreversible.

8. What Is Next

The inventory is per run, and the problem is per filesystem. Chapter 28.3 scales up to the question an engineer actually arrives at — the disk is full — and finds the case where du and df disagree by 400 MB and both are right.