Run Directories: Running a Tool Where Its Output Belongs

Part II · Module 27 · Chapter 27.3 · Run Directories: Running a Tool Where Its Output Belongs

A simulator does not ask where to put its output. It writes dump.vcd, or csrc/, or ucli.key, or simv.daidir, into the current working directory, under a name the tool chose and you did not. Chapter 27.2 got the arguments right; this chapter is about the one argument that is never on the command line — the directory the command runs in.

1. The Output Names Are Not Yours

Run the fixture's simulation and look at what appears next to you:

$ vvp build.vvp +TESTNAME=smoke +DUMP
VCD info: dumpfile dump.vcd opened for output.
TB  dumping dump.vcd depth=0 (whole run)
TB  TEST PASSED  test=smoke checks=3 errors=0
$ ls
build.vvp  dump.vcd

dump.vcd is there because the testbench called $dumpfile("dump.vcd"). The name is in the source, not on the command line — so two runs of the same testbench want the same filename by construction, and no amount of care with arguments changes that. Commercial tools make this more pronounced, because they write several such artifacts per run under fixed names.

There are two ways to keep two runs apart. You can give every artifact a unique name, or you can give every run a unique directory. The second is better for a reason that has nothing to do with taste: it works for artifacts you did not know the tool would write.

2. Four Runs, One Directory

Four runs of the same testbench with four different seeds, started concurrently, each with its own log name but all dumping to dump.vcd:

$ cd flat
$ for s in 1 2 3 4; do
>   ( vvp build.vvp +TESTNAME=random +CYCLES=20000 +SEED=$s \
>         +DUMP +DUMP_FILE=dump.vcd > run_$s.log 2>&1 ) &
> done
$ wait
$ for s in 1 2 3 4; do grep -E 'TEST (PASSED|FAILED)' run_$s.log; done
TB  TEST PASSED  test=random checks=5013 errors=0
TB  TEST PASSED  test=random checks=5032 errors=0
TB  TEST PASSED  test=random checks=4874 errors=0
TB  TEST PASSED  test=random checks=5080 errors=0

Four runs, four logs, four different check counts, four passes. The logs are fine, because each had a distinct name. Now the dump:

$ ls dump.vcd | wc -l
       1
$ wc -c < dump.vcd
 2011955

One file, where four were written. That much is unsurprising. What matters is which run's data is in it.

3. The File Has the Right Size and Belongs to Nobody

Run the same four seeds again, this time each in its own directory, and compare:

$ for s in 1 2 3 4; do mkdir -p runs/seed_$s; done
$ for s in 1 2 3 4; do
>   ( cd runs/seed_$s && vvp ../../build.vvp +TESTNAME=random +CYCLES=20000 \
>        +SEED=$s +DUMP +DUMP_FILE=dump.vcd > run.log 2>&1 ) &
> done
$ wait
$ md5 -q runs/seed_*/dump.vcd
de523f25a589e8aa2dcabee9928994ae      # seed_1, 2011955 bytes
6f8db9a83a4fb16b187116660eda796d      # seed_2, 2011934 bytes
056189455515d116cb7b571161b3281c      # seed_3, 2002480 bytes
80bf2759afd4d376aebaaecbd7a93b46      # seed_4, 2011321 bytes

Four distinct dumps, four distinct sizes. And the collided file from the previous section:

$ md5 -q flat/dump.vcd
c6850cdc4a00b2f1cfc5e411332067e0

That hash matches none of the four. The file is not seed 1's dump with the others lost; it is the four runs' writes interleaved into one file. And look at its size again: 2,011,955 bytes — byte for byte the size of seed 1's complete dump. A size check passes. A non-empty check passes. An extension check passes.

Then the structure:

$ printf 'collided: $date=%s $enddefinitions=%s $var=%s\n' \
>   "$(grep -c '^\$date' flat/dump.vcd)" \
>   "$(grep -c enddefinitions flat/dump.vcd)" \
>   "$(grep -c '^\$var' flat/dump.vcd)"
collided: $date=1 $enddefinitions=1 $var=21
$ printf 'seed_1  : $date=%s $enddefinitions=%s $var=%s\n' \
>   "$(grep -c '^\$date' runs/seed_1/dump.vcd)" \
>   "$(grep -c enddefinitions runs/seed_1/dump.vcd)" \
>   "$(grep -c '^\$var' runs/seed_1/dump.vcd)"
seed_1  : $date=1 $enddefinitions=1 $var=21

Identical header counts. One $date, one $enddefinitions, twenty-one $var declarations — a structurally well-formed VCD file.

And the metadata inside the file cannot help you either. The only identifying field a VCD carries is the date:

$ grep -A1 '^\$date' flat/dump.vcd
$date
	Fri Oct  9 02:37:05 2026

A wall-clock timestamp. It does not name a seed, a test, a command or a directory. Whatever the run was, the file does not know.

4. Making the Directory, and Entering It Correctly

The mechanism is three lines, and each has a reason.

rundir="$RUNS/${test}_s${seed}"
mkdir -p "$rundir" || exit 2
( cd "$rundir" && exec vvp "$simexe" "${rargs[@]}" ) > "$rundir/run.log" 2>&1

mkdir -p with its status checked. -p makes parents and does not complain if the directory exists, which is what you want for a rerun. The || exit 2 matters because a failed mkdir followed by a cd that also fails leaves the tool running in the wrong directory — the one case where losing the status is actively dangerous rather than merely untidy.

The subshell. ( cd … && … ) changes directory inside a subshell, so the parent's working directory is untouched. Without the parentheses the script's own cd persists and every later relative path in the script means something different — a defect that appears several commands later and looks unrelated.

cd … && exec, not cd … ; exec. With ; the command runs even if the cd failed, in whatever directory the script happened to be in. This is the specific mistake that writes a run's output into the source tree.

There is one interaction with Module 26 that bites immediately, and it bit my own wrapper:

$ cat runs/icarus_smoke_s1/compile.log
rtl/arb.v: No such file or directory
No top level modules, and no -s option.

The filelist contains relative source paths, and Chapter 26.6 measured that a tool resolves those against the current directory. Moving the compile into the run directory moved the base of every path in the filelist. The fix is to keep the compile where the paths resolve and send only the output into the run directory:

# compile at the repository root, where the filelist's paths resolve;
# only the artifact goes to the run directory
( cd "$root" && iverilog -o "$rundir/sim.vvp" -f "$root/filelists/sim.f" ) \
    > "$rundir/compile.log" 2>&1
# run in the run directory, where the tool's self-named output belongs
( cd "$rundir" && exec vvp sim.vvp "${rargs[@]}" ) > "$rundir/run.log" 2>&1

Two stages, two working directories, chosen for two different reasons. This is the normal arrangement in a real flow and it is worth stating plainly: the directory a stage runs in is part of that stage's interface, and the right answer can differ per stage.

5. Naming the Directory

A run directory's name is the first thing anybody reads, so it should carry the identity of the run and nothing else:

# the axes that distinguish one run from another, in a stable order
rundir="$RUNS/${tool}_${test}_s${seed}"

Three rules, each learned by having the alternative:

Encode every axis you vary. If you sweep seeds, the seed is in the name. A directory called random tells you nothing when there are twenty of them.

Do not put the timestamp in the name when the axes already identify the run. A timestamp makes every rerun a new directory, so the disk grows without bound and two runs of the same configuration cannot be compared by path. Keep the time inside the directory, which Chapter 27.9 does.

Keep it sortable and greppable. _s7 sorts and matches cleanly; (seed 7) does neither, and a space in a run directory name will find every unquoted expansion in your flow — which is a real way to discover them, but not a cheap one.

6. Listing What You Have

Once every run has a directory, the question becomes which one to look at. That is a one-screen report:

#!/bin/bash
# runs [DIR] — one line per run directory: verdict, dump, size, when.
set -u
base=${1:-runs}
[ -d "$base" ] || { echo "runs: no such directory: $base" >&2; exit 2; }
printf '%-26s %-11s %-12s %10s  %s\n' RUN VERDICT DUMP BYTES WHEN
for d in "$base"/*/; do
  [ -d "$d" ] || continue
  name=$(basename "$d")
  log=""; for c in "$d/run.log" "$d"/*.log; do [ -r "$c" ] && { log=$c; break; }; done
  if   [ -z "$log" ];                      then verdict=NO-LOG
  elif grep -q 'TEST PASSED' "$log";       then verdict=PASS
  elif grep -q 'TEST FAILED' "$log";       then verdict=FAIL
  else                                          verdict=NO-VERDICT; fi
  dump=$(ls "$d"*.vcd "$d"*.fst 2>/dev/null | head -1)
  if [ -n "$dump" ]; then dn=$(basename "$dump"); db=$(wc -c < "$dump" | tr -d ' ')
  else dn=-; db=0; fi
  when=$( [ -n "$log" ] && date -r "$log" '+%Y-%m-%d %H:%M:%S' || echo - )
  printf '%-26s %-11s %-12s %10s  %s\n' "$name" "$verdict" "$dn" "$db" "$when"
done

$ runs runs
RUN                        VERDICT     DUMP              BYTES  WHEN
icarus_bogus_s1            FAIL        dump.vcd           1291  2026-10-09 02:45:34
icarus_fairness_s1         PASS        dump.vcd           1677  2026-10-09 02:45:34
icarus_smoke_s1            PASS        dump.vcd           1470  2026-10-09 02:45:34
nvc_bogus_s1               FAIL        dump.fst            537  2026-10-09 02:45:35
nvc_fairness_s1            PASS        dump.fst            554  2026-10-09 02:45:35
nvc_smoke_s1               PASS        dump.fst            549  2026-10-09 02:45:35

Note the fourth verdict the script can produce. NO-VERDICT is not a stylistic flourish: it is the case where a log exists and contains neither a pass nor a fail line, which is what a killed or hung run leaves behind. A classifier with only PASS and FAIL has to put that run in one of the two, and whichever it chooses is a lie. The three-state rule comes from Chapter 24.6 and survives every later use.

Practice

1. Run the fixture's simulation twice in the same directory with dumping on, and confirm the second overwrites the first with no message.
2. Run four concurrent seeds in one directory with distinct log names, and confirm all four logs report PASSED.
3. Hash the single surviving dump and compare it against four isolated dumps of the same seeds. Confirm it matches none.
4. Compare its size against the four. Note which one it equals.
5. Count $date, $enddefinitions and $var in the collided file and in a good one. Confirm they are identical.
6. Find the only identifying field inside a VCD and decide whether it can tell you which run wrote the file.
7. Write the three-line run-directory idiom and prove the parent shell's working directory is unchanged afterwards.
8. Replace && with ; between cd and the tool, make the cd fail, and find where the output landed.
9. Reproduce the filelist base-directory collision by compiling from inside the run directory, and then fix it by compiling at the root.
10. Write the runs lister and confirm it produces four distinct states, including NO-VERDICT for a run you kill.
11. Put a space in a run directory name and count how many places in your own flow break. Then decide whether to keep the space or the flow.

Check Your Understanding

1. Why does giving artifacts unique names not solve the problem that unique directories solve?
2. Why did the four logs survive the shared directory when the four dumps did not?
3. What three properties did the collided dump have that a good dump has?
4. Which run's data was in the collided file?
5. What was its size, and why does that matter?
6. What is the only identifying metadata in a VCD, and what does it identify?
7. Why must mkdir -p's exit status be checked here specifically?
8. What does the subshell around cd protect, and what breaks without it?
9. Why can the correct working directory differ between the compile stage and the run stage?
10. Why does a run lister need a fourth state beyond PASS, FAIL and no log?

7. Remember

• The tool names its own output from inside the source, so two runs want the same filenames by construction.
• A unique directory beats unique names because it covers artifacts you did not know about.
• Four concurrent runs in one directory produced one dump of exactly the right size, with a valid header, matching no run. Every check you could run on the file passes.
• The logs survived and said PASSED, which is what makes the collision hard to see.
• A VCD's only identifying field is a timestamp. The file cannot tell you which run wrote it.
• mkdir -p "$d" || exit 2, then ( cd "$d" && exec … ) — the subshell protects the parent, and && stops the tool from running in the wrong place.
• The working directory is part of a stage's interface, and the right one can differ per stage because filelist paths resolve against the cwd.
• Keep the run's axes in the directory name and the timestamp inside it.

Chapter 28.2 inventories one of these directories file by file and finds the sizes and the values ordered oppositely.

8. What Is Next

Every run has its own directory — and Chapter 31.6 measures what four parallel runs do to each other when that is not true: one surviving log and two passing tests recorded as failures. Chapter 27.4 fills in what varies between them — the test name, the seed, the options a testbench reads at run time — and measures what happens when one of those names is misspelled.