Waveform, Log and Clean Targets

A flow that compiles and runs is not yet a flow anybody wants to use. When a test fails, the first thing an engineer asks for is the waveform, and the second is the log. When the disk fills up, the thing they reach for is clean. These three targets are what make the difference between a Makefile that technically works and one a team actually adopts.

They are also where the most dangerous bug in this whole module lives, and it is not in the waveform.

1. What a Run Produces

Chapter 5.3 ended with a run directory containing one log. A real run produces more than that, and the pieces have genuinely different natures:

• The log is written every time, whether the test passes or fails. It is the evidence.
• The verdict — did this test pass? — is not a file at all yet. Something has to decide it.
• The waveform is optional, and as §2 measures, expensive enough that it must be.
• The image is shared by every run and lives outside the run directories.

2. What a Waveform Actually Costs

"Waveform dumping is expensive" is repeated everywhere and almost never quantified. Here it is quantified, on the nine-signal counter from §5.3, running two million clock cycles with and without dumping. Four paired runs:

run 0: no dump  2037 ms | dumping  6337 ms | 3.11x | vcd 101M
run 1: no dump  2086 ms | dumping  7303 ms | 3.50x | vcd 101M
run 2: no dump  1916 ms | dumping  6547 ms | 3.42x | vcd 101M
run 3: no dump  2591 ms | dumping  6751 ms | 2.61x | vcd 101M

Dumping made the run between 2.6 and 3.5 times slower, and produced a 101 MB file — reported as 101 MB on all four dumping runs, and measured exactly once at 105,809,534 bytes — from a design with nine signals in it.

Sit with that last part. Nine signals. A real block has tens of thousands, a full chip-level testbench has millions, and a regression runs hundreds of tests overnight. A flow that dumps by default does not produce a slightly slower regression; it produces a regression that fills the disk and then fails for a reason that has nothing to do with the design.

3. Making the Dump Opt-In

There are two ways to make dumping optional, and the choice matters more than it looks.

The approach most flows use is a compile-time define. The testbench wraps its dump calls in `ifdef DUMP_VCD, and the flow builds a second image with that macro set. It works, but it means two images, two compiles, and a recompile the moment somebody wants waves for a test they already ran.

The better approach, where the tool supports it, keeps the decision at run time. Icarus does support it: $dumpfile accepts a string the testbench received as a plusarg, so one image can do both.

reg [8*64-1:0] test, vcd;

initial begin
  if (!$value$plusargs("TEST=%s", test)) test = "smoke";
  if (!$value$plusargs("SEED=%d", seed)) seed = 1;
  $display("TB: test=%0s seed=%0d", test, seed);

  if ($value$plusargs("VCD=%s", vcd)) begin
    $display("TB: dumping to %0s", vcd);
    $dumpfile(vcd);
    $dumpvars(0, tb_counter);
  end
  // ... the test body
end

The if is the whole mechanism. $value$plusargs returns false when the plusarg is absent, so no $dumpfile call happens and no file is created. Measured, without +VCD:

TB: test=smoke seed=1
TB: count=10
TB: PASSED

No VCD appeared in the directory at all. And with the plusarg:

TB: test=smoke seed=1
TB: dumping to wave.vcd
VCD info: dumpfile wave.vcd opened for output.
TB: count=10
TB: PASSED

A real 1,037-byte VCD containing nine signals, from the same image, with no recompile. That is the property worth having: a test fails in a regression, and you get waves for it by re-running one command, not by rebuilding the design.

4. The Waveform Target

The waveform is a file in the run directory, so it gets a file target, exactly like the log:

RUNDIR := $(BUILD)/$(TEST)_$(SEED)
RUNLOG := $(RUNDIR)/run.log
VCD    := $(RUNDIR)/dump.vcd

.PHONY: waves
waves: $(VCD)

$(VCD): $(SIMV)
	@mkdir -p $(@D)
	@echo "  [waves] test=$(TEST) seed=$(SEED) -> $@"
	@vvp $(SIMV) +TEST=$(TEST) +SEED=$(SEED) +VCD=$@ > $(RUNLOG)
	@echo "  [waves] $$(wc -c < $@) bytes, $$(grep -c '^\$$var' $@) signals"

  [waves] test=smoke seed=1 -> build/smoke_1/dump.vcd
  [waves]     1149 bytes, 9 signals

Three details in that recipe are deliberate.

+VCD=$@ uses the automatic variable, so the dump path is the target path by construction and the two can never disagree. Chapter 3.3 is where $@ was introduced; this is the kind of place it earns its keep.

The doubled $$ escapes Make, leaving a single $ for the shell. $$(wc -c < $@) is a shell command substitution that Make must not touch — Chapter 1.4 §8 covers the escaping rule, and '^\$$var' is the same rule applied inside a quoted pattern that needs a literal dollar.

Asking for waves re-runs the simulation, because the waveform did not exist. That is correct: the VCD is genuinely not there, and the only way to produce it is to simulate again. It writes over run.log in the same directory, which is also correct — it is the log of the run that just happened.

5. The Log, and Who Decides Pass or Fail

Now the important half of the chapter. The log is written — but who decides whether the test passed?

Not the simulator. Chapter 5.1 §4 measured this and it is worth re-confirming, because every instinct says otherwise. A run that printed TB: FAILED exits zero:

TB: test=smoke seed=1
TB: count=10
TB: FAILED
   vvp exit status = 0

The reflex at this point is to blame the pipe. If the recipe is vvp ... | tee $(RUNLOG), then the status Make sees belongs to tee, not vvp — a real hazard that Chapter 4.3 §4 covered, and set -o pipefail in §5 is the real fix for it. But it does not help here, and measuring that is what makes the point land:

   without pipefail: rc=0
   with pipefail:    rc=0

Both zero. The pipe was never the problem. vvp genuinely exits 0 on a failed test, because from the simulator's point of view nothing went wrong — it was asked to execute a testbench, and it executed it to completion. "The design misbehaved" is not a tool error.

So the verdict has to be read out of the log:

   f.log -> grep PASSED rc=1
   p.log -> grep PASSED rc=0

That is the contract the testbench and the Makefile share: the testbench promises to print exactly one agreed string on success, and the Makefile promises to look for it. It is a thin contract, and it is the one every real verification flow runs on.

6. Never Make the Log the Target

Here is the trap, and it is the most dangerous thing in Module 5.

The natural way to combine §5's check with 5.3's run target is to put both in one recipe, with the log as the target:

$(RUNLOG): $(SIMV)
	@mkdir -p $(@D)
	@echo "  [run] test=$(TEST) seed=$(SEED) -> $@"
	@vvp $(SIMV) +TEST=$(TEST) +SEED=$(SEED) $(EXTRA) > $@
	@grep -q "TB: PASSED" $@ || { echo "  [FAIL] see $@"; exit 1; }
	@echo "  [PASS] $@"

Run a failing test. It behaves exactly as you would hope:

  [compile] -> build/simv
  [run] test=smoke seed=1 -> build/smoke_1/run.log
  [FAIL] see build/smoke_1/run.log
make: *** [build/smoke_1/run.log] Error 1
    make rc=2

A clear failure and a non-zero exit. Now run the identical command again:

make: Nothing to be done for `run'.
    make rc=0

Make exited zero. The log file exists, it is newer than the image, so by Make's rules the target is up to date and there is nothing to do. The failure has been forgotten — and a regression script, a CI job, or a make run && make report chain reads that second status as success.

The fix that is almost right

Chapter 2.3 introduced the special target for exactly this class of problem. Add one line:

.DELETE_ON_ERROR:

Now a recipe that fails takes its target down with it:

  [run] test=smoke seed=1 -> build/smoke_1/run.log
  [FAIL] build/smoke_1/run.log
make: *** [build/smoke_1/run.log] Error 1
make: *** Deleting file `build/smoke_1/run.log'

The staleness is genuinely fixed — asking again re-runs and fails again, as it must. But read what Make deleted. It deleted the log. The one artifact that explains why the test failed is gone, and the engineer debugging it has to re-run to get it back — which on a real test is twenty minutes, and on an intermittent failure may not reproduce at all.

.DELETE_ON_ERROR is the right instinct and belongs in most Makefiles. It is simply the wrong tool for this particular target, because here the half-written output is the thing you want.

The fix that is right

Separate the two jobs. The log is evidence and should always survive. The verdict is what Make should track, so make the verdict its own file:

RUNDIR := $(BUILD)/$(TEST)_$(SEED)
RUNLOG := $(RUNDIR)/run.log
STATUS := $(RUNDIR)/passed.stamp

.PHONY: run
run: $(STATUS)

# The log is written, then judged. The TARGET is the stamp, not the log,
# so a failure leaves the log on disk to read and leaves no stamp behind.
$(STATUS): $(SIMV)
	@mkdir -p $(@D)
	@echo "  [run] test=$(TEST) seed=$(SEED) -> $(RUNLOG)"
	@vvp $(SIMV) +TEST=$(TEST) +SEED=$(SEED) $(EXTRA) > $(RUNLOG)
	@grep -q "TB: PASSED" $(RUNLOG) || { echo "  [FAIL] $(RUNLOG)"; exit 1; }
	@touch $@
	@echo "  [PASS] $(TEST) seed $(SEED)"

This is 5.3 §7's stamp-file idea reused, and the ordering is the point: touch $@ is the last line, so the stamp exists only if the grep before it succeeded. Measured, in sequence:

$ make run EXTRA=+FORCE_FAIL
  [compile] -> build/simv
  [run] test=smoke seed=1 -> build/smoke_1/run.log
  [FAIL] build/smoke_1/run.log
make: *** [build/smoke_1/passed.stamp] Error 1
   make rc=2
   log on disk?  YES
   stamp?        no
   log says:     TB: FAILED

$ make run EXTRA=+FORCE_FAIL
  [run] test=smoke seed=1 -> build/smoke_1/run.log
  [FAIL] build/smoke_1/run.log
make: *** [build/smoke_1/passed.stamp] Error 1
   make rc=2

$ make run
  [run] test=smoke seed=1 -> build/smoke_1/run.log
  [PASS] smoke seed 1
   make rc=0
   stamp now?    YES

$ make run
make: Nothing to be done for `run'.
   make rc=0

Every property we wanted, at once. The failing run keeps its log and writes no stamp. Asking again re-runs and fails again — the failure persists until the design is fixed, so no script can mistake it for success. Once it genuinely passes, the stamp appears and make run exits 0. And the run after that is cached, because a passed test against an unchanged image has nothing to redo.

The general rule is worth remembering past this chapter: a target should be the thing whose existence means success, not the thing produced along the way. Logs, intermediate files and dumps are side products. Give the verdict its own file and the dependency graph tells the truth.

7. Reading the Log Without Hunting For It

Per-run directories solved the overwriting problem in 5.3, at the cost of a path nobody wants to type. A tiny phony target fixes that:

.PHONY: log
log:
	@test -f $(RUNLOG) || { echo "no log for $(TEST) seed $(SEED)"; exit 1; }
	@cat $(RUNLOG)

TB: test=smoke seed=1
TB: count=10
TB: FAILED
tb/tb_counter.v:24: $finish called at 112000 (1ps)

make log TEST=axi_burst SEED=42 prints that run's log, because RUNLOG is built from the same variables the run used. The test -f guard matters: without it, cat on a missing file prints a confusing error about a path the user never typed, while the guard names the test and seed they actually asked for.

This target is phony and takes no prerequisites on purpose. It must never cause a simulation to run — someone typing make log wants to read, not to launch a twenty-minute job.

8. Clean, and the Character That Makes It Dangerous

A clean target has one job: remove everything the flow generated, and nothing a person wrote.

.PHONY: clean
clean:
	@rm -rf $(BUILD)
	@echo "  [clean] removed $(BUILD)/"

It works because 5.3's design put every generated thing under one directory. Measured, after a run and a waveform:

before: 4 generated files
  [clean] removed build/
survivors:
   Makefile
   filelists
   rtl
   tb

Four hand-maintained inputs, nothing else. That is the test of a good clean: run it, and what remains should be exactly what you would commit to git.

Now the dangerous part. Someone passes an empty value — by typo, or from a wrapper script with an unset variable:

make clean BUILD=

  [clean] removed /

That message is alarming, and the truth is more interesting than either panic or relief. Nothing was deleted. rm -rf $(BUILD) expanded to rm -rf with no operand at all, and measured directly, that exits 0 and removes nothing — a witness file in the directory survived untouched. The -f flag is what makes it silent; rm -r with no operand prints a usage message instead.

So the recipe was harmless. The echo lied, because $(BUILD)/ rendered as a bare /.

The reason to care is how close the harmless version sits to the catastrophic one. These two are what many real Makefiles contain, and with BUILD empty they expand like this — printed with echo, not executed:

would run: rm -rf /
would run: rm -rf /*

A trailing slash, or a trailing /*, turns a no-op into rm -rf /. The habit that prevents it costs two lines:

.PHONY: clean
clean:
	@test -n "$(BUILD)" || { echo "  [clean] BUILD is empty - refusing"; exit 1; }
	@test "$(BUILD)" != "/" || { echo "  [clean] BUILD is / - refusing"; exit 1; }
	@rm -rf $(BUILD)
	@echo "  [clean] removed $(BUILD)/"

$ make -f Makefile.guard clean
  [clean] removed build/

$ make -f Makefile.guard clean BUILD=
  [clean] BUILD is empty - refusing
make: *** [clean] Error 1

$ make -f Makefile.guard clean BUILD=/
  [clean] BUILD is / - refusing
make: *** [clean] Error 1

clean and distclean

A convention worth following: clean removes build output, distclean removes everything clean does plus generated files that are expensive or awkward to rebuild — generated filelists, dependency files, packaged archives.

.PHONY: clean distclean
clean:
	@test -n "$(BUILD)" || { echo "  [clean] BUILD is empty - refusing"; exit 1; }
	@rm -rf $(BUILD)
	@echo "  [clean] removed $(BUILD)/"

distclean: clean
	@rm -f $(BUILD).tar.gz
	@echo "  [distclean] done"

distclean: clean makes clean a prerequisite, so distclean always does strictly more. That ordering is free and it is the reason the convention survives: nobody has to remember which one is bigger.

9. Common Mistakes

Dumping waveforms by default. Measured at 2.6–3.5× slower and 101 MB from nine signals (§2). On a real regression that is the difference between finishing and not.

Making the waveform need its own compile. If the tool allows a run-time dump path, one image serves both and a failing test gets waves without a rebuild (§3).

Trusting the simulator's exit status. vvp exits 0 on a test that printed FAILED (§5).

Blaming the pipe. set -o pipefail is the right fix for | tee in general, but measured here it changes nothing, because the simulator itself returns 0 (§5).

Making the log the target. The worst bug in this module: a failed run leaves a log, and the next invocation exits 0 without running anything (§6).

Reaching for .DELETE_ON_ERROR on a log target. It fixes the staleness by deleting the evidence you need (§6).

touching the stamp before the check. Then a failure records itself as a pass. The touch goes last (§6).

Giving make log prerequisites. Someone asking to read a log should never trigger a simulation (§7).

An unguarded rm -rf $(VAR)/. With VAR empty that is rm -rf /, and Make expands an unset variable silently (§8).

Scattering generated files outside one directory. Then clean becomes a list of paths that goes stale, and the first thing it fails to delete is the thing you needed gone.

10. Check Your Understanding

Question 1. A Makefile has $(RUNLOG): $(SIMV) with a recipe that runs the simulation and then greps the log for TB: PASSED, failing the recipe if it is absent. The test fails. An engineer runs the same make run command a second time. State the exit status and explain it.

Question 2. Adding .DELETE_ON_ERROR: fixes Question 1's staleness. Why is it still the wrong fix here, and what should the target be instead?

Question 3. Why does touch $@ have to be the last line of the run recipe rather than the first?

Question 4. make clean BUILD= printed [clean] removed / and nothing was deleted. Explain why nothing was deleted, and give the one-character change to the recipe that would have made it catastrophic.

Question 5. The waves target re-runs a simulation even for a test that already passed. Is that a bug?

11. Exercise

Build the complete Module 5 flow and verify each behaviour. Every transcript is real output.

Step 1. Run a failing test. Confirm the log survives and no stamp is written:

make run EXTRA=+FORCE_FAIL

  [compile] -> build/simv
  [run] test=smoke seed=1 -> build/smoke_1/run.log
  [FAIL] build/smoke_1/run.log
make: *** [build/smoke_1/passed.stamp] Error 1

   log on disk?  YES
   stamp?        no
   log says:     TB: FAILED

Step 2. Ask again. The important assertion of the whole module — it must re-run, not report success:

make run EXTRA=+FORCE_FAIL
echo "rc=$?"

  [run] test=smoke seed=1 -> build/smoke_1/run.log
  [FAIL] build/smoke_1/run.log
make: *** [build/smoke_1/passed.stamp] Error 1
rc=2

Step 3. Read the log without typing the path:

make log

TB: test=smoke seed=1
TB: count=10
TB: FAILED
tb/tb_counter.v:24: $finish called at 112000 (1ps)

Step 4. Let it pass, and confirm the stamp appears:

make run

  [run] test=smoke seed=1 -> build/smoke_1/run.log
  [PASS] smoke seed 1

Step 5. Ask once more. A passed test against an unchanged image is cached:

make run

make: Nothing to be done for `run'.

Step 6. Ask for waves, and read the size the recipe reports:

make waves

  [waves] test=smoke seed=1 -> build/smoke_1/dump.vcd
  [waves]     1149 bytes, 9 signals

Step 7. Inspect the run directory, then clean and confirm only inputs remain:

find build -type f | sort
make clean
ls -1

build/simv
build/smoke_1/dump.vcd
build/smoke_1/passed.stamp
build/smoke_1/run.log
  [clean] removed build/
Makefile
filelists
rtl
tb

Then extend it, and check your work by measurement rather than by reading. Add the BUILD guard from §8 and prove it fires with make clean BUILD=. Then deliberately move touch $@ to the top of the run recipe, run a failing test twice, and confirm the second invocation now reports success — that is the bug the ordering prevents, and seeing it yourself is worth more than the rule. Put it back afterwards.

12. Remember

• Waveform dumping is measurably expensive. 2.6–3.5× slower and a 101 MB VCD from a nine-signal design, over four paired two-million-cycle runs.
• So dumping is opt-in, and asking for it must be one easy command rather than a reason to change the default.
• Prefer a run-time dump path where the tool allows it. Icarus accepts a plusarg string in $dumpfile, so one image serves both and a failing test gets waves with no recompile.
• Use $@ for the dump path, so the file written and the target claimed can never disagree.
• The simulator does not know whether the test passed. vvp exits 0 on a run that printed FAILED.
• set -o pipefail is the right fix for | tee, and it does not help here — measured identical with and without, because the simulator itself returns 0.
• The verdict comes from the log, via an agreed string the testbench promises to print.
• Never make the log the target. A failing run leaves a log that Make reads as up to date, and the next invocation exits 0 having run nothing — the most dangerous bug in this module.
• .DELETE_ON_ERROR fixes the staleness and destroys the evidence. Keep it in your Makefiles; do not point it at a log.
• Make the verdict its own file. A passed.stamp touched last means the target exists only if the test really passed, and the log always survives to be read.
• A target should be the thing whose existence means success, not a by-product of trying.
• Give make log no prerequisites, so reading can never launch a simulation.
• Guard anything that deletes. With BUILD unset, rm -rf $(BUILD) is a harmless no-op but rm -rf $(BUILD)/ is rm -rf /, and Make expands an unset variable without a word.
• clean removes build output; distclean: clean removes more. Making clean a prerequisite means nobody has to remember which is bigger.

Module 5 is complete. The flow compiles only when sources change, runs any test without rebuilding, keeps every run's results apart, produces waveforms on request, and tells the truth about pass and fail through an exit status a script can rely on. All of it for one test at a time.

That is the limit to break next. Verification does not run one test; it runs a list of tests across a range of seeds, overnight, and needs one answer at the end. Module 6 turns this single-run flow into a regression: selecting tests from the command line, expanding a test list into many runs, and collecting all those passed.stamp files back into a report a lead can read in ten seconds.