Waveform Dumping On Request

Part II · Module 25 · Chapter 25.5 · Waveform Dumping On Request

1. Why Off by Default

REGRESSION = [("axi_smoke", "PASS"), ("axi_reset", "PASS"), ("axi_burst", "FAIL"),
              ("axi_soak", "PASS"), ("axi_hang", "NO_FINISH"), ("axi_cfg", "PASS")]

waves_always = len(REGRESSION)
waves_on_need = sum(1 for _, s in REGRESSION if s != "PASS")

print(f"{'test':<11} {'status':<10} waves worth keeping?")
for test, status in REGRESSION:
    print(f"{test:<11} {status:<10} {status != 'PASS'}")

print()
print("waves if always on :", waves_always)
print("waves anybody opens:", waves_on_need)
print("useless waveforms  :", waves_always - waves_on_need,
      f"of {waves_always}")

test        status     waves worth keeping?
axi_smoke   PASS       False
axi_reset   PASS       False
axi_burst   FAIL       True
axi_soak    PASS       False
axi_hang    NO_FINISH  True
axi_cfg     PASS       False

waves if always on : 6
waves anybody opens: 2
useless waveforms  : 4 of 6

Two thirds of the waveforms a waves-always regression produces are never opened. They are also, per Chapter 24.4 Section 4, the largest artefact a run makes — so the wasted fraction is the dominant term in both the disk bill and the runtime.

The runtime cost is the less obvious half. Dumping every signal makes a simulation meaningfully slower, which means a waves-always nightly finishes later and tests fewer seeds in the same window. The waveform you did not need cost you the coverage you did.

So: off by default, on when asked. Chapter 25.1's waves=False keyword-only parameter is that decision, and the rest of this chapter is what the flag actually has to do.

2. The Run-Step Half

if waves:
    cmd += ["+fsdb+all", f"+fsdbfile={fsdb_path}"]

from pathlib import Path


def run_command(binary, test, seed, *, waves=False, run_dir="."):
    cmd = [binary, f"+UVM_TESTNAME={test}", f"+ntb_random_seed={seed}"]
    if waves:
        fsdb_path = Path(run_dir) / f"waves_{test}_{seed}.fsdb"
        cmd += ["+fsdb+all", f"+fsdbfile={fsdb_path}"]
    return cmd


for waves in (False, True):
    cmd = run_command("./simv", "axi_burst", 1839274,
                      waves=waves, run_dir="runs/axi_burst_1839274")
    print(f"waves={waves!s:<6} {len(cmd)} args")
    for part in cmd:
        print("      " + part)
    print()

waves=False  3 args
      ./simv
      +UVM_TESTNAME=axi_burst
      +ntb_random_seed=1839274

waves=True   5 args
      ./simv
      +UVM_TESTNAME=axi_burst
      +ntb_random_seed=1839274
      +fsdb+all
      +fsdbfile=runs/axi_burst_1839274/waves_axi_burst_1839274.fsdb

Two things about that path are deliberate, and both are rules from earlier chapters doing their job.

The filename carries the test and the seed. Chapter 24.3 Section 2 required it of the run directory and Chapter 24.4 Section 2 showed what a keep-list does when it assumes a constant name. A waveform called waves.fsdb is fine until two runs share a directory, and then one silently overwrites the other.

It is inside the run directory, not scratch. Chapter 24.3 Section 1 classified scratch as the discard half; a waveform on a failing run is the one large artefact worth keeping, with Chapter 24.4's seven-day horizon. Putting it in scratch would delete the very thing waves were enabled for.

+fsdb+all is Verdi's spelling and +fsdbfile= names the output. Questa writes .wlf by default and Verilator writes .vcd with --trace, so like everything vendor-shaped this belongs in Chapter 23.11's adapter rather than in the runner.

3. The Flag Does Not Stay in the Run Step

VISIBILITY = {
    "vcs": ("compile", "-debug_access+all"),
    "xrun": ("compile", "-access +rwc"),
    "vsim": ("compile", "-voptargs=+acc"),
    "verilator": ("compile", "--trace"),
}

print(f"{'simulator':<11} {'step':<9} flag")
for sim, (step, flag) in VISIBILITY.items():
    print(f"{sim:<11} {step:<9} {flag}")

print()
print("every one of them is a COMPILE flag:",
      all(step == "compile" for step, _ in VISIBILITY.values()))
print()
print("so asking for waves changes:")
print("   the run command   :", "+fsdb+all, +fsdbfile=")
print("   the compile command:", "-debug_access+all")
print("   therefore the binary:", True)

simulator   step      flag
vcs         compile   -debug_access+all
xrun        compile   -access +rwc
vsim        compile   -voptargs=+acc
verilator   compile   --trace

every one of them is a COMPILE flag: True

so asking for waves changes:
   the run command   : +fsdb+all, +fsdbfile=
   the compile command: -debug_access+all
   therefore the binary: True

A waveform needs visibility compiled in. By default a simulator optimises away the internal nets nobody asked to observe, and no run-time plusarg can bring back a signal that is not in the binary.

The failure mode is the characteristic one for this whole module:

SCENARIOS = [
    ("compiled with debug, run with +fsdb+all", True, True, "a full waveform"),
    ("compiled with debug, run without", True, False, "no waveform — correct, none asked for"),
    ("compiled WITHOUT debug, run with +fsdb+all", False, True, "an empty or near-empty file"),
    ("compiled WITHOUT debug, run without", False, False, "no waveform"),
]

print(f"{'scenario':<44} {'error?':<8} result")
for what, compiled, ran, outcome in SCENARIOS:
    errored = False
    print(f"{what:<44} {str(errored):<8} {outcome}")

print()
print("the third row is the dangerous one:")
print("   the run passes, the file exists, and it has nothing in it")
print("   somebody opens it in Verdi and concludes the testbench is broken")

scenario                                     error?   result
compiled with debug, run with +fsdb+all      False    a full waveform
compiled with debug, run without             False    no waveform — correct, none asked for
compiled WITHOUT debug, run with +fsdb+all   False    an empty or near-empty file
compiled WITHOUT debug, run without          False    no waveform

the third row is the dangerous one:
   the run passes, the file exists, and it has nothing in it
   somebody opens it in Verdi and concludes the testbench is broken

No error on any row. The third produces a file that exists, has a plausible name, and contains nothing useful — which is worse than no file, because a missing file is a question and an empty one is a wrong answer.

4. Therefore It Is in the Compile Key

import hashlib

SOURCES = ["core/alu.sv", "core/fifo.sv"]


def compile_key(sources, defines, sim, waves):
    canonical = "\n".join([sim, f"waves={int(waves)}",
                           *sorted(f"{k}={v}" for k, v in defines.items()),
                           *sorted(sources)])
    return hashlib.sha256(canonical.encode("utf-8")).hexdigest()[:12]


def bad_key(sources, defines, sim, waves):
    """The one that reads as obviously fine. waves is 'a run-time thing'."""
    canonical = "\n".join([sim, *sorted(f"{k}={v}" for k, v in defines.items()),
                           *sorted(sources)])
    return hashlib.sha256(canonical.encode("utf-8")).hexdigest()[:12]


for name, fn in (("with waves in the key", compile_key), ("without", bad_key)):
    off = fn(SOURCES, {}, "vcs", False)
    on = fn(SOURCES, {}, "vcs", True)
    print(f"{name:<22} waves=False -> {off}  waves=True -> {on}  differ={off != on}")

print()
print("with the bad key, the reproduce-with-waves run:")
print("   finds a cached binary          : True")
print("   that binary has no debug access: True")
print("   produces                       : an empty waveform")
print("   reports                        : nothing wrong")

with waves in the key  waves=False -> 4a2936ef9228  waves=True -> 51069367f626  differ=True
without                waves=False -> 41e1d57b222e  waves=True -> 41e1d57b222e  differ=False

with the bad key, the reproduce-with-waves run:
   finds a cached binary          : True
   that binary has no debug access: True
   produces                       : an empty waveform
   reports                        : nothing wrong

This is exactly the failure Chapter 25.3 Section 7 named — anything that changes the binary and is not in the key is a stale-binary bug — and it is the instance most likely to be missed, because waves reads as a run-time concern and lives in the run-step half of the code.

It is also the instance with the nastiest symptom. A stale binary from a changed define produces wrong results; a stale binary from a missing waves key produces an empty waveform on the one run somebody enabled waves to debug. The cost lands precisely when somebody is already stuck.

5. Reproducing a Failure

RESULT = {"test": "axi_burst", "seed": 1839274, "sim": "vcs", "status": "FAIL",
          "run_dir": "runs/axi_burst_1839274"}


def reproduce_command(result, *, waves=True):
    """The one command that re-runs a failing result, with waves."""
    return ["flow", "run", "--test", result["test"],
            "--seed", str(result["seed"]), "--sim", result["sim"]] + (["--waves"] if waves else [])


print("from the result record:")
for k in ("test", "seed", "sim", "status"):
    print(f"   {k:<7} {RESULT[k]}")

print()
print("reproduce:", " ".join(reproduce_command(RESULT)))
print()
print("what must be identical to the original run:")
for item in ("the test name", "the seed", "the simulator", "the source set"):
    print("   " + item)
print()
print("what is allowed to differ:")
print("   the debug-access compile flag — it changes visibility, not stimulus")

from the result record:
   test    axi_burst
   seed    1839274
   sim     vcs
   status  FAIL

reproduce: flow run --test axi_burst --seed 1839274 --sim vcs --waves

what must be identical to the original run:
   the test name
   the seed
   the simulator
   the source set

what is allowed to differ:
   the debug-access compile flag — it changes visibility, not stimulus

The canonical purpose for this chapter asks for exactly that: re-running one failing seed with waves is a single command, and every argument in it comes straight out of the result record Chapter 25.1 defined.

The distinction in the last two blocks is the one that matters.

The seed must not change, and Chapter 25.4 Section 5 is one of the ways it silently does. A reproduce run that rerolls the seed exercises different stimulus, usually passes, and produces the conclusion "it was flaky" — which is wrong, unactionable, and closes the ticket.

Recompiling with debug access is safe, because visibility does not alter behaviour: the same seed drives the same constrained-random stimulus through the same logic. That is what makes the recompile acceptable rather than a different experiment. The one caveat worth knowing is that some tools disable optimisations along with adding visibility, so a reproduce run can be noticeably slower — a cost, not a correctness problem.

6. All, Nothing, or a Window

class WaveSpec:
    """What to dump. All-or-nothing is the expensive default."""

    def __init__(self, enabled=False, scope=None, start=None, stop=None):
        if not enabled and (scope or start is not None or stop is not None):
            raise ValueError("a scope or window was given but waves are disabled")
        if start is not None and stop is not None and stop <= start:
            raise ValueError(f"window end {stop} must be after start {start}")
        self.enabled, self.scope, self.start, self.stop = enabled, scope, start, stop

    def plusargs(self, path):
        if not self.enabled:
            return []
        args = ["+fsdb+all" if self.scope is None else f"+fsdb+scope={self.scope}",
                f"+fsdbfile={path}"]
        if self.start is not None:
            args.append(f"+fsdb+start={self.start}")
        if self.stop is not None:
            args.append(f"+fsdb+stop={self.stop}")
        return args


SPECS = [("off", WaveSpec()),
         ("everything", WaveSpec(enabled=True)),
         ("one scope", WaveSpec(enabled=True, scope="tb_top.dut.axi_if")),
         ("a window", WaveSpec(enabled=True, start=140000, stop=150000)),
         ("scope + window", WaveSpec(enabled=True, scope="tb_top.dut", start=140000))]

for name, spec in SPECS:
    args = spec.plusargs("runs/t_1/waves_t_1.fsdb")
    print(f"{name:<15} {len(args)} arg(s)  {args}")

print()
for bad, why in (((False, "tb_top", None, None), "scope without enabling"),
                 ((True, None, 200000, 100000), "window that ends before it starts")):
    try:
        WaveSpec(*bad)
    except ValueError as exc:
        print(f"rejected ({why}): {exc}")

off             0 arg(s)  []
everything      2 arg(s)  ['+fsdb+all', '+fsdbfile=runs/t_1/waves_t_1.fsdb']
one scope       2 arg(s)  ['+fsdb+scope=tb_top.dut.axi_if', '+fsdbfile=runs/t_1/waves_t_1.fsdb']
a window        4 arg(s)  ['+fsdb+all', '+fsdbfile=runs/t_1/waves_t_1.fsdb', '+fsdb+start=140000', '+fsdb+stop=150000']
scope + window  3 arg(s)  ['+fsdb+scope=tb_top.dut', '+fsdbfile=runs/t_1/waves_t_1.fsdb', '+fsdb+start=140000']

rejected (scope without enabling): a scope or window was given but waves are disabled
rejected (window that ends before it starts): window end 100000 must be after start 200000

Sections 1 to 5 treated waves as a boolean, and for a first reproduce that is right — you do not yet know where to look. Once you do, all-or-nothing is leaving most of the cost on the table.

A failing run's log names a time and a hierarchy path. Chapter 20.9 extracted exactly those two things from an assertion failure, and Chapter 20.10 found the first error's context. A second reproduce aimed at tb_top.dut.axi_if between 140000 and 150000 dumps a tiny fraction of what +fsdb+all would and contains everything the first error pointed at.

Three properties of that class are worth copying.

A scope or window without enabled is an error, not a silent no-op. Somebody who writes WaveSpec(scope="tb_top.dut") and forgets enabled=True means to dump waves; returning [] and letting the run finish without a waveform wastes the run. This is the same instinct as Chapter 25.1 Section 9's status validation: reject at the boundary rather than produce something inexplicable later.

The window is validated. A stop before start is a typo with a silent consequence — most tools would dump nothing — and the error message quotes both numbers so the typo is visible.

start without stop is allowed, because from this time onwards is the common case: you know when the interesting thing began and not when it ends.

7. Waves On Failure, Automatically

def should_dump(status, *, policy, attempt):
    """Waves on a first attempt are a cost; waves on a reproduce are the point."""
    if policy == "never":
        return False
    if policy == "always":
        return True
    return attempt > 1 and status != "PASS"


RUNS = [("axi_smoke", "PASS", 1), ("axi_burst", "FAIL", 1), ("axi_burst", "FAIL", 2),
        ("axi_hang", "NO_FINISH", 2), ("axi_cfg", "PASS", 2)]

for policy in ("never", "on-retry", "always"):
    dumped = [t for t, s, a in RUNS if should_dump(s, policy=policy, attempt=a)]
    print(f"{policy:<9} dumps {len(dumped)} of {len(RUNS)}: {dumped}")

never     dumps 0 of 5: []
on-retry  dumps 2 of 5: ['axi_burst', 'axi_hang']
always    dumps 5 of 5: ['axi_smoke', 'axi_burst', 'axi_burst', 'axi_hang', 'axi_cfg']

on-retry dumps two waveforms where always dumps five, and the two are the ones anybody will open.

The policy is a function of the status and the attempt number, which is the piece people leave out. Waves on the first attempt of a failing test are not possible — the flow does not know it is going to fail — so the only honest automatic policy is fail, then re-run with waves, and that re-run is Section 5's command issued by the regression rather than by a person.

That shape belongs in Module 26's regression layer, not in the runner. Chapter 25.1 Section 8 put retry on the must-not list for exactly this reason: run_test runs one test once, and deciding to run it again with waves is a policy about a set of runs.

What the runner owes the policy is the two things it already provides — an accurate status and a waves parameter — which is why a clean contract makes the layer above easy to write.

8. Predict, Run, Observe

import hashlib
from pathlib import Path


def key(sim, waves, sources):
    return hashlib.sha256("\n".join([sim, f"waves={int(waves)}",
                                     *sorted(sources)]).encode()).hexdigest()[:8]


def cmd(test, seed, waves, run_dir):
    c = [f"+UVM_TESTNAME={test}", f"+ntb_random_seed={seed}"]
    if waves:
        c += ["+fsdb+all", f"+fsdbfile={Path(run_dir) / f'waves_{test}_{seed}.fsdb'}"]
    return c


# A
print("A:", len(cmd("t", 1, False, "d")), len(cmd("t", 1, True, "d")))

# B
print("B:", cmd("t", 1, True, "d")[-1])

# C
print("C:", key("vcs", False, ["a.sv"]) == key("vcs", True, ["a.sv"]))

# D
print("D:", key("vcs", True, ["a.sv"]) == key("xrun", True, ["a.sv"]))

# E
print("E:", int(False), int(True))

# F
print("F:", "waves.fsdb" == f"waves_{'t'}_{1}.fsdb")

# G
policy = lambda s, a: a > 1 and s != "PASS"
print("G:", policy("FAIL", 1), policy("FAIL", 2), policy("PASS", 2))

# H
print("H:", str(Path("runs/d") / "waves_t_1.fsdb"))

The verified output:

A: 2 4
B: +fsdbfile=d/waves_t_1.fsdb
C: False
D: False
E: 0 1
F: False
G: False True False
H: runs/d/waves_t_1.fsdb

A — the flag adds two arguments, not one: the enable and the destination.

B — the destination carries the test and the seed.

C — waves changes the compile key, which is Section 4's whole point.

D — so does the simulator, as Chapter 25.3 established.

E — int(bool) is how a flag becomes a stable component of a hash input.

F — a constant wave filename is not the same as a per-run one, which is what Chapter 24.4's keep-list met.

G — the policy fires only on a non-passing retry.

H — the file lands in the run directory, which is the keep half.

9. Exercises

Exercise 1 — count the waste. For your last regression, compute how many waveforms a waves-always policy would have produced and how many anybody opened.

Exercise 2 — the other cost. Explain how a waves-always nightly reduces coverage, not just free disk.

Exercise 3 — two arguments. Implement the run-step half and show the argument count with and without.

Exercise 4 — name the file. Put the test and seed in the wave filename and say which two earlier chapters require it.

Exercise 5 — run_dir or scratch. Argue in two sentences for the run directory, citing Chapter 24.3.

Exercise 6 — the other vendors. Find the wave flags for every simulator in your flow. Say where they belong.

Exercise 7 — compile-time visibility. List the debug-access flag for each of your simulators and confirm all are compile flags.

Exercise 8 — the empty waveform. Describe what a run-only +fsdb+all produces against a non-debug binary, and why it is worse than nothing.

Exercise 9 — the key. Add waves to your compile key and show the digest changing.

Exercise 10 — write the test. Add the fourth assertion to Chapter 25.3's key coverage test.

Exercise 11 — why this one is missed. Explain in two sentences why waves is the likeliest omission from a compile key.

Exercise 12 — one command. Build a reproduce command from a result record. Confirm every argument comes from the record.

Exercise 13 — what must not change. List what a reproduce must hold identical, and name one way a seed silently changes.

Exercise 14 — flaky. Describe the conclusion a rerolled seed produces and why it closes the ticket wrongly.

Exercise 15 — why recompiling is safe. Say in one sentence why debug access does not change the result.

Exercise 16 — the slow reproduce. Explain the surprise and name the wrong fix people reach for.

Exercise 17 — a second cached binary. Describe the debug-access cache and what it costs.

Exercise 18 — three granularities. Build the plusargs for all, one scope, and a window. Report the argument counts.

Exercise 19 — reject the no-op. Show WaveSpec(scope=...) without enabled raising, and say what it saves.

Exercise 20 — validate the window. Give a stop before a start and record the message. Say what most tools would do without the check.

Exercise 21 — from the log. Take a real assertion failure and derive a scope and a window from it, citing Chapter 20.9.

Exercise 22 — upstream of the symptom. Describe a failure whose cause lies outside the reported scope, and say how far you would widen.

Exercise 23 — the policy. Implement never, on-retry and always, and report the counts for your own regression.

Exercise 19 — why attempt matters. Say why a first attempt cannot dump waves on failure.

Exercise 20 — the right layer. Say why the retry-with-waves policy belongs in Module 26 and not in run_test.

10. Summary

• Most waveforms are never opened — in the example, four of six — and they are the largest artefact a run produces.
• The runtime cost reduces coverage: a slower nightly tests fewer seeds in the same window.
• Off by default, on when asked — Chapter 25.1's waves=False keyword-only parameter.
• The run-step half adds two arguments: the enable and the destination.
• Put the test and the seed in the wave filename, or two runs overwrite each other.
• The waveform belongs in the run directory, not scratch — it is the one large keep-on-failure artefact.
• Wave flags are vendor-specific and belong in Chapter 23.11's adapter.
• Visibility must be compiled in. Every simulator's debug-access flag is a compile flag.
• A run-only +fsdb+all against a non-debug binary produces an empty file and no error, which is worse than no file.
• So waves must be in the compile cache key — Chapter 25.3's stale-binary rule, in the instance most likely to be missed.
• Its symptom lands at the worst moment: an empty waveform on the one run somebody enabled waves to debug.
• Write it as a test, because there is no error to notice.
• Reproducing a failure is one command built from the result record.
• The seed must not change — a rerolled seed produces a passing re-run and a wrong "it was flaky".
• Recompiling with debug access is safe, because visibility does not alter stimulus.
• A reproduce-with-waves costs a compile, and the wrong fix for that surprise is dropping waves from the key.
• All-or-nothing is the expensive default; once a failing log names a time and a scope, a windowed dump costs a fraction.
• A scope or window without enabled must raise, not silently produce no waveform and waste the run.
• Validate the window and quote both numbers, because a reversed one dumps nothing with no complaint.
• start without stop is the common case — you know when it began, not when it ends.
• Widen generously: a bug is usually upstream of its symptom in both hierarchy and time, and a too-narrow dump costs a third reproduce.
• An automatic policy can only be fail-then-retry-with-waves, because a first attempt cannot know it will fail.
• That policy belongs in Module 26's regression layer, not in run_test — which owes it only an accurate status and a waves parameter.

The next chapter takes everything these four steps produced and turns it into the single structured result the rest of this curriculum consumes.