Skip to content
VLSI Mentor

Python Part II · Module 31 · Generating RTL and Verification Artifacts

Generating Directed Tests From a Scenario Spec

A directed test suite is mostly boilerplate wrapped around a handful of parameters, which makes a scenario table an obvious trade: twenty rows of data instead of twenty files of near-identical SystemVerilog. The generator is one line, and the canonical purpose contains the whole difficulty in its second clause — keep the table, not the generated tests, as the thing engineers edit. That is hard not because it is subtle but because the generated tests are what people open when something fails, and an edit made there disappears the next time anybody regenerates.

Advanced20 min readPythonPart IIAutomationCode GenerationVerification

Part II · Module 31 · Chapter 31.6 · Generating Directed Tests From a Scenario Spec

1. A Table Instead of Twenty Files

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
from dataclasses import dataclass


@dataclass(frozen=True)
class Scenario:
    name: str
    din: int
    pulses: int
    reset_mid: bool
    expect: int


SCENARIOS = [
    Scenario("single_byte", 0x5A, 1, False, 0x5A),
    Scenario("two_bytes", 0xA5, 2, False, 0xA5),
    Scenario("reset_midway", 0x3C, 2, True, 0x00),
    Scenario("zero_data", 0x00, 1, False, 0x00),
]

print(f"{'scenario':<14} {'din':>5} {'pulses':>7} {'reset':>6} {'expect':>7}")
for s in SCENARIOS:
    print(f"{s.name:<14} 0x{s.din:02X} {s.pulses:>7} "
          f"{str(s.reset_mid):>6} 0x{s.expect:02X}")

print()
print("rows in the table :", len(SCENARIOS))
print("fields per row    :", 5)
print("test files implied:", len(SCENARIOS))
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
scenario         din  pulses  reset  expect
single_byte    0x5A       1  False 0x5A
two_bytes      0xA5       2  False 0xA5
reset_midway   0x3C       2   True 0x00
zero_data      0x00       1  False 0x00

rows in the table : 4
fields per row    : 5
test files implied: 4

Four rows carrying twenty facts. Each generated test will be thirty lines of which twenty-six are identical, so by Chapter 31.1's metric this is a sound target — and unlike Chapter 31.4's comparator bank, there is no HDL loop that can do it, because the data is in a table outside the HDL.

2. The Annotation

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
for sc in scenarios:
    (out / f"{sc.name}_test.sv").write_text(render_test(sc))
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
from dataclasses import dataclass
from pathlib import Path


@dataclass(frozen=True)
class Scenario:
    name: str
    din: int
    expect: int


def render_test(sc):
    return (f"// test for {sc.name}\n"
            f"// din=0x{sc.din:02X} expect=0x{sc.expect:02X}\n")


SCENARIOS = [Scenario("single_byte", 0x5A, 0x5A),
             Scenario("two_bytes", 0xA5, 0xA5)]

out = Path("tests")
out.mkdir(exist_ok=True)
for sc in SCENARIOS:
    (out / f"{sc.name}_test.sv").write_text(render_test(sc))

files = sorted(p.name for p in out.iterdir())
print("files written:", files)
print("count        :", len(files))
print("scenarios    :", len(SCENARIOS))
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
files written: ['single_byte_test.sv', 'two_bytes_test.sv']
count        : 2
scenarios    : 2

Two scenarios, two files, and the counts agree. They agree because these two names are already valid identifiers — which is the assumption Section 3 removes.

3. Four Scenarios, One File

Scenario names in a real table are written by people, for people:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import re
from pathlib import Path

NAMES = ["burst read", "burst-read", "burst_read", "Burst Read"]


def sanitise(name):
    return re.sub(r"[^a-z0-9_]+", "_", name.lower()).strip("_")


print(f"{'scenario name':<16} -> filename")
for n in NAMES:
    print(f"{n!r:<16} -> {sanitise(n)}_test.sv")

filenames = [sanitise(n) for n in NAMES]
print()
print("distinct scenario names:", len(set(NAMES)))
print("distinct filenames     :", len(set(filenames)))
print("scenarios lost         :", len(set(NAMES)) - len(set(filenames)))
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
scenario name    -> filename
'burst read'     -> burst_read_test.sv
'burst-read'     -> burst_read_test.sv
'burst_read'     -> burst_read_test.sv
'Burst Read'     -> burst_read_test.sv

distinct scenario names: 4
distinct filenames     : 1
scenarios lost         : 3

Four distinct scenarios become one file. Spaces, hyphens and capitals are all normal in a human-maintained table, and any sanitiser that produces valid identifiers collapses them.

Now watch the annotation lose three tests without a word:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import re
from dataclasses import dataclass
from pathlib import Path


@dataclass(frozen=True)
class Scenario:
    name: str
    din: int


def sanitise(name):
    return re.sub(r"[^a-z0-9_]+", "_", name.lower()).strip("_")


SCENARIOS = [Scenario("burst read", 0x11), Scenario("burst-read", 0x22),
             Scenario("burst_read", 0x33), Scenario("Burst Read", 0x44)]

out = Path("tests")
out.mkdir(exist_ok=True)
for sc in SCENARIOS:
    (out / f"{sanitise(sc.name)}_test.sv").write_text(
        f"// {sc.name}\n// din=0x{sc.din:02X}\n", encoding="utf-8")

files = sorted(p.name for p in out.iterdir())
print("scenarios :", len(SCENARIOS))
print("files      :", len(files), files)
print()
print("surviving file contents:")
print((out / files[0]).read_text(encoding="utf-8"), end="")
print()
print("exceptions raised:", 0)
print("the surviving scenario is:", "whichever was written last")
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
scenarios : 4
files      : 1 ['burst_read_test.sv']

surviving file contents:
// Burst Read
// din=0x44

exceptions raised: 0
the surviving scenario is: whichever was written last

One file, holding the last scenario, and nothing raised. write_text truncates an existing file, so three tests were created and then destroyed by their successors.

Note which one survived: Burst Read, the fourth row. The surviving scenario is a function of table order, so adding a row at the top of the table silently changes which of the colliding tests exists — a result that depends on nothing anybody would think to look at.

4. Refuse the Collision, Then Account for Every Row

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import re
from dataclasses import dataclass
from pathlib import Path


@dataclass(frozen=True)
class Scenario:
    name: str
    din: int


def sanitise(name):
    return re.sub(r"[^a-z0-9_]+", "_", name.lower()).strip("_")


def plan(scenarios):
    """Map every scenario to a filename and refuse collisions before writing."""
    mapping = {}
    for sc in scenarios:
        mapping.setdefault(sanitise(sc.name), []).append(sc.name)
    collisions = {k: v for k, v in mapping.items() if len(v) > 1}
    empty = [sc.name for sc in scenarios if not sanitise(sc.name)]
    return mapping, collisions, empty


BAD = [Scenario("burst read", 0x11), Scenario("burst-read", 0x22),
       Scenario("burst_read", 0x33), Scenario("Burst Read", 0x44),
       Scenario("---", 0x55)]
GOOD = [Scenario("burst read", 0x11), Scenario("wrap read", 0x22),
        Scenario("single beat", 0x33)]

for label, scs in (("bad table", BAD), ("good table", GOOD)):
    mapping, coll, empty = plan(scs)
    print(f"{label}:")
    print(f"   scenarios={len(scs)} filenames={len(mapping)}")
    if coll:
        for fn, names in coll.items():
            print(f"   COLLISION {fn}_test.sv <- {names}")
    if empty:
        print(f"   EMPTY NAME after sanitising: {empty}")
    if not coll and not empty:
        print("   ok")
    print()
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
bad table:
   scenarios=5 filenames=2
   COLLISION burst_read_test.sv <- ['burst read', 'burst-read', 'burst_read', 'Burst Read']
   EMPTY NAME after sanitising: ['---']

good table:
   scenarios=3 filenames=3
   ok

Two problems in the bad table, and the second one matters as much as the first: "---" sanitises to the empty string, which would produce a file called _test.sv — a valid filename that no `include would ever name on purpose.

With the plan validated, the write becomes exclusive and the run is accounted for:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import re
from dataclasses import dataclass
from pathlib import Path


@dataclass(frozen=True)
class Scenario:
    name: str
    din: int


def sanitise(name):
    return re.sub(r"[^a-z0-9_]+", "_", name.lower()).strip("_")


def write_exclusive(path, text):
    with open(path, "x", encoding="utf-8") as f:
        f.write(text)


SCENARIOS = [Scenario("burst read", 0x11), Scenario("wrap read", 0x22)]
out = Path("tests")
out.mkdir(exist_ok=True)

written = []
for sc in SCENARIOS:
    p = out / f"{sanitise(sc.name)}_test.sv"
    write_exclusive(p, f"// GENERATED from scenarios.csv — do not edit\n"
                       f"// {sc.name}\n// din=0x{sc.din:02X}\n")
    written.append(p.name)

print("written:", sorted(written))
print("scenarios:", len(SCENARIOS), "files:", len(written),
      "accounted:", len(written) == len(SCENARIOS))
print()
try:
    write_exclusive(out / "burst_read_test.sv", "second attempt\n")
except FileExistsError as exc:
    print("a second write to the same path:", type(exc).__name__)
print()
print("the first file still holds its original scenario:",
      "burst read" in (out / "burst_read_test.sv").read_text(encoding="utf-8"))
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
written: ['burst_read_test.sv', 'wrap_read_test.sv']
scenarios: 2 files: 2 accounted: True

a second write to the same path: FileExistsError

the first file still holds its original scenario: True

FileExistsError instead of a silent overwrite, and the surviving content is the first scenario rather than the last — which does not matter, because the generator now stops rather than choosing.

5. An Edit Survives Until the Next Regeneration

The canonical purpose's second clause is about what happens after generation, and it is worth measuring rather than asserting:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import re
from dataclasses import dataclass
from pathlib import Path


@dataclass(frozen=True)
class Scenario:
    name: str
    din: int
    expect: int


def render(sc):
    return (f"// GENERATED from scenarios.csv — do not edit\n"
            f"task {sc.name};\n"
            f"  begin din = 8'h{sc.din:02X}; expect = 8'h{sc.expect:02X}; end\n"
            f"endtask\n")


def generate(scenarios, out):
    out.mkdir(exist_ok=True)
    for sc in scenarios:
        (out / f"{sc.name}_test.sv").write_text(render(sc), encoding="utf-8")


SCEN = [Scenario("single_byte", 0x5A, 0x5A)]
out = Path("tests")

generate(SCEN, out)
p = out / "single_byte_test.sv"
print("--- as generated")
print(p.read_text(encoding="utf-8"), end="")

text = p.read_text(encoding="utf-8")
p.write_text(text.replace("8'h5A; expect = 8'h5A",
                          "8'h5A; expect = 8'h5B  // fixed by hand"),
             encoding="utf-8")
print("--- after a hand edit")
print(p.read_text(encoding="utf-8"), end="")
print("edit present:", "fixed by hand" in p.read_text(encoding="utf-8"))

generate(SCEN, out)
print("--- after the next regeneration")
print(p.read_text(encoding="utf-8"), end="")
print("edit present:", "fixed by hand" in p.read_text(encoding="utf-8"))
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
--- as generated
// GENERATED from scenarios.csv — do not edit
task single_byte;
  begin din = 8'h5A; expect = 8'h5A; end
endtask
--- after a hand edit
// GENERATED from scenarios.csv — do not edit
task single_byte;
  begin din = 8'h5A; expect = 8'h5B  // fixed by hand; end
endtask
edit present: True
--- after the next regeneration
// GENERATED from scenarios.csv — do not edit
task single_byte;
  begin din = 8'h5A; expect = 8'h5A; end
endtask
edit present: False

The edit was present and then it was not. Nobody deleted it deliberately; the generator ran, as it is supposed to, and the file went back to what the table says.

The do-not-edit header was there the whole time, which is the point. A header is a request, and a request is not a mechanism — Chapter 31.7 is about the difference. What makes this chapter's artefact especially exposed is that a failing test is the thing somebody opens at midnight, so the generated directory is precisely where the temptation to make a local fix is highest.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
DEFENCES = [
    ("a do-not-edit header", "a request", "no"),
    ("generate into a build directory", "deleted every run", "yes"),
    ("make the files read-only", "an explicit chmod to defeat", "partly"),
    ("keep them out of version control", "an edit cannot be committed", "yes"),
    ("regenerate-and-diff in CI", "an edit fails the build", "yes"),
]

print(f"{'defence':<36} {'what it is':<28} structural?")
for name, what, structural in DEFENCES:
    print(f"{name:<36} {what:<28} {structural}")

print()
print("defences that are requests  :", sum(1 for d in DEFENCES if d[2] == "no"))
print("defences that are mechanisms:", sum(1 for d in DEFENCES if d[2] == "yes"))
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
defence                              what it is                   structural?
a do-not-edit header                 a request                    no
generate into a build directory      deleted every run            yes
make the files read-only             an explicit chmod to defeat  partly
keep them out of version control     an edit cannot be committed  yes
regenerate-and-diff in CI            an edit fails the build      yes

defences that are requests  : 1
defences that are mechanisms: 3

Three of the five are mechanisms, and the strongest combination is cheap: generate into a directory that is deleted on every build and excluded from version control, so an edit cannot be committed and does not survive a clean build. The regenerate-and-diff check is Chapter 31.7's subject and catches the case where somebody commits the generated directory anyway.

The corollary for the table is the part people skip: if the generated tests are not editable, the table has to be expressive enough to say everything a test needs. A scenario that cannot be expressed as a row is a hand-written test, and it belongs in a different directory — not as an edited copy of a generated one.

6. Compile the Suite, Run It, and Count What Reported

A scenario table earns its keep only if the expansion is checked, and the check is the one Chapter 30.1 Section 3 described: compare against the expected set.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import re
import subprocess
from dataclasses import dataclass
from pathlib import Path


@dataclass(frozen=True)
class Scenario:
    name: str
    din: int
    pulses: int
    reset_mid: bool
    expect: int


SCENARIOS = [
    Scenario("single_byte", 0x5A, 1, False, 0x5A),
    Scenario("two_bytes", 0xA5, 2, False, 0xA5),
    Scenario("reset_midway", 0x3C, 2, True, 0x00),
    Scenario("zero_data", 0x00, 1, False, 0x00),
]

Path("dut.v").write_text("""module dut(input clk, input rst_n, input [7:0] din,
                   input dv, output [7:0] dout);
  reg [7:0] r;
  always @(posedge clk) begin
    if (!rst_n) r <= 8'h00;
    else if (dv) r <= din;
  end
  assign dout = r;
endmodule
""", encoding="utf-8")


def render_task(sc):
    body = [f"  task sc_{sc.name};", "    begin"]
    body.append("      rst_n = 1; @(negedge clk);")
    for i in range(sc.pulses):
        if sc.reset_mid and i == sc.pulses - 1:
            body.append("      rst_n = 0; @(posedge clk); @(negedge clk);")
        else:
            body.append(f"      din = 8'h{sc.din:02X}; dv = 1; "
                        "@(posedge clk); #0 dv = 0; @(negedge clk);")
    body.append(f"      $display(\"SCENARIO {sc.name} dout=%h expect=%h %s\", "
                f"dout, 8'h{sc.expect:02X}, "
                f"(dout === 8'h{sc.expect:02X}) ? \"PASS\" : \"FAIL\");")
    body += ["    end", "  endtask"]
    return "\n".join(body)


tasks = "\n".join(render_task(sc) for sc in SCENARIOS)
calls = "\n".join(f"    sc_{sc.name};" for sc in SCENARIOS)

Path("suite.v").write_text(
    "// GENERATED from scenarios.csv — do not edit\n"
    "module suite;\n"
    "  reg clk = 0, rst_n = 0, dv = 0;\n"
    "  reg [7:0] din;\n"
    "  wire [7:0] dout;\n"
    "  dut u(.clk(clk), .rst_n(rst_n), .din(din), .dv(dv), .dout(dout));\n"
    "  always #1 clk = ~clk;\n"
    f"{tasks}\n"
    "  initial begin\n"
    f"{calls}\n"
    "    $finish;\n"
    "  end\n"
    "endmodule\n", encoding="utf-8")

c = subprocess.run(["iverilog", "-g2005", "-o", "suite.vvp", "dut.v", "suite.v"],
                   stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
print("compile returncode:", c.returncode)
r = subprocess.run(["vvp", "suite.vvp"], stdout=subprocess.PIPE,
                   stderr=subprocess.STDOUT, text=True)

reported = {}
for line in r.stdout.splitlines():
    m = re.match(r"SCENARIO (\w+) dout=(\w+) expect=(\w+) (PASS|FAIL)", line)
    if m:
        reported[m.group(1)] = m.group(4)
        print("  ", line)

expected = {sc.name for sc in SCENARIOS}
print()
print("scenarios in the table :", len(expected))
print("scenarios that reported:", len(reported))
print("missing                :", sorted(expected - set(reported)) or "none")
print("failures               :",
      sorted(n for n, v in reported.items() if v == "FAIL") or "none")
print("every row accounted for:", set(reported) == expected)
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
compile returncode: 0
   SCENARIO single_byte dout=5a expect=5a PASS
   SCENARIO two_bytes dout=a5 expect=a5 PASS
   SCENARIO reset_midway dout=00 expect=00 PASS
   SCENARIO zero_data dout=00 expect=00 PASS

scenarios in the table : 4
scenarios that reported: 4
missing                : none
failures               : none
every row accounted for: True

Four rows expanded, compiled, simulated, and each one reported a result that was matched back against its table row by name.

Three things in that output are doing work. The reset_midway row produced a genuinely different test body — a reset pulse instead of a second data write — which is the table driving structure rather than only constants. Each scenario prints its own name, so the suite is self-identifying rather than positional. And the missing line is the accounting step: a scenario that was dropped by a name collision, or whose task failed to compile out, shows up there and nowhere else.

The pattern is the one from Chapter 30.7: the generator's contract is a property of its output, and the only way to check it is to run the output and compare against the input.

7. A Table Can Contradict Itself

The collision check validates the scenario names. The rows themselves can be internally inconsistent, and a generator that expands them faithfully produces tests that cannot pass.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
from dataclasses import dataclass
from typing import List


@dataclass(frozen=True)
class Scenario:
    name: str
    din: int
    pulses: int
    reset_mid: bool
    expect: int


def validate(scs: List[Scenario], data_bits=8):
    problems = []
    limit = 1 << data_bits

    too_wide = [s.name for s in scs if s.din >= limit or s.expect >= limit]
    if too_wide:
        problems.append(f"din or expect wider than {data_bits} bits: {too_wide}")

    no_pulse = [s.name for s in scs if s.pulses < 1]
    if no_pulse:
        problems.append(f"no write pulses, so nothing is exercised: {no_pulse}")

    # a reset at the end must leave the register at its reset value
    bad_reset = [s.name for s in scs if s.reset_mid and s.expect != 0]
    if bad_reset:
        problems.append(
            f"reset at the end but a non-zero expect: {bad_reset}")

    # no reset means the last written value survives
    bad_keep = [s.name for s in scs
                if not s.reset_mid and s.pulses >= 1 and s.expect != s.din]
    if bad_keep:
        problems.append(
            f"expect disagrees with the last written din: {bad_keep}")

    return problems


GOOD = [Scenario("single_byte", 0x5A, 1, False, 0x5A),
        Scenario("reset_midway", 0x3C, 2, True, 0x00)]
BAD = [Scenario("too_wide", 0x1FF, 1, False, 0x1FF),
       Scenario("no_pulse", 0x11, 0, False, 0x11),
       Scenario("reset_but_expects", 0x22, 2, True, 0x22),
       Scenario("wrong_expect", 0x33, 1, False, 0x44)]

print("good table:", validate(GOOD) or "no problems")
print()
print("bad table:")
for p in validate(BAD):
    print("   -", p)
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
good table: no problems

bad table:
   - din or expect wider than 8 bits: ['too_wide']
   - no write pulses, so nothing is exercised: ['no_pulse']
   - reset at the end but a non-zero expect: ['reset_but_expects']
   - expect disagrees with the last written din: ['wrong_expect']

Four rows, four distinct contradictions, and every one of them would expand into a test that compiles and fails.

The last two are the interesting pair, because they are the same rule seen from opposite sides: the expected value is not an independent field. Given the inputs and the reset behaviour, the DUT's output is determined, so expect is derivable — which raises an obvious question the generator has to answer deliberately:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
from dataclasses import dataclass


@dataclass(frozen=True)
class Scenario:
    name: str
    din: int
    pulses: int
    reset_mid: bool
    expect: int


def predict(sc):
    """What the register holds at the end, from the spec alone."""
    return 0x00 if sc.reset_mid else sc.din


SCEN = [Scenario("single_byte", 0x5A, 1, False, 0x5A),
        Scenario("two_bytes", 0xA5, 2, False, 0xA5),
        Scenario("reset_midway", 0x3C, 2, True, 0x00)]

print(f"{'scenario':<14} {'stated':>7} {'predicted':>10} agree")
for s in SCEN:
    p = predict(s)
    print(f"{s.name:<14} 0x{s.expect:02X} {'':>3} 0x{p:02X} {'':>3} {s.expect == p}")

print()
print("should the generator DERIVE expect instead of reading it?")
for answer, why in (("no", "then the table cannot disagree, and cannot catch a model bug"),
                    ("yes", "then a typo in expect is impossible")):
    print(f"   {answer:<4} {why}")
print()
print("the choice made here:", "state it AND predict it, and require agreement")
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
scenario        stated  predicted agree
single_byte    0x5A     0x5A     True
two_bytes      0xA5     0xA5     True
reset_midway   0x00     0x00     True

should the generator DERIVE expect instead of reading it?
   no   then the table cannot disagree, and cannot catch a model bug
   yes  then a typo in expect is impossible

the choice made here: state it AND predict it, and require agreement

Deriving expect would remove a whole class of table typos and would also remove the check, because a derived expectation is a restatement of the generator's model of the DUT rather than an independent claim about it. If the model is wrong, every generated test agrees with it and every test passes.

So the answer is neither: state the expected value in the table and predict it in the validator, and require the two to agree. A disagreement is either a typo in the row or a bug in the model, and both are worth stopping for — which is the same reason Chapter 31.3 Section 8 re-parsed its offsets out of the emitted artefacts instead of comparing against the spec it had just read.

This is the redundancy Chapter 21.1 Section 3 warned against, deliberately kept. The difference is that there the two copies were stored in one record and could drift silently; here they are compared on every run, so the redundancy is a check rather than a liability.

8. Predict, Run, Observe

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import re
from pathlib import Path

san = lambda s: re.sub(r"[^a-z0-9_]+", "_", s.lower()).strip("_")

# A
print("A:", san("burst read"), san("burst-read"), san("Burst Read"))

# B
print("B:", len({san(n) for n in ["burst read", "burst-read", "Burst Read"]}))

# C
print("C:", repr(san("---")))

# D
Path("x.txt").write_text("first\n", encoding="utf-8")
Path("x.txt").write_text("second\n", encoding="utf-8")
print("D:", Path("x.txt").read_text(encoding="utf-8").strip())

The verified output:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
A: burst_read burst_read burst_read
B: 1
C: ''
D: second

A — three human-readable names, one identifier.

B — one filename for three scenarios.

C — a name that sanitises to nothing at all, which would produce _test.sv.

D — write_text truncates, which is why a collision is a silent loss rather than an error.

9. Exercises

Exercise 1 — your boilerplate ratio. Take two of your directed tests and count the lines that differ.

Exercise 2 — no HDL loop. Explain why a scenario table cannot be expressed as a generate for.

Exercise 3 — the five fields. Write the scenario table for one existing test suite of yours.

Exercise 4 — names from a table. List the scenario names in your spec and sanitise each one.

Exercise 5 — find a collision. Construct two names in your own vocabulary that collapse to one filename.

Exercise 6 — four into one. Run the annotation over colliding names and report how many files appear.

Exercise 7 — which one survives. Say which scenario is left after a collision, and what that depends on.

Exercise 8 — reorder the table. Move a colliding row and confirm the surviving content changes.

Exercise 9 — no exclusive mode. Confirm that write_text has no keyword to refuse an existing file.

Exercise 10 — open(path, "x"). Switch your generator to exclusive writes and show the exception.

Exercise 11 — the empty name. Find a scenario name that sanitises to nothing, and say what file it would produce.

Exercise 12 — plan before writing. Add a planning pass that maps every scenario to a filename and refuses collisions.

Exercise 13 — edit and regenerate. Edit a generated test, run the generator, and confirm the edit is gone.

Exercise 14 — a request is not a mechanism. Explain why the do-not-edit header did not help.

Exercise 15 — why this directory. Say why generated tests are more likely to be edited than a generated filelist.

Exercise 16 — three mechanisms. Pick the combination of structural defences you would apply, and justify the choice.

Exercise 17 — out of version control. Say what changes in your repository if the generated tests are not committed.

Exercise 18 — expressive enough. Find a test in your suite that could not be expressed as a table row, and say where it should live.

Exercise 19 — not an edited copy. Explain why a hand-written test must not start life as an edited generated one.

Exercise 20 — structure from the table. Add a field that changes the shape of the generated test, not just a constant.

Exercise 21 — self-identifying output. Make each generated test print its own scenario name, and say what that buys over positional output.

Exercise 22 — count what reported. Compare your table's row count against the number of scenarios that produced a result.

Exercise 23 — drop a row silently. Break your generator so one scenario vanishes, and confirm the accounting check fires.

Exercise 24 — compile the suite. Run your generated suite through a compiler and simulator, and report both returncodes.

10. Summary

  • Four rows carried twenty facts, and each generated test is thirty lines of which twenty-six are identical — a sound target, with no HDL loop available because the data lives outside the HDL.
  • The annotation works while the names are already identifiers, which is the assumption that fails first.
  • Four distinct scenario names sanitise to one filename. Spaces, hyphens and capitals are all normal in a human-maintained table.
  • write_text truncates, so the collision is a silent loss — three tests created and destroyed, nothing raised, and the survivor determined by table order.
  • Path.write_text has no exclusive mode; open(path, "x") raises FileExistsError, which is the right default because writing one path twice in a run is always a bug.
  • A name that sanitises to the empty string produces _test.sv — a valid filename nothing would include on purpose.
  • Plan the filenames before writing any, and refuse both collisions and empty names.
  • An edit to a generated test survives exactly until the next regeneration, with the do-not-edit header present the whole time.
  • A header is a request, not a mechanism. Three of five defences are structural: a build directory deleted every run, exclusion from version control, and a regenerate-and-diff check.
  • Generated tests are the most-edited generated artefact, because a failing test is what somebody opens at midnight.
  • If the tests are not editable, the table must be expressive enough, and a scenario that cannot be a row is a hand-written test that belongs elsewhere.
  • Let the table drive structure, not just constants — the reset-midway row produced a different test body.
  • Make each generated test print its own name, so the suite is self-identifying rather than positional.
  • Count the rows against the scenarios that reported. A scenario lost to a collision shows up there and nowhere else.
  • A table can contradict itself — a value too wide for the bus, zero pulses, a reset with a non-zero expectation, or an expectation that disagrees with the last write.
  • The expected value is not an independent field; given the inputs and the reset behaviour it is derivable.
  • Do not derive it, and do not merely state it — state it in the table, predict it in the validator, and require agreement, because a derived expectation is a restatement of the generator's model rather than a claim about the DUT.
  • A disagreement is either a typo in the row or a bug in the model, and both are worth stopping for. This is Chapter 21.1's redundancy kept deliberately, because it is compared on every run.

The last chapter of the module makes the do-not-edit header into something that can actually be enforced.

Where this fits

Part of the Python curriculum.