Python Part II · Module 31 · Generating RTL and Verification Artifacts
Generating Filelists and Build Configuration
Chapter 31.1 set the boundary; this is the easiest thing inside it and the one with the best return. A filelist is one shape repeated, it is read by three or four tools that must agree about it, and keeping those copies consistent by hand is the kind of task that works for a year and then wastes a day. The generator is ten lines. What makes the chapter worth a chapter is that a filelist is not a list of strings — it is an ordered document in a tool-specific dialect, and its worst failure mode is a line that a real compiler accepts, ignores, and never mentions again.
Intermediate19 min readPythonPart IIAutomationCode GenerationBuild
Part II · Module 31 · Chapter 31.2 · Generating Filelists and Build Configuration
1. Three Consumers, One Fact
A build's source list is consulted by more than one tool, and each wants it differently:
CONSUMERS = [
("simulator", "a filelist plus +define+ options"),
("lint", "the same files, a different option spelling"),
("synthesis", "the same files, usually a Tcl read_verilog list"),
("the Makefile", "the same files, as prerequisites"),
]
print(f"{'consumer':<14} wants")
for name, wants in CONSUMERS:
print(f"{name:<14} {wants}")
print()
print("consumers:", len(CONSUMERS))
print("facts :", 1)
print("hand-maintained copies that can drift:", len(CONSUMERS))consumer wants
simulator a filelist plus +define+ options
lint the same files, a different option spelling
synthesis the same files, usually a Tcl read_verilog list
the Makefile the same files, as prerequisites
consumers: 4
facts : 1
hand-maintained copies that can drift: 4Four copies of one fact is the problem, and it does not announce itself: the day somebody adds a file to the simulator's list and not to lint's, everything still builds and lint quietly stops checking the new file. Chapter 29.1 Section 8's provenance argument applies — the question "which list is right?" has no answer when there are four.
2. The Annotation
for src in srcs:
print(src, file=f)
for d in defines:
print(f"+define+{d}", file=f)from pathlib import Path
SRCS = ["rtl/pkg_types.sv", "rtl/alu.v", "rtl/fifo.v", "tb/tb_top.v"]
DEFINES = ["SIM", "FIFO_DEPTH=16"]
with open("files.f", "w", encoding="utf-8") as f:
for src in SRCS:
print(src, file=f)
for d in DEFINES:
print(f"+define+{d}", file=f)
print(Path("files.f").read_text(encoding="utf-8"), end="")
print()
print("lines:", len(Path("files.f").read_text(encoding="utf-8").splitlines()))rtl/pkg_types.sv
rtl/alu.v
rtl/fifo.v
tb/tb_top.v
+define+SIM
+define+FIFO_DEPTH=16
lines: 6Six lines, one shape repeated, and by Chapter 31.1's metric this is an ideal generator target. Everything that follows is about the content rather than the loop.
3. A Duplicated Source Is a Compile Error
The first defect appears as soon as the list has two contributors:
import subprocess
from pathlib import Path
Path("pkg.v").write_text(
"module pkg_dummy; endmodule\n", encoding="utf-8")
Path("alu.v").write_text(
"module alu(input a, output b); assign b = ~a; endmodule\n", encoding="utf-8")
# two subsystem lists, each correctly listing the package it depends on
CORE = ["pkg.v", "alu.v"]
PERIPH = ["pkg.v"]
SRCS = CORE + PERIPH
Path("files.f").write_text("\n".join(SRCS) + "\n", encoding="utf-8")
print("filelist:", SRCS)
c = subprocess.run(["iverilog", "-g2005", "-o", "sim.vvp", "-f", "files.f"],
stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
print("returncode:", c.returncode)
print("compiler :", c.stdout.strip().splitlines()[0] if c.stdout.strip() else "(none)")filelist: ['pkg.v', 'alu.v', 'pkg.v']
returncode: 2
compiler : pkg.v:1: error: 'pkg_dummy' has already been declared in this scope.Returncode 2, and the message is about a redeclared module rather than about a duplicate line. Nothing in the error says "your filelist lists a file twice", so the first half-hour of debugging goes into the RTL.
This happens whenever two subsystem lists each name a shared package, which is the normal and correct way for those lists to be written. The generator has to deduplicate — and the obvious way to deduplicate is the second defect.
4. A set Destroys the Order That Matters
SRCS = ["rtl/pkg_types.sv", "rtl/alu.v", "rtl/pkg_types.sv", "rtl/fifo.v",
"tb/tb_top.v", "rtl/alu.v"]
print("as specified :", SRCS)
print()
print("dict.fromkeys:", list(dict.fromkeys(SRCS)))
print("sorted(set) :", sorted(set(SRCS)))
print()
first = list(dict.fromkeys(SRCS))
srt = sorted(set(SRCS))
print("same contents:", set(first) == set(srt))
print("same order :", first == srt)
print("package first in dict.fromkeys:", first[0])
print("package first in sorted(set) :", srt[0])as specified : ['rtl/pkg_types.sv', 'rtl/alu.v', 'rtl/pkg_types.sv', 'rtl/fifo.v', 'tb/tb_top.v', 'rtl/alu.v']
dict.fromkeys: ['rtl/pkg_types.sv', 'rtl/alu.v', 'rtl/fifo.v', 'tb/tb_top.v']
sorted(set) : ['rtl/alu.v', 'rtl/fifo.v', 'rtl/pkg_types.sv', 'tb/tb_top.v']
same contents: True
same order : False
package first in dict.fromkeys: rtl/pkg_types.sv
package first in sorted(set) : rtl/alu.vSame four files, two orders, and the second one puts the package after the module that uses it. For a language with compilation-order dependencies — a SystemVerilog package, a include chain, a VHDL library — that is a build failure, and sorting alphabetically is the most natural-looking way to produce it.
dict.fromkeys deduplicates while preserving first-seen order, which is the behaviour a filelist wants: the spec's order is the dependency order somebody intended.
import subprocess
from pathlib import Path
Path("pkg.v").write_text("module pkg_dummy; endmodule\n", encoding="utf-8")
Path("alu.v").write_text(
"module alu(input a, output b); assign b = ~a; endmodule\n", encoding="utf-8")
RAW = ["pkg.v", "alu.v", "pkg.v"]
DEDUPED = list(dict.fromkeys(RAW))
for label, srcs in (("raw", RAW), ("deduplicated", DEDUPED)):
Path("files.f").write_text("\n".join(srcs) + "\n", encoding="utf-8")
c = subprocess.run(["iverilog", "-g2005", "-o", "sim.vvp", "-f", "files.f"],
stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
print(f"{label:<14} files={len(srcs)} returncode={c.returncode}")
print()
print("order preserved:", DEDUPED == ["pkg.v", "alu.v"])raw files=3 returncode=2
deduplicated files=2 returncode=0
order preserved: TrueOne line of Python moves the build from failing to passing, and it is the line that preserves order rather than the one that removes duplicates.
5. A Relative Filelist Works in One Directory
Chapter 30.6 Section 5 found this in a report. It is worse in a build input, because a build is run from more places:
import os
import subprocess
from pathlib import Path
Path("rtl").mkdir(exist_ok=True)
Path("rtl/alu.v").write_text(
"module alu(input a, output b); assign b = ~a; endmodule\n", encoding="utf-8")
Path("build").mkdir(exist_ok=True)
Path("files_rel.f").write_text("rtl/alu.v\n", encoding="utf-8")
Path("files_abs.f").write_text(str(Path("rtl/alu.v").resolve()) + "\n",
encoding="utf-8")
here = os.getcwd()
for label, flist in (("relative", "files_rel.f"), ("absolute", "files_abs.f")):
for cwd in (here, os.path.join(here, "build")):
c = subprocess.run(
["iverilog", "-g2005", "-o", os.path.join(here, "s.vvp"),
"-f", os.path.join(here, flist)],
cwd=cwd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
where = "top" if cwd == here else "build/"
print(f"{label:<9} run from {where:<7} rc={c.returncode}")relative run from top rc=0
relative run from build/ rc=1
absolute run from top rc=0
absolute run from build/ rc=0The relative filelist fails from a subdirectory and the absolute one does not. A filelist is handed to a simulator from the run directory, to lint from the project root, and to a synthesis script from a tool-specific build area, so "correct from one directory" is not a property worth having.
The generator knows where it is writing and what the paths are relative to, so it is the right place to resolve them:
from pathlib import Path
ROOT = Path("/proj/chip")
SRCS = ["rtl/alu.v", "rtl/fifo.v"]
print("as specified in the spec:")
for s in SRCS:
print(" ", s)
print()
print("emitted into the filelist:")
for s in SRCS:
print(" ", (ROOT / s).as_posix())
print()
print("every emitted path is absolute:",
all((ROOT / s).is_absolute() for s in SRCS))
print("the spec stays readable and relative:", all(not Path(s).is_absolute() for s in SRCS))as specified in the spec:
rtl/alu.v
rtl/fifo.v
emitted into the filelist:
/proj/chip/rtl/alu.v
/proj/chip/rtl/fifo.v
every emitted path is absolute: True
the spec stays readable and relative: TrueThe spec a human edits stays short and relative; the artefact a tool reads is absolute. That split is the whole value of having a generator in between, and it recurs in Chapter 31.3 for register offsets and Chapter 31.5 for port lists.
6. A Misspelt Option Is Silently Ignored
The annotation's second loop emits +define+NAME, which is a simulator's command-line syntax. The interesting question is not whether another tool accepts it — it is what happens when the spelling is slightly wrong:
import subprocess
from pathlib import Path
Path("d.v").write_text("""module d(output o);
`ifdef SIM
assign o = 1'b1;
`else
assign o = 1'b0;
`endif
endmodule
""", encoding="utf-8")
Path("tb.v").write_text("""module tb;
wire o; d u(.o(o));
initial begin #1 $display("o=%b", o); $finish; end
endmodule
""", encoding="utf-8")
def build(option_line, label):
Path("f.f").write_text("d.v\ntb.v\n" + option_line + "\n", encoding="utf-8")
c = subprocess.run(["iverilog", "-g2005", "-o", "x.vvp", "-f", "f.f"],
stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
if c.returncode:
print(f" {label:<26} rc={c.returncode} REJECTED")
return
r = subprocess.run(["vvp", "x.vvp"], stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, text=True)
o = [l for l in r.stdout.splitlines() if l.startswith("o=")][0]
print(f" {label:<26} rc=0 {o} SIM "
f"{'defined' if o == 'o=1' else 'NOT defined'}")
build("+define+SIM", "+define+SIM")
build("+defien+SIM", "+defien+SIM (typo)")
build("+DEFINE+SIM", "+DEFINE+SIM (wrong case)")
build("-DSIM", "-DSIM inside the filelist") +define+SIM rc=0 o=1 SIM defined
+defien+SIM (typo) rc=0 o=0 SIM NOT defined
+DEFINE+SIM (wrong case) rc=0 o=0 SIM NOT defined
-DSIM inside the filelist rc=1 REJECTEDFour lines, and they split into two groups by a rule nobody writes down.
+define+SIM works. Icarus understands the VCS-style plus-option in a command file, so the annotation's output is more portable than it looks.
A misspelt plus-option compiles cleanly and does nothing. +defien+SIM and +DEFINE+SIM both return zero, and the design builds with SIM undefined — which means it builds the other branch of every `ifdef. The build passes. The wrong gates are in the netlist.
A command-line flag inside the filelist is rejected — -DSIM returns 1.
So the error-reporting behaviour is exactly backwards from what is useful: the form that is merely in the wrong dialect is caught, and the form that is silently wrong is not.
BEHAVIOUR = [
("+define+SIM", "correct", "honoured", "—"),
("+defien+SIM", "a typo", "IGNORED", "builds the wrong ifdef branch"),
("+DEFINE+SIM", "wrong case", "IGNORED", "builds the wrong ifdef branch"),
("-DSIM", "wrong place", "rejected, rc=1", "caught immediately"),
]
print(f"{'line':<16} {'what it is':<12} {'what happens':<16} consequence")
for line, what, happens, cons in BEHAVIOUR:
print(f"{line:<16} {what:<12} {happens:<16} {cons}")
silent = [b for b in BEHAVIOUR if b[2] == "IGNORED"]
print()
print("forms that fail loudly :", sum(1 for b in BEHAVIOUR if "rejected" in b[2]))
print("forms that fail silently:", len(silent))
print("so a clean compile does not prove the options were understood.")line what it is what happens consequence
+define+SIM correct honoured —
+defien+SIM a typo IGNORED builds the wrong ifdef branch
+DEFINE+SIM wrong case IGNORED builds the wrong ifdef branch
-DSIM wrong place rejected, rc=1 caught immediately
forms that fail loudly : 1
forms that fail silently: 2
so a clean compile does not prove the options were understood.The deeper point is the one the canonical purpose makes: keep the fact dialect-free and let each emitter spell it.
SPEC = {
"srcs": ["rtl/pkg.sv", "rtl/alu.v"],
"defines": {"SIM": None, "FIFO_DEPTH": 16},
"incdirs": ["rtl/include"],
}
def emit_vcs_filelist(spec):
out = list(spec["srcs"])
out += [f"+incdir+{d}" for d in spec["incdirs"]]
out += [f"+define+{k}" if v is None else f"+define+{k}={v}"
for k, v in spec["defines"].items()]
return out
def emit_icarus_args(spec):
out = [f"-I{d}" for d in spec["incdirs"]]
out += [f"-D{k}" if v is None else f"-D{k}={v}"
for k, v in spec["defines"].items()]
return out
def emit_make_fragment(spec):
return [f"SRCS := {' '.join(spec['srcs'])}",
"DEFINES := " + " ".join(f"-D{k}" if v is None else f"-D{k}={v}"
for k, v in spec["defines"].items())]
print("vcs-style filelist:")
for l in emit_vcs_filelist(SPEC):
print(" ", l)
print()
print("icarus arguments:", emit_icarus_args(SPEC))
print()
print("make fragment:")
for l in emit_make_fragment(SPEC):
print(" ", l)
print()
print("places the name SIM is spelt by a human:", 1)
print("places it is spelt by an emitter :", 3)vcs-style filelist:
rtl/pkg.sv
rtl/alu.v
+incdir+rtl/include
+define+SIM
+define+FIFO_DEPTH=16
icarus arguments: ['-Irtl/include', '-DSIM', '-DFIFO_DEPTH=16']
make fragment:
SRCS := rtl/pkg.sv rtl/alu.v
DEFINES := -DSIM -DFIFO_DEPTH=16
places the name SIM is spelt by a human: 1
places it is spelt by an emitter : 3The name SIM is typed by a human once and spelt by three emitters, which is the only arrangement under which a misspelling is impossible rather than merely unlikely. Note FIFO_DEPTH: 16 against SIM: None: a define with a value and a define that is merely present are different things in every dialect, and representing the second as None rather than as an empty string is Chapter 29.2 Section 5's distinction again.
7. Checking the Three Emitters Agree
Three independent emitters over one spec is the structure Chapter 30.7 Section 8 cross-checked, and the same check applies:
import re
SPEC = {
"srcs": ["rtl/pkg.sv", "rtl/alu.v", "rtl/fifo.v"],
"defines": {"SIM": None, "FIFO_DEPTH": 16},
"incdirs": ["rtl/include"],
}
def emit_vcs(spec):
return (list(spec["srcs"])
+ [f"+incdir+{d}" for d in spec["incdirs"]]
+ [f"+define+{k}" if v is None else f"+define+{k}={v}"
for k, v in spec["defines"].items()])
def emit_icarus(spec):
return ([f"-I{d}" for d in spec["incdirs"]]
+ [f"-D{k}" if v is None else f"-D{k}={v}"
for k, v in spec["defines"].items()]
+ list(spec["srcs"]))
def emit_make(spec):
return [f"SRCS := {' '.join(spec['srcs'])}",
"DEFINES := " + " ".join(f"-D{k}" if v is None else f"-D{k}={v}"
for k, v in spec["defines"].items())]
def srcs_of_vcs(lines):
return [l for l in lines if not l.startswith(("+", "-"))]
def srcs_of_icarus(args):
return [a for a in args if not a.startswith("-")]
def srcs_of_make(lines):
return lines[0].split(":=", 1)[1].split()
def defines_of(text):
return set(re.findall(r"(?:\+define\+|-D)([A-Za-z_][A-Za-z0-9_]*)", text))
vcs, ica, mak = emit_vcs(SPEC), emit_icarus(SPEC), emit_make(SPEC)
srcs = {"vcs": srcs_of_vcs(vcs), "icarus": srcs_of_icarus(ica),
"make": srcs_of_make(mak)}
for k, v in srcs.items():
print(f" {k:<8} {len(v)} srcs {v}")
print()
print("all three source lists identical:",
len({tuple(v) for v in srcs.values()}) == 1)
defs = {"vcs": defines_of(" ".join(vcs)),
"icarus": defines_of(" ".join(ica)),
"make": defines_of(" ".join(mak))}
for k, v in defs.items():
print(f" {k:<8} defines {sorted(v)}")
print()
print("all three define sets identical:", len({frozenset(v) for v in defs.values()}) == 1)
print("spec define names :", sorted(SPEC["defines"])) vcs 3 srcs ['rtl/pkg.sv', 'rtl/alu.v', 'rtl/fifo.v']
icarus 3 srcs ['rtl/pkg.sv', 'rtl/alu.v', 'rtl/fifo.v']
make 3 srcs ['rtl/pkg.sv', 'rtl/alu.v', 'rtl/fifo.v']
all three source lists identical: True
vcs defines ['FIFO_DEPTH', 'SIM']
icarus defines ['FIFO_DEPTH', 'SIM']
make defines ['FIFO_DEPTH', 'SIM']
all three define sets identical: True
spec define names : ['FIFO_DEPTH', 'SIM']Note the three source lists are compared as ordered tuples, not as sets. Order is semantic here (Section 4), so the check has to be stricter than a membership test — a different check from Chapter 30.7's multiset, for the opposite reason.
The define extraction deliberately reads the emitted text with one regex rather than the spec, so a bug in any emitter shows up as a disagreement. An emitter that dropped a define would still pass a check written against the spec.
8. The Generator, Assembled
import subprocess
from dataclasses import dataclass, field
from pathlib import Path
from typing import Dict, List, Optional
@dataclass
class BuildSpec:
root: Path
srcs: List[str]
defines: Dict[str, Optional[object]] = field(default_factory=dict)
incdirs: List[str] = field(default_factory=list)
def resolved_srcs(self):
seen = list(dict.fromkeys(self.srcs))
return [(self.root / s).as_posix() for s in seen]
def duplicates(self):
out, seen = [], set()
for s in self.srcs:
if s in seen:
out.append(s)
seen.add(s)
return out
def emit_icarus(spec):
return ([f"-I{(spec.root / d).as_posix()}" for d in spec.incdirs]
+ [f"-D{k}" if v is None else f"-D{k}={v}"
for k, v in spec.defines.items()]
+ spec.resolved_srcs())
def emit_filelist(spec, path):
lines = (spec.resolved_srcs()
+ [f"+incdir+{(spec.root / d).as_posix()}" for d in spec.incdirs]
+ [f"+define+{k}" if v is None else f"+define+{k}={v}"
for k, v in spec.defines.items()])
Path(path).write_text("// GENERATED by gen_build.py — do not edit\n"
+ "\n".join(lines) + "\n", encoding="utf-8")
return lines
root = Path.cwd()
(root / "rtl").mkdir(exist_ok=True)
(root / "rtl" / "pkg.v").write_text("module pkg_dummy; endmodule\n", encoding="utf-8")
(root / "rtl" / "alu.v").write_text(
"module alu(input a, output b);\n"
"`ifdef SIM\n assign b = ~a;\n`else\n assign b = a;\n`endif\n"
"endmodule\n", encoding="utf-8")
spec = BuildSpec(root=root,
srcs=["rtl/pkg.v", "rtl/alu.v", "rtl/pkg.v"],
defines={"SIM": None, "FIFO_DEPTH": 16})
print("duplicates in the spec:", spec.duplicates())
print("files after dedup :", len(spec.resolved_srcs()))
print("order preserved :",
[Path(p).name for p in spec.resolved_srcs()] == ["pkg.v", "alu.v"])
print()
lines = emit_filelist(spec, "files.f")
print("filelist lines:", len(lines) + 1)
print()
args = emit_icarus(spec)
print("icarus args :", [a.split("/")[-1] if a.startswith("/") else a for a in args])
c = subprocess.run(["iverilog", "-g2005", "-o", "sim.vvp"] + args,
stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
print("compile rc :", c.returncode)
print("compiler says :", c.stdout.strip() or "(nothing)")
print()
print("paths absolute:", all(a.startswith("/") for a in args if not a.startswith("-")))
print("SIM reached the compiler:", "-DSIM" in args)duplicates in the spec: ['rtl/pkg.v']
files after dedup : 2
order preserved : True
filelist lines: 5
icarus args : ['-DSIM', '-DFIFO_DEPTH=16', 'pkg.v', 'alu.v']
compile rc : 0
compiler says : (nothing)
paths absolute: True
SIM reached the compiler: TrueOne spec, a duplicate removed in order, absolute paths, dialect-correct options, and a real compiler returning zero. The generated filelist carries the do-not-edit header that Chapter 31.7 makes mandatory.
The duplicates() method is worth keeping even though the generator fixes the problem: a spec that lists a file twice is usually a sign that two subsystem lists were concatenated, and reporting it is how somebody eventually tidies the spec rather than relying on the generator forever.
9. A Spec That Names a File That Is Not There
The generator resolves paths and deduplicates them, and it has one more cheap check available: the files it is about to name can be confirmed to exist.
import subprocess
from pathlib import Path
Path("rtl").mkdir(exist_ok=True)
Path("rtl/alu.v").write_text(
"module alu(input a, output b); assign b = ~a; endmodule\n", encoding="utf-8")
SRCS = ["rtl/alu.v", "rtl/deleted_block.v"]
Path("files.f").write_text("\n".join(SRCS) + "\n", encoding="utf-8")
c = subprocess.run(["iverilog", "-g2005", "-o", "s.vvp", "-f", "files.f"],
stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
print("filelist names:", SRCS)
print("returncode :", c.returncode)
for l in c.stdout.strip().splitlines():
print(" ", l[:64])filelist names: ['rtl/alu.v', 'rtl/deleted_block.v']
returncode : 1
rtl/deleted_block.v: No such file or directory
Preprocessor failed with 1 errors.The compiler does catch it, which is the good case — but read the second line. The missing file produced a second, misleading error: "no top level modules" is what a reader sees when the design is empty, and it sends somebody looking for a missing top module rather than a missing source file.
The generator can say it better, because it knows the spec:
from pathlib import Path
Path("rtl").mkdir(exist_ok=True)
Path("rtl/alu.v").write_text("module alu; endmodule\n", encoding="utf-8")
def check_exists(root, srcs):
missing = [s for s in srcs if not (Path(root) / s).is_file()]
dirs = [s for s in srcs if (Path(root) / s).is_dir()]
return {"named": len(srcs), "present": len(srcs) - len(missing),
"missing": missing, "directories": dirs}
for label, srcs in (("all present", ["rtl/alu.v"]),
("one deleted", ["rtl/alu.v", "rtl/deleted_block.v"]),
("a directory", ["rtl/alu.v", "rtl"])):
r = check_exists(".", srcs)
print(f"{label:<14} named={r['named']} present={r['present']} "
f"missing={r['missing']} dirs={r['directories']}")
print()
print("the message the generator can give:",
"spec/files.yaml names rtl/deleted_block.v, which does not exist")all present named=1 present=1 missing=[] dirs=[]
one deleted named=2 present=1 missing=['rtl/deleted_block.v'] dirs=[]
a directory named=2 present=1 missing=['rtl'] dirs=['rtl']
the message the generator can give: spec/files.yaml names rtl/deleted_block.v, which does not existTwo checks for four lines of code, and the second one is the non-obvious half: a path that is a directory is not missing, so an existence test alone passes it, and the compiler then fails with something even less helpful. is_file() rather than exists() is the right predicate, and this is the same distinction Chapter 24.3 found with Path("").
The error the generator can produce — naming the spec, the line and the path — is the difference between a thirty-second fix and a twenty-minute one. It is worth stating why the generator is the right place for this check rather than the build:
PLACES = [
("the spec, by hand", "nobody checks a path by eye", "no"),
("the generator", "it knows the spec file and the path", "yes"),
("the compiler", "it knows the path and not the spec", "partly"),
("a lint step on the filelist", "a second place to maintain", "redundant"),
]
print(f"{'where the check could live':<30} {'what it knows':<38} useful")
for where, knows, useful in PLACES:
print(f"{where:<30} {knows:<38} {useful}")
print()
print("the generator knows both halves:", "the spec location AND the resolved path")
print("so its message can name both.")where the check could live what it knows useful
the spec, by hand nobody checks a path by eye no
the generator it knows the spec file and the path yes
the compiler it knows the path and not the spec partly
a lint step on the filelist a second place to maintain redundant
the generator knows both halves: the spec location AND the resolved path
so its message can name both.Only the generator knows both halves. The compiler can report that rtl/deleted_block.v is absent and cannot say which line of which spec asked for it, which is the information somebody needs — and it is free, because the generator read that line a moment ago.
10. Predict, Run, Observe
SRCS = ["pkg.sv", "alu.v", "pkg.sv"]
# A
print("A:", list(dict.fromkeys(SRCS)))
# B
print("B:", sorted(set(SRCS)))
# C
print("C:", list(dict.fromkeys(SRCS))[0], sorted(set(SRCS))[0])
# D
print("D:", f"+define+{'SIM'}", f"-D{'SIM'}")The verified output:
A: ['pkg.sv', 'alu.v']
B: ['alu.v', 'pkg.sv']
C: pkg.sv alu.v
D: +define+SIM -DSIMA — deduplicated in first-seen order; the package stays first.
B — deduplicated alphabetically; the package moves after the module that uses it.
C — the two first elements, which is the whole difference between a working build and a compilation-order failure.
D — two spellings of one fact. The spec holds SIM; only the emitters know the punctuation.
11. Exercises
Exercise 1 — count your copies. List every file in your project that names the same source list, and say which is authoritative.
Exercise 2 — the silent divergence. Describe what happens in your flow when a file is added to the simulator's list and not to lint's.
Exercise 3 — duplicate a source. Add a file twice to a filelist and compile. Report the returncode and the message.
Exercise 4 — the message is not about the filelist. Explain why "already declared in this scope" sends a reader to the RTL.
Exercise 5 — two subsystems, one package. Construct the realistic case where two correct lists produce a duplicate.
Exercise 6 — set versus fromkeys. Deduplicate your own list both ways and diff the results.
Exercise 7 — order is semantic. Find a file in your project that must be compiled before another, and say what breaks if it is not.
Exercise 8 — which occurrence wins. Decide whether your deduplication keeps the first or last mention, and write the comment explaining why.
Exercise 9 — run from elsewhere. Take a relative filelist and compile from a subdirectory. Report what happens.
Exercise 10 — how many directories. List the working directories your filelist is used from.
Exercise 11 — resolve in the generator. Make your generator emit absolute paths while keeping the spec relative.
Exercise 12 — the spec stays readable. Explain why the human-edited file should keep relative paths.
Exercise 13 — a dialect mismatch. Put +define+SIM in a filelist your compiler reads and record the error.
Exercise 14 — the error points elsewhere. Say why "I can't open the file +define+SIM" is a misleading message.
Exercise 15 — three emitters. Write emitters for the three tools in your own flow from one spec.
Exercise 16 — value or presence. Choose the representation for a define that is merely present, and justify it against an empty string.
Exercise 17 — cross-check the emitters. Compare the source lists from your emitters as ordered tuples.
Exercise 18 — ordered, not set. Explain why this check must be stricter than Chapter 30.7's multiset comparison.
Exercise 19 — read the emitted text. Extract the defines from each emitted artefact with one regex, and say what bug that catches which a spec-based check would not.
Exercise 20 — a dropped define. Break one emitter so it loses a define, and confirm your cross-check fires.
Exercise 21 — compile the generated filelist. Hand your generator's output to a real compiler and report the returncode.
Exercise 22 — report the duplicates. Add a duplicates() report to your generator and run it on your real spec.
Exercise 23 — fix the spec, not just the output. Say why reporting a duplicate is better than silently removing it forever.
Exercise 24 — the do-not-edit header. Add it to your generated filelist and say what Chapter 31.7 will add to it.
12. Summary
- Four tools consult one source list, and four hand-maintained copies drift without announcing it — lint quietly stops checking the file somebody forgot to add.
- The annotation is an ideal generator target by Chapter 31.1's metric: one shape repeated. Every defect here is in the content, not the loop.
- A duplicated source is a compile error, not a warning — returncode 2 and "already been declared in this scope", a message that points at the RTL.
- Two correct subsystem lists naming one shared package produce exactly that duplicate.
sorted(set(...))destroys compilation order, putting a package after the module that uses it;dict.fromkeysdeduplicates in first-seen order.- Deduplicating is a decision about which occurrence wins. Keep the first, and say so in a comment.
- A relative filelist works from one directory and a build is launched from several; resolve paths in the generator and keep the spec relative.
- Icarus honours
+define+SIMinside a-fcommand file, so the annotation's output is more portable than it looks — which makes the next bullet worse, not better. - A misspelt plus-option compiles cleanly and does nothing.
+defien+SIMand+DEFINE+SIMboth return zero withSIMundefined, so the design builds the other branch of every`ifdefand the netlist is wrong with a green build. - A command-line flag inside a filelist is rejected (
-DSIMreturns 1), so the error behaviour is backwards from useful: the wrong-dialect form is caught and the silently-wrong form is not. - A clean compile does not prove the configuration was applied. Generate the options from a spec so a typo is a
KeyError, and probe that a define took effect where it selects behaviour that matters. - Keep the fact dialect-free and emit per tool. Only the emitters know the punctuation.
- A define with a value and a define that is merely present are different things; represent the second as
None, not as an empty string. - Cross-check the emitters against each other, comparing source lists as ordered tuples rather than sets, because order is semantic here.
- Extract the cross-check's data from the emitted text, so a bug in an emitter shows up; a check written against the spec cannot see one.
- Report duplicates even though the generator removes them, so the spec eventually gets tidied.
- Check that every file the spec names exists, and use
is_file()rather thanexists()so a directory is caught too. - The compiler's message for a missing source is misleading — it adds "no top level modules", which sends a reader after a missing top rather than a missing file.
- Only the generator knows both halves — which spec line asked for the path and what the path resolved to — so its error message is the one worth having.
The next chapter takes the generator that pays for itself fastest, and the one where a silent defect has the longest reach: the register map.
Where this fits
Part of the Python curriculum.