Python Part II · Module 23 · Running EDA Tools from Python
Driving VCS, Xcelium, Questa and Verdi
This module has built the pieces one at a time: the list form, the return code, the log file, the run directory, the timeout, the launch handler, the recorded command. A real flow has to assemble them — and then survive the request that arrives six months later, which is to run the same regression on a different simulator. That request is where most flows are discovered to be a VCS flow with a configuration file rather than a flow that happens to use VCS, and the difference is one design decision about where the tool-specific part stops.
Advanced20 min readPythonPart IIAutomationEDA ToolsVerification Flow
Part II · Module 23 · Chapter 23.11 · Driving VCS, Xcelium, Questa and Verdi
1. Three Simulators, Three Command Shapes
The same job — compile a design and run one test with a seed — in three vendors' spellings:
TEST, SEED = "axi_smoke", 1839274
xcelium = ["xrun", "-64bit", "-f", "files.f", "-top", "tb_top",
"+UVM_TESTNAME=" + TEST, "+ntb_random_seed=" + str(SEED),
"-l", "sim.log"]
vcs_build = ["vcs", "-full64", "-sverilog", "-f", "files.f", "-top", "tb_top",
"-l", "compile.log", "-o", "simv"]
vcs_run = ["./simv", "+UVM_TESTNAME=" + TEST, "+ntb_random_seed=" + str(SEED),
"-l", "sim.log"]
questa_build = ["vlog", "-sv", "-f", "files.f", "-l", "compile.log"]
questa_run = ["vsim", "-c", "tb_top", "-do", "run -all; quit -f",
"+UVM_TESTNAME=" + TEST, "-sv_seed", str(SEED), "-l", "sim.log"]
for label, steps in (("xcelium", [xcelium]),
("vcs", [vcs_build, vcs_run]),
("questa", [questa_build, questa_run])):
print(f"{label:<8} {len(steps)} invocation(s)")
for step in steps:
print(f" {step[0]:<8} {len(step):>2} arguments")xcelium 1 invocation(s)
xrun 10 arguments
vcs 2 invocation(s)
vcs 11 arguments
./simv 5 arguments
questa 2 invocation(s)
vlog 6 arguments
vsim 10 argumentsLook past the flag spellings at the first column of that output. The three tools do not have the same shape.
xrun is one invocation: it elaborates and runs in a single call. vcs is two — vcs builds a binary called simv, and then you run ./simv, which is a different executable with entirely different arguments. Questa is also two, vlog then vsim, and vsim takes its commands through a -do string.
The seed is spelled three ways — +ntb_random_seed, the same on ./simv, and -sv_seed — and the test is passed as a plusarg in all three. Those differences are the easy half. The invocation count is the hard half, and it is the one that decides whether a design works.
2. The Seam in the Wrong Place
Here is the design almost everybody writes first, and what happens to it:
def build_cmd(sim, test, seed):
"""One function, a branch per simulator."""
if sim == "xrun":
return ["xrun", "-64bit", "-f", "files.f", "-top", "tb_top",
f"+UVM_TESTNAME={test}", f"+ntb_random_seed={seed}", "-l", "sim.log"]
if sim == "vcs":
return ["vcs", "-full64", "-sverilog", "-f", "files.f", "-top", "tb_top",
"-l", "compile.log", "-o", "simv"]
raise ValueError(f"unknown simulator: {sim}")
print("xrun:", " ".join(build_cmd("xrun", "axi_smoke", 1839274)[:3]), "...")
print("vcs :", " ".join(build_cmd("vcs", "axi_smoke", 1839274)[:3]), "...")
print()
print("but the vcs command never received the test or the seed:")
cmd = build_cmd("vcs", "axi_smoke", 1839274)
print(" contains axi_smoke:", any("axi_smoke" in part for part in cmd))
print(" contains the seed :", any("1839274" in part for part in cmd))xrun: xrun -64bit -f ...
vcs : vcs -full64 -sverilog ...
but the vcs command never received the test or the seed:
contains axi_smoke: False
contains the seed : FalseThe function returns one command, so the VCS branch can only return the build — and the test and the seed, which belong to the second invocation, are silently dropped. The signature was fixed before the shapes were understood.
The usual next move is a flag: build_cmd(sim, test, seed, phase="run"). That works, and it puts the knowledge that VCS has two phases into every caller. Six months later the flow has if sim == "vcs" in four places, and the request to add a fourth simulator touches all of them.
The seam has to go where the tools actually differ. They differ in how many invocations a run takes, so that is what an adapter has to be allowed to decide.
3. An Adapter per Tool
def xrun_cmd(test, seed, waves):
"""Xcelium: one invocation elaborates and runs."""
cmd = ["xrun", "-64bit", "-f", "files.f", "-top", "tb_top",
f"+UVM_TESTNAME={test}", f"+ntb_random_seed={seed}", "-l", "sim.log"]
if waves:
cmd += ["-input", "waves.tcl"]
return cmd
def vcs_cmd(test, seed, waves):
"""VCS: vcs builds ./simv, then ./simv runs the test."""
build = ["vcs", "-full64", "-sverilog", "-f", "files.f", "-top", "tb_top",
"-l", "compile.log", "-o", "simv"]
if waves:
build += ["-debug_access+all"]
run = ["./simv", f"+UVM_TESTNAME={test}", f"+ntb_random_seed={seed}", "-l", "sim.log"]
return [build, run]
def questa_cmd(test, seed, waves):
"""Questa: vlog compiles, vsim runs with a -do script."""
build = ["vlog", "-sv", "-f", "files.f", "-l", "compile.log"]
do = "run -all; quit -f"
run = ["vsim", "-c", "tb_top", "-do", do,
f"+UVM_TESTNAME={test}", "-sv_seed", str(seed), "-l", "sim.log"]
if waves:
run.insert(2, "-voptargs=+acc")
return [build, run]
ADAPTERS = {"vcs": vcs_cmd, "xrun": xrun_cmd, "vsim": questa_cmd}
cmd = ADAPTERS["xrun"]("axi_smoke", 1839274, False)
print("xrun ->", type(cmd).__name__, "of", type(cmd[0]).__name__)
cmd = ADAPTERS["vcs"]("axi_smoke", 1839274, False)
print("vcs ->", type(cmd).__name__, "of", type(cmd[0]).__name__)xrun -> list of str
vcs -> list of listThat is the canonical annotation:
ADAPTERS = {"vcs": vcs_cmd, "xrun": xrun_cmd, "vsim": questa_cmd}
cmd = ADAPTERS[sim](test, seed, waves)A dictionary whose values are functions. That is worth pausing on if it is new: a function name without parentheses is just a name bound to the function object, exactly like any other value, so it can be stored in a dict, passed as an argument or returned. ADAPTERS[sim] looks one up and (test, seed, waves) calls it — the lookup and the call are two separate steps, and there is no if anywhere in the flow.
Chapter 9.8 posed this same problem — dispatching on a simulator name once a chain of elif branches stops reading well — and answered it with match. A dict of functions is the other answer, and it has one property match does not: the set of supported simulators is data, so it can be listed, counted and extended without touching the dispatch site at all. Section 7 is that property being measured.
But look at the two return types in the output. xrun_cmd returns a list of strings — one command. vcs_cmd returns a list of lists — two. The annotation's single cmd is exactly right for the first and ambiguous for the second, and that ambiguity will reach subprocess.run as a TypeError on the day somebody switches simulator.
4. Always a List of Steps
The fix is to make every adapter return the same thing, which means the general shape, not the common one:
def normalise(result):
"""An adapter may return one command or several. Always yield a list of commands."""
if result and isinstance(result[0], str):
return [result]
return list(result)
def xrun_cmd(test, seed, waves):
return ["xrun", "-64bit", "-f", "files.f", "-top", "tb_top",
f"+UVM_TESTNAME={test}", f"+ntb_random_seed={seed}", "-l", "sim.log"]
def vcs_cmd(test, seed, waves):
return [["vcs", "-full64", "-sverilog", "-f", "files.f", "-top", "tb_top",
"-l", "compile.log", "-o", "simv"],
["./simv", f"+UVM_TESTNAME={test}", f"+ntb_random_seed={seed}", "-l", "sim.log"]]
def questa_cmd(test, seed, waves):
return [["vlog", "-sv", "-f", "files.f", "-l", "compile.log"],
["vsim", "-c", "tb_top", "-do", "run -all; quit -f",
f"+UVM_TESTNAME={test}", "-sv_seed", str(seed), "-l", "sim.log"]]
ADAPTERS = {"vcs": vcs_cmd, "xrun": xrun_cmd, "vsim": questa_cmd}
for sim in sorted(ADAPTERS):
steps = normalise(ADAPTERS[sim]("axi_smoke", 1839274, False))
names = " then ".join(step[0] for step in steps)
print(f"{sim:<6} {len(steps)} step(s): {names}")
print(f" every step is a list of str: "
f"{all(isinstance(p, str) for step in steps for p in step)}")vcs 2 step(s): vcs then ./simv
every step is a list of str: True
vsim 2 step(s): vlog then vsim
every step is a list of str: True
xrun 1 step(s): xrun
every step is a list of str: TrueOne uniform type: a list of commands, each a list of strings. The one-invocation tool returns a list of one, which costs nothing and means the flow has a single case to handle.
normalise is three lines and it is the whole generalisation. It also keeps the canonical annotation's convenience — an adapter author writing a single-invocation tool can return a bare command and it still works.
5. The Shape of the Runner
Everything on the left of that diagram is the same for every simulator, and everything a vendor cares about is in the middle. The test of the design is whether that is actually true, and Section 7 measures it.
6. The Runner
import os
import sys
# A stand-in for a simulator: writes a log, maybe a binary, exits with a code.
name = os.path.basename(sys.argv[0])
code = int(os.environ.get("FAKE_RC", "0"))
with open("sim.log", "a", encoding="utf-8") as handle:
handle.write(f"{name}: started with {len(sys.argv) - 1} arguments\n")
if code:
handle.write("UVM_ERROR drv.sv(88) @ 141900: response mismatch\n")
handle.write("TEST DONE\n")
open("simv", "w").write("binary\n")
sys.exit(code)import json
import os
import shlex
import shutil
import subprocess
import sys
from pathlib import Path
TOOL = Path("fake_tool.py").resolve()
class ToolError(Exception):
"""The tool could not be launched — a setup failure, not a tool failure."""
def normalise(result):
if result and isinstance(result[0], str):
return [result]
return list(result)
def xrun_cmd(test, seed, waves):
return [sys.executable, str(TOOL), "-64bit", "-f", "files.f",
f"+UVM_TESTNAME={test}", f"+ntb_random_seed={seed}", "-l", "sim.log"]
def vcs_cmd(test, seed, waves):
return [[sys.executable, str(TOOL), "-full64", "-sverilog", "-o", "simv"],
[sys.executable, str(TOOL), f"+UVM_TESTNAME={test}",
f"+ntb_random_seed={seed}", "-l", "sim.log"]]
ADAPTERS = {"vcs": vcs_cmd, "xrun": xrun_cmd}
TIMEOUTS = {"axi_smoke": 600, "axi_soak": 28800}
DEFAULT_TIMEOUT_S = 3600
def run_one(sim, test, seed, waves=False, rc=0):
"""One test, one simulator, every lesson in this module."""
if sim not in ADAPTERS:
raise ToolError(f"unknown simulator '{sim}' — known: {sorted(ADAPTERS)}")
run_dir = Path("runs") / f"{sim}_{test}_{seed}"
run_dir.mkdir(parents=True, exist_ok=True)
steps = normalise(ADAPTERS[sim](test, seed, waves))
limit = TIMEOUTS.get(test, DEFAULT_TIMEOUT_S)
(run_dir / "cmd.sh").write_text(
"#!/bin/sh\n# written by the flow\n"
+ "".join("exec " + shlex.join(step) + "\n" if i == len(steps) - 1
else shlex.join(step) + " || exit $?\n"
for i, step in enumerate(steps)),
encoding="utf-8")
os.chmod(run_dir / "cmd.sh", 0o755)
env = dict(os.environ, FAKE_RC=str(rc))
record = {"sim": sim, "test": test, "seed": seed, "steps": len(steps),
"run_dir": str(run_dir), "log": str(run_dir / "sim.log"),
"limit_s": limit}
for index, step in enumerate(steps):
try:
with open(run_dir / "sim.log", "a", encoding="utf-8") as log:
cp = subprocess.run(step, cwd=run_dir, stdout=log,
stderr=subprocess.STDOUT, timeout=limit, env=env)
except FileNotFoundError as exc:
raise ToolError(f"'{step[0]}' not found while running {test}") from exc
except subprocess.TimeoutExpired:
record.update(status="NO_FINISH", reason="TIMEOUT", rc=None,
failed_step=index)
return record
if cp.returncode != 0:
record.update(status="FAIL", rc=cp.returncode, failed_step=index)
return record
record.update(status="PASS", rc=0, failed_step=None)
return record
Path("files.f").write_text("rtl/alu.sv\n", encoding="utf-8")
for sim, test, rc in (("xrun", "axi_smoke", 0), ("vcs", "axi_smoke", 0),
("vcs", "axi_reset", 1), ("xrun", "axi_soak", 0)):
record = run_one(sim, test, 1839274, rc=rc)
print(f"{record['sim']:<5} {record['test']:<10} {record['status']:<9} "
f"steps={record['steps']} failed_step={record['failed_step']} "
f"limit={record['limit_s']}")
try:
run_one("vsim", "axi_smoke", 1)
except ToolError as exc:
print("ToolError:", exc)xrun axi_smoke PASS steps=1 failed_step=None limit=600
vcs axi_smoke PASS steps=2 failed_step=None limit=600
vcs axi_reset FAIL steps=2 failed_step=0 limit=3600
xrun axi_soak PASS steps=1 failed_step=None limit=28800
ToolError: unknown simulator 'vsim' — known: ['vcs', 'xrun']Count what that function contains and where each piece came from.
The list form and shell=False by default — Chapters 23.2 and 23.3. The return code checked per step, and the loop stopping at the first non-zero — Chapter 23.4. The tool's output appended to a log file rather than captured — Chapter 23.5. A run directory per simulator, test and seed, with cwd= and mkdir(parents=True) — Chapter 23.7. A timeout per test with a default — Chapter 23.8. FileNotFoundError converted to ToolError with from exc — Chapter 23.9. cmd.sh via shlex.join, with exec on the last step only — Chapter 23.10. And a record carrying the status, the log, the run directory and the limit — Chapters 20.11 and 21.1.
Three decisions in it are worth arguing for explicitly.
failed_step is in the record. With two-invocation tools, "it failed" is ambiguous — a failure at step 0 is a compile error and at step 1 is a test failure, which Chapter 20.11 would classify differently and which send people to completely different places. The index is the cheapest way to say which.
The loop stops at the first failure. Running ./simv after vcs failed produces a second, confusing error about a missing binary on top of the real one. Chapter 23.4's point that a non-zero code is a decision, not a log line.
cmd.sh uses || exit $? between steps and exec only on the last. Pasted by hand, it stops where the flow stopped, and the final exec makes the last tool's status the script's own — Chapter 23.10 Section 5. A naive cmd.sh listing both commands unconditionally reproduces neither the sequencing nor the exit status.
7. The Test: Add a Fourth Simulator
A design claim is worth what its measurement is worth, so here is the measurement.
def xrun_cmd(test, seed, waves):
return ["xrun", "-64bit", f"+UVM_TESTNAME={test}", f"+ntb_random_seed={seed}"]
def vcs_cmd(test, seed, waves):
return [["vcs", "-full64", "-o", "simv"],
["./simv", f"+UVM_TESTNAME={test}", f"+ntb_random_seed={seed}"]]
def questa_cmd(test, seed, waves):
return [["vlog", "-sv", "-f", "files.f"],
["vsim", "-c", "tb_top", "-do", "run -all; quit -f",
f"+UVM_TESTNAME={test}", "-sv_seed", str(seed)]]
# --- the fourth simulator: everything new is below this line ---
def verilator_cmd(test, seed, waves):
"""Verilator: build a C++ model, then run the produced binary."""
build = ["verilator", "--binary", "-f", "files.f", "--top-module", "tb_top",
"--Mdir", "obj_dir"]
if waves:
build += ["--trace"]
return [build, ["./obj_dir/Vtb_top", f"+UVM_TESTNAME={test}", f"+verilator+seed+{seed}"]]
ADAPTERS = {"vcs": vcs_cmd, "xrun": xrun_cmd, "vsim": questa_cmd,
"verilator": verilator_cmd}
# --- nothing above this line changed ---
def normalise(result):
if result and isinstance(result[0], str):
return [result]
return list(result)
print(f"{'sim':<10} {'steps':>5} first argv[0] last argv[0]")
for sim in sorted(ADAPTERS):
steps = normalise(ADAPTERS[sim]("axi_smoke", 1839274, False))
print(f"{sim:<10} {len(steps):>5} {steps[0][0]:<18} {steps[-1][0]}")
print()
print("adapters :", len(ADAPTERS))
print("flow functions :", "run_one, normalise")
print("flow lines changed :", 0)
print("new adapter returns:", "2 steps, same uniform type as the others")sim steps first argv[0] last argv[0]
vcs 2 vcs ./simv
verilator 2 verilator ./obj_dir/Vtb_top
vsim 2 vlog vsim
xrun 1 xrun xrun
adapters : 4
flow functions : run_one, normalise
flow lines changed : 0
new adapter returns: 2 steps, same uniform type as the othersZero lines of flow logic changed. That number is the claim — not that the design is elegant, but that extending it costs one function in one place.
Verilator is a good fourth case precisely because it is the least like the other three: it compiles to a C++ model, puts it in a directory of its own, and the thing you run afterwards has a name derived from the top module. None of that reached the runner, because the runner only ever asked for a list of steps.
And the one-step case still works. xrun returns a bare command and normalise wraps it, so the simplest adapter stayed the simplest to write.
8. What Does Not Belong in an Adapter
FLOW_CONCERNS = [
("the run directory", "every tool needs one; the policy is the same"),
("the timeout", "a property of the test, not of the simulator"),
("launch failure handling", "the same two exceptions for every tool"),
("writing cmd.sh", "identical given a list of steps"),
("status classification", "PASS / FAIL / NO_FINISH / ERROR is the flow's vocabulary"),
("the record", "one schema, or nothing can compare two runs"),
]
ADAPTER_CONCERNS = [
("flag spelling", "-full64 versus -64bit"),
("how the seed is passed", "+ntb_random_seed versus -sv_seed"),
("how many invocations", "one, or build-then-run"),
("what the built artefact is called", "simv, or obj_dir/Vtb_top"),
("how waves are enabled", "-debug_access+all versus --trace"),
]
print("belongs in the FLOW (written once):")
for name, why in FLOW_CONCERNS:
print(f" {name:<26} {why}")
print()
print("belongs in an ADAPTER (per tool):")
for name, why in ADAPTER_CONCERNS:
print(f" {name:<26} {why}")
print()
print("adapter does:", "build argument lists")
print("adapter does not:", "run anything, time anything, classify anything")belongs in the FLOW (written once):
the run directory every tool needs one; the policy is the same
the timeout a property of the test, not of the simulator
launch failure handling the same two exceptions for every tool
writing cmd.sh identical given a list of steps
status classification PASS / FAIL / NO_FINISH / ERROR is the flow's vocabulary
the record one schema, or nothing can compare two runs
belongs in an ADAPTER (per tool):
flag spelling -full64 versus -64bit
how the seed is passed +ntb_random_seed versus -sv_seed
how many invocations one, or build-then-run
what the built artefact is called simv, or obj_dir/Vtb_top
how waves are enabled -debug_access+all versus --trace
adapter does: build argument lists
adapter does not: run anything, time anything, classify anythingThe fourth line of the adapter list runs past its column, because one of those names is longer than the width — a reminder that a fixed width is a claim about your data, which Chapter 4.13's width and alignment control makes easy to set and easy to get wrong.
The rule underneath both lists is a single sentence: an adapter builds argument lists and does nothing else.
The moment an adapter calls subprocess.run itself, four things happen. Each new tool re-implements the timeout and gets it slightly different. The cmd.sh from one adapter and another drift apart. A fix to the launch handler has to be applied n times. And testing an adapter now requires the tool to exist, where a list-building function can be tested by asserting on the list — which is the property that made Section 7's measurement possible at all.
9. Predict, Run, Observe
def one(test, seed, waves):
return ["xrun", f"+UVM_TESTNAME={test}"]
def two(test, seed, waves):
return [["vcs", "-o", "simv"], ["./simv", f"+ntb_random_seed={seed}"]]
def normalise(result):
if result and isinstance(result[0], str):
return [result]
return list(result)
ADAPTERS = {"xrun": one, "vcs": two}
# A
print("A:", type(one("t", 1, False)[0]).__name__, type(two("t", 1, False)[0]).__name__)
# B
print("B:", len(normalise(one("t", 1, False))), len(normalise(two("t", 1, False))))
# C
print("C:", all(isinstance(p, str)
for sim in ADAPTERS
for step in normalise(ADAPTERS[sim]("t", 1, False))
for p in step))
# D
print("D:", normalise(two("t", 1839274, False))[-1][-1])
# E
print("E:", callable(ADAPTERS["vcs"]), ADAPTERS["vcs"] is two)
# F
try:
ADAPTERS["vsim"]("t", 1, False)
except KeyError as exc:
print("F:", type(exc).__name__, exc)
# G
print("G:", sorted(ADAPTERS))The verified output:
A: str list
B: 1 2
C: True
D: +ntb_random_seed=1839274
E: True True
F: KeyError 'vsim'
G: ['vcs', 'xrun']A — the two adapters return different shapes: a list of str and a list of list.
B — normalise makes both a list of steps, of length 1 and 2.
C — and every step is a list of strings, which is the uniform type the runner depends on.
D — the seed reached the second invocation, which the Section 2 design could not express.
E — the dictionary values are the functions themselves, not calls to them.
F — an unknown simulator raises KeyError, which is why Section 6's runner checks membership and raises ToolError with the known names instead.
G — the known simulators are just the dictionary's keys, which is what makes that error message free.
10. Exercises
Exercise 1 — three shapes. For the simulators you have access to, write out the full command for one test with a seed and count the invocations each needs.
Exercise 2 — the spelling table. Tabulate how each passes the test name, the seed and the filelist.
Exercise 3 — the wrong seam. Write the single-if version and show that the two-invocation simulator silently loses the seed.
Exercise 4 — the phase flag. Add a phase= parameter instead and say what knowledge that pushes onto every caller.
Exercise 5 — adapters. Write adapters for two simulators, one single-invocation and one build-then-run.
Exercise 6 — the dispatch table. Build ADAPTERS and call one by name with no if anywhere.
Exercise 7 — the two return types. Show the shapes differ, and say what reaches subprocess.run if nothing normalises them.
Exercise 8 — normalise. Write it in three lines and confirm both adapters produce a list of lists of strings.
Exercise 9 — the general shape. Argue in three sentences for returning a list of one over returning one command.
Exercise 10 — assemble. Write a run_one using every piece from this module and name the chapter each came from.
Exercise 11 — failed_step. Say what a failure at step 0 means versus step 1, and where each sends somebody.
Exercise 12 — stop early. Explain what happens if the loop continues after a failed build.
Exercise 13 — the cmd.sh for two steps. Write it with || exit $? and a final exec, and say what each buys.
Exercise 14 — measure the extension. Add a fourth simulator to your own runner and count the flow lines you changed. Report the number.
Exercise 15 — the least similar tool. Pick the tool least like your others and say which of its differences reached your flow.
Exercise 16 — draw the line. Classify ten concerns from your own flow as adapter or flow. Defend the two hardest.
Exercise 17 — what an adapter must not do. Say what breaks when an adapter calls subprocess.run itself, and name four consequences.
Exercise 18 — testable without the tool. Write a test that asserts on an adapter's returned list and say why it needs no simulator.
Exercise 19 — the second adapter. Sketch the interface for a log-interpretation adapter, and say which of Module 20's patterns are UVM's rather than a vendor's.
11. Summary
- Three simulators are three command shapes, and the flag spellings are the easy half.
- They differ in the number of invocations:
xrunis one; VCS isvcsthen./simv; Questa isvlogthenvsim. - A single-command signature cannot express that, so the two-invocation tool silently loses the test and the seed.
- A
phase=flag works and spreads the knowledge that one tool has two phases into every caller. - Put the seam where the tools actually differ, which is the invocation count, not the flag spelling.
ADAPTERSis a dict of functions — a name without parentheses is a value — soADAPTERS[sim](...)replaces everyif, and Chapter 9.8's dispatch problem gets an answer where the supported set is data.- Make every adapter return a list of steps, each a list of strings; a
normaliseof three lines lets single-invocation adapters stay simple. - Prefer the general shape over the common one — a function returning sometimes one thing and sometimes many pushes a type check onto every caller.
- The runner holds everything from this module: list form,
shell=False, per-step return codes, a log file,cwd=,timeout=,ToolError,cmd.sh, and the record. - Record
failed_step— step 0 is a compile error and step 1 is a test failure, and they send people to different places. - Stop at the first failing step, or the real error arrives buried under a second one about a missing binary.
cmd.shneeds|| exit $?between steps andexecon the last, so a pasted run sequences and exits like the flow did.- Adding a fourth simulator changed zero lines of flow logic, and that zero is the design's only real claim.
- An adapter builds argument lists and nothing else. Running, timing, classifying and recording are the flow's.
- An adapter that runs things cannot be tested without the tool, where a list-building function can be tested by asserting on the list.
- What is separated here is command construction. Log interpretation differs between vendors too, and deserves an adapter of its own.
That completes Module 23. You can now launch a tool from Python safely, read what it reports, bound how long it takes, survive the failures that happen before it starts, record exactly what ran, and do all of it without naming a simulator in the flow itself.
What the module has deliberately not settled is the gap Chapter 23.10 Section 7 named: a command says which tool only in the sense of a name, and which binary that name finds is decided by PATH, by tool homes, by a licence variable and by whichever version happens to be installed. The next module is about that half — the environment a run actually used, and how to pass it deliberately instead of inheriting it by accident.
Where this fits
Part of the Python curriculum.