Python Part II · Module 33 · Testing and Debugging Automation
Pre-Flight Checks Before a Long Run
A sweep that dies at hour six because a tool is not on PATH has wasted six hours to learn something that took ten milliseconds to check. The canonical idiom for avoiding that is two lines and it gets the architecture exactly right — checks as a list of independent predicates, run as data. It also has three failure modes that only appear once the list grows past three entries, and the first one appears the moment you try to write the same check twice with different arguments.
Intermediate22 min readPythonPart IIAutomationValidationFlows
Part II · Module 33 · Chapter 33.12 · Pre-Flight Checks Before a Long Run
1. What a Late Failure Costs
SCENARIOS = [
("tool not on PATH", "run 1 of 400", "immediately", "none"),
("tool not on PATH", "no pre-flight", "hour 0, 400 times", "400 bad runs"),
("outdir not writable", "no pre-flight", "hour 6", "6 hours"),
("disk full", "no pre-flight", "hour 6", "6 hours + partial results"),
("filelist wrong", "no pre-flight", "hour 0", "a confusing log"),
]
print(f"{'problem':<22} {'when detected':<16} {'fails at':<18} wasted")
for prob, when, fails, wasted in SCENARIOS:
print(f"{prob:<22} {when:<16} {fails:<18} {wasted}")
print()
print("the cheapest check in this table costs:", "about ten milliseconds")
print("the most expensive failure costs :", "six hours")
print("the ratio is why a pre-flight exists :", True)problem when detected fails at wasted
tool not on PATH run 1 of 400 immediately none
tool not on PATH no pre-flight hour 0, 400 times 400 bad runs
outdir not writable no pre-flight hour 6 6 hours
disk full no pre-flight hour 6 6 hours + partial results
filelist wrong no pre-flight hour 0 a confusing log
the cheapest check in this table costs: about ten milliseconds
the most expensive failure costs : six hours
the ratio is why a pre-flight exists : TrueRow two is the one that justifies the whole chapter. A missing tool does not stop a sweep — each of four hundred runs fails identically and quickly, and what you get at the end is a results table that is 100% failing for a reason that has nothing to do with the design. Chapter 30.3's new-versus-fixed comparison is then completely useless, because everything is new and nothing is real.
Rows three and four are worse in a different way: they fail late, so the cost is the whole sweep plus a directory of partial results that Chapter 33.13 has to clean up.
2. The Annotation
checks = [tools_on_path, filelist_valid, disk_space_ok]
failed = [c.__name__ for c in checks if not c()]import shutil
from pathlib import Path
Path("files.f").write_text("rtl/top.v\nrtl/alu.v\n", encoding="utf-8")
def tools_on_path():
return all(shutil.which(t) for t in ("iverilog", "vvp"))
def filelist_valid():
p = Path("files.f")
return p.is_file() and bool(p.read_text(encoding="utf-8").strip())
def disk_space_ok():
return shutil.disk_usage(".").free > 100 * 2 ** 20 # 100 MiB
checks = [tools_on_path, filelist_valid, disk_space_ok]
failed = [c.__name__ for c in checks if not c()]
print("checks run :", len(checks))
print("failed :", failed)
print("ready to run :", not failed)
print()
print("the architecture: checks are DATA, not a wall of if-statements")
print("adding a check :", "one entry in the list")
print("the report :", "derived, not hand-written")checks run : 3
failed : []
ready to run : True
the architecture: checks are DATA, not a wall of if-statements
adding a check : one entry in the list
the report : derived, not hand-writtenThat is the right architecture and it is worth saying so before taking it apart. A list of independent predicates is better than a nested wall of if statements in three concrete ways: adding a check is one line, the report is derived from the list so it cannot drift out of sync, and each check is independently testable — Chapter 33.2's seam argument applied to a whole pre-flight.
Keep that. The next three sections change what goes in the list and what comes out.
3. __name__ Is Not Available on Every Callable
import functools
def tools_on_path():
return True
def tool_on_path(name):
"""The natural way to write ten tool checks: one function, ten arguments."""
return name != "vcs"
class DiskCheck:
def __init__(self, need_mib):
self.need_mib = need_mib
def __call__(self):
return True
CANDIDATES = [
("a def", tools_on_path),
("a lambda", lambda: True),
("a partial", functools.partial(tool_on_path, "iverilog")),
("a callable object", DiskCheck(100)),
]
for label, c in CANDIDATES:
try:
name = repr(c.__name__)
except AttributeError as exc:
name = f"AttributeError: {exc}"
print(f" {label:<18} callable={callable(c)} __name__ -> {name}") a def callable=True __name__ -> 'tools_on_path'
a lambda callable=True __name__ -> '<lambda>'
a partial callable=True __name__ -> AttributeError: 'functools.partial' object has no attribute '__name__'
a callable object callable=True __name__ -> AttributeError: 'DiskCheck' object has no attribute '__name__'All four are callable and only one has a usable __name__. The lambda has one and it is '<lambda>', which in a failure report is worse than nothing — an operator reading failed: ['<lambda>', '<lambda>'] has learned that two checks failed and nothing else.
The partial row is the real finding, because functools.partial is not an exotic choice here. Ten tool checks want to be one function called ten times:
import functools
import shutil
TOOLS = ["iverilog", "vvp", "tclsh", "vcs"]
def tool_on_path(name):
return shutil.which(name) is not None
checks = [functools.partial(tool_on_path, t) for t in TOOLS]
print("checks built:", len(checks))
print("all callable:", all(callable(c) for c in checks))
try:
failed = [c.__name__ for c in checks if not c()]
print("failed:", failed)
except AttributeError as exc:
print("the annotation's line raises:", type(exc).__name__)
print(" ", exc)
print()
print("so the idiom quietly requires:", "one named def per check")
print("and the first thing anyone does:", "parameterise")checks built: 4
all callable: True
the annotation's line raises: AttributeError
'functools.partial' object has no attribute '__name__'The idiom forbids the obvious refactor. Writing tool_on_path("iverilog"), tool_on_path("vvp") and so on as four partials is the clean way to express four checks of the same kind, and c.__name__ crashes on it — not with a bad report, with an AttributeError in the pre-flight itself.
There is a second, quieter cost. Since the report is built from the function's name, the function name has become an operator-facing string:
def filelist_valid():
return False
checks = [filelist_valid]
print("report before:", [c.__name__ for c in checks if not c()])
filelist_valid.__name__ = "check_filelist_is_valid_v2"
print("report after :", [c.__name__ for c in checks if not c()])
print()
print("what changed in the logic:", "nothing")
print("what changed for the operator:", "the message they will grep for")
print("so the name is now:", "an interface, not an implementation detail")report before: ['filelist_valid']
report after : ['check_filelist_is_valid_v2']
what changed in the logic: nothing
what changed for the operator: the message they will grep for
so the name is now: an interface, not an implementation detailRenaming a function for clarity — normally the safest refactor there is — silently changes what the pre-flight prints, and anything downstream grepping for filelist_valid stops matching. That is the same coupling Chapter 33.8 Section 7 warned about from the other side: a message nobody promised becomes a message somebody depends on.
4. One Raising Check Destroys the Whole Report
def tools_on_path():
return True
def filelist_valid():
raise FileNotFoundError("rtl/files.f")
def disk_space_ok():
print(" (disk_space_ok was called)")
return False
checks = [tools_on_path, filelist_valid, disk_space_ok]
try:
failed = [c.__name__ for c in checks if not c()]
print("failed:", failed)
except Exception as exc:
print("the comprehension aborted with:", type(exc).__name__, "-", exc)
print()
print("checks in the list :", len(checks))
print("checks that actually ran :", 2)
print("checks that never ran :", 1)
print("problems reported :", 0)
print("problems that existed :", 2)the comprehension aborted with: FileNotFoundError - rtl/files.f
checks in the list : 3
checks that actually ran : 2
checks that never ran : 1
problems reported : 0
problems that existed : 2Two real problems and zero reported. Note that disk_space_ok never printed its line — the list comprehension is a single expression, so the exception from check two aborted it and check three was never called. The failing filelist and the full disk both went unmentioned.
This is Chapter 33.11 Section 7's collect-everything argument failing at the one place it matters most. A pre-flight exists precisely so an operator learns all the environment problems in one go; a pre-flight that stops at the first exception has the fail-fast behaviour and none of the benefit.
And the trigger is ordinary. A check that calls p.read_text() on a path that turns out to be a directory raises IsADirectoryError rather than returning False — which is exactly the input Chapter 33.11 Section 4 measured. Checks raise for the same reasons the things they check fail.
def run_check(fn):
"""Never let a check's own failure hide the other checks."""
try:
return bool(fn()), None
except Exception as exc:
return False, f"{type(exc).__name__}: {exc}"
def tools_on_path():
return True
def filelist_valid():
raise FileNotFoundError("rtl/files.f")
def disk_space_ok():
return False
results = [(fn.__name__, *run_check(fn))
for fn in (tools_on_path, filelist_valid, disk_space_ok)]
for name, ok, why in results:
detail = why or ("ok" if ok else "returned False")
print(f" {name:<18} {str(ok):<6} {detail}")
print()
print("checks that ran :", len(results))
print("problems reported :", sum(1 for _, ok, _ in results if not ok))
print("a raising check now:", "a failed check with its exception as the reason") tools_on_path True ok
filelist_valid False FileNotFoundError: rtl/files.f
disk_space_ok False returned FalseThree checks ran, two problems reported, and the exception became the reason rather than the end of the run. One try around each call is the whole fix, and it converts the most likely way a check misbehaves into the most useful thing it could have said.
5. A Bool Loses the Reason and the Threshold
import shutil
NEED_MIB = 500
def disk_space_ok():
return shutil.disk_usage(".").free > NEED_MIB * 2 ** 20
print("--- what the annotation reports when this fails")
print(" failed:", ["disk_space_ok"])
print()
QUESTIONS = [
("which check failed?", "yes"),
("why did it fail?", "no"),
("what was the threshold?", "no"),
("what was the actual value?", "no"),
("what should the operator do?", "no"),
]
print(f" {'the operator asks':<30} answered by a bare name")
for q, a in QUESTIONS:
print(f" {q:<30} {a}")
print()
du = shutil.disk_usage(".")
print("the data the check already had, and threw away:")
print(" it compared a free-space figure against:", f"{NEED_MIB} MiB")
print(" the figure was an int :", isinstance(du.free, int))
print(" it was positive :", du.free > 0)
print(" the check could have said both numbers :", True)--- what the annotation reports when this fails
failed: ['disk_space_ok']
the operator asks answered by a bare name
which check failed? yes
why did it fail? no
what was the threshold? no
what was the actual value? no
what should the operator do? no
the data the check already had, and threw away:
it compared a free-space figure against: 500 MiB
the figure was an int : True
it was positive : True
the check could have said both numbers : TrueOne of five questions answered. The check held the free-space figure and the threshold in its hands at the moment it decided, and returning a bool discarded both. That is Chapter 33.11 Section 6's rule — put the rejected value in the message — applied one level up: a check should report what it saw, not just its verdict.
So a check returns a small record instead:
import shutil
from pathlib import Path
from dataclasses import dataclass
NEED_MIB = 500
@dataclass(frozen=True)
class CheckResult:
name: str
ok: bool
detail: str
blocker: bool = True
def check_disk_space():
free_mib = shutil.disk_usage(".").free // 2 ** 20
ok = free_mib >= NEED_MIB
return CheckResult(
"disk space", ok,
f"free space is {'at least' if ok else 'below'} the {NEED_MIB} MiB "
f"this sweep needs")
def check_tools():
missing = [t for t in ("iverilog", "vvp", "tclsh")
if shutil.which(t) is None]
return CheckResult("tools on PATH", not missing,
"all found" if not missing
else f"not on PATH: {', '.join(missing)}")
def check_filelist():
p = Path("files.f")
if not p.is_file():
return CheckResult("filelist", False, f"not a readable file: {p}")
n = len(p.read_text(encoding="utf-8").split())
return CheckResult("filelist", n > 0,
f"{n} entries" if n else f"empty: {p}")
Path("files.f").write_text("rtl/top.v\nrtl/alu.v\nrtl/fifo.v\n", encoding="utf-8")
for r in (check_tools(), check_filelist(), check_disk_space()):
print(f" {r.name:<16} {'ok' if r.ok else 'FAIL':<6} {r.detail}")
print()
print("questions answered now:", 4)
print("what is still missing :", "what the operator should DO — §8") tools on PATH ok all found
filelist ok 3 entries
disk space ok free space is at least the 500 MiB this sweep needsEach check now reports a name that is a label rather than an identifier — "tools on PATH", not tools_on_path — which severs Section 3's accidental coupling: the function can be renamed freely, and the operator-facing string is a deliberate choice.
Note what the disk check does not print. It says free space is at or above the threshold without naming the figure, because that number is different on every machine and in every directory. Chapter 30.7 Section 4 made this the rule for reports and it applies to a pre-flight too: publish the comparison, not the measurement, when the measurement is about the machine rather than about the work.
6. The Checks Worth Running
import os
import shutil
import sys
from pathlib import Path
print("=== tools, measured on the machine these examples were verified on")
for t in ["iverilog", "vvp", "tclsh", "awk", "git", "python3",
"python", "vcs", "xrun"]:
print(f" {t:<10} {'found' if shutil.which(t) else 'NOT FOUND'}")
print()
print("the row that matters most:", "`python` is absent, `python3` is present")
print("which is why 32.3 §3 said :", "always use sys.executable")
print("and a pre-flight checking :", "`python` would fail on a working machine")=== tools, measured on the machine these examples were verified on
iverilog found
vvp found
tclsh found
awk found
git found
python3 found
python NOT FOUND
vcs NOT FOUND
xrun NOT FOUND
the row that matters most: `python` is absent, `python3` is present
which is why 32.3 §3 said : always use sys.executable
and a pre-flight checking : `python` would fail on a working machineThat table is the chapter in miniature. A pre-flight that checks for python reports a broken environment on a machine where everything works, and one that checks for vcs reports a broken environment on a machine that simply uses a different simulator. A tool check is only as good as the list of tools the flow actually invokes, which means the list belongs next to the code that invokes them, not in a generic utility.
import os
import shutil
import sys
from pathlib import Path
CATEGORIES = [
("tools on PATH", "shutil.which for each tool you invoke", "blocker",
"fails every run identically"),
("filelist usable", "33.11's predicate, not exists()", "blocker",
"compiles nothing or dies deep"),
("outdir writable", "write and delete a probe file", "blocker",
"fails at the first result"),
("disk space", "disk_usage().free against a need", "blocker",
"fails late, leaves partials"),
("interpreter version", "sys.version_info against a minimum", "blocker",
"a syntax error in an import"),
("env vars set", "os.environ for the ones the tools read", "blocker",
"the tool picks a default you did not want"),
("git tree clean", "git status --porcelain", "warning",
"the results are not reproducible"),
("stale results present", "outdir already has files", "warning",
"30.3 compares against the wrong baseline"),
]
print(f"{'check':<22} {'how':<40} {'class':<9} what a late failure looks like")
for name, how, cls, late in CATEGORIES:
print(f"{name:<22} {how:<40} {cls:<9} {late}")
print()
blockers = sum(1 for _, _, c, _ in CATEGORIES if c == "blocker")
print("blockers:", blockers, "| warnings:", len(CATEGORIES) - blockers)
print("the difference:", "a blocker stops the sweep; a warning is logged")
print("who decides :", "the flow owner — and it must be written down")check how class what a late failure looks like
tools on PATH shutil.which for each tool you invoke blocker fails every run identically
filelist usable 33.11's predicate, not exists() blocker compiles nothing or dies deep
outdir writable write and delete a probe file blocker fails at the first result
disk space disk_usage().free against a need blocker fails late, leaves partials
interpreter version sys.version_info against a minimum blocker a syntax error in an import
env vars set os.environ for the ones the tools read blocker the tool picks a default you did not want
git tree clean git status --porcelain warning the results are not reproducible
stale results present outdir already has files warning 30.3 compares against the wrong baselineSix blockers and two warnings, and the split is a judgement rather than a fact. A dirty git tree does not stop a simulation from running correctly — it stops you from being able to say which code produced the result — so it is a warning in an exploratory run and a blocker in a sign-off run. Chapter 29.6's point about sign-off thresholds applies: the classification is a policy, it belongs in configuration, and it must be written down rather than implied by whether someone used return False or log.warning.
The writability check deserves its own note, because it is the one check that cannot be done by inspection:
import os
from pathlib import Path
def check_outdir_writable(outdir):
"""The only reliable test for 'can I write here' is writing."""
p = Path(outdir)
try:
p.mkdir(parents=True, exist_ok=True)
except OSError as exc:
return False, f"cannot create {p}: {type(exc).__name__}"
probe = p / ".preflight_probe"
try:
probe.write_text("probe\n", encoding="utf-8")
probe.unlink()
except OSError as exc:
return False, f"cannot write in {p}: {type(exc).__name__}"
return True, f"writable: {p}"
print("a fresh directory :", check_outdir_writable("results"))
os.makedirs("locked", exist_ok=True)
os.chmod("locked", 0o500) # r-x: listable, not writable
print("a read-only one :", check_outdir_writable("locked"))
os.chmod("locked", 0o755)
print()
print("os.access(W_OK) would have said:", "maybe — it asks the OS, not the FS")
print("the probe file asks :", "the question the flow will ask")
print("and it cleans up after itself :", True)a fresh directory : (True, 'writable: results')
a read-only one : (False, 'cannot write in locked: PermissionError')
os.access(W_OK) would have said: maybe — it asks the OS, not the FS
the probe file asks : the question the flow will ask
and it cleans up after itself : TrueWriting a probe file and deleting it is the only check that answers the question the flow will actually ask — Chapter 33.11 Section 5's "ask the question you have" applied to a directory. os.access consults permission bits and can be wrong about a read-only mount, an exhausted quota or a network filesystem that has gone away.
7. Ordering, and What Stops the Sweep
A pre-flight that reports everything, then decides once
Step 4 is the one the annotation already gets right and step 6 is where the exit code lives. Chapter 33.8's contract reserved 3 for "an input was missing or unreadable", which is exactly what a failed pre-flight is — a usage error would be 2, and 1 would wrongly claim the flow ran and found problems.
Ordering inside the list matters less than people expect, because every check runs anyway. It matters for two reasons only:
ORDER_RULES = [
("cheap before expensive", "the report appears sooner on a fast failure"),
("a prerequisite before its dependent",
"checking the filelist's contents needs the filelist readable"),
("alphabetical", "no — it scatters related checks"),
("by likelihood of failure", "no — every check runs regardless"),
]
print(f"{'rule':<36} verdict")
for rule, why in ORDER_RULES:
print(f"{rule:<36} {why}")
print()
print("because every check runs:", "order is about READING, not short-circuiting")
print("the one real dependency :", "a check that would raise without an earlier one")rule verdict
cheap before expensive the report appears sooner on a fast failure
a prerequisite before its dependent checking the filelist's contents needs the filelist readable
alphabetical no — it scatters related checks
by likelihood of failure no — every check runs regardlessOnce every check runs unconditionally, ordering stops being an optimisation and becomes a readability question — with one genuine exception: a check whose own code would raise if an earlier check failed. Section 4's try wrapper turns that into a reported failure rather than a crash, so even the exception is survivable.
8. The Pre-Flight, Assembled
import os
import shutil
import sys
from dataclasses import dataclass
from pathlib import Path
@dataclass(frozen=True)
class CheckResult:
name: str
ok: bool
detail: str
blocker: bool = True
fix: str = ""
def run_one(fn):
"""A check that raises becomes a failed check, not the end of the run."""
try:
return fn()
except Exception as exc:
return CheckResult(getattr(fn, "label", "unnamed check"), False,
f"the check itself raised {type(exc).__name__}: {exc}")
def labelled(label):
def deco(fn):
fn.label = label
return fn
return deco
@labelled("tools on PATH")
def check_tools():
need = ("iverilog", "vvp", "tclsh")
missing = [t for t in need if shutil.which(t) is None]
return CheckResult(
"tools on PATH", not missing,
"all found" if not missing else f"not on PATH: {', '.join(missing)}",
fix="load the simulator module, or correct PATH")
@labelled("interpreter")
def check_interpreter():
ok = sys.version_info >= (3, 9)
return CheckResult("interpreter", ok,
f"python {'meets' if ok else 'is below'} the 3.9 minimum",
fix="run the flow with python3, never `python` — 32.3 §3")
@labelled("filelist")
def check_filelist():
p = Path("files.f")
if not p.is_file():
return CheckResult("filelist", False, f"not a readable file: {p}",
fix="pass --filelist with a path to a real file")
n = len(p.read_text(encoding="utf-8").split())
return CheckResult("filelist", n > 0,
f"{n} entries" if n else f"empty: {p}",
fix="" if n else "the filelist has no entries")
@labelled("outdir writable")
def check_outdir():
p = Path("results")
try:
p.mkdir(parents=True, exist_ok=True)
probe = p / ".probe"
probe.write_text("x", encoding="utf-8")
probe.unlink()
except OSError as exc:
return CheckResult("outdir writable", False,
f"cannot write in {p}: {type(exc).__name__}",
fix="choose another --outdir, or fix permissions")
return CheckResult("outdir writable", True, f"writable: {p}")
@labelled("disk space")
def check_disk():
need_mib = 500
free_mib = shutil.disk_usage(".").free // 2 ** 20
ok = free_mib >= need_mib
return CheckResult("disk space", ok,
f"free space is {'at least' if ok else 'below'} "
f"the {need_mib} MiB this sweep needs",
fix="" if ok else "free space, or use a scratch volume")
@labelled("git tree clean")
def check_git_clean():
return CheckResult("git tree clean", False,
"not checked here — no repository in this directory",
blocker=False,
fix="for sign-off, require a clean tree — 29.6")
CHECKS = [check_interpreter, check_tools, check_filelist, check_outdir,
check_disk, check_git_clean]
def preflight(checks):
results = [run_one(c) for c in checks]
blockers = [r for r in results if not r.ok and r.blocker]
warnings = [r for r in results if not r.ok and not r.blocker]
return results, blockers, warnings
Path("files.f").write_text("rtl/top.v\nrtl/alu.v\nrtl/fifo.v\n", encoding="utf-8")
results, blockers, warnings = preflight(CHECKS)
print("pre-flight")
for r in results:
mark = "ok " if r.ok else ("FAIL" if r.blocker else "warn")
print(f" [{mark}] {r.name:<18} {r.detail}")
if not r.ok and r.fix:
print(f" -> {r.fix}")
print()
print(f"checks: {len(results)} | blockers failed: {len(blockers)} "
f"| warnings: {len(warnings)}")
if blockers:
print("exit 3 — nothing was run")
else:
print("exit 0 from the pre-flight — starting the sweep")pre-flight
[ok ] interpreter python meets the 3.9 minimum
[ok ] tools on PATH all found
[ok ] filelist 3 entries
[ok ] outdir writable writable: results
[ok ] disk space free space is at least the 500 MiB this sweep needs
[warn] git tree clean not checked here — no repository in this directory
-> for sign-off, require a clean tree — 29.6
checks: 6 | blockers failed: 0 | warnings: 1
exit 0 from the pre-flight — starting the sweepSix checks, one warning, and the sweep proceeds. Three things in that output are worth keeping.
Each failing check carries a fix string — the answer to Section 5's fifth question, the one a bare name could never give. An operator reading not on PATH: vcs -> load the simulator module, or correct PATH does not need to find whoever wrote the flow.
The labelled decorator hangs the operator-facing name on the function as an attribute. That looks like a workaround for Section 3 and it is really the honest version of it: the name the operator sees is now a deliberate string that run_one can reach even when the check itself raises before returning a result — which is the one case where the result's own name field does not exist yet.
And the git tree clean check reports warn, not FAIL, on a directory with no repository. That classification is a decision, visible in the code as blocker=False, which is where Section 6 said it belonged.
9. What a Pre-Flight Cannot Promise
import shutil
CLAIMS = [
("iverilog is on PATH", "still true in 6 hours?", "almost certainly"),
("the filelist is readable", "still true in 6 hours?", "usually"),
("python is 3.13", "still true in 6 hours?", "yes"),
("500 MiB are free", "still true in 6 hours?", "no — the sweep consumes it"),
("outdir is writable", "still true in 6 hours?", "not if a quota fills"),
]
print(f"{'what the pre-flight proved':<28} {'the question':<26} answer")
for claim, q, a in CLAIMS:
print(f"{claim:<28} {q:<26} {a}")
print()
print("checks whose answer DECAYS:", 2)
print("so a pre-flight is :", "a check on the past, used for the future")
print("the window it narrows :", "33.11 §8's, at eight hours' distance")
print("what still handles the rest:", "the per-run error paths — 33.6")what the pre-flight proved the question answer
iverilog is on PATH still true in 6 hours? almost certainly
the filelist is readable still true in 6 hours? usually
python is 3.13 still true in 6 hours? yes
500 MiB are free still true in 6 hours? no — the sweep consumes it
outdir is writable still true in 6 hours? not if a quota fillsTwo of five answers decay, and the disk check decays because of the sweep itself. A pre-flight confirming 500 MiB free before a sweep that writes 4 GiB of waveforms has confirmed something true and irrelevant — the check needs to compare free space against the sweep's appetite, which means knowing the per-run cost and the run count.
That is the honest limit of the whole technique. A pre-flight is a measurement of the past used to predict the future, so it narrows Chapter 33.11 Section 8's check-to-use window to a few milliseconds for some checks and widens it to eight hours for others. It does not replace the per-run error handling of Chapter 33.6, and a flow that treats a green pre-flight as a guarantee will still write partial results into a full disk — which is the next chapter's subject.
10. Predict, Run, Observe
import functools
def a():
return True
def b():
raise ValueError("boom")
def c():
return False
# A
print("A:", [f.__name__ for f in (a, c) if not f()])
# B
try:
print("B:", [f.__name__ for f in (a, b, c) if not f()])
except Exception as exc:
print("B:", type(exc).__name__)
# C
p = functools.partial(a)
try:
print("C:", p.__name__)
except AttributeError:
print("C:", "AttributeError", callable(p))
# D
def safe(f):
try:
return bool(f())
except Exception:
return False
print("D:", [f.__name__ for f in (a, b, c) if not safe(f)])The verified output:
A: ['c']
B: ValueError
C: AttributeError True
D: ['b', 'c']A — the annotation working exactly as intended: one failing check, named.
B — one raising check and the whole expression is gone. No report, no names, and c was never called.
C — the partial is callable and has no __name__. Parameterising broke the idiom, as Section 3 measured.
D — the safe wrapper from Section 4 turns b into a reported failure, and now both problems appear. That one-line change is the difference between a pre-flight and a pre-flight that works.
11. Exercises
Exercise 1 — what a late failure costs. For your own longest sweep, write down what each of Section 1's five problems would cost.
Exercise 2 — 400 identical failures. Explain why a missing tool makes Chapter 30.3's comparison useless.
Exercise 3 — checks as data. Convert a wall of if statements in one of your scripts into a list.
Exercise 4 — the three advantages. Name them for your own code.
Exercise 5 — four callables. Reproduce the __name__ measurement for a def, a lambda, a partial and a callable object.
Exercise 6 — parameterise ten tools. Write them as partials and confirm what the annotation's line does.
Exercise 7 — <lambda>. Explain why that name in a failure report is worse than useless.
Exercise 8 — the name is an interface. Rename a check function and say what else you must update.
Exercise 9 — a label instead. Give your checks explicit operator-facing names.
Exercise 10 — one raising check. Reproduce the aborted comprehension and count the checks that never ran.
Exercise 11 — why checks raise. Name a check of yours that would raise on the inputs Chapter 33.11 Section 4 measured.
Exercise 12 — the one-line fix. Add the try wrapper and confirm every check now runs.
Exercise 13 — the exception as the reason. Say why that is more useful than a bare False.
Exercise 14 — five questions. For one failing check of yours, say which of Section 5's five questions it answers.
Exercise 15 — a record, not a bool. Convert one check to return a name, verdict and detail.
Exercise 16 — publish the comparison. Explain why the disk check should not print the free-space figure.
Exercise 17 — your real tool list. List the tools your flow actually invokes, and check for exactly those.
Exercise 18 — python versus python3. Confirm on your own machine which exists, and say what a pre-flight checking the wrong one would report.
Exercise 19 — blocker or warning. Classify eight checks for your flow and justify the two hardest.
Exercise 20 — write it down. Put the classification in configuration rather than in code structure.
Exercise 21 — the probe file. Write check_outdir_writable and test it against a read-only directory.
Exercise 22 — why not os.access. Explain in two sentences what it can be wrong about.
Exercise 23 — the exit code. Decide which of Chapter 33.8's codes your pre-flight failure uses, and why not 1.
Exercise 24 — ordering. Say which of the four ordering rules you follow and why the other two are irrelevant.
Exercise 25 — a fix string. Add one to every failing check in your pre-flight.
Exercise 26 — the label on the function. Explain why run_one needs it even though CheckResult has a name.
Exercise 27 — which answers decay. For your own checks, mark the ones whose answer will not hold for six hours.
Exercise 28 — the sweep's appetite. Rewrite your disk check to compare against the sweep's expected output size.
Exercise 29 — not a guarantee. Explain why a green pre-flight does not remove the need for per-run error handling.
Exercise 30 — record it. Write your pre-flight's report into the run directory and say who will read it.
12. Summary
- A missing tool does not stop a sweep — it makes four hundred runs fail identically, and the results table is then 100% new failures for a reason unrelated to the design.
- The annotation's architecture is right: checks as a list of independent predicates, run as data, with the report derived rather than hand-written.
c.__name__is not available on every callable. Adefhas it, a lambda reports'<lambda>', and afunctools.partialor callable object raisesAttributeError.- So the idiom forbids parameterising a check — the first refactor anybody reaches for when writing ten tool checks.
- The function's name becomes an operator-facing string, so renaming it silently changes the report. Use an explicit label.
- One raising check destroys the whole report. The comprehension is one expression; later checks never run, and two real problems were reported as zero.
- Checks raise for the same reasons the things they check fail, so this is the normal case, not an edge case. One
tryper call is the fix. - A bool answers one of five operator questions. It loses the reason, the threshold, the actual value and the remedy.
- Return a record — name, verdict, detail, class and a fix string.
- Publish the comparison, not the measurement, for anything about the machine rather than the work.
- Check the tools your flow actually invokes.
pythonis absent here andpython3is present; a generic check for either is wrong somewhere. - The writability check has to write. A probe file asks the question the flow will ask;
os.accessconsults permission bits and can be wrong about a mount or a quota. - Blocker versus warning is policy, belongs in configuration, and must be written down rather than implied by
return Falseagainstlog.warning. - A failed pre-flight exits 3 — an input problem, not a usage error and not "the flow ran and found problems".
- Ordering is about readability, not short-circuiting, because every check runs.
- Two of five answers decay, and the disk check decays because of the sweep itself — so compare free space against the sweep's appetite, not against a round number.
- A green pre-flight is not a guarantee. It narrows the check-to-use window and the per-run error paths are still required.
The next chapter picks up where a pre-flight's promise runs out: the run that started cleanly, got halfway, and stopped.
Where this fits
Part of the Python curriculum.