Python Part II · Module 32 · Python, Tcl and Bash in EDA Flows
Python Generating Tcl for a Tool to Run
This is the division the book has argued for since Chapter 32.1: Python decides which corners to run and the tool's own interpreter does the running. The annotation is two lines and the arrangement is right. What makes it a chapter is the thing in the middle — Python is writing source code in another language, and that language evaluates what it is handed. Module 30 escaped text for a reader; this escapes text for an interpreter, and getting it wrong does not break a table, it runs a command.
Advanced21 min readPythonPart IIAutomationTclCode Generation
Part II · Module 32 · Chapter 32.4 · Python Generating Tcl for a Tool to Run
1. Why Generate the Script at All
OPTIONS = [
("hand-written run.tcl, edited per run",
"no generator", "every corner list change is a manual edit"),
("run.tcl reading a spec file itself",
"Tcl does the data work", "32.2 §7: untestable without the tool"),
("Python generates run.tcl",
"Python decides, Tcl executes", "one boundary, both sides testable"),
("Python replaces the Tcl entirely",
"re-implement report_timing", "impossible — 32.5"),
]
print(f"{'approach':<42} {'who decides':<26} note")
for name, who, note in OPTIONS:
print(f"{name:<42} {who:<26} {note}")
print()
print("approaches that leave the decision testable:", 1)
print("approaches that are impossible :", 1)approach who decides note
hand-written run.tcl, edited per run no generator every corner list change is a manual edit
run.tcl reading a spec file itself Tcl does the data work 32.2 §7: untestable without the tool
Python generates run.tcl Python decides, Tcl executes one boundary, both sides testable
Python replaces the Tcl entirely re-implement report_timing impossible — 32.5
approaches that leave the decision testable: 1
approaches that are impossible : 1The third row is the annotation's arrangement and it is the right one: the corner list is data, deciding which corners to run is a policy, and both are testable in Python without a tool licence — while report_timing stays where the timing graph is.
2. The Annotation
tcl_path.write_text(render_sta_tcl(corners))
subprocess.run(["dc_shell", "-f", str(tcl_path)])import subprocess
from pathlib import Path
CORNERS = ["ss_0p72v_125c", "ff_0p88v_m40c"]
def render_sta_tcl(corners):
lines = ["# GENERATED by gen_sta.py — do not edit"]
for c in corners:
lines.append(f'puts "reporting {c}"')
return "\n".join(lines) + "\n"
tcl_path = Path("run.tcl")
tcl_path.write_text(render_sta_tcl(CORNERS))
print(tcl_path.read_text(encoding="utf-8"), end="")
r = subprocess.run(["tclsh", str(tcl_path)], stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, text=True,
stdin=subprocess.DEVNULL)
print()
print("returncode:", r.returncode)
print("output :", r.stdout.strip()[:50])# GENERATED by gen_sta.py — do not edit
puts "reporting ss_0p72v_125c"
puts "reporting ff_0p88v_m40c"
returncode: 0
output : reporting ss_0p72v_125c
reporting ff_0p88v_m40cThe structure works and there are two small things to note before the real subject.
write_text has no encoding=, which Chapter 31.6 flagged: the platform default decides the bytes, and a corner name with a non-ASCII character would be written differently on different machines. And dc_shell does not exist here, so tclsh runs the script. The -f flag is not passed on: it is dc_shell's spelling, and tclsh -f script.tcl returns 0 having run nothing at all — a silent no-op of exactly the kind Chapter 31.2 Section 6 found in a filelist, and a reminder that a tool's flags are part of its interface and do not transfer with its language.
3. Tcl Evaluates What You Hand It
Now the subject. Paths and corner names come from a spec file, and Tcl performs substitution inside double quotes:
import subprocess
from pathlib import Path
HOSTILE = "/proj/[exec echo PWNED]/run"
Path("t.tcl").write_text(f'set p "{HOSTILE}"\nputs $p\n', encoding="utf-8")
print("the generated line:")
print(" " + Path("t.tcl").read_text(encoding="utf-8").splitlines()[0])
r = subprocess.run(["tclsh", "t.tcl"], stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, text=True,
stdin=subprocess.DEVNULL)
print()
print("the value Python intended:", HOSTILE)
print("the value Tcl produced :", r.stdout.strip())
print("returncode :", r.returncode)
print("a command was executed :", "PWNED" in r.stdout)the generated line:
set p "/proj/[exec echo PWNED]/run"
the value Python intended: /proj/[exec echo PWNED]/run
the value Tcl produced : /proj/PWNED/run
returncode : 0
a command was executed : TrueTcl ran the bracket. [...] is command substitution inside a double-quoted Tcl string, so a value Python interpolated became a command the tool executed — with returncode 0, because nothing went wrong from Tcl's point of view.
This is Chapter 30.6's injection with teeth. There, an unescaped < restructured a page; here, an unescaped [ runs a command inside a tool session that has a design loaded and write access to the project.
The dollar sign is the other substitution, and it fails differently:
import subprocess
from pathlib import Path
for label, value in (("a bracket", "/proj/[exec echo X]/run"),
("a dollar", "/proj/$HOME/run"),
("a plain path", "/proj/chip/run")):
Path("t.tcl").write_text(f'set p "{value}"\nputs $p\n', encoding="utf-8")
r = subprocess.run(["tclsh", "t.tcl"], stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, text=True,
stdin=subprocess.DEVNULL)
first = r.stdout.strip().splitlines()[0] if r.stdout.strip() else ""
print(f" {label:<14} rc={r.returncode} {first[:46]}") a bracket rc=0 /proj/X/run
a dollar rc=1 can't read "HOME": no such variable
a plain path rc=0 /proj/chip/runThe bracket succeeds quietly and the dollar fails loudly — $HOME is a Tcl variable reference, not an environment lookup, and there is no Tcl variable called HOME, so the script aborts with rc=1.
That asymmetry is the familiar one from this whole book, in a new place: the loud failure costs an hour and the quiet one executes a command. A generator must handle both, and the next section measures the three ways people try.
4. list Does Not Help, and Braces Are Not Safe
import subprocess
from pathlib import Path
HOSTILE = "/proj/[exec echo PWNED]/run"
ATTEMPTS = {
"double quotes": f'set p "{HOSTILE}"',
"braces": f'set p {{{HOSTILE}}}',
"list": f'set p [list {HOSTILE}]',
}
for label, line in ATTEMPTS.items():
Path("t.tcl").write_text(line + "\nputs $p\n", encoding="utf-8")
r = subprocess.run(["tclsh", "t.tcl"], stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, text=True,
stdin=subprocess.DEVNULL)
out = r.stdout.strip()
print(f" {label:<16} rc={r.returncode} intact={out == HOSTILE} {out[:40]}") double quotes rc=0 intact=False /proj/PWNED/run
braces rc=0 intact=True /proj/[exec echo PWNED]/run
list rc=0 intact=False /proj/PWNED/runlist does not help. That is the result worth dwelling on, because list is the standard advice for building Tcl commands safely — and it is correct advice inside Tcl, where list quotes values a script already holds. Here the substitution happens when Tcl parses the line, before list is ever called, so the bracket has already run.
The distinction is between quoting at runtime and quoting at generation time. Python is doing the latter and only a generation-time escape can help.
Braces do work on that value, and then:
import subprocess
from pathlib import Path
VALUES = ["/proj/run", "/proj/a{b", "/proj/a}b", "/proj/a{b}c"]
for v in VALUES:
Path("t.tcl").write_text(f'set p {{{v}}}\nputs $p\n', encoding="utf-8")
r = subprocess.run(["tclsh", "t.tcl"], stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, text=True,
stdin=subprocess.DEVNULL)
out = r.stdout.strip().splitlines()[0] if r.stdout.strip() else ""
print(f" {v!r:<16} rc={r.returncode} intact={out == v} {out[:42]}") '/proj/run' rc=0 intact=True /proj/run
'/proj/a{b' rc=1 intact=False missing close-brace
'/proj/a}b' rc=1 intact=False extra characters after close-brace
'/proj/a{b}c' rc=0 intact=True /proj/a{b}cBraces break on an unbalanced brace. A value containing { or } on its own — which a generated test name or a bus expression easily can — produces a Tcl parse error, and the fourth row shows why a naive check for "does it contain a brace" is not enough: a balanced pair is fine.
So braces are a partial fix with a sharp edge, which is exactly the situation Chapter 30.5 was in with Markdown.
5. The Emitter That Works
import subprocess
from pathlib import Path
SPECIAL = '\\$"[]{}'
def tcl_str(value):
"""Emit `value` as a Tcl double-quoted literal, escaping what Tcl reads."""
out = []
for ch in str(value):
if ch in SPECIAL:
out.append("\\")
out.append(ch)
return '"' + "".join(out) + '"'
VALUES = ["/proj/chip/run",
"/proj/[exec echo PWNED]/run",
"/proj/$HOME/run",
"/proj/a{b",
'/proj/say"hi"/run']
print(f"{'value':<30} {'rc':>3} intact as emitted")
for v in VALUES:
Path("t.tcl").write_text(f"set p {tcl_str(v)}\nputs $p\n", encoding="utf-8")
r = subprocess.run(["tclsh", "t.tcl"], stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, text=True,
stdin=subprocess.DEVNULL)
out = r.stdout.strip()
print(f"{v!r:<30} {r.returncode:>3} {str(out == v):<6} {tcl_str(v)[:28]}")
print()
print("values round-tripped byte-intact:", len(VALUES))
print("characters escaped:", list(SPECIAL))value rc intact as emitted
'/proj/chip/run' 0 True "/proj/chip/run"
'/proj/[exec echo PWNED]/run' 0 True "/proj/\[exec echo PWNED\]/r
'/proj/$HOME/run' 0 True "/proj/\$HOME/run"
'/proj/a{b' 0 True "/proj/a\{b"
'/proj/say"hi"/run' 0 True "/proj/say\"hi\"/run"
values round-tripped byte-intact: 5
characters escaped: ['\\', '$', '"', '[', ']', '{', '}']All five values round-trip byte-intact at returncode zero, including the two that defeated braces and the two that defeated double quotes.
Six characters, one loop, and the backslash is escaped first by virtue of being in the same pass — the ordering bug Chapter 30.5 Section 3 warned about cannot occur here because each character is examined once rather than the string being rewritten repeatedly.
6. The Decision Stays in Python
The canonical purpose says to leave the decision in Python, and it is worth showing what that looks like concretely — because the tempting alternative is to emit the decision as Tcl:
CORNERS = [
{"name": "ss_0p72v_125c", "enabled": True, "mode": "setup"},
{"name": "ff_0p88v_m40c", "enabled": True, "mode": "hold"},
{"name": "tt_0p80v_25c", "enabled": False, "mode": "setup"},
]
print("decision in Python — the Tcl is a list of actions:")
chosen = [c for c in CORNERS if c["enabled"]]
for c in chosen:
print(f' report_timing -delay_type {c["mode"]} ;# {c["name"]}')
print()
print("decision in Tcl — the Tcl is a program:")
print(" foreach c $corners {")
print(" if {[dict get $c enabled]} {")
print(" report_timing -delay_type [dict get $c mode]")
print(" }")
print(" }")
print()
print("corners in the spec :", len(CORNERS))
print("actions in the generated Tcl:", len(chosen))
print("conditionals in the generated Tcl:", 0)decision in Python — the Tcl is a list of actions:
report_timing -delay_type setup ;# ss_0p72v_125c
report_timing -delay_type hold ;# ff_0p88v_m40c
decision in Tcl — the Tcl is a program:
foreach c $corners {
if {[dict get $c enabled]} {
report_timing -delay_type [dict get $c mode]
}
}
corners in the spec : 3
actions in the generated Tcl: 2
conditionals in the generated Tcl: 0Zero conditionals in the generated Tcl. The disabled corner simply is not in the file, which has three consequences worth naming.
The generated script is a record of what was asked for, readable without evaluating anything — Chapter 31.1's review argument, now about a generated Tcl file rather than generated RTL.
The decision is testable in Python with a dictionary and no tool. The Tcl version's if can only be exercised by launching the tool.
And by Chapter 31.1's shape metric, two report_timing lines are one shape repeated; the foreach version is a program with its own control flow, which is the thing that chapter said not to generate.
7. How a Tcl Script Reports Failure
The second line of the annotation launches the tool and ignores what comes back. What can come back is worth measuring:
import subprocess
from pathlib import Path
SCRIPTS = {
"success": 'puts "ok"\n',
"uncaught error": 'puts "working"\nerror "constraint file missing"\n',
"bare exit 1": 'puts "working"\nexit 1\n',
"catch then exit 4": ('puts "working"\n'
'if {[catch {error "no such corner"} msg]} {\n'
' puts "caught: $msg"\n exit 4\n}\n'),
}
for label, src in SCRIPTS.items():
Path("t.tcl").write_text(src, encoding="utf-8")
r = subprocess.run(["tclsh", "t.tcl"], stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, text=True,
stdin=subprocess.DEVNULL)
lines = r.stdout.strip().splitlines()
print(f" {label:<20} rc={r.returncode} lines={len(lines)} "
f"{lines[-1][:34] if lines else ''}") success rc=0 lines=1 ok
uncaught error rc=1 lines=5 (file "t.tcl" line 2)
bare exit 1 rc=1 lines=1 working
catch then exit 4 rc=4 lines=2 caught: no such cornerThree ways to fail and three different shapes.
An uncaught error gives rc=1 and prints a five-line Tcl traceback into the tool's log — message, while executing, the offending command, invoked from within, and a file-and-line footer — which is noise a parser then has to recognise.
A bare exit 1 gives rc=1 with no explanation at all, so the Python side knows something failed and nothing about what.
catch then exit 4 gives a classified code and a message the generated script controls. That is the only one of the three that lets Python distinguish failure kinds, and since Python generated the script it can emit the catch — which makes this a generator decision rather than a Tcl style question.
CODES = {0: "ok", 2: "a constraint file was missing",
3: "a corner was not defined in the library",
4: "timing was not met"}
print("the contract the generator emits and the caller reads:")
for code, meaning in CODES.items():
print(f" exit {code} {meaning}")
print()
print("codes the generator defines:", len(CODES))
print("codes a bare `exit 1` provides:", 1)
print("who decides the mapping:", "Python, because Python wrote the script")the contract the generator emits and the caller reads:
exit 0 ok
exit 2 a constraint file was missing
exit 3 a corner was not defined in the library
exit 4 timing was not met
codes the generator defines: 4
codes a bare `exit 1` provides: 1
who decides the mapping: Python, because Python wrote the scriptBoth halves of that contract are generated by the same program, which is the quiet advantage of this arrangement: the exit codes cannot drift from the code that interprets them, because one file defines both.
8. Keep the Script Beside the Log
Python decides, the tool executes, the run directory keeps both
from pathlib import Path
RUN_DIR = Path("runs/sta_ss_0p72v_125c")
RUN_DIR.mkdir(parents=True, exist_ok=True)
ARTEFACTS = [
("run.tcl", "what the tool was told to do", "generated, do-not-edit"),
("tool.log", "what the tool said", "the tool writes it"),
("timing.rpt", "what the tool found", "the tool writes it"),
("cmd.txt", "the exact command line", "23.10's paste-and-re-run"),
]
for name, what, who in ARTEFACTS:
(RUN_DIR / name).write_text(f"# {what}\n", encoding="utf-8")
print(f"{'artefact':<14} {'holds':<32} who writes it")
for name, what, who in ARTEFACTS:
print(f"{name:<14} {what:<32} {who}")
print()
print("files in the run directory:", len(sorted(RUN_DIR.iterdir())))
print("can a reviewer tell a generator bug from a tool bug?", True)
print("because:", "run.tcl and tool.log are side by side")artefact holds who writes it
run.tcl what the tool was told to do generated, do-not-edit
tool.log what the tool said the tool writes it
timing.rpt what the tool found the tool writes it
cmd.txt the exact command line 23.10's paste-and-re-run
files in the run directory: 4
can a reviewer tell a generator bug from a tool bug? True
because: run.tcl and tool.log are side by sideFour files, and the first one is the one teams discard. A generated script written to a temporary path and deleted after the run leaves a log that reports errors about commands nobody can see — so every investigation starts by re-running the generator and hoping it produces the same script, which Chapter 31.7 showed is not guaranteed if anything in the header moves.
9. The Generator, Assembled
import subprocess
from dataclasses import dataclass
from pathlib import Path
from typing import List
SPECIAL = '\\$"[]{}'
HDR = "# GENERATED by gen_sta.py from corners.csv — do not edit"
def tcl_str(value):
out = []
for ch in str(value):
if ch in SPECIAL:
out.append("\\")
out.append(ch)
return '"' + "".join(out) + '"'
@dataclass(frozen=True)
class Corner:
name: str
mode: str
enabled: bool
lib: str
def render_sta_tcl(corners: List[Corner], out_dir: str):
chosen = [c for c in corners if c.enabled]
L = [HDR,
f"# corners in spec: {len(corners)} enabled: {len(chosen)}",
"set rc 0"]
for c in chosen:
L.append("if {[catch {")
L.append(f" puts \"reading {tcl_str(c.lib)[1:-1]}\"")
L.append(f" puts \"report_timing -delay_type {c.mode} for \""
f" ; puts {tcl_str(c.name)}")
L.append(f'}} msg]}} {{ puts "FAILED {c.name}: $msg" ; set rc 3 }}')
L.append(f"puts {tcl_str('wrote reports to ' + out_dir)}")
L.append("exit $rc")
return "\n".join(L) + "\n"
CORNERS = [
Corner("ss_0p72v_125c", "max", True, "/libs/[tt]/slow.db"),
Corner("ff_0p88v_m40c", "min", True, "/libs/fast.db"),
Corner("tt_0p80v_25c", "max", False, "/libs/typ.db"),
]
run_dir = Path("runs/sta")
run_dir.mkdir(parents=True, exist_ok=True)
tcl_path = run_dir / "run.tcl"
tcl_path.write_text(render_sta_tcl(CORNERS, str(run_dir)), encoding="utf-8")
print(tcl_path.read_text(encoding="utf-8"), end="")
print()
r = subprocess.run(["tclsh", str(tcl_path)], stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, text=True,
stdin=subprocess.DEVNULL)
print("returncode:", r.returncode)
for line in r.stdout.strip().splitlines():
print(" ", line)
print()
print("the hostile library path survived:", "/libs/[tt]/slow.db" in r.stdout)
print("disabled corners in the script :", 0)
print("header present :", tcl_path.read_text(encoding="utf-8").startswith("# GENERATED"))# GENERATED by gen_sta.py from corners.csv — do not edit
# corners in spec: 3 enabled: 2
set rc 0
if {[catch {
puts "reading /libs/\[tt\]/slow.db"
puts "report_timing -delay_type max for " ; puts "ss_0p72v_125c"
} msg]} { puts "FAILED ss_0p72v_125c: $msg" ; set rc 3 }
if {[catch {
puts "reading /libs/fast.db"
puts "report_timing -delay_type min for " ; puts "ff_0p88v_m40c"
} msg]} { puts "FAILED ff_0p88v_m40c: $msg" ; set rc 3 }
puts "wrote reports to runs/sta"
exit $rc
returncode: 0
reading /libs/[tt]/slow.db
report_timing -delay_type max for
ss_0p72v_125c
reading /libs/fast.db
report_timing -delay_type min for
ff_0p88v_m40c
wrote reports to runs/sta
the hostile library path survived: True
disabled corners in the script : 0
header present : TrueEverything the chapter argued for, in one artefact: a library path containing [tt] survived into the tool as text rather than executing, the disabled corner is absent rather than conditional, each action is wrapped in a catch that sets a classified return code, the script carries a do-not-edit header naming its source, and it lives in the run directory beside what the tool will write there.
10. How the Tool Answers Back
Everything so far has gone one way: Python writes, the tool reads, and one integer comes back. Real steps need data back, and the channel chosen for it decides how fragile the flow is.
CHANNELS = [
("parse the tool's log", "text written for humans",
"Module 28 — layout changes with options"),
("parse the tool's report", "text written for humans",
"Module 28 — better, still text"),
("have the Tcl write a machine-readable file", "a format you chose",
"stable because you own both ends"),
("the exit code", "one integer",
"a classification only — 32.7 §4"),
]
print(f"{'return channel':<44} {'what it carries':<26} stability")
for name, what, stab in CHANNELS:
print(f"{name:<44} {what:<26} {stab}")
print()
print("channels where you control the format:", 1)
print("channels Module 28 exists to cope with :", 2)return channel what it carries stability
parse the tool's log text written for humans Module 28 — layout changes with options
parse the tool's report text written for humans Module 28 — better, still text
have the Tcl write a machine-readable file a format you chose stable because you own both ends
the exit code one integer a classification only — 32.7 §4
channels where you control the format: 1
channels Module 28 exists to cope with : 2The third row is available only because Python generated the script. If the Tcl is hand-written by somebody else, you parse what the tool emits and Module 28 is your chapter; if you wrote the Tcl, you can make the tool emit a format you already have a parser for.
import json
import subprocess
from pathlib import Path
SPECIAL = '\\$"[]{}'
def tcl_str(v):
out = []
for ch in str(v):
if ch in SPECIAL:
out.append("\\")
out.append(ch)
return '"' + "".join(out) + '"'
CORNERS = [("ss_0p72v_125c", -0.043), ("ff_0p88v_m40c", 0.112)]
# every emitted line goes through tcl_str — §5's emitter, used on itself
json_lines = ["{", ' "corners": [']
for i, (name, wns) in enumerate(CORNERS):
comma = "," if i < len(CORNERS) - 1 else ""
json_lines.append(f' {{"corner": "{name}", "wns": {wns}}}{comma}')
json_lines += [" ]", "}"]
lines = ["# GENERATED by gen_sta.py — do not edit",
"set fh [open results.json w]"]
for jl in json_lines:
lines.append(f"puts $fh {tcl_str(jl)}")
lines += ["close $fh", "puts {wrote results.json}", "exit 0"]
Path("emit.tcl").write_text("\n".join(lines) + "\n", encoding="utf-8")
r = subprocess.run(["tclsh", "emit.tcl"], stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, text=True,
stdin=subprocess.DEVNULL)
print("tcl rc:", r.returncode, r.stdout.strip())
print()
print(Path("results.json").read_text(encoding="utf-8"), end="")
print()
data = json.loads(Path("results.json").read_text(encoding="utf-8"))
print("parsed by json.loads:", True)
print("corners :", [c["corner"] for c in data["corners"]])
print("wns values are floats:", all(isinstance(c["wns"], float)
for c in data["corners"]))
print("regex needed :", 0)tcl rc: 0 wrote results.json
{
"corners": [
{"corner": "ss_0p72v_125c", "wns": -0.043},
{"corner": "ff_0p88v_m40c", "wns": 0.112}
]
}
parsed by json.loads: True
corners : ['ss_0p72v_125c', 'ff_0p88v_m40c']
wns values are floats: True
regex needed : 0Zero regexes. The values arrive as floats rather than as strings needing conversion — which is Chapter 30.4 Section 4's every-value-is-a-string problem avoided rather than handled.
Note that every emitted line goes through tcl_str from Section 5. The first draft of this example did not, and wrote puts $fh " \"corners\": [" — escaping the inner quotes and leaving the [ alone, so Tcl began a command substitution and the script failed at line 4. The escaping function is not optional for JSON fragments; it is more necessary there, because JSON is made of exactly the characters Tcl reserves.
Three things make this worth the extra generated lines.
The format is stable because nothing else writes it. A tool upgrade changes the log's layout and cannot change this file, because the file's producer is a script this flow generates.
The data is typed on arrival. Chapter 28.1 spent a chapter extracting -0.043 from a line of text; here it is a JSON number.
And the file is in the run directory beside the log, so Section 9's record is complete: what the tool was told, what it said, and what it found, in three files that cannot disagree about which run they belong to.
import json
GOOD = '{"corners": [{"corner": "ss", "wns": -0.043}]}'
BAD = '{"corners": [{"corner": "s"s", "wns": -0.043}]}'
for label, text in (("well-formed", GOOD), ("a quote in a corner name", BAD)):
try:
data = json.loads(text)
print(f" {label:<26} parsed, {len(data['corners'])} corner(s)")
except json.JSONDecodeError as exc:
print(f" {label:<26} JSONDecodeError at column {exc.colno}")
print()
print("the malformed file fails loudly:", True)
print("which beats:", "a corner silently missing from the comparison") well-formed parsed, 1 corner(s)
a quote in a corner name JSONDecodeError at column 28
the malformed file fails loudly: True
which beats: a corner silently missing from the comparisonA JSONDecodeError with a column number is the loud failure, and Chapter 32.4's whole argument is that a loud failure at the boundary is worth a great deal more than a quiet one — so this is the rare case where the fragile-looking choice is the safe one, provided the reader validates.
11. Predict, Run, Observe
import subprocess
from pathlib import Path
def tcl(line):
Path("t.tcl").write_text(line + "\nputs $p\n", encoding="utf-8")
r = subprocess.run(["tclsh", "t.tcl"], stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, text=True,
stdin=subprocess.DEVNULL)
return r.returncode, r.stdout.strip()
# A
print("A:", tcl('set p "/a/[exec echo X]/b"'))
# B
print("B:", tcl('set p {/a/[exec echo X]/b}'))
# C
print("C:", tcl('set p [list /a/[exec echo X]/b]'))
# D
print("D:", tcl('set p "/a/\\[exec echo X\\]/b"'))The verified output:
A: (0, '/a/X/b')
B: (0, '/a/[exec echo X]/b')
C: (0, '/a/X/b')
D: (0, '/a/[exec echo X]/b')A — double quotes execute the bracket.
B — braces suppress it, and would break on an unbalanced brace.
C — list does not help, because substitution happens first.
D — escaping works, and unlike braces it has no value that defeats it.
12. Exercises
Exercise 1 — four approaches. Say which of Section 1's four your flow uses for its tool scripts.
Exercise 2 — the untestable decision. Find a decision inside one of your Tcl scripts and say what it would take to test it.
Exercise 3 — add the encoding. Find a write_text in your generators without encoding= and fix it.
Exercise 4 — inject a bracket. Put [exec echo X] in a path your generator interpolates, and run it.
Exercise 5 — where could a bracket come from? Name three places in your flow where a path or name is not under your control.
Exercise 6 — the dollar. Interpolate a value containing $HOME and report what Tcl does.
Exercise 7 — loud versus quiet. Say which of the bracket and the dollar you would rather have, and why.
Exercise 8 — try list. Confirm on your own tclsh that list does not prevent generation-time substitution.
Exercise 9 — runtime versus generation time. Explain the distinction in two sentences.
Exercise 10 — break braces. Find a value containing an unbalanced brace and show the parse error.
Exercise 11 — balanced is fine. Explain why checking for "contains a brace" is the wrong test.
Exercise 12 — write the emitter. Implement tcl_str and round-trip five hostile values.
Exercise 13 — one pass, not many. Say why examining each character once avoids the escape-ordering bug.
Exercise 14 — three languages, three escapes. Write the one-line rule for Markdown, HTML and Tcl.
Exercise 15 — zero conditionals. Take a generated Tcl script of yours and count its if statements.
Exercise 16 — the absent corner. Convert one generated conditional into an omission and say what it buys.
Exercise 17 — the shape metric on Tcl. Apply Chapter 31.1's metric to your generated tool script.
Exercise 18 — three failure shapes. Reproduce Section 7's table on your own tclsh.
Exercise 19 — the traceback in the log. Say what an uncaught error costs your log parser.
Exercise 20 — emit the catch. Add a catch and a classified exit code to one generated script.
Exercise 21 — one file, both halves. Explain why generating the codes and their interpretation together prevents drift.
Exercise 22 — keep the script. Check whether your flow keeps its generated tool scripts, and where.
Exercise 23 — generator bug or tool bug. Describe how a reviewer tells them apart with and without the script kept.
Exercise 24 — the whole artefact. Reproduce Section 9's generator for one real step of your flow.
13. Summary
- Python decides and the tool's interpreter executes — the only one of four approaches that leaves the decision testable without a licence.
- Tcl substitutes inside double quotes, so a generated path containing
[...]runs a command with returncode zero. Chapter 30.6's injection, now inside a tool session with a design loaded. - The bracket fails quietly and the dollar fails loudly —
$HOMEis a Tcl variable reference, so the script aborts with rc=1. listdoes not help. It is a runtime quoting tool and the substitution happens at parse time, beforelistis called.- Braces suppress substitution and break on an unbalanced brace — and a balanced pair is fine, so "contains a brace" is the wrong test.
- Escaping
\ $ " [ ] { }in one pass round-trips every hostile value byte-intact, and examining each character once makes the escape-ordering bug impossible. - The generalising rule is a question, not a function: what does the consuming language do with the text? Markdown splits, HTML parses, Tcl executes.
- Leave the decision in Python: zero conditionals in the generated Tcl. A disabled corner is absent, not guarded — readable without evaluation, testable without a tool, and one shape by Chapter 31.1's metric.
- Three ways a Tcl script reports failure: an uncaught
erroradds a traceback to the log, a bareexit 1says nothing, andcatchplus a classified code is the only one Python can act on. - Since Python wrote the script it can emit the
catch, so the exit-code contract and its interpretation live in one file and cannot drift. - Keep the generated script in the run directory beside the log and the reports — it is the record of what the tool was told to do, and without it every investigation restarts with the generator.
- Choose the return channel deliberately. Parsing the tool's log is Module 28's problem; having your own generated Tcl write a machine-readable file is not, because you own both ends.
- The values arrive typed —
json.loadsgives floats, avoiding Chapter 30.4's every-value-is-a-string problem rather than handling it. - Emitting JSON from Tcl needs the escaping function more, not less, because JSON is made of exactly the characters Tcl reserves — the first draft of that example failed on an unescaped
[. - Validate the file you read. A
JSONDecodeErrorwith a column number is a far better outcome than a corner silently missing from a comparison.
The next chapter draws the line this one has been leaning on: what can only happen inside the tool.
Where this fits
Part of the Python curriculum.