Linux & Bash · Part II · Module 35 · Reusable Flow Tooling and Hand-Off
Driving Tool Scripts: Bash Around Tcl
Every EDA tool takes a Tcl script, and almost every flow generates that script from the shell — which puts two languages that both use the dollar sign in the same heredoc. This chapter measures what one forgotten backslash does, finds a report stage that analysed nothing and exited 0, and then builds a generator that cannot lose the record of what the tool was asked to do.
Advanced28 min readLinuxPart IITclEDA ToolsCode Generation
Part II · Module 35 · Chapter 35.5 · Bash Around Tcl
Chapter 35.4 handed data to Python through a file. This boundary is different in a way that makes it the most dangerous of the three: what crosses it is not data but instructions, and the receiving interpreter lives inside a tool that will do whatever the generated script says.
$ echo 'puts [info patchlevel]' | tclsh
8.5.91. Two Languages, One Dollar Sign
An EDA tool's interface is Tcl. A flow wants to tell it which block to work on. So the flow writes a script:
#!/bin/bash
TOP=$1
cat > bad.tcl <<EOF
set top $TOP
set stage reports
foreach f [glob -nocomplain \$stage/*.rpt] {
puts "found \$f for \$top"
}
set n [llength [glob -nocomplain reports/*.rpt]]
puts "count=\$n"
EOFEvery dollar sign in that heredoc has to be classified. $TOP is bash's — it must expand. \$stage, \$f, \$top, \$n are Tcl's — they must survive. The backslash is what tells them apart, and the author has to get the decision right once per variable.
Here it is right, and the result works:
$ cat bad.tcl
set top chip_a
set stage reports
foreach f [glob -nocomplain $stage/*.rpt] {
puts "found $f for $top"
}
set n [llength [glob -nocomplain reports/*.rpt]]
puts "count=$n"
$ tclsh bad.tcl
found reports/hold.rpt for chip_a
found reports/power.rpt for chip_a
found reports/setup.rpt for chip_a
count=3Note what the correct version looks like in the source: a thicket of \$. The eye stops checking them after the third one.
2. One Forgotten Backslash
The same generator, with one backslash missing — on $stage in the glob:
cat > trap.tcl <<EOF
set top $TOP
set stage reports
set files [glob -nocomplain $stage/*.rpt]
puts "top=\$top files=[llength \$files]"
foreach f \$files { puts " \$f" }
EOFWhat bash wrote:
$ cat trap.tcl
set top chip_a
set stage reports
set files [glob -nocomplain /*.rpt]
foreach f $files { puts " $f" }$stage/*.rpt became /*.rpt. Bash has no variable called stage, so it expanded to nothing — and what was meant to be a glob of the reports directory is now a glob of the root of the filesystem.
$ tclsh trap.tcl
top=chip_a files=0
rc=0A report stage that analysed zero reports, printed that it had analysed zero reports, and exited 0.
This is Chapter 34.7's failure in its purest form, and it is worth being precise about why this particular instance is so hard to catch:
- the generated script is valid Tcl — nothing is malformed
- it runs to completion — no error, no stack trace
glob -nocomplainis there deliberately, and it is doing exactly its job- the output contains the number 0, which is a true statement about
/ - the status is 0, because nothing failed
Every downstream check this track has built would pass it. Chapter 32.3's extractor would find no failures. Chapter 32.7's gate would see a zero status. The only visible symptom is a count, in a line somebody has to read.
3. The Literal Dollar Needs Three Characters
There is a second escaping trap, and it bites the case where the Tcl script needs a $ that is not a variable:
cat > d.tcl <<EOF
set a "one backslash: \$100"
set b "two backslashes: \\\$100"
EOF$ cat -v d.tcl
set a "one backslash: $100"
set b "two backslashes: \$100"
$ tclsh d.tcl
can't read "100": no such variable
while executing
"set a "one backslash: $100""
(file "d.tcl" line 1)\$ is the escape that is correct for bash — it produces a literal $ in the file. But a literal $ in Tcl source is a variable reference, and $100 means "the variable named 100". To get a dollar that Tcl also treats as literal you need \$ in the generated file, which means \\\$ in the heredoc — three characters for one dollar sign.
So the rule for an unquoted heredoc generating Tcl is:
| in the heredoc | in the file | Tcl sees |
|---|---|---|
$x | the value of bash's x | that text |
\$x | $x | Tcl variable x |
\\\$x | \$x | a literal $x |
Three forms, one character apart, no diagnostic distinguishing them. Which is the argument for not using an unquoted heredoc at all.
4. The Quoted Heredoc and argv
cat > good.tcl <<'EOF'
if {$argc != 1} { puts stderr "good.tcl: need <top>"; exit 2 }
set top [lindex $argv 0]
set stage reports
set files [lsort [glob -nocomplain $stage/*.rpt]]
if {[llength $files] == 0} {
puts stderr "good.tcl: no reports in $stage"
exit 3
}
puts "top=$top files=[llength $files]"
foreach f $files { puts " $f" }
EOFQuoting the delimiter — <<'EOF' — turns off every bash expansion inside the heredoc, so what lands on disk is byte-identical to what is in the generator. No backslashes to classify, and therefore no backslash to forget. Chapter 12.4 introduced the quoted form; this is the case where it is not a stylistic preference but the difference between a working flow and the one in Section 2.
Values then arrive the way they arrive for any other program — as arguments:
$ tclsh good.tcl chip_a
top=chip_a files=3
reports/hold.rpt
reports/power.rpt
reports/setup.rpt
rc=0
$ tclsh good.tcl
good.tcl: need <top>
rc=2
$ cd empty && tclsh ../good.tcl chip_a
good.tcl: no reports in reports
rc=3Usage 2, no data 3 — Chapter 35.4's contract, in Tcl, with the empty case a refusal rather than a cheerful zero.
5. Tcl's Exit Status Says Less Than You Need
Before building the runner, one measurement that constrains its design. Three completely different Tcl failures:
$ tclsh err.tcl # set x [expr {1/0}]
divide by zero
while executing
"expr {1/0}"
(file "err.tcl" line 2)
rc=1
$ tclsh cmd.tcl # report_timing -delay max
invalid command name "report_timing"
(file "cmd.tcl" line 1)
rc=1
$ tclsh brace.tcl # if {1} { puts "x"
missing close-brace
(file "brace.tcl" line 1)
rc=1All three exit 1. A division by zero, a command that does not exist, and a syntactically broken file are indistinguishable by status — and 1 is the value this track reserves for a test that failed (Chapter 21.4). So a flow step that reads only the status cannot tell "the design has a problem" from "my generator wrote garbage".
Explicit statuses do pass through faithfully:
exit 0 -> rc=0
exit 1 -> rc=1
exit 2 -> rc=2
exit 3 -> rc=3Which is why good.tcl above uses exit 2 and exit 3 explicitly, and why the runner must translate: anything the interpreter itself decided is not a test result.
6. A Dry-Run Check, and Its Measured Limit
bash -n (Chapter 20.8) parses a script without running it. Tcl has no -n — but it has info complete, which answers "is this source syntactically finished?" without evaluating it:
# tclcheck.tcl <script> -- the nearest thing Tcl has to `bash -n`.
# Reads the file WITHOUT executing it and reports whether it is syntactically
# complete. Exits 0 complete, 1 incomplete, 2 unreadable.
if {$argc != 1} { puts stderr "tclcheck: need <script>"; exit 2 }
set p [lindex $argv 0]
if {[catch {open $p r} ch]} { puts stderr "tclcheck: $ch"; exit 2 }
set src [read $ch]; close $ch
if {[string length [string trim $src]] == 0} {
puts stderr "tclcheck: $p is empty"; exit 1
}
if {![info complete $src]} {
puts stderr "tclcheck: $p is syntactically incomplete (unbalanced brace/bracket/quote)"
exit 1
}
puts "tclcheck: $p complete ([llength [split [string trim $src] \n]] lines)"
exit 0Proven able to fail, on three different inputs:
| input | result | rc |
|---|---|---|
good.tcl | complete (11 lines) | 0 |
brace.tcl | syntactically incomplete (unbalanced brace/bracket/quote) | 1 |
zero.tcl | is empty | 1 |
nosuch.tcl | couldn't open "nosuch.tcl": no such file or directory | 2 |
And its limit, measured rather than assumed:
| input | real defect | tclcheck says |
|---|---|---|
cmd.tcl | report_timing is not a command | complete (1 lines) |
d.tcl | $100 is an unset variable | complete (4 lines) |
info complete checks structure, not names — exactly like bash -n, which also accepts a call to a command that does not exist. It catches the generator that stopped halfway and the generator that wrote nothing; it cannot catch the generator that wrote a plausible wrong thing. Publishing that boundary matters, because a check whose limits are unknown gets trusted beyond them.
A tool's own Tcl is a further step away: report_timing is not a command in tclsh and is one inside the tool, so tclcheck run on a tool script can only ever answer the structural question. That is still worth having — it is the difference between the broken and gone cases below being caught before the tool licence is taken out.
7. The Runner That Keeps the Evidence
Here is the shape almost every flow actually has, and the line that ruins it:
TCL=/tmp/flow.$$.tcl
cat > "$TCL" <<EOF
…
EOF
tclsh "$TCL"
rc=$?
rm -f "$TCL" # <-- the evidence$ /bin/bash runflow_bad.sh chip_a
analysed 0 reports for chip_a
flow rc=0
what is left on disk to explain that? -> 0 file(s)Zero files. The step did nothing, said so in one line, exited 0, and deleted the only artefact that could explain it. Reproducing it means re-running the generator — which, since it produced the bug, will produce it again and leave nothing again. Chapter 27.3 argued for keeping a run directory; a generated tool script is the single highest-value thing in it, because it is the only record of what the tool was asked to do.
So:
#!/bin/bash
# runtcl.sh -- generate a Tcl script, keep it, check it, then run it.
#
# CONTRACT
# run_tcl <script> [args...]
# 0 the script ran and exited 0
# 2 usage, or the script is absent/empty/incomplete (did NOT run)
# 3 the script ran and exited non-zero
# The generated script is NEVER deleted. Its path and digest are logged so
# the run can be reproduced from the artefact rather than from the generator.
set -u
FLOW_NAME=${FLOW_NAME:-runtcl}
die() { printf '%s: %s\n' "$FLOW_NAME" "$1" >&2; exit "${2:-2}"; }
note() { printf '%s: note: %s\n' "$FLOW_NAME" "$1" >&2; }
gen_report_script() {
# $1 = output path. QUOTED heredoc: bash expands NOTHING, so what is
# written is byte-identical to what is read here. Values arrive via argv.
cat > "$1" <<'TCL'
if {$argc != 2} { puts stderr "report.tcl: need <top> <stage>"; exit 2 }
set top [lindex $argv 0]
set stage [lindex $argv 1]
if {![file isdirectory $stage]} {
puts stderr "report.tcl: not a directory: $stage"; exit 3
}
set files [lsort [glob -nocomplain [file join $stage *.rpt]]]
if {[llength $files] == 0} {
puts stderr "report.tcl: no .rpt files in $stage"; exit 3
}
puts "top=$top stage=$stage reports=[llength $files]"
foreach f $files { puts " [file tail $f]" }
exit 0
TCL
}
run_tcl() {
[ $# -ge 1 ] || die "run_tcl needs a script path"
local s=$1; shift
[ -f "$s" ] || die "no such script: $s"
[ -s "$s" ] || die "generated script is EMPTY: $s (generation failed)"
tclsh "$(dirname "$0")/tclcheck.tcl" "$s" >/dev/null || die "will not run: $s"
note "running $s ($(/sbin/md5 -q "$s" | cut -c1-8)) args: $*"
tclsh "$s" "$@"
local rc=$?
[ "$rc" -eq 0 ] || { note "script exited $rc"; return 3; }
return 0
}and the driver:
#!/bin/bash
. "$(dirname "$0")/runtcl.sh"
mkdir -p gen
case ${1:-ok} in
ok) gen_report_script gen/report.tcl ;;
empty) : > gen/report.tcl ;;
broken) printf 'if {1} { puts "x"\n' > gen/report.tcl ;;
gone) rm -f gen/report.tcl ;;
esac
run_tcl gen/report.tcl "${2:-chip_a}" "${3:-reports}"
rc=$? # captured BEFORE anything else runs
note "flow finished rc=$rc"
exit "$rc" # the driver's status IS the step's statusSix cases, each a different way the step can go wrong, with the status read outside the driver:
| case | what happened | output | driver rc |
|---|---|---|---|
ok | generated, checked, ran | top=chip_a stage=reports reports=3 | 0 |
empty | generation produced 0 bytes | generated script is EMPTY … (generation failed) | 2 |
broken | generation stopped mid-brace | syntactically incomplete … / will not run | 2 |
gone | no script at all | no such script: gen/report.tcl | 2 |
nodata | ran; stage held no reports | report.tcl: no .rpt files in nostage | 3 |
nodir | ran; stage does not exist | report.tcl: not a directory: no_such_dir | 3 |
The three 2s never launched the interpreter, which is the distinction Section 5 said the status could not make and the runner therefore has to: 2 means the flow is broken, 3 means the tool ran and refused, 0 means it worked. And the artefact survives:
$ ls -l gen/report.tcl
486 bytes gen/report.tcl
$ tclsh gen/report.tcl chip_a reports
top=chip_a stage=reports reports=3
hold.rpt
power.rpt
setup.rpt
rc=0 (no bash, no drive.sh, no regeneration)8. What This Means for a Real Tool
tclsh stands in here for a tool interpreter, and three differences matter in a real flow.
The tool's Tcl is a superset. report_timing, get_cells, create_clock exist inside the tool and nowhere else, so tclcheck answers only the structural question and the tool's own parse is the real check. Many tools have a mode that reads a script and exits — that mode, run early, is this chapter's Section 6 with the tool's own vocabulary.
A licence makes the pre-flight checks worth more. The empty, broken and gone cases above were rejected without launching anything. Chapter 34.3 measured what contention costs; a flow that takes out a licence to run a zero-byte script pays that cost for nothing, and then reports success.
And the tool's exit status is its own contract, which Chapter 27.1 measured and found varied: Icarus returns its error count modulo 256, so 256 errors exits 0. run_tcl above returns 3 for "the script ran and exited non-zero", which deliberately throws away the tool's value — because a value whose meaning is unknown is worse than a value whose meaning is fixed by the flow.
The division from Chapter 35.4 extends cleanly:
The shell decides what to run and keeps the record. Tcl says what the tool should do. Neither interpolates the other's variables.
Practice
- Write a generator with an unquoted heredoc that interpolates one bash value and uses three Tcl variables, and run it.
- Count the backslashes you had to get right.
- Remove exactly one backslash and read the generated file.
- Run it and read the output and the status.
- List every reason the result passes the checks this track has built.
- Add
set -uand read both the diagnostic and the exit status. - Add
set -euand then check whether the output file exists and how big it is. - Run an empty Tcl script and read its status.
- Describe the three-mechanism chain that makes a failed generation report success.
- Generate a literal
$100and count the characters it took. - Build the three-row escaping table from your own measurements.
- Rewrite the generator with a quoted heredoc and
argv. - Diff the generated file against the heredoc body and confirm they are identical.
- Give the Tcl script usage and no-data exit statuses and test both.
- State the generate-structure-not-values rule and its four benefits.
- Measure the digest across two runs.
- Cause three different Tcl errors and compare their exit statuses.
- Say why status 1 is ambiguous in this flow.
- Write a
tclcheckusinginfo complete. - Prove it able to fail on at least three inputs.
- Find two defects it cannot catch and say what that tells you about
bash -n. - Write the generate-run-delete pattern and then ask what explains its output.
- Build a runner that refuses to launch an absent, empty or incomplete script.
- Give it three distinct statuses and say what each means.
- Read its status from outside on all six cases.
- Check your own driver for the trailing-
echobug. - Reproduce the step by hand from the kept artefact alone.
- Say what a licence adds to the pre-flight checks.
Check Your Understanding
- Why does a bash-generated Tcl script need every dollar sign classified?
- What did one forgotten backslash change in the generated file?
- What did the resulting script glob, and what did it report?
- What was its exit status?
- Give four reasons the bad run passes downstream checks.
- Does
set -ucatch it? What happens to the status? - What does
set -euleave on disk, and why? - What does an empty Tcl script do?
- State the three-mechanism chain.
- How many characters make a literal Tcl dollar in an unquoted heredoc?
- What does
\$xproduce, and what does Tcl do with it? - What does quoting the heredoc delimiter do?
- How do values reach the script then?
- What statuses does
good.tcluse, and for what? - State the generate-structure-not-values rule.
- Name its four benefits.
- What is the limiting case, and how do you handle it?
- What status do a divide-by-zero, an unknown command and a missing brace produce?
- Why is that a problem for a flow step?
- Do explicit
exitvalues pass through? - What is Tcl's nearest equivalent to
bash -n? - Name three inputs it was proven able to reject.
- Name two defects it cannot catch, and what that says about
bash -n. - What is wrong with generate-run-delete?
- What did the bad flow leave on disk to explain its output?
- What do the runner's statuses 0, 2 and 3 mean?
- Which cases never launch the interpreter, and why does that matter?
- What bug did the first driver have, and why was it invisible from inside?
- How many times has that mechanism appeared in Part II?
- Why does
run_tcldiscard the tool's own exit value? - State the three-way division for this boundary.
9. Remember
- An unquoted heredoc generating Tcl puts two languages' dollar signs in one block, and the author must classify every one of them correctly.
- One forgotten backslash globbed
/instead ofreports/, found 0 files, printedfiles=0, and exited 0 — valid Tcl, no error, true output, clean status. set -ucatches it and the status is still lost;set -euaborts but>has already created a zero-byte file, and an empty Tcl script exits 0.- A literal Tcl dollar takes three characters —
\\\$— because\$produces a Tcl variable reference. - Quote the delimiter.
<<'EOF'turns off all expansion, so the file is byte-identical to the source and there is no backslash to forget. - Generate structure, never values. Pass values as arguments — for no escaping, a stable digest (
ef2979f9…twice), a script you can run by hand, and an artefact worth reviewing. - Every Tcl failure exits 1 — divide by zero, unknown command, missing brace — which is the same value as a failing test, so the runner must translate.
info completeis Tcl'sbash -n: it rejected an unbalanced brace, an empty file and a missing file, and accepted both an invalid command name and the$100bug. Structure, not names.- Never delete the generated script. The bad flow left zero files to explain a step that analysed nothing.
- Three statuses: 0 worked, 2 the flow is broken and nothing launched, 3 the tool ran and refused — and the three
2s cost no licence. - Capture
rc=$?immediately andexit "$rc". My own corrected driver exited 0 on a failing run because a reportingechowas its last command — the fourth instance of that mechanism in Part II. - The shell decides what to run and keeps the record; Tcl says what the tool should do; neither interpolates the other's variables.
10. What Is Next
All three hand-off boundaries are now drawn, and the flow is a library, a configuration file, a dependency graph, a reporting script and a generated tool script. That is more moving parts than one person holds in their head after a month away — which is Chapter 35.6's subject: what has to be written down so somebody else can run and repair this without asking you, and what is not worth writing because the script should say it instead.
Where this fits
Part of the Linux & Bash curriculum.
