Skip to content
VLSI Mentor

Linux & Bash · Part II · Module 35 · Reusable Flow Tooling and Hand-Off

Driving Tool Scripts: Bash Around Tcl

Every EDA tool takes a Tcl script, and almost every flow generates that script from the shell — which puts two languages that both use the dollar sign in the same heredoc. This chapter measures what one forgotten backslash does, finds a report stage that analysed nothing and exited 0, and then builds a generator that cannot lose the record of what the tool was asked to do.

Advanced28 min readLinuxPart IITclEDA ToolsCode Generation

Part II · Module 35 · Chapter 35.5 · Bash Around Tcl

Chapter 35.4 handed data to Python through a file. This boundary is different in a way that makes it the most dangerous of the three: what crosses it is not data but instructions, and the receiving interpreter lives inside a tool that will do whatever the generated script says.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
$ echo 'puts [info patchlevel]' | tclsh
8.5.9

1. Two Languages, One Dollar Sign

An EDA tool's interface is Tcl. A flow wants to tell it which block to work on. So the flow writes a script:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
#!/bin/bash
TOP=$1
cat > bad.tcl <<EOF
set top $TOP
set stage reports
foreach f [glob -nocomplain \$stage/*.rpt] {
  puts "found \$f for \$top"
}
set n [llength [glob -nocomplain reports/*.rpt]]
puts "count=\$n"
EOF

Every dollar sign in that heredoc has to be classified. $TOP is bash's — it must expand. \$stage, \$f, \$top, \$n are Tcl's — they must survive. The backslash is what tells them apart, and the author has to get the decision right once per variable.

Here it is right, and the result works:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
$ cat bad.tcl
set top chip_a
set stage reports
foreach f [glob -nocomplain $stage/*.rpt] {
  puts "found $f for $top"
}
set n [llength [glob -nocomplain reports/*.rpt]]
puts "count=$n"

$ tclsh bad.tcl
found reports/hold.rpt for chip_a
found reports/power.rpt for chip_a
found reports/setup.rpt for chip_a
count=3

Note what the correct version looks like in the source: a thicket of \$. The eye stops checking them after the third one.

2. One Forgotten Backslash

The same generator, with one backslash missing — on $stage in the glob:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
cat > trap.tcl <<EOF
set top $TOP
set stage reports
set files [glob -nocomplain $stage/*.rpt]
puts "top=\$top files=[llength \$files]"
foreach f \$files { puts "  \$f" }
EOF

What bash wrote:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
$ cat trap.tcl
set top chip_a
set stage reports
set files [glob -nocomplain /*.rpt]
foreach f $files { puts "  $f" }

$stage/*.rpt became /*.rpt. Bash has no variable called stage, so it expanded to nothing — and what was meant to be a glob of the reports directory is now a glob of the root of the filesystem.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
$ tclsh trap.tcl
top=chip_a files=0
  rc=0

A report stage that analysed zero reports, printed that it had analysed zero reports, and exited 0.

This is Chapter 34.7's failure in its purest form, and it is worth being precise about why this particular instance is so hard to catch:

  • the generated script is valid Tcl — nothing is malformed
  • it runs to completion — no error, no stack trace
  • glob -nocomplain is there deliberately, and it is doing exactly its job
  • the output contains the number 0, which is a true statement about /
  • the status is 0, because nothing failed

Every downstream check this track has built would pass it. Chapter 32.3's extractor would find no failures. Chapter 32.7's gate would see a zero status. The only visible symptom is a count, in a line somebody has to read.

3. The Literal Dollar Needs Three Characters

There is a second escaping trap, and it bites the case where the Tcl script needs a $ that is not a variable:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
cat > d.tcl <<EOF
set a "one backslash: \$100"
set b "two backslashes: \\\$100"
EOF
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
$ cat -v d.tcl
set a "one backslash: $100"
set b "two backslashes: \$100"

$ tclsh d.tcl
can't read "100": no such variable
    while executing
"set a "one backslash: $100""
    (file "d.tcl" line 1)

\$ is the escape that is correct for bash — it produces a literal $ in the file. But a literal $ in Tcl source is a variable reference, and $100 means "the variable named 100". To get a dollar that Tcl also treats as literal you need \$ in the generated file, which means \\\$ in the heredoc — three characters for one dollar sign.

So the rule for an unquoted heredoc generating Tcl is:

in the heredocin the fileTcl sees
$xthe value of bash's xthat text
\$x$xTcl variable x
\\\$x\$xa literal $x

Three forms, one character apart, no diagnostic distinguishing them. Which is the argument for not using an unquoted heredoc at all.

4. The Quoted Heredoc and argv

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
cat > good.tcl <<'EOF'
if {$argc != 1} { puts stderr "good.tcl: need <top>"; exit 2 }
set top [lindex $argv 0]
set stage reports
set files [lsort [glob -nocomplain $stage/*.rpt]]
if {[llength $files] == 0} {
  puts stderr "good.tcl: no reports in $stage"
  exit 3
}
puts "top=$top files=[llength $files]"
foreach f $files { puts "  $f" }
EOF

Quoting the delimiter — <<'EOF' — turns off every bash expansion inside the heredoc, so what lands on disk is byte-identical to what is in the generator. No backslashes to classify, and therefore no backslash to forget. Chapter 12.4 introduced the quoted form; this is the case where it is not a stylistic preference but the difference between a working flow and the one in Section 2.

Values then arrive the way they arrive for any other program — as arguments:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
$ tclsh good.tcl chip_a
top=chip_a files=3
  reports/hold.rpt
  reports/power.rpt
  reports/setup.rpt
  rc=0

$ tclsh good.tcl
good.tcl: need <top>
  rc=2

$ cd empty && tclsh ../good.tcl chip_a
good.tcl: no reports in reports
  rc=3

Usage 2, no data 3 — Chapter 35.4's contract, in Tcl, with the empty case a refusal rather than a cheerful zero.

5. Tcl's Exit Status Says Less Than You Need

Before building the runner, one measurement that constrains its design. Three completely different Tcl failures:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
$ tclsh err.tcl          # set x [expr {1/0}]
divide by zero
    while executing
"expr {1/0}"
    (file "err.tcl" line 2)
  rc=1

$ tclsh cmd.tcl          # report_timing -delay max
invalid command name "report_timing"
    (file "cmd.tcl" line 1)
  rc=1

$ tclsh brace.tcl        # if {1} { puts "x"
missing close-brace
    (file "brace.tcl" line 1)
  rc=1

All three exit 1. A division by zero, a command that does not exist, and a syntactically broken file are indistinguishable by status — and 1 is the value this track reserves for a test that failed (Chapter 21.4). So a flow step that reads only the status cannot tell "the design has a problem" from "my generator wrote garbage".

Explicit statuses do pass through faithfully:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
  exit 0 -> rc=0
  exit 1 -> rc=1
  exit 2 -> rc=2
  exit 3 -> rc=3

Which is why good.tcl above uses exit 2 and exit 3 explicitly, and why the runner must translate: anything the interpreter itself decided is not a test result.

6. A Dry-Run Check, and Its Measured Limit

bash -n (Chapter 20.8) parses a script without running it. Tcl has no -n — but it has info complete, which answers "is this source syntactically finished?" without evaluating it:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
# tclcheck.tcl <script> -- the nearest thing Tcl has to `bash -n`.
# Reads the file WITHOUT executing it and reports whether it is syntactically
# complete. Exits 0 complete, 1 incomplete, 2 unreadable.
if {$argc != 1} { puts stderr "tclcheck: need <script>"; exit 2 }
set p [lindex $argv 0]
if {[catch {open $p r} ch]} { puts stderr "tclcheck: $ch"; exit 2 }
set src [read $ch]; close $ch
if {[string length [string trim $src]] == 0} {
    puts stderr "tclcheck: $p is empty"; exit 1
}
if {![info complete $src]} {
    puts stderr "tclcheck: $p is syntactically incomplete (unbalanced brace/bracket/quote)"
    exit 1
}
puts "tclcheck: $p complete ([llength [split [string trim $src] \n]] lines)"
exit 0

Proven able to fail, on three different inputs:

inputresultrc
good.tclcomplete (11 lines)0
brace.tclsyntactically incomplete (unbalanced brace/bracket/quote)1
zero.tclis empty1
nosuch.tclcouldn't open "nosuch.tcl": no such file or directory2

And its limit, measured rather than assumed:

inputreal defecttclcheck says
cmd.tclreport_timing is not a commandcomplete (1 lines)
d.tcl$100 is an unset variablecomplete (4 lines)

info complete checks structure, not names — exactly like bash -n, which also accepts a call to a command that does not exist. It catches the generator that stopped halfway and the generator that wrote nothing; it cannot catch the generator that wrote a plausible wrong thing. Publishing that boundary matters, because a check whose limits are unknown gets trusted beyond them.

A tool's own Tcl is a further step away: report_timing is not a command in tclsh and is one inside the tool, so tclcheck run on a tool script can only ever answer the structural question. That is still worth having — it is the difference between the broken and gone cases below being caught before the tool licence is taken out.

7. The Runner That Keeps the Evidence

Here is the shape almost every flow actually has, and the line that ruins it:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
TCL=/tmp/flow.$$.tcl
cat > "$TCL" <<EOF
…
EOF
tclsh "$TCL"
rc=$?
rm -f "$TCL"          # <-- the evidence
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
$ /bin/bash runflow_bad.sh chip_a
analysed 0 reports for chip_a
  flow rc=0
  what is left on disk to explain that? ->  0 file(s)

Zero files. The step did nothing, said so in one line, exited 0, and deleted the only artefact that could explain it. Reproducing it means re-running the generator — which, since it produced the bug, will produce it again and leave nothing again. Chapter 27.3 argued for keeping a run directory; a generated tool script is the single highest-value thing in it, because it is the only record of what the tool was asked to do.

So:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
#!/bin/bash
# runtcl.sh -- generate a Tcl script, keep it, check it, then run it.
#
# CONTRACT
#   run_tcl <script> [args...]
#     0  the script ran and exited 0
#     2  usage, or the script is absent/empty/incomplete  (did NOT run)
#     3  the script ran and exited non-zero
#   The generated script is NEVER deleted. Its path and digest are logged so
#   the run can be reproduced from the artefact rather than from the generator.
set -u
FLOW_NAME=${FLOW_NAME:-runtcl}
die()  { printf '%s: %s\n' "$FLOW_NAME" "$1" >&2; exit "${2:-2}"; }
note() { printf '%s: note: %s\n' "$FLOW_NAME" "$1" >&2; }

gen_report_script() {
  # $1 = output path.  QUOTED heredoc: bash expands NOTHING, so what is
  # written is byte-identical to what is read here. Values arrive via argv.
  cat > "$1" <<'TCL'
if {$argc != 2} { puts stderr "report.tcl: need <top> <stage>"; exit 2 }
set top   [lindex $argv 0]
set stage [lindex $argv 1]
if {![file isdirectory $stage]} {
    puts stderr "report.tcl: not a directory: $stage"; exit 3
}
set files [lsort [glob -nocomplain [file join $stage *.rpt]]]
if {[llength $files] == 0} {
    puts stderr "report.tcl: no .rpt files in $stage"; exit 3
}
puts "top=$top stage=$stage reports=[llength $files]"
foreach f $files { puts "  [file tail $f]" }
exit 0
TCL
}

run_tcl() {
  [ $# -ge 1 ] || die "run_tcl needs a script path"
  local s=$1; shift
  [ -f "$s" ] || die "no such script: $s"
  [ -s "$s" ] || die "generated script is EMPTY: $s (generation failed)"
  tclsh "$(dirname "$0")/tclcheck.tcl" "$s" >/dev/null || die "will not run: $s"
  note "running $s ($(/sbin/md5 -q "$s" | cut -c1-8)) args: $*"
  tclsh "$s" "$@"
  local rc=$?
  [ "$rc" -eq 0 ] || { note "script exited $rc"; return 3; }
  return 0
}

and the driver:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
#!/bin/bash
. "$(dirname "$0")/runtcl.sh"
mkdir -p gen
case ${1:-ok} in
  ok)     gen_report_script gen/report.tcl ;;
  empty)  : > gen/report.tcl ;;
  broken) printf 'if {1} { puts "x"\n' > gen/report.tcl ;;
  gone)   rm -f gen/report.tcl ;;
esac
run_tcl gen/report.tcl "${2:-chip_a}" "${3:-reports}"
rc=$?                      # captured BEFORE anything else runs
note "flow finished rc=$rc"
exit "$rc"                 # the driver's status IS the step's status

Six cases, each a different way the step can go wrong, with the status read outside the driver:

casewhat happenedoutputdriver rc
okgenerated, checked, rantop=chip_a stage=reports reports=30
emptygeneration produced 0 bytesgenerated script is EMPTY … (generation failed)2
brokengeneration stopped mid-bracesyntactically incomplete … / will not run2
goneno script at allno such script: gen/report.tcl2
nodataran; stage held no reportsreport.tcl: no .rpt files in nostage3
nodirran; stage does not existreport.tcl: not a directory: no_such_dir3

The three 2s never launched the interpreter, which is the distinction Section 5 said the status could not make and the runner therefore has to: 2 means the flow is broken, 3 means the tool ran and refused, 0 means it worked. And the artefact survives:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
$ ls -l gen/report.tcl
486 bytes  gen/report.tcl
$ tclsh gen/report.tcl chip_a reports
top=chip_a stage=reports reports=3
  hold.rpt
  power.rpt
  setup.rpt
  rc=0 (no bash, no drive.sh, no regeneration)

8. What This Means for a Real Tool

tclsh stands in here for a tool interpreter, and three differences matter in a real flow.

The tool's Tcl is a superset. report_timing, get_cells, create_clock exist inside the tool and nowhere else, so tclcheck answers only the structural question and the tool's own parse is the real check. Many tools have a mode that reads a script and exits — that mode, run early, is this chapter's Section 6 with the tool's own vocabulary.

A licence makes the pre-flight checks worth more. The empty, broken and gone cases above were rejected without launching anything. Chapter 34.3 measured what contention costs; a flow that takes out a licence to run a zero-byte script pays that cost for nothing, and then reports success.

And the tool's exit status is its own contract, which Chapter 27.1 measured and found varied: Icarus returns its error count modulo 256, so 256 errors exits 0. run_tcl above returns 3 for "the script ran and exited non-zero", which deliberately throws away the tool's value — because a value whose meaning is unknown is worse than a value whose meaning is fixed by the flow.

The division from Chapter 35.4 extends cleanly:

The shell decides what to run and keeps the record. Tcl says what the tool should do. Neither interpolates the other's variables.

Practice

  1. Write a generator with an unquoted heredoc that interpolates one bash value and uses three Tcl variables, and run it.
  2. Count the backslashes you had to get right.
  3. Remove exactly one backslash and read the generated file.
  4. Run it and read the output and the status.
  5. List every reason the result passes the checks this track has built.
  6. Add set -u and read both the diagnostic and the exit status.
  7. Add set -eu and then check whether the output file exists and how big it is.
  8. Run an empty Tcl script and read its status.
  9. Describe the three-mechanism chain that makes a failed generation report success.
  10. Generate a literal $100 and count the characters it took.
  11. Build the three-row escaping table from your own measurements.
  12. Rewrite the generator with a quoted heredoc and argv.
  13. Diff the generated file against the heredoc body and confirm they are identical.
  14. Give the Tcl script usage and no-data exit statuses and test both.
  15. State the generate-structure-not-values rule and its four benefits.
  16. Measure the digest across two runs.
  17. Cause three different Tcl errors and compare their exit statuses.
  18. Say why status 1 is ambiguous in this flow.
  19. Write a tclcheck using info complete.
  20. Prove it able to fail on at least three inputs.
  21. Find two defects it cannot catch and say what that tells you about bash -n.
  22. Write the generate-run-delete pattern and then ask what explains its output.
  23. Build a runner that refuses to launch an absent, empty or incomplete script.
  24. Give it three distinct statuses and say what each means.
  25. Read its status from outside on all six cases.
  26. Check your own driver for the trailing-echo bug.
  27. Reproduce the step by hand from the kept artefact alone.
  28. Say what a licence adds to the pre-flight checks.

Check Your Understanding

  1. Why does a bash-generated Tcl script need every dollar sign classified?
  2. What did one forgotten backslash change in the generated file?
  3. What did the resulting script glob, and what did it report?
  4. What was its exit status?
  5. Give four reasons the bad run passes downstream checks.
  6. Does set -u catch it? What happens to the status?
  7. What does set -eu leave on disk, and why?
  8. What does an empty Tcl script do?
  9. State the three-mechanism chain.
  10. How many characters make a literal Tcl dollar in an unquoted heredoc?
  11. What does \$x produce, and what does Tcl do with it?
  12. What does quoting the heredoc delimiter do?
  13. How do values reach the script then?
  14. What statuses does good.tcl use, and for what?
  15. State the generate-structure-not-values rule.
  16. Name its four benefits.
  17. What is the limiting case, and how do you handle it?
  18. What status do a divide-by-zero, an unknown command and a missing brace produce?
  19. Why is that a problem for a flow step?
  20. Do explicit exit values pass through?
  21. What is Tcl's nearest equivalent to bash -n?
  22. Name three inputs it was proven able to reject.
  23. Name two defects it cannot catch, and what that says about bash -n.
  24. What is wrong with generate-run-delete?
  25. What did the bad flow leave on disk to explain its output?
  26. What do the runner's statuses 0, 2 and 3 mean?
  27. Which cases never launch the interpreter, and why does that matter?
  28. What bug did the first driver have, and why was it invisible from inside?
  29. How many times has that mechanism appeared in Part II?
  30. Why does run_tcl discard the tool's own exit value?
  31. State the three-way division for this boundary.

9. Remember

  • An unquoted heredoc generating Tcl puts two languages' dollar signs in one block, and the author must classify every one of them correctly.
  • One forgotten backslash globbed / instead of reports/, found 0 files, printed files=0, and exited 0 — valid Tcl, no error, true output, clean status.
  • set -u catches it and the status is still lost; set -eu aborts but > has already created a zero-byte file, and an empty Tcl script exits 0.
  • A literal Tcl dollar takes three characters — \\\$ — because \$ produces a Tcl variable reference.
  • Quote the delimiter. <<'EOF' turns off all expansion, so the file is byte-identical to the source and there is no backslash to forget.
  • Generate structure, never values. Pass values as arguments — for no escaping, a stable digest (ef2979f9… twice), a script you can run by hand, and an artefact worth reviewing.
  • Every Tcl failure exits 1 — divide by zero, unknown command, missing brace — which is the same value as a failing test, so the runner must translate.
  • info complete is Tcl's bash -n: it rejected an unbalanced brace, an empty file and a missing file, and accepted both an invalid command name and the $100 bug. Structure, not names.
  • Never delete the generated script. The bad flow left zero files to explain a step that analysed nothing.
  • Three statuses: 0 worked, 2 the flow is broken and nothing launched, 3 the tool ran and refused — and the three 2s cost no licence.
  • Capture rc=$? immediately and exit "$rc". My own corrected driver exited 0 on a failing run because a reporting echo was its last command — the fourth instance of that mechanism in Part II.
  • The shell decides what to run and keeps the record; Tcl says what the tool should do; neither interpolates the other's variables.

10. What Is Next

All three hand-off boundaries are now drawn, and the flow is a library, a configuration file, a dependency graph, a reporting script and a generated tool script. That is more moving parts than one person holds in their head after a month away — which is Chapter 35.6's subject: what has to be written down so somebody else can run and repair this without asking you, and what is not worth writing because the script should say it instead.

Where this fits

Part of the Linux & Bash curriculum.