Bash Launches the Tool

Part II · Module 28 · Chapter 28.1 · Bash Launches the Tool

Module 27 built the Tcl a project lives on. This module is about where Tcl stops — and the first boundary is the one CI crosses.

1. What the wrapper is for

Four jobs, and only four:

• Set the environment the tool needs — licence variables, library paths, TCLLIBPATH.
• Create the run directory, so the tool writes somewhere known.
• Invoke the tool, with the log going to a file inside that directory.
• Propagate the exit status, so the caller learns what happened.

The flow itself stays in Tcl (Chapter 28.4 is about why). The wrapper is the adapter between CI's world and the tool's.

And the fourth job is the one that goes wrong.

2. A pipe destroys the exit status

=== a tool that exits 3 ===
    direct                      : $? = 3
    through a pipe to tee       : $? = 0

3. Where the real status went, and three traps on it

The status is not lost; it is in PIPESTATUS.

=== where the real status went ===
    PIPESTATUS, read immediately: (3 0)  first = 3
    one ':' command in between  : (0)  first = 0

Trap one: it is destroyed by the very next command — including :.

: is the shell's no-op. It does nothing at all, and running it replaces the array. So PIPESTATUS must be captured in the statement immediately following the pipeline, and anything at all in between — a local declaration, an echo, a comment-like : — is too late. Measured identically in bash and zsh.

=== the same array under zsh ===
    ${PIPESTATUS[0]} (bash spelling) : ''
    $pipestatus      (zsh spelling)  : (3 0)

Trap two: the name is not portable. Under zsh, ${PIPESTATUS[0]} is the empty string — the array is called pipestatus and it is 1-based, not 0-based. A wrapper written for bash and run as zsh wrapper.sh gets '', and [ '' -ne 0 ] is a syntax error rather than a failure.

Trap three: it is an array, so quoting it correctly is its own small problem — "${PIPESTATUS[@]}" to copy, ${PIPESTATUS[0]} to read the first, and a bare $PIPESTATUS gives only element zero in bash and the whole array in zsh.

4. Two fixes, and which to prefer

=== the two fixes ===
    set -o pipefail             : $? = 3
    no pipe: redirect to a file : $? = 3  (log has 2 lines)

set -o pipefail makes a pipeline's status the first non-zero one. It works, it is one line, and it exists in this bash — 3.2.57, the macOS system shell — which is worth knowing because much advice says pipefail is bash 4 and up. zsh has it as setopt pipefail, also verified.

Not piping at all is the one this chapter prefers:

tclsh "$PROJ/flow/stage.tcl" "$DESIGN" "$STAGE" > "$LOG" 2>&1
status=$?

The status is preserved with no shell option, no array, and no portability question — and the log still lands on disk, which was the only reason for the pipe. If somebody also wants it on the terminal, cat "$LOG" afterwards costs nothing and cannot lose a status.

5. The wrapper

#!/usr/bin/env bash
set -u                 # an unset variable is an error
set -o pipefail        # a pipeline's status is its FIRST failure, not its last

# ---- locate ourselves, not the caller's directory -----------------------
HERE=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
PROJ=$(dirname "$HERE")

# ---- the environment the tool needs -------------------------------------
export FLOW_ROOT="$PROJ/flow"
export TCLLIBPATH="$PROJ/flow/lib"
export RUN_DIR="$OUTROOT/$DESIGN/$STAGE"
mkdir -p "$RUN_DIR"
LOG="$RUN_DIR/run.log"

# ---- invoke the tool ----------------------------------------------------
/usr/bin/tclsh "$PROJ/flow/stage.tcl" "$DESIGN" "$STAGE" > "$LOG" 2>&1
status=$?

case $status in
    0) echo "PASS  $DESIGN/$STAGE  ($(wc -l < "$LOG" | tr -d ' ') log lines)" ;;
    2) echo "SETUP $DESIGN/$STAGE  — see $LOG" >&2 ;;
    *) echo "FAIL  $DESIGN/$STAGE  status $status — see $LOG" >&2 ;;
esac
exit $status

=== the wrapper, four invocations ===
    a complete run                     exit 0
        PASS  core_top/synth  (5 log lines)
    a missing option                   exit 2
        -s STAGE is required
        usage: run_stage.sh -d DESIGN -s STAGE [-o OUTROOT]
    an unknown stage                   exit 2
        SETUP core_top/bogus  — see /private/tmp/m28/work/core_top/bogus/run.log
    a tool failure                     exit 1
        FAIL  core_top/synth  status 1 — see /private/tmp/m28/work/core_top/synth/run.log

Six decisions worth naming, each one a lesson from Module 27 restated in shell.

HERE=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) is the shell's file normalize [info script] (Chapter 27.1 §3). ${BASH_SOURCE[0]} rather than $0 because $0 is the shell's name when the script is sourced rather than executed, and cd … && pwd is what makes it absolute.

set -u turns a typo'd variable into an error instead of an empty string. That matters most in paths: mkdir -p "$OUTROT/$DESIGN" with -u fails loudly, and without it creates a directory called /core_top.

export, not set. The tool is a child process, so the environment is the channel — and TCLLIBPATH is how a vendor interpreter finds the shared libraries of Chapter 27.3.

Every expansion is quoted. "$LOG", "$RUN_DIR", "$PROJ". An unquoted path with a space becomes two arguments, which is the shell's version of Chapter 6.13's {*} problem and fails only on the one project whose directory has a space in it.

The status is read once into a variable on the line after the command. Anything else in between loses it, for the §3 reason — and this is a plain $? rather than PIPESTATUS precisely because there is no pipe.

Three outcome classes, three statuses — 0, 2 for a setup problem, anything else for a flow failure — which is Chapter 27.1 §7's split surviving the hand-off to the shell. The case maps the tool's own convention onto the wrapper's message, so CI's log says which queue a failure belongs to without reading the run log.

6. What the wrapper must not contain

Being specific, because this file grows if nobody says so.

No flow logic. No parsing of reports, no computing utilisation, no looping over corners. Chapter 28.4 argues this with a measurement; the short version is that the data is already structured inside the tool session and extracting it to manipulate in the shell is throwing away the structure.

No arithmetic on the flow's numbers. Bash cannot do floating-point arithmetic at all (Chapter 28.4 §3), so any attempt will be routed through bc or awk — which is two more languages in the wrapper and a third set of numeric semantics.

No decisions that belong to Make. "Has the netlist changed since the SDC" is Chapter 28.2's question, and answering it in shell with [ file1 -nt file2 ] reimplements Make badly.

No cd whose effect outlives it. A wrapper that cds and leaves the caller's shell elsewhere is an unpleasant surprise; a subshell — ( cd "$X" && … ) — confines it.

7. Exercises

Exercise 1. What are the wrapper's four jobs?

Exercise 2. A tool exits 3 and the shell reports 0. Why?

Exercise 3. Why is that the reason CI goes green on a broken run?

Exercise 4. PIPESTATUS read immediately gave (3 0); with one : in between it gave (0). What is the rule?

Exercise 5. What does ${PIPESTATUS[0]} give under zsh, and what is the array actually called there?

Exercise 6. Give the two fixes, and say which this chapter prefers and why.

Exercise 7. State the general principle behind both fixes.

Exercise 8. Why ${BASH_SOURCE[0]} rather than $0?

Exercise 9. What does set -u prevent, with a concrete path example?

Exercise 10. Why does the wrapper not use set -e?