Bash Launches Perl: The CI Entry Point

Part II · Module 26 · Chapter 26.1 · Bash Launches Perl: The CI Entry Point

1. One Number, Passed Up a Chain

Here is how a timing check usually runs in a real project. Nobody types the Perl command. A CI job runs make sta_check; the Makefile recipe runs a Bash script; the Bash script runs the Perl parser.

The arrows point down the chain, because that is the direction commands travel. The information that matters travels the other way: the parser's exit status has to come back up through every link. Chapter 15.4 established that the exit status is the only machine-readable thing a program returns. This chapter is about what happens to that number on its way back up — and the honest answer is that the most common Bash wrappers lose it.

There is a reason this matters more in a flow than at a terminal. At a terminal you read the error message. A CI job does not read anything. If the wrapper exits 0, the job is green, the sign-off dashboard shows the corner as clean, and the violation printed three lines above the end of the job log is seen by nobody.

2. The Perl End: An Exit Contract

A wrapper can only propagate a number that means something. So the Perl side comes first, and its job is to make the number documented and deliberate.

The parser below reads one timing report and counts violations. Its header comment states what each exit status means, and every exit is explicit — because, as Chapter 15.4 measured, die exits with whatever $! happens to hold, which makes its number unusable as a contract.

#!/usr/bin/perl
# parse_sta.pl -- count setup violations in one timing report.
#
# Exit contract. This is the ONLY thing a caller can act on:
#   0  report complete, no violations
#   1  report complete, at least one violation
#   2  the report could not be used (missing, unreadable, or incomplete)
use strict;
use warnings;
use Getopt::Long;

my $report;
GetOptions("report=s" => \$report) or exit 2;
unless (defined $report) {
    print STDERR "usage: parse_sta.pl --report FILE\n";
    exit 2;
}

open my $fh, "<", $report or do {
    print STDERR "parse_sta: cannot open $report: $!\n";
    exit 2;
};

my ($paths, $viol, $worst, $complete) = (0, 0, undef, 0);
while (my $line = <$fh>) {
    $complete = 1 if $line =~ /^End of report$/;
    next unless $line =~ /^\s+slack \((MET|VIOLATED)\)\s+(-?\d+\.\d+)\s*$/;
    $paths++;
    $viol++ if $1 eq "VIOLATED";
    $worst = $2 if !defined $worst || $2 < $worst;
}
close $fh;

unless ($complete) {
    print STDERR "parse_sta: $report has no end marker -- incomplete\n";
    exit 2;
}
printf "paths=%d violations=%d worst=%s\n", $paths, $viol, $worst // "n/a";
exit($viol ? 1 : 0);

Two reports to run it against. These are synthetic teaching reports in a small format defined for this module — not the output of any real STA tool, whose formats differ from vendor to vendor and from version to version. The format is deliberately simple so that the chapter is about the hand-off, not about parsing.

Report : timing (synthetic teaching format)
Design : chip_top
Corner : ss_0p72v_125c

Startpoint: u_core/r_reg[3]
Endpoint  : u_core/acc_reg[7]
  slack (VIOLATED)   -0.043

Startpoint: u_core/cnt_reg[0]
Endpoint  : u_core/cnt_reg[5]
  slack (MET)         0.120

End of report

Report : timing (synthetic teaching format)
Design : chip_top
Corner : tt_0p80v_25c

Startpoint: u_core/r_reg[3]
Endpoint  : u_core/acc_reg[7]
  slack (MET)         0.212

Startpoint: u_core/cnt_reg[0]
Endpoint  : u_core/cnt_reg[5]
  slack (MET)         0.388

End of report

Run directly, with no wrapper in between, the parser keeps its contract on all three cases — a clean corner, a violating corner, and a report that does not exist:

use strict;
use warnings;

my @CASES = (
    [ "clean corner",     "reports/tt_0p80v_25c.rpt",  0 ],
    [ "violating corner", "reports/ss_0p72v_125c.rpt", 1 ],
    [ "missing report",   "reports/ff_0p88v_m40c.rpt", 2 ],
);

for my $c (@CASES) {
    my ($label, $rpt, $want) = @$c;
    my $out = qx{perl parse_sta.pl --report $rpt 2>&1};
    my $got = $? >> 8;
    chomp $out;
    printf "%-17s exit %d (contract says %d)  %s\n", $label, $got, $want, $out;
}

clean corner      exit 0 (contract says 0)  paths=2 violations=0 worst=0.212
violating corner  exit 1 (contract says 1)  paths=2 violations=1 worst=-0.043
missing report    exit 2 (contract says 2)  parse_sta: cannot open reports/ff_0p88v_m40c.rpt: No such file or directory

The report paths here are fixed, safe strings, so the backtick form is acceptable for this one demonstration; section 6 shows why it is not acceptable for anything a user supplies.

3. Eight Wrappers, Measured

Now put a Bash script between the caller and the parser. Each of the eight below is a pattern that appears in real flow repositories. Read them before reading the measurement, and predict which ones keep the contract.

The first two do some work after the parser finishes — a log line, a tidy-up:

#!/bin/bash
perl parse_sta.pl "$@"
echo "parse step finished"

#!/bin/bash
perl parse_sta.pl "$@"
echo "parse step finished"
exit $?

The next two keep a copy of the parser's output in a log file:

#!/bin/bash
perl parse_sta.pl "$@" | tee parse.log

#!/bin/bash
set -o pipefail
perl parse_sta.pl "$@" | tee parse.log

The next two use Bash's strict mode, one of them through a function with its own failure branch:

#!/bin/bash
set -euo pipefail
perl parse_sta.pl "$@"
echo "parse step finished"

#!/bin/bash
set -euo pipefail
parse() {
    perl parse_sta.pl "$@"
    echo "parse step finished"
}
if ! parse "$@"; then
    echo "parse step FAILED" >&2
    exit 1
fi

And the last two are the two correct shapes — one that does work afterwards and still keeps the number, and one that hands the whole process over to Perl:

#!/bin/bash
set -uo pipefail
rc=0
perl parse_sta.pl "$@" || rc=$?
echo "parse step finished with status $rc" >&2
exit "$rc"

#!/bin/bash
set -euo pipefail
exec perl parse_sta.pl "$@"

The measurement runs each wrapper on the same three inputs and records the exit status the caller receives. The helper silences the wrappers' own output by forking and redirecting in the child, so the table shows only the numbers.

use strict;
use warnings;

my @WRAPPERS = qw(
    wrap_echo_after  wrap_exit_status  wrap_tee        wrap_tee_pipefail
    wrap_set_e       wrap_function_if  wrap_capture    wrap_exec
);
my @CASES = (
    [ clean   => "reports/tt_0p80v_25c.rpt",  0 ],
    [ viol    => "reports/ss_0p72v_125c.rpt", 1 ],
    [ missing => "reports/ff_0p88v_m40c.rpt", 2 ],
);

# Run a command with its stdout and stderr discarded; return what the
# CALLER sees: an exit code, or "sigN" if it died by a signal (14.3).
sub status_of {
    my @cmd = @_;
    my $pid = fork // die "fork: $!\n";
    if ($pid == 0) {
        open STDOUT, ">", "/dev/null" or die;
        open STDERR, ">", "/dev/null" or die;
        exec @cmd or exit 127;
    }
    waitpid $pid, 0;
    return $? & 127 ? "sig" . ($? & 127) : $? >> 8;
}

printf "%-19s %-6s %-6s %-8s %s\n", "wrapper", "clean", "viol", "missing", "verdict";
printf "%-19s %-6s %-6s %-8s %s\n", "(contract)", map({ $_->[2] } @CASES), "";
my $swallowed = 0;
for my $w (@WRAPPERS) {
    my @got = map { status_of("bash", "$w.sh", "--report", $_->[1]) } @CASES;
    my $bad = grep { $got[$_] ne $CASES[$_][2] } 0 .. $#CASES;
    $swallowed++ if $bad;
    printf "%-19s %-6s %-6s %-8s %s\n", $w, @got,
        $bad ? "SWALLOWS ($bad of 3 wrong)" : "keeps the contract";
}
printf "\n%d of %d wrappers report a different status from the parser\n",
    $swallowed, scalar @WRAPPERS;

wrapper             clean  viol   missing  verdict
(contract)          0      1      2        
wrap_echo_after     0      0      0        SWALLOWS (2 of 3 wrong)
wrap_exit_status    0      0      0        SWALLOWS (2 of 3 wrong)
wrap_tee            0      0      0        SWALLOWS (2 of 3 wrong)
wrap_tee_pipefail   0      1      2        keeps the contract
wrap_set_e          0      1      2        keeps the contract
wrap_function_if    0      0      0        SWALLOWS (2 of 3 wrong)
wrap_capture        0      1      2        keeps the contract
wrap_exec           0      1      2        keeps the contract

4 of 8 wrappers report a different status from the parser

Four of eight. And look at which number goes wrong: in every swallowing row, the failures become 0. Not a different failure code — success. A wrapper that loses the status does not make a failed run look different; it makes it look fine.

4. Why Each One Swallows

Each failure has a precise cause, and each cause is a rule of the shell rather than a quirk.

wrap_echo_after — a script exits with the status of its last command. The last command is echo, and echo succeeds. The parser's 1 or 2 was in $? for exactly one line, and nothing read it.

wrap_exit_status — $? is the status of the most recent command, which is the echo. This one looks like a fix and is the most deceptive of the eight, because exit $? reads as "exit with the parser's status". It is not. By the time exit runs, $? has been overwritten by the line in between. Any statement between the command and the place you read $? destroys it — which is why the correct shape (wrap_capture) copies it into a variable on the same line as the command.

wrap_tee — a pipeline's status is the status of its last stage. tee copied the output to parse.log perfectly and exited 0, so the pipeline exited 0. The parser's status was discarded by the shell before the script even got to look at it.

wrap_tee_pipefail keeps the contract, because set -o pipefail changes the rule: the pipeline's status becomes the last non-zero status of any stage. That one line is the difference between rows three and four.

wrap_set_e keeps the contract, because set -e makes the shell exit as soon as a command fails — with that command's status. The echo after the parser simply never runs on a failure.

wrap_function_if is the trap in strict mode. It uses set -e, it has an explicit failure branch, and the failure branch never runs. Bash suspends set -e for any command that is being tested as a condition — after if, after !, on the left of && or || — and for everything inside a function called in that position. So inside parse, the failed perl line no longer stops anything, the echo runs, the function returns the echo's 0, and if ! parse sees success.

Here is that happening, with the output visible this time:

#!/bin/bash
echo "--- violating corner, through wrap_function_if.sh"
bash wrap_function_if.sh --report reports/ss_0p72v_125c.rpt 2>&1
echo "caller sees: $?"

--- violating corner, through wrap_function_if.sh
paths=2 violations=1 worst=-0.043
parse step finished
caller sees: 0

The parser reported its violation, then parse step finished printed after a failure that strict mode was supposed to stop at, and the script's own parse step FAILED branch never appeared. The author wrote a failure handler and the shell made it unreachable.

Two pipefail consequences worth knowing

pipefail is the right default, and it has a cost that surprises people the first time. A stage that legitimately exits non-zero now fails the whole pipeline. The common case is grep, which exits 1 when it selects no lines:

#!/bin/bash
set -o pipefail
# a parser run that succeeded and printed only a comment line
perl -e 'print "# 0 paths checked\n"; exit 0' | grep -v '^#' | cat
echo "with pipefail:    $?"
set +o pipefail
perl -e 'print "# 0 paths checked\n"; exit 0' | grep -v '^#' | cat
echo "without pipefail: $?"

with pipefail:    1
without pipefail: 0

The Perl step succeeded; the pipeline reports failure because a filter found nothing to keep. That is not a reason to turn pipefail off. It is a reason not to put a filter whose "nothing matched" status means success inside a pipeline whose status you act on — filter in Perl instead, where "no lines" is a value rather than an exit code.

5. A Killed Parser: exec Keeps the Signal, a Plain Child Converts It

There is a fourth way to lose information, and it is subtler than swallowing: the wrapper reports a failure, but the wrong kind.

When a job queue kills a job for exceeding its memory or wall-clock limit, it sends a signal. Chapter 14.3 showed how a Perl parent tells a signal death from an exit code: $? & 127 holds the signal number. That only works if the parent is looking at the process that was killed. With a wrapper in between, it usually is not.

The stand-in for a killed parser is a script that sends itself SIGTERM, which is what a queue's kill would do:

# stands in for a parser killed by the job queue mid-run
kill "TERM", $$;
sleep 5;

#!/bin/bash
perl killed_parse.pl

#!/bin/bash
exec perl killed_parse.pl

use strict;
use warnings;

for my $w (qw(call_child call_exec)) {
    my $pid = fork // die "fork: $!\n";
    if ($pid == 0) {
        open STDERR, ">", "/dev/null" or die;   # bash's job message
        exec "bash", "$w.sh" or exit 127;
    }
    waitpid $pid, 0;
    printf "%-11s raw %-6d  exit code %-4d  signal %d\n",
        $w, $?, $? >> 8, $? & 127;
}

call_child  raw 36608   exit code 143   signal 0
call_exec   raw 15      exit code 0     signal 15

Same death, two different reports.

Through call_child, the signal has become an ordinary exit code, 143. Bash ran Perl as a child, saw it die from signal 15, and recorded that in its own $? using the shell's convention: 128 plus the signal number. Then the script ended and Bash exited with that value. The Perl parent at the top receives a process that exited normally with status 143 — signal 0. Chapter 14.3's decoder would print "exited with status 143", not "killed by signal 15".

Through call_exec, the signal survives, because exec does not start a child at all. It replaces the Bash process with Perl, keeping the same process ID, so the process the parent is waiting for is the one that was killed.

This is worth recognising in CI logs, because the numbers recur:

A status above 128 coming out of a wrapper almost always means "a signal, converted by a shell". It does not prove it — a program may legitimately exit 143 — which is exactly why a wrapper that execs is better: it does not create the ambiguity.

6. Arguments: Only One Form Is Right

The wrapper's other job is passing arguments through. A corner name, a report path, an option — whatever the Makefile gave the wrapper must reach Perl unchanged. A small script makes the difference visible by printing exactly what it received, one bracketed argument at a time:

printf "%d args:", scalar @ARGV;
print " [$_]" for @ARGV;
print "\n";

#!/bin/bash
printf '%-14s' '"$@"';  perl show_args.pl "$@"
printf '%-14s' '$@';    perl show_args.pl $@
printf '%-14s' '"$*"';  perl show_args.pl "$*"
printf '%-14s' '$*';    perl show_args.pl $*

#!/bin/bash
bash forward_forms.sh --corner "ss 0p72v" "Rev A.rpt" 'reports/*.rpt'

"$@"          4 args: [--corner] [ss 0p72v] [Rev A.rpt] [reports/*.rpt]
$@            7 args: [--corner] [ss] [0p72v] [Rev] [A.rpt] [reports/ss_0p72v_125c.rpt] [reports/tt_0p80v_25c.rpt]
"$*"          1 args: [--corner ss 0p72v Rev A.rpt reports/*.rpt]
$*            7 args: [--corner] [ss] [0p72v] [Rev] [A.rpt] [reports/ss_0p72v_125c.rpt] [reports/tt_0p80v_25c.rpt]

The caller passed four arguments. Only "$@" delivered four.

• Unquoted $@ and $* are word-split and glob-expanded. ss 0p72v became two arguments, and the literal pattern reports/*.rpt — which the caller deliberately quoted — was expanded by the wrapper into the two report files that happen to exist. Seven arguments where four were sent, and only one of the seven is unchanged.
• "$*" joins everything into one string. One argument, with all the boundaries gone.
• "$@" is special-cased by the shell to expand to each original argument as a separate, untouched word. It is the only form that forwards arguments.

Corner names rarely contain spaces. Paths do — a home directory, a mounted share, a release called Rev A. The unquoted form works in every test the author runs and fails the first time someone else's path is involved, which is why it survives code review.

7. The Entry Point

Putting the measured rules together gives the shape below. It is short because every line has a job.

#!/bin/bash
# run_sta.sh -- the entry point a Makefile or CI job calls.
# Exit status is parse_sta.pl's, unchanged: 0 clean, 1 violations, 2 unusable.
set -euo pipefail

here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)

if ! command -v perl >/dev/null 2>&1; then
    echo "run_sta.sh: perl not found on PATH" >&2
    exit 2
fi

exec perl "$here/parse_sta.pl" "$@"

Line by line:

set -euo pipefail — stop on the first failing command, treat an unset variable as an error, and make a pipeline report its failing stage. Section 4 showed where -e stops protecting you; it is still the right default for everything above the last line.

here=$(...) finds the directory the script lives in, so the wrapper works when the Makefile calls it from a different directory. Chapter 17.1 makes the same argument for a Perl script anchoring itself to the project root. BASH_SOURCE[0] is the script's own path even when it is sourced; $0 is not.

command -v perl checks the interpreter exists before handing over, and fails with the contract's "unusable" status and a message that names the problem. Without it, a missing perl gives Bash's command not found and status 127 — a number outside the contract.

exec perl ... "$@" forwards every argument untouched and replaces the shell with Perl, so the exit status and any signal reach the caller exactly as Perl produced them. Nothing runs after an exec, so there is nowhere for the status to be lost.

The canonical example for this chapter writes exit $? after the Perl line instead of exec. Under set -e that works — but the exit $? is only reached when Perl succeeds, because a failure has already ended the script. It is harmless, and it is also the line that turns into wrap_exit_status the moment somebody adds an echo above it. exec cannot be broken that way.

When the wrapper genuinely has work to do afterwards — copying the report to an archive, writing a status file — use wrap_capture's shape instead: capture the status on the same line as the command, do the work, exit "$rc".

#!/bin/bash
for rpt in reports/tt_0p80v_25c.rpt reports/ss_0p72v_125c.rpt reports/ff_0p88v_m40c.rpt; do
    bash run_sta.sh --report "$rpt" 2>&1
    echo "  -> run_sta.sh exited $?"
done

paths=2 violations=0 worst=0.212
  -> run_sta.sh exited 0
paths=2 violations=1 worst=-0.043
  -> run_sta.sh exited 1
parse_sta: cannot open reports/ff_0p88v_m40c.rpt: No such file or directory
  -> run_sta.sh exited 2

All three statuses arrive intact, and the messages for the two failures are the parser's own.

8. What Make Does to the Number

The next link up is Make. A recipe line that exits non-zero stops the target — that much everybody expects. What Make reports upwards is less well known.

REPORT ?= reports/tt_0p80v_25c.rpt

sta_check:
	bash run_sta.sh --report $(REPORT)
	@echo "sta_check passed"

#!/bin/bash
for rpt in reports/tt_0p80v_25c.rpt reports/ss_0p72v_125c.rpt reports/ff_0p88v_m40c.rpt; do
    echo "--- REPORT=$rpt"
    make sta_check REPORT="$rpt" 2>&1
    echo "make exited $?"
done

--- REPORT=reports/tt_0p80v_25c.rpt
bash run_sta.sh --report reports/tt_0p80v_25c.rpt
paths=2 violations=0 worst=0.212
sta_check passed
make exited 0
--- REPORT=reports/ss_0p72v_125c.rpt
bash run_sta.sh --report reports/ss_0p72v_125c.rpt
paths=2 violations=1 worst=-0.043
make: *** [sta_check] Error 1
make exited 2
--- REPORT=reports/ff_0p88v_m40c.rpt
bash run_sta.sh --report reports/ff_0p88v_m40c.rpt
parse_sta: cannot open reports/ff_0p88v_m40c.rpt: No such file or directory
make: *** [sta_check] Error 2
make exited 2

Make stopped correctly on both failures and printed the recipe's real status — Error 1 for violations, Error 2 for an unusable report. But Make's own exit status was 2 both times. GNU Make exits 2 whenever a target fails, whatever the recipe's number was.

So the contract survives to Make's log and no further. Above Make, the CI job can tell "it failed" from "it passed" and nothing more. If the CI system needs to tell violations from a broken run — and it should, because the first is a design problem and the second is an infrastructure problem with a different owner — that information has to travel in a file the parser writes, not in the exit status. Chapter 26.3 is about exactly that hand-off.

This output was captured with GNU Make 3.81. Newer versions print the failing recipe's file and line in the *** message and quote target names differently, but they exit 2 in the same situation.

9. Exercises

1. Predict, then check. A colleague's wrapper ends with perl parse_sta.pl "$@" | grep -v '^DEBUG' | tee parse.log and has set -euo pipefail at the top. For each of the three reports in this chapter, predict the wrapper's exit status, then work out the input on which the wrapper fails even though the parser exited 0. What is the smallest change that removes that false failure without turning pipefail off?

2. A wrapper with a post-step. Extend wrap_capture.sh so that it also writes the parser's status to parse.status in the run directory. The file must never be seen half-written by a dashboard that polls the directory. Which Perl chapter's technique applies to a shell script here, and what is the shell equivalent?

3. Make swallows the distinction. Design, without writing the CI side, the smallest change to parse_sta.pl that lets a CI job distinguish "timing violations" from "unusable report" even though Make collapses both to status 2. What must be true of that change so that a missing or stale file is never read as a clean result?

4. A 137 in the log. A nightly job's log ends with run_sta.sh exited 137. List what you can conclude, what you cannot, and the one change to run_sta.sh that would have let a Perl parent see the original signal directly. Then explain why that change would not help if the queue killed the wrapper's parent instead.

5. Find the swallow. In any flow repository you have access to, search the Bash scripts for a Perl or Python invocation followed by another command on the next line with no set -e, no exec, and no captured status. For each one, decide whether the status matters to anything above it. Most of them will be harmless; the exercise is in being able to say which ones are not.

10. What To Remember

• Every layer above the parser acts on one number, and a CI job reads nothing else.
• Give the Perl tool a documented exit contract and exit explicitly; die's status is $! and is not a contract (15.4).
• Measured: 4 of 8 common wrappers swallow the parser's status, and every one turns failure into 0, not into a different failure.
• A script exits with its last command's status — an echo after the parser erases the failure.
• exit $? after any other command exits with that command's status. Capture rc on the same line as the command.
• A pipeline's status is its last stage's unless set -o pipefail is on.
• set -e is suspended inside if, !, &&, || — and inside any function called there. Measured: the wrapper's own failure branch never ran.
• pipefail makes a filter's "nothing matched" (grep's 1) fail a good run. Filter in Perl, not in the status-carrying pipeline.
• A plain child killed by SIGTERM reaches the caller as exit 143 (128 + 15), signal 0. With exec, the signal survives.
• 143, 137 and 130 in a CI log usually mean TERM, KILL and INT, converted by a shell.
• Forward arguments with "$@" and nothing else. Unquoted $@ split ss 0p72v and expanded a quoted glob into two files.
• The entry point is set -euo pipefail, find the script's directory, check for perl, then exec perl ... "$@".
• Make prints the recipe's real status (Error 1, Error 2) but exits 2 for any failure. Detail the CI job needs must travel in a file.

Chapter 26.2 turns to the other half of the boundary: getting values into the Perl program safely — through arguments, the environment, a file, or standard input — and the quoting failures that appear only when a path contains a space or a name contains a quote.