Why Scripts Fail Silently

Part I · Module 20 · Chapter 20.1 · Why Scripts Fail Silently

Here is a three-line script. It is wrong in a way that no reviewer would catch and that no test would fail on.

#!/bin/bash
cd /nonexistent-dir
rm -f junk.txt
echo "work done"

$ ./clean.sh
./clean.sh: line 2: cd: /nonexistent-dir: No such file or directory
work done
$ echo $?
0

Read that carefully. The cd failed and said so. Then rm -f junk.txt ran — in whatever directory the caller happened to be in, deleting a file that was never meant to be touched. Then echo printed a confident message. And the script exited 0.

Everything after line 2 operated on a premise that had already been proved false, and the caller was told the run succeeded.

1. The Default Is a Decision, Not an Accident

It is worth understanding why this is the default, because "Bash is badly designed" is both unfair and useless.

Bash is a command language, and its original job was an interactive prompt. At a prompt, continuing after a failure is obviously right: you mistype cd /usr/loca, it complains, you retype it. A shell that exited on your first typo would be unusable.

A script is the same language with the same rules, so it inherits that behaviour. Nothing in the language distinguishes "a human is typing and will react to the error" from "a script is running unattended and nobody will". Chapter 15.1 made that continuity a feature — a script is just the commands you would have typed — and this is the price.

So the defaults are correct for the interactive case and wrong for the automated one, and every chapter in this module is about replacing them deliberately.

2. Seven Shapes, All Measured

The pattern is always the same: the shell reports the problem, then carries on, and the status is set by something else. Here are the forms you will actually meet.

One — a failed cd, and work in the wrong place. The opening example. The scariest of the seven, because the subsequent commands are not merely useless; they are destructive somewhere unintended.

Two — the trailing message that masks the failure.

$ cat c1.sh
#!/bin/bash
grep -q NEVERMATCH /etc/hosts
echo "done"

$ ./c1.sh
done
$ echo $?
0

grep -q returned 1. echo returned 0. The script's status is its last command's — Chapter 15.6 — so the search result was discarded by the line that announced it. This is Chapter 18.3 §3's function-level trap at script scale.

Three — a pipeline hiding a failed producer.

$ bash -c 'exit 3' | wc -l > /dev/null
$ echo $?
0

Chapter 12.8: a pipeline's status is its last stage's. wc succeeded. The thing that produced its input died with status 3, and nothing in the exit status says so.

Four — an unset variable in a path.

$ out=
$ printf 'writing to [%s]\n' "$out/result.txt"
writing to [/result.txt]

Not an error. An unset or empty variable expands to nothing, so "$out/result.txt" became /result.txt — a path at the root of the filesystem. Chapter 20.4 is entirely about how bad this gets when the command is rm.

Five — an unmatched glob in a loop.

$ for f in ./*.nope; do printf 'processing [%s]\n' "$f"; done
processing [./*.nope]

Chapter 9.6's rule and Chapter 17.1 §6's consequence: the body runs once on a file that does not exist. Whatever it does then fails, and if that failure is also swallowed you have two silent failures stacked.

Six — a die swallowed by command substitution.

$ die() { echo "FATAL: $*" >&2; exit 1; }
$ f() { die "bad"; }
$ v=$(f); echo "continued, v=[$v]"
FATAL: bad
continued, v=[]

Chapter 18.3 §7: exit ends the process it runs in, and $( ) is a subshell. The fatal message printed and the script carried on with an empty value.

Seven — a command that does not exist.

$ cat c2.sh
#!/bin/bash
nosuchtool --flag
echo "carried on anyway"

$ ./c2.sh
c2.sh: line 2: nosuchtool: command not found
carried on anyway
$ echo $?
0

A typo in a tool name, or a tool missing from PATH on one machine — and the script proceeds as though the step had been done. Chapter 11.1 measured that command not found is status 127; nothing read it.

3. The Common Thread

Look back at all seven. In every single one, the shell knew:

• cd printed an error and returned non-zero.
• grep -q returned 1.
• The pipeline's first stage returned 3, and PIPESTATUS recorded it.
• The command-not-found returned 127.
• The die returned 1 from its subshell, and $? held it.

The information was there. Nothing acted on it. That is the whole problem, and it is why the fix is mostly about reading statuses you are already being given rather than about detecting anything new.

Two of the seven are different in kind and worth separating out, because no amount of status-checking helps:

• Four (the empty variable) produced no failure at all. /result.txt is a perfectly valid path. There is no status to check, because nothing went wrong as far as the shell is concerned — the script's intent was violated, not any rule. That needs a declared expectation, which is Chapters 20.3 and 20.4.
• Five (the unmatched glob) is the same: ./*.nope is a legal word and the loop ran correctly over a one-item list.

So the module has two jobs, not one: read the statuses you are given, and state the expectations the shell cannot infer.

4. The Nightly Regression That Ran Nothing

A composite, because the shapes rarely arrive alone. This is what they look like together:

#!/bin/bash
# nightly.sh — the version that lies

cd "$RESULTS_ROOT/nightly"              # 1: RESULTS_ROOT unset
rm -rf old/*                            # 4: expands to rm -rf old/* in $HOME
testlist=$(cat tests.txt)               # 7: tests.txt not here; testlist empty
for t in $testlist; do                  # 5: empty list, zero iterations
  run_sim "$t" > "$t.log"
done
failed=$(grep -lc UVM_ERROR ./*.log | wc -l)
echo "nightly complete: $failed failures"

$ ./nightly.sh
./nightly.sh: line 4: cd: /nightly: No such file or directory
cat: tests.txt: No such file or directory
grep: ./*.log: No such file or directory
nightly complete:        0 failures
$ echo $?
0

"0 failures", exit 0, and not one test was run. Three error lines went to stderr, which in a cron job goes to a mail nobody reads or a log nobody greps. The one line that reaches the dashboard is the last one, and it is a lie.

Chapter 20.8's tool answers "what did it actually do" in six lines:

$ env -u RESULTS_ROOT bash -x ./nightly.sh 2>&1 | grep '^+'
+ cd /nightly
+ rm -rf 'old/*'
++ cat tests.txt
+ testlist=
++ grep -lc UVM_ERROR './*.log'
++ wc -l
+ failed='       0'
+ echo 'nightly complete:        0 failures'

Every lie is visible there. cd /nightly — the variable vanished. rm -rf 'old/*' — quoted by the trace because the glob matched nothing and stayed literal, so this run deleted nothing, though the same line in a directory that does have an old/ would. testlist= — empty. And failed=' 0' carrying BSD wc's padding, which is where the gap in the output line comes from.

Every chapter in this module removes one of those lines' ability to lie:

And one thing that is not on that list: none of those fixes would have told you what the script actually did. That is Chapter 20.8, and it is why tracing is in this module rather than being an optional extra.

5. What "Defensive" Does Not Mean

Two honest limits, so the module is not oversold.

It does not mean set -euo pipefail and stop thinking. That line is Chapter 20.2's subject and it is genuinely valuable; it is also surrounded by exemptions that make it do nothing in several of the cases above. Chapter 20.2 measures exactly where, and the module's own description promises to be honest about it. A script with that line at the top and no other care is not a defended script.

It does not mean checking everything. A guard that cannot fail is noise, and noise makes the real guards harder to see. Chapter 16.4 §9's note applies to the whole module: a check is a statement at a moment in time, and for many operations the better pattern is to attempt the thing and handle the failure. The question for each guard is what would go wrong without it, and would I rather find out here or three steps later.

The goal is a script that, when something is wrong, stops at the point of the problem and says what it was — and that tells the truth in its exit status either way.

Practice

1. Write the three-line cd script and run it from a directory containing a file called junk.txt. Confirm the file is gone and the exit status is 0. Then say which of the three lines was actually at fault.
2. Write grep -q NEVERMATCH /etc/hosts followed by echo done in a script. Print $?. Then remove the echo and print $? again.
3. Run bash -c 'exit 3' | wc -l > /dev/null; echo $?. Then print "${PIPESTATUS[*]}" instead and explain where the 3 went.
4. Set out= and printf '%s\n' "$out/result.txt". Explain why no error is reported, and say what would have to be true for the shell to catch it.
5. Reproduce the die inside $( ) case. Confirm the message prints and the script continues. Which chapter explains it?
6. Type out nightly.sh and run it in an empty directory with RESULTS_ROOT unset. Count the stderr lines, the stdout lines, and the exit status.
7. For each of the seven shapes, write down whether the shell reported a failure and whether a status was available. Two of the seven differ — which, and why does that change the fix?
8. Find a script in your own work whose last line is an echo or a printf. Determine its exit status when the step before the echo fails.

Check Your Understanding

1. A script's cd fails. What happens to the rest of the script, and what is its exit status?
2. Why is continuing after a failure the right default for an interactive shell?
3. A script ends with echo "done". What does that do to its exit status?
4. bash -c 'exit 3' | wc -l exits 0. Why, and where is the 3 recorded?
5. Two of the seven shapes cannot be fixed by checking a status. Which, and what do they need instead?
6. Why is exiting 0 wrongly worse than printing a wrong number?
7. Does set -euo pipefail make a script defended?

6. Remember

• By default a failed step does not stop the script, and the script's status is its last command's.
• That default is right for a prompt and wrong for automation, and nothing in the language distinguishes the two.
• Seven shapes to recognise: a failed cd; a trailing message masking a failure; a pipeline hiding its producer; an empty variable in a path; an unmatched glob in a loop; exit swallowed by $( ); and a missing command.
• In five of the seven, a status was available and nothing read it. The fix is mostly to read what you are already given.
• Two of the seven are not failures at all — an empty expansion and an unmatched glob are legal. They need a declared expectation.
• Exit zero wrongly is the worst outcome, because a false success propagates into CI, make and && chains.
• Errors go to stderr, which in cron goes nowhere anyone reads. The one line on the dashboard is the last one.
• Proving a script fails when it should is a test, not a hope — Chapter 21.8 builds the suite and then breaks the script to prove the suite can notice.
• Defensive does not mean one magic line, and it does not mean checking everything. It means stopping at the problem, naming it, and telling the truth in the status.

Chapter 20.2 turns on the three options that fix most of this — and measures, case by case, the places where the most famous line in shell scripting does nothing at all.