The Exit Status of a Pipeline

Part I · Module 12 · Chapter 12.8 · The Exit Status of a Pipeline

Chapter 11.3 noted it, Chapter 12.5 named it as a problem, and Chapter 12.6 showed it breaking a build report. Here is the answer.

1. Only the Last Stage Counts

$ false | true;  echo "false | true  -> $?"
false | true  -> 0

$ true  | false; echo "true  | false -> $?"
true  | false -> 1

A pipeline's exit status is the status of its last command. Everything earlier is discarded, exactly as a semicolon-separated list discards all but its final status (Chapter 11.2).

That is a defensible default — the shell has one $? and the last stage is the one that produced the output you are looking at. It is also the cause of a specific, expensive class of silence:

$ nosuchcmd | wc -l
bash: nosuchcmd: command not found
       0
$ echo "status: $?"
status: 0

The command did not exist, and the pipeline reports success. wc was handed an empty stream, counted zero lines quite happily, and exited zero. The error message went to your terminal (Chapter 12.5 — errors bypass the pipe), where in a script redirected to a log nobody will read it.

2. pipefail

$ ( set -o pipefail; false | true );  echo "pipefail: false | true -> $?"
pipefail: false | true -> 1

$ ( set -o pipefail; nosuchcmd 2>/dev/null | wc -l >/dev/null ); echo "pipefail: missing | wc -> $?"
pipefail: missing | wc -> 127

With pipefail on, a pipeline's status is the rightmost non-zero status, or zero if every stage succeeded.

set -o pipefail

This is the pipefail in set -euo pipefail (Chapter 10.6), and it is the single most valuable third of that line. It turns "did the last thing work?" into "did all of it work?", which is nearly always the question.

It is off by default and must be switched on per shell — Chapter 10.6 noted that options are not inherited, so a script needs its own.

What it costs

$ yes | head -3 >/dev/null;                       echo "  plain:    $?"
  plain:    0
$ ( set -o pipefail; yes | head -3 >/dev/null );  echo "  pipefail: $?"
  pipefail: 141

141 is 128 + 13, and signal 13 is SIGPIPE. Chapter 12.5 explained it: head took its three lines and exited, the pipe closed, and yes was killed on its next write. Nothing went wrong — that is the mechanism working — but pipefail sees a non-zero status and reports failure.

3. PIPESTATUS

pipefail gives one combined answer. PIPESTATUS gives every stage's status individually.

$ false | true | false
$ echo "statuses: ${PIPESTATUS[@]}"
statuses: 1 0 1

$ grep zzz names.txt | sort | wc -l > /dev/null
$ echo "statuses: ${PIPESTATUS[@]}"
statuses: 1 0 0

It is an array (Chapter 19.2), indexed from zero in pipeline order:

./regress.sh 2>&1 | tee regress.log
status=${PIPESTATUS[0]}

${PIPESTATUS[0]} is the first stage — the command you actually care about when the last stage is tee.

Checking every stage

./produce.sh | ./filter.sh | ./consume.sh
statuses=("${PIPESTATUS[@]}")

for i in "${!statuses[@]}"; do
  if [ "${statuses[$i]}" -ne 0 ]; then
    echo "stage $i failed with ${statuses[$i]}" >&2
  fi
done

That tells you which stage failed, which pipefail cannot. Worth it for a pipeline of tools where the distinction matters; overkill for most.

4. Choosing

5. Avoiding the Problem

Sometimes the better answer is not to need a pipeline status at all.

Split the pipeline:

./regress.sh > /tmp/raw.log 2>&1
status=$?
grep -v '^INFO' /tmp/raw.log > regress.log
[ "$status" -eq 0 ] || exit "$status"

Two commands, two ordinary statuses, nothing to remember. It costs a temporary file and removes a whole class of mistake — often the right trade in a script somebody else will maintain.

Use process substitution instead of a pipe:

./regress.sh > >(tee regress.log) 2>&1
status=$?

The tee is no longer a pipeline stage, so $? is the regression's. (Chapter 12.5 introduced <(…); >(…) is the same idea in the other direction. Note that the parent may not wait for the substituted process, so the log can be a moment behind.)

Let the tool avoid the early close:

grep -m1 ERROR huge.log          # instead of grep ERROR huge.log | head -1
sort -u file                     # instead of sort file | uniq
grep -c ERROR file               # instead of grep ERROR file | wc -l

Fewer stages means fewer statuses to reason about, and each of these is faster as well.

6. The Module, Assembled

#!/bin/bash
set -uo pipefail

run="${1:?usage: $0 <run-name>}"
logdir="$HOME/runs/$run"
mkdir -p "$logdir" || { echo "cannot create $logdir" >&2; exit 2; }

log="$logdir/regress.log"

command -v vcs > /dev/null 2>&1 || { echo "vcs not on PATH" >&2; exit 3; }

# Watch it live and keep the record; capture the REGRESSION's status, not tee's.
./regress.sh --name "$run" < /dev/null 2>&1 | tee "$log"
status=${PIPESTATUS[0]}

# grep returning 1 for "no matches" is a result, not an error — 11.1.
errors=$(grep -c 'ERROR' "$log" || true)
warns=$(grep -c 'WARNING' "$log" || true)

{
  echo "run      : $run"
  echo "status   : $status"
  echo "errors   : $errors"
  echo "warnings : $warns"
} | tee "$logdir/summary.txt"

exit "$status"

Every chapter of the module is in there:

• 12.1 — diagnostics on stream three, results on stream two.
• 12.2 — > for the summary, a per-run directory rather than an appended log.
• 12.3 — 2>&1 before the pipe, and > /dev/null 2>&1 on the check.
• 12.4 — < /dev/null so the regression cannot hang on a prompt.
• 12.5 — the pipe itself, with the redirect in the right place.
• 12.6 — tee for live-and-saved, twice.
• 12.7 — both streams discarded on the existence check, and nowhere else.
• 12.8 — ${PIPESTATUS[0]} on the next line, and || true on the grep counts.

Practice

mkdir -p /tmp/pls && cd /tmp/pls
printf 'alpha\nbeta\n' > names.txt

1 — Confirm the default.

false | true;  echo "false | true  -> $?"
true  | false; echo "true  | false -> $?"

2 — Hide a missing command.

nosuchcmd | wc -l
echo "  status: $?"

3 — Switch on pipefail.

( set -o pipefail; nosuchcmd 2>/dev/null | wc -l >/dev/null; echo "  pipefail: $?" )

4 — Meet the head false positive.

yes | head -3 >/dev/null;                      echo "  plain:    $?"
( set -o pipefail; yes | head -3 >/dev/null ); echo "  pipefail: $?"

5 — Read every stage.

false | true | false
echo "  statuses: ${PIPESTATUS[@]}"
grep zzz names.txt | sort | wc -l >/dev/null
echo "  statuses: ${PIPESTATUS[@]}"

6 — Destroy PIPESTATUS, then do it right.

false | true; echo "something"; echo "  too late: ${PIPESTATUS[0]}"
false | true; s=${PIPESTATUS[0]}; echo "  captured: $s"

7 — Rescue a tee pipeline.

false 2>&1 | tee t.log >/dev/null; echo "  plain:         $?"
false 2>&1 | tee t.log >/dev/null; echo "  PIPESTATUS[0]: ${PIPESTATUS[0]}"

8 — Remove a stage instead.

grep alpha names.txt | wc -l
grep -c alpha names.txt
grep alpha names.txt | head -1
grep -m1 alpha names.txt

9 — Clean up. cd /tmp && rm -rf /tmp/pls

Check Your Understanding

1. What is the exit status of false | false | true?

2. nosuchcmd | wc -l reports success. Explain both halves.

3. What does pipefail change, and is it on by default?

4. yes | head -3 reports 141 under pipefail. Did anything go wrong?

5. ./job.sh | tee log and you need the job's status. Give two ways.

6. Why must ${PIPESTATUS[0]} be captured on the very next line?

7. Remember

• A pipeline's status is its last command's by default. Every earlier stage is discarded.
• So a missing first command piped into wc reports success — and the error went to the terminal, not the log.
• This is why a CI step can pass on a failing build, most often through | tee.
• set -o pipefail makes the pipeline report the rightmost non-zero status. It is off by default and not inherited.
• It is the right default for every script, and the most valuable third of set -euo pipefail.
• | head under pipefail can report 141 — SIGPIPE, from a reader that closed early. Nothing went wrong.
• Handle that with || true, or a tool flag such as grep -m1 that removes the pipe entirely.
• PIPESTATUS is an array of every stage's status, in order, indexed from zero.
• ${PIPESTATUS[0]} is what you want after | tee.
• Capture it on the very next line. Any command in between replaces it.
• PIPESTATUS is Bash-only; pipefail is not in every older sh.
• Fewer stages means fewer statuses. grep -c, grep -m1 and sort -u each remove a pipe.

That completes Module 12, and with it the structural half of Part I. You can now say where every byte a command produces goes, send each stream somewhere different, connect programs into a pipeline that processes more than fits in memory, keep a record while you watch, discard what you genuinely do not need, and find out afterwards whether any of it worked.

Module 13 stops building plumbing and starts using it. It is a module of tools designed to be pipeline stages — selecting columns, sorting, counting duplicates, substituting text — and the reason it comes after this one is that every single one of them is useless without the pipe.