Guarding Destructive Commands

Part I · Module 20 · Chapter 20.4 · Guarding Destructive Commands

Everything in this module so far has been about a script doing the wrong thing. This chapter is about a script doing the wrong thing irreversibly.

1. How This Chapter Was Measured

Every hazardous command below was run with echo in front of it, so what you see is the real command line the shell built and nothing was deleted. The only genuine rm in the chapter runs inside a mktemp sandbox on files created for the purpose.

Reproduce it the same way. If you want to see what an expansion does, put echo in front or turn on tracing — never "just try it and see". That habit is Chapter 14.7's "preview, then act" rule, and this is the chapter where it stops being advice.

2. What an Empty Variable Builds

$ bash -c 'set -x; dir=""; echo rm -rf "$dir/"' 2>&1 | grep '^+'
+ dir=
+ echo rm -rf /

rm -rf /.

Not an error. Not a warning. "$dir/" is the expansion of an empty variable followed by a literal slash, and that is the string /. The shell did exactly what it was told, and the command line it built names the root of the filesystem.

Without the trailing slash it is harmless:

$ bash -c 'set -x; dir=""; echo rm -rf "$dir"' 2>&1 | grep '^+'
+ dir=
+ echo rm -rf ''

rm -rf '' is an empty argument, which rm rejects. So the trailing slash — added by habit, or because the variable is documented as "a directory without a trailing slash" — is what turns a harmless mistake into a catastrophic one.

And the glob form is worse than either:

$ bash -c 'set -x; dir=""; echo rm -rf "$dir"/*' 2>&1 | grep '^+'
+ dir=
+ echo rm -rf /Applications /Library /System /Users /Volumes /bin /cores /dev /etc /home /opt /private /sbin /tmp /usr /var

"$dir"/* became /*, which globbed against the root directory. Sixteen arguments, every one of which exists. rm -rf / on a modern system is often refused by rm itself as a special case; rm -rf /Applications /Library /System … is not special to rm at all — it is just a list of directories.

3. set -u Does Not Catch This

It is worth being explicit, because this is the chapter where people assume Chapter 20.2's line has them covered:

$ bash -c 'set -u; dir=""; echo rm -rf "$dir/"' 2>&1 | tail -1
rm -rf /

set -u objects to a variable that was never set. dir="" is set — to nothing. There is no rule being broken, so there is nothing for set -u to report.

And an empty value is not an exotic case. It is what you get from:

• dir=$1 with no first argument;
• dir=$(some_command) where the command failed or printed nothing;
• dir=${CONFIG_DIR} where the config file had a blank line;
• dir=$(dirname "$x") where x had no slash in it;
• a wrapper script that passed an unset variable of its own through.

Every one of those is routine, and none of them is an error the shell will tell you about.

4. The Guard: Refuse to Expand

Chapter 20.3's ${VAR:?} is the answer, used at the point of danger rather than only at the top:

$ bash -c 'dir=""; echo rm -rf -- "${dir:?run_dir must be set}"/*'
bash: dir: run_dir must be set

$ bash -c 'unset dir; echo rm -rf -- "${dir:?run_dir must be set}"/*'
bash: dir: run_dir must be set

$ bash -c 'dir=/tmp/safe; echo rm -rf -- "${dir:?}"/*'
rm -rf -- /tmp/safe/*

The command was never built. The shell stops at the expansion, prints the message, and exits — so there is no window in which a wrong command line exists.

That is the key property, and it is why ${dir:?} beats a preceding if:

# an if can be skipped, moved, or separated from the rm by later edits
if [ -z "$dir" ]; then die "dir must be set"; fi
...
rm -rf -- "$dir"/*                  # is the check still above this? who knows

# the guard is attached to the dangerous command itself
rm -rf -- "${dir:?}"/*

The second cannot be separated from the thing it protects, because it is the thing it protects. Write both — the readable check at the top from Chapter 20.3, and the :? on the destructive line — because they fail at different times and the second is the one that survives a refactor.

And write the colon. Measured again, because it matters most here:

$ bash -c 'dir=""; echo "[${dir?unset only}]"'
[]
$ bash -c 'dir=""; echo "[${dir:?empty or unset}]"'
bash: dir: empty or unset

${dir?} lets the empty case through, which is the case you are actually defending against.

5. The Other Four Habits

${var:?} handles the empty value. Four more habits handle the rest of the ways a deletion goes wrong.

Use -- before the operands. Chapter 17.2 §2 measured a file named -rf; the same applies to a directory name that starts with a dash:

rm -rf -- "${dir:?}"/*

Without --, a dir of -rf or a glob result beginning with a dash is read as an option.

Delete contents, not the directory, when that is what you mean. rm -rf -- "$dir" and rm -rf -- "$dir"/* are different operations, and the second is usually the one intended for a working directory that must continue to exist. Note that "$dir"/* skips dotfiles (Chapter 3.5) — if you need those too, find "$dir" -mindepth 1 -delete is the clearer tool than reaching for dotglob.

Refuse targets that cannot be right. A positive list of things you will never delete costs four lines and has caught real accidents:

case $dir in
  /|/usr|/etc|/home|/var|/opt|"$HOME")
    die "refusing to clean $dir" ;;
esac

That is Chapter 16.6's pattern branch used as a safety rail. It is not complete — nothing is — but / and $HOME are the two targets an empty-or-wrong variable most often produces, and both are covered.

Require the target to look like yours. Stronger than a deny-list, and better when you can state the rule:

case $dir in
  "$RESULTS_ROOT"/*) ;;                      # must be under the results tree
  *) die "refusing to clean outside $RESULTS_ROOT: $dir" ;;
esac

Now no empty value, no .., and no absolute path from somewhere else can get through, because the requirement is positive rather than a list of known-bad cases.

6. A Complete Script

A real guarded cleanup, with every habit from §4 and §5, measured in a sandbox:

#!/usr/bin/env bash
# clean-run.sh — remove the contents of a run directory, safely
# usage: clean-run.sh [-n] <run_dir>
#
# RESULTS_ROOT must be set; run_dir must be inside it.
#
# exit: 0 cleaned (or would have) · 2 unusable invocation or target

set -o errexit
set -o nounset
set -o pipefail
shopt -s nullglob

note() { printf '%s: %s\n' "${0##*/}" "$*" >&2; }
die()  { note "$*"; exit 2; }

dry_run=0
case ${1:-} in
  -n) dry_run=1; shift ;;
esac

: "${RESULTS_ROOT:?must be set — export RESULTS_ROOT=/path/to/results}"

run_dir=${1:-}

# 1. the readable check, at the top
[ -n "$run_dir" ] || die "usage: ${0##*/} [-n] <run_dir>"
[ -d "$run_dir" ] || die "not a directory: $run_dir"

# normalise to an absolute path, so the positive check in 3 can match
run_dir=$(cd -- "$run_dir" 2>/dev/null && pwd) \
  || die "not a usable directory: ${1:-}"

# 2. refuse anything that cannot be right
case $run_dir in
  /|/usr|/etc|/home|/var|/opt|"$HOME") die "refusing to clean $run_dir" ;;
esac

# 3. require the target to be ours — positive, not a deny-list
case $run_dir in
  "$RESULTS_ROOT"/*) ;;
  *) die "refusing to clean outside $RESULTS_ROOT: $run_dir" ;;
esac

# 4. collect first, so the preview and the action use the same list
declare -a victims=("$run_dir"/* "$run_dir"/.[!.]*)

if [ "${#victims[@]}" -eq 0 ]; then
  note "$run_dir is already empty"
  exit 0
fi

if [ "$dry_run" -eq 1 ]; then
  printf 'WOULD REMOVE %d item(s) from %s:\n' "${#victims[@]}" "$run_dir"
  printf '  %s\n' "${victims[@]}"
  exit 0
fi

printf 'removing %d item(s) from %s\n' "${#victims[@]}" "$run_dir"

# 5. the guard on the dangerous line itself, plus --
rm -rf -- "${victims[@]:?nothing to remove}"

$ export RESULTS_ROOT=$PWD/results
$ mkdir -p results/run42
$ touch results/run42/a.log 'results/run42/my report.log' results/run42/.hidden

$ ./clean-run.sh -n results/run42
WOULD REMOVE 3 item(s) from /…/results/run42:
  /…/results/run42/a.log
  /…/results/run42/my report.log
  /…/results/run42/.hidden
$ echo $?
0

$ ./clean-run.sh results/run42
removing 3 item(s) from /…/results/run42
$ echo $?
0
$ ls -a results/run42
.  ..

$ ./clean-run.sh results/run42
clean-run.sh: /…/results/run42 is already empty
$ echo $?
0

$ ./clean-run.sh ""
clean-run.sh: usage: clean-run.sh [-n] <run_dir>
$ echo $?
2

$ ./clean-run.sh /
clean-run.sh: refusing to clean /
$ echo $?
2

$ ./clean-run.sh "$HOME"
clean-run.sh: refusing to clean /home/engineer
$ echo $?
2

$ env -u RESULTS_ROOT ./clean-run.sh results/run42
./clean-run.sh: line 23: RESULTS_ROOT: must be set — export RESULTS_ROOT=/path/to/results
$ echo $?
1

Six decisions:

victims is collected once and used for both the preview and the deletion. A dry run that builds the list differently from the real run is a dry run that can lie. This is Chapter 19.5's dry-run rule applied where it matters most.

"$run_dir"/* "$run_dir"/.[!.]* catches dotfiles without dotglob. The .[!.]* pattern means "a dot followed by something that is not a dot", which excludes . and .. — the reason rm -rf .* is one of the classic ways to destroy a parent directory.

shopt -s nullglob is essential here, not optional. Measured on an empty directory with the line removed:

$ grep -v nullglob clean-run.sh > no-nullglob.sh && chmod +x no-nullglob.sh
$ ./no-nullglob.sh -n results/run42
WOULD REMOVE 2 item(s) from /…/results/run42:
  /…/results/run42/*
  /…/results/run42/.[!.]*
$ echo $?
0

Two "items", both of them literal patterns. The count check passed, the early-exit message never printed, and the real run would have handed rm -rf two paths that do not exist. rm -rf on a non-existent path is harmless — but the script has just told you it removed two things, which is a lie it will repeat every night.

The normalising cd … && pwd converts a relative path to absolute so the positive check in step 3 can match. Remove it and a path that is genuinely inside the tree is rejected for being spelled differently:

$ grep -v 'cd -- "$run_dir"' clean-run.sh > no-normalise.sh && chmod +x no-normalise.sh
$ ./no-normalise.sh -n results/run42
no-normalise.sh: refusing to clean outside /…/results: results/run42
$ echo $?
2

That is the honest cost of a positive rule: it rejects by default, including things it should not, until you make the comparison canonical. A deny-list has the opposite failure — it accepts by default — which is why the positive rule plus normalisation is the better trade, not a free one.

${victims[@]:?...} on the rm line. The count was already checked, so this can never fire; it is there because the check and the rm are eleven lines apart and a future edit could separate them. A guard that cannot fire today is cheap insurance against tomorrow's refactor.

my report.log survived with its space through the array, the preview and the deletion — Chapter 19.3, and the reason victims is an array rather than a string.

Practice

1. Run bash -c 'set -x; dir=""; echo rm -rf "$dir/"' 2>&1 | grep '^+'. Write down the command line it built. Keep the echo.
2. Do the same for "$dir" with no slash and for "$dir"/*. Rank the three by how much damage they would do and explain the ranking.
3. Add set -u to the first one and confirm the command line is unchanged. State in one sentence what set -u is actually about.
4. List three routine ways a variable ends up holding an empty string. For each, say whether the shell reported anything.
5. Compare ${dir:?msg} and ${dir?msg} with dir="". Which one is the guard, and why is the other one almost useless here?
6. Explain why rm -rf -- "${dir:?}"/* is safer than an if [ -z "$dir" ] ten lines above it. Then argue for writing both.
7. Create a directory named -rf in a sandbox and write a loop that passes it to echo rm. Fix it with -- and with ./.
8. Write the positive check case $dir in "$RESULTS_ROOT"/*) ;; *) die ;; esac and feed it a relative path that is genuinely inside the tree. Then fix it with the normalising cd … && pwd.
9. Remove shopt -s nullglob from clean-run.sh and dry-run it on an empty directory. What does the preview claim it would remove?
10. Remove the .[!.]* pattern and dry-run on a directory containing only a dotfile. What does the script report, and what would it have left behind?
11. Add a --force flag to clean-run.sh that skips the deny-list. Then argue, in two sentences, for or against having written it.

Check Your Understanding

1. dir="". What command line does rm -rf "$dir/" build?
2. Why is the version without the trailing slash harmless?
3. Why is the glob form worse than either?
4. Does set -u prevent any of this?
5. What does ${dir:?} do that an earlier if does not?
6. What does -- protect against?
7. Why is a positive "must be under this tree" check better than a deny-list, and what does it cost?

Chapter 28.4 is where these guards meet the script that deletes by the thousand: it prints what an unset root expands to — sixteen top-level directories — without running it.

7. Remember

• Measure hazardous expansions with echo in front or with tracing. Never "try it and see".
• dir="" makes rm -rf "$dir/" into rm -rf /. Measured, and not an error.
• The glob form is worse: "$dir"/* becomes /* and expands to every top-level directory.
• set -u does not help — it is about unset, not empty, and an empty value is routine.
• ${dir:?message} stops the command being built, so no wrong command line exists. Write the colon.
• Put the guard on the dangerous line itself, as well as the readable check at the top. They fail at different times.
• Always -- before the operands. A name beginning with a dash is otherwise an option.
• "$dir"/* and "$dir" are different operations. Add .[!.]* for dotfiles — never .*, which includes ...
• shopt -s nullglob is load-bearing when you collect victims into an array: without it an empty directory yields literal patterns.
• Prefer a positive "must be inside this tree" rule to a deny-list — and normalise the path with cd … && pwd first, or a correct relative path will be rejected.
• Collect the list once and use it for both the preview and the deletion. A dry run built differently can lie — Chapter 21.5 makes the dry run a general property rather than an ad-hoc one.

Chapter 20.5 handles what happens when a script stops in the middle — whether by failure, by set -e, or by someone pressing Ctrl-C — and leaves a half-finished directory behind.