Expansion Forms: ${name}, Defaults and Length

Part I · Module 8 · Chapter 8.2 · Expansion Forms: Defaults and Length

Chapter 8.1 used one form of expansion — a dollar sign and a name — and noted in passing that an unset variable silently expands to nothing. This chapter is about the forms that go around the name, and about doing something deliberate with the missing case instead of hoping.

1. Why the Braces Exist

Start with a failure that has nothing to do with defaults.

$ file=run_42

$ echo "$file_v2"


$ echo "${file}_v2"
run_42_v2

The first echo printed an empty line. Nothing is broken — the shell read the name as file_v2, because _ is a legal character in a variable name (Chapter 8.1), and file_v2 has never been set.

Braces say where the name ends. ${file}_v2 is unambiguous: the name is file, and _v2 is ordinary text that follows it.

The rule generalises to every character that could be part of a name:

Letters, digits and underscores need braces. Everything else — dots, slashes, hyphens, spaces — ends the name on its own.

2. Four Ways to Handle a Missing Value

A variable that is unset expands to nothing, silently. Sometimes that is fine. Usually you want one of four things instead, and Bash has a form for each.

Use a fallback

$ unset dir

$ echo "${dir:-/tmp}"
/tmp

$ dir=/scratch
$ echo "${dir:-/tmp}"
/scratch

The variable is left alone. The fallback is substituted into this one expansion and nothing is stored:

$ unset dir
$ echo "${dir:-/tmp}"
/tmp

$ echo "dir is still [${dir}]"
dir is still []

This is the form you will use most. It is the whole of "sensible default" in six characters:

logdir="${LOGDIR:-./logs}"
jobs="${PARALLEL_JOBS:-4}"

Use it and keep it

$ unset u

$ echo "[${u:=ASSIGNED}]"
[ASSIGNED]

$ echo "now u=[$u]"
now u=[ASSIGNED]

Same substitution, but the variable is also assigned, so every later use sees the value too. Useful at the top of a script when a default should hold for the whole run rather than for one line.

Refuse to continue

$ unset CFG

$ echo "${CFG:?configuration directory not set}"
bash: CFG: configuration directory not set

There is no sensible default, so stop. The shell prints the variable's name, your message, and in a script, exits.

This is the right form whenever a wrong guess is more expensive than a clean failure:

project="${PROJECT_ROOT:?set PROJECT_ROOT before running this}"

The error names the variable, so somebody reading the failure three months later knows exactly what to set — which is a great deal better than a script that proceeds with an empty path and deletes the wrong directory.

Only if it is set

$ unset u
$ s=yes

$ echo "u -> [${u:+WAS_SET}]"
u -> []

$ echo "s -> [${s:+WAS_SET}]"
s -> [WAS_SET]

The inverse of the other three: the text is used when the variable has a value, and the whole expansion is empty otherwise. Note that the variable's own value is not used — only its presence.

The classic application is building an optional flag without an if:

# add --seed N only when SEED is set to something
./run_sim.sh ${SEED:+--seed "$SEED"}

3. The Colon Is Not Decoration

Every form above has a version without the colon, and the difference is precise and verified.

With a colon, "missing" means unset or empty. Without it, "missing" means unset only.

$ unset u
$ e=""
$ s="set"

$ echo "unset  :-  [${u:-DEF}]"
unset  :-  [DEF]
$ echo "unset   -  [${u-DEF}]"
unset   -  [DEF]
$ echo "empty  :-  [${e:-DEF}]"
empty  :-  [DEF]
$ echo "empty   -  [${e-DEF}]"
empty   -  []
$ echo "set    :-  [${s:-DEF}]"
set    :-  [set]

The whole distinction lives in one row:

They agree in two cases out of three. They differ exactly when the variable is set to the empty string — and that is a real situation, because an empty value frequently arrives from somewhere legitimate: a command that produced no output, a configuration line with nothing after the equals sign, a user who pressed Enter at a prompt.

4. Length

$ name=run_42
$ echo "${#name}"
6

$ empty=""
$ echo "${#empty}"
0

$ p="/home/engineer/project"
$ echo "${#p}"
22

${#name} is the number of characters in the value. It is the one form where the # is not a comment and not a trim operator — which is confusing, and worth noticing now so that Chapter 8.3 does not surprise you. The difference is position: ${#name} has it before the name, ${name#pattern} has it after.

The common use is the one place where unset and empty genuinely must be told apart, in combination with the colonless forms:

if [ "${#value}" -eq 0 ]; then
  echo "no value supplied"
fi

Testing with if properly is Module 16's subject; the expansion is the part that belongs here.

5. The Forms Together

Every one of them belongs inside double quotes in real use, for exactly the reason Chapter 8.1 gave:

cp "$src" "${dest:-/tmp}"

Without the quotes, a default containing a space would be split into two arguments — and the default is the case least likely to have been tested.

6. A Realistic Header

Put the chapter together as the opening of a script, which is where these forms actually live:

#!/bin/bash
set -u

# Required — there is no sensible guess, so fail loudly and name the variable.
project="${PROJECT_ROOT:?set PROJECT_ROOT to your project directory}"

# Optional, with defaults that hold for the whole run.
: "${RUN_NAME:=run_$(date +%H%M%S)}"
: "${PARALLEL_JOBS:=4}"

logdir="${LOG_DIR:-${project}/sim/logs}"

echo "project     : ${project}"
echo "run         : ${RUN_NAME}"
echo "parallel    : ${PARALLEL_JOBS}"
echo "logs        : ${logdir}"

Four things in there worth pointing at:

• :? for the one value that must come from outside. With set -u already on, an unset variable would fail anyway — but with a message about an "unbound variable" rather than one explaining what to do about it.
• : "${VAR:=default}" is the idiom for "apply this default and move on". The : is a built-in command that does nothing and ignores its arguments, so the line exists purely for the assignment side effect of :=. It reads oddly the first time and is completely standard.
• ${LOG_DIR:-${project}/sim/logs} — a default built from another variable, which works because the fallback is an ordinary expansion in its own right.
• Every expansion is braced, and every one that reaches a command is quoted.

That header is about eight lines, and it means the script either runs with values somebody can see, or stops with a sentence explaining what is missing. Neither outcome involves an empty path.

Practice

1 — Make the adjacency failure happen.

file=run_42
echo "$file_v2"
echo "${file}_v2"
echo "$file.log"

Explain why the third one works without braces.

2 — Walk the six-cell table yourself.

unset u; e=""; s=set
for v in u e s; do
  eval "echo \"$v  colon=[\${$v:-DEF}]  plain=[\${$v-DEF}]\""
done

(eval is doing something advanced here — run it for the output, and do not adopt it as a habit.) Or simply type the six echo lines out individually, which is the better way to see it.

3 — Watch := differ from :-.

unset a b
echo "[${a:-ONE}]";  echo "a is now [${a-still unset}]"
echo "[${b:=TWO}]";  echo "b is now [${b-still unset}]"

4 — Fail on purpose.

unset CFG
echo "${CFG:?you must set CFG}"
echo "this line does not run in a script"

Try it at a prompt, then put the same two lines in a script and run it. The script stops; your interactive shell does not close.

5 — Build an optional flag.

unset SEED
echo running ${SEED:+--seed "$SEED"}
SEED=7
echo running ${SEED:+--seed "$SEED"}

6 — Measure lengths.

for v in "" "a" "run_42" "/home/engineer/project"; do
  echo "${#v}  <- [$v]"
done

7 — Write the header. Type out the script from §6, run it with PROJECT_ROOT unset, read the error, then run it with PROJECT_ROOT=/tmp ./header.sh and read the output.

Check Your Understanding

1. version=3 and echo "$version_rc1" prints nothing. Why, and what are two ways to fix it?

2. What is the difference between ${d:-/tmp} and ${d=/tmp}, in two respects?

3. LOGDIR is set to the empty string. Does ${LOGDIR:-./logs} give you ./logs? Does ${LOGDIR-./logs}?

4. When would you choose :? over :-?

5. ${name:+--verbose} — when does this produce --verbose, and what does the variable's own value contribute?

6. Why is "${dest:-/tmp}" quoted, given that /tmp has no spaces in it?

7. Remember

• Braces delimit the name. ${file}_v2 is the file plus text; $file_v2 is a different variable entirely.
• Only letters, digits and underscores continue a name. Dots and slashes end it on their own — but writing ${name} everywhere means never having to decide.
• "${name}" is the form to write as a reflex — braced and quoted.
• ${name:-alt} substitutes a fallback and leaves the variable alone.
• ${name:=alt} substitutes and assigns, so the default holds for the rest of the script.
• ${name:?message} fails with your message when there is no sensible default. Use it for the values that must come from outside.
• ${name:+alt} is the inverse — it produces alt only when the variable has a value, and ignores what that value is.
• The colon decides whether empty counts as missing. With it, unset and empty behave alike; without it, only unset triggers the modifier.
• Use the colon forms by default. An empty value is almost always just as unusable as an absent one.
• ${#name} is the character count — the # before the name, which is a different operator from the # after it.
• : "${VAR:=default}" is the standard idiom for applying a default as a statement of its own.
• Quote every expansion, defaults included. The default branch is the one you tested least.
• A $VAR inside a filelist is expanded by the tool or by the flow, never by neither — Chapter 26.2 §5 puts the options and the variables where every tool sees them.

You can now read a variable safely whatever state it is in. The next chapter stops reading values whole and starts taking them apart — stripping an extension, extracting a file name from a path, and replacing text inside a value without starting a single extra process.