Linux & Bash · Part I · Module 9 · Quoting and Shell Expansion

Brace and Tilde Expansion

The first two stages of the pipeline are the two most useful ones to type by hand. Braces turn one word into many; the tilde saves you writing out a home directory. Both are routinely confused with the file-matching stage, and the difference between generating a name and finding one decides which tool you need.

Intermediate17 min readLinuxPart IFundamentalsQuotingExpansion

Part I · Module 9 · Chapter 9.5 · Brace and Tilde Expansion

Chapter 9.1 put braces at stage one and the tilde at stage two, proved that braces run before variables, and moved on. This chapter is the detail — and the distinction that makes braces worth understanding rather than just copying.

1. Braces Generate; Globs Find

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ echo nope_{1..3}.log
nope_1.log nope_2.log nope_3.log

$ echo nope_*.log
nope_*.log

There are no files called nope_1.log in that directory. The brace produced three names anyway. The glob produced nothing — it stayed literal, because nothing matched.

The distinction

A brace is a text generator. It knows nothing about the filesystem and never consults it. It turns one word into several, whatever is or is not on disk.

A glob is a filter over names that exist. It can only ever produce files that are really there.

So braces are for making things and globs are for finding them:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

mkdir -p run_{1..8}/logs      # creating eight directories that do not exist yet
grep ERROR run_*/logs/*.log   # searching the ones that do

Reaching for the wrong one is the usual mistake, and it fails in opposite directions — a glob that creates nothing, or a brace that cheerfully names forty files you do not have.

2. The Brace Forms

Lists

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ echo {apple,banana,cherry}
apple banana cherry

$ echo file.{txt,bak}
file.txt file.bak

Anything attached before or after the braces is repeated onto each item. That is what makes it useful rather than just a way to type commas.

Ranges

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ echo {1..5}
1 2 3 4 5

$ echo {a..e}
a b c d e

Ranges work on integers and on single letters, and they count downward too — {5..1} gives 5 4 3 2 1.

Cross products

Put two braces in one word and you get every combination, left varying slowest:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ echo {A,B}{1,2}
A1 A2 B1 B2

$ echo run_{1..3}_seed{7,9}
run_1_seed7 run_1_seed9 run_2_seed7 run_2_seed9 run_3_seed7 run_3_seed9

Six directories from one short line. This is where braces earn their place.

A brace needs a comma or a range

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ echo {single}
{single}
$ echo {}
{}
$ echo {a}
{a}

All literal. With nothing to vary, there is no expansion, and the braces are left as ordinary characters. That is deliberate and occasionally useful, but it mostly matters as an explanation for why a brace you expected to expand did not: check that it contains either a comma or a ...

The backup idiom

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ echo cp config.yml{,.bak}
cp config.yml config.yml.bak

The empty item before the comma contributes the bare word. It reads strangely the first time and is extremely common:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

cp config.yml{,.bak}          # back it up
mv run.log{,.old}             # rotate it

Two brace forms need Bash 4

Measured on the verification machine, which runs Bash 3.2 — still what macOS ships:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ echo {1..10..3}
{1..10..3}

$ echo {05..10}
5 6 7 8 9 10

On Bash 4 and later the first gives 1 4 7 10 (a step) and the second gives 05 06 07 08 09 10 (zero-padding preserved).

Neither produces an error on 3.2 — the stepped form stays literal and the padded form silently loses its zeros. A script relying on either works on Linux and then produces a directory called {1..10..3} on a colleague's laptop.

seq handles both portably:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

seq 1 3 10            # 1 4 7 10
seq -w 5 10           # 05 06 07 08 09 10

3. Braces and Variables, Again

Chapter 9.1 proved braces run first. In practice that means one specific thing does not work:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ n=5
$ echo {1..$n}
{1..5}

The three ways round it:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

echo $(seq 1 "$n")                              # a command, so stage 3
for ((i = 1; i <= n; i++)); do ...; done        # evaluated at run time
eval "echo {1..$n}"                             # works, and see 9.4 §6

Prefer the second in a script and the first at a prompt. The third is here only so you recognise it; eval turns the whole line back into code, for the reasons Chapter 9.4 gave.

Note the asymmetry: a variable inside the braces is too late, but a variable next to them is fine, because that part is just text being repeated:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ prefix=run
$ echo ${prefix}_{1..3}
run_1 run_2 run_3

Stage one copies the literal text ${prefix}_ onto each item, and stage three then substitutes it in all three copies.

4. The Tilde

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ echo ~
/home/engineer

$ echo ~/work
/home/engineer/work

~ at the start of a word becomes your home directory. Not in the middle — /tmp/~/x is a literal tilde — and only as the first character.

The four forms

Form	Becomes
`~`	your home directory
`~user`	that user's home directory
`~+`	the current directory — the same as `$PWD`
`~-`	the previous directory — the same as `$OLDPWD`

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ echo ~root
/var/root

$ cd /tmp
$ echo ~+
/tmp
$ echo ~-
/home/engineer/project

~- pairs with the cd - from Chapter 2.4 — same idea, usable inside a longer path:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

cp report.txt ~-/archive/

Quotes switch it off

Chapter 9.3 covered this and it is worth repeating here, because it is the one place where the quote-everything habit actively hurts:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ echo ~/project
/home/engineer/project

$ echo "~/project"
~/project

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

mkdir -p ~/backups          # correct
mkdir -p "~/backups"        # creates a DIRECTORY NAMED ~ in the current dir

That second line does not fail. It silently makes a directory called ~, which then sits there confusing people and is awkward to delete, because rm -rf ~ means something else entirely. (rm -rf ./~ is the safe spelling.)

In a script, use $HOME

$HOME is an ordinary variable, so it behaves predictably with the quoting habit everything else in this module has taught:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

cp "$file" "$HOME/backups/"
logdir="$HOME/runs/$run_name"

~ is for typing at a prompt, where it is quicker. "$HOME" is for scripts, where consistency matters more than three saved characters — and where someone will eventually quote the whole path.

A tilde from a variable is never expanded

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ t="~"
$ echo $t
~

$ t="~/project"
$ cd $t
bash: cd: ~/project: No such file or directory

Tilde expansion is stage two; the variable is not substituted until stage three. The tilde arrives as data, too late to be syntax — exactly the same mechanism as the quote character in Chapter 9.1 §5.

This bites when a path comes from a configuration file or a prompt. If a user types ~/logs, you receive two literal characters and a path. Store "$HOME/logs" instead, or substitute it yourself:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

dir="${dir/#\~/$HOME}"

That is the anchored replacement from Chapter 8.3, with the tilde escaped so it is matched literally.

5. Using Them Together

Setting up a regression tree is the natural worked example, and it is one line:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ mkdir -p ~/runs/sweep_{1..3}/{logs,waves,reports}

$ find ~/runs/sweep_1 -type d | sort
/home/engineer/runs/sweep_1
/home/engineer/runs/sweep_1/logs
/home/engineer/runs/sweep_1/reports
/home/engineer/runs/sweep_1/waves

Nine directories, from one command. The cross product gave sweep_1/logs, sweep_1/waves, sweep_1/reports, sweep_2/logs and so on; -p created the intermediate levels.

Then, once they contain something, the other tool:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

grep -l ERROR ~/runs/sweep_*/logs/*.log

Brace to create, glob to search — in two consecutive commands, which is the clearest possible statement of §1.

Check a destructive brace with echo first

A brace expands to exactly what you wrote, with no filesystem to stop you:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ echo rm -rf ~/runs/sweep_{1..30}
rm -rf /home/engineer/runs/sweep_1 /home/engineer/runs/sweep_2 ...

Putting echo in front and reading the result costs two seconds. Unlike a glob, a brace gives you no feedback at all when the names are wrong — it will happily generate thirty paths that do not exist, or thirty that do and should not have been touched.

Practice

1 — Generate.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

echo {1..5}
echo {a..e}
echo {5..1}
echo {apple,banana}
echo file.{txt,bak,log}

2 — Cross-multiply.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

echo {A,B}{1,2}
echo run_{1..3}_seed{7,9}
echo {A,B}{1,2}{x,y}

3 — Find the ones that stay literal.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

echo {single}; echo {}; echo {a}; echo {1.5}

4 — Check your Bash version against the two risky forms.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

bash --version | head -1
echo {1..10..3}
echo {05..10}
seq 1 3 10
seq -w 5 10

5 — Prove braces precede variables.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

n=4
echo {1..$n}
echo $(seq 1 "$n")
prefix=run; echo ${prefix}_{1..3}

6 — Build and then search.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

mkdir -p /tmp/bt/sweep_{1..3}/{logs,waves}
find /tmp/bt -type d | sort
touch /tmp/bt/sweep_{1,3}/logs/run.log
echo /tmp/bt/sweep_*/logs/*.log

Note that the glob found two files, not three — sweep_2 has no log, and the glob only ever names what exists.

7 — Walk the tilde forms.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

echo ~
echo ~root
cd /tmp && echo ~+ && echo ~-

8 — Make the quoting mistake safely.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

cd /tmp/bt
mkdir -p "~/oops"
ls -d ./~
rm -rf "./~"

9 — Watch a tilde from a variable stay literal.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

t="~/project"; echo "$t"; echo "${t/#\~/$HOME}"

10 — Clean up. rm -rf /tmp/bt

Check Your Understanding

1. What is the difference between echo run_{1..3}.log and echo run_*.log in an empty directory?

2. echo {single} prints {single}. Is something wrong?

3. n=4; mkdir run_{1..$n} creates one directory. What is it called, and why?

4. mkdir -p "~/backups" succeeds but your home directory is unchanged. What happened?

5. A config file contains logdir = ~/logs and your script does cd "$logdir", which fails. Why, and what are two fixes?

6. You want eight directories run_01 to run_08, zero-padded, on any machine. How?

Answers

1. The brace prints three names — run_1.log run_2.log run_3.log — because it generates text without consulting the filesystem. The glob prints run_*.log unchanged, because nothing matched and an unmatched pattern is passed through literally (Chapter 9.6).

2. No. A brace needs something to vary — a comma or a .. range. With neither, there is nothing to expand, so the braces stay as ordinary characters.

3. One directory literally named run_{1..4}. Brace expansion is stage one and variable expansion is stage three, so when the brace stage ran, the range end was the text $n. By the time it became 4, the brace stage was over.

4. The quotes switched off tilde expansion, so mkdir created a directory named ~ in the current directory. Nothing failed, which is what makes it confusing. Use ~/backups unquoted, or "$HOME/backups".

5. The tilde arrived as data at stage three, long after stage two where tildes are expanded, so $logdir holds a literal ~ character. Either store $HOME/logs in the config instead, or substitute it in the script: logdir="${logdir/#\~/$HOME}".

6. The Bash 4 brace form run_{01..08} is shortest, but it loses its zeros on Bash 3.2, so it is not the portable answer. Use a loop with printf, which gives you the padding explicitly:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

for i in $(seq 1 8); do mkdir -p "$(printf 'run_%02d' "$i")"; done

6. Remember

A brace generates text; a glob finds files. The brace never consults the filesystem; the glob can only name what exists.
Brace to create, glob to search. mkdir -p run_{1..8} to make them, then grep ERROR run_*/*.log to search them.
Lists and ranges: {a,b,c} and {1..5} and {a..e}, with attached text repeated onto each item.
Two braces in one word give a cross product, left varying slowest.
A brace with no comma and no range stays literal.
cp config.yml{,.bak} is the backup idiom — the empty item gives the bare word.
{1..10..3} and zero-padded {05..10} need Bash 4, and macOS ships 3.2. Use seq and seq -w for portability.
A variable cannot supply a brace range (stage one precedes stage three), but a variable beside a brace is fine.
~ expands only at the start of a word: ~, ~user, ~+ for here, ~- for the previous directory.
Quotes switch off the tilde, and mkdir "~/x" silently creates a directory called ~.
Use "$HOME" in scripts and ~ at the prompt.
A tilde arriving from a variable is never expanded — it is data by then. Fix it with "${v/#\~/$HOME}".
Put echo in front of a destructive brace before running it. A brace gives no feedback when the names are wrong.

Two stages left unexamined, and they are the two that cause damage: what happens when an expansion produces nothing, and what happens when it produces far too much.

Where this fits

Part of the Linux & Bash curriculum.