Trimming and Replacing Inside a Value

Part I · Module 8 · Chapter 8.3 · Trimming and Replacing Inside a Value

Chapter 8.2 was about what to do when a value is missing. This chapter is about a value you have and want only part of.

1. The One Sentence

There are four trimming operators. They look cryptic and they are arranged on two perfectly regular axes:

• # removes from the front. % removes from the back.
• One character removes the shortest match. Two characters remove the longest.

That is all of it. Everything in §2 and §3 is that sentence applied.

2. The Four Operators

Take one value and apply all four:

$ p="project/rtl/alu.sv"

$ echo "${p#*/}"
rtl/alu.sv

$ echo "the front-trim forms"
alu.sv

$ echo "${p%/*}"
project/rtl

$ echo "${p%%/*}"
project

Four different, useful answers from one value. Read them against the two axes:

• ${p#*/} — from the front, shortest match of "anything then a slash". The shortest such match is project/, so what remains is rtl/alu.sv.
• the front-trim forms — from the front, longest. The longest match of "anything then a slash" is project/rtl/, leaving alu.sv — the file name.
• ${p%/*} — from the back, shortest match of "a slash then anything". That is /alu.sv, leaving project/rtl — the directory.
• ${p%%/*} — from the back, longest. That is /rtl/alu.sv, leaving project — the top-level directory.

The two you will actually type

$ p="/home/engineer/project/rtl/alu.sv"

$ echo "the front-trim forms"
alu.sv

$ echo "${p%/*}"
/home/engineer/project/rtl

the front-trim forms is the file name and ${p%/*} is the directory. They are the built-in equivalents of two commands you may already know:

$ basename "$p"
alu.sv

$ dirname "$p"
/home/engineer/project/rtl

Same answers. The difference is that basename and dirname are external programs (Chapter 1.7), so each call starts a process; the expansions are done by the shell itself and start nothing.

3. Extensions

The same four operators, with a dot instead of a slash:

$ f="run_42.tar.gz"

$ echo "${f%.*}"
run_42.tar

$ echo "${f%%.*}"
run_42

$ echo "${f#*.}"
tar.gz

$ echo "${f##*.}"
gz

• ${f%.*} removes the last extension.
• ${f%%.*} removes all extensions — the stem.
• ${f##*.} is the extension itself, with no dot.

And the everyday case, with a literal pattern rather than a wildcard:

$ file="run_42.log"
$ echo "${file%.log}"
run_42

${file%.log} removes exactly that text, and only if it is at the end. If the value does not end in .log, nothing is removed and you get the value unchanged — which is a benign, silent no-op worth knowing about, because it means a typo in the suffix produces no error at all.

The classic use is changing an extension:

$ log="run_42.log"
$ echo "${log%.log}.summary"
run_42.summary

4. Replacing

Trimming works at the ends. Replacement works anywhere.

$ s="test_a test_b test_c"

$ echo "${s/test/sim}"
sim_a test_b test_c

$ echo "${s//test/sim}"
sim_a sim_b sim_c

• One slash replaces the first match.
• Two slashes replace every match.

The same doubling idea as the trim operators — one does the minimum, two does the lot.

Deleting is replacing with nothing:

$ echo "${s//test_/}"
a b c

And the replacement can be anchored to one end:

$ path="rtl/alu.sv"

$ echo "${path/#rtl/src}"
src/alu.sv

$ echo "${path/%.sv/.v}"
rtl/alu.v

/# anchors at the start and /% anchors at the end — reusing the same two characters, with the same meanings, in a third place.

5. Substrings by Position

$ v="run_20260407_142233"

$ echo "${v:4}"
20260407_142233

$ echo "${v:4:8}"
20260407

$ echo "${v:0:3}"
run

$ echo "a negative substring offset"
142233

${v:start} takes everything from that position; ${v:start:length} takes a fixed number of characters. Positions count from zero.

Positional substrings are worth using only when the input genuinely has fixed-width fields — a timestamp, a formatted ID. For anything with separators, the trim operators are correct and positions are a trap, because they break the moment a field changes width.

6. Four Things That Will Catch You

These are glob patterns, not regular expressions

The * in the front-trim forms is the filename-matching wildcard from Chapter 3.5, not the regular-expression one. The vocabulary is the small one:

There is no +, no \d, no grouping, no alternation and no capture. Anything needing those is a job for sed and awk in Module 13, or grep -E in Module 14.

They cannot be chained

$ p=/a/b/c.log
$ echo "${the front-trim forms%.log}"
bash: ${the front-trim forms%.log}: bad substitution

One operator per expansion. To do two things, use two steps:

$ p=/a/b/c.log
$ base="the front-trim forms"
$ echo "${base%.log}"
c

This is a hard error rather than a silent wrong answer, which is the good kind of limitation — you find out immediately.

They work on one value, not on lines

These operators see a variable's value as a single string. If that value contains newlines, a * will happily match across them, and there is no concept of "do this to each line".

Per-line work belongs to sed, awk or a loop with read (Chapter 8.6). Using expansions on multi-line text usually produces something that looks nearly right, which is worse than failing.

Case conversion is newer than you might assume

Bash 4 added ${v^^} for uppercase and ${v,,} for lowercase. They are convenient and they are not universal — Bash 3.2 rejects them outright:

$ v=hello
$ echo "${v^^}"
bash: ${v^^}: bad substitution

That matters because macOS still ships Bash 3.2, so a script written on a modern Linux box can fail on a colleague's laptop in a way that names a line rather than a feature. Check with bash --version or use tr '[:lower:]' '[:upper:]', which works everywhere.

7. Doing Real Work With It

A loop that turns every log into a summary file, and extracts the seed from the name:

for log in sim/logs/*.log; do
  name="${log##*/}"        # run_42_seed7.log
  stem="${name%.log}"      # run_42_seed7
  seed="${stem##*seed}"    # 7
  out="${log%.log}.summary"

  echo "log=$log  name=$name  seed=$seed  out=$out"
done

Four string operations per file, no external commands at all. Written with basename and sed it would be three processes per iteration and about the same number of characters — and over a directory of two thousand logs, a visible wait instead of none.

The for loop is Chapter 17.1's subject; the glob is Chapter 3.5. Everything between the loop markers is this chapter.

8. When to Stop and Use a Real Tool

Expansions are the right answer for small, structural edits on a single value. They are the wrong answer for:

• anything spanning lines — use sed, awk, or a read loop;
• anything needing a real regular expression — alternation, repetition counts, capture groups;
• anything you would struggle to read in six months. A triple-nested sequence of trims saves three processes and costs an hour of somebody's time later. Add a comment, or use the obvious tool.

The honest rule is that these forms earn their place in loops and in the small path arithmetic every script does. Reaching for them to avoid sed on principle is how a script becomes unreadable.

Practice

1 — Walk the matrix. Type all four against one relative path and confirm Figure 1:

p="project/rtl/alu.sv"
echo "#   -> ${p#*/}"
echo "##  -> the front-trim forms"
echo "%   -> ${p%/*}"
echo "%%  -> ${p%%/*}"

Then repeat with an absolute path, /home/engineer/project/rtl/alu.sv, and work out why %% now gives nothing at all.

2 — Compare with the commands.

p="/home/engineer/project/rtl/alu.sv"
echo "the front-trim forms  vs  $(basename "$p")"
echo "${p%/*}   vs  $(dirname "$p")"

3 — Take extensions apart.

f="run_42.tar.gz"
echo "last off : ${f%.*}"
echo "all off  : ${f%%.*}"
echo "ext      : ${f##*.}"
echo "inner    : ${f#*.}"

4 — Replace, then replace everything.

s="test_a test_b test_c"
echo "${s/test/sim}"
echo "${s//test/sim}"
echo "${s//test_/}"

5 — Anchor a replacement.

path="rtl/alu.sv"
echo "${path/#rtl/src}"
echo "${path/%.sv/.v}"

6 — Fall into the negative-offset trap on purpose.

v="run_20260407_142233"
echo "no space : ${v:-6}"
echo "with     : a negative substring offset"

7 — Confirm chaining fails. Run echo "${${v##*_}%3}" and read the error. Then do it in two steps.

8 — Check your Bash. Run bash --version, then v=hello; echo "${v^^}". If it fails, you now know what the version number was telling you.

9 — Build the loop. Create a few files named run_1_seed3.log, run_2_seed9.log, and run §7's loop over them.

Check Your Understanding

1. the front-trim forms and ${p%/*} on /home/engineer/notes.txt — what does each give, and what are their everyday names?

2. Why does ${f%.log} leave report.txt completely unchanged rather than reporting an error?

3. What is the difference between ${s/a/b} and ${s//a/b}?

4. ${v:-8} on a variable set to run_12345678 gives the whole value, not 12345678. Why?

5. You want to turn a.log into a.txt for two thousand files in a loop. Give the expansion, and say what you avoided by not using sed.

6. When should you stop using these and use sed or awk instead?

9. Remember

• One sentence covers the four trim operators: # removes from the front and % from the back; one character takes the shortest match and two take the longest.
• the front-trim forms is the file name and ${p%/*} is the directory — the two you will type most, and the built-in equivalents of basename and dirname.
• The expansions start no process. In a loop over thousands of files that is the whole runtime difference.
• ${f%.ext} strips a suffix; ${f##*.} is the extension. A pattern that does not match removes nothing, silently.
• ${v/old/new} replaces the first match, ${v//old/new} every match — the same single-versus-double rule again.
• /# and /% anchor a replacement to the start and the end.
• ${v:start:length} counts from zero, and a negative offset needs a leading space or it is read as the default operator.
• These are glob patterns, not regular expressions. *, ? and character sets only.
• Operators cannot be chained. Two edits means two steps — and the error is loud, not silent.
• They operate on one value, not on lines. Multi-line work belongs to sed, awk or a read loop.
• ${v^^} and ${v,,} need Bash 4, and macOS ships Bash 3.2. Use tr when portability matters.
• Quote the result anyway. A clever expansion still produces a value that has to reach a command intact.

Everything so far has worked on values you typed in yourself. The next chapter is the one that changes what a shell can do: taking the output of a command and using it as a value.