Acting on What find Found: -exec, xargs and NUL Safety

Part I · Module 14 · Chapter 14.7 · Acting on What find Found

Chapter 14.6 produced lists of paths. This chapter does something with them without breaking anything.

1. find Can Run the Command Itself

$ find . -name '*.txt' -exec echo "called with:" {} \;
called with: ./two words.txt
called with: ./it's.txt
called with: ./-leading.txt
called with: ./normal.txt
called with: ./sub/nested.txt

-exec COMMAND {} \; runs the command once for each match, with {} replaced by the path.

Two pieces of syntax:

• {} is where the filename goes. It can appear more than once.
• \; ends the command. The semicolon must be escaped, or the shell takes it as a command separator (Chapter 11.2).

Every one of those five names arrived intact — including the one with a space, the one with an apostrophe and the one starting with a hyphen. find passes each path as a single argument directly to the command, with no shell in between, so there is nothing to split it.

2. Once Per File, or Once For All

$ find . -name '*.txt' -exec echo "called with:" {} +
called with: ./two words.txt ./it's.txt ./-leading.txt ./normal.txt ./sub/nested.txt

+ batches the paths into as few invocations as possible instead of one per file.

The difference is performance, and on a large tree it is substantial:

find . -name '*.tmp' -exec rm {} \;     # 4000 files → 4000 processes
find . -name '*.tmp' -exec rm {} +      # 4000 files → 2 or 3 processes

3. The Other Way, and Why It Breaks

xargs reads a list on standard input and builds command lines from it. The obvious pairing:

$ find . -name '*.txt' | xargs -n1 echo "got:"
got: ./two
got: words.txt
xargs: unterminated quote

Two separate failures in three lines of output.

• ./two words.txt became two arguments. xargs splits its input on whitespace, and a space inside a name is indistinguishable from a space between names.
• Then it died. ./it's.txt contains an apostrophe, which xargs treats as an opening quote, and the input ended before a closing one.

Neither failure is hypothetical and both are silent until they are not. The first would have deleted, moved or edited two files that do not exist — or worse, two that do.

4. Null Separation

$ find . -name '*.txt' -print0 | xargs -0 -n1 echo "got:"
got: ./two words.txt
got: ./it's.txt
got: ./-leading.txt
got: ./normal.txt
got: ./sub/nested.txt

All five, intact.

• -print0 makes find separate paths with a null byte instead of a newline.
• -0 makes xargs expect that separator.

A file name can contain any byte except / and null. So null is the only separator that cannot appear inside a name — which is why this pair exists and why there is no alternative that is merely "good enough".

5. Which to Use

Both approaches work. They differ in three ways that decide it.

6. xargs in Detail

$ find . -name '*.txt' -print0 | xargs -0 -I{} echo "would copy {} to {}.bak"
would copy ./two words.txt to ./two words.txt.bak

-t is the one to use while developing. It shows exactly what is about to run:

find . -name '*.tmp' -print0 | xargs -0 -t rm

And the parallel form, which is the real reason to choose xargs:

find . -name '*.sv' -print0 | xargs -0 -P 8 -n 1 ./lint_one.sh

Eight lint processes at once (Chapter 7.5's concurrency, with the scheduling done for you). On an eight-core machine that is close to an eight-fold speed-up, and it is the standard way to parallelise a per-file operation.

7. The Built-In Actions

For two common cases, find has an action and you need neither mechanism.

$ find d -name '*.tmp' -delete

-delete removes matching files, with no subprocess at all. It is faster than -exec rm {} + and there is no quoting to get wrong.

Three cautions:

• It is not undoable and there is no confirmation. Run the same find without -delete first and read the list. Every time.
• -delete implies -depth, so directories are processed after their contents — which is what makes it able to remove directory trees, and also changes the traversal order in ways that can interact with -prune.
• It only removes empty directories. For a non-empty tree you still want rm -rf.

$ find . -name 'filelist.f' -exec dirname {} \;

-exec with a command that takes one argument, which is the \; case from §2.

8. A Shell Loop, and When Not To

The third approach:

find . -name '*.log' -print0 | while IFS= read -r -d '' f; do
  echo "processing $f"
done

read -d '' reads up to a null byte rather than a newline (Chapter 8.6), which is what makes this safe.

It is the right tool when the body needs more than one command, a conditional, or a variable that survives:

while IFS= read -r -d '' f; do
  lines=$(wc -l < "$f")
  [ "$lines" -gt 1000 ] && echo "$f is large: $lines lines"
done < <(find . -name '*.log' -print0)

Note < <(find …) rather than a pipe — Chapter 12.5 and Chapter 11.4 both showed that a piped loop runs in a subshell and loses its variables. With the process substitution, a counter accumulated in that loop survives.

For a single command per file, the loop is slower and noisier than -exec … +. Use it when there is logic, not when there is a command.

9. In Practice

# preview, then delete
find . -name '*.tmp'
find . -name '*.tmp' -delete

# compress every old log, batched
find /scratch -name '*.log' -mtime +7 -exec gzip {} +

# search inside files a condition selected
find . -name '*.v' -newer last_build -print0 | xargs -0 grep -ln 'clk_div'

# rename, which needs one at a time
find . -name '*.vh' -exec sh -c 'mv "$1" "${1%.vh}.svh"' _ {} \;

# lint in parallel across eight cores
find . -name '*.sv' -print0 | xargs -0 -P 8 -n 1 ./lint_one.sh

# the total size of everything a search matched
find . -name '*.log' -print0 | xargs -0 du -ch 2>/dev/null | tail -1

# edit only the files that actually contain the pattern
grep -rlZ --include='*.v' 'OLD_MACRO' . | xargs -0 sed -i.bak 's/OLD_MACRO/NEW_MACRO/g'

Two of those are worth a closer look.

The rename uses sh -c '…' _ {} — the positional-parameter form from Chapter 9.4. The inner script is single-quoted so the outer shell leaves it alone, and the filename arrives as $1 rather than being pasted into text. Without that, a name containing a space or a quote would break the inner shell.

The last one combines Chapter 14.5's -lZ with this chapter's xargs -0 and Chapter 13.6's portable sed -i.bak. It edits only the files that need it, and it is safe with any file name.

Practice

mkdir -p /tmp/act/sub && cd /tmp/act
touch "normal.txt" "two words.txt" "it's.txt" "-leading.txt" sub/nested.txt
mkdir -p d && touch d/a.tmp d/b.tmp "d/c d.tmp"

1 — See -exec handle every name.

find . -name '*.txt' -exec echo "called with:" {} \;

2 — Compare \; with +.

find . -name '*.txt' -exec echo "called with:" {} \;
find . -name '*.txt' -exec echo "called with:" {} +

Five lines versus one.

3 — Watch the naive pipeline fail twice.

find . -name '*.txt' | xargs -n1 echo "got:"

A split name, then a crash.

4 — Fix it.

find . -name '*.txt' -print0 | xargs -0 -n1 echo "got:"

5 — Use the placeholder.

find . -name '*.txt' -print0 | xargs -0 -I{} echo "would copy {} to {}.bak"

6 — Preview with -t.

find d -name '*.tmp' -print0 | xargs -0 -t echo rm

7 — Preview, then delete.

find d -name '*.tmp'
find d -name '*.tmp' -delete
find d -name '*.tmp' | wc -l

8 — Run things in parallel and feel the difference.

time (for i in 1 2 3 4; do sleep 1; done)
time (printf '1\n2\n3\n4\n' | xargs -P 4 -n 1 sleep)

Four seconds versus one.

9 — Use a loop where logic is needed.

n=0
while IFS= read -r -d '' f; do
  n=$(( n + 1 ))
done < <(find . -name '*.txt' -print0)
echo "counted $n files, and the counter survived"

10 — Clean up. cd /tmp && rm -rf /tmp/act

Check Your Understanding

1. What is the difference between -exec cmd {} \; and -exec cmd {} +?

2. find . -name '*.txt' | xargs rm on a directory containing two words.txt. What happens?

3. Why is a null byte the right separator and not, say, a newline?

4. When is xargs worth the extra risk over -exec?

5. Why does the rename example use sh -c '…' _ {} rather than mv {} …?

6. Why must a while read loop over find output use < <(…) rather than a pipe?

10. Remember

• -exec cmd {} \; runs once per file; -exec cmd {} + batches. Prefer +.
• \; is needed when the command takes one file, when {} is not last, or when you want a per-file status.
• -exec is inherently safe with awkward names, because find passes each path directly with no shell in between.
• find | xargs splits names at spaces and dies on quotes. Both failures are measurable and neither is rare.
• -print0 with xargs -0 is one unit of notation. Null is the only byte a file name cannot contain.
• One without the other is worse than neither.
• Use xargs for parallelism (-P), placeholders (-I{}), batch sizes (-n), or a non-find source.
• xargs -t prints each command before running it — use it while developing.
• Empty input differs: GNU runs the command once with nothing, BSD does not. -exec … + sidesteps it.
• -delete needs no subprocess, is not undoable, and only removes empty directories.
• Preview, then act. Run the find without the action and read the list — every time.
• A while read -r -d '' loop is right when there is logic, and must use < <(…) so the loop is not a subshell. Chapter 17.2 measures why the NUL form is needed at all, and Chapter 17.7 is the subshell half.
• To transform a name, pass it as $1 to sh -c, never paste it into the script text.
• A list of files handed to a tool is a filelist — Chapter 26.1 measures why it exists: 40,000 paths is 2 MB against a 1 MB ARG_MAX, and the glob compile dies with status 126.

That is both halves of the module: finding files, and finding what is in them. The last chapter is about choosing between them — and between these tools and the ones from Module 13 — from the question rather than from habit.