Batch Editing and Safe File Selection

Part III · Module 8 · Chapter 8.3 · Batch Editing and Safe File Selection

1. The List Is the Deliverable

Chapter 7.2's procedure proved the edit before running it. This chapter proves the list.

They fail differently. A wrong edit shows up in a diff of files you were expecting. A wrong list touches files you were not thinking about, or misses files you were — and neither appears in a diff you are reading, because you are reading the diff of the files you expected.

The discipline is four steps, and the first three write nothing:

   1. BUILD    find … -print0        produce the list
   2. READ     | xargs -0 -n1 echo   look at it
   3. COUNT    | wc -l               is that the number you expected?
   4. EDIT     | xargs -0 sed -i …   only now

2. The Tree

A small RTL tree, with one filename chosen to be awkward in a completely ordinary way:

find rtl -type f | sort

rtl/alu.sv
rtl/fifo.sv
rtl/regfile.sv
rtl/sub dir/odd name.sv

Four files. The fourth has a space in the directory name and in the filename. Nothing exotic — a directory called sub dir is the kind of thing that appears in a vendor drop, an extracted archive, or anywhere a Windows tool has been involved.

3. Step 1 and 2 — Build the List and Read It

find rtl -name '*.sv' -print0 | xargs -0 -n1 echo

rtl/fifo.sv
rtl/alu.sv
rtl/sub dir/odd name.sv
rtl/regfile.sv

Four entries, each on its own line, the awkward one intact. -print0 makes find separate entries with a NUL byte instead of a newline, and -0 tells xargs to expect that. NUL is the one byte that cannot appear in a filename, which is what makes it safe.

This is the dry run for the file list — the counterpart to Chapter 7.2's dry run for the edit. xargs -0 -n1 echo runs echo once per entry, so what you are reading is exactly the argument list the real command will receive.

Count it:

find rtl -name '*.sv' -print0 | xargs -0 -n1 echo | wc -l

       4

Four. If you expected four, continue. If you expected three, you have just discovered a file you did not know about, for free.

4. Why NUL — the Measurement

Drop -print0 and -0 and run the same listing:

find rtl -name '*.sv' | xargs -n1 echo

rtl/fifo.sv
rtl/alu.sv
rtl/sub
dir/odd
name.sv
rtl/regfile.sv

Six entries instead of four. rtl/sub dir/odd name.sv became three separate words — rtl/sub, dir/odd, name.sv — because xargs splits on whitespace by default, and a space in a filename is whitespace like any other.

None of those three paths exists.

5. Step 3 — Count the Hits, Not Just the Files

Knowing which files are in scope is half of it. The other half is how much each one will change, which is Chapter 7.3's counting discipline applied across a tree:

grep -rc '/proj/old/' rtl/ | grep -v ':0$'

rtl/fifo.sv:1
rtl/alu.sv:1
rtl/sub dir/odd name.sv:1
rtl/regfile.sv:1

grep -rc prints a count for every file including zeros; the second grep drops the ones with none, leaving the files that will actually change and by how much.

Four files, one hit each — four changes expected. That is the number to check against git diff --stat afterwards. An outlier here is the signal Chapter 7.3 described: the file with 40 hits where everything else has 1 is where a too-wide pattern will do its damage.

6. Step 4 — The Two Safe Forms

find … -print0 | xargs -0 sed -i reuses the list you just read:

find rtl -name '*.sv' -print0 | xargs -0 sed -i '' 's|/proj/old/|/proj/new/|g'

Exit status 0. Verify, including the awkward file:

grep -rn '/proj/old/' rtl/ || echo 'no stale references'

no stale references

cat 'rtl/sub dir/odd name.sv'

module odd;
  wire /proj/new/y;
endmodule

The space-named file was edited correctly.

find … -exec … + does the same job without a pipe:

find rtl -name '*.sv' -exec sed -i '' 's|/proj/old/|/proj/new/|g' {} +

Exit 0, and the same four files updated. The trailing + means "pass as many filenames per invocation as will fit", which is what makes it efficient; \; instead would run one sed per file.

Which to use:

• -exec … + needs no second tool, cannot be confused by any delimiter, and is a single command. It is the better default.
• -print0 | xargs -0 is better when you want the list as its own step — which is this chapter's whole argument — because the same find … -print0 can be piped to echo to review and to sed to apply, with nothing changing in between.

Note the -i '' in both: that is the BSD/macOS spelling from Chapter 7.1. On GNU sed it is -i with no argument, and a script that runs on both needs the branch from that chapter.

7. What xargs Does Not Tell You

Give the command one operand it cannot read, alongside four it can:

find rtl -name '*.sv' -print0 | xargs -0 sed -n 's/module/M/p' /nonexistent

sed: /nonexistent: No such file or directory
M fifo;
endM
M alu;
endM
M odd;
endM
M regfile;
endM

Exit status 1 — and look at the output above it. Four files were processed completely. The non-zero status is real, but it does not distinguish "nothing ran" from "almost everything ran and one thing failed".

That matters for a sweep with -i, because the files that succeeded are already written. A failed xargs is not a transaction you can roll back by re-running; it is a partial edit that needs the same verification as a successful one.

The practical consequences:

• Check the status — set -o pipefail from Chapter 8.1, since the xargs is the last stage of a pipe and its status is the one you want.
• Do not assume a non-zero status means nothing happened. Verify with git status --short regardless.
• Prefer a committed tree, which Chapter 7.2 argued, because a partial sweep is exactly the situation where git checkout -- . is the only clean recovery.

8. The Composed Procedure

Putting Chapter 7.2's edit discipline together with this chapter's list discipline gives the whole thing:

   git status --short                                  tree clean?
   find rtl -name '*.sv' -print0 | xargs -0 -n1 echo   the list — read it
   … | wc -l                                           the count — expected?
   grep -rc 'pattern' rtl/ | grep -v ':0$'             hits per file
   sed 's|old|new|g' one_file | diff -u one_file -     the edit, on one file
   find rtl -name '*.sv' -print0 | xargs -0 sed -i …   apply
   git diff --stat                                     does it match the count?
   grep -rn 'old' rtl/ || echo clean                   nothing stale survives

Eight commands, six of which write nothing. Steps 1, 5, 7 and 8 are Chapter 7.2's; steps 2, 3, 4 and 6 are this chapter's. The point of writing them out together is that neither half is sufficient: a proven edit applied to an unexamined list is as dangerous as a careless edit applied to a careful one.

9. Think It Through

1. Why is find … | xargs sed -i … dangerous even on a tree you believe has no spaces in filenames?

Because you are betting on a property of the tree that nothing enforces, and the failure is silent — section 4 measured four files becoming six bogus words, with the real file skipped, three stderr lines nobody reads, and the rest of the sweep succeeding. -print0 and -0 cost eight characters and remove the bet entirely.

2. What is the file-list equivalent of Chapter 7.2's dry run?

find … -print0 | xargs -0 -n1 echo. It runs the harmless command once per entry, so you read exactly the argument list the real command will get. Pair it with | wc -l so you have a number to check against afterwards.

3. -exec … + or -print0 | xargs -0?

-exec … + by default: one command, no pipe, nothing to get the delimiters wrong. Switch to the xargs form when you want the list as a separate, reviewable step, or when you need xargs features such as -P for parallelism or -n to limit batch size.

4. Your sweep exits 1. Did it change anything?

Probably, and you cannot tell from the status. Section 7 measured xargs exiting 1 after processing four files successfully and failing one. Check with git status --short and git diff --stat, and treat a partial sweep as needing the same verification as a complete one.

5. How would you edit only the files that actually contain the pattern?

grep -rl --null 'pattern' rtl/ | xargs -0 sed -i '' 's|old|new|g' — -l lists matching filenames and --null separates them with NUL. It is faster on a large tree and the list is self-documenting: every file in it is there because it matched.

Write --null in full. The short form -Z is a GNU spelling, and section 7's callout measures what it does on the two other greps available here — which is not this.

6. A sweep was supposed to touch 12 files and git diff --stat says 15. What now?

Stop and read the three extras. Most likely your find predicate was wider than you thought — -name '*.sv' also matches *.sv files in directories you forgot, and there is no depth limit unless you add -maxdepth. That is the list being the bug, and it is the exact failure this chapter's step 2 exists to catch before the edit, not after.

10. Summary

• The file list is a deliverable. Build it, read it, count it, then edit. The first three steps write nothing.
• Use NUL delimiters always: find … -print0 | xargs -0. A single ordinary filename with spaces turned four targets into six words, three of which do not exist.
• That failure is silent in practice — the real file is skipped, the rest of the sweep succeeds, and the evidence is stderr lines and an exit status nothing checks.
• xargs -0 -n1 echo is the dry run for the list, the counterpart to Chapter 7.2's dry run for the edit.
• find's order is filesystem order, not sorted. Use sort -z to keep the chain NUL-safe when a human will read the list.
• Count hits per file first: grep -rc 'pat' dir/ | grep -v ':0$' gives the number to check git diff --stat against, and shows outliers.
• -exec … + is the better default — one command, no pipe. The xargs form wins when the list should be its own reviewable step.
• -i '' here is the BSD spelling from Chapter 7.1; GNU takes no argument.
• A non-zero xargs status does not mean nothing happened. Four files were already written when it exited 1. Verify regardless, and keep the tree committed.
• Edit only what matches with grep -rl --null … | xargs -0 sed -i …. Write --null in full: -Z is GNU-only, and on both greps here it emitted newlines, which made xargs -0 treat the whole list as one filename.

That is the mechanical half of Part III complete: you can write the edit, prove it, apply it to a list you have checked, and recover if it was wrong. The remaining question is the one this track raised in Chapter 1.3 and has deferred ever since — whether the job should have been a sed job at all. Chapter 8.4 — When to Use Tcl or Python Instead answers it, and it closes Part III.