In-Place Editing and Portability

Part III · Module 7 · Chapter 7.1 · In-Place Editing and Portability

1. The Option, in Both Spellings

-i makes sed write its result back to the input file instead of to standard output. The two seds you will meet spell it differently, and the spellings are not interchangeable:

   sed -i    's|old|new|g' file      GNU sed      — no argument
   sed -i '' 's|old|new|g' file      BSD / macOS  — an explicit empty argument

On GNU sed, -i takes an optional suffix attached directly to it (-i.bak). On BSD sed the suffix is a separate argument and is mandatory — an empty string means "no backup". Section 8 shows what happens when you get that wrong.

A script that must run on both needs a branch. There is no single spelling that works everywhere, and that fact alone makes -i the least portable thing in sed.

2. What "In Place" Actually Means

It does not mean what it says. The sequence is:

3. Proving It: the Inode Changes

Every file on disk has an inode number — the identity of the file itself, as distinct from the name pointing at it. If -i edited the file, the inode would stay the same.

printf '/proj/old/alu.sv\n/proj/old/fifo.sv\n' > filelists/top.f
stat -f "mode=%Sp inode=%i links=%l" filelists/top.f

mode=-rw-r--r-- inode=85155273 links=1

sed -i '' 's|/proj/old/|/proj/new/|g' filelists/top.f
stat -f "mode=%Sp inode=%i links=%l" filelists/top.f

mode=-rw-r--r-- inode=85155275 links=1

The inode changed — 85155273 became 85155275. The content is correct:

cat filelists/top.f

/proj/new/alu.sv
/proj/new/fifo.sv

…but the file holding it is a new one. The old file was unlinked when the rename happened.

For most work this is invisible and harmless. For three situations it is not, and they are the rest of this chapter.

4. A Hard Link Breaks, Silently

Two names for one file. This is how some flows share a common filelist or constraint file between directories without duplicating it.

printf 'original\n' > hard_a.txt
ln hard_a.txt hard_b.txt
stat -f "links=%l inode=%i" hard_a.txt hard_b.txt

links=2 inode=85155276
links=2 inode=85155276

Two names, one inode, link count two — one file. Now edit through one of the names:

sed -i '' 's/original/edited/' hard_a.txt
stat -f "links=%l inode=%i" hard_a.txt hard_b.txt

links=1 inode=85155279
links=1 inode=85155276

Two different inodes, each with one link. The two names are now two separate files. And the contents have diverged:

cat hard_a.txt

edited

cat hard_b.txt

original

5. A Symlink Is Refused

A symbolic link is a name pointing at another name. Edit through it:

printf 'target content\n' > real.txt
ln -s real.txt link.txt
sed -i '' 's/target/EDITED/' link.txt

sed: link.txt: in-place editing only works for regular files

Exit status 1, and nothing changed — link.txt is still a symlink and real.txt still says target content. BSD sed refuses the operation outright.

That is the good outcome, and it is worth knowing it is not universal. GNU sed's documented default is to follow a different path: it replaces the symlink itself with a regular file containing the edited text, breaking the link, unless you pass --follow-symlinks. No GNU sed was available on the machine these examples were verified against, so that is stated from its documentation rather than shown — but it means the same command can refuse on a Mac and quietly destroy a symlink on a build server.

If your tree has symlinks, resolve them before editing, and know which sed you are on. Chapter 2.3 showed how to tell.

6. What Is and Is Not Preserved

Here the honest answer is narrower than the usual warning.

File mode is preserved on the sed used here. An executable script:

chmod 755 exec.sh
sed -i '' 's/a/b/' exec.sh
stat -f %Sp exec.sh

-rwxr-xr-x

Still executable. And a private file:

chmod 600 priv.txt
sed -i '' 's/a/b/' priv.txt
stat -f %Sp priv.txt

-rw-------

Still 600. So the common claim that -i "loses your permissions" was not true in these runs.

Ownership is a different question and was not measured. Changing a file's owner requires privileges, so the risk is real in principle — if a root-owned file is edited by another user who can write the directory, the replacement is owned by that user. Testing that needs a second user account, which was not available here, so this chapter does not claim a result either way. Treat it as a reason to check ls -l after a sweep over files you do not own, not as something these examples demonstrated.

Extended attributes, ACLs and timestamps are in the same category: plausibly affected by a create-and-rename, not measured here.

7. Permissions Belong to the Directory, Not the File

This is the most counter-intuitive consequence, and it falls straight out of Figure 1.

Mark a file read-only and edit it:

printf 'a\n' > ro.txt
chmod 444 ro.txt
sed -i '' 's/a/b/' ro.txt

Exit status 0, no complaint:

cat ro.txt

b

A read-only file was edited. Not because sed ignored the permission, but because it never wrote to that file — it created a new file and renamed it, and neither operation needs write permission on the old file. They need write permission on the directory.

Confirm it from the other side. Same read-only file, this time in a directory that is not writable:

chmod 555 rodir
sed -i '' 's/a/b/' rodir/f.txt

sed: rodir/f.txt: Permission denied

Exit status 1, and the file is unchanged. The file's own mode was identical in both runs; only the directory differed.

The practical consequence: chmod 444 is not protection against a sed sweep. If you are relying on read-only bits to keep a generated or golden file from being rewritten, that is not what they do here.

8. The BSD -i Trap

The commonest portability mistake with -i is forgetting the mandatory argument on BSD. The option then eats your script:

sed -i 's/q/Q/' n1.txt n2.txt

sed: 1: "n1.txt
": extra characters at the end of n command

Exit status 1, and both files are unchanged. Trace it: -i took s/q/Q/ as the backup suffix, so the first remaining operand — n1.txt — became the script. sed read n as the next-line command and then found 1.txt after it, which is not valid, so it gave up.

Chapter 2.3's exercises met the same failure with a different filename and a different message. The message depends on what letter your filename starts with, which makes it confusing to recognise; the cause is always the same.

9. What This Chapter Does Not Cover

-i is the mechanism. Using it safely is a separate discipline:

• proving the edit is right before anything is written;
• keeping a backup that is worth having, and knowing why -i.bak is usually the weaker option;
• using version control as the rollback path;
• choosing the file list so a sweep touches what you intended.

The first three are Chapter 7.2, immediately next. The fourth is Chapter 8.3. Nothing in this chapter should be run on a real tree until you have read 7.2 — every example here was run on throwaway files created for the purpose.

10. Think It Through

1. Your colleague says -i edits the file in place, so it is safe for a file that other names point at. What do you show them?

The inode, before and after — section 3 measured it changing. Then the hard-link case from section 4: two names, one file, and after a -i edit through one name they are two files with different contents and no diagnostic. "In place" is the name of the option, not a description of the operation.

2. Why did a chmod 444 file get edited, and what would have stopped it?

Because sed wrote a new file and renamed it; neither step touches the old file's permissions. Only the directory's write permission matters, which section 7 confirmed by making the directory read-only and getting Permission denied, exit 1. To protect a file from a sweep, make its directory read-only, or keep it out of the file list — the latter being the real answer, and Chapter 8.3's subject.

3. Write a portable one-liner that rewrites a path in a filelist, without using -i.

sed 's|/proj/old/|/proj/new/|g' top.f > top.f.new && mv top.f.new top.f. The && matters: if sed fails, the move does not run and the original survives. Never > top.f — Chapter 2.4 measured that emptying the file with exit status 0.

4. sed -i 's/a/b/' *.sv on your Mac prints an error naming one of your files as a command. What happened?

-i consumed s/a/b/ as its backup suffix, so the first .sv file became the script. The exact message depends on that filename's first character — section 8 got extra characters at the end of n command from a file called n1.txt. The fix is sed -i '' 's/a/b/' *.sv, and the better fix is the detection branch in section 8.

5. You run a sweep over a tree and a downstream block keeps compiling the old content. Where do you look first?

For hard links: find . -links +1 -type f. If a file has more than one name, the sweep edited one name and left the other holding the original, exactly as section 4 showed — and that is invisible in the diff of the files you edited. Symlinks are the other candidate, though on BSD sed those refuse loudly rather than diverging silently.

6. Why does this chapter insist every example ran on throwaway files?

Because -i is the first thing in this track that cannot be undone by closing the terminal. The habit of trying a new sed construct on a copy is not timidity; it is the only way to find out what a command does without the answer being expensive. The next chapter turns that habit into a procedure.

11. Summary

• -i does not edit your file. It writes a new file and renames it over the old name.
• Proved by the inode changing — 85155273 became 85155275 on an ordinary edit.
• The spellings differ and are not interchangeable: GNU -i, BSD/macOS -i ''. On BSD the argument is mandatory.
• A hard link breaks silently. Two names sharing one inode became two files with different contents, exit 0, no message. Check with find . -links +1 -type f.
• A symlink is refused on BSD — in-place editing only works for regular files, exit 1. GNU's documented default instead replaces the link with a regular file unless --follow-symlinks.
• File mode was preserved in these runs (755 and 600 both survived). Ownership was not measured and is not claimed either way.
• A chmod 444 file was edited anyway, exit 0 — because the write goes to the directory. The same file in a 555 directory gave Permission denied, exit 1.
• Forgetting BSD's mandatory argument makes your first filename the script, producing a message that depends on its first letter.
• The portable escape is sed … file > file.new && mv file.new file, never > file.

You now know what the option does to your disk. What you do not yet have is any way of knowing whether the edit is right before it lands — and an edit that is wrong is far more common than an edit that is unportable. Chapter 7.2 — Dry Runs, Backups, and Diff Validation is the procedure: how to see exactly what would change, which backup is worth keeping, and why a commit beforehand beats every backup file.