Common sed Errors and Debugging

Part III · Module 7 · Chapter 7.4 · Common sed Errors and Debugging

1. Loud Failures Are the Easy Ones

Chapter 7.3 covered the dangerous class: sed succeeds, exit status 0, and the output is quietly wrong. This chapter covers the other kind, and it is worth saying plainly that these are the good failures. A command that refuses to run cannot damage anything, and the message tells you where to look.

What makes them feel unhelpful is that sed describes its own parse, not your intention. When it says bad flag in substitute command: 'o', it is not commenting on your path — it is telling you that it stopped reading your command three characters in, and the next character it found where a flag should be was o.

Read them that way and they become precise.

2. How to Read Any sed Error

Every message has the same shape:

   sed: 1: "s/^module/// SUPPRESSED ...": bad flag in substitute command: '/'
        │   └────────────┬────────────┘   └──────────────┬──────────────┘
        │        what sed was parsing            what went wrong
        └─ which line of the script

The quoted fragment is the most useful part and the most ignored. It shows you how much of your command sed had consumed when it gave up. If the fragment stops in the middle of something you thought was one unit — a path, a label, a filename — that is the bug.

3. The Catalogue

Every message below is real output from the sed these examples were verified against. All exit with status 1 unless noted.

Two rows deserve a note. unused label is the only one in the table that does not stop sed — it is a warning, the script runs, and it exits 0. Chapter 6.3 measured what that costs: the input passed through completely unchanged while reporting success. And No such file or directory does not stop the run either; sed processes the remaining operands and exits 1, which Chapter 2.4 measured producing a plausible but wrong line count.

4. The Three You Will Actually Hit

A slash in your data. The most common error in practice:

sed 's/^module/// SUPPRESSED: module/' rtl/top.sv

sed: 1: "s/^module/// SUPPRESSED ...": bad flag in substitute command: '/'

The third / closed the command. sed read the pattern as ^module, the replacement as empty, and then found / where it expected a flag. Change the delimiter:

sed 's|^module|// SUPPRESSED: module|' rtl/top.sv

// SUPPRESSED: module top (
  input clk
);
endmodule

Chapter 3.3 is the full treatment; this is the error it produces.

A missing delimiter. One / short:

sed 's/old/new' rtl/top.sv

sed: 1: "s/old/new
": unescaped newline inside substitute pattern

The wording is confusing — there is no newline in what you typed. What sed means is that it reached the end of your script while still reading the replacement, and the end of the script is a newline to it. Count delimiters: s, three separators, done.

An address with nothing after it. Usually a p lost while editing:

sed '/module/' rtl/top.sv

sed: 1: "/module/
": command expected

5. The Mistake Whose Visibility Depends on Your Filename

Here is the one worth remembering, because it is genuinely surprising and it explains an entire category of confusion.

Forget the script — give sed only a filename. Chapter 2.3 showed one outcome:

sed filelists/top.f

sed: 1: "filelists/top.f
": invalid command code f

Exit 1. sed took the filename as the script, and f is not a command. Loud, obvious, fixable.

Now make exactly the same mistake with a different file:

sed rtl/top.sv

Nothing. Exit status 0. No error, no output, no indication that anything is wrong.

The reason is that r is a valid sed command — it reads a file and queues its contents for output. So rtl/top.sv parses as r followed by the filename tl/top.sv:

ls tl/top.sv

ls: tl/top.sv: No such file or directory

That file does not exist — and r on a missing file is silently ignored, by design:

printf 'x\n' | sed 'r nosuchfile'

x

While r on a file that does exist appends it:

printf 'x\n' | sed 'r rtl/top.sv'

x
module top (
  input clk
);
endmodule

So sed rtl/top.sv means "read the non-existent file tl/top.sv", which does nothing, and with no file operand sed reads standard input. Pipe something in and you can watch it happen:

printf 'HELLO\n' | sed rtl/top.sv

HELLO

Exit 0. The input came straight through.

6. When the Message Makes No Sense

Two situations where the diagnostic points somewhere unhelpful.

The fragment stops mid-path. If the quoted part of the message ends in the middle of something you consider one unit, sed disagreed about where that unit ended. A delimiter collision (section 4), a label swallowing a line (Chapter 6.3), or -i eating your script (Chapter 7.1) all look like this. Read the fragment, not the explanation.

The error names something you never typed. That usually means the shell changed your command before sed saw it. An unquoted $, a ! in double quotes, a glob that expanded — sed is reporting what arrived, which is not what you wrote. The quick test is to echo the command:

   echo sed 's/a/b/' file

If what you see is not what you meant, the problem is the shell and Chapter 8.2 is the chapter. Until then: single-quote every sed script, which Chapter 3.1 recommended for this reason.

A third habit worth having: simplify until it runs. Strip the command back to the plainest thing that should work — sed -n '/word/p' file with no operators at all — and add pieces back one at a time. The piece that breaks it is the bug, and this finds it faster than re-reading the original.

7. Think It Through

1. Why are this chapter's failures easier than Chapter 7.3's?

Because they stop. A command that refuses to run changes nothing and hands you a message naming the place it gave up. An over-match runs to completion, exits 0, and the only evidence is in a diff nobody has read yet. Loud failure is cheap; silent success is expensive.

2. sed 's/a/b/' *.f on a Mac reports extra characters at the end of n command. There is no n in your command. Where did it come from?

From a filename. Something consumed your script — almost certainly a -i without its mandatory BSD argument (Chapter 7.1 section 8) — so the first operand became the script, and that filename started with n. The message names a letter from your file list, which is the section 6 pattern: read the quoted fragment and you will recognise it as a path.

3. A pipeline stage does nothing and exits 0. What do you check first?

Whether the sed has a script at all. Section 5 showed sed rtl/top.sv passing input straight through with exit 0, because r is a command. Then check for a swallowed label (Chapter 6.3) and a GNU-only construct failing silently (\b, \|, \U, 0,/re/). All four produce "it ran and did nothing", and all four exit 0.

4. What does unescaped newline inside substitute pattern mean when your command is on one line?

That sed reached the end of your script while still looking for a delimiter — the end of the script reads as a newline to it. You are one delimiter short. Count: an s command has exactly three.

5. You get undefined label '; s/^/ /'. What is the actual problem?

A b or t whose target ran to the end of the line and swallowed the next command, so the label name is ; s/^/ / and no such label exists. Put the branch and the following command on separate -e fragments or separate lines of a -f file — Chapter 6.3 section 4. Note the sibling message unused label is the same mistake in the other direction and does not stop sed, which is far worse.

6. Build a one-line check you could put in a flow script to catch a sed that did nothing.

Compare the output with the input: sed … file | diff -q file - >/dev/null && echo "WARNING: sed changed nothing". diff exits 0 when they are identical, so the && fires exactly when the edit was a no-op — Chapter 7.2's inverted exit status, used as a guard. That one line catches the silent script-forgotten case, the swallowed label, and every GNU-only construct that fails quietly.

8. Summary

• These are the good failures. sed refuses to run, changes nothing, and names where it stopped.
• Read the quoted fragment in the message — it shows how much of your command sed consumed before giving up. If it stops mid-path, that is the bug.
• Section 3 is the catalogue: fourteen real messages mapped to causes and fixes.
• bad flag in substitute command: '/' means a slash in your data ended the s early. Change the delimiter.
• unescaped newline inside substitute pattern means you are one delimiter short, even on a single line.
• unused label exits 0 and is the only entry in the catalogue that lets a broken script run — Chapter 6.3 measured the damage.
• Forgetting the script fails loudly or silently depending on your filename. sed filelists/top.f gave invalid command code f, exit 1; sed rtl/top.sv gave nothing, exit 0, because r is the read-file command and r on a missing file is silently ignored.
• A message naming something you never typed usually means the shell got there first. Single-quote your scripts; Chapter 8.2 is the full treatment.
• Simplify until it runs, then add pieces back one at a time.
• sed … file | diff -q file - exits 0 when the edit did nothing — the cheapest guard against every silent failure in this track.

That closes Module 7 and the core of safe editing. You can now apply an edit without destroying anything, prove it before it lands, diagnose it when it is too wide, and read sed's diagnostics when it refuses. What is still missing is everything around the sed command — getting a script safely through the shell, composing it with grep and find, choosing a file list that does not include surprises, and knowing when to stop. Module 8 — sed in Shell Automation is that, and it ends with the chapter this track has been pointing at since Chapter 1.3: deciding when the job belongs in Tcl or Python instead.