Running Your First sed Commands

Part I · Module 1 · Chapter 1.4 · Running Your First sed Commands

1. The Fixture

Everything in this chapter uses one small file — a generated compile filelist, the artifact class Chapter 1.2 identified as sed's natural home. Create it so you can follow along:

+incdir+/proj/rtl_old/include
/proj/rtl_old/alu_core.sv
/proj/rtl_old/regfile.sv
/proj/rtl_old/tb/top_tb.sv

Four lines: an include directory and three source files, all under a path that is about to change. Small enough to check every result by eye, which is the point — you are learning to verify, and you cannot learn that on a file you cannot read.

2. The Shape of the Command

Every command in this chapter has the same three parts:

sed  's/old/new/'  filename
 │         │           │
 │         │           └── the input: a file to read
 │         └── the script: what to do to each line
 └── the program

The script is one instruction, s, which Chapter 1.1 introduced: substitute. The single quotes keep the shell from touching it — section 7 shows what happens without them. The filename is what sed reads.

Here is a first substitution. Rename one source file:

sed 's/alu_core/alu_top/' filelists/top.f

+incdir+/proj/rtl_old/include
/proj/rtl_old/alu_top.sv
/proj/rtl_old/regfile.sv
/proj/rtl_old/tb/top_tb.sv

Four lines in, four lines out. The second line changed; the other three were printed exactly as they arrived, because a line with no match is passed through untouched. That is sed's default and it is why the output is a complete file rather than just the changed part.

3. The Output Went to Your Screen

Look at the file again:

cat filelists/top.f

+incdir+/proj/rtl_old/include
/proj/rtl_old/alu_core.sv
/proj/rtl_old/regfile.sv
/proj/rtl_old/tb/top_tb.sv

Unchanged. alu_core is still alu_core.

This is the behaviour Chapters 1.1 and 1.2 kept promising, and now you have run it yourself. sed read the file, applied your instruction to each line, and wrote the result to standard output — the terminal. It did not write to the file. Making the change permanent is a separate decision with its own risks, and Module 7 is devoted to doing it safely.

While you are learning, this is the feature that makes experimenting free. Any command you try costs you a wrong-looking screenful and nothing else.

4. A Real Change: The Path Prefix

The rename above was a warm-up. The actual job is that /proj/rtl_old/ has become /proj/rtl/.

The pattern contains slashes, and so does the replacement. Chapter 1.2 showed why that matters: with / as the separator you must escape every slash inside, which works but is hard to review. Choosing a separator the text does not contain removes the problem. Any character can follow the s:

sed 's|/proj/rtl_old/|/proj/rtl/|' filelists/top.f

+incdir+/proj/rtl/include
/proj/rtl/alu_core.sv
/proj/rtl/regfile.sv
/proj/rtl/tb/top_tb.sv

All four lines corrected, including the +incdir+ line, because sed matched the path prefix wherever it appeared and had no opinion about what kind of entry each line was.

Note what is not here: no g at the end. Each of these lines contains the prefix once, so the first-match default is sufficient. Chapter 1.2's section 7 walked through what happens on a line holding it twice.

5. Two Ways to Feed sed

Everything so far passed a filename. sed can also read from standard input, which is what makes it work in a pipeline.

cat filelists/top.f | sed 's|/proj/rtl_old/|/proj/rtl/|'

+incdir+/proj/rtl/include
/proj/rtl/alu_core.sv
/proj/rtl/regfile.sv
/proj/rtl/tb/top_tb.sv

Identical output. In the first form sed opened the file itself; in the second, cat read the file and the pipe handed its output to sed, which read from the pipe instead. Same text in, same text out.

For a single file the direct form is better — it is shorter, and it avoids running a second program to do what sed already does. The pipeline form earns its place when the input is not a file sitting on disk: it is the output of another command. Narrow first, then rewrite:

grep -v '^+incdir+' filelists/top.f | sed 's|/proj/rtl_old/|/proj/rtl/|'

/proj/rtl/alu_core.sv
/proj/rtl/regfile.sv
/proj/rtl/tb/top_tb.sv

Three lines, not four. grep -v dropped the include-directory line, so sed never saw it. Each stage does one job — the composition pattern Chapter 1.3 described.

6. The Hard Half: Did It Actually Work?

Now the part that matters more than the syntax.

Run a substitution whose pattern is not in the file — here with alu_kore, a typo:

sed 's/alu_kore/alu_top/' filelists/top.f

+incdir+/proj/rtl_old/include
/proj/rtl_old/alu_core.sv
/proj/rtl_old/regfile.sv
/proj/rtl_old/tb/top_tb.sv

Four lines of output, no error message, and the exit status is 0 — success. Nothing changed, because nothing matched, and sed considers that a perfectly normal day's work: it was asked to substitute where the pattern occurs, and it occurred nowhere.

This is the trap. That output is indistinguishable at a glance from a correct run on a file that needed no changes. Exit status 0 means sed ran; it does not mean your text changed. On a four-line file you can see the difference. On four hundred lines scrolling past, you cannot.

So verify deliberately. The most direct check is to compare the output against the input:

sed 's/alu_kore/alu_top/' filelists/top.f | diff filelists/top.f -

No output at all. diff prints nothing when two inputs are identical, so silence here is the finding: the command changed nothing. The - is the second filename and means read this one from the pipe, so you are comparing the file on disk against what sed produced.

Now the same check on the command that did work:

sed 's|/proj/rtl_old/|/proj/rtl/|' filelists/top.f | diff filelists/top.f -

1,4c1,4
< +incdir+/proj/rtl_old/include
< /proj/rtl_old/alu_core.sv
< /proj/rtl_old/regfile.sv
< /proj/rtl_old/tb/top_tb.sv
---
> +incdir+/proj/rtl/include
> /proj/rtl/alu_core.sv
> /proj/rtl/regfile.sv
> /proj/rtl/tb/top_tb.sv

Four lines changed, < showing what you had and > what you would get. Output from diff means the command did something; you then read it to confirm it did the right something.

A second habit, useful before you substitute at all, is to count what the pattern will hit:

grep -c '/proj/rtl_old/' filelists/top.f

4

Four lines contain the prefix, and the substitution changed four lines. When those two numbers disagree, something in your pattern is not what you thought.

7. Quoting, Briefly

The single quotes around the script are not decoration, and two cases explain why.

Without quotes, the shell splits on spaces. A script containing a space is torn into pieces before sed ever sees it:

sed s/alu_core/alu top/ filelists/top.f

sed reports an error about the substitute command being malformed — it received only the fragment up to the space, and top/ arrived as a separate argument. The exact wording differs between sed versions, so read it as "my script did not arrive intact" rather than memorising the text. Quoting prevents the whole class of problem.

Single quotes keep everything literal; double quotes let the shell substitute first. That difference is a tool, not a trap, once you know which you want:

echo 'TEST = $NAME' | sed 's/\$NAME/apb_smoke/'

TEST = apb_smoke

Here the text really contains the characters $NAME, and single quotes let the pattern say so. When you want a shell variable's value in the command instead, double quotes are the way:

NEW=alu_top; echo "/proj/rtl_old/alu_core.sv" | sed "s/alu_core/$NEW/"

/proj/rtl_old/alu_top.sv

The shell replaced $NEW with alu_top and handed the finished script to sed. Use single quotes by default and reach for double quotes only when a value must expand — and remember the value must not contain your delimiter. Chapter 8.2 covers quoting in full.

8. Debug This One

A colleague says this command "does nothing" and asks you to look. Predict what happens and why before reading on.

sed 's|/proj/rtl_OLD/|/proj/rtl/|' filelists/top.f

What happens.

+incdir+/proj/rtl_old/include
/proj/rtl_old/alu_core.sv
/proj/rtl_old/regfile.sv
/proj/rtl_old/tb/top_tb.sv

The whole file, unchanged, exit status 0 — section 6's silent no-op exactly.

Why. The pattern says rtl_OLD and the file says rtl_old. sed matches text literally and case-sensitively, so the pattern occurs nowhere. There is nothing wrong with the command as a command: the delimiter choice is good, the syntax is valid, it ran successfully. It simply described text that does not exist.

How to find that without guessing. Ask whether the pattern matches anything at all, before worrying about the replacement:

grep -c '/proj/rtl_OLD/' filelists/top.f

0

Zero. The problem is in the pattern, not the replacement, not the delimiter and not the file. That is the diagnostic order worth internalising: does it match? comes before does it replace correctly?

The fix is the command from section 4, with the case as it appears in the file. Run the diff check from section 6 afterwards and you will see four changed lines rather than silence.

9. Summary

• A sed command is three parts: the program, a quoted script, and the input.
• Name a file, or pipe into it. Both read the same text; the pipeline form matters when the input is another command's output.
• Non-matching lines pass through untouched, so the output is a whole file, not just the changed part.
• The result goes to standard output. Your input file is not modified — Module 7 covers making a change permanent.
• Choose a delimiter your text does not contain. For paths, s|...|...| avoids escaping every slash.
• Exit status 0 means sed ran, not that anything changed. A pattern that matches nothing is a normal, successful, silent no-op.
• Verify deliberately. Pipe the output into diff against the input: silence means nothing changed, and a listing shows exactly what would.
• Count the matches first with grep -c; when that number and the number of changed lines disagree, your pattern is not what you thought.
• Quote the script in single quotes by default; use double quotes only when a shell value must expand.
• Diagnose in order: does the pattern match at all, and only then, does it replace correctly?

Everything in this chapter used a literal pattern — plain text, matched exactly. That is also its limit: you cannot yet say "any digit", "the start of a line", or "only lines mentioning an include directory". Those are regular expressions and addresses, and they are what Module 2 onward is built on, starting with the execution cycle that explains how sed applies a script to each line in turn.

That also completes Module 1. You now know what sed is for, where it belongs in a chip flow, when a different tool is the better answer, and how to run one and prove it worked.