Pattern Space and the Execution Cycle

Part I · Module 2 · Chapter 2.2 · Pattern Space and the Execution Cycle

1. One Buffer, One Line at a Time

sed has a working buffer. It is called the pattern space, and it holds exactly one line of text at a time.

When sed reads a line, it does not hand that line to your commands. It copies the line into the pattern space — without its trailing newline — and then runs your commands against the buffer. At the end, it prints the buffer and empties it. The input line was only a seed.

That distinction sounds pedantic until you have two commands. Then it is the whole story: the second command does not see your input, it sees the pattern space as the first command left it.

2. The Cycle

Here is what sed does, once per input line, from the first line to the last.

Three consequences fall straight out of the diagram, and they explain most of what Module 1 showed you without justifying:

• Step 4 is why you get a whole file back. A line that matched nothing still reaches step 4 and still gets printed. Substitution does not filter; it edits in passing.
• Step 3 is why command order matters. Two commands are not two independent passes over your input. They are two edits to one buffer.
• Step 5 is why sed cannot count. The buffer is cleared, so there is nothing to accumulate in. Chapter 2.1 made the same point from the memory side.

3. Auto-Print, and Turning It Off

Step 4 happens whether or not you asked for it. The easiest way to see it is a script that does nothing at all:

sed '' bus.sv

logic axi_awvalid;
logic axi_awready;
logic axi_wvalid;

An empty script, and the file comes out intact — that output is step 4 and nothing else. Add -n and the same empty script produces nothing:

sed -n '' bus.sv

-n does exactly one thing: it suppresses step 4. The cycle still runs, the commands still execute, but sed stops printing the buffer for you. From then on, anything that appears in the output has to be asked for explicitly with the p command.

So these two produce the same result for opposite reasons:

sed -n 'p' bus.sv

logic axi_awvalid;
logic axi_awready;
logic axi_wvalid;

In sed '' the output came from auto-print. In sed -n 'p' auto-print is off and p printed the buffer by hand. And if you ask for both at once, you get both:

sed 'p' bus.sv

logic axi_awvalid;
logic axi_awvalid;
logic axi_awready;
logic axi_awready;
logic axi_wvalid;
logic axi_wvalid;

Every line twice: once from p at step 3, once from auto-print at step 4. This is the reason -n and p are almost always written together — sed -n '/pattern/p' is a filter, while sed '/pattern/p' is a filter that duplicates the lines it found:

sed '/wvalid/p' bus.sv

logic axi_awvalid;
logic axi_awvalid;
logic axi_awready;
logic axi_wvalid;
logic axi_wvalid;

Both matching lines are doubled and the non-matching one is not. If you have ever seen duplicated lines in sed output and assumed something had gone wrong with your pattern, this was it.

4. Two Commands, One Buffer

The registry's own example for this chapter is a two-step rename, and it is worth running because the result is correct and most people's mental model does not predict it.

cat -n bus.sv

     1	logic axi_awvalid;
     2	logic axi_awready;
     3	logic axi_wvalid;

Now a script with two substitutions. The first adds an m_ prefix to the AXI signals; the second renames one specific signal — note that it refers to the signal by its new name:

sed -e 's/axi_/m_axi_/' -e 's/m_axi_awvalid/m_axi_awvalid_q/' bus.sv

logic m_axi_awvalid_q;
logic m_axi_awready;
logic m_axi_wvalid;

Both edits landed on line 1. Trace the cycle for that line:

1. Step 1 — pattern space becomes logic axi_awvalid;
2. Step 2 — s/axi_/m_axi_/ runs. The buffer is now logic m_axi_awvalid;
3. Step 3 — s/m_axi_awvalid/m_axi_awvalid_q/ runs against that. m_axi_awvalid is present, so the buffer becomes logic m_axi_awvalid_q;
4. Step 4 — the buffer is printed.

The second command matched a string that did not exist in the input file. It existed in the buffer, which is the only thing the second command can see.

Reverse the two -e fragments and the script quietly does half the job:

sed -e 's/m_axi_awvalid/m_axi_awvalid_q/' -e 's/axi_/m_axi_/' bus.sv

logic m_axi_awvalid;
logic m_axi_awready;
logic m_axi_wvalid;

No _q anywhere. Now the rename runs first, against a buffer that still says axi_awvalid, finds no m_axi_awvalid, and does nothing. The prefix is then added, and the output looks plausible enough to commit. Nothing failed, nothing warned, and the exit status was 0.

5. Three Lines, Three Different Wrong Answers

The previous section was the benign case. Here is the one that costs an afternoon.

A two-stage clock rename, of the kind that shows up when a block is re-integrated: clk becomes core_clk, and core_clk becomes gated_clk.

cat -n clocks.sv

     1	assign clk     = ref_clk;
     2	assign core_clk = clk;
     3	assign div_clk  = core_clk;

sed -e 's/clk/core_clk/' -e 's/core_clk/gated_clk/' clocks.sv

assign gated_clk     = ref_clk;
assign core_gated_clk = clk;
assign div_gated_clk  = core_clk;

Every line is wrong, and each one is wrong differently. Two facts explain all three; both are properties of s that Module 3 covers in full, stated here only as far as this trace needs them:

• s without the g flag replaces only the leftmost match on the line.
• s is not a word-aware rename. clk matches wherever those three characters occur, including inside core_clk.

Line 1 — double-applied. assign clk = ref_clk;. The first command replaces the leftmost clk, giving assign core_clk = ref_clk;. The second command now finds core_clk — the string the first command just created — and renames it. Result assign gated_clk = ref_clk;. A signal intended to be renamed once was renamed twice.

Line 2 — a match inside a longer name. assign core_clk = clk;. The first command looks for clk and the leftmost occurrence is inside core_clk, at index 5:

c o r e _ c l k   = clk;
0 1 2 3 4 5 6 7
          ^^^^^ leftmost "clk"

So the buffer becomes assign core_core_clk = clk;. The second command looks for core_clk in core_core_clk; the leftmost occurrence starts at index 5, not 0, so it replaces that and leaves the core_ in front. Result assign core_gated_clk = clk; — a signal name that appears nowhere in the design.

Line 3 — right-looking, wrong target. assign div_clk = core_clk;. The first command hits the clk inside div_clk, giving div_core_clk. The second then rewrites core_clk inside that, producing div_gated_clk. Meanwhile the genuine core_clk on the right-hand side of the assignment is untouched, because s already used up its one substitution earlier in the line. Result assign div_gated_clk = core_clk;.

Changing the order does not rescue it — it just moves the damage:

sed -e 's/core_clk/gated_clk/' -e 's/clk/core_clk/' clocks.sv

assign core_clk     = ref_clk;
assign gated_core_clk = clk;
assign div_core_clk  = gated_clk;

6. Addresses See the Buffer Too

Everything above was about substitutions. The same rule applies to the part of a command that decides whether it runs.

A plausible log filter — downgrade one known-benign error, then drop whatever errors remain:

printf 'UVM_ERROR arb\n' | sed -e 's/UVM_ERROR/UVM_WARNING/' -e '/UVM_ERROR/d'

UVM_WARNING arb

The line was not deleted. By the time /UVM_ERROR/d was reached, the buffer said UVM_WARNING arb — the substitution had already removed the very text the address was looking for. The address is matched against the current pattern space, not against the input line, and the delete never fires.

Reverse the order and it does:

printf 'UVM_ERROR arb\n' | sed -e '/UVM_ERROR/d' -e 's/UVM_ERROR/UVM_WARNING/'

Nothing at all. d deletes the pattern space and ends the cycle immediately — step 4 never happens for this line, and neither does the substitution after it. That is d's second effect and the one people forget: it is not only a delete, it is a jump to the next line.

A clean demonstration, with auto-print off so every printed line is accounted for:

printf 'a\nb\nc\n' | sed -n -e '/b/p' -e '/b/d' -e 'p'

a
b
c

Each line appears exactly once. For a and c, the first two commands do not match and the trailing p prints them. For b, the first p prints it, then d ends the cycle — so the trailing p never runs, and b is printed once rather than twice. If d had merely emptied the buffer and continued, b would have appeared twice.

The same applies to s. This pair looks like it should produce CHANGED, and does not:

printf 'keep\ndrop\n' | sed -e '/drop/d' -e 's/drop/CHANGED/'

keep

The matching line is gone before the substitution is reached. Swap them and the substitution runs first, after which the address no longer matches — so the line survives, renamed:

printf 'keep\ndrop\n' | sed -e 's/drop/CHANGED/' -e '/drop/d'

keep
CHANGED

Two orderings, two completely different results, no error either way. This is why a sed script is read as a sequence of edits to one buffer and not as a set of rules.

7. What the Cycle Does Not Cover

The cycle in Figure 1 is the POSIX one; it behaves the same on the BSD sed used for these transcripts and on GNU sed, so nothing in this chapter is a platform quirk.

It is also the default cycle, and sed has deliberate exits from it:

• -n removes step 4, as section 3 showed.
• d cuts the cycle short at step 3.
• The hold space is a second buffer that step 5 does not clear, which is how sed can carry something from one line to the next. Commands that move text between the two buffers, and those that pull an extra input line into the pattern space, are Module 6's subject. They do not change the cycle; they add steps inside it.

Until then, the model in Figure 1 is complete enough to predict every script in Part I and most scripts you will meet in a real flow.

8. Think It Through

1. Without running it, what does this print, and why?

printf 'one\n' | sed -e 's/one/two/' -e 's/two/three/'

three. The first command puts two in the buffer; the second sees two and replaces it. This is section 4's benign case and section 5's hazard in miniature — whether the compounding is what you wanted is the only difference between the two.

2. A colleague's filter is supposed to delete DEBUG lines and prefix the rest. It deletes nothing. The script is sed -e 's/^/ /' -e '/^DEBUG/d'. What is wrong?

The order. The first command indents every line by two spaces, so by the time the address /^DEBUG/ is tested, no line begins with DEBUG any more — it begins with two spaces. The anchor ^ is matched against the buffer, which the first command has already changed. Putting the d first fixes it, and so does loosening the anchor, but the first is the honest fix.

3. Why does sed -n '/UVM_ERROR/p' sim.log print each error once, while sed '/UVM_ERROR/p' sim.log prints matching lines twice and non-matching lines once?

Because p and auto-print are two separate prints. With -n, step 4 is off and only p produces output. Without it, a matching line is printed by p at step 3 and again by auto-print at step 4, while a non-matching line is printed only by auto-print. The doubling is not a pattern bug.

4. You need to know how many lines a filter removed. Can the cycle tell you?

Not by itself — step 5 clears the buffer, so there is nowhere to keep a count. The practical answers are outside sed: compare wc -l before and after, or use grep -c. Chapter 1.3 made this call explicitly, and it is the same boundary here seen from the inside.

9. Summary

• Commands act on the pattern space, not on your input line. The input line only seeds the buffer at step 1.
• The cycle is: read, run every command in order, print, clear, repeat — once per input line.
• Each command sees what the previous one left. A later command can match text that no line of the file contains.
• Addresses are matched against the buffer too, so a substitution can stop a later address from firing — silently, with exit status 0.
• d ends the cycle, not just the line: commands after it never run for that line.
• Auto-print at step 4 is why a non-matching line still appears. -n removes it; p without -n therefore duplicates.
• If command B's pattern can match what command A produces, the two are not independent. Trace the cycle; do not read the script as a list of intentions.

The cycle is now the mental model for everything that follows. What is still missing is the vocabulary around it — the options that modify it, and how to take apart an unfamiliar sed invocation into options, script, and files so you can tell which part is doing what. Chapter 2.3 — sed Syntax, Commands, and Options does that, and finishes the job -n and -e started here.