Anchors and Precise Matching

Part II · Module 4 · Chapter 4.3 · Anchors and Precise Matching

1. The Default Is a Substring Match

A sed pattern does not have to match a whole line, or a whole word, or anything in particular. It has to appear somewhere on the line. That is the entire rule, and almost every over-matching bug in sed comes from forgetting it.

Two characters change it:

   ^pattern      the pattern must start at the beginning of the line
   pattern$      the pattern must end at the end of the line
   ^pattern$     the line must be exactly the pattern, and nothing else

They are called anchors, and they are the only positional tool basic sed gives you. Everything in this chapter is a consequence.

2. /module/ Also Matches endmodule

Start with a module file:

cat -n rtl/top.sv

     1	module top (
     2	  input  logic clk,
     3	  input  logic rst_n,
     4	  output logic [31:0] data_o,
     5	  inout  wire  sda
     6	);
     7	  logic [7:0] count;
     8	  wire        ready;
     9	endmodule

Find the module declaration:

sed -n '/module/p' rtl/top.sv

module top (
endmodule

Two lines. endmodule contains the six letters module, so it matched — correctly, by the rules, and not what anybody asking this question wants.

Anchor it:

sed -n '/^module /p' rtl/top.sv

module top (

One line. Two changes did that: ^ requires the match at the start of the line, and the trailing space requires a word boundary of sorts — module followed by a space, which rules out modulename as well.

This is the whole chapter in miniature. The unanchored pattern was not broken; it was less specific than the question.

3. The One That Breaks RTL

The substring problem gets expensive when the text is a signal name. Here are three names that all contain clk:

printf 'clk\nclk_div\ngclk\n' | sed 's/clk/CLK/'

CLK
CLK_div
gCLK

Three lines changed where you wanted one. clk_div is a different signal; gclk is a gated clock. Both were rewritten, both exit status 0, and in a sweep across two hundred RTL files nothing would have told you.

Anchoring to the whole line fixes it when the name is alone on the line:

printf 'clk\nclk_div\ngclk\n' | sed 's/^clk$/CLK/'

CLK
clk_div
gclk

Exactly one. ^clk$ says "the line is clk and nothing else", so neither the prefix in gclk nor the suffix in clk_div can satisfy it.

4. An Anchor Matches a Position, Not a Character

This is worth stating directly, because it explains a behaviour that otherwise looks like a special case.

^ does not match the first character of the line. It matches the place before it — a position of zero width. Same for $ at the other end. So when you substitute an anchor, there is no character to replace, and the replacement is simply inserted.

Chapter 3.1 used this to prefix lines:

printf 'clk\nrst\n' | sed 's/^/top./'

top.clk
top.rst

Nothing was consumed; top. was inserted at the position ^ names. The same at the other end:

printf 'alu.sv\n' | sed 's/$/ \\/'

alu.sv \

And the most useful version of this in practice — stripping trailing whitespace, which is the one sed command worth knowing by heart. cat -et marks the end of each line with $ so the change is visible. Line 5 of the filelist has three trailing spaces:

cat -et filelists/top.f | sed -n '5p'

./rtl/fifo.sv   $

Now the same line after the substitution:

sed 's/[[:space:]]*$//' filelists/top.f | cat -et | sed -n '5p'

./rtl/fifo.sv$

The spaces are gone. Read the pattern as "any amount of whitespace, positioned at the end of the line" — and note that because * allows zero, this matches on every line, harmlessly replacing nothing with nothing on the ones that were already clean.

Which lines actually had trailing whitespace? A pattern that requires at least one:

sed -n '/[[:space:]][[:space:]]*$/=' filelists/top.f

5

One line, number 5. The doubled class is the portable "one or more" from Chapter 4.2 — [[:space:]]\{1,\}$ says the same thing.

5. Anchoring a Prefix, and a Test That Proved Nothing

The registered example for this chapter strips a leading ./ from filelist entries:

sed 's|^\./||' filelists/top.f

+define+DEBUG
+incdir+/proj/old/rtl

rtl/alu.sv
rtl/fifo.sv   
# a commented-out file
/proj/old/tb/env.sv

#another comment
rtl/top.sv

Three entries lost their ./ and nothing else changed. Note the \. — an unescaped dot would match any character, which Chapter 3.3 measured.

Now, does the ^ actually matter here? The obvious way to find out is to drop it and compare. That test is worth showing because it does not work:

printf './rtl/./sub/alu.sv\n' | sed 's|\./||'

rtl/./sub/alu.sv

printf './rtl/./sub/alu.sv\n' | sed 's|^\./||'

rtl/./sub/alu.sv

Identical. A line with an interior ./ as well as a leading one, and the anchor changed nothing — because without g a substitution replaces only the leftmost match, and the leftmost ./ is the leading one. The anchor was redundant on this input.

Two inputs do discriminate it. Either add g:

printf './rtl/./sub/alu.sv\n' | sed 's|\./||g'

rtl/sub/alu.sv

printf './rtl/./sub/alu.sv\n' | sed 's|^\./||g'

rtl/./sub/alu.sv

Now the difference is plain: unanchored and global strips both, anchored and global strips only the leading one. Or — the realistic case — use a file where some line's ./ is not at the start:

cat -n filelists/dots.f

     1	./rtl/alu.sv
     2	rtl/./fifo.sv

sed 's|\./||' filelists/dots.f

rtl/alu.sv
rtl/fifo.sv

sed 's|^\./||' filelists/dots.f

rtl/alu.sv
rtl/./fifo.sv

Line 2 is the evidence. The unanchored command reached into the middle of a path and removed a ./ that was meaningful; the anchored one left it alone.

That is the third time this module has needed a fixture built on purpose to tell two commands apart, and the first time the obvious attempt failed. The method matters more than the example: when you add an anchor, ask what input would behave differently without it — and if your first test gives the same answer either way, the test is wrong, not the anchor.

6. In the Middle of a Pattern, They Are Just Characters

A detail that saves confusion. ^ is special only at the start of a pattern, and $ only at the end. Elsewhere they are ordinary text, and need no escaping:

printf 'a^b\n' | sed -n '/a^b/p'

a^b

printf 'a$b\n' | sed -n '/a$b/p'

a$b

Both matched their lines. This is why a Tcl variable reference or a shell parameter can sit in the middle of a basic-mode pattern without ceremony, and why a stray $ mid-pattern is not an error — it is a dollar sign.

Two related meanings already covered elsewhere, collected here so the three uses of these characters do not blur:

• ^ first inside brackets negates a set — [^/] is "not a slash". Chapter 4.2 section 4.
• $ after a substitution's closing delimiter is nothing special, because a replacement is not a pattern — s/\.sv$/.svh/ has an anchor in the pattern and a plain dot in the replacement. Chapter 3.3 section 5.

7. Think It Through

1. sed -n '/ready/p' rtl/top.sv is meant to find the ready wire and also prints a line declaring data_ready. Fix it two ways.

Anchor the surroundings you know about. /wire *ready;/ requires the declaration shape — wire, some spaces, then ready; — and tested against a decoy pair it keeps wire ready; and correctly rejects wire data_ready;.

What does not work is anchoring only the end. /ready;$/ matches both lines, because data_ready; also ends in ready;. That is worth running yourself: an anchor constrains the edge you attach it to and says nothing about what precedes it.

Both are approximations. The honest answer is that "the identifier ready" needs a word boundary, which is Chapter 9.2. Anchors get you a long way on structured lines and cannot express word-ness.

2. Why does s/$/ \\/ append rather than replace, when s/x/y/ replaces?

Because $ matches a zero-width position, not a character. There is nothing for the replacement to consume, so the text is inserted at that point. The same reasoning applies to ^. If anchors matched a character, s/^/top./ would delete the first letter of every line.

3. You run sed 's/[[:space:]]*$//' file and the output looks unchanged. Did it work?

Probably — trailing whitespace is invisible, so "looks unchanged" is the expected appearance of success. Check with cat -et before and after, as section 4 did, or count the affected lines first with sed -n '/[[:space:]][[:space:]]*$/='. If that returns nothing, the file had no trailing whitespace and the command genuinely had nothing to do.

4. Is /^$/ the right way to find blank lines?

Only if you mean strictly empty ones. A line containing spaces or a tab is not empty and ^$ will not match it, though it looks blank in any editor. /^[[:space:]]*$/ covers both, and Chapter 4.2 section 8 measured the difference on a file built to show it.

5. A colleague says anchors make patterns safer. When are they not enough, and when are they actively wrong?

Not enough: whenever the target sits inside a line among other text, as any identifier in real code does — anchors can pin the line's edges and nothing in between. Actively wrong: whenever the thing you are matching genuinely can appear anywhere, such as a path fragment inside a longer compile option. Chapter 3.4 section 5 had a published example that matched nothing precisely because a ^ was committing it to a line shape the file did not have. An anchor is a claim about position; add it when you know the position, not as a reflex.

8. Summary

• A sed pattern is unanchored by default and matches a substring anywhere on the line. This is the source of most over-matching.
• /module/ also matches endmodule. /^module / does not.
• s/clk/CLK/ changed clk, clk_div and gclk. s/^clk$/CLK/ changed exactly one.
• ^pattern$ means the line is exactly that and nothing else — the right tool for filelists, one-item-per-line data and report fields.
• Anchors cannot express word boundaries. For an identifier inside a line of code you need \b (GNU) or [[:<:]] (BSD), which is Chapter 9.2 — and which is spelled differently on the two seds.
• An anchor matches a position, not a character, so substituting one inserts. s/^/top./ prefixes; s/$/ \\/ appends.
• s/[[:space:]]*$// strips trailing whitespace on every line, harmlessly on the clean ones. Verify it with cat -et, since the change is invisible.
• ^ and $ are literal in the middle of a pattern and need no escaping.
• When you add an anchor, construct the input that would differ without it. My first attempt here did not discriminate, because without g the leftmost match was the anchored one anyway.

You can now describe text precisely and say where on the line it must sit. The remaining question is the one every example in this module has quietly been using: which lines does the command run on at all? sed -n '/pattern/p' has been an address and a command all along. Chapter 4.4 — Line Numbers, Pattern Addresses, and Ranges makes that explicit, and it closes Module 4 with the range behaviour that surprises almost everybody.