Capture Groups and Backreferences

Part II · Module 3 · Chapter 3.4 · Capture Groups and Backreferences

1. The Problem With Retyping

You have three AXI signals to rename from axi_ to m_axi_:

axi_awvalid
axi_awready
axi_wvalid

You could write three substitutions, one per signal. That works and does not scale — a real bus has thirty signals, and the thirty-first one added next week will be missed.

What you want to say is: match axi_ followed by whatever the rest of the name is, and put the rest back unchanged behind a new prefix. The "whatever it is" and the "put it back" are the two halves of this chapter.

2. Mark a Piece, Put It Back

Two pieces of syntax:

• Parentheses around part of a pattern make it a group — sed remembers what that part matched.
• \1 in the replacement means "whatever group 1 matched".

s / axi_\(aw[a-z]*\) / m_axi_\1 /
         └────┬────┘        └┬┘
          group 1       put group 1 back

In words: match axi_, then capture a run of lowercase letters starting with aw, and rebuild it as m_axi_ followed by exactly what was captured.

A first run, on a bare list of port names:

cat -n rtl/ports.f

     1	    axi_awvalid,
     2	    axi_awready,
     3	    axi_wvalid,

sed -E 's/axi_(aw[a-z]+)/m_axi_\1/' rtl/ports.f

    m_axi_awvalid,
    m_axi_awready,
    axi_wvalid,

One command, both aw signals renamed, each keeping its own suffix. awvalid stayed awvalid and awready stayed awready without either being mentioned in the command.

Line 3 is a useful control: axi_wvalid does not start with aw, so the pattern did not match and the line is untouched. The group did its job and the pattern did its job — a rename that is specific rather than a rename that catches everything nearby.

3. How This Differs From &

Chapter 3.3's & inserts the whole match. A group inserts a chosen part of it. Compare them on the same input:

printf 'clk\n' | sed 's/clk/&_i, &_o/'

clk_i, clk_o

Here & was the entire matched text, clk. That is all & can ever be. When you need to keep the tail of a match while replacing its head — which is what a prefix rename is — & cannot express it, because it would bring the old prefix back with it.

Groups are the general tool; & is the convenient special case for "the whole thing".

4. Two Spellings: BRE and ERE

sed has two regular-expression dialects, and they disagree about parentheses. This is the single most common source of confusion in this chapter.

• Basic (the default): parentheses are ordinary characters. To make a group you escape them — \( and \).
• Extended (with -E): parentheses are grouping operators. You write them plainly — ( and ).

The same job, both ways. Here is an STA corner script:

cat -n sta/corners.tcl

     1	set corner ss_0p72v_125c
     2	read_sdc constraints/ss_0p72v_125c.sdc
     3	report_timing -corner ss_0p72v_125c > reports/ss_0p72v_125c.rpt

The corner is being retargeted from 125 °C to −40 °C, keeping the voltage. In basic form, with escaped parentheses:

sed 's/\(ss_0p72v\)_125c/\1_m40c/' sta/corners.tcl

set corner ss_0p72v_m40c
read_sdc constraints/ss_0p72v_m40c.sdc
report_timing -corner ss_0p72v_m40c > reports/ss_0p72v_125c.rpt

And in extended form, with -E:

sed -E 's/(ss_0p72v)_125c/\1_m40c/g' sta/corners.tcl

set corner ss_0p72v_m40c
read_sdc constraints/ss_0p72v_m40c.sdc
report_timing -corner ss_0p72v_m40c > reports/ss_0p72v_m40c.rpt

Note that the backreference is \1 in both dialects. Only the parentheses change.

Get it wrong and basic mode tells you so, loudly:

sed 's/(ss_0p72v)_125c/\1_m40c/' sta/corners.tcl

sed: 1: "s/(ss_0p72v)_125c/\1_m4 ...": \1 not defined in the RE

Exit status 1. The unescaped parentheses were treated as literal text, so there was no group 1 for \1 to refer to — and sed said so instead of guessing. After Chapter 3.3, that is a welcome change.

5. A Published Example That Does Nothing

This chapter's registered example is worth running exactly as written, because the result is instructive:

cat -n rtl/bus.sv

     1	module bus (
     2	    input  logic axi_awvalid,
     3	    output logic axi_awready,
     4	    input  logic axi_wvalid
     5	);
     6	endmodule

sed -E 's/^([[:space:]]*)axi_(aw[a-z]+)/\1m_axi_\2/' rtl/bus.sv

module bus (
    input  logic axi_awvalid,
    output logic axi_awready,
    input  logic axi_wvalid
);
endmodule

Nothing changed. Exit status 0. No diagnostic.

The command is not malformed — it has two valid groups and two valid backreferences. It simply does not match this file, and the reason is the ^ anchor. Read the pattern as a sequence of claims about the line:

^              the line starts here
([[:space:]]*) then: any amount of whitespace  ← group 1
axi_           then: immediately axi_

That says axi_ must come immediately after the indentation. On line 2 the text after the indentation is input logic axi_awvalid, — so between the whitespace and axi_ there are thirteen characters the pattern does not allow for. No match.

The pattern describes a bare port list, not a declaration. And on a bare port list it works perfectly:

sed -E 's/^([[:space:]]*)axi_(aw[a-z]+)/\1m_axi_\2/' rtl/ports.f

    m_axi_awvalid,
    m_axi_awready,
    axi_wvalid,

Same command, different file, and the indentation is preserved exactly — that is what group 1 is for.

To make it work on the declaration, drop the anchor and let the pattern match wherever axi_ appears:

sed -E 's/axi_(aw[a-z]+)/m_axi_\1/' rtl/bus.sv

module bus (
    input  logic m_axi_awvalid,
    output logic m_axi_awready,
    input  logic axi_wvalid
);
endmodule

Both aw signals renamed, axi_wvalid correctly untouched, and no group needed for the indentation because nothing before axi_ was consumed.

6. Several Groups, and Nesting

You can have more than one group. They are numbered in the order their opening parenthesis appears, left to right — that single rule covers nesting too.

Two groups, swapped:

printf 'alu 1.842\n' | sed -E 's/^([a-z]+) ([0-9.]+)$/\2 \1/'

1.842 alu

Group 1 took the name, group 2 took the number, and the replacement put them back in the other order. Turning a report into a different column order is this, repeated.

Now nesting. The outer parenthesis opens first, so it is group 1:

printf 'ss_0p72v_125c\n' | sed -E 's/^((ss)_[0-9a-z]+)_([0-9]+c)$/[\1][\2][\3]/'

[ss_0p72v][ss][125c]

Count the opening parentheses from the left: the first is the outer one (ss_0p72v), the second is the inner one (ss), the third is the temperature (125c). Nothing about being "inside" or "outside" matters — only the position of the (.

One more property, easy to miss and genuinely useful: a group that matches nothing at all still exists. The whitespace group in section 5 matched zero characters on an unindented line, and \1 expanded to an empty string rather than failing:

printf 'axi_awvalid\n' | sed -E 's/^([[:space:]]*)axi_/\1m_axi_/'

m_axi_awvalid

No indentation, no error, correct output. That is why the capture-the-indentation idiom is safe to use on files whose lines are not all indented the same way.

7. Groups Do Not Change the Occurrence Rules

Everything from Chapter 3.2 still applies — a substitution with groups replaces one match per line unless you say otherwise. Look again at line 3 of the corner script, which mentions the corner twice:

sed 's/\(ss_0p72v\)_125c/\1_m40c/' sta/corners.tcl

set corner ss_0p72v_m40c
read_sdc constraints/ss_0p72v_m40c.sdc
report_timing -corner ss_0p72v_m40c > reports/ss_0p72v_125c.rpt

The -corner argument was retargeted; the report filename at the end of the same line was not. The script now writes −40 °C results into a file named for 125 °C, which is the kind of mistake that survives until somebody reads the wrong report in a review.

Add g:

sed 's/\(ss_0p72v\)_125c/\1_m40c/g' sta/corners.tcl

set corner ss_0p72v_m40c
read_sdc constraints/ss_0p72v_m40c.sdc
report_timing -corner ss_0p72v_m40c > reports/ss_0p72v_m40c.rpt

Both. Groups are about what you keep; the flag is about how many places you change it. You need both right, and they fail independently.

8. Think It Through

1. Write one command that renames every axi_ signal to m_axi_ while keeping its suffix, on a bare port list.

sed -E 's/axi_([a-z]+)/m_axi_\1/' rtl/ports.f. The group captures the suffix and \1 restores it. Add g if a line can mention two signals. Note this is wider than section 2's version, which required aw — whether you want the wider or narrower form is a judgement about the file, and Chapter 9.2 is where identifier renaming gets the care it deserves, including why this command is not safe to run across real RTL without word boundaries.

2. Why is \1 not defined in the RE a better failure than most of Chapter 3.3's?

Because it stops. sed found a backreference with no group to match it, refused to run, and exited 1 — you cannot commit the result of a command that never produced one. Contrast the unescaped & and the GNU \U, which both ran happily and wrote wrong output with exit status 0. A loud error is cheap; a quiet one costs a review cycle or a tapeout.

3. In s/\(a\)\(b\(c\)\)/\3\1/, what are the three groups?

Count opening parentheses left to right: group 1 is a, group 2 is bc, group 3 is c. The replacement produces c followed by a. Nesting does not create a separate numbering space — there is one counter, and it increments at every (.

4. A colleague's anchored rename "does nothing" and they are sure the pattern is right. What is your first question?

Not about the pattern — about what sits between the anchor and the target. ^ commits you to describing the whole line from character one. Ask them to run the command with -n and a p flag: empty output proves the pattern does not match. Then sed -n '2p' file | cat -et shows exactly what the start of the line really contains — -e marks the line end with $ and -t shows tabs as ^I, so a line indented with a tab is immediately distinguishable from one indented with spaces. (On the BSD cat that macOS ships, -A does not exist — it exits 1 with illegal option; -et is the portable spelling.)

5. You have ss_0p72v_125c and ss_0p88v_125c and want to change only the temperature on both, keeping each voltage.

Capture the voltage and reuse it: sed -E 's/(ss_[0-9p]+v)_125c/\1_m40c/g' file. One command for both corners, and for the third one added next quarter. Writing it as two literal substitutions would also work today and would need editing every time the library grows — which is the argument for groups in one sentence.

9. Summary

• Parentheses mark part of a match; \1 puts that part back. This is how you change a prefix while keeping every suffix.
• & is the whole match; a group is a chosen piece of it. & cannot express "keep the tail, replace the head".
• Basic mode needs \( and \); -E needs ( and ). The backreference is \1 in both. Mixing them up gives \1 not defined in the RE and exit 1 — a loud, safe failure.
• Groups are numbered by the position of their opening parenthesis, left to right, nesting included.
• A group that matches the empty string still defines its backreference, which is what makes the capture-the-indentation idiom safe.
• An anchored pattern is a claim about everything before your target. The registered example for this chapter changes nothing on a real port declaration, exits 0, and says nothing — because ^[[:space:]]*axi_ describes a bare port list, not input logic axi_awvalid. Dropping the anchor fixes it.
• Check a pattern before you trust it: sed -n 's/…/…/p' on the real file. Empty output means it does not match.
• Groups do not change the occurrence rules. Without g, the corner script kept a 125 °C filename on a −40 °C run.

That closes Module 3. You can now express most real substitutions: the right matches, on the right lines, keeping the parts that should not change. What you cannot yet do is describe the patterns precisely — everything here has been plain text plus a dot, a star and a bracket used without explanation. Module 4 — Regular Expressions and Addressing supplies that language, starting with the two dialects this chapter had to keep apologising for, and then the addresses that decide which lines a command runs on at all.