Capture Groups and $1 $2 $3

Part I · Module 11 · Chapter 11.10 · Capture Groups and $1 $2 $3

1. Parentheses Keep What They Matched

Chapter 11.9 used parentheses for grouping. The same parentheses also remember.

use strict;
use warnings;

my $line = "u_alu/A 0.42 ns";

# Parentheses keep the part they matched
my ($pin, $val, $unit) = $line =~ m{^(\S+)\s+(\S+)\s+(\S+)$};

printf "pin  : %s\n", $pin;
printf "val  : %s\n", $val;
printf "unit : %s\n", $unit;

pin  : u_alu/A
val  : 0.42
unit : ns

One statement, three named variables. The pattern simultaneously validated the line — three whitespace-separated fields, nothing else, anchored at both ends — and extracted the three values. That combination is the thing patterns are genuinely good at, and it is why this idiom appears in almost every Perl script in the field.

This is the list-context return Chapter 11.2 measured, now being used for its intended purpose. The parentheses on the left are essential: my ($pin, ...) puts the match in list context. Without them you get 1.

2. The Numbered Variables

The same captures are also available as $1, $2 and so on.

use strict;
use warnings;

my $line = "u_alu/A 0.42 ns";

if ($line =~ m{^(\S+)\s+(\S+)\s+(\S+)$}) {
    printf "\$1 : %s\n", $1;
    printf "\$2 : %s\n", $2;
    printf "\$3 : %s\n", $3;
    printf "\$& : %s   <- the whole match\n", $&;
}

$1 : u_alu/A
$2 : 0.42
$3 : ns
$& : u_alu/A 0.42 ns   <- the whole match

$1 through $3 hold the three captures, and $& holds everything the pattern matched — here the whole line, because the pattern is anchored at both ends.

Note where these are read: inside the if. That placement is not stylistic, and sections four and five are about why.

3. How the Numbers Are Assigned

Numbering follows the opening parenthesis from left to right, and nesting does not get its own scheme.

use strict;
use warnings;

my $s = "u_alu/A 0.42 ns";

# Numbering follows the OPENING parenthesis, left to right, nesting included
my @c = $s =~ m{^((\w+)/(\w+))\s+(\S+)\s+(\w+)$};

for my $i (0 .. $#c) {
    printf "  \$%d = %s\n", $i + 1, $c[$i];
}

  $1 = u_alu/A
  $2 = u_alu
  $3 = A
  $4 = 0.42
  $5 = ns

Five groups, five values, and the outer group is $1 while the two groups inside it are $2 and $3.

The rule is mechanical: count opening parentheses from the left. An outer group always gets a lower number than anything inside it, and a nested group does not interrupt the numbering of what follows — $4 is the value field, which comes after the whole nested structure.

This is also why $1 can overlap $2: u_alu/A contains u_alu. Captures are not a partition of the match, they are five independent recordings of what five different parts of the pattern consumed. Getting both the whole pin and its two halves out of one match, as here, is a legitimate and common use.

4. Nothing Clears a Capture Variable

Here is the measurement that makes this chapter Critical.

use strict;
use warnings;

my @lines = ("slack=-0.12", "no numbers here", "slack=0.44");

# WRONG: reading $1 without checking whether the match succeeded
for my $l (@lines) {
    $l =~ /slack=(\S+)/;
    printf "  %-18s \$1 = %s\n", $l, (defined $1 ? $1 : "undef");
}
print "\nthe middle line did not match, yet \$1 still holds -0.12\n";

  slack=-0.12        $1 = -0.12
  no numbers here    $1 = -0.12
  slack=0.44         $1 = 0.44

the middle line did not match, yet $1 still holds -0.12

The middle line contains no slack= field at all. The pattern did not match it. And $1 reported -0.12 — the value from the line above.

A failed match leaves the capture variables exactly as the last successful match left them. There is no reset. So the loop above produces three rows of output from two matching lines, and the extra row carries a real number, from the right file, attached to the wrong line.

Think about what that looks like downstream. Every value in the column is plausible. Every value actually appeared in the input. The total is wrong, the worst path may be attributed to the wrong pin, and no test that checks "are the values in range" will notice. If the non-matching line happens to be a header or a blank, you get a duplicate of the row above it.

5. The Two Guards That Work

The first guard is to read captures only inside the success branch.

use strict;
use warnings;

my @lines = ("slack=-0.12", "no numbers here", "slack=0.44");

# Also correct: read $1 only INSIDE the success branch
for my $l (@lines) {
    if ($l =~ /slack=(\S+)/) {
        printf "  %-18s -> %s\n", $l, $1;
    }
    else {
        printf "  %-18s -> skipped\n", $l;
    }
}

  slack=-0.12        -> -0.12
  no numbers here    -> skipped
  slack=0.44         -> 0.44

The second, and the one to prefer, is a list assignment whose result you test.

use strict;
use warnings;

my @lines = ("slack=-0.12", "no numbers here", "slack=0.44");

# RIGHT: a list assignment, and test what came back
for my $l (@lines) {
    if (my ($v) = $l =~ /slack=(\S+)/) {
        printf "  %-18s -> %s\n", $l, $v;
    }
    else {
        printf "  %-18s -> (no slack field)\n", $l;
    }
}

  slack=-0.12        -> -0.12
  no numbers here    -> (no slack field)
  slack=0.44         -> 0.44

Both report the two real values and handle the third line explicitly. The list-assignment form is better for two reasons: the captured value gets a meaningful name at the point of extraction, and the value is a lexical that cannot be stale, because it is created fresh each time round the loop.

There is a subtlety in that guard which is worth measuring rather than asserting, because it looks like it should be fragile.

use strict;
use warnings;

# "count=" matches, but the capture is the EMPTY STRING (false).
my $line = "count=";

# A list assignment in boolean context yields the NUMBER OF ELEMENTS
# on the right-hand side -- not the truth of the captured value.
if (my ($v) = $line =~ /count=(\d*)/) {
    printf "list-assign guard : matched, capture=[%s]\n", $v;
}
else {
    print  "list-assign guard : reported NO match\n";
}

# A scalar assignment tests the VALUE, so an empty capture looks like failure
if (my $v = ($line =~ /count=(\d*)/)[0]) {
    printf "scalar-value guard: matched, capture=[%s]\n", $v;
}
else {
    print  "scalar-value guard: reported NO match  <- WRONG, it did match\n";
}

my $n = () = $line =~ /count=(\d*)/;
printf "\nelements returned by the match : %d\n", $n;

list-assign guard : matched, capture=[]
scalar-value guard: reported NO match  <- WRONG, it did match

elements returned by the match : 1

The subject count= does match the pattern, and the capture is the empty string — a legitimate result meaning "the field is present and blank".

The list-assignment guard got this right. A list assignment used as a boolean yields the number of elements on its right-hand side, which is 1 here, as the last line confirms. It does not look at whether the captured value is true. The guard that tested the value itself got it wrong, reporting no match for a line that matched.

That distinction is the reason if (my ($v) = ...) is the recommended idiom and not merely a stylistic preference: it separates "did the match succeed" from "is the captured value true", which are different questions that a scalar test conflates.

6. A Captured Zero Is a Real Value

The same conflation has a more common form.

use strict;
use warnings;

# A capture can legitimately be "0" or "", which are FALSE.
my @lines = ("count=0", "count=5", "count=");

for my $l (@lines) {
    my ($n) = $l =~ /count=(\d*)/;

    printf "  %-10s capture=%-5s %-9s %s\n", $l, "[$n]",
        (defined $n ? "defined" : "undef"),
        ($n ? "truthy" : "FALSE");
}
print "\ntest DEFINED, not truth -- a captured 0 is a real value\n";

  count=0    capture=[0]   defined   FALSE
  count=5    capture=[5]   defined   truthy
  count=     capture=[]    defined   FALSE

test DEFINED, not truth -- a captured 0 is a real value

All three captures are defined. Two of the three are false.

So if ($n) { ... } after a capture is wrong whenever zero or the empty string is a value your data can legitimately contain — and in this field it constantly is. A count of zero violations, a slack of exactly 0.00, an empty optional column: all three are real answers, and all three are false.

Test defined when you mean "did I get a value". This is Chapter 2's truth table doing real work: Perl has only a handful of false values, and two of them are things a report will hand you.

7. A Worked Case: the Row That Appears Twice

Section four showed the stale capture on three tidy lines. Here is what it does to a report with the furniture a real tool emits — a banner, a blank line, and separator rules.

use strict;
use warnings;

# A timing report as a tool emits it: a banner, a blank, a separator,
# and three real data rows.
my @report = (
    "Timing Report",
    "",
    "----------------------------",
    "u_alu/A    -0.12",
    "u_fifo/B    0.44",
    "----------------------------",
    "u_regf/C   -0.05",
);

# --- the unguarded version: read $1 after every attempt
my (@rows_bad, @rows_good);
for my $l (@report) {
    $l =~ /^(\S+)\s+(-?\d+\.\d+)/;
    push @rows_bad, [$1, $2] if defined $1;
}

# --- the guarded version
for my $l (@report) {
    if (my ($pin, $slack) = $l =~ /^(\S+)\s+(-?\d+\.\d+)/) {
        push @rows_good, [$pin, $slack];
    }
}

printf "input lines        : %d\n", scalar @report;
printf "real data rows     : 3\n\n";

printf "unguarded rows : %d\n", scalar @rows_bad;
printf "  %s\n", join("  ", map { "$_->[0]=$_->[1]" } @rows_bad);
printf "guarded rows   : %d\n", scalar @rows_good;
printf "  %s\n", join("  ", map { "$_->[0]=$_->[1]" } @rows_good);

my $sum_bad  = 0; $sum_bad  += $_->[1] for @rows_bad;
my $sum_good = 0; $sum_good += $_->[1] for @rows_good;
printf "\nsum of slack, unguarded : %.2f\n", $sum_bad;
printf "sum of slack, guarded   : %.2f\n", $sum_good;

input lines        : 7
real data rows     : 3

unguarded rows : 4
  u_alu/A=-0.12  u_fifo/B=0.44  u_fifo/B=0.44  u_regf/C=-0.05
guarded rows   : 3
  u_alu/A=-0.12  u_fifo/B=0.44  u_regf/C=-0.05

sum of slack, unguarded : 0.71
sum of slack, guarded   : 0.27

The report has three data rows. The unguarded parser produced four, and u_fifo/B appears twice.

The duplicate comes from the separator line that follows it. That line does not match, so $1 and $2 still hold u_fifo/B and 0.44, and the defined $1 test — which looks like a careful guard — passes, because $1 is indeed defined. It is defined from the previous line.

Notice which lines did not produce a phantom row. The banner, the blank and the first separator all precede any successful match, so $1 was still undefined and defined $1 correctly rejected them. That is why this bug is so good at hiding: the obvious test cases, where junk lines come first, behave perfectly. It only misfires on a non-matching line that comes after a match — which in a real report is every separator, every page break, and every per-group subtotal.

The consequence is in the totals. The guarded sum is 0.27, which is the real one. The unguarded sum is 0.71, having counted 0.44 twice. Neither number looks obviously wrong on its own.

8. What /g Returns With Captures

One shape to know before Chapter 11.12 covers /g properly.

use strict;
use warnings;

my $line = "u_alu/A u_fifo/B u_regf/C";

# A /g match in list context returns every capture from every match
my @pins = $line =~ m{(\w+)/(\w+)}g;
printf "flat list : %d item(s)\n", scalar @pins;
printf "            %s\n", join(" | ", @pins);

# Counting matches rather than captures
my $count = () = $line =~ m{\w+/\w+}g;
printf "matches   : %d\n", $count;

flat list : 6 item(s)
            u_alu | A | u_fifo | B | u_regf | C
matches   : 3

Three matches of a two-group pattern returned six items, in one flat list. There is no nesting and no grouping by match — you get every capture of every match, in order.

That is convenient when the groups come in pairs you want to feed straight into a hash, and a trap when you expected one item per match. The arithmetic to remember: the list length is the number of matches times the number of groups, so if you need the match count, count matches rather than dividing. The my $count = () = ... idiom on the last line does that — it puts the match in list context, assigns the list to an empty list, and then reads the number of elements in scalar context.

9. What To Remember

• Parentheses capture what they matched, so one pattern validates and extracts at once.
• Numbering follows the opening parenthesis, left to right, nesting included — an outer group gets a lower number than the groups inside it, and captures may overlap.
• $& is the whole match.
• Nothing clears $1. After a failed match it still holds the last successful match's capture — so a loop reading $1 unguarded reports the previous line's value, with no warning.
• Read captures only where success is established: inside the if, or from a list assignment you tested.
• if (my ($v) = $line =~ /.../) tests the number of elements returned, not the truth of $v. That is why it is correct even when the capture is legitimately empty — measured: it reported a match where a value test reported none.
• A captured 0 or "" is defined and false. Test defined, not truth.
• /g in list context returns every capture of every match as one flat list — 3 matches of a 2-group pattern gave 6 items.
• my $n = () = $line =~ /.../g counts matches.

Chapter 11.11 removes the numbering problem altogether. Naming a group means reading $+{pin} instead of counting parentheses to work out which number the pin ended up as — and on a pattern with five fields, that is the difference between a line a colleague can read and one they have to decode.