Finding Every Match With /g

Part I · Module 11 · Chapter 11.12 · Finding Every Match With /g

1. Without /g, One Match

use strict;
use warnings;

my $line = "u_alu/A 0.42  u_fifo/B 0.44";

my @one = $line =~ /(-?\d+\.\d+)/;
my @all = $line =~ /(-?\d+\.\d+)/g;

printf "without /g : %d item(s)  %s\n", scalar @one, join(" | ", @one);
printf "with    /g : %d item(s)  %s\n", scalar @all, join(" | ", @all);

without /g : 1 item(s)  0.42
with    /g : 2 item(s)  0.42 | 0.44

The line holds two numbers. Without the modifier the match returns one item, because the match operator stops at the first success. With it, both.

A plain match is not "search the line"; it is "find the first place this pattern fits". Everything in this chapter follows from removing that stop.

2. List Context: Every Match at Once

use strict;
use warnings;

my $line = "u_alu/A 0.42  u_fifo/B 0.44  u_regf/C -0.05";

# LIST context: every match at once
my @vals = $line =~ /(-?\d+\.\d+)/g;
printf "list context  : %d match(es)  %s\n", scalar @vals, join(" | ", @vals);

# SCALAR context in a while loop: one at a time
my $n = 0;
while ($line =~ /(-?\d+\.\d+)/g) {
    $n++;
    printf "  iteration %d : %s\n", $n, $1;
}
printf "while loop    : %d iteration(s)\n", $n;

list context  : 3 match(es)  0.42 | 0.44 | -0.05
  iteration 1 : 0.42
  iteration 2 : 0.44
  iteration 3 : -0.05
while loop    : 3 iteration(s)

Three values, and both forms found all three.

The list form is what you want whenever the whole collection is the answer: every slack value on the line, every instance name in a netlist line, every option in a command. It is one statement and it reads as a noun — my @vals = $line =~ /.../g; is "the values in this line".

The while form is what you want when each match needs work done on it, especially work that might next, last, or push onto several different places at once. It also matters when the subject is large: the list form builds the entire result in memory before you see any of it, while the loop hands you one at a time.

Note that $1 is valid inside the loop body. Each iteration is a fresh successful match, so this is one of the places where reading a numbered capture is safe — the match succeeded, or the loop would have ended. Chapter 11.10's warning is about reading $1 after a match whose success you have not established; here the while condition establishes it.

3. Where Perl Keeps Its Place

The while form needs to remember where it stopped, and it stores that on the string.

use strict;
use warnings;

my $s = "aXbXc";

while ($s =~ /X/g) {
    printf "matched at pos %d\n", pos($s);
}
printf "after the loop, pos is %s\n", (defined pos($s) ? pos($s) : "undef");

matched at pos 2
matched at pos 4
after the loop, pos is undef

pos($s) reports the offset just past the last match — 2 and then 4 for the two X characters at indices 1 and 3.

The last line is the important one. When the match finally fails and ends the loop, Perl resets pos to undef. That is deliberate and useful: it means the same string can be scanned again from the start by another loop later, without you clearing anything. If the bookmark persisted after failure, a second loop over the same variable would find nothing.

The consequence to watch for is the other way round. If you last out of a /g loop early, the bookmark is left where it was, so a later loop over that same variable resumes from the middle rather than starting over. Chapter 11.20 is about using that behaviour on purpose; here it is just something to know, because an early last plus a second scan is a confusing bug.

4. Two Groups per Match, Straight Into a Hash

Chapter 11.10 measured that /g with captures returns one flat list. That shape has a use.

use strict;
use warnings;

my $line = "pin=u_alu/A slack=-0.12 unit=ns";

# Two groups per match, so /g in list context yields a flat key,value,... list
my %field = $line =~ /(\w+)=(\S+)/g;

for my $k (sort keys %field) {
    printf "  %-6s => %s\n", $k, $field{$k};
}
printf "fields: %d\n", scalar keys %field;

  pin    => u_alu/A
  slack  => -0.12
  unit   => ns
fields: 3

Three matches, two groups each, so six items in key, value, key, value order — which is exactly what a hash assignment consumes. One line turns a key=value string into a hash.

This is the idiom for parsing an attribute string, and it is worth recognising because the alternative is a loop with a $h{$1} = $2 inside it. Both are fine; this one is shorter and says "this line is a set of fields".

The usual caution applies: the result depends on the pattern having exactly two groups. Add a third capturing group and the pairing silently shifts, which is Chapter 11.9's renumbering hazard in a new costume. If the pattern needs internal grouping, use (?:...) for it.

5. A Pattern That Can Match Nothing

Folklore says a /g loop on a pattern that can match the empty string hangs. It does not.

use strict;
use warnings;

# A pattern that can match the EMPTY string -- does /g loop forever?
my $s = "ab";
my $n = 0;
while ($s =~ /x*/g) {
    $n++;
    printf "  iteration %d at pos %s matched [%s]\n",
        $n, (defined pos($s) ? pos($s) : "undef"), $&;
    last if $n > 10;
}
printf "iterations: %d\n", $n;

  iteration 1 at pos 0 matched []
  iteration 2 at pos 1 matched []
  iteration 3 at pos 2 matched []
iterations: 3

Three iterations on a two-character string, each matching the empty string, and then it stopped. The last if $n > 10 guard never fired — I put it there expecting to need it.

Perl protects you here: after a zero-length match it advances the bookmark by one character rather than matching at the same place again. So the loop terminates, and pos walks 0, 1, 2 to the end of the string.

6. A Worked Case: Both Modes on One Block

Here is a slice of tool output held in one string, read twice — once for the collection, once for the pairing.

use strict;
use warnings;

# A block of tool output held in one string, as a slurp would give it.
my $log = join "", map { "$_\n" } (
    "Startpoint: u_alu/A",
    "Endpoint:   u_regf/D",
    "  slack -0.12",
    "Startpoint: u_fifo/B",
    "Endpoint:   u_mem/Q",
    "  slack 0.44",
    "Startpoint: u_ctl/C",
    "Endpoint:   u_regf/E",
    "  slack -0.05",
);

# List context: every slack value in one expression
my @slack = $log =~ /^\s+slack\s+(-?[\d.]+)$/mg;
printf "paths found : %d\n", scalar @slack;
printf "values      : %s\n", join(", ", @slack);

my $violations = grep { $_ < 0 } @slack;
printf "violations  : %d\n", $violations;

# While loop: pair each startpoint with the slack that follows it
my @pairs;
while ($log =~ /^Startpoint: (\S+)\n.*?^\s+slack\s+(-?[\d.]+)$/msg) {
    push @pairs, "$1=$2";
}
printf "paired      : %d  (%s)\n", scalar @pairs, join(" ", @pairs);

paths found : 3
values      : -0.12, 0.44, -0.05
violations  : 2
paired      : 3  (u_alu/A=-0.12 u_fifo/B=0.44 u_ctl/C=-0.05)

Three paths, two of them violations, and then the same three paired with their start points.

The two halves show each mode doing what it is for. The list form answers an aggregate question — "how many violations" needs every value and nothing else, so one expression plus a grep is the whole computation. The while form answers a relational question: each slack belongs to a particular start point, and that association exists only because the loop handles one match at a time and can read both captures together.

Note the modifiers on the second pattern. /m so ^ anchors to each line, /s so .*? can cross the lines between a start point and its slack, /g to repeat — and the quantifier is lazy, because a greedy one would run from the first start point to the last slack and pair the wrong values. That is Chapter 11.7's lesson in a pattern that needs it; Chapter 11.16 covers /m and /s properly.

7. What To Remember

• Without /g a match finds the first hit and stops. /g removes the stop.
• In list context /g returns every match at once — the right form when the collection is the answer.
• In a while condition /g returns one match per iteration, which is the right form when each match needs work, or when the subject is large enough that building the whole list first would be wasteful.
• $1 is safe inside a /g while loop, because the condition established the match.
• pos($str) is the bookmark, reporting the offset just past the last match.
• The failing match that ends the loop resets pos to undef, so the string can be re-scanned from the start. But an early last leaves the bookmark mid-string.
• With captures, /g in list context gives one flat list — two groups per match assigns straight to a hash. A third capturing group silently breaks the pairing; use (?:...).
• A /g loop on a zero-width-capable pattern does not hang. Perl advances past a zero-length match, so it runs once per character — safe, and a sign your pattern has no mandatory part.

Chapter 11.13 turns from finding to changing. s/// is the substitution operator: it finds a pattern, replaces what it matched, edits the variable in place, and hands back a count of how many replacements it made.