qr// : Compiled, Reusable Patterns

Part I · Module 11 · Chapter 11.18 · qr// : Compiled, Reusable Patterns

1. A Pattern With a Name

use strict;
use warnings;

my $pat = qr{^(\w+)/(\w+)$};

printf "ref        : %s\n", ref($pat);
printf "stringified: %s\n", "$pat";

for my $s ("u_alu/A", "u_alu", "u_fifo/B") {
    my @c = $s =~ $pat;
    printf "  %-9s -> %s\n", $s, (@c ? join("/", @c) : "(no match)");
}

ref        : Regexp
stringified: (?^:^(\w+)/(\w+)$)
  u_alu/A   -> u_alu/A
  u_alu     -> (no match)
  u_fifo/B  -> u_fifo/B

ref reports Regexp, so this is a real object, and it binds with =~ exactly like a literal pattern. Captures work; everything works.

The stringification is worth looking at: (?^:^(\w+)/(\w+)$). Perl wrapped the pattern in a group that records which modifiers it was compiled with. The ^ inside (?^: means "default modifiers" — and that wrapper is the mechanism behind the next section.

Being able to name a pattern is the first benefit and the most mundane. my $pin_name = qr{...}; puts the format rule in one place with a name that says what it describes, and the match site then reads if ($line =~ $pin_name), which needs no explanation.

2. A Stored Pattern Carries Its Own Modifiers

use strict;
use warnings;

my $ci = qr/error/i;
my $cs = qr/error/;

printf "qr/error/i stringified : %s\n", "$ci";
printf "qr/error/  stringified : %s\n", "$cs";

printf "'ERROR' =~ \$ci : %s\n", ("ERROR" =~ $ci ? "match" : "no match");
printf "'ERROR' =~ \$cs : %s\n", ("ERROR" =~ $cs ? "match" : "no match");

# The flag travels when the qr is interpolated into a LARGER pattern
printf "'x ERROR y' =~ /x \$ci y/ : %s   <- /i survived interpolation\n",
    ("x ERROR y" =~ /x $ci y/ ? "match" : "no match");
printf "'x ERROR y' =~ /x \$cs y/ : %s\n",
    ("x ERROR y" =~ /x $cs y/ ? "match" : "no match");

qr/error/i stringified : (?^i:error)
qr/error/  stringified : (?^:error)
'ERROR' =~ $ci : match
'ERROR' =~ $cs : no match
'x ERROR y' =~ /x $ci y/ : match   <- /i survived interpolation
'x ERROR y' =~ /x $cs y/ : no match

The two stringifications differ by one character — (?^i:error) against (?^:error) — and that character is the whole guarantee.

The last two lines are the test that matters. Each stored pattern was interpolated into a larger pattern that itself has no modifiers, and the case-insensitive one still matched ERROR. The flag travelled with the pattern.

This is why composition is safe. If modifiers were applied at the outer pattern's level, building a big pattern out of parts would be a trap: a part that needed /i would silently lose it, or an outer /x would start eating whitespace inside a part that depended on it. The (?^...) wrapper prevents both. Contrast this with interpolating a plain string, which carries nothing — a string holding error interpolated into a pattern is just those five characters, and any case-insensitivity has to be reapplied by whoever uses it.

3. Building a Pattern Out of Parts

Once patterns compose safely, a complicated format can be described in layers.

use strict;
use warnings;

# Build a pattern from named parts
my $name  = qr{\w+};
my $path  = qr{$name (?: / $name )*}x;
my $num   = qr{-? \d+ (?: \. \d+ )?}x;

my $row = qr{^ ($path) \s+ ($num) \s+ (\w+) $}x;

for my $s ("u_core/u_alu/A -0.12 ns", "u_alu 0.42 ps", "bad row here") {
    if (my ($p, $v, $u) = $s =~ $row) {
        printf "  %-26s pin=%-16s val=%-6s unit=%s\n", $s, $p, $v, $u;
    }
    else {
        printf "  %-26s (no match)\n", $s;
    }
}

  u_core/u_alu/A -0.12 ns    pin=u_core/u_alu/A   val=-0.12  unit=ns
  u_alu 0.42 ps              pin=u_alu            val=0.42   unit=ps
  bad row here               (no match)

Four patterns, each defined in terms of the ones above it. $name is an identifier; $path is a name plus any number of slash-separated segments, which is Chapter 11.9's grouped quantifier; $num is a signed decimal, which is Chapter 11.6's optional group; and $row assembles the three into a line.

The payoff is that each layer can be read and reviewed on its own. "Is a path a name followed by slash-name segments?" is a question you can answer by looking at one line. Asked of the equivalent single pattern, the same question requires counting parentheses.

Two details worth noticing. The parts use /x individually and the composed pattern uses /x too — and because of section two, those choices are independent, so a part written without /x would keep its literal spaces inside an /x outer pattern. And the capture groups are placed in $row, not in the parts: $path uses (?:...) internally so the only numbered groups are the three the row cares about. If $path had used a capturing group, every capture number in $row would shift — Chapter 11.9's hazard, which composition makes easier to trip over because the groups are now out of sight.

4. A Hash of Formats

This is the use the canonical description singles out, and it is the one that earns qr// its place.

use strict;
use warnings;

# One script, several input formats
my %format = (
    slack  => qr{^ (?<pin>\S+) \s+ (?<val>-?[\d.]+) \s+ ns $}x,
    kv     => qr{^ (?<key>\w+) = (?<val>\S+) $}x,
    banner => qr{^ -{3,} $}x,
);

my @lines = ("u_alu/A -0.12 ns", "mode=setup", "-----", "???");

for my $l (@lines) {
    my $hit = "unrecognised";
    for my $kind (sort keys %format) {
        if ($l =~ $format{$kind}) { $hit = $kind; last }
    }
    printf "  %-18s -> %s\n", $l, $hit;
}

  u_alu/A -0.12 ns   -> slack
  mode=setup         -> kv
  -----              -> banner
  ???                -> unrecognised

Three named formats, four lines, and each line classified — including the one that matches nothing, which is reported rather than silently skipped.

The shape of that code is the point. Adding a fourth format means adding one entry to the hash; the loop does not change. The format names are data, so they can be printed in an error message — "line 42 unrecognised" is much more useful when the script can also say which formats it tried. And because each pattern uses named captures from Chapter 11.11, the code that handles a classified line reads $+{pin} regardless of which format matched, so long as the formats agree on field names.

5. The Speed Claim, Measured

The usual argument for qr// is that it compiles the pattern once, so it saves work in a loop. I believed that until I measured it.

use strict;
use warnings;
use Time::HiRes qw(time);

my @lines = map { "u_alu/bit$_ 0.42 ns" } 1 .. 100000;
my $pat   = qr{^(\S+)\s+([\d.]+)\s+ns$};

# THE HYPOTHESIS UNDER TEST (and it turns out to be wrong):
# "interpolating a string costs a recompile each iteration, so qr// wins".
my $str = '^(\S+)\s+([\d.]+)\s+ns$';

my $t0 = time; my $c1 = 0;
for my $l (@lines) { $c1++ if $l =~ $pat }
my $t1 = time; my $c2 = 0;
for my $l (@lines) { $c2++ if $l =~ /$str/ }
my $t2 = time;

printf "lines        : %d\n", scalar @lines;
printf "qr// matched : %d\n", $c1;
printf "string matched: %d\n", $c2;
printf "both agree   : %s\n", ($c1 == $c2 ? "yes" : "no");

lines        : 100000
qr// matched : 100000
string matched: 100000
both agree   : yes

The block prints only the counts, because a wall-clock number is not a reproducible output. The comment inside that script states the hypothesis I started with — that interpolating a string costs a recompile each iteration, so qr// wins — and labels it as wrong, because the numbers below say it is.

Timing four strategies over the same 100,000 lines, three passes each, gave this:

The plain interpolated string was fastest, in every pass. Perl compiles an interpolated constant pattern once and caches the compiled form, so there is no repeated compilation for qr// to save; going through a Regexp object adds a small indirection instead.

The one strategy that is clearly bad is rebuilding a qr// inside the loop, at roughly 1.7 times the cost of the cheapest option. That is a real mistake and worth avoiding — but it is a mistake you only make because you are using qr//, so it is not an argument for it.

6. What To Remember

• qr// makes a pattern a value of class Regexp, usable with =~ like any pattern.
• It stringifies as (?^flags:...), which records the modifiers it was built with.
• A stored pattern carries its own modifiers into a larger pattern. Measured: a qr/error/i interpolated into an unmodified outer pattern still matched ERROR. A plain string carries nothing.
• That is what makes composition safe — build $name, then $path, then $row, and review each layer on its own.
• Put capture groups in the outer pattern and (?:...) in the parts, or composition hides Chapter 11.9's renumbering from you.
• A hash of named formats lets one loop classify several input shapes and report the ones it does not recognise. Adding a format is one hash entry.
• Order matters when formats can overlap — use an array of pairs, most specific first, rather than sort keys.
• It is not faster. Over 100,000 lines a constant interpolated string beat qr// in every pass; only rebuilding a qr// inside the loop was clearly slow. Use qr// for structure.

Chapter 11.19 adds the constructs that let a pattern check its surroundings without consuming them: lookahead and lookbehind. They are how you accept a number only where it is followed by the right unit, or a pin name only where it is not followed by a suffix.