Alternation and Grouping

Part I · Module 11 · Chapter 11.9 · Alternation and Grouping

1. Either-Or

A vertical bar separates alternatives. The pattern matches if any one of them does.

use strict;
use warnings;

my @lines = ("ERROR: x", "FATAL: y", "WARNING: z", "INFO: w");

for my $l (@lines) {
    printf "%-13s /ERROR|FATAL/ : %s\n", $l, ($l =~ /ERROR|FATAL/ ? "match" : "-");
}

ERROR: x      /ERROR|FATAL/ : match
FATAL: y      /ERROR|FATAL/ : match
WARNING: z    /ERROR|FATAL/ : -
INFO: w       /ERROR|FATAL/ : -

Two of four matched, which is the behaviour you would predict. | is how you write "any of these severities", "either spelling of this option", or "the three ways this tool labels a failure".

Note immediately what this is not. A character class picks one character from a set; alternation picks one string from a list. Chapter 11.5 asserted that [setup|hold] does not mean "setup or hold", and this is the chapter where that gets measured — section seven.

2. Alternation Binds Last

Here is the measurement that justifies the chapter's existence.

use strict;
use warnings;

my @subjects = ("ERROR", "FATAL", "FATALX", "xERROR", "xFATAL");

printf "%-9s %-20s %s\n", "SUBJECT", "/^ERROR|FATAL\$/", "/^(?:ERROR|FATAL)\$/";
for my $s (@subjects) {
    printf "%-9s %-20s %s\n", $s,
        ($s =~ /^ERROR|FATAL$/     ? "match" : "-"),
        ($s =~ /^(?:ERROR|FATAL)$/ ? "match" : "-");
}
print "\n";
print "the first reads as (^ERROR) OR (FATAL\$) -- two different anchors\n";

SUBJECT   /^ERROR|FATAL$/      /^(?:ERROR|FATAL)$/
ERROR     match                match
FATAL     match                match
FATALX    -                    -
xERROR    -                    -
xFATAL    match                -

the first reads as (^ERROR) OR (FATAL$) -- two different anchors

Four of the five rows agree. The fifth is the whole point: xFATAL matches the ungrouped pattern.

Almost everyone reads /^ERROR|FATAL$/ as "the whole string is either ERROR or FATAL". Perl reads it as two complete alternatives, because | binds more loosely than everything else in the pattern — looser than the anchors, looser than quantifiers, looser than concatenation. So the pattern is:

• ^ERROR — starts with ERROR, with anything allowed after it, or
• FATAL$ — ends with FATAL, with anything allowed before it.

xFATAL satisfies the second. So would the run was FATAL. And on the other side, ERROR: setup violation satisfies the first — which in this case you probably wanted, but not for the reason you thought.

The fix is a pair of parentheses that bound the alternation, so the anchors apply to the whole thing. (?:...) is the non-capturing form, and it is the right default here because the grouping is purely structural — you are not trying to extract the severity, only to test it.

3. First Match, Not Longest Match

The order of alternatives is not cosmetic.

use strict;
use warnings;

my $s = "setup_check";

my ($a) = $s =~ /^(setup|setup_check)/;
my ($b) = $s =~ /^(setup_check|setup)/;

printf "subject                      : %s\n", $s;
printf "/^(setup|setup_check)/       : [%s]\n", $a;
printf "/^(setup_check|setup)/       : [%s]\n", $b;
print  "\nalternation takes the FIRST alternative that works, not the longest\n";

subject                      : setup_check
/^(setup|setup_check)/       : [setup]
/^(setup_check|setup)/       : [setup_check]

alternation takes the FIRST alternative that works, not the longest

The same subject, two patterns differing only in the order of the alternatives, two different captures.

Perl tries the alternatives left to right and stops at the first one that allows the overall match to succeed. In the first pattern setup works — the pattern is unanchored at the end, so matching only part of the token is fine — and the engine never considers the longer alternative. The captured "check type" is therefore setup for a line that is actually a setup_check.

Some references stop here and advise ordering your alternatives longest-first. That works, but it is a rule you have to remember forever, including in six months when you add a seventh alternative. There is a better fix.

use strict;
use warnings;

my @subjects = ("setup", "setup_check", "hold");

printf "%-13s %-24s %s\n", "SUBJECT", "unanchored group", "anchored group";
for my $s (@subjects) {
    my ($un) = $s =~ /^(setup|setup_check)/;
    my ($an) = $s =~ /^(setup|setup_check)$/;
    printf "%-13s %-24s %s\n", $s,
        (defined $un ? "[$un]" : "(no match)"),
        (defined $an ? "[$an]" : "(no match)");
}

SUBJECT       unanchored group         anchored group
setup         [setup]                  [setup]
setup_check   [setup]                  [setup_check]
hold          (no match)               (no match)

With $ after the group, the alternatives are in the same wrong order and the result is correct anyway.

The mechanism is backtracking, the same process Chapter 11.7 described. The engine still tries setup first, but now the anchor demands the end of the string immediately after, and _check is in the way. So the match fails, the engine backtracks and tries the next alternative, and setup_check succeeds.

That is the general principle and it is worth stating plainly: anchor your pattern and the order of alternatives stops mattering. Ordering longest-first is a workaround for a pattern that was not specific enough about where it ends.

4. Grouping So a Quantifier Has Something To Repeat

Chapter 11.6 measured that a quantifier binds to exactly one item. Parentheses are how you make a longer thing into one item.

use strict;
use warnings;

# Parentheses let a quantifier apply to more than one character.
# ^\w+(?:/\w+)*$  =  a name, then ANY NUMBER of "/name" segments
my @subjects = (
    "u_alu",
    "u_alu/A",
    "u_core/u_alu/A",
    "u_alu/",            # trailing slash with no segment after it
    "/u_alu",            # leading slash
    "u_core//A",         # empty segment
);

printf "%-18s %-12s %s\n", "SUBJECT", "(?:/\\w+)*", "(?:/\\w+)";
for my $s (@subjects) {
    printf "%-18s %-12s %s\n", $s,
        ($s =~ m{^\w+(?:/\w+)*$} ? "accept" : "reject"),
        ($s =~ m{^\w+(?:/\w+)$}  ? "accept" : "reject");
}
print "\n";
print "the * on the GROUP allows any depth; without it, exactly one segment\n";

SUBJECT            (?:/\w+)*    (?:/\w+)
u_alu              accept       reject
u_alu/A            accept       accept
u_core/u_alu/A     accept       reject
u_alu/             reject       reject
/u_alu             reject       reject
u_core//A          reject       reject

the * on the GROUP allows any depth; without it, exactly one segment

^\w+(?:/\w+)*$ is a complete description of a hierarchical name: one identifier, then any number of slash-plus-identifier segments. It accepts a bare name, a one-level path and a two-level path, and that is the * on the group doing work no single-character quantifier could do.

The second column shows the group's contribution as a difference. Without the *, exactly one segment is allowed, so the bare name and the two-level path are both rejected.

The three rejected subjects are worth reading as a set, because they show the pattern is tight rather than merely permissive. u_alu/ is rejected because a / must be followed by at least one \w. /u_alu is rejected because the pattern must start with \w+. u_core//A is rejected for the same reason as the first — the second slash has no identifier after it. A hand-written split "/" check would need explicit code for each of those; here they fall out of the shape.

5. Capturing Changes the Numbering

Parentheses capture by default, and (?:...) is the form that does not. That choice has consequences beyond tidiness.

use strict;
use warnings;

my $s = "WARN: fifo depth 8";

# With a CAPTURING group, the level occupies $1 and the number is $2
my ($lvl, $num) = $s =~ /^(WARN|ERROR): \w+ \w+ (\d+)/;
printf "with ()   : first=[%s] second=[%s]\n", $lvl, $num;

# With a NON-capturing group, the number is the only capture
my ($only) = $s =~ /^(?:WARN|ERROR): \w+ \w+ (\d+)/;
printf "with (?:) : first=[%s]\n", $only;

with ()   : first=[WARN] second=[8]
with (?:) : first=[8]

Both patterns match the same line. The capturing version returns two values, the first being the severity; the non-capturing version returns one, the number.

So (?:...) is not just a micro-optimisation — it changes the shape of what comes back. Use a capturing group when you want the text; use (?:...) when the parentheses are there for structure and you do not.

6. Why That Default Matters

The cost of capturing a group you did not need appears when you edit the pattern later.

use strict;
use warnings;

my $s = "ERROR: u_alu/A 0.42";

# Adding a group at the FRONT shifts every later number
my ($a1, $a2) = $s =~ m{^\w+: (\S+) (\S+)};
printf "two groups        : \$1=[%s] \$2=[%s]\n", $a1, $a2;

my ($b1, $b2, $b3) = $s =~ m{^(\w+): (\S+) (\S+)};
printf "group added first : \$1=[%s] \$2=[%s] \$3=[%s]\n", $b1, $b2, $b3;
print  "\nthe pin moved from \$1 to \$2 without the pin part of the pattern changing\n";

two groups        : $1=[u_alu/A] $2=[0.42]
group added first : $1=[ERROR] $2=[u_alu/A] $3=[0.42]

the pin moved from $1 to $2 without the pin part of the pattern changing

In the first pattern the pin is the first capture. Someone then extends the pattern to also grab the severity, by adding parentheses at the front — a one-character edit to a part of the pattern that has nothing to do with the pin.

The pin is now the second capture. Every line of code downstream that read the first capture is now reading the severity, and since both are strings, nothing complains. A report column headed "pin" fills up with the word ERROR.

Capture numbering is assigned by the position of the opening parenthesis, counting left to right through the pattern, including groups nested inside other groups. That is a perfectly sensible rule and it is also fragile under editing, which is the problem Chapter 11.11 solves by giving groups names instead of numbers.

7. A Set Is Not an Alternation

Chapter 11.5 asserted this; here it is measured.

use strict;
use warnings;

# A classic beginner error: [setup|hold] is a SET OF CHARACTERS
my @subjects = ("setup", "hold", "zzz", "p", "|");

printf "%-9s %-22s %s\n", "SUBJECT", "[setup|hold] (a SET)", "(?:setup|hold)";
for my $s (@subjects) {
    printf "%-9s %-22s %s\n", "[$s]",
        ($s =~ /^[setup|hold]+$/   ? "match" : "-"),
        ($s =~ /^(?:setup|hold)$/  ? "match" : "-");
}

SUBJECT   [setup|hold] (a SET)   (?:setup|hold)
[setup]   match                  match
[hold]    match                  match
[zzz]     -                      -
[p]       match                  -
[|]       match                  -

The two columns agree on the first three subjects, which is exactly why this mistake survives review. On the inputs an author tests — the two values they meant to allow, and one they meant to reject — the broken pattern behaves correctly.

The last two rows expose it. [setup|hold] is a character class containing the nine distinct characters s e t u p | h o l d, so the single letter p matches it, and so does a bare vertical bar. With + it would also accept pupu, dole and plot.

The diagnostic is simple: if you see | inside square brackets, it is almost certainly a bug. A class never contains alternatives, only characters.

8. What To Remember

• | means either, and it chooses between strings, where a character class chooses between characters.
• Alternation has the lowest precedence of anything in a pattern. /^ERROR|FATAL$/ is (^ERROR)|(FATAL$), which is why xFATAL matched.
• Parenthesise every alternation unless you have decided otherwise on purpose.
• Alternation is first-match, not longest-match. /^(setup|setup_check)/ captured setup from setup_check.
• Anchoring beats reordering. With $ after the group, the wrong order gives the right answer, because the engine is forced to backtrack into the longer alternative.
• Parentheses give a quantifier something longer than one character to repeat. ^\w+(?:/\w+)*$ is a hierarchical name of any depth, and it correctly rejects a trailing slash, a leading slash and an empty segment.
• (?:...) groups without capturing, and changes what the match returns.
• Capture numbers come from the opening parenthesis, left to right. Adding a group at the front shifted the pin from $1 to $2 with no error — prefer (?:...) by default and list assignment over $1.
• | inside square brackets is a bug. [setup|hold] matches p.

Chapter 11.10 turns to the capturing half in earnest: how to get values out, why $1 is the fragile way to read them, and the measured trap where a failed match leaves a previous line's capture sitting in $1 for your code to read as though it were current.