When Not to Use a Regex

Part I · Module 11 · Chapter 11.22 · When Not to Use a Regex

1. Choosing by the Shape of the Input

The decision is not about taste. It follows from what the text actually is.

Only one of those five boxes is a pattern. The rest of this chapter measures each of the other four against the pattern-based version of the same job.

2. A Fixed Word: index

Chapter 11.1 opened the module with this and it is worth closing with it.

use strict;
use warnings;

my @lines = ("ERROR: setup", "clean run", "ERROR: hold");

# A fixed word: index says yes/no AND where
my $hits = grep { index($_, "ERROR") >= 0 } @lines;
printf "index   found : %d\n", $hits;

my $re = grep { /ERROR/ } @lines;
printf "pattern found : %d\n", $re;
printf "same answer   : %s\n", ($hits == $re ? "yes" : "no");

# and the bug a pattern invites
my $needle = "a.b";
my @subjects = ("a.b", "axb");
printf "\nsearching for the literal %s\n", $needle;
printf "  index   : %d of 2\n", scalar grep { index($_, $needle) >= 0 } @subjects;
printf "  /\$needle/: %d of 2   <- the dot matched 'x'\n", scalar grep { /$needle/ } @subjects;

index   found : 2
pattern found : 2
same answer   : yes

searching for the literal a.b
  index   : 1 of 2
  /$needle/: 2 of 2   <- the dot matched 'x'

On the easy case both tools agree. On the second case the pattern is wrong: searching for the literal a.b it matched axb too, because the needle came from a variable and its dot was read as syntax — Chapter 11.4's measurement, in its natural habitat.

index has no syntax to misread. For a fixed string, that is the whole argument, and it holds even when the string is a constant you typed: index tells a reader "this is literal text" without them having to check for metacharacters.

3. Delimited Fields: split

use strict;
use warnings;

my $row = "u_alu/A,setup,-0.12,ns";

# split states the structure: four comma-separated fields
my @f = split /,/, $row;
printf "split  : %d field(s)  %s\n", scalar @f, join(" | ", @f);

# the pattern version has to name every field and is longer and more fragile
if (my @g = $row =~ /^([^,]*),([^,]*),([^,]*),([^,]*)$/) {
    printf "pattern: %d field(s)  %s\n", scalar @g, join(" | ", @g);
}

# and split handles a row with a different field COUNT; the pattern does not
my $short = "u_alu/A,setup";
printf "\nshort row with split  : %d field(s)\n", scalar(my @h = split /,/, $short);
printf "short row with pattern: %d field(s)\n",
    scalar(my @i = $short =~ /^([^,]*),([^,]*),([^,]*),([^,]*)$/);

split  : 4 field(s)  u_alu/A | setup | -0.12 | ns
pattern: 4 field(s)  u_alu/A | setup | -0.12 | ns

short row with split  : 2 field(s)
short row with pattern: 0 field(s)

On the well-formed row both give four fields. The difference is everything else about them.

split /,/, $row says "this row is comma-separated" and nothing more. The pattern says the same thing in forty characters, repeats ([^,]*) four times, and hard-codes the field count — so adding a fifth column means editing the pattern, while split needs no change at all.

The short row shows the trade honestly. split returned 2; the pattern returned 0. If you require exactly four fields, the pattern's refusal is a feature — it validated and rejected. If you want to handle rows of varying width, or to report which field is missing, split plus a count check (@f == 4 or warn ...) gives you a better error message than a match that simply failed.

The rule: split to divide, then validate the pieces. That separation is easier to debug than one pattern doing both jobs, because when it fails you can see which half failed.

4. Fixed-Width Columns: substr

This is where the pattern approach does not merely lose on style — it loses on correctness.

use strict;
use warnings;

# A fixed-width report: columns are at known offsets, and fields may be BLANK
my @rows = (
    "u_alu/A   -0.12  ns",
    "u_fifo/B         ns",      # the value column is empty
);

for my $r (@rows) {
    my $pin  = substr($r,  0, 10);
    my $val  = substr($r, 10,  7);
    my $unit = substr($r, 17);
    $_ =~ s/^\s+|\s+$//g for ($pin, $val, $unit);
    printf "substr : pin=[%-9s] val=[%-5s] unit=[%s]\n", $pin, $val, $unit;
}

print "\n";
# \s+ cannot describe an EMPTY column: the fields shift left
for my $r (@rows) {
    my @f = split ' ', $r;
    printf "split  : %d field(s)  %s\n", scalar @f, join(" | ", @f);
}

substr : pin=[u_alu/A  ] val=[-0.12] unit=[ns]
substr : pin=[u_fifo/B ] val=[     ] unit=[ns]

split  : 3 field(s)  u_alu/A | -0.12 | ns
split  : 2 field(s)  u_fifo/B | ns

The second row has an empty value column — a real thing in timing reports, where an unconstrained path has no slack to print.

substr read it correctly: three fields, the middle one blank. Because it addresses columns by position, an empty column is simply a run of spaces at a known offset.

The whitespace split reported two fields, and ns has moved into the slot where the value belongs. Downstream, that row now says the slack is ns. Nothing errored.

This is also the counter-argument to Chapter 11.5's advice about \s+. That advice was right for its case — a report whose columns are always populated and whose alignment may change. For a report with blank columns the two pieces of advice conflict, and position wins.

5. When the Target Is a Rule

The one box in the diagram that is a pattern.

Everything above has a fixed word, a fixed delimiter, or a fixed offset. A pattern earns its place when the thing you want is described by a rule: a signed decimal of unknown length, an identifier followed by optional slash-segments, a line that starts with a severity keyword. Chapter 11.1 measured this at the start of the module — index plus substr cannot express "stop at the end of the number" without you writing a number parser by hand, and \d+(?:\.\d+)? is that parser.

So the test to apply is: can I name the exact text, the exact delimiter, or the exact offset? If yes, use the plain tool. If the answer is "it varies, but it follows a rule", that is what patterns are for.

6. Quoting: the Escalation

The last case is different in kind. It is not that a pattern is clumsier; it is that no pattern will do.

use strict;
use warnings;

# A quoted field containing the delimiter. This is where patterns lose.
my $row = 'u_alu/A,"setup, hold",-0.12,ns';

my @naive = split /,/, $row;
printf "naive split  : %d field(s)\n", scalar @naive;
printf "               %s\n", join(" | ", @naive);

# A pattern that tries to respect quotes
my @clever = $row =~ /("[^"]*"|[^,]+)/g;
printf "quote-aware  : %d field(s)\n", scalar @clever;
printf "               %s\n", join(" | ", @clever);

printf "\nexpected     : 4 fields, the second being: setup, hold\n";

naive split  : 5 field(s)
               u_alu/A | "setup |  hold" | -0.12 | ns
quote-aware  : 4 field(s)
               u_alu/A | "setup, hold" | -0.12 | ns

expected     : 4 fields, the second being: setup, hold

The naive split gave five fields, having cut the quoted field in half at its internal comma. A quote-aware pattern — alternation preferring a quoted run over a bare run, which is Chapter 11.9's construct — gave the correct four.

So far the pattern won. Now change the data in the way real data changes.

use strict;
use warnings;

# The quote-aware pattern works until the data escapes a quote.
my @rows = (
    'a,"b, c",d',          # a quoted comma
    'a,"b "" c",d',        # a DOUBLED quote inside a quoted field
);

for my $row (@rows) {
    my @f = $row =~ /("[^"]*"|[^,]+)/g;
    printf "%-16s -> %d field(s)  %s\n", $row, scalar @f, join(" | ", @f);
}
print "\nthe second row should be 3 fields with the middle one: b \" c\n";

a,"b, c",d       -> 3 field(s)  a | "b, c" | d
a,"b "" c",d     -> 4 field(s)  a | "b " | " c" | d

the second row should be 3 fields with the middle one: b " c

The first row is still right: three fields. The second row — which uses the standard CSV convention of doubling a quote to mean a literal quote — gave four fields where three are correct, splitting the middle field into "b " and " c".

You can patch the pattern for that. And then the data will contain a newline inside a quoted field, and you will patch it again. Each patch buys exactly one more case, and the pattern grows until nobody can review it — which is precisely the state Chapter 11.17's /x was invented to postpone and cannot prevent.

The diagnostic, stated so you can recognise it in your own work: if you have patched the same pattern twice for new input shapes, stop patching. Two patches means the input has structure your pattern is not modelling, and the third case is already on its way.

7. What To Remember

• index for a fixed word. It has no syntax to misread — measured: a pattern built from the needle a.b also matched axb.
• split for delimited data, then validate the pieces. It states the structure in one phrase, does not hard-code the field count, and degrades gracefully on a short row where an anchored pattern returns nothing.
• substr (or unpack) for fixed-width columns. Measured: on a row with an empty value column, substr gave three fields with a blank middle and whitespace-splitting gave two, with the unit in the value's place.
• A whitespace delimiter and an absent field are indistinguishable. No pattern can recover that; only column offsets carry it. This overrides the \s+ advice from Chapter 11.5 when columns can be blank.
• A pattern is right when the target is a rule rather than a fixed word, delimiter or offset. That is the extraction job from Chapter 11.1.
• No pattern handles quoting. A quote-aware pattern parsed a quoted comma correctly and then failed on a doubled quote, giving 4 fields where 3 are right.
• Two patches to the same pattern means stop patching. The input has structure the pattern is not modelling.
• Use Text::CSV for delimited data with quoting, and the existing parser for anything structured.

That completes Module 11. Twenty-two chapters: the pattern language itself from literal text through metacharacters, classes, quantifiers, greediness, anchors, alternation and both styles of capture; then /g, substitution, transliteration, the mode modifiers, /x, compiled patterns, lookaround and progressive matching; and finally how to debug a pattern and how to recognise when not to write one. Module 12 turns to organising code into modules and packages, and Module 13 to the input side — opening files, reading them line by line, and writing results back out — which is where almost every pattern in this module will actually be applied.