Matching a Severity Tag Without False Positives

Part II · Module 18 · Chapter 18.4 · Matching a Severity Tag Without False Positives

1. The Five False Positives

Here are eight lines. Exactly one is a genuine UVM_ERROR message.

1  UVM_ERROR tb/env/sb.sv(151) @ 2400: [SB_MISMATCH] expected 0x0f     <- the real one
2  UVM_ERROR :    1                                                    <- the summary's count
3  UVM_INFO @ 0: [CFG] NO_UVM_ERROR mode is enabled                    <- the opposite meaning
4  UVM_INFO @ 9600: the string "UVM_ERROR" is quoted here              <- a quoted mention
5  UVM_INFO @ 0: reporter [RNTST] Running test uvm_error_injection_test <- a test name
6  UVM_INFO @ 0: [CFG] UVM_ERROR_LIMIT set to 10                       <- a config token
7  # UVM_ERROR mentioned in a comment line                             <- a comment
8  UVM_WARNING tb/env/drv.sv(88) @ 2401: [DRV] backpressure            <- a different severity

Every one of lines 2, 3, 4, 6 and 7 contains the exact string UVM_ERROR. Line 5 contains it only in lowercase. Line 8 contains a different severity that shares the prefix UVM_.

The question is which pattern distinguishes line 1 from all seven others.

2. The Ladder

use strict;
use warnings;

my @line = (
 'UVM_ERROR tb/env/sb.sv(151) @ 2400: [SB_MISMATCH] expected 0x0f',
 'UVM_ERROR :    1',
 'UVM_INFO @ 0: [CFG] NO_UVM_ERROR mode is enabled',
 'UVM_INFO @ 9600: the string "UVM_ERROR" is quoted here',
 'UVM_INFO @ 0: reporter [RNTST] Running test uvm_error_injection_test...',
 'UVM_INFO @ 0: [CFG] UVM_ERROR_LIMIT set to 10',
 '# UVM_ERROR mentioned in a comment line',
 'UVM_WARNING tb/env/drv.sv(88) @ 2401: [DRV] backpressure',
);

# each rung adds ONE restriction to the one above it
my @rung = (
 [ 'substring   /UVM_ERROR/',                qr/UVM_ERROR/                 ],
 [ 'anchored    /^UVM_ERROR/',               qr/^UVM_ERROR/                ],
 [ 'bounded     /^UVM_ERROR\b/',             qr/^UVM_ERROR\b/              ],
 [ 'tokenised   /^UVM_ERROR(?=\s)/',         qr/^UVM_ERROR(?=\s)/          ],
 [ 'line shape  /^UVM_ERROR\s+\S+\(\d+\)/',  qr/^UVM_ERROR\s+\S+\(\d+\)/   ],
);

printf "%-36s %5s  %s\n", "PATTERN", "hits", "matched lines";
for my $r (@rung) {
    my @hit = grep { $line[$_ - 1] =~ $r->[1] } 1 .. scalar @line;
    printf "%-36s %5d  %s\n", $r->[0], scalar @hit, join(",", @hit);
}

print "\nonly line 1 is a genuine UVM_ERROR message.\n";
print "2 summary count | 3 NO_UVM_ERROR | 4 quoted | 5 test name (lowercase)\n";
print "6 UVM_ERROR_LIMIT | 7 a comment | 8 a different severity\n";

PATTERN                               hits  matched lines
substring   /UVM_ERROR/                  6  1,2,3,4,6,7
anchored    /^UVM_ERROR/                 2  1,2
bounded     /^UVM_ERROR\b/               2  1,2
tokenised   /^UVM_ERROR(?=\s)/           2  1,2
line shape  /^UVM_ERROR\s+\S+\(\d+\)/     1  1

only line 1 is a genuine UVM_ERROR message.
2 summary count | 3 NO_UVM_ERROR | 4 quoted | 5 test name (lowercase)
6 UVM_ERROR_LIMIT | 7 a comment | 8 a different severity

Read down the hits column: 6, then 2, 2, 2, then 1.

The substring finds six. Five false positives, and it misses nothing — it also matches line 7, a comment, which some logs contain and some tools emit.

Anchoring drops it to two. ^ removes the quoted mention, the config token, the NO_UVM_ERROR line and the comment in one move, because in all four cases the token is not at the start. This is the single most valuable character in the pattern.

\b and (?=\s) change nothing. Both rungs still find two. That is the result worth pausing on.

Only the line shape finds one.

3. Why Anchoring Is Not Enough

The line that survives anchoring is line 2, the summary's own count:

UVM_ERROR :    1

That line genuinely begins with UVM_ERROR, and the next character genuinely is whitespace. So:

• ^UVM_ERROR matches — the token is at the start
• ^UVM_ERROR\b matches — there is a word boundary after R
• ^UVM_ERROR(?=\s) matches — the next character is a space

All three refinements are asking about the token, and the token is correct. The thing that is wrong is the rest of the line: a message continues with a file and line number, and a count continues with a colon and a digit.

So the pattern has to describe the line, not just its first word:

qr/^UVM_ERROR\s+\S+\(\d+\)/        # severity, then file(line)

That requires a non-space run followed by a parenthesised number, which UVM_ERROR : 1 cannot satisfy. Chapter 18.1 measured the cost of not doing this — a count of 2 where the truth was 1, and a count of 1 on a passing test.

4. The Pattern To Actually Use

Putting it together, with the normalisation from Chapter 18.3 in front of it:

my $NORM     = \&normalise;                       # 18.3's shared stripper
my %SEVERITY = map { $_ => qr/^UVM_\Q$_\E\s+\S+\(\d+\)/ }
               qw(INFO WARNING ERROR FATAL);

Four things that gives you:

One pattern per severity, built once. A table rather than four literals, so the shape is defined in one place and a change applies to all of them. Chapter 18.2's qr// argument.

\Q...\E around the interpolated name. Chapter 18.2 measured why: a severity read from a configuration file containing a . would otherwise become a wildcard, and UVM.ERROR matched both UVM_ERROR and UVM.ERROR.

The line shape, not the token. As measured.

No /i.

And the counting loop, with the cross-check Chapter 18.1 recommended:

my (%count, %summary);
while (my $raw = <$fh>) {
    my $line = $NORM->($raw);

    # the summary's own counts -- captured deliberately, as a cross-check
    if ($line =~ /^UVM_(\w+)\s*:\s*(\d+)\s*\z/) { $summary{$1} = $2; next }

    for my $sev (keys %SEVERITY) {
        $count{$sev}++, last if $line =~ $SEVERITY{$sev};
    }
}

for my $sev (sort keys %count) {
    next unless defined $summary{$sev};
    next if $summary{$sev} == ($count{$sev} // 0);
    printf STDERR "warning: counted %d %s but the log's summary says %d\n",
        $count{$sev} // 0, "UVM_$sev", $summary{$sev};
}

The next on the summary branch is what removes line 2 from the message counts — it is recognised as a count and consumed as one. And then the two numbers are compared, which is the cheapest possible check on your own parser: if your count and the tool's disagree, one of them is wrong and you would rather know during development than during signoff.

5. What To Remember

• A log mentions its own severity tokens constantly — in its summary, in config messages, in test names and inside quoted strings.
• Measured on 8 lines with 1 real error: substring 6 hits, anchored 2, \b 2, (?=\s) 2, line shape 1.
• ^ is the single most valuable character — it removes the quoted mention, the config token, NO_UVM_ERROR and the comment at once.
• \b and (?=\s) add nothing here, because the surviving false positive has a correct token.
• The summary's UVM_ERROR : 1 begins with the token and a space, so no token-level refinement can reject it.
• Match the LINE SHAPE: ^UVM_ERROR\s+\S+\(\d+\) — severity, then file(line).
• A count line continues with : and a digit; a message continues with a file and line number. That is the difference.
• Line 5 escaped every pattern only because it is lowercase — luck, not design.
• Never use /i on a severity pattern. UVM severities are upper-case, and the lower-case spelling is a class or test name.
• Build one qr// per severity in a table, with \Q...\E around the interpolated name.
• Recognise the summary branch and next, so a count is consumed as a count.
• Then cross-check your count against the summary's and warn on disagreement — the cheapest possible test of your own parser.
• A severity count is not a verdict. Zero errors can still be a failing test, and errors can be deliberately demoted.

Chapter 18.5 moves from deciding whether a line matters to getting the values out of it. The capture is the easy part; the chapter is about the line that matched and captured nothing.