Anchors and Word Boundaries

Part II · Module 19 · Chapter 19.7 · Anchors and Word Boundaries

1. Assertions Match Positions, Not Characters

Everything so far has consumed characters: a class matches one, a quantifier matches several. An anchor matches a position and consumes nothing:

import re

print("^ at the start :", re.search(r"^UVM", "UVM_ERROR here").group())
print("^ not at start :", re.search(r"^UVM", " UVM_ERROR here"))
print("$ at the end   :", re.search(r"ns$", "slack -0.142ns").group())
print("$ not at end   :", re.search(r"ns$", "slack -0.142ns extra"))
print()
m = re.search(r"^UVM", "UVM_ERROR")
print("span of ^UVM   :", m.span(), "-- the ^ itself took no characters")

^ at the start : UVM
^ not at start : None
$ at the end   : ns
$ not at end   : None

span of ^UVM   : (0, 3) -- the ^ itself took no characters

Two things to read there. ns$ returned just ns, not the whole line — an anchor constrains where the match may sit and adds nothing to what comes back.

And the span is (0, 3): three characters for UVM and none for the ^. That zero-width property is what makes anchors composable.

2. Anchoring a Whole Line

A pattern that must describe the entire line uses both:

import re

CANDIDATES = ["smoke,1234,PASS", "  smoke,1234,PASS", "smoke,1234,PASS  ", "#smoke,1234,PASS!"]

for c in CANDIDATES:
    loose = bool(re.search(r"\w+,\d+,\w+", c))
    strict = bool(re.search(r"^\w+,\d+,\w+$", c))
    print(f"{c!r:<22} loose={str(loose):<6} anchored={strict}")

'smoke,1234,PASS'      loose=True   anchored=True
'  smoke,1234,PASS'    loose=True   anchored=False
'smoke,1234,PASS  '    loose=True   anchored=False
'#smoke,1234,PASS!'    loose=True   anchored=False

The loose pattern accepts all four, including two with stray whitespace and one with junk on both sides. Anchored, it accepts only the well-formed line.

Which you want depends on the job. A parser scanning a log for an error line wants search with no anchors — the line has a timestamp and a prefix and the interesting part is in the middle. A validator checking that a line conforms to a format wants anchors, because the whole point is to reject anything else.

Chapter 19.11's fullmatch does the same job as ^...$ and reads better for validation; the anchors are what you use when the pattern is embedded in a larger one.

3. The NO_ERROR Problem

The bug the canonical purpose names:

import re

LINES = [
    "UVM_ERROR drv.sv(88) bad handshake",
    "checker reports NO_ERROR for this transaction",
    "status: NO-ERROR",
    "an ERROR was found",
]

for line in LINES:
    naive = bool(re.search(r"ERROR", line))
    print(f"{str(naive):<6} {line}")

True   UVM_ERROR drv.sv(88) bad handshake
True   checker reports NO_ERROR for this transaction
True   status: NO-ERROR
True   an ERROR was found

Four lines, four matches, and two of them are wrong. A checker reporting NO_ERROR is reporting success, and a script counting ERROR matches has just recorded two failures that did not happen.

This is not a contrived example. NO_ERROR, ERROR_COUNT: 0, error_threshold and max_errors all appear in real tool output, and all of them match a bare substring search.

4. \b, and the Detail That Decides Whether It Works

A word boundary is the position between a word character and a non-word character:

import re

LINES = [
    "UVM_ERROR drv.sv(88) bad handshake",
    "checker reports NO_ERROR for this transaction",
    "status: NO-ERROR",
    "an ERROR was found",
]

for line in LINES:
    bounded = bool(re.search(r"\bERROR\b", line))
    print(f"{str(bounded):<6} {line}")

False  UVM_ERROR drv.sv(88) bad handshake
False  checker reports NO_ERROR for this transaction
True   status: NO-ERROR
True   an ERROR was found

Three of those four results are worth reading individually.

NO_ERROR is correctly rejected. The character before ERROR is _, which Chapter 19.4 established is a word character — so there is no boundary between O_ and E, and the assertion fails. This is the fix working.

UVM_ERROR is also rejected, by exactly the same mechanism — and here that is probably not what you wanted. UVM_ERROR is a real error and \bERROR\b just filtered it out. The correct pattern for a UVM log is \bUVM_ERROR\b, matching the whole token.

NO-ERROR is accepted, which is the opposite of NO_ERROR and means the same thing. Worth isolating:

import re

for subject in ["NO_ERROR", "NO-ERROR", "NO ERROR", "ERROR"]:
    hit = bool(re.search(r"\bERROR\b", subject))
    before = subject[subject.index("ERROR") - 1] if subject.index("ERROR") > 0 else "(start)"
    print(f"{subject:<10} char before ERROR: {before!r:<9} \\bERROR\\b matches: {hit}")

NO_ERROR   char before ERROR: '_'       \bERROR\b matches: False
NO-ERROR   char before ERROR: '-'       \bERROR\b matches: True
NO ERROR   char before ERROR: ' '       \bERROR\b matches: True
ERROR      char before ERROR: '(start)' \bERROR\b matches: True

NO-ERROR matches. The hyphen is not a word character, so there is a boundary between - and E, and \bERROR\b happily matches the ERROR half of a token that means the opposite.

That is the detail worth taking away:

\b is defined entirely by \w. It separates NO_ERROR because the underscore is a word character, and fails to separate NO-ERROR because the hyphen is not.

So "use \b to avoid substring false positives" is true for underscore-joined tokens and false for hyphen-joined ones — and which a tool uses is a property of the tool, not something you can assume.

5. Three Ways to Fix It, and When Each Applies

import re

LINES = [
    "UVM_ERROR drv.sv(88) bad handshake",
    "checker reports NO_ERROR",
    "status: NO-ERROR",
    "an ERROR was found",
    "ERROR_COUNT: 0",
]

print(f"{'whole token':<13}{'anchored':<11}{'separator-aware':<17}line")
for line in LINES:
    whole = bool(re.search(r"\bUVM_ERROR\b", line))
    anchored = bool(re.search(r"^UVM_ERROR\b", line))
    sep = bool(re.search(r"(?:^|\s)ERROR(?:\s|:|$)", line))
    print(f"{str(whole):<13}{str(anchored):<11}{str(sep):<17}{line}")

whole token  anchored   separator-aware  line
True         True       False            UVM_ERROR drv.sv(88) bad handshake
False        False      False            checker reports NO_ERROR
False        False      False            status: NO-ERROR
False        False      True             an ERROR was found
False        False      False            ERROR_COUNT: 0

Three different correct answers to three different questions.

Match the whole token — \bUVM_ERROR\b — when you know exactly what the tool writes. This is almost always the right answer for a known format, and it sidesteps the hyphen problem entirely because you are matching the complete thing rather than a fragment of it.

Anchor to the start — ^UVM_ERROR\b — when the token must be the first thing on the line. Stricter still, and it rejects a line that merely mentions UVM_ERROR in a message.

Be explicit about separators — (?:^|\s)ERROR(?:\s|:|$) — when you genuinely need the bare word and cannot rely on \b. It is uglier and it says precisely what counts as a separator. (The (?:...) is a non-capturing group; Chapter 19.8 explains the parentheses and Chapter 19.10 the |.)

Note the last row: ERROR_COUNT: 0 is rejected by all three, which is correct — it is a field name, not an error.

6. ^ Means Start of String, Not Start of Line

The property that catches people the first time a whole file is read at once:

import re

LINE = "UVM_ERROR first"
WHOLE_FILE = "UVM_ERROR first\nUVM_ERROR second\nUVM_ERROR third\n"

print("on one line     :", re.findall(r"^UVM_ERROR", LINE))
print("on a whole file :", re.findall(r"^UVM_ERROR", WHOLE_FILE))
print("unanchored      :", len(re.findall(r"UVM_ERROR", WHOLE_FILE)))

on one line     : ['UVM_ERROR']
on a whole file : ['UVM_ERROR']
unanchored      : 3

One match on a file containing three. By default ^ means the start of the whole string, so on a file read with read() it matches only at character zero.

This is the same read-line-by-line versus read-whole distinction Chapter 19.5 Section 7 raised for negated classes, and it has the same shape: a pattern that is correct while you test it on single lines behaves differently the day somebody passes it a whole file.

Two fixes. Iterate the file and anchor per line, which is Chapter 15.7's streaming default and usually right. Or use the MULTILINE flag, which redefines ^ and $ to mean line boundaries — Chapter 19.14's subject.

$ has a related wrinkle worth knowing now:

import re

print("$ before a trailing newline:", bool(re.search(r"complete$", "Simulation complete\n")))
print("\\Z is strictly the end     :", bool(re.search(r"complete\Z", "Simulation complete\n")))
print("without the newline        :", bool(re.search(r"complete\Z", "Simulation complete")))

$ before a trailing newline: True
\Z is strictly the end     : False
without the newline        : True

$ matches before a final newline as well as at the very end, which is almost always what you want when a line still has its terminator attached (Chapter 15.3). \Z is the strict version, for the rare case where the trailing newline matters.

7. Predict, Run, Observe

import re

SUBJECT = "ERROR_COUNT: 0, errors: 3, NO_ERROR seen, an ERROR here"

print("A:", len(re.findall(r"ERROR", SUBJECT)))
print("B:", len(re.findall(r"\bERROR\b", SUBJECT)))
print("C:", re.findall(r"\bERROR\w*", SUBJECT))
print("D:", bool(re.search(r"^ERROR", SUBJECT)))
print("E:", bool(re.search(r"here$", SUBJECT)))
print("F:", len(re.findall(r"\berrors\b", SUBJECT)))
print("G:", re.search(r"\BERROR", SUBJECT).group())

A: 3
B: 1
C: ['ERROR_COUNT', 'ERROR']
D: True
E: True
F: 1
G: ERROR

A is 3 and B is 1. The bare substring found ERROR_COUNT, NO_ERROR and the real one; the bounded version found only the real one. That gap is the whole chapter.

C shows \b at one end only. \bERROR\w* requires a boundary before ERROR and then allows word characters after it, so it matched ERROR_COUNT (boundary before, _COUNT after) and the bare ERROR — but not NO_ERROR, which has no boundary before. Anchoring one end is a different question from anchoring both.

D is True because the string genuinely starts with ERROR_COUNT, and ^ERROR asserts only the start and the five letters — it says nothing about what follows.

F is 1 and matching is still case-sensitive: errors matched the lowercase occurrence and not the uppercase ones.

G is \B, the inverse assertion, finding an ERROR that is not at a word boundary — the ERROR inside NO_ERROR. It is occasionally useful for finding exactly the false positives you are trying to exclude, which makes it a debugging tool more than a parsing one.

8. Exercises

Exercise 1 — zero width. Confirm ^UVM has a span of three characters. Explain what the ^ contributed.

Exercise 2 — validator versus scanner. Write one pattern with anchors and one without for the same format. Feed both a line with trailing whitespace.

Exercise 3 — the false positive. Count ERROR matches in a log containing NO_ERROR and ERROR_COUNT. Report how many are real.

Exercise 4 — the separator decides. Show \bERROR\b rejecting NO_ERROR and accepting NO-ERROR. Say why.

Exercise 5 — three fixes. Apply the whole-token, anchored and separator-aware fixes to the same log. Tabulate which lines each accepts.

Exercise 6 — the whole file. Anchor a pattern with ^ and run it against a three-line string read whole. Explain the single match.

Exercise 7 — the trailing newline. Compare $ and \Z on a line that still has its newline. Say which you want when iterating a file.

Exercise 8 — find the false positives. Use \B to list every occurrence in a log that a bounded pattern would reject. Check they are all genuinely not errors.

9. Summary

• An anchor matches a position and consumes nothing, so it constrains where a match may occur without changing what it returns.
• ^ and $ pin a pattern to the start and end; together they make a validator rather than a scanner.
• A bare substring search matches ERROR inside NO_ERROR — the most common false positive in log parsing.
• \b is a word boundary, defined entirely by \w.
• It separates NO_ERROR because the underscore is a word character, and fails to separate NO-ERROR because the hyphen is not.
• So \b works for underscore-joined tokens and not hyphen-joined ones, and which a tool uses is not something to assume.
• Matching the whole token — \bUVM_ERROR\b — is the fix that does not depend on the separator.
• ^ means start of string, not start of line, so on a file read whole it matches once; iterate the file or use MULTILINE.
• $ matches before a trailing newline; \Z is the strict end.
• \B finds the non-boundary matches, which makes it a tool for auditing your own false positives.

The next chapter is the one the module has been building toward: turning a test into an extraction.