search, match, fullmatch, findall, finditer

Part II · Module 19 · Chapter 19.11 · search, match, fullmatch, findall, finditer

1. Five Jobs

All five take the same two arguments — a pattern and a subject — and differ in where they look and what they hand back:

Two columns of that table are worth reading as a pair. The first three return a match or None and need Chapter 19.8's guard. The last two return a collection and never return None — an empty list or an iterator that yields nothing is what "no matches" looks like there, which means no guard is needed and if m is None would be a bug.

Everything in this chapter is one of those five choices. None of them changes pattern syntax at all.

2. search: Anywhere

The default, and the one every chapter so far has used:

import re

LINE = "UVM_ERROR drv.sv(88) @ 12500: seed=1839274"

print("first digits:", re.search(r"\d+", LINE).group(0))
print("severity    :", re.search(r"UVM_\w+", LINE).group(0))
print("missing     :", re.search(r"slack", LINE))

first digits: 88
severity    : UVM_ERROR
missing     : None

re.search scans left to right and stops at the first position where the pattern can match. The first digits in that line are the 88 inside the parentheses, not the seed — which is a useful reminder that \d+ on its own says nothing about which number you wanted. The fix is context in the pattern, not a different function.

3. match: Anchored at Position Zero

re.match tries the pattern only at the start of the subject:

import re

LINE = "UVM_ERROR drv.sv(88) @ 12500: seed=1839274"

print("match UVM_ :", re.match(r"UVM_\w+", LINE))
print("match digit:", re.match(r"\d+", LINE))
print("match drv  :", re.match(r"drv", LINE))
print("search drv :", re.search(r"drv", LINE))

match UVM_ : <re.Match object; span=(0, 9), match='UVM_ERROR'>
match digit: None
match drv  : None
search drv : <re.Match object; span=(10, 13), match='drv'>

re.match(r"drv", ...) is None even though the line plainly contains drv, because position 0 is a U. re.search finds it at offset 10. The pattern is identical; only the starting rule differs.

This is where a line-by-line misreading starts. re.match is not the ^ anchor and it is not a per-line test:

import re

TEXT = """UVM_ERROR drv.sv(88) @ 12500: seed=1839274
UVM_WARNING mon.sv(51) @ 99300: seed=1839274
UVM_FATAL env.sv(12) @ 141900: seed=771002
"""

print("match  :", re.match(r"UVM_WARNING", TEXT))
print("match ^:", re.match(r"^UVM_WARNING", TEXT))
print("search^:", re.search(r"^UVM_WARNING", TEXT))
print("search :", re.search(r"UVM_WARNING", TEXT).span())

match  : None
match ^: None
search^: None
search : (43, 54)

All three of the first results are None, for two different reasons that are easy to conflate. re.match fails because the subject does not begin with UVM_WARNING. The two patterns with ^ fail because — as Chapter 19.7 established — ^ means start of the string, not start of a line, and this three-line subject is one string. Only re.search finds the second line's severity, 43 characters in.

The practical consequence: to test lines, iterate over lines. for line in text.splitlines() and then re.match on each is a per-line start-of-line test; re.match on the whole text is not.

4. fullmatch: The Whole Field or Nothing

re.fullmatch requires the pattern to consume the entire subject, and it is the function to use when the question is is this field valid:

import re

FIELDS = ["1839274", "1839274 ", "18x9274", ""]

print(f"{'field':<12} {'fullmatch':>9} {'match':>7} {'search':>7}")
for field in FIELDS:
    print(f"{field!r:<12} {str(bool(re.fullmatch(r'\d+', field))):>9}"
          f" {str(bool(re.match(r'\d+', field))):>7}"
          f" {str(bool(re.search(r'\d+', field))):>7}")

field        fullmatch   match  search
'1839274'         True    True    True
'1839274 '       False    True    True
'18x9274'        False    True    True
''               False   False   False

Read the second and third rows carefully, because they are the reason this function exists. A seed field with a trailing space and a seed field with an x in the middle are both accepted by match and by search, and both rejected by fullmatch. match is happy because the field starts with digits; search is happy because it contains digits somewhere.

A validator built on search approves 18x9274 as a seed. One built on fullmatch does not. If you are checking a value rather than finding one, this is the function.

fullmatch also makes the anchors from Chapter 19.7 unnecessary — it is exactly equivalent to wrapping the pattern in ^...$ — and the function form is harder to get wrong, because an anchor accidentally left outside a group is Chapter 19.10 Section 3's trap and a missing function argument is not.

5. findall: Three Return Shapes, Chosen by Your Pattern

re.findall returns every non-overlapping match, left to right, as a list. With no group in the pattern it is exactly what you expect:

import re

TEXT = """UVM_ERROR drv.sv(88) @ 12500: seed=1839274
UVM_WARNING mon.sv(51) @ 99300: seed=1839274
UVM_FATAL env.sv(12) @ 141900: seed=771002
"""

print(re.findall(r"UVM_\w+", TEXT))
print(len(re.findall(r"UVM_\w+", TEXT)))

['UVM_ERROR', 'UVM_WARNING', 'UVM_FATAL']
3

Three whole matches, as strings. Now add a group — and note that adding a group has never changed anything in this module before:

import re

TEXT = """UVM_ERROR drv.sv(88) @ 12500: seed=1839274
UVM_WARNING mon.sv(51) @ 99300: seed=1839274
UVM_FATAL env.sv(12) @ 141900: seed=771002
"""

print(re.findall(r"UVM_(ERROR|FATAL|WARNING)", TEXT))

['ERROR', 'WARNING', 'FATAL']

The UVM_ is gone. With exactly one group, findall returns that group rather than the whole match — the canonical annotation's comment, groups, not whole matches, in one line of output. Nothing about search behaves this way; m.group(0) was always available there.

With more than one group it changes shape again:

import re

TEXT = """UVM_ERROR drv.sv(88) @ 12500: seed=1839274
UVM_WARNING mon.sv(51) @ 99300: seed=1839274
UVM_FATAL env.sv(12) @ 141900: seed=771002
"""

rows = re.findall(r"(\w+\.sv)\((\d+)\)", TEXT)
print(rows)
for src, num in rows:
    print(f"{src:<8} {int(num):>4}")

[('drv.sv', '88'), ('mon.sv', '51'), ('env.sv', '12')]
drv.sv     88
mon.sv     51
env.sv     12

A list of tuples, one tuple per match, one element per group — and tuple unpacking in the loop reads well enough that this is a genuinely good use of findall. Named groups make no difference at all here: findall returns the same tuples whether the groups are named or not, because a tuple has no room for the names.

So the rule, in three lines:

One more wrinkle, which follows from Chapter 19.10 Section 6 — a non-participating optional group becomes an empty string in findall, not None:

import re

print(re.findall(r"a(b)?", "ab a"))

['b', '']

findall has nowhere to put a None in a list of strings, so it substitutes ''. That means findall cannot distinguish an absent optional field from an empty one, and the moment your pattern has an optional group you need the next function instead.

6. finditer: Match Objects, One at a Time

re.finditer yields a full match object per occurrence, so everything from Chapters 19.8 and 19.9 is available:

import re

TEXT = """UVM_ERROR drv.sv(88) @ 12500: seed=1839274
UVM_WARNING mon.sv(51) @ 99300: seed=1839274
UVM_FATAL env.sv(12) @ 141900: seed=771002
"""

ISSUE = r"UVM_(?P<sev>ERROR|FATAL|WARNING) (?P<file>\w+\.sv)\((?P<line>\d+)\) @ (?P<cycle>\d+):"

for m in re.finditer(ISSUE, TEXT):
    f = m.groupdict()
    print(f"{f['sev']:<8} {f['file']:<8} {int(f['line']):>4} {int(f['cycle']):>7}  at char {m.start()}")

ERROR    drv.sv     88   12500  at char 0
WARNING  mon.sv     51   99300  at char 43
FATAL    env.sv     12  141900  at char 88

Names, numbers, groupdict, span, start — all of it, per match, which findall cannot give you at any price. The canonical annotation's loop is this function:

for m in re.finditer(r"seed=(\d+)", text):
    seeds.append(int(m.group(1)))

And finditer is lazy. It produces matches on demand rather than building a list, which matters on the kind of file this module exists for:

import re

TEXT = "".join(f"UVM_ERROR drv.sv({i}) @ {i * 100}: seed={i}\n" for i in range(1, 200001))

print("lines in subject    :", TEXT.count("\n"))

every = re.findall(r"seed=(\d+)", TEXT)
print("findall list length :", len(every))
print("findall first, last :", every[0], every[-1])

first_three = []
for m in re.finditer(r"seed=(\d+)", TEXT):
    first_three.append(int(m.group(1)))
    if len(first_three) == 3:
        break
print("finditer stopped at :", first_three)

lines in subject    : 200000
findall list length : 200000
findall first, last : 1 200000
finditer stopped at : [1, 2, 3]

findall built a list of two hundred thousand strings whether or not anything needed them. finditer found three and stopped, and the break meant the engine never scanned the rest of the subject. On a report you are scanning for the first failure — which Module 20 does — that difference is the whole cost of the operation.

7. Choosing, on a Real Question

Four questions a flow script actually asks, and the function each one wants:

import re

TEXT = """UVM_INFO top.sv(142) @ 0: starting test smoke_rw
UVM_ERROR drv.sv(88) @ 12500: bad handshake
UVM_WARNING mon.sv(51) @ 99300: late sample
UVM_ERROR drv.sv(104) @ 141900: timeout
"""

# 1. Did anything fail at all?  -> search, one answer, stops early
failed = re.search(r"UVM_(ERROR|FATAL)", TEXT) is not None
print("1 failed       :", failed)

# 2. How many of each severity? -> findall, no positions needed
counts = {}
for sev in re.findall(r"UVM_(ERROR|FATAL|WARNING|INFO)", TEXT):
    counts[sev] = counts.get(sev, 0) + 1
print("2 counts       :", counts)

# 3. Where is the FIRST failure? -> finditer plus break, or search
m = re.search(r"UVM_(?:ERROR|FATAL)[^\n]*", TEXT)
print("3 first failure:", m.group(0))

# 4. Is this field a clean seed? -> fullmatch
print("4 seed valid   :", bool(re.fullmatch(r"\d+", "1839274")), bool(re.fullmatch(r"\d+", "1839274 ")))

1 failed       : True
2 counts       : {'INFO': 1, 'ERROR': 2, 'WARNING': 1}
3 first failure: UVM_ERROR drv.sv(88) @ 12500: bad handshake
4 seed valid   : True False

The pattern in question 3 contains (?:ERROR|FATAL), which is Chapter 19.12's construct — a group that bounds the alternation without capturing. It is used here because the alternation needs parentheses and nothing needs the group, and [^\n]* extends the match to the end of the line without crossing into the next one, which is Chapter 19.5's negated class doing the job . cannot do here.

The decision in each case came from what the question needed, not from what the pattern looked like:

• A yes-or-no question wants search, because it can stop at the first hit.
• A count wants findall, because positions and groups are irrelevant and a list is the easiest thing to measure.
• A record — fields, positions, context — wants finditer, and takes a break when only the first one matters.
• A validation wants fullmatch, because search and match both approve a field with junk in it.
• match wants data that is already one record per call.

8. Predict, Run, Observe

import re

TEXT = "slack -0.142\nslack 0.031\n"

# A
print("A:", re.match(r"slack", TEXT) is not None)

# B
print("B:", re.match(r"0.031", TEXT) is not None)

# C
print("C:", re.findall(r"slack (-?\d+\.\d+)", TEXT))

# D
print("D:", re.findall(r"(slack) (-?\d+\.\d+)", TEXT))

# E
print("E:", [m.start() for m in re.finditer(r"slack", TEXT)])

# F
print("F:", re.findall(r"timing", TEXT), re.search(r"timing", TEXT))

# G
print("G:", bool(re.fullmatch(r"slack -?\d+\.\d+", "slack -0.142")))

The verified output:

A: True
B: False
C: ['-0.142', '0.031']
D: [('slack', '-0.142'), ('slack', '0.031')]
E: [0, 13]
F: [] None
G: True

A — the subject starts with slack, so match succeeds at position 0.

B — 0.031 is in the subject but not at position 0, so match fails. search would find it.

C — one group, so findall returns the group and the word slack is nowhere in the result.

D — two groups, so tuples. Same matches, different shape, decided entirely by the pattern.

E — finditer gives positions; the second slack starts at 13, which findall could not have told you.

F — no matches: findall returns [] and search returns None. Different falsy values from the same absence, which is why the two families need different guards.

G — fullmatch consumed the whole subject, so the field is valid in its entirety.

9. Exercises

Exercise 1 — search versus match. Take one log line and find a word in its middle with both functions. Record both results and explain the None in one sentence.

Exercise 2 — match is not ^. Build a three-line string whose second line starts with UVM_WARNING. Try re.match(r"UVM_WARNING", text), re.match(r"^UVM_WARNING", text) and re.search(r"^UVM_WARNING", text). Say why all three fail, giving two distinct reasons.

Exercise 3 — iterate to test lines. Fix Exercise 2 by iterating over splitlines() and using re.match per line. Print which line index matched.

Exercise 4 — the validator. Build the four-row table from Section 4 for a pattern of your own — a hex address, say — including one field with trailing whitespace. Say which function you would use in a config checker.

Exercise 5 — the three shapes. Run findall with the same subject and three patterns: no group, one group, two groups. Print all three results and state what decided each return type.

Exercise 6 — the silent edit. Write a working findall loop with no group, then add a group to the pattern without changing the loop. Record what the loop now prints and why nothing raised.

Exercise 7 — optional group in findall. Use a pattern with an optional group in findall and show that the absent group arrives as ''. Then get a None for the same data using finditer, and say which you would want in a CSV row.

Exercise 8 — positions. Use finditer to print every severity in a log extract with the character offset where it starts. Say what findall would have to give up.

Exercise 9 — stop early. Generate a 50,000-line synthetic log and find only the first three seeds, using finditer and break. Then do it with findall and say exactly what the second version did that the first did not.

Exercise 10 — four questions. For each, name the function and justify it in one sentence: (a) count the warnings; (b) is this string a legal instance path; (c) report the first error with its cycle number; (d) does the log mention a fatal anywhere.

10. Summary

• Five entry points, same pattern syntax. The choice is about where the search starts and what comes back.
• search looks anywhere and stops at the first position that matches — the default, and the right one most of the time.
• match is anchored at position 0, and returns None for a pattern that appears later in the subject.
• match is not ^, and neither is a line-by-line test. ^ is start of string; to test lines, iterate over lines.
• fullmatch requires the pattern to consume the whole subject — the function for validating a field.
• search approves 18x9274 as a seed and fullmatch does not. Trailing whitespace is the version of this that reaches production.
• findall returns every non-overlapping match as a list, and its return shape depends on your pattern: whole matches with no group, group 1 with one group, tuples with two or more.
• Adding a group to a working findall silently changes every value in the loop and raises nothing.
• findall turns a non-participating optional group into '', so it cannot distinguish absent from empty.
• finditer yields match objects — names, numbers, groupdict, span, start — and is lazy, so a break stops the scan.
• finditer is an iterator and good for one pass. Build a list deliberately if you need two.
• The first three return a match or None and need a guard; the last two return a collection and never do — [] and None are different absences.
• Pick from the question: yes-or-no wants search, a count wants findall, a record wants finditer, a check wants fullmatch.

The next chapter uses these to build one real pattern from nothing, adding a single construct at a time until no symbol in it is unexplained.