Capturing the Value You Actually Need

Part II · Module 19 · Chapter 19.8 · Capturing the Value You Actually Need

1. One Pair of Parentheses

Here is the pattern from Chapter 19.6, and the same pattern with two characters added:

import re

LINE = "UVM_INFO top.sv(142) @ 12500: uvm_test_top [CFG] test=smoke_rw seed=1839274"

print("test   :", bool(re.search(r"seed=\d+", LINE)))

m = re.search(r"seed=(\d+)", LINE)
print("extract:", m.group(1))

test   : True
extract: 1839274

The first line is everything Chapters 19.1 to 19.7 could do: it reports that the line mentions a seed. The second line reports which seed, and the only difference between the two patterns is ( and ).

Parentheses in a regex mean remember what matched here. The region they enclose is called a group, and a pattern may contain as many as it needs. A group changes nothing about whether the pattern matches — seed=\d+ and seed=(\d+) match exactly the same lines — it changes what you can ask about the match afterwards.

That is the whole mechanism, and it is the one the rest of this curriculum runs on:

m = re.search(r"seed=(\d+)", line)
seed = int(m.group(1))

Two lines. The first finds, the second converts. Sections 3 and 6 are about why those two lines are not quite enough in a real script.

2. group(0), group(1), and span

A match object carries more than the captured text, and it is worth seeing all of it once:

import re

LINE = "UVM_INFO top.sv(142) @ 12500: uvm_test_top [CFG] test=smoke_rw seed=1839274"

m = re.search(r"seed=(\d+)", LINE)

print("group(0) :", repr(m.group(0)))
print("group(1) :", repr(m.group(1)))
print("group()  :", repr(m.group()))
print("groups() :", m.groups())
print("span(0)  :", m.span(0))
print("span(1)  :", m.span(1))

group(0) : 'seed=1839274'
group(1) : '1839274'
group()  : 'seed=1839274'
groups() : ('1839274',)
span(0)  : (63, 75)
span(1)  : (68, 75)

Five things to read off that output.

group(0) is the whole match, including the seed= that the pattern required but you did not want. Group zero always exists, in every match, whether or not the pattern has any parentheses at all. It is what m.group() with no argument returns.

group(1) is the first pair of parentheses — the part you asked to be remembered. The numbering starts at 1 precisely because 0 is taken.

groups() returns every captured group as a tuple, and excludes group 0. There is one group here, so the tuple has one element — and notice the trailing comma, which is Chapter 6.2's one-element tuple showing up in real output.

span(n) gives the half-open character range of group n in the subject string, in exactly the form Chapter 19.2 introduced. span(1) starting at 68 while span(0) starts at 63 is the five characters of seed= measured for you.

Spans matter when you need to edit rather than read. m.span(1) tells you where the number sits, so LINE[:m.start(1)] + "0" + LINE[m.end(1):] replaces the seed and leaves the rest of the line untouched — without a single assumption about field positions.

3. A Group Is Always a String

This is the part that bites in week one, and it never produces an error — only a wrong answer:

import re

LINE = "resolution: seed=9 retries=10 slack=-0.5"

a = re.search(r"seed=(\d+)", LINE).group(1)
b = re.search(r"retries=(\d+)", LINE).group(1)

print("types   :", type(a).__name__, type(b).__name__)
print("as text :", a > b)
print("as ints :", int(a) > int(b))

types   : str str
as text : True
as ints : False

\d+ matched digits, so it is natural to assume a number came back. It did not. Every group is a slice of the subject string, and a slice of a string is a string. "9" > "10" is True because text comparison works character by character and '9' sorts after '1' — exactly the trap Chapter 2.11 described, now arriving through a regex instead of through input().

So the second line of the canonical pattern is not decoration:

import re

LINE = "UVM_INFO top.sv(142) @ 12500: uvm_test_top [CFG] test=smoke_rw seed=1839274"

seed = int(re.search(r"seed=(\d+)", LINE).group(1))
cycle = int(re.search(r"@ (\d+):", LINE).group(1))
slack = float(re.search(r"\((\d+)\)", LINE).group(1))

print(seed + 1, cycle // 1000, slack)

1839275 12 142.0

Convert at the point of extraction, not three functions later. A string that reaches a comparison still looking like a number is a defect that passes every test written with small numbers and fails the first time a seed crosses a digit boundary.

4. Several Groups, Numbered Left to Right

A pattern may capture as many fields as the line contains:

import re

LINE = "UVM_INFO top.sv(142) @ 12500: uvm_test_top [CFG] test=smoke_rw seed=1839274"

m = re.search(r"UVM_(\w+) (\w+\.sv)\((\d+)\)", LINE)

print("1:", m.group(1))
print("2:", m.group(2))
print("3:", m.group(3))
print("all:", m.groups())
print("two at once:", m.group(1, 3))

1: INFO
2: top.sv
3: 142
all: ('INFO', 'top.sv', '142')
two at once: ('INFO', '142')

Groups are numbered left to right by their opening parenthesis, starting at 1. m.group(1, 3) with several numbers returns a tuple of just those groups, which is occasionally tidier than indexing groups() twice.

A group number that does not exist is an error, not a None:

import re

m = re.search(r"seed=(\d+)", "test=smoke_rw seed=1839274")
print(m.group(1))
print(m.group(2))

1839274
Traceback (most recent call last):
  File "group_range.py", line 5, in <module>
    print(m.group(2))
          ~~~~~~~^^^
IndexError: no such group

IndexError: no such group means the pattern has no group 2, which is a bug in your code rather than in the data. It is a different failure from the one in Section 6, and telling them apart quickly is worth the thirty seconds it takes to learn both messages.

5. Nested Groups Are Still Numbered by Opening Parenthesis

Groups can contain groups, and the numbering rule does not change — which surprises people who expect the outer group to somehow come last:

import re

LINE = "UVM_ERROR drv.sv(88) @ 12500: write to 0x40001000 returned 0xdeadbeef"

m = re.search(r"((\w+)\.sv)\((\d+)\)", LINE)

print("1:", m.group(1))
print("2:", m.group(2))
print("3:", m.group(3))
print("spans:", m.span(1), m.span(2), m.span(3))

1: drv.sv
2: drv
3: 88
spans: (10, 16) (10, 13) (17, 19)

Read the parentheses left to right and number them as you meet the opening one: the outer ( comes first, so the full filename is group 1; the inner ( comes second, so the stem is group 2; the \( for the literal bracket is escaped and captures nothing, so the line number is group 3.

The spans confirm the nesting physically. Group 2 (10, 13) sits inside group 1 (10, 16) — same start, three characters shorter, because .sv belongs to the outer group only.

Nesting like this is genuinely useful when you want both a whole field and a part of it without matching twice. It is also the main reason a six-group pattern becomes unmaintainable, because adding one set of parentheses anywhere on the left renumbers everything to its right — and every group(4) in your script silently starts meaning something else. Chapter 19.9 fixes that permanently.

6. The Check That Is Not Optional

Here is the canonical two-liner again, run over a real log extract rather than a single line that was chosen to match:

import re

LOG = """UVM_INFO top.sv(142) @ 0: uvm_test_top [CFG] test=smoke_rw seed=1839274
UVM_ERROR drv.sv(88) @ 12500: write to 0x40001000 returned 0xdeadbeef
UVM_INFO top.sv(210) @ 99000: uvm_test_top [CFG] test=burst_wr seed=771002
"""

for line in LOG.splitlines():
    m = re.search(r"seed=(\d+)", line)
    seed = int(m.group(1))
    print(seed)

1839274
Traceback (most recent call last):
  File "unguarded.py", line 10, in <module>
    seed = int(m.group(1))
               ^^^^^^^
AttributeError: 'NoneType' object has no attribute 'group'

The first line matched, printed, and gave no warning at all. The second line has no seed, so re.search returned None, and None.group(1) is an AttributeError — the exact shape Chapter 14.3 catalogued, now arriving from a regex.

This is the single most common crash in hand-written flow scripts, and the reason is worth stating plainly: re.search returns a match object or None, and nothing about the two-line idiom reminds you of the second possibility. It works perfectly on the one line you tested it against.

The fix is a guard, and there are two shapes worth knowing.

Skip what does not match — the right choice when most lines are irrelevant, which in a simulation log is nearly all of them:

import re

LOG = """UVM_INFO top.sv(142) @ 0: uvm_test_top [CFG] test=smoke_rw seed=1839274
UVM_ERROR drv.sv(88) @ 12500: write to 0x40001000 returned 0xdeadbeef
UVM_INFO top.sv(210) @ 99000: uvm_test_top [CFG] test=burst_wr seed=771002
UVM_WARNING mon.sv(51) @ 99300: late sample ignored
UVM_INFO top.sv(142) @ 120000: uvm_test_top [CFG] test=random_rw seed=904113
"""

CFG = r"test=(\w+) seed=(\d+)"

print(f"{'test':<12} {'seed':>9}")
for line in LOG.splitlines():
    m = re.search(CFG, line)
    if m is None:
        continue
    print(f"{m.group(1):<12} {int(m.group(2)):>9}")

test              seed
smoke_rw       1839274
burst_wr        771002
random_rw       904113

Five lines in, three rows out, no crash. The if m is None: continue is doing the filtering that Chapters 19.1 to 19.7 did with bool(...) — except now the same call does both jobs, which is why search plus a None check is the shape you will write a thousand times.

Report what does not match — the right choice when every line should parse, and one that does not means the tool changed its format:

import re

ROWS = [
    "slack (VIOLATED)   -0.142",
    "slack (MET)         0.031",
    "slack -- no path --",
]

for row in ROWS:
    m = re.search(r"(-?\d+\.\d+)", row)
    if m is None:
        print(f"UNPARSED: {row!r}")
    else:
        print(f"{float(m.group(1)):>8.3f}")

  -0.142
   0.031
UNPARSED: 'slack -- no path --'

Both shapes are three lines. Neither is clever. The difference between them is a decision about the data — is a non-matching line expected or is it a surprise? — and Chapter 17.7 called that deciding what failure means. A script that silently skips a line the tool renamed will report a clean run on a report it could not read.

7. What This Buys: Three Fields From One Pass

With groups available, a filter becomes a report. Each line is searched once and yields every field the summary needs:

import re

LOG = """UVM_INFO top.sv(142) @ 0: uvm_test_top [CFG] test=smoke_rw seed=1839274
UVM_ERROR drv.sv(88) @ 12500: uvm_test_top.env.agt.drv [WR] bad handshake
UVM_WARNING mon.sv(51) @ 99300: uvm_test_top.env.agt.mon [SMP] late sample
UVM_ERROR drv.sv(104) @ 141900: uvm_test_top.env.agt.drv [WR] timeout
"""

ISSUE = r"UVM_(ERROR|WARNING) (\w+\.sv)\((\d+)\) @ (\d+):"

print(f"{'sev':<8} {'file':<10} {'line':>5} {'cycle':>8}")
print("-" * 34)
for line in LOG.splitlines():
    m = re.search(ISSUE, line)
    if m is None:
        continue
    sev, src, num, cycle = m.groups()
    print(f"{sev:<8} {src:<10} {int(num):>5} {int(cycle):>8}")

sev      file        line    cycle
----------------------------------
ERROR    drv.sv        88    12500
WARNING  mon.sv        51    99300
ERROR    drv.sv       104   141900

Three things in that script are worth naming.

m.groups() unpacked straight into four variables is the tidiest way to use a multi-group pattern, and it is Chapter 6.5's tuple unpacking with no regex-specific rule at all. It also fails loudly if you change the pattern's group count and forget to change the unpacking — a ValueError rather than a silently shifted field.

The pattern contains UVM_(ERROR|WARNING), which uses one construct this module has not covered yet. Chapter 19.10 explains it; for now read it as either word here, and note that it is captured like any other group because it sits in parentheses.

The UVM_INFO line was not matched and not reported, by design. That is the first guard shape doing its job, and the table has three rows because the log had three issues — not because a pattern happened to fit.

8. Predict, Run, Observe

Work through this before reading the output. Four of the five lines depend on something in this chapter rather than on the pattern.

import re

LINE = "resolution: seed=9 retries=10 slack=-0.5"

m = re.search(r"(seed)=(\d+)", LINE)

# A
print("A:", m.groups())

# B
print("B:", repr(m.group(0)))

# C
print("C:", m.span(2))

# D
m2 = re.search(r"(sl(ack))=(\S+)", LINE)
print("D:", m2.groups())

# E
print("E:", re.search(r"cycles=(\d+)", LINE))

The verified output:

A: ('seed', '9')
B: 'seed=9'
C: (17, 18)
D: ('slack', 'ack', '-0.5')
E: None

A — two groups, so groups() has two elements, and both are strings. '9' has quotes around it in a tuple repr for a reason.

B — group 0 is the whole match, seed=9, not just the second group.

C — group 2 is the single character 9, so the span is one wide. Section 2's rule with the smallest possible example.

D — numbered by opening parenthesis: outer first (slack), inner second (ack), then the value. The inner group captured a fragment nobody wanted, which is what happens when parentheses are used for grouping rather than for capturing — Chapter 19.12 shows the form that avoids it.

E — no match, so None, and print is happy to display it. One .group(1) away from Section 6's traceback.

9. Exercises

Exercise 1 — one pair of parentheses. Take r"@ \d+:" and add a group so that m.group(1) is the cycle number alone. Print the group and its type name.

Exercise 2 — group 0 against group 1. For r"seed=(\d+)" on a real UVM line, print group(0), group(1) and both spans. Say in one sentence what the five-character difference between the two spans is.

Exercise 3 — the string result. Capture two numbers from one line where the shorter one is numerically larger, and show that > gives different answers before and after int().

Exercise 4 — three fields, one pass. Write a pattern with three groups that captures severity, source file and line number from UVM_ERROR drv.sv(88) @ 12500: ..., and print them as one aligned row.

Exercise 5 — renumbering. Take your Exercise 4 pattern and add a group at its left end. Say what every existing group(n) in your code now means, without running it, then run it and check.

Exercise 6 — nested. Write a pattern that captures both uvm_test_top.env.agt.drv and the final drv from one match. Print the two spans and state which contains which.

Exercise 7 — both guards. Parse a four-line extract where one line has no seed. Write it once with if m is None: continue and once with an UNPARSED: report, and say which you would ship for a nightly regression and why.

Exercise 8 — the unguarded crash. Reproduce the AttributeError from Section 6 deliberately. Record the message and say which object was None and why.

Exercise 9 — group that does not exist. Ask for group(3) from a two-group pattern. Record the error and say how its message differs from Exercise 8's, and what each one tells you about where the bug is.

Exercise 10 — unpack. Rewrite Exercise 4 to use tuple unpacking from m.groups() instead of numbered group() calls. Then change the pattern to have four groups without changing the unpacking, and record what fails.

10. Summary

• Parentheses mean remember what matched here. They change what you can ask about a match, never whether the pattern matches.
• group(0) is the whole match and always exists; group(1) is the first captured group, which is why numbering starts at 1.
• groups() returns every captured group as a tuple and excludes group 0.
• span(n) gives the half-open range of group n, which is what you need to edit a line rather than read it.
• Every group is a string, even when the pattern was \d+ — convert with int() or float() at the point of extraction.
• "9" > "10" is True, so an unconverted group reaches a comparison and gives a wrong answer with no error.
• Groups are numbered left to right by opening parenthesis, and nesting does not change that rule.
• Adding a group on the left renumbers everything to its right, which is how a long pattern becomes unmaintainable.
• A group number the pattern does not have raises IndexError: no such group — a bug in your code.
• re.search returns a match object or None, and None.group(1) raises AttributeError — the most common crash in hand-written flow scripts.
• Two guard shapes: skip with if m is None: continue when non-matching lines are expected, report when they are a surprise.
• Never write re.search(...).group(1) in a script — it removes the only place the check could go.

The next chapter replaces group(3) with a name, which is the difference between a pattern you wrote and a pattern someone else can maintain.