Extracting the Test Name and Seed

Part II · Module 20 · Chapter 20.8 · Extracting the Test Name and Seed

1. Why Not the Directory Name

A regression sweep produces directories that look like this:

runs/axi_smoke_test__1839274/sim.log
runs/axi_burst_test__771002/sim.log

Both facts are right there in the path, and splitting on __ is two lines. It also stops working the moment anything touches the directory — and in a real flow, several things do:

• a cleanup script archives last week's sweep into one tarball with flattened names
• an engineer copies the failing log alone into a bug report
• a rerun under runs/retry/ keeps the log and loses the name
• a different sweep uses runs/0001/, because it indexes rather than names

Every one of those leaves the log intact and the path useless. The log is the artefact that travels with the failure; the directory is context that only exists where the run happened. Reading the facts from the log costs two patterns and survives all four.

That is not an argument against using the path when it is there. It is an argument for the log being the source of truth, and Chapter 20.16 records both.

2. Both Facts, From the Header

A simulator writes its command line and its seed before the test starts:

xcelium> run
Loading snapshot worklib.tb_top:sv ....................... Done
xcelium: *N,RUNCMD: run -f cmd.f +UVM_TESTNAME=axi_smoke_test +UVM_VERBOSITY=UVM_LOW
Simulation random seed = 1839274
UVM_INFO @ 0: reporter [RNTST] Running test axi_smoke_test...
UVM_ERROR drv.sv(88) @ 141900: uvm_test_top.env.agt.drv [WR] bad handshake

import re

with open("sim.log", encoding="utf-8") as handle:
    header = handle.read()

m = re.search(r"\+UVM_TESTNAME=(\w+)", header)
s = re.search(r"random seed\s*=\s*(\d+)", header)

print("test:", m.group(1) if m else None)
print("seed:", int(s.group(1)) if s else None)

test: axi_smoke_test
seed: 1839274

That is the canonical annotation, run. Two things in it are deliberate.

\+ is escaped. A + is a quantifier — Chapter 19.6 — so an unescaped \+UVM_TESTNAME would mean one or more of the previous thing, which here is nothing at all and is a pattern error. Chapter 19.3's rule, on a character that appears in every UVM plusarg.

\s*=\s* rather than =. Chapter 19.12 Stage 3's argument: the tool aligns, and a literal space fails on the release that pads differently.

And int(s.group(1)) converts at the boundary, because a seed that stays a string sorts wrong and cannot be compared with the one in a results table.

3. The Log Names the Test Twice

A UVM log carries the test name in two independent places: the simulator's command line, and the report server's own announcement.

xcelium: *N,RUNCMD: run -f cmd.f +UVM_TESTNAME=axi_smoke_test +UVM_VERBOSITY=UVM_LOW
Simulation random seed = 1839274
UVM_INFO @ 0: reporter [RNTST] Running test axi_smoke_test...
UVM_ERROR drv.sv(88) @ 141900: uvm_test_top.env.agt.drv [WR] bad handshake

import re

PLUSARG = re.compile(r"\+UVM_TESTNAME=(?P<test>\w+)")
BANNER = re.compile(r"\[RNTST\] Running test (?P<test>\w+)")

with open("sim.log", encoding="utf-8") as handle:
    header = handle.read()

asked = PLUSARG.search(header)
ran = BANNER.search(header)

print("command line :", asked["test"] if asked else None)
print("report server:", ran["test"] if ran else None)
print("agree        :", bool(asked and ran and asked["test"] == ran["test"]))

command line : axi_smoke_test
report server: axi_smoke_test
agree        : True

Normally they agree and the second is redundant. When they disagree, something specific and nasty has happened: the command line asked for one test and the testbench built another. That happens for real — a stale cmd.f, a factory override left in a base test, a typo in a test name that fell back to a default. The run is not the run you think it is, and every conclusion drawn from it is about the wrong test.

Checking costs one extra pattern and one comparison, and it is the same instinct as Chapter 20.7 Section 6: when the data states a fact twice, compare the two statements.

4. The Seed Is Spelled Differently by Every Simulator

The test name is a UVM plusarg, so it is the same string in every tool. The seed is not — it is the simulator's own, and each vendor names it:

xcelium> run
xcelium: *N,RUNCMD: run -f cmd.f +UVM_TESTNAME=axi_smoke_test
Simulation random seed = 1839274
UVM_INFO @ 0: reporter [RNTST] Running test axi_smoke_test...

Chronologic VCS simulator copyright 1991-2024
Command: ./simv +UVM_TESTNAME=axi_burst_test +ntb_random_seed=771002
UVM_INFO @ 0: reporter [RNTST] Running test axi_burst_test...

import re

TESTNAME = re.compile(r"\+UVM_TESTNAME=(\w+)")

SEED_PATTERNS = [
    re.compile(r"random seed\s*=\s*(\d+)"),
    re.compile(r"\+ntb_random_seed=(\d+)"),
    re.compile(r"\+svseed=(\d+)"),
]


def find_seed(text):
    """The first seed any known simulator spelling turns up, or None."""
    for pattern in SEED_PATTERNS:
        m = pattern.search(text)
        if m is not None:
            return int(m.group(1))
    return None


for path in ["xcelium.log", "vcs.log"]:
    with open(path, encoding="utf-8") as handle:
        header = handle.read()
    t = TESTNAME.search(header)
    print(f"{path:<12} test={t.group(1) if t else None:<16} seed={find_seed(header)}")

xcelium.log  test=axi_smoke_test   seed=1839274
vcs.log      test=axi_burst_test   seed=771002

A list of patterns, tried in order is the right shape here, and it is Chapter 19.18's "keep a sample from each tool and version" showing up in production code rather than in a test. Each entry is one tool's spelling, each is independently testable, and adding a fourth simulator is one line.

What it is not is one pattern with three alternatives glued together. (?:random seed\s*=\s*|\+ntb_random_seed=|\+svseed=)(\d+) works and is worse: it cannot tell you which spelling matched, it is harder to test a single tool's format against, and Chapter 19.10 Section 4's ordering rule starts to matter the moment two spellings share a prefix.

5. Stop at the Header

Both facts are in the first few lines, and a parser that reads a 20 GB log looking for them is doing Chapter 10.7's mistake deliberately. Read until the test actually starts, then stop:

xcelium> run
Loading snapshot worklib.tb_top:sv ....................... Done
xcelium: *N,RUNCMD: run -f cmd.f +UVM_TESTNAME=axi_smoke_test
Simulation random seed = 1839274
UVM_INFO @ 0: reporter [RNTST] Running test axi_smoke_test...
UVM_INFO tb_top.sv(142) @ 0: uvm_test_top [CFG] clk period 2000 ps
UVM_ERROR drv.sv(88) @ 141900: uvm_test_top.env.agt.drv [WR] bad handshake
UVM_INFO seq.sv(140) @ 200000: uvm_test_top.env.agt.seqr [SEQ] done

import re

TESTNAME = re.compile(r"\+UVM_TESTNAME=(\w+)")
SEED = re.compile(r"random seed\s*=\s*(\d+)")
BANNER = re.compile(r"\[RNTST\] Running test (\w+)")

test = None
seed = None
read = 0

with open("sim.log", encoding="utf-8") as handle:
    for line in handle:
        read += 1
        if test is None:
            m = TESTNAME.search(line) or BANNER.search(line)
            if m is not None:
                test = m.group(1)
        if seed is None:
            m = SEED.search(line)
            if m is not None:
                seed = int(m.group(1))
        if test is not None and seed is not None:
            break

print("test       :", test)
print("seed       :", seed)
print("lines read :", read)

test       : axi_smoke_test
seed       : 1839274
lines read : 4

Four lines read out of eight, and it would be four out of twenty million. The break fires as soon as both facts are known — Chapter 10.7's "stop a 20 GB scan at the first fatal", applied to a header field instead.

Two details worth naming. TESTNAME.search(line) or BANNER.search(line) uses the command line when it is there and falls back to the banner when it is not, which is Chapter 9.4's short-circuit or returning the first truthy value — and a match object is truthy. And if test is None before each search means neither pattern runs again once its fact is found, which on a long header is most of the saving.

6. When They Are Not There

Some logs genuinely lack them: a truncated run that died before the banner, a directly-invoked testbench with no plusarg, a tool whose seed spelling nobody has added yet.

UVM_INFO tb_top.sv(142) @ 0: uvm_test_top [CFG] clk period 2000 ps
UVM_ERROR drv.sv(88) @ 141900: uvm_test_top.env.agt.drv [WR] bad handshake

import re

TESTNAME = re.compile(r"\+UVM_TESTNAME=(\w+)")
SEED = re.compile(r"random seed\s*=\s*(\d+)")

with open("headless.log", encoding="utf-8") as handle:
    header = handle.read()

t = TESTNAME.search(header)
s = SEED.search(header)

record = {
    "test": t.group(1) if t else None,
    "seed": int(s.group(1)) if s else None,
}

print("record:", record)
print("reproducible:", record["test"] is not None and record["seed"] is not None)

name = record["test"] or "unknown"
seed = record["seed"]
print("report row  :", f"{name:<16} seed={seed if seed is not None else '-':>8}")

record: {'test': None, 'seed': None}
reproducible: False
report row  : unknown          seed=       -

None, not a guess. The two tempting defaults are both worse than nothing:

seed = 0 is a real seed. A results table containing it says the test ran with seed zero, and somebody will eventually try to reproduce a failure with it and get a passing run.

seed = "unknown" is a string in an integer column. It survives printing and fails at the first comparison or int(), usually inside a report two modules away.

None is the honest value, and the or "unknown" belongs in the formatting, where it cannot be mistaken for data. That is Chapter 19.10 Section 6's rule — or for display, is not None for the question — and Chapter 20.6's contract said the key is always present while the value may legitimately be absent.

reproducible as its own boolean is worth having. A regression report that cannot reproduce a failure should say so next to it, rather than printing a row that looks complete.

7. What It Is For

The two facts exist to produce one line:

record = {"test": "axi_smoke_test", "seed": 1839274, "sim": "xcelium"}

if record["test"] is not None and record["seed"] is not None:
    print(f"reproduce: run_sim.py --test {record['test']} --seed {record['seed']}")
else:
    print("reproduce: not possible — test or seed missing from the log")

reproduce: run_sim.py --test axi_smoke_test --seed 1839274

A failure report that includes the command to reproduce it is a different artefact from one that does not. The engineer who picks it up at 9am does not have to find the sweep, work out the naming convention, or ask who ran it — and that is the entire return on the two patterns in this chapter.

Chapter 20.16 puts test and seed into the record; Chapter 25.4 is where run_sim.py becomes real.

8. Predict, Run, Observe

import re

HEADER = ("Command: ./simv +UVM_TESTNAME=axi_burst_test +ntb_random_seed=771002\n"
          "Simulation random seed = 1839274\n"
          "UVM_INFO @ 0: reporter [RNTST] Running test axi_smoke_test...\n")

# A
print("A:", re.search(r"\+UVM_TESTNAME=(\w+)", HEADER).group(1))

# B
print("B:", re.search(r"\[RNTST\] Running test (\w+)", HEADER).group(1))

# C
print("C:", re.search(r"random seed\s*=\s*(\d+)", HEADER).group(1))

# D
print("D:", re.findall(r"random seed\s*=\s*(\d+)|\+ntb_random_seed=(\d+)", HEADER))

# E
print("E:", type(re.search(r"\+ntb_random_seed=(\d+)", HEADER).group(1)).__name__)

# F
print("F:", re.search(r"\+UVM_TESTNAME=(\w+)", "no plusarg here"))

# G
print("G:", (re.search(r"\+UVM_TESTNAME=(\w+)", "nothing") or
             re.search(r"Running test (\w+)", HEADER)).group(1))

The verified output:

A: axi_burst_test
B: axi_smoke_test
C: 1839274
D: [('', '771002'), ('1839274', '')]
E: str
F: None
G: axi_smoke_test

A and B — the two sources disagree on this header. That is the Section 3 bug, visible in two lines of output: the command line asked for axi_burst_test and the report server announced axi_smoke_test.

C — the xcelium spelling matched, even though a vcs-style seed appears first in the text. A pattern finds its own spelling wherever it is.

D — findall with two groups returns tuples with an empty string for the non-participating one, which is Chapter 19.11 Section 5's rule. This is why a list of separate patterns reads better than one alternation.

E — the group is a str. int() is still yours.

F — no match is None, which is what Section 6 stores rather than a default.

G — or between two searches returns the first truthy match object, which is the fallback in Section 5.

9. Exercises

Exercise 1 — the four ways a path is lost. Name a cleanup, archive or rerun step in your own flow that would destroy the run-directory name while keeping the log.

Exercise 2 — both facts. Extract the test name and the seed from a log header of your own. Convert the seed at the point of extraction.

Exercise 3 — the escaped plus. Write the plusarg pattern without escaping the +. Record what happens and say which chapter's rule you broke.

Exercise 4 — the alignment. Replace \s*=\s* with a literal = and run it against a header that pads differently. Record the result.

Exercise 5 — two sources. Extract the test name from both the command line and the RNTST banner. Build a header where they disagree and report the disagreement.

Exercise 6 — what disagreement means. Describe one flow mistake that would produce two different test names in one log, and say what a report should do about it.

Exercise 7 — a list of seed patterns. Write patterns for three simulator spellings and try them in order. Add a fourth with one line.

Exercise 8 — one pattern instead. Write the same thing as a single alternation. List two things you lose.

Exercise 9 — stop early. Count the lines your header scan reads before it has both facts. Say what that number would be on a twenty-million-line log.

Exercise 10 — absent. Run your extractor on a log with no header. Confirm both values are None and that nothing raises.

Exercise 11 — the two bad defaults. Say exactly what goes wrong, and when, if a missing seed becomes 0, and if it becomes the string "unknown".

Exercise 12 — the reproduce line. Print a reproduce command when both facts are present and an honest refusal when they are not.

10. Summary

• Both facts are usually in the directory name, and that is why reading them from there fails — archives, copies, reruns and indexed sweeps all keep the log and lose the path.
• The log travels with the failure. The path is context that exists only where the run happened.
• \+UVM_TESTNAME=(\w+) needs the + escaped; unescaped it is a quantifier.
• \s*=\s*, never a literal = — the tool aligns and the next release may pad differently.
• Convert the seed at the boundary. A seed left as a string sorts wrong and cannot be compared.
• A UVM log names the test twice: the +UVM_TESTNAME plusarg and the report server's RNTST banner.
• When the two disagree, the command line asked for one test and the testbench ran another — a stale filelist, a factory override, a typo falling back to a default.
• When the data states a fact twice, compare the two statements. Same instinct as Chapter 20.7's self-check.
• The test name is methodology and the seed is the simulator's, so one pattern covers every tool for the name and a list of spellings is needed for the seed.
• A list of patterns beats one alternation: it says which spelling matched, it is testable per tool, and it avoids Chapter 19.10's prefix-ordering rule.
• Stop reading once both facts are known — three lines instead of twenty million.
• Absent is None, never 0 and never "unknown". One is a real seed; the other is a string in an integer column.
• or "unknown" belongs in the formatting, where it cannot be mistaken for data.
• A reproducible boolean is worth carrying, so a report that cannot reproduce a failure says so rather than looking complete.

The next chapter pulls the same kind of structure out of a different message entirely: an SVA assertion failure, which names the property, the time and the instance path that broke.