Assertion Failures, Times and Hierarchy Paths

Part II · Module 20 · Chapter 20.9 · Assertion Failures, Times and Hierarchy Paths

1. The Run That Passes and Is Broken

UVM_INFO @ 0: reporter [RNTST] Running test axi_smoke_test...
UVM_INFO tb_top.sv(142) @ 0: uvm_test_top [CFG] clk period 2000 ps
Assertion tb_top.u_dut.u_fifo.no_overflow_a failed at time 171200
    File: fifo.sv, Line: 64
UVM_INFO scb.sv(120) @ 180000: uvm_test_top.env.scb [CHK] 160 compared
Assertion tb_top.u_dut.u_arb.grant_onehot_a failed at time 182400
--- UVM Report Summary ---
UVM_INFO :    3
UVM_WARNING :    0
UVM_ERROR :    0
UVM_FATAL :    0
TEST DONE

import re

UVM_FAIL = re.compile(r"^UVM_(ERROR|FATAL)\b.*@\s+\d+:")
SVA = re.compile(r"Assertion (?P<name>[\w.]+) failed at time (?P<t>\d+)")

with open("asrt.log", encoding="utf-8") as handle:
    text = handle.read()

uvm_errors = sum(1 for line in text.splitlines() if UVM_FAIL.match(line))
assertions = SVA.findall(text)

print("UVM errors     :", uvm_errors)
print("SVA failures   :", len(assertions))
print("summary says   : UVM_ERROR : 0")
print()
print("verdict on UVM severities alone:", "PASS" if uvm_errors == 0 else "FAIL")
print("verdict including assertions   :", "PASS" if uvm_errors == 0 and not assertions else "FAIL")

UVM errors     : 0
SVA failures   : 2
summary says   : UVM_ERROR : 0

verdict on UVM severities alone: PASS
verdict including assertions   : FAIL

Read the three facts at the top together, because they are all true at once.

There really are zero UVM errors. The report summary block really does say UVM_ERROR : 0, so Chapter 20.7's self-check compares its count against the simulator's and they agree perfectly — the parser is working exactly as designed and the run is broken.

Two properties were violated in the RTL. A FIFO overflowed and an arbiter granted two requesters at once, and the checker everything up to Chapter 20.8 built reports PASS.

The reason is structural rather than a gap in the patterns. An SVA failure is reported by the simulator, not by the testbench. The UVM report server never sees it, so it has no UVM_ severity, no @ <time>: in the UVM format, and no entry in the tallies the report server prints. Nothing Chapter 20.7 wrote could have matched it.

2. The Pattern

UVM_INFO @ 0: reporter [RNTST] Running test axi_smoke_test...
Assertion tb_top.u_dut.u_fifo.no_overflow_a failed at time 171200
    File: fifo.sv, Line: 64
Assertion tb_top.u_dut.u_arb.grant_onehot_a failed at time 182400
TEST DONE

import re

SVA = re.compile(r"Assertion (?P<name>[\w.]+) failed at time (?P<t>\d+)")

with open("asrt.log", encoding="utf-8") as handle:
    for line in handle:
        m = SVA.search(line)
        if m is None:
            continue
        print(f"t={int(m['t']):>7}  {m['name']}")

t= 171200  tb_top.u_dut.u_fifo.no_overflow_a
t= 182400  tb_top.u_dut.u_arb.grant_onehot_a

The canonical annotation, with two details worth naming.

[\w.]+ and not \w+. An assertion name arrives fully qualified, and \w excludes the dot — Chapter 19.4 Section 3's fact. With \w+ the group stops at tb_top, the rest of the pattern then demands failed at time and finds .u_dut, and the whole match fails. The result is not a wrong name: it is None, and a real assertion failure that is never counted at all.

search, not match. Some simulators indent the line or prefix it; anchoring at position 0 would drop those. Chapter 19.11 Section 3's distinction, and here the looser choice is the right one because the Assertion ... failed at time phrase is specific enough on its own.

3. The Name Is Already a Path

tb_top.u_dut.u_fifo.no_overflow_a is a hierarchy path and a property name joined by a dot, and separating them needs no regex at all:

NAMES = [
    "tb_top.u_dut.u_fifo.no_overflow_a",
    "tb_top.u_dut.u_arb.grant_onehot_a",
    "no_scope_a",
]

for name in NAMES:
    scope, dot, prop = name.rpartition(".")
    print(f"{prop:<18} scope={scope or '(top level)'}  found_dot={dot == '.'}")

no_overflow_a      scope=tb_top.u_dut.u_fifo  found_dot=True
grant_onehot_a     scope=tb_top.u_dut.u_arb  found_dot=True
no_scope_a         scope=(top level)  found_dot=False

rpartition splits at the last dot, which is the one separating the instance path from the property — Chapter 19.19 Section 3's argument arriving inside a regex module: the field is already delimited, so a string method beats a pattern. And the third row shows why rpartition rather than rsplit or a slice: a name with no dot returns the whole string as the property and an empty scope, with nothing raised.

That split is what makes the output actionable. The property name says what rule was broken; the scope says which instance broke it. A report with both points an engineer at one module, and a report with only the full dotted string makes them read it.

4. A Record Per Failure

UVM_INFO @ 0: reporter [RNTST] Running test axi_smoke_test...
Assertion tb_top.u_dut.u_fifo.no_overflow_a failed at time 171200
    File: fifo.sv, Line: 64
UVM_INFO scb.sv(120) @ 180000: uvm_test_top.env.scb [CHK] 160 compared
Assertion tb_top.u_dut.u_arb.grant_onehot_a failed at time 182400
Assertion tb_top.u_dut.u_fifo.no_overflow_a failed at time 190100
TEST DONE

import re

SVA = re.compile(r"Assertion (?P<name>[\w.]+) failed at time (?P<t>\d+)")

failures = []
with open("asrt.log", encoding="utf-8") as handle:
    for line in handle:
        m = SVA.search(line)
        if m is None:
            continue
        scope, _, prop = m["name"].rpartition(".")
        failures.append({"property": prop, "scope": scope, "time": int(m["t"])})

print(f"{'time':>8}  {'property':<18} scope")
for f in failures:
    print(f"{f['time']:>8}  {f['property']:<18} {f['scope']}")

by_property = {}
for f in failures:
    by_property[f["property"]] = by_property.get(f["property"], 0) + 1

print()
print("distinct properties:", len(by_property))
for prop in sorted(by_property):
    print(f"  {prop:<18} {by_property[prop]}")
print("first failure at   :", min(f["time"] for f in failures))

    time  property           scope
  171200  no_overflow_a      tb_top.u_dut.u_fifo
  182400  grant_onehot_a     tb_top.u_dut.u_arb
  190100  no_overflow_a      tb_top.u_dut.u_fifo

distinct properties: 2
  grant_onehot_a     1
  no_overflow_a      2
first failure at   : 171200

Three failures, two distinct properties. That distinction matters for triage: a property that fires once is a specific event; one that fires repeatedly is usually a condition that stayed broken, and the interesting one is the first occurrence. Grouping by property and reporting the earliest time is a better summary than a list of three.

min(f["time"] for f in failures) works because the times are integers. Had int() been skipped, min would compare strings and "190100" would sort before "99300" — Chapter 19.8 Section 3, costing the wrong answer rather than an error.

Note sorted(by_property) in the loop. Dict iteration order is insertion order and that is deterministic here, but sorting makes the report stable regardless of what order the failures happened to arrive in, which is what a diff between two nights needs.

5. Every Simulator Spells It Differently

As with the seed in Chapter 20.8, the format is the tool's, not the methodology's:

import re

SVA_PATTERNS = [
    re.compile(r"Assertion (?P<name>[\w.]+) failed at time (?P<t>\d+)"),
    re.compile(r'"(?P<file>[\w.]+)", (?P<line>\d+): (?P<name>[\w.]+): '
               r'started at \d+\w* failed at (?P<t>\d+)'),
]

SAMPLES = [
    "Assertion tb_top.u_dut.u_fifo.no_overflow_a failed at time 171200",
    '"fifo.sv", 64: tb_top.u_dut.u_fifo.no_overflow_a: started at 171200s failed at 171200s',
    "UVM_INFO scb.sv(120) @ 180000: uvm_test_top.env.scb [CHK] 160 compared",
]


def parse_assertion(line):
    """The first known spelling that matches, as (name, time), or None."""
    for pattern in SVA_PATTERNS:
        m = pattern.search(line)
        if m is not None:
            return m["name"], int(m["t"])
    return None


for sample in SAMPLES:
    result = parse_assertion(sample)
    print(f"{sample[:46]!r:<50} {result}")

'Assertion tb_top.u_dut.u_fifo.no_overflow_a fa'   ('tb_top.u_dut.u_fifo.no_overflow_a', 171200)
'"fifo.sv", 64: tb_top.u_dut.u_fifo.no_overflow'   ('tb_top.u_dut.u_fifo.no_overflow_a', 171200)
'UVM_INFO scb.sv(120) @ 180000: uvm_test_top.en'   None

Two tools, two spellings, one function and the same tuple out of both. The UVM line is correctly rejected.

The second pattern illustrates something the first does not: it carries the source file and line of the property, which the first spelling does not provide and the log's indented File: fifo.sv, Line: 64 continuation line does. Getting that field means either parsing the continuation — which needs the state Chapter 20.15 introduces — or accepting that it is tool-dependent and leaving it out of the record. Chapter 20.16 leaves it out, and says so.

6. Folding It Into the Verdict

The parser now has two independent sources of failure, and the verdict has to consider both:

UVM_INFO @ 0: reporter [RNTST] Running test axi_smoke_test...
Assertion tb_top.u_dut.u_fifo.no_overflow_a failed at time 171200
UVM_INFO scb.sv(120) @ 180000: uvm_test_top.env.scb [CHK] 160 compared
Assertion tb_top.u_dut.u_arb.grant_onehot_a failed at time 182400
--- UVM Report Summary ---
UVM_INFO :    2
UVM_WARNING :    0
UVM_ERROR :    0
UVM_FATAL :    0
TEST DONE

import re

UVM_FAIL = re.compile(r"^UVM_(?P<sev>ERROR|FATAL)\b.*@\s+(?P<t>\d+):")
SVA = re.compile(r"Assertion (?P<name>[\w.]+) failed at time (?P<t>\d+)")

record = {"errors": 0, "fatals": 0, "assertions": 0,
          "first_failure_time": None, "first_failure_kind": None}


def note_first(record, time, kind):
    """Keep the earliest failure of any kind."""
    if record["first_failure_time"] is None or time < record["first_failure_time"]:
        record["first_failure_time"] = time
        record["first_failure_kind"] = kind


with open("asrt.log", encoding="utf-8") as handle:
    for line in handle:
        m = UVM_FAIL.match(line)
        if m is not None:
            key = "errors" if m["sev"] == "ERROR" else "fatals"
            record[key] += 1
            note_first(record, int(m["t"]), "UVM_" + m["sev"])
            continue
        a = SVA.search(line)
        if a is not None:
            record["assertions"] += 1
            note_first(record, int(a["t"]), "assertion")

failed = record["errors"] or record["fatals"] or record["assertions"]
print("record :", record)
print("verdict:", "FAIL" if failed else "PASS")

record : {'errors': 0, 'fatals': 0, 'assertions': 2, 'first_failure_time': 171200, 'first_failure_kind': 'assertion'}
verdict: FAIL

assertions is a third counter alongside errors and fatals, present always — Chapter 20.6's rule — and the verdict is an or across all three rather than a test on errors.

note_first compares times, not positions in the file. That is a deliberate change from Chapter 20.7, where first-by-position and first-by-time were the same thing because every line came from one ordered stream. They are no longer guaranteed to be: the simulator's assertion reporter and the UVM report server are different writers, and on a buffered or multi-threaded run their lines can interleave out of time order. Comparing the time makes the answer correct either way.

record["first_failure_kind"] says which kind won, which is the difference between "go and look at the testbench" and "go and look at the RTL".

7. Predict, Run, Observe

import re

SVA = re.compile(r"Assertion (?P<name>[\w.]+) failed at time (?P<t>\d+)")

LINE = "Assertion tb_top.u_dut.u_fifo.no_overflow_a failed at time 171200"
NARROW = re.compile(r"Assertion (?P<name>\w+) failed at time (?P<t>\d+)")

# A
print("A:", SVA.search(LINE)["name"])

# B
print("B:", NARROW.search(LINE))

# C
print("C:", "tb_top.u_dut.u_fifo.no_overflow_a".rpartition("."))

# D
print("D:", "no_scope_a".rpartition("."))

# E
print("E:", type(SVA.search(LINE)["t"]).__name__)

# F
print("F:", min(["190100", "99300"]), min([190100, 99300]))

# G
print("G:", SVA.match("    Assertion a.b failed at time 10"),
      bool(SVA.search("    Assertion a.b failed at time 10")))

The verified output:

A: tb_top.u_dut.u_fifo.no_overflow_a
B: None
C: ('tb_top.u_dut.u_fifo', '.', 'no_overflow_a')
D: ('', '', 'no_scope_a')
E: str
F: 190100 99300
G: None True

A — [\w.]+ captures the whole qualified name.

B — \w+ cannot reach failed at time, because after tb_top the subject continues .u_dut. The whole match fails and the result is None: not a wrong name, but a failure that is silently never counted.

C — rpartition gives scope, separator and property.

D — with no dot, the scope is empty and the property is the whole string. Nothing raises.

E — the time is a string until you convert it.

F — as text, "190100" is less than "99300"; as integers it is not. An unconverted time makes min report the wrong first failure.

G — an indented line is invisible to match and found by search.

8. Exercises

Exercise 1 — the passing broken run. Build a log with zero UVM errors, a clean summary block and one failed assertion. Run a UVM-severity-only checker against it and record the verdict.

Exercise 2 — why the self-check agrees. Explain in two sentences why Chapter 20.7's count-versus-summary comparison does not catch this.

Exercise 3 — ask the question. Find out whether your own testbench binds assertion failures into UVM. Say how you found out, and what your parser should do if the answer differs per block.

Exercise 4 — the dotted name. Extract an assertion name with \w+ and with [\w.]+. Record what the first one returns and say which of this module's failure classes it belongs to.

Exercise 5 — match or search. Run your pattern against an indented assertion line with both. Say which you would ship.

Exercise 6 — split the name. Use rpartition to separate scope from property, including a name with no dot. Say why rpartition beats split(".")[-1] here.

Exercise 7 — distinct properties. Parse a log with the same property failing three times and two others once each. Report distinct properties and the earliest time for each.

Exercise 8 — unconverted times. Compute min over the times as strings and as integers on data where they differ. Record both answers.

Exercise 9 — two spellings. Write patterns for two simulator formats and one function that tries both. Confirm a UVM line is rejected by both.

Exercise 10 — different group sets. Fold your two patterns into one alternation and say what m["file"] now means when it is None.

Exercise 11 — earliest of any kind. Write note_first and feed it a UVM error at 180000 and an assertion at 171200, in that order in the file. Confirm the assertion wins and say why comparing times rather than positions matters.

9. Summary

• An SVA failure is reported by the simulator, not the testbench — no UVM severity, no entry in the report summary.
• So a run can have zero UVM errors, a clean summary, and violated properties, and everything through Chapter 20.8 reports PASS.
• Chapter 20.7's self-check still agrees, because the parser's count and the simulator's tally are both correctly zero.
• Some testbenches bind assertions into UVM and some do not, and it varies between blocks — a parser handling both is correct either way.
• [\w.]+, not \w+. An assertion name is fully qualified and \w excludes the dot, so with \w+ the match fails entirely and the failure is never counted.
• search, not match — some simulators indent the line, and the phrase is specific enough without an anchor.
• The name is already a path. rpartition(".") splits scope from property with no regex, and handles an undotted name without raising.
• The property says what rule broke; the scope says which instance broke it. A report needs both to point at one module.
• Group by property and report the earliest time. A property firing three times is usually one condition, not three events.
• Convert the time. As text, "190100" sorts before "99300", so min returns the wrong first failure.
• The spelling is tool-specific, so a list of patterns tried in order, exactly as for Chapter 20.8's seed.
• Do not fold them into one alternation — the spellings have different group sets, so a None would mean both "this tool does not print it" and "this instance had none".
• assertions is a third always-present counter, and the verdict is an or across all three.
• Compare times, not file positions, because the assertion reporter and the report server are different writers and can interleave.

The next chapter keeps the first failure together with the lines around it, because the failing line on its own is rarely enough to start from.