Skip to content
VLSI Mentor

Python Part II · Module 29 · Coverage Automation

Coverage Metrics and Thresholds

Chapter 29.1 produced records that each name a metric, and that field has been carried along without being examined. It turns out to be the field that decides everything: the four common coverage metrics count different things, have different achievable ceilings, and support different claims about a design. A single pass mark applied to all of them is not a strict policy or a lax one — it is simultaneously both, failing designs that are fully verified while passing ones that are barely tested.

Intermediate19 min readPythonPart IIAutomationCoverageSignoff

Part II · Module 29 · Chapter 29.2 · Coverage Metrics and Thresholds

1. Four Metrics, Four Questions

The metric field on a coverage record is not a label. It says which question the number answers:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
METRICS = [
    ("statement", "was this line executed?", "the compiler", "high 90s"),
    ("branch", "was each direction of each decision taken?", "the compiler", "low 90s"),
    ("toggle", "did each bit go 0->1 and 1->0?", "the netlist", "varies wildly"),
    ("functional", "did the scenarios I described happen?", "you", "100 at signoff"),
]

print(f"{'metric':<12} {'question':<42} {'denominator by':<14} typical target")
for name, question, who, target in METRICS:
    print(f"{name:<12} {question:<42} {who:<14} {target}")

print()
print("metrics whose denominator you choose:",
      [m[0] for m in METRICS if m[2] == "you"])
print("metrics whose denominator is handed to you:",
      [m[0] for m in METRICS if m[2] != "you"])
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
metric       question                                   denominator by typical target
statement    was this line executed?                    the compiler   high 90s
branch       was each direction of each decision taken? the compiler   low 90s
toggle       did each bit go 0->1 and 1->0?             the netlist    varies wildly
functional   did the scenarios I described happen?      you            100 at signoff

metrics whose denominator you choose: ['functional']
metrics whose denominator is handed to you: ['statement', 'branch', 'toggle']

The last two lines are the division that matters, and Section 8 returns to it. Three of the four metrics have a denominator the tools compute from your code; exactly one has a denominator you wrote by hand. That difference changes what a high number means, and it is the single most misunderstood thing about coverage.

2. The Annotation

The canonical annotation for this chapter is a dictionary:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
THRESHOLDS = {"functional": 95.0, "branch": 90.0}

Unlike most annotations in Part II, the shape here is right, and it is worth saying so plainly before taking it apart. A threshold keyed by metric is the whole argument of this chapter expressed in one literal: functional and branch get different numbers because they measure different things.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
THRESHOLDS = {"functional": 95.0, "branch": 90.0}

REPORT = {"functional": 96.2, "branch": 91.4}

for metric, value in sorted(REPORT.items()):
    floor = THRESHOLDS[metric]
    print(f"{metric:<12} {value:>6.2f}  floor {floor:>6.2f}  "
          f"{'PASS' if value >= floor else 'FAIL'}")
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
branch        91.40  floor  90.00  PASS
functional    96.20  floor  95.00  PASS

Two metrics, two thresholds, two verdicts, and the gate reads correctly. Everything from Section 4 onwards is about what the dictionary leaves out.

3. Why One Number Cannot Work

Before fixing the dictionary, it is worth seeing what the alternative costs. Toggle coverage is the metric that makes a flow-wide threshold impossible, and the reason is arithmetic rather than opinion:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
SIGNALS = [
    ("addr", 32, 12, "upper 20 bits are tied off for this configuration"),
    ("data", 64, 64, "fully exercised"),
    ("status", 8, 3, "5 reserved bits never driven"),
]

total_bits = sum(w for _, w, _, _ in SIGNALS)
used_bits = sum(u for _, _, u, _ in SIGNALS)

print(f"{'signal':<9} {'width':>6} {'in use':>7} {'ceiling':>9}  why")
for name, width, used, why in SIGNALS:
    print(f"{name:<9} {width:>6} {used:>7} {100.0*used/width:>8.2f}%  {why}")

print()
print(f"bits in the netlist      : {total_bits}")
print(f"bits anything can toggle : {used_bits}")
print(f"design-wide toggle ceiling: {100.0*used_bits/total_bits:.2f}%")
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
signal     width  in use   ceiling  why
addr          32      12    37.50%  upper 20 bits are tied off for this configuration
data          64      64   100.00%  fully exercised
status         8       3    37.50%  5 reserved bits never driven

bits in the netlist      : 104
bits anything can toggle : 79
design-wide toggle ceiling: 75.96%

A perfectly verified version of this design reports 75.96% toggle coverage and cannot do better, because twenty-five bits have nothing driving them. Now run a single flow-wide threshold against a realistic set of numbers:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
REPORT = {"statement": 96.8, "branch": 92.1, "toggle": 74.0, "functional": 61.5}

for single in (90.0, 60.0):
    verdicts = {m: ("PASS" if v >= single else "FAIL") for m, v in sorted(REPORT.items())}
    wrong = []
    if verdicts["toggle"] == "FAIL" and single == 90.0:
        wrong.append("toggle, which cannot exceed 75.96")
    if verdicts["functional"] == "PASS" and single == 60.0:
        wrong.append("functional at 61.5, which is nowhere near signoff")
    print(f"single threshold {single}:")
    for m, v in verdicts.items():
        print(f"   {m:<12} {REPORT[m]:>6.2f}  {v}")
    print("   wrong verdicts:", "; ".join(wrong) or "none")
    print()
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
single threshold 90.0:
   branch        92.10  PASS
   functional    61.50  FAIL
   statement     96.80  PASS
   toggle        74.00  FAIL
   wrong verdicts: toggle, which cannot exceed 75.96

single threshold 60.0:
   branch        92.10  PASS
   functional    61.50  PASS
   statement     96.80  PASS
   toggle        74.00  PASS
   wrong verdicts: functional at 61.5, which is nowhere near signoff

90 fails a design that is fully verified. 60 passes a design that is not verified at all. There is no single number between them that is right, because the two metrics are not on the same scale — which is why the annotation's shape is correct and why the rest of this chapter is about completing it rather than replacing it.

4. The Metric With No Threshold

Now the defect. The report in a real flow carries more metrics than the dictionary mentions, and the natural way to write the lookup hides that completely:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
THRESHOLDS = {"functional": 95.0, "branch": 90.0}
REPORT = {"functional": 96.2, "branch": 91.4, "toggle": 61.0, "statement": 88.3}

failures = [m for m, v in sorted(REPORT.items()) if v < THRESHOLDS.get(m, 0)]

print("metrics in the report  :", sorted(REPORT))
print("metrics with a threshold:", sorted(THRESHOLDS))
print("gate verdict           :", failures or "PASS")
print()
print("statement at 88.3 was compared against:", THRESHOLDS.get("statement", 0))
print("toggle at 61.0 was compared against   :", THRESHOLDS.get("toggle", 0))
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
metrics in the report  : ['branch', 'functional', 'statement', 'toggle']
metrics with a threshold: ['branch', 'functional']
gate verdict           : PASS

statement at 88.3 was compared against: 0
toggle at 61.0 was compared against   : 0

The gate says PASS, and half of it never ran. THRESHOLDS.get(m, 0) turns every unmentioned metric into a comparison against zero, which no real number can fail. The dictionary did not lie; the lookup invented a policy that nobody wrote down.

This is the same class of defect as Chapter 8.3's missing-key default, and the fix is the same: a default is only safe when it is the answer you would have written deliberately. Zero is not. Here is the gate that cannot do this:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
THRESHOLDS = {"functional": 95.0, "branch": 90.0}
REPORT = {"functional": 96.2, "branch": 91.4, "toggle": 61.0, "statement": 88.3}


def gate(report, thresholds):
    """Every reported metric must have a policy. Raise rather than assume one."""
    ungoverned = sorted(set(report) - set(thresholds))
    if ungoverned:
        raise KeyError(f"no threshold for reported metric(s): {ungoverned}")
    return sorted(m for m, v in report.items() if v < thresholds[m])


try:
    gate(REPORT, THRESHOLDS)
except KeyError as exc:
    print("refused:", exc)

print()
print("the gate cannot return PASS until somebody decides about")
print("statement and toggle — which is the point.")
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
refused: "no threshold for reported metric(s): ['statement', 'toggle']"

the gate cannot return PASS until somebody decides about
statement and toggle — which is the point.

The gate now refuses to produce a verdict it cannot justify. This is Chapter 21.7's validate-before-you-run argument applied to a policy rather than a configuration: the failure arrives when somebody adds a metric to the report, which is the moment a human can still cheaply decide what it should mean.

5. Report, But Do Not Gate

Refusing every unmentioned metric is right and it is not usable on its own, because some metrics genuinely should be reported without gating — toggle during early development, or a metric whose ceiling nobody has established yet. The answer is to make that a decision rather than an omission:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
THRESHOLDS = {
    "functional": 95.0,
    "branch": 90.0,
    "statement": 90.0,
    "toggle": None,     # reported, deliberately not gated: ceiling unknown
}
REPORT = {"functional": 96.2, "branch": 91.4, "toggle": 61.0, "statement": 88.3}

ungoverned = sorted(set(REPORT) - set(THRESHOLDS))
assert not ungoverned, ungoverned

gated = {m: v for m, v in REPORT.items() if THRESHOLDS[m] is not None}
ungated = {m: v for m, v in REPORT.items() if THRESHOLDS[m] is None}
failures = sorted(m for m, v in gated.items() if v < THRESHOLDS[m])

print("gated  :", sorted(gated))
print("ungated:", sorted(ungated), "(reported for information)")
print("failures:", failures)
print()
for m in sorted(REPORT):
    floor = THRESHOLDS[m]
    mark = "—" if floor is None else f"{floor:.1f}"
    status = "info" if floor is None else ("PASS" if REPORT[m] >= floor else "FAIL")
    print(f"  {m:<12} {REPORT[m]:>6.2f}  floor {mark:>5}  {status}")
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
gated  : ['branch', 'functional', 'statement']
ungated: ['toggle'] (reported for information)
failures: ['statement']

  branch        91.40  floor  90.0  PASS
  functional    96.20  floor  95.0  PASS
  statement     88.30  floor  90.0  FAIL
  toggle        61.00  floor     —  info

None as a threshold means reported, deliberately not gated, and it is a completely different statement from a missing key. One says somebody looked at toggle coverage and decided not to gate on it yet; the other says nobody has looked. And with the policy complete, the gate found the failure the original annotation's lookup had hidden: statement coverage at 88.3 against a floor of 90.

6. Gating on a Rounded Percentage

Chapter 29.1 Section 6 argued for keeping the hits/total pair. Here is the first place it pays, and the defect is invisible in every dashboard:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
hits, total = 9496, 10000
exact = 100.0 * hits / total
printed = float(f"{exact:.1f}")

print(f"hits / total        : {hits} / {total}")
print(f"exact percentage    : {exact}")
print(f"as the report prints: {exact:.1f}")
print()
FLOOR = 95.0
print(f"floor                        : {FLOOR}")
print(f"gate on the printed value    : {'PASS' if printed >= FLOOR else 'FAIL'}  ({printed})")
print(f"gate on hits/total           : {'PASS' if exact >= FLOOR else 'FAIL'}  ({exact})")
print()
print("bins short of the floor:", total * 95 // 100 - hits)
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
hits / total        : 9496 / 10000
exact percentage    : 94.96
as the report prints: 95.0

floor                        : 95.0
gate on the printed value    : PASS  (95.0)
gate on hits/total           : FAIL  (94.96)

bins short of the floor: 4

The report rounded 94.96 up to 95.0, and a gate reading the printed string passed a run that is four bins short of the policy. Four bins is a small number and that is the point: the error is bounded by the report's own precision, so it will never be large, and it will always sit exactly where decisions are made — at the boundary. A threshold is only ever consulted near its own value.

The rule is short: gate on the pair where you have it, and record the precision you gated at where you do not.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
RECORDS = [
    ("functional", 95.0, 9496, 10000),
    ("branch", 90.0, None, None),
]

for metric, floor, hits, total in RECORDS:
    if hits is None:
        print(f"{metric:<12} gated on a printed percentage — precision 0.05 points")
    else:
        exact = 100.0 * hits / total
        need = -(-floor * total // 100)
        print(f"{metric:<12} exact {exact:.4f}  needs {int(need)} of {total}  "
              f"short by {int(need) - hits}")
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
functional   exact 94.9600  needs 9500 of 10000  short by 4
branch       gated on a printed percentage — precision 0.05 points

-(-floor * total // 100) is a ceiling division, and it answers the question an engineer actually asks when a gate fails: not what percentage am I at but how many more bins do I need. Four. That is a number somebody can act on this afternoon, and it does not exist anywhere in a percentage.

7. A Threshold Is a Policy, Not a Measurement

Everything in Sections 4 to 6 assumed the numbers 95.0 and 90.0 are correct. They are not measurements, and nothing in a coverage report can tell you what they should be:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
ORIGINS = [
    ("functional", 100.0, "signoff requires every described scenario to occur"),
    ("branch", 90.0, "agreed with the design lead; the gap is reviewed, not chased"),
    ("statement", 90.0, "same review as branch"),
    ("toggle", None, "ceiling not yet established for this configuration"),
]

print(f"{'metric':<12} {'floor':>6}  where the number comes from")
for metric, floor, why in ORIGINS:
    print(f"{metric:<12} {str(floor) if floor is not None else '—':>6}  {why}")

print()
print("thresholds that came from a measurement:", 0)
print("thresholds that came from a decision   :", sum(1 for o in ORIGINS if o[1] is not None))
print("decisions with a recorded reason       :", sum(1 for o in ORIGINS if o[2]))
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
metric        floor  where the number comes from
functional    100.0  signoff requires every described scenario to occur
branch         90.0  agreed with the design lead; the gap is reviewed, not chased
statement      90.0  same review as branch
toggle            —  ceiling not yet established for this configuration

thresholds that came from a measurement: 0
thresholds that came from a decision   : 3
decisions with a recorded reason       : 4

Not one of these numbers is derived from data, and that is the honest position. Chapter 28.6 measured a timing threshold from the flow's own run-to-run spread because timing noise is a physical property of the tool; a coverage floor is a statement about how much risk a project will accept, which no amount of running the flow can discover. Chapter 29.6 shows the one part of coverage gating that can be measured — the run-to-run spread of a random-seed regression — and it is a different question from where the floor sits.

What makes this policy usable is the fourth column. Carrying the reason beside the number turns the threshold file into the document a review actually needs, and means the next person to ask "why 90?" gets an answer instead of a shrug.

8. 100% Functional Is a Weaker Claim Than 90% Branch

Here is the thing about coverage metrics that costs teams the most, and it follows directly from Section 1's split between denominators you control and denominators handed to you:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
CLAIMS = [
    ("functional", 4, 4, "bins I wrote by hand"),
    ("branch", 360, 400, "branches the compiler found"),
]

for metric, covered, total, what in CLAIMS:
    print(f"{metric:<12} {covered:>4}/{total:<4} = {100.0*covered/total:>6.2f}%   {what}")

print()
f_cov = CLAIMS[0][1]
b_cov = CLAIMS[1][1]
print(f"things proven exercised by the 100% claim: {f_cov}")
print(f"things proven exercised by the 90% claim : {b_cov}")
print(f"ratio: {b_cov // f_cov}x in favour of the lower percentage")
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
functional      4/4    = 100.00%   bins I wrote by hand
branch        360/400  =  90.00%   branches the compiler found

things proven exercised by the 100% claim: 4
things proven exercised by the 90% claim : 360
ratio: 90x in favour of the lower percentage

100% of four bins proves four things. 90% of four hundred branches proves three hundred and sixty. The higher percentage is the weaker claim by a factor of ninety, and the reason is that functional coverage has a denominator the verification engineer wrote — so reaching 100% can be achieved either by writing more tests or by writing a smaller covergroup, and the number alone cannot distinguish those.

This is why a signoff review asks for the covergroup definitions and not just the percentage, and why Chapter 29.6's evidence record carries bin counts. A functional coverage percentage is uninterpretable without its denominator:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
QUARTERS = [
    ("Q1", 40, 40),
    ("Q2", 95, 100),
    ("Q3", 240, 260),
]

print(f"{'when':<5} {'covered':>8} {'bins':>6} {'pct':>8}   reading")
for when, covered, total in QUARTERS:
    pct = 100.0 * covered / total
    print(f"{when:<5} {covered:>8} {total:>6} {pct:>7.2f}%   "
          f"{'bins grew by ' + str(total - 40) if total > 40 else 'baseline'}")

print()
print("the percentage fell from Q1 to Q3:", 100.0*240/260 < 100.0)
print("things proven rose from Q1 to Q3 :", 240 - 40, "bins")
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
when   covered   bins      pct   reading
Q1          40     40  100.00%   baseline
Q2          95    100   95.00%   bins grew by 60
Q3         240    260   92.31%   bins grew by 220

the percentage fell from Q1 to Q3: True
things proven rose from Q1 to Q3 : 200 bins

A team that went from forty proven scenarios to two hundred and forty watched its functional coverage fall from 100% to 92.31%, and a threshold of 95 would fail them for it. That is not a hypothetical: writing a richer covergroup is the normal consequence of understanding a design better, and it always moves the number the wrong way. Chapter 29.4 is about exactly this, and it is the reason a gate on functional coverage needs the denominator in the record.

9. The Gate, Assembled

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
from dataclasses import dataclass
from typing import Optional

THRESHOLDS = {
    "functional": 95.0,
    "branch": 90.0,
    "statement": 90.0,
    "toggle": None,
}
REASONS = {
    "functional": "signoff policy",
    "branch": "agreed with design lead",
    "statement": "agreed with design lead",
    "toggle": "ceiling not established for this configuration",
}


@dataclass(frozen=True)
class MetricResult:
    metric: str
    pct: float
    hits: Optional[int]
    total: Optional[int]
    floor: Optional[float]
    verdict: str
    detail: str


def evaluate(report, thresholds, reasons):
    ungoverned = sorted(set(report) - set(thresholds))
    if ungoverned:
        raise KeyError(f"no threshold for reported metric(s): {ungoverned}")
    missing_reason = sorted(m for m in thresholds if not reasons.get(m))
    if missing_reason:
        raise ValueError(f"threshold with no recorded reason: {missing_reason}")

    out = []
    for metric in sorted(report):
        pct, hits, total = report[metric]
        floor = thresholds[metric]
        if floor is None:
            out.append(MetricResult(metric, pct, hits, total, None, "INFO",
                                    reasons[metric]))
            continue
        if hits is not None:
            exact = 100.0 * hits / total
            need = int(-(-floor * total // 100))
            ok = exact >= floor
            detail = "meets policy" if ok else f"short by {need - hits} of {total} bins"
        else:
            exact = pct
            ok = exact >= floor
            detail = "meets policy" if ok else f"short by {floor - exact:.2f} points"
        out.append(MetricResult(metric, round(exact, 4), hits, total, floor,
                                "PASS" if ok else "FAIL", detail))
    return out


REPORT = {
    "functional": (95.0, 9496, 10000),
    "branch": (91.4, 2742, 3000),
    "statement": (88.3, None, None),
    "toggle": (61.0, 790, 1040),
}

results = evaluate(REPORT, THRESHOLDS, REASONS)

print(f"{'metric':<12} {'value':>8} {'floor':>6} {'verdict':<8} detail")
for r in results:
    floor = "—" if r.floor is None else f"{r.floor:.1f}"
    print(f"{r.metric:<12} {r.pct:>8.2f} {floor:>6} {r.verdict:<8} {r.detail}")

failed = [r.metric for r in results if r.verdict == "FAIL"]
print()
print("gated metrics :", sum(1 for r in results if r.verdict != "INFO"))
print("informational :", sum(1 for r in results if r.verdict == "INFO"))
print("failures      :", failed)
print("overall       :", "FAIL" if failed else "PASS")
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
metric          value  floor verdict  detail
branch          91.40   90.0 PASS     meets policy
functional      94.96   95.0 FAIL     short by 4 of 10000 bins
statement       88.30   90.0 FAIL     short by 1.70 points
toggle          61.00      — INFO     ceiling not established for this configuration

gated metrics : 3
informational : 1
failures      : ['functional', 'statement']
overall       : FAIL

Compare this against the annotation's two-line gate. The same report that returned a bare PASS now returns two failures, one of them with an actionable count — four bins — and one informational row carrying the reason it is not gated. Nothing here is cleverer than the original; it simply refuses to leave a policy decision implicit.

The functional row is the one to notice: its printed percentage was 95.0, which passes, and its exact value is 94.96, which does not. Section 6 in a single row of a table.

10. Predict, Run, Observe

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
THRESHOLDS = {"functional": 95.0, "branch": 90.0}
REPORT = {"functional": 96.0, "branch": 91.0, "toggle": 12.0}

# A
print("A:", [m for m, v in sorted(REPORT.items()) if v < THRESHOLDS.get(m, 0)])

# B
print("B:", sorted(set(REPORT) - set(THRESHOLDS)))

# C
print("C:", 100.0 * 9496 / 10000 >= 95.0, float(f"{100.0*9496/10000:.1f}") >= 95.0)

# D
print("D:", int(-(-95.0 * 10000 // 100)) - 9496)

The verified output:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
A: []
B: ['toggle']
C: False True
D: 4

A — toggle at 12.0 passes, because .get(m, 0) compared it against zero. A gate that returns an empty failure list is not the same as a gate that checked everything.

B — the one line that would have caught it, and it costs a set difference.

C — the exact value fails and the printed value passes. Section 6.

D — four bins. The number an engineer can act on.

11. Exercises

Exercise 1 — four questions. For each of the four metrics, write the question it answers in your own words, and name who owns its denominator.

Exercise 2 — your toggle ceiling. Pick one bus in your design, state its width and how many bits are driven, and compute the toggle ceiling for that signal.

Exercise 3 — no single number. Using your own coverage report, find a single threshold that gives a wrong verdict in both directions, as Section 3 does.

Exercise 4 — the silent half. Run THRESHOLDS.get(m, 0) against a report with more metrics than your policy file, and print which metrics were compared against zero.

Exercise 5 — refuse instead. Add the ungoverned-metric check and show the error it raises on your own report.

Exercise 6 — report but do not gate. Choose one metric in your flow that should be reported and not gated, write it as None, and record the reason.

Exercise 7 — absent versus deliberate. Explain in two sentences what a reviewer can conclude from "toggle": None that they cannot conclude from a missing toggle key.

Exercise 8 — the rounding boundary. Find a hits/total pair in your flow whose printed percentage crosses a threshold its exact value does not.

Exercise 9 — how many bins. Implement the ceiling-division "short by N bins" message and run it on a failing metric.

Exercise 10 — where did 90 come from. For each threshold in your flow, write one line saying who decided it and when. Note any you cannot answer.

Exercise 11 — the weaker claim. Construct a pair of metrics where the higher percentage proves fewer things, and give the ratio.

Exercise 12 — a richer covergroup. Describe what happens to your functional coverage percentage the week somebody adds forty bins, and what the gate does about it.

Exercise 13 — the ceiling-relative gate. Write a gate of the form value >= 0.95 * ceiling for toggle coverage and say what you need to know to use it.

Exercise 14 — reasons as a gate. Add the check that every threshold has a recorded reason, and run it against your real policy file.

Exercise 15 — the ceiling-relative report. For one signal, report toggle coverage both absolutely and as a fraction of its ceiling.

Exercise 16 — who owns the denominator. For each metric in your flow, name what computes its denominator and whether you can change it.

Exercise 17 — both wrong at once. Show that a single threshold on your data is simultaneously too strict for one metric and too lax for another.

Exercise 18 — the policy file. Write your thresholds as a dict with None for anything reported-but-not-gated, and a reason for every entry.

Exercise 19 — add a metric. Add a new metric to your report without touching the policy, and show what your current gate does.

Exercise 20 — the boundary is where it matters. Explain in two sentences why a rounding error that is always small is still a correctness problem for a gate.

Exercise 21 — ceiling division. Derive the -(-floor * total // 100) form and say what it returns when the floor is already met.

Exercise 22 — a weaker 100%. Find a covergroup in your design small enough that its 100% proves less than a lower code-coverage number.

Exercise 23 — the quarter that went backwards. Describe a period in your project when understanding the design better lowered the reported number.

Exercise 24 — gate on the pair. Convert one gate from a printed percentage to hits/total and report any verdict that changed.

12. Summary

  • The metric field says which question a number answers — statement, branch and toggle have denominators the tools compute; functional has one you wrote.
  • A flow-wide threshold is simultaneously too strict and too lax: 90 fails a design whose toggle ceiling is 75.96%, and 60 passes functional coverage of 61.5.
  • A ceiling below 100 is information. Record it and gate relative to it rather than chasing a gap nothing can close.
  • The annotation's shape is right — a threshold per metric is the whole argument in one literal.
  • THRESHOLDS.get(m, 0) invents a policy nobody wrote. Every unmentioned metric passes against a floor of zero, and the gate reports PASS with half of it absent.
  • Make a missing threshold an error, so the decision happens when a metric is added rather than silently never.
  • None means reported-but-not-gated, which is a completely different statement from a missing key — and a reviewer can only tell them apart if the policy says so.
  • Gate on hits/total, not on the printed percentage. 9496/10000 is 94.96 and prints as 95.0, and a threshold is only ever consulted at its own boundary.
  • Report the shortfall in bins, not points. "Short by 4 of 10000" is actionable; "94.96 against 95.0" is not.
  • A coverage floor is a policy, not a measurement — no amount of running the flow can discover it, so record the reason beside the number.
  • 100% of four bins is a weaker claim than 90% of four hundred branches, by a factor of ninety.
  • A richer covergroup always moves the percentage down, which is why a functional gate needs the denominator in the record.
  • Four of the five metric rows a report prints may have no policy at all, and the gate will still say PASS.
  • A rounding error is bounded by the report's precision and always lands at the boundary, because a threshold is only consulted near its own value.
  • -(-floor * total // 100) gives the bins needed, which is the number an engineer can act on this afternoon.
  • Record who decided each threshold and when — the fourth column is what makes the policy file reviewable.
  • A richer covergroup lowers the number, so understanding a design better looks like a regression to a naive gate.

The next chapter takes a design's worth of these per-metric numbers and asks what a closure meeting should be shown.

Where this fits

Part of the Python curriculum.