Coverage Philosophy

Functional Coverage · Module 19 · Page 19.1

The Engineering Problem

The Scoreboards module built automated checking — the scoreboard decides whether the DUT is correct on every transaction (Module 18). But a passing regression answers only one question: was the DUT correct on what you ran? It says nothing about a second, equally vital question: did you run enough? A test can pass while never exercising the corner cases where bugs hide — a FIFO that's never filled, an error response that's never triggered, a combination of modes that's never tried. The bug in the unexercised scenario escapes a green regression silently: the tests passed because they never hit the buggy case. Checking tells you "it was right"; it cannot tell you "we tested enough." So passing tests alone give false confidence — and constrained-random (which generates huge volumes randomly) makes this worse: how do you know the random stimulus hit the important cases? The problem this chapter solves is measuring completeness: functional coverage — whether verification has exercised the design's important scenarios.

Functional coverage measures what the verification has exercised — which scenarios, values, and combinations the design saw — relative to a plan of what should be exercised. It is orthogonal to checking: checking answers "is the DUT correct?" (the scoreboard); coverage answers "did we exercise enough?" Both are required — checking without coverage gives false confidence (passing tests that missed the buggy scenario), and coverage without checking exercises scenarios but doesn't verify them. Functional coverage is intent-centric (written by the verifier to measure the spec's features and scenarios), distinct from code coverage (automatic, but only "did the RTL execute?"). It is what makes "done" definable: verification is done not when tests pass (you could pass forever without hitting the corners) but when coverage is closed — every planned scenario exercised and checked. And it is the feedback constrained-random needs: it shows which scenarios the random stimulus hit (and the gaps), so you can steer toward closure. This chapter is the coverage philosophy: the completeness question, its orthogonality to checking, why passing isn't done, and coverage as feedback and closure.

Why is checking not enough — what is the second question (did we exercise enough?) that functional coverage answers, how is it orthogonal to correctness, and why do passing tests not mean verification is done?

Motivation — why checking isn't enough

A checked but un-measured verification effort has graded what it ran but doesn't know what it skipped. The reasons coverage is essential:

• Passing tells you about what you ran, not what you skipped. The scoreboard checks every transaction you generate — but if you never generate the corner case, the scoreboard never checks it, and the bug there escapes. Passing covers the cases you ran; it says nothing about the cases you didn't.
• Bugs hide in unexercised scenarios. Bugs cluster in the corners — full buffers, error paths, mode combinations, boundary values. If verification never reaches those corners, no amount of checking finds the bugs there. Coverage measures whether you reached them.
• Constrained-random needs feedback. Random stimulus generates huge volume, but blindly — you don't know which scenarios it hit. Coverage is the feedback: it shows what the random stimulus covered, so you can see the gaps and steer (constrain, add directed tests) toward them. Random without coverage is driving blind.
• "Done" can't be defined by passing. You could run passing tests forever without increasing what's exercised. Passing is not a measure of thoroughness. Coverage closure — every planned scenario exercised — is what defines "done".
• Functional coverage measures intent, not just execution. Code coverage (lines/branches) is automatic but measures "did the RTL run?" — not "did we test the feature?". Functional coverage, written by the verifier from the spec, measures the interesting functional space — the scenarios that matter.

The motivation, in one line: a passing regression tells you the DUT was correct on what you ran but nothing about whether you ran enough — and bugs hide in unexercised corners, constrained-random needs feedback on what it hit, and "done" can't be defined by passing — so functional coverage (measuring which spec scenarios were exercised, intent-centric) is essential: it's the orthogonal measure of completeness that checking (correctness) cannot provide.

Mental Model

Hold functional coverage as a map of explored territory — checking verifies each visited place, but the blank spots are where bugs hide unseen:

Functional coverage is a map of explored territory: as verification runs, it shades in the regions it has visited, and the blank spots are the scenarios never exercised. Checking verifies that each visited place is correct, but it says nothing about the blank spots — and bugs hide in the blank spots, unseen, because you never went there. Passing tests are places visited and verified; coverage is the whole map, showing how much remains blank. Picture exploring an unknown territory (the design's behavior space). As verification runs stimulus, it visits regions — exercises scenarios — and functional coverage shades them in on a map. The shaded regions are what you've exercised; the blank spots are the scenarios you've never reached (the full FIFO, the error path, the rare mode combination). Now, checking (the scoreboard) is a separate function: it verifies that each place you visit is correct — every shaded region is checked. But checking says nothing about the blank spots — it can't verify a place you never went. And here's the danger: bugs hide in the blank spots — in the scenarios you never exercised — unseen, because you never went there to check. A passing regression is a fully-verified set of visited places — every place you went, you checked, and it was correct. But if half the map is blank, you have no idea whether the unvisited half works — and a bug there is invisible (the tests passed because they never went to the buggy region). The map (coverage) is what reveals the blank spots — it shows how much of the territory remains unexplored, so you know what's left to exercise. Without the map, you'd think you were done (all visited places pass) while half the territory — and its bugs — remained unexplored.

So functional coverage is a map of explored territory: it shades in the scenarios exercised and reveals the blank spots (the unexercised scenarios where bugs hide unseen). Checking (the scoreboard) verifies each visited place is correct — but says nothing about the blank spots. Together: coverage tells you where you've been (and where you haven't), and checking tells you each place was right — so confidence requires exploring the whole map (coverage) and verifying each place (checking). Passing tests are visited-and-verified places; coverage is the whole map — and the blank spots are where bugs hide.

Visual Explanation — the two orthogonal questions

The defining picture is the two questions: correctness (checking) and completeness (coverage) — orthogonal, both required.

The figure shows the two orthogonal questions verification asks. Correctness: is the DUT right on what we ran? — answered by checking (the scoreboard, Module 18). Completeness: did we exercise enough of the design's scenarios? — answered by functional coverage (this module). The crucial reading is that these are orthogonal — independent axes. A test can be correct but incomplete (it passes, but exercises little) or complete but unchecked (it exercises much, but doesn't verify it). Neither alone is verification: checking without coverage gives false confidence (you checked what you ran, but ran little); coverage without checking gives unverified exercise (you hit the scenarios, but don't know if the DUT was right). Confidence (the warning-colored outcome) requires both — exercise the important scenarios (coverage) and check each is correct (checking). The brand-colored questions (correctness, completeness) map to the success-colored answers (checking, coverage), and both feed confidence. The verification insight is that the whole of verification needs this two-dimensional view: the scoreboard (Module 18) handles one axis (correctness), and functional coverage (this module) handles the other (completeness) — and they're orthogonal, so progress on one does not imply progress on the other. This is why a project tracks both a pass rate (correctness) and a coverage number (completeness): 100% passing with 60% coverage means you've verified the 60% you exercised — the other 40% is unknown. The diagram is the two-axis model of verification: correctness (checking) and completeness (coverage), orthogonal, both required for confidence — and this module is about the second axis, the one a passing regression alone leaves blank.

RTL / Simulation Perspective — coverage measures, it doesn't check

In code, functional coverage is written by the verifier to sample the scenarios of interest — measuring what's exercised, separate from the scoreboard that checks. The code sketches the distinction.

// === FUNCTIONAL COVERAGE: a covergroup samples the SCENARIOS of interest (this module's mechanism) ===
covergroup txn_cg with function sample(bus_txn t);
  cp_op:   coverpoint t.op    { bins ops[] = {READ, WRITE, RMW}; }       // did we exercise each op?
  cp_size: coverpoint t.size  { bins small={[1:4]}; bins large={[5:64]};} // each size class?
  cp_resp: coverpoint t.resp  { bins ok={OKAY}; bins err={SLVERR,DECERR};}// did we hit ERROR responses?
  x_op_size: cross cp_op, cp_size;                                        // each op x size combination?
endgroup
// the coverage RECORDS which scenarios occurred — it makes NO judgment about CORRECTNESS
 
// === SCOREBOARD (Module 18): CHECKS correctness — a SEPARATE concern ===
//   sb.check(observed, expected);   // is the DUT right? — orthogonal to coverage
 
// === both run on every transaction, answering DIFFERENT questions ===
function void monitor_or_sb::write(bus_txn t);
  txn_cg.sample(t);     // COVERAGE: did we exercise this scenario? (completeness)
  sb.check(t);          // CHECKING: was the DUT right? (correctness)
endfunction
// ✗ passing tests with LOW coverage = false confidence: checked what you ran, but ran too little (DebugLab)

The code shows the orthogonality of coverage and checking. The covergroup (txn_cg) — the coverage mechanism (Module 19.2) — samples the scenarios of interest: did we exercise each op (READ/WRITE/RMW)? each size class? the error resp? each op × size combination (cross)? It records which scenarios occurred — it makes no judgment about correctness. The scoreboard (Module 18) checks correctness — is the DUT right? — a separate concern. Both run on every transaction (txn_cg.sample(t) for coverage, sb.check(t) for checking), answering different questions: coverage asks "did we exercise this scenario?" (completeness), the scoreboard asks "was the DUT right?" (correctness). The closing ✗ comment marks the thesis: passing tests with low coverage = false confidence — you checked what you ran, but ran too little (the DebugLab). The shape to carry: coverage and checking are orthogonal and both run — coverage measures the scenarios exercised (a covergroup sampling the interesting space), the scoreboard checks each is correct. The coverage makes no correctness judgment (it just records "this scenario happened"); the scoreboard makes no completeness judgment (it just checks "this transaction was right"). You need both: coverage to know you exercised the important scenarios, checking to know each was correct. The covergroup here is the mechanism the rest of the module details (covergroups 19.2, coverpoints 19.3, bins 19.4, cross 19.5); this chapter is why it exists — to measure completeness, the axis checking leaves blank.

Verification Perspective — the four quadrants of checking × coverage

Because checking and coverage are orthogonal, a verification effort sits in one of four quadrants — and only one gives confidence. Seeing all four shows why both are required.

The figure shows the four quadrants of checking × coverage — and why only one gives confidence. Checked + covered: the important scenarios were exercised AND verified → real confidence (verification's goal). Checked, not covered: what ran was verified, but little was exercised → false confidence (bugs hide in the unexercised scenarios — the most dangerous quadrant, because it looks done: all tests pass). Covered, not checked: scenarios were exercised but not verified → hit but unknown (you hit the cases but don't know if the DUT was right). Neither: no verification value. The verification insight is that only the checked-and-covered quadrant gives confidence — the other three are gaps. The most insidious is checked-but-not-covered (the warning-colored upper): it's false confidence because everything you ran passed, so it looks like success — but you ran too little, so the unexercised scenarios (and their bugs) are unverified. This is exactly the trap of passing tests: a green regression in the checked-but-not-covered quadrant feels done but isn't. The covered-but-not-checked quadrant (exercising without checking) is less common but also a gap — you hit the scenarios but learned nothing about correctness. The brand-colored checked-and-covered is the only quadrant that is verification: exercise the important scenarios (coverage) and check each (checking). The figure is the argument for both dimensions: verification is not "tests pass" (could be the false-confidence quadrant) nor "lots of stimulus" (could be the unchecked quadrant) — it's the conjunction: exercised AND verified. So a project must track both — and the coverage number is what reveals whether a passing regression is in the real-confidence quadrant or the false-confidence one. Passing alone can't tell you which quadrant you're in; coverage can.

Runtime / Execution Flow — coverage as the feedback loop

At run time, coverage acts as the feedback loop that steers verification toward closure: run stimulus, measure coverage, find the gaps, target them, repeat. The flow shows the loop.

The flow shows coverage as the feedback loop toward closure. Run + check (step 1): run stimulus (constrained-random or directed), each checked by the scoreboard (Module 18). Measure (step 2): record which planned scenarios were exercised — the completeness measurement. Find gaps (step 3): the planned scenarios still unexercised — the blank spots on the map. Target (step 4): constrain the randomization or add directed tests to hit them; repeat until every planned scenario is exercised and checked (closure, Module 19.7). The runtime insight is that coverage is what makes verification a directed process rather than blind repetition. Without coverage, you'd run constrained-random forever — generating stimulus blindly, not knowing what you've covered or what's left — and stop on a guess ("enough tests passed"). With coverage, you have a feedback loop: measure what you hit, see the gaps, target them, converge to closure. This is especially vital for constrained-random (Module 8): random stimulus explores broadly but unpredictably, so coverage is the instrument that tells you which of the important scenarios the randomness actually reached — and which it kept missing (so you constrain toward them). The loop — run, measure, find gaps, target, repeat — is how a verification effort progresses toward completeness deterministically, even with random stimulus. The flow is coverage as the steering: it turns blind random generation into a directed march toward closure by providing the feedback — what's covered, what's not — that no amount of passing tests can give. Coverage is the map that tells you where to explore next; without it, you wander.

Waveform Perspective — coverage records scenarios as they occur, and shows gaps

Coverage is visible on a timeline as sampling: as scenarios occur, coverage records (hits) the corresponding bins — and unhit bins are the gaps. The waveform shows coverage measuring exercise.

The waveform shows coverage measuring exercise and revealing a gap. As transactions occur, the operation (op) takes values, and coverage records which scenarios were hit: cov_read pulses when a READ occurs, cov_write when a WRITE occurs. Over this run, READ and WRITE are hit — but cov_rmw stays low: the RMW (read-modify-write) scenario was never exercised — a coverage gap. The crucial reading is the separation of coverage and checking: the scoreboard checked every transaction (sb_check pulses on each) — but it could only check what ran, and RMW never ran, so a bug in RMW would escape (the scoreboard never saw an RMW to check). The cov_rmw staying low is the visible gap: a planned scenario never exercised, invisible to the passing regression (every transaction that ran passed), revealed by coverage. The picture to carry is that coverage records what's exercised (the hit bins — READ, WRITE) and reveals what's not (the unhit bin — RMW), independent of checking (which verifies the hit scenarios). The gap (cov_rmw low) is exactly the blank spot on the map — the scenario where a bug could hide unseen (the tests passed because they never hit RMW). Reading coverage this way — which scenarios were hit? which planned ones are still unhit (gaps)? — is reading the completeness measurement. The hit bins recording exercise and the unhit bin revealing a gap is the signature of functional coverage: it measures what verification exercised and shows what it missed — the completeness axis that checking (which only verifies what ran) leaves blank. Coverage shows the gap; close it before you call it done.

DebugLab — the green regression that shipped a bug in an unexercised corner

✗ SIGN OFF on PASSING tests alone (no coverage measurement):
  // 100% passing, zero failures → "verified, ship it"
  // BUT: the FIFO-overflow-under-back-pressure scenario was NEVER exercised
  // the scoreboard checks what RUNS — it never saw the overflow → never checked it → bug ESCAPED
  // passing = "correct on what we ran"; it says NOTHING about whether we ran the corner
 
✓ MEASURE functional coverage; require CLOSURE before sign-off:
  covergroup ...; coverpoint fifo_state { bins overflow = {FULL_BACKPRESSURE}; }  // plan the corner
  // coverage shows overflow bin = 0% → GAP → not done. Add a test to exercise overflow.
  // now the overflow scenario RUNS, the scoreboard CHECKS it → the bug is CAUGHT before sign-off

Common Mistakes

• Signing off on passing tests alone. Passing means correct on what ran, not that you ran enough; require coverage closure before sign-off.
• Not measuring functional coverage. Without it, thoroughness is unknown — you can't tell the real-confidence quadrant from the false-confidence one.
• Mistaking code coverage for functional coverage. Code coverage measures "did the RTL execute?"; functional coverage measures "did we test the feature?" — write the functional plan from the spec.
• Running constrained-random without coverage. Random stimulus needs coverage as feedback; without it, you generate blindly and don't know what you hit.
• Treating coverage and checking as the same. They're orthogonal — coverage measures completeness, checking measures correctness; you need both.
• Not planning the corners. Bugs hide in corners (overflow, error paths, mode combinations); the coverage plan must include them.

Senior Design Review Notes

Interview Insights

Exercises

1. State the two questions. Name the two orthogonal questions verification asks and what answers each.
2. Place the quadrant. For a regression that's 100% passing with 50% coverage, name the quadrant and what it means.
3. Find the gap. Explain how coverage would reveal that an error-response scenario was never exercised, and why passing tests wouldn't.
4. Define done. Explain why passing tests don't define done, and what does.

Summary

• Functional coverage measures what the verification has exercised — which scenarios, values, and combinations the design saw — relative to a plan; it answers "did we exercise enough?", the question orthogonal to checking's "is the DUT correct?".
• Both are required: checking without coverage gives false confidence (passing tests that never hit the buggy scenario — the checked-but-not-covered quadrant), and coverage without checking exercises scenarios but doesn't verify them.
• Passing tests don't mean done: a passing regression is correct on what you ran, but bugs hide in unexercised corners that passing never reveals; coverage closure (every planned scenario exercised and checked) is what defines done.
• Coverage is intent-centric (the spec's features and scenarios, written by the verifier — distinct from automatic code coverage) and is the feedback constrained-random needs: it shows which scenarios the random stimulus hit, and the gaps, so you can steer toward closure.
• The durable rule of thumb: measure functional coverage, not just the pass rate — verification asks two orthogonal questions (is it correct? did we exercise enough?), and passing answers only the first; plan the important scenarios from the spec (including the corners where bugs hide), require coverage closure before sign-off, and never mistake a green regression for done, because a passing regression with low coverage is a map mostly blank, and the blank spots are where the shipping bugs hide.

Next — Covergroups: the mechanism that measures coverage — the covergroup: how it samples transactions to record which scenarios occurred, where it's instantiated and triggered, the structure that holds coverpoints and crosses, and how it turns the coverage plan into a concrete, measurable artifact.