Coverage Analysis

Functional Coverage · Module 19 · Page 19.6

The Engineering Problem

With covergroups, coverpoints, bins, and crosses built (Modules 19.2–19.5), the measurement now exists — and produces a coverage report. But a report is not an answer; it's data that must be read and acted on. The naive reading is to look at the headline number — "94%" — and treat it as a progress bar: push it toward 100%, declare done near the top. This misreads what coverage is for. The number is a summary; the useful information is which specific bins are unhit and why. A high overall percentage can hide a critical coverpoint at 0% (an entire error path never exercised) buried in the aggregate. And the last few percent — the gaps that remain — are both the hardest to close and the most likely to harbor bugs (the corners). Worse, each unhit bin has a different cause — a reachable scenario the randomizer never produced, a reserved value that can't occur, a forbidden combination, or a testbench limitation — and each demands a different action. The problem this chapter solves is coverage analysis: reading the report to find and classify the gaps, and turning each into the right next action.

Coverage analysis is reading the coverage report — navigating its hierarchy (overall → covergroup → coverpoint → bin / cross-cell) to the specific unhit bins — and classifying each gap into an action. The number is not the goal; the gap list is. Each unhit bin is one of four kinds: a stimulus gap (the scenario is reachable but the randomization never produced it → constrain or add a directed test), an unreachable bin (the scenario can't occur — reserved value, impossible combination → ignore_bins, it shouldn't be a counting bin), a forbidden combination (must never occur → illegal_bins, a check, not a chase), or a testbench limitation (the bench can't generate it → fix the testbench). Coverage is merged across many regression seeds (the union of all runs' hits, in the coverage database) — you analyze the merged result, not individual runs. The report drives the closure loop: find gaps → classify → act → re-run → re-merge → re-analyze. And the pitfall is the aggregate: a high overall % can mask a 0% critical coverpoint, so you drill into the specific bins, prioritize by criticality, and never dismiss the last few percent as noise. This chapter is coverage analysis: the report hierarchy, gap classification, merging, the closure loop, and the aggregate pitfall.

How do you read a coverage report — navigate its hierarchy to the specific unhit bins, classify each gap (stimulus gap, unreachable bin, forbidden combination, or testbench limitation) into the right action, merge coverage across regression runs, and avoid letting a high headline number hide the critical gaps the last few percent contain?

Motivation — why the report must be read, not just totalled

The coverage number is easy to misuse — as a progress bar to maximize — when its real job is to direct the next work. The reasons analysis matters:

• The gap list, not the number, is the actionable output. A percentage tells you how much is covered, not what's missing. The what's missing — the specific unhit bins — is what you act on. Chasing the number without reading the gaps is flying blind.
• A high aggregate hides critical gaps. A covergroup at 97% can contain a coverpoint at 0% — an entire error path never exercised — averaged away in the headline. Drilling in is the only way to see it.
• Each gap has a different cause and cure. An unhit bin might be a stimulus gap (constrain toward it), an unreachable bin (ignore it), a forbidden combination (make it illegal), or a testbench limitation (fix the bench). Misclassifying it wastes effort (chasing an unreachable bin) or hides a real hole (ignoring a reachable one).
• Coverage lives across many runs, merged. No single seed closes coverage; the union of many seeds does. You analyze the merged database, and merge correctly (consistent definitions) or coverage fragments.
• The last few percent are the hard, important part. The first 90% is easy (common scenarios); the last 10% is the corners — hardest to hit and most likely to harbor bugs. Dismissing the remaining gaps as noise is exactly where escaped bugs live.

The motivation, in one line: a coverage report is data, not an answer — its value is the gap list (the specific unhit bins), not the headline number, so analysis is essential: drill in (the aggregate hides critical gaps), classify each gap (stimulus / unreachable / forbidden / bench — each a different action), merge across seeds (the union), and prioritize the last few percent (the corners, where bugs hide) — reading the report to direct the next work, not maximizing a progress bar.

Mental Model

Hold coverage analysis as route-planning on a map of explored territory — each blank region gets a diagnosis and an expedition, not just a glance at the percent-explored:

The coverage report is a map of explored territory with the blank regions marked, and analysis is route-planning: you don't just read the "percent explored" headline — you look at each blank region and decide why it's blank and what to do. Some regions are simply unvisited but reachable, so you send an expedition (constrain the stimulus). Some are unreachable terrain that was wrongly drawn on the map, so you correct the map (ignore the bin). Some are forbidden territory that should trip an alarm if ever entered (make it illegal). And some are blank because the road there is washed out — the testbench can't get there — so you fix the road. The percent-explored number is a summary; the real work is diagnosing each blank region and dispatching the right response. Recall the map of explored territory from Module 19.1: as verification runs, it shades in the regions exercised, and the blank spots are the unexercised scenarios. The coverage report is that map, rendered — with a headline "percent explored" and, underneath, the specific blank regions (the unhit bins). The novice reads only the headline — "94% explored, nearly done" — and pushes to inch it up. The expert does route-planning: they open the map, find each blank region, and diagnose it. A region that's unvisited but reachable (a valid scenario the random expeditions simply haven't reached yet) → send a targeted expedition (constrain the randomizer, or add a directed test, to drive toward it). A region that's unreachable terrain — drawn on the map by mistake (a reserved value, an impossible combination that can't occur) → correct the map (ignore_bins it; it never should have been a required region). A region that's forbidden territory (a combination that must never occur) → post an alarm (illegal_bins, so entering it fails the run). And a region blank because the road is washed out — the testbench physically can't generate that scenario (a missing sequence, a constraint that blocks it) → fix the road (extend the testbench). The percent-explored number is a summary of how much of the map is shaded; the real analysis is diagnosing each blank region and dispatching the right response — and prioritizing the blank regions that matter (a blank error-handling region matters far more than nudging a well-explored region fuller).

So coverage analysis is route-planning on the map: the report is the map with blank regions (unhit bins), and you diagnose each: unvisited-but-reachable → send an expedition (constrain / direct); unreachable terrain → correct the map (ignore); forbidden → post an alarm (illegal); road washed out → fix the road (testbench). The percent-explored is a summary; the substance is the blank-region diagnosis, prioritized by which regions matter. Don't just read the percent explored — diagnose each blank region and dispatch the right response.

Visual Explanation — the report hierarchy and drilling to the bin

The defining picture is the drill-down: the report is a hierarchy, and analysis navigates it from the headline number to the specific unhit bin.

The figure shows the report hierarchy. The top level is the overall coverage % (the warning-colored headline — warning because it's the most misused level). It breaks down per covergroup (each interface / transaction type), then per coverpoint and per cross (each feature and combination), then per bin and per cross-cell (the leaves — the success-colored bottom, where the actual hit counts live). The crucial reading is the direction of useful analysis: down. The headline number is a summary that averages away detail — a 97% overall can contain a covergroup at 80%, which can contain a coverpoint at 0%, which is an entire feature never exercised — invisible at the top. Analysis means drilling down — from the overall %, into the low covergroups, into the low coverpoints/crosses, to the specific unhit bins (the leaves) — because that's where the actionable gaps are. The brand-colored coverpoint/cross level is where you start finding the gaps, and the success-colored bin level is where the gaps actually are (a named bin, 0 hits). The reason the leaves matter is that an action attaches to a specific bin: "the error_resp bin of the cp_resp coverpoint has 0 hits" is actionable (add a test that triggers an error response); "coverage is 94%" is not. So the report is a tree, and analysis is traversal — descending from the summary to the specific gaps, never stopping at the headline. The diagram is the navigation map: overall → covergroup → coverpoint/cross → bin/cell, with the actionable information at the leaves — read the report by drilling to the unhit bins, not by reading the top number. The headline summarizes; the leaves are where you act.

RTL / Simulation Perspective — classifying a gap into an action

In analysis, every unhit bin gets a classification that determines the action. The example walks an unhit-bin list and shows the decision per bin.

// A coverage report shows these UNHIT bins. For EACH, classify the cause → take the action:
 
// (1) cp_resp.error_resp = 0 hits — error responses are REACHABLE but never randomized
//     CLASSIFY: stimulus gap   → ACTION: add a test/constraint that injects an error response
//     constraint err_c { resp dist { OKAY := 90, SLVERR := 10 }; }   // bias toward errors
 
// (2) cp_mode.bins[5..7] = 0 hits — modes 5–7 are RESERVED; the DUT can never produce them
//     CLASSIFY: unreachable    → ACTION: ignore_bins them (they should not be counting bins)
//     ignore_bins rsvd = {[5:7]};   // remove from the denominator so the coverpoint can close (19.4)
 
// (3) x_op_size cell (WRITE × size==0) = 0 hits — a WRITE at size 0 must NEVER occur
//     CLASSIFY: forbidden      → ACTION: illegal_bins it (sampling one should FAIL the test)
//     illegal_bins wr_zero = binsof(cp_op.wr) && binsof(cp_size) intersect {0};
 
// (4) cp_seq.back_to_back = 0 hits — the testbench's sequence NEVER issues two back-to-back
//     CLASSIFY: testbench limitation → ACTION: fix the sequence/driver to generate the pattern
//     // the stimulus generator literally can't produce it → extend the testbench, not the constraints
 
// ✗ MISTAKE: reading only "94%" and pushing random seeds — never drilling to see bins (1)..(4)
//   bin (1) is a real hole; bin (2) blocks closure; bin (3) is an uncaught violation risk

The code shows the classification decision — the heart of analysis. For each unhit bin, classify the cause, then take the action. (1) cp_resp.error_resp = 0 — error responses are reachable but never randomized → stimulus gap → add a test/constraint biasing toward errors (resp dist {...}). (2) cp_mode.bins[5..7] = 0 — modes 5–7 are reserved, the DUT can't produce them → unreachable → ignore_bins them (they shouldn't be counting bins — Module 19.4), so the coverpoint can close. (3) x_op_size cell (WRITE × size 0) = 0 — a WRITE at size 0 must never occur → forbidden → illegal_bins it (sampling one fails the test). (4) cp_seq.back_to_back = 0 — the testbench's sequence never issues two back-to-back → testbench limitation → fix the sequence/driver (the stimulus generator literally can't produce it; constraints won't help — extend the testbench). The mistake (commented) is reading only "94%" and pushing random seeds without drilling to see bins (1)–(4) — where bin (1) is a real hole, bin (2) blocks closure, and bin (3) is an uncaught violation risk. The shape to carry: analysis is per-bin classification — each unhit bin maps to one of four causes, and each cause has one right action (stimulus gap → constrain/direct; unreachable → ignore; forbidden → illegal; bench limit → fix the bench). The classification is what turns the report into work — and misclassifying (or not classifying at all, just chasing the number) wastes effort or hides holes. The four actions are distinct and non-interchangeable: constraining won't close an unreachable bin, ignoring a reachable hole hides it, and more seeds won't fix a testbench that can't generate the scenario. Read each unhit bin, classify its cause, take the matching action.

Verification Perspective — the four gap classifications

The central skill is the classification — and seeing the four kinds together, each with its tell and its action, is the analysis toolkit.

The figure shows the four classifications of an unhit bin, each with its action. Stimulus gap: the scenario is reachable but the randomizer never produced it → constrain or add a directed test to drive toward it. Unreachable: the scenario can't occur (reserved value, impossible combination) → ignore_bins it (so the coverpoint can close). Forbidden: it must never occur → illegal_bins it (so a violation fails the test). Testbench limitation: the bench can't generate it → fix the sequence or driver. The verification insight is that every unhit bin is exactly one of these four, and each demands a different response — so the classification is not optional bookkeeping, it's what determines the work. The tells distinguish them: a stimulus gap is a reachable scenario (the spec allows it, the DUT can do it) the randomizer just missed — more/biased stimulus closes it. An unreachable bin is a reserved or impossible value — no stimulus can close it, so it must be ignored (the 19.4 closure-blocker). A forbidden bin is a violation — it should never be hit, so making it illegal turns the coverage into a check. A testbench limit is a scenario the bench architecture can't produce (a missing sequence type, a structural constraint) — no amount of seeds or constraints helps; the bench must be extended. The brand-colored classifications map to the success/default-colored actions, and getting the classification wrong is costly: chasing an unreachable bin with stimulus (wasted weeks, the 19.4 trap), ignoring a reachable hole (hiding a real gap), constraining toward a testbench limit (futile). The figure is the analysis toolkit: one unhit bin → four possible causes → four distinct actions — and the skill is reading each bin's tell to classify it right. Classify before you act: the same unhit bin means four different jobs depending on why it's unhit.

Runtime / Execution Flow — the closure loop

At the level of a regression, analysis is a loop: run many seeds, merge, analyze the gaps, act, and repeat — driving toward closure. The flow shows the operational cycle.

The flow shows the closure loop. Run (step 1): a regression of many constrained-random seeds, each checked by the scoreboard and each collecting coverage. Merge (step 2): combine all seeds' coverage into the union — total coverage is what all runs hit together, not any one run (no single seed closes coverage; the merge of many does). Analyze (step 3): drill the merged report to the unhit bins; classify each (stimulus / unreachable / forbidden / bench-limited). Act (step 4): constrain/direct, ignore, make illegal, or fix the bench; then re-run, re-merge, re-analyze until the meaningful gaps close. The runtime insight is that coverage analysis is iterative and operates on the merged result. The merge is essential: constrained-random spreads coverage across seeds (each seed hits a different random subset), so you never judge coverage from one run — you merge the regression and analyze the union. The loop is the operational form of 19.1's feedback: the merged report reveals the gaps, the classification assigns the actions, the actions change the stimulus/coverage model, and the next regression moves coverage — converging toward closure. Each iteration narrows the gap list: stimulus gaps close as constraints drive toward them, unreachable bins vanish from the denominator as they're ignored, forbidden bins become checks, and bench limits close as the bench is extended. The flow is the closure cycle: run → merge → analyze → act → repeat — and the report is the steering at every iteration, telling you what to do next. The loop ends not at "100%" but when the meaningful gaps are closed (the closure criterion, Module 19.7). Analyze the merged union, classify the gaps, act, and iterate — the report steers each lap.

Waveform Perspective — coverage accumulation and the stubborn corner

Coverage accumulates over a run (and a regression) as scenarios occur: common bins fill early, the curve rises fast then plateaus, and a corner bin fills only when a targeted effort reaches it — or never. The waveform shows the shape and the stubborn gap.

The waveform shows the shape of coverage accumulation. Common scenarios occur constantly, so their bins fill in the first transactions and overall coverage rises steeply (cov_rising). As the common bins saturate, the curve plateaus (plateau) — new transactions mostly hit already-covered bins, so the number barely moves. A corner bin (the error path, cov_error) stays unhit through the plateau — random stimulus rarely reaches it — until a directed test injects the error late in the run (err_inject), finally closing it. The crucial reading is the gap between the plateau and closure: the headline number plateaus (say at 90%) long before the meaningful gap (the error path) closes — so a verifier watching only the number would see it "stuck" and might conclude diminishing returns and stop, leaving the error corner (the cov_error bin) unexercised. The last few percent are exactly this stubborn corner: the common bins gave the first 90% cheaply, but the remaining 10% is the error paths, boundaries, and rare combinations that random stimulus rarely hits — the hardest to close and the most likely to harbor a bug (an error path that's never exercised is an error path that's never checked). The picture to carry is that coverage accumulation is front-loaded (steep then flat), and the flat part is where the important work is: the plateau is not "done" — it's the signal to switch from random to targeted (directed tests, biased constraints) to reach the stubborn corners. Reading the curve this way — the plateau means the easy bins are done; the remaining gaps need targeted effort — is reading coverage as a process, not a number. The error corner closing only with a directed test is the signature of the last-few-percent reality: the headline plateaus, but the meaningful gap closes only when you target it. The plateau is where analysis earns its keep: target the stubborn corners, don't mistake the flat curve for done.

DebugLab — the 0% error path hidden under 97%

✗ TRACK only the headline number:
  // overall coverage = 97% → "essentially complete, sign off"
  // BUT drilling in would have shown:
  //   error_handling_cg ............ 5%      ← critical, almost entirely unexercised
  //     cp_resp.error_resp ......... 0 hits  ← error responses NEVER triggered
  //     cp_resp.ok_resp ............ 100%    ← normal path fully covered
  // the 0% error path is averaged away under thousands of covered normal-path bins → HIDDEN
 
✓ ANALYZE the gap list, prioritized by criticality:
  // drill to the unhit bins, sort by importance — an error path at 0% outranks a common path at 95%→96%
  // classify error_resp=0 as a STIMULUS GAP → add a directed test injecting malformed requests
  // re-run: error_resp now hit, the scoreboard CHECKS the error path → the hang bug is CAUGHT
  // sign off on CLOSURE of meaningful bins, not on a high aggregate number

Common Mistakes

• Chasing the headline number instead of the gaps. The percentage is a summary; drill to the specific unhit bins, which are the actionable output.
• Signing off on a high aggregate. A high overall can hide a 0% critical coverpoint; gate sign-off on the closure of meaningful bins, not the headline.
• Misclassifying gaps. Chasing an unreachable bin with stimulus wastes weeks; constraining toward a testbench limitation is futile — classify before acting.
• Over-aggressively ignoring bins. Ignoring a reachable scenario to "close" coverage hides a real hole; ignore only genuinely unreachable values.
• Analyzing a single run instead of the merged result. No seed closes coverage alone; merge the regression and analyze the union.
• Dismissing the last few percent as noise. The remaining gaps are the corners — hardest to hit and most likely to harbor bugs; prioritize them.

Senior Design Review Notes

Interview Insights

Exercises

1. Drill the hierarchy. Given a 96% overall with one covergroup at 40%, describe the path you'd drill to find the actionable gap.
2. Classify the gaps. For four unhit bins — an error response, a reserved mode, a forbidden combination, and a never-issued sequence — name each classification and its action.
3. Explain the merge. Explain why analyzing a single seed's coverage misleads, and what the merged union represents.
4. Defend the corner. Explain why a 0% error-path coverpoint under a 97% overall is more urgent than raising the overall to 98%.

Summary

• Coverage analysis is reading the report to find and classify the gaps, then acting — the number is a summary, but the value is the gap list: navigate the hierarchy (overall → covergroup → coverpoint/cross → bin/cell) to the specific unhit bins.
• Classify each gap into its action: a stimulus gap (reachable, not produced → constrain/direct), an unreachable bin (can't occur → ignore_bins), a forbidden combination (must never occur → illegal_bins), or a testbench limitation (bench can't generate → fix the bench) — each demands a different response.
• Coverage is merged across many seeds (the union of all runs' hits, with consistent definitions) — analyze the merged result, not individual runs; the report drives the closure loop: find gaps → classify → act → re-run → re-merge → re-analyze.
• The aggregate pitfall: a high overall % can hide a 0% critical coverpoint (an error path) averaged away in the headline, and the last few percent are the corners — hardest to hit and most bug-prone — so drill into the specific bins and prioritize by criticality.
• The durable rule of thumb: the coverage number is a summary, but the gap list is the substance — drill to the specific unhit bins, classify each (stimulus gap → constrain; unreachable → ignore; forbidden → illegal; bench limit → fix the bench), analyze the merged regression union, and prioritize gaps by criticality, because a high overall percentage can hide a 0% critical coverpoint, and the last few percent — the stubborn corners — are exactly where the unexercised, unchecked, bug-harboring scenarios live.

Next — Coverage Closure: the module's culmination — closure: deciding when verification is actually done. What coverage closure means, how a coverage plan and sign-off criteria define the target, why 100% is neither necessary nor sufficient, how closure combines functional coverage with checking and code coverage, and how to reach a defensible, signed-off "done."