Coverage Metrics & What They Miss

Chapter 6 · Section 6.2 · Coverage & Signoff

Project thread — the FSM's 98.9% test coverage (6.1) is stuck-at only; this lesson asks what that number doesn't promise (timing, masking, DT-vs-PT).

1. Why Should I Learn This?

A coverage number is only as trustworthy as your critical reading of it — the blind spots decide whether it's honest.

• vs one model — stuck-at coverage ≠ timing/bridging coverage (need a portfolio, 2.1).
• Denominator/masking can inflate it (6.1); X-masking (Ch7) can overstate real detection.
• 'Detected' may include possibly-detected — optimistic if full-credited.
• Toggle/functional coverage are verification metrics — not fault/test coverage; the DPPM link is statistical (1.5).

2. Real Silicon Story — the 99% that shipped escapes

A team signed off at a proud '99% coverage' and set a loose DPPM target on the strength of it. The number was real — and deeply misleading in three ways at once.

First, it was 99% stuck-at only — it said nothing about timing defects (2.3), which then escaped at speed (the 2.3 story's cousin). Second, the flow had masked several observation points to handle X (Chapter 7) but still counted those faults toward coverage — overstating real detection. Third, a batch of possibly-detected (PT) faults were full-credited as detected, though they'd only be caught if the X resolved favorably. The honest test coverage, on a portfolio with a clean denominator and PT counted conservatively, was materially lower — and the escapes proved it.

Lesson: a coverage number is a claim with fine print. Always ask which model, which denominator, what's masked, DT-vs-PT — a 99% that hides its assumptions can ship the very escapes coverage exists to prevent.

3. Factory Perspective — reading coverage through each lens

• What the test engineer sees: the number plus its assumptions — model, denominator, masking, DT/PT, mode — and reports coverage per model with caveats.
• What the yield engineer sees: that escapes can appear despite 'high coverage' when the number was stuck-at-only or inflated — a model/masking gap, not a mystery.
• What the RTL/DV engineer sees: that toggle/functional (verification) coverage is not test coverage — a block can be fully verified (no bugs) yet poorly testable (low fault coverage), and vice versa.
• What management cares about: that a headline coverage number is only a quality commitment if it's per-model, honest-denominator, and DT-strict — otherwise it's a hidden liability against the DPPM target (1.5).

4. Concept — the blind spots, one by one

1) Coverage is vs one model (2.1):

• Stuck-at coverage measures stuck-at faults only — timing (transition, 2.3) and bridging/leakage (2.4) are different lists.
• A high number in one model implies nothing about the others → you need a portfolio, reported per model.

2) Denominator games (6.1):

• Removing unproven faults (aborts) or shrinking the denominator via masking inflates test coverage — a dishonest number.

3) X-masking overstates detection (Chapter 7):

• To stop unknown (X) values corrupting the compare, flows mask certain observation points. A masked point doesn't observe its faults there — so crediting them as detected overstates real detection.

4) Possibly-detected (PT) is optimistic:

• A PT fault's effect reaches an output as X; it's detected only if the X resolves to the right value. Full-crediting PT as detected is optimistic — count it conservatively.

5) Per pattern-set / model / effort / mode:

• Coverage is for this pattern set at this model, effort, and test mode (e.g. slow vs at-speed) — not a universal claim.

6) Not verification coverage:

• Toggle, code, and functional coverage are verification metrics — they measure whether the testbench exercised the design (finding bugs). Fault/test coverage measures manufacturing defect detection. Different worlds (bugs vs defects, 1.1) — never conflate.

7) The DPPM link is statistical:

• Higher coverage → statistically fewer escapes → lower DPPM (1.5) — but 99% coverage ≠ exactly 1% escapes; it's a distribution, dependent on defect density and the unmodeled tail.

5. Mental Model — a restaurant's star rating

A coverage number is like a restaurant's star rating — useful, but easy to over-trust.

• One dimension: '5 stars for pizza' says nothing about their sushi (stuck-at coverage says nothing about timing). You need ratings per dish (a model portfolio).
• Denominator games: a place that only counts invited reviewers (removes unproven/masked faults) can show a higher average — inflated by who's in the sample.
• Optimistic credit: counting a 'might have been good' meal as 5 stars (a possibly-detected fault as detected) pads the score.
• Wrong metric: the health inspection score (verification: is the kitchen clean / no bugs) is not the food rating (test: are the dishes actually good / no defects) — different audits.
• Statistical link: 4.9 stars doesn't guarantee your specific meal is perfect (coverage → DPPM is a distribution, not a promise).

Trust the rating only after asking which dish, which reviewers, what got padded, which audit — exactly the reflex for coverage.

6. Working Example — an annotated coverage report

Read a coverage number with its caveats:

# Coverage number, read CRITICALLY — REPRESENTATIVE, SIMPLIFIED, tool-neutral:
  Headline: "coverage = 99.0%"
  -> WHICH MODEL?      stuck-at ONLY. Timing (2.3) + bridging/leakage (2.4) NOT covered -> add models (portfolio, 2.1)
  -> WHICH METRIC?     TEST coverage (DT/(all-RE-AU)). Fault coverage is lower. State BOTH (6.1)
  -> DENOMINATOR ok?   RE/AU are PROVEN? aborts NOT removed? no MASKING-based removals? -> else inflated (6.1)
  -> DT vs PT?         Are POSSIBLY-DETECTED faults full-credited? Count PT conservatively -> honest DT lower
  -> WHAT'S MASKED?    X-masked observation points -> those faults not truly observed there -> don't over-credit (Ch7)
  -> WHICH MODE?       slow (stuck-at) capture -- says nothing about AT-SPEED (2.3)
# HONEST read: "99.0% stuck-at test coverage, this pattern set/mode, PT counted conservatively, no masking inflation"
#   -> and the coverage->DPPM link is STATISTICAL (1.5), not "1% escapes".

# NOT test coverage (verification metrics) — REPRESENTATIVE:
  toggle coverage / code coverage / functional coverage  = VERIFICATION (did the testbench exercise the design? bugs)
  fault coverage / test coverage                         = MANUFACTURING TEST (are physical defects detected? defects)
# A block can be 100% functionally covered (no bugs) and still have LOW fault coverage (poorly testable) -- and vice versa.

7. Industry Flow — the critical-reading checklist

Every coverage number should pass a critical-reading gate before signoff:

8. Debugging Session — the reassuring number that wasn't

9. Common Mistakes

• Trusting one model's number as 'coverage.' Stuck-at ≠ timing/bridging — report a portfolio (2.1).
• Crediting masked/possibly-detected faults fully. Masking overstates; PT is optimistic — count conservatively.
• Confusing verification and test coverage. Toggle/functional = bugs; fault/test = defects — different (1.1).
• Reading coverage as a DPPM guarantee. The link is statistical (1.5), not '99% → 1% escapes'.
• Ignoring mode/effort. Slow-capture coverage says nothing about at-speed (2.3).

10. Industry Best Practices

• Report coverage per model, with metric/denominator and masking/PT policy stated.
• Count PT conservatively; don't credit masked observation points.
• Keep verification and test coverage separate — they answer different questions.
• Set DPPM statistically — coverage is a lever, not a guarantee (1.5).
• Interrogate every headline number — which model, denominator, mask, DT/PT, mode.

11. Senior Engineer Thinking

• Beginner: "We're at 99% coverage — quality is basically guaranteed."
• Senior: "99% of which model? Stuck-at only? What's the denominator — any masking or unproven removals? Are possibly-detected faults full-credited? Is this at-speed or slow? And that's not our functional coverage. Only after those questions is 99% a quality statement — and even then the DPPM link is statistical."

The senior interrogates every coverage number against the blind spots before treating it as a commitment.

12. Silicon Impact

Coverage is the headline metric of DFT, which is exactly why its blind spots are dangerous — a falsely reassuring number directly causes escapes and a missed DPPM (1.5), because teams set quality expectations against it. The blind spots are independent and compounding: a number can be high because it's stuck-at-only (ignoring timing/bridging, 2.1/2.3/2.4), because its denominator was inflated (masking or unproven removals, 6.1), because possibly-detected faults were full-credited, or because it was measured in a mode (slow) that says nothing about another (at-speed). Any one of these can turn a materially lower honest coverage into a comforting 99%, and the story shows all three at once. The discipline that prevents this is a reflexive critical read — which model, which denominator, what's masked, DT-vs-PT, which mode — plus reporting per model with caveats so signoff is a real commitment, not a padded one. Two confusions deserve special vigilance: verification coverage (toggle/functional — about bugs) is not test coverage (about defects), so a fully-verified block can still be a testability liability; and the coverage-to-DPPM link is statistical, so coverage is a lever on the escape distribution, not a per-part guarantee — which is why SLT/burn-in insurance for the unmodeled tail remains part of any serious quality plan. For the RTL/DV engineer, the takeaway is to never hand up (or accept) a bare coverage number: attach its assumptions, keep verification and test metrics distinct, and treat a high number as a starting question, not a conclusion — the habit that makes the FSM's 98.9% (6.1) an honest input to signoff rather than a trap.

13. Engineering Checklist

• Stated which model(s) — reported a portfolio (stuck-at + timing/bridging as needed, 2.1).
• Confirmed the metric/denominator (test vs fault; only proven-untestable removed, 6.1).
• Did not credit masked observation points; counted PT conservatively.
• Kept verification (toggle/functional) coverage separate from test coverage.
• Set DPPM statistically (1.5); stated the mode/effort the number applies to.

14. Try Yourself

1. Take a '99% coverage' claim and list five questions to make it honest (model, denominator, masking, DT/PT, mode).
2. Show how a stuck-at-only 99% can coexist with timing escapes (2.3).
3. Explain how X-masking (Ch7) can overstate coverage if masked faults are credited.
4. Contrast toggle/functional coverage (verification) with fault/test coverage (test) — different questions.
5. Argue why 99% coverage ≠ exactly 1% escapes (statistical link, defect density, unmodeled tail, 1.5).

The critical-reading skill is tool-neutral. Real coverage/masking come from the ATPG flow. No paid tool required.

15. Interview Perspective

• Weak: "High coverage means the chip is well tested."
• Good: "Coverage is versus a model and a denominator, so you have to check what it includes."
• Senior: "A coverage number has fine print. It's vs one model — 99% stuck-at says nothing about timing (2.3) or bridging (2.4), so I want a portfolio. The denominator can be inflated by masking or removing unproven faults (6.1). 'Detected' can include possibly-detected faults (X-resolves-favorably) that are optimistic. It's for one mode/effort — slow says nothing about at-speed. And toggle/functional coverage are verification metrics (bugs), not fault/test coverage (defects). So for any number I ask which model, which denominator, what's masked, DT-vs-PT, which mode — and I treat coverage→DPPM as statistical, not a guarantee."

16. Interview / Review Questions

17. Key Takeaways

• A coverage number is a claim with fine print — read it critically, or a falsely reassuring number ships escapes.
• It is measured against one model (2.1): stuck-at coverage says nothing about timing (2.3) or bridging/leakage (2.4) → report a portfolio, per model.
• The denominator can be inflated (masking or unproven removals, 6.1), and 'detected' may include optimistic possibly-detected faults — don't credit masked points, count PT conservatively.
• Toggle/code/functional coverage are verification metrics (bugs) — not manufacturing fault/test coverage (defects); never conflate them (1.1); and coverage is per pattern-set/model/effort/mode.
• The coverage-to-DPPM link is statistical (1.5) — 99% coverage ≠ exactly 1% escapes — so for any number, ask which model, which denominator, what's masked, DT-vs-PT, which mode. Next: 6.3 — debugging coverage loss.

18. Quick Revision

Coverage blind spots — read critically. A coverage number has FINE PRINT: (1) vs ONE model (stuck-at ≠ timing 2.3 / bridging 2.4 → need a PORTFOLIO, 2.1); (2) denominator can be INFLATED (masking / unproven removals, 6.1); (3) X-MASKING (Ch7) means masked faults aren't truly observed → over-credit OVERSTATES; (4) possibly-detected (PT) = optimistic (count conservatively); (5) per pattern-set/model/effort/MODE (slow ≠ at-speed); (6) toggle/functional = VERIFICATION (bugs) ≠ fault/test coverage (defects, 1.1); (7) coverage→DPPM is STATISTICAL (1.5), not '99% → 1% escapes'. Reflex: ask which model, which denominator, what's masked, DT-vs-PT, which mode. Next: 6.3 — debugging coverage loss.