Scoreboard Debugging

Scoreboards · Module 18 · Page 18.7

The Engineering Problem

The scoreboard flags a mismatch — and the natural instinct is to believe it and debug the DUT. But this module has shown, chapter by chapter, that a mismatch is as likely to be a checker bug as a DUT bug: a circular check passes a broken DUT (18.1); a predictor state divergence floods false mismatches (18.2); a reference model copied from the RTL hides bugs (18.3); a don't-care comparison false-flags (18.4); in-order checking on a reordering DUT false-mismatches (18.5); an ID-reuse collision drops and mis-pairs (18.6). So the scoreboard's accusation is not trustworthy on its own — the checker is as likely to be wrong as the DUT. The cost of trusting it blindly is brutal: days lost debugging an innocent DUT for a bug that lives in the checker. The problem this chapter solves is debugging a scoreboard failure — determining whether a mismatch is a real DUT bug or a checker bug, and which, by a systematic audit rather than a blind assumption.

Scoreboard debugging is the systematic method for localizing a mismatch to its true cause — a DUT bug or one of four checker components: the monitor (observation), the predictor/reference model (expected), the pairing/ordering logic, or the comparison (data checking). The cardinal rule: do not trust the scoreboard's accusation — audit the checker before (or in parallel with) debugging the DUT. The audit asks four questions, each localizing a cause: is the observed value what the DUT actually produced (waveform → else a monitor bug, Module 13.5)? is the expected value what the spec says for this input (spec → else a predictor/reference bug)? are these the right two transactions to compare (pairing → else an ordering bug)? should this field even be compared (don't-care → else a comparison bug)? Only if the observed is faithful, the expected is spec-correct, the pairing is right, and the comparison is meaningful — and they still differ — is it a real DUT bug. The signature patterns from the prior chapters accelerate the audit. This chapter is the methodology: the two causes, the four-question audit, the signature patterns, and the discipline of auditing the checker first.

How do you debug a scoreboard mismatch — determine whether it's a real DUT bug or a checker bug (monitor, predictor/reference, pairing, or comparison) — by auditing the observed against the waveform, the expected against the spec, the pairing, and the comparison, rather than trusting the scoreboard's accusation?

Motivation — why you must audit the checker, not trust it

The scoreboard's mismatch is an accusation, and auditing the accusation is essential because the checker is fallible. The reasons:

• A mismatch has two possible causes. A real DUT bug (the DUT produced the wrong result) or a checker bug (the checker is wrong about what's right). The scoreboard can't tell you which — it only knows observed ≠ expected.
• Checker bugs are common — this whole module catalogued them. Every prior chapter showed a checker bug that masquerades as a DUT failure (or hides a real one). So a large fraction of "DUT mismatches" are checker bugs. Trusting the accusation blindly sends you debugging the wrong thing.
• Trusting the accusation wastes days on an innocent DUT. The worst outcome: you believe the scoreboard, debug the DUT for days, find nothing (because the DUT is correct), and finally discover the checker was wrong all along. The audit would have found it in minutes.
• The checker has four fallible components. Monitor (observation), predictor/reference (expected), pairing (which two compared), comparison (which fields). A bug in any produces a mismatch — and the audit localizes which.
• The signature patterns are diagnostic. The prior chapters' bugs have recognizable signatures (passes-an-injected-bug, flood-from-a-point, same-field-every-time, swapped-values, throughput-correlated) — so recognizing the pattern often names the checker bug immediately.

The motivation, in one line: a scoreboard mismatch has two possible causes — a DUT bug or a checker bug (and this module showed checker bugs are common) — so trusting the accusation and debugging the DUT blindly risks days lost on an innocent DUT; you must audit the checker (observed, expected, pairing, comparison) — the checker errs as often as the DUT — and the signature patterns often name the checker bug at a glance.

Mental Model

Hold scoreboard debugging as auditing a disputed grade before failing the student — the grader errs as often as the student:

Scoreboard debugging is auditing a disputed grade: before you fail the student (the DUT) over a flagged wrong answer, you audit the grading — was the student's answer recorded correctly, is the answer key correct, was the right question's key used, and was the grading fair? The grader errs as often as the student, so you never just trust the failing mark. Picture a flagged wrong answer (a mismatch) on a graded exam. The naive move is to assume the student (the DUT) got it wrong — the grade said so. But a good auditor knows the grading itself can be wrong, so before failing the student, they audit four things. First, was the student's answer recorded correctly? — did the grader transcribe the student's actual answer faithfully, or mis-copy it (the observed vs what the DUT actually produced — the monitor)? Second, is the answer key correct? — does the key hold the right answer per the subject (the expected vs the spec — the reference model)? Third, was the right question's key used? — did the grader compare the student's answer-to-question-3 against the key for question 3, or accidentally against question 5's key (the pairing)? Fourth, was the grading fair? — did the grader mark a margin doodle wrong that doesn't count (a don't-care — the comparison)? Only if the answer was recorded right, the key is correct, the right key was used, and the grading was fair — and the answer still differs from the key — is the student actually wrong (a real DUT bug). If any audit fails, the grade was wrong — the grader (checker) erred, not the student. So a good auditor never fails the student on the grade alone — they audit the grading first, because the grader errs as often as the student.

So scoreboard debugging is auditing the grading before failing the student: a mismatch is a flagged wrong answer, and you audit — answer recorded right (observed vs waveform → monitor), key correct (expected vs spec → reference), right key used (pairing → ordering), grading fair (don't-care → comparison) — before concluding the student (DUT) is wrong. Only if all the grading is correct and the answer still differs is it a real DUT bug. The checker errs as often as the DUT, so never trust the failing mark alone — audit the grading.

Visual Explanation — two causes, four checker components

The defining picture is the fork: a mismatch is either a DUT bug or a checker bug (one of four components), and the first job is to determine which.

The figure shows the fork a scoreboard debugger faces. A reported mismatch (observed ≠ expected) is either a real DUT bug (the DUT is wrong) or a checker bug (the checker is wrong about what's right). The checker bug can live in any of four components: the monitor (the observed isn't what the DUT produced — Module 13.5), the predictor/reference (the expected isn't spec-correct — 18.2/18.3), the pairing/ordering (the wrong two transactions were compared — 18.5/18.6), or the comparison (a don't-care field was compared — 18.4). The crucial reading is that the warning-colored mismatch forks into five possible causes (one DUT, four checker), and the scoreboard can't tell which — it only knows observed ≠ expected. The naive debugger follows only the DUT branch (the default-colored "real DUT bug"), ignoring the four success-colored checker suspects — which is why they lose days on an innocent DUT when the real cause is a checker bug. The brand-colored "checker bug" branch — with its four components — is exactly what the audit must examine. The diagram is the map of suspects: five possible causes for every mismatch, and the first job is not to debug the DUT but to determine which of the five it is — by auditing the four checker components (is the observed faithful? the expected spec-correct? the pairing right? the comparison meaningful?) and only concluding DUT bug when all four checker audits pass. Five suspects, one mismatch — audit the four checker suspects before convicting the DUT.

RTL / Simulation Perspective — the four-question audit

In code (and in inspection), the audit is four checks, each localizing a cause. The example shows how to audit a mismatch.

// A mismatch was reported:  "MISMATCH id=5: got data=B0, expected data=C1"
// AUDIT THE CHECKER before debugging the DUT — four questions, each localizing a cause:
 
// (1) OBSERVED audit — is "got" what the DUT ACTUALLY produced? (check the WAVEFORM)
//     waveform shows DUT drove data=B0 on the bus for id=5  → observed FAITHFUL (monitor OK)
//     if the waveform showed a DIFFERENT value → MONITOR bug (Module 13.5), not the DUT
 
// (2) EXPECTED audit — is "expected" what the SPEC says for this input? (check the SPEC)
//     for this input, the spec says the result should be C1  → expected SPEC-CORRECT (reference OK)
//     if the spec said B0 (or something else) → PREDICTOR/REFERENCE bug (18.2/18.3), not the DUT
 
// (3) PAIRING audit — are these the RIGHT two transactions to compare? (check the IDs/order)
//     observed id=5 paired with expected id=5  → pairing CORRECT
//     if observed id=5 was paired with expected id=3 → PAIRING/ORDERING bug (18.5/18.6)
 
// (4) COMPARISON audit — should the 'data' field even be compared? (check the spec: meaningful?)
//     data is a spec-defined, meaningful field  → comparison VALID
//     if 'data' were a don't-care → COMPARISON bug (18.4), a false mismatch
 
// VERDICT: observed faithful + expected spec-correct + pairing right + comparison valid + they DIFFER
//          → REAL DUT BUG (the DUT produced B0 where the spec requires C1). NOW debug the DUT.

The example shows the four-question audit applied to a concrete mismatch (got B0, expected C1). (1) Observed audit: is got=B0 what the DUT actually produced? Check the waveform — if the DUT drove B0 on the bus, the observed is faithful (monitor OK); if the waveform showed a different value, it's a monitor bug (Module 13.5), not the DUT. (2) Expected audit: is expected=C1 what the spec says for this input? Check the spec — if the spec says C1, the expected is spec-correct (reference OK); if the spec says something else, it's a predictor/reference bug (18.2/18.3), not the DUT. (3) Pairing audit: are these the right two transactions? Check the IDs/order — if observed id=5 is paired with expected id=5, the pairing is correct; if paired with the wrong expected, it's a pairing/ordering bug (18.5/18.6). (4) Comparison audit: should data even be compared? Check the spec — if data is meaningful, the comparison is valid; if it's a don't-care, it's a comparison bug (18.4). Verdict: only if all four pass — observed faithful, expected spec-correct, pairing right, comparison valid — and they still differ is it a real DUT bug (debug the DUT now). The shape to carry: each audit question targets a specific checker component and either rules it out (pass) or names it as the culprit (fail). The audit is fast (four checks) and conclusive (it localizes the cause to one of five). Crucially, the audit precedes debugging the DUT — you rule out the four checker components before concluding the DUT is wrong, so you never burn time debugging an innocent DUT (the DebugLab). Audit observed, expected, pairing, comparison — then, and only then, the DUT.

Verification Perspective — the signature patterns accelerate the audit

The prior chapters' checker bugs have recognizable signatures — so recognizing the pattern often names the cause before the full audit. Knowing the signatures is fast-pathing the diagnosis.

The figure shows the signature patterns — each a fast path from a symptom to a checker-bug cause. Passes everything, including an injected bug → a circular check (expected from the DUT, 18.1) or a reference copied from the RTL (18.3) — the check can't see the bug. A flood of mismatches from a point, with plausible observed values → a predictor state divergence (18.2) — the predictor's state drifted. The same field mismatching every time (others match) → a don't-care being compared (18.4) — a field the predictor can't predict. Swapped values where the observed set matches the expected set → an ordering mismatch (18.5) — positional pairing on a reordered stream. Failures correlating with throughput or ID reuse → an ID-reuse collision (18.6) — a single-slot map. (And a specific, implausible observed value, not matching any checker-bug signature → likely a real DUT bug.) The verification insight is that the audit (Figure 1's four questions) is fast-pathed by pattern recognition: an experienced verifier recognizes the signature and jumps to the likely cause — then confirms with the targeted audit question. The warning-colored symptoms point at the brand-colored causes. This is why the module's chapters matter for debugging: each taught a checker bug with its signature, and collectively they form a diagnostic vocabulary — a symptom-to-cause map. So scoreboard debugging is not a blind four-question audit every time; it's pattern recognition first (does the symptom match a known signature? → jump to that cause), then the targeted audit to confirm, falling back to the full four-question audit when the pattern is unfamiliar. The figure is the diagnostic accelerator: the signatures turn recognizing the symptom into naming the cause — and a symptom that matches no checker-bug signature (a specific, implausible observed value, correctly paired and compared) is the signature of a real DUT bug. Recognize the pattern, name the cause, confirm with the targeted audit.

Runtime / Execution Flow — the systematic debugging process

At run time (well, at debug time), scoreboard debugging is a fixed process: recognize the pattern, audit the four checker components, and conclude DUT bug only if all pass. The flow shows it.

The flow shows the systematic process. Read (step 1): read the precise mismatch — which field, got vs expected, context (the starting evidence, Module 18.4). Recognize the pattern (step 2): if the symptom matches a known signature (passes-injected, flood-from-a-point, same-field, swapped, throughput-correlated), jump to that cause and confirm. Audit (step 3): otherwise (or to confirm), run the four-question audit — observed vs waveform (monitor), expected vs spec (reference), pairing (ordering), comparison (data-check) — each rules a cause in or out. Verdict (step 4): conclude a real DUT bug only if all four audits pass and the values still differ. The runtime insight is that this process guarantees you never debug the DUT until the checker is ruled out — the cardinal rule. The pattern recognition (step 2) is the fast path (a known signature → the cause in seconds); the four-question audit (step 3) is the complete path (works for any mismatch, unfamiliar or not); and the verdict (step 4) only reaches "DUT bug" after the checker is cleared. This is the discipline the module builds toward: each prior chapter taught a checker bug (so you recognize the signatures) and the audit question that localizes it (so you can confirm). Together, they make scoreboard debugging systematic: not "the scoreboard said the DUT is wrong, so debug the DUT" but "a mismatch was reported — which of the five causes is it?" — audited, localized, confirmed. The flow is the complete debugging loop for a scoreboard: read (evidence) → recognize (pattern) → audit (four checker components) → verdict (DUT bug only if the checker is clean) — a deterministic path from a reported mismatch to its true cause, whichever of the five it is. Audit the accusation before acting on it.

Waveform Perspective — auditing the observed against the DUT

The first audit question — is the observed value what the DUT actually produced? — is literally a waveform read. The waveform shows confirming the observed is faithful (ruling out the monitor) so the bug is elsewhere.

The waveform shows the first audit question answered directly. The scoreboard reported "got=B0, expected=C1". The first audit: is got=B0 what the DUT actually produced? The waveform shows the DUT drove dut_out=B0 on the bus, and the scoreboard's observed (sb_got) equals B0 — so the observed is faithful (obs_ok — the monitor is OK), confirmed by the waveform. The crucial reading is that the waveform directly settles the observed audit: dut_out (the ground truth of what the DUT produced) equals sb_got (what the scoreboard observed), so the monitor faithfully reported the DUT's output — the observed is not the bug. This rules out the monitor (Module 13.5's concern), narrowing the cause to the other checker components (expected, pairing, comparison) or a real DUT bug. The next audit step is the expected (sb_exp=C1 — is that what the spec says? — which the waveform can't show; that needs the spec). The picture to carry is that the observed audit is a waveform read: does the scoreboard's got match the DUT's actual output on the bus? — yes → monitor faithful (rule it out); no → monitor bug (found it). The dut_out equals sb_got confirmation is the signature of a faithful observation — the first checker component cleared, so the audit proceeds to the expected, pairing, and comparison. Reading the waveform this way — is the observed what the DUT produced? — is the first, fastest audit step, and it immediately either finds a monitor bug or clears the monitor and narrows the hunt. The waveform answers the first audit question; the spec answers the second.

DebugLab — the days lost debugging an innocent DUT

✗ TRUST the scoreboard, debug the DUT (no checker audit):
  // mismatch: "got=B0, expected=C1"
  // assume DUT wrong → debug RTL/pipeline/datapath for a WEEK → find nothing (DUT computes B0 correctly)
  // the DUT was CORRECT; the bug was in the EXPECTED (reference model misread the spec)
 
✓ AUDIT the checker FIRST (four questions, ~minutes):
  // (1) observed: waveform shows DUT drove B0 → faithful (monitor OK)
  // (2) expected: SPEC says the result should be B0, NOT C1 → REFERENCE MODEL BUG (found it!)
  //     the reference computed C1; the spec requires B0 → the EXPECTED is wrong, not the DUT
  // (3) pairing, (4) comparison: not needed — the expected audit already localized it
  // → fix the reference model (compute B0 per spec); the mismatch disappears; DUT was always right

Common Mistakes

• Trusting the scoreboard's accusation. A mismatch is as likely a checker bug as a DUT bug; audit the checker before debugging the DUT.
• Debugging the DUT first. Run the four-question audit (observed, expected, pairing, comparison) before touching the DUT — it's faster and rules out the checker.
• Not checking the expected against the spec. A wrong expected (reference/predictor bug) looks like a DUT bug; verify the expected is spec-correct.
• Not checking the observed against the waveform. A wrong observed (monitor bug) looks like a DUT bug; verify the observed is what the DUT produced.
• Ignoring the signature patterns. The module's symptom-to-cause map fast-paths diagnosis; recognize passes-injected, flood, same-field, swapped, and throughput-correlated.
• Treating the checker as infallible. The predictor, reference model, pairing, and comparison all have bugs; scrutinize them like the DUT.

Senior Design Review Notes

Interview Insights

Exercises

1. Run the audit. For a mismatch "got=X, expected=Y", list the four audit questions and what each rules in or out.
2. Match the signature. For each — passes an injected bug, flood from a point, same field every time, swapped values, throughput-correlated — name the likely checker bug.
3. Diagnose the innocent DUT. A persistent mismatch where the DUT appears correct. Explain the audit step that finds the real cause.
4. Identify a real DUT bug. Describe the audit outcome that confirms a mismatch is a real DUT bug, not a checker bug.

Summary

• Scoreboard debugging localizes a mismatch to its true cause — a DUT bug or one of four checker components (monitor, predictor/reference, pairing/ordering, comparison) — because the scoreboard's accusation is as likely a checker bug as a DUT bug.
• The cardinal rule: do not trust the accusation — audit the checker before (or in parallel with) debugging the DUT, or you risk days lost on an innocent DUT.
• The four-question audit, each localizing a cause: observed vs waveform (monitor), expected vs spec (predictor/reference), pairing (ordering), comparison (don't-care) — conclude a real DUT bug only if all four pass and the values still differ.
• The signature patterns accelerate the audit: passes-an-injected-bug → circular/copied reference; flood-from-a-point → predictor state divergence; same-field-every-time → don't-care comparison; swapped-values → ordering mismatch; throughput-correlated → ID-reuse collision; specific-implausible-value → likely a real DUT bug.
• The durable rule of thumb: never trust the scoreboard's accusation — a mismatch is as likely a checker bug as a DUT bug, so audit the checker first (observed vs waveform, expected vs spec, pairing, comparison), using the signature patterns to fast-path the diagnosis, and conclude a real DUT bug only when all four checker components are cleared and the values still differ; the checker errs as often as the DUT, and auditing it first saves the days otherwise lost debugging an innocent DUT.

Next — Coverage Philosophy: the next module turns from checking correctness to measuring completeness. A passing test tells you the DUT was right on what you ran — but did you run enough? That is functional coverage's question. The module opens with the coverage philosophy: why checking isn't enough, and how coverage measures whether verification has exercised the design's important scenarios.