Debug Checklist

Debugging without a procedure is browsing. Module 27 built the method; this chapter is the reviewable form of it.

SYMPTOM  ->  BOUNDARY  ->  INVARIANT  ->  EVIDENCE  ->  ROOT CAUSE

The discipline the whole chapter rests on is a three-way separation people collapse constantly:

1. The Procedure

Twelve steps. The ones people skip are 5, 7 and 12.

DBG-04. Was the cheapest evidence that could discriminate actually used?

Two cases from this module where a counter beat a waveform outright.

The duplicate commit. Three rigs, identical termination counts, identical stored values in two of them. A waveform of any one rig looks perfect. One counter separates them: commits 2 / 2 / 8.

The misrouted response. Every port protocol-shaped, every handshake legal, a full waveform dump entirely unremarkable. One field separates it: the owner at phase start, recorded beside the recipient — 133 misdeliveries, first at cycle 6.

In both, the waveform contains the answer and does not contain a way to find it.

2. Discriminating Evidence

DBG-07. Would this observation have come out differently under the other hypothesis?

If not, it is not evidence — it is confirmation. Worked, from an ambiguous symptom:

=== SIM K - DEBUG REVIEW ===

  SYMPTOM
    scoreboard reports 1 data mismatch on a partial write;
    protocol violations 0; assertions 0; unknowns 0.

  HYPOTHESES
    H1  the slave merged the lanes wrongly
    H2  the fabric delivered the wrong byte mask
    H3  the monitor recorded the wrong value
    H4  the reference model expected the wrong value

  DISCRIMINATING OBSERVATION - the delivered mask
    rig 1: asked 2, delivered f
    rig 2: asked 2, delivered 2

One field, and it does work in both directions:

  ELIMINATION
    rig 1  delivered mask differs from the mask asked for,
           so H2 survives and H1 is not yet needed.
    rig 2  delivered mask is correct, so H1 and H2 are both
           rejected and the divergence is downstream of the
           design entirely.

  ROOT CAUSE
    rig 1  a DESIGN defect: the partial write was widened.
    rig 2  a TESTBENCH defect: the reference model ignores
           the byte selects. The design was correct.

Two identical symptoms. Same count, same kind, same transfer. One is the design and one is the testbench, and the only thing that separates them is a field no protocol checker computes.

DBG-09. Has the possibility that the checker is wrong been eliminated, or merely not considered?

Rig 2 is the case people do not look for. A scoreboard mismatch is a statement that two things disagree; the instinct to indict the design first is a bias, and in this module's evidence it is wrong about a third of the time.

3. First Divergence, And Causal Ordering

DBG-08. Is the claimed root cause earlier than the symptom it explains?

Every worked debug review records six fields:

And the ordering rule, checked rather than asserted:

root-cause event  ≤  first divergence  ≤  final symptom

A claimed cause that happens after its symptom is a coincidence with a story attached. Sort divergences by first occurrence, never by count — the boundary with the largest count is usually the furthest downstream, and therefore the least likely to be the cause.

4. Two Fixes That Are Not Fixes

DBG-11. Does the fix remove the mechanism, or the evidence of it?

Raising the timeout. Symptom: transfers do not return. Chapter 27.1 measured what doubling the patience buys when a return path drops acknowledgements — twice the wait and the same one completion, while the slave answered on every one of those clocks. A timeout is a policy for surviving a missing response. It repairs nothing.

Avoiding the failing address. Symptom: a readback is wrong. Change the test to stop reading that location and the suite goes green with the decoder exactly as broken as before. Chapter 27.6 ran that experiment: 4 operations retired, 0 mismatches, 0 violations, defect untouched.

A third, and the one this module adds: widening the tolerance on a scoreboard comparison because "the model is probably wrong". Sometimes it is — SIM K rig 2 — but that is a conclusion you reach by looking at the delivered mask, not by relaxing the check. Relaxing it destroys the evidence for both cases at once.

The six conditions a root-cause fix must meet, of which "the test passes now" is the fifth:

1. a mechanism that explains the symptom;
2. evidence that the mechanism actually occurred;
3. a first divergence consistent with it;
4. a correction that removes the divergence;
5. the original failing test passing;
6. a regression that failed before the fix and is clean after.

Condition 6 is the one that gets dropped. A regression that passed before proves nothing about the fix.

5. Debug Readiness

DBG-01. Can this system be debugged at all?

This is a design review item, not a debug one, and it is cheapest to fix before anybody needs it. Four fields, each free at design time and unrecoverable afterwards:

Chapter 27.4 found a checker that reported zero misdeliveries on a rig built to misdeliver every response, because it was watching the one boundary the defect removes the evidence from. Place the probe where the evidence still exists, and expose it on a port so a reviewer can ask for it.

6. False Confidence

"The waveform looks right." Proves: nothing that was looked at was obviously wrong. Does not prove: that what was wrong was on the screen. The misrouted-response rig is protocol-shaped on every port. Missing evidence: a census of something the waveform does not display — owner, target, commit count.

"The test passes now." Proves: the observable changed. Does not prove: that the mechanism was removed. Both non-fixes above produce passing tests. Missing evidence: conditions 1, 2, 3 and 6.

7. The Debug Checklist

8. What To Carry Forward

• Observation, hypothesis, proof. Collapsing any two is how a week goes missing.
• Climb the ladder from the top and stop where the hypotheses separate. Every case here settled at a census.
• Evidence that cannot come out the other way is confirmation.
• Sort by first occurrence, never by count.
• Check causal ordering mechanically. A cause later than its symptom is a story.
• A regression that passed before the fix proves nothing.
• Debug observability is a design-review item. Four fields, free before tape-in, unrecoverable after.

Chapter 30.6 turns the same discipline on the thing you say out loud.