Verification Checklist

The question a verification review must answer is not how many tests are there. It is:

Could this environment have detected the defects this design is at risk of containing?

Which decomposes into five things, none of which is a test count:

VERIFICATION QUALITY =
    OBSERVABILITY + INDEPENDENCE + SENSITIVITY + COVERAGE + NEGATIVE CONTROLS

1. Coverage Answers A Different Question

VER-04. Is the stimulus sensitive to the defect, or merely present in the bin?

Two sentences that sound the same and are not:

The same defect, the same partial-write bin, two stimuli:

=== SIM D - COVERAGE IS NOT DEFECT SENSITIVITY ===

  the same defect, the same partial-write bin, two stimuli

  stimulus            mask asked  mask delivered  scoreboard  detected?
  zero background              2               f           0   NO
  non-zero background          2               f           1  yes
  control, no defect           2               2           0  n/a

  committed value     blind 0x00000000   sensitive 0x0000aa00   control 0xffffaaff

The defect is armed in both rows. The delivered mask is f where 2 was asked for, in each. The first stimulus wrote zeros into a location holding zeros, so the wrong mask produced the right answer — and a coverage report would show the partial-write bin covered, in green, on a broken design.

The review artifact this produces:

PASS: every covered feature has an injection that the intended detector catches. FAIL: any bin covered by stimulus that cannot discriminate.

2. Can The Checker Count?

VER-11. Can every PASS-deciding metric represent the number it claims to count?

This item exists because of a defect found in this curriculum's own testbenches, and it is the sharpest thing in this chapter.

Every error counter in Modules 27, 28 and 29 was written like this:

               // THE ONE-BIT FORM, inline in $display exactly as Modules
               // 27-29 wrote it
               (cond[0]!=0)+(cond[1]!=0)+(cond[2]!=0)+(cond[3]!=0)
              +(cond[4]!=0)+(cond[5]!=0)+(cond[6]!=0)+(cond[7]!=0),

EXECUTED TESTBENCH — the defective form, reproduced deliberately in SIM E.

Verilog sizes a self-determined expression by its widest operand. Each comparison is one bit. The sum is evaluated at one bit and wraps. Run it:

=== SIM E - CHECKER ARITHMETIC WIDTH ===

  the same conditions, three ways of counting them

  conditions  expected  1-bit sum (bad)  widened sum  module count
  00000000         0              0            0             0
  00000001         1              1            1             1
  00000011         2              0            2             2
  00011111         5              1            5             5

Read the third row. Two simultaneous failures report as zero.

  READ THE 2 AND THE 5.
  Two simultaneous failures report as 0 in the one-bit form.
  Five report as 1. An even count is indistinguishable from
  a clean run, and every gate downstream believes it.

A genuine zero still prints zero, which is why the published results of Modules 27–29 survived and were re-verified after the correction. The method did not survive: a gate that can only distinguish zero from odd is not a counter.

The three properties the replacement has, and a review should demand:

  // $countones returns an integer, so nothing here is evaluated at the
  // width of a single comparison. That is the entire fix.
  logic [WIDTH-1:0] n_now, n_bad;
  assign n_now = WIDTH'($countones(active));
  assign n_bad = WIDTH'($countones(bad));

EXECUTED RTL — wb_rev_count.sv.

It saturates rather than wraps, so an overflow is visible:

  SATURATION
  accumulating 8 failures per clock for 12000 clocks
    count      65535
    saturated  1

And it reports unknowns rather than reading them as false:

  X-SAFETY
  one true condition and one unknown one
    safe count    1
    unknown count 1

The review item: for every metric that decides PASS, record its declared width, the width of the expression that produces it, the maximum value it must represent, and whether a negative control has ever shown it reporting a nonzero number. In this module: 53 widened terms, 0 unwidened, and the negative control covers 0, 1, 2 and 5 simultaneous failures.

A checker nobody has seen report a failure is a checker nobody has tested.

3. Independence

VER-06. Does the checker derive expected behaviour from the design's internals?

A reference model that copies the design's arithmetic agrees with the design's bugs. Classify every input to the predictor:

PASS: zero design-internal signals feed the expected-behaviour computation. FAIL: any, unless explicitly justified — and the justification has to survive the question what bug would this model now agree with?

4. Who Checks The Checker

VER-08. Have testbench-side defects been exercised?

A scoreboard mismatch is a statement that two things disagree. It contains no information about which of them is wrong.

Two rigs, identical symptom:

  DISCRIMINATING OBSERVATION - the delivered mask
    rig 1: asked 2, delivered f
    rig 2: asked 2, delivered 2

Rig 1 is a design defect. Rig 2 is a reference model with the wrong byte-lane semantics, on a design that did exactly what it was asked. Same mismatch count, same failing transfer, same expected and observed values — with the roles of expected and observed swapped, which no report prints.

Required negative controls, testbench side: at minimum a monitor that samples early and a predictor with wrong semantics. Chapter 30.5 runs the full discrimination.

5. Detector Discrimination

Every detector in the environment should have a known defect it is supposed to catch, and evidence that it did.

Intended detections missed: 1, and it is the point of SIM D — the row exists to show that the miss is a property of the stimulus, not of the detector. With seeded data the same detector catches the same defect.

6. False Confidence

"Coverage is 100%." Proves: every bin you wrote was entered. Does not prove: that entering them could have failed. SIM D's first row is a covered bin on a broken design. Missing evidence: an injection per covered feature.

"The scoreboard passed." Proves: two streams agreed. Does not prove: that either was right, or that anything was compared. A scoreboard with no input agrees with everything — Chapter 27.4 measured zero mismatches from zero comparisons. Missing evidence: the comparison count beside the mismatch count.

"No protocol assertion fired." Proves: the handshakes were well-formed. Does not prove: anything about destination, payload, ownership or commit count — nine of the eleven defects in this module produce zero protocol violations. Missing evidence: the system-invariant checks that no rule encodes.

7. The Verification Checklist

8. What To Carry Forward

• Coverage says the situation occurred. Sensitivity says the observation would have differed. Only the second is evidence.
• Audit the arithmetic that decides PASS. Two failures reporting as zero is not hypothetical; it happened here.
• A checker that has never fired is untested. Give every detector a defect it should catch, and record that it did.
• A predictor fed from design internals agrees with design bugs.
• Report comparisons beside mismatches. Zero over zero is not a pass.
• An unknown must fail the check, never skip it.

Chapter 30.3 reviews the one invariant no Wishbone rule encodes at all.