Bug Injection, Waveform Debug, and Design Review

Five chapters built a controller and an environment that says it is correct. This chapter stops believing either.

A loopback test passes nine of the thirteen bugs injected below, including a bit-order fault that inverts every word in both directions. A test that passes is not evidence until you know what it is unable to fail on.

1. What Bug Injection Is Actually Measuring

Not the design. Every mutation below is reverted immediately, and the controller shipped at the end of this module is the one Chapter 20.3 published.

What is being measured is the environment: given a realistic defect, does something fire, and is it something that names the problem? A suite that has only ever seen a correct design has never been asked that question.

One rule makes the whole exercise valid, and it is easy to skip:

2. The Detection Matrix

Thirteen defects, five detectors. Numbers are failure counts; PASS means that detector did not notice.

  id   requirement          loop  direct   check    crv    vhdl  psl  caught
  B1   REQ-MODE-002/003       12      16       5     45       5    0  directed+checking+crv+vhdl
  B2   REQ-FUNC-002           12      16       4     33       4    0  directed+checking+crv+vhdl
  B3   REQ-FUNC-003         PASS    PASS    PASS   PASS    PASS    0  EQUIVALENT (identical traces)
  B4   REQ-TIM-003          PASS       8      10   PASS      10    0  directed+checking+vhdl
  B5   REQ-MODE-001         PASS    PASS       1   PASS       1    0  checking+vhdl
  B6   REQ-CFG-001          PASS       2    PASS   PASS    PASS    0  directed
  B7   REQ-MODE-003           24      33      10     85      10    0  directed+checking+crv+vhdl
  B8   REQ-ERR-001          PASS       6    PASS      1    PASS    0  directed+crv
  B9   REQ-ABT-001          PASS       2    PASS   PASS    PASS    0  directed
  B2b  REQ-FUNC-002         PASS      65      20    152      20    0  directed+checking+crv+vhdl
  B3b  REQ-FUNC-003         PASS      34      24     87      24    0  directed+checking+crv+vhdl
  B11  REQ-FUNC-006           48     203      67    340   crash    1  directed+checking+crv+vhdl+psl
  B10  REQ-FUNC-004         PASS    PASS      12   PASS      12    0  checking+vhdl

  mutations applied            : 13
  genuine escapes              : 0
  equivalent mutants           : 1  -> B3
  MISSED BY THE LOOPBACK BENCH : 9  -> B3 B4 B5 B6 B8 B9 B2b B3b B10

3. Three Results Worth More Than The Score

B2b — the loopback bench is blind to a symmetric bit-order fault

B2 inverts only the transmit alignment, which makes the loop asymmetric, and the loopback bench catches it. B2b inverts the flag that feeds both paths — which is what a real bit-order bug looks like, because there is one flag — and the reversal applied going out is undone coming back.

   loopback  PASS       24 transfers, every mode, every width
   directed  65 failures
   crv       152 failures

The loopback bench is not a bad test. It exercises the divider, the counters, the chip select, the state machine and the whole datapath end to end at every mode and width, and it is the first test anybody writes because it needs no model. What it cannot do is notice a transformation that its own topology inverts — and no amount of extra loopback stimulus changes that, because the blindness is structural.

This is also why Chapter 20.1 §10 insisted on non-palindromic data. With a5 or 5a, even the external-model suite would have missed it.

B3 — an equivalent mutant, proven rather than assumed

B3 relaxes the sample guard from below the width to at or below the width. Nothing detects it, and nothing should: the frame produces exactly 2N transitions of which exactly N are sample events, so there is no N+1th sample event for the guard to stop. The guard is redundant by construction.

Saying so is not enough, because the environment missed it and there was nothing to miss look identical in a pass/fail table. The harness distinguishes them by comparing full transcripts against the baseline:

   B3   EQUIVALENT (identical traces)

Byte-identical output from all three benches means the mutated design behaved exactly like the original. That is evidence of equivalence rather than an argument for it — and B3b, which adds two real edges so the extra samples actually occur, is caught by every detector with 24 to 87 failures. The guard matters when the edge count is wrong, which is precisely what it is there for.

B11 — the only bug PSL catches, and the only one it can

Twelve of thirteen mutations produce psl 0. That is not a weakness in the temporal layer; it is an accurate measurement of what it is for. None of the other twelve breaks a temporal obligation — they break data, levels, counts and configuration, all of which a per-cycle predicate sees.

B11 removes the condition that ends the transfer phase, so the frame clocks forever. Every safety property still holds, indefinitely — Chapter 20.5 works through why, property by property.

   procedural benches   watchdog timeout -- "a frame never completed"
   PSL                  assertion failure on eventually! (done = '1')

Both detect it. Only one names the broken obligation. A timeout says something hung; the PSL failure says an accepted request did not complete, and points at the line that promised it would.

4. Waveform Debug, With A Trap In It

Take B7 — shift-then-present — and debug it the way Chapter 18's method requires: symptom, hypotheses, discriminator, evidence.

Symptom. Modes 0 and 2 pass every width. Modes 1 and 3 return words shifted one position, in both directions at once.

Hypotheses. (a) the mode decoder's parity is wrong; (b) the CPHA-conditional term is wrong; (c) the shift ordering is wrong for one CPHA value; (d) the device model is wrong.

Discriminator. (a) would fail all four modes, not two — so the failure being conditioned on CPHA and not CPOL eliminates it immediately. (d) is eliminated by the device model being shared with the passing modes. Between (b) and (c): if the decoder's CPHA term were wrong, the edge on which data moved would change, which is visible as MOSI changing on the wrong SCLK transition. If the shift ordering were wrong, MOSI would change on the right transition carrying the wrong bit.

Evidence. So the measurement is: does MOSI move on the expected transition? Not is the data right, which is already known to be wrong.

Root cause. On a launch, the code shifted the transmit register and then presented its new top bit. For CPHA = 0 that is correct, because a bit was already presented when the chip select asserted and the next launch genuinely wants the following bit. For CPHA = 1 nothing has been presented, so bit N-1 is discarded.

Fix. One uniform rule — present, then shift — used by the pre-launch and by every launch.

Regression. All four modes, all four widths, both orders: 284 checks, 0 failures, and the fix is what Chapter 20.3 publishes.

5. The Two Gaps Injection Found In The Environment

Bug injection is only worth doing if a miss is allowed to change something.

B10 — the device model answered any select

The wrong chip select was asserted and nothing noticed. The reason is that the bench's device model responds to any select being low, so a controller that asserted cs_n[1] when asked for cs_n[0] transferred the correct data to the same model and every check passed. P1 did not help either: one select was still low, just not the right one.

The fix was a new scoreboard field — capture which index went low at the assert cycle, and compare it against the requested device. After it:

   B10   checking 12 failures, vhdl 12 failures

The general form is worth stating: a monitor that observes that something happened cannot check which thing happened. Any property phrased over a reduction — an OR of the selects, a count of active lines — has this shape.

B5 — the directed suite checked the parked level one cycle too early

B5 makes SCLK park at the inverse of cfg_cpol, and Chapter 20.3's directed suite does check a parked level — yet it reports PASS. The check samples immediately after busy falls, and the idle re-park is registered, so it lands one cycle later. What the directed suite actually validates is the level at frame end, which is guaranteed by the even transition count rather than by the idle logic.

The property monitor in Chapter 20.5 catches it, because it watches every cycle rather than one sampling point. That is a real division of labour rather than a redundancy, and it is worth recording that the two checks which look like the same check are not.

6. Synthesis Review

Read against the failure classes rather than run through a synthesiser, since none is installed. Each line is a claim about the published RTL.

Parameter corners, elaborated

7. Requirements Traceability — Closing The Matrix

Nineteen requirements, all with evidence. Four have no injected bug — three because the directed evidence is a measurement rather than a comparison, and one (REQ-FUNC-007) because refusing a request while busy has no mutation that is not also a mutation of something else. That is recorded rather than filled in with a ceremonial check.

8. The Final Review

The questions a reviewer should ask, and the answers this project can support.

What happens if configuration changes one cycle after acceptance? Nothing. REQ-CFG-001 captures every field at the accepting edge, G7 rewrites all six live fields five cycles into a frame and confirms the frame is unaffected, and B6 proves the check has teeth by making the edge count read the live width.

What exactly defines the first launch edge for CPHA = 1? The first SCLK transition of the frame, index 0, which is by definition the one that leaves the parked level. MOSI holds a defined low from the chip select until that transition, so the pin visibly differs from CPHA = 0 throughout the lead — which is what Figure 3 of Chapter 20.3 shows.

What prevents a 9-bit transfer from producing ten sample events? The edge count, not the datapath guard. The frame makes 2N transitions of which the decoder classifies N as sample events, so there is no tenth to produce. B3 proves the datapath guard is redundant by producing byte-identical traces; B3b proves it earns its place by adding two edges and watching the bit-count checks fire.

Which requirement proves the chip select cannot release early? REQ-TIM-003, measured at the pins in G2 as (cfg_lag + 1) × t_half and checked both against the specification's >= and against that exact form. B4 removes the lag and is caught by three detectors.

What part of the testbench is independent of the RTL implementation? The reference model, and deliberately only it. It contains a mask, a bit reversal and one multiplication — no shift register, no counter, no divider, no state. Its independence rests on one assumption, stated plainly: that the device model is right about what it returns. That is mitigated by checking the device's received word against an independent prediction too, by exercising the model in all four modes, and by B1/B2b/B7 confirming the pair detects faults a mutually-confused pair would not.

Which coverage hole would concern you most? None of the closed ones — the four missing bins. Chapter 20.6 names the four the model omits. Three of them are exercised by the directed suite, so they are tested and unmeasured; the default arm is neither. A reviewer should treat the 49-bin model as a list of what somebody thought of.

What does simulation still not prove about the board? The MISO round trip and the reset release. Chapter 20.4 computes the round trip at 19.0 ns for illustrative delays and derives cfg_div ≥ 1 — a number no tool in the flow checks, because the path leaves the chip and returns through a device the timing tool has never heard of. And reset release synchronisation cannot be established by simulation at all, for the scheduling reason Chapter 20.4 §5 sets out.

What assumption breaks first if SCLK were doubled? The MISO round trip, and it breaks immediately: at cfg_div = 0 the available half-period is 10.0 ns against 19.0 ns required. Not the logic, not the datapath, not any of the nineteen requirements — a path through a part on the other side of the board.

Which behaviour is guaranteed by RTL and which requires constraints? Guaranteed by RTL: every functional and mode requirement, the cycle-exact half-period, and the ordering of every phase. Requires constraints: that each pin's delay relative to clk is what the budget assumed, and that the reset release path is analysed for recovery and removal rather than setup. Requires neither and is nobody's tool's job: the round trip.

What happens if reset arrives mid-frame? REQ-RST-002, checked in G5: every select released in the same cycle, no done, busy low, SCLK at a defined low level — which for a mode-2 or mode-3 device is the wrong idle level until one cycle after release. That is recorded in Chapter 20.1 as a decision, and it is safe because the device is deselected and REQ-TIM-002 guarantees a settled level before the first edge that matters.

Known limitations, collected

   1  no multi-word transaction   the select releases every frame; a command-then-data
                                  sequence cannot be expressed. The most significant
                                  functional limit, and it follows from a stated non-goal
   2  no request queue            REQ-TIM-004 is a floor; the measured gap includes the
                                  requester's reaction time, so no throughput figure from
                                  this interface describes the controller alone
   3  lead/lag in half-periods    the delivered CS setup changes 4x between cfg_div 3
                                  and 0 with no change to the configuration field
   4  DATA_W limited to 31        cfg_width is five bits; a 32-bit datapath has an
                                  unreachable top bit
   5  reset release assumed       synchronisation is the integrator's, and simulation
                                  cannot check it
   6  cfg_div >= 1 for reads      a board-and-device budget, checked by no tool
   7  SCLK re-park can coincide   with the select assertion, reachable only immediately
      with CS assertion           after reset or after a CPOL change plus an immediate
                                  request. Safe via REQ-TIM-002, not structurally
   8  four coverage bins missing  abort, mid-frame reset, back-to-back, default arm

Eight limitations, all of them consequences of decisions made in the open rather than discoveries. That is the difference a specification makes: a review of this controller argues about trade-offs, not about mistakes.

9. The Loopback Bench

Published because it is instructive in its failure, not as a recommendation.

// spi_capstone_loop_tb.sv
//
// Chapter 20.7 -- the LOOPBACK bench, included because it is WRONG in an instructive
// way and because it is the first test almost everybody writes.
//
// It ties MISO to MOSI and checks that the received word equals the transmitted word.
// That is a real check: it exercises the divider, the counters, the chip select, the
// state machine, and the whole transmit and receive datapath end to end, at every mode
// and every width. It passes on the correct controller.
//
// AND IT CANNOT SEE A BIT-ORDER BUG, because the reversal applied on the way out is
// undone on the way back in. Nor can it see anything about which EDGE the data moved
// on, because both ends of the loop move on the same edge by construction.
//
// Chapter 20.7's injection matrix runs this bench beside the real one against the same
// mutations, and the column where it reports PASS is the point of the whole chapter:
// a test that passes is not evidence until you know what it is unable to fail on.

`timescale 1ns/1ps

module spi_capstone_loop_tb;

    reg         clk, rst_n;
    reg         cfg_cpol, cfg_cpha, cfg_lsb;
    reg  [4:0]  cfg_width;
    reg  [7:0]  cfg_div;
    reg  [1:0]  cfg_dev;
    reg  [3:0]  cfg_lead, cfg_lag, cfg_idle;
    reg         start, abort;
    reg  [15:0] tx_data;
    wire        busy, done, cfg_err;
    wire [15:0] rx_data;
    wire [4:0]  bits_done;
    wire        sclk, mosi;
    wire [3:0]  cs_n;

    integer n_chk, n_err;

    // THE LOOP. One wire, and every limitation of this bench follows from it.
    wire miso = mosi;

    spi_capstone_ctrl #(.DATA_W(16), .MIN_WIDTH(4), .NDEV(4)) dut (
        .clk(clk), .rst_n(rst_n),
        .cfg_cpol(cfg_cpol), .cfg_cpha(cfg_cpha), .cfg_lsb_first(cfg_lsb),
        .cfg_width(cfg_width), .cfg_div(cfg_div), .cfg_dev(cfg_dev),
        .cfg_lead(cfg_lead), .cfg_lag(cfg_lag), .cfg_idle(cfg_idle),
        .start(start), .tx_data(tx_data), .abort(abort),
        .busy(busy), .done(done), .cfg_err(cfg_err),
        .rx_data(rx_data), .bits_done(bits_done),
        .sclk(sclk), .mosi(mosi), .cs_n(cs_n), .miso(miso)
    );

    always #5 clk = ~clk;

    function [15:0] mask;
        input [4:0] w;
        reg [16:0] one;
        begin one = 17'd1; mask = ((one << w) - 17'd1); end
    endfunction

    task fire;
        input [15:0] d;
        begin
            @(negedge clk); tx_data = d; start = 1'b1;
            @(negedge clk); start = 1'b0;
        end
    endtask

    task wait_idle;
        input integer maxc; output gotd;
        integer g; reg seen;
        begin
            g=0; seen=1'b0;
            while (g < maxc) begin
                @(negedge clk); g=g+1;
                if (done) seen=1'b1;
                if (!busy && seen) g=maxc;
                else if (!busy && g>4) g=maxc;
            end
            gotd = seen;
        end
    endtask

    integer mi, oi, wi;
    reg [4:0] WID [0:2];
    reg gd;

    initial begin
        clk=1'b0; rst_n=1'b0; start=1'b0; abort=1'b0; tx_data=16'd0;
        cfg_cpol=1'b0; cfg_cpha=1'b0; cfg_lsb=1'b0; cfg_width=5'd8;
        cfg_div=8'd1; cfg_dev=2'd0; cfg_lead=4'd2; cfg_lag=4'd2; cfg_idle=4'd2;
        n_chk=0; n_err=0;
        WID[0]=5'd4; WID[1]=5'd8; WID[2]=5'd16;

        $display("=== Chapter 20.7 -- loopback bench (MISO tied to MOSI) ===");
        repeat (4) @(posedge clk);
        @(negedge clk); rst_n = 1'b1;
        repeat (2) @(posedge clk);

        for (mi = 0; mi < 4; mi = mi + 1)
          for (oi = 0; oi < 2; oi = oi + 1)
            for (wi = 0; wi < 3; wi = wi + 1) begin
                cfg_cpol = mi[1]; cfg_cpha = mi[0]; cfg_lsb = oi[0];
                cfg_width = WID[wi]; cfg_div = 8'd1; cfg_dev = 2'd0;
                fire(16'hB39D);
                wait_idle(8000, gd);
                n_chk = n_chk + 1;
                if (!gd) begin
                    n_err = n_err + 1;
                    $display("    FAIL no done, mode %0d w %0d", mi, WID[wi]);
                end
                n_chk = n_chk + 1;
                // The only comparison this bench can make. Note that the expected
                // value does not mention bit order ANYWHERE -- which is exactly why
                // it cannot detect a bit-order fault.
                if (rx_data !== (16'hB39D & mask(WID[wi]))) begin
                    n_err = n_err + 1;
                    $display("    FAIL loop mode %0d %0s w %0d got %04h exp %04h",
                             mi, oi[0] ? "lsb" : "msb", WID[wi], rx_data,
                             16'hB39D & mask(WID[wi]));
                end
            end

        $display("    24 loopback transfers, %0d failures", n_err);
        $display("=== SUMMARY checks=%0d negatives=0 failures=%0d : %0s ===",
                 n_chk, n_err, (n_err == 0) ? "PASS" : "FAIL");
        $finish;
    end

    initial begin
        #8000000;
        $display("    FATAL global timeout");
        $display("=== SUMMARY checks=%0d negatives=0 failures=%0d : FAIL ===", n_chk, n_err+1);
        $finish;
    end

endmodule

10. Summary

Thirteen deliberate defects were injected into the capstone controller and run against five detectors, with the baseline verified clean first and every mutation's anchor confirmed to have applied. Zero genuine escapes. One mutation — a redundant guard relaxed — was proven to be an equivalent mutant by byte-identical transcripts across all three benches rather than assumed to be one, and its observable sibling was caught by every detector.

The loopback bench missed nine of thirteen, including a bit-order fault that inverts every word in both directions. That is not a criticism of loopback testing; it is the measurement of what a test whose topology cancels its own transformation can and cannot establish, and it is why every suite in this module uses an external device model and non-palindromic data.

PSL contributed exactly one detection, and it was the only detection available: a frame that never completes leaves all eight safety properties true forever, and only a liveness operator names the obligation that was broken. The procedural benches saw a watchdog timeout.

Two gaps in the environment were found by injection and fixed. The device model answered any chip select, so asserting the wrong one went unnoticed until the scoreboard began capturing which index went low — and the same reduction weakness was then found in three more properties. The directed suite's parked-level check samples one cycle before the idle re-park lands, so what it validates is the frame-end level; the per-cycle property catches the idle level, and the two checks that looked redundant are not.

Nineteen requirements close with evidence. Eight limitations are recorded, every one a consequence of a decision made in the open — no multi-word transaction, no request queue, lead and lag in half-periods, DATA_W usable only to 31, reset release assumed, a read-path divider floor no tool checks, an SCLK re-park that can coincide with a select assertion, and four coverage bins nobody modelled.

11. The End Of The Track

Twenty modules and 129 chapters ago, SPI was four wires. It is still four wires, and the distance covered is in what can now be said about them: that CPOL and CPHA are two independent bits and a parity rather than a table to memorise; that the read path is limited by a round trip through a part on the other side of the board and not by any logic; that a testbench's expected values must be written in the specification's vocabulary or they will agree with the design's bugs; and that the most dangerous test is the one that passes for a reason nobody has examined.

The capstone is finished and it is not finished — it has eight recorded limitations and a specification that would need revising before the first of them is addressed. That is the normal state of a design that has had a review, and it is a better ending than a claim of completeness.

For anyone whose weak spots showed up here: the mode reasoning is Module 3, the timing that decides the divider is Module 15, the debugging method is Module 18, and the integration context is Module 19. The method itself — specify, commit, model independently, count exemptions, review the generator, classify holes, inject defects — is not about SPI, and the most useful thing to do with it is to apply it to a protocol this track never covered.