Wait States

Chapter 6.4 built a registered slave that answers one cycle late. Real peripherals are less uniform: a value may take several cycles to produce, and the whole transfer sits on the bus while it does.

What happens when the slave cannot complete the read immediately, and what must stay coherent while it cannot?

1. What the Master Must Hold

The master's obligations during a wait are not new — they are RULE 3.60's qualification read across a multi-cycle presentation, which Chapter 5.3 established generally. What is new is that a read has a specific list.

WE_O is the row worth pausing on. A transfer whose direction flips while outstanding is not one transfer. In Chapter 6.1's master we_o is a hard-wired constant precisely so this cannot happen — a constant satisfies a stability obligation more convincingly than a register does.

And what the master must not do: capture DAT_I, report completion, or release the bus. Chapter 6.3 measured a master that captured during the wait and returned zero on every read.

The master needs no "wait-state mode". It simply does not complete until a termination arrives. wb_read_master has no special handling at all — the level-sampled termination in its always_ff is the whole implementation, and it works for zero waits and for a hundred.

2. What the Slave Must Hold

Less is required of the slave, but one thing matters.

It must not terminate before its data is ready. RULE 3.65 makes the termination and the data one statement (Chapter 6.4 §1). A slave that acknowledges early has promised something it cannot deliver.

It must eventually terminate. The STB_O signal description says a slave asserts one of ACK_I, ERR_I or RTY_I "in response to every assertion of the STB_O signal". Silence forever is the one response that is never acceptable — Chapter 4.8 made this the argument for erroring on unknown offsets rather than ignoring them.

It may take as long as it needs. No rule bounds slave latency.

3. The Side-Effecting Read

This is the section the chapter exists for, and it is where waiting stops being a timing detail.

Chapter 6.1 §2 noted that a read is not necessarily passive: read-to-clear status bits, pop-on-read FIFOs, and counters that latch or advance when observed are ordinary peripheral designs.

Combine a side effect with a wait and the presented-versus-accepted distinction becomes load-bearing.

Chapter 5.3 measured a write firing four times for one transfer on a fully conformant bus. For a read-to-clear register the same bug is worse, because the discarded state is events that were never reported to anyone.

4. RTL — A Delayed Read Slave With a Side Effect

// ─────────────────────────────────────────────────────────────────────────
// wb_delayed_read_slave — the running peripheral, with latency and a
// read-to-clear register.
//
// PURPOSE. Two things at once, because they interact:
//   (a) a read that takes LAT cycles to produce its value;
//   (b) a register whose semantics are READ-TO-CLEAR.
//
// WHAT THE LATENCY MODEL IS. A down-counter, which is NOT realistic
// hardware latency — no peripheral is slow because of a counter. It is an
// EDUCATIONAL MODEL standing in for the real sources:
//   * an SRAM or register-file with a pipelined output stage
//   * a status value assembled from logic that needs a cycle to settle
//   * a bridge crossing to another clock or another bus
//   * a measurement that must be sampled and converted
// What matters for the protocol is only that the answer is not ready yet.
//
// Reset: SYNCHRONOUS, ACTIVE HIGH (RULES 2.30, 3.00).
// ─────────────────────────────────────────────────────────────────────────
module wb_delayed_read_slave #(
  parameter int unsigned OFF_AW = 3,
  parameter int unsigned DW     = 32,
  parameter int unsigned LAT    = 3        // cycles before the answer
) (
  input  logic              clk_i,
  input  logic              rst_i,
  input  logic              cyc_i,
  input  logic              stb_i,
  input  logic              we_i,
  input  logic [OFF_AW-1:0] adr_i,
  output logic [DW-1:0]     dat_o,
  output logic              ack_o,
  output logic              err_o,
  // device side
  input  logic [7:0]        event_set_i,   // events arriving from hardware
  output logic [7:0]        irq_flags_o,   // observable, for QA
  output logic [7:0]        clears_o       // side-effect COUNT, for QA
);
  localparam logic [OFF_AW-1:0] O_STATUS = 3'd0;
  localparam logic [OFF_AW-1:0] O_COUNT  = 3'd1;
  localparam logic [OFF_AW-1:0] O_IRQ    = 3'd5;   // READ-TO-CLEAR
  localparam logic [OFF_AW-1:0] O_ID     = 3'd4;

  localparam logic [DW-1:0] ID_VALUE = 32'h5742_0601;

  logic [DW-1:0] count_q;
  logic [7:0]    irq_q;
  assign irq_flags_o = irq_q;

  logic xfer;
  assign xfer = cyc_i & stb_i;                  // RULES 3.30 & 3.35

  // ── OFFSET LEGALITY ───────────────────────────────────────────────────
  logic known_off;
  always_comb begin
    unique case (adr_i)
      O_STATUS, O_COUNT, O_IRQ, O_ID: known_off = 1'b1;
      default:                        known_off = 1'b0;
    endcase
  end

  logic illegal;
  assign illegal = ~known_off | we_i;           // this slave is read-only

  // ── THE LATENCY COUNTER ───────────────────────────────────────────────
  // Counts only while a transfer is presented, and RESETS the moment it is
  // not. That reset is what satisfies RULE 3.50 structurally: when STB_I
  // goes away the counter returns to zero and no termination can be
  // asserted, so a stale response cannot outlive its transfer.
  // `ready` is declared and assigned BEFORE the block that uses it.
  // Icarus rejects a reference that precedes its declaration, and the
  // ordering also reads better: the readiness condition is what the
  // counter is counting towards.
  logic [7:0] waited_q;
  logic       ready;
  assign ready = (waited_q >= 8'(LAT));

  always_ff @(posedge clk_i) begin
    if (rst_i)        waited_q <= '0;
    else if (!xfer)   waited_q <= '0;
    else if (!ready)  waited_q <= waited_q + 8'd1;
    else              waited_q <= '0;           // one termination per transfer
  end

  // ── TERMINATION ───────────────────────────────────────────────────────
  // ACCEPTED is true for exactly one cycle per transfer. PRESENTED (xfer)
  // is true for LAT+1 of them. Everything with a consequence hangs off
  // ACCEPTED.
  assign err_o = xfer & ready &  illegal;
  assign ack_o = xfer & ready & ~illegal;

  // ── READ DATA ─────────────────────────────────────────────────────────
  // RULE 3.65: qualified by the termination, so it is driven for exactly
  // one cycle of a multi-cycle transfer — which is precisely the window
  // Chapter 6.3's master captures in.
  always_comb begin
    dat_o = '0;
    if (ack_o) begin
      unique case (adr_i)
        O_STATUS: dat_o = {24'd0, 8'hA5};
        O_COUNT:  dat_o = count_q;
        O_IRQ:    dat_o = {24'd0, irq_q};
        O_ID:     dat_o = ID_VALUE;
        default:  dat_o = '0;
      endcase
    end
  end

  // ── THE SIDE EFFECT ───────────────────────────────────────────────────
  // read_accept is the acceptance event: one cycle, at the termination.
  // Gating the clear on it — rather than on xfer — is the whole
  // correctness of the read-to-clear register.
  //
  // Replace read_accept with xfer below and this slave clears the flags on
  // all LAT+1 cycles of every read, discarding events that arrive during
  // the wait and were never returned to anyone (Section 3).
  logic read_accept;
  assign read_accept = ack_o & (adr_i == O_IRQ);

  always_ff @(posedge clk_i) begin
    if (rst_i) begin
      count_q  <= '0;
      irq_q    <= '0;
      clears_o <= '0;
    end else begin
      count_q <= count_q + 32'd1;

      // Events arriving from hardware are OR-ed in every cycle. The clear
      // and the set are combined in one expression so an event arriving in
      // the same cycle as a read is not silently lost: it is set after the
      // clear, and will be reported by the NEXT read.
      //
      // Getting this wrong in the other direction — clearing after setting
      // — would drop an event that arrived exactly at the acceptance edge.
      if (read_accept) begin
        irq_q    <= event_set_i;              // cleared, then this cycle's events
        clears_o <= clears_o + 8'd1;
      end else begin
        irq_q    <= irq_q | event_set_i;
      end
    end
  end
endmodule

Reading it

Purpose. Make a read take time, and make one register change state when it is read — so the interaction between the two is visible.

Interface. A standard Wishbone slave plus three device-side signals. irq_flags_o and clears_o are not Wishbone signals; they exist so a testbench can measure what happened, and a production peripheral would not export them.

State. The latency counter, the free-running counter, the interrupt flags, and a side-effect count for QA.

Combinational logic. xfer, offset legality, ready, the two terminations, the read multiplexer, and read_accept.

Sequential logic. The latency counter; the flags register with its combined set/clear; the QA counter.

Read start. No start event — only the first cycle in which xfer is true. A slave that needs a start event must manufacture one, and here waited_q transitioning from zero is it.

Address. adr_i is consumed combinationally in the cycle of the termination. Because the master holds it still under RULE 3.60, reading it at the termination rather than latching it is safe — and Chapter 6.4 §4 showed the alternative choice and why a registered slave should latch instead.

Data return. Gated on ack_o, driven for one cycle.

ACK. Asserted when waited_q reaches LAT, and only while the transfer is still presented.

Capture. The master's, at the termination edge.

Waiting. waited_q counts 0, 1, …, LAT. xfer is true throughout; ack_o for one cycle.

Reset. Synchronous, active high. waited_q and the flags clear; note the counter also clears whenever the transfer goes away, which is what prevents a response outliving its request.

Failure modes. Section 7.

Simplifications. Fixed latency rather than a real source. The read-to-clear register clears all flags rather than only those reported — a real design often clears write-one-to-clear style instead, which avoids the whole hazard and is worth preferring where the interface allows it.

5. Waveform — Four Cycles Presented, One Accepted

Cycles 2 to 5 are identical on the bus. Same qualifiers, same address, same direction. Nothing distinguishes them from one another — which is exactly why a side effect keyed on them fires four times.

Cycle 5 is the only special one, and what makes it special is produced by the slave: its own acknowledge.

irq holds 0x05 through the whole wait and clears at the edge after the acknowledge. clears goes 0 → 1 and stops.

With the side effect gated on xfer instead, clears reaches 4 and irq is zero from cycle 3 onward — while CYC_O, STB_O, ADR_O, ACK_I and DAT_I are pixel-for-pixel identical. The bus cannot see the difference.

6. Simulation — Coherence and the Single Clear

Simulation C — what stays coherent during a wait.

  === SIMULATION C - delayed read (LAT=3) ===
    cycles the transfer was presented       4
    ADR_O changes while outstanding         0
    WE_O changes while outstanding          0
    SEL_O changes while outstanding         0
    CYC_O deasserted while outstanding      0
    STB_O deasserted while outstanding      0
    master completions before termination   0
    value captured                     0x00000007
    captured in the termination cycle?     yes

Every qualified signal held still for all four cycles, and the master completed exactly once, at the termination.

The captured value is 0x07, not the 0x05 that was pending when the read began — an event worth 0x02 arrived during the wait and was OR-ed into the flags before the acceptance. That is correct: the read reports what is pending at the moment it is accepted, not at the moment it was requested. Section 6's second table shows why that matters.

Simulation D — the side effect, correct and broken.

Two otherwise-identical slaves read the interrupt register once, with an event arriving during the wait.

  === SIMULATION D - read-to-clear with LAT=3 ===
                                   gated on ACCEPT   gated on PRESENT
    cycles presented                      4                4
    terminations returned                 1                1
    CLEAR fired                           1                4
    flags pending at the start         0x05             0x05
    event arriving during the wait     0x02             0x02
    events the read should report         3                3
    value returned to the master       0x07             0x00
    events actually reported              3                0
    flags left pending afterwards      0x00             0x00
    EVENTS LOST                           0                3

The correct slave reported all three events and lost none. Two were pending when the read began; a third arrived during the wait and was OR-ed in before the acceptance. All three came back in 0x07, and all three were then cleared — correctly, because all three had been delivered.

The broken slave reported none and lost all three. It cleared on the first presented cycle, discarding the two that were pending. The third arrived in cycle 2 and was cleared in cycle 2. By the acceptance cycle there was nothing left to return, so the master received 0x00.

Note what the two slaves have in common: four presented cycles, one termination, identical bus traces, and both masters reporting a successful read. EVENTS LOST is internal to the slave and appears nowhere on the bus.

And note the shape of the failure. It is not that the broken slave returned a wrong value — 0x00 is a perfectly plausible "no interrupts pending". It returned a value that was true only because the slave itself had destroyed the evidence, which is why software has no way to detect it.

7. Failure Modes and Discriminating Evidence

Symptom: a peripheral's interrupts are occasionally missed, more often under load.

Candidate causes. A read-to-clear register whose clear is gated on the transfer being presented rather than accepted.

Discriminating evidence. Compare the number of clears against the number of completed reads, not against cycles. If the count tracks the wait length rather than the read count, that is conclusive. The load correlation is the clue that points here: load introduces latency, latency lengthens the wait, and the wait is the multiplier.

Likely RTL location. The side effect's enable — a missing termination term.

Property. P2 in Section 8.

Symptom: a read returns the wrong register, only when the slave is slow.

Candidate causes. The master's address does not hold still — rewritten while waiting, or driven from an unlatched client input.

Discriminating evidence. Watch ADR_O across the whole strobe assertion. Any change before the termination is a RULE 3.60 violation. Chapter 6.2 §7 separates the two causes by whether the bus address tracks the client.

Property. P1.

Symptom: the master completes before the slave produced data.

Candidate causes. The master captured during the wait rather than at the termination, or the slave terminated before its value was ready.

Discriminating evidence. Check whether the slave's DAT_O was valid in the acknowledged cycle. Valid at the slave and wrong at the master isolates the capture (Chapter 6.3); invalid at the slave means it acknowledged too early, which is the coherence failure of Chapter 6.4 §1.

Symptom: a read hangs and the slave's latency counter is stuck.

Candidate causes. The counter advances only under a condition that stopped being true — for instance gated on something other than the presented transfer.

Discriminating evidence. xfer asserted at the slave with waited_q not advancing. That is conclusive and localises inside the slave immediately.

Likely RTL location. The counter's increment condition.

Symptom: the master gives up and the system continues incorrectly.

Candidate causes. A master implementing a timeout by negating CYC_O.

Discriminating evidence. CYC_O negated with STB_O still asserted, or both dropped before any termination.

Correct approach. There is no protocol-level abandon — Chapter 5.2 §2 covered why, and the slave may have begun a side effect the master will never learn about. RECOMMENDATION 3.10 puts the watchdog in the interconnect so the transfer is terminated rather than orphaned.

8. Verification

// ─────────────────────────────────────────────────────────────────────────
// wb_read_wait_checker — properties for a waited read.
//
// P1 is SPECIFICATION (RULE 3.60 across a multi-cycle presentation).
// P2 is a DESIGN OBLIGATION — the specification does not know what side
// effects a slave has, so it cannot require them to happen once. It is
// nonetheless the property that catches the bug this chapter is about.
// P3 is SPECIFICATION (RULE 3.50).
// ─────────────────────────────────────────────────────────────────────────
module wb_read_wait_checker #(
  parameter int unsigned AW = 30,
  parameter int unsigned DW = 32
) (
  input logic          clk_i,
  input logic          rst_i,
  input logic          cyc_i,
  input logic          stb_i,
  input logic          we_i,
  input logic [AW-1:0] adr_i,
  input logic [DW/8-1:0] sel_i,
  input logic          ack_o,
  input logic          err_o,
  input logic [7:0]    clears_q     // white-box: side-effect counter
);
  default disable iff (rst_i);

  logic xfer, terminated;
  assign xfer       = cyc_i & stb_i;
  assign terminated = ack_o | err_o;

  // P1 — SPECIFICATION (RULE 3.60). Everything the strobe qualifies holds
  //      still for the whole wait. The end boundary is the TERMINATION;
  //      omitting it would forbid a legal back-to-back transfer.
  property p_request_frozen_while_waiting;
    @(posedge clk_i) (xfer && !terminated)
      |=> ($stable(adr_i) && $stable(we_i) && $stable(sel_i) &&
           cyc_i && stb_i);
  endproperty
  a_request_frozen_while_waiting :
    assert property (p_request_frozen_while_waiting)
    else $error("RULE 3.60: a qualified signal moved during the wait");

  // P2 — DESIGN OBLIGATION, and the one this chapter exists for. A side
  //      effect happens only in an ACCEPTED cycle, never in a merely
  //      PRESENTED one. Fires on the FIRST extra clear rather than when a
  //      lost-interrupt symptom eventually surfaces.
  //
  //      No bus-level property can catch this: the bus is conformant
  //      throughout, so the checker needs the counter.
  property p_side_effect_once_per_accept;
    @(posedge clk_i) $changed(clears_q) |-> $past(ack_o);
  endproperty
  a_side_effect_once_per_accept :
    assert property (p_side_effect_once_per_accept)
    else $error("side effect fired outside an accepted transfer");

  // P3 — SPECIFICATION (RULE 3.50, OBSERVATION 3.10). The termination is
  //      negated in response to STB_I negating, so a slow response cannot
  //      outlive the transfer that provoked it.
  property p_no_termination_without_strobe;
    @(posedge clk_i) !stb_i |-> !terminated;
  endproperty
  a_no_termination_without_strobe :
    assert property (p_no_termination_without_strobe)
    else $error("RULE 3.50: termination asserted with STB_I negated");
endmodule

P2 requires white-box access and there is no alternative. The bus during the lost-interrupt bug is fully conformant — one presented transfer, one termination, stable signals, correct data. This is the fifth time this course has met that boundary, after lost atomicity (4.9), repeated writes (5.3), the RULE 3.55 stall (5.4) and capture-edge bugs (6.3). Chapter 5.8 §8 collected the pattern: conformance is necessary, finite, and not sufficient.

Tooling limitation. Icarus Verilog has no SVA support and cannot execute any of these. They are reviewed by inspection; the slave is elaborated and both simulations above were run.

9. Common Mistakes

"No ACK means something went wrong."

Wrong mental model: absence of a response is a failure.

Concrete bug: a master that treats a slow slave as an error and abandons the transfer by negating CYC_O.

Observable evidence: transfers abandoned against slower peripherals; a slave that has begun a side effect nobody will learn the outcome of.

Correct model: withholding the termination is how a slave throttles the cycle. It is neither error nor retry, and the master has no legal abandon.

"A read is safe to repeat, so a wait cannot hurt."

Wrong mental model: reads are idempotent.

Concrete bug: a read-to-clear register whose clear fires once per presented cycle.

Observable evidence: interrupts lost under load, with a perfectly conformant bus and a successful read returning a plausible value.

Correct model: a read may change state, and a side effect belongs to the accepted transfer, not the presented one.

"The master needs a wait-state mode."

Wrong mental model: waiting is a distinct protocol feature to be implemented.

Concrete bug: none directly — but it invites special-case logic that diverges from the simple path and is tested less.

Correct model: a master that samples the termination as a level and holds its outputs still is already correct for any latency. wb_read_master has no wait-state logic at all.

10. Interview Reasoning

11. Understanding Check

12. What's Next

Waiting is now fully specified for a read: the slave throttles by withholding, the master freezes everything it drives, and any side effect belongs to the accepted transfer rather than the presented one.

Every chapter so far has explained a mechanism and then shown a waveform of it. The final chapter reverses that. Given an unfamiliar trace and no explanation, the question becomes what it actually shows — and where the evidence says the fault lies.

How do you read an arbitrary Wishbone read off a waveform, and localise a failure from evidence alone?

Chapter 6.6 — Waveform Analysis answers it. The full path is on the Wishbone curriculum index.