Skip to content

PCIe · Module 22

Credit Bottlenecks — A Healthy Link That Cannot Legally Transmit

The Link is in L0, there are no errors, the transmitter is idle and a packet is queued — and sending it would be illegal. Proving that from counters requires per-class attribution that only counts when a packet of that class is actually pending.

Chapter 22.1 counted no_credit cycles. It never said which credit was missing.

This chapter is that investigation, and it starts from the most disorienting state in PCIe performance work: the Link is trained and in L0, no errors are logged, no replay is in progress, the transmitter has nothing in flight, a packet is sitting at the head of the queue — and transmitting it would violate the protocol.

Nothing is broken. The transmitter simply does not own permission.

1. Sources, Scope, and the Six Chapters This One Stands On

2. Permission, Not Bandwidth

Four lines, then this chapter moves on (Module 16 owns the rest).

A credit is permission to occupy receiver buffer space — advertised by the receiver, spent by the transmitter, returned when the receiver frees the space (16.1 §1, 16.6 §1).

There are six pools because there are three TLP classes and two kinds of information in each (16.1 §4): PH/PD, NPH/NPD, CPLH/CPLD.

Header and data are separate resources. A packet needs both for its class — and one can be exhausted while the other is plentiful (§3).

And it is point-to-point (16.1 §1). Credits describe the link partner's buffer, not the ultimate destination's — which is why a switch's downstream congestion does not appear directly in your credit counters.

3. Starvation Is Not Failure

4. Which Resource Blocked First

Credits are not the only permission a packet needs, and the performance question is always which one ran out first.

ResourceExhausted meansOwner
NPH creditcannot issue the request at all16.3
Tagsno identity available for a new outstanding request20.3 §3
local completion contextnowhere to put the data when it returns16.4 §3
PD creditcannot send data-bearing Posted traffic16.2
link readinessnothing may transmit18.6

These produce identical external symptoms and completely different fixes. A design with free Tags and NPH = 0 issues nothing; so does one with NPH available and no free Tags. From outside, both are "reads stopped".

Which is why §9's instrumentation records the first blocking resource per attempt rather than a set of flags. A cycle in which three resources are all unavailable ranks them, exactly as Chapter 22.1 §6 ranks cycle categories — and for the same reason: a set of simultaneously-true conditions cannot be prioritized after the fact.

Chapter 16.3 §4 already established the conceptual half of this — NPH is not the outstanding-request table. This chapter adds the measurement: a counter per resource, incremented only for the one that actually blocked.

5. No Work Is Not No Credit

6. The Credit Window Ceiling

7. Head-of-Line Credit Blocking, Quantified

One transmit FIFO holds packets of all three classes. The head is a Posted packet needing PD, and PD is zero. Behind it sits a Non-Posted request needing only NPH — which is available.

Nothing moves. The NP packet is eligible and unreachable, blocked by a packet in a different class with a different resource problem.

This is an implementation consequence, not a PCIe requirement. The protocol does not mandate a single queue; PCIe requires the classes to be independently flow-controlled precisely so an implementation can keep them separate (16.1 §4).

§11 Model 16 measured the cost over 100,000 cycles with mixed-class queues and a 35% per-packet block probability:

Transmit structureLost issue opportunities
single shared FIFO34,768 (34.8%)
per-class queues6,332 (6.3%)
recovered by structure alone28.4% of all cycles

28.4% of cycles is a larger effect than most protocol-level optimizations produce, and it costs nothing but queue structure — which is why §9's scheduler evaluates eligibility across classes rather than inspecting one head.

And the residual 6.3% is meaningful too. Per-class queues do not eliminate blocking; they reduce it to cycles when every class at its head is blocked. That residue is the real credit bottleneck, and §5's qualification rule is what makes it measurable.

8. The Waveform

Two different starvations, one healthy Link, and a class that keeps moving

10 cycles
Ten cycles of a PCIe transmitter. Link ready is high throughout and a Posted packet is pending throughout. Posted header credit is unavailable in cycles 2, 3 and 4. Posted data credit is unavailable in cycles 6, 7 and 8. Posted transmission occurs in cycles 0, 1, 5 and 9. A flow control update arrives at cycle 4 and at cycle 8. A Non-Posted packet transmits in cycles 6 and 7 while Posted traffic is data-credit starved. Credit blocked is asserted in cycles 2, 3, 4, 6, 7 and 8.PH starved — PD is plentifulPH starved — PD isplentifulFC update returns header creditFC update returns headercreditPD starved — PH is plentifulPD starved — PH isplentifulanother class still transmitsanother class stilltransmitsclklink_readyp_pendingph_availpd_availp_txnp_txfc_updatecred_blockt0t1t2t3t4t5t6t7t8t9
Figure 1 — ten cycles at a transmitter whose Link is healthy throughout. A Posted packet is pending in every cycle. Transmission stops twice for two different reasons: header credit is exhausted while data credit remains, and later data credit is exhausted while header credit remains. Each Flow Control update makes the class eligible again, and a Non-Posted packet transmits during the data-credit stall because its own pools are healthy.

Four things to read out of the figure.

link_ready never falls. Every stalled cycle here is a healthy Link (§3). A design that reports "link stall" for these cycles has misnamed the finding.

Cycles 2–4 and 6–8 are different bugs to chase. Header starvation points at the receiver's header buffer or update policy; data starvation points at payload buffering and packet sizes — and the two are conflated by any counter that tracks one aggregate "credit" figure (§9).

cred_block is asserted only because p_pending is high (§5). Had the queue been empty in cycles 2–4, those would be no_work cycles and this signal would stay low — the difference §11 measured at 81.8%.

And cycles 6–7 are the argument for per-class queues. Posted traffic is data-starved and Non-Posted traffic transmits anyway (§7). In a single-FIFO design with a Posted packet at the head, those two cycles are lost.

9. RTL — Cost, Ledger, Reservation, Scheduling, Attribution

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SYNTHESIZABLE. Credit classes and cost derivation.
// SOURCED via Chapter 16.1 §1: six tracked types; the data credit unit is
// 4 DW = 16 bytes; n = Roundup(Length / FC unit size).
package credit_pkg;
 
  parameter int N_CLASS      = 3;                    // P, NP, Cpl (VC0 only, §1)
  parameter int CLASS_W      = (N_CLASS <= 1) ? 1 : $clog2(N_CLASS);
  parameter int CREDIT_BYTES = 16;                   // SOURCED: 4 DW
  parameter int CRED_W       = 12;                   // credit counter width
  parameter int LEN_W        = 16;                   // payload bytes
  parameter int N_PROD       = 3;                    // producers contending
  parameter int PROD_W       = (N_PROD <= 1) ? 1 : $clog2(N_PROD);
 
  typedef enum logic [CLASS_W-1:0] {
    CL_POSTED = '0,
    CL_NONPOSTED = CLASS_W'(1),
    CL_COMPLETION = CLASS_W'(2)
  } tlp_class_e;
 
  // =================================================================
  // THE CEIL DIVISION, WITH THE ZERO GUARD.
  // A header-only TLP carries no data and therefore needs NO data
  // credit. The naive form ((bytes-1)/U)+1 UNDERFLOWS at bytes = 0 and
  // in unsigned 32-bit arithmetic returns 268,435,456 units (§11
  // Model 12) -- a number large enough to look like "plenty available".
  // It is correct for every non-zero input, which is why it survives
  // every test that does not include a header-only packet.
  // =================================================================
  function automatic logic [CRED_W-1:0] data_credits(input logic [LEN_W-1:0] bytes);
    if (bytes == '0) return '0;                                  // THE GUARD
    return CRED_W'((bytes + LEN_W'(CREDIT_BYTES - 1)) / CREDIT_BYTES);
  endfunction
 
  // One header credit per TLP, per Chapter 16.1's header-unit definition.
  function automatic logic [CRED_W-1:0] header_credits(input logic has_tlp);
    return has_tlp ? CRED_W'(1) : CRED_W'(0);
  endfunction
 
  typedef struct packed {
    logic             valid;
    tlp_class_e       tlp_class;
    logic             has_data;
    logic [LEN_W-1:0] payload_bytes;
    logic [CRED_W-1:0] need_hdr;
    logic [CRED_W-1:0] need_data;
  } pkt_cost_t;
 
endpackage
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import credit_pkg::*;
 
// SYNTHESIZABLE. Packet credit calculator (§9).
// Header and data are computed INDEPENDENTLY -- a packet may need header
// credit and zero data credit, and collapsing them is mutation 27.
module packet_credit_cost (
  input  logic             pkt_valid,
  input  tlp_class_e       pkt_class,
  input  logic             pkt_has_data,
  input  logic [LEN_W-1:0] pkt_bytes,
  output pkt_cost_t        cost
);
  always_comb begin
    cost.valid         = pkt_valid;
    cost.tlp_class     = pkt_class;
    cost.has_data      = pkt_has_data;
    cost.payload_bytes = pkt_bytes;
    cost.need_hdr      = header_credits(pkt_valid);
    // has_data gates the computation as well as the guard inside it --
    // belt and braces, because this is the corner that ships broken.
    cost.need_data     = (pkt_valid && pkt_has_data) ? data_credits(pkt_bytes)
                                                     : CRED_W'(0);
  end
endmodule
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import credit_pkg::*;
 
// SYNTHESIZABLE. Multi-class credit ledger.
// Three states per class: AVAILABLE (may be reserved), RESERVED (owned by
// a specific packet), SPENT (transmitted, awaiting an FC update).
// Arithmetic is WIDENED and checked BEFORE commit -- unsigned underflow
// turns 0 minus 1 into the maximum value (§11 Model 18).
module credit_ledger (
  input  logic clk,
  input  logic rst_n,
 
  input  logic [CRED_W-1:0] advertised_hdr  [N_CLASS],
  input  logic [CRED_W-1:0] advertised_data [N_CLASS],
  input  logic              init_done,
 
  // reservation grant from the manager below -- never from a producer
  input  logic              resv_commit,
  input  tlp_class_e        resv_class,
  input  logic [CRED_W-1:0] resv_hdr,
  input  logic [CRED_W-1:0] resv_data,
 
  input  logic              resv_release,     // cancellation
  input  logic              resv_consume,     // transfer completed
 
  // normalized FC return (Chapter 16.6 owns the real DLLP path)
  input  logic              fc_valid,
  input  logic              fc_ready,
  input  tlp_class_e        fc_class,
  input  logic [CRED_W-1:0] fc_hdr_delta,
  input  logic [CRED_W-1:0] fc_data_delta,
 
  output logic [CRED_W-1:0] avail_hdr  [N_CLASS],
  output logic [CRED_W-1:0] avail_data [N_CLASS],
  output logic [CRED_W-1:0] reserved_hdr  [N_CLASS],
  output logic [CRED_W-1:0] reserved_data [N_CLASS],
  output logic              err_underflow,    // sticky
  output logic              err_overflow      // sticky
);
  logic [CRED_W-1:0] ah_q [N_CLASS], ad_q [N_CLASS];
  logic [CRED_W-1:0] rh_q [N_CLASS], rd_q [N_CLASS];
  logic uf_q, of_q;
 
  always_comb begin
    for (int c = 0; c < N_CLASS; c++) begin
      avail_hdr[c]     = ah_q[c];  avail_data[c]    = ad_q[c];
      reserved_hdr[c]  = rh_q[c];  reserved_data[c] = rd_q[c];
    end
  end
  assign err_underflow = uf_q;
  assign err_overflow  = of_q;
 
  // Widened by one bit so the borrow/carry is VISIBLE rather than wrapped.
  function automatic logic [CRED_W:0] wsub(input logic [CRED_W-1:0] a,
                                           input logic [CRED_W-1:0] b);
    return {1'b0, a} - {1'b0, b};
  endfunction
  function automatic logic [CRED_W:0] wadd(input logic [CRED_W-1:0] a,
                                           input logic [CRED_W-1:0] b);
    return {1'b0, a} + {1'b0, b};
  endfunction
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      for (int c = 0; c < N_CLASS; c++) begin
        ah_q[c] <= '0; ad_q[c] <= '0; rh_q[c] <= '0; rd_q[c] <= '0;
      end
      uf_q <= 1'b0; of_q <= 1'b0;                 // reset FABRICATES NOTHING
    end else begin
      if (init_done)
        for (int c = 0; c < N_CLASS; c++) begin
          if (ah_q[c] == '0 && ad_q[c] == '0 && rh_q[c] == '0 && rd_q[c] == '0) begin
            ah_q[c] <= advertised_hdr[c];
            ad_q[c] <= advertised_data[c];
          end
        end
 
      // ---- RESERVE: available -> reserved -------------------------
      if (resv_commit) begin
        automatic logic [CRED_W:0] nh = wsub(ah_q[resv_class], resv_hdr);
        automatic logic [CRED_W:0] nd = wsub(ad_q[resv_class], resv_data);
        if (nh[CRED_W] || nd[CRED_W]) uf_q <= 1'b1;      // BORROW -> report
        else begin
          ah_q[resv_class] <= nh[CRED_W-1:0];
          ad_q[resv_class] <= nd[CRED_W-1:0];
          rh_q[resv_class] <= rh_q[resv_class] + resv_hdr;
          rd_q[resv_class] <= rd_q[resv_class] + resv_data;
        end
      end
 
      // ---- RELEASE (cancel): reserved -> available -----------------
      if (resv_release) begin
        if (rh_q[resv_class] < resv_hdr || rd_q[resv_class] < resv_data)
          uf_q <= 1'b1;                                   // release without reserve
        else begin
          rh_q[resv_class] <= rh_q[resv_class] - resv_hdr;
          rd_q[resv_class] <= rd_q[resv_class] - resv_data;
          ah_q[resv_class] <= ah_q[resv_class] + resv_hdr;
          ad_q[resv_class] <= ad_q[resv_class] + resv_data;
        end
      end
 
      // ---- CONSUME (transferred): reserved -> spent -----------------
      // Credit stays SPENT. It returns only via an FC update (16.6 §1).
      if (resv_consume) begin
        if (rh_q[resv_class] < resv_hdr || rd_q[resv_class] < resv_data) uf_q <= 1'b1;
        else begin
          rh_q[resv_class] <= rh_q[resv_class] - resv_hdr;
          rd_q[resv_class] <= rd_q[resv_class] - resv_data;
        end
      end
 
      // ---- FC RETURN: applied ONCE, on the accepted beat ------------
      if (fc_valid && fc_ready) begin
        automatic logic [CRED_W:0] nh = wadd(ah_q[fc_class], fc_hdr_delta);
        automatic logic [CRED_W:0] nd = wadd(ad_q[fc_class], fc_data_delta);
        // Returning above what was advertised means the loop is broken.
        if ((nh + {1'b0, rh_q[fc_class]}) > {1'b0, advertised_hdr[fc_class]} ||
            (nd + {1'b0, rd_q[fc_class]}) > {1'b0, advertised_data[fc_class]})
          of_q <= 1'b1;
        else begin
          ah_q[fc_class] <= nh[CRED_W-1:0];
          ad_q[fc_class] <= nd[CRED_W-1:0];
        end
      end
    end
  end
endmodule
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import credit_pkg::*;
 
// SYNTHESIZABLE. THE FLAGSHIP BLOCK. Centralized reservation.
// Producers may OBSERVE credit; only this block may GRANT it. Several
// producers each seeing "one credit available" and each deciding to send
// is the double-spend Chapter 16.5 §4 proves; §11 Model 14 measured that
// producers collectively want more than is available in 65.6% of cycles.
module reservation_manager (
  input  logic clk,
  input  logic rst_n,
 
  input  logic       [N_PROD-1:0] req_valid,
  input  tlp_class_e              req_class [N_PROD],
  input  logic [CRED_W-1:0]       req_hdr   [N_PROD],
  input  logic [CRED_W-1:0]       req_data  [N_PROD],
 
  input  logic [CRED_W-1:0]       avail_hdr  [N_CLASS],
  input  logic [CRED_W-1:0]       avail_data [N_CLASS],
 
  output logic [N_PROD-1:0]       grant,
  output logic                    commit,
  output tlp_class_e              commit_class,
  output logic [CRED_W-1:0]       commit_hdr,
  output logic [CRED_W-1:0]       commit_data,
  output logic [N_PROD-1:0]       blocked_by_credit
);
  logic [PROD_W-1:0] rr_q;
  logic [N_PROD-1:0] eligible;
 
  // Eligibility needs BOTH resources for the packet's own class (§2).
  always_comb
    for (int i = 0; i < N_PROD; i++)
      eligible[i] = req_valid[i]
                 && (avail_hdr [req_class[i]] >= req_hdr [i])
                 && (avail_data[req_class[i]] >= req_data[i]);
 
  // A producer is credit-blocked only if it HAS a request (§5).
  always_comb
    for (int i = 0; i < N_PROD; i++)
      blocked_by_credit[i] = req_valid[i] && !eligible[i];
 
  // EXACTLY ONE grant per cycle. This is the whole point: the decision is
  // made once, centrally, against one snapshot of the ledger.
  logic [PROD_W-1:0] pick; logic found;
  always_comb begin
    pick = '0; found = 1'b0;
    for (int k = N_PROD-1; k >= 0; k--) begin
      int idx = (int'(rr_q) + k) % N_PROD;
      if (eligible[idx]) begin pick = PROD_W'(idx); found = 1'b1; end
    end
    grant        = '0;
    if (found) grant[pick] = 1'b1;
    commit       = found;
    commit_class = req_class[pick];
    commit_hdr   = req_hdr[pick];
    commit_data  = req_data[pick];
  end
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) rr_q <= '0;
    else if (found) rr_q <= (pick == PROD_W'(N_PROD-1)) ? '0 : (pick + PROD_W'(1));
  end
endmodule
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import credit_pkg::*;
 
// SYNTHESIZABLE. The reservation TRAVELS WITH THE PACKET (§9).
// Held stable under stall; terminated exactly once, by consume OR release.
module reservation_record (
  input  logic clk,
  input  logic rst_n,
  input  logic              grant_valid,
  input  tlp_class_e        grant_class,
  input  logic [CRED_W-1:0] grant_hdr,
  input  logic [CRED_W-1:0] grant_data,
 
  input  logic              tx_valid,
  input  logic              tx_ready,
  input  logic              tx_eop,
  input  logic              cancel,
 
  output logic              resv_valid,
  output tlp_class_e        resv_class,
  output logic [CRED_W-1:0] resv_hdr,
  output logic [CRED_W-1:0] resv_data,
  output logic              do_consume,
  output logic              do_release,
  output logic              err_double_terminate    // sticky
);
  logic v_q, e_q; tlp_class_e c_q; logic [CRED_W-1:0] h_q, d_q;
 
  assign resv_valid = v_q;   assign resv_class = c_q;
  assign resv_hdr   = h_q;   assign resv_data  = d_q;
  assign err_double_terminate = e_q;
 
  // Terminal events are mutually exclusive and require a live reservation.
  assign do_consume = v_q && tx_valid && tx_ready && tx_eop && !cancel;
  assign do_release = v_q && cancel;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin v_q<=1'b0; c_q<=CL_POSTED; h_q<='0; d_q<='0; e_q<=1'b0; end
    else begin
      if ((do_consume || do_release) && !v_q) e_q <= 1'b1;   // cannot happen: prove it
      if (do_consume || do_release) v_q <= 1'b0;
      else if (!v_q && grant_valid) begin
        v_q <= 1'b1; c_q <= grant_class; h_q <= grant_hdr; d_q <= grant_data;
      end
      // Under stall NOTHING above fires, so the record is stable by
      // construction -- P14 proves it rather than trusting it.
    end
  end
endmodule
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import credit_pkg::*;
 
// SYNTHESIZABLE. Class-aware scheduler (§7). A class whose credits are
// exhausted must not block a class whose credits are healthy. This is
// LOCAL POLICY -- PCIe does not require separate queues; it makes them
// possible by flow-controlling the classes independently (16.1 §4).
// §11 Model 16: single FIFO lost 34.8% of issue opportunities, per-class
// queues 6.3% -- 28.4% of all cycles recovered by structure alone.
module class_scheduler (
  input  logic clk,
  input  logic rst_n,
  input  logic [N_CLASS-1:0] queue_valid,
  input  logic [N_CLASS-1:0] credits_ok,
  input  logic               tx_ready,
  input  logic               link_operational,
 
  output logic [N_CLASS-1:0] eligible,
  output logic               sel_valid,
  output tlp_class_e         sel_class,
  output logic [N_CLASS-1:0] blocked_pending    // pending AND not eligible (§5)
);
  logic [CLASS_W-1:0] rr_q;
 
  always_comb begin
    for (int c = 0; c < N_CLASS; c++) begin
      eligible[c]        = queue_valid[c] && credits_ok[c] && link_operational;
      blocked_pending[c] = queue_valid[c] && !credits_ok[c];   // qualified, §5
    end
  end
 
  logic [CLASS_W-1:0] pick; logic found;
  always_comb begin
    pick='0; found=1'b0;
    for (int k = N_CLASS-1; k >= 0; k--) begin
      int idx = (int'(rr_q) + k) % N_CLASS;
      if (eligible[idx]) begin pick = CLASS_W'(idx); found = 1'b1; end
    end
    sel_valid = found;
    sel_class = tlp_class_e'(pick);
  end
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) rr_q <= '0;
    else if (found && tx_ready)
      rr_q <= (pick == CLASS_W'(N_CLASS-1)) ? '0 : (pick + CLASS_W'(1));
  end
endmodule
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import credit_pkg::*;
 
// SYNTHESIZABLE. Bottleneck attribution instrumentation (§4, §5).
// Per class, per resource, QUALIFIED BY PENDENCY. An unqualified counter
// inflated the credit-blocked total by 81.8% (§11 Model 17).
module credit_stall_counters (
  input  logic clk,
  input  logic rst_n,
  input  logic [N_CLASS-1:0] pending,
  input  logic [N_CLASS-1:0] hdr_short,      // pending packet needs more header
  input  logic [N_CLASS-1:0] data_short,
  input  logic               tags_short,     // §4: a different resource entirely
  input  logic               link_down,
  input  logic [CRED_W-1:0]  avail_hdr  [N_CLASS],
  input  logic [CRED_W-1:0]  avail_data [N_CLASS],
  input  logic               clear,
 
  output logic [31:0] hdr_block_cycles  [N_CLASS],
  output logic [31:0] data_block_cycles [N_CLASS],
  output logic [31:0] tag_block_cycles,
  output logic [31:0] link_block_cycles,
  output logic [CRED_W-1:0] min_hdr  [N_CLASS],   // watermarks, §9
  output logic [CRED_W-1:0] min_data [N_CLASS]
);
  logic [31:0] hb_q [N_CLASS], db_q [N_CLASS], tb_q, lb_q;
  logic [CRED_W-1:0] mh_q [N_CLASS], md_q [N_CLASS];
  logic init_q;
 
  always_comb begin
    for (int c = 0; c < N_CLASS; c++) begin
      hdr_block_cycles[c]  = hb_q[c];
      data_block_cycles[c] = db_q[c];
      min_hdr[c]  = mh_q[c];
      min_data[c] = md_q[c];
    end
    tag_block_cycles  = tb_q;
    link_block_cycles = lb_q;
  end
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n || clear) begin
      for (int c = 0; c < N_CLASS; c++) begin
        hb_q[c] <= '0; db_q[c] <= '0;
        mh_q[c] <= {CRED_W{1'b1}}; md_q[c] <= {CRED_W{1'b1}};  // start HIGH
      end
      tb_q <= '0; lb_q <= '0; init_q <= 1'b0;
    end else begin
      init_q <= 1'b1;
      // ---- FIRST BLOCKING RESOURCE, ranked (§4). A cycle blocked by
      // several resources indicts ONE, or the shares cannot be compared.
      if (link_down) begin
        if (!(&lb_q)) lb_q <= lb_q + 32'd1;
      end else begin
        for (int c = 0; c < N_CLASS; c++) begin
          if (pending[c] && hdr_short[c]) begin              // pendency, §5
            if (!(&hb_q[c])) hb_q[c] <= hb_q[c] + 32'd1;
          end else if (pending[c] && data_short[c]) begin
            if (!(&db_q[c])) db_q[c] <= db_q[c] + 32'd1;
          end
        end
        if (tags_short && !(&tb_q)) tb_q <= tb_q + 32'd1;
      end
 
      // ---- WATERMARKS: monotone DOWNWARD only, never re-raised.
      for (int c = 0; c < N_CLASS; c++) begin
        if (avail_hdr[c]  < mh_q[c]) mh_q[c] <= avail_hdr[c];
        if (avail_data[c] < md_q[c]) md_q[c] <= avail_data[c];
      end
    end
  end
endmodule
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import credit_pkg::*;
 
// VERIFICATION-ONLY. Conservation monitor: per class, the three states
// plus what is awaiting an FC update must equal what was advertised.
// A SUMMED-ACROSS-CLASSES version of this check passes while two classes
// are wrong in opposite directions -- the lesson of Chapter 21.4 §7.
module credit_conservation_monitor (
  input logic [CRED_W-1:0] advertised [N_CLASS],
  input logic [CRED_W-1:0] avail      [N_CLASS],
  input logic [CRED_W-1:0] reserved   [N_CLASS],
  input logic [CRED_W-1:0] spent      [N_CLASS],
  output logic err_conservation
);
  always_comb begin
    err_conservation = 1'b0;
    for (int c = 0; c < N_CLASS; c++)
      if ((avail[c] + reserved[c] + spent[c]) != advertised[c])
        err_conservation = 1'b1;               // PER CLASS, deliberately
  end
endmodule

Classification: six synthesizable, one verification-only.

Failure — eight. The zero-byte ceil-div underflow (268,435,456 units, §11). Unsigned credit subtraction without a borrow check (0 − 1 = 255). Producers acting on observed availability instead of a grant. A reservation that does not survive a stall. Releasing a reservation twice. Applying an FC update on every valid cycle. Unqualified stall counters (81.8%, §11). And one aggregate credit counter in place of six.

10. Same-Cycle Audit and Assertions

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// ==================================================================
// COST DERIVATION (§9) -- the arithmetic that ships broken.
// ==================================================================
 
// P1: a header-only TLP requires ZERO data credit. §11 Model 12: the
// naive ceil form returns 268,435,456 units at bytes = 0.
property p_zero_bytes_zero_data_credit;
  @(posedge clk) disable iff (!rst_n)
    (cost.valid && !cost.has_data) |-> (cost.need_data == '0);
endproperty
 
// P2: the ceil relation holds exactly -- SOURCED n = Roundup(Length/unit).
property p_ceil_division_exact;
  @(posedge clk) disable iff (!rst_n)
    (cost.valid && cost.has_data && (cost.payload_bytes != '0)) |->
      (cost.need_data ==
        CRED_W'((cost.payload_bytes + LEN_W'(CREDIT_BYTES-1)) / CREDIT_BYTES));
endproperty
 
// P3: an exact multiple of the unit consumes no extra credit.
property p_exact_multiple_no_roundup;
  @(posedge clk) disable iff (!rst_n)
    (cost.valid && cost.has_data && ((cost.payload_bytes % CREDIT_BYTES) == '0)
      && (cost.payload_bytes != '0)) |->
      (cost.need_data == CRED_W'(cost.payload_bytes / CREDIT_BYTES));
endproperty
 
// P4: one byte past a unit boundary costs exactly one more credit.
property p_one_byte_over_costs_one_more;
  @(posedge clk) disable iff (!rst_n)
    (cost.valid && cost.has_data
      && ((cost.payload_bytes % CREDIT_BYTES) == LEN_W'(1))) |->
      (cost.need_data == CRED_W'(cost.payload_bytes / CREDIT_BYTES) + CRED_W'(1));
endproperty
 
// P5: every TLP costs header credit, whether or not it carries data.
property p_every_tlp_costs_header;
  @(posedge clk) disable iff (!rst_n)
    cost.valid |-> (cost.need_hdr == CRED_W'(1));
endproperty
 
// ==================================================================
// LEDGER ARITHMETIC (§9) -- no wrap, either direction.
// ==================================================================
 
// P6: credit never underflows. §11 Model 18: unsigned 0 minus 1 is 255,
// after which the engine believes it owns 255 credits.
property p_no_underflow;
  @(posedge clk) disable iff (!rst_n)
    (avail_hdr[0] <= advertised_hdr[0]) && (avail_data[0] <= advertised_data[0]);
endproperty
 
// P7: a reservation that would borrow is REFUSED and reported.
property p_borrow_refused_and_flagged;
  @(posedge clk) disable iff (!rst_n)
    (resv_commit && (resv_hdr > avail_hdr[resv_class])) |=> err_underflow;
endproperty
 
// P8: available + reserved never exceeds what was advertised.
property p_no_overflow_above_advertised;
  @(posedge clk) disable iff (!rst_n)
    ((avail_hdr[0] + reserved_hdr[0]) <= advertised_hdr[0]);
endproperty
 
// P9: per-class conservation. A summed version passes while two classes
// are wrong oppositely (Chapter 21.4 §7's lesson, applied to credit).
property p_conservation_per_class;
  @(posedge clk) disable iff (!rst_n)
    !err_conservation;
endproperty
 
// ==================================================================
// RESERVATION -- availability is not ownership (§9).
// ==================================================================
 
// P10: EXACTLY ONE grant per cycle. Two producers observing the same last
// credit is the double-spend of Chapter 16.5 §4.
property p_single_grant;
  @(posedge clk) disable iff (!rst_n)
    $onehot0(grant);
endproperty
 
// P11: a grant implies both resources sufficed for THAT producer's class.
property p_grant_implies_both_resources;
  @(posedge clk) disable iff (!rst_n)
    grant[0] |-> ((avail_hdr [req_class[0]] >= req_hdr [0])
               && (avail_data[req_class[0]] >= req_data[0]));
endproperty
 
// P12: header sufficiency alone never grants -- mutation 2.
property p_header_alone_insufficient;
  @(posedge clk) disable iff (!rst_n)
    (req_valid[0] && (avail_data[req_class[0]] < req_data[0])) |-> !grant[0];
endproperty
 
// P13: data sufficiency alone never grants -- mutation 1.
property p_data_alone_insufficient;
  @(posedge clk) disable iff (!rst_n)
    (req_valid[0] && (avail_hdr[req_class[0]] < req_hdr[0])) |-> !grant[0];
endproperty
 
// P14: no transmission without a live reservation.
property p_no_issue_without_reservation;
  @(posedge clk) disable iff (!rst_n)
    (tx_valid && tx_ready) |-> resv_valid;
endproperty
 
// P15: the reservation record is STABLE under stall. Availability may
// change underneath; ownership may not.
property p_reservation_stable_under_stall;
  @(posedge clk) disable iff (!rst_n)
    (resv_valid && !(tx_valid && tx_ready && tx_eop) && !cancel)
      |=> (resv_valid && $stable(resv_class)
                      && $stable(resv_hdr) && $stable(resv_data));
endproperty
 
// P16: the reserved class cannot change while the reservation is held --
// the scheduler may not swap the packet under its own reservation.
property p_class_cannot_change_under_stall;
  @(posedge clk) disable iff (!rst_n)
    (resv_valid && $past(resv_valid)) |-> $stable(resv_class);
endproperty
 
// P17: terminal events are mutually exclusive -- consume XOR release.
property p_terminal_exclusive;
  @(posedge clk) disable iff (!rst_n)
    !(do_consume && do_release);
endproperty
 
// P18: a reservation terminates AT MOST ONCE. Releasing twice returns
// credit that was never reserved and lifts the pool above advertised.
property p_single_termination;
  @(posedge clk) disable iff (!rst_n)
    (do_consume || do_release) |=> !(do_consume || do_release) until_with grant_valid;
endproperty
 
// P19: a terminal event requires a live reservation.
property p_terminate_requires_live;
  @(posedge clk) disable iff (!rst_n)
    (do_consume || do_release) |-> resv_valid;
endproperty
 
// P20: cancellation releases the reservation exactly once, and the credit
// returns to AVAILABLE rather than being lost.
property p_cancel_returns_credit;
  @(posedge clk) disable iff (!rst_n)
    do_release |=> (avail_hdr[$past(resv_class)]
                     == $past(avail_hdr[resv_class]) + $past(resv_hdr));
endproperty
 
// ==================================================================
// FC RETURN (§9).
// ==================================================================
 
// P21: an update is applied ONCE, on the accepted beat -- not on every
// cycle `fc_valid` is held.
property p_fc_applied_once;
  @(posedge clk) disable iff (!rst_n)
    (fc_valid && !fc_ready) |=> ($stable(avail_hdr[fc_class])
                              && $stable(avail_data[fc_class]));
endproperty
 
// P22: an update credits the class it names, and no other.
property p_fc_credits_named_class_only;
  @(posedge clk) disable iff (!rst_n)
    (fc_valid && fc_ready && (fc_class != CL_POSTED))
      |=> $stable(avail_hdr[CL_POSTED]);
endproperty
 
// P23: a same-cycle return and reservation are ordered -- the reservation
// may use the returned credit, never the stale value (§10).
property p_same_cycle_return_then_reserve;
  @(posedge clk) disable iff (!rst_n)
    (fc_valid && fc_ready && resv_commit && (fc_class == resv_class)) |=>
      (avail_hdr[resv_class]
        == $past(avail_hdr[resv_class]) + $past(fc_hdr_delta) - $past(resv_hdr));
endproperty
 
// ==================================================================
// RESET (§10) -- must not invent permission.
// ==================================================================
 
// P24: reset zeroes credit. It must NEVER fabricate the advertised total.
property p_reset_zeroes_credit;
  @(posedge clk)
    (!rst_n) |=> ((avail_hdr[0] == '0) && (avail_data[0] == '0));
endproperty
 
// P25: no reservation survives reset.
property p_reset_clears_reservations;
  @(posedge clk)
    (!rst_n) |=> (!resv_valid && (reserved_hdr[0] == '0));
endproperty
 
// ==================================================================
// CLASS INDEPENDENCE AND ATTRIBUTION (§5, §7).
// ==================================================================
 
// P26: a starved class does not block an eligible one. §11 Model 16:
// 28.4% of cycles recovered by queue structure alone.
property p_starved_class_does_not_block;
  @(posedge clk) disable iff (!rst_n)
    (queue_valid[1] && credits_ok[1] && link_operational) |-> sel_valid;
endproperty
 
// P27: exactly one class is selected.
property p_single_class_selected;
  @(posedge clk) disable iff (!rst_n)
    sel_valid |-> (eligible[sel_class]);
endproperty
 
// P28: a credit-stall counter advances only when a packet of that class
// is PENDING. §11 Model 17: unqualified counting inflated by 81.8%.
property p_stall_requires_pending;
  @(posedge clk) disable iff (!rst_n)
    (hdr_block_cycles[0] != $past(hdr_block_cycles[0]))
      |-> $past(pending[0] && hdr_short[0]);
endproperty
 
// P29: an idle class never accrues credit-stall cycles.
property p_no_work_never_counts_as_credit_stall;
  @(posedge clk) disable iff (!rst_n)
    (!pending[0]) |=> ($stable(hdr_block_cycles[0]) && $stable(data_block_cycles[0]));
endproperty
 
// P30: one blocking resource per cycle -- header and data stalls for the
// same class are not both counted (§4's ranking).
property p_one_blocking_resource_per_class;
  @(posedge clk) disable iff (!rst_n)
    (!link_down) |=>
      !((hdr_block_cycles[0]  != $past(hdr_block_cycles[0])) &&
        (data_block_cycles[0] != $past(data_block_cycles[0])));
endproperty
 
// P31: a down Link is attributed to the Link, never to credit (§3).
property p_link_down_outranks_credit;
  @(posedge clk) disable iff (!rst_n)
    link_down |=> ($stable(hdr_block_cycles[0]) && $stable(data_block_cycles[0]));
endproperty
 
// P32: the watermark is monotone DOWNWARD between clears -- it records
// the worst moment, and a recovery must not erase it.
property p_watermark_monotone_down;
  @(posedge clk) disable iff (!rst_n)
    (!clear) |=> (min_hdr[0] <= $past(min_hdr[0]));
endproperty
 
// P33: the watermark never claims less than was ever available.
property p_watermark_bounded_by_history;
  @(posedge clk) disable iff (!rst_n)
    (init_q && !clear) |-> (min_hdr[0] <= avail_hdr[0]);
endproperty
 
// P34: the diagnostic counters never drive the credit state or the
// scheduler -- a monitor that throttles is not a monitor.
property p_monitor_does_not_drive;
  @(posedge clk) disable iff (!rst_n)
    $stable({grant, commit, sel_valid}) or
      !$stable({hdr_block_cycles[0], min_hdr[0]});
endproperty

Thirty-four properties. P1–P5 guard the arithmetic that ships broken. P10–P20 are the ownership contract — availability is not permission, and a reservation is a lease that terminates exactly once. P26–P34 are what makes this a performance chapter: without pendency qualification and single-resource attribution, the counters produce numbers that are precise and wrong.

11. Measured Behaviour

12. Verification — DV and Mutations

DV, against an independent integer credit ledger — never the DUT's own functions: Posted header-only · Posted with data · Non-Posted request · Completion with data · header starvation · data starvation · FC return · spending the exact last credit · simultaneous return and reserve · two producers, one remaining credit · cancellation · reset with reservations outstanding · class isolation · single-FIFO head-of-line · per-class queues · payload of zero bytes · payload of exactly one credit unit · an exact multiple · one byte over a unit boundary · an out-of-range class index.

#MutationSymptomCaught by
1Check data credit, not headerheader pool overruns; receiver header buffer overflowsP13
2Check header credit, not datapayload overruns the receiver's data bufferP12
3Let producers act on observed availabilitydouble-spend; §11 Model 14: the race arises 65.6% of cyclesP10, P14
4Grant two producers in one cycleover-commitment of the same creditP10
5Subtract credit on tx_valid instead of the reservationstalls spend credit repeatedlyP14, P15
6Drop the reservation when TX stallscredit reallocated under a live packetP15
7Leak the reservation on cancellationcredit permanently lost; throughput decays over timeP20
8Release the reservation twicepool rises above advertised; receiver overrun followsP8, P18
9Apply the FC update every fc_valid cyclecredit inflates while the update is stalledP21
10Unsigned subtract with no borrow check0 − 1 = 255; engine believes it owns 255 (§11)P6, P7
11Allow returns above the advertised totalreceiver buffer overflow; loop is brokenP8
12Use ((bytes−1)/U)+1 unguarded268,435,456 units at bytes = 0 (§11 Model 12)P1
13Ceil-divide with > instead of >= at the boundaryexact multiples charged one extra creditP3
14Round down instead of upone byte over a unit is under-charged; overrunP4
15Charge a header-only TLP one data creditNP requests consume PD/NPD needlesslyP1, P5
16Charge no header credit for a data-bearing TLPheader pool overrunsP5
17PD starvation blocks Non-Posted traffic28.4% of issue opportunities lost (§11 Model 16)P26
18Select a class whose credits are exhaustedillegal transmissionP27
19Count credit-blocked cycles with nothing pending81.8% inflation; wrong subsystem blamed (§11)P28, P29
20Count header and data stalls for the same classshares exceed the window; causes unrankableP30
21Attribute a down-Link cycle to creditPHY problem reported as a buffer problemP31
22Let the watermark rise after recoverythe worst moment is erasedP32
23Initialise the watermark to zeroit reports zero forever; uselessP33
24Let a diagnostic counter gate the schedulerthe instrument throttles the systemP34
25Credit an FC update to the wrong classone pool inflates while another starvesP22
26Treat PH as NPHPosted returns unblock Non-Posted illegallyP22
27Conflate CPLH and CPLDCompletion header/data accounting driftsP9, P22
28Alias an out-of-range class index onto Postedone class's credit spent by anotherP9
29Fabricate the advertised total at resettransmitter owns buffer never advertised (§10)P24
30Let a reservation survive resetstale ownership across re-initialisationP25
31Swap the packet under a held reservationcredits reserved for class A spent by class BP16
32Use a stale availability value on a same-cycle returnthe last credit is refused although it existsP23
33Check conservation summed across classespasses while two classes are wrong oppositelyP9
34Treat a replay as needing a fresh TL credit allocationover-claims what 16.5 §9 declines to claimdesign review
35Present the single-FIFO structure as a PCIe requirementit is local policy (§7)design review
36Publish a fixed credit-return delay as a PCIe constantit is implementation-specific (§6)design review

Two counterexamples worth stating explicitly.

Mutation 12 is the most beautiful bug in this chapter, because it is a parameter-arithmetic failure with no bad intent anywhere. ((bytes − 1) / 16) + 1 is the standard ceiling idiom, it is correct for every non-zero payload, and it is written by someone who has thought about rounding carefully. At bytes = 0 the unsigned subtraction wraps and the function returns 268,435,456. The failure mode is the worst possible direction: the packet appears to need far more credit than exists, so it never issues, and the engine stalls forever on a header-only TLP while every credit counter reads healthy. P1 is one line and it is the only thing standing between a correct design and that.

Mutation 19 is the one that survives review, because the counter it produces looks like exactly the counter you wanted. It increments when credits are low; low credits are the thing you are hunting. It is wrong only because it does not ask whether anything wanted to send — and §11 Model 17 measured that as 81.8% fabricated cycles. The engineer reading it concludes the receiver needs a bigger buffer, orders that change, and the throughput does not move, because the real finding was an idle source. P29 is the falsifying property: an idle class must never accrue credit-stall cycles.

13. Debugging

Scenario — Link in L0, no errors, TX idle, packet queued. This is the chapter's opening state, and it has a fixed procedure. Read the six pools separately. Identify the head packet's class. Compute its header and data cost independently. Then: is the shortfall header or data? A design with one aggregate credit number cannot answer any of this, which is why §9 keeps six.

Scenario — reads burst, then stop for a long time, then burst again. Sawtooth on a Non-Posted path is the credit window (§6) or the Tag pool (20.3 §3) — and they look identical from outside. Distinguish with the attribution counters (§4): nph_block_cycles versus tag_block_cycles. Only one of them will be large, and they call for opposite fixes.

Scenario — writes stall, reads keep flowing. Posted resources, not the Link (§3). Check PH and PD. If PD is the empty one, look at payload sizes and the receiver's data buffering — Chapter 22.4 owns the payload-size analysis.

Scenario — every class stalls at once. Now suspect something shared: link readiness (§3), a scheduler that blocks all classes on one head (§7, mutation 17), or genuine exhaustion everywhere. The single-FIFO structure is the most common cause, and it is distinguishable in one step: if per-class blocked_pending shows one class blocked and others eligible while nothing transmits, the queue structure is the bug, not the credits.

Scenario — throughput is well below the link rate and no counter shows starvation. Check the watermarks (§9). If min_hdr or min_data reached zero at some point while the stall counters read low, the system is skirting the credit window — brief, frequent starvation that a coarse counter averages away. Then apply §6's ceiling with the measured advertised capacity and return delay.

Scenario — the system runs for hours, then stops permanently. A leak. Credit reserved and never released, a Tag never returned, or a completion context never freed. The signature is monotone: available credit trends downward across the run and never recovers. §9's watermark makes this visible in one read, and the conservation monitor identifies which class.

Scenario — a credit counter suddenly reads a huge value. Unsigned underflow (§11 Model 18), or the zero-byte ceil bug (Model 12). Both produce implausibly large numbers rather than small ones, so "credit looks plentiful and nothing sends" is a strong signature for exactly these two bugs.

A general rule: read the six pools separately, and never believe an unqualified stall counter. Both halves of that sentence were measured — one aggregate hides which class starved, and one unqualified counter inflates by 81.8%.

14. Misconceptions

"Credits are bandwidth tokens the sender generates." No — they are advertised by the receiver and describe its buffer (16.1 §1).

"One credit counter is enough." There are six (16.1 §4), and §13's procedure is impossible with one.

"If the Link is in L0, the transmitter can send." No. L0 plus a pending packet plus insufficient credit is a legal, silent stall (§3).

"A packet needs data credit." It needs header credit always, and data credit only if it carries data (§9, P5).

"A header-only TLP needs one data credit." Zero — and the naive ceiling idiom returns 268,435,456 instead (§11 Model 12).

"Every producer can check availability and decide." Availability is not ownership. The race arises in 65.6% of loaded cycles (§11 Model 14), and 16.5 §4 proves why the check is unsound.

"A flow-control stall means the Link failed." It means the receiver's buffer is full or its updates are slow (§3).

"Credit starvation blocks all traffic." It blocks one class. Whether it blocks the rest is a property of your queue structure — 28.4% of cycles hang on that choice (§7).

"Replay needs a fresh Transaction-Layer credit allocation." This chapter makes no such claim, and neither does 16.5 §9 (mutation 34).

"available − 1 at zero stays at zero in RTL." It becomes the maximum value (§11 Model 18).

"Credit return is immediate." It is a loop with a delay, and that delay is a throughput ceiling (§6).

"A bigger receiver buffer always means more throughput." Only if credit was the binding constraint — §6's last rows are already link-limited.

"Credit tells you about the far end of the fabric." Flow control is point-to-point (16.1 §1); it describes your link partner's buffer, not the destination's.

15. Understanding Check

Q1. Two request engines simultaneously see one remaining NPH credit and both compute credit_ok = 1. What is wrong, and where must the fix live? Both computations are correct and the conclusion is not. credit_ok is an observation of availability, and availability is not ownership — nothing in either engine's check prevents the other from acting on the same credit. The fix must be a single point that grants: one reservation per cycle, made against one snapshot of the ledger (§9, P10). §11 Model 14 measured this race arising in 65.6% of loaded cycles, so it is the normal case rather than a corner.

Q2. A Memory Read request carries no payload. How much data credit does it need, and what does the standard ceiling idiom return? Zero. The naive ((bytes − 1) / 16) + 1 underflows at bytes = 0 and returns 268,435,456 in unsigned 32-bit arithmetic (§11 Model 12). The packet then appears unaffordable forever, so the engine stalls on a header-only TLP while every credit counter reads healthy — a stall with no visible cause.

Q3. Your pd_block_cycles counter reads 40% of the run. What have you actually learned? Possibly nothing. If the counter is unqualified it also counted cycles when no Posted packet was pending, which inflated the total by 81.8% in §11 Model 17. Only pending && !credit_ok is a credit finding; the unqualified version is a mixture of credit starvation and an idle source, and those two indict different teams.

Q4. A link carries 64 bytes/cycle. The receiver advertises 256 bytes of data credit and the return loop is 64 cycles. What is the sustained ceiling, and does a faster link help? 4 bytes/cycle — about 6% of the link (§6, §11 Model 15). A faster link changes nothing, because the ceiling C / RTT contains no link term. The levers are more advertised capacity or a shorter return loop; doubling the signalling rate is the one change guaranteed to be useless.

Q5. PD is exhausted. A Non-Posted request with NPH available sits behind a Posted packet in one FIFO. What happens, and is PCIe at fault? Nothing transmits, and PCIe is not at fault — it flow-controls the classes independently precisely so an implementation can keep them separate (§7). The single queue is local policy. §11 Model 16 measured the cost: 34.8% of issue opportunities lost with one FIFO versus 6.3% with per-class queues.

Q6. After a reset, should the ledger show the advertised totals? No — it must show zero. Credit is granted by the receiver's initialisation (16.1 §6), and a reset that fabricates the totals gives the transmitter permission to occupy buffer space the receiver has not advertised since re-initialising. P24 asserts zero and P25 clears every reservation — reset removes permission, it does not create it.

16. Module 22 So Far, and What Remains

Three questions, three chapters, one method.

22.1 — how much can the architecture sustain? Capacity is arithmetic; achieved throughput is a measurement, and every non-transferring cycle must be attributed to exactly one cause.

22.2 — how long does one unit of work take? Latency is an interval between two named events, decomposed across queueing, serialization, transport, service and return, and measured per transaction.

22.3 — which resource stops the pipeline? Credit is permission, not bandwidth; starvation is per class; and proving it requires attribution qualified by pendency.

The method is the same each time, and it is the transferable part: name the measurement point, rank the causes so they sum to the whole, and qualify every counter so it cannot count something it did not observe. Three chapters, three subsystems, one discipline — and in all three the naive instrument fails in the optimistic direction, which is why it survives review.

Three chapters remain, and none of their material has been consumed here.

22.4 — Payload Size Effects. How MPS and MRRS change packetization: the header-to-payload ratio, the credit cost per packet that §9 computed but never optimized, and the serialization terms 22.2 §4 named.

22.5 — Link Efficiency. The exact accounting of what the Link carries that is not payload — headers, framing, DLLPs, ordered sets and encoding — which 22.1 §5 deliberately named as one term and refused to derive.

22.6 — Benchmark Interpretation. Reading a real result without lying to yourself. Every chapter in this batch was a prerequisite for it: 22.1 §3's measurement point, 22.2 §3's event pair, and this chapter's insistence that a counter must be qualified before it means anything.