Skip to content

PCIe · Module 16

Non-Posted Credits — Free Tags Are Not Permission

A Memory Read costs 1 NPH and no data credit at all. Yet it is the request most likely to be blocked, because launching one needs four independent resources at once — and NPH is not the outstanding-request table, however much it looks like it.

Chapter 16.2 costed a packet that carries payload and expects nothing back. This chapter costs the opposite shape: a Memory Read carries no payload at all, and something is waiting for it.

A Memory Read Request costs 1 NPH. That is the entire cost — no data credit, because the request has no data.

And it is still the request most likely to be blocked. A requester can hold free Tags, free replay-buffer entries and an idle Link, and be unable to issue a single read.

What do NPH and NPD represent, why is a read so much harder to launch than to pay for, and why does adding more Tags not help?

1. The Verified Costs

2. Cheap to Buy, Hard to Launch

The asymmetry that makes this chapter different from the last one.

Memory Write (16.2)Memory Read
Credit cost1 PH + n PD1 NPH
Payloadyesnone
Needs a Tagnoyes
Something waits for itnoyes — a Completion
Resources to launch34
Usual blockerPDcontext or NPH

A read is the cheapest request in the protocol to pay for and the most constrained to issue.

3. NPD Exists, and Not for Reads

The chapter must resist making itself symmetric with Chapter 16.2, because the protocol is not.

Posted traffic has a large, frequently-exercised data pool. Every Memory Write consumes PD proportional to its payload, and PD is usually the binding constraint (Chapter 16.2 §10).

Non-Posted traffic barely uses NPD at all. From §1's table:

RequestNPD cost
Memory Readnone
I/O Read, Configuration Readnone
I/O Write, Configuration Write1 — at most one aligned DW
AtomicOpn

Three of the four common Non-Posted requests consume no NPD whatsoever, and the fourth consumes exactly one.

4. NPH Is Not the Outstanding-Request Table

This is the chapter's most important distinction, and the one that costs engineers the most time.

Both quantities limit how many reads can be in flight. They are different resources, owned by different components, sized independently, and released by different events.

NPH creditRequester context / Tag
Whose resourcethe remote receiver's bufferthe local requester's table
Advertised bythe far side, via InitFC/UpdateFCnobody — it is your own RTL
Released whenthe receiver frees the request bufferthe last Completion arrives
Typical lifetimeshorta full round trip
Sized bythe far side's designeryou
Runs out becausethe far side is slow to acceptyou have too many reads outstanding

A third confusion, worth naming separately: NPH has nothing to do with Max_Read_Request_Size.

MRRS bounds how much data one read may request (Chapter 12.1). NPH bounds how many request headers the receiver can hold. A read for 4096 bytes and a read for 4 bytes both cost exactly one NPH. Raising MRRS changes the number of reads needed to move a given amount of data — it does not change what any one of them costs, and it does not increase NPH.

5. What NP Credit Actually Buys the Receiver

Worth being precise, because "the receiver must service it later" invites an overclaim.

A Non-Posted Request creates future work: the receiver must accept it, keep enough information to service it, and eventually produce a Completion where required.

But NP credit accounts for the receive buffer, not for the servicing machinery.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
NP Request arrives
  → occupies the receiver's NP receive buffer      ← NPH accounts for THIS
  → moves into internal servicing machinery        ← implementation-defined
  → receive buffer is free
  → credit can be returned
  → ... later ... the Completion is produced       ← CPLH/CPLD, other direction

Why the separate pool matters is Chapter 16.1 §4's forward-progress argument, and here it becomes concrete: Completions must be able to make progress while requests are backing up. If Non-Posted Requests could consume the capacity a Completion needs to land, a requester waiting on a Completion could be blocked by its own outstanding requests. Separate pools make that dependency unconstructable. This chapter states the principle; the full argument belongs with Chapter 13.4 and 16.4.

6. A Trace

Internal teaching signals, not PCIe wire signals. NPH capacity 3, 4 requester contexts.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
step             1     2     3     4     5     6     7     8
read_valid       1     1     1     1     1     1     0     0
 
ctx_free         4     3     2     1     1     1     1     2
nph_available    3     2     1     0     0     1     1     1
 
eligible         1     1     1     0     0     1     -     -
launch           1     1     1     0     0     1     -     -
 
nph_return       0     0     0     0     1     0     0     0
cpl_arrives      0     0     0     0     0     0     0     1

Read steps 1–3. Three reads launch. Each takes one context and one NPH.

Read step 4 — the block, and which resource caused it. ctx_free is 1, so contexts are not the constraint. nph_available is 0. The requester has a free Tag and cannot use it — §14's advanced question in one line of trace.

Read step 5. An NPH return arrives, but note eligible is still low in that cycle's column because the trace shows the value before the return lands; by step 6 availability is 1 and the read goes.

Read step 6. The fourth read launches, consuming the last context.

Read step 8 — the asymmetry. A Completion arrives and ctx_free rises to 2. nph_available does not change, because a Completion returns a context, not a credit (§4). Those two counters move on completely different events, and a design in which they move together has the bug P11 forbids.

7. RTL — Non-Posted Credit Cost Deriver

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SYNTHESIZABLE. Derive Non-Posted flow-control cost from a normalized
// packet descriptor.
// The per-class costs are NORMATIVE (section 1). The descriptor shape and
// the error report are ILLUSTRATIVE.
package np_fc_pkg;
 
  // VERIFIED (section 1), identical to the Posted data unit.
  localparam int FC_DATA_UNIT_BYTES = 16;
 
  typedef enum logic [2:0] {
    NP_MEM_READ    = 3'd0,   // 1 NPH            -- NO data credit
    NP_IO_READ     = 3'd1,   // 1 NPH
    NP_CFG_READ    = 3'd2,   // 1 NPH
    NP_IO_WRITE    = 3'd3,   // 1 NPH + 1 NPD    -- at most one aligned DW
    NP_CFG_WRITE   = 3'd4,   // 1 NPH + 1 NPD
    NP_ATOMIC      = 3'd5,   // 1 NPH + n NPD
    NP_NOT_NP      = 3'd6    // Posted -- Chapter 16.2
  } np_kind_e;
 
  // Only AtomicOp needs this. Same ceiling form and same widening
  // discipline as Chapter 16.2 section 6.
  function automatic logic [11:0]
      npd_credit_cost(input logic [13:0] payload_bytes);
    logic [14:0] widened;
    widened = {1'b0, payload_bytes} + 15'(FC_DATA_UNIT_BYTES - 1);
    return 12'(widened / 15'(FC_DATA_UNIT_BYTES));
  endfunction
 
endpackage
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import np_fc_pkg::*;
 
module np_credit_cost #(
  parameter int LEN_W  = 14,
  parameter int COST_W = 12
) (
  input  np_kind_e            pkt_kind,
  // Meaningful ONLY for the payload-carrying kinds. For a read this field
  // describes how much data is REQUESTED, and charging NPD for it would
  // charge the request for the Completion's payload -- on the wrong pool,
  // in the wrong direction (section 3).
  input  logic [LEN_W-1:0]    payload_bytes,
 
  output logic                uses_nph,
  output logic [COST_W-1:0]   nph_cost,
  output logic                uses_npd,
  output logic [COST_W-1:0]   npd_cost,
 
  output logic                cost_error
);
 
  wire is_np = (pkt_kind != NP_NOT_NP);
 
  // A CONSTANT, exactly as in Chapter 16.2: every NP request costs one
  // header credit, whatever its header size (Chapter 16.1 section 5).
  assign uses_nph = is_np;
  assign nph_cost = is_np ? COST_W'(1) : '0;
 
  // NOT A MIRROR OF THE POSTED DERIVER (section 3). A lookup, because the
  // protocol is lopsided here and the code should say so.
  always_comb begin
    uses_npd = 1'b0;
    npd_cost = '0;
    unique case (pkt_kind)
      NP_MEM_READ, NP_IO_READ, NP_CFG_READ : begin
        // NO DATA CREDIT. The request carries nothing.
        uses_npd = 1'b0;
        npd_cost = '0;
      end
      NP_IO_WRITE, NP_CFG_WRITE : begin
        // Exactly one, always. The source states the data written is never
        // more than one aligned DW, so no division is possible or needed.
        uses_npd = 1'b1;
        npd_cost = COST_W'(1);
      end
      NP_ATOMIC : begin
        uses_npd = (payload_bytes != '0);
        npd_cost = (payload_bytes != '0)
                     ? COST_W'(npd_credit_cost(14'(payload_bytes))) : '0;
      end
      default : ;                       // NP_NOT_NP: nothing, reported below
    endcase
  end
 
  // A Posted kind reaching this deriver, or a read claiming a payload it
  // cannot carry. Reported rather than silently costed.
  assign cost_error = (!is_np)
                   || ((pkt_kind inside {NP_MEM_READ, NP_IO_READ, NP_CFG_READ})
                       && (payload_bytes != '0) && uses_npd);
 
endmodule

Classification: synthesizable (combinational) plus a compile-time helper.

Architecture. A constant for the header, and a five-way lookup for the data — with a division reachable from exactly one arm.

The shape is the lesson. Chapter 16.2's deriver was a division because Posted data cost genuinely varies. This one is a table because Non-Posted data cost mostly does not, and writing it as a division with most inputs happening to yield zero would obscure that.

Cost table produced by this module:

Packetnph_costuses_npd / npd_cost
Memory Read, 4096 bytes requested10 / 0
Memory Read, 4 bytes requested10 / 0
I/O Write11 / 1
Configuration Write11 / 1
AtomicOp, 8 bytes11 / 1
AtomicOp, 17 bytes11 / 2
Posted kind00 / 0, cost_error

Failure — five. Charging NPD for a Memory Read's Length confuses requested data with carried data (§3). Mirroring Chapter 16.2's division across all kinds produces the same bug wherever payload_bytes is non-zero for a read. Floor division in the AtomicOp arm — the same defect as 16.2 §12's counterexample. Charging more than one NPD for an I/O or Configuration Write wastes a pool whose advertised minimum is one unit. And deriving the kind from a raw Type field re-decodes what Chapter 11.7 already normalized.

Deliberately simplified: one packet per cycle; AtomicOp treated only as a credit cost, not semantically.

8. RTL — Memory Read Resource Admission

The chapter's centrepiece: four resources, four owners, one decision — and an explicit answer to what happens between accepting a request and sending it.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import np_fc_pkg::*;
 
// SYNTHESIZABLE. Multi-resource admission for a Non-Posted Request.
// The NPH/NPD costs are NORMATIVE (section 1). That a read ALSO needs a
// requester context, replay-buffer space and a transmit opportunity comes
// from Chapters 12.1, 14.4 and 16.1 -- this module composes them and owns
// none of them.
// The single-cycle commit model is ILLUSTRATIVE (section 8's callout).
module np_read_admission #(
  parameter int CRED_W = 12,
  parameter int COST_W = 12,
  parameter int LEN_W  = 14
) (
  input  logic clk,
  input  logic rst_n,
 
  // ---- Local read request ------------------------------------------------
  input  logic                local_read_valid,
  output logic                local_read_ready,
  input  np_kind_e            pkt_kind,
  input  logic [LEN_W-1:0]    payload_bytes,
 
  // ---- The four resources, each owned elsewhere -------------------------
  // Chapter 12.1: a free requester context / Tag. NOT allocated here.
  input  logic                ctx_available,
  // Chapter 14.4: local replay storage for the transmitted packet.
  input  logic                replay_space,
  // The transmit path, this cycle.
  input  logic                tx_ready,
  // Credit plumbing for the NP pools.
  input  logic                init_valid,
  input  logic [CRED_W-1:0]   init_nph_capacity,
  input  logic                init_nph_infinite,
  input  logic [CRED_W-1:0]   init_npd_capacity,
  input  logic                init_npd_infinite,
  input  logic                nph_return_valid,
  input  logic [CRED_W-1:0]   nph_return_count,
  input  logic                npd_return_valid,
  input  logic [CRED_W-1:0]   npd_return_count,
 
  // ---- Outputs ------------------------------------------------------------
  output logic                launch,          // ctx alloc + credit consume
  output logic [CRED_W-1:0]   nph_available,
  output logic [CRED_W-1:0]   npd_available,
 
  // WHICH resource is missing. Section 11's first debugging question, and
  // without these the answer costs a simulation run.
  output logic                blocked_on_ctx,
  output logic                blocked_on_nph,
  output logic                blocked_on_npd,
  output logic                blocked_on_replay,
  output logic                blocked_on_tx,
  output logic                np_error
);
 
  logic uses_nph, uses_npd, cost_err, credit_eligible, pair_err;
  logic [COST_W-1:0] nph_cost, npd_cost;
 
  np_credit_cost #(.LEN_W(LEN_W), .COST_W(COST_W)) u_cost (
    .pkt_kind, .payload_bytes,
    .uses_nph, .nph_cost, .uses_npd, .npd_cost, .cost_error(cost_err)
  );
 
  // ATOMIC ACROSS THE NP POOLS (Chapter 16.1 section 9). For a Memory Read
  // only the header pool is engaged, and the data pool must be neither
  // charged nor consulted -- a read blocked by a full NPD pool would be a
  // read blocked by a resource it does not use.
  credit_pair_gate #(.CRED_W(CRED_W), .COST_W(COST_W)) u_pair (
    .clk, .rst_n,
    .pkt_valid(local_read_valid),
    .needs_header(uses_nph), .header_cost(nph_cost),
    .needs_data(uses_npd),   .data_cost(npd_cost),
    .send_fire(launch),
    .hdr_return_valid(nph_return_valid), .hdr_return_count(nph_return_count),
    .dat_return_valid(npd_return_valid), .dat_return_count(npd_return_count),
    .init_valid,
    .init_hdr_capacity(init_nph_capacity),
    .init_hdr_infinite(init_nph_infinite),
    .init_dat_capacity(init_npd_capacity),
    .init_dat_infinite(init_npd_infinite),
    .eligible(credit_eligible),
    .hdr_available(nph_available), .dat_available(npd_available),
    .pool_error(pair_err)
  );
 
  // =====================================================================
  // THE CONJUNCTION. All four, in one cycle, or nothing.
  //
  // ACCEPTANCE IS LAUNCH: local_read_ready is exactly the condition under
  // which the packet also leaves. There is no window between accepting the
  // request and committing its resources, so there is no reservation state
  // and no release path to get wrong (section 8's callout).
  // =====================================================================
  wire all_ready = credit_eligible && ctx_available && replay_space && tx_ready;
 
  assign local_read_ready = all_ready;
  assign launch           = local_read_valid && all_ready;
 
  // Reported only while a request is actually waiting.
  assign blocked_on_ctx    = local_read_valid && !ctx_available;
  assign blocked_on_nph    = local_read_valid && uses_nph
                                              && (nph_available < CRED_W'(nph_cost));
  assign blocked_on_npd    = local_read_valid && uses_npd
                                              && (npd_available < CRED_W'(npd_cost));
  assign blocked_on_replay = local_read_valid && !replay_space;
  assign blocked_on_tx     = local_read_valid && !tx_ready;
  assign np_error          = cost_err | pair_err;
 
endmodule

Classification: synthesizable.

Architecture. A cost lookup, Chapter 16.1's atomic pair, and one conjunction with five reason flags. It allocates no context and stores no packet — it decides, and the owners of each resource act on launch.

launch is the single commit event, and every resource must key off it. A design in which the context table allocates on local_read_valid while credit is consumed on launch is the drift bug the §8 callout warns about — and P11 asserts the two move together.

Cycle behaviour.

Missing resourcelocal_read_readylaunchCredit moved?
none11yes, both pools as costed
context00no
NPH00no
replay space00no
transmit path00no

Every row but the first moves nothing. That uniformity is the property worth having: there is exactly one path that changes state.

Failure — five. Allocating a context on validity while consuming credit on launch drifts the two apart. Accepting locally and sending later without reservation state loses the resource guarantee (§8's callout). A single blocked flag cannot distinguish an NPH shortage from a context shortage, and those have opposite remedies. Consulting the NPD pool for a Memory Read blocks a request on a resource it does not use. And releasing a context on an NPH return frees a Tag whose Completion has not arrived (§4) — the corruption case.

9. Independent Reference Model

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// VERIFICATION-ONLY. Independent reference for the Non-Posted credit cost,
// written from section 1's table rather than from section 7's structure.
typedef struct { int nph; int npd; } np_cost_t;
 
function automatic np_cost_t ref_np_cost(string kind, int payload_bytes);
  np_cost_t c;
  c.nph = 1;                                   // every NP request, always
  case (kind)
    "MEM_READ", "IO_READ", "CFG_READ" : c.npd = 0;
    "IO_WRITE", "CFG_WRITE"           : c.npd = 1;
    "ATOMIC"                          : begin
      // Counted, not divided -- deliberately a different algorithm from
      // section 7, so a rounding bug cannot be present in both.
      c.npd = 0;
      while (payload_bytes > 0) begin
        c.npd = c.npd + 1;
        payload_bytes = payload_bytes - 16;
      end
    end
    default                           : begin c.nph = 0; c.npd = 0; end
  endcase
  return c;
endfunction
 
// The independent local-resource model. It tracks contexts from OBSERVED
// launches and OBSERVED completions -- never from the DUT's ctx_available.
class np_resource_model;
  int contexts_free;
  int nph, npd;
  function void on_launch(np_cost_t c);
    contexts_free--; nph -= c.nph; npd -= c.npd;
  endfunction
  // A COMPLETION returns a CONTEXT. It returns NO credit (section 4).
  function void on_completion();
    contexts_free++;
  endfunction
  // An UPDATE returns CREDIT. It returns NO context.
  function void on_nph_return(int n); nph += n; endfunction
endclass

Classification: verification-only.

The class is where the §4 distinction becomes executable. on_completion touches only contexts; on_nph_return touches only credit. A DUT that moved them together would diverge from this model within a few transactions, and the divergence names the bug.

10. Assertions

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SVA over np_credit_cost and np_read_admission, composed with Chapter
// 16.1's primitives and bound against Chapter 12.1's context table.
// These assert the VERIFIED NP costs and the local admission contract.
// Nothing here asserts credit-return timing or completion timing (both
// environment properties -- Chapter 16.1 section 15), the Completion rows
// (16.4), spend policy (16.5), or the update protocol (16.6).
 
// ---- ENVIRONMENT ------------------------------------------------------
// A1: pkt_kind is Chapter 11.7's normalized class, not re-decoded here.
assume property (@(posedge clk) disable iff (!rst_n)
  local_read_valid |-> (pkt_kind != NP_NOT_NP));
// A2: the request is stable while offered and not accepted.
assume property (@(posedge clk) disable iff (!rst_n)
  (local_read_valid && !local_read_ready)
    |=> ($stable(pkt_kind) && $stable(payload_bytes)));
 
// ---- COST -------------------------------------------------------------
 
// P1: EVERY Non-Posted request costs exactly one header credit.
property p_nph_cost_is_one;
  @(posedge clk) disable iff (!rst_n)
  uses_nph |-> (nph_cost == COST_W'(1));
endproperty
a_nph_one : assert property (p_nph_cost_is_one);
 
// P2: A MEMORY READ CONSUMES NO DATA CREDIT -- the chapter's headline
// cost, and the one a mirror of Chapter 16.2's deriver gets wrong.
// Asserted for ALL payload_bytes values, including large ones: a read's
// Length describes REQUESTED data, not carried data (section 3).
property p_read_costs_no_npd;
  @(posedge clk) disable iff (!rst_n)
  (pkt_kind inside {NP_MEM_READ, NP_IO_READ, NP_CFG_READ})
    |-> (!uses_npd && (npd_cost == '0));
endproperty
a_read_no_npd : assert property (p_read_costs_no_npd);
 
// P3: an I/O or Configuration Write costs exactly one NPD -- never n.
property p_io_cfg_write_one_npd;
  @(posedge clk) disable iff (!rst_n)
  (pkt_kind inside {NP_IO_WRITE, NP_CFG_WRITE})
    |-> (uses_npd && (npd_cost == COST_W'(1)));
endproperty
a_io_npd : assert property (p_io_cfg_write_one_npd);
 
// P4: the AtomicOp data cost matches an INDEPENDENT model, and is a
// CEILING (Chapter 16.2 section 11's pair, applied to the one NP kind that
// needs a division).
property p_atomic_npd_ceiling;
  @(posedge clk) disable iff (!rst_n)
  ((pkt_kind == NP_ATOMIC) && (payload_bytes != '0))
    |-> ((COST_W'(npd_cost) * 16 >= payload_bytes)
      && ((COST_W'(npd_cost) - 1) * 16 < payload_bytes));
endproperty
a_atomic_ceiling : assert property (p_atomic_npd_ceiling);
 
// ---- ADMISSION --------------------------------------------------------
 
// P5: no NP request launches without sufficient NPH.
property p_no_launch_without_nph;
  @(posedge clk) disable iff (!rst_n)
  (launch && uses_nph) |-> (nph_available >= CRED_W'(nph_cost));
endproperty
a_nph_gate : assert property (p_no_launch_without_nph);
 
// P6: ALL FOUR RESOURCES, or none. The chapter's central admission
// property -- a launch that satisfied three of four would either overflow
// the receiver, lose replayability, or strand a Completion with no context.
property p_launch_needs_all_four;
  @(posedge clk) disable iff (!rst_n)
  launch |-> (credit_eligible && ctx_available && replay_space && tx_ready);
endproperty
a_conjunction : assert property (p_launch_needs_all_four);
 
// P7: NO RESOURCE IS TAKEN WITHOUT A LAUNCH. There is exactly one path
// that changes state (section 8).
property p_no_resource_without_launch;
  @(posedge clk) disable iff (!rst_n)
  (local_read_valid && !launch)
    |=> ($stable(nph_available) && $stable(npd_available)
      && $stable(dut_ctx_table.allocated_count));
endproperty
a_no_phantom : assert property (p_no_resource_without_launch);
 
// P8: ATOMICITY across the NP pools -- a failed NPD leaves NPH untouched.
property p_failed_npd_leaves_nph;
  @(posedge clk) disable iff (!rst_n)
  (local_read_valid && uses_npd && (npd_available < CRED_W'(npd_cost)))
    |=> $stable(nph_available);
endproperty
a_atomic : assert property (p_failed_npd_leaves_nph);
 
// P9: a Memory Read is never blocked by the NPD pool -- it does not use it.
property p_read_not_blocked_by_npd;
  @(posedge clk) disable iff (!rst_n)
  ((pkt_kind == NP_MEM_READ) && local_read_valid) |-> !blocked_on_npd;
endproperty
a_read_free_of_npd : assert property (p_read_not_blocked_by_npd);
 
// P10: a starved request is STABLE -- metadata unchanged while it waits.
property p_starved_stable;
  @(posedge clk) disable iff (!rst_n)
  (local_read_valid && !local_read_ready)
    |=> (local_read_valid && $stable(pkt_kind) && $stable(payload_bytes));
endproperty
a_stable : assert property (p_starved_stable);
 
// ---- CREDIT AND CONTEXT ARE DIFFERENT RESOURCES (section 4) ------------
 
// P11: THE CORRUPTION PROPERTY. An NPH return NEVER frees a requester
// context. A design that released a Tag on a credit return would free a
// context whose Completion has not arrived -- and that Completion would
// later match a Tag reallocated to a different read.
property p_credit_return_frees_no_context;
  @(posedge clk) disable iff (!rst_n)
  (nph_return_valid && !dut_cpl_arrives)
    |=> $stable(dut_ctx_table.allocated_count);
endproperty
a_no_ctx_from_credit : assert property (p_credit_return_frees_no_context);
 
// P12: and the mirror -- a Completion returns a context, never credit.
property p_completion_returns_no_credit;
  @(posedge clk) disable iff (!rst_n)
  (dut_cpl_arrives && !nph_return_valid && !npd_return_valid)
    |=> ($stable(nph_available) && $stable(npd_available));
endproperty
a_no_credit_from_cpl : assert property (p_completion_returns_no_credit);
 
// P13: context and credit are committed TOGETHER on launch. Catches the
// drift bug: allocate at acceptance, consume at send (section 8).
property p_ctx_and_credit_commit_together;
  @(posedge clk) disable iff (!rst_n)
  launch |=> ((dut_ctx_table.allocated_count == $past(dut_ctx_table.allocated_count) + 1)
           && (nph_available == $past(nph_available) - $past(nph_cost)));
endproperty
a_together : assert property (p_ctx_and_credit_commit_together);
 
// ---- CLASS ISOLATION --------------------------------------------------
 
// P14: a Posted packet never consumes NP credit through this engine.
property p_posted_never_spends_np;
  @(posedge clk) disable iff (!rst_n)
  (dut_posted_engine.send_fire && !launch)
    |=> ($stable(nph_available) && $stable(npd_available));
endproperty
a_class_isolation : assert property (p_posted_never_spends_np);
 
// P15: NPH exhaustion never blocks Posted traffic.
property p_np_starvation_does_not_block_posted;
  @(posedge clk) disable iff (!rst_n)
  ((nph_available == '0) && dut_posted_engine.credit_eligible
                         && dut_posted_engine.pkt_valid)
    |-> dut_posted_engine.pkt_ready;
endproperty
a_no_cross_block : assert property (p_np_starvation_does_not_block_posted);
 
// P16: a full replay buffer blocks the launch and touches no credit.
property p_replay_is_not_credit;
  @(posedge clk) disable iff (!rst_n)
  !replay_space |=> ($stable(nph_available) && $stable(npd_available));
endproperty
a_not_replay : assert property (p_replay_is_not_credit);

P11, P12 and P13 are the chapter's most valuable properties, and none of them can be written inside a single module. P11 forbids a credit return from freeing a context — the silent-corruption case, where a Tag is reallocated while its Completion is still in flight. P12 is the mirror. P13 requires the two to move together on launch, which is what catches the drift the §8 callout describes.

P2 is asserted for all payload_bytes, deliberately. A property that only checked reads with zero length would pass a design that charges NPD from the Length field, because reads with a real Length are exactly the ones that break.

P15 is a forward-progress property in miniature. It states that exhausting the NP pool does not impede Posted traffic — the isolation from Chapter 16.1 §4, made checkable. The full argument belongs to Chapter 16.4 and 13.4; this is the local half.

No liveness is assertedChapter 16.1 §15's argument applies unchanged. Neither credit return nor Completion arrival is within this design's control.

11. Verification and Fault Injection

The scoreboard uses §9's model — a flat cost table and an independent resource tracker — and never reads ctx_available, nph_available or the DUT's eligibility. Contexts are tracked from observed launches and observed Completions; credit from observed launches and observed updates. Mirroring the DUT's admission helper would make the whole exercise circular.

Cost

  • Memory Read with Length 4, 128, 4096. Verify nph_cost = 1 and npd_cost = 0 every time (P2) — the required test, and the one a mirrored deriver fails at every non-zero length.
  • I/O Write and Configuration Write. Verify exactly 1 NPH + 1 NPD (P3).
  • AtomicOp at 8, 16, 17, 32, 33 bytes. The ceiling boundaries (P4) — the same 16/17 pair as Chapter 16.2 §12.
  • A Posted kind offered to this deriver. Verify cost_error and no cost.

Admission — one resource at a time

Withhold exactly one resource and verify the launch is blocked, the right flag is set, and nothing is consumed:

  • Contexts exhausted, NPH available. blocked_on_ctx, and NPH untouched.
  • NPH exhausted, contexts free. blocked_on_nph, and no context allocatedthe required test, and §14's advanced question.
  • Replay buffer full, everything else free (P16).
  • tx_ready low, everything else free.
  • All four available. Verify launch, and that context and credit move in the same cycle (P13).

The §4 distinction

  • A Completion arrives with no UpdateFC. Verify a context frees and credit does not (P12).
  • An UpdateFC arrives with no Completion. Verify credit rises and no context frees (P11) — the corruption test.
  • Both in the same cycle. Verify both apply, independently.
  • The Tag-doubling experiment (§4): run to steady state, double the context pool, and verify throughput improves only when contexts were the binding constraint.

Sustained and mixed

  • Reads until NPH is exhausted, then returns resume traffic.
  • Mixed Posted and Non-Posted traffic with NPH at zero. Verify writes continue (P15).
  • NPD exhausted with Memory Reads offered. Verify reads are unaffected (P9).
  • Reset mid-admission, and reset with a request waiting.

Mutations

#MutationCaught bySilicon symptom
1Memory Read charged to PH instead of NPHP14writes stall; reads drain the wrong pool
2NPH decremented on local_read_valid before launchP7NPH drains with no reads on the wire
3context allocated but credit not reservedP13counters drift; stall once they disagree enough
4credit consumed but context allocation failsP13credit leaks one per attempt; reads stop permanently
5NPD charged for a header-only Memory ReadP2, P9reads blocked by a pool they never use
6NPH allowed below zero16.1 P1availability wraps huge; receiver overflow
7NPH return credited to the PH poolP14, and the scoreboardPH grows without bound; NPH starves
8request launched with no replay capacityP6an unreplayable packet; data lost on the first error
9request metadata mutated while waitingA2, P10wrong address or length sent
10Posted write blocked by NPH exhaustionP15all traffic stops when reads back up
11context released on an NPH returnP11Completion matches a reallocated Tag — silent corruption
12AtomicOp NPD floored instead of ceilingedP4receiver overflow on unaligned AtomicOps

12. Performance — What NPH Actually Limits

NPH bounds how many Non-Posted request headers may be in flight toward the receiver. Nothing else.

When it is the bottleneck, read concurrency is capped regardless of free Tags, available bandwidth, or how quickly the completer could respond. And it is a remote resource, so no local change fixes it.

And note the term this chapter cannot supply: Completions consume CPLH/CPLD on the return direction, so a read can be blocked by capacity the requester advertised, not the completer. Chapter 16.4 closes that loop.

13. Debugging

Read throughput collapses but free Tags remain

The canonical NP symptom, and §8's flags answer it immediately.

  1. blocked_on_nph? Then the far side has not returned NPH. No local change helps — not more Tags, not a larger MRRS, not a faster requester.
  2. Is NPH ever returned? A pool that only falls means the receiver is not freeing request buffers, or the update path is not decoding (Chapter 15.2 §13).
  3. Was NPH advertised at all? An uninitialised pool refuses everything (Chapter 16.1 P4) — a failed FC initialisation looks exactly like this.
  4. Was it advertised as infinite? Zero means unlimited (§1); a design storing it as capacity blocks that class forever.

Free Tags with no throughput is the signature that eliminates the entire local design in one observation.

NPH decreases with no outbound Request

The consumption boundary is wrong (Chapter 16.1 §10).

Credit is being spent on acceptance or on scheduler selection rather than on launch. Check whether the decrement correlates with local_read_valid or with launch.

Or it is the drift bug (§8): credit consumed at send while contexts were allocated at acceptance, with the two now out of step. P13 distinguishes them — the drift bug shows credit and context counts changing on different cycles.

Writes continue while reads stop

Check whether this is a bug at all.

It is exactly what pool isolation is for (Chapter 16.1 §4). NPH exhausted while PH and PD have capacity means Posted traffic proceeds and Non-Posted does not — the architecture working.

It becomes a bug only if reads stop while NPH has capacity, and then it is a context shortage, a replay-buffer shortage, or the transmit path — which §8's other three flags name directly.

A read is accepted locally and never launches

In this chapter's model, that cannot happen — acceptance is launch (§8).

So observing it means the design is not this model. Either a reservation architecture is in use and its release path is wrong, or the accept-then-hope pattern was built and lost a resource in the window. Check whether local_read_ready ever asserts without launch — one waveform separates the two.

14. Common Misconceptions

  • "NPH equals the Tag count." Different resource, different owner, different lifetime (§4).
  • "NPH equals the number of outstanding reads." It bounds request headers the receiver can accept, not transactions you are awaiting (§5).
  • "A Non-Posted Request waits for credit after being transmitted." Credit is checked before transmission. Nothing waits for credit on the wire (§1).
  • "A Memory Read consumes NPD because data comes back." It consumes 1 NPH and nothing else. The returning data is charged to CPLH/CPLD, in the other direction (§1, §3).
  • "A Completion returns NPH." A Completion returns a context. An UpdateFC returns credit (§4, P11/P12).
  • "Posted and Non-Posted traffic share one pool." Six pools, and the isolation is the mechanism (Chapter 16.1 §4).
  • "MRRS determines NPH." Unrelated. A 4-byte read and a 4096-byte read both cost one NPH (§4).
  • "Free Tags guarantee a read may launch." Four resources are needed simultaneously (§8).
  • "NPH returns only when the Completion reaches the requester." It can return once the receiver frees the request buffer — potentially much earlier (§5).
  • "NPH exhaustion means the receiver is broken." It means the far side is not accepting requests as fast as you are issuing them (§13).
  • "A Non-Posted Request is stored in the replay buffer instead of receiver buffers." Both: local replay storage and remote receive capacity, independently (Chapter 16.1 §3).
  • "All Non-Posted requests consume NPD." Reads consume none; I/O and Configuration Writes consume exactly one (§3).
  • "Symmetry with Posted credits means NPD works like PD." PD scales with payload on every write; NPD is barely used (§3).

15. Understanding Check

16. What's Next

Non-Posted Requests cost 1 NPH — and a Memory Read costs nothing else at all, because it carries nothing. NPD exists for a narrow set: one credit for an I/O or Configuration Write, n for an AtomicOp, and none for any read.

The chapter's real content was not the cost. It was that NPH is not the Tag pool, that credit and context are freed by different agents at different points in a transaction's life, and that a design conflating them corrupts data without raising a single error.

And it was where the resources finally composed: four owners, four chapters, one conjunction — with the cleanest answer to the reservation problem being to have no window at all.

Chapter 16.4 — Completion Credits closes the loop with CPLH and CPLD, and answers the question this chapter deliberately left open: a read can be blocked by capacity the requester advertised, on the return direction, for the Completions it asked for.

Chapter 16.5 then takes spend policy — including the reservation architecture §8 declined to build — and 16.6 the update protocol that has been returning credit throughout.

The idea to carry forward: a resource you did not size is a resource you cannot fix — and the first job in any stall is to find out whose it was.