Skip to content

PCIe · Module 16

Posted Credits — One Header, and a Ceiling Division

A Memory Write costs 1 PH plus Roundup(Length / 16 bytes) PD. The header cost is a constant; the data cost is a division whose off-by-one is invisible until a payload misses a boundary — and then it overflows the receiver.

Chapter 10.3 established that a Posted Request receives no Completion. The transaction is over when the packet leaves.

That single fact produces the most durable misconception in PCIe flow control: if nothing comes back, nothing is owed — so posted traffic must be free.

It is not. A Memory Write lands in a receiver's buffer exactly like everything else, and it must be paid for before it is sent — from two pools, in two different units, with two different arithmetic.

What does a Posted TLP actually cost, and why is one of those two costs the most reliable source of receiver-overflow bugs in the entire protocol?

1. The Verified Costs

2. Posted Does Not Mean Free

The inference is natural and it is wrong at three separate layers.

"Posted means…"What is actually still true
no Completion✓ correct — Chapter 10.3
…so no acknowledgement✗ the Link still ACKs it (Chapter 14.2)
…so no replay state✗ it is retained until that ACK (Chapter 14.4)
…so no receive buffer✗ it lands in one, like everything else
…so no credits1 PH + n PD, paid before it is sent (§1)

3. Which Packets Use the Posted Pools

The rule the source supports, and the boundary it does not.

Memory Write Requests are the canonical Posted TLP, and §1's table gives their cost exactly: 1 PH + n PD.

Messages appear in both Posted rows"Message Requests without Data: 1 PH unit" and "Message Requests with Data: 1 PH + n PD units" — so a Message that reaches the Posted pools costs a header credit always and payload credits when it carries data.

4. The Two Costs Are Not the Same Kind of Computation

This is the chapter's structural point, and it is why the mutations cluster on one side.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
PH cost:   1                              a CONSTANT
PD cost:   Roundup(bytes / 16)            a CEILING DIVISION

The header cost cannot be got wrong. There is no length, no boundary, no rounding. Chapter 16.1 §5 explained why: the unit is sized for the largest possible header plus Digest, so a 3DW header and a 4DW header with a Digest both cost exactly one.

The data cost is where every bug lives, and there are three distinct ways to write it wrong:

Wrong formResultDetected by
bytes / 16under-charges by 1 whenever not a multiple of 16receiver overflow, eventually
bytes / 16 + 1over-charges by 1 on exact multiplesslow, never unsafe
(bytes + 16) / 16over-charges by 1 on exact multiplesslow, never unsafe
(bytes + 15) / 16correct

Note the asymmetry, because it decides how to prioritise testing. Two of the wrong forms are safe and slow; one is fast and catastrophic. The floor is the only one that under-charges, and it is also the one a tired engineer writes.

5. A Trace

Internal teaching signals, not PCIe wire signals. PH capacity 4, PD capacity 8 (128 bytes at 16 bytes per credit).

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
step             1     2     3     4     5     6     7
pkt_valid        1     1     1     1     1     1     0
pkt_bytes       64    17    64    64    64     -     -
pd_cost          4     2     4     4     4     -     -
 
ph_available     4     3     2     1     1     1     2
pd_available     8     4     2     2     2     6     6
 
eligible         1     1     0     0     1     0     -
send_fire        1     1     0     0     1     0     -
 
pd_return        0     0     0     0     0     4     0

Read step 2 — the boundary case. 17 bytes. Roundup(17/16) is 2, not 1. A floor would charge 1 here and the receiver would be short by 16 bytes of storage.

Read steps 3–4 — the refusal. PD is 2 and the packet needs 4. eligible is low, send_fire is low, and neither pool moves — note ph_available stays at 1 across both cycles even though PH alone would have been sufficient. That is the atomic pair working (Chapter 16.1 §9); a sequential design would have drained PH to 0 by step 4 and permanently.

Read step 5. Nothing changed and it is still blocked — the trace shows the stall persisting, which is what a credit-starved packet looks like: stable, valid, and going nowhere.

Read step 6. A PD return of 4 arrives. Available rises to 6. The packet is now affordable and will go on the next cycle the transmit path is free.

And note what never happens: ph_available never falls except on a send_fire.

6. RTL — Posted Credit Cost Deriver

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SYNTHESIZABLE. Derive Posted flow-control cost from a normalized packet
// descriptor.
// 1 PH per Posted TLP, and n PD where n = Roundup(Length / FC unit size)
// with the data unit 4 DW: NORMATIVE (section 1). The descriptor shape and
// the illegal-length report: ILLUSTRATIVE.
package posted_fc_pkg;
 
  // VERIFIED (section 1): the data credit unit is 4 DW = 16 bytes.
  localparam int FC_DATA_UNIT_BYTES = 16;
 
  // Normalized packet class, from Chapter 11.7's decode. NOT re-derived
  // from Fmt/Type here (section 3).
  typedef enum logic [1:0] {
    PKT_MEM_WRITE       = 2'd0,   // Posted:     1 PH + n PD
    PKT_MSG_NO_DATA     = 2'd1,   // Posted:     1 PH
    PKT_MSG_WITH_DATA   = 2'd2,   // Posted:     1 PH + n PD
    PKT_NOT_POSTED      = 2'd3    // uses the NP pools -- Chapter 16.3
  } posted_kind_e;
 
  // CEILING DIVISION, in the only form that is correct at every boundary.
  //
  //   bytes/16          under-charges by 1 unless bytes is a multiple of 16
  //   bytes/16 + 1      over-charges by 1 on exact multiples
  //   (bytes+16)/16     over-charges by 1 on exact multiples
  //   (bytes+15)/16     CORRECT                                (section 4)
  //
  // COMPILE-TIME SAFE: the addition is performed at a width that cannot
  // wrap, then the result is narrowed once the division has shrunk it.
  function automatic logic [11:0]
      pd_credit_cost(input logic [13:0] payload_bytes);
    logic [14:0] widened;              // one bit wider than the input, so
    widened = {1'b0, payload_bytes}    // + (UNIT-1) cannot overflow
            + 15'(FC_DATA_UNIT_BYTES - 1);
    return 12'(widened / 15'(FC_DATA_UNIT_BYTES));
  endfunction
 
endpackage
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import posted_fc_pkg::*;
 
module posted_credit_cost #(
  parameter int LEN_W  = 14,        // payload bytes; 14 bits covers 4096
  parameter int COST_W = 12
) (
  input  posted_kind_e        pkt_kind,
  input  logic [LEN_W-1:0]    payload_bytes,
 
  output logic                uses_ph,
  output logic [COST_W-1:0]   ph_cost,
  output logic                uses_pd,
  output logic [COST_W-1:0]   pd_cost,
 
  // A Posted packet claiming payload it cannot have, or a non-Posted kind
  // reaching this deriver at all. Reported rather than silently costed.
  output logic                cost_error
);
 
  wire is_posted = (pkt_kind != PKT_NOT_POSTED);
  wire has_data  = (pkt_kind == PKT_MEM_WRITE)
                || (pkt_kind == PKT_MSG_WITH_DATA);
 
  // THE HEADER COST IS A CONSTANT (section 4). There is nothing to compute
  // and therefore nothing to get wrong -- the unit is sized for the largest
  // header plus Digest, so a 3DW and a 4DW header both cost one.
  assign uses_ph = is_posted;
  assign ph_cost = is_posted ? COST_W'(1) : '0;
 
  // A ZERO-LENGTH payload consumes NO data credits. Charging one would
  // drain PD against packets that store nothing.
  wire data_present = has_data && (payload_bytes != '0);
 
  assign uses_pd = data_present;
  assign pd_cost = data_present
                     ? COST_W'(pd_credit_cost(14'(payload_bytes)))
                     : '0;
 
  assign cost_error = (!is_posted)
                   || (!has_data && (payload_bytes != '0));
 
endmodule

Classification: synthesizable (combinational) plus a compile-time helper function.

Architecture. A classification, a constant, and one ceiling division. The division is the only arithmetic in the module, which is deliberate — it is the part that needs auditing, and putting anything else beside it would dilute the review.

The widening in pd_credit_cost is not decoration. payload_bytes + 15 at 14 bits would wrap for lengths above 16369, producing a tiny cost for the largest packets — exactly the wrong direction. The addition is done at 15 bits, and the result is narrowed only after the division has made it small.

Cost table produced by this module:

Packetuses_ph / ph_costuses_pd / pd_cost
Memory Write, 64 bytes1 / 11 / 4
Memory Write, 17 bytes1 / 11 / 2
Memory Write, 16 bytes1 / 11 / 1
Message without data1 / 10 / 0
Message with data, 4 bytes1 / 11 / 1
Non-Posted kind0 / 00 / 0, cost_error

Failure — five. Floor division under-charges by one for every non-multiple of 16 (§4) — the dangerous one. Adding the full unit rather than unit−1 over-charges on exact multiples — safe but wasteful. Narrow addition wraps at the largest payloads. Charging PD for a zero-length payload drains the pool against packets that store nothing. And deriving has_data from a raw Type field rather than the normalized kind reintroduces the second decode §3 forbids.

Deliberately simplified: one packet per cycle; payload_bytes assumed already computed from Length (Chapter 11.3); no MPS check — that is a legality question, not a costing one.

7. RTL — Posted Issue Gate

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import posted_fc_pkg::*;
 
// SYNTHESIZABLE. Compose the cost deriver with Chapter 16.1's atomic pair
// and multi-resource gate for one Posted transmit path.
// The PH/PD costs are NORMATIVE (section 1). The composition, the reason
// flags and the consume-on-send boundary are Chapter 16.1's ILLUSTRATIVE
// implementation policy.
module posted_issue_gate #(
  parameter int CRED_W = 12,
  parameter int COST_W = 12,
  parameter int LEN_W  = 14
) (
  input  logic clk,
  input  logic rst_n,
 
  // ---- Head of the Posted transmit queue --------------------------------
  input  logic                pkt_valid,
  input  posted_kind_e        pkt_kind,
  input  logic [LEN_W-1:0]    payload_bytes,
  output logic                pkt_ready,
 
  // ---- Other launch resources (Chapter 16.1 section 10) -----------------
  input  logic                replay_space,
  input  logic                tx_path_ready,
 
  // ---- Credit plumbing ---------------------------------------------------
  input  logic                init_valid,
  input  logic [CRED_W-1:0]   init_ph_capacity,
  input  logic                init_ph_infinite,
  input  logic [CRED_W-1:0]   init_pd_capacity,
  input  logic                init_pd_infinite,
  input  logic                ph_return_valid,
  input  logic [CRED_W-1:0]   ph_return_count,
  input  logic                pd_return_valid,
  input  logic [CRED_W-1:0]   pd_return_count,
 
  // ---- Outputs ------------------------------------------------------------
  output logic                send_fire,
  output logic [CRED_W-1:0]   ph_available,
  output logic [CRED_W-1:0]   pd_available,
  output logic                blocked_on_ph,
  output logic                blocked_on_pd,
  output logic                blocked_on_replay,
  output logic                blocked_on_path,
  output logic                posted_error
);
 
  logic uses_ph, uses_pd, cost_err, credit_eligible;
  logic [COST_W-1:0] ph_cost, pd_cost;
 
  posted_credit_cost #(.LEN_W(LEN_W), .COST_W(COST_W)) u_cost (
    .pkt_kind, .payload_bytes,
    .uses_ph, .ph_cost, .uses_pd, .pd_cost, .cost_error(cost_err)
  );
 
  // ATOMIC ACROSS BOTH POOLS (Chapter 16.1 section 9). Eligibility is
  // evaluated for PH and PD together, and the two consume strobes come from
  // one commit -- so a short PD pool can never leak a PH credit.
  logic pair_err;
  credit_pair_gate #(.CRED_W(CRED_W), .COST_W(COST_W)) u_pair (
    .clk, .rst_n,
    .pkt_valid,
    .needs_header(uses_ph), .header_cost(ph_cost),
    .needs_data(uses_pd),   .data_cost(pd_cost),
    .send_fire,
    .hdr_return_valid(ph_return_valid), .hdr_return_count(ph_return_count),
    .dat_return_valid(pd_return_valid), .dat_return_count(pd_return_count),
    .init_valid,
    .init_hdr_capacity(init_ph_capacity),
    .init_hdr_infinite(init_ph_infinite),
    .init_dat_capacity(init_pd_capacity),
    .init_dat_infinite(init_pd_infinite),
    .eligible(credit_eligible),
    .hdr_available(ph_available), .dat_available(pd_available),
    .pool_error(pair_err)
  );
 
  // CREDIT IS ONE INPUT TO A LAUNCH, NOT THE WHOLE OF IT.
  credit_resource_gate u_gate (
    .clk, .rst_n,
    .pkt_valid, .pkt_ready,
    .credit_eligible, .replay_space, .tx_path_ready,
    .send_fire,
    .blocked_on_credit(), .blocked_on_replay, .blocked_on_path
  );
 
  // Split the credit reason by pool. "Blocked on credit" is not actionable;
  // "blocked on PD" tells an engineer which pool to watch (section 11).
  assign blocked_on_ph = pkt_valid && uses_ph && (ph_available < CRED_W'(ph_cost));
  assign blocked_on_pd = pkt_valid && uses_pd && (pd_available < CRED_W'(pd_cost));
  assign posted_error  = cost_err | pair_err;
 
endmodule

Classification: synthesizable.

Architecture. Three composed blocks and nothing else: cost → atomic pair → multi-resource gate. The chapter adds no new accounting; it supplies the PCIe-specific costs to Chapter 16.1's primitives.

The consumption boundary is send_fire — the irrevocable launch, not the scheduler's selection (Chapter 16.1 §10). A deeper transmit pipeline that must commit earlier needs reservation state, which is Chapter 16.5's.

Splitting the blocked reason by pool is the difference between a debug session and a guess (§11).

Failure — four. A single combined blocked_on_credit cannot tell PH exhaustion from PD exhaustion, and the two have completely different causes. Bypassing the pair gate to check PH and PD independently reintroduces the leak. Consuming on pkt_valid drains both pools while the transmit path stalls. And letting a packet whose cost_error is set proceed charges a Non-Posted packet to the Posted pools (§3).

8. Credit Starvation Propagates Backward

A write queue is where remote receive capacity becomes visible to local logic, and the chain is worth following end to end.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
remote receiver stops draining
   → PD credits stop returning
   → head-of-queue write is ineligible
   → posted queue stops dequeuing
   → queue fills
   → local producer sees its own ready go low
   → the producer stalls

Nothing in that chain is a Link error. Every block is behaving correctly. The far side's buffer occupancy has propagated backward through six stages of local design and surfaced as backpressure on a producer that has no idea PCIe exists.

9. RTL — Independent Cost Model

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// VERIFICATION-ONLY. Independent reference for the Posted credit cost.
// Deliberately NOT the same algorithm as section 6: this counts units,
// section 6 divides. Two implementations of one rule, and a mismatch means
// one of them is wrong.
function automatic int ref_pd_cost(int payload_bytes);
  int n, remaining;
  n = 0;
  remaining = payload_bytes;
  while (remaining > 0) begin
    n = n + 1;
    remaining = remaining - 16;      // 4 DW, verified (section 1)
  end
  return n;                          // zero-length yields 0
endfunction
 
function automatic int ref_ph_cost(bit is_posted);
  return is_posted ? 1 : 0;          // a constant, verified (section 1)
endfunction

Classification: verification-only.

The loop is the point. It cannot share a rounding bug with a division, and it makes the semantics unambiguous: every started 16-byte unit costs one credit. For 17 bytes it iterates twice; for 16, once; for 0, never.

10. Performance — The Posted Window

The steady-state write rate is set by how much PD the receiver advertised and how fast it returns it — not by the Link.

The relationship, derived rather than quoted:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
in-flight posted bytes  ≤  PD credits available × 16 bytes

A transmitter can burst until the advertised window is spent, then it waits for returns. If returns arrive at a steady rate, the sustained throughput is the return rate times the unit — and the Link's raw bandwidth does not enter the expression at all until it becomes the smaller limit.

Do not confuse this with a PCIe-defined minimum throughput. The relationship above is arithmetic on advertised capacity, not a protocol guarantee — PCIe defines the accounting, not a rate.

11. Assertions

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SVA over posted_credit_cost and posted_issue_gate, composed with Chapter
// 16.1's primitives. These assert the VERIFIED Posted costs and the local
// issue contract. Nothing here asserts credit-return timing (an environment
// property), the NP or Completion rows (Chapters 16.3-16.4), spend policy
// (16.5), or the update protocol (16.6).
 
// ---- ENVIRONMENT ------------------------------------------------------
// A1: pkt_kind is the NORMALIZED class from Chapter 11.7, not re-decoded.
assume property (@(posedge clk) disable iff (!rst_n)
  pkt_valid |-> (pkt_kind != PKT_NOT_POSTED));
// A2: payload_bytes is stable while the packet is offered.
assume property (@(posedge clk) disable iff (!rst_n)
  (pkt_valid && !pkt_ready) |=> $stable(payload_bytes));
 
// ---- COST -------------------------------------------------------------
 
// P1: THE HEADER COST IS EXACTLY ONE, always (section 1). No length
// dependence, no header-size dependence.
property p_ph_cost_is_one;
  @(posedge clk) disable iff (!rst_n)
  uses_ph |-> (ph_cost == COST_W'(1));
endproperty
a_ph_one : assert property (p_ph_cost_is_one);
 
// P2: THE CHAPTER'S CENTRAL PROPERTY. The data cost equals an INDEPENDENT
// model (section 9's counting loop, not section 6's division). A shared
// rounding bug cannot satisfy this.
property p_pd_cost_matches_reference;
  @(posedge clk) disable iff (!rst_n)
  uses_pd |-> (pd_cost == COST_W'(ref_pd_cost(int'(payload_bytes))));
endproperty
a_pd_ref : assert property (p_pd_cost_matches_reference);
 
// P2b: CEILING, NOT FLOOR -- stated directly, so the failure names itself.
// n*16 must COVER the payload, and (n-1)*16 must not.
property p_pd_cost_is_ceiling;
  @(posedge clk) disable iff (!rst_n)
  (uses_pd && (payload_bytes != '0))
    |-> ((COST_W'(pd_cost) * 16 >= payload_bytes)
      && ((COST_W'(pd_cost) - 1) * 16 < payload_bytes));
endproperty
a_ceiling : assert property (p_pd_cost_is_ceiling);
 
// P3: a Posted packet with no payload consumes NO data credits.
property p_no_payload_no_pd;
  @(posedge clk) disable iff (!rst_n)
  (pkt_kind == PKT_MSG_NO_DATA) |-> (!uses_pd && (pd_cost == '0));
endproperty
a_no_pd : assert property (p_no_payload_no_pd);
 
// ---- ISSUE ------------------------------------------------------------
 
// P4: no Posted packet issues without sufficient PH.
property p_no_issue_without_ph;
  @(posedge clk) disable iff (!rst_n)
  (send_fire && uses_ph) |-> (ph_available >= CRED_W'(ph_cost));
endproperty
a_ph_gate : assert property (p_no_issue_without_ph);
 
// P5: no payload-carrying Posted packet issues without sufficient PD.
property p_no_issue_without_pd;
  @(posedge clk) disable iff (!rst_n)
  (send_fire && uses_pd) |-> (pd_available >= CRED_W'(pd_cost));
endproperty
a_pd_gate : assert property (p_no_issue_without_pd);
 
// P6: ATOMICITY. A failed PD eligibility consumes NO PH -- the leak in
// Chapter 16.1 section 9, in its Posted form.
property p_failed_pd_leaves_ph;
  @(posedge clk) disable iff (!rst_n)
  (pkt_valid && uses_pd && (pd_available < CRED_W'(pd_cost)))
    |=> $stable(ph_available);
endproperty
a_atomic : assert property (p_failed_pd_leaves_ph);
 
// P7: and the mirror -- a failed PH eligibility consumes no PD.
property p_failed_ph_leaves_pd;
  @(posedge clk) disable iff (!rst_n)
  (pkt_valid && uses_ph && (ph_available < CRED_W'(ph_cost)))
    |=> $stable(pd_available);
endproperty
a_atomic_mirror : assert property (p_failed_ph_leaves_pd);
 
// P8: a header-only Posted packet never touches PD.
property p_header_only_leaves_pd;
  @(posedge clk) disable iff (!rst_n)
  (send_fire && !uses_pd) |=> $stable(pd_available);
endproperty
a_isolation : assert property (p_header_only_leaves_pd);
 
// P9: CREDIT IS CONSUMED EXACTLY ONCE PER ISSUED PACKET. Ghost counters
// are testbench state.
property p_consumed_once;
  @(posedge clk) disable iff (!rst_n)
  send_fire |=> ((g_ph_spent == $past(g_ph_spent) + $past(ph_cost))
              && (g_pd_spent == $past(g_pd_spent) + $past(pd_cost)));
endproperty
a_once : assert property (p_consumed_once);
 
// P10: no credit consumed without an issue (Chapter 16.1 P8, Posted form).
property p_no_phantom_consumption;
  @(posedge clk) disable iff (!rst_n)
  (pkt_valid && !send_fire)
    |=> ($stable(ph_available) && $stable(pd_available));
endproperty
a_no_phantom : assert property (p_no_phantom_consumption);
 
// P11: a credit-starved packet is STABLE and still offered.
property p_starved_stable;
  @(posedge clk) disable iff (!rst_n)
  (pkt_valid && !pkt_ready)
    |=> (pkt_valid && $stable(pkt_kind) && $stable(payload_bytes));
endproperty
a_stable : assert property (p_starved_stable);
 
// ---- CROSS-POOL NON-INTERFERENCE --------------------------------------
 
// P12: A NON-POSTED PACKET NEVER CONSUMES POSTED CREDIT through this
// engine (section 3). The mutation that drains PH with I/O writes.
property p_np_never_spends_posted;
  @(posedge clk) disable iff (!rst_n)
  (dut_np_engine.send_fire && !send_fire)
    |=> ($stable(ph_available) && $stable(pd_available));
endproperty
a_class_isolation : assert property (p_np_never_spends_posted);
 
// P13: a PD return never lands in the PH pool, and vice versa.
property p_returns_do_not_cross;
  @(posedge clk) disable iff (!rst_n)
  (pd_return_valid && !ph_return_valid) |=> $stable(ph_available);
endproperty
a_return_pool : assert property (p_returns_do_not_cross);
 
// P14: a full replay buffer blocks the launch and touches no credit.
property p_replay_is_not_credit;
  @(posedge clk) disable iff (!rst_n)
  !replay_space |=> ($stable(ph_available) && $stable(pd_available));
endproperty
a_not_replay : assert property (p_replay_is_not_credit);

P2 and P2b are deliberately redundant, and both earn their place. P2 compares against an independently written model — it catches any disagreement but does not say which side is wrong. P2b states the ceiling property directly: n×16 covers the payload and (n−1)×16 does not. When P2b fails, the message names the defect; when only P2 fails, you still have to work out which implementation is at fault.

P6 and P7 are the same atomicity in both directions, and both are needed. A design that guards PH-before-PD but not PD-before-PH passes one and leaks on the other.

P12 is a cross-engine non-interference property in the style of Chapter 15.5 §11. It cannot be written inside either engine — neither sees the other's send event — and it is the only mechanism that catches an I/O Write charged to the Posted pools, because both engines are individually correct.

12. Verification and Fault Injection

The scoreboard uses §9's counting model, never §6's division, and maintains its own PH/PD pools from observed initialisations, returns and sends.

Cost boundaries — the required set

PayloadExpected PDWhy this value
00no payload, no data credit
11the smallest non-zero payload
151just under one unit
161exactly one unit
172one byte over — the floor-division killer
312just under two
322exactly two
644a common aligned size
4095 / 4096256 / 256the maximum-payload pair

The 16/17 and 32/33 pairs are the tests that matter. Aligned power-of-two payloads — which is what most directed tests generate — cost the same under a floor and a ceiling, so a test suite made only of those will pass a broken design.

Issue

  • PH available, PD insufficient. Verify neither pool moves (P6) — required.
  • PD available, PH insufficient. The mirror (P7).
  • Both available. Verify both move by exactly the derived costs (P9).
  • A Message without data with PD at zero. Verify it issues and PD is untouched (P8).
  • Both pools exhausted. Verify the packet waits, stable (P11).
  • A PD return arriving while the packet is stalled. Verify it becomes eligible that cycle (Chapter 16.1 §8's contract).
  • Continuous maximum-size writes until PD is exhausted, then returns. The sustained-throughput case (§10).
  • Replay buffer full with credit available. Verify no launch, no credit moved (P14).
  • tx_path_ready low with everything else available. Verify no consumption (P10).
  • A queue full behind a stalled head. Verify the backpressure chain of §8.
  • Reset mid-issue, and reset with a stalled packet.

Mutations

#MutationCaught bySilicon symptom
1payload_bytes / 16 — floor instead of ceilingP2, P2b at 17 bytesreceiver overflow on unaligned payloads under load
2PH consumed when PD is shortP6slow PH leak; writes stop permanently after minutes
3PD consumed on a stalled packetP10PD drains with no packets on the wire
4Memory Read charged to the Posted poolsP12reads drain PH; writes stall for no visible reason
5stale payload_bytes used for the costA2, P2intermittent under- or over-charge; load-dependent
6a PD return credited to PHP13PH grows without bound, PD starves
7one packet consumes credit twiceP9throughput halves; pools drain at 2× the send rate
8the queue entry freed before send_fireP10, and the scoreboardpackets lost locally, credit spent
9blocked_on_pd computed from the PH poolno property — a debug-visibility bugdebugging leads to the wrong pool
10payload_bytes + 15 computed at LEN_WP2 at 4096largest packets cost almost nothing; immediate overflow
11(bytes + 16)/16 — off-by-one the safe wayP2b at 16 bytesthroughput loss only; never unsafe
12zero-length payload charged 1 PDP3PD drains against packets that store nothing

13. Debugging

Small writes work, large writes stall forever

Almost certainly PD, and the split reason flag says so immediately.

  1. blocked_on_pd asserted? Then the derived cost exceeds available PD.
  2. Compare pd_cost against §9's model for that payload. A mismatch is §6's division.
  3. If the cost is right, is PD ever returned? A pool that only falls is a receiver that is not draining, or an update path that is not decoding (Chapter 15.2 §13).
  4. Is the advertised PD large enough for one maximum payload at all? §10: if the receiver advertised less than MPS/16, the largest legal write can never be issued — the stall is permanent by construction and no amount of waiting helps.

Step 4 is the one that gets missed, and it looks exactly like a credit-return problem until you compare the advertisement against the packet size.

The consumption boundary is wrong (Chapter 16.1 §10).

Check whether the decrement correlates with pkt_valid or with send_fire. If PH falls while the transmit path is stalled, credit is being spent at scheduler selection rather than at launch.

Or it is the leak (§12's mutation 2): PH decremented before PD eligibility failed. Distinguish them in one waveform — the leak only decrements when PD is short, the boundary bug decrements whenever a packet is offered.

Memory Reads reduce Posted credits

Classification, not accounting (§3).

A read is Non-Posted and must charge NPH. If PH is falling on reads, the normalized packet kind is wrong upstream, or the credit engine is selected by direction rather than by posting semantics.

P12 catches it in simulation. In the lab, the tell is that PH and NPH move together, or that NPH never moves at all.

Only payloads just above a boundary fail

The ceiling division — §12's counterexample, and this symptom is nearly diagnostic on its own.

If 16, 32 and 64 bytes work and 17, 33 and 65 fail, the cost function is flooring. No other bug produces that pattern, because no other bug is sensitive to the payload modulo 16.

14. Common Misconceptions

  • "Posted traffic needs no credits." It needs 1 PH + n PD, paid before transmission (§1, §2).
  • "Posted means no acknowledgement." No Completion. The Link still ACKs it and retains it for replay (Chapter 14.2, 14.4).
  • "A Memory Write only consumes data credit." It consumes a header credit too — every TLP does (§1).
  • "Header credit represents bytes." It represents one header, sized for the largest possible (Chapter 16.1 §5).
  • "PD cost can be rounded down." Roundup — the source says so, and flooring overflows the receiver (§4, §12).
  • "A 16-byte and a 17-byte write cost the same." 1 PD and 2 PD (§4).
  • "Credit is consumed when the TLP enters the queue." On the irrevocable send (§7, P10).
  • "An ACK restores Posted credit." An ACK returns replay storage; UpdateFC returns credit (P14, Chapter 16.1 §3).
  • "Posted credit exhaustion means the transaction failed." Nothing failed. The transmitter is waiting for capacity (§8).
  • "All write-like packets use Posted credits." I/O and Configuration Writes are Non-Posted1 NPH + 1 NPD (§3).
  • "A bigger MPS always means better throughput." Bigger packets cost proportionally more PD. With a fixed advertised pool, fewer are in flight (§10).
  • "The replay buffer and PH/PD are one resource." Local TX storage versus remote RX capacity (Chapter 16.1 §3).
  • "A zero-length write costs one PD." It costs zero — there is nothing to store (P3).

15. Understanding Check

16. What's Next

Posted traffic is one-way and fully accounted. 1 PH always, and Roundup(Length / 16) PD when there is payload — a constant and a ceiling division, with all the danger in the division.

The atomic pair is not stylistic here: a short PD pool leaking PH credits kills a traffic class permanently, and the only reliable defence is a structure with no ordering to exploit.

Chapter 16.3 — Non-Posted Credits takes NPH and NPD, and a packet with the opposite shape: a Memory Read carries no payload and costs 1 NPH alone — yet it is the request most likely to be resource-blocked, because it needs three other things at the same time. Why a requester with free Tags, free replay space and an idle Link still cannot issue a read is the question there.

Chapter 16.4 then closes the loop with CPLH and CPLD.

The idea to carry forward: the header cost is a constant you cannot get wrong; the data cost is a division you can get wrong once, quietly, for years.