PCIe · Module 16
Posted Credits — One Header, and a Ceiling Division
A Memory Write costs 1 PH plus Roundup(Length / 16 bytes) PD. The header cost is a constant; the data cost is a division whose off-by-one is invisible until a payload misses a boundary — and then it overflows the receiver.
Chapter 10.3 established that a Posted Request receives no Completion. The transaction is over when the packet leaves.
That single fact produces the most durable misconception in PCIe flow control: if nothing comes back, nothing is owed — so posted traffic must be free.
It is not. A Memory Write lands in a receiver's buffer exactly like everything else, and it must be paid for before it is sent — from two pools, in two different units, with two different arithmetic.
What does a Posted TLP actually cost, and why is one of those two costs the most reliable source of receiver-overflow bugs in the entire protocol?
1. The Verified Costs
2. Posted Does Not Mean Free
The inference is natural and it is wrong at three separate layers.
| "Posted means…" | What is actually still true |
|---|---|
| no Completion | ✓ correct — Chapter 10.3 |
| …so no acknowledgement | ✗ the Link still ACKs it (Chapter 14.2) |
| …so no replay state | ✗ it is retained until that ACK (Chapter 14.4) |
| …so no receive buffer | ✗ it lands in one, like everything else |
| …so no credits | ✗ 1 PH + n PD, paid before it is sent (§1) |
3. Which Packets Use the Posted Pools
The rule the source supports, and the boundary it does not.
Memory Write Requests are the canonical Posted TLP, and §1's table gives their cost exactly: 1 PH + n PD.
Messages appear in both Posted rows — "Message Requests without Data: 1 PH unit" and "Message Requests with Data: 1 PH + n PD units" — so a Message that reaches the Posted pools costs a header credit always and payload credits when it carries data.
4. The Two Costs Are Not the Same Kind of Computation
This is the chapter's structural point, and it is why the mutations cluster on one side.
PH cost: 1 a CONSTANT
PD cost: Roundup(bytes / 16) a CEILING DIVISIONThe header cost cannot be got wrong. There is no length, no boundary, no rounding. Chapter 16.1 §5 explained why: the unit is sized for the largest possible header plus Digest, so a 3DW header and a 4DW header with a Digest both cost exactly one.
The data cost is where every bug lives, and there are three distinct ways to write it wrong:
| Wrong form | Result | Detected by |
|---|---|---|
bytes / 16 | under-charges by 1 whenever not a multiple of 16 | receiver overflow, eventually |
bytes / 16 + 1 | over-charges by 1 on exact multiples | slow, never unsafe |
(bytes + 16) / 16 | over-charges by 1 on exact multiples | slow, never unsafe |
(bytes + 15) / 16 | correct | — |
Note the asymmetry, because it decides how to prioritise testing. Two of the wrong forms are safe and slow; one is fast and catastrophic. The floor is the only one that under-charges, and it is also the one a tired engineer writes.
5. A Trace
Internal teaching signals, not PCIe wire signals. PH capacity 4, PD capacity 8 (128 bytes at 16 bytes per credit).
step 1 2 3 4 5 6 7
pkt_valid 1 1 1 1 1 1 0
pkt_bytes 64 17 64 64 64 - -
pd_cost 4 2 4 4 4 - -
ph_available 4 3 2 1 1 1 2
pd_available 8 4 2 2 2 6 6
eligible 1 1 0 0 1 0 -
send_fire 1 1 0 0 1 0 -
pd_return 0 0 0 0 0 4 0Read step 2 — the boundary case. 17 bytes. Roundup(17/16) is 2, not 1. A floor would charge 1 here and the receiver would be short by 16 bytes of storage.
Read steps 3–4 — the refusal. PD is 2 and the packet needs 4. eligible is low, send_fire is low, and neither pool moves — note ph_available stays at 1 across both cycles even though PH alone would have been sufficient. That is the atomic pair working (Chapter 16.1 §9); a sequential design would have drained PH to 0 by step 4 and permanently.
Read step 5. Nothing changed and it is still blocked — the trace shows the stall persisting, which is what a credit-starved packet looks like: stable, valid, and going nowhere.
Read step 6. A PD return of 4 arrives. Available rises to 6. The packet is now affordable and will go on the next cycle the transmit path is free.
And note what never happens: ph_available never falls except on a send_fire.
6. RTL — Posted Credit Cost Deriver
// SYNTHESIZABLE. Derive Posted flow-control cost from a normalized packet
// descriptor.
// 1 PH per Posted TLP, and n PD where n = Roundup(Length / FC unit size)
// with the data unit 4 DW: NORMATIVE (section 1). The descriptor shape and
// the illegal-length report: ILLUSTRATIVE.
package posted_fc_pkg;
// VERIFIED (section 1): the data credit unit is 4 DW = 16 bytes.
localparam int FC_DATA_UNIT_BYTES = 16;
// Normalized packet class, from Chapter 11.7's decode. NOT re-derived
// from Fmt/Type here (section 3).
typedef enum logic [1:0] {
PKT_MEM_WRITE = 2'd0, // Posted: 1 PH + n PD
PKT_MSG_NO_DATA = 2'd1, // Posted: 1 PH
PKT_MSG_WITH_DATA = 2'd2, // Posted: 1 PH + n PD
PKT_NOT_POSTED = 2'd3 // uses the NP pools -- Chapter 16.3
} posted_kind_e;
// CEILING DIVISION, in the only form that is correct at every boundary.
//
// bytes/16 under-charges by 1 unless bytes is a multiple of 16
// bytes/16 + 1 over-charges by 1 on exact multiples
// (bytes+16)/16 over-charges by 1 on exact multiples
// (bytes+15)/16 CORRECT (section 4)
//
// COMPILE-TIME SAFE: the addition is performed at a width that cannot
// wrap, then the result is narrowed once the division has shrunk it.
function automatic logic [11:0]
pd_credit_cost(input logic [13:0] payload_bytes);
logic [14:0] widened; // one bit wider than the input, so
widened = {1'b0, payload_bytes} // + (UNIT-1) cannot overflow
+ 15'(FC_DATA_UNIT_BYTES - 1);
return 12'(widened / 15'(FC_DATA_UNIT_BYTES));
endfunction
endpackageimport posted_fc_pkg::*;
module posted_credit_cost #(
parameter int LEN_W = 14, // payload bytes; 14 bits covers 4096
parameter int COST_W = 12
) (
input posted_kind_e pkt_kind,
input logic [LEN_W-1:0] payload_bytes,
output logic uses_ph,
output logic [COST_W-1:0] ph_cost,
output logic uses_pd,
output logic [COST_W-1:0] pd_cost,
// A Posted packet claiming payload it cannot have, or a non-Posted kind
// reaching this deriver at all. Reported rather than silently costed.
output logic cost_error
);
wire is_posted = (pkt_kind != PKT_NOT_POSTED);
wire has_data = (pkt_kind == PKT_MEM_WRITE)
|| (pkt_kind == PKT_MSG_WITH_DATA);
// THE HEADER COST IS A CONSTANT (section 4). There is nothing to compute
// and therefore nothing to get wrong -- the unit is sized for the largest
// header plus Digest, so a 3DW and a 4DW header both cost one.
assign uses_ph = is_posted;
assign ph_cost = is_posted ? COST_W'(1) : '0;
// A ZERO-LENGTH payload consumes NO data credits. Charging one would
// drain PD against packets that store nothing.
wire data_present = has_data && (payload_bytes != '0);
assign uses_pd = data_present;
assign pd_cost = data_present
? COST_W'(pd_credit_cost(14'(payload_bytes)))
: '0;
assign cost_error = (!is_posted)
|| (!has_data && (payload_bytes != '0));
endmoduleClassification: synthesizable (combinational) plus a compile-time helper function.
Architecture. A classification, a constant, and one ceiling division. The division is the only arithmetic in the module, which is deliberate — it is the part that needs auditing, and putting anything else beside it would dilute the review.
The widening in pd_credit_cost is not decoration. payload_bytes + 15 at 14 bits would wrap for lengths above 16369, producing a tiny cost for the largest packets — exactly the wrong direction. The addition is done at 15 bits, and the result is narrowed only after the division has made it small.
Cost table produced by this module:
| Packet | uses_ph / ph_cost | uses_pd / pd_cost |
|---|---|---|
| Memory Write, 64 bytes | 1 / 1 | 1 / 4 |
| Memory Write, 17 bytes | 1 / 1 | 1 / 2 |
| Memory Write, 16 bytes | 1 / 1 | 1 / 1 |
| Message without data | 1 / 1 | 0 / 0 |
| Message with data, 4 bytes | 1 / 1 | 1 / 1 |
| Non-Posted kind | 0 / 0 | 0 / 0, cost_error |
Failure — five. Floor division under-charges by one for every non-multiple of 16 (§4) — the dangerous one. Adding the full unit rather than unit−1 over-charges on exact multiples — safe but wasteful. Narrow addition wraps at the largest payloads. Charging PD for a zero-length payload drains the pool against packets that store nothing. And deriving has_data from a raw Type field rather than the normalized kind reintroduces the second decode §3 forbids.
Deliberately simplified: one packet per cycle; payload_bytes assumed already computed from Length (Chapter 11.3); no MPS check — that is a legality question, not a costing one.
7. RTL — Posted Issue Gate
import posted_fc_pkg::*;
// SYNTHESIZABLE. Compose the cost deriver with Chapter 16.1's atomic pair
// and multi-resource gate for one Posted transmit path.
// The PH/PD costs are NORMATIVE (section 1). The composition, the reason
// flags and the consume-on-send boundary are Chapter 16.1's ILLUSTRATIVE
// implementation policy.
module posted_issue_gate #(
parameter int CRED_W = 12,
parameter int COST_W = 12,
parameter int LEN_W = 14
) (
input logic clk,
input logic rst_n,
// ---- Head of the Posted transmit queue --------------------------------
input logic pkt_valid,
input posted_kind_e pkt_kind,
input logic [LEN_W-1:0] payload_bytes,
output logic pkt_ready,
// ---- Other launch resources (Chapter 16.1 section 10) -----------------
input logic replay_space,
input logic tx_path_ready,
// ---- Credit plumbing ---------------------------------------------------
input logic init_valid,
input logic [CRED_W-1:0] init_ph_capacity,
input logic init_ph_infinite,
input logic [CRED_W-1:0] init_pd_capacity,
input logic init_pd_infinite,
input logic ph_return_valid,
input logic [CRED_W-1:0] ph_return_count,
input logic pd_return_valid,
input logic [CRED_W-1:0] pd_return_count,
// ---- Outputs ------------------------------------------------------------
output logic send_fire,
output logic [CRED_W-1:0] ph_available,
output logic [CRED_W-1:0] pd_available,
output logic blocked_on_ph,
output logic blocked_on_pd,
output logic blocked_on_replay,
output logic blocked_on_path,
output logic posted_error
);
logic uses_ph, uses_pd, cost_err, credit_eligible;
logic [COST_W-1:0] ph_cost, pd_cost;
posted_credit_cost #(.LEN_W(LEN_W), .COST_W(COST_W)) u_cost (
.pkt_kind, .payload_bytes,
.uses_ph, .ph_cost, .uses_pd, .pd_cost, .cost_error(cost_err)
);
// ATOMIC ACROSS BOTH POOLS (Chapter 16.1 section 9). Eligibility is
// evaluated for PH and PD together, and the two consume strobes come from
// one commit -- so a short PD pool can never leak a PH credit.
logic pair_err;
credit_pair_gate #(.CRED_W(CRED_W), .COST_W(COST_W)) u_pair (
.clk, .rst_n,
.pkt_valid,
.needs_header(uses_ph), .header_cost(ph_cost),
.needs_data(uses_pd), .data_cost(pd_cost),
.send_fire,
.hdr_return_valid(ph_return_valid), .hdr_return_count(ph_return_count),
.dat_return_valid(pd_return_valid), .dat_return_count(pd_return_count),
.init_valid,
.init_hdr_capacity(init_ph_capacity),
.init_hdr_infinite(init_ph_infinite),
.init_dat_capacity(init_pd_capacity),
.init_dat_infinite(init_pd_infinite),
.eligible(credit_eligible),
.hdr_available(ph_available), .dat_available(pd_available),
.pool_error(pair_err)
);
// CREDIT IS ONE INPUT TO A LAUNCH, NOT THE WHOLE OF IT.
credit_resource_gate u_gate (
.clk, .rst_n,
.pkt_valid, .pkt_ready,
.credit_eligible, .replay_space, .tx_path_ready,
.send_fire,
.blocked_on_credit(), .blocked_on_replay, .blocked_on_path
);
// Split the credit reason by pool. "Blocked on credit" is not actionable;
// "blocked on PD" tells an engineer which pool to watch (section 11).
assign blocked_on_ph = pkt_valid && uses_ph && (ph_available < CRED_W'(ph_cost));
assign blocked_on_pd = pkt_valid && uses_pd && (pd_available < CRED_W'(pd_cost));
assign posted_error = cost_err | pair_err;
endmoduleClassification: synthesizable.
Architecture. Three composed blocks and nothing else: cost → atomic pair → multi-resource gate. The chapter adds no new accounting; it supplies the PCIe-specific costs to Chapter 16.1's primitives.
The consumption boundary is send_fire — the irrevocable launch, not the scheduler's selection (Chapter 16.1 §10). A deeper transmit pipeline that must commit earlier needs reservation state, which is Chapter 16.5's.
Splitting the blocked reason by pool is the difference between a debug session and a guess (§11).
Failure — four. A single combined blocked_on_credit cannot tell PH exhaustion from PD exhaustion, and the two have completely different causes. Bypassing the pair gate to check PH and PD independently reintroduces the leak. Consuming on pkt_valid drains both pools while the transmit path stalls. And letting a packet whose cost_error is set proceed charges a Non-Posted packet to the Posted pools (§3).
8. Credit Starvation Propagates Backward
A write queue is where remote receive capacity becomes visible to local logic, and the chain is worth following end to end.
remote receiver stops draining
→ PD credits stop returning
→ head-of-queue write is ineligible
→ posted queue stops dequeuing
→ queue fills
→ local producer sees its own ready go low
→ the producer stallsNothing in that chain is a Link error. Every block is behaving correctly. The far side's buffer occupancy has propagated backward through six stages of local design and surfaced as backpressure on a producer that has no idea PCIe exists.
9. RTL — Independent Cost Model
// VERIFICATION-ONLY. Independent reference for the Posted credit cost.
// Deliberately NOT the same algorithm as section 6: this counts units,
// section 6 divides. Two implementations of one rule, and a mismatch means
// one of them is wrong.
function automatic int ref_pd_cost(int payload_bytes);
int n, remaining;
n = 0;
remaining = payload_bytes;
while (remaining > 0) begin
n = n + 1;
remaining = remaining - 16; // 4 DW, verified (section 1)
end
return n; // zero-length yields 0
endfunction
function automatic int ref_ph_cost(bit is_posted);
return is_posted ? 1 : 0; // a constant, verified (section 1)
endfunctionClassification: verification-only.
The loop is the point. It cannot share a rounding bug with a division, and it makes the semantics unambiguous: every started 16-byte unit costs one credit. For 17 bytes it iterates twice; for 16, once; for 0, never.
10. Performance — The Posted Window
The steady-state write rate is set by how much PD the receiver advertised and how fast it returns it — not by the Link.
The relationship, derived rather than quoted:
in-flight posted bytes ≤ PD credits available × 16 bytesA transmitter can burst until the advertised window is spent, then it waits for returns. If returns arrive at a steady rate, the sustained throughput is the return rate times the unit — and the Link's raw bandwidth does not enter the expression at all until it becomes the smaller limit.
Do not confuse this with a PCIe-defined minimum throughput. The relationship above is arithmetic on advertised capacity, not a protocol guarantee — PCIe defines the accounting, not a rate.
11. Assertions
// SVA over posted_credit_cost and posted_issue_gate, composed with Chapter
// 16.1's primitives. These assert the VERIFIED Posted costs and the local
// issue contract. Nothing here asserts credit-return timing (an environment
// property), the NP or Completion rows (Chapters 16.3-16.4), spend policy
// (16.5), or the update protocol (16.6).
// ---- ENVIRONMENT ------------------------------------------------------
// A1: pkt_kind is the NORMALIZED class from Chapter 11.7, not re-decoded.
assume property (@(posedge clk) disable iff (!rst_n)
pkt_valid |-> (pkt_kind != PKT_NOT_POSTED));
// A2: payload_bytes is stable while the packet is offered.
assume property (@(posedge clk) disable iff (!rst_n)
(pkt_valid && !pkt_ready) |=> $stable(payload_bytes));
// ---- COST -------------------------------------------------------------
// P1: THE HEADER COST IS EXACTLY ONE, always (section 1). No length
// dependence, no header-size dependence.
property p_ph_cost_is_one;
@(posedge clk) disable iff (!rst_n)
uses_ph |-> (ph_cost == COST_W'(1));
endproperty
a_ph_one : assert property (p_ph_cost_is_one);
// P2: THE CHAPTER'S CENTRAL PROPERTY. The data cost equals an INDEPENDENT
// model (section 9's counting loop, not section 6's division). A shared
// rounding bug cannot satisfy this.
property p_pd_cost_matches_reference;
@(posedge clk) disable iff (!rst_n)
uses_pd |-> (pd_cost == COST_W'(ref_pd_cost(int'(payload_bytes))));
endproperty
a_pd_ref : assert property (p_pd_cost_matches_reference);
// P2b: CEILING, NOT FLOOR -- stated directly, so the failure names itself.
// n*16 must COVER the payload, and (n-1)*16 must not.
property p_pd_cost_is_ceiling;
@(posedge clk) disable iff (!rst_n)
(uses_pd && (payload_bytes != '0))
|-> ((COST_W'(pd_cost) * 16 >= payload_bytes)
&& ((COST_W'(pd_cost) - 1) * 16 < payload_bytes));
endproperty
a_ceiling : assert property (p_pd_cost_is_ceiling);
// P3: a Posted packet with no payload consumes NO data credits.
property p_no_payload_no_pd;
@(posedge clk) disable iff (!rst_n)
(pkt_kind == PKT_MSG_NO_DATA) |-> (!uses_pd && (pd_cost == '0));
endproperty
a_no_pd : assert property (p_no_payload_no_pd);
// ---- ISSUE ------------------------------------------------------------
// P4: no Posted packet issues without sufficient PH.
property p_no_issue_without_ph;
@(posedge clk) disable iff (!rst_n)
(send_fire && uses_ph) |-> (ph_available >= CRED_W'(ph_cost));
endproperty
a_ph_gate : assert property (p_no_issue_without_ph);
// P5: no payload-carrying Posted packet issues without sufficient PD.
property p_no_issue_without_pd;
@(posedge clk) disable iff (!rst_n)
(send_fire && uses_pd) |-> (pd_available >= CRED_W'(pd_cost));
endproperty
a_pd_gate : assert property (p_no_issue_without_pd);
// P6: ATOMICITY. A failed PD eligibility consumes NO PH -- the leak in
// Chapter 16.1 section 9, in its Posted form.
property p_failed_pd_leaves_ph;
@(posedge clk) disable iff (!rst_n)
(pkt_valid && uses_pd && (pd_available < CRED_W'(pd_cost)))
|=> $stable(ph_available);
endproperty
a_atomic : assert property (p_failed_pd_leaves_ph);
// P7: and the mirror -- a failed PH eligibility consumes no PD.
property p_failed_ph_leaves_pd;
@(posedge clk) disable iff (!rst_n)
(pkt_valid && uses_ph && (ph_available < CRED_W'(ph_cost)))
|=> $stable(pd_available);
endproperty
a_atomic_mirror : assert property (p_failed_ph_leaves_pd);
// P8: a header-only Posted packet never touches PD.
property p_header_only_leaves_pd;
@(posedge clk) disable iff (!rst_n)
(send_fire && !uses_pd) |=> $stable(pd_available);
endproperty
a_isolation : assert property (p_header_only_leaves_pd);
// P9: CREDIT IS CONSUMED EXACTLY ONCE PER ISSUED PACKET. Ghost counters
// are testbench state.
property p_consumed_once;
@(posedge clk) disable iff (!rst_n)
send_fire |=> ((g_ph_spent == $past(g_ph_spent) + $past(ph_cost))
&& (g_pd_spent == $past(g_pd_spent) + $past(pd_cost)));
endproperty
a_once : assert property (p_consumed_once);
// P10: no credit consumed without an issue (Chapter 16.1 P8, Posted form).
property p_no_phantom_consumption;
@(posedge clk) disable iff (!rst_n)
(pkt_valid && !send_fire)
|=> ($stable(ph_available) && $stable(pd_available));
endproperty
a_no_phantom : assert property (p_no_phantom_consumption);
// P11: a credit-starved packet is STABLE and still offered.
property p_starved_stable;
@(posedge clk) disable iff (!rst_n)
(pkt_valid && !pkt_ready)
|=> (pkt_valid && $stable(pkt_kind) && $stable(payload_bytes));
endproperty
a_stable : assert property (p_starved_stable);
// ---- CROSS-POOL NON-INTERFERENCE --------------------------------------
// P12: A NON-POSTED PACKET NEVER CONSUMES POSTED CREDIT through this
// engine (section 3). The mutation that drains PH with I/O writes.
property p_np_never_spends_posted;
@(posedge clk) disable iff (!rst_n)
(dut_np_engine.send_fire && !send_fire)
|=> ($stable(ph_available) && $stable(pd_available));
endproperty
a_class_isolation : assert property (p_np_never_spends_posted);
// P13: a PD return never lands in the PH pool, and vice versa.
property p_returns_do_not_cross;
@(posedge clk) disable iff (!rst_n)
(pd_return_valid && !ph_return_valid) |=> $stable(ph_available);
endproperty
a_return_pool : assert property (p_returns_do_not_cross);
// P14: a full replay buffer blocks the launch and touches no credit.
property p_replay_is_not_credit;
@(posedge clk) disable iff (!rst_n)
!replay_space |=> ($stable(ph_available) && $stable(pd_available));
endproperty
a_not_replay : assert property (p_replay_is_not_credit);P2 and P2b are deliberately redundant, and both earn their place. P2 compares against an independently written model — it catches any disagreement but does not say which side is wrong. P2b states the ceiling property directly: n×16 covers the payload and (n−1)×16 does not. When P2b fails, the message names the defect; when only P2 fails, you still have to work out which implementation is at fault.
P6 and P7 are the same atomicity in both directions, and both are needed. A design that guards PH-before-PD but not PD-before-PH passes one and leaks on the other.
P12 is a cross-engine non-interference property in the style of Chapter 15.5 §11. It cannot be written inside either engine — neither sees the other's send event — and it is the only mechanism that catches an I/O Write charged to the Posted pools, because both engines are individually correct.
12. Verification and Fault Injection
The scoreboard uses §9's counting model, never §6's division, and maintains its own PH/PD pools from observed initialisations, returns and sends.
Cost boundaries — the required set
| Payload | Expected PD | Why this value |
|---|---|---|
| 0 | 0 | no payload, no data credit |
| 1 | 1 | the smallest non-zero payload |
| 15 | 1 | just under one unit |
| 16 | 1 | exactly one unit |
| 17 | 2 | one byte over — the floor-division killer |
| 31 | 2 | just under two |
| 32 | 2 | exactly two |
| 64 | 4 | a common aligned size |
| 4095 / 4096 | 256 / 256 | the maximum-payload pair |
The 16/17 and 32/33 pairs are the tests that matter. Aligned power-of-two payloads — which is what most directed tests generate — cost the same under a floor and a ceiling, so a test suite made only of those will pass a broken design.
Issue
- PH available, PD insufficient. Verify neither pool moves (P6) — required.
- PD available, PH insufficient. The mirror (P7).
- Both available. Verify both move by exactly the derived costs (P9).
- A Message without data with PD at zero. Verify it issues and PD is untouched (P8).
- Both pools exhausted. Verify the packet waits, stable (P11).
- A PD return arriving while the packet is stalled. Verify it becomes eligible that cycle (Chapter 16.1 §8's contract).
- Continuous maximum-size writes until PD is exhausted, then returns. The sustained-throughput case (§10).
- Replay buffer full with credit available. Verify no launch, no credit moved (P14).
tx_path_readylow with everything else available. Verify no consumption (P10).- A queue full behind a stalled head. Verify the backpressure chain of §8.
- Reset mid-issue, and reset with a stalled packet.
Mutations
| # | Mutation | Caught by | Silicon symptom |
|---|---|---|---|
| 1 | payload_bytes / 16 — floor instead of ceiling | P2, P2b at 17 bytes | receiver overflow on unaligned payloads under load |
| 2 | PH consumed when PD is short | P6 | slow PH leak; writes stop permanently after minutes |
| 3 | PD consumed on a stalled packet | P10 | PD drains with no packets on the wire |
| 4 | Memory Read charged to the Posted pools | P12 | reads drain PH; writes stall for no visible reason |
| 5 | stale payload_bytes used for the cost | A2, P2 | intermittent under- or over-charge; load-dependent |
| 6 | a PD return credited to PH | P13 | PH grows without bound, PD starves |
| 7 | one packet consumes credit twice | P9 | throughput halves; pools drain at 2× the send rate |
| 8 | the queue entry freed before send_fire | P10, and the scoreboard | packets lost locally, credit spent |
| 9 | blocked_on_pd computed from the PH pool | no property — a debug-visibility bug | debugging leads to the wrong pool |
| 10 | payload_bytes + 15 computed at LEN_W | P2 at 4096 | largest packets cost almost nothing; immediate overflow |
| 11 | (bytes + 16)/16 — off-by-one the safe way | P2b at 16 bytes | throughput loss only; never unsafe |
| 12 | zero-length payload charged 1 PD | P3 | PD drains against packets that store nothing |
13. Debugging
Small writes work, large writes stall forever
Almost certainly PD, and the split reason flag says so immediately.
blocked_on_pdasserted? Then the derived cost exceeds available PD.- Compare
pd_costagainst §9's model for that payload. A mismatch is §6's division. - If the cost is right, is PD ever returned? A pool that only falls is a receiver that is not draining, or an update path that is not decoding (Chapter 15.2 §13).
- Is the advertised PD large enough for one maximum payload at all? §10: if the receiver advertised less than MPS/16, the largest legal write can never be issued — the stall is permanent by construction and no amount of waiting helps.
Step 4 is the one that gets missed, and it looks exactly like a credit-return problem until you compare the advertisement against the packet size.
PH drains even though no packet leaves the Link
The consumption boundary is wrong (Chapter 16.1 §10).
Check whether the decrement correlates with pkt_valid or with send_fire. If PH falls while the transmit path is stalled, credit is being spent at scheduler selection rather than at launch.
Or it is the leak (§12's mutation 2): PH decremented before PD eligibility failed. Distinguish them in one waveform — the leak only decrements when PD is short, the boundary bug decrements whenever a packet is offered.
Memory Reads reduce Posted credits
Classification, not accounting (§3).
A read is Non-Posted and must charge NPH. If PH is falling on reads, the normalized packet kind is wrong upstream, or the credit engine is selected by direction rather than by posting semantics.
P12 catches it in simulation. In the lab, the tell is that PH and NPH move together, or that NPH never moves at all.
Only payloads just above a boundary fail
The ceiling division — §12's counterexample, and this symptom is nearly diagnostic on its own.
If 16, 32 and 64 bytes work and 17, 33 and 65 fail, the cost function is flooring. No other bug produces that pattern, because no other bug is sensitive to the payload modulo 16.
14. Common Misconceptions
- "Posted traffic needs no credits." It needs 1 PH + n PD, paid before transmission (§1, §2).
- "Posted means no acknowledgement." No Completion. The Link still ACKs it and retains it for replay (Chapter 14.2, 14.4).
- "A Memory Write only consumes data credit." It consumes a header credit too — every TLP does (§1).
- "Header credit represents bytes." It represents one header, sized for the largest possible (Chapter 16.1 §5).
- "PD cost can be rounded down."
Roundup— the source says so, and flooring overflows the receiver (§4, §12). - "A 16-byte and a 17-byte write cost the same." 1 PD and 2 PD (§4).
- "Credit is consumed when the TLP enters the queue." On the irrevocable send (§7, P10).
- "An ACK restores Posted credit." An ACK returns replay storage; UpdateFC returns credit (P14, Chapter 16.1 §3).
- "Posted credit exhaustion means the transaction failed." Nothing failed. The transmitter is waiting for capacity (§8).
- "All write-like packets use Posted credits." I/O and Configuration Writes are Non-Posted —
1 NPH + 1 NPD(§3). - "A bigger MPS always means better throughput." Bigger packets cost proportionally more PD. With a fixed advertised pool, fewer are in flight (§10).
- "The replay buffer and PH/PD are one resource." Local TX storage versus remote RX capacity (Chapter 16.1 §3).
- "A zero-length write costs one PD." It costs zero — there is nothing to store (P3).
15. Understanding Check
16. What's Next
Posted traffic is one-way and fully accounted. 1 PH always, and Roundup(Length / 16) PD when there is payload — a constant and a ceiling division, with all the danger in the division.
The atomic pair is not stylistic here: a short PD pool leaking PH credits kills a traffic class permanently, and the only reliable defence is a structure with no ordering to exploit.
Chapter 16.3 — Non-Posted Credits takes NPH and NPD, and a packet with the opposite shape: a Memory Read carries no payload and costs 1 NPH alone — yet it is the request most likely to be resource-blocked, because it needs three other things at the same time. Why a requester with free Tags, free replay space and an idle Link still cannot issue a read is the question there.
Chapter 16.4 then closes the loop with CPLH and CPLD.
The idea to carry forward: the header cost is a constant you cannot get wrong; the data cost is a division you can get wrong once, quietly, for years.