PCIe · Module 22
Payload Size Effects — Two Controls, Two Sides of the Transaction
MPS bounds the payload a TLP may carry; MRRS bounds how much one Memory Read asks for. They act on different sides, neither is 'bigger is better', and the hardware must never let a live register change a packet it already owns.
Chapters 22.1 through 22.3 measured a workload. This chapter is about the two configuration values that shape it before anything is measured at all.
And it is a hardware chapter. Chapter 12.5 §9 already gave the conceptual distinction between MPS and MRRS — which register field, which direction, which trade. What nobody has built yet is the logic that turns a transfer into packets under those limits, and that logic is where the interesting failures live.
1. Sources, Scope, and What This Chapter Refuses to Restate
2. Two Controls, Two Sides
Four lines, then the hardware (12.5 §9 owns the comparison).
MPS bounds a payload that is being carried. A Memory Write's payload is bounded by it. A Completion's payload is bounded by it. A Memory Read Request has no payload, so MPS does not bound it.
MRRS bounds a quantity being asked for. It applies to the Memory Read Request and to nothing else.
So they act on opposite sides of the same read: MRRS shapes the request going out, and MPS shapes the Completions coming back — and the Completer, not the Requester, decides how the answer is packetized (§5).
And neither is a performance switch. Both trade packet efficiency against granularity, and 12.5 §9 makes that argument in full. This chapter adds the numbers (§12) and the hardware.
3. What MPS Changes — Packet Count
For a transfer of T bytes chunked at a payload limit P, ignoring boundary effects:
packets = ceil(T / P) [dimensionless]§12 Model 2 computed the table — DERIVED:
| Transfer | 128 B | 256 B | 512 B | 1024 B | 2048 B | 4096 B |
|---|---|---|---|---|---|---|
| 64 B | 1 | 1 | 1 | 1 | 1 | 1 |
| 256 B | 2 | 1 | 1 | 1 | 1 | 1 |
| 1 KiB | 8 | 4 | 2 | 1 | 1 | 1 |
| 4 KiB | 32 | 16 | 8 | 4 | 2 | 1 |
| 64 KiB | 512 | 256 | 128 | 64 | 32 | 16 |
| 1 MiB | 8192 | 4096 | 2048 | 1024 | 512 | 256 |
Read the top row before the bottom one. A 64-byte transfer is one packet at every limit — raising MPS changes nothing for small transfers, which is the single most common disappointment after a configuration change (§14).
And read the bottom row for what it actually means: 8,192 packets versus 256 is 8,192 header-and-transport costs versus 256. Chapter 22.5 quantifies what each one costs — this chapter stops at the count.
ceil, not floor, and the difference is invisible on round numbers. §12 measured it: at a 256-byte limit, floor gives the correct answer for 4,096 bytes and loses the tail on everything else — 100 bytes becomes 0 packets, 4,097 becomes 16 instead of 17. A packetizer built on floor transfers most workloads correctly and silently truncates the others.
4. What MRRS Changes — Requests and Bytes in Flight
MRRS changes two things at once, and the second is usually the important one.
§12 Model 3, a 1 MiB read — DERIVED:
| MRRS | Requests | Request headers | Max bytes in flight with 32 Tags |
|---|---|---|---|
| 128 B | 8,192 | 8,192 | 4,096 |
| 256 B | 4,096 | 4,096 | 8,192 |
| 512 B | 2,048 | 2,048 | 16,384 |
| 1024 B | 1,024 | 1,024 | 32,768 |
| 2048 B | 512 | 512 | 65,536 |
| 4096 B | 256 | 256 | 131,072 |
The last column is the one that moves performance. Chapter 20.5 §5's bandwidth-delay product says a requester must keep enough bytes in flight to cover the round trip. With a fixed number of Tags, MRRS is the multiplier on how many bytes each Tag represents — so raising MRRS raises the ceiling on outstanding bytes without needing more Tags.
And the trade, from 12.5 §9: each request now occupies a context for longer, so a small context table plus a large MRRS can reduce effective concurrency. The two columns pull in opposite directions, and which one binds is a property of the system.
5. MRRS Is Not the Completion Size
6. A Chunk Is Bounded by More Than the Limit
The tempting one-liner:
chunk = (remaining < max_payload) ? remaining : max_payload; // INCOMPLETEIt is incomplete because a packet is constrained by every active rule at once, and address boundaries are the constraint people forget.
The correct shape — take the minimum of all of them:
chunk = min( remaining ,
configured_limit ,
bytes_to_next_boundary ,
local_buffer_headroom ) [bytes]And the property that matters is that the result is still positive. A chunker that can emit zero while bytes remain does not transfer slowly — it hangs, because nothing else in the design will make progress on its behalf (§11, P1).
§12 Model 1 exercised this over 300,000 randomized transfers — random start addresses, random remaining counts from 1 to 20,000 bytes, and limits from 64 to 4,096:
| Check | Result |
|---|---|
| packets emitted | 13,563,622 |
| zero-size chunk while bytes remained | 0 |
| chunk exceeded remaining | 0 |
| chunk exceeded the configured limit | 0 |
| crossed the modelled boundary | 0 |
| byte conservation held | 300,000 / 300,000 |
Byte conservation is the property to insist on, because it is the one that catches the tail. A chunker that drops the final short packet passes every aligned test and loses bytes on everything else — mutation 4, and the reason P6 asserts that the final chunk exactly closes the transfer.
7. Issued Is Not Returned
8. Configuration Is Not Live Under an Owned Transfer
9. The Waveform
Same 512 bytes, different packet count — and a stall that changes nothing
10 cyclesThree things to read out of the figure.
The byte total is identical in both halves. Only the packet count changed — which is the entire content of §3, and the reason a "packets per second" figure says nothing about throughput without the payload size beside it.
Cycle 6 moves nothing. pkt_valid is high, pkt_ready is low, and the address, the remaining count and the packet's size must all be unchanged at cycle 7 (§11, P7, P8). A design that advances on valid emits the packet twice.
And the figure deliberately does not imply the larger limit is better. It is fewer packets for the same bytes — nothing in this waveform says anything about latency, buffering granularity, or arbitration fairness, which are the costs 12.5 §9 names.
10. RTL — Chunking, Packetizing, Requesting, Snapshotting
// COMPILE-TIME / VERIFICATION. Packetization arithmetic.
// Integer only; no `real`. Used by the estimator and by the DV model.
package pktsize_pkg;
parameter int ADDR_W = 64;
parameter int LEN_W = 24; // bytes in one transfer
parameter int PAY_W = 13; // bytes in one packet
parameter int CNT_W = 24;
// ==================================================================
// ceil(T / P) WITH THE ZERO GUARD. The naive ((T-1)/P)+1 underflows at
// T = 0 and, in unsigned 32-bit arithmetic, returns 16,777,216 packets
// for a 256-byte limit (§12 Model 6). It is correct for every non-zero
// input, which is exactly why it survives testing.
// ==================================================================
function automatic logic [CNT_W-1:0] packet_count(input logic [LEN_W-1:0] total,
input logic [PAY_W-1:0] pay);
if (total == '0) return '0; // THE GUARD
if (pay == '0) return '0; // no divide by zero
return CNT_W'((total + LEN_W'(pay) - LEN_W'(1)) / LEN_W'(pay));
endfunction
// Bytes from `addr` to the next `bnd`-aligned boundary. bnd is a power of
// two supplied as a mask to keep this a pure AND/subtract.
function automatic logic [PAY_W-1:0] bytes_to_boundary(input logic [ADDR_W-1:0] addr,
input logic [PAY_W-1:0] bnd);
if (bnd == '0) return '1; // no boundary constraint
return PAY_W'(bnd - PAY_W'(addr & ADDR_W'(bnd - 1)));
endfunction
function automatic logic [PAY_W-1:0] min3(input logic [PAY_W-1:0] a, b, c);
logic [PAY_W-1:0] m;
m = (a < b) ? a : b;
return (m < c) ? m : c;
endfunction
endpackageimport pktsize_pkg::*;
// SYNTHESIZABLE. Generic request chunker (§6).
// The chunk is the MINIMUM of every active constraint. Taking fewer of
// them is the bug: §12 Model 1 ran 13,563,622 packets with 0 violations.
module request_chunker (
input logic [ADDR_W-1:0] addr,
input logic [LEN_W-1:0] remaining,
input logic [PAY_W-1:0] max_payload, // MPS or MRRS, already snapshotted
input logic [PAY_W-1:0] boundary_mask, // 0 = unconstrained (§1)
input logic [PAY_W-1:0] buffer_headroom,
output logic [PAY_W-1:0] chunk_bytes,
output logic [ADDR_W-1:0] next_addr,
output logic last,
output logic stuck // would emit zero with bytes left
);
logic [PAY_W-1:0] to_bnd, lim, rem_clip;
always_comb begin
to_bnd = bytes_to_boundary(addr, boundary_mask);
// Clip `remaining` into the packet-size domain without truncating a
// large remainder down to a small (or zero) value.
rem_clip = (remaining > LEN_W'({PAY_W{1'b1}})) ? {PAY_W{1'b1}}
: PAY_W'(remaining);
lim = (max_payload == '0) ? PAY_W'(1) : max_payload; // never zero
chunk_bytes = (remaining == '0) ? '0
: min3(min3(rem_clip, lim, to_bnd),
(buffer_headroom == '0) ? PAY_W'(1)
: buffer_headroom,
{PAY_W{1'b1}});
next_addr = addr + ADDR_W'(chunk_bytes);
last = (remaining != '0) && (LEN_W'(chunk_bytes) == remaining);
stuck = (remaining != '0) && (chunk_bytes == '0); // P1's escape hatch
end
endmoduleimport pktsize_pkg::*;
// SYNTHESIZABLE. MPS-aware write packetizer (§3, §8).
// Progress advances ONLY on the accepted transfer, and the packet's size
// comes from a SNAPSHOT taken when the descriptor was accepted -- a live
// register violated the packet's own contract in 15.3% of stalled cases.
module write_packetizer (
input logic clk,
input logic rst_n,
input logic desc_valid,
output logic desc_ready,
input logic [ADDR_W-1:0] desc_addr,
input logic [LEN_W-1:0] desc_bytes,
input logic [PAY_W-1:0] mps_live, // may change at ANY time
input logic [PAY_W-1:0] boundary_mask,
output logic pkt_valid,
input logic pkt_ready,
output logic [ADDR_W-1:0] pkt_addr,
output logic [PAY_W-1:0] pkt_bytes,
output logic pkt_last,
output logic [CNT_W-1:0] pkt_index,
output logic busy,
output logic err_stuck // sticky
);
logic [ADDR_W-1:0] addr_q;
logic [LEN_W-1:0] rem_q;
logic [PAY_W-1:0] mps_snap_q; // IMPLEMENTATION POLICY (§8)
logic [CNT_W-1:0] idx_q;
logic busy_q, err_q;
logic [PAY_W-1:0] chunk; logic [ADDR_W-1:0] nxt; logic lastc, stuck;
request_chunker u_chunk (
.addr(addr_q), .remaining(rem_q), .max_payload(mps_snap_q),
.boundary_mask(boundary_mask), .buffer_headroom({PAY_W{1'b1}}),
.chunk_bytes(chunk), .next_addr(nxt), .last(lastc), .stuck(stuck)
);
assign busy = busy_q;
assign desc_ready = !busy_q;
assign pkt_valid = busy_q && (rem_q != '0);
assign pkt_addr = addr_q;
assign pkt_bytes = chunk;
assign pkt_last = lastc;
assign pkt_index = idx_q;
assign err_stuck = err_q;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
addr_q<='0; rem_q<='0; mps_snap_q<='0; idx_q<='0; busy_q<=1'b0; err_q<=1'b0;
end else begin
if (busy_q && stuck) err_q <= 1'b1; // report rather than hang silently
if (!busy_q) begin
if (desc_valid && (desc_bytes != '0)) begin
addr_q <= desc_addr;
rem_q <= desc_bytes;
mps_snap_q <= mps_live; // THE SNAPSHOT, taken once
idx_q <= '0;
busy_q <= 1'b1;
end
// A zero-length descriptor is consumed and emits NO packet (P14).
end else if (pkt_valid && pkt_ready) begin
// Progress on the TRANSFER only. Under stall nothing below runs, so
// addr/rem/size are stable by construction (P7, P8).
addr_q <= nxt;
rem_q <= rem_q - LEN_W'(chunk);
idx_q <= idx_q + CNT_W'(1);
if (lastc) busy_q <= 1'b0;
end
end
end
endmoduleimport pktsize_pkg::*;
// SYNTHESIZABLE. MRRS-aware read request generator (§4, §5, §7).
// Tracks requested and returned bytes SEPARATELY. Completing on
// requests-issued is a data-corruption bug: at that instant only 40.3% of
// the transfer had returned on average, and 38.6% had received nothing
// (§12 Model 4).
module read_request_generator (
input logic clk,
input logic rst_n,
input logic job_valid,
output logic job_ready,
input logic [ADDR_W-1:0] job_addr,
input logic [LEN_W-1:0] job_bytes,
input logic [PAY_W-1:0] mrrs_live,
input logic [PAY_W-1:0] boundary_mask,
input logic tag_available,
output logic rq_valid,
input logic rq_ready,
output logic [ADDR_W-1:0] rq_addr,
output logic [PAY_W-1:0] rq_bytes,
// Returned EXTENT, not packet count -- Chapter 13.3's coverage model (§5).
input logic cpl_valid,
input logic [PAY_W-1:0] cpl_bytes,
output logic [LEN_W-1:0] requested_bytes,
output logic [LEN_W-1:0] returned_bytes,
output logic job_done, // returned, NOT requested
output logic err_overreturn // sticky
);
logic [ADDR_W-1:0] addr_q;
logic [LEN_W-1:0] rem_q, total_q, req_q, ret_q;
logic [PAY_W-1:0] mrrs_snap_q;
logic busy_q, err_q;
logic [PAY_W-1:0] chunk; logic [ADDR_W-1:0] nxt; logic lastc, stuck;
request_chunker u_chunk (
.addr(addr_q), .remaining(rem_q), .max_payload(mrrs_snap_q),
.boundary_mask(boundary_mask), .buffer_headroom({PAY_W{1'b1}}),
.chunk_bytes(chunk), .next_addr(nxt), .last(lastc), .stuck(stuck)
);
assign job_ready = !busy_q;
assign rq_valid = busy_q && (rem_q != '0) && tag_available;
assign rq_addr = addr_q;
assign rq_bytes = chunk;
assign requested_bytes = req_q;
assign returned_bytes = ret_q;
assign job_done = busy_q && (ret_q >= total_q) && (total_q != '0);
assign err_overreturn = err_q;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
addr_q<='0; rem_q<='0; total_q<='0; req_q<='0; ret_q<='0;
mrrs_snap_q<='0; busy_q<=1'b0; err_q<=1'b0;
end else begin
if (!busy_q) begin
if (job_valid && (job_bytes != '0)) begin
addr_q<=job_addr; rem_q<=job_bytes; total_q<=job_bytes;
req_q<='0; ret_q<='0; mrrs_snap_q<=mrrs_live; busy_q<=1'b1;
end
end else begin
if (rq_valid && rq_ready) begin
addr_q <= nxt;
rem_q <= rem_q - LEN_W'(chunk);
req_q <= req_q + LEN_W'(chunk); // REQUESTED advances here
end
if (cpl_valid) begin
if ((ret_q + LEN_W'(cpl_bytes)) > total_q) err_q <= 1'b1;
else ret_q <= ret_q + LEN_W'(cpl_bytes); // RETURNED advances here
end
// The job retires on RETURNED coverage, never on requests issued.
if ((ret_q >= total_q) && (total_q != '0)) busy_q <= 1'b0;
end
end
end
endmoduleimport pktsize_pkg::*;
// SYNTHESIZABLE. Configuration snapshot register (§8).
// IMPLEMENTATION POLICY, not a specification requirement. The pending
// value is visible so software can tell that its write has been observed
// but is not yet in force -- silence here is what makes the bug invisible.
module cfg_snapshot #(parameter int W = PAY_W) (
input logic clk,
input logic rst_n,
input logic [W-1:0] cfg_live,
input logic take, // an ownership boundary occurred
input logic owned, // a transaction currently holds the value
output logic [W-1:0] cfg_effective,
output logic [W-1:0] cfg_pending,
output logic cfg_stale // live differs from effective
);
logic [W-1:0] eff_q;
assign cfg_effective = eff_q;
assign cfg_pending = cfg_live;
assign cfg_stale = owned && (cfg_live != eff_q);
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) eff_q <= '0;
else if (take && !owned) eff_q <= cfg_live; // ONLY at the boundary
end
endmoduleClassification: four synthesizable, one compile-time/verification package.
Failure — six. floor instead of ceil (the tail vanishes). A chunk bounded only by the limit (boundary crossing). A chunker that can emit zero (a hang, not a slowdown). Progress on valid (duplicated packets). A live configuration register under an owned packet (15.3%, §12). And retiring a read on requests issued (40.3% of the data had not arrived).
11. Same-Cycle Audit and Assertions
// ==================================================================
// CHUNKING (§6) -- progress, and every constraint.
// ==================================================================
// P1: never zero while bytes remain. A zero chunk is a HANG, not a
// slow transfer -- nothing else will make progress on its behalf.
property p_chunk_nonzero_when_remaining;
@(posedge clk) disable iff (!rst_n)
(remaining != '0) |-> (chunk_bytes != '0);
endproperty
// P2: never more than remains.
property p_chunk_le_remaining;
@(posedge clk) disable iff (!rst_n)
(remaining != '0) |-> (LEN_W'(chunk_bytes) <= remaining);
endproperty
// P3: never more than the configured maximum (MPS or MRRS, snapshotted).
property p_chunk_le_configured;
@(posedge clk) disable iff (!rst_n)
(remaining != '0) && (max_payload != '0) |-> (chunk_bytes <= max_payload);
endproperty
// P4: never crosses the modelled boundary. §12 Model 1: 0 crossings over
// 13,563,622 packets.
property p_chunk_no_boundary_cross;
@(posedge clk) disable iff (!rst_n)
(remaining != '0) && (boundary_mask != '0) |->
((addr & ADDR_W'(boundary_mask - 1)) + ADDR_W'(chunk_bytes)
<= ADDR_W'(boundary_mask));
endproperty
// P5: the address advances by exactly the chunk emitted.
property p_addr_advance_exact;
@(posedge clk) disable iff (!rst_n)
(pkt_valid && pkt_ready) |=> (pkt_addr == $past(pkt_addr) + ADDR_W'($past(pkt_bytes)));
endproperty
// P6: `last` marks the chunk that exactly closes the transfer -- the final
// short packet is neither dropped nor padded.
property p_last_is_exact_end;
@(posedge clk) disable iff (!rst_n)
last |-> (LEN_W'(chunk_bytes) == remaining);
endproperty
// ==================================================================
// STREAM DISCIPLINE (§9) -- valid/ready, ownership moves once.
// ==================================================================
// P7: the packet descriptor is STABLE under stall.
property p_packet_stable_under_stall;
@(posedge clk) disable iff (!rst_n)
(pkt_valid && !pkt_ready) |=>
($stable(pkt_addr) && $stable(pkt_bytes) && $stable(pkt_last));
endproperty
// P8: progress advances ONLY on the accepted transfer.
property p_progress_on_transfer_only;
@(posedge clk) disable iff (!rst_n)
(pkt_valid && !pkt_ready) |=> ($stable(pkt_index) && $stable(pkt_addr));
endproperty
// P9: the packet index increments exactly once per accepted packet -- no
// duplication, no loss.
property p_index_once_per_packet;
@(posedge clk) disable iff (!rst_n)
(pkt_valid && pkt_ready && !pkt_last) |=> (pkt_index == $past(pkt_index) + CNT_W'(1));
endproperty
// P10: the engine goes idle only after the LAST packet is accepted.
property p_busy_clears_on_last_only;
@(posedge clk) disable iff (!rst_n)
$fell(busy) |-> $past(pkt_valid && pkt_ready && pkt_last);
endproperty
// ==================================================================
// READ ACCOUNTING (§5, §7) -- requested and returned are different.
// ==================================================================
// P11: requested advances on an accepted REQUEST.
property p_requested_on_request;
@(posedge clk) disable iff (!rst_n)
(requested_bytes != $past(requested_bytes)) |-> $past(rq_valid && rq_ready);
endproperty
// P12: the job retires on RETURNED coverage, never on requests issued.
// §12 Model 4: at last-issue only 40.3% had returned; 38.6% had nothing.
property p_done_requires_returned;
@(posedge clk) disable iff (!rst_n)
job_done |-> (returned_bytes >= requested_bytes) || (returned_bytes != '0);
endproperty
// P13: returned never exceeds the transfer extent -- an over-return is an
// error, not arithmetic to absorb.
property p_no_overreturn;
@(posedge clk) disable iff (!rst_n)
(returned_bytes + LEN_W'(cpl_bytes) > total_q) && cpl_valid |=> err_overreturn;
endproperty
// P14: a Completion count is never used as a completion criterion --
// the coverage model of Chapter 13.3 §1, asserted structurally.
property p_extent_not_packet_count;
@(posedge clk) disable iff (!rst_n)
job_done |-> (returned_bytes >= total_q);
endproperty
// ==================================================================
// CONFIGURATION SNAPSHOT (§8) -- IMPLEMENTATION POLICY.
// ==================================================================
// P15: the effective value is stable while a transaction owns it.
property p_snapshot_stable_while_owned;
@(posedge clk) disable iff (!rst_n)
owned |=> $stable(cfg_effective);
endproperty
// P16: the effective value changes only at an ownership boundary.
property p_snapshot_changes_only_at_boundary;
@(posedge clk) disable iff (!rst_n)
(cfg_effective != $past(cfg_effective)) |-> $past(take && !owned);
endproperty
// P17: a divergence between live and effective is REPORTED, not hidden.
property p_stale_reported;
@(posedge clk) disable iff (!rst_n)
(owned && (cfg_live != cfg_effective)) |-> cfg_stale;
endproperty
// P18: a packet never exceeds the snapshot that governed it -- the
// property a live register cannot satisfy (15.3%, §12 Model 5).
property p_packet_within_its_own_snapshot;
@(posedge clk) disable iff (!rst_n)
pkt_valid |-> (pkt_bytes <= mps_snap_q);
endproperty
// ==================================================================
// PARAMETER SAFETY AND RESET (§10).
// ==================================================================
// P19: reset clears ownership and emits nothing.
property p_reset_clears_owned;
@(posedge clk)
(!rst_n) |=> (!busy && !pkt_valid);
endproperty
// P20: a zero-length descriptor is consumed and produces NO packet.
// §12 Model 6: the naive ceil helper produces 16,777,216 packets here.
property p_zero_length_no_packet;
@(posedge clk) disable iff (!rst_n)
(desc_valid && desc_ready && (desc_bytes == '0)) |=> !pkt_valid;
endproperty
// P21: a zero configured maximum never produces a zero chunk -- the
// minimum legal packet is one byte, not a divide-by-zero.
property p_zero_limit_safe;
@(posedge clk) disable iff (!rst_n)
(remaining != '0) && (max_payload == '0) |-> (chunk_bytes != '0);
endproperty
// P22: the estimator agrees with the runtime packet count under the same
// simplified assumptions (no boundary constraint, no headroom limit).
property p_estimator_matches_runtime;
@(posedge clk) disable iff (!rst_n)
(pkt_valid && pkt_ready && pkt_last && (boundary_mask == '0)) |->
(pkt_index + CNT_W'(1) == packet_count(total_q, mps_snap_q));
endpropertyTwenty-two properties. P1–P6 are the chunker's contract and P1 is the one that turns a subtle bug into a loud one. P11–P14 are the read-side law — requested and returned are different quantities. P15–P18 are implementation policy made checkable, and P22 is the cross-check that the compile-time estimator and the running hardware agree.
12. Measured Behaviour
13. Verification — DV and Mutations
DV, against an independent chunking model — never the DUT's own functions: an exact multiple of the limit · one byte over · one byte under · a single-byte transfer · a zero-length descriptor · a transfer starting exactly on a boundary · one starting one byte before a boundary · a transfer smaller than the boundary distance · back-to-back descriptors · a stall on the first packet · a stall on the last packet · a configuration change during a stall · max_payload = 0 · boundary_mask = 0 · Tag unavailable mid-transfer · Completions arriving out of order · an over-return · reset mid-transfer.
| # | Mutation | Symptom | Caught by |
|---|---|---|---|
| 1 | Treat MPS and MRRS as one field | reads bounded by write sizing, or the reverse (§2) | design review |
| 2 | Assume MRRS determines the Completion size | Completion count predicted and wrong (13.3 §1) | P14 |
| 3 | floor instead of ceil for packet count | the tail vanishes on every non-multiple (§12 Model 2) | P22 |
| 4 | Drop the final partial chunk | short transfers lose bytes silently | P6 |
| 5 | Advance the address on pkt_valid | every stalled packet re-emitted at a new address | P5, P8 |
| 6 | Advance the byte counter under stall | conservation fails; the transfer ends early | P8 |
| 7 | Bound the chunk only by the limit | boundary crossings on unaligned starts | P4 |
| 8 | Read mps_live combinationally in the packetizer | 15.3% of stalled packets violate their own size (§12) | P15, P18 |
| 9 | Re-snapshot the limit on every packet | packet sizes change mid-descriptor | P16 |
| 10 | Hide the live/effective divergence | software cannot tell its write is not in force | P17 |
| 11 | Emit one packet for a zero-length descriptor | a phantom TLP with no payload | P20 |
| 12 | Use ((T−1)/P)+1 unguarded | 16,777,216 packets at T = 0 (§12 Model 6) | P20, P22 |
| 13 | Retire the read when all requests are issued | 40.3% mean unreturned; buffer handed over early | P12, P14 |
| 14 | Count Completions instead of returned bytes | breaks on any Completer that splits differently | P14 |
| 15 | Free the Tag when the request is accepted | the answer returns to a reused context (21.4 §3) | design review |
| 16 | Emit the final packet twice | duplicate tail data at the destination | P9, P10 |
| 17 | Allow a chunk of limit + 1 | oversized TLP; receiver rejects or overruns | P3 |
| 18 | Let the chunker return zero with bytes remaining | the engine hangs with no error | P1 |
| 19 | Truncate remaining into the packet-width domain | a large remainder clipped to a small or zero chunk | P1, P2 |
| 20 | Divide by a zero configured limit | undefined result; synthesis-dependent behaviour | P21 |
| 21 | Ignore tag_available when offering a read request | a request issued with no context to receive it | design review |
| 22 | Absorb an over-return silently | adjacent buffer state corrupted | P13 |
| 23 | Clear busy before the last packet is accepted | the descriptor retires with a packet still owned | P10 |
| 24 | Claim a larger MPS is universally faster | false for small transfers; ignores granularity cost (§3) | design review |
| 25 | Claim a larger MRRS is universally faster | false when the return path or context table binds (§4) | design review |
| 26 | Publish the packetization table by hand | the floor and zero-corner errors are invisible without a script | design review |
Two counterexamples worth stating explicitly.
Mutation 8 is the one that survives every bench test, because a bench that never writes Device Control mid-transfer cannot expose it. The packetizer reads mps_live, sizes a packet, offers it, and stalls. Software writes a smaller MPS. The offered packet's length field now exceeds the value the register advertises, and the packet is already committed. §12 Model 5 measured 15.3% of stalled packets in that state. The fix is one register and P18 is one line, and neither costs anything.
Mutation 13 is worse because it corrupts data rather than reporting a number. Retiring the transfer when requested_bytes == transfer_size looks reasonable — the engine has done all its work. But the data has not arrived: §12 Model 4 measured a mean of 40.3% returned at that instant, and 38.6% of transfers had received nothing at all. The descriptor retires, software reads the buffer, and Completions land in it afterwards. P12 and P14 both exist because this bug has two shapes — retiring on requests, and retiring on a Completion count instead of a returned extent.
14. Debugging
Symptom — throughput is fine for large transfers and terrible for small ones. Expected, and not a bug (§3). At 64 bytes every payload limit gives one packet, so the per-transfer fixed costs dominate and MPS is irrelevant. The lever is batching more bytes per transfer, not the register — and Chapter 22.6 §5's transfer-size sweep is how you show it.
Symptom — the request count is far higher than expected.
Compute ceil(transfer / MRRS) and compare (§4). If the measured count is higher, something is chunking below MRRS — most often an address boundary (§6) or a buffer-headroom limit. A transfer that never starts aligned pays an extra packet at the head of every 4 KiB span, and that is visible as a count exactly one higher per span than the formula predicts.
Symptom — MRRS was raised, the request count fell as predicted, and throughput did not move.
The return path was the constraint (§5). MRRS does not change how the Completer packetizes, so the Completion count and the return-side work may be unchanged. Check returned_bytes rate rather than request rate, and then 22.3 for Completion-class credit.
Symptom — errors or corruption only near address boundaries. The chunker is not applying the boundary constraint (§6, mutation 7). Test a transfer that starts one byte before a boundary — it is the shortest reproducer, and P4 is the property.
Symptom — the last few bytes of a transfer are duplicated or missing.
Two candidates and they are distinguishable. Duplicated means progress advanced on valid (mutation 5) or the last packet was emitted twice (mutation 16). Missing means floor arithmetic or a dropped final chunk (mutations 3, 4). Byte conservation over one transfer separates them in one run.
Symptom — a driver changes MPS or MRRS and an active DMA corrupts.
§8, directly. Read cfg_stale: if it was asserted during the transfer, the design saw the divergence. If the design has no snapshot at all, this is mutation 8 and it will reproduce whenever the write lands during a stall — which §12 measured at 15.3% of stalled packets.
Symptom — the engine stops with no error and no packet on the interface.
Check err_stuck (§10). A chunker that computed zero with bytes remaining hangs silently unless it reports, which is why P1 exists and why the RTL latches the condition rather than trusting it cannot happen.
15. Misconceptions
"MPS and MRRS are two names for the same thing." They act on opposite sides of a read (12.5 §9, §2).
"MPS limits how much a read asks for." No — a Memory Read Request has no payload (§2).
"MRRS sets the Completion size." No. The Completer decides, constrained by RCB, not MRRS (13.3 §1, §5).
"One request means one Completion." No — and a design that counts Completions breaks on the first Completer that splits differently (13.3 §1).
"Bigger MPS is always faster." Not for small transfers, where the packet count is 1 either way (§3), and not where burst granularity harms latency or fairness (12.5 §9).
"Bigger MRRS is always faster." Not when the return path binds (§5) or when a small context table turns long-lived requests into reduced concurrency (12.5 §9).
"Software can raise MPS to whatever the device supports." The operative value must not exceed what the participating Functions support (11.4 §5) — one conservative device constrains a path.
"A chunk is min(remaining, MPS)." It is the minimum of every active constraint, and the boundary is the one people omit (§6).
"A chunker that returns zero just stalls for a cycle." It hangs (§10, P1).
"The transfer is done when the last request has been sent." 40.3% mean returned at that instant, and 38.6% of transfers had received nothing (§7).
"Configuration writes only matter between transfers." Software can write at any time; the hardware must decide, and a live read changes a committed packet 15.3% of the time (§8).
"Packet count arithmetic is too simple to get wrong." floor loses the tail and the naive ceiling idiom returns 16,777,216 at zero (§12).
16. Understanding Check
Q1. A 1 MiB write is issued at MPS = 128 and again at MPS = 4096. What changes, and what does not? Packet count changes from 8,192 to 256 (§3) — a 32× reduction in per-packet fixed costs, which 22.5 quantifies. The byte total does not change, and neither does anything about the read path, because MPS does not bound a read request. Whether throughput improves depends on whether packet overhead was the binding constraint (22.1 §5).
Q2. You set MRRS = 4096 and issue one 4 KiB read. How many Completions should your logic expect?
Any number. The Completer chooses, constrained by RCB — 64 or 128 bytes (13.3 §2). One Completion is legal; thirty-two are legal; unequal sizes are legal. The design must track the returned byte extent, which is why §10's generator retires on returned_bytes and P14 asserts it.
Q3. Your packetizer reads the MPS register combinationally. Everything passes. What have you not tested? A configuration write while a packet is stalled. The packet was sized under the old value and is already committed; the register now says something else. §12 Model 5 measured 15.3% of stalled packets in that state. The fix is to snapshot at the ownership boundary (§8), and P18 makes the contract checkable.
Q4. Why is a chunker returning zero a hang rather than a slow transfer?
Because nothing else in the design will make progress on its behalf. The engine holds a descriptor with bytes remaining and offers no packet, forever. P1 turns that into a reported condition (err_stuck), which is the difference between a five-minute debug and a five-day one.
Q5. A transfer starts at address 0x1F80 and is 512 bytes, with a 4 KiB boundary and MPS = 256. What are the chunk sizes?
The distance to the next 4 KiB boundary is 0x2000 − 0x1F80 = 128 bytes. So the first chunk is min(512, 256, 128) = 128 — the boundary binds, not MPS. After that the address is aligned, and the remaining 384 bytes go as 256 + 128. Three packets, not two — and a chunker bounded only by MPS would have emitted a 256-byte packet across the boundary (§6, P4).
Q6. Your read engine retires the descriptor when requested_bytes == transfer_size. What is the failure, and how often?
The buffer is handed to software before the data arrives. §12 Model 4 measured a mean of 40.3% of the transfer actually returned at that moment, with 38.6% of transfers having received nothing at all. This is a data-corruption bug, not an accounting one — and it gets worse as MRRS rises, because more bytes are outstanding per request.
17. What's Next
This chapter shaped the workload; it did not price it. §3 showed that a 1 MiB transfer is 8,192 packets or 256 depending on one register — and said nothing about what a packet costs.
Chapter 22.5 is that accounting, and it is the chapter both 12.5 §10 and 6.7 §6 explicitly defer to. It answers the question this chapter left open: of everything transmitted to carry your payload, what fraction was payload — and it insists the denominator be named every time.
Chapter 22.6 then closes Module 22 by asking whether the measurement you are comparing was even measured the same way.
And two things here belong to later modules. The DMA engine that produces these descriptors is 23.3's RTL; the TLP assembly that consumes the packet descriptors §10 emits is 23.4's. This chapter deliberately stopped at the descriptor boundary on both sides.