PCIe · Module 22
Link Efficiency — Naming the Denominator Before Quoting the Number
Payload over what? The same workload is 94% or 80% efficient depending only on which transport the denominator includes — and replay, control traffic and idle time each belong to a different question.
"PCIe is about 95% efficient" is not a fact. It is a fragment of one.
Chapter 6.7 §6 named protocol overhead as its layer 6 and did not derive it. Chapter 12.5 §10 explained at length why it published no efficiency percentage. Chapter 22.4 §3 counted the packets and said nothing about what one costs.
All three deferred to this chapter, and this chapter's first job is to explain why they were right to be careful.
1. Sources, Scope, and a Deliberate Refusal
2. Payload Over What?
The question that makes the chapter tractable is not "how efficient is PCIe" but "efficient with respect to which resource".
Four denominators, four legitimate questions:
| Denominator | The question it answers | Who asks it |
|---|---|---|
| payload + per-packet fixed cost | is my packet size sensible? | the DMA architect |
| + replayed packet bytes | is my link clean? | the SI / validation engineer |
| + control traffic | what is my protocol spending? | the controller architect |
| + all occupied link time | what fraction of the link did my application get? | the system owner |
They are nested, so each is smaller than the last, and the difference between two adjacent rows is itself the useful number — it isolates one cost.
§13 Model 8 makes this concrete on one workload and finds 94.12% / 90.50% / 89.55% / 80.00%. The 3.62-point step from the first to the second is the replay cost. The 0.95-point step to the third is the control cost. The 9.55-point step to the fourth is everything else, which is mostly idle — and idle is 22.1's subject, not this chapter's (§8).
So the rule this chapter enforces everywhere: state the numerator, state the denominator, state the workload, state the direction. A number missing any of those cannot be compared with another number, which is Chapter 22.6's entire subject.
3. The Efficiency Stack
Each layer wraps the one above it, and each adds something that is not application payload.
| Layer | What it adds | Expressible identically in every generation? |
|---|---|---|
| application payload | — | yes |
| transaction layer | header, and optionally a digest | yes, with the header form named |
| data link layer | sequence and integrity information, DLLPs | yes |
| physical layer | framing, encoding, ordered sets | no — see §9 |
| occupied link time | idle, retraining, everything else | yes, but it is utilisation (§8) |
The fourth row is why this chapter cannot publish one universal formula. Gen1–Gen5 apply a line code that expands every byte; Gen6 does not, and relocates the overhead into the FLIT (6.7 §2). A single "efficiency" number that silently spans that boundary is comparing two different quantities (§9).
4. Header Amortization
5. Four Denominators, One Workload
§13 Model 8's workload, stated completely so the numbers are reproducible: 1,000 accepted packets of 256-byte payload, per-packet fixed cost F = 16 B, 40 packets replayed, 3,000 bytes of control traffic, and 320,000 byte-times of occupied link.
Unique application payload delivered: 256,000 bytes. That number is the numerator in all four rows and it never changes.
| Model | Denominator includes | Efficiency |
|---|---|---|
| A | payload + per-packet fixed cost | 94.12% |
| B | A + bytes spent retransmitting replayed packets | 90.50% |
| C | B + control traffic | 89.55% |
| D | total occupied link byte-times | 80.00% |
The differences are the diagnosis:
A − B = 3.62 points ← the replay tax
B − C = 0.95 points ← the control-traffic tax
C − D = 9.55 points ← idle and everything else (22.1's question)Which is why §11's RTL keeps four independent counters rather than one efficiency register. A single ratio computed in hardware forecloses every one of these questions, and the four counters cost less than the divider would.
6. Replay Adds Transport and No Payload
7. Control Traffic Is Real and Workload-Dependent
Flow Control updates and acknowledgements travel as DLLPs (16.6 §1) and occupy link opportunities that carry no application payload.
But their share is not a constant, and any chapter quoting a fixed DLLP overhead percentage has invented it. §13 Model 10 shows why — the same absolute control traffic against different packet sizes:
| Payload per packet | TLP bytes | Control bytes | Control share |
|---|---|---|---|
| 64 B | 80,000 | 3,000 | 3.61% |
| 256 B | 272,000 | 3,000 | 1.09% |
| 1024 B | 1,040,000 | 3,000 | 0.29% |
| 4096 B | 4,112,000 | 3,000 | 0.07% |
A 50× spread from packet size alone, with the control traffic held fixed. And in reality it is not fixed — update frequency depends on the receiver's policy and its drain rate (16.6 §8).
So the only defensible way to state control overhead is to measure it, which is what §11's control_byte_counter exists for. measured_control_bytes / total_transport_bytes is a number about your system; anything else is a number about someone's memory.
8. Idle Is Not Overhead
9. Gen6 Needs a Different Basis, Not a Different Constant
10. The Waveform
Same numerator, larger denominator — replay and control at work
10 cyclesThree things to read out of the figure.
transport is asserted in cycles 6 and 7 and unique_pay is not. That divergence is the efficiency loss, and it is visible only because the two are counted separately (§11).
Cycle 6 is the replay. A counter that increments unique_pay here reports the second window as being as efficient as the first — blind to exactly what it was installed to see (§6).
And the two snapshots must not be mixed. Cycle 4 closes window A and cycle 9 closes window B; a ratio built from window A's numerator and window B's denominator is meaningless, which is why §11 publishes all counters from one atomic snapshot and P13 asserts it.
11. RTL — Four Counters That Must Not Become One
// SYNTHESIZABLE. Efficiency measurement types.
// Four independent accumulators. The generation basis is a compile-time
// selection because Gen6 is NOT the Gen3-5 model with a different constant
// (§9) -- an unsupported combination must fail to elaborate, not silently
// compute the wrong thing.
package eff_pkg;
parameter int CNT_W = 48;
parameter int BYTES_W = 13;
typedef enum logic [1:0] {
BASIS_LINECODE = 2'd0, // Gen1-Gen5: a line code expands every byte
BASIS_FLIT = 2'd1, // Gen6: no line code; overhead inside the FLIT
BASIS_NONE = 2'd2 // byte-domain only; no physical-layer claim
} basis_e;
function automatic logic [CNT_W-1:0] sat_add(input logic [CNT_W-1:0] v,
input logic [CNT_W-1:0] inc);
logic [CNT_W:0] wide;
wide = {1'b0, v} + {1'b0, inc};
return wide[CNT_W] ? {CNT_W{1'b1}} : wide[CNT_W-1:0];
endfunction
function automatic bit is_sat(input logic [CNT_W-1:0] v);
return (v == {CNT_W{1'b1}});
endfunction
endpackageimport eff_pkg::*;
// SYNTHESIZABLE. UNIQUE application payload -- the NUMERATOR (§6).
// Increments on FIRST successful delivery only. A replay of the same
// packet must not increment it: §13 Model 9 shows a counter that does is
// blind to the exact condition it exists to detect.
module unique_payload_counter (
input logic clk,
input logic rst_n,
input logic measuring,
input logic tlp_accepted, // handshake at the measurement point
input logic is_replay, // this transmission is a retransmit
input logic [BYTES_W-1:0] payload_bytes,
output logic [CNT_W-1:0] unique_payload,
output logic saturated,
output logic replay_seen // sticky: qualifies the whole window
);
logic [CNT_W-1:0] p_q;
logic rs_q;
assign unique_payload = p_q;
assign saturated = is_sat(p_q);
assign replay_seen = rs_q;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin p_q <= '0; rs_q <= 1'b0; end
else if (measuring && tlp_accepted) begin
if (is_replay) rs_q <= 1'b1; // qualify, do not count
else p_q <= sat_add(p_q, CNT_W'(payload_bytes)); // FIRST delivery only
end
end
endmoduleimport eff_pkg::*;
// SYNTHESIZABLE. Transport bytes -- the DENOMINATOR's core term (§5).
// Counts EVERY transmission, including replays. The measurement point is
// the local transmit interface, which sits ABOVE the physical layer: this
// is represented transport, NOT a wire capture (§1, mutation 16).
module transport_byte_counter (
input logic clk,
input logic rst_n,
input logic measuring,
input logic tx_valid,
input logic tx_ready,
input logic [BYTES_W-1:0] tx_bytes, // header + payload for this packet
output logic [CNT_W-1:0] transport_bytes,
output logic saturated
);
logic [CNT_W-1:0] t_q;
assign transport_bytes = t_q;
assign saturated = is_sat(t_q);
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) t_q <= '0;
// The HANDSHAKE, never valid alone -- Chapter 22.1 §7's rule, unchanged.
else if (measuring && tx_valid && tx_ready) t_q <= sat_add(t_q, CNT_W'(tx_bytes));
end
endmoduleimport eff_pkg::*;
// SYNTHESIZABLE. Replay bytes, counted SEPARATELY so the replay tax is
// isolable (§5: A - B is exactly this quantity). Diagnostic only -- the
// functional replay controller is Chapter 14.4's and is untouched here.
module replay_byte_counter (
input logic clk,
input logic rst_n,
input logic measuring,
input logic tx_valid,
input logic tx_ready,
input logic is_replay,
input logic [BYTES_W-1:0] tx_bytes,
output logic [CNT_W-1:0] replay_bytes,
output logic [31:0] replay_packets,
output logic saturated
);
logic [CNT_W-1:0] r_q; logic [31:0] n_q;
assign replay_bytes = r_q;
assign replay_packets = n_q;
assign saturated = is_sat(r_q);
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin r_q <= '0; n_q <= '0; end
else if (measuring && tx_valid && tx_ready && is_replay) begin
r_q <= sat_add(r_q, CNT_W'(tx_bytes));
if (!(&n_q)) n_q <= n_q + 32'd1;
end
end
endmoduleimport eff_pkg::*;
// SYNTHESIZABLE, CONDITIONAL ON INTERFACE VISIBILITY (§1, §7).
// Only instantiate where the controller exposes categorized control
// transmissions. Where it does not, the honest design has NO counter --
// claiming to measure invisible traffic is mutation 16.
module control_byte_counter #(parameter bit HAS_CONTROL_VISIBILITY = 1'b1) (
input logic clk,
input logic rst_n,
input logic measuring,
input logic ctrl_valid,
input logic ctrl_ready,
input logic [BYTES_W-1:0] ctrl_bytes,
output logic [CNT_W-1:0] control_bytes,
output logic control_measured // 0 => the field is UNKNOWN
);
logic [CNT_W-1:0] c_q;
assign control_bytes = HAS_CONTROL_VISIBILITY ? c_q : '0;
assign control_measured = HAS_CONTROL_VISIBILITY;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) c_q <= '0;
else if (HAS_CONTROL_VISIBILITY && measuring && ctrl_valid && ctrl_ready)
c_q <= sat_add(c_q, CNT_W'(ctrl_bytes));
end
endmoduleimport eff_pkg::*;
// SYNTHESIZABLE. Atomic snapshot of the whole window (§5, §10).
// ALL counters are published from ONE cycle. A ratio assembled from two
// different windows is meaningless, and it is the easiest bug to commit
// when counters are read one register at a time.
module efficiency_snapshot #(parameter basis_e BASIS = BASIS_NONE) (
input logic clk,
input logic rst_n,
input logic window_close,
input logic [CNT_W-1:0] unique_payload,
input logic [CNT_W-1:0] transport_bytes,
input logic [CNT_W-1:0] replay_bytes,
input logic [CNT_W-1:0] control_bytes,
input logic [CNT_W-1:0] active_cycles, // 22.1's denominator, kept apart (§8)
input logic control_measured,
input logic replay_seen,
input logic any_saturated,
output logic snap_valid,
output logic [CNT_W-1:0] snap_unique,
output logic [CNT_W-1:0] snap_transport,
output logic [CNT_W-1:0] snap_replay,
output logic [CNT_W-1:0] snap_control,
output logic [CNT_W-1:0] snap_active,
output logic snap_control_valid,
output logic snap_replay_seen,
output logic snap_saturated,
output basis_e snap_basis
);
assign snap_basis = BASIS;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
snap_valid<=1'b0; snap_unique<='0; snap_transport<='0; snap_replay<='0;
snap_control<='0; snap_active<='0; snap_control_valid<=1'b0;
snap_replay_seen<=1'b0; snap_saturated<=1'b0;
end else begin
snap_valid <= window_close;
if (window_close) begin
// ONE cycle, ALL fields. This is the whole point of the block.
snap_unique <= unique_payload;
snap_transport <= transport_bytes;
snap_replay <= replay_bytes;
snap_control <= control_bytes;
snap_active <= active_cycles;
snap_control_valid <= control_measured;
snap_replay_seen <= replay_seen;
snap_saturated <= any_saturated;
end
end
end
endmoduleimport eff_pkg::*;
// VERIFICATION-ONLY. Sanity monitor for the ratios software will form.
// Payload can never exceed the transport that carried it, and replay
// bytes are a SUBSET of transport -- not an addition to it.
module efficiency_sanity_monitor (
input logic [CNT_W-1:0] snap_unique,
input logic [CNT_W-1:0] snap_transport,
input logic [CNT_W-1:0] snap_replay,
input logic snap_saturated,
output logic err_payload_exceeds_transport,
output logic err_replay_exceeds_transport
);
always_comb begin
err_payload_exceeds_transport =
!snap_saturated && (snap_unique > snap_transport);
err_replay_exceeds_transport =
!snap_saturated && (snap_replay > snap_transport);
end
endmoduleClassification: five synthesizable (one gated on declared interface visibility), one verification-only.
Failure — seven. Counting replayed payload as new (blind to replay). Counting transport on valid (inflated denominator under stall). One efficiency register instead of four counters (every question in §2 foreclosed). Mixing two windows in one ratio. Claiming to measure control traffic an interface does not expose. Wrapping counters. And applying the Gen1–Gen5 basis to Gen6 — which the basis_e parameter exists to make an elaboration-time decision rather than a silent one.
12. Same-Cycle Audit and Assertions
// ==================================================================
// NUMERATOR (§6) -- unique payload, once.
// ==================================================================
// P1: unique payload advances only on an accepted, NON-replay TLP.
property p_unique_on_first_delivery;
@(posedge clk) disable iff (!rst_n)
(unique_payload != $past(unique_payload)) |->
$past(measuring && tlp_accepted && !is_replay);
endproperty
// P2: a replay NEVER advances unique payload. §13 Model 9: a counter that
// breaks this reports 94.12% at every replay rate.
property p_replay_adds_no_payload;
@(posedge clk) disable iff (!rst_n)
(measuring && tlp_accepted && is_replay) |=> $stable(unique_payload);
endproperty
// P3: the increment is exactly the payload accepted.
property p_unique_increment_exact;
@(posedge clk) disable iff (!rst_n)
(measuring && tlp_accepted && !is_replay && !saturated) |=>
(unique_payload == $past(unique_payload) + CNT_W'($past(payload_bytes)));
endproperty
// P4: a replay in the window is REPORTED stickily -- the window is
// qualified even if the replay bytes are small.
property p_replay_seen_sticky;
@(posedge clk) disable iff (!rst_n)
replay_seen |=> replay_seen;
endproperty
// ==================================================================
// DENOMINATOR (§5) -- transport, on the handshake.
// ==================================================================
// P5: transport advances only on an accepted transmission.
property p_transport_on_handshake;
@(posedge clk) disable iff (!rst_n)
(transport_bytes != $past(transport_bytes)) |->
$past(measuring && tx_valid && tx_ready);
endproperty
// P6: an offered but unaccepted packet adds nothing (Chapter 22.1 §7).
property p_no_transport_on_stall;
@(posedge clk) disable iff (!rst_n)
(tx_valid && !tx_ready) |=> $stable(transport_bytes);
endproperty
// P7: transport counts replays TOO -- that asymmetry against P2 is the
// entire mechanism by which replay shows up as efficiency loss.
property p_transport_includes_replay;
@(posedge clk) disable iff (!rst_n)
(measuring && tx_valid && tx_ready && is_replay && !saturated) |=>
(transport_bytes != $past(transport_bytes));
endproperty
// P8: replay bytes are a SUBSET of transport, never an addition.
property p_replay_subset_of_transport;
@(posedge clk) disable iff (!rst_n)
(!saturated) |-> (replay_bytes <= transport_bytes);
endproperty
// P9: control bytes accumulate independently of TLP transport.
property p_control_independent;
@(posedge clk) disable iff (!rst_n)
(ctrl_valid && ctrl_ready && measuring && control_measured) |=>
(control_bytes != $past(control_bytes));
endproperty
// ==================================================================
// COUNTER HYGIENE.
// ==================================================================
// P10: counters saturate, never wrap.
property p_no_wrap;
@(posedge clk) disable iff (!rst_n)
(transport_bytes == {CNT_W{1'b1}}) |=> (transport_bytes == {CNT_W{1'b1}});
endproperty
// P11: saturation invalidates the window explicitly, so software discards
// it rather than computing a ratio from a truncated denominator.
property p_saturation_flagged;
@(posedge clk) disable iff (!rst_n)
(snap_valid && $past(any_saturated)) |-> snap_saturated;
endproperty
// P12: payload can never exceed the transport that carried it. This is
// the cheapest possible check that the two counters share a window.
property p_payload_le_transport;
@(posedge clk) disable iff (!rst_n)
(snap_valid && !snap_saturated) |-> (snap_unique <= snap_transport);
endproperty
// ==================================================================
// SNAPSHOT ATOMICITY (§5) -- one window, one set of numbers.
// ==================================================================
// P13: every published field comes from the SAME cycle.
property p_snapshot_atomic;
@(posedge clk) disable iff (!rst_n)
snap_valid |-> ((snap_unique == $past(unique_payload))
&& (snap_transport == $past(transport_bytes))
&& (snap_replay == $past(replay_bytes))
&& (snap_active == $past(active_cycles)));
endproperty
// P14: exactly one snapshot per window close -- no double publication.
property p_one_snapshot_per_close;
@(posedge clk) disable iff (!rst_n)
snap_valid |-> $past(window_close);
endproperty
// P15: a boundary-cycle transfer is inside the CLOSING window (§12).
property p_boundary_transfer_in_closing_window;
@(posedge clk) disable iff (!rst_n)
(window_close && tx_valid && tx_ready) |=>
(snap_transport >= CNT_W'($past(tx_bytes)));
endproperty
// P16: the unmeasured control field is marked UNKNOWN, not zero-as-fact.
// Reporting an unobservable quantity as 0 is mutation 16.
property p_control_validity_declared;
@(posedge clk) disable iff (!rst_n)
(snap_valid && !snap_control_valid) |-> (snap_control == '0);
endproperty
// P17: reset publishes nothing and clears every accumulator.
property p_reset_clears_window;
@(posedge clk)
(!rst_n) |=> (!snap_valid && (unique_payload == '0) && (transport_bytes == '0));
endproperty
// P18: a window with no traffic yields a zero denominator -- defined, and
// the consumer's problem, not a fabricated 100%.
property p_zero_traffic_defined;
@(posedge clk) disable iff (!rst_n)
(snap_valid && (snap_transport == '0)) |-> (snap_unique == '0);
endproperty
// P19: the generation basis travels WITH the snapshot, so a consumer
// cannot apply the Gen1-Gen5 model to a Gen6 window (§9).
property p_basis_accompanies_snapshot;
@(posedge clk) disable iff (!rst_n)
snap_valid |-> (snap_basis inside {BASIS_LINECODE, BASIS_FLIT, BASIS_NONE});
endproperty
// P20: the monitor never drives the transmit interface.
property p_monitor_does_not_drive;
@(posedge clk) disable iff (!rst_n)
$stable({tx_valid, tx_ready}) or !$stable({transport_bytes, unique_payload});
endpropertyTwenty properties. P1, P2 and P7 together are the chapter — the deliberate asymmetry between a numerator that ignores replay and a denominator that does not. P13–P16 exist because a ratio is only as trustworthy as the atomicity of the two numbers in it, and P16 is the one that keeps an honest design from reporting a quantity it cannot see.
13. Measured Behaviour
14. Verification — DV and Mutations
DV, against an independent byte-accounting model — never the DUT's own counters: a clean window with no replay · a window that is entirely replay · a single packet · a zero-payload packet · maximum payload · control-only traffic · a transfer exactly on the window boundary · two consecutive window closes · counter saturation · reset mid-window · HAS_CONTROL_VISIBILITY = 0 · a window with no traffic at all · payload deliberately forced above transport (must be rejected).
| # | Mutation | Symptom | Caught by |
|---|---|---|---|
| 1 | Count replayed payload as new unique payload | reports 94.12% at every replay rate; blind to replay (§13) | P2 |
| 2 | Omit replay bytes from transport | the replay tax vanishes from the denominator | P7, P8 |
| 3 | Count transport on tx_valid alone | denominator inflated under stall; efficiency understated | P5, P6 |
| 4 | Count unique payload twice for one delivery | efficiency above 100% | P3, P12 |
| 5 | Publish one efficiency register instead of four counters | every question in §2 foreclosed | design review |
| 6 | Assemble a ratio from two different windows | numerator and denominator describe different runs | P13 |
| 7 | Publish two snapshots per window close | software double-counts a window | P14 |
| 8 | Credit the boundary-cycle transfer to the next window | every window misreports its edge | P15 |
| 9 | Report unmeasured control bytes as 0 | an unknown presented as a measurement (§7) | P16 |
| 10 | Claim wire-level measurement from an interface above the PHY | a denominator that omits physical-layer transport (§1) | design review |
| 11 | Let counters wrap | a busy window reports a small denominator | P10, P11 |
| 12 | Ignore saturation when publishing | a truncated denominator inflates efficiency | P11 |
| 13 | Allow payload > transport silently | an impossible ratio published as fact | P12 |
| 14 | Change the denominator basis between rows of a table | the trend is an artifact of the basis (§5) | design review |
| 15 | Use one header size for every TLP | 3 DW and 4 DW forms differ (11.3 §2) | design review |
| 16 | Conflate the link-layer CRC with an end-to-end digest | different layers, different presence conditions (§1) | design review |
| 17 | Quote a fixed DLLP overhead percentage | measured spread was 3.61% to 0.07% by packet size (§13) | design review |
| 18 | Apply × 128/130 to Gen6 | "wrong and confidently precise" (6.7 §2) | P19 |
| 19 | Compare a Gen6 post-FLIT figure with a Gen5 post-encoding figure | different bases; not a valid ratio (§9) | P19 |
| 20 | Sum TX and RX into a one-direction efficiency | doubles the numerator against a one-way denominator | design review |
| 21 | Report idle cycles as packet-format overhead | sends the fix to packet sizing on a starved system (§8) | design review |
| 22 | Equate 100% link utilisation with 100% payload efficiency | §13 Model 11 row 3: 100% busy at 80.00% efficient | design review |
| 23 | Divide when the window had no traffic | division by zero, or a fabricated 100% | P18 |
| 24 | Type the efficiency tables by hand | the basis errors above are invisible without a script | design review |
Two counterexamples worth stating explicitly.
Mutation 1 is self-concealing, which makes it the worst kind. A unique_payload counter that increments on every accepted TLP — replay or not — inflates the numerator by exactly the same bytes it inflates the denominator with. The reported efficiency is therefore identical at 0% and 25% replay: 94.12% in both cases (§13 Model 9 computed the true values as 94.12% and 75.29%). The instrument was installed to detect link quality problems and it is precisely blind to them. P2 is the one-line property that separates the two counters' behaviour on a replay.
Mutation 6 needs no bad code at all. Software reads unique_payload, then reads transport_bytes a few microseconds later, after a window boundary has passed. Both registers are correct; the ratio is not, because it mixes one window's numerator with another's denominator. The defect is the absence of an atomic snapshot, and it appears in real systems as an efficiency figure that occasionally exceeds 100% for no reproducible reason. P13 asserts atomicity and P12 catches the symptom.
15. Debugging
Symptom — the link looks busy and the application receives little.
Separate the two questions first (§8). Read active_cycles alongside the byte counters: high activity with a low payload-to-transport ratio is an efficiency problem (small packets, or replay); high ratio with low activity is a utilisation problem and belongs to 22.1 §6. They have no fix in common.
Symptom — efficiency is far below what the packet size predicts.
Read replay_bytes and replay_packets (§11). Compare §4's curve for your payload size against §13 Model 9's replay column: if the shortfall matches the replay rate, this is a link-quality problem and belongs to 14.4, not to packet sizing.
Symptom — a tiny-TLP workload saturates the link and delivers almost nothing. Expected (§4). At F = 16, a 4-byte payload is 20% efficient; four fifths of the link carried no application data. The fix is batching in the producer, and 22.4 §3 shows what the packet count becomes.
Symptom — hardware counters and a benchmark disagree about efficiency. Almost always a denominator mismatch (§5). Ask which of Model A/B/C/D each side computed, and whether both used the same direction. A 14-point disagreement is exactly the A-to-D spread, so a discrepancy of that size is a strong hint that nobody is wrong and the bases differ — the reconciliation procedure is 22.6 §3's.
Symptom — a link analyzer reports more payload than the DMA engine did.
The analyzer is counting replayed bytes as payload (§6) or measuring at a different point (22.1 §3). Reconcile by subtracting replay_bytes from the analyzer's total and checking whether the residual matches; if it does, the instruments agree and only their definitions differed.
Symptom — a Gen6 device looks less efficient than a Gen5 one. Check the bases before believing it (§9). A Gen6 post-FLIT figure has already paid overhead that a Gen5 post-encoding figure has not, so the comparison is invalid as posed (6.7 §2). Recompute both in the byte domain — payload over transport, which is generation-independent — and compare those.
Symptom — the efficiency figure occasionally exceeds 100%. Not a rounding artifact. Either the numerator counted a replay (mutation 1 combined with 2), or the ratio mixed two windows (mutation 6). P12 exists precisely so this is caught in simulation rather than in a report.
16. Misconceptions
"PCIe is about 95% efficient." With respect to which denominator? The same workload measured 94.12% and 80.00% in §13 (§2).
"Efficiency is a property of the generation." It is a property of the workload, the packet size, the link quality and the chosen denominator — the generation sets the capacity (6.7).
"Larger packets are always dramatically better." 64 B → 256 B buys 14 points; 1 KiB → 4 KiB buys 1.15 (§4). Most of the gain arrives early.
"Replayed bytes are throughput." They are transport that delivers no new payload (§6).
"A replay-heavy run just measures a bit lower." At 25% replay, efficiency fell 18.82 points while delivering identical data (§13).
"DLLP overhead is about N%." Measured spread was 3.61% to 0.07% by packet size alone (§7).
"Idle time is protocol overhead." It is utilisation loss, a different question with different fixes (§8).
"100% link utilisation means 100% efficiency." §13 Model 11 row 3: 100% busy at 80.00% efficient (§8).
"One header size is close enough." Header forms are 3 DW or 4 DW (11.3 §2), and the digest is conditional (§1).
"LCRC and ECRC are the same protection." Different layers, different scope, different presence conditions (§1, mutation 16).
"Gen6 just uses a different efficiency constant." Gen6 is 1b/1b at the line with overhead relocated into the FLIT — a different basis, not a different constant (6.7 §2, §9).
"Full duplex doubles efficiency." Efficiency is per direction, like capacity (6.7 §4).
"Hardware should just report an efficiency percentage." That forecloses every question in §2 and hides which cost dominated (§11).
17. Understanding Check
Q1. Two engineers measure the same run and report 94% and 80%. Who is wrong? Probably neither (§5). 94.12% is payload over payload-plus-per-packet-cost; 80.00% is payload over total occupied link time. The 14.12-point difference is itself the finding — it decomposes into the replay tax (3.62), the control tax (0.95) and idle (9.55). The correct response is to name both denominators, not to pick a winner.
Q2. Your efficiency counter reads 94.12% whether the replay rate is 0% or 25%. What is broken?
The numerator counts replayed payload as new (mutation 1). It inflates numerator and denominator identically, so the ratio is invariant to the very condition it should expose. True values are 94.12% and 75.29% (§13 Model 9). P2 is the property, and the fix is that unique_payload advances only on a first delivery.
Q3. Raising the payload from 1 KiB to 4 KiB — how much efficiency do you gain, and is it the best available lever? About 1.15 points at F = 16 (98.46% → 99.61%, §4). Almost certainly not the best lever: the same table shows 64 B → 256 B buying 14 points, and §13 Model 9 shows a 5% replay rate costing 4.48. If replay is non-zero, link quality is worth more than the last doubling of MPS — and 22.4 §3 notes the granularity costs of the larger value.
Q4. Why does §11 keep four counters instead of computing one ratio in hardware? Because the differences between the nested denominators are the diagnosis (§5): A−B is the replay tax, B−C the control tax, C−D idle. One ratio destroys all three and cannot be recovered afterwards. The four counters are also cheaper than the divider they replace.
Q5. A window reports zero transport bytes. What should software display? Not 100%, and not 0%. The denominator is zero, so the ratio is undefined — P18 guarantees the numerator is zero too, so the honest display is "no traffic" (§12). A design that fabricates 100% here reports its best-ever efficiency during a completely idle window.
Q6. Why does this chapter refuse to publish a cross-generation efficiency table? Because Gen6 has no line encoding and relocates its overhead into the FLIT (6.7 §2), so a Gen6 "after overhead" figure and a Gen5 "after encoding" figure are not on the same basis (§9). A table spanning both would change denominator mid-row — mutation 14, which is the exact sin §2 exists to prevent. The generation-independent comparison is bytes of payload over bytes of transport, and §4 publishes that.
18. What's Next
Three chapters have now measured the same system three ways. 22.1 counted the cycles, 22.4 counted the packets, and this chapter priced them. Each insisted on the same discipline: name what you divided by.
Chapter 22.6 closes Module 22 by applying that discipline to other people's numbers. Everything here is a prerequisite for it — a benchmark that does not state its denominator, its direction, its transfer size and its error state is not a weaker measurement than yours; it is an uninterpretable one.
And a specific handoff. §5's four models are exactly the ambiguity that makes two honest benchmark results differ by 14 points with neither being wrong. 22.6 turns that observation into a procedure.