CXL Transport on UCIe

Chapter 11.2 and Chapter 11.3 built semantics: obligations about what an agent may observe, what it owns, and what it owes. Both chapters assumed those semantics arrive at the far die intact, exactly once, in an order the protocol permits.

This chapter is that assumption, examined — and it is a larger job than the equivalent chapter for PCIe was, for a reason worth stating immediately.

CXL does not arrive at UCIe as a bare protocol. It arrives with a multiplexer, a link layer, a flit definition, a credit scheme, and a retry mechanism of its own. Every one of those has a UCIe counterpart. So the mapping problem is not "how do I frame this" but "which of two mechanisms owns each job, and what happens at the seam" — and getting that wrong produces a coherence action delivered twice, which Chapter 11.3 §25 showed corrupts distributed state rather than one location.

1. The One-Sentence Model

Transport may retransmit a CXL flit as often as reliability requires; the memory or coherence semantics above it must observe the transaction exactly once, in an order the protocol permits, with its meaning unchanged.

Chapter 10.2 §1 said almost the same sentence about PCIe. The words are similar and the stakes are not: a duplicated PCIe memory write corrupts one address, while a duplicated ownership grant leaves two agents believing they may write — and there is no address to inspect afterwards, only a distributed state that no longer agrees with itself.

2. Sourcing, and Where the Line Is

3. Three Reasons This Is Not the PCIe Mapping Renamed

Chapter 10.2 concluded that PCIe's data-link functions are largely replaced by the Adapter's own mechanisms — that asking which flit a flow-control DLLP travels in is asking about a substituted mechanism. It is tempting to carry that conclusion across. Do not, and here is why in three structural differences.

First: CXL is three protocols, not one. Chapter 11.1 §4 established that CXL.io, CXL.cache and CXL.mem are dynamically multiplexed, and the UCIe Consortium's own description maps them to distinct usage models — discovery and DMA to CXL.io, memory to CXL.mem, caching to CXL.cache. So the mapping layer faces several concurrent traffic classes with different semantics and different ordering obligations, which is why §22 and §24 exist and have no counterpart in Module 10.

Second: CXL already has a multiplexer. The CXL specification's ARB/MUX arbitrates between the CXL link layers, multiplexes to the physical layer, and demultiplexes on receive by decoding the flit. Meanwhile UCIe's Adapter, per the UCIe Consortium, defines the underlying arbitration mechanism when multiple protocols are supported. Two arbitration layers meet here, and §6 is about what that means for an integration.

Third — and this is the important one: CXL.cache/CXL.mem has its own link layer with its own CRC and its own retry. PCIe's data-link functions could be described as replaced because UCIe's Adapter provides equivalents. Here the situation is materially different, because the Adapter's reliability is optional and the CXL link layer's is integral to the flit definition — 2 bytes of CRC inside a 528-bit flit is not a separable wrapper. §14 is the section that exists only because of this.

PCIe over UCIe was a substitution problem. CXL over UCIe is a division-of-ownership problem, and the two have different failure modes.

4. Layer Ownership

The two middle layers are the chapter. Each is a real, specified layer with real mechanisms, and the interesting questions are all about their interface: which one frames, which one protects, which one arbitrates, and which one is allowed to know what a message means.

The answer to that last one is fixed and non-negotiable, and it is Chapter 10.1 §7's rule with higher stakes: nothing below the mapping boundary may interpret CXL semantics. §10 is what happens when it does.

5. The Protocol Classes Are Not One Traffic Stream

// Illustrative internal classification. NOT a UCIe or CXL encoding, and NOT a
// Protocol ID value — no such value is asserted anywhere in this chapter.
// The enum has three members because the MAPPING LAYER's own decisions need
// three, and no more.
typedef enum logic [1:0] {
  CLS_IO    = 2'd0,   // discovery, configuration, DMA — the PCIe-shaped path
  CLS_MEM   = 2'd1,   // host access to device-attached memory (Ch 11.2)
  CLS_CACHE = 2'd2    // device caching of host memory (Ch 11.3)
} cxl_class_t;

Architecture. The class is the coarsest fact about a transport object that the mapping layer legitimately needs, and it needs it for four separate reasons: to select the right queue (§16), to apply the right ordering rule (§22), to arbitrate (§24), and to route to the right engine on reconstruction (§19).

Note what the class is not for. It is not a licence to inspect the object's contents. The mapping layer knows which family an object belongs to; it does not know whether the object is a read, a snoop, or a grant, and §10 is why that boundary is worth defending.

State. A class field alongside each object — per-object lifetime, assigned at acceptance and never changed. A class that can change mid-object is a malformed object, and §19's reconstruction must detect it rather than absorb it.

Cycle behaviour. Assigned once, read many times.

Contract. Everything downstream keys queueing, ordering and delivery on it.

Failure. Misclassification delivers a memory object to the coherence engine or vice versa. The first produces an engine receiving something meaningless; the second silently drops a coherence action, which Chapter 11.3 §26 showed leaves distributed state inconsistent with no error reported.

Why the enum is deliberately coarse. Inventing a finer-grained CXL message-type encoding here would be exactly the fabrication §2 forbids. The real classification comes from the interface definition of the revision you implement.

6. Two Multiplexers, One Link

The structural fact that surprises people, and it needs stating plainly.

CXL's ARB/MUX exists. The specification describes it as arbitrating between requests from the CXL link layers, multiplexing data to the physical layer, decoding received flits to determine which link layer to forward them to, maintaining virtual link state machines per link-layer interface, and generating link management packets to coordinate power-state transitions across the link on each link layer's behalf.

UCIe's Adapter also arbitrates. The UCIe Consortium states that when multiple protocols are supported, the Adapter defines the underlying arbitration mechanism.

So an integration has to answer, for its revision: where does the multiplexing of CXL.io against CXL.cache/CXL.mem actually happen, and is it happening twice?

The design consequence, independent of the answer. Whatever arbitrates must be told, per class, whether that class currently has anything eligible to send — and eligibility is not the same as "has data queued", because a queued object may be blocked on ordering (§22) or on credit. §24's arbiter takes eligibility as its input for exactly this reason.

7. The Path and Its Buffers

Read the figure as five lifetimes that do not coincide, which is the same lesson as Chapter 10.2 §4 with two extra stages:

Every bug in this chapter is a case where two of those five were assumed to share a lifetime. §27 is the table that keeps them apart.

8. The Transport Object

// ILLUSTRATIVE internal transport object — NOT a CXL flit layout and NOT a
// UCIe flit layout. No bit position, header field, CRC position, protocol-ID
// width or payload arrangement is asserted. Symbolic widths throughout.
localparam int CXL_META_W = 128;                   // symbolic
localparam int CXL_DATA_W = 512;                   // symbolic
localparam int CXL_BYTES  = CXL_DATA_W / 8;
 
typedef struct packed {
  cxl_class_t             cls;          // §5 — assigned once, never changed
  logic [CXL_META_W-1:0]  meta;         // opaque below the mapping boundary
  logic [CXL_DATA_W-1:0]  data;
  logic [CXL_BYTES-1:0]   byte_valid;   // see the note below
  logic [OBJ_LEN_W-1:0]   extent;       // declared length, for §19
  logic [MON_ID_W-1:0]    mon_id;       // VERIFICATION ONLY — §21
} cxl_obj_t;

Architecture. Class, descriptor, payload, extent — consumed together, therefore travelling together, in one object advanced by one enable. Chapter 10.2 §7's rule.

On byte_valid, because CXL makes this concrete. The CXL link layer's own treatment is instructive and directly relevant: when all bytes are enabled it does not transmit the byte-enable bits, clearing a header field instead, and the receiver must regenerate the all-ones value. So a byte-enable mask is not always carried explicitly, and a mapping layer that assumes it is — or that carries a cleared field through and hands all-zeros to the far side — turns a full-line write into a write that touches nothing, or the reverse. A design that accepts byte_valid and then quietly ignores it produces a full-line write where a partial write was intended, and nothing about the object is malformed.

On mon_id, explicitly. It is a verification-only tag that lets a scoreboard follow one semantic object end to end. It is not a CXL identity, not a transport sequence number, and not present in silicon. Chapter 11.2 §16 listed the real CXL identities — the CXL.mem request tag pre-allocated for the transaction's duration, and the CXL.cache queue identifiers — and the reason a verification tag is needed in addition is that those are reused, so they cannot distinguish "this object again" from "a different object with the same identity".

On the metadata/payload rule, and why it is worse here than in Module 10. Chapter 10.2 §7 showed that pairing descriptor N with payload N−1 manufactures a well-formed PCIe transaction carrying the wrong data. In the coherence case the pairing error is worse still: a snoop response descriptor paired with the previous response's data supplies the wrong line's data in answer to a probe, which is Chapter 11.3 §20's silent stale-read failure produced by a pipeline-depth bug rather than by a coherence bug — and the coherence engine will be blamed for it.

// Illustrative — one object, one pipeline, one enable. Misalignment is not
// prevented by review; it is made unrepresentable.
cxl_obj_t obj_q, obj_q2;
 
always_ff @(posedge clk or negedge rst_n) begin
  if (!rst_n) begin
    obj_q  <= '0;
    obj_q2 <= '0;
  end else if (pipe_en) begin
    obj_q  <= obj_in;
    obj_q2 <= obj_q;
  end
end

// Illustrative — the whole bundle holds still under backpressure, including
// the class field. A class that shifted relative to its payload would route a
// coherence object to the memory engine.
property p_object_stable_under_stall;
  @(posedge clk) disable iff (!rst_n)
    (obj_valid && !obj_ready)
      |=> ($stable(obj_q) && $stable(obj_valid));
endproperty
a_object_stable_under_stall: assert property (p_object_stable_under_stall);

9. The Adapter Must Not Understand Coherence

The layering rule, and the violation is more tempting here than in Module 10 because the information is genuinely useful.

// WRONG — transport behaviour derived from coherence semantics.
if (cxl_line_state == C_MODIFIED)
  adapter_retry_priority = 2'b11;      // "dirty data is more urgent"

Why someone writes this. It is not stupid. A flit carrying the only current copy of a modified line genuinely is more urgent than one carrying a speculative read, and prioritising it would improve tail latency. The optimisation is real.

Why it is still wrong, in four escalating steps.

It requires the Adapter to parse CXL. To know a line's state, the Adapter must decode fields whose layout belongs to the CXL revision in force. It now breaks when that revision changes — and Chapter 11.3 §5 showed the coherence model itself was replaced between CXL revisions, so this is not a hypothetical.

It makes a protocol-agnostic layer protocol-specific. The UCIe Consortium describes the Adapter's jobs as link state management, parameter negotiation, arbitration, and optional reliable delivery. None of those needs semantic content. A design that adds semantic dependence has given up the property that lets the same Adapter carry PCIe (Module 10), raw streaming (Module 9) and CXL.

It creates a correctness coupling out of a performance idea. Once retry behaviour depends on coherence state, a coherence bug becomes a transport bug and vice versa, and §29's three error domains collapse into one — which destroys the diagnostic separation that makes any of this debuggable.

And it does not even work reliably, because the Adapter sees a transport object. On a retry it may see the same object again, whose state information is now stale, and prioritise on a fact that has since changed.

The right shape. If the transport genuinely needs a priority hint, the mapping layer — which is allowed to know the class and is allowed to be told a hint by the semantic layer — attaches an opaque priority field that the Adapter compares without interpreting:

// Illustrative — an opaque hint the transport orders by but never interprets.
// The mapping layer sets it; the Adapter compares it. No semantics cross.
logic [PRIO_W-1:0] obj_priority;    // meaning owned entirely above

One number, no protocol knowledge, and the layering survives. That is the general pattern for every case where a lower layer appears to need an upper layer's information: pass a hint, not a meaning.

10. Flit Format Is Configuration State With a Lifetime

CXL has more than one flit format, and which one is in force is a fact the whole path depends on.

What is verified. CXL 1.1 and 2.0 support a 68-byte flit up to 32 GT/s; CXL 3.0 adds a 256-byte flit up to 64 GT/s, in a standard and a latency-optimised variant, and the Consortium places enhanced coherency and memory sharing on the 256-byte format. Separately, the CXL.cache/CXL.mem link layer's flit is specified as a fixed 528 bits with 2 bytes of CRC and four 16-byte slots. On the UCIe side, the Consortium states that a 256-byte flit defines the underlying transfer mechanism when the Adapter is responsible for reliable transfer, and Chapter 10.2 §2 recorded the protocol-and-format pairings that Consortium and vendor-technical material describe for the UCIe stack.

What follows architecturally, independent of the encodings. The active format determines the transfer quantum, the buffer sizing, the packing rules, and how the far side interprets what arrives. It is therefore configuration state with an epoch, and it must not change while objects framed under the old one are in flight.

// ILLUSTRATIVE format-mode state. The enum members are DELIBERATELY not named
// after flit sizes: which formats a given UCIe revision permits for CXL is not
// asserted here (§2). What is modelled is the LIFETIME discipline, which is
// the same whatever the formats turn out to be.
typedef enum logic [1:0] {
  CXL_FMT_A = 2'd0,
  CXL_FMT_B = 2'd1
} cxl_fmt_t;
 
cxl_fmt_t fmt_requested_q;   // what negotiation or software has asked for
cxl_fmt_t fmt_active_q;      // what TX frames with and RX decodes with
logic     fmt_change_armed_q;
 
// A format change is COMMITTED only at a quiescent point. This is the same
// requested/committed discipline as Chapter 5.2 §8's parameter negotiation and
// Chapter 8's training configuration — and for the same reason.
wire fmt_quiescent = (map_queue_occupancy == '0)
                  && (replay_occupancy     == '0)
                  && (recon_open_objects   == '0);
 
always_ff @(posedge clk or negedge rst_n) begin
  if (!rst_n) begin
    fmt_active_q       <= CXL_FMT_A;
    fmt_requested_q    <= CXL_FMT_A;
    fmt_change_armed_q <= 1'b0;
  end else begin
    if (fmt_request_valid) begin
      fmt_requested_q    <= fmt_request_value;
      fmt_change_armed_q <= (fmt_request_value != fmt_active_q);
    end
    // Commit only when nothing is in flight anywhere in the path.
    if (fmt_change_armed_q && fmt_quiescent) begin
      fmt_active_q       <= fmt_requested_q;
      fmt_change_armed_q <= 1'b0;
    end
  end
end
 
// New objects are refused while a change is armed, so the path can drain.
assign fmt_allows_accept = !fmt_change_armed_q;

Classification: synthesizable, illustrative.

Architecture. Two registers instead of one, and a commit condition. The requested value can change at any time; the active value changes only at a point where changing it cannot be observed by anything in flight.

State. Per-format-epoch lifetime — a lifetime that spans many objects and many transactions, and which both dies must agree on. That last clause is what makes this harder than a local configuration register: a format epoch is a distributed agreement, like Chapter 11.1 §25's coherent epoch.

Cycle behaviour. Arming is immediate; committing waits for quiescence; acceptance is blocked while armed, which is what causes quiescence to arrive.

Contract. TX frames with fmt_active_q; RX decodes with its own copy; the two must be the same epoch. Nothing in the path may be framed under one and interpreted under the other.

Failure. §11.

Deliberately simplified: the quiescence condition is local to one side. A real change requires a coordinated sequence with the far side, because the far side's decode must switch at a point consistent with the transmitter's — and that coordination is a protocol matter this chapter does not invent.

11. Wrong RTL — Changing Format Under Live Traffic

// WRONG — the active format follows the request immediately.
always_ff @(posedge clk) begin
  if (fmt_request_valid) fmt_active_q <= fmt_request_value;
end

Architecture. One register, no epoch, no quiescence. It is the smallest possible implementation and it reads as obviously sufficient.

Cycle behaviour. The change takes effect on the next cycle, part-way through whatever the path is doing.

Failure, and it is deterministic corruption rather than a race. Objects already framed under the old format are in the queues, in the replay store, and on the wire. The transmitter now frames new ones differently. The receiver switches its decode at some other point, determined by when its own request arrived.

The result is that a boundary exists in the stream where framing and interpretation disagree, and every object crossing it is misinterpreted:

• Field boundaries land in the wrong places, so a descriptor's contents become another field's contents. Every field is individually a legal value.
• Payload lengths are read from the wrong offsets, so reconstruction accumulates the wrong extent — which §19's exact-match check catches, but only if it is an exact match rather than a >=.
• If the formats differ in how many objects a transfer unit may carry, the receiver may split one object into two or merge two into one.
• And the replay store now holds objects framed under a format that is no longer active, so a retry after the change retransmits something the receiver cannot parse — turning a recoverable transport error into an unrecoverable one.

Why the symptom is confusing. The corruption begins at a specific moment and then either stops (if both sides settle into the new format) or continues indefinitely. So the failure looks like a burst of errors correlated with a configuration event, and the natural hypothesis is that the new format is broken — when in fact both formats work and the transition was unmanaged.

// Illustrative — the format cannot move while anything is outstanding.
property p_format_stable_while_outstanding;
  @(posedge clk) disable iff (!rst_n)
    (map_queue_occupancy != '0 || replay_occupancy != '0 || recon_open_objects != '0)
      |=> $stable(fmt_active_q);
endproperty
a_format_stable_while_outstanding: assert property (p_format_stable_while_outstanding);
 
// And every object was framed under the format that is still active — the
// property that makes a retry after a change impossible rather than unlikely.
property p_replay_entry_matches_active_format;
  @(posedge clk) disable iff (!rst_n)
    replay_entry_valid[e] |-> (replay_entry_fmt[e] == fmt_active_q);
endproperty
a_replay_entry_matches_active_format:
  assert property (p_replay_entry_matches_active_format);

DV. Request a format change with objects in every stage — queued, in replay, and partially reconstructed — one stage at a time and then all three. Verify the change does not commit, that acceptance is blocked, and that the change commits once the path drains. Then inject a transport error during the drain, which is the case where a retry and a pending format change interact.

12. Two Reliability Mechanisms, One Link

The section that exists only for CXL, and the one an implementer must not skip.

What is verified on each side:

Every row has an entry on both sides. That is the whole problem.

The corollary about credits, which is easier and worth getting right. Chapter 9.5 §14 established that credits can legitimately exist at two layers protecting different storage, and CXL is the clearest example in the curriculum: its link layer returns credits per message class — and Chapter 11.3 §28 showed that the per-class nature is precisely what makes the coherence protocol deadlock-free. So UCIe link credits protecting transport buffers do not replace CXL's per-class credits protecting the coherence engine's buffers. Collapsing them would remove the property §11.3 depends on, and no counter bound would notice.

13. Wrong Architecture — Stacked Retry

// WRONG — a mapping layer that adds its own retry on top of a transport that
// already provides one, "for safety".
assign map_retry_start = map_timeout_expired && !map_ack_received;

Architecture. Two mechanisms, two timeouts, two notions of what has been delivered, and no coordination between them.

Cycle behaviour. In the common case the lower mechanism recovers the error long before the upper timeout, and the upper mechanism never fires. The design appears to work, and the redundant layer appears free.

Failure, and it appears only under the conditions that matter. When errors are frequent enough — the retry-storm regime of Chapter 9.4 §15 — the lower mechanism's recovery time approaches the upper mechanism's timeout. Then:

• The upper layer retransmits an object the lower layer is already retransmitting. The far side now receives it twice, and unless the delivery fence (§19) suppresses the duplicate, a coherence action is processed twice — Chapter 11.3 §25's failure, with distributed-state corruption.
• The two mechanisms' notions of order diverge. Lower-layer replay preserves order structurally; an upper-layer retransmission inserted into a stream that is already being replayed does not, and Chapter 11.3 §24 established that the device owns its own ordering — so a reordering here manifests as a coherence bug.
• Each mechanism's error counters undercount, because each attributes recoveries to itself. The link looks healthier than it is, exactly when accurate telemetry matters most.
• Escalation logic misfires. If either layer escalates on a retry count, and retries are being split between two mechanisms, neither reaches its threshold and a genuinely failing link is never escalated.

Why it survives review. The redundant mechanism is defensive, well-intentioned, and inert in every test that does not stress the error rate. It reads as prudence.

One reliability mechanism per traffic class per link. If two exist, the second one's only measurable effect is to convert a recoverable transport error into a duplicated semantic action.

14. The Boundary Queue and the Acceptance Gate

// Illustrative per-class boundary storage between the CXL side and the
// Adapter. Per-class rather than shared, for Chapter 11.3 §28's reason: a
// blocked class must not be able to block another class's progress.
cxl_obj_t map_queue_q [3][MAP_Q_DEPTH];
logic [MAP_Q_W-1:0] map_occ_q [3];
 
// Illustrative acceptance gate. Accepting a CXL object commits every stage
// between here and the far side's engine.
assign cxl_obj_ready =
    map_queue_space[obj_cls]    // this CLASS has room — not "some class does"
 && replay_space                // Ch 9.4 — retention is possible
 && (tx_credit_q != '0)         // Ch 9.5 — the receiver has a slot
 && ucie_operational            // the transport can carry flits at all
 && fmt_allows_accept;          // §10 — no format change is armed

Architecture. Five vetoes, each owned by a different mechanism, and the first is indexed by class.

Why per-class queue space rather than a shared pool. Because a shared pool recreates the dependency Chapter 11.3 §28 showed the protocol is structured to avoid. If a full pool of queued memory writes prevents a coherence response from being accepted, and that response is what would let the memory writes drain, the result is a deadlock at the mapping boundary — a layer below the protocol, recreating a hazard the protocol layer carefully eliminated. Every safety assertion passes.

State. Per-class occupancy counters with per-object lifetime in aggregate; the queue entries themselves are the accepted-object storage that Chapter 10.1 §10's rule protects — an accepted object cannot vanish.

Failure, and the diagnostic value is in the differences. Without per-class space the failure is the deadlock above. Without replay_space the object becomes unrecoverable the moment the link errors. Without credit the remote buffer overflows. Without ucie_operational the object is accepted into a path that cannot move it. Without fmt_allows_accept it is framed under a format that is about to change (§11). Five omissions, five different symptoms, several on different dies.

15. Wrong RTL — UCIe Ready Alone

// WRONG — acceptance derived from the transport's readiness.
assign cxl_obj_ready = ucie_tx_ready;

Architecture. The most available signal substituted for the composed condition. This is the fifth appearance of this shape in the curriculum — Chapter 10.2 §8, Chapter 10.3 §6, Chapter 10.4 §7, Chapter 11.2 §14 — and by now the count is itself the argument: the shape recurs because the available signal is always more available than the correct one.

Failure. A semantic transaction is accepted with no storage that can hold it and no replay entry that can recover it. The semantic layer has been told the object is safely handed over. The object then does not exist anywhere.

For a memory write that is a lost write. For a coherence action it is worse: the initiating agent has moved its line into a transient state waiting for a response to a transaction that was never transmitted, so the line sits transient until a timeout — and Chapter 11.3 §27 established that a coherence timeout cannot be recovered from by guessing. One missing veto term produces an unrecoverable coherence state.

16. Reconstruction and the Semantic Delivery Fence

The heart of the chapter, and the mechanism the whole thing exists to build.

// Illustrative reconstruction and delivery. A partial CXL object is not a
// smaller CXL object; it is not an object at all.
logic                      obj_open_q;        // reconstruction in progress
cxl_class_t                obj_open_cls_q;    // the class it was opened with
logic [OBJ_LEN_W-1:0]      obj_bytes_q;       // accumulated so far
logic [OBJ_LEN_W-1:0]      obj_expected_q;    // declared extent
logic                      obj_complete;
 
// EXACT match, never >=. A >= accepts an over-length object, which means an
// extent mismatch went undetected and the payload boundary is wrong.
assign obj_complete = obj_open_q && (obj_bytes_q == obj_expected_q);
 
// The delivery fence. THREE independent conditions, and the second is the one
// that makes transport replay invisible above this line.
logic transport_resolved_q;   // the transport has confirmed, not merely arrived
logic obj_is_duplicate;       // this transport object has been seen before
 
assign semantic_deliver = obj_complete
                       && transport_resolved_q
                       && !obj_is_duplicate
                       && delivery_order_ok;   // §17 — dependencies satisfied
 
assign deliver_to_class = obj_open_cls_q;      // §5 — routed by class, once

Architecture. Four conditions, and each excludes a different failure. Completeness excludes handing a truncated object to an engine. Transport resolution and duplicate suppression together exclude the replay from being observed. Ordering excludes a legal object arriving at an illegal moment.

State. An open flag, a class, an accumulator, and the declared extent — per-object lifetime, cleared on completion or invalidation. Plus duplicate-detection state, which is Chapter 9.4 §12's last-delivered identity.

Cycle behaviour. Accumulates as transport units arrive; semantic_deliver fires at most once per semantic object, regardless of how many times its transport representation crossed the link.

Contract. The semantic engine may assume every delivery is a distinct transaction. That assumption is the entire reason CXL semantics survive the crossing, and Chapter 11.3 §25 is what happens without it.

Failure. §17, immediately.

On malformed detection. A mapping layer can detect impossible states: a continuation with no object open, an accumulated extent exceeding the declared one, and — specific to a multi-class transport — a class change mid-object, which is the signature of a framing or format error rather than a data error:

// Illustrative — malformed conditions are reported abstractly and refused,
// never absorbed. Do NOT invent UCIe or CXL error codes for these; the
// reporting mechanism is defined by the interface you implement.
assign obj_malformed = (unit_is_continuation && !obj_open_q)
                    || (obj_bytes_q > obj_expected_q)
                    || (obj_open_q && (unit_cls != obj_open_cls_q));

17. Wrong RTL — Semantic Processing on First Arrival

// WRONG — semantic delivery keyed on transport arrival.
assign semantic_deliver = flit_received && flit_crc_ok;

Architecture. Arrival treated as resolution. The two differ in exactly one case, and it is a case that happens.

Cycle behaviour. Correct whenever nothing goes wrong, and correct even for most errors — because a corrupted flit that is retried was never delivered the first time.

Failure, and it needs the specific mechanism to be understood. Chapter 9.4 §12 established that a retry also occurs when a confirmation is lost — the original flit arrived intact, its acknowledgement did not, and the transmitter retransmits. The receiver now sees the same transport object twice, both times with a passing CRC.

With delivery keyed on arrival, the far-side engine processes it twice. What that costs depends on the class, and the range is the reason this section is the chapter's core:

The CXL.cache row is why the fence is not optional. A duplicated probe generates a response to a snoop the host is not tracking — and Chapter 11.3 §15 recorded that only one snoop per line per device may be outstanding, so the host has no context for it. A duplicated grant may be applied after a subsequent local transaction has opened, overwriting its transient state and producing two write owners. Neither has an address you can inspect afterwards.

And none of it produces an error. The CRC passed both times. The transport did precisely what it was designed to do.

The rule, restated because it is the chapter:

Transport retirement and semantic retirement are different events. The transport retires an object when delivery is confirmed. A CXL transaction retires according to CXL's own rules — a response returning, a tag freeing, an ownership transfer completing. Neither is the other's trigger.

18. SVA — Exactly-Once Semantic Delivery

// Illustrative. Uses the VERIFICATION-ONLY monitor tag of §8, because the
// protocol identities are reused and therefore cannot distinguish "this
// object again" from "a different object with the same identity".
 
// SAFETY — at most one semantic delivery per source object. This is the
// chapter's thesis as a property, and it is checkable at every effort level.
property p_semantic_delivery_at_most_once;
  @(posedge clk) disable iff (!rst_n)
    semantic_deliver |-> !delivered_mon[deliver_mon_id];
endproperty
a_semantic_delivery_at_most_once: assert property (p_semantic_delivery_at_most_once);
 
// SAFETY — never deliver a partial object.
property p_no_delivery_before_complete;
  @(posedge clk) disable iff (!rst_n)
    semantic_deliver |-> (obj_bytes_q == obj_expected_q);
endproperty
 
// SAFETY — an over-length object is detected, not delivered. Catches a >= that
// should have been ==.
property p_no_overlength_object;
  @(posedge clk) disable iff (!rst_n)
    obj_open_q |-> (obj_bytes_q <= obj_expected_q);
endproperty
 
// SAFETY — an object is delivered to the engine of its own class. A class that
// drifted would hand a coherence action to the memory engine.
property p_delivered_to_own_class;
  @(posedge clk) disable iff (!rst_n)
    semantic_deliver |-> (deliver_to_class == obj_open_cls_q);
endproperty
 
// CONSERVATION — a transport retry does not re-allocate a replay entry.
// Chapter 9.4 §8's invariant, restated because §13's stacked retry breaks it.
property p_retry_does_not_reallocate;
  @(posedge clk) disable iff (!rst_n)
    transport_retry_start |=> (replay_occupancy == $past(replay_occupancy));
endproperty
 
// CONSERVATION — a retained object is not modified while it is retained. A
// replay must retransmit what was sent, not what the queue now holds.
property p_replay_entry_immutable;
  @(posedge clk) disable iff (!rst_n)
    (replay_entry_valid[e] && !replay_entry_retire[e])
      |=> $stable(replay_entry_obj[e]);
endproperty
 
// BOUNDEDNESS — per-class occupancy never exceeds its depth. Assert the bound;
// never saturate it (Ch 9.5 §8).
property p_class_occupancy_bounded;
  @(posedge clk) disable iff (!rst_n)
    map_occ_q[c] <= MAP_Q_DEPTH;
endproperty
 
// LEGALITY — the arbiter only ever selects a class with an eligible head.
property p_selected_class_has_eligible_head;
  @(posedge clk) disable iff (!rst_n)
    arb_grant_valid |-> class_eligible[arb_grant_class];
endproperty

On the exactly-once pair, and being honest about which half is which. p_semantic_delivery_at_most_once is safety — it can be checked in any run, and it is the half that catches the bug. "Exactly once" additionally requires liveness: that every accepted object is eventually delivered. That half cannot be asserted without assumptions (the transport eventually succeeds, the far side eventually accepts, no error injection is active), and it is properly checked as an end-of-test reconciliation in the scoreboard (§25) rather than as a temporal property with an invented bound. Claiming a bounded-latency property here and then waiving it during injection tests is worse than not writing it.

19. Ordering Across Classes

The rule, and the discipline against over-generalising it.

What must not be done. Chapter 9.3's per-stream FIFO model was built for a protocol UCIe does not interpret, where the implementer defined the domain. Applying it to CXL means inventing an ordering model for three protocols whose rules are the specification's.

What is verified and can be stated. Two things, and they point in opposite directions, which is exactly why a single generic rule fails:

CXL.mem's request classes must drain independently. Its forward-progress rules state that the request-without-data and request-with-data classes each need to be credited independently and must eventually drain without dependency on any other traffic type. So a mapping layer must not create a dependency between them.

CXL.cache's ordering is the device's responsibility. Chapter 11.3 §24 quoted it: the host will not preserve the order of CXL.cache requests as delivered by the device, and the device must maintain ordering where it matters.

Read together, those give the mapping layer its brief precisely. It must not introduce dependencies that the protocol forbids, and it must not destroy orderings the semantic layer is relying on having enforced by issue control. Those are different obligations and both are the mapping layer's:

The transport must not introduce reorderings the protocol layer does not expect, and must not introduce cross-class dependencies the protocol layer's forward-progress rules forbid. Whatever order it preserves must be at least as strong as what the carried protocol requires between any two objects.

Three practical consequences.

Do not merge classes into one ordering domain. A single FIFO across CXL.io, CXL.mem and CXL.cache satisfies "preserve order" and violates independent drain. It is the strictest possible ordering and the most likely to deadlock.

Do not let the arbiter reorder within a class beyond what that class permits. The safe default, where the mapping layer does not model the protocol's rules, is to preserve acceptance order within a class and to arbitrate freely between classes.

Model the right number of expectation queues. Chapter 9.3 §13's rule — one expectation queue per ordering domain — still applies, and the domains here are the carried protocols', not one per class by assumption. Getting the count wrong produces false failures if too strict and silent escapes if too loose.

20. Retry and Ordering Interact — and This Is Where They Bite

A specific scenario, because the general statement above is easy to nod at and hard to apply.

Coherence action A changes ownership of line X. Action B targets the same line and must not be observed before A.

1. A is accepted, queued, selected, transmitted.
2. B is accepted, queued, selected, transmitted.
3. A's flit is corrupted. B's arrives intact.
4. Go-back-N replay retransmits A, then B.
5. B therefore arrives a second time.

What must happen. B, which arrived intact at step 3, must not be delivered before A. And B's second arrival at step 5 must not be delivered at all, because B has already been delivered — or, if it was held for ordering, it must be delivered exactly once from whichever copy.

Three mechanisms are simultaneously load-bearing here, and that conjunction is the reason §26's regression list singles it out:

Ordering hold — B waits for A, which is head-of-line blocking (Chapter 9.3 §7) at the semantic layer rather than the transport layer.

Duplicate suppression — B's second arrival is recognised and dropped (§17).

Reconstruction state maintained across the retry — B's held state must survive the replay, not be discarded and rebuilt, because discarding it and then suppressing the rebuild loses B entirely.

Get any one wrong and the symptom is a coherence bug. B delivered before A is an ownership transition applied in the wrong order. B delivered twice is Chapter 11.3 §25. B delivered never is a transaction that times out and, per Chapter 11.3 §27, cannot be safely recovered from. Three distinct coherence failures, one transport cause, and the coherence engine is blameless in all three.

21. Arbitration Across Protocol Classes

// ILLUSTRATIVE arbiter across the three protocol classes. The UCIe Consortium
// states that the Adapter defines the underlying arbitration mechanism when
// multiple protocols are supported; the POLICY BELOW IS THIS MODEL'S OWN and
// is NOT a claim about UCIe or CXL arbitration or QoS.
module cxl_class_arbiter #(
  parameter int NUM_CLS  = 3,
  parameter int WEIGHT_W = 4
) (
  input  logic                  clk,
  input  logic                  rst_n,
 
  // A class is ELIGIBLE, not merely non-empty: its head must also be
  // orderable now (§19) and its resources available. Eligibility is computed
  // outside this module precisely so the arbiter stays a pure scheduler.
  input  logic [NUM_CLS-1:0]    class_eligible,
  input  logic                  downstream_ready,
 
  output logic                  grant_valid,
  output logic [$clog2(NUM_CLS)-1:0] grant_class
);
 
  // Deficit-style rotation: each class carries a credit that increases while
  // it is passed over and is spent when it is served. A class that is
  // continuously eligible therefore cannot be starved by another class that
  // is also continuously eligible — which is §22's failure.
  logic [WEIGHT_W-1:0] deficit_q [NUM_CLS];
  logic [$clog2(NUM_CLS)-1:0] last_q;
 
  // Pick the eligible class with the largest deficit; ties break by rotation
  // from the last grant, so the tie-break is fair rather than fixed.
  logic [$clog2(NUM_CLS)-1:0] best;
  logic                       found;
 
  always_comb begin
    found = 1'b0;
    best  = '0;
    for (int k = 0; k < NUM_CLS; k++) begin
      // Rotate the scan start so equal deficits do not always favour class 0.
      int c = (int'(last_q) + 1 + k) % NUM_CLS;
      if (class_eligible[c]) begin
        if (!found || (deficit_q[c] > deficit_q[best])) begin
          best  = c[$clog2(NUM_CLS)-1:0];
          found = 1'b1;
        end
      end
    end
  end
 
  // grant_valid depends on eligibility and on downstream_ready, and NOT the
  // other way round: no combinational path from a requester's valid to its
  // own ready (Ch 5.5 §2).
  assign grant_valid = found && downstream_ready;
  assign grant_class = best;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      for (int c = 0; c < NUM_CLS; c++) deficit_q[c] <= '0;
      last_q <= '0;
    end else if (grant_valid) begin
      last_q <= grant_class;
      for (int c = 0; c < NUM_CLS; c++) begin
        if (c == int'(grant_class)) begin
          deficit_q[c] <= '0;                       // served: reset
        end else if (class_eligible[c] && (deficit_q[c] != {WEIGHT_W{1'b1}})) begin
          deficit_q[c] <= deficit_q[c] + 1'b1;      // passed over: accrue
        end
      end
    end
  end
 
endmodule

Classification: synthesizable, illustrative. The arbitration policy is this model's, stated as such.

Architecture. A scheduler with a fairness memory. The deficit is what turns "pick the best" into "pick the best, but not the same one forever".

State. NUM_CLS deficit counters plus a rotation pointer, with per-link-epoch lifetime — fairness state has no meaning across a link that has been re-established, and re-initialising it is correct.

Cycle behaviour. Combinational selection over registered deficits. The deficit of a passed-over eligible class increases; an ineligible class does not accrue, which matters: a class that is empty is not being starved, and letting it accrue would give it a burst of unearned priority the moment it becomes eligible.

Contract. The arbiter selects only eligible classes, and the downstream stage is ready when a grant is issued.

Failure. §22 without the deficit; a burst of unearned priority if ineligible classes accrue.

Deliberately simplified: the deficit saturates rather than wrapping, so an extremely long starvation interval degrades to plain rotation instead of wrapping to zero and restarting the starvation. Weights are uniform; a real design may weight classes by required bandwidth, which is a system-level decision.

Note the WEIGHT_W saturation explicitly, because it is the kind of arithmetic that gets written without a bound: deficit_q[c] + 1'b1 on a saturated counter would wrap to zero, converting the anti-starvation mechanism into a starvation mechanism with a period. The guard is one comparison.

22. Wrong Scheduler — Starvation Is Not a Performance Bug Here

// WRONG — strict priority, chosen because memory traffic "matters more".
always_comb begin
  grant_valid = downstream_ready && (class_eligible != '0);
  if      (class_eligible[CLS_MEM])   grant_class = CLS_MEM;
  else if (class_eligible[CLS_IO])    grant_class = CLS_IO;
  else                                grant_class = CLS_CACHE;
end

Architecture. Fixed priority with no fairness memory. It is smaller, faster, and easier to reason about, and every correctness property holds.

Cycle behaviour. With continuously eligible memory traffic, CLS_CACHE is never granted.

Failure — and the reason it is in this chapter rather than in a performance one. Every counter stays in range. Every safety assertion passes. Occupancy is bounded. No object is lost, duplicated, or reordered. And the coherence protocol stops making progress.

Trace what that means using Chapter 11.3's state:

• A coherence probe response cannot be transmitted, so the host never resolves its snoop.
• By its own rules the host cannot issue another snoop to that line, so that line is now permanently in limbo.
• The device's line stays in a transient state, and the local access waiting on it stays blocked.
• The blocked local access eventually blocks whatever the accelerator was computing.
• Eventually a coherence timeout fires — and Chapter 11.3 §27 established that a coherence timeout cannot be recovered from by restoring the previous state, so a starvation bug in a transport arbiter has produced an unrecoverable coherence condition.

In a coherent system, arbitration fairness is a correctness concern, not a performance one. A starved response channel converts into a stalled coherence protocol, and from there into an error path that has no safe recovery.

And this is precisely Chapter 11.3 §28's deadlock hazard arriving from below. The protocol's channel classification eliminated it at the protocol layer; a transport arbiter beneath that layer reintroduced it. A property established at one layer can be destroyed by a layer that has never heard of it, which is the general lesson worth carrying out of this section.

// Illustrative bounded-progress property. ASSUMPTIONS, which must be stated:
//   - the class remains continuously eligible
//   - the downstream stage becomes ready within the window
//   - no error injection is active
// The bound is an engineering choice, not a UCIe or CXL value.
property p_no_class_starved;
  @(posedge clk) disable iff (!rst_n || error_injection_active)
    (class_eligible[c] && downstream_ready)[*MAX_STARVE_CYCLES]
      |-> (grant_valid && (grant_class == c))[->1];
endproperty

23. Buffer and State Lifetimes

The table this chapter exists to produce, and the widest one in Module 11.

Four rows carry most of the weight.

Replay entries do not re-allocate on retry — Chapter 9.4 §8's invariant, and §13's stacked retry is what breaks it.

Reconstruction state is held across a retry, not rebuilt. §20's scenario depends on this, and the failure mode of getting it wrong — discard and then suppress the rebuild — loses the object entirely, which is the one outcome worse than duplicating it.

The format epoch is a distributed agreement, not local configuration. Both dies must be in the same epoch, which makes re-establishing it after a recovery a coordination problem rather than a register write.

And the coherence transient state has no clean answer, which the table records honestly rather than papering over. Chapter 11.3 §26 is the section; the row exists here because this layer's recovery is what triggers it, and a mapping layer that clears its own state without signalling upward leaves the coherence engine with no way to know it must escalate.

24. Three Error Domains, Not Two

Chapter 10.3 §19 argued for three domains in the PCIe case. Here the third is larger and has its own failure catalogue, so the separation is worth restating with CXL's contents.

Transport errors — UCIe's domain. Flit CRC failures, retries, link recovery, credit accounting. Modules 8 and 9 own the mechanisms. Symptom: counters. These are the errors that announce themselves.

Protocol errors — CXL's domain. Illegal coherence transitions, ownership violations, memory-semantic faults, poisoned or viral data. Chapters 11.2 and 11.3 own the reasoning. Symptom: wrong values, or a hung transaction with a clean link.

Mapping and integration errors — this chapter's domain, and the one with no owner unless you assign it. Wrong class, wrong format epoch, duplicated semantic delivery, metadata/payload misalignment, a starved class, a cleared queue, a lost accepted object. Symptom: whatever the protocol layer above happens to do when handed something impossible — which is why these errors are consistently misattributed to the layer that reports them rather than the layer that caused them.

The third domain's defining property is that its failures are reported by other layers. A duplicated coherence action is reported, if at all, as a coherence inconsistency. A format-epoch mismatch is reported as data corruption. A starved class is reported as a coherence timeout. Nothing in the mapping layer reports anything, unless you build it — which is what §26's diagnostic counters are for.

25. Two Scoreboards, Because There Are Two Contracts

Chapter 10.2 §15 introduced the split; here each model has more to hold, and the pairing answers a question neither can.

The transport model tracks what Chapter 9.4 defined, plus what this chapter adds:

per transport object:
  allocated / transmitted / unconfirmed / retired
  retry_count, replay_occupancy_at_each_point
  format_epoch_when_framed        — §11
  class_as_carried                — §5
 
per class:
  objects_selected, cycles_since_last_grant     — §22

Its invariant: every object was delivered at least once, eventually retired, framed and interpreted under one epoch, and no class went unserved beyond the fairness bound.

The CXL semantic model tracks what the engines see, and it is the models of the two previous chapters, reused:

per class:
  offered_objects[]     — every object the local engine handed over, in order
  delivered_objects[]   — every object the remote engine received
 
CXL.mem model            — Chapter 11.2 §30: address map, value per address,
                           outstanding requests, exactly-once retirement
CXL.cache model          — Chapter 11.3 §30: value, owner, sharers, pending
                           transactions and probes per line

Its invariant: exactly once, in an order the carried protocol permits, unmodified, and with the resulting semantic state legal.

The check that only the pairing can make. delivered_objects equals offered_objects, per class, regardless of how many retransmissions the transport model recorded. A transport-only model cannot see a duplicate delivery, because from its point of view retransmitting is correct behaviour. A semantic-only model can see the duplicate but cannot explain it.

Why both are needed, in one sentence each. The semantic model tells you that — a probe was processed twice, an ownership transition happened in the wrong order, a write was lost. The transport model tells you why — the object was in the unconfirmed window when the confirmation was dropped, or the format epoch changed while it was queued, or its class had not been granted for a long time.

And the reconciliation that must happen at end of test, because it is the liveness half §18 could not assert: every offered object appears exactly once in delivered_objects, and every transport object was retired. A run that ends with an object offered and not delivered has found a loss that no in-run property fired on.

26. Diagnostic State Worth Building

// Illustrative diagnostic counters. Not a UCIe or CXL register. Each exists to
// separate the three domains of §24 — which is the only thing that makes a
// mapping-layer bug attributable to the mapping layer.
logic [31:0] obj_accepted_q   [3];   // per class
logic [31:0] obj_delivered_q  [3];   // per class, far side
logic [31:0] dup_suppressed_q [3];   // §17 — a repeat arrived and was dropped
logic [31:0] held_for_order_q [3];   // §20 — waited for a predecessor
logic [31:0] malformed_q;            // §16 — including class-change mid-object
logic [31:0] fmt_change_count_q;     // §10 — every committed epoch change
logic [31:0] fmt_change_denied_q;    // armed but path not quiescent
logic [15:0] max_starve_cycles_q [3];// §22 — worst observed per class
logic [31:0] veto_blocked_q      [5];// §14 — which term, separately

Architecture. dup_suppressed_q is the single most valuable counter in the list, and it is the one nobody builds. A non-zero value proves the delivery fence is doing work — that duplicates genuinely arrive and are genuinely suppressed. A value that is always zero means either the link is very clean or the mechanism has never been exercised, and those are indistinguishable without deliberately injecting a lost confirmation.

State. Diagnostic lifetime — these must survive link recovery and format changes and clear only on a broad, deliberate reset. A counter cleared by the event under investigation is worse than no counter, because it looks like evidence.

Failure. Saturating silently. And max_starve_cycles_q in particular must be a high-water mark rather than an instantaneous value, because the interesting number is the worst case and it will not be present when you look.

27. Flagship Trace — a Coherence Action Across a Retry

Illustrative timing throughout. Two coherence objects, A then B, to the same line, so ordering matters. Generic action names.

Seven things to read off it, and together they are the chapter.

Cycle 6: B arrived first and was not delivered. Ordering required A first. Head-of-line blocking at the semantic layer, not the transport layer.

Cycle 8: the format change is armed, not applied. §10's discipline. Note that it is armed during a retry, which is the worst possible moment and exactly why arming is separate from committing.

Cycle 9: replay does not allocate. Occupancy is unchanged; the send pointer rewound (Chapter 9.4 §8). §18's p_retry_does_not_reallocate.

Cycle 10: B is re-sent although it was fine. Go-back-N's cost, paid to preserve order structurally.

Cycle 13: B arrives twice and is delivered once. §17's rule doing its work. Without it, an ownership action is applied twice, and there is no address to inspect afterwards.

Cycle 14: B is delivered from the state held since cycle 6, not rebuilt from the cycle-13 copy. §23's reconstruction row. Either source works if they are identical; what matters is that discarding the held state and then suppressing the rebuild would have lost B entirely.

Cycle 17: the format epoch changes only after everything drains. Nine cycles after it was requested, through a corruption and a full replay. §11's assertion held throughout, and the alternative — applying it at cycle 8 — would have made the cycle-9 replay retransmit objects the receiver could no longer parse, converting a recoverable error into an unrecoverable one.

And the row that never appears: no cycle in which the coherence engines learn that any of this happened. That invisibility is the deliverable.

28. Error Injection, and What Each Case Targets

Three of these are the ones regressions omit, and all three are marked. A lost confirmation is the only injection that produces a genuine duplicate, so without it the entire delivery fence ships unexercised. A format change during a retry is the only way to reach the interaction of two independently-correct mechanisms. And deliberate starvation is the only way to reach §22, which will otherwise look like a healthy design with good throughput numbers.

29. Coverage

// Illustrative coverage for CXL-over-UCIe mapping. Not UCIe- or CXL-defined.
covergroup cg_cxl_transport @(posedge clk iff obj_event);
 
  cp_class      : coverpoint obj_cls;             // all three, §5
  cp_fmt        : coverpoint fmt_active_q;        // every format epoch, §10
  cp_fmt_change : coverpoint fmt_change_state {
    bins idle = {0}; bins armed = {1}; bins committed = {2};
  }
  cp_outstand   : coverpoint objects_outstanding {
    bins one = {1}; bins few = {[2:3]}; bins many = {[4:$]};
  }
  cp_retry      : coverpoint obj_retry_count {
    bins none = {0}; bins one = {1}; bins several = {[2:$]};
  }
  cp_dup        : coverpoint obj_was_duplicate;   // §17 — a repeat arrived
  cp_held       : coverpoint obj_held_for_order;  // §20
  cp_recon      : coverpoint recon_partial;
  cp_veto       : coverpoint which_veto_blocked;  // §14 — each term separately
  cp_grant      : coverpoint arb_grant_class;     // §21 — every class served
  cp_starve     : coverpoint starve_cycles_bucket;// §22
  cp_recovery   : coverpoint recovery_with_semantic_outstanding;
 
  // The cross Module 10 already argued for, now per class: a duplicate
  // arriving while another object is held for ordering.
  x_dup_held      : cross cp_dup, cp_held, cp_class;
  // A retry with several objects outstanding, per class.
  x_retry_out     : cross cp_retry, cp_outstand, cp_class;
  // THE cross unique to this chapter: a format change armed while a retry is
  // in progress. Two independently-correct mechanisms interacting.
  x_fmt_retry     : cross cp_fmt_change, cp_retry;
  // Starvation observed for each class — none of these should be in the
  // highest bucket, and covering the bucket at all is a finding.
  x_starve_class  : cross cp_starve, cp_grant;
  // Recovery with a semantic transaction outstanding, per class. The CXL.cache
  // bin is the hardest case in Module 11.
  x_recovery_cls  : cross cp_recovery, cp_class;
 
endgroup

Why x_fmt_retry is this chapter's signature cross. A format change with an idle path exercises §10 alone. A retry with a stable format exercises §17 alone. A format change armed while a retry is in flight exercises the epoch discipline, the replay store's immutability, and the drain condition simultaneously — the configuration of §27's trace, and the one where three separately-verified mechanisms first meet.

And x_recovery_cls is the batch's hardest bin. Recovery with a CXL.cache transaction outstanding reaches Chapter 11.3 §26's distributed-state problem through this layer's mechanism, which is the only way to reach it at all.

30. Diagnostic Taxonomy

The two rows to internalise are the third and the sixth, because both are mapping-layer bugs that present as CXL-layer failures. A duplicated ownership transition looks like a coherence engine bug. A coherence timeout looks like a coherence engine bug. Neither is, and the counters of §26 are what make that visible in one look instead of a week.

31. Debug Checklist

1. Which protocol class generated the object? §5 — a surprising answer ends the investigation.
2. Was it accepted exactly once at the boundary? §14 — and if not, which veto was low.
3. Which per-class queue entry held it? Follow the verification-only monitor tag.
4. Which format epoch was it framed under? §10.
5. Did the format epoch change during its lifetime? §11 — check the assertion, and the denied-change counter.
6. Did metadata and payload stay bundled? §8 — correlate failures with differing consecutive descriptors.
7. Was a replay entry allocated exactly once? §18 — occupancy across the retry.
8. Did the transport retry, and how many times?
9. Was a confirmation lost? That is the only mechanism that produces a genuine duplicate.
10. Did the object arrive more than once? §26's duplicate counter — and was the repeat suppressed?
11. Did the far side reconstruct one complete object? §16 — exact extent match, and no class change mid-object.
12. Was semantic delivery performed exactly once? §18 for the safety half; §25's end-of-test reconciliation for the rest.
13. Was the object held for ordering, and released in the right order? §19, §20.
14. Was its class starved? §26's high-water mark. A coherence timeout with a healthy link starts here.
15. Did a recovery occur, and which rows of §23 survived it? Check each against the policy, not against intuition.
16. Do the transport and semantic scoreboards agree? §25 — the single question that routes the whole investigation.

Step 16 is the highest-yield one. A clean transport model with a diverged semantic model puts the bug in this layer or above it. A diverged transport model makes everything above it a consequence.

32. Common Misconceptions

"CXL over UCIe is the PCIe mapping with different names." Three structural differences (§3): CXL is three concurrently-multiplexed protocols rather than one; CXL already has its own multiplexer with per-interface virtual link states; and CXL.cache/CXL.mem has its own link layer with its own CRC and retry, which PCIe's data-link functions did not survive as. PCIe over UCIe was a substitution problem; this is a division-of-ownership problem.

"The Adapter should understand cache ownership." It should not, and the tempting version of the violation is a performance optimisation — prioritising flits carrying dirty data. It requires parsing a CXL revision whose coherence model was replaced between revisions, makes a protocol-agnostic layer protocol-specific, turns a performance idea into a correctness coupling, and does not even work reliably because the Adapter may be looking at a replayed object whose state is stale. Pass an opaque hint, not a meaning (§9).

"Any flit format can change while traffic is active." A format change under live traffic creates a boundary in the stream where framing and interpretation disagree, and it leaves the replay store holding objects the receiver can no longer parse — converting a recoverable error into an unrecoverable one. Request and commit are two registers and a quiescence condition (§10, §11).

"Transport replay may be visible as a second coherence request." Then a probe is processed twice, or a grant applied twice. Unlike a duplicated memory write, there is no address to inspect afterwards — only distributed state that no longer agrees (§17).

"Clean CRC means the CXL semantics are correct." CRC proves the bytes arrived as sent. Misclassification, format-epoch mismatch, metadata misalignment, premature delivery, and duplicate delivery all pass CRC (§24, §30).

"All CXL protocol classes share one ordering rule." CXL.mem's request classes must drain without dependency on any other traffic type, while CXL.cache's ordering is explicitly the device's responsibility because the host does not preserve it. A single FIFO across all classes satisfies "preserve order" and violates independent drain (§19).

"One scoreboard is enough." A transport model cannot see a duplicate delivery; a semantic model cannot explain one. And the semantic model here is two models — the memory model of 11.2 and the coherence model of 11.3 — because the classes have different invariants (§25).

"A UCIe recovery can clear all the CXL semantic state." Link-epoch state is re-baselined correctly; semantic transaction state and coherence transient state are owned above the link, and clearing them leaves a coherence engine with no way to know it must escalate (§23).

"Metadata/payload misalignment would be caught by CRC." Every field is individually valid. In the coherence case it supplies the wrong line's data in answer to a probe, which presents as a coherence bug in a blameless engine (§8).

"Fair arbitration is only a performance concern." A starved coherence response channel means the host never resolves its snoop, cannot issue another to that line, and the device's line stays transient until a timeout that Chapter 11.3 §27 showed has no safe recovery. In a coherent system, fairness is a correctness property (§22).

"Two reliability mechanisms are safer than one." Two timeouts, two notions of delivery, no coordination. Under a raised error rate the upper mechanism retransmits what the lower one is already retransmitting, both undercount, escalation thresholds are never reached, and the observable outcome is a duplicated semantic action (§13).

33. Understanding Check

34. Summary and What Comes Next

Transport may retransmit a CXL flit as often as reliability requires; the semantics above it must observe the transaction exactly once, in an order the protocol permits, with its meaning unchanged.

The premise had to be corrected first, and in the opposite direction from Module 10. PCIe over UCIe was a substitution problem; its data-link functions were largely replaced by the Adapter's. CXL over UCIe is a division-of-ownership problem, because CXL arrives with three multiplexed protocols, its own ARB/MUX with per-interface virtual link states, and — decisively — its own link layer with CRC inside its flit and a retry mechanism of its own, meeting an Adapter whose reliability is explicitly optional.

The mechanisms: class assigned once and never interpreted below the boundary, with an opaque hint rather than a meaning where the transport appears to need semantic information. Format as an epoch, with request and commit separated by a quiescence condition, because a change under live traffic leaves the replay store holding objects the receiver can no longer parse. Five acceptance vetoes, one of them indexed by class so that a full queue of one class cannot block another and recreate below the protocol a deadlock the protocol carefully eliminated. A delivery fence with four conditions, of which duplicate suppression is the one that makes retry invisible. A fair arbiter, because in a coherent system a starved class becomes a stalled protocol and then an unrecoverable timeout. And reconstruction state held across a retry rather than rebuilt, because losing an object is worse than duplicating it.

The rule the chapter exists for: transport retirement and semantic retirement are different events, and neither is the other's trigger. A lost confirmation makes the transport deliver an object twice; the mapping layer must deliver it once, and for a coherence action the alternative corrupts distributed state with no address to inspect afterwards.

For verification, three error domains and two scoreboards — where the third domain's defining property is that its failures are reported by other layers, so a duplicated ownership transition and a coherence timeout both present as coherence bugs in a blameless engine. And the injections regressions omit are named: a lost confirmation, a format change during a retry, and deliberate starvation.

Memory semantics, coherence semantics, and the transport that must carry both without changing their meaning are now individually understood. What remains is composing them: a complete coherent chiplet, brought up from reset through discovery, capability negotiation, window configuration, coherence initialisation and first traffic, with every one of this module's lifetimes in play at once:

• 11.5 — CXL-over-UCIe Integration — the integration model for a CXL-coherent chiplet on UCIe.

Browse the full path on the UCIe tutorials index.