The Three CXL Protocols

Chapter 4.4 put the three protocols side by side and answered the question that comes first: who is allowed to start a transaction. Direction is the discriminator, and everything in that chapter followed from it.

Module 6 opens by asking the question that comes next, and it is a harder one. Direction tells you which way the arrow points. It does not tell you what either end has to build.

1. The Engineering Problem — A Protocol Is a Standing Obligation

An architect writes "the device will support CXL.cache" in a specification. That sentence commits real silicon on both ends of the link, and most of it is not on the datapath:

• the device must hold a cache, and an agent that tracks its coherence state;
• the host must track which lines that device may be holding, so it can find them again;
• both must keep that state consistent across every reset, error and retrain.

Now write "the device will support CXL.mem" instead. The device needs a memory controller and no cache at all. The host needs its coherence resolution logic but does not need to track device-held copies of host memory, because the device is not holding any.

Same link, same three names, completely different obligations. A team that chooses protocols from a list of names and discovers the state obligations at implementation is a team that has already committed its area budget.

2. The One-Sentence Model

Each protocol is a contract about who owns a resource and who must keep state about it — .io owns control and needs no coherence state, .cache puts host memory in a device cache and so obliges both ends to track it, .mem puts device memory under host management and so obliges the device to be a memory and the host to manage it.

Call it the ownership-and-state model. Chapter 4.4's who-asks model tells you the direction; this one tells you the bill.

3. What This Chapter Owns

Deliberately not repeated from 4.4: the who-asks model, the direction table, the switch-behaviour row, and the coexistence arithmetic. Those stand. This chapter starts from them and asks what the arrows cost.

4. The Ownership Boundary

Everything in this chapter follows from one picture: there are two memories, and two of the three protocols cross between them in opposite directions.

Read the two labelled arrows as a pair. .cache goes right-to-left and .mem goes left-to-right, and each ends in the memory the other side owns. That crossing is what creates the state obligation: whoever reaches across must be tracked by whoever owns.

.io touches neither memory. That is why it costs no coherence state — and also why it is the one protocol every device must have, since discovery and configuration have to work before any ownership question can be asked.

5. The State Obligation Table

This is the chapter's central artefact. For each protocol, what does each side have to build and remember?

What each side must hold.

Who owns what, and whether you can skip it.

The requester follows from the owner: .cache is the one the device initiates (4.4), because it is the one reaching into memory it does not own. The coherence role follows too — under .cache the device becomes a caching agent, and under .mem device memory joins the host's space.

Three rows deserve emphasis.

"Resource owner" is the row that generates the other two. Whoever owns the memory must track who else may be holding it. .cache puts host memory in a device, so the host gains tracking state. .mem puts device memory under host management, so the device gains the obligation to be a well-behaved memory.

"Optional?" is short but decisive. .io is mandatory for all devices; .cache and .mem are optional and usage specific. So a device with no .cache is not a degraded device — it is a normal one, and Chapter 5.6 showed what happens to a compatibility model that treats a missing optional protocol as a fault.

Host state differs between .cache and .mem, and that is not obvious. Both are coherent, so it is tempting to assume they impose the same host cost. They do not: .cache requires the host to track copies held elsewhere, while .mem requires the host to manage a memory that is not local. RTL 1 makes the difference checkable, and mutation M2 is what happens when a design assumes they are the same.

Read the middle and bottom rows as two separate bills. The .cache column charges both sides; the .mem column charges both sides differently; and the two coherent columns share no structure at all — which is the fact RTL 1 makes checkable and Debug Lab 2 shows a design getting wrong.

6. Teaching-model boundary

7. RTL 1 — Advertising a Protocol Commits State

module state_obligation #(
  parameter bit IGNORE_STATE = 1'b0     // 1 = advertise without the state
) (
  input  logic       clk, rst_n,
  input  logic       en_io, en_cache, en_mem,
  // device-side structures actually built
  input  logic       has_cfg_space,     // .io needs config/control state
  input  logic       has_dev_cache,     // .cache needs a cache + DCOH agent
  input  logic       has_mem_ctrl,      // .mem needs a memory controller + HDM
  // host-side structures actually built
  input  logic       host_has_home,     // Home Agent resolves coherence
  input  logic       host_tracks_dev,   // host must track device-held copies
  input  logic       valid,
  output logic       dev_state_ok, host_state_ok,
  output logic [2:0] advertised,
  output logic [2:0] backed,            // advertised AND actually backed
  output logic       advertised_unbacked_err, io_missing_err
);
  logic io_ok, cache_ok, mem_ok;
 
  // .io needs configuration state on the device only.
  assign io_ok    = en_io    && has_cfg_space;
  // .cache: the DEVICE holds copies of HOST memory, so the device needs a
  // cache and a coherence agent, and the host needs to track what the device
  // holds. Both sides gain state -- this is the asymmetric-complexity trade.
  assign cache_ok = en_cache && has_dev_cache && host_has_home && host_tracks_dev;
  // .mem: the HOST reaches DEVICE memory, so the device needs a memory
  // controller and the host needs its Home Agent. The device does NOT need a
  // cache for .mem alone.
  assign mem_ok   = en_mem   && has_mem_ctrl && host_has_home;
 
  assign advertised = {en_mem, en_cache, en_io};
  assign backed     = IGNORE_STATE ? advertised : {mem_ok, cache_ok, io_ok};
 
  assign dev_state_ok  = (!en_io    || has_cfg_space) &&
                         (!en_cache || has_dev_cache) &&
                         (!en_mem   || has_mem_ctrl);
  assign host_state_ok = (!en_cache || (host_has_home && host_tracks_dev)) &&
                         (!en_mem   || host_has_home);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      advertised_unbacked_err <= 1'b0; io_missing_err <= 1'b0;
    end else if (valid) begin
      if ((backed & ~{mem_ok, cache_ok, io_ok}) != 3'd0) advertised_unbacked_err <= 1'b1;
      // CXL.io is mandatory for all devices.
      if ((en_cache || en_mem) && !en_io) io_missing_err <= 1'b1;
    end
  end
endmodule

Purpose. To make "the device supports CXL.cache" a claim the design can be held to, rather than a sentence in a specification.

Architecture position. Between the capability registers and anything that enables an engine. It is the last place a design can notice that it is about to advertise something it did not build.

State. No datapath state; two sticky diagnostics. The structure inputs are parameters of the build, not runtime signals — in a real design they would be tied off by the configuration that selects which engines are instantiated, which is exactly why they can drift from the advertisement (5.6's phantom capability).

Synthesis. A handful of AND gates and two set-reset flops. The cost is nil; the value is that a derivative part which removes an engine cannot silently keep advertising it.

DV. Sweep the eight enable combinations against each structure being present or absent, and assert backed per bit. The interesting cases are the ones that remove one structure and check what survives — which is where mutation M2 lives.

  no device cache   : dev_ok=0 backed=101 (cache bit drops)
  .mem survives the loss of a device cache        : ok
  no memory ctrl    : dev_ok=0 backed=011 (mem bit drops)
  .cache survives the loss of a memory controller : ok
  host cannot track : host_ok=0 backed=101  <-- HOST state gates .cache
  .mem is unaffected -- the two need different host state : ok

Those three lines are §5's table proved rather than asserted. Removing the device cache kills .cache and leaves .mem standing; removing the memory controller does the reverse; removing the host's tracking kills .cache and leaves .mem alone. The two coherent protocols are not two flavours of one thing — they depend on disjoint structures on both sides.

8. RTL 2 — The Ownership Boundary, in Logic

module resource_ownership #(
  parameter bit SWAP_DIRECTION = 1'b0   // 1 = the reversed-arrow bug
) (
  input  logic       clk, rst_n, req_valid,
  input  logic [1:0] req_class,        // 0=.io 1=.cache 2=.mem
  input  logic       tgt_is_host_mem, tgt_is_dev_mem, initiator_is_dev,
  output logic       route_ok, to_host_mem, to_dev_mem,
  output logic       wrong_owner_err, wrong_initiator_err
);
  localparam logic [1:0] C_IO = 2'd0, C_CACHE = 2'd1, C_MEM = 2'd2;
  logic cache_req, mem_req;
 
  assign cache_req = req_valid && (req_class == C_CACHE);
  assign mem_req   = req_valid && (req_class == C_MEM);
 
  // .cache targets HOST memory and is initiated by the DEVICE.
  // .mem   targets DEVICE memory and is initiated by the HOST.
  assign to_host_mem = SWAP_DIRECTION ? mem_req   : cache_req;
  assign to_dev_mem  = SWAP_DIRECTION ? cache_req : mem_req;
 
  assign route_ok = req_valid &&
                    ((req_class == C_IO) ||
                     (cache_req && tgt_is_host_mem &&  initiator_is_dev) ||
                     (mem_req   && tgt_is_dev_mem  && !initiator_is_dev));
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      wrong_owner_err <= 1'b0; wrong_initiator_err <= 1'b0;
    end else if (req_valid) begin
      // A request must land in the address space its class owns.
      if (to_host_mem && !tgt_is_host_mem) wrong_owner_err <= 1'b1;
      if (to_dev_mem  && !tgt_is_dev_mem)  wrong_owner_err <= 1'b1;
      // And be raised by the end its class allows to raise it.
      if (cache_req && !initiator_is_dev)  wrong_initiator_err <= 1'b1;
      if (mem_req   &&  initiator_is_dev)  wrong_initiator_err <= 1'b1;
    end
  end
endmodule

  .cache dev->host mem : route_ok=1 to_host=1 to_dev=0
  .mem  host->dev mem  : route_ok=1 to_host=0 to_dev=1
  .cache raised by HOST: route_ok=0  <-- wrong initiator
  .mem  raised by DEV  : route_ok=0  <-- wrong initiator

Two independent conditions, two diagnostics. wrong_owner_err says the request landed in the wrong memory; wrong_initiator_err says the wrong end raised it. A design can get either wrong independently — mutation M4 swaps the directions and trips the first, M6 drops the initiator check and trips the second — and merging them into one flag would make the report ambiguous about which invariant broke.

wrong_initiator_err is worth naming as a security-adjacent check rather than a performance one. A host-initiated .cache request is not a slow request; it is a request from an end that has no business raising it.

9. RTL 3 — The Type Is Derived, Not Declared

Consortium material gives the type table directly: Type 1 is .io + .cache, Type 2 adds .memory, Type 3 is .io + .memory. The engineering point is the direction of the implication.

module device_type_derive #(
  parameter bit TRUST_DECLARED = 1'b0   // 1 = believe a declared type field
) (
  input  logic       clk, rst_n,
  input  logic       en_io, en_cache, en_mem,
  input  logic [1:0] declared_type,     // what a config field claims
  input  logic       valid,
  output logic [1:0] derived_type, reported_type,
  output logic       type_disagree_err, io_absent_err
);
  logic [1:0] d;
 
  always_comb begin
    if      (en_io && en_cache && en_mem) d = 2'd2;   // Type 2
    else if (en_io && en_cache)           d = 2'd1;   // Type 1
    else if (en_io && en_mem)             d = 2'd3;   // Type 3
    else                                  d = 2'd0;   // io only: no CXL type
  end
 
  assign derived_type  = valid ? d : 2'd0;
  assign reported_type = valid ? (TRUST_DECLARED ? declared_type : d) : 2'd0;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      type_disagree_err <= 1'b0; io_absent_err <= 1'b0;
    end else if (valid) begin
      if (reported_type != derived_type) type_disagree_err <= 1'b1;
      if ((en_cache || en_mem) && !en_io) io_absent_err <= 1'b1;
    end
  end
endmodule

Usage examples are the Consortium's own: PGAS NIC and NIC atomics for Type 1; GP-GPU and dense computation for Type 2; memory bandwidth expansion, capacity expansion and storage-class memory for Type 3. Chapter 6.3 reverses this arrow — starting from a requirement and deriving the engine set.

  io cache mem | derived
  1  0     0   | io-only        <-- no CXL device type
  1  0     1   | Type 3
  1  1     0   | Type 1
  1  1     1   | Type 2

The io-only row has no CXL type, and that is not an omission in the table. A device with only .io holds no coherent copies and offers no system memory — it is a PCIe device on a CXL-capable link. The three types enumerate the coherent combinations, which is why there are three and not four.

10. RTL 4 — The Class Names the Engine, Not the Address

module class_engine_route #(
  parameter bit ROUTE_BY_ADDR = 1'b0    // 1 = route on address, ignoring class
) (
  input  logic       clk, rst_n, req_valid,
  input  logic [1:0] req_class,         // 0=.io 1=.cache 2=.mem
  input  logic       addr_is_dev_mem,
  input  logic [2:0] engines_present,   // {mem, cache, io}
  output logic       to_io_eng, to_cache_eng, to_mem_eng, refuse,
  output logic       class_lost_err, no_engine_err
);
  localparam logic [1:0] C_IO = 2'd0, C_CACHE = 2'd1, C_MEM = 2'd2;
  logic want_io, want_cache, want_mem, have;
 
  // Routing on the ADDRESS instead of the CLASS is the classic bug: a .cache
  // request whose address happens to fall in device memory gets handed to the
  // memory engine, which has no idea what coherence state to keep for it.
  assign want_io    = req_valid && (ROUTE_BY_ADDR ? (!addr_is_dev_mem && (req_class == C_IO))
                                                  : (req_class == C_IO));
  assign want_cache = req_valid && (ROUTE_BY_ADDR ? 1'b0 : (req_class == C_CACHE));
  assign want_mem   = req_valid && (ROUTE_BY_ADDR ? addr_is_dev_mem : (req_class == C_MEM));
 
  assign have = (want_io    && engines_present[0]) ||
                (want_cache && engines_present[1]) ||
                (want_mem   && engines_present[2]);
 
  assign to_io_eng    = want_io    && engines_present[0];
  assign to_cache_eng = want_cache && engines_present[1];
  assign to_mem_eng   = want_mem   && engines_present[2];
  assign refuse       = req_valid && !have;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      class_lost_err <= 1'b0; no_engine_err <= 1'b0;
    end else if (req_valid) begin
      if (to_cache_eng && (req_class != C_CACHE)) class_lost_err <= 1'b1;
      if (to_mem_eng   && (req_class != C_MEM))   class_lost_err <= 1'b1;
      if (to_io_eng    && (req_class != C_IO))    class_lost_err <= 1'b1;
      // A class with no engine must be refused, never silently dropped.
      if (!have && !refuse) no_engine_err <= 1'b1;
    end
  end
endmodule

  .cache req, addr in DEVICE memory:
    class-routed : io=0 cache=1 mem=0
    addr-routed  : io=0 cache=0 mem=1  <-- class lost
  .cache req with NO cache engine : refuse=1

A .cache request whose address happens to fall in device memory is still a .cache request. The address does not determine the protocol; the protocol determines what the address means and what state must be kept for it. Routing by address hands the request to a memory engine that keeps no coherence state for it — and the failure surfaces later, as stale data, far from the router.

Note the refuse path. A class with no engine must be refused, not dropped: mutation M10 removes the refusal and the request simply vanishes, which is the hardest failure of all to debug because nothing anywhere reports it.

11. RTL 5 — Per-Class Telemetry

module per_class_counters (
  input  logic        clk, rst_n, req_valid,
  input  logic [1:0]  req_class,
  input  logic        accepted, refused,
  output logic [15:0] n_offered_q, n_io_q, n_cache_q, n_mem_q, n_refused_q,
  output logic        accounting_err, both_outcomes_err
);
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_offered_q <= '0; n_io_q <= '0; n_cache_q <= '0;
      n_mem_q <= '0; n_refused_q <= '0;
      accounting_err <= 1'b0; both_outcomes_err <= 1'b0;
    end else begin
      if (req_valid) n_offered_q <= n_offered_q + 16'd1;
      if (req_valid && accepted) begin
        case (req_class)
          2'd0: n_io_q    <= n_io_q    + 16'd1;
          2'd1: n_cache_q <= n_cache_q + 16'd1;
          2'd2: n_mem_q   <= n_mem_q   + 16'd1;
          default: ;
        endcase
      end
      if (req_valid && refused) n_refused_q <= n_refused_q + 16'd1;
 
      // Every offered request is accepted into exactly one class, or refused.
      if (n_offered_q != n_io_q + n_cache_q + n_mem_q + n_refused_q)
        accounting_err <= 1'b1;
      if (req_valid && accepted && refused) both_outcomes_err <= 1'b1;
    end
  end
endmodule

  offered=60 io=20 cache=20 mem=0 refused=20
  conservation offered == io+cache+mem+refused : 60 == 60
  every count matched an independent reference   : ok
  .mem engine absent -> 20 requests refused, 0 serviced

A single merged traffic counter cannot answer the question that validates the architecture. "Is this device actually using .cache?" is the measurement that tells a Type 1 or Type 2 design whether the coherent engine it paid for is earning its area — and Chapter 6.4 makes that the basis of a design-review argument.

The conservation law is checked and every count is compared against an independent reference computed from the stimulus. Chapter 5.5 established why both are needed: conservation proves the parts are consistent with each other and cannot prove any part is correct.

12. Quantitative — What Each Protocol Costs

Public material gives one hard number: .cache and .mem target near-CPU-cache coherent latency, under 200 ns load-to-use. That single figure constrains everything else about the two coherent protocols, and it is worth working through what it implies.

At a 200 ns budget, the round trip must cover link traversal both ways, protocol processing at both ends, and the actual memory access — and every structure in §5's table sits somewhere in that path. A device cache lookup, a DCOH state check, a host Home Agent resolution and a memory controller access all have to fit inside the same envelope.

Set against .io, which has no latency target of that kind at all, the shape of the trade becomes clear:

The cost of a coherent protocol is not bandwidth, it is state and the latency budget that state must fit into. A design that budgets .cache as "some more queues" has mis-modelled it; the queues are cheap and the tracking structures, plus their timing closure inside a sub-200 ns loop, are not.

That framing is also why the combinatorics matter: a Type 2 device holds both a cache with its DCOH agent and a memory controller with host-managed memory, and both must meet the same latency envelope simultaneously — which is Chapter 6.2's subject and the reason Type 2 is the hardest class to build.

13. Assertions

Concurrent SVA execution: NOT SUPPORTED BY Icarus Verilog. Not executed; each maps to a procedural stand-in and a mutation.

// SAFETY -------------------------------------------------------------------
// V1 — nothing is backed whose state is not built.
a_backed_needs_state: assert property (@(posedge clk) disable iff (!rst_n)
  valid |-> ((backed & ~{mem_ok, cache_ok, io_ok}) == '0));
 
// V2 — a coherent protocol is never enabled without .io.
a_io_mandatory: assert property (@(posedge clk) disable iff (!rst_n)
  (en_cache || en_mem) |-> en_io);
 
// V3 — .cache requires BOTH device cache and host tracking.
a_cache_both_sides: assert property (@(posedge clk) disable iff (!rst_n)
  cache_ok |-> (has_dev_cache && host_has_home && host_tracks_dev));
 
// V4 — .mem does NOT depend on a device cache.
//      Stated as an independence property: the negative form catches a design
//      that over-couples the two coherent protocols.
a_mem_independent: assert property (@(posedge clk) disable iff (!rst_n)
  (en_mem && has_mem_ctrl && host_has_home) |-> mem_ok);
 
// V5 — .cache lands in host memory and is device-initiated.
a_cache_ownership: assert property (@(posedge clk) disable iff (!rst_n)
  (req_valid && (req_class == C_CACHE) && route_ok)
    |-> (tgt_is_host_mem && initiator_is_dev));
 
// V6 — .mem lands in device memory and is host-initiated.
a_mem_ownership: assert property (@(posedge clk) disable iff (!rst_n)
  (req_valid && (req_class == C_MEM) && route_ok)
    |-> (tgt_is_dev_mem && !initiator_is_dev));
 
// V7 — the engine reached matches the class asked for.
a_class_preserved: assert property (@(posedge clk) disable iff (!rst_n)
  to_cache_eng |-> (req_class == C_CACHE));
 
// V8 — a request with no engine is refused, never dropped.
a_no_silent_drop: assert property (@(posedge clk) disable iff (!rst_n)
  req_valid |-> (to_io_eng || to_cache_eng || to_mem_eng || refuse));
 
// V9 — the reported type equals the type the engines imply.
a_type_derived: assert property (@(posedge clk) disable iff (!rst_n)
  valid |-> (reported_type == derived_type));
 
// V10 — CONSERVATION: every offered request lands in one class or is refused.
a_class_conserved: assert property (@(posedge clk) disable iff (!rst_n)
  n_offered_q == n_io_q + n_cache_q + n_mem_q + n_refused_q);

V4 is the unusual one and is deliberately an independence property. V3 says .cache needs its structures; V4 says .mem must not acquire a dependency it does not have. Most verification plans check that things are required and never that things are not required — which is exactly the gap mutation M2 exploits.

14. Mutation Testing

Twelve mutations. Clean code restored after each.

12/12 killed, 0 escaped

Three escaped on the first run, in three different categories — and only two were testbench problems.

M2 was a missing check of an unusual kind: a missing independence check. The tests verified that removing the device cache kills .cache, and never verified that it leaves .mem standing. Positive requirements are natural to write; the assertion that something is not required is not, and it is where an over-coupled design hides.

M5 was an attribution failure in a positive test. The diagnostic wrong_initiator_err has two triggering conditions — host-initiated .cache and device-initiated .mem — and the positive test only asked whether the flag had ever fired. Deleting one condition left the other to satisfy it. The fix was to observe the flag immediately after the specific stimulus, before the second condition could run. A shared diagnostic needs per-condition observation, not an aggregate _ever latch.

M7 was an equivalent mutant — §9's callout explains why it was never killable, and it was replaced rather than chased.

15. Debug Lab

// The device has a cache, so .cache is supported.
assign cache_ok = en_cache && has_dev_cache;

  host cannot track : host_ok=0 backed=101  <-- .cache should have dropped
  advertised_unbacked_err=1

assign cache_ok = en_cache && has_dev_cache && host_has_home && host_tracks_dev;

// Coherent protocols need the coherence hardware.
assign mem_ok = en_mem && has_mem_ctrl && host_has_home && has_dev_cache;

  no device cache : backed=101   (correct: .mem survives)
  mutated design  : backed=001   (.mem lost with the cache)

assign mem_ok = en_mem && has_mem_ctrl && host_has_home;   // no cache dependency

// Device-memory addresses go to the memory engine.
assign to_mem_eng = req_valid && addr_is_dev_mem;

  .cache req, addr in DEVICE memory:
    class-routed : io=0 cache=1 mem=0
    addr-routed  : io=0 cache=0 mem=1  <-- class lost

assign to_cache_eng = req_valid && (req_class == C_CACHE) && engines_present[1];
assign to_mem_eng   = req_valid && (req_class == C_MEM)   && engines_present[2];
if (to_cache_eng && (req_class != C_CACHE)) class_lost_err <= 1'b1;

// The product configuration says what type this is.
assign reported_type = cfg_declared_type;

  io cache mem | derived  | trust-declared reports
  1  0     1   | Type 3   |  2
  type_disagree_err=1

// Derive it, and check any declared field against the derivation.
assign derived_type  = /* from en_io, en_cache, en_mem */;
assign reported_type = derived_type;
if (reported_type != derived_type) type_disagree_err <= 1'b1;

// Route to whichever engine matches.
assign to_cache_eng = req_valid && (req_class == C_CACHE) && engines_present[1];
// ...and if none matches, nothing happens.

  .cache req with NO cache engine : refuse=1   (correct)
  mutated design                  : refuse=0, no engine reached
  no_engine_err=1

assign refuse = req_valid && !(to_io_eng || to_cache_eng || to_mem_eng);
if (!have && !refuse) no_engine_err <= 1'b1;

// Count transactions.
if (req_valid && accepted) n_total_q <= n_total_q + 16'd1;

  offered=60 io=20 cache=20 mem=0 refused=20
  merged counter would report: 40 accepted, and nothing else

case (req_class)
  2'd0: n_io_q    <= n_io_q    + 16'd1;
  2'd1: n_cache_q <= n_cache_q + 16'd1;
  2'd2: n_mem_q   <= n_mem_q   + 16'd1;
endcase
if (req_valid && refused) n_refused_q <= n_refused_q + 16'd1;
if (n_offered_q != n_io_q + n_cache_q + n_mem_q + n_refused_q) accounting_err <= 1'b1;

16. Verification Plan

Rows 2 and 5 are this chapter's contributions to the standard plan. Row 2 catches over-coupling that every positive test misses; row 5 exists because a diagnostic with two triggering conditions cannot be validated by asking whether it ever fired.

17. Design Review

• For each advertised protocol, what state does it commit on each end, and did we check both?
• Does .mem in this design have any dependency on cache hardware? Why?
• Is the device type derived from the engines, or stored in a field that can drift?
• Does the router discriminate on protocol class or on address? What happens when they disagree?
• Can any request reach neither an engine nor a refusal?
• Can telemetry answer "how much .cache traffic did this part actually see"?
• Does anything in the design assume .cache and .mem impose the same host-side cost?
• What is the latency budget for the coherent path, and which structures sit inside it?

18. How This Appears in Real Engineering

The state obligation is discovered at floorplan, not at specification. "Support CXL.cache" costs a cache, a coherence agent and host-side tracking, and none of that is visible in the sentence. Teams that write the protocol list before sizing the structures re-plan.

Derivative parts break the coupling assumptions. Debug Labs 2 and 4 are both derivative-silicon bugs: the lead part has every engine, so an over-coupled dependency or a stale type constant is harmless until an engine is removed.

Address-versus-class routing is a Type 2 problem specifically. On Type 1 and Type 3 the two discriminators rarely disagree, so a router that uses the address survives. Type 2 is where a .cache request can name device memory, and that is where the design first meets the case it got wrong.

Per-class telemetry gets added after it is needed. The question "is .cache earning its area" arrives during the next part's architecture phase, and can only be answered from counters that shipped in the current one.

19. Common Misconceptions

20. Interview Reasoning

21. Exercises

1. Explain. For each of the three protocols, name what the host must build and what the device must build. Then state which structure, if removed, disables exactly one protocol and leaves the others intact.

2. Calculate. A .cache round trip must fit in a 200 ns load-to-use budget. Allocate that budget across link traversal (both directions), device cache lookup, DCOH state check, host Home Agent resolution and host memory access, stating your assumptions. Which term would you attack first, and what does that imply about where the coherent engine must sit on the die?

3. RTL task. Extend state_obligation so a protocol can be built but not enabled by policy, distinct from not built. State which existing diagnostic becomes ambiguous, and what new one is required.

4. DV task. Write the independence property that would have caught mutation M2, and explain why every positive "X requires Y" test in the plan missed it.

5. Debug task. A Type 2 device returns stale data only for addresses in device-attached memory, only under concurrent access. Give your investigation order and name the single trace that would settle it in one step.

6. Design review. A team proposes a Type 1 accelerator and reports that "asymmetric complexity means the device side is cheap". List the device-side structures they still owe, and the host-side ones their platform must have.

22. Summary

The three protocols are three contracts about who owns a resource and who must keep state about it.

• .io — control, mandatory for every device, no coherence state.
• .cache — the device holds copies of host memory; the device needs a cache and a DCOH agent, and the host must track what it holds.
• .mem — the host reaches device memory under its own management; the device needs a memory controller, the host its Home Agent, and no device cache is required.
• The structures for the two coherent protocols are disjoint on both sides — removing one leaves the other standing, and a design that couples them breaks on its first derivative part.
• The device type is derived from the engines, never declared. .io-only has no CXL type at all.
• The class determines what an address means, not the reverse — a router that discriminates on address loses exactly the information that says what state to keep.
• The cost of a coherent protocol is state and a sub-200 ns latency budget, not bandwidth.
• Verification lessons: assert independence as well as dependence, and a diagnostic with two triggering conditions needs per-condition observation.

Chapter 6.2 takes the next step and the hardest one in this module: these three protocols do not get a link each. They share one — and not in the way most descriptions suggest.