CXL 2.0 Architectural Changes

20.1 and 20.2 covered the two features CXL 2.0 is sold on. Neither of them is a change to the device.

The switch is a new component and pooling is a capability built out of it, but both rest on a set of changes to the adapter — the block inside every CXL device that terminates the link and decodes the address. A CXL 1.1 adapter cannot participate in either feature, and the reasons are specific: it has one HDM decoder, it assumes the host on the other end owns it, it treats link integrity as optional, it reports errors at device scope, and it expects to be present at boot.

This chapter is that list, made precise. It is the least glamorous chapter in Module 20 and the one a device team spends its schedule on.

1. The Engineering Problem — Two Generations Have To Coexist

A 2.0 device will be plugged into a 1.1 host and a 1.1 device into a 2.0 host, and both links have to come up. The rule is the lower of the two capabilities, and a negotiation that takes the more capable end's version brings up a link one end cannot drive. Section 5.

One decoder became several. A 1.1 adapter decodes one HDM range because it serves one host; a 2.0 adapter decodes one per logical device, and that is decoder state, not a mode bit. Section 6.

The host stopped owning the device. In 1.1 the device belongs to whatever it is wired to. In 2.0 a pooled device is assigned by a fabric manager, and a host that claims one directly is doing something the architecture no longer permits. Section 7.

Link integrity stopped being optional, and a MAC on every flit is bandwidth the throughput model has to have been carrying all along. Section 8.

Capacity can arrive after boot, which the address map has to have left room for. Section 9.

And an error has to name a logical device, because a device-scoped error on a pooled device tells every attached host that something they cannot identify has failed. Section 11.

This chapter against 20.1 and 20.2, stated precisely. Those own the switch and the capacity model. This one owns the adapter delta that makes a device eligible for either — and section 15 shows a device that negotiates 2.0 and has none of it.

2. The One-Sentence Model

A device is a CXL 2.0 device when it negotiates to the lower of the two ends' capabilities, decodes more than one range, accepts assignment from a fabric manager, carries link integrity inside its bandwidth budget, accepts capacity after boot, and reports errors per logical device — and every defect below is a device that advertises 2.0 and is missing one of them.

3. What This Chapter Owns

Deferred:

4. Teaching-Model Boundary

Every model here is a small synchronous block isolating one property. A real adapter is a link layer, a transaction layer, a decoder array, a coherence engine and a management interface, and none of it is reproduced. Each model captures the arithmetic or the decision that changed between generations.

Each is built twice, and the broken build in this chapter is almost always "the 1.1 behaviour, carried forward" — not a mistake anybody made, but a correct design that stopped being correct when the surrounding architecture changed underneath it.

Figure 1 — Three added blocks and one dashed edge. The dashed path is section 15's feature-list definition: a device that negotiates 2.0 on the link and reaches the label without the decoder array, the management port or the integrity block.

5. RTL 1 — Negotiation Takes The Lower End

// RTL 1 - backward compatibility. A 2.0 device must work with a 1.1 host and a
// 1.1 device must work with a 2.0 host, at the lower of the two capabilities.
module version_negotiate #(parameter int ASSUME_PEER = 0) (
  input  logic clk, rst_n,
  input  logic       negotiate,
  input  logic [3:0] host_ver, device_ver,   // 11 = CXL 1.1, 12 = CXL 2.0
  output logic [3:0] agreed_ver,
  output logic       compatible, downgraded,
  output logic [7:0] n_links, n_downgraded,
  output logic       overclaim_err
);
  logic [3:0] lower;
  assign lower = (host_ver < device_ver) ? host_ver : device_ver;
  logic [3:0] higher;
  assign higher = (host_ver > device_ver) ? host_ver : device_ver;
  // A link runs at the lower of the two capabilities. The assuming build takes
  // the more capable end's version and expects the other to keep up, which is
  // how a negotiation written as "use the newest features available" behaves.
  assign agreed_ver = (ASSUME_PEER != 0) ? higher : lower;
  assign compatible = (agreed_ver <= host_ver) && (agreed_ver <= device_ver);
  assign downgraded = (agreed_ver < host_ver) || (agreed_ver < device_ver);
  // A link brought up at a version one of the two ends does not support.
  assign overclaim_err = negotiate && !compatible;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_links <= 8'd0; n_downgraded <= 8'd0;
    end else if (negotiate) begin
      n_links <= n_links + 8'd1;
      if (downgraded) n_downgraded <= n_downgraded + 8'd1;
    end
  end
endmodule

Four links.

Two downgrades, and the assuming build overclaimed twice.

compatible has two halves and both of them fire, in opposite rows. Row two overclaims against the device; row three overclaims against the host. Section 18 records that the host half was dead until the broken build was changed from "take my own version" to "take the more capable end's" — because a build that takes its own version can never exceed its own version, which made half the check unreachable.

Downgrading is not a failure. Rows two and three are correct behaviour and the whole point of the compatibility requirement: a 2.0 host with a 1.1 device gets a working 1.1 link, and every feature in this chapter is simply unavailable on it. A fleet mid-migration is mostly rows two and three, which is section 14.

6. RTL 2 — One Decoder Became Several

// RTL 2 - the HDM decoder went from one range to many. A 1.1 device decoded one
// window; a 2.0 device decodes several, one per logical device.
module decoder_count #(parameter int SINGLE_RANGE = 0) (
  input  logic clk, rst_n,
  input  logic       program_req,
  input  logic [3:0] ranges_wanted, ranges_supported,
  input  logic [3:0] range_index,
  output logic       count_ok, index_ok, may_program,
  output logic [3:0] ranges_unused,
  output logic [7:0] n_programs, n_refused,
  output logic       over_decode_err
);
  logic [3:0] effective_support;
  // A 1.1 adapter has one decoder however many the fabric manager asks for.
  assign effective_support = (SINGLE_RANGE != 0) ? 4'd1 : ranges_supported;
  assign count_ok = (ranges_wanted <= effective_support);
  assign index_ok = (range_index < ranges_wanted);
  assign may_program = program_req && count_ok && index_ok;
  assign ranges_unused = (ranges_wanted >= effective_support) ? 4'd0
                                                             : (effective_support - ranges_wanted);
  // Programming a decoder range the adapter does not have.
  assign over_decode_err = may_program && !count_ok;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_programs <= 8'd0; n_refused <= 8'd0;
    end else if (program_req) begin
      n_programs <= n_programs + 8'd1;
      if (!may_program) n_refused <= n_refused + 8'd1;
    end
  end
endmodule

An adapter supporting eight ranges.

Two refusals against four.

The index is checked against the requested count, not the supported count, and the difference matters. An adapter supporting eight ranges configured for four has index 3 as its last valid one — index 4 through 7 name decoders that exist in silicon and hold nothing. Checking the index against ranges_supported would accept a program into a decoder the fabric manager has not allocated, which is 20.1 section 5's overlapping-range failure reached by a different route.

Row five is the interoperability case, and it is why a 1.1 adapter is usable at all in a 2.0 fleet: asked for one range, it has one. It just cannot be pooled, because pooling means more than one host and therefore more than one range.

Why the broken build is not a strawman. One decoder is not a limitation of a 1.1 adapter; it is a correct design for a device with one host. The decoder array is real silicon — base, size and logical-device identifier per range, read on every access — and a device team that did not need it did not build it.

The count is a product decision with a long tail. A device supporting four ranges can be pooled between four hosts and not five, and that number is fixed at tapeout while the fleet it will live in is not. Section 12 prices the array; what section 12 cannot price is the deployment three years later that needs one more host than the silicon can present. This is the same shape as 20.1's port count — a topology ceiling chosen by a component designer and discovered by somebody else.

The ranges_unused output exists for that conversation. An adapter supporting eight ranges and configured for four carries four decoders' worth of comparators on every access for nothing, which is a power number as well as an area one. Over-provisioning the array is not free either, and the counter is what makes the trade visible in a deployed fleet rather than argued from a spreadsheet at design time.

7. RTL 3 — The Host Stopped Owning The Device

// RTL 3 - the fabric manager became a required component. In 1.1 the host owned
// the device; in 2.0 something outside both assigns it.
module ownership_model #(parameter int HOST_OWNS = 0) (
  input  logic clk, rst_n,
  input  logic       claim,
  input  logic [1:0] claimant,      // 0 host, 1 another host, 2 fabric manager
  input  logic       device_pooled,
  output logic       claim_valid, needs_fm,
  output logic [7:0] n_claims, n_refused,
  output logic       unmanaged_claim_err
);
  // A pooled device is assigned by the fabric manager. A directly attached one
  // belongs to the host it is wired to.
  assign needs_fm = device_pooled;
  assign claim_valid = (HOST_OWNS != 0) ? (claimant != 2'd1)
                     : (needs_fm ? (claimant == 2'd2) : (claimant == 2'd0));
  // A pooled device claimed by a host rather than assigned by the fabric manager.
  assign unmanaged_claim_err = claim && claim_valid && needs_fm && (claimant != 2'd2);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_claims <= 8'd0; n_refused <= 8'd0;
    end else if (claim) begin
      n_claims <= n_claims + 8'd1;
      if (!claim_valid) n_refused <= n_refused + 8'd1;
    end
  end
endmodule

One unmanaged claim.

Row three is why the model has a device_pooled input at all. Direct attachment did not go away in CXL 2.0 — a device wired to one host still belongs to that host, and a check written as "only the fabric manager may claim anything" breaks every 1.1-style attachment in the fleet. The ownership rule is a property of the topology, not of the protocol version.

The unmanaged_claim_err condition is gated on needs_fm for the same reason. A host claiming a device it is wired to is not an escalation; a host claiming a pooled device is.

8. RTL 4 — Integrity Stopped Being Optional

// RTL 4 - link integrity became mandatory rather than optional, and the flit
// budget has to absorb it.
module flit_overhead #(parameter int IGNORE_MAC = 0) (
  input  logic clk, rst_n,
  input  logic        sample,
  input  logic [15:0] payload_b, mac_b, header_b, link_gbps,
  output logic [15:0] on_wire_b, useful_gbps, overhead_pct,
  output logic        meets_target,
  input  logic [15:0] target_gbps,
  output logic [7:0]  n_samples, n_missed,
  output logic        target_miss_err
);
  logic [31:0] u_q, o_q;
  // The MAC is on the wire whether the model counts it or not.
  assign on_wire_b = (IGNORE_MAC != 0) ? (payload_b + header_b)
                                       : (payload_b + header_b + mac_b);
  assign u_q = (on_wire_b == 16'd0) ? 32'd0
                                    : (({16'd0, link_gbps} * {16'd0, payload_b}) / {16'd0, on_wire_b});
  assign useful_gbps = (u_q > 32'd65535) ? 16'hFFFF : u_q[15:0];
  assign o_q = (on_wire_b == 16'd0) ? 32'd0
                                    : ((({16'd0, on_wire_b} - {16'd0, payload_b}) * 32'd100) / {16'd0, on_wire_b});
  assign overhead_pct = (o_q > 32'd65535) ? 16'hFFFF : o_q[15:0];
  assign meets_target = (useful_gbps >= target_gbps);
  assign target_miss_err = sample && !meets_target;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_samples <= 8'd0; n_missed <= 8'd0;
    end else if (sample) begin
      n_samples <= n_samples + 8'd1;
      if (!meets_target) n_missed <= n_missed + 8'd1;
    end
  end
endmodule

A 400 Gbps link, a 16-byte MAC and an 8-byte header, a 280 Gbps target.

Two misses.

The overhead is a function of payload size and nothing else. At 256 bytes the fixed 24 bytes cost 8%; at 16 bytes they cost 60%. That is the same argument 19.2 made about security overhead, arriving here as an architectural requirement rather than a policy choice — in 1.1 a device could decline to pay it, and in 2.0 it cannot.

Row four is what "optional" meant in practice. A 1.1 device with integrity disabled runs at 355 Gbps of useful bandwidth where the same device with a mandatory MAC runs at 290. The 65 Gbps difference is not a regression anybody introduced; it is a capability that stopped being optional, and a throughput model carried forward from a 1.1 datasheet is wrong by 22%.

Row six is the guard, and it earns its place. A flit with no payload, no header and no MAC is not a physical thing; it is what a model produces when a configuration register reads back zero, which happens during bring-up more often than anybody plans for. Without the guard the useful-rate division is by zero, and the number reaching a performance dashboard is an X or a 65,535 — either of which somebody will screenshot.

The interaction with 20.1 is worth stating. That chapter's switch adds latency on every crossing and this chapter's MAC adds bytes to every flit, and a pooled deployment pays both. A 64-byte access through a switch, with integrity, is 290 Gbps of a 400 Gbps link at 300 ns instead of 200 — and neither penalty appears in a component datasheet, because each component reports its own number correctly.

9. RTL 5 — Capacity Can Arrive After Boot

// RTL 5 - hot-plug. A 1.1 device is present at boot; a 2.0 device can appear and
// disappear under a running host, which the address map has to survive.
module hotplug_capable #(parameter int BOOT_ONLY = 0) (
  input  logic clk, rst_n,
  input  logic       event_valid,
  input  logic       at_boot, map_reserved, host_supports_hotplug,
  output logic       can_accept, map_ok,
  output logic [7:0] n_events, n_rejected,
  output logic       map_break_err
);
  // Capacity appearing after boot needs address space that was reserved for it
  // and a host that knows what to do with the notification.
  assign map_ok = at_boot || map_reserved;
  assign can_accept = (BOOT_ONLY != 0) ? at_boot
                                       : (map_ok && (at_boot || host_supports_hotplug));
  // Capacity accepted after boot with no address space reserved for it.
  assign map_break_err = event_valid && can_accept && !map_ok;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_events <= 8'd0; n_rejected <= 8'd0;
    end else if (event_valid) begin
      n_events <= n_events + 8'd1;
      if (!can_accept) n_rejected <= n_rejected + 8'd1;
    end
  end
endmodule

Two refusals against three.

Row five is the case that keeps map_ok honest. Capacity present at boot needs no reservation, because the address map is constructed around what is there. The reservation requirement applies only to capacity that arrives later, which is exactly what at_boot || map_reserved says — and section 18 records that the at_boot half was untested until this row was driven.

Row four is the half nobody controls. The map is fine and the device is willing, and a host whose firmware and operating system do not understand a capacity-arrival notification cannot use it. Hot-plug is a three-party agreement, and the device is only one of the three.

10. Waveform — A 2.0 Device Meeting A 1.1 Host

That waveform is a success. A 2.0 device met a 1.1 host and produced a working link with none of the 2.0 features on it — which is the interoperability requirement being met, not a degradation. The failure is only in the assumed row, where a negotiation that reaches for the newer capability produces a link one end cannot terminate.

11. RTL 6 — An Error Has To Name A Logical Device

// RTL 6 - error reporting moved from a device-level event to a per-logical-device
// one, and a 1.1 error path cannot express which logical device failed.
module error_scope #(parameter int DEVICE_SCOPE = 0) (
  input  logic clk, rst_n,
  input  logic       err,
  input  logic [3:0] failing_ld, ld_count,
  output logic [3:0] hosts_notified,
  output logic       scoped, ld_valid,
  output logic [7:0] n_errors, n_overnotified,
  output logic       overnotify_err
);
  assign ld_valid = (failing_ld < ld_count);
  // A 1.1 error path has one scope: the device. A 2.0 path names the logical
  // device, so only its host is told.
  assign scoped = (DEVICE_SCOPE == 0) && ld_valid;
  assign hosts_notified = scoped ? 4'd1 : ld_count;
  // Telling hosts about a failure in a logical device that is not theirs.
  assign overnotify_err = err && (hosts_notified > 4'd1);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_errors <= 8'd0; n_overnotified <= 8'd0;
    end else if (err) begin
      n_errors <= n_errors + 8'd1;
      if (hosts_notified > 4'd1) n_overnotified <= n_overnotified + 8'd1;
    end
  end
endmodule

Five errors on a device with eight logical devices.

Two overnotifications by the scoped build against four.

Row two is why 1.1 never needed this. A device with one logical device tells one host either way, so device scope and logical-device scope are the same thing — and the 1.1 error path was correct until the day a second logical device existed.

Rows three and five are the identifier boundary from both sides. An identifier of 9 against a count of 8 is obviously invalid; an identifier of exactly 8 against a count of 8 is the off-by-one, and it is the one that survives review. The scoped build overnotifies on both, which is the right failure mode: an error it cannot attribute is an error everybody has to hear about.

This is 20.1 section 13 one level down. That chapter kept the source port across a switch; this one keeps the logical device inside a device. Both are the same requirement — an error must arrive with the identity of what produced it — and both are cheapest to violate.

12. RTL 7 — What The Delta Costs In Silicon

// RTL 7 - the adapter cost of 2.0. Every added capability is silicon, and the
// question at design time is what a 1.1 adapter has to grow by.
module adapter_area #(parameter int REUSE_11 = 0) (
  input  logic clk, rst_n,
  input  logic        estimate,
  input  logic [15:0] base_kgate, decoder_kgate, integrity_kgate, mgmt_kgate,
  output logic [15:0] total_kgate, added_kgate, growth_pct,
  output logic        within_budget,
  input  logic [15:0] budget_kgate,
  output logic [7:0]  n_estimates, n_over,
  output logic        budget_err
);
  logic [31:0] g_q;
  // A 1.1 adapter reused unchanged gets none of the 2.0 capabilities. Costing it
  // as if it did is how a 2.0 programme gets approved on 1.1 silicon.
  assign added_kgate = (REUSE_11 != 0) ? 16'd0
                                       : (decoder_kgate + integrity_kgate + mgmt_kgate);
  assign total_kgate = base_kgate + added_kgate;
  assign g_q = (base_kgate == 16'd0) ? 32'd0
                                     : (({16'd0, added_kgate} * 32'd100) / {16'd0, base_kgate});
  assign growth_pct = (g_q > 32'd65535) ? 16'hFFFF : g_q[15:0];
  assign within_budget = (total_kgate <= budget_kgate);
  assign budget_err = estimate && !within_budget;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_estimates <= 8'd0; n_over <= 8'd0;
    end else if (estimate) begin
      n_estimates <= n_estimates + 8'd1;
      if (!within_budget) n_over <= n_over + 8'd1;
    end
  end
endmodule

A 400 kgate 1.1 adapter, a 650 kgate budget. Decoders 60, integrity 90, management 50.

One over budget.

A 50% area growth on the adapter is the honest number for a 1.1 design becoming 2.0, and it is why "we will support 2.0 in firmware" is not an available answer. The decoder array is registers, the integrity block is a datapath, and the management port is a second interface with its own state — none of the three is software.

Row four is the alternative that usually wins. Designing a new 2.0 adapter rather than growing a 1.1 one costs 200 kgates of new blocks and no growth percentage at all, because there is no baseline to grow. The 50% figure is a property of the migration, not of the architecture.

Figure 3 — Which end is scarcer changes as a migration proceeds, and the usable fraction follows it. Buying 2.0 devices for a fleet of 1.1 hosts moves the lower node and not the answer.

13. RTL 8 — Advertising What You Have

// RTL 8 - a capability a device claims and does not implement is worse than one
// it does not claim, because software will use it.
module capability_honesty #(parameter int CLAIM_ALL = 0) (
  input  logic clk, rst_n,
  input  logic       probe,
  input  logic [3:0] implemented_mask, advertised_mask,
  input  logic [1:0] feature,
  output logic [3:0] effective_mask, phantom_mask,
  output logic       implemented, advertised,
  output logic [7:0] n_probes, n_phantom,
  output logic       phantom_capability_err
);
  logic [3:0] fbit;
  assign fbit = 4'b0001 << feature;
  // A device must advertise only what it implements. The claiming build
  // advertises the full 2.0 feature set regardless.
  assign effective_mask = (CLAIM_ALL != 0) ? 4'b1111 : advertised_mask;
  assign implemented = |(implemented_mask & fbit);
  assign advertised  = |(effective_mask & fbit);
  // Every bit advertised and not implemented.
  assign phantom_mask = effective_mask & ~implemented_mask;
  // Software probing a feature it will be told exists and that does not.
  assign phantom_capability_err = probe && advertised && !implemented;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_probes <= 8'd0; n_phantom <= 8'd0;
    end else if (probe) begin
      n_probes <= n_probes + 8'd1;
      if (advertised && !implemented) n_phantom <= n_phantom + 8'd1;
    end
  end
endmodule

A device implementing features 0 and 1.

The claiming build advertises 1111 throughout, giving a phantom mask of 1100 and two phantom probes.

Row three is safe and wasteful, and worth naming because it is the opposite error. A device that implements more than it advertises has spent silicon nobody will use — an efficiency problem with no correctness consequence. Row four is the dangerous direction: software will read the capability register, conclude the feature exists, and use it.

The asymmetry is the whole model. Under-advertising costs money. Over-advertising costs correctness, in a place — driver initialisation, months after the device shipped — where the failure is attributed to the software.

14. RTL 9 — A Fleet Migrates In Pieces

// RTL 9 - the migration. A fleet does not replace every device at once, so a
// 2.0 capability is only usable where every element of the path supports it.
module fleet_migration #(parameter int ASSUME_UNIFORM = 0) (
  input  logic clk, rst_n,
  input  logic       plan,
  input  logic [7:0] hosts_2p0, hosts_total, devices_2p0, devices_total,
  output logic [7:0] usable_pct,
  output logic       path_capable, uniform,
  output logic [7:0] n_plans, n_partial,
  output logic       assume_uniform_err
);
  logic [15:0] u_q;
  logic [7:0] limiting;
  // A path is 2.0-capable only if both ends are, so the usable fraction is
  // bounded by the scarcer of the two.
  assign limiting = (hosts_2p0 < devices_2p0) ? hosts_2p0 : devices_2p0;
  assign u_q = (hosts_total == 8'd0) ? 16'd0
                                     : (({8'd0, limiting} * 16'd100) / {8'd0, hosts_total});
  assign usable_pct = (ASSUME_UNIFORM != 0) ? 8'd100
                    : ((u_q > 16'd255) ? 8'hFF : u_q[7:0]);
  assign uniform = (hosts_2p0 == hosts_total) && (devices_2p0 == devices_total);
  assign path_capable = (limiting != 8'd0);
  // A plan that counts every path as 2.0-capable when the fleet is mixed.
  assign assume_uniform_err = plan && (usable_pct == 8'd100) && !uniform;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_plans <= 8'd0; n_partial <= 8'd0;
    end else if (plan) begin
      n_plans <= n_plans + 8'd1;
      if (!uniform) n_partial <= n_partial + 8'd1;
    end
  end
endmodule

A fleet of 100 hosts and 100 devices.

Five mixed plans, and the assuming plan was wrong on all five.

The scarcer end bounds it, and which end that is changes. Rows one and five are device-limited and host-limited respectively, and the answer swaps without anything about the fleet's size changing. Buying 2.0 devices for a fleet of 1.1 hosts buys nothing, and the same is true in reverse — which is the migration-sequencing question every platform team has to answer and few do explicitly.

Row three is the only correct 100%, and it requires both ends fully migrated. Section 18 records that the uniformity check's two halves were untestable until rows five and six were driven — every earlier case had both ends incomplete.

Row four is the state a programme starts in, and the one the assuming plan is most wrong about. Nothing migrated, no capable path, and a plan reporting full usability because it never looked. That is not a hypothetical: a capability plan written before any hardware arrives has exactly those inputs, and the number it produces gets quoted for the life of the programme.

The path_capable output separates two different zeros. A fleet at 0% because nothing has been migrated is a programme that has not started. A fleet at 0% because one end migrated and the other did not is a programme that has spent money and bought nothing. The percentage is identical and the conversation is not.

Figure 4 — A procurement process stops at the first decision. The right-hand terminal is section 22's device: honest about its link version and wrong about everything the link version was taken to imply.

15. RTL 10 — The 2.0 Delta Assembled

// RTL 10 - the 2.0 delta assembled. Everything a 1.1 device has to become.
module architecture_delta #(parameter int FEATURE_LIST_ONLY = 0) (
  input  logic clk, rst_n,
  input  logic       evaluate,
  input  logic       version_negotiated,  // the link runs at the lower capability
  input  logic       decoders_multiple,   // more than one HDM range
  input  logic       fm_owns_assignment,  // the fabric manager, not the host
  input  logic       integrity_budgeted,  // the MAC is in the bandwidth model
  input  logic       hotplug_supported,   // capacity may arrive after boot
  input  logic       errors_per_ld,       // an error names a logical device
  output logic       is_2p0,
  output logic [5:0] fail_mask,
  output logic [7:0] n_eval, n_2p0,
  output logic       false_2p0_err
);
  assign fail_mask[0] = ~version_negotiated;
  assign fail_mask[1] = ~decoders_multiple;
  assign fail_mask[2] = ~fm_owns_assignment;
  assign fail_mask[3] = ~integrity_budgeted;
  assign fail_mask[4] = ~hotplug_supported;
  assign fail_mask[5] = ~errors_per_ld;
  // The feature-list build checks that the device advertises 2.0 and calls it a
  // 2.0 device, which is what a datasheet comparison does.
  assign is_2p0 = (FEATURE_LIST_ONLY != 0) ? version_negotiated : (fail_mask == 6'd0);
  assign false_2p0_err = evaluate && is_2p0 && (fail_mask != 6'd0);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_eval <= 8'd0; n_2p0 <= 8'd0;
    end else if (evaluate) begin
      n_eval <= n_eval + 8'd1;
      if (is_2p0) n_2p0 <= n_2p0 + 8'd1;
    end
  end
endmodule

One of six, and four false claims.

The feature-list definition is a datasheet comparison, and it is the one a procurement process runs. The device says 2.0; the datasheet says 2.0; the box says 2.0. Row two is that device with one HDM decoder, which cannot be pooled, which means 20.2 is unavailable on it — and nothing in the negotiation, the datasheet or the box says so.

16. Quantitative Reasoning

Negotiation. Four links, two downgraded and both correctly. The assuming build overclaimed twice, in opposite directions — once against a 1.1 device and once against a 1.1 host.

Decoders. An eight-range adapter configured for four has index 3 as its last valid one, and four of its decoders hold nothing. A 1.1 adapter refused four of five programming requests and accepted the one asking for a single range.

Ownership. Four claims, two refused by the 2.0 model against one by the 1.1 model, and the difference is a host claiming a pooled device — permitted by the older rule, forbidden by the newer.

Integrity. 64 bytes of payload behind 24 bytes of header and MAC: 88 on the wire, 290 Gbps useful of 400, a 27% overhead. At 16 bytes it is 60%; at 256 bytes, 8%. With integrity disabled — the 1.1 default — the same link delivers 355 Gbps, so a throughput model inherited from 1.1 is 22% optimistic.

Hot-plug. Five events, two refused by the 2.0 build and three by the boot-only one. The refusals are an unreserved map and a host that cannot be told.

Error scope. Five errors on eight logical devices. The scoped build told one host three times and eight hosts twice — both on invalid identifiers. The device-scope build told eight hosts four times of five.

Area. A 400 kgate 1.1 adapter grows by 200 kgates, 50%, and a heavier integrity block takes it to 310 and 78%, past a 650 kgate budget. A clean-sheet 2.0 adapter is 200 kgates with no growth to report.

Capabilities. A device implementing two of four features and advertising all four has a phantom mask of 1100 and produced two phantom probes of four.

Migration. 50 of 100 hosts and 80 of 100 devices gives 50% of paths usable; 90 hosts and 30 devices gives 30%. The assuming plan claimed 100% on five of six.

The assembled model. Six properties, six configurations, one 2.0 device. The feature-list definition reported five.

17. Assertions

Every check is an explicit comparison against an exact value. Icarus Verilog 13.0 has no concurrent assertion support, so each is a procedural comparison against 1'b1, and every one is an equality.

Negotiation. Both overclaim directions are asserted, which is what makes both halves of the compatibility check live.

chk(vAg == 4'd11, "a 1.1 host also brings the link to 1.1");
chk(aAg == 4'd12, "the assuming build takes the more capable end");
chk(aCo == 1'b0,  "which the 1.1 host does not support");
chk(aOe == 1'b1,  "so it overclaims in this direction too");

Decoders. The index boundary is asserted at exactly the requested count.

chk(dCo == 1'b1, "four ranges is within eight");
chk(dIo == 1'b0, "but index 4 is one past the last of four");

Ownership. A direct device claimed by its own host is asserted valid, which is what stops a rule written as "only the fabric manager".

Integrity. The exact-target boundary and the empty flit are both driven.

chk(fUg == 16'd280, "exactly 280 Gbps useful");
chk(fMt == 1'b1,    "which meets a 280 Gbps target exactly");
chk(fOw == 16'd0,  "an empty flit is nothing on the wire");

Hot-plug. Capacity at boot with nothing reserved is asserted acceptable, which is the half of map_ok a post-boot-only stimulus never reaches.

chk(pMo == 1'b1, "capacity present at boot needs no reservation");

Error scope. The identifier boundary is asserted from both sides, and the single-logical-device case is asserted identical in both builds.

chk(sLv == 1'b0, "identifier 8 is one past the last of eight");
chk(gOe == 1'b0, "so neither overnotifies");
chk(sOe == 1'b0, "which is why 1.1 never needed this");

Area. The exact budget is driven, and the zero-base case is asserted to report no growth.

Capabilities. Implemented-but-not-advertised is asserted to produce no phantom, separating the safe direction from the dangerous one.

Migration. Each end uniform alone is asserted non-uniform, which is what makes both halves of the check live.

chk(jUn == 1'b0, "and the fleet is not uniform, whatever the hosts have done");

The assembled model. Every fail mask is asserted as an exact six-bit value, including the hot-plug bit alone.

Totals: 240 checks across two testbenches, 119 on the front five models and 121 on the back five, all passing on the unmutated sources.

18. Mutation Testing

Forty-five mutations were injected one at a time.

45 injected, 45 killed, after eight survivors were diagnosed — the largest survivor set in the batch, and the diagnoses fall into three groups.

Group one: a dead half of a compound condition, which was a design problem. The mutation that reduced compatible to its device half survived because the broken build took its own version, which can never exceed its own version — making the host half unreachable by construction. The fix was not more stimulus; it was a better broken build. A negotiation that takes the more capable end's version is both more realistic — it is how "use the newest available features" is naturally written — and makes both halves live. A survivor that cannot be killed by any stimulus is a survivor telling you the model is wrong.

Group two: five boundaries and halves the stimulus never reached. The exact-target bandwidth, the empty flit, capacity at boot with nothing reserved, an identifier exactly equal to the logical-device count, and each end of the fleet uniform alone. All five are guards or comparisons that were correct throughout and invisible. The last one is worth naming: every migration case in the testbench had both ends incomplete, so uniform was false for the same reason twice and neither half of it was ever the deciding term.

Group three: a mask bit never set. The hot-plug bit of the assembled model survived being tied to zero, because no configuration in the testbench had hotplug_supported low. Five of six bits were exercised and the sixth was not. An assembled model needs each of its inputs driven false alone, and counting the configurations is not the same as covering the bits.

19. Verification Strategy

What a testbench for a real adapter must cover.

Every mask bit driven false alone. Section 18's third group. A six-bit fail mask needs six single-bit configurations plus the all-clear, and a testbench that drives interesting combinations can still leave a bit untouched.

Both halves of every compound condition, with a broken build that can exercise both. The compatibility check is the case worth generalising: when a mutation cannot be killed by any stimulus, check whether the parameterised alternative can even produce the failing input. A broken build that is too weak makes half the correct build untestable.

Both versions in both roles. A 2.0 device with a 1.1 host and a 1.1 device with a 2.0 host are different tests, and the asymmetry is exactly where the negotiation logic is wrong.

Every identifier boundary from both sides. The decoder index against the requested count, the failing logical device against the logical-device count. Both are < comparisons where <= is the natural mistake, and both need the exactly-equal case driven.

The cases that are correct and look like failures. A downgraded link. A 1.1 adapter refusing a four-range program. A device implementing more than it advertises. A device-scoped error on a device with one logical device. Each trips a naive checker.

What a real adapter needs that these models do not have. Concurrency — a decoder reprogrammed while the link is passing traffic. Ordering — a hot-add notification racing a link retrain. Reset semantics — what survives a link down, and what a fabric manager has to reassert afterwards. Each is where a generation transition actually breaks, and none is visible in combinational logic.

20. Synthesis and Implementation Reality

The decoder array is registers on a critical path. Each range is a base, a size and a logical-device identifier, compared in parallel on every access. Eight ranges is eight comparator pairs plus a priority encoder, and section 12's 60 kgates is mostly the comparison logic rather than the storage.

The integrity block is a datapath with a latency, not just an area. A MAC computed over every flit adds pipeline stages in both directions, which is part of 20.1 section 7's per-crossing latency and part of the device's own access latency. It is the one 2.0 addition that costs bandwidth, area and latency simultaneously.

The management port is a second interface with its own reset domain. It has to remain reachable when the primary link is down, because that is precisely when a fabric manager needs to reassign the device — which means separate clocking, separate reset and separate power, and it is why section 12 costs 50 kgates for what looks like a register file.

Per-logical-device error reporting is a fan-in problem. Sixteen logical devices, each with its own error state, feeding an interface that presents them to sixteen different hosts. The cheap implementation aggregates and loses the attribution, which is section 11's broken build arriving as an area decision rather than a design decision.

Reset semantics are the part nobody scopes. A link that goes down and retrains has to come back with its decoder state intact, or the fabric manager has to reassert every assignment — and which of those it is decides whether a link blip is invisible or a fleet-wide reconfiguration event. In 1.1 the question barely existed, because the host that owned the device was also the host that would notice. In 2.0 the owner of the state and the party that observes the outage are different components entirely.

None of this is firmware. The recurring answer in a 1.1-to-2.0 conversation is that the delta will be handled in the management processor, and every item above is a counter-example. The 50% area growth in section 12 is the real answer to "can we support 2.0 on this silicon".

21. Silicon Observability

The unattributable-error counter is the one to insist on. Errors per logical device is the obvious ask and it is not sufficient: an error the device could not attribute to a logical device does not appear in any per-logical-device counter, and without a separate bucket it is simply absent. A telemetry system where the failures it cannot classify are invisible will report perfect classification.

22. Debug Lab

Symptom. A newly purchased CXL 2.0 memory device is installed behind a switch in a pooled configuration. The fabric manager reports the device present and healthy. Attempts to assign capacity to a second host fail with a generic error. The first host works perfectly.

Step 1 — is the link at 2.0? Read the negotiated version on both the switch's downstream port and the device. Both report 2.0. The link is not the problem, and section 5 is not the chapter.

Step 2 — is it a fabric-manager problem? Assignment to the first host succeeded through the same path minutes earlier, and re-running it succeeds again. The management path works.

Step 3 — read the decoder configuration. The device reports ranges supported: 1. The first host's assignment consumed it. The second assignment has no decoder to program, and section 6 is the chapter.

Step 4 — but it advertises 2.0. Read the capability mask against the implemented mask. The device advertises multi-logical-device support and implements one HDM decoder. That is section 13's phantom capability, and it is why the fabric manager tried at all — it read the capability, concluded the device could be pooled, and issued an assignment the hardware cannot accept.

Step 5 — what is the device, actually? A CXL 1.1 adapter with a 2.0 link layer. It negotiates 2.0, carries integrity, and has one decoder, host-scoped errors and no fabric-manager-controlled assignment beyond the first. Section 15's row two, in a rack.

The finding. Not a fault, and not exactly a lie either — the device does implement part of CXL 2.0. It advertises a capability it does not have, and every layer above took the advertisement at face value because there is no other source of truth.

The fix. Use the device as a directly attached expander, where its one decoder is sufficient and its 2.0 link layer buys integrity. Then, for procurement: stop qualifying devices on the negotiated version. Section 15's six properties are each independently observable through the counters in section 21, and a bring-up script that reads all six takes an afternoon to write.

What made this hard. Every layer reported success at its own level. The link negotiated 2.0 correctly. The fabric manager issued a correct command. The device correctly refused a program it had no decoder for. The only incorrect statement in the whole chain was a bit in a capability register.

23. Design Review

1. Does the negotiation take the lower of the two advertised versions, and is that tested with each version in each role? Section 5, and both directions.

2. How many HDM decoder ranges does the adapter have? Not how many the specification permits. Section 6.

3. Is the decoder index checked against the configured count or the supported count? Two different checks with the same shape. Section 6.

4. Can a host claim a pooled device directly? And is direct attachment still permitted for devices that are wired that way? Section 7.

5. Is the throughput budget written with the MAC in it? If it came from a 1.1 datasheet, it is 22% optimistic. Section 8.

6. What address space is reserved for capacity that arrives after boot, and who reserved it? Section 9, and the host is the answer to the second half.

7. Does an error name a logical device, and where does an unattributable error go? The second half is section 21's insistence.

8. What is the adapter's area growth from 1.1, and has anyone costed the alternative of a clean-sheet design? Section 12, and the answer is frequently the second one.

9. Does the advertised capability mask match the implemented mask, and is that checked mechanically? Section 13, and section 22 is what it costs not to.

10. Which of the six properties does the procurement process actually verify? Section 15 exists because the answer is usually one.

24. How This Appears In Real Engineering

A device team scoping a 2.0 product does section 12 first, and the honest version of the question is whether to grow the 1.1 adapter or design a new one. The growth path looks cheaper and carries the entire 1.1 architecture's assumptions forward — one decoder, host ownership, device-scoped errors — each of which then has to be found and changed individually.

A platform team qualifying a device runs into section 22. The lesson generalises: the negotiated link version is one of six properties and the only one with a standard way to read it, so a qualification process built around it will pass devices that cannot do the thing they were bought for. Each of the other five is observable, and none of them is observable by default.

A fleet architect sequencing a migration owns section 14, and the decision is which end to migrate first. Buying 2.0 devices for 1.1 hosts strands the capability in the devices; buying 2.0 hosts for 1.1 devices strands it in the hosts. The right answer depends on which end has the shorter replacement cycle, and it is a question with a real answer that is rarely asked.

A verification team finds section 18's first group is the interesting one. A mutation that no stimulus can kill is usually reported as an equivalent mutant and dismissed. Here it was a signal that the broken build was too weak to exercise the correct one — which is a class of finding that only appears when the design under test is deliberately parameterised into correct and incorrect variants.

25. Common Misconceptions

"The device negotiated 2.0, so it is a 2.0 device." One property of six. Section 15's feature-list definition called five of six configurations a 2.0 device.

"CXL 2.0 replaces 1.1." It interoperates with it, at 1.1 capability, and a fleet spends years in that state. Sections 5 and 14.

"A downgraded link is a problem." It is the compatibility requirement working. Section 5.

"Multi-logical-device support is a mode." It is a decoder array. Sections 6 and 20.

"The 2.0 delta can be handled in firmware." Decoder registers, an integrity datapath and a second management interface are not firmware. Section 20.

"Integrity overhead is small." 27% at a 64-byte payload and 60% at 16 bytes. Section 8.

"We can reuse the 1.1 throughput numbers." Not with a mandatory MAC. 22% optimistic. Section 8.

"Hot-plug is a device capability." It needs reserved address space and a host that understands the notification. Three parties. Section 9.

"Per-logical-device errors are just a reporting change." They are a fan-in structure with sixteen destinations. Section 20.

"Advertising a capability we plan to add is harmless." Software will use it the day the device ships. Sections 13 and 22.

26. Interview Reasoning

Q. A CXL 2.0 device is plugged into a CXL 1.1 host. What happens?

The link comes up at 1.1 and works, with every 2.0 feature unavailable. That is the requirement being met, not a failure. The follow-up worth reaching: the same is true in reverse, and a negotiation that instead reaches for the newer version brings up a link one end has no logic to drive.

Q. What stops a CXL 1.1 device from being pooled?

One HDM decoder. Pooling means more than one host, which means more than one range, which is a decoder array the 1.1 adapter does not have. The good follow-up is that this is not a limitation anybody chose — one decoder is the correct design for a device with one host.

Q. A device advertises CXL 2.0 and a second host assignment fails. Where do you look?

The decoder count, and then the advertised capability mask against what is implemented. A device can negotiate 2.0 on the link and have a 1.1 adapter behind it, and the capability register is the only place the discrepancy exists — which is why nothing else in the stack reported an error.

Q. Why did link integrity becoming mandatory matter to a throughput model?

Because a 1.1 device could decline to pay it and a 2.0 device cannot. A 16-byte MAC on a 64-byte payload is 22% of the useful bandwidth, and a model inherited from a 1.1 datasheet is optimistic by exactly that. The follow-up is how it scales — the overhead is a function of payload size, 60% at 16 bytes and 8% at 256.

Q. Why does an error need to name a logical device?

Because a device-scoped error on a device with eight logical devices tells eight hosts that something failed and none of them which. The follow-up worth being ready for: where does an error the device cannot attribute go? If the answer is "nowhere", the telemetry will report perfect attribution.

Q. Your fleet has 2.0 hosts and 1.1 devices. What can you use?

Nothing 2.0-specific — a path is only 2.0-capable if both ends are, so the usable fraction is bounded by the scarcer end. The interesting follow-up is which end to migrate first, and the answer depends on replacement cycles rather than on the protocol.

27. Exercises

1. Extend RTL 1 to negotiate a feature mask rather than a version number, and show that a per-feature negotiation permits combinations a version comparison cannot express.

2. Add a decoder-programming sequence to RTL 2 and show that ranges must be programmed before the logical device is bound, which is 20.2 section 12's ordering at the register level.

3. Extend RTL 3 to model a fabric-manager takeover of a directly attached device, and decide what the host's existing decoder state means afterwards.

4. Model RTL 4's latency as well as its bandwidth: add the MAC's pipeline stages and show the payload size at which latency rather than bandwidth becomes the binding constraint.

5. Add a hot-remove path to RTL 5 and show that the address-space reservation must persist after the capacity leaves, or the next hot-add has nowhere to go.

6. Give RTL 6 a separate unattributable-error counter and show that a device with one such error and a perfect per-logical-device record is indistinguishable from a correct one without it.

7. Extend RTL 7 to model area against a target frequency, and show that the decoder array's comparison depth makes the two trade against each other.

8. Make RTL 8's capability mask readable and writable, and show how a bring-up script could verify each advertised bit by exercising it — which is section 22's real fix.

9. Extend RTL 9 to a migration schedule over time and find the order of host and device replacement that maximises the integral of usable paths.

10. Add a seventh property to RTL 10. If it is implied by one of the six, say which; if not, give the device it catches that the current mask calls a 2.0 device.

28. Summary

Beneath the switch and the pool is the adapter, and this is what it had to become.

Negotiation takes the lower of the two ends. Two of four links downgraded, correctly — and a build reaching for the more capable end overclaimed in both directions, against a 1.1 device and against a 1.1 host.

One decoder became several. An eight-range adapter configured for four has index 3 as its last valid one, and a 1.1 adapter refused four of five programming requests — accepting only the one asking for a single range, which is why it can still be attached and cannot be pooled.

The host stopped owning the device, and direct attachment did not go away — the ownership rule is a property of the topology, so a check written as "only the fabric manager" breaks every 1.1-style attachment in the fleet.

Integrity stopped being optional. 88 bytes on the wire for 64 useful: 290 Gbps of 400, a 27% overhead — 60% at a 16-byte payload — and a throughput model inherited from a 1.1 datasheet is 22% optimistic.

Capacity can arrive after boot, which needs reserved address space and a host that understands the notification. Three parties, and the device is one of them.

An error has to name a logical device. A device-scoped error told eight hosts of eight that something they cannot identify had failed — and the errors that cannot be attributed need their own counter, or telemetry will report perfect attribution.

The delta is 200 kgates and 50% area growth on a 1.1 adapter, and the alternative that usually wins is a clean-sheet design with no growth percentage to report. None of it is firmware.

A capability advertised and not implemented is worse than one not advertised, because software will use it — and section 22 is a rack full of that exact bit.

A fleet migrates in pieces. 50 of 100 hosts and 80 of 100 devices gives 50% of paths usable, bounded by the scarcer end, and which end that is swaps without the fleet changing size.

And a mutation no stimulus could kill turned out to be a broken build too weak to exercise the correct one — the batch's most useful finding, because the instinct is to call it equivalent and move on.

Negotiating 2.0 is one property of six. The feature-list definition — the one a procurement process runs — called five of six devices a 2.0 device when one was.

20.4 — CXL 2.0 New Capabilities takes the remaining additions the generation brought, hot-plug in full and the single-level fabric's implications, and closes Module 20.