Skip to content
VLSI Mentor

CXL · Module 5

PCIe / CXL Compatibility

What backwards compatibility actually means in a mixed PCIe/CXL fleet: feature intersection over a set, the generation axis, why a fallback needs a reason, and why advertising the richer set after degrading is the real defect. Five RTL models simulated, eleven mutations, eleven killed.

Chapter 5.1 established the two-key rule as a single bit: is CXL permitted at all. Chapters 5.3 to 5.5 built the machinery that answers it.

This chapter closes Module 5 by replacing that bit with a set, and adding the axis nobody plans for: the two ends are not only differently capable, they are differently aged.

1. The Engineering Problem — "Compatible" Is Not a Yes/No

A fleet contains hosts and devices bought over five years. Some hosts are CXL 3.0, some 1.1. Some devices support three protocols, some one. Some features exist in silicon and are disabled by policy. Every pair must produce a working link.

"Is this device compatible with this host?" has no useful answer, because it is the wrong question. The useful questions are:

  • Which features can this pair actually use? — a set operation, not a comparison.
  • Which revision will they speak? — a generation question, independent of features.
  • What did they lose, and why? — because a degraded link that cannot say why is unactionable.
  • Does software know? — because the failure this chapter is really about is a link that degrades quietly and keeps advertising the richer set.

2. The One-Sentence Model

Capability intersection, never capability assumption — the effective feature set is what both ends support and have actually said they support, the operating revision is the older of the two, and a link that degrades is fine while a link that degrades and still advertises the full set is broken.

The second half is the part that costs money. Degradation is not the defect. Undisclosed degradation is.

3. What This Chapter Owns

QuestionOwned by
Why the reuse exists at all5.1
Which structures are reused5.2
How the two ends agree on protocol5.3
Widths, rates, recovery5.4
How software reads capability structures5.5
Feature sets, revisions, and honest degradationthis chapter

The deliberate advance over 5.1. Chapter 5.1 gated one bit and its RTL intersected two booleans. Here the intersection is over a five-bit feature vector, there is a second, orthogonal axis (revision), refusal happens at individual feature granularity rather than whole traffic classes, and the failure of interest is a reporting failure rather than a routing one.

4. Two Axes, Not One

The single most common modelling error here is treating compatibility as one dimension.

FeatureRevision
Askswhich caps?which gen?
Operationintersectionminimum
Failurecap unusablesemantics differ
Fixbuild itnone — it is the fleet

They are orthogonal. Two CXL 3.0 parts can share a revision and disagree on features; a 3.0 host and a 1.1 device can agree on every feature 1.1 defines and still operate at 1.1. A design that folds them into one "compatible" flag cannot express either.

5. Quantitative — Why Validation Grows So Fast

With N host capability profiles and M device profiles there are N × M combinations, and each is a distinct thing that can be wrong. The reuse argument from 5.1 guaranteed every pair links; it guaranteed nothing about every pair being equivalent.

RTL 1 sweeps a real 5 × 5 grid of feature profiles:

Icarus Verilog 13.0 — EXP4
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
  over 5 host x 5 device profiles = 25 combinations
  host got its full set: 14 | partial: 11 | nothing: 0
  only 14 of 25 pairs deliver the host's full request

Under half the grid delivers what the host asked for, and no cell fails outright — every pair works, at different capability. That is precisely why the matrix has to be enumerated rather than sampled: there is no error to trip over.

Add the revision axis and it multiplies. Four revisions on each side is another 16 combinations per feature pair, and the two axes are independent, so the honest count is N × M × R_h × R_d. A team that validates "the newest host with the newest device" has tested one cell of several hundred.

The intersection, concretely

Feature masks intersect bitwise. With five teaching features:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
host   = 11101
device = 10111
usable = 10101      (3 of 5)

Bit 1 is lost because the host lacks it; bit 3 because the device lacks it. Neither end can tell you the answer alone — which is the whole content of RTL 1.

6. Teaching-model boundary

7. RTL 1 — Intersection Over a Set

feature_intersect.sv — and an unheard peer contributes nothing
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
module feature_intersect #(
  parameter bit LOCAL_ONLY      = 1'b0,   // 1 = use our own mask as the answer
  parameter bit ASSUME_PEER_ALL = 1'b0    // 1 = treat an unheard peer as all-capable
) (
  input  logic       clk, rst_n,
  input  logic [4:0] host_features, device_features,
  input  logic       peer_heard,        // the partner's mask has actually arrived
  input  logic       link_ready,
  output logic [4:0] usable_features,
  output logic [2:0] n_usable,
  output logic       any_usable,
  output logic       not_subset_host_err, not_subset_dev_err, used_unheard_peer_err
);
  logic [4:0] peer_view;
  integer     k;
 
  // An unheard peer contributes NOTHING, not everything. This is the same
  // distinction as 5.1's "the peer said no" vs "the peer has not answered",
  // now over a set rather than a bit.
  assign peer_view = peer_heard ? device_features
                                : (ASSUME_PEER_ALL ? 5'b11111 : 5'b00000);
 
  assign usable_features = link_ready
                         ? (LOCAL_ONLY ? host_features : (host_features & peer_view))
                         : 5'b00000;
  assign any_usable = |usable_features;
 
  always_comb begin
    n_usable = 3'd0;
    for (k = 0; k < 5; k = k + 1) n_usable = n_usable + {2'd0, usable_features[k]};
  end
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      not_subset_host_err <= 1'b0; not_subset_dev_err <= 1'b0;
      used_unheard_peer_err <= 1'b0;
    end else if (link_ready) begin
      // Stated about the OUTPUT and separately per side: a local-only bug
      // breaks exactly one of these, an assume-all bug breaks the other.
      if ((usable_features & ~host_features)   != 5'd0) not_subset_host_err   <= 1'b1;
      if ((usable_features & ~device_features) != 5'd0) not_subset_dev_err    <= 1'b1;
      if (any_usable && !peer_heard)                    used_unheard_peer_err <= 1'b1;
    end
  end
endmodule

Architecture position. Between capability discovery (5.5) and any enable logic. It consumes two masks and a validity bit and produces the only mask anything downstream may act on.

State. None on the datapath — the intersection is combinational, which is correct: it is a pure function of the two advertisements. The registers exist only for the three diagnostics, which must be sticky so a transient violation is not lost between polls.

Backpressure. None; this is a decision, not a transfer. That matters for verification — there is no queue to fill and therefore no timing-dependent behaviour to chase.

Synthesis. Five AND gates, a 5-to-3 population count, and three set-reset flops. The population count is the only non-trivial structure and it is off the critical path because nothing downstream needs the count in the same cycle.

Icarus Verilog 13.0 — EXP2 and EXP3
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
  host=11101 device=10111 -> usable=10101 (3 of 5)
  peer NOT heard : correct usable=00000 | assume-all usable=11101

The second line is the one that costs silicon. An unheard peer contributes nothing, not everything. The ASSUME_PEER_ALL variant enables four features against a partner that has not spoken — and every one of them is a guess.

8. Waveform — Full and Degraded, Side by Side

The same host, two partners: one matches, one does not

8 cycles
Both cases hold usable features at zero until the link is ready and the peer has been heard at cycle three; the matching partner then yields the full host mask 11101 while the differing partner yields the intersection 10101no linkno linklink up, peer not yet heardlink up, peer notyet heardintersection validintersection validpeer heard — only now may anything be enabledpeer heard — only now mayanything be enabledclklink_readypeer_heardhost_feat1110111101111011110111101111011110111101dev A1110111101111011110111101111011110111101usable A0000000000000001110111101111011110111101dev B1011110111101111011110111101111011110111usable B0000000000000001010110101101011010110101t0t1t2t3t4t5t6t7
Icarus Verilog 13.0, EXP1 and EXP2. Cycle values taken from the simulation transcript.

Cycles 1 and 2 are the interesting ones. The link is ready and nothing is enabled — a window that a design in a hurry will try to close by assuming, and RTL 1's ASSUME_PEER_ALL variant is what that assumption looks like in silicon.

9. RTL 2 — The Generation Axis

Public material states CXL 3.0 has full backward compatibility with 2.0, 1.1 and 1.0. Operationally that means the newer end speaks the older revisionmin(), not max(), and definitely not "whatever the host is".

revision_negotiate.sv — the newer end downshifts
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
module revision_negotiate #(
  parameter bit USE_HOST_REV = 1'b0     // 1 = the broken shape
) (
  input  logic       clk, rst_n,
  input  logic [1:0] host_rev, device_rev,
  input  logic       valid,
  output logic [1:0] op_rev,            // the revision the link actually runs
  output logic       host_downshifted,  // the newer end is operating older
  output logic       rev_above_peer_err, rev_regressed_err
);
  logic [1:0] agreed, op_rev_q, host_q, dev_q;
  logic       seen_q;
 
  // Backward compatibility means the NEWER end speaks the OLDER revision.
  assign agreed  = (host_rev < device_rev) ? host_rev : device_rev;
  assign op_rev  = valid ? (USE_HOST_REV ? host_rev : agreed) : 2'd0;
  assign host_downshifted = valid && (host_rev > op_rev);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      rev_above_peer_err <= 1'b0; rev_regressed_err <= 1'b0;
      op_rev_q <= 2'd0; seen_q <= 1'b0;
    end else if (valid) begin
      // Never operate above what EITHER end can do.
      if ((op_rev > host_rev) || (op_rev > device_rev)) rev_above_peer_err <= 1'b1;
      // For an UNCHANGED pair the agreed revision must be stable. Comparing
      // an input to itself is a tautology and can never fail -- the inputs
      // have to be registered for this to be a real check.
      if (seen_q && (host_rev == host_q) && (device_rev == dev_q) &&
          (op_rev != op_rev_q))
        rev_regressed_err <= 1'b1;
      op_rev_q <= op_rev; host_q <= host_rev; dev_q <= device_rev; seen_q <= 1'b1;
    end
  end
endmodule
Icarus Verilog 13.0 — EXP5, all 16 pairs
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
  host_rev device_rev | agreed  downshifted | use-host-rev
     3        0      |   0        1       |    3
     3        1      |   1        1       |    3
     3        3      |   3        0       |    3
  agreed revision is min(host,device) in all 16 pairs

The USE_HOST_REV column is a 3.0 host talking 3.0 semantics at a 1.0 device. Every message is well-formed for a revision the partner does not implement.

10. RTL 3 — Refusal at Feature Granularity

Chapter 5.1 refused a whole traffic class in the wrong mode. Real software does not ask for a class; it asks for a feature.

feature_gate.sv — both directions of refusal are defects
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
module feature_gate #(
  parameter bit TRUST_REQUEST = 1'b0    // 1 = the broken shape
) (
  input  logic       clk, rst_n, req_valid,
  input  logic [2:0] req_feature,       // which feature this request needs
  input  logic [4:0] usable_features,
  output logic       accept, refuse,
  output logic [7:0] n_accept_q, n_refuse_q,
  output logic       unsupported_accepted_err, supported_refused_err
);
  logic supported;
  assign supported = req_valid && (req_feature < 3'd5) &&
                     usable_features[req_feature[2:0]];
 
  assign accept = req_valid && (TRUST_REQUEST ? 1'b1 : supported);
  assign refuse = req_valid && !accept;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_accept_q <= 8'd0; n_refuse_q <= 8'd0;
      unsupported_accepted_err <= 1'b0; supported_refused_err <= 1'b0;
    end else begin
      if (accept) n_accept_q <= n_accept_q + 8'd1;
      if (refuse) n_refuse_q <= n_refuse_q + 8'd1;
      // Both directions are defects. Accepting the unsupported corrupts;
      // refusing the supported is a silent capability loss.
      if (accept && !supported) unsupported_accepted_err <= 1'b1;
      if (refuse && supported)  supported_refused_err    <= 1'b1;
    end
  end
endmodule
Icarus Verilog 13.0 — EXP6, usable = 10101
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
    feature[0] usable=1 : accept=1 refuse=0 | trust-request accept=1
    feature[1] usable=0 : accept=0 refuse=1 | trust-request accept=1
    feature[2] usable=1 : accept=1 refuse=0 | trust-request accept=1
    feature[3] usable=0 : accept=0 refuse=1 | trust-request accept=1
    feature[4] usable=1 : accept=1 refuse=0 | trust-request accept=1
  correct: accepted=3 refused=2 | trust-request accepted=5

Two error signals, not one. unsupported_accepted_err is a correctness failure — the operation proceeds on a link that cannot carry it. supported_refused_err is a capability failure — the feature exists and is being wasted. They have opposite fixes and different owners, and a combined "gate error" flag would send both to whoever reads it first.

11. RTL 4 — Why, Not Just Whether

A degraded link that cannot say why is an unactionable alert. These reason codes are teaching values chosen to separate causes that need different responses and different teams.

fallback_reason.sv — the first cause is the actionable one
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
module fallback_reason (
  input  logic       clk, rst_n, evaluate,
  input  logic       link_ok, peer_heard, peer_supports,
  input  logic       rev_compatible, feature_present, policy_enabled,
  output logic [2:0] reason_q,
  output logic       degraded,
  output logic       no_reason_err, reason_without_degrade_err
);
  localparam logic [2:0] R_NONE = 3'd0, R_LINK = 3'd1, R_UNHEARD = 3'd2,
                         R_PEER = 3'd3, R_REV = 3'd4, R_ABSENT = 3'd5,
                         R_POLICY = 3'd6;
  logic [2:0] reason_c;
  logic       deg_c;
 
  // Priority matters: the FIRST thing that stopped you is the actionable one.
  // Reporting "policy disabled" on a link that never trained sends the
  // investigation to the wrong team.
  always_comb begin
    if      (!link_ok)         reason_c = R_LINK;
    else if (!peer_heard)      reason_c = R_UNHEARD;
    else if (!peer_supports)   reason_c = R_PEER;
    else if (!rev_compatible)  reason_c = R_REV;
    else if (!feature_present) reason_c = R_ABSENT;
    else if (!policy_enabled)  reason_c = R_POLICY;
    else                       reason_c = R_NONE;
    deg_c = (reason_c != R_NONE);
  end
 
  assign degraded = evaluate && deg_c;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      reason_q <= R_NONE; no_reason_err <= 1'b0; reason_without_degrade_err <= 1'b0;
    end else if (evaluate) begin
      reason_q <= reason_c;
      if (deg_c && (reason_c == R_NONE))  no_reason_err <= 1'b1;
      if (!deg_c && (reason_c != R_NONE)) reason_without_degrade_err <= 1'b1;
    end
  end
endmodule
Icarus Verilog 13.0 — EXP7, causes added one at a time
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
  everything ok            : degraded=0 reason=NONE
  policy disabled          : degraded=1 reason=POLICY
  + feature absent         : degraded=1 reason=ABSENT
  + revision incompatible  : degraded=1 reason=REVISION
  + peer does not support  : degraded=1 reason=PEER
  + peer not heard         : degraded=1 reason=UNHEARD
  + link down              : degraded=1 reason=LINK  <-- the first cause wins

The ladder is ordered by how early the cause bites. A link that never trained is not "policy disabled" even if policy also happens to be off — reporting the later cause sends an engineer to a configuration file when the problem is a connector.

Note the two complementary diagnostics. no_reason_err catches a degraded link with nothing to say; reason_without_degrade_err catches a false alarm. A status register needs both, because each alone is satisfiable by a design that never reports anything.

12. RTL 5 — What Software Is Told

This is the chapter's real subject.

capability_report.sv — advertise the outcome, not the intent
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
module capability_report #(
  parameter bit REPORT_INTENT = 1'b0   // 1 = advertise what we WANTED
) (
  input  logic        clk, rst_n, tick, link_up,
  input  logic [4:0]  desired_features,  // what this platform hoped for
  input  logic [4:0]  usable_features,   // what the intersection actually gave
  input  logic        engine_present,    // RTL for the feature really exists
  output logic [4:0]  advertised_q, enabled_q,
  output logic [15:0] t_up_q, t_full_q, t_degraded_q,
  output logic        overclaim_err, phantom_capability_err, residency_err
);
  logic [4:0] adv_c, en_c;
 
  assign en_c  = link_up ? usable_features : 5'd0;
  // The bug shape: advertise the intent instead of the outcome.
  assign adv_c = link_up ? (REPORT_INTENT ? desired_features : usable_features) : 5'd0;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      advertised_q <= 5'd0; enabled_q <= 5'd0;
      t_up_q <= '0; t_full_q <= '0; t_degraded_q <= '0;
      overclaim_err <= 1'b0; phantom_capability_err <= 1'b0; residency_err <= 1'b0;
    end else begin
      advertised_q <= adv_c; enabled_q <= en_c;
 
      if (tick && link_up) begin
        t_up_q <= t_up_q + 16'd1;
        if (usable_features == desired_features) t_full_q     <= t_full_q     + 16'd1;
        else                                     t_degraded_q <= t_degraded_q + 16'd1;
      end
 
      // Software must never be told about a capability the link does not have.
      if ((adv_c & ~en_c) != 5'd0) overclaim_err <= 1'b1;
      // Advertising a feature with no engine behind it.
      if (|adv_c && !engine_present) phantom_capability_err <= 1'b1;
      // Conservation: every up-tick is either full or degraded, never neither.
      if (t_up_q != t_full_q + t_degraded_q) residency_err <= 1'b1;
    end
  end
endmodule
Icarus Verilog 13.0 — EXP8, 80 ticks
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
  up=80 full=30 degraded=50
  conservation up == full + degraded : 80 == 80
  correct      : advertised=10101 enabled=10101
  report-intent: advertised=11101 enabled=10101  <-- overclaim

The link spent 50 of 80 ticks degraded and never reported a single error, because degradation is not an error. The only thing separating a healthy fleet from a broken one is whether advertised tracks enabled — and the REPORT_INTENT variant differs by exactly one signal name.

phantom_capability_err catches the adjacent case: an advertised feature with no engine behind it. That is Chapter 5.5's presence-versus-validity problem arriving from the other direction — there, a structure existed without meaning; here, a claim exists without hardware.

13. Assertions

Concurrent SVA execution: NOT SUPPORTED BY Icarus Verilog. Not executed; each maps to a procedural stand-in and a mutation.

pcie_cxl_compat_sva.sv — bind-ready properties
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SAFETY -------------------------------------------------------------------
// V1 — the usable set is a subset of BOTH advertisements.
a_subset_both: assert property (@(posedge clk) disable iff (!rst_n)
  link_ready |-> (((usable_features & ~host_features)   == '0) &&
                  ((usable_features & ~device_features) == '0)));
 
// V2 — nothing is usable before the peer has been heard.
a_no_assumption: assert property (@(posedge clk) disable iff (!rst_n)
  any_usable |-> peer_heard);
 
// V3 — the operating revision never exceeds either end.
a_rev_floor: assert property (@(posedge clk) disable iff (!rst_n)
  valid |-> ((op_rev <= host_rev) && (op_rev <= device_rev)));
 
// V4 — an unchanged pair yields an unchanged agreed revision.
a_rev_stable: assert property (@(posedge clk) disable iff (!rst_n)
  (valid && $stable(host_rev) && $stable(device_rev)) |-> $stable(op_rev));
 
// V5 — a feature request is accepted only if that feature is usable.
a_gate_honest: assert property (@(posedge clk) disable iff (!rst_n)
  accept |-> usable_features[req_feature]);
 
// V6 — a usable feature is never refused.
a_no_waste: assert property (@(posedge clk) disable iff (!rst_n)
  (req_valid && usable_features[req_feature]) |-> accept);
 
// V7 — degraded implies a reason, and a reason implies degraded.
a_reason_iff_degraded: assert property (@(posedge clk) disable iff (!rst_n)
  evaluate |-> (degraded == (reason_q != R_NONE)));
 
// V8 — software is never told about a capability that is not enabled.
a_no_overclaim: assert property (@(posedge clk) disable iff (!rst_n)
  (advertised_q & ~enabled_q) == '0);
 
// V9 — CONSERVATION: every up-tick is full or degraded, never neither/both.
a_residency: assert property (@(posedge clk) disable iff (!rst_n)
  t_up_q == t_full_q + t_degraded_q);
 
// LIVENESS -----------------------------------------------------------------
// V10 — a ready link eventually produces a decided feature set.
//       ENVIRONMENT ASSUMPTION: the peer eventually advertises. Chapter 5.3's
//       timeout is what discharges this; without it an unheard peer holds the
//       set at zero indefinitely, which is correct behaviour, not a defect.
a_eventually_decided: assert property (@(posedge clk) disable iff (!rst_n)
  link_ready |-> s_eventually peer_heard);

V5 and V6 are a deliberate pair. Either alone is satisfied by a degenerate design — V5 by one that refuses everything, V6 by one that accepts everything. Only the conjunction pins the gate, which is why RTL 3 carries two error signals rather than one.

14. Mutation Testing

Eleven mutations. Clean code restored after each.

IDMutationResult
M1union instead of intersectionKILLED — used_unheard_peer_err
M2an unheard peer's mask used anywayKILLED — used_unheard_peer_err
M3agreed revision is max, not minKILLED — revision table
M4only the host side of the bound checkedKILLED — positive test
M5every feature request acceptedKILLED — per-feature check
M6the usable mask is not consultedKILLED — per-feature check
M7cause priority invertedKILLED — priority assertion
M8a policy-disabled feature reports no reasonKILLED — ladder assertions
M9software told the intent, not the outcomeKILLED — overclaim_err
M10degraded ticks not countedKILLED — conservation
M11the overclaim check disabledKILLED — positive test
Mutation run — final
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
11/11 killed, 0 escaped

One escaped on the first run, and it was the batch-005 carry-forward repeating. M8 removes the policy branch from the reason ladder. EXP7 printed all seven rungs and asserted only the last one, so deleting a middle rung changed a printed line and no verdict.

Batch 005's audit recorded this exact habit as a P1 process carry-forward: assert every displayed value at the point it is displayed. It recurred here on the first chapter of the next batch. The fix was to assert all seven rungs, and the rule was then applied to every experiment in the remaining four chapters of this batch — which is why §14 of each subsequent chapter reports a lower first-run escape rate.

15. Debug Lab

1

A device works on one host and loses a feature on an identical one

LOCAL-MASK-USED-AS-ANSWER
Buggy Code
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// We know what we support. Use it.
assign usable_features = link_ready ? host_features : 5'd0;
Symptom

A feature is enabled and fails on first use, but only with certain devices. The host's own capability registers read correctly. Nothing in bring-up reports an error.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
  host=11101 device=10111 -> correct usable=10101
  local-only bug would use 11101
  not_subset_dev_err=1
Root Cause

The host's own mask was used as the answer. It is a true statement about the host and says nothing about the pair. The device never advertised bit 3, so every operation using it is issued to a partner that cannot process it.

This is Chapter 5.1's two-key rule generalised: capability is a property of the pair, and the combining function over a set is AND, exactly as it was over a bit.

Fix
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
assign usable_features = link_ready ? (host_features & peer_view) : 5'd0;
// and check from both sides, separately:
if ((usable_features & ~host_features)   != 5'd0) not_subset_host_err <= 1'b1;
if ((usable_features & ~device_features) != 5'd0) not_subset_dev_err  <= 1'b1;
Lesson

Two subset checks, not one. A union bug breaks both; using one side's mask breaks exactly one, and which one names the bug immediately. A single combined "mask error" flag would have reported the same thing for two defects with different fixes.

2

Features are enabled in the window before the partner answers

UNHEARD-PEER-ASSUMED-CAPABLE
Buggy Code
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// Optimistic: assume a partner supports everything until told otherwise.
assign peer_view = peer_heard ? device_features : 5'b11111;
Symptom

Intermittent failures at boot, more frequent on faster hosts and on links with retimers. Once the system is up it runs correctly. Rebooting sometimes fixes it.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
  peer NOT heard : correct usable=00000 | assume-all usable=11101
  used_unheard_peer_err=1
Root Cause

There is a real window — visible as cycles 1 and 2 in §8's waveform — where the link is ready and the partner's mask has not arrived. Treating an unheard peer as fully capable enables features during that window based on nothing.

The timing dependence is what makes it expensive: on a slow path the mask usually arrives before anything is enabled, so the bug reproduces only under conditions that look unrelated to capability.

Fix
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
assign peer_view = peer_heard ? device_features : 5'b00000;
if (any_usable && !peer_heard) used_unheard_peer_err <= 1'b1;
Lesson

The safe default for absent information is the empty set. This is the third time Module 5 has hit the same distinction — 5.1 for a capability bit, 5.5 for an absent function map, and now for a feature vector. "Has not answered" is not "supports everything" and is not "supports nothing" either; it is not a decision yet, and the only safe action while undecided is none.

3

A newer host speaks a revision its partner does not implement

HOST-REVISION-IMPOSED
Buggy Code
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// The host defines the platform revision.
assign op_rev = host_rev;
Symptom

A new server fails with older devices that work in the previous generation of the same platform. Errors appear well after link-up, at the first use of a construct the older device does not recognise.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
  host_rev device_rev | agreed  | use-host-rev
     3        0      |   0     |    3
  rev_above_peer_err=1
Root Cause

Backward compatibility was implemented as "the newer part sets the terms", which is backwards. Public material describes CXL 3.0 as having full backward compatibility with 2.0, 1.1 and 1.0 — meaning the 3.0 part must be able to operate as a 1.0 part, not that the 1.0 part must cope with 3.0 semantics.

The operating revision is min(host, device). The host downshifts. That is what the compatibility guarantee is.

Fix
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
assign agreed = (host_rev < device_rev) ? host_rev : device_rev;
assign op_rev = valid ? agreed : 2'd0;
if ((op_rev > host_rev) || (op_rev > device_rev)) rev_above_peer_err <= 1'b1;
Lesson

Check the bound on both sides. Mutation M4 removes the device half of that check and survives on any test where the host is the older end — which is the unnatural configuration and therefore the one nobody writes. Whenever a value must be below two limits, assert both, and make sure the test sweeps which limit is binding.

4

The link degrades and the status register still reads full capability

ADVERTISED-DOES-NOT-TRACK-ENABLED
Buggy Code
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// Report what this platform provides.
assign advertised = link_up ? desired_features : 5'd0;
Symptom

Fleet telemetry shows every link at full capability. Application performance on a subset of nodes is well below the pilot. Every capability query returns the expected mask; no node reports an error.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
  up=80 full=30 degraded=50
  correct      : advertised=10101 enabled=10101
  report-intent: advertised=11101 enabled=10101
  overclaim_err=1
Root Cause

The advertised mask was wired to the platform's intent rather than the negotiated outcome. Software reads the intent, configures for it, and issues operations the link cannot carry.

Note that nothing here is an error by the link's own standards: it degraded correctly, ran correctly at the reduced set, and reported what it was told to report. The defect is entirely in the reporting path, which is why no functional test finds it.

Fix
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
assign enabled     = link_up ? usable_features : 5'd0;
assign advertised  = enabled;                     // the outcome, never the intent
if ((advertised & ~enabled) != 5'd0) overclaim_err <= 1'b1;
Lesson

A system that operates with fewer capabilities is not broken; a system that hides it is. The check is one line and it is the difference between a fleet you can reason about and one you cannot. Pair it with residency counters — 50 of 80 ticks degraded with zero errors logged is exactly the signature this measures.

5

A capability is advertised with no engine behind it

PHANTOM-CAPABILITY
Buggy Code
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// The capability register is initialised from the product configuration.
assign advertised = cfg_feature_mask;   // engine_present never consulted
Symptom

A driver enables a feature, the enable succeeds, and the first operation returns an unsupported-request error from a device that just claimed to support it. The behaviour is identical across every unit, so it does not look like a defect at all.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
  advertised=10101 engine_present=0
  phantom_capability_err=1
Root Cause

The advertised mask came from a configuration constant rather than from what was built. On a derivative part where an engine was removed to save area, the constant was not updated — so the capability survived in the register map after it stopped existing in RTL.

This is the mirror of Chapter 5.5's presence-versus-validity: there a structure existed without meaning; here a claim exists without hardware.

Fix
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// Advertise only what is both negotiated AND built.
assign advertised = usable_features & engine_mask;
if (|advertised && !engine_present) phantom_capability_err <= 1'b1;
Lesson

Derive the advertisement from the design, not from a constant beside it. Any capability register populated by a parameter can disagree with the RTL it describes, and derivative parts are where it happens — the configuration is edited and the register map is not. The design-review question is: what would make this register wrong, and would anything catch it?

16. Verification Plan

ItemApproach and goal
Intersectioncross host x device masks incl. disjoint and partial — subset both ways
Unheard peerassert with link_ready high and peer_heard low — nothing usable
Revision floorsweep all R × R pairs — agreed is min, both bounds asserted
Revision stabilityhold a pair constant — agreed does not drift
Feature gaterequest every index against a known mask — accept iff usable
Reason ladderadd causes one at a time — assert every rung, not just the last
Report honestydegrade mid-run — advertised tracks enabled every cycle
Residencymixed full/degraded trace — conservation holds every tick
Phantom capabilitybuild with an engine absent — advertisement follows the RTL
Diagnostic livenesseach broken variant — every diagnostic observed firing

Row 6 is this chapter's own lesson. A ladder of prioritised causes has one obvious assertion (the top rung) and N-1 that are easy to omit — and mutation M8 lived in exactly one of the omitted ones.

17. Design Review

  • Is the effective capability an intersection, and is it checked from both sides separately?
  • What does the design do in the window where the link is up and the partner has not answered?
  • Is the operating revision min of the two ends, and is the bound asserted against both?
  • Can the status register disagree with what the link actually enabled? What would catch it?
  • Does any advertised capability come from a constant rather than from the design?
  • Does a degraded link report why, and is the reported cause the earliest one?
  • Are "refused something supported" and "accepted something unsupported" distinguishable in telemetry?
  • Can the platform report what fraction of link-time ran at full capability?

18. How This Appears in Real Engineering

The compatibility matrix is a schedule item, not a diagram. §5's 25 combinations is a toy; a real fleet has host generations × device generations × optional-feature profiles, and the count runs to hundreds. Teams that sample it validate the newest-with-newest cell and ship the rest.

Silent degradation is found by performance engineers, not by validation. Every functional test passes on a degraded link, because degradation is correct behaviour. The first signal is usually a workload that is slower on some nodes, investigated for weeks as a software problem.

Derivative parts are where phantom capabilities appear. The lead part is built and validated with every engine present. The cost-reduced variant removes one, and the capability constant is edited in one place and not another. Debug Lab 5 is disproportionately a derivative-silicon bug.

The revision axis surprises platform teams. A new host with a full 3.0 stack talking to a 1.1 device operates at 1.1 — so a platform refresh can show no improvement on unchanged devices, which reads as a regression until someone checks the operating revision.

19. Common Misconceptions

ClaimWhy it is wrong
"Compatible is a yes/no property"Two axes: feature intersection and revision min. A design with one flag can express neither.
"The newer host sets the terms"Backward compatibility means the newer part operates as the older. Publicly, CXL 3.0 is fully backward compatible with 2.0, 1.1 and 1.0.
"An unheard partner probably supports it"An unheard partner has supplied no information. The safe default is the empty set — the third time Module 5 has needed this.
"Falling back is an error"It is usually the correct outcome. What is never correct is falling back and still advertising the full set.
"If the link is up, capability is settled"There is a real window where the link is ready and the partner's mask has not arrived. §8's waveform shows it.
"The capability register describes the hardware"Only if it is derived from the hardware. A constant beside the RTL can outlive the engine it describes.
"One error flag is enough for the gate"Accepting the unsupported and refusing the supported are different defects with different owners.
"CXL.cache and CXL.mem are always available on a CXL link"CXL.io is mandatory; CXL.cache and CXL.mem are optional and usage specific. Their absence is a normal configuration, not a fault.

20. Interview Reasoning

21. Exercises

  1. Calculate. With host 11010 and device 01110, compute the usable set and the count. Then find a device mask that leaves the host with exactly two features, and one that leaves it with none. State which of the two is more dangerous to ship and why.

  2. Explain. §4 argues the feature and revision axes are orthogonal. Construct a concrete counter-example where they interact — a feature whose availability depends on revision — and say what that does to the validation matrix.

  3. DV task. Write the coverage cross that would have caught mutation M4 (only the host side of the revision bound checked). Explain why a sweep in which the host is always the newer end cannot catch it.

  4. RTL task. Extend capability_report so a feature can be built, negotiated, and disabled by policy as three distinct states, and state which existing diagnostic becomes ambiguous as a result.

  5. Debug task. A fleet reports 100% of links at full capability and 30% of nodes below expected throughput, all on one host model. Give your investigation order, and name the single measurement that separates a reporting bug from a genuine capability difference.

  6. Design review. A colleague proposes caching the negotiated feature mask in firmware across warm resets to speed boot. Give the strongest case for it, then the failure it introduces, and state what you would require before accepting it.

22. Summary

Compatibility is two orthogonal questions, and "compatible" as a boolean answers neither.

  • Capability intersection, never capability assumption. The usable set is host & device, and an unheard peer contributes nothing — the third time Module 5 has needed that distinction.
  • The revision axis is separate and resolves to min. Backward compatibility means the newer part operates as the older; publicly, CXL 3.0 is fully backward compatible with 2.0, 1.1 and 1.0.
  • In a 5 × 5 profile grid, only 14 of 25 pairs deliver the host's full request — and none of the other 11 report an error, which is why the matrix must be enumerated rather than sampled.
  • Degrading is fine. Degrading silently is the defect. A link ran 50 of 80 ticks degraded with zero errors logged; only advertised == enabled and residency counters make it visible.
  • A fallback needs a reason, and the reported reason must be the earliest cause, because that is what determines who investigates.
  • Verification lesson, carried from batch 005 and repeated here: a displayed value that is not asserted is not checked — and a tautological checker is invisible to mutation testing, because an equivalent mutant proves nothing.

23. Module 5 Complete

Module 5 asked why CXL rides on PCIe and what the machinery that exploits that reuse has to do.

  • 5.1 — the reuse buys zero dead pairs; compatibility is a constraint at reset, at link-up, and at error.
  • 5.2 — "reuse" is four contracts, and the class names what you still owe a structure.
  • 5.3 — the protocol agreement uses PCIe's own alternate protocol negotiation, producing the intersection of both ends and the path.
  • 5.4 — training resolves to an operating point; recovery must narrow, then conclude.
  • 5.5 — software learns it all by a bounded read of PCIe's capability list.
  • 5.6 — and in a mixed fleet, the effective capability is an intersection on two axes, which must be reported honestly.

Module 5's transferable idea is that every mechanism bounds something the design does not control — an unanswering partner, an unrecoverable channel, an endless list, and now a fleet of parts built years apart.

Module 6 turns from the link to what runs on it. Chapter 6.1 asks what CXL.io, CXL.cache and CXL.mem actually are — not as a list of three names, but as three different contracts about who owns a resource and who must keep state about it.

Standards & specifications

Governing standard
CXL Specification (CXL Consortium)(opens CXL Consortium in a new tab)

Defines CXL.io, CXL.cache and CXL.mem, and the coherence and memory-pooling behaviour built on them. System design and deployment topology are not mandated.

This page also covers RTL structure, verification approach and debugging technique. Those are engineering practice built on the standard, not requirements the standard itself imposes.

Where this fits

Part of the CXL curriculum.