CXL · Module 31
“CXL Replaces PCIe”
Replacement means displacement, not capability. Nine tests of the claim — layers reused, what would have to disappear, two independent questions, the devices that gain nothing, what trained the link, rate against semantics, the coherence crossover, three device kinds, and the four conditions that would have to hold.
"CXL replaces PCIe" is the first thing almost everybody believes about CXL, and it is believed for a reason that is entirely sound: CXL is newer, it does things PCIe does not, and the two appear in the same sentence in every introduction. Two true premises, and a conclusion that does not follow from them.
The question this chapter turns on:
What would have to disappear for this to be a replacement — and has any of it?
"Replaces" is a claim about displacement, not about capability. A protocol can be newer, faster, and strictly more capable, and still not replace anything — because replacement means the older thing stops being needed, and that is a question you settle by counting rather than by arguing.
1. Two True Statements Are Not An Argument
The belief is assembled from premises that are individually correct.
| Premise | True? | Implies it? |
|---|---|---|
| CXL is newer | yes | no — newer is a date |
| CXL does things PCIe does not | yes | no — that is addition |
| Both appear in the same talks | yes | no — so do a road and a lorry |
| Therefore CXL replaces PCIe | — | does not follow |
The error is not factual, it is structural, which is why it survives contact with accurate documentation: everything the believer knows is true, and the conclusion is still wrong. A claim of the form "A replaces B" needs evidence about B, and every premise above is about A.
This chapter is nine ways of asking for evidence about B, each one a working model, each one measurable, and each one arriving at the same answer from a different direction.
2. How To Use This Chapter
Each of the nine dimensions below is a working test of the claim, and every one answers the same seven questions:
| Facet | What it settles |
|---|---|
| The claim under test | the specific form of "replaces" being examined |
| What replacement would require | the condition that would have to hold |
| The measurement | what the model computes, and from what |
| What the shortcut build reports | the reasoning the misconception uses |
| Why the belief is reasonable | the true fact it is built on |
| What it costs to hold | the engineering decision it leads to |
| What to say instead | the one-sentence correction |
The last row matters most in an interview. "No, that is wrong" is not an answer; the answer is the relationship, and the relationship is more interesting than the misconception.
3. The One-Sentence Model
CXL is a set of protocols that run over the PCIe physical foundation and are selected on a link that PCIe brought up — so the relationship is layering, not succession, and every test in this chapter reaches that conclusion by counting something different.
4. What This Chapter Owns
| Ground | Owner |
|---|---|
| What CXL is, and the problem it addresses | 1.1 |
| The three protocols and what each carries | Module 3 |
| Device types, and which protocols each uses | Module 4 |
| Reviewing an architecture, an RTL block, a testbench | Module 30 |
| Preparing to explain any of it | 30.8 |
| Why "replaces" is the wrong verb | this chapter |
The boundary with 30.8 is worth stating. That chapter says a claim needs a boundary and a trade-off named. This chapter is that discipline applied to one specific claim, in public, with the arithmetic shown — and it is the chapter to read if you want to see what "state the conditions under which it would be true" looks like when it is actually done.
5. Teaching-Model Boundary And Source Discipline
Every model in this chapter is a teaching model, and this chapter needs a stronger statement of that than any other in the batch, because it is the only one that refutes a claim about two named real interconnects.
Every model here computes a property of a CLAIM, not of a protocol. The inputs are layer counts, device counts, condition flags and illustrative latencies. Not one model mentions CXL or PCIe — they speak of a base protocol and an advanced one, a simple attach and a coherent one. The naming happens only in prose, where it is a statement about a general architectural relationship rather than a claim about a document.
Nothing in this chapter states a normative detail of either specification. No opcode, packet layout, bit position, field width, response encoding, training step, negotiation sequence, register definition, timing guarantee, link rate or specification revision appears anywhere — checked by a scan over the finished page as well as by writing the models that way.
| Claim class | How it is marked |
|---|---|
| General architectural relationship | stated plainly, at the level of layering |
| Teaching abstraction | declared in the model header |
| Illustrative parameter | every concrete figure in a model or table |
| Simulator-derived result | quoted from a run and asserted |
| Derived arithmetic | shown with its inputs |
And this is the chapter's own argument, turned on itself: the refutation does not need normative detail. Replacement is settled by counting layers, devices and conditions — none of which requires a specification citation. If the argument needed a bit position to work, it would not be an argument about displacement.
6. Test 1 — Count The Layers Reused
The claim under test. That a newer protocol has displaced the stack it runs on.
What replacement would require. That the layers of the older thing stopped being used.
The measurement. A stack of five layers with four reused and two added:
// RTL 1 - a protocol that reuses most of a stack has not replaced it.
//
// "Replaces" is a claim about DISPLACEMENT: the old thing is gone. A protocol
// that runs on the electrical signalling, the connector, the training sequence
// and the physical layer of an existing interconnect, and adds its own
// semantics above them, has not displaced that interconnect - it has taken a
// dependency on it. The arithmetic is the whole argument.
//
// BAD : "it is faster and newer, so it replaces the old one"
// GOOD : count the layers reused against the layers added, and say which
// of the old ones stopped being needed
//
// TEACHING MODEL. It counts a property of a LAYERING CLAIM using illustrative
// layer counts. It is not a model of CXL or PCIe, and contains no opcode,
// packet layout, field width, encoding, register definition, timing guarantee
// or specification revision from any published standard.
module layer_reuse #(parameter int NEWER_MEANS_REPLACES = 0) (
input logic clk, rst_n,
input logic assess,
input logic [7:0] layers_total, layers_reused, layers_added,
output logic [7:0] layers_displaced, n_assessments, n_overclaims,
output logic [15:0] reuse_pct,
output logic is_replacement, claimed_replacement,
output logic replaces_err
);
logic [31:0] r_q;
logic [7:0] reused_c;
// A stack cannot reuse more layers than it has.
assign reused_c = (layers_reused > layers_total) ? layers_total : layers_reused;
assign layers_displaced = layers_total - reused_c;
assign r_q = (layers_total == 8'd0) ? 32'd0
: (({24'd0, reused_c} * 32'd100) / {24'd0, layers_total});
assign reuse_pct = r_q[15:0];
// The truth: a replacement displaces the whole stack. Reusing any layer of
// it means the old stack is still required for the new one to work.
assign is_replacement = (reused_c == 8'd0) && (layers_total != 8'd0);
// The whole review point: what the claim is judged on.
assign claimed_replacement = (NEWER_MEANS_REPLACES != 0) ? (layers_added != 8'd0)
: is_replacement;
assign replaces_err = assess && claimed_replacement && !is_replacement;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
n_assessments <= 8'd0; n_overclaims <= 8'd0;
end else if (assess) begin
n_assessments <= n_assessments + 8'd1;
if (claimed_replacement && !is_replacement) n_overclaims <= n_overclaims + 8'd1;
end
end
endmodule5 layers, 4 reused, 2 added : reuse=80% displaced=1 replacement=0 newer_says=1Eighty percent reuse and one layer displaced. A protocol that runs on the electrical signalling, the connector, the training sequence and the physical layer of an existing interconnect, and adds its own semantics above them, has not displaced that interconnect — it has taken a dependency on it.
What the shortcut build reports. Replacement, on the grounds that something is newer. That is the entire reasoning, and it is the reasoning in the sentence the chapter is named after.
Why the belief is reasonable. The added layers are the interesting ones. They are what gets written about, demonstrated and asked about in interviews, and the reused ones are invisible precisely because they work.
What it costs to hold. An architecture discussion that treats the physical layer as a solved historical detail, and a bring-up schedule that discovers it is not.
What to say instead. "It reuses most of the stack and adds semantics above it — the arithmetic is four of five layers reused, so the relationship is layering, not succession."
7. Test 2 — Ask What Would Have To Disappear
The claim under test. That the older interconnect is retired.
What replacement would require. That nothing it provides is still needed.
The measurement. An older interconnect providing ten things, of which the newer one also provides six and adds three of its own:
// RTL 2 - what would have to disappear?
//
// The test for "A replaces B" is not whether A is better. It is whether, after
// A arrives, B stops being needed. Count the things B provides that A does not:
// if that count is not zero, B is still there, and the relationship is
// "A depends on B" or "A sits beside B", not replacement.
//
// BAD : list what A adds
// GOOD : list what B still provides that A does not, and count it
//
// TEACHING MODEL. Illustrative counts.
module displacement_test #(parameter int ADDITIONS_COUNT = 0) (
input logic clk, rst_n,
input logic assess,
input logic [7:0] b_provides, a_also_provides, a_adds,
output logic [7:0] still_needed, n_assessments, n_wrong_calls,
output logic [15:0] displaced_pct,
output logic b_retired, claimed_retired,
output logic disp_err
);
logic [31:0] d_q;
logic [7:0] covered;
assign covered = (a_also_provides > b_provides) ? b_provides : a_also_provides;
assign still_needed = b_provides - covered;
assign d_q = (b_provides == 8'd0) ? 32'd100
: (({24'd0, covered} * 32'd100) / {24'd0, b_provides});
assign displaced_pct = d_q[15:0];
// The truth: B is retired only when nothing it provides is still needed.
assign b_retired = (still_needed == 8'd0);
// The whole review point: judging by what A adds rather than what B still does.
assign claimed_retired = (ADDITIONS_COUNT != 0) ? (a_adds != 8'd0) : b_retired;
assign disp_err = assess && claimed_retired && !b_retired;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
n_assessments <= 8'd0; n_wrong_calls <= 8'd0;
end else if (assess) begin
n_assessments <= n_assessments + 8'd1;
if (claimed_retired && !b_retired) n_wrong_calls <= n_wrong_calls + 8'd1;
end
end
endmoduleold provides 10, new covers 6, adds 3 : still_needed=4 retired=0 additions_say=1Four things are still needed and the older thing is therefore still there. The three additions are real, and they are not evidence about the four.
This is the structural error in one line. The additions-count build looks at what A adds and concludes B is retired. The test for "A replaces B" is a count over B, and no fact about A can settle it.
The degenerate case is a decision, not an accident
The run drives an older interconnect that provides nothing, and the model reports 100 percent displaced. That is vacuously true and it is a choice: if the old thing provides nothing, everything it provided has been displaced.
The mutation that changes that answer to zero survived the first campaign, because only non-zero provision counts had been driven. Section 19 records it. The case was reachable, the answer was a decision, and nothing in the testbench had made the decision explicit.
Why the belief is reasonable. Listing what a new thing adds is the natural way to evaluate it, and it is the right way to evaluate whether to use it. It is simply not the test for whether something else has gone away.
What it costs to hold. A roadmap that stops investing in the older attach, for a fleet that still needs it.
What to say instead. "Four of the ten things it provided are still only provided by it, so it has not been retired — the relationship is addition."
8. Test 3 — Notice That There Are Two Questions
The claim under test. That answering the coherence question settles the attachment question.
What replacement would require. That the two questions were the same question.
The measurement. A device that needs both, with only the coherence question answered:
// RTL 3 - "how do I attach it" and "does it need coherence" are two questions.
//
// The misconception collapses two independent decisions into one. Attachment is
// about how a device is enumerated, configured and reached. Coherence is about
// whether its accesses participate in the memory system's ownership rules.
// A device can need one, both, or neither, and answering only the coherence
// question leaves the attachment question unanswered - it does not remove it.
//
// BAD : "coherent attach, so the old attach question is settled"
// GOOD : answer both, and notice that one of the answers is unchanged
//
// TEACHING MODEL.
module two_questions #(parameter int ONE_ANSWER_SETTLES_BOTH = 0) (
input logic clk, rst_n,
input logic assess, needs_attach, needs_coherence,
input logic attach_answered, coherence_answered,
output logic [7:0] questions_open, n_assessments, n_premature,
output logic both_answered, claimed_settled,
output logic question_err
);
logic attach_open, coh_open;
assign attach_open = needs_attach && !attach_answered;
assign coh_open = needs_coherence && !coherence_answered;
assign questions_open = {7'd0, attach_open} + {7'd0, coh_open};
// The truth: the design is settled when every question it raised is answered.
assign both_answered = (questions_open == 8'd0);
// The whole review point.
assign claimed_settled = (ONE_ANSWER_SETTLES_BOTH != 0) ? coherence_answered
: both_answered;
assign question_err = assess && claimed_settled && !both_answered;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
n_assessments <= 8'd0; n_premature <= 8'd0;
end else if (assess) begin
n_assessments <= n_assessments + 8'd1;
if (claimed_settled && !both_answered) n_premature <= n_premature + 8'd1;
end
end
endmodulecoherence answered, attach not : open=1 both_answered=0 one_answer_says=1One question remains open and the one-answer build reports the design settled. That is the misconception in its most consequential form, because it is the form that produces an incomplete design rather than an incorrect opinion.
Attachment and coherence are independent decisions. Attachment is about how a device is enumerated, configured and reached. Coherence is about whether its accesses participate in the memory system's ownership rules. A device can need one, both or neither — and answering only the second leaves the first exactly where it was.
Why the belief is reasonable. The two questions are answered by the same technology in the cases that get discussed most, so they look like one decision in every example somebody has seen.
What it costs to hold. A design review that closes with an unanswered enumeration question, discovered at bring-up by a device nothing can find.
What to say instead. "Coherence is one question and attachment is another. Answering one does not remove the other; it leaves it unchanged."
9. Test 4 — Count The Devices That Gain Nothing
The claim under test. That everything moves to the newer attach.
What replacement would require. That the newer attach serves every device the older one served, at least as well.
The measurement. Twenty devices, four of which participate in the memory system's ownership rules:
// RTL 4 - the devices that still want the simpler attach.
//
// A replacement has to serve everything the old thing served. Most devices in a
// system do not participate in the memory system's ownership rules at all - a
// storage controller, a network adapter, a display output - and for those the
// coherent path is cost without benefit. Counting them is the fastest way to
// see that displacement was never on the table.
//
// BAD : "everything will move to the new attach"
// GOOD : count the devices that gain nothing from it
//
// TEACHING MODEL. Illustrative device counts, not a census of any real system.
module installed_base #(parameter int ASSUME_ALL_MIGRATE = 0) (
input logic clk, rst_n,
input logic assess,
input logic [7:0] devices_total, devices_wanting_coherence,
output logic [7:0] devices_served_better_by_old, n_assessments, n_overclaims,
output logic [15:0] migrate_pct, stranded_pct,
output logic all_migrate, claimed_all_migrate,
output logic base_err
);
logic [31:0] m_q, s_q;
logic [7:0] want_c;
assign want_c = (devices_wanting_coherence > devices_total)
? devices_total : devices_wanting_coherence;
assign devices_served_better_by_old = devices_total - want_c;
assign m_q = (devices_total == 8'd0) ? 32'd0
: (({24'd0, want_c} * 32'd100) / {24'd0, devices_total});
assign s_q = (devices_total == 8'd0) ? 32'd0
: (({24'd0, devices_served_better_by_old} * 32'd100) / {24'd0, devices_total});
assign migrate_pct = m_q[15:0];
assign stranded_pct = s_q[15:0];
// The truth: everything migrates only if nothing is served better by the old.
assign all_migrate = (devices_served_better_by_old == 8'd0) && (devices_total != 8'd0);
// The whole review point.
assign claimed_all_migrate = (ASSUME_ALL_MIGRATE != 0) ? 1'b1 : all_migrate;
assign base_err = assess && claimed_all_migrate && !all_migrate;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
n_assessments <= 8'd0; n_overclaims <= 8'd0;
end else if (assess) begin
n_assessments <= n_assessments + 8'd1;
if (claimed_all_migrate && !all_migrate) n_overclaims <= n_overclaims + 8'd1;
end
end
endmodule20 devices, 4 want coherence : migrate=20% stranded=80% all_migrate=0 assumed=1Twenty percent have something to gain and eighty percent do not. For a storage controller, a network adapter or a display output, the coherent path is cost without benefit — and the counting is the fastest way to see that displacement was never on the table.
The device counts are illustrative parameters, not a census of any real system. The shape of the result is the durable part, and the shape does not depend on the exact fraction: as long as some devices gain nothing, the older attach has a constituency.
Why the belief is reasonable. The devices that gain from coherence are the ones being written about — accelerators, memory expanders, shared-working-set workloads. The other sixteen are boring, which is a statement about coverage rather than about count.
What it costs to hold. A platform plan that provisions for a migration most of the installed base has no reason to make.
What to say instead. "Most devices do not participate in ownership at all, so for them the simpler attach is strictly cheaper. Replacement would require serving them too."
Figure 2 — the eighty percent is the half of the picture the misconception never sees. Both branches end in a correct answer; only one of them is the answer the claim assumes.
10. Test 5 — Ask What Trained The Link
The claim under test. That a link running the newer protocol is not the older one.
What replacement would require. That the newer protocol brings up its own link.
The measurement. A link that trains in its base mode and then selects the advanced one:
// RTL 5 - one physical link, more than one protocol above it.
//
// If a link brings itself up using the older interconnect's own training and
// then selects which protocol will run over it, the older one is not a
// competitor - it is the thing that got the link working. A selector is not a
// replacement; it is evidence of a shared foundation.
//
// BAD : "the link is CXL, so it is not the other thing"
// GOOD : ask what trained it, and what would happen if selection failed
//
// TEACHING MODEL. The two modes here are abstract A and B. No training
// sequence, negotiation step, timing or encoding from any specification appears.
module shared_foundation #(parameter int MODE_IS_THE_LINK = 0) (
input logic clk, rst_n,
input logic start_link, base_trained, select_advanced, assess,
output logic [7:0] mode_now, n_links, n_fallbacks,
output logic link_up, foundation_shared, fell_back,
output logic claimed_independent, mode_err
);
logic base_q, adv_q;
assign link_up = base_q;
// Mode 2 is the advanced protocol; mode 1 is the base one the link trained as.
assign mode_now = base_q ? (adv_q ? 8'd2 : 8'd1) : 8'd0;
// The truth: the advanced mode exists only on top of a trained base link.
assign foundation_shared = adv_q ? base_q : 1'b1;
assign fell_back = base_q && !adv_q;
// The whole review point: whether selecting the advanced mode is taken to
// mean the base one was not involved.
assign claimed_independent = (MODE_IS_THE_LINK != 0) ? adv_q : 1'b0;
assign mode_err = assess && claimed_independent && base_q;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
base_q <= 1'b0; adv_q <= 1'b0; n_links <= 8'd0; n_fallbacks <= 8'd0;
end else begin
if (start_link && base_trained) begin
base_q <= 1'b1;
n_links <= n_links + 8'd1;
end
// The advanced protocol can only be selected on a link that came up.
if (base_q && select_advanced) adv_q <= 1'b1;
if (base_q && !select_advanced && !adv_q) n_fallbacks <= n_fallbacks + 8'd1;
end
end
endmodulebase trained then advanced selected : mode=2 foundation_shared=1 mode_is_link_says=1The advanced mode exists only on top of a trained base link. A selector is not a replacement; it is evidence of a shared foundation — and the selection could not happen at all without the thing it appears to supersede.
The strongest form of this test is the failure case. If selection does not happen, the link is still up in its base mode and the device still works. That is what a foundation does, and a protocol that can be fallen back from is a protocol that was layered on something.
Why the belief is reasonable. From software, a link that has selected the advanced protocol behaves as the advanced protocol. The foundation is invisible once it has done its job, which is the defining property of a good foundation and the reason it gets left out of the mental model.
What it costs to hold. A bring-up plan with no owner for the base-mode training, and a debug session that cannot explain why a link is up and the advanced protocol is not.
What to say instead. "The link trains in the base protocol and then selects what runs over it. If selection fails you still have a working link — which is the clearest possible evidence that one is built on the other."
11. Test 6 — Hold The Rate Equal
The claim under test. That the difference between the two is speed.
What replacement would require. That speed were the axis at all.
The measurement. Two protocols at the same signalling rate, differing in whether their accesses participate in ownership:
// RTL 6 - the same signalling rate does not make two protocols the same, and
// a faster one does not make the slower one obsolete.
//
// Speed is the argument people reach for and it is the weakest one available.
// Two protocols sharing a physical rate deliver the same bits per second and
// differ entirely in what those bits MEAN - whether an access participates in
// ownership, whether a write must be seen by others, whether a read may be
// served from a stale copy. Semantics, not rate, is the axis.
//
// BAD : compare the headline rates
// GOOD : hold the rate equal and ask what changes
//
// TEACHING MODEL. Rates are illustrative integers in arbitrary units.
module rate_is_not_semantics #(parameter int RATE_DECIDES = 0) (
input logic clk, rst_n,
input logic compare,
input logic [7:0] rate_a, rate_b,
input logic a_participates, b_participates,
output logic [7:0] n_comparisons, n_shallow,
output logic same_rate, semantics_differ, meaningful_difference,
output logic claimed_difference, differ_err
);
assign same_rate = (rate_a == rate_b);
assign semantics_differ = (a_participates != b_participates);
// The truth: the difference that matters is in the semantics.
assign meaningful_difference = semantics_differ;
// The whole review point: judging the comparison on the rate instead.
assign claimed_difference = (RATE_DECIDES != 0) ? !same_rate : semantics_differ;
// A comparison concluded a difference the rates cannot support, or missed one
// the semantics do.
assign differ_err = compare && (claimed_difference != meaningful_difference);
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
n_comparisons <= 8'd0; n_shallow <= 8'd0;
end else if (compare) begin
n_comparisons <= n_comparisons + 8'd1;
if (claimed_difference != meaningful_difference) n_shallow <= n_shallow + 8'd1;
end
end
endmodulesame rate, different participation : same_rate=1 semantics_differ=1 rate_says=0Same bits per second, entirely different meaning. Whether an access participates in ownership, whether a write must be seen by others, whether a read may be served from a stale copy — none of that is a rate, and the rate-decides build reports no meaningful difference between two protocols that differ in every way that matters.
Speed is the argument people reach for and it is the weakest one available. It is also the one most easily refuted in an interview, because holding the rate equal takes one sentence and the difference does not go away.
Why the belief is reasonable. Rates are the numbers on every comparison chart, and they are comparable across generations in a way semantics are not. A number that can be compared gets compared.
What it costs to hold. A selection decision made on a bandwidth figure for a workload whose problem was never bandwidth.
What to say instead. "Hold the rate equal and ask what changes. The answer is what the bits mean, which is the axis the comparison is actually on."
12. Test 7 — Price The Coherence
The claim under test. That the coherent path is better.
What replacement would require. That it were better for every workload.
The measurement. A base access cost of 100 units, a coherence cost of 20, and a workload sharing 10 percent of its accesses:
// RTL 7 - coherence is not free, which is why it is not universal.
//
// Participating in the memory system's ownership rules costs something on the
// access path: a lookup, a possible interrogation of other holders, and a wait
// for the answer. For a workload that shares data that cost buys correctness
// nobody wants to write by hand. For a workload that streams, it is pure
// overhead - which is the whole reason a simpler attach continues to exist.
//
// BAD : "coherent is better"
// GOOD : state the overhead, state the sharing rate, and find the crossover
//
// TEACHING MODEL. All latencies are illustrative integers in arbitrary units.
// None is a CXL or PCIe figure.
module coherence_cost #(parameter int COHERENT_ALWAYS_WINS = 0) (
input logic clk, rst_n,
input logic evaluate,
input logic [7:0] base_lat, coherence_lat, sharing_pct,
output logic [15:0] coherent_cost, simple_cost, overhead_pct,
output logic [7:0] n_evals, n_wrong_picks,
output logic coherent_better, picked_coherent,
output logic lat_err
);
logic [31:0] o_q, c_q, s_q;
// Coherent: every access pays the ownership lookup.
assign c_q = {24'd0, base_lat} + {24'd0, coherence_lat};
// Simple: no lookup, but shared data must be resolved in software, which is
// charged here as an illustrative penalty proportional to the sharing rate.
assign s_q = {24'd0, base_lat}
+ (({24'd0, sharing_pct} * {24'd0, coherence_lat} * 32'd3) / 32'd100);
assign coherent_cost = (c_q > 32'd65535) ? 16'd65535 : c_q[15:0];
assign simple_cost = (s_q > 32'd65535) ? 16'd65535 : s_q[15:0];
assign o_q = (base_lat == 8'd0) ? 32'd0
: (({24'd0, coherence_lat} * 32'd100) / {24'd0, base_lat});
assign overhead_pct = o_q[15:0];
// The truth: coherent wins when it costs less than resolving sharing by hand.
assign coherent_better = (coherent_cost < simple_cost);
// The whole review point.
assign picked_coherent = (COHERENT_ALWAYS_WINS != 0) ? 1'b1 : coherent_better;
assign lat_err = evaluate && picked_coherent && !coherent_better;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
n_evals <= 8'd0; n_wrong_picks <= 8'd0;
end else if (evaluate) begin
n_evals <= n_evals + 8'd1;
if (picked_coherent && !coherent_better) n_wrong_picks <= n_wrong_picks + 8'd1;
end
end
endmodulebase 100, coherence 20, sharing 10% : coherent=120 simple=106 overhead=20%The simple attach wins at this sharing rate and loses at a higher one. Coherent pays the ownership lookup on every access — 100 + 20 = 120. The simple attach pays nothing on most accesses and pays a hand-written synchronisation cost on the shared fraction, which at 10 percent sharing comes to 106.
All latencies here are illustrative integers in arbitrary units. None is a CXL or PCIe figure, and the model header says so. The crossover is the durable result, not the numbers that produce it.
The crossover is the answer, and it is derivable
Coherent wins when 20 is less than the synchronisation cost the simple path pays, which in this model is sharing x 3 x coherence over 100. Setting them equal gives a crossover at roughly 33 percent sharing — below it the simple attach wins, above it the coherent one does.
That single number is what turns a preference into an engineering answer, and it is the part 30.8 section 11 identifies as the one most often left out.
Why the belief is reasonable. Coherence removes a class of bugs that are genuinely awful to write around by hand. For a sharing workload it is unambiguously the right answer, and those are the workloads that motivate the technology.
What it costs to hold. Overhead on every access of a streaming workload, bought to solve a problem that workload does not have.
What to say instead. "Coherence costs a lookup on the access path. Below roughly a third sharing the simpler attach wins; above it the coherent one does. That crossover is why both exist."
13. Test 8 — Name The Device Kind First
The claim under test. That one attach serves everything.
What replacement would require. That one answer fit all device kinds.
The measurement. A bulk-moving device offered a coherent attach:
// RTL 8 - device kinds, and the one the misconception forgets.
//
// A device that only moves bulk data in and out wants the cheapest possible
// attach. A device that shares working data with the processor wants ownership
// participation. A device that presents memory wants something different again.
// Three kinds, three answers - and "replaces" would require one answer to serve
// all three, which is the claim the taxonomy disproves.
//
// BAD : one attach for everything
// GOOD : name the kind, then name the attach it wants
//
// TEACHING MODEL. Three abstract device kinds. No device type, class code or
// capability from any specification appears.
module device_kinds #(parameter int ONE_ANSWER_FITS_ALL = 0) (
input logic clk, rst_n,
input logic assess,
input logic [7:0] kind,
output logic [7:0] attach_wanted, n_devices, n_misfitted,
output logic fits, claimed_fits,
output logic tier_err
);
// Kind 1 bulk mover, kind 2 sharer, kind 3 memory presenter.
// Attach 1 simple, 2 coherent, 3 memory-semantics.
assign attach_wanted = (kind == 8'd1) ? 8'd1
: (kind == 8'd2) ? 8'd2
: (kind == 8'd3) ? 8'd3 : 8'd0;
// The truth: a single coherent attach fits only the sharer.
assign fits = (attach_wanted == 8'd2);
// The whole review point: assuming the coherent attach suits every kind.
assign claimed_fits = (ONE_ANSWER_FITS_ALL != 0) ? (kind != 8'd0) : fits;
assign tier_err = assess && claimed_fits && !fits;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
n_devices <= 8'd0; n_misfitted <= 8'd0;
end else if (assess) begin
n_devices <= n_devices + 8'd1;
if (claimed_fits && !fits) n_misfitted <= n_misfitted + 8'd1;
end
end
endmodulea bulk mover offered a coherent attach : wants=1 fits=0 one_attach_says=1Three kinds, three answers. A device that only moves bulk data in and out wants the cheapest possible attach. A device that shares working data with the processor wants ownership participation. A device that presents memory wants something different again — and the run drives that third case too, where a coherent attach does not fit either.
"Replaces" would require one answer to serve all three, which is the claim the taxonomy disproves without any argument at all. Three kinds is the refutation.
The run also drives an unrecognised kind, and the model reports that nothing fits rather than defaulting to something. A taxonomy that always produces an answer is not a taxonomy, which is the same property 30.7 section 8 requires of an ordering.
Why the belief is reasonable. Most introductory material discusses the sharing device, because it is the one the technology was built for. Two of the three kinds are underrepresented in the material rather than in the world.
What it costs to hold. An attach chosen for a device before anybody asked what the device does.
What to say instead. "Name the kind first. A bulk mover, a sharer and a memory presenter want three different things, and no single attach is the answer to all three."
14. Test 9 — Write Down What Would Have To Be True
The claim under test. All of them, at once.
What replacement would require. Four conditions, and all four.
| Condition | Would have to be true |
|---|---|
| the stack is not reused | the newer protocol brings its own physical foundation |
| every device is served | including the ones that gain nothing from coherence |
| the older attach is retired | nothing it provides is still only provided by it |
| no workload is worse off | including the streaming ones |
// RTL 9 - what would have to be true for "replaces" to hold?
//
// The disciplined way to handle a sweeping claim is to write down the
// conditions under which it WOULD be true, and then check them. Four are enough
// here: the stack is not reused, every device is served, the older attach is
// retired, and no workload is worse off. Checking them turns an argument into
// an arithmetic problem, and the answer is usually visible immediately.
//
// BAD : argue about whether it replaces it
// GOOD : list what would have to be true, and count how many are
//
// TEACHING MODEL.
module replacement_conditions #(parameter int MOST_IS_ENOUGH = 0) (
input logic clk, rst_n,
input logic assess,
input logic stack_not_reused, all_devices_served,
input logic old_attach_retired, no_workload_worse,
output logic [7:0] conditions_met, n_assessments, n_overclaims,
output logic [15:0] met_pct,
output logic would_replace, claimed_replace,
output logic repl_err
);
logic [31:0] m_q;
assign conditions_met = {7'd0, stack_not_reused} + {7'd0, all_devices_served}
+ {7'd0, old_attach_retired} + {7'd0, no_workload_worse};
assign m_q = ({24'd0, conditions_met} * 32'd100) / 32'd4;
// No clamp: four one-bit values over four cannot exceed a hundred.
assign met_pct = m_q[15:0];
// The truth: a replacement needs every condition, not most of them.
assign would_replace = (conditions_met == 8'd4);
// The whole review point.
assign claimed_replace = (MOST_IS_ENOUGH != 0) ? (conditions_met >= 8'd2)
: would_replace;
assign repl_err = assess && claimed_replace && !would_replace;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
n_assessments <= 8'd0; n_overclaims <= 8'd0;
end else if (assess) begin
n_assessments <= n_assessments + 8'd1;
if (claimed_replace && !would_replace) n_overclaims <= n_overclaims + 8'd1;
end
end
endmoduleThe measurement. Two of the four conditions met:
2 of 4 conditions : met=50% would_replace=0 most_is_enough_says=1Fifty percent, and the most-is-enough build calls it a replacement. That is the disciplined form of the misconception — not a refusal to check, but a willingness to round up.
This is the technique worth taking away from the chapter, and it generalises to every sweeping claim you will be handed:
Write down the conditions under which the claim WOULD be true, then check them. The argument becomes arithmetic, and the answer is usually visible immediately.
Why the belief is reasonable. Nobody writes the four conditions down, because the claim arrives as a sentence rather than as a proposition. Once it is a proposition it is trivially checkable, and it takes about a minute.
What it costs to hold. A strategy argument that runs for a year and could have been settled in the first meeting by a list.
What to say instead. "Here are the four things that would have to be true. Two of them are, and the other two are not — so the relationship is layering plus addition, not replacement."
15. The Misconception Assembled
Nine dimensions, one summary — and the same trap this batch has found at every level.
// RTL 10 - the misconception examined. Nine dimensions, one summary.
// "It is newer and faster" is bit 0: two true statements, and one sixth of an
// argument about replacement.
module mis_review_signoff #(parameter int NEWER_IS_PROOF = 0) (
input logic clk, rst_n,
input logic review,
input logic newer_and_faster, stack_displaced, both_questions_answered,
input logic every_device_served, cost_stated, conditions_checked,
output logic [5:0] fail_mask,
output logic [15:0] conditions_met, sound_pct,
output logic sound,
output logic [7:0] n_reviews, n_sound, n_claimed,
output logic mis_err
);
logic [31:0] s_q;
logic truly_sound, claimed;
assign fail_mask[0] = ~newer_and_faster;
assign fail_mask[1] = ~stack_displaced;
assign fail_mask[2] = ~both_questions_answered;
assign fail_mask[3] = ~every_device_served;
assign fail_mask[4] = ~cost_stated;
assign fail_mask[5] = ~conditions_checked;
assign conditions_met = {15'd0, newer_and_faster} + {15'd0, stack_displaced}
+ {15'd0, both_questions_answered} + {15'd0, every_device_served}
+ {15'd0, cost_stated} + {15'd0, conditions_checked};
assign s_q = ({16'd0, conditions_met} * 32'd100) / 32'd6;
// No clamp: six one-bit values over six cannot exceed a hundred.
assign sound_pct = s_q[15:0];
assign truly_sound = (fail_mask == 6'd0);
assign claimed = (NEWER_IS_PROOF != 0) ? newer_and_faster : truly_sound;
assign sound = claimed;
assign mis_err = review && !truly_sound && claimed;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
n_reviews <= 8'd0; n_sound <= 8'd0; n_claimed <= 8'd0;
end else if (review) begin
n_reviews <= n_reviews + 8'd1;
if (truly_sound) n_sound <= n_sound + 8'd1;
if (claimed) n_claimed <= n_claimed + 8'd1;
end
end
endmoduleThe measurement. Two views of the same argument:
the stack was not displaced : mask=000010 met=5 sound=83%
it is newer and faster, and nothing else : mask=111110 met=1 sound=16%The first line is a serious argument with one condition unmet — bit 1, the stack was not displaced. That single unmet condition is fatal to the claim, and it is worth noticing that five of six can be met and the conclusion still fail: displacement is not a majority vote.
The second line is the misconception itself. Bit 0 is clear — it is newer and faster, and both of those are true — and not one of the other five was checked.
Sixteen percent of an argument, and it is the most-repeated sentence about CXL there is.
Figure 4 — bit 0 is the only one the misconception ever evaluates, and it evaluates to true. That is why the belief is so stable: the one thing it checks, it checks correctly.
16. Quantitative Reasoning
Every figure here is a teaching parameter or a value derived from one and asserted by the testbench. None is a measurement of a real system, and none is a figure from either specification.
Layer reuse, derived. Five layers with four reused gives 4 x 100 / 5 = 80 percent reuse and 5 − 4 = 1 layer displaced. The general form is displaced = total − reused, and "replaces" requires that number to equal the total. At 80 percent reuse it is off by a factor of four.
Displacement, derived. An older interconnect providing 10 things, of which the newer covers 6, leaves 10 − 6 = 4 still needed and 6 x 100 / 10 = 60 percent displaced. The three additions do not appear in either figure, which is the arithmetic statement of the structural error: still needed is a function of B and the coverage of B, and no term in it is a property of A alone.
Open questions. Two independent decisions and one answered leaves 1 open. The general form is open = (needs attach and not answered) + (needs coherence and not answered), and answering one term cannot change the other — which is the whole content of the item, expressed as an addition.
Installed base, derived. Twenty devices with four participating gives 4 x 100 / 20 = 20 percent migrating and 16 x 100 / 20 = 80 percent with nothing to gain. The stranded fraction is the complement, and replacement would require it to be zero. At sixteen devices it is not close to zero, and the conclusion survives any plausible change to the fraction: the claim needs no device to prefer the older attach.
Link foundation. The advanced mode is reachable only from a trained base link, so the reachable mode set is 2 and mode 2 requires mode 1 to have happened. Mode 2 without mode 1 is not a state the model can reach, and that unreachability is the argument.
Rate against semantics. Two protocols at the same rate, differing in participation, have a rate difference of 0 and a semantic difference of 1. A comparison on rate returns "no meaningful difference" between two protocols that differ in everything that matters — which is the sharpest single number in the chapter, because it is a zero produced by looking at the wrong axis.
Coherence cost, derived. A base access cost of 100 with a coherence cost of 20 gives a coherent path of 100 + 20 = 120 on every access. The simple path pays the base cost plus a hand-written synchronisation cost on the shared fraction: at 10 percent sharing that is 10 x 20 x 3 / 100 = 6, for a total of 106. The overhead is 20 x 100 / 100 = 20 percent.
The crossover, derived. The two are equal when coherence = sharing x 3 x coherence / 100, which reduces to sharing = 33 percent and is independent of the coherence cost itself. Below a third sharing the simple attach wins; above it the coherent one does. The factor of three is the model's illustrative penalty for hand-written synchronisation, declared as such — and the structure of the result, a crossover that exists, does not depend on it.
Device kinds. Three kinds and three attaches, with exactly one correct pairing each. A single attach applied to all three is correct for one of three, or 33 percent — and an unrecognised kind maps to nothing, which is a taxonomy behaving correctly rather than a gap.
Conditions, derived. Four conditions with two met is 2 x 100 / 4 = 50 percent, and the claim requires four of four. A conjunction is not a percentage, which is precisely the error the most-is-enough build makes.
The sign-off arithmetic. Six conditions; five met is 5 x 100 / 6 = 83 percent under integer division, and one met is 100 / 6 = 16 percent. Five of six is still false, because displacement is a conjunction.
17. Verification Method
Order of work
compile → inspect warnings → legal baseline → reset → boundaries → simultaneous events → abuse and error cases → configuration contrasts → structural gates → PASS → mutation campaign
A mutation campaign on a failing baseline is invalid, and the campaign in this chapter ran against a green one. The baseline was re-run after every testbench modification before the campaign was re-run.
Independent oracles
| Model | Oracle |
|---|---|
| layer reuse | 5 layers, 4 reused → 80 percent, 1 displaced, not a replacement |
| displacement test | provides 10, covered 6, adds 3 → 4 still needed, not retired |
| two questions | coherence answered, attach not → 1 open, not settled |
| installed base | 20 devices, 4 participating → 20 percent migrate, 80 percent stranded |
| shared foundation | base trained then advanced selected → mode 2, foundation shared |
| rate is not semantics | equal rates, different participation → same rate, semantics differ |
| coherence cost | base 100, coherence 20, sharing 10 → coherent 120, simple 106 |
| device kinds | bulk mover → wants simple; coherent attach does not fit |
| replacement conditions | 2 of 4 → 50 percent, would not replace |
| sign-off | five of six → 83 percent; one of six → 16 percent |
chkv prints got against expected, which is what lets an oracle be wrong out loud. In this chapter it caught one, mine, recorded in section 18.
X and Z rejected explicitly
chk(c, …) tests c !== 1'b1, so an X-valued condition fails rather than passing. chkv(got, exp, …) reduces the result and reports an explicit X/Z failure before comparing. The undriven-output sweep 30.7 produced was run over these ten models with no findings.
Pulses are latched, never sampled
Every evidence output — replaces_err, disp_err, question_err, base_err, mode_err, differ_err, lat_err, tier_err, repl_err, mis_err — is caught by a continuous always @(posedge clk) monitor into a sticky bit, asserted outside any conditional, in a window containing a clock edge.
Stimulus never lands on the active edge, and reset is released after it
step_clk is @(posedge clk); #1;, and reset release lands one delta after the edge. Carried forward from the race 30.5 exposed.
Both builds are always instantiated
Every model has both its counting build and its shortcut build wired to the same stimulus. In nine of the ten, the shortcut build computes the honest figure internally and reports a different conclusion from it — the difference under review is never the information available, always the inference drawn.
Safety, liveness and performance kept apart
Safety-of-claim is this chapter's safety class. A replacement is never reported while any part of the older thing is still needed. A design is never reported settled with a question open. An attach is never reported as fitting a kind it does not serve. None requires an assumption.
Liveness — one claim, stated with its assumption: the link comes up, assuming the base protocol trains. That assumption is the chapter's central point, and the model makes it structural rather than asserted.
Performance — one model is a cost comparison, and it is explicitly a performance claim with a crossover rather than a correctness claim. Section 12 states the crossover, and section 16 derives it.
18. Baseline Defects Found Before Mutation
RTL defects — none. Testbench defects — none.
The ten models compiled and ran clean on the first attempt. That is recorded with the caveat 30.7 earned: a green first run is evidence about the tests that existed at that moment, not about the models. The structural gates and the undriven-output sweep were run separately, and the mutation campaign is what actually probed them — it found two things the first run did not.
Wrong oracles — one, mine
The fallback-cycle count. I expected two fallback cycles while the link was up in its base mode with no advanced mode selected; there was one, because the counter only advances on a clock edge and the stimulus had spent one.
It is the same class as every oracle error in this batch: a when error rather than an arithmetic one. Across 30.5, 30.6, 30.7 and this chapter, fifteen wrong oracles, and fourteen of them were about when a value is available rather than about what it should be.
Coverage gaps found by the structural gates
| Gate | Finding | Closed by |
|---|---|---|
outscan | 4 unasserted output nets, all in the shared-foundation model | value assertions |
banned, excheck, splitcheck, domcheck, displaycheck | none | — |
All four gaps were in one model, which is the pattern worth noting: unasserted outputs cluster in the model with the most state, because its outputs are the ones whose values depend on when you look.
Source discipline, checked rather than assumed
This chapter refutes a claim about two named real interconnects, which makes it the highest-risk chapter in the batch for stating a normative detail that is not defensible. Three things were done about that, and they are recorded as verification steps rather than as intentions:
- Every model computes a property of a CLAIM, not of a protocol. The inputs are layer counts, device counts, condition flags and illustrative latencies.
- No opcode, packet layout, bit position, field width, response encoding, training step, negotiation sequence, register definition, timing guarantee or specification revision appears anywhere — checked by a scan over the finished page as well as by writing the models that way.
- Neither protocol is named inside any model. The models speak of a base protocol and an advanced one, a simple attach and a coherent one.
Compiler-warning findings
Under -Wall the ten models produce no truncation, select-width, signedness, cast, multiplication-width, shift-width or loop-width warnings.
One width result was reasoned rather than trusted. The coherence-cost model multiplies three 8-bit quantities in a 32-bit context and divides by 100. At the illustrative ranges used the product peaks well under 32 bits, and the result is clamped to 16 bits explicitly rather than relying on the assignment to truncate. The tool warns on none of this, which is 30.2 section 9's rule applied to the arithmetic of a Foundation page.
Simulator constraints
Icarus Verilog 13.0 rejects ref task arguments, carried forward from every chapter in this module.
19. Mutation Testing
71 mutations attempted, 71 non-equivalent, 71 killed. Zero unexplained survivors, zero equivalent mutants withdrawn.
| Reported separately | Count |
|---|---|
| Mutants attempted | 71 |
| Withdrawn as equivalent | 0 |
| Non-equivalent mutants | 71 |
| Killed | 71 |
| Unexplained survivors | 0 |
| Model | Dimension | Muts |
|---|---|---|
| m1 | layer reuse | 8 |
| m2 | displacement test | 7 |
| m3 | two questions | 6 |
| m4 | installed base | 7 |
| m5 | shared foundation | 7 |
| m6 | rate is not semantics | 6 |
| m7 | coherence cost | 7 |
| m8 | device kinds | 5 |
| m9 | replacement conditions | 6 |
| m10 | review sign-off | 12 |
Sixty-nine of seventy-one on the first run, and both survivors were classified before anything was changed.
Survivor 1 — a degenerate input whose answer was a decision
The mutation changed the displacement model's answer for an older interconnect that provides nothing, from 100 percent displaced to zero. It survived because only non-zero provision counts had been driven.
Classified as a stimulus gap, not an equivalent mutant — the separating input exists and is reachable. And the case is a decision rather than an accident: if the old thing provides nothing, then everything it provided has been displaced, vacuously. The model had made that decision and the testbench had never recorded it, which is the same defect shape as an undocumented contract, one level down.
Survivor 2 — a counter asserted before the last state transition
The mutation dropped the guard that stops fallback cycles being counted once the advanced mode is up. The state was driven — the run continues past selection — and the fallback counter was only ever asserted before it.
Classified as a missing checker. The stimulus reached the state; nothing looked at it there.
Both gaps are at the edges of the run
Neither is a weakness in a checker. One is a degenerate input at the top of a model and one is a state at the end of a sequence, and both are places the stimulus stopped one step short of where the model kept going.
That is a distinct shape from the boundary family the rest of the batch produced, and it is worth naming separately:
Drive the degenerate input, and assert every counter after the last state transition as well as before it.
Across the five chapters in this batch the survivor families are now: a boundary approached and never landed on (eighteen), the middle of a conjunction (one), a decisive experiment the model could not express (two), a model ambiguity (one), checkers placed where the builds agree (five), and the edges of the run (two).
The classification rule
Never add an assertion for a survivor before classifying it.
| Class | Means, and what to do |
|---|---|
| Equivalent | no input tells the two apart — withdraw it, never count a kill |
| Stimulus gap | the case is never driven — extend the stimulus |
| Missing checker | the case is driven and nothing looks — add the checker |
| Vacuous checker | the check cannot fail — fix the check, not the design |
| Model cannot express it | the decisive experiment has no representation — rebuild the model |
| Model ambiguity | the model has not decided what it means — decide, then re-mutate |
| Unreachable | its guard never holds — fix the guard |
| Masked | another mechanism hides it — expose it, or say why you cannot |
| Coincidental | the arithmetic happens to agree — change the stimulus |
| Missing config | the build that differs is never built — instantiate it |
| Other | anything else — state it precisely |
20. Synthesis And Implementation Reality
These models are not meant to be synthesised, and saying so is part of the source discipline this chapter is subject to. What follows is the honest reading of what they would cost, because the reasoning transfers to the instrumentation a real platform would want.
Every counting model here is a small adder and a comparator. A four-condition conjunction is a four-input AND. A parts-still-needed count is an 8-bit subtraction with a clamp. The entire chapter, as hardware, is smaller than one entry of any queue in 30.6.
The percentage models are the only arithmetic of any size, and the conclusion is the same one every chapter in this batch reached: publish the numerator and the denominator and let the reader divide. Four dividers exist here because the models publish a percentage the prose quotes; a real platform would publish "16 of 20" and nothing else, which is smaller, exact, and not rounded.
The device-kind model is a small decode, three comparators and a priority encode, and its most important property costs nothing: an unrecognised kind maps to zero rather than to a default. A default there would be the misconception in gates.
The foundation model is two flops — base trained, advanced selected — and the property that matters is structural rather than logical: mode 2 is unreachable without mode 1, and no amount of gate-level cleverness changes that. An architecture where the dependency is in the state graph does not need an assertion to enforce it.
The dual-build structure costs exactly double, and it is a verification cost rather than a design cost. No shipped design instantiates two versions of itself — it is what makes "the information was available and the inference differed" checkable rather than asserted.
No area, frequency or power figures appear in this chapter, because none was measured and none would mean anything if it had been.
21. Silicon Observability
A platform can publish the evidence this chapter counts, and several of these are things a real system already has or could have for almost nothing. None of them is a specification-mandated register, and none is claimed to be.
| Telemetry | What it would settle |
|---|---|
| the mode a link is operating in, readable per link | whether the advanced protocol was selected, or the link fell back |
| a fallback counter per link | how often selection does not happen, which the misconception predicts is never |
| a count of attached devices by kind | the migrate-against-stranded ratio, as two numbers rather than a belief |
| the base-trained and advanced-selected bits, separately | that one is a precondition of the other, observably |
| per-device attach type as configured | whether an attach was matched to a kind or assumed |
| shared-access fraction per device | which side of the coherence crossover a workload is on |
The second and the last are the two that change decisions. A fallback counter reading non-zero is a direct observation that the base protocol is load-bearing; a shared-access fraction below a third is a direct observation that a coherent attach is buying overhead.
Publish counts, not percentages — the same conclusion 30.5 reached about denominators and 30.8 reached about revision coverage. "Sixteen of twenty" is exact and "eighty percent" is rounded, and only the first lets a reader compute a different ratio than the one the publisher chose.
One pattern is worth reading for. A fleet where every link reports the advanced mode and the fallback counters all read zero is a fleet where the base protocol has done its job perfectly and invisibly — which is exactly the condition under which people conclude it is not there.
22. DebugLabs
These labs debug decisions made from the misconception, using the structure the rest of the module uses to debug a design. The symptom is always a project that went wrong, and the root cause is always a claim about A used as evidence about B.
Lab 1 — A platform plan budgeted for a migration that did not happen
Symptom. A platform roadmap provisions coherent attach capacity for the whole device population. Two years in, most devices have not moved.
Evidence. The plan's justification lists what the newer attach adds. It contains no count of devices that would gain from it.
Hypothesis. The plan tested capability, not displacement.
Investigation. Count the attached devices by kind. Four of twenty participate in ownership.
Root cause. A claim about A used as evidence about B. Every fact in the justification is true and none of them is about the installed base.
Fix. Provision for the fraction that gains, and keep the simpler attach as a first-class path.
Prevention. Any "everything will move" claim needs a count of what will not.
Observability. Devices by kind, as two numbers. Not a percentage — the counts, so the reader can compute their own ratio.
Lab 2 — A link is up and the advanced protocol is not, and nobody owns it
Symptom. A link trains and works, and the advanced protocol never comes up. The bring-up team has no owner for the problem.
Evidence. The mental model in the plan was "the link is the advanced protocol", so base-mode training had no assignee.
Hypothesis. The foundation was assumed rather than owned.
Investigation. Read the two state bits separately. Base trained, advanced not selected.
Root cause. A model in which selection is not a step, because the foundation was believed to have been replaced.
Fix. Own the base training as its own bring-up milestone, with its own exit criteria.
Prevention. Ask what trains the link, at the planning stage. The answer is a name on a schedule.
Observability. The two bits, readable separately, and a fallback counter.
Lab 3 — A design review closed with the enumeration question unanswered
Symptom. A device cannot be found by system software after integration. Every coherency question in the review was answered thoroughly.
Evidence. The review minutes contain a long coherence discussion and no enumeration discussion.
Hypothesis. The two questions were collapsed into one.
Investigation. Ask how the device is discovered and configured. Nobody has an answer, and nobody noticed.
Root cause. Answering the coherence question was read as settling the design. It settled one question and left the other exactly where it was.
Fix. Two agenda items, always, with two owners.
Prevention. A review checklist with attachment and coherence as separate rows. They are separate because they are separate.
Observability. Per-device attach type as configured, readable — which makes an unanswered attach question visible before bring-up.
Lab 4 — A streaming workload got slower after an upgrade
Symptom. A bulk-data device moved to a coherent attach and its throughput fell.
Evidence. The move was justified as an upgrade. No sharing fraction was measured.
Hypothesis. The workload pays the ownership lookup and shares almost nothing.
Investigation. Measure the shared-access fraction. It is under five percent.
Root cause. A coherent path bought to solve a problem the workload does not have. The overhead is on every access; the benefit is on the fraction that shares.
Fix. Move it back, and record the sharing fraction as the selection criterion.
Prevention. State the crossover before choosing. Below roughly a third sharing in this model's terms, the simpler attach wins.
Observability. Shared-access fraction per device. It is the number that decides the question and it is rarely measured.
Lab 5 — A comparison was made on bandwidth and the problem was not bandwidth
Symptom. A protocol was selected on a headline rate. The latency-sensitive workload that motivated the project did not improve.
Evidence. The comparison table has one column: rate.
Hypothesis. The comparison was made on the wrong axis.
Investigation. Hold the rate equal and ask what differs. Everything that mattered to the workload.
Root cause. Rate is comparable across generations and semantics are not, so the comparable thing got compared.
Fix. Compare on what the bits mean: participation, ordering, what a read may return.
Prevention. Hold the rate equal as a review question. If the two options are still different, rate was not the axis.
Observability. None for this one — it is a review discipline. The artefact is a comparison table with more than one column.
Lab 6 — An attach was chosen before anybody asked what the device does
Symptom. A memory-presenting device was given a coherent attach and does not work as intended.
Evidence. The attach was chosen from a platform default.
Hypothesis. The device kind was never named.
Investigation. Ask which of the three kinds it is. It is the third, and the attach serves the second.
Root cause. One answer applied to a taxonomy with three entries.
Fix. Name the kind, then name the attach it wants.
Prevention. A device-kind field in the platform inventory, filled in before an attach is chosen.
Observability. Attached devices by kind, and attach type as configured. A mismatch between the two columns is a one-query diagnosis.
Lab 7 — A strategy argument ran for a year with no exit criteria
Symptom. Two groups argue about whether to invest in the older attach. The argument does not converge.
Evidence. Neither side has written down what would settle it.
Hypothesis. The claim was never turned into a proposition.
Investigation. Write the four conditions replacement would require. Check them. Two hold.
Root cause. A sentence treated as a position rather than as a claim with conditions.
Fix. The list, in the first meeting.
Prevention. For any sweeping claim, ask what would have to be true. It converts an argument into arithmetic, and it takes about a minute.
Observability. The list itself. A strategy claim with no condition list is 30.8's unbounded claim at organisational scale.
Lab 8 — Every link reports the advanced mode, so the foundation was declared unnecessary
Symptom. A fleet review concludes the base protocol can be deprecated. Every link in the fleet reports the advanced mode.
Evidence. The fallback counters were not read. They are all zero.
Hypothesis. Zero fallbacks is being read as "the base protocol is not used", and it means "the base protocol worked every time".
Investigation. Read the base-trained bit. It is set on every link, on every power-up, before the mode was selected.
Root cause. Silence read as absence. It is the same error as 30.7 section 14 — a thing that never fails looks like a thing that is not there.
Fix. Read the precondition bit, not only the outcome.
Prevention. A foundation working perfectly is indistinguishable from a foundation that was removed, unless you measure the precondition.
Observability. Base-trained and advanced-selected as two separate bits. The whole diagnosis is that the first is set.
23. Coverage Reasoning
Coverage of an argument has the same failure mode as coverage of a design: it measures what was considered, not whether anything was checked.
Four coverage models are worth keeping over any claim of this shape:
Premise-direction coverage. A bin per premise on whether it is a statement about A or about B. The bin the misconception can never fill is "a checked statement about B", and that empty bin is the proof — every premise in the belief is about A.
Condition coverage on the conjunction. Four conditions, and four cases with exactly one unmet. Section 15 shows that five of six can be met and the conclusion still fail; a conjunction has no partial credit, and this model is what makes that checkable rather than asserted.
Device-kind coverage. Three kinds, plus an unrecognised one. The cell that matters is the unrecognised kind, because a taxonomy that answers there is defaulting rather than classifying.
Degenerate-input coverage. A bin for zero on every count in the argument: zero layers, zero devices, zero things provided. This chapter's first mutation survivor lived in exactly one of those cells, and the cell was reachable.
The bin the shortcut build cannot hit is the most valuable bin in any model. In section 6 it is "high reuse reported as not a replacement". In section 9 it is "some devices prefer the older attach". In section 12 it is "the simple path wins". Each is unreachable in the shortcut build and trivial in the counting one.
24. How This Appears In Real Engineering
The misconception is not confined to conversation, and every failure in section 22 is a real class of project decision.
Roadmaps are written from capability lists, because that is what a technology brief contains, and a capability list cannot settle a displacement question.
Bring-up plans inherit their milestones from the mental model in the architecture document. If the foundation is believed to be replaced, nobody is assigned to bring it up.
Design reviews collapse attachment and coherence because the two are answered together in every example in the introductory material, and reviews are short.
Selection decisions are made on the only comparable number, which is a rate, for workloads whose problem is semantic.
Platform defaults are applied to device kinds nobody classified, because classifying is a step and a default is not.
Strategy arguments run without exit criteria because a claim arrives as a sentence, and turning it into a proposition feels pedantic until the second quarter of arguing.
And the most durable form: a foundation that works perfectly is indistinguishable from one that is not there. Every reliable layer in a system is invisible for the same reason, which is why "we do not need that any more" is a claim that deserves a precondition bit rather than an opinion.
25. Where The Misconception Comes From
It is worth being precise about why this belief is so stable, because "people are careless" is not the explanation and it is not a useful thing to say in an interview.
The premises are true. Newer, faster, more capable — all correct. A belief built from true premises does not fall over when you check the premises, which is the normal way beliefs get corrected.
The added layer is the interesting one. Everything written about the newer protocol is about what it adds, because what it reuses is not news. The literature is systematically biased toward the difference, and the difference is the small part of the stack.
The reused layers are invisible because they work. A physical layer that trains reliably every time produces no incidents, no escalations and no articles. Reliability and invisibility are the same property observed from outside.
The interesting devices are the minority. Accelerators and memory expanders motivate the technology and dominate the examples. The sixteen boring devices are underrepresented in the material rather than in the machine room.
Succession is the dominant narrative in computing. One bus replaced another, one interface replaced another, and the pattern is real often enough that applying it is a reasonable prior. This is a case where the prior is wrong, and knowing why it is usually right is what makes the correction convincing.
Nobody writes the conditions down. The claim arrives as a sentence. Turning it into a proposition takes a minute and almost never happens, which is section 14's entire point and the one habit worth taking from this chapter.
26. Common Misconceptions
"CXL replaces PCIe." Replacement means the older thing stops being needed. Count what it still provides that nothing else does.
"It is newer, so it supersedes it." Newer is a date. Succession is a claim about displacement, and displacement is counted.
"It is faster, so the old one is obsolete." Hold the rate equal and ask what changes. The answer is semantics, which is the axis the comparison is actually on.
"The link is CXL, so it is not PCIe." What trained it? A selector on a trained link is evidence of a shared foundation, not of independence.
"It adds coherence, so the attach question is settled." Coherence and attachment are two questions. Answering one leaves the other unchanged.
"Everything will migrate." Most devices do not participate in ownership at all. For them the coherent path is cost without benefit.
"Coherent is better." For a sharing workload, yes. For a streaming one it is overhead on every access, which is why a crossover exists and why both attaches exist.
"One attach for everything." Three device kinds want three different things. "Replaces" would require one answer to serve all three.
"Most of the conditions hold, so it basically replaces it." A conjunction has no partial credit. Five of six is false.
"Nobody uses the base protocol any more — every link reports the advanced mode." Every link also trained in the base one. A foundation working perfectly looks exactly like a foundation that was removed.
"It is newer and faster." Both true, and together they are bit 0 — one sixth of an argument about replacement.
27. Interview And Design-Review Questions
The claim itself
1. What does "A replaces B" actually claim? That after A arrives, B stops being needed. It is a claim about displacement, not about capability.
2. What kind of evidence settles it? Evidence about B. Every premise in the misconception is about A.
3. Why does the belief survive contact with accurate documentation? Because its premises are true. Newer, faster and more capable are all correct, and the conclusion still does not follow.
4. What is the single fastest refutation? Count the layers reused. Four of five is not displacement.
5. What would a genuine replacement look like in that count? Every layer displaced and zero reuse — the newer protocol bringing its own signalling, connector, training and link layer. At four of five layers reused the count is off by a factor of four, which is not a close call.
6. What is the right verb? It depends on it, and it adds semantics above it. The relationship is layering.
Layering and foundation
7. What brings a link up before any protocol runs over it? The base protocol's own training. The advanced mode is selected on a link that already exists.
8. Why is a selector evidence against replacement? Because selection requires something to select on. A protocol that can be fallen back from was layered on something.
9. What happens if selection does not occur? The link is up in its base mode and the device works. That is what a foundation does.
10. Why is the foundation invisible in most mental models? Because it works. Reliability and invisibility are the same property seen from outside.
11. Every link in a fleet reports the advanced mode and every fallback counter reads zero. What have you learned? That the base protocol worked every time. Not that it is unused — read the base-trained bit.
12. What is the general form of that error? Silence read as absence. It is the same shape as a fault that stops reproducing when you instrument it.
The two questions, and the devices
13. Name the two independent questions the misconception collapses. How a device is attached — enumerated, configured, reached — and whether its accesses participate in ownership.
14. Can a device need one and not the other? Yes, in both directions, which is what makes them independent.
15. What does answering only the coherence question leave? The attachment question, exactly where it was. The two are independent, so answering one cannot change the other — and the danger is that the design review feels finished, which is how an enumeration question reaches bring-up unanswered.
16. Why do they look like one question? Because the examples that get discussed answer both with the same technology.
17. What fraction of a typical device population participates in ownership? A minority. This chapter's illustrative count is four of twenty; the durable point is that the number is not all of them.
18. Name three devices that gain nothing from coherence. A storage controller, a network adapter, a display output — anything that moves bulk data in and out.
19. What does replacement require of those devices? That they be served at least as well. A coherent attach serves them worse, at higher cost.
20. Name the three device kinds. A bulk mover, a sharer, and a memory presenter. Three kinds, three answers.
21. Why is the taxonomy itself a refutation? Because "replaces" would require one answer to serve all three.
22. What should a taxonomy do with an unrecognised kind? Report that nothing fits. A taxonomy that always answers is defaulting, not classifying.
Rate, cost and crossover
23. Why is speed the weakest argument available here? Because two protocols at the same rate can differ in everything that matters, and holding the rate equal takes one sentence.
24. What differs when you hold the rate equal? What the bits mean — whether an access participates in ownership, whether a write must be seen by others, whether a read may return a stale copy.
25. Why does rate get compared anyway? It is comparable across generations. Semantics are not, so the comparable thing gets compared.
26. What does coherence cost, mechanically? A lookup on the access path, a possible interrogation of other holders, and a wait for the answer — on every access.
27. What does it buy? Correctness nobody has to write by hand, on the accesses that share.
28. At a base cost of 100 and a coherence cost of 20, what does the coherent path cost per access? 120, on every access — the ownership lookup is paid whether or not that access shares anything. The simple path pays 100 on every access and a synchronisation penalty only on the shared fraction, which is why a crossover exists.
29. And the simple path at 10 percent sharing, with a threefold hand-synchronisation penalty? 106 — the base cost plus the penalty on the shared tenth.
30. Where is the crossover in that model? Around a third sharing, and it is independent of the coherence cost itself.
31. Why is the crossover the answer rather than the two costs? Because it is what makes the choice actionable. Two costs are two facts.
Method, and saying it well
32. State the disciplined way to handle a sweeping claim. Write down the conditions under which it would be true, then check them. The argument becomes arithmetic.
33. Give the four conditions for this one. The stack is not reused, every device is served, the older attach is retired, no workload is worse off.
34. Two of the four hold. What follows? Nothing is replaced. A conjunction has no partial credit.
35. Five of six conditions met — is the claim 83 percent true? No. It is false, and the percentage is a description of how much work was done rather than of how close the claim is.
36. What would you say to a colleague who states the misconception? Give the relationship, not the contradiction. The layering is more interesting than the correction.
37. Why does that matter in an interview? Because "that is wrong" ends a conversation and the relationship continues it — which is what the interviewer is there to find out.
38. What is the cheapest sentence that demonstrates you understand the relationship? "It reuses most of the stack and adds semantics above it."
39. What does this chapter share with the performance chapter? Publish counts, not percentages. "Sixteen of twenty" is exact and "eighty percent" is rounded.
40. What does it share with the interview chapter? A claim with no stated boundary is the easiest thing to disprove, and the disproof is remembered.
41. What does it share with the debug chapter? Silence read as evidence — there a fault that stopped appearing, here a foundation that never fails.
42. What does it share with the integration chapter? A shared contract nobody wrote down: attachment and coherence are two questions, and collapsing them is the same failure as two teams reading one interface differently.
43. One mutation here survived on a degenerate input. What was the lesson? The answer for a zero count was a decision the model had made and the testbench had never recorded.
44. The other survived at the end of a run. What was the lesson? Assert every counter after the last state transition as well as before it.
45. Why does this chapter contain no specification detail at all? Because the refutation does not need any. Displacement is settled by counting, and a bit position would not help.
46. State the single question this chapter turns on. What would have to disappear for this to be a replacement — and has any of it?
28. Exercises
1 — Design review · Foundation. Builds: turning a claim into a count. Take the sentence "CXL replaces PCIe". Bounded scope: write the four conditions that would have to be true for it to hold, mark each as true or false, and write the one-sentence correction you would give a colleague. Hint: the correction should state the relationship, not the contradiction.
2 — Analysis · Intermediate. Builds: separating a claim about A from evidence about B. Take any three statements you have heard in support of the belief. Bounded scope: for each, say whether it is a statement about the newer protocol or about the older one, and say what evidence about the older one would be needed instead. Hint: a correct statement can still be the wrong kind of evidence.
3 — Architecture · Intermediate. Builds: counting the installed base rather than assuming it. Take a system you know — a workstation, a server, a development board. Bounded scope: list its attached devices, classify each as bulk mover, sharer or memory presenter, and compute the fraction that would gain from a coherent attach. Hint: classify before you count, and record the ones you cannot classify.
4 — Trade-off · Advanced. Builds: deriving a crossover instead of holding a preference. Using this chapter's model, a base access cost of 100 and a coherence cost of 30. Bounded scope: compute the coherent cost, compute the simple cost at 10, 30 and 50 percent sharing, find the crossover, and say whether it moved when the coherence cost changed. Hint: the second half of that question is the interesting one.
5 — Debug · Advanced. Builds: distinguishing a working foundation from an absent one. A fleet reports every link in the advanced mode and every fallback counter at zero. Bounded scope: state the two possible readings, name the single bit that separates them, and say what you would conclude from each value. Hint: this is the same error as a fault that stops reproducing under instrumentation.
6 — Design review · Advanced. Builds: keeping two questions apart. Review a device integration proposal that answers the coherence question thoroughly. Bounded scope: list what remains unanswered, say at which project stage each omission would surface, and write the two review agenda rows that would have prevented it. Hint: the second question does not become easier because the first was answered.
7 — Analysis · Intermediate. Builds: comparing on the axis that matters. Find a comparison table between two interconnects that uses rate as its primary column. Bounded scope: hold the rate equal, list what still differs, and rewrite the table with semantics as the first column. Hint: if nothing differs once the rate is equal, the comparison was about rate after all — say so.
8 — Verification · Expert. Builds: testing an argument the way a campaign tests a model. Take a sweeping claim from your own field. Bounded scope: write it as a conjunction of conditions, construct the cases with exactly one condition unmet, identify the degenerate inputs at the edges of each condition, and say which case you would have skipped. Hint: both of this chapter's mutation survivors lived at edges — a zero count, and a state after the last transition.
29. Summary
"Replaces" is a claim about displacement, not about capability. A protocol can be newer, faster and strictly more capable and displace nothing.
Every premise in the belief is a statement about the newer protocol, and the claim can only be settled by evidence about the older one.
Count the layers reused. Four of five reused and one displaced is a dependency, not a succession.
Ask what would have to disappear. Four of ten things still only provided by the older interconnect means it has not been retired.
Attachment and coherence are two questions. Answering one leaves the other exactly where it was — and that is the form of the misconception that produces an incomplete design rather than a wrong opinion.
Most devices do not participate in ownership at all. For them the coherent path is cost without benefit, and replacement would require serving them too.
Ask what trained the link. A selector on a trained link is evidence of a shared foundation, and a protocol that can be fallen back from was layered on something.
Hold the rate equal. Two protocols at the same rate can differ in everything that matters, and speed is the weakest argument available.
Coherence costs a lookup on every access and buys correctness on the fraction that shares. The crossover — around a third sharing in this model — is why both attaches exist.
Three device kinds want three different things, and "replaces" would require one answer to serve all three.
Write down what would have to be true, then check it. The argument becomes arithmetic, and it takes about a minute.
A conjunction has no partial credit. Five of six conditions met is still false.
A foundation working perfectly is indistinguishable from a foundation that was removed — which is the deepest reason this belief is so hard to shake, and the reason the precondition bit is worth reading.
Six conditions, and "it is newer and faster" is one of them. Two true premises, checked correctly, are 16 percent of an argument.
Continue learning
Related tutorials
- Related topic
Relationship to PCIe
What CXL shares with PCIe and what it adds on top, why reuse was the decisive choice, and what that reuse costs in RTL — traffic classification, class arbitration and starvation, per-class outstanding budgets and PCIe-first mode selection, all simulated with measured evidence.
- Related topic
CXL Layered Architecture
Protocol, link and physical responsibilities in a CXL stack, the contracts between them, and what breaks when a responsibility lands in the wrong layer — measured on layer backpressure, metadata truncation, retry duplication and error misrouting.
- Related topic
Relationship to the PCIe Stack
Exactly where CXL stops being PCIe, resolved per layer rather than as a single boundary — inherited, extended and added — what CXL.io's payload budget costs, and the mode space a dual stack must validate.
- Related topic
Why CXL Uses PCIe
The engineering reasons CXL rides on PCIe rather than defining its own interconnect: the deployed-base argument done in numbers, what a new PHY would have cost in schedule and risk, and the two-key rule that makes mixed systems work. Four RTL models simulated, six mutations, six killed.
Standards & specifications
- Governing standard
- CXL Specification (CXL Consortium)(opens CXL Consortium in a new tab)
Defines CXL.io, CXL.cache and CXL.mem, and the coherence and memory-pooling behaviour built on them. System design and deployment topology are not mandated.
This page also covers RTL structure, verification approach and debugging technique. Those are engineering practice built on the standard, not requirements the standard itself imposes.
Where this fits
Part of the CXL curriculum.
