Device-Type Question

27.5 designed a system. This chapter chooses a part — which CXL device type is right for this workload — and it is the question where a candidate who has memorised the three types and one who understands them give visibly different answers.

It is a Type 3. That is right most of the time, because most CXL devices are memory expanders, and being right most of the time from a fact rather than from a decision is exactly what the question is testing for.

1. The Engineering Problem — The Type Is An Answer

The question is about the workload. Eight traits decide the type; knowing three of them is thirty-seven percent of a decision and a hundred and ten units of spend, most of it rework. Section 5.

Two questions decide the type and neither is about the device. Does the device need to cache host memory, and does the host need the device's. Being an accelerator is not the reason — sparse access to host memory is. Section 6.

The wrong type works. A thousand accesses at two units native and twenty by workaround is eighteen thousand units of penalty — nine hundred percent over — paid on every access, forever, for a procurement decision. Section 8.

Type 2 is the expensive one. Sixty units of extra silicon against thirty of gain is a capability that does not pay for itself, and the more-capable view books the gain with no cost against it. Section 9.

The type you specify and the type you can buy are different lists. A design that needs a part nobody sells has chosen a schedule rather than a device. Section 10.

This chapter against 27.5, stated precisely. That one owns arranging devices into a system. This one owns which device — which is why every model here is a decision with a cost on both sides, and why section 14's weak definition is a correct fact deployed as a substitute for a decision.

2. The One-Sentence Model

A device type is the right answer when a type was chosen at all, when the workload it serves has been characterised, when the question of whether the device needs host lines has been answered, when the question of whether the host needs device memory has been answered, when the resulting type can actually be bought, and when its extra capability is paid for by what the workload does with it — and "it is a Type 3" is one of those six.

3. What This Chapter Owns

The taxonomy first, stated as what it is rather than as three names.

The three types are the three useful combinations of two optional protocols. CXL.io is mandatory on all of them. CXL.cache and CXL.mem are each present or absent, which gives four combinations, and one of them — neither present — is a PCIe device.

Type 1 is io plus cache. An accelerator with no memory of its own that needs coherent access to the host's. A network or crypto accelerator working on host buffers in place.

Type 2 is io plus cache plus mem. An accelerator with memory of its own, where the device caches host memory and the host addresses device memory. A GPU-class part where both sides need coherent access in both directions.

Type 3 is io plus mem. A memory expander. No accelerator, no coherent device attach, no CXL.cache at all. This is most of the market, and it is what somebody means by "a CXL device" when they do not say which.

The two questions that decide it. Does the device need to cache host memory? — which is a question about access density, answered in section 6. Does the host need the device's memory? — which is a question about spare capacity and host demand, answered in section 7. Yes-no gives Type 1; yes-yes gives Type 2; no-yes gives Type 3; no-no is a PCIe device and does not need CXL at all.

That last case is worth saying out loud. An accelerator that moves whole buffers densely, has no memory to share, and needs no coherence is a PCIe device, and choosing CXL for it buys a flit layer, an alternate-protocol negotiation and a link budget for no return. "No" to both is a legitimate answer to this question and an answer that cannot produce it has not been reasoning.

The numbering is not a ranking and it is worth saying so, because the names invite the reading that Type 3 is the most advanced. They are a list in the order the specification introduced them, and the capability ordering — if there is one — runs Type 3, Type 1, Type 2, which is nobody's reading of the names. A candidate who says "the highest type" has told the interviewer which mental model is in use.

Everything below the protocol is the same on all three. The same flit layer, the same link training, the same alternate-protocol negotiation over PCIe, the same lane counts and the same speeds. The type is a statement about which protocols ride the link, not about the link, which is why 27.1's ninety-second answer works for all three without qualification and why a type change does not change the electrical design.

And the type decides who the device's memory belongs to, which is the part with consequences for software. A Type 3's memory is host-managed and appears in the host's map as a NUMA node the operating system allocates from. A Type 2's device memory is host-addressable but the device also uses it, and which side has the current copy of a line is managed by bias — 27.3's subject and the reason a Type 2 is not simply a Type 1 and a Type 3 in one package.

4. Teaching-Model Boundary

Every model in this chapter is a teaching model, not a selection tool. It computes the one relationship the section is about and nothing else. There is no device, no workload trace and no procurement system anywhere in this file.

Each model is built twice from one source. A parameter selects between the measured build, which computes what a decision costs, and the type-first build, which computes what follows from naming a type and reasoning backwards. Every section's headline number is the gap between them.

5. RTL 1 — The Question Is About The Workload

Start with the thing the question is actually testing, because it is the part that is easy to skip and impossible to recover from.

A device type is an output. It is determined by what the workload does — how it touches host memory, whether it has memory the host wants, how those change over its phases. Naming a type before describing the workload is producing an answer to a question that has not been asked, and being right is then a matter of luck rather than of reasoning.

The cost structure is what makes this worth modelling. Characterising costs something now. Being wrong costs that plus rework later, and the rework is section 12's subject.

It is worth naming the traits, because "characterise the workload" is otherwise an instruction nobody can act on. What does the device read from host memory, and how densely — section 6's ratio. How much memory does the device carry, and how much of it does the device use — section 7. Is the host short of memory at all — without which the CXL.mem side has no customer. Does the access pattern change between phases — section 11. How long does the workload run in each phase, without which section 11 cannot weigh anything. What is the software's fallback if the device cannot do it natively — section 8's workaround cost. What does the programme's schedule allow — section 10. And when could this decision still be changed — section 12. Eight, and a team that can answer three of them usually has the three that are easiest to ask rather than the three that decide the most.

// RTL 1 - the question is about the workload. A device type is an answer, and
// naming one before describing what the thing has to do is answering a
// question nobody asked.
module workload_first #(parameter int PICK_THE_TYPE_FIRST = 0) (
  input  logic clk, rst_n,
  input  logic        evaluate,
  input  logic [15:0] traits_needed, traits_known, char_cost, rework_cost,
  output logic [15:0] known_ok, unknown_traits, reach_pct, spend_now,
  output logic        workload_characterised,
  output logic [7:0]  n_evals, n_blind,
  output logic        blind_err
);
  logic [15:0] true_unknown;
  logic [31:0] r_q, s_q;
  logic        truly_blind;
  // A characterisation cannot know more traits than the workload has.
  assign known_ok = (traits_known > traits_needed) ? traits_needed : traits_known;
  assign true_unknown = traits_needed - known_ok;
  assign unknown_traits = (PICK_THE_TYPE_FIRST != 0) ? 16'd0 : true_unknown;
  assign r_q = (traits_needed == 16'd0) ? 32'd100
             : (({16'd0, known_ok} * 32'd100) / {16'd0, traits_needed});
  assign reach_pct = (r_q > 32'd100) ? 16'd100 : r_q[15:0];
  // Characterising costs something now; being wrong costs rework later.
  assign s_q = (true_unknown == 16'd0) ? {16'd0, char_cost}
             : ({16'd0, char_cost} + {16'd0, rework_cost});
  assign spend_now = (s_q > 32'hFFFF) ? 16'hFFFF : s_q[15:0];
  assign workload_characterised = (unknown_traits == 16'd0);
  assign truly_blind = (true_unknown != 16'd0) && (traits_needed != 16'd0);
  assign blind_err = evaluate && truly_blind && workload_characterised;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_evals <= 8'd0; n_blind <= 8'd0;
    end else if (evaluate) begin
      n_evals <= n_evals + 8'd1;
      if (truly_blind) n_blind <= n_blind + 8'd1;
    end
  end
endmodule

Eight traits that decide the type with three of them known is thirty-seven percent of a decision — and the spend already carries the rework, because a choice made from three eighths of a workload is a choice that will be revisited.

The sixth case is the sharpest. Characterisation that costs nothing and was not done — free measurement, five unknown traits, a hundred units of rework already in the estimate — is the worst configuration in the model, and it is common because free measurement is usually free only in the sense that nobody has been asked to do it.

The clamp is worth a line because it says something real. Measuring more traits than the decision needs is waste, not blindness — the model clamps it to what the decision uses and reports full characterisation, because a workload study that answered questions nobody was going to ask is inefficient rather than wrong.

The reason this gets skipped is not laziness and it is worth being precise about, because the fix depends on the cause. The type has to be chosen before the workload exists. Silicon is procured against a workload that is still being written, and the honest position at decision time is that nobody knows what it will do. Characterisation then means characterising the predecessor — the version running on the current hardware, or the closest analogue anybody has — and the quality of the decision is the quality of that analogy. A team that says "we cannot characterise it, it does not exist yet" is describing the problem correctly and then usually skipping the achievable version of the work, which is to characterise what it most resembles and say how confident that makes them.

6. RTL 2 — Does The Device Need To Cache Host Memory

The first of the two questions, and the one where a plausible-sounding rule gives the wrong answer.

The rule that sounds right is accelerators need coherent access to host memory. It is wrong, and the reason it is wrong is the whole of 27.3 section 12: coherence pays per access and transfer pays per buffer. An accelerator that streams whole buffers densely gains nothing from CXL.cache and pays for it; one that touches a large structure sparsely gains a great deal.

The question is about the access pattern, not about what the device is called.

// RTL 2 - does the device need to cache host memory. This is the first of the
// two questions that decide the type, and the answer is a property of the
// access pattern rather than of how clever the device is.
module cache_requirement #(parameter int AN_ACCELERATOR_NEEDS_COHERENCE = 0) (
  input  logic clk, rst_n,
  input  logic        evaluate,
  input  logic [15:0] host_lines_touched, touch_density, transfer_grain, dev_is_accel,
  output logic [15:0] moved_by_transfer, moved_by_cache, waste_bytes, cache_gain_pct,
  output logic        cache_needed,
  output logic [7:0]  n_evals, n_needing,
  output logic        cache_err
);
  logic [15:0] true_waste;
  logic [31:0] t_q, c_q, g_q;
  logic        truly_needs;
  // A transfer moves whole grains of whatever the device touches.
  assign t_q = {16'd0, host_lines_touched} * {16'd0, transfer_grain};
  assign moved_by_transfer = (t_q > 32'hFFFF) ? 16'hFFFF : t_q[15:0];
  assign c_q = {16'd0, host_lines_touched} * {16'd0, touch_density};
  assign moved_by_cache = (c_q > 32'hFFFF) ? 16'hFFFF : c_q[15:0];
  assign true_waste = (moved_by_transfer > moved_by_cache)
                      ? (moved_by_transfer - moved_by_cache) : 16'd0;
  assign waste_bytes = true_waste;
  assign g_q = (moved_by_transfer == 16'd0) ? 32'd0
             : (({16'd0, true_waste} * 32'd100) / {16'd0, moved_by_transfer});
  assign cache_gain_pct = (g_q > 32'd100) ? 16'd100 : g_q[15:0];
  // Being an accelerator is not the reason; sparse access to host memory is.
  assign cache_needed = (AN_ACCELERATOR_NEEDS_COHERENCE != 0)
                        ? (dev_is_accel != 16'd0) : (true_waste != 16'd0);
  assign truly_needs = (true_waste != 16'd0) && (host_lines_touched != 16'd0);
  assign cache_err = evaluate && (true_waste == 16'd0) && cache_needed
                     && (host_lines_touched != 16'd0);

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_evals <= 8'd0; n_needing <= 8'd0;
    end else if (evaluate) begin
      n_evals <= n_evals + 8'd1;
      if (truly_needs) n_needing <= n_needing + 8'd1;
    end
  end
endmodule

A hundred host lines touched at sixty-four bytes each, transferred in four-kilobyte grains, moves nearly sixty thousand bytes for six thousand four hundred touched — ninety percent of the traffic avoidable. That device wants CXL.cache. The same accelerator touching densely wants nothing of the kind.

Figure 1 — the rule and the measurement, on the same device. The upper path gives the right answer for this workload and gives the same answer for every workload, which is what makes it a rule rather than a decision. The lower path asks what the access pattern does, and on a dense workload it answers no — which is the answer that saves an accelerator team from buying a coherency agent it will never benefit from.

The sixth case is the one that inverts the intuition and is worth sitting with. A transfer grain smaller than what the device touches wastes nothing — the transfer is already moving less than the device reads, and a cache would add nothing. Fine-grained DMA is doing what a coherent cache would do, more cheaply, and a device with it does not need CXL.cache however sparse its access is.

The third case is why the wrong rule survives. A non-accelerator with dense access is answered "no" by both the rule and the measurement, and so is every memory expander anybody has ever specified. The rule agrees with the measurement on the entire Type 3 market, which is most of the devices anybody sees.

It is worth being concrete about what CXL.cache actually buys, because "coherent access" is a phrase that can be repeated without content. The device gets a cache of host lines that the host's coherency protocol knows about. The device reads a line, holds it, and the host's directory records that it is held; if a host core writes that line, the device is snooped and its copy is invalidated. What the device gains is that it can read one 64-byte line without a driver, a descriptor, a DMA setup or a completion, and it can hold that line across several uses without re-fetching. What it costs is a caching agent on the device, participation in the host's snoop traffic, and — as 27.3 sets out at length — a latency the device cannot control, because a snoop arrives when the host decides rather than when the device is ready.

That cost structure is why the ratio decides it. A device touching a hundred lines scattered through a gigabyte cannot transfer efficiently: every four-kilobyte grain it pulls carries sixty-four useful bytes. A device streaming a contiguous buffer uses every byte of every grain, gains nothing from holding lines, and pays the snoop exposure for the privilege. The same silicon is excellent for one and pure overhead for the other, and the only thing distinguishing them is the access pattern.

The measurement itself is cheap and worth stating as a recipe. Take the device's read set over a window, count the distinct lines, multiply by the line size, and divide by what the transfer mechanism would have moved to deliver them. A profiler that records addresses gives this directly; a profiler that records only byte counts does not, which is the usual reason the question gets answered by rule instead.

7. RTL 3 — Does The Host Need The Device's Memory

The second question, and the one whose answer is usually obvious and occasionally expensive in the direction of yes.

A device with memory on it does not automatically have memory to give. Only what the device is not using itself can be exposed to the host, and a device that exposes memory it is using has made a promise it cannot keep — the host maps it, the allocator counts it, and the device cannot have it back.

// RTL 3 - does the host need the device's memory. The second of the two
// questions, and the one whose answer is usually obvious and occasionally
// expensive to get wrong in the direction of yes.
module mem_requirement #(parameter int MEMORY_ON_IT_MEANS_EXPOSE_IT = 0) (
  input  logic clk, rst_n,
  input  logic        evaluate,
  input  logic [15:0] dev_gb, host_wants_gb, dev_uses_gb, host_gb_short,
  output logic [15:0] exposable_gb, host_served_gb, kept_by_device, served_pct,
  output logic        mem_needed,
  output logic [7:0]  n_evals, n_needing,
  output logic        mem_err
);
  logic [15:0] spare_gb, true_served;
  logic [31:0] s_q;
  logic        truly_needs;
  // Only what the device is not using itself can be given to the host.
  assign spare_gb = (dev_uses_gb > dev_gb) ? 16'd0 : (dev_gb - dev_uses_gb);
  assign exposable_gb = (MEMORY_ON_IT_MEANS_EXPOSE_IT != 0) ? dev_gb : spare_gb;
  assign true_served = (host_wants_gb > spare_gb) ? spare_gb : host_wants_gb;
  assign host_served_gb = (host_wants_gb > exposable_gb) ? exposable_gb : host_wants_gb;
  assign kept_by_device = dev_gb - exposable_gb;
  assign s_q = (host_wants_gb == 16'd0) ? 32'd100
             : (({16'd0, true_served} * 32'd100) / {16'd0, host_wants_gb});
  assign served_pct = (s_q > 32'd100) ? 16'd100 : s_q[15:0];
  assign mem_needed = (host_gb_short != 16'd0) && (spare_gb != 16'd0);
  assign truly_needs = mem_needed;
  // Exposing memory the device is using is a promise it cannot keep.
  assign mem_err = evaluate && (host_served_gb > true_served)
                   && (host_wants_gb != 16'd0);

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_evals <= 8'd0; n_needing <= 8'd0;
    end else if (evaluate) begin
      n_evals <= n_evals + 8'd1;
      if (truly_needs) n_needing <= n_needing + 8'd1;
    end
  end
endmodule

Five hundred and twelve gigabytes on the device with three hundred and eighty-four in use leaves a hundred and twenty-eight spare against a host that wants two hundred and fifty-six — half the request met. The memory-on-it view exposes the whole device and serves all two hundred and fifty-six.

The sixth case is why the whole-device rule looks right. A device that uses none of its own memory — a pure memory expander — exposes everything, correctly, and both views agree. Every Type 3 device in existence is this case, which is exactly the market on which the wrong rule cannot be distinguished from the right one.

The fifth case is the other direction and it is the one worth checking before specifying a Type 2. A host that is not short of memory has no reason to want the device's, and the model reports the capacity as servable while declining to call it needed. Adding CXL.mem to an accelerator because the accelerator has DRAM on it, when no host in the deployment is memory-constrained, is section 9's mistake arriving through this door.

There is a subtlety in "spare" that the model treats as static and reality does not. A device's own usage changes with its workload. An accelerator that uses three hundred and eighty-four gigabytes during a training phase may use forty between runs, and the spare capacity is therefore a function of time rather than a number. Exposing the maximum spare means the host will map memory the device needs back; exposing the minimum spare means most of the capacity is never offered. The honest figure is the device's usage at its peak, which is the only one that can be exposed without a mechanism for taking it back — and a mechanism for taking it back is 27.5's reassignment, with all of its unmap-before-reassign discipline, running inside a single device.

The other half of the question is whether the host can use it at all. Device-attached memory arrives in the host's map as a separate NUMA node with its own latency, and an application that is not NUMA-aware will allocate from it as though it were local and run slower. A host that is short of memory and has software that can place allocations deliberately is the case where CXL.mem pays; a host that is short of memory and allocates blindly gets capacity and a performance complaint. The memory question therefore has a software half that the hardware specification cannot answer, and asking it early is the difference between a deployment and a return.

8. RTL 4 — What The Wrong Type Costs

Sections 6 and 7 decide the type. This one prices getting it wrong, and the price has a shape worth knowing: it is not a failure, it is a tax.

A memory expander asked to do an accelerator's job works. The data moves by explicit transfer, the software is written around it, and the system functions. The cost is paid per access, it does not decrease, and it is invisible in any test that does not compare against the alternative.

// RTL 4 - what the wrong type costs. A memory expander asked to do an
// accelerator's job works: the data moves by explicit transfer, the software
// is written round it, and the cost is paid on every access forever.
module type_penalty #(parameter int ANY_TYPE_WILL_DO = 0) (
  input  logic clk, rst_n,
  input  logic        evaluate,
  input  logic [15:0] accesses, native_per_acc, workaround_per_acc, type_is_right,
  output logic [15:0] native_cost, actual_cost, penalty, penalty_pct,
  output logic        type_fits,
  output logic [7:0]  n_evals, n_mismatched,
  output logic        fit_err
);
  logic [15:0] true_actual, true_penalty;
  logic [31:0] n_q, a_q, p_q;
  logic        truly_mismatched;
  assign n_q = {16'd0, accesses} * {16'd0, native_per_acc};
  assign native_cost = (n_q > 32'hFFFF) ? 16'hFFFF : n_q[15:0];
  assign a_q = (type_is_right != 16'd0)
             ? ({16'd0, accesses} * {16'd0, native_per_acc})
             : ({16'd0, accesses} * {16'd0, workaround_per_acc});
  assign true_actual = (a_q > 32'hFFFF) ? 16'hFFFF : a_q[15:0];
  assign actual_cost = (ANY_TYPE_WILL_DO != 0) ? native_cost : true_actual;
  assign true_penalty = (true_actual > native_cost)
                        ? (true_actual - native_cost) : 16'd0;
  assign penalty = (ANY_TYPE_WILL_DO != 0) ? 16'd0 : true_penalty;
  assign p_q = (native_cost == 16'd0) ? 32'd0
             : (({16'd0, true_penalty} * 32'd100) / {16'd0, native_cost});
  assign penalty_pct = (p_q > 32'd999) ? 16'd999 : p_q[15:0];
  assign type_fits = (penalty == 16'd0);
  assign truly_mismatched = (true_penalty != 16'd0) && (accesses != 16'd0);
  assign fit_err = evaluate && truly_mismatched && type_fits;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_evals <= 8'd0; n_mismatched <= 8'd0;
    end else if (evaluate) begin
      n_evals <= n_evals + 8'd1;
      if (truly_mismatched) n_mismatched <= n_mismatched + 8'd1;
    end
  end
endmodule

A thousand accesses at two units native and twenty by workaround is eighteen thousand units of penalty — nine hundred percent over what the right type would have cost.

Figure 2 — why the wrong type is not a failure. Nothing on either path is broken: the device works, the software works, the workload completes. The upper path is what a specification says the device costs and the lower is what the system pays, and the gap is a tax that is charged on every access for the life of the deployment. It never appears as an error and it never appears in a test unless somebody builds the other version to compare against.

The last case is the one that actually ships. A thirty percent penalty is not dramatic, does not look like a defect, and is easy to accept as the cost of doing business — and it is a third of the performance of the system, permanently. The nine-hundred-percent case gets found; the thirty-percent case gets absorbed.

The reason it gets absorbed is structural rather than cultural. There is no baseline to compare against. A system built on one device type is the only version of itself that exists; its performance is whatever it is, and the number that would reveal the tax — what the same workload costs on the other type — was never measured because building it would have meant buying the other part. The comparison the model makes trivially is the one a programme cannot make cheaply, which is why the decision has to be made from a model rather than from a measurement, and why section 5's characterisation is the only lever.

What the workaround actually is depends on which direction the type is wrong in, and the two have different shapes. A Type 3 asked to do a Type 1's job moves data explicitly: the software copies the region it needs into device memory, works on it, copies it back, and pays for the parts of the region it did not need. A Type 1 asked to do a Type 3's job is worse, because there is no device memory to expose at all and the capacity simply is not there — the workaround is not slower, it is absent, and the system runs on less memory than it was sized for. The second failure is easier to notice and cheaper to fix precisely because it does not silently work.

The third case is the honest limit. A workaround as cheap as the native path is not a penalty, and there are workloads where the type genuinely does not matter — where the software would do the same thing either way. An answer that treats the type as always decisive has overclaimed.

9. RTL 5 — Type 2 Is The Expensive One

The type that is most often over-specified, and the reason is that it sounds like the safe choice.

Type 2 is both other types at once: the device caches host memory, the host addresses device memory, and the device carries a coherency agent, a memory controller, bias management and the silicon for all of it. A workload that wants only one of the two directions pays for both.

The argument for over-specifying is always the same and always sounds prudent: we might need it later. The models here price that, because the extra capability is not free and "might" is not a gain.

// RTL 5 - Type 2 is the expensive one. Both protocols, both directions,
// coherency in the device and bias management on top, and the workload has to
// want all of it or the extra silicon is silicon that does nothing.
module type_two_cost #(parameter int MORE_CAPABLE_IS_BETTER = 0) (
  input  logic clk, rst_n,
  input  logic        evaluate,
  input  logic [15:0] type3_silicon, type2_silicon, gain_units, unit_value,
  output logic [15:0] extra_silicon, gain_value, net_value, gain_pct,
  output logic        justified,
  output logic [7:0]  n_evals, n_unjustified,
  output logic        cost_err
);
  logic [15:0] true_net;
  logic [31:0] g_q, p_q;
  logic        truly_unjustified;
  assign extra_silicon = (type2_silicon > type3_silicon)
                         ? (type2_silicon - type3_silicon) : 16'd0;
  assign g_q = {16'd0, gain_units} * {16'd0, unit_value};
  assign gain_value = (g_q > 32'hFFFF) ? 16'hFFFF : g_q[15:0];
  assign true_net = (gain_value > extra_silicon)
                    ? (gain_value - extra_silicon) : 16'd0;
  assign net_value = (MORE_CAPABLE_IS_BETTER != 0) ? gain_value : true_net;
  assign p_q = (extra_silicon == 16'd0) ? 32'd100
             : (({16'd0, gain_value} * 32'd100) / {16'd0, extra_silicon});
  assign gain_pct = (p_q > 32'd999) ? 16'd999 : p_q[15:0];
  assign justified = (net_value != 16'd0);
  assign truly_unjustified = (extra_silicon >= gain_value) && (gain_units != 16'd0);
  assign cost_err = evaluate && truly_unjustified && justified;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_evals <= 8'd0; n_unjustified <= 8'd0;
    end else if (evaluate) begin
      n_evals <= n_evals + 8'd1;
      if (truly_unjustified) n_unjustified <= n_unjustified + 8'd1;
    end
  end
endmodule

Sixty units of extra silicon against a hundred and fifty of gain is a two-hundred-and-fifty percent return and an obvious yes. The same silicon against thirty of gain is nothing net, and the more-capable view books the thirty and calls it a win.

The third case is the boundary and the model puts it on the side that matters. Breaking even is not justification — equal gain and silicon means the extra capability cost exactly what it returned, which is a great deal of design effort for nothing, and a decision that lands there should be deliberate. The more-capable view calls it a win, which is how a break-even Type 2 gets specified.

The fifth case is the one that makes the answer easy when it applies. A Type 2 that costs no more than a Type 3 — because the silicon was going to be there anyway, or the part is bought rather than built — is always justified, and both views agree. That is a real situation and it is the honest case for choosing the more capable part.

The silicon figure is the visible part of the cost and it is not the largest part. A caching agent has to be verified, and verifying one means proving that the device responds correctly to every snoop in every state, under every interleaving the host can produce — which is a coherency verification effort with its own team, its own formal work and its own schedule. The bias machinery has to be verified with it, because a line the device believes it owns while the host believes it is shared is a data-corruption bug that will not reproduce. Next to that, the gate count is a rounding error, and a cost model that counts only silicon area will always make Type 2 look cheaper than it is.

The gain side is usually estimated and the cost side is usually known, which biases the comparison in a predictable direction. Silicon area and verification effort are quantities an engineering organisation can put a number on; the benefit of coherent device access to host memory on a workload that does not exist yet is a projection. Two quantities of very different confidence being subtracted is the shape of most over-specification, and the discipline is not to make the projection better — it usually cannot be — but to write the confidence next to it, so that a break-even result is recognised as the coin flip it is.

There is a deployment answer that changes the arithmetic and is worth knowing. A Type 2 operated with its CXL.cache side disabled is a Type 3, and it works — the device exposes its memory, the host maps it, and the caching agent sits idle. That makes the extra capability an option rather than a commitment, which is exactly what "we might need it later" wants. The model's contribution is that an option has a price and this one is paid at tape-out, in silicon and in a verification programme, whether or not the option is ever exercised.

10. RTL 6 — The Type You Can Specify And The Type You Can Buy

The sixth consideration, and the one that is not an engineering question at all.

A design can specify any of the three types. A programme can only use one that exists, ships, and arrives before the programme needs it. Those are three separate conditions and none of them is in the specification.

This matters more for CXL than for most interconnects because the market is uneven: Type 3 parts are plentiful, Type 2 parts are few and are usually tied to a specific accelerator, and a Type 1 part is often something a team builds rather than buys.

// RTL 6 - the type you can specify and the type you can buy. A design that
// needs a part nobody sells has chosen a schedule rather than a device, and
// the lead time is the answer.
module procurability #(parameter int IF_ITS_IN_THE_SPEC_ITS_REAL = 0) (
  input  logic clk, rst_n,
  input  logic        evaluate,
  input  logic [15:0] type_wanted, types_offered, lead_weeks, slack_weeks,
  output logic [15:0] offered_ok, over_by_weeks, delay_weeks, slack_pct,
  output logic        procurable,
  output logic [7:0]  n_evals, n_unavailable,
  output logic        procure_err
);
  logic [15:0] true_over;
  logic [31:0] s_q;
  logic        truly_unavailable;
  // A type is offered if the market has it; the specification says nothing.
  assign offered_ok = ((types_offered & type_wanted) != 16'd0) ? 16'd1 : 16'd0;
  assign true_over = (lead_weeks > slack_weeks) ? (lead_weeks - slack_weeks) : 16'd0;
  assign over_by_weeks = (IF_ITS_IN_THE_SPEC_ITS_REAL != 0) ? 16'd0 : true_over;
  assign delay_weeks = (offered_ok == 16'd0) ? 16'hFFFF : true_over;
  assign s_q = (slack_weeks == 16'd0) ? 32'd0
             : (({16'd0, lead_weeks} * 32'd100) / {16'd0, slack_weeks});
  assign slack_pct = (s_q > 32'd999) ? 16'd999 : s_q[15:0];
  assign procurable = (IF_ITS_IN_THE_SPEC_ITS_REAL != 0)
                      ? 1'b1 : ((offered_ok != 16'd0) && (over_by_weeks == 16'd0));
  assign truly_unavailable = ((offered_ok == 16'd0) || (true_over != 16'd0))
                             && (type_wanted != 16'd0);
  assign procure_err = evaluate && truly_unavailable && procurable;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_evals <= 8'd0; n_unavailable <= 8'd0;
    end else if (evaluate) begin
      n_evals <= n_evals + 8'd1;
      if (truly_unavailable) n_unavailable <= n_unavailable + 8'd1;
    end
  end
endmodule

A type nobody offers is an unbounded delay whatever the lead time says. A type that is offered and takes twenty weeks against twelve of slack is eight weeks late — a hundred and sixty-six percent of the schedule's slack — and the it-is-in-the-specification view reports a part either way.

The last case is the realistic one for a part at the edge of the market. Neither offered nor in time — the model reports both problems, and the two have different responses: a part nobody sells means changing the design, and a part that arrives late means changing the schedule or the design. Reporting them as one failure loses the distinction that decides what to do.

The boundary is driven deliberately. A lead time exactly equal to the slack is in time, and a model that failed it would be punishing a programme that had done the arithmetic and planned to the edge of it.

The market's shape is worth understanding rather than memorising, because it follows from the economics. Type 3 parts are plentiful because a memory expander is a controller and some DRAM, sells to anybody with a memory-bound workload, and has no coupling to what the host is computing. Type 2 parts are few because the device is an accelerator first: it exists because somebody built a GPU or a domain-specific engine, and the CXL type is a property of that part rather than a product in itself. You do not buy a Type 2; you buy a particular accelerator that happens to be one. Type 1 parts are rarest as products because a coherent accelerator with no memory is usually something a team builds for its own workload, and there is little reason to sell one on the open market.

Second-sourcing follows directly from that and it is the question a procurement team will ask. A Type 3 has alternatives; a design that depends on one has a fallback if a vendor slips. A Type 2 usually has exactly one supplier, and a design that depends on one has taken a single-vendor dependency on top of everything else. That is not automatically wrong — plenty of programmes depend on one accelerator — but it should be a decision rather than a discovery, and it belongs in the same conversation as the lead time rather than in a later one.

11. RTL 7 — A Workload Has Phases

The seventh consideration, and the one that makes a well-characterised workload still produce the wrong answer.

Section 5 says characterise the workload. This one says: the characterisation is a distribution, not a number. A workload with a load phase, a compute phase and a write-back phase may want CXL.mem in the first, CXL.cache in the second and nothing in the third, and a single type serves whichever of those is dominant.

A type chosen from the average of a phased workload is a type chosen for a workload that never runs.

// RTL 7 - a workload has phases and the phases want different types. A model
// that picks a type from the average of a workload has picked a type for a
// workload that never runs.
module phase_variation #(parameter int ONE_WORKLOAD_ONE_TYPE = 0) (
  input  logic clk, rst_n,
  input  logic        evaluate,
  input  logic [15:0] phases, phases_wanting_cache, phases_wanting_mem, phase_len,
  output logic [15:0] phases_covered, phases_missed, missed_time, covered_pct,
  output logic        steady_type,
  output logic [7:0]  n_evals, n_varying,
  output logic        phase_err
);
  logic [15:0] dominant, true_missed;
  logic [31:0] m_q, c_q;
  logic        truly_varying;
  // A single type serves whichever need is dominant and misses the other.
  assign dominant = (phases_wanting_cache > phases_wanting_mem)
                    ? phases_wanting_cache : phases_wanting_mem;
  assign phases_covered = (dominant > phases) ? phases : dominant;
  assign true_missed = phases - phases_covered;
  assign phases_missed = (ONE_WORKLOAD_ONE_TYPE != 0) ? 16'd0 : true_missed;
  assign m_q = {16'd0, phases_missed} * {16'd0, phase_len};
  assign missed_time = (m_q > 32'hFFFF) ? 16'hFFFF : m_q[15:0];
  assign c_q = (phases == 16'd0) ? 32'd100
             : (({16'd0, phases_covered} * 32'd100) / {16'd0, phases});
  assign covered_pct = (c_q > 32'd100) ? 16'd100 : c_q[15:0];
  assign steady_type = (phases_missed == 16'd0);
  assign truly_varying = (true_missed != 16'd0) && (phase_len != 16'd0);
  assign phase_err = evaluate && truly_varying && steady_type;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_evals <= 8'd0; n_varying <= 8'd0;
    end else if (evaluate) begin
      n_evals <= n_evals + 8'd1;
      if (truly_varying) n_varying <= n_varying + 8'd1;
    end
  end
endmodule

Ten phases with six wanting a cache and four wanting memory is four phases on the wrong type — four hundred units of run time, sixty percent of the workload covered.

The sixth case is the worst shape and it is a real one. An even split between the two needs covers half the workload whichever type is chosen — there is no dominant need, and the decision is a coin flip between two wrong answers. That is the configuration where a Type 2 earns its price, and it is the strongest argument section 9 has against itself.

The degenerate case is worth a note. A phase of no duration costs nothing to miss, and the model declines to report it — a workload whose phases are characterised but whose durations are not is a characterisation that cannot support this decision, and reporting a failure from it would be inventing one.

The model counts phases and the decision needs time, which is the distinction that makes this section usable. Six phases of one second and four of ten seconds is four phases missed and eighty percent of the run time on the wrong device — the phase count says sixty percent covered and the duration says twenty. Whenever the durations are uneven the count is the wrong summary, and it is the summary a profile usually reports. Weight by time, always, and if the durations are not known then say that the decision cannot be made from the data rather than making it from the count.

A phased workload does not automatically want a Type 2. Three other answers are usually cheaper. The phases may be separable across devices — a memory expander and an accelerator, each doing the phase it suits, which costs two parts and a data movement between them. The workload may be restructurable so the phases overlap less. Or the dominant phase may simply be dominant enough that eighty or ninety percent coverage is worth more than the Type 2 premium. The even split is the case where none of those works, and it is genuinely the strongest argument for the expensive part — which is why it is worth checking whether the split is real before reaching for it.

12. RTL 8 — What It Costs To Find Out Late

The eighth consideration, and the reason sections 5 through 7 are worth doing at all.

A device type is not a setting. Changing it means a different part, and depending on how far the programme has run it can mean a different driver, a different board, a different memory map and in the worst case a different silicon programme. The work done before the mistake is found is work done again.

// RTL 8 - what it costs to find out late. A type chosen wrongly is not a
// setting to change; it is a different part, a different driver and in the
// worst case a different silicon programme, and the price rises with how far
// the project has run.
module change_cost #(parameter int WE_CAN_SWAP_IT_LATER = 0) (
  input  logic clk, rst_n,
  input  logic        evaluate,
  input  logic [15:0] weeks_in, cost_per_week, part_cost, caught_at_week,
  output logic [15:0] sunk_cost, rework_cost, total_cost, sunk_pct,
  output logic        caught_early,
  output logic [7:0]  n_evals, n_late,
  output logic        migrate_err
);
  logic [15:0] weeks_ok, true_sunk;
  logic [31:0] s_q, t_q, d_q, p_q;
  logic        truly_late;
  // Work done before the mistake is found is work done again.
  assign weeks_ok = (caught_at_week > weeks_in) ? weeks_in : caught_at_week;
  assign s_q = {16'd0, weeks_ok} * {16'd0, cost_per_week};
  assign true_sunk = (s_q > 32'hFFFF) ? 16'hFFFF : s_q[15:0];
  assign sunk_cost = (WE_CAN_SWAP_IT_LATER != 0) ? 16'd0 : true_sunk;
  assign rework_cost = part_cost;
  assign t_q = {16'd0, sunk_cost} + {16'd0, rework_cost};
  assign total_cost = (t_q > 32'hFFFF) ? 16'hFFFF : t_q[15:0];
  // The denominator is taken in 32 bits so a large sunk cost plus a large part
  // cost cannot wrap to zero and turn the division into an X.
  assign d_q = {16'd0, true_sunk} + {16'd0, part_cost};
  assign p_q = (d_q == 32'd0) ? 32'd0 : (({16'd0, true_sunk} * 32'd100) / d_q);
  assign sunk_pct = (p_q > 32'd100) ? 16'd100 : p_q[15:0];
  assign caught_early = (sunk_cost == 16'd0);
  assign truly_late = (true_sunk != 16'd0) && (part_cost != 16'd0);
  assign migrate_err = evaluate && truly_late && caught_early;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_evals <= 8'd0; n_late <= 8'd0;
    end else if (evaluate) begin
      n_evals <= n_evals + 8'd1;
      if (truly_late) n_late <= n_late + 8'd1;
    end
  end
endmodule

Thirty weeks of work done before the mistake is found, at a hundred units a week, against a five-thousand-unit part is eight thousand in total, thirty-seven percent of it sunk before anybody knew. The swap-it-later view charges the part alone.

The last case is the shape of the argument. Caught at week ten it is sixteen percent; caught at week thirty it is thirty-seven. The cost is linear in when the mistake is found, which means the value of section 5's characterisation is highest at the start and falls continuously — and is negative after the point where the rework exceeds what a better decision would have saved.

The second case is the target. Caught before any work is done costs the part and nothing else, which is the situation a design review at the right moment creates.

That phrasing hides the actual difficulty, which is that the review has to happen when the information is worst. Week two is when a change is cheapest and also when least is known about the workload, and week thirty is when the most is known and the change costs thirty-seven percent of the programme. The value of a review is the product of what it can still change and what it can still learn, and those move in opposite directions — which is an argument for more than one review rather than for placing a single one perfectly.

The practical version is a decision with a written expiry. Choose the type at week two, record which of section 5's eight traits were assumptions rather than measurements, and set a checkpoint at the first moment those assumptions can be tested — usually when the first real workload runs on the predecessor hardware. If they hold, the decision stands and nobody has spent anything. If they do not, the change is being made at week eight instead of week thirty. What makes this work is not the review, it is the written list of assumptions, because a programme that did not record what it assumed has nothing to check at the checkpoint and will find out the way section 12 prices.

13. RTL 9 — The Order The Questions Are Asked In

Nine sections of inputs. This one is about the sequence, and it is the cheapest decision in the chapter because it costs nothing to make correctly.

Two questions decide the type. They do not cost the same to answer. The memory question — does the host need the device's memory — is usually a capacity conversation with numbers people already have. The cache question needs an access-pattern study, which is real work. And one of them settles a substantial fraction of designs outright: a device with no memory to share cannot be Type 2 or Type 3, and the cache question is then the only one left.

// RTL 9 - the order the questions are asked in. Two questions decide the type
// and one of them is cheap to answer; asking the expensive one first is a
// decision that costs nothing to get right and is made by habit.
module question_order #(parameter int START_WITH_THE_HARD_ONE = 0) (
  input  logic clk, rst_n,
  input  logic        plan_it,
  input  logic [15:0] cheap_q_cost, hard_q_cost, cheap_settles_pct, designs,
  output logic [15:0] cost_cheap_first, cost_hard_first, cost_now, cost_best,
  output logic        order_right,
  output logic [7:0]  n_plans, n_wasteful,
  output logic        order_err
);
  logic [15:0] settled, unsettled;
  logic [31:0] s_q, a_q, b_q;
  logic        truly_wasteful;
  // Asking the cheap question first settles some designs outright.
  // Taken in 32 bits: a thousand designs at a hundred percent overflows a
  // 16-bit product and settles a number smaller than it started with.
  assign s_q = ({16'd0, designs} * {16'd0, cheap_settles_pct}) / 32'd100;
  assign settled = (cheap_settles_pct > 16'd100) ? designs
                 : ((s_q > {16'd0, designs}) ? designs : s_q[15:0]);
  assign unsettled = designs - settled;
  assign a_q = ({16'd0, designs} * {16'd0, cheap_q_cost})
             + ({16'd0, unsettled} * {16'd0, hard_q_cost});
  assign cost_cheap_first = (a_q > 32'hFFFF) ? 16'hFFFF : a_q[15:0];
  assign b_q = ({16'd0, designs} * {16'd0, hard_q_cost})
             + ({16'd0, designs} * {16'd0, cheap_q_cost});
  assign cost_hard_first = (b_q > 32'hFFFF) ? 16'hFFFF : b_q[15:0];
  assign cost_best = (cost_cheap_first < cost_hard_first)
                     ? cost_cheap_first : cost_hard_first;
  assign cost_now = (START_WITH_THE_HARD_ONE != 0) ? cost_hard_first : cost_best;
  assign order_right = (cost_now <= cost_best);
  assign truly_wasteful = (cost_cheap_first < cost_hard_first) && (designs != 16'd0);
  assign order_err = plan_it && truly_wasteful && (cost_now == cost_hard_first);

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_plans <= 8'd0; n_wasteful <= 8'd0;
    end else if (plan_it) begin
      n_plans <= n_plans + 8'd1;
      if (truly_wasteful) n_wasteful <= n_wasteful + 8'd1;
    end
  end
endmodule

A cheap question that settles six designs in ten costs five hundred asked first and eleven hundred asked second — a factor of more than two, on a hundred designs, from a decision that costs nothing to make.

The last case is the one that makes the principle precise. The settling question wins even when it costs more to ask — a question at ten units that settles sixty percent beats one at one unit that settles nothing. The ordering value is in what a question resolves, not in what it costs, and conflating the two is how the wrong order gets chosen by people who are trying to be efficient.

The second case is the honest limit. A cheap question that settles nothing has no ordering value at all, and both orders cost the same. The technique needs a question that actually eliminates possibilities, and knowing which of your questions does that is itself a piece of knowledge about the decision.

What makes the memory question the settling one in practice is worth spelling out, because it is a fact about the type table rather than about cost. Two of the three types have device memory the host uses. A device with nothing to expose is a Type 1 or it is not a CXL device at all, and the cache question is then the only remaining one — the memory question has halved the space at the cost of asking somebody how much DRAM is on the board. A device that does have memory to expose is a Type 2 or a Type 3, and the cache question distinguishes them. Either way the memory question is answered from a datasheet and a capacity conversation, and the cache question needs a profile.

The ordering principle generalises past this decision, which is the reason it is in the chapter rather than in a footnote. Any sequence of questions with different costs and different resolving power should be asked in order of resolution per unit cost, and most engineering investigations are that shape — the debug ladder in section 21 is ordered the same way, with four reads of existing data before anything that requires building something. The mistake the model is built to catch is sorting by cost alone, which feels efficient, answers the cheap questions first, and can leave the expensive one still to ask on every design.

14. RTL 10 — A Device-Type Choice Assembled

Nine inputs. This one puts them in one place and makes the confident answer visible as what it is: one bit of six.

// RTL 10 - a device-type choice assembled. Everything that must hold before a
// type is the right answer, with "it is a Type 3" as one of the six rather
// than the whole decision.
module type_choice_signoff #(parameter int ITS_A_TYPE_THREE = 0) (
  input  logic clk, rst_n,
  input  logic       evaluate,
  input  logic       type_named,             // a type was chosen at all
  input  logic       workload_characterised, // the access pattern is known
  input  logic       cache_answered,         // does the device need host lines
  input  logic       mem_answered,           // does the host need device memory
  input  logic       type_procurable,        // the type can actually be bought
  input  logic       cost_justified,         // the extra capability is paid for
  output logic       choice_sound,
  output logic [5:0] fail_mask,
  output logic [7:0] n_eval, n_sound,
  output logic       false_type_err
);
  assign fail_mask[0] = ~type_named;
  assign fail_mask[1] = ~workload_characterised;
  assign fail_mask[2] = ~cache_answered;
  assign fail_mask[3] = ~mem_answered;
  assign fail_mask[4] = ~type_procurable;
  assign fail_mask[5] = ~cost_justified;
  // The it-is-a-Type-3 build stops at the first bit.
  assign choice_sound = (ITS_A_TYPE_THREE != 0) ? type_named : (fail_mask == 6'd0);
  assign false_type_err = evaluate && choice_sound && (fail_mask != 6'd0);

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_eval <= 8'd0; n_sound <= 8'd0;
    end else if (evaluate) begin
      n_eval <= n_eval + 8'd1;
      if (choice_sound) n_sound <= n_sound + 8'd1;
    end
  end
endmodule

The stimulus walks all six bits one at a time. When a type has been named and any one of the other five fails, the assembled model reports that the choice is not sound and the Type-3 view reports a decision.

Across the eight evaluations, the assembled model calls one choice sound and the Type-3 view calls six of them a decision.

The bit order is by how much of the decision each condition supplies. Bit 0 is the output. Bit 1 is the input to everything. Bits 2 and 3 are the decision itself. Bit 4 is a constraint from outside engineering entirely, and bit 5 is the check that the answer was worth its price — last because it can only be applied once the answer exists.

"It is a Type 3" is bit 0 and it is usually correct. That is what makes it the right weak definition for this chapter: it is not a wrong answer, it is a right answer arrived at without the decision that would justify it. Most CXL devices are memory expanders, so a candidate who says Type 3 by default will be right more often than not, and being right more often than not is indistinguishable from understanding until the case arrives where the default does not hold — a sparse accelerator, a phased workload, a host that is not short of memory. The bit is set and the choice is not sound, and the model's whole purpose is to make those two statements sit next to each other.

The five other bits are each a section's worth of work and they fail independently, which is the property that makes the mask useful rather than decorative. An uncharacterised workload can still produce a correct type by luck. An unanswered cache question can coexist with a correct memory answer. A type that cannot be bought is a complete engineering decision that a programme cannot execute. And a capability that does not pay for itself is a decision that was made carefully and should have landed elsewhere. None of the five implies any of the others, which is why the summary is a mask rather than a score.

Figure 4 — the mask ordered by how much of the decision each condition supplies, which is also roughly the order in which the answers become available. The first two are work done before a type is named, the middle two are the decision itself, and the last two are checks applied to an answer that already exists — which is why a programme that reaches them and fails one has to go back to the start rather than adjust something.

15. Quantitative Reasoning

Three of eight traits known is thirty-seven percent of a decision, and the estimate already carries the rework because a choice made from three eighths will be revisited.

Ninety percent of the traffic avoidable for a sparse workload on a four-kilobyte grain. For a dense one, zero — and the same accelerator label covers both.

A hundred and twenty-eight gigabytes spare against a two-hundred-and-fifty-six gigabyte request is half the request met, on a device with five hundred and twelve gigabytes on it.

Eighteen thousand units of penalty on two thousand of native cost — nine hundred percent, paid per access, forever. And thirty percent in the case that actually ships.

Sixty units of silicon against thirty of gain is nothing net. Against a hundred and fifty it is a two-hundred-and-fifty percent return, and the same view books both as wins.

Eight weeks past twelve of slack is a hundred and sixty-six percent of the schedule's margin, on a part that is at least offered. A part nobody offers is unbounded.

Four phases of ten on the wrong type is four hundred units of run time — forty percent of a correctly-characterised workload, lost to averaging.

Thirty-seven percent of the total cost sunk by finding out at week thirty of forty. Sixteen percent at week ten.

Five hundred against eleven hundred from asking two questions in the other order, on a hundred designs, at no cost.

One choice of eight sound; the Type-3 view counts six. The assembled model's summary, and the chapter's.

16. Assertions

The testbenches carry 523 checks across ten models.

Every output of every model is asserted as a value, in both builds. The output listing step reported twenty-five and two were real gaps — the procurement model's weeks-past-slack and the phase model's wrong-type run time. Both are parameter-dependent values whose neighbours were pinned, which is now the fifth chapter running where the outscan gap had exactly that shape.

Both builds are asserted on every degenerate case. A decision with no inputs, a device touching no host memory, a host wanting nothing, an unused device, a capability with no gain, a design wanting no type, an uncharacterised workload, a project that has not started, nothing to decide.

Every clamp that an input can reach is driven past its limit exactly once. More traits measured than the decision needs, a usage figure above the device's memory, a catch week beyond the project, a phase count above the workload's, a settle rate above a hundred percent, costs that saturate.

Every error output is checked in both directions in every case. Section 6's third and sixth cases, section 7's sixth, section 8's third and section 9's second exist to assert the quiet half — a dense non-accelerator, a fine transfer grain, a pure memory device, a workaround as cheap as the native path, and a question that settles nothing. Each is a case where the naive rule is right, and a model that alarmed on them would be unusable.

17. Mutation Testing

106 mutations, 106 killed. Fifty against the first testbench, fifty-six against the second.

Two mutations were reported as ANCHOR xN rather than as kills, one matching twice and one matching nothing — and both were mine. The first anchored on a fragment that appears in two expressions in the same model; the second on a variable name that had been changed during a widening fix and not updated in the mutation list.

That is the guard batch 025 lost its only point for not having, doing its job twice in one chapter. An unmatched anchor and a compile error are not kills, and a harness that silently counted them would have reported 106 of 106 while injecting 104. The cost of the guard is that the campaign has to be re-run after fixing them; the cost of not having it is a score that means nothing.

Both campaigns were otherwise clean on the first run, which is the second chapter running. The two rules that got them there — unbalance every counter split, and drive every zero-guard with the other inputs non-zero — are now applied while writing the stimulus rather than after a survivor asks.

18. Verification Strategy

Characterise before choosing. Section 5. The list of traits that decide the type is short and writing it down is the whole of the method.

Measure access density against the transfer grain. Section 6. This is the cache question and it is a ratio, not a judgement about what the device is.

Ask how much of the device's memory it is using itself. Section 7. Spare capacity, not installed capacity.

Price the workaround before accepting the type. Section 8. A thirty percent penalty is the one that gets absorbed, and it is only visible against the alternative.

Put the extra silicon next to the extra gain. Section 9. Type 2 is chosen by comparison or by hope.

Check availability and lead time before the design depends on the type. Section 10.

Characterise the phases, not the average. Section 11. A workload with a temperature or a protocol distribution needs the distribution.

Hold the review early enough to matter. Section 12. The cost of a change is linear in when it is made.

19. Synthesis and Implementation Reality

Type 1 is rare as a product and common as a design. Few vendors sell one; teams building their own accelerator reach it by needing CXL.cache and having no memory to share.

Type 2 carries a coherency agent, and that is the expensive part. Not the CXL.mem side — a memory controller is a known quantity — but the caching agent, the snoop path and the bias machinery, which is 27.3's whole chapter of constraints arriving as silicon.

A Type 2 can be operated as a Type 3. If the CXL.cache side is unused the device is a memory expander with extra silicon, which is what "we might need it later" actually buys. Whether that is worth its price is section 9.

Type 3 is the cheapest to build and the cheapest to buy, which is why the market looks the way it does and why the default answer is right so often.

A device's type is fixed at silicon. Nothing about it is a setting, which is why section 12's cost is a programme cost rather than a configuration change.

A device advertises its type in configuration space, through the DVSEC structures the host reads during enumeration, and the host builds its view of the device from that — which protocols to negotiate, whether to expect a memory range, whether to enable a coherency domain. The type is therefore discoverable at run time even though it is unchangeable, and a host that finds a Type 3 where the platform expected a Type 2 will enumerate it correctly and quietly, as a memory expander. Nothing reports that the system got a different device than it designed for, which is one more reason the wrong type is a silent condition rather than a loud one.

The three types do not partition the market evenly and the gap is widening. Memory expanders ship in volume because the workload that wants them — anything memory-bound on a host whose DIMM slots are full — is common and does not care what the device computes. Coherent accelerators are tied to particular compute products and ship at those products' volumes. Anyone reasoning about "CXL devices" in general is, in practice, reasoning about Type 3, and the interview question exists to find out whether that is a default or a conclusion.

20. Silicon Observability

Free, and on paper. Which protocols a part implements. This is the type and it is in the datasheet.

Cheap. The workload's access density against the transfer grain, from an existing profile. Section 6 is a ratio from numbers a performance team usually has.

Cheap. The device's own memory usage against its capacity. Section 7.

Moderate. Per-phase access behaviour, which section 11 needs and which a summary profile averages away. This is the measurement most likely to be missing.

Moderate. The penalty of the workaround, which requires building or modelling the alternative. Section 8's number is the hardest one on this list to get honestly.

Unobtainable before the fact. What the workload will do in two years. Section 9's "we might need it later" is a bet on this, and the model's contribution is to price the bet rather than to resolve it.

Free, once the part is in a system. The device's advertised type, from configuration space. This is the one observation that is unambiguous, and it answers the question nobody was confused about.

Expensive, and usually the one that decides an argument. What the other type would have cost. It requires the other part, a port of the software, and a comparable measurement — which is a programme of its own, and the reason section 8's tax is argued about rather than measured.

21. Debug Lab

An accelerator programme has a CXL device and the system underperforms the projection.

Step 1 — check which type was chosen and what decided it. Section 5. If the answer is "it is what we could get", the rest of this list is about a constraint rather than a mistake.

Step 2 — measure access density against the transfer grain. Section 6. If the device is a Type 3 and the access is sparse, the penalty is section 8's and it is structural.

Step 3 — check whether the host is actually using the device's memory. Section 7. A Type 2 whose CXL.mem side is idle is a Type 3 that cost more.

Step 4 — profile per phase, not overall. Section 11. A workload that is right for the type on average can be wrong for it forty percent of the time.

Step 5 — price the workaround against the native path. Section 8. This is the number that decides whether a change is worth making.

Step 6 — if a change looks warranted, compute the sunk cost first. Section 12. Late enough in a programme, the right answer is to live with the penalty.

Steps 1 to 4 are all reads from existing data, which makes this a cheap investigation whose result is usually a decision rather than a fix.

Step 7 — check whether the type was even the constraint. A system underperforming its projection has many candidate causes, and the device type is one of the more dramatic and less common. Link width and speed, a device latency worse than the model assumed, a NUMA-blind allocator placing hot pages on the far node — all of those produce the same symptom and all are cheaper to fix. Reach for the type only when the access pattern and the protocol set actually disagree, which steps 2 and 3 establish before anything is concluded.

The result of this ladder is usually one of three sentences, and it is worth knowing which one you are heading for. The type is right and something else is slow — go elsewhere. The type is wrong and it is early — change it. The type is wrong and it is late — section 12, and the answer is often to keep it and stop investigating.

22. Design Review

What does the workload do, and which traits decide the type?

What is the access density against the transfer grain — and has anybody computed it, or is the answer "it is an accelerator"?

How much of the device's memory does the device use itself?

If the type is wrong, what does the workaround cost per access?

For a Type 2: what is the extra silicon, and what is the gain that pays for it?

Is the type offered by anybody, and what is the lead time against the schedule's slack?

Does the workload have phases with different needs, and was the type chosen from the distribution or the average?

How late in the programme could this decision change, and what would it cost then?

23. How This Appears In Real Engineering

The part is chosen early, for a good reason, from an incomplete picture. The programme runs. The system works and underperforms.

What usually happened is section 6 answered by rule rather than by measurement — it is an accelerator, so it needs coherence — and the workload turned out to stream. The device is correct, the software is correct, and every access pays section 8's tax. Nothing reports it, because there is no error and no comparison.

The second shape is section 9 and it is a procurement story. A Type 2 was specified because it was the capable choice, the CXL.cache side was never used, and the programme paid for a coherency agent that sat idle. "We might need it later" is a real argument and it has a price, and the price is rarely written next to the argument.

The third is section 11, and it is the one that survives a good process. The workload was characterised, carefully, into a summary. The summary said cache, the device was chosen, and the load and write-back phases — forty percent of the run — are on the wrong device. The characterisation was correct and the averaging destroyed it, which is a failure of the representation rather than of the work.

The pattern is that the type decision is made once, early, from the least information the programme will ever have, and everything in this chapter is about getting more of that information before the decision than after it.

The fourth shape is procurement and it is the one nobody writes up. The design needed a Type 2, the part did not exist in time, and a Type 3 plus explicit data movement shipped instead. That is a correct decision under a constraint and it looks identical, two years later, to the mistake in the first shape — a memory expander doing an accelerator's job and paying per access. The difference is only in whether somebody wrote down that it was a constraint. A design record that says which of the eight traits were assumed and which were conceded is the only thing that distinguishes a compromise from an error afterwards, and it costs a paragraph at the time.

24. Common Misconceptions

"It is a Type 3." Usually right, and right from a fact rather than a decision. Section 14.

"It is an accelerator, so it needs CXL.cache." It needs it if its access to host memory is sparse. Section 6.

"It has memory, so expose it." Only what the device is not using. Section 7.

"The wrong type still works." It does. That is what makes the tax invisible. Section 8.

"Type 2 is the safe choice." It is the expensive one, and safety costs silicon. Section 9.

"We will specify what we need." And buy what exists, when it ships. Section 10.

"We characterised the workload." Into an average, or into a distribution? Section 11.

"We can change it later." Not without a different part, and the cost is linear in how late. Section 12.

25. Interview Reasoning

"Pick a device type for this workload." Ask what the workload does first. Two questions decide it: does the device need coherent access to host memory — which is about access density, not about being an accelerator — and does the host need the device's memory. Yes-no is Type 1, yes-yes is Type 2, no-yes is Type 3, no-no is a PCIe device.

"Why not just use a Type 2 and keep the options open?" Because it carries a coherency agent, bias machinery and the silicon for both directions. Put the extra cost next to the gain the workload actually realises. If the cache side is unused it is a Type 3 that cost more.

"How do you decide whether it needs CXL.cache?" Compare what a transfer would move against what the device actually touches. Sparse access to a large structure is the case; dense streaming is not, whatever the device is called.

"The workload has phases that want different things. What then?" That is the case where Type 2 earns its price, and the check is whether the phases' durations make it worth the silicon. Choosing from the average serves the dominant phase and runs on the wrong device for the rest.

"What if the type you want isn't available?" Then the design has chosen a schedule. The part being offered, shipping, and arriving inside the slack are three separate conditions and none is in the specification.

"You realise at week thirty of a forty-week programme that the type is wrong. What do you do?" Compute the sunk cost and the remaining penalty and compare. Late enough, the right answer is to live with it — which is why the review belongs at week two.

"Which of the two questions do you ask first, and why?" The memory question, because it is answered from a datasheet and a capacity conversation while the cache question needs an access-pattern study, and because two of the three types have memory the host uses — so the cheap question halves the space. The principle is resolution per unit cost, not cost alone.

"Is a Type 2 just a Type 1 and a Type 3 in one package?" No, and that is the useful part of the answer. Both protocols on one device means the device caches host lines while the host addresses device memory, and something has to decide which side holds the current copy of a given line. That is bias, and it is the machinery — plus its verification — that makes Type 2 expensive rather than the sum of two parts.

26. Exercises

1. Twelve traits decide a type and seven are known. Characterisation costs 20 and rework 300. Compute the described fraction and the expected spend. What is the spend if all twelve are measured?

2. A device touches 400 host lines at 128 bytes each. Compute the traffic at grains of 128, 512 and 4,096 bytes, and the avoidable fraction at each. At which grain does CXL.cache stop being worth having?

3. A device has 1 TB and uses 600 GB. The host wants 500 GB and is 500 GB short. How much is truly servable? What does a whole-device view promise, and what breaks when it is believed?

4. A workload makes 5,000 accesses. The native path costs 3 units each and the workaround 11. Compute the penalty and the percentage. At what workaround cost does the penalty reach 30%?

5. Type 3 silicon is 200 units and Type 2 is 340. The workload realises 60 units of gain at 3 units each. Is it justified? What gain would be needed to double the return?

6. A type is offered with a 26-week lead time against 18 weeks of slack. How late is it? What slack would be needed, and what is the alternative?

7. A workload has 20 phases: 8 want a cache, 5 want memory, 7 want neither, each 50 units long. How many phases does a single type serve, and how much run time is on the wrong device?

8. Extend the assembled model with a seventh bit for a condition this chapter does not cover. Justify its position using the rule that the ordering is by how much of the decision each condition supplies.

9. A workload has 6 phases wanting a cache at 1 second each and 4 wanting memory at 10 seconds each. Compute the coverage by phase count and by time. Which summary would a profile report, and which decides the type?

27. Summary

The type is an answer. Naming one before describing the workload is being right by luck rather than by reasoning.

Two questions decide it, and neither is about the device. Access density to host memory, and spare device memory against host demand.

Being an accelerator is not the reason for CXL.cache. Sparse access is, and a fine transfer grain removes the case entirely.

Only spare device memory can be exposed, and a whole-device view promises memory the device is using.

The wrong type works. Nine hundred percent in the case that gets found, thirty percent in the case that ships.

Type 2 is the expensive one, and "we might need it later" is a bet with a price that is rarely written down.

The type you specify and the type you can buy are different lists, and a part nobody sells is an unbounded delay.

A workload has phases, and the average of a phased workload is a workload that never runs.

The cost of changing is linear in when you find out — sixteen percent at week ten, thirty-seven at week thirty.

Six bits, and "it is a Type 3" is one of them. One choice of eight is sound; the Type-3 view counts six.