CXL.cache Question

27.2 took the first of the three protocols. This chapter takes the second, and it is the one where the wrong answer is not merely incomplete — it is frequently backwards.

Explain CXL.cache. — "the device has a cache". True, and it describes a device that can read stale data quickly. "the device caches its own memory" — that is CXL.mem with the words rearranged, and it is the most common wrong answer in the module.

1. The Engineering Problem — A Cache Is Not Coherence

A copy nobody can correct is a fast way to read stale data. Sixty-four lines held with sixteen the host does not track, against a hundred host writes, is twenty-five stale reads in a thousand — and every one of them is served at cache speed. Section 5.

The direction is the answer. CXL.cache is the device holding the host's lines. The opposite direction is a different protocol with a different name, and swapping them is the commonest wrong answer there is. Section 6.

Coherence is something the host does to the device. Sixteen lines the host never sends a snoop for are sixteen lines the host cannot correct, on a device that answers every snoop it receives. Section 7.

The state of a line decides what may be done to it. Twenty-five writes against ten owned lines is fifteen writes to lines the device only had a copy of — an error the host cannot detect afterwards, because there is nothing left to detect it with. Section 8.

A snoop response is on the host's critical path. A three-hundred-nanosecond device cache lookup against a hundred-nanosecond budget stops eight host cores, and the device's cache design has become the host's memory latency. Section 9.

This chapter against 27.2, stated precisely. That one owns a protocol that is PCIe with different framing. This one owns a protocol that has no PCIe equivalent at all — which is why every model here is about what the two sides must agree on rather than about what carries over, and why section 14's weak definition is a device feature rather than a protocol property.

2. The One-Sentence Model

A device caching host memory is coherent when it can hold host lines at all, when it is the device holding the host's memory rather than the reverse, when the host can reach in and invalidate, when the state of a line governs what the device may do to it, when the snoop response is inside the host's tolerance, and when the host's directory can track what the device holds — and "the device has a cache" is one of those six.

3. What This Chapter Owns

The content first, because the models are about the relationship and the relationship has to be stated before it can be modelled.

CXL.cache lets a device hold and modify host memory with the host's coherency protocol extended over the link. An accelerator reads a line of host memory, holds it in its own cache, works on it, and the host's cores see a consistent view throughout — the same guarantee they have from each other.

The device is a caching agent and the host is the home. This asymmetry is the architectural decision that makes CXL implementable. The host owns the coherency protocol, tracks who holds what, and issues the snoops; the device responds. A device vendor does not have to implement a CPU's coherency fabric, only a compliant agent within it.

The three things a device does with a line. It requests a line in some state — shared if it only needs to read, exclusive or modified if it intends to write. It holds it, subject to the state's rules. And it responds to snoops, giving the line up or downgrading its state when the host asks.

The three things the host does. It tracks which lines the device holds, in a structure with a finite number of entries. It snoops the device when a core or another agent needs a line the device may hold. And it takes lines back when it runs out of tracking capacity, whether or not the device is finished with them.

What this buys. The alternative for an accelerator is explicit transfer: DMA the buffer in, work on it, DMA it out, with software flushes to make the boundaries safe. That is efficient for large buffers touched densely and ruinous for sparse access to a large structure, where the transfer moves a great deal that will never be read. Coherence is a per-access cost against DMA's per-buffer cost, and section 12 puts the crossover on a graph.

There is a second benefit that is easy to overlook and is often the real reason a design chooses coherence. The software model changes. Explicit transfer requires the programmer to know where every buffer is, when it is safe to touch, and which flushes go where. Coherent attach means a pointer is a pointer on both sides, and the failure mode of getting it wrong is a performance problem rather than a correctness one. For a large accelerated application with complex, pointer-rich data structures, that difference can dominate the per-access arithmetic entirely — which is a legitimate answer to why coherence and is not the answer the models in this chapter measure.

What it costs. A cache in the device, snoop handling in the device, tracking capacity in the host, and a device cache lookup placed squarely on the host's memory latency path. Sections 9, 10 and 13 are the three prices.

The device types, because they follow from which protocols are present. A Type 1 device speaks CXL.io and CXL.cache: an accelerator with no memory of its own that wants coherent access to the host's. A Type 2 device adds CXL.mem: an accelerator with its own memory, where both sides need coherent access in both directions. A Type 3 device speaks CXL.io and CXL.mem and not CXL.cache at all: a memory expander, which caches nothing and is therefore outside this chapter entirely. CXL.cache is what makes a device Type 1 or Type 2, which is a cleaner way to remember the taxonomy than memorising the three lists.

The states, named, because an interviewer will ask. CXL.cache's device-side states are the familiar modified, exclusive, shared and invalid set. A line held modified has been written and the device owes the host the data when asked for it. Exclusive means the device is the only holder and may write without asking again. Shared means read-only, possibly held by others. Invalid means the device does not have it. The device requests a transition, the host grants or denies, and a snoop is the host requesting a transition in the other direction — usually toward invalid or shared.

What is worth saying beyond the list is why the list is short. A coherency protocol's state count is a design trade: more states allow finer decisions and more transitions to get right. CXL's device-side set is deliberately the well-understood one, because the whole architectural bet is that a device vendor can implement a compliant agent without implementing a CPU's coherency fabric, and an exotic state set would lose that bet.

Bias, for Type 2 devices. A device with its own memory can hold a region in device bias — the device accesses it directly without host coordination — or host bias, where the host has coherent access and the device's own accesses take the longer route. Section 11 is the cost model, and the point is that the right answer depends on the access mix rather than on the device.

4. Teaching-Model Boundary

Every model in this chapter is a teaching model, not a coherency implementation. It computes the one relationship the section is about and nothing else. There is no cache line, no state machine, no snoop filter and no interconnect anywhere in this file.

Each model is built twice from one source. A parameter selects between the measured build, which computes what coherence actually requires, and the has-a-cache build, which computes what follows from treating the device's cache as the whole of the story. Every section's headline number is the gap between them.

5. RTL 1 — A Cache Is Not Coherence

Start with the distinction the one-liner erases, because everything else in the chapter is a consequence of it.

A cache is a place to keep a copy. Building one is a solved problem and a device vendor can do it without talking to anybody. Coherence is an agreement about what happens to that copy when the original changes, and it cannot be built by one side alone — it requires the host to know the copy exists and to be able to reach it.

A device that has the first and not the second has not built half a coherent cache. It has built a mechanism for reading data that was correct at some point in the past, at full cache speed, with no indication of which reads those are.

// RTL 1 - a cache is not coherence. A device that can hold a copy of host
// memory and cannot be told when that copy is wrong has not built a cache; it
// has built a way to read stale data quickly.
module cache_without_coherence #(parameter int A_CACHE_IS_COHERENCE = 0) (
  input  logic clk, rst_n,
  input  logic        evaluate,
  input  logic [15:0] lines_held, lines_tracked, host_writes, reads_served,
  output logic [15:0] tracked_ok, untracked, stale_reads, coherent_pct,
  output logic        cache_is_coherent,
  output logic [7:0]  n_evals, n_stale,
  output logic        stale_err
);
  logic [15:0] true_stale;
  logic [31:0] s_q, c_q;
  logic        truly_stale;
  // The host cannot track more of the device's lines than the device holds.
  assign tracked_ok = (lines_tracked > lines_held) ? lines_held : lines_tracked;
  assign untracked  = lines_held - tracked_ok;
  // A host write to an untracked line leaves a copy nobody will correct.
  assign s_q = (lines_held == 16'd0) ? 32'd0
             : (({16'd0, untracked} * {16'd0, host_writes}) / {16'd0, lines_held});
  assign true_stale = (s_q > {16'd0, reads_served}) ? reads_served : s_q[15:0];
  assign stale_reads = (A_CACHE_IS_COHERENCE != 0) ? 16'd0 : true_stale;
  assign c_q = (reads_served == 16'd0) ? 32'd100
             : ((({16'd0, (reads_served - true_stale)}) * 32'd100)
                / {16'd0, reads_served});
  assign coherent_pct = (c_q > 32'd100) ? 16'd100 : c_q[15:0];
  assign cache_is_coherent = (stale_reads == 16'd0);
  assign truly_stale = (true_stale != 16'd0);
  assign stale_err = evaluate && truly_stale && cache_is_coherent;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_evals <= 8'd0; n_stale <= 8'd0;
    end else if (evaluate) begin
      n_evals <= n_evals + 8'd1;
      if (truly_stale) n_stale <= n_stale + 8'd1;
    end
  end
endmodule

Sixty-four lines held with forty-eight tracked leaves sixteen the host does not know about. Against a hundred host writes that is twenty-five stale reads of a thousand — ninety-seven percent coherent, which is the worst possible number to see because it looks like a rounding error.

Figure 1 — the same device, read two ways. The upper path is not observing anything wrong, and there is nothing wrong to observe from where it stands: the cache hits, the lines are present, the reads return promptly. The lower path counts the difference between what the device holds and what the host knows about, which is a quantity neither side has on its own. Ninety-seven percent coherent is the worst number on the page, because it is small enough to be mistaken for noise and unbounded in what it can cost.

The fifth case is the one that makes this ship. Untracked lines that the host never writes to produce no stale read at all — the cache is fully coherent, by luck, and both builds agree. That is the state a bring-up system is in: one writer, no contention, and a tracking gap that costs nothing until a second agent starts writing.

The last stimulus case draws the baseline. A device holding no lines at all cannot go stale — a thousand reads, all of them fully coherent, because reading through is always correct and never fast. That is what coherence is measured against, and it is the honest answer to why not just not cache.

The clamp here is worth a line. A tracking report naming more lines than the device holds is a directory read taken while lines were being evicted, and a model that believed it would compute negative untracked lines and report a cache more coherent than perfect.

The asymmetry worth carrying out of this section is about who can detect the problem. The device knows exactly which lines it holds. The host knows exactly which lines it tracks. Neither knows the difference, because the difference is between two records held in two places, and no instrument in either sees both. That is the general shape of every failure in this chapter and it is why section 21's debug lab needs data from both sides at nearly every step — a property that makes these sessions organisationally slow in a way that has nothing to do with their technical difficulty.

6. RTL 2 — Which Side Caches Which Memory

Now the mistake that costs candidates the question, and it is a single word in the wrong place.

CXL.cache is the device caching the host's memory. The device reaches across the link, takes a line of host DRAM, and holds it coherently. The traffic flows device-to-host for the request and host-to-device for the data and the snoops.

CXL.mem is the host accessing the device's memory. Different direction, different protocol, different use case — a memory expander rather than an accelerator.

An answer that says "the device caches its own memory coherently" has described neither. An answer that says "the host caches the device's memory" has described CXL.mem and called it CXL.cache.

// RTL 2 - which side caches which memory. CXL.cache is the DEVICE holding the
// HOST's lines. The opposite direction is a different protocol with a
// different name, and swapping them is the commonest wrong answer there is.
module cache_direction #(parameter int EITHER_SIDE_WILL_DO = 0) (
  input  logic clk, rst_n,
  input  logic        score_it,
  input  logic [15:0] dev_holds_host, host_holds_dev, lines_claimed, protocol_named,
  output logic [15:0] on_cache, on_mem, mis_attributed, right_pct,
  output logic        direction_right,
  output logic [7:0]  n_scores, n_swapped,
  output logic        direction_err
);
  logic [15:0] true_mis;
  logic [31:0] r_q;
  logic        truly_swapped;
  // CXL.cache carries the device's copies of host lines. Host copies of
  // device memory travel on CXL.mem and are not this protocol's subject.
  assign on_cache = dev_holds_host;
  assign on_mem   = host_holds_dev;
  assign true_mis = (host_holds_dev > lines_claimed) ? lines_claimed : host_holds_dev;
  assign mis_attributed = (EITHER_SIDE_WILL_DO != 0) ? 16'd0 : true_mis;
  assign r_q = (lines_claimed == 16'd0) ? 32'd100
             : ((({16'd0, (lines_claimed - true_mis)}) * 32'd100)
                / {16'd0, lines_claimed});
  assign right_pct = (r_q > 32'd100) ? 16'd100 : r_q[15:0];
  assign direction_right = (mis_attributed == 16'd0) && (protocol_named != 16'd0);
  assign truly_swapped = (true_mis != 16'd0) && (protocol_named != 16'd0);
  assign direction_err = score_it && truly_swapped && direction_right;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_scores <= 8'd0; n_swapped <= 8'd0;
    end else if (score_it) begin
      n_scores <= n_scores + 8'd1;
      if (truly_swapped) n_swapped <= n_swapped + 8'd1;
    end
  end
endmodule

Forty lines, all of them the host holding device memory, is zero lines on CXL.cache and forty on CXL.mem — zero percent right, with the protocol confidently named. The either-side view attributes nothing wrongly because it does not distinguish the two.

Figure 2 — one word in the wrong place. Both descriptions are of real CXL protocols and both are things the architecture does; only one of them is the answer to the question that was asked. The reason this is worth a figure rather than a sentence is that the two are not confusable in the specification and are highly confusable in speech, where "the device and the host share memory coherently" covers both and commits to neither.

The sixth case is the one that reaches an interview most often, and it is harder to correct than the fully-wrong one. Thirty lines right and ten wrong is seventy-five percent, which sounds like a candidate who mostly understands it. In practice a mostly-right answer here usually means the two protocols are held as one concept with a fuzzy boundary, and the ten wrong lines are the cases where the boundary was needed.

The clamp in this model has a meaning worth stating. A claim covering fewer lines than the answer described is what happens when somebody enumerates more cases than they actually addressed, and the model grades what was claimed rather than what was listed. That is the right way round: an answer is not improved by mentioning cases it did not resolve.

The degenerate case is worth reading because it inverts the usual polarity. An answer that names no protocol at all is reported as wrong-direction by both builds — not because the direction was swapped but because there is no direction to evaluate. The model requires a protocol to be named before it will grade the attribution, which is the difference between wrong and absent.

The one-line version to have ready: the device caches the host's memory, and the host caching the device's memory is the other protocol. That sentence answers the question and pre-empts the follow-up.

There is a reason the confusion is so durable and it is worth understanding rather than just correcting. Both protocols make a pointer work across the link, which is the property most people carry away from an introduction to CXL, and at that level of description they are the same thing. The distinction only appears when you ask which side's memory and who holds the copy — two questions that a summary sentence about coherent shared memory does not raise.

The practical test, for anybody trying to keep them straight: ask where the DRAM is. If the DRAM is on the host and a device wants to work on it, that is CXL.cache. If the DRAM is on the device and the host wants to use it, that is CXL.mem. A Type 2 device does both, in different directions, for different regions — which is exactly why Type 2 is where people get confused.

7. RTL 3 — The Host Must Be Able To Reach In

Section 5 established that the host has to know the copy exists. This section is the mechanism, and the property it establishes is uncomfortable if you are used to thinking of a cache as a private thing.

Coherence is not a property the device has. It is a thing the host does to the device. When a core needs a line the device may be holding, the host sends a snoop, and the device must give the line up or downgrade its state. A device that cannot be snooped for a line is a device holding a copy the host has no way to correct — and from the host's point of view that line simply does not participate in coherence at all.

// RTL 3 - the host must be able to reach into the device's cache. Coherence is
// not a property the device has; it is a thing the host does to the device, and
// a line the host cannot snoop is a line the host cannot correct.
module snoop_reach #(parameter int THE_DEVICE_KEEPS_ITSELF_HONEST = 0) (
  input  logic clk, rst_n,
  input  logic        probe_it,
  input  logic [15:0] lines_held, snoops_sent, snoops_answered, host_writes,
  output logic [15:0] answered_ok, unanswered, unsnoopable, reach_pct,
  output logic        snoopable,
  output logic [7:0]  n_probes, n_unreachable,
  output logic        snoop_err
);
  logic [15:0] true_unsnoopable;
  logic [31:0] r_q;
  logic        truly_unreachable;
  // A device cannot answer more snoops than were sent to it.
  assign answered_ok = (snoops_answered > snoops_sent) ? snoops_sent : snoops_answered;
  assign unanswered  = snoops_sent - answered_ok;
  // Lines the host never even sent a snoop for are the ones it cannot reach.
  assign true_unsnoopable = (snoops_sent > lines_held)
                            ? 16'd0 : (lines_held - snoops_sent);
  assign unsnoopable = (THE_DEVICE_KEEPS_ITSELF_HONEST != 0) ? 16'd0 : true_unsnoopable;
  assign r_q = (lines_held == 16'd0) ? 32'd100
             : (({16'd0, answered_ok} * 32'd100) / {16'd0, lines_held});
  assign reach_pct = (r_q > 32'd100) ? 16'd100 : r_q[15:0];
  assign snoopable = (unsnoopable == 16'd0) && (unanswered == 16'd0);
  assign truly_unreachable = ((true_unsnoopable != 16'd0) || (unanswered != 16'd0))
                             && (host_writes != 16'd0);
  assign snoop_err = probe_it && truly_unreachable && snoopable;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_probes <= 8'd0; n_unreachable <= 8'd0;
    end else if (probe_it) begin
      n_probes <= n_probes + 8'd1;
      if (truly_unreachable) n_unreachable <= n_unreachable + 8'd1;
    end
  end
endmodule

Sixty-four lines held with snoops sent for forty-eight leaves sixteen lines the host cannot reach at all — three quarters of the cache reachable — on a device that answered every snoop it was sent. The device's behaviour is impeccable and sixteen lines are outside the protocol.

The fourth case is the failure everybody already knows how to look for, and the model is explicit that it is a different one. A device that receives snoops and does not answer them is caught by both builds, because an outstanding snoop is visible from the host side. Silence is loud. The failure this section owns is the quieter one: a line nobody ever sends a snoop for, because nobody knows it is held.

The two failures need different instruments and that is the practical content of the distinction. An unanswered snoop is visible from the host side alone — the host sent it, no response arrived, and a timeout or a counter records that. An unsent snoop is visible from neither side alone, because the host has no reason to send one and the device has no reason to expect one. Only the comparison of two counts finds it, which is why it is the one that reaches production.

The sixth and seventh cases are the same configuration one writer apart. Unreachable lines with no host writes cost nothing — no stale data, no symptom, no alarm. Add a writer and the same configuration is a correctness failure. That is the shape 26.3 spends a chapter on from the other end.

It is worth being precise about what "unsnoopable" means here, because the word suggests a device defect and usually is not one. A line becomes unreachable through bookkeeping, not through capability: the device holds it, the host's record says it does not, and so no snoop is ever generated. The device's snoop logic is fine and would answer correctly if asked. The failure is upstream of the device entirely, in whatever caused the host's record and the device's contents to diverge — a dropped completion, a tracking entry evicted without an invalidate, a reset that cleared one side and not the other.

That is why the measurement is a count of snoops sent against lines held rather than an attempt to snoop the device. You cannot detect this by testing the device, which is the single most useful thing to know about it.

8. RTL 4 — The State Decides What May Be Done

A device holding a line holds it in a state, and the state is a contract about what the device may do next.

The rules are the familiar ones. A line held shared may be read, and other agents may hold it shared too. A line held exclusive or modified may be written, and the device is the only holder. The device must acquire the right state before it acts, not after.

What makes this worth a model is the failure mode. A device that writes a line it holds shared has not made a mistake the host can detect later. Other agents believe their copies are valid, because a shared line's copies are valid by definition. The divergence is silent, permanent, and invisible to every mechanism the protocol has.

// RTL 4 - the state of a line decides what may be done to it. A device holding
// a line shared and writing to it has not made a mistake the host can detect
// later; it has made one the host cannot detect at all.
module state_permission #(parameter int HOLDING_IT_MEANS_OWNING_IT = 0) (
  input  logic clk, rst_n,
  input  logic        evaluate,
  input  logic [15:0] lines_shared, lines_owned, writes_taken, reads_taken,
  output logic [15:0] writes_allowed, writes_illegal, acts_taken, legal_pct,
  output logic        states_honoured,
  output logic [7:0]  n_evals, n_illegal,
  output logic        state_err
);
  logic [15:0] true_illegal;
  logic [31:0] l_q;
  logic        truly_illegal;
  // Only an owned line may be written. A shared line may be read.
  assign writes_allowed = (writes_taken > lines_owned) ? lines_owned : writes_taken;
  assign true_illegal   = writes_taken - writes_allowed;
  assign writes_illegal = (HOLDING_IT_MEANS_OWNING_IT != 0) ? 16'd0 : true_illegal;
  assign acts_taken = writes_taken + reads_taken;
  assign l_q = (acts_taken == 16'd0) ? 32'd100
             : ((({16'd0, (acts_taken - true_illegal)}) * 32'd100)
                / {16'd0, acts_taken});
  assign legal_pct = (l_q > 32'd100) ? 16'd100 : l_q[15:0];
  assign states_honoured = (writes_illegal == 16'd0);
  assign truly_illegal = (true_illegal != 16'd0) && (lines_shared != 16'd0);
  assign state_err = evaluate && truly_illegal && states_honoured;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_evals <= 8'd0; n_illegal <= 8'd0;
    end else if (evaluate) begin
      n_evals <= n_evals + 8'd1;
      if (truly_illegal) n_illegal <= n_illegal + 8'd1;
    end
  end
endmodule

Twenty-five writes taken against ten owned lines is fifteen writes the device was not entitled to — eighty-eight percent of its acts legal. The holding-means-owning view counts no illegal write, because from inside the device a write to a held line is just a write.

The fifth case draws a boundary that matters. Illegal writes on a device holding nothing shared — more writes than owned lines, with no shared lines at all — is a capacity or a bookkeeping fault, not a coherency one, and the model declines to claim it. Getting this wrong in the other direction would make the model fire on every device that had simply run out of ownership.

The eighty-eight percent is the number to take away rather than the fifteen. Twelve percent of acts illegal is not twelve percent wrong; it is a cache that cannot be trusted at all, because nothing distinguishes the lines the illegal writes touched from the ones they did not.

The mechanism behind the silence deserves a sentence, because it is what makes this the most dangerous failure in the chapter. When a device writes a line it holds shared, the other holders' copies remain valid according to the protocol — they were shared, sharing is legal, and no rule was visibly broken from their side. There is no snoop, no invalidate and no state change anywhere except inside the offending device. Every subsequent read by every other agent returns data that the protocol guarantees is current, and is not.

Compare that with section 7's failure, which is at least detectable as a discrepancy between two counts. This one leaves no discrepancy anywhere. It has to be prevented by construction, which is why section 18's plan says to drive it deliberately rather than to look for it.

9. RTL 5 — A Snoop Response Is On The Host's Critical Path

Every section so far has been about correctness. This one is about the price, and it is the price that most often decides whether CXL.cache is used at all.

When a host core needs a line the device may hold, it does not proceed until the device responds. The device's cache lookup latency has become the host's memory latency, on that access, for that core. This is not a background query that the device can schedule; it is a core stopped, waiting.

The consequence for a device designer is that a cache structure chosen for the device's own hit rate may be the wrong structure. A large, deeply associative cache with a slow lookup is excellent for the device and is a tax on every host access that has to snoop it.

// RTL 5 - a snoop response is on the host's critical path. The device is not
// answering a background query; a host core is stopped until it replies, and
// the device's cache lookup latency becomes the host's memory latency.
module snoop_latency #(parameter int THE_DEVICE_HAS_ALL_DAY = 0) (
  input  logic clk, rst_n,
  input  logic        measure,
  input  logic [15:0] snoop_lat_ns, host_budget_ns, snoop_rate, cores,
  output logic [15:0] over_budget_ns, stall_ns, cores_stalled, budget_pct,
  output logic        snoop_bounded,
  output logic [7:0]  n_measures, n_over,
  output logic        latency_err
);
  logic [15:0] true_over;
  logic [31:0] s_q, b_q;
  logic        truly_over;
  assign true_over = (snoop_lat_ns > host_budget_ns)
                     ? (snoop_lat_ns - host_budget_ns) : 16'd0;
  assign over_budget_ns = (THE_DEVICE_HAS_ALL_DAY != 0) ? 16'd0 : true_over;
  assign s_q = {16'd0, true_over} * {16'd0, snoop_rate};
  assign stall_ns = (s_q > 32'hFFFF) ? 16'hFFFF : s_q[15:0];
  // Every core waiting on a snoop is a core doing nothing.
  assign cores_stalled = (true_over == 16'd0) ? 16'd0 : cores;
  assign b_q = (host_budget_ns == 16'd0) ? 32'd0
             : (({16'd0, snoop_lat_ns} * 32'd100) / {16'd0, host_budget_ns});
  assign budget_pct = (b_q > 32'd999) ? 16'd999 : b_q[15:0];
  assign snoop_bounded = (over_budget_ns == 16'd0);
  assign truly_over = (true_over != 16'd0) && (snoop_rate != 16'd0);
  assign latency_err = measure && truly_over && snoop_bounded;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_measures <= 8'd0; n_over <= 8'd0;
    end else if (measure) begin
      n_measures <= n_measures + 8'd1;
      if (truly_over) n_over <= n_over + 8'd1;
    end
  end
endmodule

A three-hundred-nanosecond snoop response against a hundred-nanosecond budget is two hundred nanoseconds over — three hundred percent of the budget, with eight cores waiting. The device-has-all-day view reports a bounded snoop, because from the device's side the response was sent promptly.

The percentage in this model saturates at nine hundred and ninety-nine rather than a hundred, for the same reason 27.1 section 5's does: exceeding the budget is the result, not an arithmetic error, and clamping at the whole would delete the number the section exists to produce. A clamp copied from a neighbouring model would have done exactly that.

The fifth case is the one that explains why this ships. A slow snoop that nothing ever sends costs nothing today — the latency is there, the budget is exceeded, and no traffic exercises it. A device validated against its own access patterns and not against a host contending for the same lines will pass.

A design consequence follows that is worth stating as a rule, because it cuts against a device team's instincts. The snoop path should be optimised for latency and the device's own path for throughput, and where they share a structure the snoop's requirement wins. A device that queues snoops behind its own in-flight work has added that work's duration to a host core's memory access — which means a deeper device pipeline, normally an unambiguous improvement, makes the host slower.

The number that makes this concrete is the one in the table. Three hundred nanoseconds is not an outrageous cache lookup for a large structure. A hundred nanoseconds is not an unreasonable host tolerance. The two are perfectly ordinary and their combination stops eight cores, which is the whole argument for measuring them against each other rather than separately.

10. RTL 6 — A Device Cache Is A Fixed Size

The second price, and the one that turns a correctness feature into a performance problem.

A device cache holds a fixed number of lines. A working set does not. When the set exceeds the cache, the excess is evicted and re-fetched, and every one of those evictions and fetches is coherency traffic on the link — not payload, not useful work, just the cost of having promised to participate.

The unpleasant property is that this cost scales with the mismatch rather than with the work. A device doing the same amount of useful work with a set twice too large for its cache generates coherency traffic that grows while the output does not.

// RTL 6 - a device cache is a fixed size and a working set is not. When the
// set does not fit, the coherency traffic generated by evicting and re-fetching
// is the cost, and it grows while the useful work does not.
module cache_capacity #(parameter int THE_CACHE_IS_BIG_ENOUGH = 0) (
  input  logic clk, rst_n,
  input  logic        measure,
  input  logic [15:0] cache_lines, working_set, accesses, coh_per_miss,
  output logic [15:0] resident, spilled, misses, coh_traffic,
  output logic        cache_fits,
  output logic [7:0]  n_measures, n_thrashing,
  output logic        thrash_err
);
  logic [15:0] true_spilled;
  logic [31:0] m_q, c_q;
  logic        truly_thrashing;
  // A cache holds what it holds; the rest of the set is re-fetched.
  assign resident     = (working_set > cache_lines) ? cache_lines : working_set;
  assign true_spilled = working_set - resident;
  assign spilled = (THE_CACHE_IS_BIG_ENOUGH != 0) ? 16'd0 : true_spilled;
  // Misses follow the spill this build believes in, so the big-enough build
  // reports none of them.
  assign m_q = (working_set == 16'd0) ? 32'd0
             : (({16'd0, spilled} * {16'd0, accesses}) / {16'd0, working_set});
  assign misses = (m_q > {16'd0, accesses}) ? accesses : m_q[15:0];
  assign c_q = {16'd0, misses} * {16'd0, coh_per_miss};
  assign coh_traffic = (c_q > 32'hFFFF) ? 16'hFFFF : c_q[15:0];
  assign cache_fits = (spilled == 16'd0);
  assign truly_thrashing = (true_spilled != 16'd0) && (accesses != 16'd0);
  assign thrash_err = measure && truly_thrashing && cache_fits;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_measures <= 8'd0; n_thrashing <= 8'd0;
    end else if (measure) begin
      n_measures <= n_measures + 8'd1;
      if (truly_thrashing) n_thrashing <= n_thrashing + 8'd1;
    end
  end
endmodule

A four-thousand-line working set in a thousand-line cache spills three thousand lines. Against ten thousand accesses that is seven and a half thousand misses and twenty-two thousand five hundred coherency messages — for a workload whose useful output is unchanged.

The last case is the limit and it is instructive rather than absurd. A device with no cache at all misses on everything — every access a miss, thirty thousand coherency messages — and the big-enough view calls it a perfectly sized cache. A device that participates in coherence without caching anything pays the protocol's full cost for none of its benefit, which is a configuration that exists more often than it should.

It arises two ways, both of them plausible. A device may have its cache disabled during bring-up or by a conservative driver setting, and nobody notices because everything still works. Or a device may have a cache that is correct and tiny relative to the workload — sixteen lines against a thousand — which the model's sixth case measures at a ninety-eight percent miss rate and saturated coherency traffic. Both look like a working coherent device from every functional test, and both produce a link carrying almost entirely protocol overhead.

The fifth case is the timing of the failure. A set that does not fit and is never accessed costs nothing, so the capacity problem is invisible until a workload arrives. That is the same shape as sections 7 and 13, and it is the reason all three of them are worth checking at design time rather than discovering at deployment.

The eviction traffic deserves one clarification, because the model's arithmetic understates it in a specific way. A miss is not one message. It is a request, potentially a snoop of other holders, a data return, and — if the line being replaced was modified — a writeback. The model's per-miss constant is a parameter for exactly that reason: the real number depends on the protocol's message set and on how dirty the evicted lines are, and a clean-evicting workload and a dirty-evicting one differ by roughly a factor of two on the same miss count.

What does not change is the shape. The traffic is proportional to the mismatch between the set and the cache, and a device that is nearly big enough generates very little of it while a device at half the required size generates a great deal.

11. RTL 7 — Which Side Pays

This section applies to Type 2 devices — an accelerator with memory of its own that both sides need coherent access to — and it is the one an interviewer reaches for to separate a memorised answer from an understood one.

A region of device memory can be held in one of two biases. In device bias the device accesses it directly, with no host coordination on each access, and a host access takes the longer coordinated route. In host bias the host has efficient coherent access and the device's own accesses are the ones that pay.

Neither is correct. The right bias is a property of the access mix, not of the device, and a workload that shifts from device-heavy to host-heavy across its phases wants the bias to shift with it.

// RTL 7 - which side pays. On a device with its own memory, a line can be
// biased toward the host or toward the device, and the same access costs a
// different amount depending on which way the bias points.
module bias_cost #(parameter int ONE_BIAS_FITS_ALL = 0) (
  input  logic clk, rst_n,
  input  logic        evaluate,
  input  logic [15:0] dev_accesses, host_accesses, near_cost, far_cost,
  output logic [15:0] cost_dev_bias, cost_host_bias, cost_now, cost_best,
  output logic        bias_right,
  output logic [7:0]  n_evals, n_misbiased,
  output logic        bias_err
);
  logic [31:0] d_q, h_q;
  logic        truly_misbiased;
  // Device bias: the device's own accesses are near, the host's are far.
  assign d_q = ({16'd0, dev_accesses} * {16'd0, near_cost})
             + ({16'd0, host_accesses} * {16'd0, far_cost});
  assign cost_dev_bias = (d_q > 32'hFFFF) ? 16'hFFFF : d_q[15:0];
  // Host bias: the other way round.
  assign h_q = ({16'd0, dev_accesses} * {16'd0, far_cost})
             + ({16'd0, host_accesses} * {16'd0, near_cost});
  assign cost_host_bias = (h_q > 32'hFFFF) ? 16'hFFFF : h_q[15:0];
  assign cost_best = (cost_dev_bias < cost_host_bias) ? cost_dev_bias : cost_host_bias;
  // The one-bias build never switches, whatever the access mix does.
  assign cost_now = (ONE_BIAS_FITS_ALL != 0) ? cost_host_bias : cost_best;
  assign bias_right = (cost_now <= cost_best);
  assign truly_misbiased = (cost_dev_bias < cost_host_bias)
                           && ((dev_accesses + host_accesses) != 16'd0);
  assign bias_err = evaluate && truly_misbiased && (cost_now == cost_host_bias);

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_evals <= 8'd0; n_misbiased <= 8'd0;
    end else if (evaluate) begin
      n_evals <= n_evals + 8'd1;
      if (truly_misbiased) n_misbiased <= n_misbiased + 8'd1;
    end
  end
endmodule

A hundred device accesses against ten host accesses, at one unit near and five far, is a hundred and fifty units under device bias and five hundred and ten under host bias — a factor of three and a half, from a choice the device does not make for itself.

The second case is the honest one. On a host-heavy mix the fixed bias happens to be right, and both builds pick the same answer — which is exactly why a fixed bias survives: it is correct for some workloads and nobody checks which. The third case is the boundary: a balanced mix costs the same either way and there is no preferred bias at all.

The fifth case is the degenerate one for the whole mechanism. When near and far cost the same, there is no bias to get right — no locality, no benefit, and the entire machinery is dead weight. That is worth knowing because it is the sanity check on any claim about bias: if the two costs are close, the bias is not the thing to tune.

The mechanism is worth one more sentence because "bias" is a word that sounds like a hint and is not. In device bias the device accesses its own memory without host coordination on each access — the host has agreed, for that region, not to cache it, so no snoop is needed and no permission is sought. In host bias the host may cache the region, so the device's own accesses have to go through the coordination path to stay coherent with whatever the host is holding.

The switch is therefore not a performance knob in the usual sense. It is an agreement about who is allowed to hold copies, and the cost difference is a consequence of that agreement rather than a tuning parameter. Saying that distinguishes an answer that has read about bias from one that understands it.

12. RTL 8 — What Coherence Buys Against What It Costs

Nine sections about how coherence works. This one is about whether to use it, which is what a good interviewer actually wants to know.

The alternative is explicit transfer. DMA the buffer to device memory, work on it locally, DMA it back, with software flushes around the boundaries to make them safe. It is well understood, it needs no coherency protocol, and for the right access pattern it is unbeatable.

The costs scale differently, and that is the whole answer. Explicit transfer pays once per buffer, regardless of how much of it is touched. Coherence pays per access, regardless of how large the buffer is. Sparse access to a large structure is coherence's best case and DMA's worst; dense access to a small buffer is the reverse.

// RTL 8 - what coherence buys against what it costs. The alternative is
// explicit transfer plus software flushes, and it is cheaper on some access
// patterns and ruinous on others - which is the actual engineering content
// behind "why would you want CXL.cache".
module coherence_worth #(parameter int COHERENCE_IS_ALWAYS_BETTER = 0) (
  input  logic clk, rst_n,
  input  logic        evaluate,
  input  logic [15:0] shared_accesses, transfer_size, flush_cost, coh_per_access,
  output logic [15:0] dma_cost, coh_cost, saving_pct, cheaper_cost,
  output logic        coherence_pays,
  output logic [7:0]  n_evals, n_wrong_choice,
  output logic        worth_err
);
  logic [15:0] chosen_cost;
  logic [31:0] d_q, c_q, s_q;
  logic        truly_worse;
  // Explicit transfer moves the whole buffer and flushes around it.
  assign d_q = {16'd0, transfer_size} + {16'd0, flush_cost};
  assign dma_cost = (d_q > 32'hFFFF) ? 16'hFFFF : d_q[15:0];
  // Coherence pays per access instead, and the accesses are what vary.
  assign c_q = {16'd0, shared_accesses} * {16'd0, coh_per_access};
  assign coh_cost = (c_q > 32'hFFFF) ? 16'hFFFF : c_q[15:0];
  assign cheaper_cost = (coh_cost < dma_cost) ? coh_cost : dma_cost;
  assign chosen_cost = (COHERENCE_IS_ALWAYS_BETTER != 0) ? coh_cost : cheaper_cost;
  assign s_q = (dma_cost == 16'd0) ? 32'd0
             : ((dma_cost > cheaper_cost)
                ? ((({16'd0, (dma_cost - cheaper_cost)}) * 32'd100)
                   / {16'd0, dma_cost}) : 32'd0);
  assign saving_pct = (s_q > 32'd100) ? 16'd100 : s_q[15:0];
  assign coherence_pays = (chosen_cost <= cheaper_cost);
  assign truly_worse = (coh_cost > dma_cost) && (shared_accesses != 16'd0);
  assign worth_err = evaluate && truly_worse && (chosen_cost == coh_cost);

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_evals <= 8'd0; n_wrong_choice <= 8'd0;
    end else if (evaluate) begin
      n_evals <= n_evals + 8'd1;
      if (truly_worse) n_wrong_choice <= n_wrong_choice + 8'd1;
    end
  end
endmodule

A hundred shared accesses against a four-kilobyte transfer with two hundred units of flush cost is two hundred coherently against four thousand two hundred and ninety-six by transfer — ninety-five percent saved. Raise the access count to ten thousand and the transfer wins, and the always-coherent view does not notice.

The third case is the number to know: the crossover is at 2,148 accesses for these parameters, and at the crossover neither is worse. A candidate who can say there is a crossover and here is what moves it has answered the question; one who says coherence is better has not.

The last case is the shape that sells CXL.cache. A sixty-thousand-unit buffer touched five times costs sixty-five thousand to move and ten to share — ninety-nine percent saved. That is the workload the protocol exists for: a large structure, sparsely touched, where transferring it is transferring mostly what will never be read.

Two caveats belong with this model, because the comparison is cleaner in arithmetic than in practice.

The costs are not in the same units in a real system. Transfer cost is bandwidth and occupancy; coherence cost is latency and link messages. A system that is bandwidth-rich and latency-sensitive will weight them differently from one that is the reverse, and the crossover moves accordingly. The model uses one unit because the shape is the point.

The access count is usually unknown at design time. That is the honest difficulty: the crossover is computable and the input to it is a property of a workload nobody has run yet. Which is an argument for supporting both paths where the cost of doing so is small, and for measuring rather than predicting where it is not.

13. RTL 9 — The Host's Tracking Structure Is Finite

The last of the three prices, and the one candidates almost never mention because it lives in the host rather than in the device.

Somewhere in the host is a record of which lines the device holds. It has to exist — a host that did not know would have to snoop the device for every access, which is the cost the structure exists to avoid. It has a fixed number of entries, and when the device holds more lines than the structure can track, the host has to reclaim entries.

Reclaiming an entry means taking a line back from the device that the device did not ask to give up. The device may still want it; it will fetch it again; and the cycle repeats. The device's cache is the right size, its access pattern is reasonable, and the limit it is hitting is in somebody else's silicon.

// RTL 9 - the host's tracking structure is finite. Somewhere in the host there
// is a record of which lines the device holds, it has a fixed number of
// entries, and when the device holds more lines than it has entries the host
// has to take lines back that nobody asked it to take back.
module directory_capacity #(parameter int THE_HOST_TRACKS_EVERYTHING = 0) (
  input  logic clk, rst_n,
  input  logic        evaluate,
  input  logic [15:0] dir_entries, lines_held, accesses, refetch_cost,
  output logic [15:0] tracked, backinval_lines, refetches, wasted_cost,
  output logic        capacity_tracked,
  output logic [7:0]  n_evals, n_overflowed,
  output logic        directory_err
);
  logic [15:0] true_backinval;
  logic [31:0] r_q, w_q;
  logic        truly_overflowed;
  // The directory tracks what it has room for; the rest must be reclaimed.
  assign tracked = (lines_held > dir_entries) ? dir_entries : lines_held;
  assign true_backinval = lines_held - tracked;
  assign backinval_lines = (THE_HOST_TRACKS_EVERYTHING != 0) ? 16'd0 : true_backinval;
  // Every line taken back is a line the device fetches again if it still wants it.
  assign r_q = (lines_held == 16'd0) ? 32'd0
             : (({16'd0, true_backinval} * {16'd0, accesses}) / {16'd0, lines_held});
  assign refetches = (r_q > {16'd0, accesses}) ? accesses : r_q[15:0];
  assign w_q = {16'd0, refetches} * {16'd0, refetch_cost};
  assign wasted_cost = (w_q > 32'hFFFF) ? 16'hFFFF : w_q[15:0];
  assign capacity_tracked = (backinval_lines == 16'd0);
  assign truly_overflowed = (true_backinval != 16'd0) && (accesses != 16'd0);
  assign directory_err = evaluate && truly_overflowed && capacity_tracked;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_evals <= 8'd0; n_overflowed <= 8'd0;
    end else if (evaluate) begin
      n_evals <= n_evals + 8'd1;
      if (truly_overflowed) n_overflowed <= n_overflowed + 8'd1;
    end
  end
endmodule

A thousand lines held against a two-hundred-and-fifty-six entry directory is seven hundred and sixty-eight lines taken back, three thousand seven hundred and fifty refetches and fifteen thousand units of wasted work — with nothing wrong on the device side at all.

The fifth case is the same timing observation as sections 7 and 10. An overflow nobody is accessing costs nothing — the lines are taken back, nothing refetches them, and the system is quiet. The cost arrives with the workload.

The last case is the degenerate one and it is a real configuration. A host with no directory at all cannot let a device hold anything: every line is taken back, every access refetches, and the device's cache is a structure that holds nothing for any length of time. The tracks-everything view is furthest from seeing this, which is the point of driving it.

The practical consequence for an answer: the device's cache size is bounded by the host's tracking capacity, not by the device's own area budget. A device designed with a larger cache than the host can track has bought silicon it cannot use, and the number is in a host datasheet rather than in the device's own specification.

There is a partial defence available that is worth knowing about. A host directory does not have to be exact — it can be conservative, tracking a superset of what the device holds and snooping more often than strictly necessary, which trades snoop traffic for tracking capacity. That changes the failure from back-invalidation to extra snoops, and extra snoops land on section 9's critical path.

So the three prices are connected rather than independent. Relieving the directory costs snoop traffic; relieving snoop latency costs device cache structure; relieving cache capacity costs directory entries. A design conversation that treats them as separate budgets will find that fixing one has moved the problem rather than solved it, which is a useful thing to say out loud in an interview about trade-offs.

14. RTL 10 — The CXL.cache Answer Assembled

Nine models, nine conditions. This one puts them in one place and makes the weak claim visible as what it is: one bit of six.

"The device has a cache" is the first thing anybody says, and it is the only one of the six that is a property of the device alone. The other five are agreements.

// RTL 10 - the CXL.cache answer assembled. Everything that must hold before a
// device caching host memory is coherent, with "the device has a cache" as one
// of the six rather than the whole claim.
module cache_answer_signoff #(parameter int IT_HAS_A_CACHE = 0) (
  input  logic clk, rst_n,
  input  logic       evaluate,
  input  logic       has_cache,        // the device can hold host lines at all
  input  logic       direction_right,  // the device caches host memory, not the reverse
  input  logic       snoopable,        // the host can reach in and invalidate
  input  logic       states_honoured,  // the line's state governs what may be done
  input  logic       snoop_bounded,    // the response is inside the host's tolerance
  input  logic       capacity_tracked, // the host's directory can track what is held
  output logic       answer_sound,
  output logic [5:0] fail_mask,
  output logic [7:0] n_eval, n_sound,
  output logic       false_cache_err
);
  assign fail_mask[0] = ~has_cache;
  assign fail_mask[1] = ~direction_right;
  assign fail_mask[2] = ~snoopable;
  assign fail_mask[3] = ~states_honoured;
  assign fail_mask[4] = ~snoop_bounded;
  assign fail_mask[5] = ~capacity_tracked;
  // The has-a-cache build stops at the first bit.
  assign answer_sound = (IT_HAS_A_CACHE != 0) ? has_cache : (fail_mask == 6'd0);
  assign false_cache_err = evaluate && answer_sound && (fail_mask != 6'd0);

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_eval <= 8'd0; n_sound <= 8'd0;
    end else if (evaluate) begin
      n_eval <= n_eval + 8'd1;
      if (answer_sound) n_sound <= n_sound + 8'd1;
    end
  end
endmodule

The stimulus walks all six bits one at a time. When the device has a cache and any one of the other five fails, the assembled model reports that the answer is not sound and the has-a-cache build reports a coherent device.

Across the eight evaluations, the assembled model calls one device coherent and the has-a-cache build calls six of them coherent.

The bit order is by how much of the answer lives outside the device. Bit 0 is entirely the device's. Bits 1 through 4 are agreements the device participates in. Bit 5 is entirely the host's, and it is last because it is the one a device-centric answer never reaches — which is also why it is the most effective follow-up an interviewer has.

Figure 4 — the mask as a follow-up order, arranged by how much of each condition lives outside the device. The first decision is a definition, the middle three are agreements the device participates in, and the last is entirely the host's — which is why a device-centric answer reaches the first four and stops.

15. Quantitative Reasoning

Numbers from the models, stated so they can be argued with rather than admired.

Sixteen untracked lines of sixty-four is twenty-five stale reads in a thousand. Ninety-seven percent coherent — a number that reads as a rounding error and is a correctness failure with no bound on its consequences.

Zero of forty lines correctly attributed, on an answer that named the protocol confidently and described the other one.

Sixteen lines out of reach on a device with a perfect snoop-response record. The device answered every snoop it received; sixteen lines never received one.

Fifteen illegal writes of twenty-five is eighty-eight percent legal acts — and a cache nothing can distinguish the good lines in.

Three hundred nanoseconds against a hundred is three hundred percent of budget and eight cores stopped. The device's cache lookup is now the host's memory latency.

Three thousand lines spilled produces twenty-two thousand five hundred coherency messages for a workload whose useful output did not change.

A hundred and fifty units against five hundred and ten, from the same accesses. A factor of three and a half decided by a bias setting, not by the device.

The coherence-versus-transfer crossover is at 2,148 accesses for a four-kilobyte buffer. Below it coherence saves ninety-five percent; above it the transfer wins.

Seven hundred and sixty-eight lines taken back by a host directory, on a device doing nothing wrong.

One of eight devices coherent; the has-a-cache view counts six. The assembled model's summary number, and the chapter's.

16. Assertions

The testbenches carry 537 checks across ten models.

Every output of every model is asserted as a value, in both builds. The output listing step reported twenty-seven on the first run and one was a real gap: the capacity model's coherency-traffic figure, which in the big-enough build reports zero on a link carrying twenty-two thousand messages. It is derived from the spill count and therefore parameter-dependent, which makes it exactly the kind of output that can be corrupted for free while the asserted neighbours stay correct.

Both builds are asserted on every degenerate case. A device with no lines, a claim with no protocol named, a cache nothing snoops, a device that only reads, a host with no budget declared, a working set of nothing, an idle device, a buffer nobody shares, a host with no directory.

Every clamp is driven past its limit exactly once. A directory naming more lines than are held, more snoop responses than snoops, more writes than owned lines, more host-side lines than the answer claimed, a set far larger than the cache, costs large enough to saturate.

Every error output is checked in both directions in every case. Section 7's fourth case, section 8's fifth case and section 11's second case exist to assert the quiet half — a device that does not answer snoops, illegal writes with nothing shared, and a fixed bias that happens to be right. In all three the measured build must stay silent.

17. Mutation Testing

104 mutations, 104 killed. Forty-nine against the first testbench, fifty-five against the second.

One mutation survived the first run, and it was a missing stimulus case rather than an equivalent mutant. Flipping the zero-guard in the staleness model — making a device that holds no lines report a saturated stale count — changed nothing, because the only case with zero lines held also had zero reads served, and the downstream clamp reduced the saturated value to zero.

The case the testbench was missing is a real and interesting one: a device holding no lines while still serving reads. That is a device reading through — always correct, never fast — and it is the baseline coherence is measured against. Adding it killed the mutation and gave section 5 the sentence it needed.

The distinction between this survivor and the batch-026 ones is worth drawing, because the response differs. An equivalent mutant created by the stimulus — a counter split evenly — is a testbench gap that a source-level check can find without running anything, which is why it became splitcheck.py. A missing stimulus case cannot be found that way: nothing in the source says which degenerate combinations are interesting. The only checks that find it are the output-listing step, which asks whether every published value is pinned, and the campaign itself. That is the argument for running the campaign even when the other checks are clean: it is the only instrument that asks whether the assertions are load-bearing rather than merely present.

Every counter was checked against the even-split rule from 26.7 section 17 before the campaign, and none needed repair — the second chapter running where that check came back clean on the first pass.

18. Verification Strategy

A verification plan for coherent device attach, which is mostly a plan for making the host's half observable.

Test with two writers, not one. Section 5's failure is invisible with a single writer. A tracking gap costs nothing until a second agent writes the line, and a bring-up environment is almost always single-writer.

Count snoops sent against lines held. Section 7's gap is a difference between two numbers the host already has, and it is the only way to find a line that is held and unsnoopable.

Drive writes against lines held shared, deliberately. Section 8's failure produces no error, no event and no divergence the protocol can see. It has to be tested by construction rather than waited for.

Measure snoop response latency under host contention, not in isolation. Section 9's cost only appears when a host core is waiting, and a device-side benchmark will not produce one.

Size the working set against the device cache, and report the coherency traffic separately from the payload. Section 10's cost is invisible in a bandwidth number that counts only data.

Sweep the bias across the access mix. Section 11's right answer moves with the workload, and a single-bias test measures one point on a curve.

Ask the host team for the directory capacity. Section 13's limit is not in the device's specification and is not discoverable from the device's side.

19. Synthesis and Implementation Reality

The device cache's lookup latency is a host-visible parameter. This is the design decision most likely to be made for the wrong reason: optimising the structure for the device's own hit rate, when the snoop path is what the host feels. A smaller, faster, less associative cache can be the better system-level choice.

Snoop handling needs a path that does not queue behind the device's own work. A snoop that waits for an in-flight compute operation has added that operation's latency to a host core's memory access.

The device cache size is bounded by the host's directory. Section 13. Area spent past that bound buys lines that will be taken back.

Bias switching costs something and needs a policy. A Type 2 device that supports both biases and switches on nothing useful has the mechanism and none of the benefit.

A coherent device is a device that can be snooped at any time. Power management, clock gating and low-power states all have to keep the snoop path alive or coordinate with the host about it, which is a constraint that catches teams late.

20. Silicon Observability

Free, already there. The device's own cache hit and miss counters. These answer section 10's capacity question directly and are usually present because the device team wanted them.

Cheap, if it exists. A count of snoops received and answered. This is section 7's measurement and it is the single most useful coherency counter a device can have.

Cheap, on the host side. Directory occupancy and back-invalidation counts. Section 13 is invisible without these and they are host-side registers a device team may not know to ask for.

Moderate. Snoop response latency, ideally as a distribution rather than a mean — 26.6 section 7's argument applies directly, because a host core waits for the slowest snoop and not the average one.

Expensive. A protocol analyser showing the coherency traffic, which answers every question in this chapter by direct observation. 26.7 is about using one well.

Unobtainable. A direct observation that a line is held and untracked. Section 5's failure is a difference between two views, and no single instrument sees both.

That last entry is the one that shapes the whole list. Four of the six conditions in section 14 are differences between two records rather than properties of one, which means the observability that matters is not a better instrument on either side — it is the ability to read both sides' counters at a comparable moment. A device with superb internal visibility and no way to line its numbers up against the host's is, for this chapter's purposes, less observable than a device with two counters and an agreed way to sample them together.

21. Debug Lab

An accelerator reads stale data occasionally. The device team reports the cache is working correctly, and it is.

Step 1 — count lines held against lines tracked. Section 5. Two numbers from two sides, and the difference is the fault.

Step 2 — count snoops sent against lines held. Section 7. A line held and never snooped is held outside coherence.

Step 3 — check whether the device ever writes a line it holds shared. Section 8. This produces no error and has to be looked for deliberately.

Step 4 — if the complaint is latency rather than correctness, measure snoop response under contention. Section 9. A device-side measurement will report a healthy number.

Step 5 — if throughput is the complaint, compare the working set against the device cache. Section 10, then 26.6.

Step 6 — ask the host for directory occupancy and back-invalidation counts. Section 13. A device being stripped of lines it still wants looks, from the device, exactly like a cache that is too small.

Step 7 — for a Type 2 device, check the bias against the actual access mix. Section 11.

Steps 1, 2 and 6 all need data from the host side, which is why this is one of the harder debug sessions to run from a device team alone.

The step worth insisting on is the first, because it is the one most often skipped in favour of something that feels more diagnostic. Counting lines held against lines tracked is two register reads and it either finds the problem or eliminates an entire class of it. Compared with attaching an analyser — which 26.7 prices at a maintenance window, a probe-loading risk and a capture that covers one link — it is free. The reason it gets skipped is that it requires asking somebody on the host team for a number, and that is a slower path than opening a tool you already have.

22. Design Review

What is the device cache's snoop-lookup latency, and what is the host's tolerance?

Can snoop handling proceed while the device's own pipeline is busy?

How many lines can the host's directory track, and is the device cache sized against that number?

Is there a counter for snoops received and answered?

Does the device ever issue a write for a line it holds shared, and how would anybody know?

For a Type 2 device: what drives a bias switch, and has the policy been tested against a phase-changing workload?

Does the device stay snoopable in every power state it can enter?

What is the expected working set, and what does the coherency traffic look like when it does not fit?

23. How This Appears In Real Engineering

The accelerator is faster in isolation and slower in the system. Both measurements are correct.

What is usually happening is section 9 or section 10 — either the device's snoop responses are stalling host cores, or the working set does not fit and the coherency traffic is consuming the link the payload needs. Neither appears in a device-side benchmark, because a device-side benchmark has no host contending for lines, and both appear immediately in a system one.

The second shape is stale data that nobody can reproduce. Sections 5 and 7 produce this, and the reason it resists reproduction is that it needs two writers and a tracking gap at the same time — which is a timing coincidence in a system that mostly has one writer. It reproduces under load and disappears in a debug environment, which is the signature to recognise.

The third is a device being stripped of lines by a host directory it has never heard of. From the device, this is indistinguishable from a cache that is too small, and a device team will respond by asking for more cache — which makes it worse, because the extra lines are also taken back. Section 13 exists because this conversation happens more than once.

The pattern underneath is the one the chapter opens with. Five of the six bits are agreements, and a device team can only see its own half of each. The device's cache works; the device answers snoops; the device obeys the states it is told about. Every one of those statements can be true while the system is incoherent.

The organisational form of that is worth naming because it determines how long these take to resolve. Each side's evidence exonerates it, correctly, and neither side's evidence is sufficient to identify the fault. A device team measuring its own cache, its own snoop responses and its own state transitions will find nothing wrong, publish that finding, and be right. A host team doing the same will do likewise. The fault lives in a difference that only appears when the two sets of numbers are put on the same page, and nothing in either team's normal workflow puts them there.

Teams that resolve these quickly have usually done one specific thing: agreed in advance on a small set of paired counters — lines held against lines tracked, snoops sent against lines held, directory occupancy against device cache occupancy — and made them readable together. That is a handful of registers and a script, decided at design time, and it converts a multi-week cross-team investigation into an afternoon.

24. Common Misconceptions

"The device has a cache, so it's CXL.cache." A cache is a copy. Coherence is an agreement about the copy. Section 5.

"The device caches its own memory." That is not this protocol. The device caches the host's memory. Section 6.

"The device answers every snoop, so it's coherent." For the lines it is snooped for. Section 7.

"It only writes lines it holds." Holding is not owning. Section 8.

"Snoop latency is the device's problem." It is the host's memory latency. Section 9.

"A bigger device cache is better." Up to the host's directory capacity, and not past it. Sections 10 and 13.

"Device bias is faster." For device-heavy access. The bias should follow the mix. Section 11.

"Coherence is better than DMA." Below the crossover. Above it, the transfer wins. Section 12.

25. Interview Reasoning

"Explain CXL.cache." The device caches the host's memory coherently — the device is a caching agent, the host is the home, the host tracks what the device holds and snoops it when a core needs the line. The device requests a line in a state, holds it under that state's rules, and gives it up when snooped. That is the answer; the rest is follow-up.

"How is that different from CXL.mem?" Direction. CXL.cache is the device holding host memory; CXL.mem is the host addressing device memory. Getting this backwards is the single most common error on this question.

"Why not just DMA the buffer?" Because the costs scale differently — DMA pays per buffer and coherence pays per access. For a large structure touched sparsely, DMA moves mostly what will never be read. A strong answer names the crossover as a thing that exists rather than picking a winner.

"What does coherence cost the host?" Three things: the snoop response is on a host core's critical path, the coherency traffic shares the link with payload, and the host needs a finite structure to track what the device holds. The third is the one that separates answers.

"A device holds more lines than the host can track. What happens?" The host takes lines back that the device did not offer. The device refetches them, and from the device's side this is indistinguishable from a cache that is too small — which is why asking for more cache makes it worse.

"When would you use host bias instead of device bias?" When the host is the one doing most of the accessing to that region. The bias should follow the access mix, and a workload with phases wants it to move.

26. Exercises

1. A device holds 256 lines and the host tracks 200. The host performs 500 writes and the device serves 4,000 reads. Compute the stale reads and the coherent percentage. Now remove the writes and recompute.

2. A device holds 512 lines and the host sends snoops for 512 but receives answers for 480. Is the cache snoopable? Which of the two failures in section 7 is this, and which instrument finds it?

3. A device takes 60 writes with 25 lines owned and 100 held shared. How many writes were illegal, and what percentage of its 300 total acts were legal? What, if anything, in the protocol would detect this?

4. A device snoop responds in 250 ns against a 120 ns host budget, at 200 snoops per interval, on a 16-core host. Compute the over-budget time, the stall and the budget percentage.

5. A 2,048-line cache holds an 8,192-line working set over 20,000 accesses at 4 coherency messages per miss. Compute the spill, the misses and the traffic. What cache size halves the traffic?

6. A Type 2 region sees 300 device accesses and 80 host accesses, with near cost 2 and far cost 9. Compute both bias totals. At what host-access count do they cross?

7. A 16 KB buffer costs 500 to flush and 2 per coherent access. Compute the crossover access count. Now double the buffer size and recompute — which direction does the crossover move, and why?

8. Extend the assembled model with a seventh bit for a condition this chapter does not cover. Justify its position using the rule that the ordering is by how much of the condition lives outside the device.

27. Summary

A cache is not coherence. A copy the host cannot correct is a fast way to read stale data, and sixteen untracked lines of sixty-four is twenty-five stale reads in a thousand.

The device caches the host's memory. The other direction is the other protocol, and swapping them is the commonest wrong answer in the module.

Coherence is something the host does to the device. A line nobody snoops is a line outside the protocol, on a device with a perfect response record.

Holding is not owning. A write to a shared line produces no error, no event and no way to find it afterwards.

A snoop response is a host core's memory latency. Three hundred nanoseconds against a hundred stops eight cores.

A device cache is a fixed size and a working set is not. Three thousand spilled lines is twenty-two thousand coherency messages for unchanged output.

The right bias follows the access mix, not the device — a factor of three and a half from one setting.

Coherence pays per access and DMA pays per buffer. There is a crossover, and naming it is the answer.

The host's directory is finite, and a device holding more than it can track is stripped of lines it still wants.

Six bits, and "the device has a cache" is one of them. One device of eight is coherent; the has-a-cache view counts six.