CXL.cache Overview

Chapter 6.1 placed CXL.cache next to its two siblings and said what it is for. This chapter goes inside it and asks what it costs — because the moment a device keeps a copy of host memory, it has signed up to answer questions about that copy for as long as it holds it.

1. The Engineering Problem — A Cached Copy Is Not a Private Copy

A device that wants host data has two ways to get it. It can read the data, use it, and forget it. Or it can keep it.

The first costs a round trip per use. The second costs a round trip once — and then costs something quite different, forever: the device now holds state that the rest of the system has an opinion about. Someone else may want to write that line. Someone else may need to know whether the device's copy is newer than memory. The device cannot answer "I'm busy" and it cannot answer "I threw it away" unless it actually did, in a way the system agreed to.

That is the trade CXL.cache offers, and it is not a small one:

The obligation is the subject of this module. A device that treats CXL.cache as "DMA with a buffer" builds hardware that is correct in isolation and wrong in a system.

2. The One-Sentence Model

Borrowed data, host-coordinated ownership. The device may hold a copy of host memory, but the host orchestrates who is allowed to do what with that line — so every line the device caches creates state the device must maintain, defend and eventually surrender in a way the system agreed to.

Call it the borrowed line. Not "the device's cache" — the data is on loan, and the loan has terms.

3. What This Chapter Owns

4. The Shape of the Thing

Four responsibilities, and it is worth being precise about each.

Host memory is where the line lives when nobody has borrowed it. It is host-owned throughout — caching a line never transfers the memory, only a copy and some permission.

Host coherence logic is the coordinator the sourced quote names. Chapter 3.4 calls this the serialisation point: it decides the order of conflicting requests to a line and it knows who holds copies.

The device coherence agent is the piece that exists only because the device caches. Its job is to be answerable — to maintain per-line state, to act when coherence reaches it, and to make sure the accelerator never sees a line the device is not entitled to serve. This is what 8.3 is about.

The device cache holds tags, state and data. The state is the part engineers under-build, and §5 is about why.

The accelerator never touches host memory directly. Every access goes through the cache, and the cache is the thing that is answerable. That indirection is the whole architecture.

5. What the Device Must Remember

A non-coherent buffer needs to remember two things about a line: which address it holds, and whether it holds anything. A coherent cache needs more, and the extra fields are not bookkeeping — each one exists because a specific question can be asked.

Architectural. The last two are the ones that surprise people.

dirty determines who has the authoritative value. If the device modified the line and memory has not been updated, then reading memory yields a stale value — and the device is the only place the real one exists. That makes the device's copy load-bearing for the whole system, which is why it cannot simply be dropped (§10).

pending exists because transitions are not instantaneous. A line whose permission is being renegotiated is in neither the old state nor the new one. It has a third status, and code that recognises only "valid" and "invalid" will serve it — which is the defect in §8.

Those four fields have a shape when you draw them out: a line moves between permission levels, and every move toward more permission passes through a state where the line is unusable.

Teaching model. These are not CXL.cache state encodings. Read three things off the shape.

Every arrow toward more permission passes through a PENDING state, and the accelerator cannot be served from either of them. That is the WAIT answer in §8, drawn.

The arrow out of OWNED says "write back, demote", not simply "demote". A modified line owes its data to the system before it gives up the right to hold it — the omission that Debug Lab 4 is about.

There is no arrow from SHARED directly to OWNED. Gaining permission is not a local decision, so the path goes through a request and a grant. A design with that shortcut arrow has the defect in Debug Lab 5.

6. Teaching-model boundary

7. RTL 1 — A Tag Match Is Not a Hit

The most common way to get a coherent cache wrong is to build a non-coherent one and add state later.

  // A hit needs THREE facts, not one: the tag matches, the line is not
  // INVALID, and no transition is in flight. Dropping any of the three is a
  // separate real bug, and HIT_IGNORES_VALID models the first.
  assign usable_hit = HIT_IGNORES_VALID
                    ? tag_match
                    : (tag_match && (state_m[rd_idx] != 2'd0) && !pend_m[rd_idx]);

=== EXP1: a tag match is not a hit ===
  valid shared line   : tag_match=1 usable_hit=1
  tag + valid + settled -> hit                       : ok
  invalid line        : tag_match=1 correct hit=0 | ignores-valid hit=1
  tag matched but the line is invalid: not a hit     : ok
  the broken variant served stale data on tag alone  : ok
  transient line      : tag_match=1 usable_hit=0
  a line mid-transition is not usable yet            : ok
  valid-line population = 2
  population counted transitions, not writes         : ok

tag_match is high in all three cases and usable_hit is high in exactly one. The tag answers "which line", not "may I use it" — and a device that conflates them serves data it invalidated, or data whose permission is mid-negotiation.

The population counter is a second, quieter lesson. It moves on the transition into and out of the valid states, not on every write:

        if ((state_m[wr_idx] == 2'd0) && (wr_state != 2'd0)) n_valid_q <= n_valid_q + 8'd1;
        if ((state_m[wr_idx] != 2'd0) && (wr_state == 2'd0)) n_valid_q <= n_valid_q - 8'd1;

That distinction is invisible while every line goes invalid-to-valid exactly once, which is why the first testbench missed it:

=== EXP11: rewriting a line that is already valid ===
  two rewrites of a live line : population 2 -> 2
  population tracked transitions, not writes        : ok
  and it decremented when the line went invalid    : ok

8. RTL 2 — Three Answers, Not Two

A non-coherent cache lookup returns hit or miss. A coherent one has a third answer, and needs a fourth concept.

  // WAIT dominates: a transition in flight settles the question later.
  assign is_wait = req_valid && tag_match && line_pending && !TRANSIENT_IS_HIT;
  assign is_hit  = req_valid && present && !is_wait &&
                   (need_write ? writable : 1'b1);
  // Present-but-not-writable is not a miss: the data is here, the PERMISSION
  // is not. Treating it as a miss re-fetches data the device already holds.
  assign need_upgrade = req_valid && present && !is_wait && need_write && !writable;
  assign is_miss = req_valid && !present && !is_wait;

=== EXP3: hit, miss, and WAIT are three answers ===
  settled shared, read : hit=1 miss=0 wait=0 upgrade=0
  transient line, read : correct wait=1 hit=0 | transient-is-hit hit=1
  a line mid-transition returns WAIT, not HIT        : ok
  the broken variant served it anyway               : ok
  absent line -> miss                               : ok
=== EXP4: present but not writable is an UPGRADE, not a miss ===
  shared line, write   : hit=0 miss=0 upgrade=1
  data present, permission absent -> upgrade         : ok

Two ideas worth keeping.

WAIT is not a miss. A miss starts a transaction. A line already mid-transition has a transaction — starting a second one is exactly the conflict §9 exists to prevent. The right answer is "ask again shortly".

An upgrade is not a miss either. The data is already in the cache; only the permission is missing. Classifying it as a miss re-fetches bytes the device is already holding — correct output, wasted bandwidth, and on a write-heavy workload it is a large waste.

9. RTL 3 — One Line, One Transaction

    // A device must not run two coherent transactions on ONE line: their
    // state transitions would interleave.
    for (k = 0; k < NENT; k = k + 1)
      if (busy_q[k[1:0]] && (line_m[k] == alloc_line)) line_busy = 1'b1;

=== EXP5: one line, one transaction ===
  two different lines : occupancy=2 allocated=2
  different lines proceed concurrently               : ok
  same line again     : allocated=2 denied=1 two-live-error=0
  the second same-line request was refused           : ok

Different lines proceed concurrently; the same line serialises. That is the whole scheduling rule, and it is why coherent caches can still be fast: the serialisation is per line, not global.

A checker that was measuring the wrong thing

The first version of this module raised an error flag when it refused a same-line request. That is backwards, and simulation caught it immediately: the good instance failed its own test while behaving correctly.

Refusing is the mechanism. The property is that two live entries never name the same line, and those are different statements — one is an action the design takes, the other is a fact the design maintains. The fix separates them:

  // The real invariant: no two LIVE entries may name the same line. Refusing a
  // request is the mechanism; this is the property that mechanism protects.
  for (k = 0; k < NENT; k = k + 1)
    for (j = 0; j < NENT; j = j + 1)
      if ((k != j) && busy_q[k[1:0]] && busy_q[j[1:0]] &&
          (line_m[k] == line_m[j])) dup_live = 1'b1;

n_same_line_q now counts refusals as an observation — useful telemetry, since a high count means the workload is hammering one line — while two_live_same_line_err is the actual safety property. Never assert on the mechanism when you can assert on the property it protects.

Freeing an entry is not the same as answering it

=== EXP7: an entry frees on COMPLETION, not on the response ===
  after response only : correct busy=1111 | free-on-response busy=0111
  the entry is still held after the response         : ok
  the broken variant released it early               : ok
  completion released it                             : ok
  the broken variant then saw a recycled entry       : ok

A response arrives; the device still has work to do with it. Releasing the entry at the response lets the next allocation take that slot while the previous transaction is still finishing — and a late message from the old transaction then lands on the new occupant. This is the same identity hazard 7.5 raised about tag reuse, and 8.5 takes it further still.

10. RTL 4 — Drop Yes, Gain No

Architectural, and a direct consequence of the sourced quote. If the host orchestrates coherency management, then a device cannot help itself to more permission than it was given.

  // Dropping permission is the device's own business. Gaining it is not:
  // a promotion needs the host's grant, or the device has invented rights.
  assign legal = req && (ALLOW_SILENT_UPGRADE ? 1'b1
                       : (promote ? host_granted : 1'b1));

=== EXP8: the device may drop permission, not grant itself more ===
  SHARED->OWNED, no grant : correct legal=0 state=1 | silent-upgrade legal=1
  promotion without a host grant was refused         : ok
  the broken variant promoted itself                : ok
  with the host's grant the promotion applied        : ok
  dropping permission needs no grant                 : ok

The asymmetry is the point. Losing permission is always safe; the device simply becomes less able to act. Gaining permission is never safe on the device's own authority, because the host may have granted that permission elsewhere.

And one thing the device may not silently do, even though it looks like dropping:

=== EXP9: a dirty line cannot simply go invalid ===
  abuse instance flagged discarding the modified copy : ok

Going to INVALID while holding the only modified copy is not dropping permission — it is destroying the authoritative value. The data has to go somewhere first. That check needed an instance reserved for illegal stimulus, because a correct device never does it and driving it into the instance under normal test would trip that instance's own checker and be scored a failure.

11. RTL 5 — Counters That Describe a Coherent Cache

=== EXP10: hit rate needs the right denominator ===
  lookups=24 completions=16 hits=8 | attempts-variant hits=12
  hit count matched an independent oracle            : ok
  counting attempts inflated the hit count           : ok
  correct hit rate = 8/16 | inflated = 12/16
  hits never exceeded completions                    : ok

Twenty-four lookups produced sixteen completions — eight lookups did not complete, because they were waits or upgrades. Counting hits per lookup reports 12/16; counting per completion reports 8/16. The first number is not merely optimistic, it is measuring a different thing.

The oracle here is a testbench-side model built from the definition of a hit, with no connection to the design's expressions. That independence is what makes the comparison worth anything.

12. The Structures, Side by Side

Five modules, and it is worth seeing which part of a cache each one is.

The guard is the one that sits in an unexpected place. It does not gate the accelerator's path at all — it gates writes into the state array, which is where every permission change lands regardless of whether it came from the accelerator, the coherence agent or an eviction. Putting the check at the storage rather than at each caller is why no call site can forget it.

13. Quantitative Reasoning — What Coherence Costs in Bits

Illustrative. Take a device cache with 256 sets, 8 ways and 64-byte lines, on a 48-bit physical address.

Now the metadata, using this chapter's teaching fields:

So the metadata array is 2048 × 38 = 77,824 bits = 9,728 bytes ≈ 9.5 KiB, which is 7.42% of the data array.

Then read the split. Of those 38 bits, only three — state and pending — exist because the cache is coherent. That is 0.59% of the cache.

Coherence is cheap in bits and expensive in everything else. Three bits per line buys the storage. What it costs is a lookup that returns three answers, a serialisation rule per line, a transaction tracker, a legality check on every transition, and a verification space that grows with the product of all of them. Anyone who budgets coherence by counting metadata bits has budgeted the wrong resource.

That sentence is the honest summary of the module, and every later chapter is an instance of it.

14. Assertions

Icarus Verilog 13.0 is the simulator available here and does not support concurrent SystemVerilog assertions, so every property is implemented as synthesisable checker logic and verified in simulation. Properties are classified, because a safety property never proves progress.

Safety

Liveness

Both are stated with their assumption attached, because neither is true unconditionally — and a design that cannot state the assumption has not established the property.

Performance goals

15. Mutation Testing

Fifteen mutations, each a plausible coherence mistake. Fifteen killed.

The first run scored 11 of 15, and the four survivors are the interesting part.

Population counted per write — missing stimulus. Every line in the test went invalid-to-valid exactly once, so the two counting rules agreed. Rewriting a live line separated them (EXP11).

Two checkers were unreachable by construction. multi_class_err could never fire, because hit requires present and miss requires !present — they are mutually exclusive by construction, so the checker was decoration. Same for hit_exceeds_complete_err, which is gated by complete on the correct design.

The fix is the one this track keeps arriving at, applied twice:

  // OVERLAP_CLASSES drops the presence test, so a present line reports HIT
  // and MISS at once -- which is what makes the one-class checker earn its keep.
  assign is_miss = req_valid && (OVERLAP_CLASSES ? 1'b1 : !present) && !is_wait;

  overlap variant flagged multi_class                : ok
  abuse instance flagged hits exceeding completions  : ok

A checker that cannot fire is not verification, it is documentation. Either build the variant that makes it fire, or delete it and admit the property is structural.

A response for no live transaction — missing stimulus, needing an abuse instance. Driving a stray response into the tracker under test trips that tracker's own checker, so the positive test got a dedicated DUT, and asserts two things: that the stray response is flagged, and that it frees nothing.

  abuse instance flagged a response with no request  : ok
  and it freed nothing: an unknown id is void       : ok

16. Verification Plan

The coverage model that matters is a cross of line state against request type: invalid / shared / owned × read / write × settled / pending. The pending column is the one a non-coherent testbench does not have, and it is where the WAIT class lives.

17. Silicon Observability

If this block misbehaves in a real device, these are the registers worth having:

The pair that resolves the most confusing class of bug is unknown-response count against denied allocations. A device that is slow because it is full looks nothing like a device that is slow because responses are arriving for transactions it has already retired, and one counter cannot tell you which.

18. Debug Lab

// Cache lookup.
assign hit = (tag_m[idx] == probe_tag);

  invalid line : tag_match=1 correct hit=0 | ignores-valid hit=1

assign usable_hit = tag_match && (state_m[idx] != INVALID) && !pend_m[idx];

assign hit = tag_match && (state != INVALID);   // pending not considered

  transient line, read : correct wait=1 hit=0 | transient-is-hit hit=1

assign is_wait = req_valid && tag_match && line_pending;   // a third answer
assign is_hit  = req_valid && present && !is_wait && ...;

if (rsp_valid) begin
  busy_q[rsp_id] <= 1'b0;        // released as soon as it is answered
end

  after response only : correct busy=1111 | free-on-response busy=0111
  the broken variant then saw a recycled entry       : ok

if (rsp_valid)  answered_q[rsp_id] <= 1'b1;    // answered
if (done_valid) busy_q[done_id]    <= 1'b0;    // finished — only now

// Accelerator no longer needs the line — drop it.
state_m[idx] <= INVALID;

  abuse instance flagged discarding the modified copy : ok

if (legal && dirty && (nxt_state == INVALID)) dirty_dropped_err <= 1'b1;

// Accelerator wants to write a line we already hold shared.
state_m[idx] <= OWNED;      // no grant consulted

  SHARED->OWNED, no grant : correct legal=0 state=1 | silent-upgrade legal=1

assign legal = req && (promote ? host_granted : 1'b1);

if (lookup && was_hit) n_hit_q <= n_hit_q + 1;   // counted per lookup

  correct hit rate = 8/16 | inflated = 12/16

if (complete && was_hit) n_hit_q <= n_hit_q + 1;

19. Design Review

1. Does a hit require state, or only a tag? If only a tag, invalidation does not work.
2. How many answers does a lookup have? Two means transient lines are being served.
3. Is an upgrade distinguished from a miss? If not, the device re-fetches data it holds.
4. What prevents two transactions on one line? And is the assertion on the mechanism or on the property?
5. When is a tracker entry released — at the response or at completion?
6. Can the device raise its own permission? It must not.
7. What happens to a dirty line on eviction? If it can go straight to invalid, data is lost.
8. Which denominator does the hit rate use?
9. Which counters distinguish "full" from "confused"?

20. How This Appears in Real Engineering

Architecture. The decision to support CXL.cache at all is the decision to build a coherence agent, and it is not incremental — it changes the cache's lookup, its eviction path, its transaction tracking and its verification plan simultaneously.

RTL. The state and pending fields are three bits that touch every path in the cache. Engineers who scope the work by counting bits under-scope it badly.

DV. The pending column doubles the state space of every lookup test, and the same-line concurrency case cannot be reached by random address streams that do not deliberately collide.

Post-silicon. Transient-line occupancy and unknown-response counts are what separate "the cache is too small" from "the coherence path is misbehaving", and they cost almost nothing to add.

Performance. The hit-rate denominator question is not pedantry: teams have sized caches from an inflated number and shipped a device that stalls on coherence rather than capacity.

21. Common Misconceptions

22. Interview Reasoning

23. Exercises

1. Analysis. A device reports lookups=1000, completions=1000, hits=980, and the accelerator stalls 40% of cycles. Nothing in those numbers is inconsistent. Name the two structures whose counters you would read next, and what each would tell you.

2. Design. The teaching metadata has one pending bit. Give a case where one bit is insufficient, state what the second bit would encode, and say what new invariant it introduces.

3. RTL task. Extend hit_classify so an upgrade on a line that is already pending returns WAIT rather than UPGRADE. State which existing assertion must be strengthened and why the current design is nevertheless not wrong.

4. DV task. Write the coverage cross for the cache lookup, then explain why a random address stream cannot close it, and which two points must be directed.

5. Debug task. An accelerator intermittently reads a value it previously wrote, but one revision old. Give your investigation order and the single counter pair that distinguishes an eviction bug from a lookup bug.

6. Design review. A colleague proposes dropping the dirty bit because "the accelerator writes through, so memory is always current." Give the strongest version of that argument, then name what it costs and the one workload that breaks it.

24. Summary

CXL.cache is a loan, and the terms are the module.

• The sourced contract is that CXL.cache "supports device caching of host memory with host processor orchestrating the coherency management" — the memory stays host-owned, and the host coordinates.
• A cached copy is not a private copy. It is state the system has an opinion about, for as long as it is held.
• A device must remember tag, state, dirty and pending. The last two are the ones people omit, and they are the ones that make the copy answerable.
• A tag match is not a hit. Invalidation changes state and leaves the tag alone.
• A coherent lookup has three answers plus an upgrade: hit, miss, wait — and present-but-not-writable, which is none of them.
• One line, one transaction. Different lines proceed concurrently; the same line serialises.
• An entry is released at completion, not at the response.
• The device may drop permission freely and gain it never — and may not drop a dirty line at all, because that is not permission, it is data.
• Coherence costs three bits per line and most of the complexity. In the illustrative cache, metadata is 7.42% of the array and the coherence-specific part is 0.59%.
• Verification lesson: a checker that cannot fire is documentation. Two of this chapter's fifteen mutations survived on exactly that.

Chapter 8.2 takes the first real consequence: when the device needs a line, memory may not hold the current value — so the device cannot simply read it.