Cache-Interaction Flows

14.1 through 14.3 treated the agents as peers.

They are not. The host manages coherency, and that single asymmetry decides what a device cache may do on its own authority, what it has to ask for, and — most counter-intuitively — why memory physically attached to the device is still not the device's to cache without permission.

1. The Engineering Problem — An Asymmetry That Is Not A Design Choice

Two caches, one link. What makes this different from two caches on one die is not the latency. It is that one of them is in charge.

The list of what a device may do alone is short, and both errors are expensive. Too little autonomy makes every eviction a round trip. Too much loses data. Section 5 enumerates the list and measures what happens at each end.

A back-invalidation is not a snoop. A snoop asks; a back-invalidation tells. But the device may be mid-transaction on that line, and forcing immediate compliance destroys whatever it was doing. Waiting is correct and is bounded by the device's own transaction rather than by the host.

Physical attachment says nothing about coherency management. Device-attached memory is nearer to the device and still managed by the host. A device that treats attachment as ownership caches a line the host may already hold, and neither side knows about the other's copy.

And messages cross. The device evicts a line at the same moment the host snoops it. Both messages are legitimate, both are in flight, and the eviction is the answer to the snoop — a host that waits for a separate one waits forever.

2. The One-Sentence Model

One side is in charge, and the other has a short list. A device may hit and drop clean lines alone; everything that changes what the host believes requires asking, including for memory the device is physically holding.

Call it ask unless it is on the list. Every defect in this chapter is an action taken without asking, or an asking that was forced when it should have waited.

3. What This Chapter Owns

Deferred:

13.5 owns the boundary as a translation problem; this chapter owns it as an authority problem. That chapter asked how two vocabularies map; this one asks who has to ask whom.

4. Teaching-Model Boundary

Every model below is a teaching model, compiled and simulated with Icarus Verilog 13.0, checked by a testbench whose oracle is structurally different from the design.

What these models are not: a device controller. There is no accelerator, no DMA engine, no address translation, no link layer and no power management. A real device cache sits behind a streaming engine whose access pattern this chapter does not model.

The conventions from Module 13 and 14.1 carry over. Every checker tests cond !== 1'b1. Unreachable monitors get an extra build of the same source. Comparisons are one source under a parameter, instantiated twice, driven from one stimulus stream. Every displayed value is a captured signal, and every combinational sample is preceded by a settle.

This chapter uses seven parameterised twin builds — SILENT_DIRTY_DROP, FORCE_IMMEDIATE, IGNORE_CROSS, DEVICE_ASSUMES_LOCAL, FAULT_INJECT on the direction gate, SHARED on the queues, and NO_CROSS_CHECK on the pair.

5. RTL 1 — What A Device May Do Alone

The list is three items long:

  // A hit with permission and a clean drop need nobody. Everything else
  // changes what the host believes, so the host has to be told.
  assign base_auto = ((action == A_RD)    && has_perm)
                  || ((action == A_WR)    && has_perm)
                  ||  (action == A_DROPC);
  assign data_lost_err = act && (action == A_DROPD) && autonomous;

Six actions were driven with permission and the same six without, against an oracle that is an explicit list of integers. Measured:

  autonomy : alone=4 must ask=8 (of 12 driven) | dirty drop lost correct=0 silent build=1

Four of twelve. A read hit and a write hit, each only with permission, and a clean drop with or without. Everything else — upgrades, fetches, dirty evictions, and hits without permission — changes what the host believes about the line and therefore requires a message.

The clean drop is the interesting entry. It is autonomous regardless of permission, because dropping a clean copy removes nothing the system needs: memory or another agent still has the value. That is 13.3's silent-eviction rule, and it is the one place the device gets something for free.

The SILENT_DIRTY_DROP build adds the dirty eviction to the list, and data_lost_err fired on the first one. That is not a corner case — a device cache under streaming pressure evicts constantly, and making dirty evictions autonomous turns every one of them into a lost line.

6. RTL 2 — A Back-Invalidation Is Not A Snoop

The host needs a line back. It is not asking:

  // FORCE_IMMEDIATE makes the device yield even mid-transaction, which tears
  // whatever it was doing. Waiting is correct and bounded by the device's own
  // transaction, not by the host.
  assign dev_must_yield = pend_q && ((FORCE_IMMEDIATE != 0) || !dev_busy);
  assign torn_txn_err   = dev_must_yield && dev_busy;

Measured:

  back-inv : yield while busy correct=0 forced=0 torn=1 (count 4) | worst wait latched=5

The correct build did not make the device yield while it had a transaction on the line. The FORCE_IMMEDIATE build did, and tore the device's transaction — four times over the run, because the device was busy for four consecutive cycles.

The distinction matters because the two designs look identical when the device happens to be idle. A back-invalidation arriving at an idle device is served immediately in both builds; only one that lands mid-transaction separates them, and that is precisely the case a directed test is least likely to construct.

The wait is latched at 5 cycles and a shorter one afterwards did not reduce it. A second request arriving while one was outstanding did not open a second back-invalidation or reset the wait — the host asking twice does not make the device faster.

7. RTL 3 — An Eviction Crossing A Snoop

Both messages are in flight and they pass each other:

  // When the two cross, the eviction IS the answer to the snoop. A host that
  // still waits for a separate acknowledgement waits forever.
  assign host_expects_ack = sn_q && !(crossed && (IGNORE_CROSS == 0));
  assign lost_ack_err     = crossed && host_expects_ack;
  // ...and a host that treats both as separate events counts one line twice.
  assign double_count_err = crossed && (IGNORE_CROSS != 0) && evict_arrived
                            && snoop_arrived;

Measured:

  crossing : crossed=1 | lost ack correct=0 ignore-cross=0 double count=1

The eviction answers the snoop. The device gave the line up before the snoop arrived; there is nothing left to invalidate and nothing further to say. A host that does not recognise the crossing waits for an acknowledgement that will never come, and one that treats the two as independent events counts one line twice in its sharer accounting.

Both failures come from the same misreading: treating two messages about one line as two facts rather than one exchange. The IGNORE_CROSS build produced both.

8. Waveform — The Host Revokes Before It Writes

Transcribed from the printed cycle trace of the assembled pair in section 15.

The unchk_er row never returns to zero. A cross-boundary single-writer violation does not resolve itself — both caches continue to believe they hold the line correctly, and nothing in either cache's own checking can see it.

9. RTL 4 — A Device Cache Is Small

Capacity evictions are the common case rather than the exception:

  // Filling over a dirty line whose eviction has not completed destroys it.
  assign overwrite_dirty_err = fill && v_q[fill_way] && d_q[fill_way] && !pend_q;

Every way was filled clean and then dirty, against a per-way integer oracle. Measured:

  cache    : fills=16 evicts=12 dirty=2 | victim dirty=1 overwrite reported=1 pending=1

Twelve evictions from sixteen fills. That ratio is what a small cache looks like under pressure, and it is why the dirty-eviction path is on the hot path for a device rather than being an occasional event.

The !pend_q term is the subtle one. Filling over a dirty way whose eviction is already outstanding is not a destroy — the line is on its way to the host and the way is legitimately being reused. Without that term the monitor fires on correct behaviour, and an alarm that fires on correct behaviour is an alarm that gets disabled. The bench drives exactly that case and confirms it is not reported.

10. RTL 5 — Attached Is Not Owned

The single most counter-intuitive fact in this module:

  // Being physically attached says nothing about who manages coherency for it.
  // DEVICE_ASSUMES_LOCAL treats attachment as ownership, which is exactly the
  // mistake the CXL.mem direction split exists to prevent.
  assign may_cache = dev_access
                     && (host_granted
                         || ((DEVICE_ASSUMES_LOCAL != 0) && is_device_attached));
  assign stale_local_err = bypassed_host_err && host_holds_copy;

Measured:

  attached : must ask=1 bypassed correct=0 assumes-local=1 stale local copy=1

The device had to ask the host for permission to cache memory bolted to its own board. The DEVICE_ASSUMES_LOCAL build cached it without asking, and stale_local_err fired because the host was holding a copy the device knew nothing about.

The reasoning is entirely about who manages coherency, not about who is nearer. The host manages coherency for device-attached memory; therefore the host knows who holds copies; therefore a device that caches without telling the host has created a copy outside the only record of copies that exists. Physical proximity is irrelevant to that argument, which is why the mistake is so natural.

11. RTL 6 — What The Device Sees While An Invalidation Is Outstanding

14.2 measured the exposure window from the writer's side. This is the same window from the reader's:

  // While the invalidation is outstanding the device's copy is still valid and
  // still old. Both facts are true at once, and neither is an error.
  assign dev_sees_old = dev_reads &&  win_q;
  assign dev_sees_new = dev_reads && !win_q && written_q;

Measured:

  visible  : old-value reads=4 new-value reads=1 window=4 peak latched=4
             read before any write counted as new=0

Four reads of the old value, every one of them correct. The device holds a valid copy and the host's write has not completed. Neither side is doing anything wrong, and a reader on the device sees a value the host believes it has already replaced.

The written_q term stops a read before any write from being counted as seeing a new value — a small guard that prevents the counter from claiming visibility of something that never happened. Without it, the first read of the run is attributed to a write that has not occurred.

12. RTL 7 — Which Protocol Carries Which Access

Four combinations of initiator and target:

  assign dev_caches_host_mem = acc &&  initiator_is_device && !target_is_device_attached;
  assign host_reads_dev_mem  = acc && !initiator_is_device &&  target_is_device_attached;
  assign host_local          = acc && !initiator_is_device && !target_is_device_attached;
  assign dev_local           = acc &&  initiator_is_device &&  target_is_device_attached
                               && (FAULT_INJECT == 0);

Measured against a literal four-row truth table:

  direction: crossing=2 local=2 | faulty build crosses a local access=1 multi=1

Two of the four cross the link. A device caching host memory and a host accessing device-attached memory both traverse the boundary; a host accessing host memory and a device accessing its own attached memory do not.

The FAULT_INJECT build sends a device-local access across the link — a round trip for data the device is physically holding — and classifies it as nothing at all, which the one-hot monitor catches. The whole combination space was swept on both builds, and the faulty one misclassifies exactly the device-local case and no other.

13. RTL 8 — The Queues Between The Two Sides

Requests cross in both directions at once:

  // With separate queues a full device-to-host path never blocks the host's
  // back-invalidation. SHARED couples them, which is where the cycle comes from.
  assign h2d_ready = (h_q < DEPTH[3:0]) && ((SHARED == 0) || (d_q < DEPTH[3:0]));
  assign deadlock_err = h2d_in && !h2d_ready && d2h_in && !d2h_ready;

Measured:

  queues   : separate d2h_ready=1 shared=0 | deadlock separate=0 shared=1

This is 13.5's channel-dependency result in its concrete form: the host's back-invalidation blocked behind the device's own requests. The device cannot drain its request queue until the host services it; the host cannot service it because its back-invalidation cannot be issued; and neither protocol's specification describes the state.

The queues were also drained and confirmed to recover, and a fifth push into a full queue was confirmed not to overflow. A boundary that overflows rather than backpressuring converts a deadlock into a data loss.

14. RTL 9 — What The Boundary Costs Each Side

  // Back-invalidations per hundred accesses: the cost the device pays for
  // caching host memory at all, which the hit rate alone does not show.
  assign bi_per_hundred = (total == 17'd0) ? 8'd0
                        : (weighted_bi / {15'd0, total});

Measured:

  cost     : hit rate=80% mean miss=100 mean back-inval=30 | 10 back-invals per 100

An 80% hit rate and ten back-invalidations per hundred accesses. The hit rate alone says the device cache is working well. The back-invalidation rate says the host is reclaiming lines at one tenth the access rate, and every one of those is a line the device fetched, used briefly, and gave back.

Those two numbers together are what decides whether the device cache is earning its area. A high hit rate with a high back-invalidation rate is a cache that is being churned by the host rather than one that is serving the device — and the fix is in the allocation policy or the working-set size, not in the cache.

15. RTL 10 — Two Caches, One Line

  // The host manages coherency, so it may revoke the device's copy. The device
  // cannot revoke the host's -- it asks, and the host does the revoking.
  assign revoke_dev  = host_wants_write && dr_q && (NO_CROSS_CHECK == 0);
  assign revoke_host = dev_wants_write  && hr_q && (NO_CROSS_CHECK == 0);
  assign cross_swmr_err = (hw_q && dw_q) || (hw_q && dr_q) || (dw_q && hr_q);

Measured:

  pair     : revokes=2 grants host=1 dev=1 | swmr checked=0 unchecked=1
             device read admitted against a host writer=0

Revocations crossed the boundary in both directions, and the writer moved from the host to the device with the invariant intact throughout. A device read arriving while the host was writing was not admitted.

The asymmetry is visible in the code: revoke_host exists, so the host's copy is revoked when the device wants to write — but the device does not perform that revocation. It asks, and the host does it, because the host is the agent that manages coherency. The signal names describe what happens to each cache, not who issued the message.

The NO_CROSS_CHECK build granted immediately and produced a writer on one side and a reader on the other — a violation that neither cache's own checking can see, because each is internally consistent.

16. Quantitative Reasoning

The autonomy list is worth a round trip per entry. Four of twelve driven actions were autonomous. Each of the other eight is a message, and on a link with a 30-cycle round trip that is 30 cycles per action. Making the clean drop autonomous — the one free entry on the list — saves a round trip on every clean eviction, which at the measured 12-evictions-per-16-fills rate is most of them.

A device cache is eviction-dominated. Measured at 12 evictions from 16 fills. That ratio is a property of a small cache behind a streaming engine, and it means the eviction path, not the fill path, is what determines the boundary's message rate.

The back-invalidation cost is bounded by the device, not the host. Measured worst wait: 5 cycles, set by the device's own transaction length. Forcing immediate compliance would remove that wait and destroy the transaction — trading a bounded latency for unbounded rework.

The back-invalidation rate is the number the hit rate hides. At an 80% hit rate and 10 back-invalidations per hundred accesses, the device is returning lines at one tenth the rate it is using them. A cache with a 95% hit rate and a 40% back-invalidation rate is being churned; the hit rate alone cannot distinguish the two.

The exposure window is a round trip, seen from the reader. Measured at 4 cycles with 4 reads of the old value, all correct. On a real link this is the full invalidation round trip, and the number of stale-but-correct reads inside it scales directly with the device's read rate.

17. Assertions

Presented as SystemVerilog and executed as procedural checkers — see section 19.

A device never drops a dirty line on its own authority.

property p_no_silent_dirty_drop;
  @(posedge clk) disable iff (!rst_n)
    (act && action == A_DROPD) |-> must_ask;
endproperty

Every action is either autonomous or asks, never both.

property p_one_outcome;
  @(posedge clk) disable iff (!rst_n)  act |-> (autonomous ^ must_ask);
endproperty

A back-invalidation never tears a device transaction.

property p_no_torn_txn;
  @(posedge clk) disable iff (!rst_n)  dev_must_yield |-> !dev_busy;
endproperty

A crossed eviction answers the snoop.

property p_crossing_answers;
  @(posedge clk) disable iff (!rst_n)  crossed |-> !host_expects_ack;
endproperty

A dirty way is never overwritten without a pending eviction.

property p_no_dirty_overwrite;
  @(posedge clk) disable iff (!rst_n)
    (fill && victim_dirty) |-> evict_pending;
endproperty

The device never caches without the host's permission.

property p_no_bypass;
  @(posedge clk) disable iff (!rst_n)  may_cache |-> host_granted;
endproperty

Attachment does not imply permission.

property p_attached_still_asks;
  @(posedge clk) disable iff (!rst_n)
    (dev_access && is_device_attached && !host_granted) |-> must_request;
endproperty

Exactly one direction class per access.

property p_one_direction;
  @(posedge clk) disable iff (!rst_n)
    acc |-> $countones({dev_caches_host_mem, host_reads_dev_mem,
                        host_local, dev_local}) == 1;
endproperty

A full queue in one direction never blocks the other.

property p_no_boundary_cycle;
  @(posedge clk) disable iff (!rst_n)  d2h_in |-> ##[0:$] d2h_ready;
endproperty

Never a writer on one side and a reader on the other.

property p_cross_swmr;
  @(posedge clk) disable iff (!rst_n)
    !((host_writer && dev_reader) || (dev_writer && host_reader));
endproperty

A device read is not admitted against a host writer.

property p_no_read_against_writer;
  @(posedge clk) disable iff (!rst_n)  host_writer |-> !dev_reader;
endproperty

A granted writer holds a readable copy.

property p_writer_has_copy;
  @(posedge clk) disable iff (!rst_n)  host_writer |-> host_reader;
endproperty

18. Mutation Testing

79 mutations were injected into the ten models, one at a time, each a single-line change a competent engineer could plausibly write. Every one must make the testbench print RESULT: FAIL.

Representative mutations, all killed:

Nine mutations survived the first run. None was patched away.

Six stimulus gaps. The bench never issued a second back-invalidation while one was outstanding, never followed the long wait with a shorter one that still ticked, never filled over a dirty way whose eviction was already outstanding, never read before any write had occurred, never drained the queues, and never swept the faulty direction gate across the whole combination space.

Two unobserved outputs. The forced-yield count on the faulty back-invalidation build, and the device-reader flag while the host was writing.

One delta-cycle defect in the bench itself, and it is worth its own paragraph. A newly added check sampled overwrite_dirty_err immediately after clearing fill, in the same delta. The continuous assignment had not propagated, so the check read the previous value and reported a destroy on four consecutive correct fills. This is the same class as the waveform defect in batch 013 — a combinational output read in the delta its driver changes — and it appeared here in a check rather than in a printed trace. A #1 settle fixed it.

A note on the autonomy sweep. The chapter's assertion count was initially below the three chapters before it. Rather than accept the taper, the autonomy table was swept a second time without permission — twelve driven actions instead of six — which is genuinely more coverage rather than more assertions: it is what proves that permission gates the two hit cases and does not gate the clean drop.

19. Verification Strategy

The oracle must not be the design. Each testbench models the same behaviour in a structurally different representation.

For device_autonomy the design is a boolean expression over an action encoding. The oracle is an explicit list written as integers, with no notion of an encoding at all:

  function integer o_auto(input integer a, input integer perm);
    begin
      o_auto = 0;
      if (a==0 && perm!=0) o_auto = 1;   // read hit
      if (a==1 && perm!=0) o_auto = 1;   // write hit
      if (a==2)            o_auto = 1;   // drop a clean line
    end
  endfunction

Note the third line has no permission term — that is the silent-eviction rule expressed as a reference rather than as an assertion.

For dev_cache_pressure the design holds two bit vectors. The oracle holds eight plain integers, one valid and one dirty per way, so a vector-indexing bug cannot appear identically in both.

For direction_gate the design is four boolean expressions. The oracle is a literal four-row truth table returning a single integer, so a design that asserts two classes cannot be reproduced by a reference that holds one.

Every displayed value is a captured signal, latched before any later event changes it.

Delta-cycle discipline, and this chapter is the reason to restate it: a combinational output sampled in the same delta as its driver changes returns the previous value, and doing that inside a check rather than a display produces a false failure rather than a false transcript. Every sample of a combinational output in these benches is preceded by a settle.

Coverage recorded: 139 assertion sites across three testbenches; the autonomy table swept twice, with and without permission; back-invalidations driven against a busy device, an idle one, and with a second request outstanding; an eviction crossing a snoop on both builds; every cache way filled clean and dirty against a per-way oracle, plus a fill over an already-pending eviction; device-attached access driven with and without permission; the visibility window driven with reads before, during and after; the direction table swept on both builds; the queues filled, blocked and drained; and the cache pair driven through a host grant, a device grant and a read against a writer.

20. Synthesis and Implementation Reality

The autonomy decision is combinational and on the device's access path. It is a small decode over the action and the permission bit, and it has to resolve before the device knows whether to issue a message. Getting it wrong in the permissive direction is not a timing problem, it is a data-loss problem.

The back-invalidation tracker is per line in flight, not per line. It holds a pending flag, the requester, and the age. The age counter feeds only the diagnostic; the yield decision reads the device's own busy signal, which is why the wait is bounded by the device rather than by a timer.

The device cache's dirty-eviction path is the hot path. At the measured 12-evictions-from-16-fills ratio, the eviction queue rather than the fill queue determines the boundary's message rate — which inverts the sizing intuition carried over from a host cache.

Separate queues in each direction are a physical commitment. As 13.5 established, coupling them cannot be fixed later by making one deeper, because the cycle is structural.

cross_swmr_err spans two caches and therefore lives at the boundary, not in either cache. Each cache's own checking is satisfied throughout the violation. That is what makes the monitor necessary and what makes it awkward: it needs visibility of both sides, which the boundary has and neither cache does.

Reset must leave the device holding nothing and the host granting nothing. A device cache that comes out of reset with valid lines has copies the host's directory does not know about — the same condition the assumes-local build produces, arriving at power-on rather than at runtime.

21. Silicon Observability

Five error signals belong in silicon. data_lost_err, torn_txn_err, overwrite_dirty_err, stale_local_err and cross_swmr_err all detect states from which no correct behaviour is possible, and each is a handful of gates over signals that already exist.

bi_per_hundred alongside the hit rate is the pair that judges a device cache. A high hit rate says the cache is serving the device; a high back-invalidation rate says the host is reclaiming lines as fast as the device can use them. Only both together distinguish a cache that is working from one that is churning.

n_bypassed is a design-time finding, not a runtime one. A non-zero value means the device is caching host-managed memory without permission, which is not a condition that arises from a transient — it means the autonomy logic is wrong, and it will be wrong on every access of that class.

22. Debug Lab

  autonomy : alone=4 must ask=8 (of 12 driven) | dirty drop lost correct=0 silent build=1

assign base_auto = ((action == A_RD) && has_perm)
                || ((action == A_WR) && has_perm)
                ||  (action == A_DROPC);          // clean only

  back-inv : yield while busy correct=0 forced=0 torn=1 (count 4) | worst wait latched=5

assign dev_must_yield = pend_q && !dev_busy;

  crossing : crossed=1 | lost ack correct=0 ignore-cross=0 double count=1

assign host_expects_ack = snoop_outstanding && !crossed;

  cache    : fills=16 evicts=12 dirty=2 | victim dirty=1 overwrite reported=1 pending=1

assign overwrite_dirty_err = fill && valid[way] && dirty[way] && !evict_pending;

  attached : must ask=1 bypassed correct=0 assumes-local=1 stale local copy=1

assign may_cache    = dev_access && host_granted;      // attachment is not a grant
assign must_request = dev_access && !host_granted;

  visible  : old-value reads=4 new-value reads=1 window=4 peak latched=4

if (len_q + 8'd1 > max_window) max_window <= len_q + 8'd1;

  direction: crossing=2 local=2 | faulty build crosses a local access=1 multi=1

assign dev_local = acc && initiator_is_device && target_is_device_attached;

  pair     : revokes=2 grants host=1 dev=1 | swmr checked=0 unchecked=1

assign revoke_dev = host_wants_write && dev_reader;
assign grant_host = host_wants_write && !dev_reader;

23. Design Review

What was built. Ten models: an autonomy gate with a silent-dirty-drop twin, a back-invalidation path with a force-immediate twin, an eviction/snoop crossing resolver with an ignore-cross twin, a small device cache with a per-way oracle, a host-managed-memory gate with an assumes-local twin, a visibility-window meter, a four-way direction gate with a fault-injection build, a two-direction queue with a shared twin, a cost model pairing the hit rate with the back-invalidation rate, and an assembled cache pair with an unchecked twin.

What was measured. Four of twelve driven actions autonomous, with the silent-drop build losing a line on the first dirty eviction. A back-invalidation that waited for a busy device against one that tore four transactions, with the worst wait latched at 5. An eviction answering a crossed snoop, against a build that both waited forever and double-counted. 12 evictions from 16 fills, with a destroy reported only when no eviction was pending. A device required to ask for memory attached to itself, against a build that cached it while the host held a copy. 4 reads of a superseded value, all correct. Two of four combinations crossing the link. A device-to-host message accepted with the host-to-device queue full, and blocked when shared. An 80% hit rate with 10 back-invalidations per hundred. Revocations crossing in both directions with the union invariant intact.

What would be different in production. A real device cache sits behind a streaming engine whose access pattern drives everything here. The autonomy decision is folded into the access pipeline rather than standing alone. The back-invalidation tracker is per outstanding reclaim. The direction gate is an address decode with real ranges. None of that changes the authority asymmetry; all of it multiplies where it can be got wrong.

The strongest argument against this design. Requiring the host's permission to cache device-attached memory adds a round trip to an access whose data is physically inches away, and on a device with a large attached memory that is a real cost on a real hot path. That argument is correct about the cost, and the answer is not to skip the request but to make it coarser: permission over a region rather than a line, refreshed rather than requested per access. That keeps the host as the single record of who holds what, which is the property the whole model rests on, while amortising the round trip. The model here is per access because per-access is the honest teaching case; region-granular permission is a Module 15 and 16 concern.

What would be built differently next time. The !pend_q term in the overwrite monitor was added after the model was written, when the bench drove a fill over an already-pending eviction and the monitor fired on correct behaviour. Writing the monitor before enumerating the legal cases is how alarms on legal traffic get created — and this chapter produced one, caught it, and the same discipline is what caught the dead monitor in 14.3.

24. How This Appears In Real Engineering

In a device architecture review, the question that determines the boundary's message rate is the eviction-to-fill ratio. Measured at 12 from 16, the eviction path is the hot path, which inverts the sizing intuition carried over from a host cache.

In driver and firmware work, the visibility window is the number that matters. Software that assumes a host write is immediately visible to the device has assumed something no protocol promised, and the latched peak is the only measurement of how wrong that assumption can be.

In bring-up, n_bypassed is the first counter to check on a device with attached memory. A non-zero value is not a transient — it means the autonomy logic is wrong for a whole access class.

In a performance investigation, the hit rate alone is misleading for a device cache. Pair it with the back-invalidation rate: 80% and 10-per-hundred is a working cache; 95% and 40-per-hundred is a cache being churned by the host.

In a verification plan review, ask whether the back-invalidation has been tested against a busy device. An idle device is served identically by a correct design and one that forces compliance, so the busy case is the only one that separates them.

In silicon debug, a violation spanning two caches is invisible to both. Each is internally consistent; only a monitor at the boundary can see it, and it has to be designed in because neither cache has the visibility to add it later.

25. Common Misconceptions

"A device cache is just a smaller host cache." It is eviction-dominated rather than fill-dominated — measured at 12 evictions from 16 fills — and it operates under an authority it does not hold.

"A device can evict its own lines." Only clean ones. Measured: the silent-drop build lost a line on the first dirty eviction, and a device under streaming pressure evicts constantly.

"Memory attached to the device belongs to the device." The host manages coherency for it. Measured: the assumes-local build cached it without asking while the host held a copy, and neither side knew about the other's.

"A back-invalidation is just a snoop that cannot be refused." It can be waited for. Forcing compliance destroys the device's own transaction, and the wait is bounded by that transaction rather than by the host.

"An eviction and a snoop crossing is a race that needs arbitration." It is not a race — the eviction is the answer. Measured: the ignore-cross build waited forever and counted the line twice, both from treating one exchange as two facts.

"A read served from a device copy during a host write is a coherency bug." It is correct. The device holds a valid line and the write has not completed. Measured: four such reads, none of them an error.

"A high device hit rate means the cache is working." Not on its own. Measured at 80% with 10 back-invalidations per hundred; a cache at 95% with 40 per hundred is being churned by the host and the hit rate cannot tell you.

"Each cache checking its own invariants is sufficient." Both were internally consistent throughout a cross-boundary violation. The union invariant lives at the boundary because that is the only place with visibility of both sides.

26. Interview Reasoning

27. Exercises

1. Calculation. A device cache sees 16 fills producing 12 evictions, of which 40% are dirty, on a link with a 30-cycle round trip. Compute the eviction message traffic per 1000 fills, then compute it again if the clean drop were not autonomous.

2. Analysis. A device reports a 95% hit rate and 40 back-invalidations per hundred accesses. State what that combination means, why the hit rate alone is misleading, and which two design parameters you would change first.

3. RTL task. Extend back_invalidate to support a bounded wait that reports rather than forces. State what the report must contain to be actionable, and why a forced yield is not an acceptable fallback.

4. Assertion task. Write the property proving a device never caches without the host's permission. Then explain why it passes trivially on a design that derives host_granted from the same signal that drives may_cache, and what independent source of the grant is required.

5. Design task. Add region-granular permission so a device can cache a range of its attached memory under one grant. State what must be revoked when the host needs part of that range back, and which of this chapter's monitors must change.

6. Testbench design. Design the stimulus that distinguishes a back-invalidation that waits from one that forces. Explain why every test with an idle device passes on both, and state the minimum stimulus that separates them.

7. Debug task. A device is caching host-managed memory and the host's directory shows no device copy. Give your investigation order, name the counter that identifies it, and explain why this is a design-time finding rather than a runtime transient.

8. Design review. A colleague proposes a shared queue at the boundary to save area, arguing that traffic in the two directions is bursty and rarely coincides. Give the strongest version of that argument, then the failure it enables, and why making the queue deeper does not fix it.

28. Summary

One side is in charge, and the other has a short list.

• Four of twelve driven actions are autonomous. Two hits with permission, and a clean drop with or without — the one entry the device gets for free.
• A dirty eviction is a message. The silent-drop build lost a line on the first one, and a device evicting 12 times per 16 fills does that constantly.
• A back-invalidation waits for a busy device. The forcing build tore four transactions in a five-cycle busy period; the worst legitimate wait was latched at 5.
• A crossed eviction answers the snoop. The build that missed the crossing both waited forever and counted the line twice — one misreading, two failures.
• A fill over a dirty way needs a pending eviction, and a fill over one that is already pending is correct. Flagging it would alarm on legal behaviour.
• Attached is not owned. The device had to ask the host for memory on its own board; the build that assumed otherwise cached it while the host held a copy.
• Four reads of a superseded value, all correct. The window is the invalidation round trip seen from the reader, and software that assumes otherwise assumed something no protocol promised.
• Two of four access combinations cross the link, and which two follows from who is accessing what rather than from any routing decision.
• Separate queues, or the boundary deadlocks. A device-to-host message was accepted with the host-to-device queue full, and blocked when the two were shared.
• 80% hit rate with 10 back-invalidations per hundred. The hit rate alone cannot tell a working cache from one being churned.
• The union invariant lives at the boundary. Both caches were internally consistent throughout a cross-boundary violation that neither could see.
• Verification: 139 assertion sites, 79 of 79 mutations killed, zero surviving. Nine first-run escapes were six stimulus gaps, two unobserved outputs, and one delta-cycle defect in the bench itself that produced a false failure on four correct fills.

Next: 14.5 State Transitions, which takes all four flows and asks which transitions actually fire, in what order relative to the messages, and what a flow that is interrupted half-way leaves behind.