Device Cache Access

Chapter 8.2 followed a request outward — the device wanting a line and needing to find its authoritative copy. This chapter reverses the direction. Coherence activity arrives at a line the device is already holding, and the device has to answer.

1. First, What This Chapter Is Not

The curriculum names this chapter "how the host accesses a coherent device cache", and that phrase is easy to read in a way that is architecturally false.

That distinction also tells you what direction the data flows. When the host needs a line the device holds modified, the value moves device to host — which looks superficially like the host "reading the device cache", and is in fact the device discharging an obligation it took on when it cached the line.

2. The One-Sentence Model

A cached copy is a promise to answer coherence questions later. Caching is not a read that finished; it is a commitment that stays open for as long as the line is held — so the device must remain able to respond, must supply the value before it gives up a modified line, and must not let its own housekeeping quietly break that promise.

Call it the standing promise. The device chose to make it, the moment it decided to cache.

3. What This Chapter Owns

4. The Exchange, From the Device's Side

Architectural. The arrows are obligations, not named messages.

Three things to read off it.

The value is supplied before the copy is dropped. Those two arrows are in that order for a reason, and §8 shows what happens when they are swapped.

"Not held" is a real answer, and a cheap one. Most inbound actions on most devices find nothing, and that path must be fast — a device that treats every action as expensive spends its cache port on non-answers.

The accelerator is stalled during the first exchange. That arrow is the honest cost of coherence, and §12 measures who wins the port.

5. Teaching-model boundary

6. RTL 1 — Stall an Action, Never Drop One

  // Ready is the whole contract: refuse to accept, never accept and discard.
  assign act_ready = DROP_WHEN_FULL ? 1'b1 : !full;

=== EXP1: a coherence action may be stalled, never dropped ===
  6 actions into depth 4 : correct level=4 in=4 stalled=2 dropped=0
  the two it could not take were stalled, not lost   : ok
  ready deasserted: back-pressure is the mechanism    : ok
  drop-when-full variant : in=4 dropped=1
  the dropping variant lost actions silently         : ok

A device may say "not yet". It may not say "yes" and then discard. The distinction is the ready signal, and it is the entire safety property of this module.

Why dropping is unrecoverable is worth being precise about. The system believes the device holds a copy of that line. An action that is dropped never gets answered, so that belief is never corrected — and there is no timeout on the device side that would notice, because from the device's point of view nothing happened at all. The system waits, or worse, proceeds on a stale assumption.

Compare that with a dropped request in 8.2: the device is the one waiting, so it has a timer and can recover. Here the device is the one being waited on, and only back-pressure keeps the conversation alive.

7. RTL 2 — The Obligation Is a Function of State

  // The obligation is a function of state, not of what the accelerator wants.
  assign owes_data    = (st_m[rd_idx] != 2'd0) && dty_m[rd_idx];
  assign owes_nothing = (st_m[rd_idx] == 2'd0);

=== EXP2: the obligation is a function of state ===
  not held      : state=0 owes_data=0 owes_nothing=1
  a line we do not hold owes nothing                 : ok
  held clean    : state=1 owes_data=0
  a clean copy owes state, not data                  : ok
  held MODIFIED : state=2 dirty=1 owes_data=1
  a modified copy owes the VALUE                     : ok

Three states, three different debts, and the comment on that first line is the one to remember: the obligation depends on what the device holds, not on what the accelerator still wants. An accelerator that has finished with a line changes nothing about what the system is owed for it.

8. RTL 3 — Supply the Value, Then Drop the Line

  // The correct order: invalidate only once nothing is owed.
  assign do_invalidate  = busy_q && (INVALIDATE_FIRST ? 1'b1 : !wb_pending_q);
  assign rsp_valid      = busy_q && (INVALIDATE_FIRST ? 1'b1 : !wb_pending_q);

=== EXP3: supply the value, THEN drop the line ===
  modified line : correct needs_wb=1 invalidate=0 rsp=0 | first-variant invalidate=1
  the correct handler waits for the value            : ok
  the broken variant dropped it immediately          : ok
  after capture : invalidate=1 rsp=1 dirty-responses=0
  value captured first, then the line was dropped    : ok
  the broken variant destroyed the only copy         : ok

rsp_valid carries the same condition as do_invalidate, and that is deliberate. Answering early is as dangerous as dropping early: the response tells the system it may proceed, and if the modified value has not yet been captured, whatever proceeds does so against a line whose only current copy is about to vanish.

Two separate errors exist for the two separate mistakes:

          // Dropping the line while its value was still owed loses it.
          if (wb_pending_q) data_lost_err <= 1'b1;
        // Answering before the value is safe tells the system it may proceed.
        if (rsp_valid && wb_pending_q) rsp_before_wb_err <= 1'b1;

9. Waveform — What a Modified Line Costs

Read busy across the two actions. The modified line holds the handler for four cycles; the clean line for one. The whole difference is the writeback obligation — and during those four cycles the cache port is unavailable to the accelerator, which is the cost §12 quantifies.

Notice also that invalidate and response rise in the same cycle, and that both wait for writeback_owed to fall. Three signals, one ordering rule, and every one of the three errors in this chapter's Debug Lab is that rule broken in a different place.

10. RTL 4 — The Evictor Does Not Know About Coherence

This is the path most likely to skip the protocol entirely, because the accelerator's reason for evicting — "I need this way for something else" — has nothing to do with coherence.

  // Two independent reasons to refuse, and they are not the same reason.
  assign evict_ok        = evict_req && (IGNORE_BUSY ? 1'b1 : !line_busy);
  assign needs_writeback = evict_ok && line_dirty && (line_state != 2'd0);

=== EXP5: the accelerator's evictor does not know about coherence ===
  evict a busy line : correct ok=0 | ignore-busy ok=1
  eviction was blocked while coherence held the line : ok
  the broken variant evicted it anyway               : ok
  evict a settled dirty line : ok=1 needs_wb=1
  a settled line may be evicted                      : ok
  and a settled dirty eviction still owes a writeback: ok

Two independent obligations, and a design can satisfy one while breaking the other. A busy line may not be evicted at all — a coherence operation holds a reference the tag array knows nothing about. A settled dirty line may be evicted, and still owes its value on the way out.

This is 8.1's invariant reappearing from the other direction, and it is worth stating as a rule: the device cannot discard coherence-relevant state merely because the accelerator no longer needs the line. The accelerator's opinion is not an input to the obligation.

11. RTL 5 — One Accepted Action, One Response

=== EXP6: one accepted action, one response ===
  2 accepted : owed=2
  each accepted action owes exactly one response      : ok
  responding cleared exactly one obligation           : ok

The conservation law of this chapter. Accepting an action creates a debt; responding discharges exactly one. A response for an action nobody accepted must clear nothing, and the abuse instance proves the checker fires:

  abuse instance flagged a response nobody owed      : ok
  and it cleared nothing: an unowed id is void       : ok

12. RTL 6 — Coherence Must Not Starve Behind the Accelerator

The accelerator and the coherence path share one tag/state port. The scheduling policy is therefore a correctness decision, not a performance one — because the system is waiting on the coherence side, and a device that stops answering is a device that has broken its promise.

  // Coherence is promoted once it has waited long enough. Without this the
  // accelerator can hold the port indefinitely.
  assign promote     = (coh_wait_q >= COH_AGE_LIMIT[7:0]);
  assign grant_coh   = coh_req && (ACCEL_ALWAYS_WINS ? !accel_req
                                                     : (!accel_req || promote));

Twenty-four cycles of full contention, both requesters asking every cycle:

=== EXP7: coherence must not starve behind the accelerator ===
  24 cycles of contention : fair accel=20 coh=4 max_coh_wait=4
                          : strict accel=24 coh=0 max_coh_wait=24
  the fair scheduler served coherence                : ok
  strict accelerator priority served it never       : ok
  worst coherence wait was exactly the age limit, 4  : ok
  under strict priority it waited all 24 cycles      : ok

Strict accelerator priority served coherence exactly zero times in twenty-four cycles. Not "rarely" — never. And the accelerator's own throughput barely changed: 24 versus 20 grants, a 17% gain bought at the price of the device no longer functioning as a coherent participant.

The aging policy bounds the worst case at exactly the age limit, which is the property worth asserting: not "coherence got some service" but "coherence waited no longer than the number we chose". That is a bound a design review can argue about.

13. The Inbound Path

The eviction gate is the block that surprises people, because it does not sit on the inbound path at all — it sits on the accelerator's path, protecting inbound work from local housekeeping. That placement is the architectural point of §10: the threat to an in-flight coherence operation comes from the device's own replacement policy, not from the coherence side.

14. Assertions

Icarus Verilog 13.0 does not support concurrent SVA here, so each property is synthesisable checker logic verified in simulation.

Safety

Liveness

The third is the one the strict-priority variant violates, and it is a liveness failure specifically: nothing unsafe happens, the device simply never answers.

Performance goals

15. Mutation Testing

Eighteen mutations. Eighteen killed.

The first run scored 14 of 18, and one survivor was a defect in my assertion rather than my stimulus.

Max coherence wait lagged by one and survived, because I had asserted a bound — "waited no more than 6" — when I could assert an exact value. The measured worst wait is exactly the age limit, 4, so the correct assertion is equality:

    if (gmw !== 8'd4) begin
      $display("FAIL max coherence wait=%0d expected exactly 4 (the age limit)", gmw);

A bound is a weaker assertion than an equality, and mutation testing is what tells you that you settled for one. If the design has a computable exact answer, assert the exact answer.

The other three were the familiar shapes: two needed instances reserved for illegal stimulus (busy-but-absent, and a response nobody owed), and one checker was unreachable until a variant made hits exceed counted actions.

16. Verification Plan

The coverage cross is line state against inbound action kind against accelerator activity: not-held / clean / modified, crossed with downgrade / invalidate / fetch-data, crossed with accelerator idle / contending. The third axis is the one that produced the arbitration finding, and a testbench that exercises coherence with an idle accelerator will never see it.

17. Silicon Observability

The pair that answers the most common escalation is dirty responses against inbound actions. A device supplying data on most inbound actions is being used as a cache of modified data by the workload, which is expensive and often unintended; a device that almost never supplies data is holding mostly clean lines and its coherence cost is dominated by lookups, not writebacks. Those two situations look identical in an aggregate latency number and have completely different fixes.

18. Debug Lab

assign act_ready = 1'b1;               // always accept
if (act_valid && !full) store(...);    // ...but only store when there is room

  drop-when-full variant : in=4 dropped=1
  the dropping variant lost actions silently         : ok

assign act_ready = !full;              // refuse instead of discarding
assign do_push   = act_valid && act_ready;

// Coherence wants the line — give it up.
state_m[idx] <= INVALID;
send_response();

  modified line : correct needs_wb=1 invalidate=0 rsp=0 | first-variant invalidate=1
  the broken variant destroyed the only copy         : ok

assign do_invalidate = busy_q && !wb_pending_q;   // nothing owed
assign rsp_valid     = busy_q && !wb_pending_q;   // same condition
if (do_invalidate && wb_pending_q) data_lost_err <= 1'b1;

if (start) send_response();          // answer immediately, write back later

  after capture : invalidate=1 rsp=1 dirty-responses=0
  value captured first, then the line was dropped    : ok

assign rsp_valid = busy_q && !wb_pending_q;
if (rsp_valid && wb_pending_q) rsp_before_wb_err <= 1'b1;

// Replacement policy picked this way.
assign evict_ok = evict_req;

  evict a busy line : correct ok=0 | ignore-busy ok=1
  the broken variant flagged evicted-while-busy      : ok

assign evict_ok = evict_req && !line_busy;   // gate on the live operation

if (evict_ok) state_m[idx] <= INVALID;   // no writeback consideration

  evict a settled dirty line : ok=1 needs_wb=1
  and a settled dirty eviction still owes a writeback: ok

assign needs_writeback = evict_ok && line_dirty && (line_state != INVALID);
// and the line may not go INVALID until the writeback is accepted

assign grant_coh   = coh_req && !accel_req;   // accelerator always wins
assign grant_accel = accel_req;

  24 cycles of contention : fair accel=20 coh=4 max_coh_wait=4
                          : strict accel=24 coh=0 max_coh_wait=24

assign promote   = (coh_wait_q >= COH_AGE_LIMIT);
assign grant_coh = coh_req && (!accel_req || promote);

if (action_in) n_hit_q <= n_hit_q + 1;   // every action counted as a hit

  actions=20 hits=13 misses=7 supplied=3 invalidated=5 stalls=4
  every inbound action landed in exactly one class   : ok

if (action_hit)  n_hit_q  <= n_hit_q  + 1;
if (action_miss) n_miss_q <= n_miss_q + 1;
if (n_hit_q + n_miss_q > n_action_q) classify_err <= 1'b1;

19. Design Review

1. Can an inbound coherence action ever be dropped? If ready and "stored" can disagree, yes.
2. Is the response gated on the same condition as the invalidate? Answering early has the same effect as dropping early.
3. What blocks eviction of a line under a live coherence operation?
4. Does a dirty eviction with no coherence traffic still write back?
5. Who wins the cache port under sustained contention, and what bounds the loser's wait?
6. Is the coherence wait bound asserted as an equality or a bound?
7. Does the hit counter count arrivals or outcomes?
8. Which counter tells you the device is holding mostly modified data?
9. What happens to outstanding obligations on reset?

20. How This Appears in Real Engineering

Architecture. The shared cache port and its arbitration policy is the decision that determines whether the device is a good coherent citizen. It is usually made early, for area reasons, and revisited late for correctness reasons.

RTL. The three-signal ordering rule — writeback, invalidate, response — is small and touched by several paths. Centralising it in one handler is what stops each path getting it slightly wrong.

DV. The eviction-under-coherence race and the sustained-contention case are both directed tests. Neither is reachable by random traffic, and both find serious bugs.

Post-silicon. Dirty-response rate against inbound action rate is the number that tells an operator whether the workload is using the device as intended.

Software. Placement again: a workload where the accelerator writes lines the CPU then reads generates a supply-the-value action every time, which is the most expensive inbound path there is.

21. Common Misconceptions

22. Interview Reasoning

23. Exercises

1. Analysis. A device reports inbound_actions=50000, hits=140, dirty_responses=138, and the accelerator stalls heavily. Nothing is inconsistent. Explain what the workload is doing and name the one change most likely to help.

2. Design. Add a downgrade action that reduces a modified line to clean rather than invalidating it. State what the device still owes, what it keeps, and which existing assertion must be weakened.

3. RTL task. Extend evict_gate so a blocked eviction is retried automatically rather than dropped. State the new liveness property and the counter that would show the retry loop is not making progress.

4. DV task. Write the coverage cross for the inbound path, then explain why the accelerator-activity axis is the one most often omitted and what it hides.

5. Debug task. Host threads touching shared lines stall for long periods while the accelerator reports excellent throughput and no errors. Give your investigation order and the single counter that identifies the cause.

6. Design review. A colleague proposes dropping inbound actions when the queue is full "because the host will retry". Give the strongest version of that argument, then name exactly what breaks and why the device cannot detect it.

24. Summary

A cached copy is a standing promise, and this chapter is what keeping it costs.

• The host does not read the device's cache. The host's operation concerns a line; the device is a holder and its job is to answer.
• An inbound action may be stalled but never dropped — the device is the one being waited on, so back-pressure is the only safe relief.
• The obligation is a function of state: nothing if not held, state if clean, the value if modified.
• Supply, then drop, then answer — and the response is gated on the same condition as the invalidate, because answering early has the same effect as dropping early.
• Eviction is not local housekeeping. A busy line may not be evicted at all; a settled dirty line may be, and still owes its value.
• The accelerator finishing changes nothing. The obligation depends on the line's state, not on its usefulness.
• Measured: a modified line costs four cycles of obligation against one for a clean line.
• Measured: strict accelerator priority served coherence zero times in twenty-four cycles, for a 17% accelerator gain. Aging bounds the worst wait at exactly the age limit.
• Verification lesson: a bound is a weaker assertion than an equality. A mutation survived because I asserted "no more than six" where the design has an exact answer of four.

Chapter 8.4 puts this machinery inside a real accelerator and asks what it does to the compute pipeline.