Why Cache Coherency Exists

Module 1 · Chapter 1.1 · Cache Coherency Foundations

Project thread — this chapter starts the running example every later CHI lesson builds on: two CPUs (CPU0, CPU1), each with a private L1 cache, sharing one memory with one shared line A.

1. Learning Outcomes

By the end of this lesson you will be able to:

• Explain the cache-coherency contract — single-writer/multiple-reader, write propagation, and value consistency — as a per-address guarantee.
• Distinguish coherency (per-address agreement) from memory consistency and ordering (cross-address behaviour, Module 12).
• Trace the exact cycle where two private cache copies of one line diverge after a write.
• Identify a stale-read bug from its signature: two caches holding different values for the same address with nothing reconciling them.
• Explain why a software lock cannot fix stale cache copies and why the guarantee must live in hardware.

2. Key Terms

• Cache line — the unit of data a cache holds and tracks (typically 64 bytes); coherency operates per line.
• Private cache — a cache owned by one core (its L1); other cores cannot see its contents directly.
• Copy — a cached line is a copy of memory; two cores caching one address hold two copies.
• Stale data — a cached copy that no longer matches the latest write to that address.
• Coherency — the per-address hardware contract that keeps all copies of a line agreeing on the latest value.
• Write propagation — making a write visible to every other core that holds (or fetches) a copy.
• Visibility — when a write performed by one core can be observed by another.

3. Why Should I Learn This?

Every fast multi-core SoC has private caches. Every private cache is a copy. Every copy can go stale. Coherency is the mechanism that keeps those copies telling the same story — and CHI is how modern Arm SoCs implement it at scale.

4. Core Concept — copies, writes, and the coherency contract

Start from the running project: CPU0 and CPU1, each with a private L1, over one shared memory holding line A. Think of it as two colleagues each holding a photocopy of one shared document — fine until one edits their copy and the other keeps reading their now out-of-date sheet.

Where copies come from. A load of A on CPU0 misses in L1, fetches the line, and caches a copy. The same load on CPU1 does the same — now two private copies of A exist. So far harmless: both are read-only and identical.

Where it breaks — the write. CPU0 stores to A, updating its own L1 copy to the new value. CPU1 still holds the old copy; nothing told it anything changed. CPU1 loads A, hits in its L1, and reads the stale value. The bug is live.

The coherency contract (per address):

• Single writer / multiple readers. At any instant either one core may write the line, or many may read it — never both.
• Write propagation. A write eventually becomes visible to every other core (by updating or invalidating their copies).
• Value consistency. A read returns the value of the most recent write to that address, wherever that write currently lives.

Coherency is per-address. It is not ordering or consistency — how writes to different addresses are ordered across cores (Module 12). This chapter answers the sharper question: for one address, do all copies agree?

5. Engineering Diagram

6. Real Silicon Story — the counter that lost increments

A dual-core microcontroller ran a shared event counter. Both cores incremented count in shared memory under a lock. Under real dual-core load the counter was low — thousands of events, hundreds counted — yet single-core tests always passed. The counter lived in a cacheable region: CPU0 read count into its L1, incremented it, and wrote it back to its own L1 copy; CPU1 did the same against its own copy. Each core happily incremented a private, diverging copy of the same address, their writes never reached each other, and increments were silently lost.

The firmware team suspected the lock. The lock was fine. The problem was one layer lower: the two L1 caches were never kept coherent, so a write by one core was invisible to the other. The fix was not more locking — it was making the shared region coherent, so a write to count in one cache invalidates or updates the copy in the other. That mechanism, at SoC scale, is exactly what AMBA CHI provides.

7. Worked Example — a shared counter diverges

Trace the lost-increment bug at the value level, no protocol yet:

# Two private caches over one shared memory. Initial: A = 5 everywhere.
# (No coherency mechanism exists — each core caches A independently.)
 
Cycle  CPU0 action              L1(CPU0)   L1(CPU1)   Memory   Note
-----  -----------------------  ---------  ---------  -------  -----------------------------
  1    load  A  (miss -> fetch)  A=5        -          A=5      CPU0 caches copy of A
  2    load  A  (CPU1, miss)     A=5        A=5        A=5      CPU1 caches its own copy
  3    store A=6 (CPU0)          A=6        A=5        A=5      CPU0 updates ONLY its copy
  4    load  A  (CPU1, HIT)      A=6        A=5        A=5      CPU1 reads STALE 5  <-- BUG
  5    store A=7 (CPU1)          A=6        A=7        A=5      based on stale 5 -> lost update

CPU1's increment was computed from a stale read, so CPU0's write to A is silently lost. No lock fixes this — the caches themselves disagree. Coherency is the layer that makes cycle 4 return 6, not 5.

8. Protocol Transaction Walkthrough — the write that never propagated

Walk the exact sequence, framed the way CHI will later frame it (requester, the line, the missing snoop):

The single missing message — the snoop/invalidate from CPU0's write to CPU1's cache — is the entire coherency problem. Every mechanism in CHI exists to make that dashed arrow real, correct, and cheap.

9. Timing Diagram — cache states diverge over time

Track each core's cached value for line A cycle by cycle. Watch the two copies split apart at the write:

The visual signature of a coherency bug: two cached copies of the same address holding different values, with no event reconciling them.

10. RTL Demonstrator — a two-core stale-data model

This is a simplified behavioral demonstrator, not a coherency controller: two 1-entry private caches over a shared memory, wired to reproduce the stale read. A store updates only the local copy — the baseline the rest of the series makes coherent.

// chi_incoherent_demo.sv — two private 1-entry caches, one shared memory.
// Reproduces the stale-read bug: a store in core0 is invisible to core1.
// SIMPLIFIED behavioral model, NOT a coherency controller.
module chi_incoherent_demo #(parameter int DW = 8)(
  input  logic          clk, rst_n,
  input  logic          c0_wr, c0_rd, c1_wr, c1_rd,
  input  logic [DW-1:0] c0_wdata, c1_wdata,
  output logic [DW-1:0] c0_rdata, c1_rdata
);
  logic [DW-1:0] mem_A, c0_cache_A, c1_cache_A;
  logic          c0_valid, c1_valid;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      mem_A <= 8'd5;                                      // A = 5 everywhere at reset
      c0_cache_A <= 8'd5; c0_valid <= 1'b0;
      c1_cache_A <= 8'd5; c1_valid <= 1'b0;
    end else begin
      if (c0_wr) begin c0_cache_A <= c0_wdata; c0_valid <= 1'b1; end          // store: own copy only
      if (c1_wr) begin c1_cache_A <= c1_wdata; c1_valid <= 1'b1; end
      if (c0_rd && !c0_valid) begin c0_cache_A <= mem_A; c0_valid <= 1'b1; end // fill on miss
      if (c1_rd && !c1_valid) begin c1_cache_A <= mem_A; c1_valid <= 1'b1; end
    end
  end
 
  assign c0_rdata = c0_cache_A;   // core0's private copy
  assign c1_rdata = c1_cache_A;   // core1's private copy (may be stale)
endmodule

Expected result: after CPU0 stores 6, CPU1's next read returns the stale 5 (not 6) because its line stayed valid and nothing invalidated it — the missing path is coherency.

11. DebugLab — "CPU1 never sees CPU0's write"

12. Common Mistakes

• Blaming the lock. A lock serializes access; it does not make one cache's write visible to another. Coherency is a separate, lower layer.
• Assuming memory is the single source of truth. With write-back caches the newest value often lives in a cache, not in memory (a "dirty" line); memory can be stale.
• Confusing coherency with consistency. Coherency is per-address agreement; consistency/ordering is cross-address (Module 12).
• Thinking "flush everything" is the answer. Flushing on every write is correct but ruinously slow — CHI exists to be coherent cheaply.
• "AXI already gives coherency." No — AXI is a per-master read/write protocol with no snoop and no cache-state tracking (1.8).
• The invariant to hold: after a write to a line settles, no two caches may report different valid values for that address — the incoherent demo violates exactly this.

13. Interview Questions

14. Engineering Checklist

• I can state the coherency contract: single-writer/multiple-reader, write propagation, value consistency — per address.
• I can show the exact cycle where two private copies diverge on a write.
• I can distinguish coherency (per-address) from consistency/ordering (cross-address, Module 12).
• I can explain why a lock does not fix stale cache copies.
• I can name the missing action — snoop/invalidate on write — a coherency protocol must add.

15. Key Takeaways

• Private caches create copies. Two cores caching the same address hold two independent copies of that line.
• Writes must propagate. A store updates only the writing core's copy; without coherency the other copy stays stale and the reader gets wrong data.
• Coherency is a per-address hardware contract: single-writer/multiple-reader, write propagation, and value consistency.
• Locks cannot fix it. Locks order access; they never reconcile diverging cache contents. Coherency must be enforced by hardware because software cannot reach into another core's private line.
• This is why CHI exists. Every CHI mechanism — nodes, channels, snoops, directories — makes the missing "invalidate the other copy on a write" both correct and cheap.

16. Quick Revision

Why cache coherency exists. Private caches = copies of shared memory. Two cores caching address A = two copies. A store updates only the writer's copy -> the other copy is stale -> reader hits old data (lost updates, broken shared state). Coherency = per-address hardware contract: single-writer/multi-reader + write propagation + value consistency. It is not locking (orders access, cannot reconcile copies) and not consistency/ordering (cross-address, Module 12). The fix = a write must snoop/invalidate other cached copies; at SoC scale that machinery is AMBA CHI. Memory is not the source of truth under write-back — the newest value may live in a cache.

17. Coming Next

Next — 1.2 The Multi-Core Problem. You will add:

• the concrete two-core race (the four-way interleaving of two loads and two stores on one line),
• why the bug is timing-dependent and vanishes single-core,
• the first read of a coherency-ordered outcome versus the broken one.

We keep the same project — CPU0, CPU1, private L1s, one shared line — and zoom from "copies can diverge" (this lesson) into "here is exactly how two cores racing make them diverge."