ACE Introduction

Module 3 · Chapter 3.4 · Why CHI Exists

Project thread — 3.3 left AXI fast but non-coherent. This chapter adds the missing piece: ACE layers snoop-based coherency onto AXI. 3.5 shows why its broadcast model hits a wall.

1. Learning Outcomes

By the end of this chapter you should be able to:

• Explain what ACE adds to AXI: snoop channels, coherent transactions, and a cache-state model.
• Name the three snoop channels — AC (snoop address), CR (snoop response), CD (snoop data).
• List ACE's five cache states — UD, UC, SD, SC, I — and their MOESI mapping.
• Trace a ReadShared coherent transaction through a broadcast snoop and back.
• Implement a representative ACE snoop-response block in SystemVerilog, Verilog-2001, and VHDL.
• See why ACE coherency is fundamentally broadcast — the setup for its scaling limit.

2. Why Should I Learn This?

ACE is the bridge between the non-coherent world (AXI) and CHI. It is the first time AMBA makes caches visible to each other in hardware, and the mechanisms it introduces — snoops, coherent transaction types, a shared cache-state model, shareability domains — are the same ideas CHI keeps and re-engineers. Understand ACE and CHI is a refinement, not a new planet.

It is also production hardware: Arm big.LITTLE systems used ACE with coherent interconnects like CCI-400/500. Knowing the snoop flow — and, next chapter, why broadcasting it does not scale — is exactly the reasoning that justifies CHI.

3. Key Terms

4. Previous Chapter Connection

Chapter 3.3 pinned AXI's one hard limit: it is non-coherent. Its five channels move data superbly but there is no snoop, no cache-state signalling, no way for one master to see another's cache. Sharing data across masters was left to software.

ACE removes exactly that limit — and it does so by adding to AXI, not replacing it. The AW/W/B/AR/R channels stay; ACE bolts on snoop channels and coherent transaction semantics above them. This is the first appearance of the ideas Module 2 taught in the abstract — invalidations, ownership, dirty data, snoops — now as real AMBA wires.

5. Core Concept — coherency layered on AXI

ACE is AXI plus four additions.

• Snoop channels (AC · CR · CD). The interconnect drives a snoop address to a cache on AC; the cache returns a snoop response on CR (do I have it, in what state) and, if it holds dirty data, the data on CD. These are the wires that let caches see each other.
• Coherent transaction types. ARSNOOP / AWSNOOP encode intent: ReadShared (I want a shareable copy), ReadUnique (I want the only copy, to write), MakeUnique / CleanUnique (invalidate others), WriteBack / Evict (give up a line). AXI's plain reads and writes gain coherency meaning.
• A five-state cache model. Every line is in one of UD, UC, SD, SC, I — ACE's actual states, mapping to MOESI: UD = Modified, UC = Exclusive, SD = Owned, SC = Shared, I = Invalid. This is the bookkeeping that keeps copies consistent.
• Shareability domains. Each transaction names a domain (Inner / Outer / System) — the set of caches the interconnect must snoop. Snoop only who might have the line.

Put together:

ACE turns AXI into a coherent fabric by making the interconnect an active snoop orchestrator: on a coherent request it broadcasts a snoop to the domain, gathers CR/CD responses, takes dirty data from the owner, updates every affected line's state, and returns data to the requester. Coherency Module 2 described in the abstract is now AC/CR/CD wires and UD/UC/SD/SC/I states.

6. Engineering Mental Model — a librarian who calls around

Picture the coherent interconnect as a librarian for one shared book (a cache line).

• A reader asks for a shareable copy (ReadShared). Before answering, the librarian calls every branch in the relevant region (broadcast snoop on AC) — "does anyone have this book, and is your copy marked up?"
• Each branch answers (CR): "no copy," or "clean copy," or "I have the only, marked-up copy." A branch with the marked-up (dirty) copy hands it over (CD).
• The librarian gives the reader the up-to-date copy (R) and updates the ledger: the branch that had the only copy is now just one of several shared holders.

The system works — but notice the librarian calls every branch every time, whether or not they hold the book. That is broadcast, and it is exactly what the next chapter shows does not scale.

7. Engineering Diagram — ACE topology

The caches never connect directly — every coherent action routes through the interconnect, which is what makes it responsible for snooping everyone in the domain.

8. The ACE Protocol Stack

ACE is best read as layers on top of AXI: the AXI channels are untouched underneath; coherency is added above.

9. Worked Example — a ReadShared snoop across two caches

CPU0 issues ReadShared for a line CPU1 currently holds UniqueDirty (the only, modified copy). Follow both caches' states and the snoop response.

The outcome is pure MOESI: CPU1's UniqueDirty line becomes SharedDirty (it still owns the dirty data and must eventually write it back), CPU0 becomes SharedClean, and the dirty data moved cache-to-cache without touching memory. Two facts to carry: the interconnect had to snoop CPU1 to discover the dirty copy, and it would have snooped every other cache in the domain too — even those holding nothing.

10. Transaction Walkthrough — one coherent read

The same ReadShared, channel by channel:

1. Issue. CPU0 drives AR with ARSNOOP = ReadShared and a shareability domain. This is an AXI AR beat carrying coherent intent.
2. Broadcast snoop. The interconnect sends the snoop address on AC to every other cache in the domain — here, CPU1.
3. Snoop response. CPU1 sees it holds the line UniqueDirty. It responds on CR (hit, was-unique, will pass a shared copy) and drives the dirty line on CD.
4. Merge and update. The interconnect takes CPU1's data, transitions CPU1 to SharedDirty, and prepares the response. Memory is not written yet — the dirty data stays cached under SD.
5. Return. The interconnect returns the data to CPU0 on R; CPU0 installs the line SharedClean.

Every coherent read pays for a broadcast snoop and a response-collection step, even in the common case where no one else holds the line.

11. Timing View — the snoop flow in cycles

The snoop is a multi-channel handshake; seeing it on a waveform makes the AC → CR → CD → R chain concrete.

Every stage is a real channel handshake. The point to carry into the next chapter: steps 1–3 happen for each snooped cache, and the interconnect must wait for all responses before it can answer.

12. RTL / Hardware View — a representative snoop-response block

The heart of an ACE cache is how it answers a snoop. Here is that logic for a ReadShared snoop: given the line's current state, produce the CR response fields and the next state. Representative and combinational — one transaction type, no full channel or domain logic.

// Representative ACE ReadShared snoop-response (educational, not full ACE).
// A snooped cache answers a ReadShared: supply data if dirty, become shared,
// never remain Unique. States map to MOESI (UD=M, UC=E, SD=O, SC=S, I=I).
module ace_snoop_readshared (
  input  logic [2:0] state,        // current line state
  output logic       data_xfer,    // this cache supplies data on CD
  output logic       is_shared,    // CR: line is now shared
  output logic [2:0] next_state    // this cache's next state
);
  localparam logic [2:0] I=3'd0, SC=3'd1, SD=3'd2, UC=3'd3, UD=3'd4;
  always_comb begin
    data_xfer = 1'b0; is_shared = 1'b0; next_state = state;   // defaults
    case (state)
      I:  next_state = I;                                             // no copy
      SC: begin is_shared = 1'b1; next_state = SC; end               // already shared-clean
      SD: begin data_xfer = 1'b1; is_shared = 1'b1; next_state = SD; end  // owner supplies, stays SD
      UC: begin is_shared = 1'b1; next_state = SC; end               // unique-clean -> shared-clean
      UD: begin data_xfer = 1'b1; is_shared = 1'b1; next_state = SD; end  // unique-dirty -> shared-dirty
      default: next_state = I;
    endcase
  end
endmodule

The same behavior in Verilog-2001:

// Representative ACE ReadShared snoop-response (Verilog-2001).
module ace_snoop_readshared (
  input      [2:0] state,
  output reg       data_xfer,
  output reg       is_shared,
  output reg [2:0] next_state
);
  localparam I=3'd0, SC=3'd1, SD=3'd2, UC=3'd3, UD=3'd4;
  always @* begin
    data_xfer = 1'b0; is_shared = 1'b0; next_state = state;
    case (state)
      I:  next_state = I;
      SC: begin is_shared = 1'b1; next_state = SC; end
      SD: begin data_xfer = 1'b1; is_shared = 1'b1; next_state = SD; end
      UC: begin is_shared = 1'b1; next_state = SC; end
      UD: begin data_xfer = 1'b1; is_shared = 1'b1; next_state = SD; end
      default: next_state = I;
    endcase
  end
endmodule

And in VHDL:

-- Representative ACE ReadShared snoop-response (VHDL).
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
 
entity ace_snoop_readshared is
  port (
    state      : in  std_logic_vector(2 downto 0);
    data_xfer  : out std_logic;
    is_shared  : out std_logic;
    next_state : out std_logic_vector(2 downto 0)
  );
end entity;
 
architecture rtl of ace_snoop_readshared is
  constant I  : std_logic_vector(2 downto 0) := "000";
  constant SC : std_logic_vector(2 downto 0) := "001";
  constant SD : std_logic_vector(2 downto 0) := "010";
  constant UC : std_logic_vector(2 downto 0) := "011";
  constant UD : std_logic_vector(2 downto 0) := "100";
begin
  process(state)
  begin
    data_xfer <= '0'; is_shared <= '0'; next_state <= state;   -- defaults
    case state is
      when I  => next_state <= I;
      when SC => is_shared <= '1'; next_state <= SC;
      when SD => data_xfer <= '1'; is_shared <= '1'; next_state <= SD;
      when UC => is_shared <= '1'; next_state <= SC;
      when UD => data_xfer <= '1'; is_shared <= '1'; next_state <= SD;
      when others => next_state <= I;
    end case;
  end process;
end architecture;

All three encode the ReadShared rule: a snooped cache loses uniqueness (UC→SC, UD→SD), supplies data when dirty (SD/UD set data_xfer), and reports the line as shared. That is MOESI, spoken in ACE's CR/CD signals.

13. Verification View — a snoop never leaves a stale unique copy

Two properties guard the coherency contract: a shared read must clear uniqueness everywhere, and dirty data must be supplied so it is never lost.

// Bind to ace_snoop_readshared.
localparam logic [2:0] I=3'd0, SC=3'd1, SD=3'd2, UC=3'd3, UD=3'd4;
 
// 1. After a ReadShared snoop, this cache is never left Unique
//    (a shared copy now exists elsewhere — no single-writer illusion).
always_comb assert (next_state != UC && next_state != UD);
 
// 2. If the line was dirty (SD or UD), the snoop MUST supply the data,
//    otherwise the only up-to-date copy would be lost.
always_comb assert (!((state == SD) || (state == UD)) || data_xfer);

The system point, beyond the two checks:

ACE's correctness rests on the interconnect hearing from every cache that could hold the line before it answers. A snoop response that hides a dirty copy, or a cache that stays Unique after a shared read, breaks Single-Writer/Multiple-Reader. That completeness is cheap with two cores and a broadcast — and it is exactly the cost that explodes as cores multiply, which the next chapter takes up.

• What it proves: the snoop downgrades correctly and never drops dirty data.
• What it does not prove: interconnect-level ordering, multi-cache merge, or domain filtering.
• Bug signature: a dirty line answered without data_xfer → the requester reads stale memory (the DebugLab).

14. Testbench — exercise every state's snoop response

Drives all five states through a ReadShared snoop and checks the response and next state.

module tb_ace_snoop_readshared;
  logic [2:0] state, next_state;
  logic       data_xfer, is_shared;
  int errors = 0;
  localparam logic [2:0] I=3'd0, SC=3'd1, SD=3'd2, UC=3'd3, UD=3'd4;
 
  ace_snoop_readshared dut (.*);
 
  task automatic check(input logic [2:0] s, input logic exp_d,
                       input logic [2:0] exp_ns, input string tag);
    state = s; #1;
    if (data_xfer !== exp_d || next_state !== exp_ns) begin
      errors++; $display("FAIL [%s] data_xfer=%b next=%0d (exp d=%b ns=%0d)",
                         tag, data_xfer, next_state, exp_d, exp_ns);
    end else
      $display("PASS [%s] data_xfer=%b is_shared=%b next=%0d", tag, data_xfer, is_shared, next_state);
  endtask
 
  initial begin
    check(I,  1'b0, I,  "I  -> stay I, no data");
    check(SC, 1'b0, SC, "SC -> stay SC, no data");
    check(SD, 1'b1, SD, "SD -> stay SD, supply data");
    check(UC, 1'b0, SC, "UC -> SC, no data");
    check(UD, 1'b1, SD, "UD -> SD, supply data");
 
    if (errors == 0) $display("ALL TESTS PASSED");
    else             $display("%0d FAILURE(S)", errors);
    $finish;
  end
endmodule

Expected output:

PASS [I  -> stay I, no data] data_xfer=0 is_shared=0 next=0
PASS [SC -> stay SC, no data] data_xfer=0 is_shared=1 next=1
PASS [SD -> stay SD, supply data] data_xfer=1 is_shared=1 next=2
PASS [UC -> SC, no data] data_xfer=0 is_shared=1 next=1
PASS [UD -> SD, supply data] data_xfer=1 is_shared=1 next=2

15. DebugLab — dirty data lost on a snoop

state  is_shared  data_xfer  next_state  note
 UD       1           0          SD       downgraded, but NO data supplied

16. Common Mistakes

• Thinking ACE replaces AXI. Assumption: a new coherent bus. Bug: expecting AXI channels to disappear. Prevention: ACE is layered on AXI — AW/W/B/AR/R are unchanged; snoop channels are added.
• Ignoring the snoop-data path. Assumption: state changes are enough. Bug: dirty data lost on a snoop (the DebugLab). Prevention: dirty (SD/UD) lines must supply data (CD) or write back on a snoop.
• Confusing the state set with CHI's. Assumption: UD/UC/SD/SC/I is CHI's model. Bug: mismatched expectations later. Prevention: UD/UC/SD/SC/I is ACE's; CHI defines its own set.
• Forgetting shareability domains. Assumption: every transaction snoops everyone. Bug: over- or under-snooping. Prevention: the domain (Inner/Outer/System) selects who is snooped.
• Treating ReadShared and ReadUnique alike. Assumption: a read is a read. Bug: sharing when you meant to gain write ownership (or vice versa). Prevention: ARSNOOP intent matters — ReadShared shares, ReadUnique invalidates others.
• Assuming broadcast is free. Assumption: snooping all caches scales fine. Bug: response latency and traffic grow with cores. Prevention: broadcast is ACE's model — and its limit (next chapter).

17. Engineering Checklist

• Identify the three snoop channels — AC (address), CR (response), CD (data).
• Read the ARSNOOP / AWSNOOP intent (ReadShared / ReadUnique / MakeUnique / WriteBack).
• Track lines in the five states — UD, UC, SD, SC, I — and know the MOESI mapping.
• On any snoop of a dirty line, ensure data is supplied or written back — never dropped.
• Scope snoops by shareability domain, not blindly to all caches.
• Remember the interconnect must hear from all snooped caches before responding.

18. Key Takeaways

• ACE = AXI + coherency: three snoop channels (AC/CR/CD), coherent transaction types on ARSNOOP/AWSNOOP, and a five-state cache model — added above unchanged AXI channels.
• ACE's five states — UD, UC, SD, SC, I — are its real model and map to MOESI; CHI defines its own set later.
• A coherent read makes the interconnect broadcast a snoop to the domain, collect CR/CD responses, move dirty data cache-to-cache, and update states.
• Correctness needs complete snooping — every cache that could hold the line — and dirty data must never be dropped on a state change.
• Broadcasting the snoop is cheap at two cores and is ACE's defining mechanism — and, as the next chapter shows, its scaling limit.
• The ACE here is aligned to the AMBA model, simplified for teaching.

19. Quick Revision

ACE introduction. AXI Coherency Extensions add coherency on top of AXI. Three snoop channels: AC (snoop address out), CR (snoop response back), CD (snoop data back). Coherent transactions via ARSNOOP/AWSNOOP: ReadShared, ReadUnique, MakeUnique, WriteBack. Five states — UD, UC, SD, SC, I — ACE's real model, mapping to MOESI (M/E/O/S/I). A coherent interconnect broadcasts a snoop to the shareability domain, gathers responses, moves dirty data cache-to-cache, and updates states. On a ReadShared, a snooped cache loses uniqueness (UC→SC, UD→SD) and supplies data if dirty. Correct only if snooping is complete and dirty data is never dropped. Broadcast is the model — and the limit CHI clears. Aligned to the AMBA ACE model.

Coming Next

Chapter 3.5 — ACE Limitations. ACE works beautifully for a handful of cores, but its coherency is broadcast: every coherent request snoops every cache in the domain, and every snoop waits for every response. The next chapter shows how that snoop traffic and latency grow with core count until broadcast becomes the bottleneck — the exact scaling wall that a directory-based design like CHI was created to break.