Cache Agents

Module 5 · Chapter 5.7 · CHI System Components

Project thread — the taxonomy is complete; this chapter opens the RN-F cache agent that implements its coherency. 5.8 maps DRAM controllers onto the SN-F role.

1. Learning Outcomes

By the end of this chapter you should be able to:

• Describe the cache agent as RN-F's coherency engine and its internal blocks.
• Read the full cache-line state machine — request-driven and snoop-driven transitions together.
• Explain the silent UC → UD upgrade and why a shared line cannot take it.
• Match each CPU event (load, store, evict) to the coherent request the agent issues.
• Trace how one state machine serves both request issue and snoop response.
• Implement a representative cache-agent state machine in SystemVerilog, Verilog-2001, and VHDL.

2. Why Should I Learn This?

The cache agent is where a CPU's loads and stores become coherent transactions. It is the single most important piece of RTL in a Request Node, and the place multi-core bugs concentrate — a wrong transition, a missing request, a mishandled snoop. Understanding the agent's one state machine, driven from both sides, is understanding how coherency is actually implemented in silicon.

It also unifies the two halves of Chapter 5.1. There, request issue and snoop response looked like separate behaviors; here you see they are one state machine per line, moved by CPU events and snoop events alike. That unification is the mental shift that makes CHI's cache side click.

3. Key Terms

4. Previous Chapter Connection

Chapter 5.1 gave RN-F's behavior from the outside: it holds lines in CHI states, issues the right coherent request, and answers every snoop. It presented request issue and snoop response as two lists of rules.

This chapter opens the agent that implements both — and reveals they are one machine. A line's state is moved by CPU events (load, store, evict) and by snoop events (SnpShared, SnpUnique), and the same next-state logic serves both. Where 5.1 was the specification of behavior, 5.7 is the engine, at the depth an implementer needs.

5. Core Concept — one engine, driven from two sides

The cache agent is a small set of blocks around a single state machine.

• The cache and its state array. Each line has data and a CHI state — I, UC, UD, SC, SD. The state array is the agent's coherency memory.
• The line-state FSM. One per-line state machine is moved by two kinds of events. CPU events: a load on a miss, a store that needs write permission, an eviction. Snoop events: SnpShared (downgrade), SnpUnique (invalidate). The same next-state function handles both.
• Request issue. When the FSM needs the fabric — a miss (ReadShared/ReadUnique), a shared-line store (MakeUnique/CleanUnique), a dirty eviction (WriteBack) — the agent issues the coherent request and tracks it in an MSHR by TxnID.
• Snoop handling. When a SNP arrives, the FSM computes the response and next state from the line's current state, and the agent supplies data if the line is dirty.

The unification that matters:

Request issue and snoop response are not two engines — they are one cache-line state machine driven from two sides. A store and a SnpUnique both change a line's state through the same logic; the agent simply issues a request on some transitions and a snoop response on others. See the FSM as the whole cache agent, and RN-F's behavior becomes a single table.

6. Engineering Mental Model — a librarian's index card per book

Picture the cache agent as a librarian keeping one index card per book (line) that records its status (state).

• When a patron (the CPU) asks to read a book the branch does not have, the librarian orders a copy (a request) and updates the card when it arrives. When the patron wants to write in a book the branch only holds a read copy of, the librarian must first recall all other copies (MakeUnique) before letting them write.
• When the head office (the Home Node) calls to say "someone else needs this book" (a snoop), the librarian updates the same card — mark it shared, or hand it back — reading the card's current status to decide.
• There is one card per book, and both the patron's requests and the head office's calls write to it. The librarian never keeps two cards for one book; one status, updated from both sides.

One index card per line, moved by the CPU and by snoops through the same rules — that is the cache agent.

7. Engineering Diagram — inside the cache agent

The FSM is the hub: the cache feeds it state, and it drives issue, snoop response, and the MSHR. Everything the cache agent does routes through that one machine.

8. The Cache-Line State Machine

Here is the per-line machine the agent runs, over the CHI core states. It is moved by CPU events (load, store, evict) and snoop events; a SnpUnique takes any state to Invalid (noted below, not drawn, to keep the diagram legible).

The transition that defines the cache agent's subtlety is UC → UD "silent": a store to a Unique-Clean line needs no bus transaction, because this cache already holds the only copy. A Shared line cannot do that — it must issue MakeUnique first, the DebugLab.

9. The Full Transition Table

The diagram shows the spine; the full behavior, including SnpUnique from every state, is a small table.

Two facts to carry: only UC upgrades silently (already sole owner) — SC and SD must issue MakeUnique — and a dirty line (UD/SD) supplies data on any snoop that removes or downgrades it (Chapter 5.1). The whole cache agent is this one table, driven from both sides.

10. Transaction Walkthrough — a store to a Shared line

CPU0 holds line A SharedClean and executes a store — it must gain ownership before writing.

1. Store hits a shared line. The FSM sees state SC and a store event. A shared line is not writable — the agent must upgrade.
2. Issue MakeUnique. The agent issues MakeUnique for line A (it already has the data, so it needs only ownership) and allocates an MSHR entry (TxnID).
3. Home invalidates others. The Home Node snoops every other sharer with SnpUnique; they invalidate. The Home returns completion.
4. Upgrade and write. The agent transitions the line SC → UD, retires the MSHR entry, and lets the store write. Line A is now uniquely held and dirty in CPU0.
5. Single-writer preserved. By the time CPU0 writes, MakeUnique has cleared every other copy — exactly one cache holds the line writable.

Contrast a store to UC: no MakeUnique, no snoops, no bus — just UC → UD and write. The difference between SC and UC on a store is the whole point, and getting it wrong is the DebugLab.

11. RTL / Hardware View — the cache-agent state machine

The agent's core is the combined next-state function: CPU events and snoop events, request issue and snoop data, in one block. Representative and combinational (the FSM logic; the cache array and MSHR are elsewhere). Note the input is named ev — event is a reserved SystemVerilog keyword.

// Representative RN-F cache-agent state machine (educational, CHI core states).
// One next-state function for CPU events (load/store/evict) and snoop events.
// Emits issue_req when the fabric is needed and data_xfer when a dirty line is
// snooped. `ev` (not `event`, a reserved word): 0 LOAD,1 STORE,2 SNP_S,3 SNP_U,4 EVICT.
module cache_agent_fsm (
  input  logic [2:0] state,      // I/UC/UD/SC/SD
  input  logic [2:0] ev,         // event
  input  logic       excl,       // load found no other holder -> Exclusive (UC)
  output logic       issue_req,  // a coherent request must be issued
  output logic       data_xfer,  // supply data (dirty line snooped)
  output logic [2:0] next_state
);
  localparam logic [2:0] I=3'd0, SC=3'd1, SD=3'd2, UC=3'd3, UD=3'd4;
  localparam logic [2:0] LOAD=3'd0, STORE=3'd1, SNP_S=3'd2, SNP_U=3'd3, EVICT=3'd4;
  always_comb begin
    issue_req = 1'b0; data_xfer = 1'b0; next_state = state;
    case (ev)
      LOAD:  if (state == I) begin issue_req = 1'b1; next_state = excl ? UC : SC; end
      STORE: case (state)
               I:       begin issue_req = 1'b1; next_state = UD; end   // ReadUnique
               SC, SD:  begin issue_req = 1'b1; next_state = UD; end   // MakeUnique
               UC:      next_state = UD;                                // silent upgrade
               default: next_state = state;                            // UD: already writable
             endcase
      SNP_S: case (state)                                              // SnpShared: downgrade
               UC:      next_state = SC;
               UD, SD:  begin data_xfer = 1'b1; next_state = SD; end
               default: next_state = state;
             endcase
      SNP_U: begin data_xfer = (state == UD) || (state == SD); next_state = I; end  // invalidate
      EVICT: begin issue_req = (state == UD) || (state == SD); next_state = I; end   // WriteBack if dirty
      default: next_state = state;
    endcase
  end
endmodule

The same behavior in Verilog-2001:

// Representative RN-F cache-agent state machine (Verilog-2001).
module cache_agent_fsm (
  input      [2:0] state,
  input      [2:0] ev,
  input            excl,
  output reg       issue_req,
  output reg       data_xfer,
  output reg [2:0] next_state
);
  localparam I=3'd0, SC=3'd1, SD=3'd2, UC=3'd3, UD=3'd4;
  localparam LOAD=3'd0, STORE=3'd1, SNP_S=3'd2, SNP_U=3'd3, EVICT=3'd4;
  always @* begin
    issue_req = 1'b0; data_xfer = 1'b0; next_state = state;
    case (ev)
      LOAD:  if (state == I) begin issue_req = 1'b1; next_state = excl ? UC : SC; end
      STORE: case (state)
               I:      begin issue_req = 1'b1; next_state = UD; end
               SC, SD: begin issue_req = 1'b1; next_state = UD; end
               UC:     next_state = UD;
               default: next_state = state;
             endcase
      SNP_S: case (state)
               UC:     next_state = SC;
               UD, SD: begin data_xfer = 1'b1; next_state = SD; end
               default: next_state = state;
             endcase
      SNP_U: begin data_xfer = (state == UD) || (state == SD); next_state = I; end
      EVICT: begin issue_req = (state == UD) || (state == SD); next_state = I; end
      default: next_state = state;
    endcase
  end
endmodule

And in VHDL:

-- Representative RN-F cache-agent state machine (VHDL).
library ieee;
use ieee.std_logic_1164.all;
 
entity cache_agent_fsm is
  port (
    state      : in  std_logic_vector(2 downto 0);
    ev         : in  std_logic_vector(2 downto 0);
    excl       : in  std_logic;
    issue_req  : out std_logic;
    data_xfer  : out std_logic;
    next_state : out std_logic_vector(2 downto 0)
  );
end entity;
 
architecture rtl of cache_agent_fsm is
  constant I  : std_logic_vector(2 downto 0) := "000";
  constant SC : std_logic_vector(2 downto 0) := "001";
  constant SD : std_logic_vector(2 downto 0) := "010";
  constant UC : std_logic_vector(2 downto 0) := "011";
  constant UD : std_logic_vector(2 downto 0) := "100";
  constant LOAD : std_logic_vector(2 downto 0) := "000";
  constant STORE: std_logic_vector(2 downto 0) := "001";
  constant SNP_S: std_logic_vector(2 downto 0) := "010";
  constant SNP_U: std_logic_vector(2 downto 0) := "011";
  constant EVICT: std_logic_vector(2 downto 0) := "100";
begin
  process(state, ev, excl)
  begin
    issue_req <= '0'; data_xfer <= '0'; next_state <= state;
    case ev is
      when LOAD  => if state = I then issue_req <= '1';
                      if excl = '1' then next_state <= UC; else next_state <= SC; end if;
                    end if;
      when STORE => if state = I then issue_req <= '1'; next_state <= UD;
                    elsif state = SC or state = SD then issue_req <= '1'; next_state <= UD;
                    elsif state = UC then next_state <= UD;                    -- silent
                    end if;
      when SNP_S => if state = UC then next_state <= SC;
                    elsif state = UD or state = SD then data_xfer <= '1'; next_state <= SD;
                    end if;
      when SNP_U => if state = UD or state = SD then data_xfer <= '1'; end if;
                    next_state <= I;
      when EVICT => if state = UD or state = SD then issue_req <= '1'; end if;
                    next_state <= I;
      when others => next_state <= state;
    end case;
  end process;
end architecture;

All three run one machine for both sides. The line that separates the cache agent from a stale-data bug is UC: next_state = UD under STORE with issue_req low — the silent upgrade — set against SC/SD under STORE with issue_req high (MakeUnique). Conflate them and the DebugLab follows.

12. Verification View — only Unique upgrades silently

Two properties: a store to a shared line always issues a request, and a store to UC never does (silent upgrade).

// Bind to cache_agent_fsm.
localparam logic [2:0] SC=3'd1, SD=3'd2, UC=3'd3, UD=3'd4;
localparam logic [2:0] STORE=3'd1;
 
// 1. A store to a SHARED line must issue a request (MakeUnique) before writing.
always_comb if ((ev == STORE) && (state == SC || state == SD))
  assert (issue_req && next_state == UD);
 
// 2. A store to UC upgrades silently — no request issued.
always_comb if ((ev == STORE) && (state == UC))
  assert (!issue_req && next_state == UD);

The system point, beyond the two checks:

The cache agent's correctness hinges on the difference between Unique and Shared on a store. A Unique line (UC) may be written with no bus transaction — this cache is provably the only holder, so no other copy can be stale. A Shared line (SC/SD) may have copies elsewhere, so writing it requires MakeUnique to invalidate them first. The silent upgrade is a real optimization (the E-state's whole value), but it is legal only from Unique. Extend it to Shared and you write while stale readers survive — the single-writer violation the DebugLab shows.

• What it proves: shared stores issue MakeUnique; UC stores are silent.
• What it does not prove: the request actually invalidated all sharers (that is the HN's job, Chapter 5.3).
• Bug signature: a shared line written with no MakeUnique — stale sharers (the DebugLab).

13. Testbench — exercise loads, stores, and snoops

Drives representative events and checks the request/data outputs and next state.

module tb_cache_agent_fsm;
  logic [2:0] state, ev, next_state;
  logic excl, issue_req, data_xfer;
  int errors = 0;
  localparam logic [2:0] I=3'd0, SC=3'd1, SD=3'd2, UC=3'd3, UD=3'd4;
  localparam logic [2:0] LOAD=3'd0, STORE=3'd1, SNP_S=3'd2, SNP_U=3'd3, EVICT=3'd4;
 
  cache_agent_fsm dut (.*);
 
  task automatic check(input logic [2:0] s, input logic [2:0] e, input logic ex,
                       input logic exp_iss, input logic exp_dat, input logic [2:0] exp_ns, input string tag);
    state = s; ev = e; excl = ex; #1;
    if (issue_req !== exp_iss || data_xfer !== exp_dat || next_state !== exp_ns) begin
      errors++; $display("FAIL [%s] iss=%b dat=%b ns=%0d", tag, issue_req, data_xfer, next_state);
    end else $display("PASS [%s] iss=%b dat=%b ns=%0d", tag, issue_req, data_xfer, next_state);
  endtask
 
  initial begin
    check(I,  LOAD,  1'b1, 1'b1, 1'b0, UC, "I load excl -> UC (issue)");
    check(I,  STORE, 1'b0, 1'b1, 1'b0, UD, "I store -> UD (ReadUnique)");
    check(UC, STORE, 1'b0, 1'b0, 1'b0, UD, "UC store -> UD (silent)");
    check(SC, STORE, 1'b0, 1'b1, 1'b0, UD, "SC store -> UD (MakeUnique)");
    check(UD, SNP_S, 1'b0, 1'b0, 1'b1, SD, "UD SnpShared -> SD (data)");
    check(SD, SNP_U, 1'b0, 1'b0, 1'b1, I,  "SD SnpUnique -> I (data)");
    check(UD, EVICT, 1'b0, 1'b1, 1'b0, I,  "UD evict -> WriteBack, I");
 
    if (errors == 0) $display("ALL TESTS PASSED");
    else             $display("%0d FAILURE(S)", errors);
    $finish;
  end
endmodule

Expected output:

PASS [I load excl -> UC (issue)] iss=1 dat=0 ns=3
PASS [I store -> UD (ReadUnique)] iss=1 dat=0 ns=4
PASS [UC store -> UD (silent)] iss=0 dat=0 ns=4
PASS [SC store -> UD (MakeUnique)] iss=1 dat=0 ns=4
PASS [UD SnpShared -> SD (data)] iss=0 dat=1 ns=2
PASS [SD SnpUnique -> I (data)] iss=0 dat=1 ns=0
PASS [UD evict -> WriteBack, I] iss=1 dat=0 ns=0

14. DebugLab — silently upgrading a Shared line

state  ev     issue_req  next_state  note
 SC    STORE     0           UD       WRONG: silent upgrade from Shared

15. Common Mistakes

• Silent upgrade from Shared. Assumption: any clean line upgrades silently. Bug: stale sharers (the DebugLab). Prevention: only UC upgrades silently; SC/SD issue MakeUnique.
• Two engines instead of one. Assumption: request and snoop logic are separate. Bug: divergent state. Prevention: one FSM per line, driven from both sides.
• Dropping dirty data on a snoop. Assumption: state change suffices. Bug: lost update. Prevention: UD/SD supply data on any removing/downgrading snoop.
• Forgetting the MSHR. Assumption: transactions complete instantly. Bug: mishandled concurrency. Prevention: track outstanding requests by TxnID; a snoop can arrive mid-transaction.
• Confusing Unique with Dirty. Assumption: they are the same. Bug: mishandling UC vs UD. Prevention: Unique = sole copy; Dirty = modified.
• Writing a line without ownership. Assumption: a cached line is writable. Bug: SWMR violation. Prevention: gain Unique before any store.

16. Engineering Checklist

• Run one per-line FSM for both CPU and snoop events.
• Upgrade silently only from UC; SC/SD must issue MakeUnique.
• Issue ReadShared / ReadUnique on a miss; WriteBack on a dirty eviction.
• Supply data on any snoop that removes or downgrades a dirty line.
• Track outstanding transactions in an MSHR by TxnID.
• Never store to a line the agent does not hold Unique.

17. Key Takeaways

• The cache agent is RN-F's coherency engine: a cache-state array, a per-line FSM, request issue, snoop handling, and an MSHR.
• Request issue and snoop response are one state machine per line, driven by CPU events and snoop events alike.
• The silent UC → UD upgrade needs no bus, because a Unique line is the sole copy; SC/SD must issue MakeUnique.
• A dirty line (UD/SD) always supplies data on a removing or downgrading snoop.
• Applying the silent upgrade to a Shared line writes with stale sharers — a single-writer violation.
• The whole cache agent is one small state table driven from two sides; the model here is representative.

18. Quick Revision

Cache agents. The coherency engine inside an RN-F: a cache + state array (I/UC/UD/SC/SD), a per-line state machine, request issue, snoop handling, and an MSHR (outstanding by TxnID). Request issue and snoop response are one FSM, moved by CPU events (load, store, evict) and snoop events (SnpShared, SnpUnique). Loads on a miss issue ReadShared/ReadUnique; a store to UC upgrades silently to UD (no bus — sole copy), but a store to SC/SD must issue MakeUnique to invalidate other holders first; a dirty (UD/SD) line supplies data on a snoop; SnpUnique → I, WriteBack on a dirty evict. The defining subtlety: silent upgrade only from Unique — do it from Shared and you write with stale sharers (SWMR violation). Representative model; 5.8 maps DRAM controllers onto SN-F.

Coming Next

Chapter 5.8 — Memory Controllers in CHI. From the cache side to the memory side. Chapter 5.8 shows how a real DRAM controller maps onto the SN-F role: how the downstream reads, writes, and writebacks a Home Node forwards become DRAM commands, how the DBID write handshake meets a scheduler and banks, and where the memory controller's own reordering is safe precisely because SN-F does no coherency. It grounds the coherent-memory subordinate of Chapter 5.5 in the silicon that actually stores the data.