CPU Clusters

Module 20 · Chapter 20.3 · Real Industry Case Studies

Project thread — 20.2 was AI accelerators. 20.3 is the canonical CPU-cluster architecture; 20.4 is coherent DMA.

1. The Domain

A CPU cluster architecture is the canonical way Arm builds a scalable coherent CPU — and the reference pattern the rest of the industry follows. It is a two-level structure: clusters of CPU cores sharing a local cache, connected by a coherent mesh that ties the clusters, a system-level cache, and memory together. It spans everything from a phone's few cores to a server's dozens, and it is the most important CHI configuration to understand, because it is the one CHI was designed for.

2. Why CHI Here

CPU clusters use CHI because it is the protocol built for scalable CPU coherence. The lower level (a shared unit) needs tight, low-latency coherence among a few cores; the upper level (a mesh) needs coherence that scales across many clusters without a shared-bus bottleneck. CHI's directory-based, targeted snooping (Chapter 15.6) and distributed home nodes (Chapter 15.3) are exactly what make the upper level scale, and its credit-based, deadlock-free fabric (Chapters 14.1, 14.3) is what makes the mesh reliable. CHI is not just used here — this two-level CPU architecture is the use case CHI exists to serve.

3. Key Terms

4. Previous Chapter Connection

This chapter is the integration of the scalability material into one architecture. Distributed home nodes with address interleaving (Chapter 15.3), the system-level cache as a source between caches and memory (Chapter 13.2), directory efficiency (Chapter 15.5), and hotspot balancing (Chapter 15.7) all appear here as design levers. The cores are RN-F (Chapter 5); the homes are HN-F (Chapter 5); the mesh is the interconnect (Chapter 6.7).

Where mobile (20.1) and AI (20.2) were single subsystems, the CPU cluster is a complete, scalable coherent system — the one the large-SoC interview answer (Chapter 19.10) described. This case study makes that answer concrete: the mesh (bisection scales), the distributed interleaved homes (coherence scales), the SLC (cut memory traffic), and balanced placement (avoid hotspots) are here as a named architecture (DSU + CMN). It is the canonical application, and the template every other CHI system varies from.

5. The CHI Configuration

The two-level CPU-cluster CHI architecture:

• Lower level — the cluster (DSU). A DynamIQ Shared Unit groups a few CPU cores (RN-F) with a shared L3 into one coherent unit — intra-cluster coherence, tight and low-latency.
• Upper level — the coherent mesh (CMN). A CMN-style mesh is the CHI interconnect connecting multiple clusters, providing inter-cluster coherence that scales.
• Distributed home nodes + SLC. The mesh has distributed home nodes (HN-F), addresses interleaved across them, each holding part of the system-level cache — so coherence tracking and the SLC are spread across the mesh.
• Memory controllers (SN). Multiple memory controllers on the mesh feed the system.

The synthesis:

A CPU cluster architecture is two levels: DynamIQ-style shared-unit clusters (cores + shared L3, RN-F) for intra-cluster coherence, connected by a CMN-style coherent mesh with distributed, address-interleaved home nodes (HN-F) — each holding part of a system-level cache — for inter-cluster coherence, plus multiple memory controllers. It scales from a few cores to dozens without a shared bus.

6. The Domain Constraints

What shapes a CPU-cluster CHI design — scalability with low latency.

• Two-level coherence. Coherence within a cluster (a few cores sharing L3) must be fast; coherence across clusters (many clusters on the mesh) must scale. The two levels have different optimization points.
• Scale without a bus. As clusters multiply, a shared bus would bottleneck — so the upper level is a mesh with distributed homes, so coherence cost tracks sharing, not cluster count (Chapter 15.3).
• Latency matters (it's a CPU). Unlike an accelerator's bandwidth focus, a CPU is latency-sensitive — a distant home on a large mesh adds latency (Chapter 15.1), so home placement and the SLC (to avoid memory round-trips) matter.
• The SLC is the shared cache. A distributed system-level cache between the clusters and memory cuts memory traffic and latency — its sizing is a central design decision (Chapter 15.5).

The point to carry:

The CPU cluster balances scale (many clusters, distributed homes, mesh) against latency (it's a CPU — a distant home or a memory round-trip hurts). The two levels exist because they optimize differently: the cluster for tight local coherence, the mesh for scalable global coherence. Getting the boundary right — cluster size, mesh size, SLC size, home distribution — is the essence of the design, and it is the most general CHI design problem, because this is the architecture CHI was built for.

7. Engineering Diagram — the two-level CPU-cluster system

Clusters (cores + shared L3) at the lower level, a CMN mesh with distributed home nodes and SLC at the upper level, and multiple memory controllers. The two coherence levels — intra-cluster in the DSU, inter-cluster on the mesh — are the defining structure. This is the reference CHI CPU architecture.

8. The Distinctive Design Decisions

What the CPU-cluster architecture does — and why.

The rule to carry: the two-level hierarchy is the defining decision, and everything else follows from it. The cluster optimizes local coherence (fast, few cores); the mesh optimizes global coherence (scalable, many clusters). The distributed homes + SLC and balanced placement are how the mesh level scales without bottlenecking. This is the architecture the large-SoC interview answer (Chapter 19.10) sketches — here it is named and concrete (DSU + CMN), the template every other CHI CPU varies from.

9. The Trade-offs

The tensions a CPU-cluster design balances.

• Cluster size vs mesh size. Bigger clusters (more cores per DSU) mean fewer mesh nodes (simpler mesh) but a larger intra-cluster coherence cost; smaller clusters mean a bigger mesh. The split balances local vs global coherence cost.
• SLC size vs area. A larger system-level cache cuts memory traffic and latency but costs area; too small and it misses often (Chapter 15.5). Sized to the workload's reuse.
• Home count vs latency. More homes spread coherence load (scale) but each is farther on average (latency); fewer homes are closer but risk hotspots. Balanced by interleaving and placement (Chapter 15.7).
• Scale vs latency. A bigger mesh scales throughput but adds hop latency to distant homes/memory (Chapter 15.1) — a CPU feels this, so placement and the SLC mitigate it.

The point to carry:

The CPU cluster's trade-offs are all sizing and placement — cluster size, mesh size, SLC size, home distribution — balancing scale against latency. There is no single right answer; it depends on core count, workload, and latency targets. This is why it is the canonical design problem: it exercises every scalability lever (distribute, interleave, cache, balance) against a latency-sensitive CPU workload. Master this trade-off space and you can architect any CHI system.

10. A Concrete Walkthrough

A cross-cluster shared-line access — the scenario that shows the two levels working.

1. Intra-cluster hit. Core 0 in cluster A reads a line another core in cluster A holds. The DSU handles it locally — fast, no mesh traffic. Intra-cluster coherence.
2. Cross-cluster miss. Core 0 in cluster A reads a line a core in cluster B holds. The DSU misses locally, so the request goes to the mesh.
3. Home lookup on the mesh. The request reaches the home node for that address (chosen by interleaving), which checks its directory and finds cluster B shares the line.
4. Targeted snoop + SLC. The home snoops cluster B (targeted, Chapter 15.6) and, if the line is in the SLC, sources it from there — no memory round-trip. Inter-cluster coherence.
5. Complete. The data returns to cluster A. The two levels cooperated: the DSU for local, the mesh for global, the SLC to avoid memory.

The access used the DSU for intra-cluster and the mesh + distributed home + SLC for inter-cluster — the two-level architecture in action. The pitfall is mis-sizing the SLC (frequent memory round-trips) or mis-distributing the homes (a hotspot home caps the mesh).

11. Representative Configuration

A two-level CPU-cluster CHI configuration. Representative.

// Representative CPU-cluster CHI configuration (educational, not a product spec).
// Two-level: DynamIQ-style clusters + a CMN-style coherent mesh. The canonical scalable CPU.
localparam int N_CLUSTERS       = 4;       // number of DSU-style clusters
localparam int CORES_PER_CLUSTER= 4;       // cores per cluster (RN-F), sharing an L3
localparam int L3_KB_PER_CLUSTER= 2048;    // shared L3 per cluster (intra-cluster level)
 
// Upper level: the CMN-style coherent mesh.
localparam int MESH_ROWS        = 4;       // mesh dimensions (bisection scales, 15.3)
localparam int MESH_COLS        = 4;
localparam int N_HOME_NODES     = 8;       // DISTRIBUTED HN-F, address-interleaved (15.3/15.6)
localparam int SLC_KB           = 16384;   // system-level cache, DISTRIBUTED across homes
localparam int N_MEM_CTRL       = 4;       // multiple memory controllers on the mesh
 
// The two coherence levels:
//   intra-cluster (DSU: cores + L3)   -- fast, local
//   inter-cluster (CMN: distributed HN-F + SLC on a mesh) -- scalable, global
// Sizing/placement balances SCALE (more clusters/homes) vs LATENCY (closer homes/SLC).

The configuration names the two levels: clusters (cores + L3) and a mesh (distributed homes + SLC + memory). The knobs — cluster count/size, mesh dimensions, home count, SLC size — are the sizing and placement trade-offs of section 9. This is the template the mobile (small) and server (large, Chapter 20.6) configurations vary from.

12. Analysis View — scale and latency together

The two forces a CPU-cluster design balances.

• Aggregate throughput scales with distributed homes. With homes distributed and addresses interleaved, aggregate coherence throughput grows with the home count (Chapter 15.3) — the mesh scales.
• Latency grows with mesh size. A larger mesh means more hops to a distant home or memory (Chapter 15.1) — and a CPU feels latency, so the design mitigates it with placement and the SLC.
• The SLC cuts the memory term. A well-sized SLC turns memory accesses into SLC hits, cutting the dominant latency term (Chapter 15.1) and the memory-bandwidth pressure (Chapter 15.4).
• Hotspots cap the mesh. A mis-distributed home creates a hot link that caps throughput despite a low average (Chapter 15.7) — balanced by interleaving and placement.

The point to carry:

The CPU-cluster analysis is scale and latency together — distribute homes to scale, place them and size the SLC to keep latency low, and balance to avoid hotspots. It is the complete performance analysis from Chapter 15, applied to one architecture: latency anatomy (15.1), distribution (15.3), memory-bandwidth knee (15.4), directory efficiency (15.5), snoop reduction (15.6), and hotspots (15.7) all appear as levers. This is why the CPU cluster is the canonical design — it is where every performance lever matters at once.

13. Design Considerations

The deeper checklist for a CPU-cluster CHI design.

• Choose the cluster/mesh split — cluster size (cores per DSU) vs mesh size, balancing local vs global coherence cost.
• Distribute the home nodes and interleave addresses — coherence scales with sharing.
• Size the SLC to the workload's reuse — cut memory traffic and latency.
• Place homes and memory controllers to avoid hotspots — keep the mesh flat.
• Mind latency — a CPU feels distant homes/memory; use placement and the SLC to mitigate.
• Balance scale vs latency — the core CPU-cluster trade-off, tuned to core count and workload.

14. The Pitfall

15. Common Mistakes

• Undersized SLC. Assumption: a small SLC suffices. Bug: frequent memory round-trips (the Pitfall). Fix: size to the reuse.
• Centralized/hotspot homes. Assumption: few homes are simpler. Bug: flat scaling / a hot link. Fix: distribute + interleave.
• Ignoring latency. Assumption: scale is everything. Bug: distant-home stalls. Fix: placement + SLC for latency.
• Wrong cluster/mesh split. Assumption: any split works. Bug: unbalanced coherence cost. Fix: balance local vs global.
• Uniform-traffic assumption. Assumption: traffic is even. Bug: hidden hotspots. Fix: verify a flat utilisation histogram.
• One coherence level. Assumption: flatten the hierarchy. Bug: no local fast path or no global scale. Fix: two levels.

16. Engineering Checklist

• Build two coherence levels — clusters (DSU) + a coherent mesh (CMN).
• Group cores + shared L3 into coherent clusters for fast local coherence.
• Distribute the home nodes and interleave addresses on the mesh.
• Distribute the system-level cache across the homes; size it to the reuse.
• Place homes and memory controllers to avoid hotspots.
• Balance scale against latency for the target core count and workload.

17. Key Takeaways

• CPU clusters are the canonical Arm coherent-CPU architecture on CHI.
• It is two levels — intra-cluster (DSU: cores + L3) and inter-cluster (CMN mesh).
• The mesh has distributed, interleaved home nodes with a system-level cache.
• It scales from a few cores to dozens without a shared bus.
• The trade-offs are all sizing and placement — scale vs latency.
• The pitfall is mis-sizing the SLC or mis-distributing the homes; the model here is representative.

18. Quick Revision

CPU clusters. The canonical Arm coherent-CPU architecture is two-level. The lower level is a DynamIQ Shared Unit (DSU)-style cluster: a few CPU cores (RN-F) with a shared L3, one coherent unit — fast, local intra-cluster coherence. The upper level is a CMN (Coherent Mesh Network)-style CHI mesh connecting multiple clusters, with distributed home nodes (HN-F) — addresses interleaved across them (one serialization point per address, many homes in parallel) — each holding part of a distributed system-level cache (SLC), plus multiple memory controllers. This provides scalable inter-cluster coherence without a shared bus, from a few cores to dozens. The defining feature is the two levels, which optimize differently: the cluster for tight local coherence, the mesh for scalable global coherence. The design questions are all sizing and placement — cluster size vs mesh size, SLC size (cut memory traffic/latency), and home distribution (scale vs latency, avoid hotspots) — balancing scale against latency (a CPU feels a distant home or memory round-trip). It exercises every performance lever from Chapter 15 (distribute, interleave, cache, balance) at once, which is why it is the reference design. The pitfall: mis-sizing the SLC (frequent memory round-trips) or mis-distributing the homes (a hot link caps the mesh, or a centralized home flattens scaling). Representative case study; 20.4 covers cache-coherent DMA.

Coming Next

Chapter 20.4 — Cache-Coherent DMA. CPU clusters were fully-coherent cores; the next case is a device that needs coherence without a cache. Chapter 20.4 covers cache-coherent DMA — how an I/O device or DMA engine accesses coherent memory via an I/O-coherent request node without holding a cache, why this eliminates software cache-maintenance around DMA, and the one-way coherence relationship it establishes.