Skip to content
VLSIMentor

AMBA CHI · Module 1 · Cache Coherency Foundations

Shared Memory Systems

Every multi-core program assumes one thing about hardware: that all cores see a single shared pool of memory where an address means the same value to everyone. This chapter defines that memory. It introduces UMA, one memory pool with uniform latency for every core, and NUMA, memory split into local and remote regions where latency depends on which core is asking. It then separates the two ideas software must never confuse: memory topology, a performance property, from the memory abstraction, a single flat cache-coherent address space that is a correctness property. Holding that abstraction true across many cores and memory controllers is exactly what AMBA CHI was built to do.

Foundation12 min readAMBA CHIShared MemoryUMANUMAAddress Space

Module 1 · Chapter 1.3 · Cache Coherency Foundations

Project thread — continuing the running example: two CPUs (CPU0, CPU1), each with a private L1 cache. This chapter zooms out to the memory both cores sit on — the single shared address space underneath them.

1. Learning Outcomes

By the end of this lesson you will be able to:

  • Explain the single cache-coherent shared-address-space abstraction and why every multi-core program silently depends on it.
  • Distinguish UMA (one pool, uniform latency) from NUMA (partitioned local/remote memory, non-uniform latency), and define ccNUMA.
  • Show that the same shared address returns the same value in both topologies — only the latency differs.
  • Separate three orthogonal axes: topology (latency), coherency (per-address correctness), and consistency/ordering (cross-address correctness).
  • Identify coherent versus non-coherent regions and explain why device/IO registers must never be cached.

2. Why Should I Learn This?

In 1.1 you saw two private caches diverge over one shared memory, and 1.2 named coherency as the property that keeps copies agreeing. But what is that shared memory? Multi-core software assumes it is one flat pool where every address holds a single value visible to every core — the cache-coherent shared-address-space abstraction. The hardware that keeps it true at scale is the whole reason CHI exists.

The trap is real: a server team once ported working code from a single-die part to a two-die package and watched throughput collapse on half the threads. They hunted a coherency bug for a week — there was none. Every value was correct; the part was simply NUMA, and those threads were reaching memory on the other die across a slower link. Topology changed the speed, never the values. Confusing the two costs weeks.

3. Core Concept — UMA, NUMA, and the single coherent address space

Zoom out from the two CPUs to the memory system they share. Two questions define it, and they are independent.

Question 1 — the abstraction (correctness). What does software see? A single, flat, cache-coherent address space: one linear range of physical addresses; every address holds exactly one value; a write by any core is eventually visible to every core that reads that address. This is the contract every multi-core program depends on. Coherency (1.1, 1.2) is the mechanism that makes it true even though private caches hold copies.

Question 2 — the topology (performance). How is the physical memory arranged behind that abstraction?

  • UMA (Uniform Memory Access). One memory pool behind a shared interconnect. Every core reaches every address in roughly the same time. Simple and symmetric — the natural model for small core counts. It does not scale: the single pool and shared path saturate as cores are added.
  • NUMA (Non-Uniform Memory Access). Memory is partitioned into nodes; each core has local memory (fast) and remote memory (slower, reached across a die/socket link). It scales bandwidth by giving each node its own memory, at the cost of latency that depends on where the address lives.

The crucial point: NUMA is still one address space. A NUMA system that keeps its caches coherent is called cache-coherent NUMA (ccNUMA) — remote memory is slower to reach, but it is the same shared, coherent address space. Topology changes the cost of an access, never its value.

Coherent vs non-coherent regions. Not all of the map is coherent. Ordinary cacheable RAM is coherent — the fabric tracks copies and keeps them in sync. Device/peripheral registers and some IO regions are non-coherent — reading a UART status register must hit the device, not a cached copy. A real memory map is a mix, and later CHI node types (coherent versus IO home nodes) exist precisely to serve both.

Keep the axes strictly apart. Topology (UMA/NUMA) is how far memory is — latency. Coherency is whether copies of one address agree — per-address correctness. Consistency/ordering (Module 12) is how writes to different addresses are ordered across cores — a separate correctness property. This chapter is about the first two and never confuses either with the third.

4. Engineering Diagram

Left shows UMA with two cores and one shared memory behind one interconnect at equal latency; right shows NUMA with each core having local memory and reaching the remote memory across a slower link, but both present one coherent address spaceCore A (UMA)equal latencyCore B (UMA)equal latencyInterconnectone shared pathShared Memoryone poolCore C (NUMA)fast localCore D (NUMA)fast localLocal Mem CnearLocal Mem Dremote to C12
Figure 1 — the same single coherent address space, two topologies. On the LEFT (UMA) both cores reach one shared memory through one interconnect at equal latency. On the RIGHT (NUMA) memory is split into node-local pools: each core reaches its LOCAL memory quickly and the OTHER node's memory across a slower link. In BOTH cases software sees ONE flat, cache-coherent address space — the same address means the same value everywhere. Topology changes latency; the coherent abstraction is identical.

5. Worked Example — same address space, different latency

Trace a shared buffer at address A, once on UMA and once on NUMA. The value is identical on both; only latency differs.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
# One coherent address space. Address A holds one value for everyone.
# UMA: every access ~1 unit. NUMA: local ~1 unit, remote ~4 units.
 
Step  Actor / access             UMA cost   NUMA cost    Value seen   Note
----  -------------------------  --------   ----------   ----------   ---------------------------
  1   Core A: store A = 42       1          1 (local)    A = 42       write into shared address A
  2   Core A: load  A            1          1 (local)    42           reads back its own write
  3   Core B: load  A            1          4 (remote)   42           SAME value, slower on NUMA
  4   Core B: store A = 43       1          4 (remote)   A = 43       write is visible to all
  5   Core A: load  A            1          4 (remote)   43           SAME value, slower on NUMA
 
# Correctness (which value) is identical on both. Only latency changes.
# NUMA never returns a different value for A — it just costs more to reach A remotely.

The abstraction is airtight: address A holds one value both cores agree on. NUMA changes steps 3 and 5 from cheap to expensive, but never from 42 to something else. That separation — coherent values, non-uniform latency — is the entire mental model of a modern shared-memory SoC.

6. Where This Appears in a Real SoC

The two-CPU picture is the smallest slice of a real coherent system. The same single-address-space abstraction — and the same topology choices — scale up across the chip:

  • CPU clusters address one physical memory map; whether that memory is local or remote to the cluster is the NUMA question.
  • Private L1/L2 and a shared last-level cache (LLC) hold copies of lines from the shared space; coherency keeps those copies agreeing, and the LLC is a natural point where the latest value is tracked.
  • The coherent interconnect routes each access to whichever memory owns that address range and carries the coherency traffic; on NUMA it decides local versus remote hops.
  • DMA, I/O, and accelerators that share buffers with CPUs address the same space; increasingly they participate in coherency to avoid costly copies.
  • Memory controllers each own a slice of the physical map — multiple controllers are exactly what turns a UMA design into a NUMA one.

(Vendor names would be architectural context only — the single coherent address space is universal.)

7. DebugLab — "half the threads got slow after the die split"

1

Half the threads got slow after the die split

NUMA LATENCY ASYMMETRY -> DATA PLACEMENT, NOT A COHERENCY BUG
Symptom

Identical code, identical cores, identical clock. After moving from a single-die part to a two-die package, throughput collapses on the threads pinned to die 1 that hammer a shared buffer. Values are always correct — no wrong reads, no lost updates. Single-die runs are fine.

Root Cause

The part is NUMA, and the hot buffer physically lives on die 0, so die-1 threads pay a remote-access latency on every touch. The single coherent address space is fully intact — the buffer's address means the same value on both dies — but memory is now partitioned: die 0's memory is local to die-0 cores and remote to die-1 cores, reached across the die-to-die link. The software was written for UMA, where every access costs the same, so it never considered where the buffer was allocated. On NUMA a remote access costs several times a local one, and a thread doing millions of remote touches stalls. This is a topology/placement issue: coherency governs which value you get, and that value was always right.

Fix

Place data in the memory local to the core that uses it most, so hot accesses stay on-die. Use NUMA-aware allocation (first-touch or explicit node binding) so each thread's working set sits in its local node; pin threads to the node holding their data. Where sharing is unavoidable, minimize the shared writable footprint and batch remote accesses. Nothing in the coherency fabric needs changing — the abstraction was never broken. The principle: UMA vs NUMA is a latency property, not a correctness one; a NUMA slowdown is fixed by data placement, never by touching coherency. At SoC scale, the fabric that routes each address to its owning memory node while keeping the one coherent space true is exactly CHI (Home Nodes over a mesh, Modules 4 and 11).

8. Common Mistakes

  • Assuming UMA everywhere. Writing multi-core code as if every access costs the same. On NUMA that is a silent performance cliff, not a correctness bug.
  • Thinking NUMA breaks the address space. It does not — ccNUMA is one coherent space. NUMA changes latency, never values.
  • Confusing topology with coherency. UMA/NUMA is where memory is; coherency is whether copies agree. Orthogonal properties.
  • Treating all regions as coherent. Device/IO registers are non-coherent by design — caching a status register is a bug.
  • Believing memory is always the source of truth. With write-back caches (1.4+) the newest value may live in a cache, not in the memory pool — even in a clean UMA topology.

9. Interview Questions

10. Engineering Checklist

  • I can define the single cache-coherent shared-address-space abstraction and why software depends on it.
  • I can distinguish UMA (uniform latency, one pool) from NUMA (local versus remote, partitioned memory).
  • I can state that UMA/NUMA is a latency property, not a correctness one — same values, different speed.
  • I can explain ccNUMA: NUMA that is still one coherent address space.
  • I can identify coherent versus non-coherent regions and why device registers must not be cached.
  • I can separate topology (UMA/NUMA) from coherency (per-address) from consistency/ordering (cross-address).

11. Key Takeaways

  • Software assumes one shared address space. A single flat range where every address holds one value visible to every core — the contract all multi-core code rests on.
  • UMA vs NUMA is topology. UMA = one pool, uniform latency, simple, does not scale. NUMA = partitioned local/remote memory, scalable bandwidth, non-uniform latency.
  • Topology is not correctness. ccNUMA is still one coherent space; NUMA changes speed, never values. A NUMA slowdown is a placement problem, not a coherency bug.
  • Not all memory is coherent. Cacheable RAM is coherent; device/IO registers are deliberately non-coherent — foreshadowing CHI's coherent versus IO node types.
  • Scale is why CHI exists. Preserving one coherent address space across many cores and memory controllers, cheaply, is exactly the job of CHI's Home Nodes, mesh routing, and directories.

12. Quick Revision

Shared memory systems. Software assumes one flat, cache-coherent address space — every address is one value, visible to every core. UMA = one memory pool, uniform latency, simple but does not scale. NUMA = memory split into local (fast) and remote (slow) nodes; ccNUMA keeps it one coherent space. Topology (UMA/NUMA) = latency; coherency = correctness — orthogonal. A NUMA slowdown is fixed by data placement, never by coherency. Some regions (device/IO) are non-coherent on purpose. Keeping one coherent space true across many cores plus memory controllers, cheaply, is exactly what AMBA CHI does (Home Nodes, mesh, directories). Coherency (per-address) is not consistency/ordering (cross-address, Module 12).

13. Coming Next

Next — 1.4 Cache Hierarchy Review. You will add:

  • the levels that sit between a core and this shared memory — private L1, often private or shared L2, and a shared last-level cache — and why each exists,
  • write-through versus write-back and why write-back means the newest value can live in a cache, not in the memory pool,
  • inclusion policy (inclusive / exclusive / non-inclusive) and why it shapes what a coherency fabric must track.

We keep the same project — CPU0, CPU1, private L1s, one shared address space — and fill in the cache levels between the cores and the memory this chapter defined, setting up exactly where copies live before coherency has to reconcile them.