AMBA CHI · Module 1 · Cache Coherency Foundations

The Write Visibility Problem

A store executing on one core does not make its new value visible to other cores at that instant. The write must first travel to a single agreed place, the point of serialization, before any other core can observe it. Until it reaches that point the write is real inside the writer yet invisible everywhere else, so a consumer core can spin for thousands of cycles on a flag that was written long ago. This chapter defines write propagation and the exact moment of global visibility, traces a write that stays invisible until it commits to a shared serialization point, and foreshadows that point as the CHI Home Node.

Foundation11 min readAMBA CHICache CoherencyWrite VisibilitySerialization PointHome Node

Module 1 · Chapter 1.6 · Cache Coherency Foundations

Project thread — same running example as the whole CHI series: two CPUs (CPU0, CPU1), each with a private L1, over one shared memory. Chapters 1.1–1.5 established what a cache is, why coherency is needed, and that a write can leave another core reading stale data. Here we sharpen that into a timing question and introduce the serialization point: after CPU0 stores, exactly when can CPU1 see it?

1. Learning Outcomes

By the end of this lesson you will be able to:

Distinguish the local completion of a store (it retires into the writer's own cache or store buffer) from its global visibility to every other core.
Define the point of serialization (point-of-coherence) and explain why a write becomes globally visible at the instant it is ordered there.
Trace the write visibility window — the span between store execution and ordering at the serialization point — where the value is real in the writer yet stale everywhere else.
Identify the future CHI Home Node as the serialization point for an address, and separate write visibility (this chapter) from ordering and consistency (Module 12).

2. Why Should I Learn This?

Chapter 1.5 showed that a write can fail to reach another core. This chapter answers the harder question every coherency protocol is built around: when does a write become visible, and where must it travel for that to happen. Get this and the Home Node stops being jargon.

Executing a store is not the same as being visible. A core can retire a store into its own cache or store buffer while every other core still reads the old value — the write exists but has not propagated.
Visibility happens at a place, not just a time. A write becomes globally visible when it reaches a single agreed point every core observes through: the point of serialization (point-of-coherence).
That point is what CHI centralises. In CHI the serialization point for an address is the Home Node (Module 4); every coherent write to a line is ordered there, and that is the moment it becomes visible to all.

Real silicon: on a dual-core SoC a producer wrote ready = 1 and moved on while the consumer spun on its own cached copy of ready for thousands of cycles, still reading 0. Adding volatile and more polling changed nothing — the store simply had no defined moment of global visibility.

3. Core Concept — write propagation and the point of serialization

Start from the running project: CPU0 and CPU1, each with a private L1, over one shared memory. Ask the precise question: after CPU0 executes a store to address A, at what instant can CPU1 observe the new value?

A store has two distinct events.

Local completion. CPU0's store retires — the new value lands in CPU0's own cached copy (and possibly a store buffer). CPU0 considers the write done. Nothing about this event touches CPU1.
Global visibility (propagation). The write reaches a place every core observes through, and from that instant any read on any core sees the new value. This is a strictly later event than local completion.

The gap between these two events is the write visibility window — the time during which the value is real inside the writer but stale everywhere else. Every stale-data bug from 1.5 is really a write that never closed this window.

The point of serialization.

To give "globally visible" a single, unambiguous meaning, the system needs one agreed place per address that every write passes through and every read is ordered against. This is the point of serialization, also called the point-of-coherence.
A write becomes globally visible at the moment it is ordered at the serialization point — not when the store executes, and not when it later reaches memory.
Because every write to A is ordered at the same point, all cores agree on a single order of writes to A. That is what makes "the latest value" a well-defined thing.

Foreshadowing CHI. In CHI the serialization point for an address range is the Home Node (HN) (Module 4). A coherent write is ordered at its Home Node, which knows who else holds the line and makes the write visible to them. For now, hold the abstraction: there exists one point per address where writes become globally visible.

4. Engineering Diagram

Figure 1 — write propagation to the point of serialization. CPU0 executes a store to A; the value lands LOCALLY in CPU0's private L1 (write complete for CPU0) but is not yet visible to CPU1, which still holds an OLD copy. The write becomes GLOBALLY VISIBLE only when it reaches the point of serialization (the point-of-coherence). Every core observes writes to A through this one point, so all agree on the latest value. In CHI this serialization point is the Home Node for the address.

The same picture as a hand-off in time — a writer, a serialization point, and the core that must be reached:

Figure 2 — how a write becomes visible. CPU0 executes a store to ready and completes it LOCALLY in its own cache (invisible to others). The write then propagates to the point of serialization, where it is ORDERED — this is the moment of global visibility. From there the serialization point makes the value observable to CPU1, whose next poll returns the new value. The dashed arrow is the local-only completion that does NOT make the write visible; the solid path through the serialization point is what does.

The whole coherency job is compressed into the middle message: ordering the write at the serialization point is the instant it becomes visible to all.

5. Worked Example — a write becomes visible only at the serialization point

Trace a producer flag at the value level. The write exists in CPU0 immediately but is invisible to CPU1 until it reaches the serialization point:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

# Producer/consumer over one shared address "ready". Initial: ready = 0 everywhere.
# "Serial-Point" = the value any core observes THROUGH the point of serialization.
 
Cycle  CPU0 (producer) action        L1(CPU0)   Serial-Point   L1(CPU1)   Note
-----  ----------------------------  ---------  ------------   ---------  ------------------------------
  1    store ready=1 (retires local) ready=1    ready=0        ready=0    executed, NOT yet propagated
  2    (write in flight to SP)       ready=1    ready=0        ready=0    CPU1 polls -> sees 0 (invisible)
  3    (write in flight to SP)       ready=1    ready=0        ready=0    CPU1 polls -> sees 0 (invisible)
  4    write ORDERED at serial point ready=1    ready=1        ready=0    MOMENT OF GLOBAL VISIBILITY
  5    CPU1 poll refetches from SP    ready=1    ready=1        ready=1    CPU1 now sees 1  <-- visible

Between cycle 1 (store executes) and cycle 4 (write ordered at the serialization point) the flag is real in CPU0 yet invisible to CPU1 — that span is the write visibility window. The value did not become globally true when the store executed; it became true when it reached the serialization point at cycle 4. Note that CPU0's own copy is 1 the whole time — local completion says nothing about what any other core can see.

6. Timing Diagram — the write-propagation window

Track the value of ready at the producer, at the serialization point, and at the consumer, cycle by cycle. Watch the visibility window open at the store and close when the write is ordered at the serialization point:

ready propagates to the ordering point — the visibility window closes on ordering

6 cycles

The visual signature of a write-visibility problem: the writer's copy holds the new value while the serialization point (and therefore every other core) still holds the old one — the window between store and ordering.

7. DebugLab — "the consumer spins on a flag the producer already set"

The consumer spins on a flag the producer already set

WRITE COMPLETE LOCALLY BUT NOT PROPAGATED -> NEEDS A SERIALIZATION POINT

Symptom

A producer/consumer hand-off intermittently hangs. CPU0 writes ready = 1 and proceeds; CPU1 spins on ready and keeps reading 0 for thousands of cycles, sometimes forever. Single-core tests pass. Adding volatile and more polling does not help.

Root Cause

The store completed locally in the producer's cache and store buffer but never propagated to a point the consumer observes through, so the new value was real in CPU0 yet invisible to CPU1. Executing the store is a local event; the write only becomes globally visible when it reaches a point of serialization every core observes. Nothing here forced the write out of CPU0's private cache toward such a point, so CPU1 kept hitting its own stale copy of ready. volatile only stops the compiler from caching the value in a register — it cannot make one core's cached write propagate to another. The bug is invisible single-core because there is only one observer; it appears only when a second core must observe a write across the visibility window.

Fix

Give writes a defined point of serialization — a single place a write to an address is ordered and from which it becomes visible to every core. At the system level this is a coherent fabric: when CPU0 writes ready, the fabric orders that write at the address's serialization point and makes it observable to CPU1, so CPU1's next poll returns 1. In CHI that serialization point is the Home Node for the address (Module 4): every coherent write is ordered there, and the Home Node propagates the new value to other holders. The principle: a write is globally visible the instant it is ordered at the point of serialization — not when the store executes — and software cannot substitute for that hardware guarantee. (Stale data is 1.5; read visibility is 1.7; why AXI cannot provide a shared serialization point is 1.8.)

8. Common Mistakes

Believing a store is visible the moment it executes. It is locally complete, not globally visible; visibility happens later, at the serialization point.
Reaching for volatile or more polling. volatile prevents register caching in software; it cannot propagate one core's cached write to another core.
Assuming memory is where visibility happens. With write-back caches the new value may sit in a cache long before memory; the visibility point is the serialization point, not necessarily DRAM.
Confusing visibility with ordering. Visibility is when a single write becomes observable; ordering/consistency is how writes to different addresses are ordered across cores (Module 12).
Thinking every store must be instantly global. Forcing that would destroy performance — which is exactly why a defined visibility point is needed instead.

9. Interview Questions

10. Engineering Checklist

I can distinguish local completion of a store from its global visibility.
I can define the point of serialization (point-of-coherence) and say why visibility happens there.
I can show the write visibility window on a timing diagram and identify the cycle it closes.
I can explain why volatile and more polling do not fix a visibility gap.
I can separate write visibility (per-address, one write) from ordering/consistency (cross-address, Module 12).
I can name the future serialization point in CHI — the Home Node — and what it does at the moment of visibility.

11. Key Takeaways

Executing a store is not visibility. A store first completes locally in the writer's cache or store buffer; that says nothing about other cores.
Visibility happens at a place and a time. A write becomes globally visible the instant it is ordered at the point of serialization for its address.
The window is where bugs live. Between store execution and ordering at the serialization point, the value is real in the writer and stale everywhere else — the write visibility window.
Software cannot fix it. volatile and more polling never propagate one core's cached write; a defined hardware visibility point must.
This is the Home Node's job. In CHI the serialization point for an address is the Home Node, distributed for scale — it orders each write and makes it visible to sharers.
Invariant: no-early-visibility — a store that has only completed locally must not be observable to any other core; the value becomes observable exactly at the instant the write is ordered at the point of serialization, and no earlier.

12. Quick Revision

The write visibility problem. A store executes locally (writer's cache / store buffer) before it has propagated. It becomes globally visible only when it reaches the point of serialization (point-of-coherence) — one agreed place per address every write passes through and every read is ordered against. The gap between execute and ordering is the write visibility window, where a consumer spins on a flag already set. volatile / more polling cannot fix it — visibility is a hardware guarantee. It is not ordering/consistency (cross-address, Module 12). In CHI the serialization point is the Home Node for the address, distributed for scale — it orders the write and makes it visible to sharers.

13. Coming Next

Next — 1.7 The Read Visibility Problem. You will add:

the mirror-image question — when a read is guaranteed to observe the latest write, not just when a write becomes visible,
how a read must be ordered against the serialization point so it cannot slip behind a write it should have seen,
why a read hitting a stale private copy is the same serialization-point gap viewed from the reader's side.

We keep the same project — CPU0, CPU1, private L1s, one shared line, one serialization point — and turn the lens from the writer's moment of visibility (this lesson) to the reader's guarantee of observing it.