Memory Ordering as a Mental Model

Part I · Module 26 · Chapter 26.8 · Memory Ordering as a Mental Model

1. The Measurement That Makes This Necessary

Two threads. Two atomic variables, both starting at zero. Each thread writes one and then reads the other:

// thread 0            // thread 1
x.store(1, MO);        y.store(1, MO);
r0 = y.load(MO);       r1 = x.load(MO);

Reason about it for a moment. Whichever store happened first, the thread that stored second must see the other's store — so r0 == 0 && r1 == 0 cannot happen. There is no interleaving of those four operations that produces it. That is what sequential consistency means: there exists some single global order of all operations consistent with what every thread saw.

Here is what 200,000 iterations produced, over eleven runs:

  seq_cst : r0==0 && r1==0 observed 0 times in 200000
  relaxed : r0==0 && r1==0 observed … times in 200000
  relaxed + seq_cst FENCE between the store and the load:
            r0==0 && r1==0 observed 0 times in 200000

With relaxed atomics the impossible happened about nine times in ten. Not occasionally, not under exotic conditions — as the normal case. Both threads stored, both read, and both saw zero, because each thread's store was still sitting in its core's store buffer when the other core read.

And the third row is why this is a result rather than an anecdote. The relaxed arm on its own proves very little: a 90% rate that high is exactly what a broken harness looks like, and my first reaction was that I had measured my own bug. So the experiment includes a control — the same relaxed code with a single std::atomic_thread_fence(std::memory_order_seq_cst) inserted between the store and the load. The outcome disappeared completely, in all eleven runs. One instruction, nothing else changed, and an outcome occurring 180,000 times became an outcome occurring zero times. That is the fence doing exactly what it claims, and it is what licenses the conclusion that the other 180,000 were real reordering rather than a counting error.

2. It Was the Hardware, Not the Compiler

There is a second suspect, and skipping the check would have been sloppy: a compiler is allowed to reorder a relaxed store and a relaxed load of a different variable, so the 90% could have been the optimiser rather than the processor.

It was not. The emitted instructions, at -O2:

__Z13relaxed_storel:                    ; @_Z13relaxed_storel
	str	x0, [x8, _c@PAGEOFF]
__Z13release_storel:                    ; @_Z13release_storel
	stlr	x0, [x8]
__Z12seqcst_storel:                     ; @_Z12seqcst_storel
	stlr	x0, [x8]
__Z12relaxed_loadPl:                    ; @_Z12relaxed_loadPl
	ldr	x8, [x8, _c@PAGEOFF]
	str	x8, [x0]
__Z12acquire_loadPl:                    ; @_Z12acquire_loadPl
	ldapr	x8, [x8]
	str	x8, [x0]
__Z11seqcst_loadPl:                     ; @_Z11seqcst_loadPl
	ldar	x8, [x8]
	str	x8, [x0]

The relaxed store is a plain str and the relaxed load a plain ldr, in program order, one after the other, no reordering by the compiler at all. The CPU reordered them at run time: the str entered the store buffer, the ldr executed before it drained, and the other core could not see it yet.

That matters for your mental model. The compiler and the processor are two independent reorderers, and memory_order constrains both. A fence that stops one is not automatically stopping the other, and reading the assembly tells you only about the first. The Chapter 26.7 §4 callout makes the same point from the other side: assembly is evidence about one of the two layers, and it has to agree with a runtime measurement before you trust the conclusion.

3. The Six Orderings, Honestly Ranked

Two of those rows deserve their own sentence.

memory_order_consume should not be used. It was intended to express data-dependent ordering more cheaply than acquire, no implementation ever implemented it as specified, and every real compiler promotes it to acquire. The standard discourages it. Treat it as a historical artefact.

relaxed is not "slightly weaker". It is no ordering whatsoever, and §1 is what that buys you: an outcome that cannot happen under sequential consistency, nine times in ten. It is correct for a statistics counter whose value nobody uses to make a decision, and wrong for essentially everything that coordinates between threads.

4. What the Orderings Cost

This is the section that should change your behaviour, because the usual reason people reach for a weaker ordering is speed. From Chapter 26.7 §3, an atomic increment, median of eleven runs:

Identical medians and identical ranges. On this machine, weakening a fetch_add from the strongest ordering to the weakest bought nothing measurable at all.

That is not because the compiler ignored the request. The instructions genuinely differ:

__Z11relaxed_addv:                      ; @_Z11relaxed_addv
	ldadd	x9, x8, [x8]
__Z10seqcst_addv:                       ; @_Z10seqcst_addv
	ldaddal	x9, x8, [x8]

ldadd against ldaddal — the same atomic add, with and without acquire-release semantics. Two different instructions, and no measurable difference in an uncontended loop, because the cost of an atomic read-modify-write here is dominated by the cache-line ownership it needs rather than by the ordering it enforces.

Now the two asymmetries from §2's listing, which are the genuinely surprising part:

A seq_cst store and a release store are the same instruction. Both are stlr. So on this machine, weakening a store from seq_cst to release cannot possibly be faster — it is the identical instruction. People weaken stores for performance all the time; here there is nothing to win.

A seq_cst load and an acquire load are different. ldar against ldapr. So whatever extra cost seq_cst carries over acquire-release on this hardware lives in the load, not the store, which is the opposite of what most people guess.

The practical conclusion is uncomfortable for the folklore. Relaxing orderings is the classic micro-optimisation, and it is the one that most reliably introduces bugs that cannot be reproduced. On this machine the measured benefit on a fetch_add was zero and the benefit on a store was structurally zero. Chapter 25.10 §7's rule applies with unusual force: if you cannot measure the improvement, you have taken on the risk for nothing — and here the risk is a 90%-probability wrong answer that your tests will not show you, because they run on one architecture.

5. happens-before, Which Is the Whole Model

Everything above is mechanism. This is the model, and it is the thing to actually remember.

Thread B is guaranteed to see thread A's write if and only if that write happens-before B's read. That is the only guarantee the language gives. No ordering keyword, cache-line argument or fence changes that sentence; they are all just ways of establishing the relation.

happens-before is built from two pieces:

Within one thread, program order. Statement one happens-before statement two. This is why single-threaded reasoning works at all.

Between threads, a synchronises-with edge. A release operation on an object synchronises-with an acquire operation on the same object that reads the value the release wrote. That edge, plus program order on each side, is what connects two threads' timelines.

Put together, the canonical shape:

// thread A -- publish
data = computeEverything();              // (1) ordinary, non-atomic write
ready.store(true, std::memory_order_release);   // (2) release

// thread B -- consume
if (ready.load(std::memory_order_acquire))      // (3) acquire, reads true
    use(data);                           // (4) sees (1), guaranteed

(1) happens-before (2) by program order. (2) synchronises-with (3) because the acquire read the value the release wrote. (3) happens-before (4) by program order. The relation is transitive, so (1) happens-before (4) — and data need not be atomic at all, because there is no longer a data race on it: the accesses are ordered.

Three consequences worth stating explicitly.

Release and acquire are useless alone. A release with no matching acquire on the same object establishes nothing. They are a pair, and the pairing is what people get wrong when they sprinkle release on a store and read it with relaxed.

A mutex gives you this for free, and that is what Chapter 26.4 §1 promised to formalise. Unlocking a mutex is a release operation; locking it is an acquire. So everything a thread did before unlocking happens-before everything the next thread does after locking, and the data you wrote inside the critical section is visible to the next holder. That is the whole of why a mutex protects data and not merely code — the mutual exclusion stops the overlap, and the happens-before edge makes the writes visible. Chapter 26.4 §6's callout argued the same point from the design side.

Thread creation and join are edges too. Everything before std::thread's constructor happens-before the new thread's first statement; everything the thread does happens-before join() returns. That is why Chapter 26.2's examples needed no atomics around data handed to a thread and collected after joining it, and it is the most-used synchronisation in practice precisely because nobody has to think about it.

6. Release and Acquire, in Practice

Where release/acquire genuinely earns its place is the publish-once pattern in §5: one thread prepares something, then flips a flag; others check the flag and read what was prepared. Two rules make it safe.

Publish with release on the store and consume with acquire on the load of the same atomic. Mismatch them and you have nothing: a relaxed load of a release-stored flag establishes no edge, so the data it announces may not be visible. This is the single commonest memory-ordering bug, and §1's measurement is what it looks like when it bites.

Never relax the flag "because it is just a bool". The flag is not carrying its own value; it is carrying the ordering for everything else. Chapter 26.7 §4 measured the plain-bool version of this mistake — the compiler deleted the loop reading it — and a relaxed atomic is the subtler relative: the loop survives, the value arrives, and the data it refers to may not have.

The honest caveat is that the version with seq_cst on both sides is also correct, is the default, and is what you should write unless a profile points at the flag. §4 is why: the store is the same instruction either way.

7. What relaxed Is Actually For

There is one clean use and it is worth knowing precisely, because it is the case where relaxed is not merely tolerable but right:

std::atomic<long> eventsProcessed{0};
// ... in the hot loop, from many threads:
eventsProcessed.fetch_add(1, std::memory_order_relaxed);
// ... much later, after joining every thread:
report(eventsProcessed.load());

A counter nobody reads to make a decision while the threads are running. Every increment is atomic, so no count is lost; nothing is ordered against anything, which is fine because no other data depends on the value. The final read happens after join, which supplies its own happens-before edge per §5.

The failure mode to recognise is the counter that is used for a decision — if (queueDepth.load(relaxed) > 0) take(); — which is Chapter 26.7 §8's two-operations-with-a-gap problem compounded by having no ordering either.

And the thing to notice about even the good case: §4 measured no speed benefit from relaxed on a fetch_add here. Which means the honest reason to write relaxed on a counter is to document that no ordering is intended, not to go faster.

8. Fences

std::atomic_thread_fence(order) orders operations without being attached to any particular variable. §1's control arm is the canonical demonstration: one seq_cst fence between a relaxed store and a relaxed load eliminated an outcome that was otherwise occurring 180,000 times in 200,000.

When a fence is the right tool: when you want one ordering point covering several variables, rather than an ordering attached to each. When it is not: almost always, because an ordering attached to the operation that needs it is easier to reason about and easier to review. A fence is action at a distance; release on the store that publishes is local and obvious.

Worth distinguishing from the thing it is often confused with. std::atomic_signal_fence only constrains the compiler, not the hardware, and exists for signal handlers running on the same thread. It is not a cheaper thread fence, and using it as one produces code that works until it does not.

9. Why Your relaxed Bug Will Not Reproduce

This is the portability point, and it is the practical reason the chapter is marked Expert.

Everything measured here is an Apple M3 — arm64, which is weakly ordered, so the store-buffer reordering in §1 is observable about nine times in ten. x86-64 is documented to implement a substantially stronger model (total store order), in which this particular reordering of a store followed by a load of a different location is the one relaxation it permits, and most other reorderings are not visible at all. I have not measured x86-64 — this machine does not have one — so I am reporting that as architecture documentation rather than as an experiment, which is the same discipline Chapter 26.5 §10 used for the sanitizer.

The consequence does not depend on the details. A memory-ordering bug is architecture-dependent, so the architecture you test on decides whether you see it. Code with missing or wrong orderings can be developed, reviewed, tested and deployed on one architecture for years and fail the week it is built for another — and Chapter 26.3 §2's argument applies in full: it was undefined behaviour the whole time, and the working builds were not evidence.

Which gives the rule that this chapter ultimately exists to justify: write seq_cst — the default — and leave it alone. It is the only ordering whose behaviour is the same everywhere, it costs nothing measurable on the operations tested here, and it is the one that does not turn a port into a debugging project.

10. The Table

11. What to Actually Do

• Leave seq_cst in place. It is the default, it is the same everywhere, and §4 found no measurable cost on the operations tested.
• Use release/acquire as a matched pair on the same atomic for the publish-once pattern, and nowhere else without a reason.
• Never relax the flag that announces other data. It is carrying the ordering, not the value.
• Use relaxed only for a counter nobody reads while the threads run — and write it to document the intent, not to go faster.
• Never use memory_order_consume.
• Prefer an ordering on the operation over a free-standing fence. A fence is action at a distance.
• Do not confuse atomic_signal_fence with atomic_thread_fence. The first constrains only the compiler.
• Reason in happens-before, not in caches, store buffers or instruction timings. The relation is the only guarantee.
• Remember that a mutex, a thread construction and a join all give you the edge for free — most code needs no explicit ordering at all.
• Treat any ordering weaker than the default as a change needing a measurement, and remember that your tests run on one architecture.
• Put a control in any reordering experiment. The relaxed arm alone proves nothing; the fence arm is what made §1 a result.

12. Mistakes Worth Making Once

Assuming writes become visible in program order. 180,823 of 200,000 say otherwise.

Believing the 90% is a broken harness. It would have been, without the fence arm; the control is what settles it.

Blaming the compiler for a reordering. The asm showed str then ldr, in order; the CPU did it.

Relaxing a store for speed. It is stlr either way.

Relaxing a fetch_add for speed. Measured identical: 1.88 ns, same range.

A release store read by a relaxed load. No edge, no guarantee, no symptom until it matters.

Guessing that seq_cst's extra cost is in the store. It is in the load — ldar against ldapr.

Using memory_order_consume. It is not implemented as specified anywhere.

Reaching for a fence instead of an ordering on the operation. Harder to review, no benefit.

Concluding your ordering is right because the tests pass on x86. That architecture hides most of these.

Reasoning in store buffers rather than happens-before. The hardware is an explanation; the relation is the contract.

13. Interview Relevance

The frozen curriculum marks this High and Expert, which is a fair description: it is rarely the first question and it is often the last one, used to find the ceiling of a candidate's understanding. Three things answer it well.

Lead with happens-before, not with the enumeration. The complete answer to "what does memory_order do?" is that the language guarantees thread B sees thread A's write exactly when that write happens-before the read, that happens-before is built from program order within a thread plus release/acquire pairs between threads, and that the keywords are simply the ways to create those edges. Saying that a mutex's unlock is a release and its lock an acquire — and that this is why a mutex protects data rather than code — shows the model is connected to the code you have actually written.

Then the measurement, because it is memorable and it is a real experiment. A store-buffer litmus produced an outcome that sequential consistency forbids in 180,823 of 200,000 iterations with relaxed atomics, 0 of 200,000 with seq_cst, and 0 of 200,000 when a single fence was added to the relaxed version. Volunteering that third arm — the control, and the reason the result is trustworthy — is worth more than the number.

Then the cost, which contradicts the folklore. fetch_add measured 1.88 ns at both relaxed and seq_cst, with identical ranges, and a seq_cst store compiles to the same instruction as a release store on arm64. So the standard micro-optimisation of weakening orderings bought nothing here while introducing an architecture-dependent bug class. The closing line to have ready is that seq_cst is the default for good reasons and that the right answer in production code is almost always to leave it alone.

14. What You Should Be Able to Say Now

• That two threads can disagree about the order of operations, and how often: an outcome forbidden by sequential consistency occurred in 180,823 of 200,000 iterations [113,691–198,665] with relaxed atomics.
• That seq_cst produced it 0 times in all 11 runs, and that adding one fence to the relaxed version also produced 0 — the control that makes the result trustworthy.
• That the compiler did not reorder anything: the assembly shows str then ldr in program order, so the processor did it, and memory_order constrains both layers.
• That relaxed means atomicity with no ordering, and that consume should never be used.
• That weakening a fetch_add measured no difference (1.88 ns, same range), that a seq_cst store is the same instruction as a release store (stlr), and that seq_cst's extra cost on this machine is in the load (ldar against ldapr).
• That the real model is happens-before: program order within a thread, plus a release that synchronises-with an acquire reading its value, plus transitivity.
• That this is why data need not be atomic in the publish-once pattern — the edge removes the race.
• That a mutex's unlock is a release and its lock an acquire, which is why a mutex protects data; and that thread construction and join are edges too.
• That release and acquire are useless unpaired, and that relaxing the flag that announces other data is the commonest ordering bug.
• That these bugs are architecture-dependent — this was arm64, which is weakly ordered, while x86-64 is documented to be stronger — so passing tests on one architecture is not evidence.

15. Exercises

Exercise 1 — the litmus test. Build §1's experiment with a barrier between iterations and 200,000 iterations per arm. Run all three arms eleven times.

Exercise 2 — the control. Remove the fence arm and convince yourself that the remaining two rows would not have persuaded a sceptical reviewer. Then put it back.

Exercise 3 — rule out the compiler. Dump the assembly for a relaxed store followed by a relaxed load and confirm they are emitted in program order.

Exercise 4 — the instructions. Compile a load and a store at relaxed, acquire/release and seq_cst and tabulate the instructions. Find the two that are identical and the two that are not.

Exercise 5 — the cost. Measure fetch_add at relaxed and seq_cst, median of eleven runs with ranges. Decide whether the ranges separate before claiming a difference.

Exercise 6 — publish once. Write §5's pattern with release/acquire and a non-atomic payload. Then weaken the consuming load to relaxed and explain precisely what guarantee you lost, even if the program still works.

Exercise 7 — the mutex edge. Write the same publish-once pattern using a mutex and no atomics, and explain which operation is the release and which the acquire.

16. Summary

• Memory is not a place where writes appear in program order. A store-buffer litmus test produced r0 == 0 && r1 == 0 — forbidden under sequential consistency — in a median 180,823 of 200,000 iterations (90.4%, range 113,691–198,665) using relaxed atomics.
• seq_cst produced it zero times in all eleven runs, and so did the relaxed version with one seq_cst fence added. That fence arm is the control, and it is what makes the 90% a measurement of hardware reordering rather than of my own harness.
• The compiler was not the culprit. It emitted a plain str followed by a plain ldr, in program order. The processor reordered them, and memory_order constrains compiler and hardware both.
• Six orderings exist; three matter. seq_cst is the default and should stay; release/acquire are a matched pair for publishing data; relaxed is atomicity with no ordering; consume must never be used.
• Weakening orderings bought nothing measurable here. fetch_add was 1.88 ns at both extremes with identical ranges, a seq_cst store is the same instruction as a release store (stlr), and the only asymmetry is in the load (ldar against ldapr) — the opposite of the common guess.
• The model is happens-before, built from program order plus release-synchronises-with-acquire, and transitive. It is the only guarantee the language makes, and it explains why the published payload need not be atomic.
• A mutex gives it for free: unlock is a release, lock is an acquire — which is the formal reason a mutex protects data rather than code. Thread construction and join are edges too, which is why most code needs no explicit ordering.
• And these bugs are architecture-dependent. This was arm64, weakly ordered; x86-64 is documented to be stronger, and I did not measure it. So tests passing on one architecture say nothing about the other, which is the whole argument for leaving the default in place.

Chapter 26.9 closes the module with the practical question all of this has been leading to: how you debug a concurrent program when the bug does not reproduce.