Atomics

Part I · Module 26 · Chapter 26.7 · Atomics

1. What std::atomic Guarantees

#include <atomic>

std::atomic<long> processed{0};

void worker()
{
    for (const auto& job : myJobs)
        { run(job); processed.fetch_add(1, std::memory_order_relaxed); }
}

std::atomic<T> gives you three things that a plain T does not.

Indivisibility. A read-modify-write such as fetch_add happens as one unit. No other thread can observe a half-finished increment, and no increment is lost. That is the fix for Chapter 26.3's lost updates.

Freedom from the data race. Accessing a std::atomic from two threads is not a data race, so the program is not undefined behaviour. This is the clause that matters most and the one most often forgotten: an atomic does not merely make the arithmetic come out right, it makes the program defined.

An ordering, which you choose. Atomic operations take a std::memory_order argument that says how strongly they are ordered against surrounding operations. The default — seq_cst — is the strongest and the one to use until you can prove you need less. Chapter 26.8 is that subject in full.

What it does not give you is a way to make two variables consistent with each other. Two atomics are two indivisible operations, not one; §8 is the limit, and it is the limit that sends most problems back to a mutex.

2. volatile Is Not Atomic

This deserves its own section because the mistake is so common, and because Chapter 26.3 §3 measured it. Four threads, 200,000 increments each, expecting 800,000:

A. four threads, 200000 increments each, expecting 800000
   plain long                   got  800000 of  800000  
   volatile long                got  …     of  800000  <-- LOST UPDATES
   atomic<long> relaxed         got  800000 of  800000

Over eleven runs the volatile long produced a median of 326,849 of 800,000, ranging from 249,186 to 354,556 — it lost between 56% and 69% of its increments. The atomic was correct every run.

volatile means "do not optimise this access away". It says nothing about indivisibility: ++vol is still a load, an add and a store, and two threads interleave those freely. It also says nothing about ordering between different variables, which is what a concurrent algorithm needs.

So the honest summary is that volatile solves a different problem — memory-mapped hardware registers, where every read must actually happen — and solves none of the concurrency ones. C++20 deprecated ++ on a volatile scalar, which is the committee saying the same thing:

race.cpp:16:54: warning: increment of object of volatile-qualified type 'volatile long' is deprecated [-Wdeprecated-volatile]

And yet volatile is not useless for the flag in §4, which is the confusing part — it keeps the load in the loop, which is enough for a single boolean on this hardware and not enough in general. §4 measures both.

3. What It Costs

100,000 increments of one counter, minimum of 31 repetitions per run, median of eleven runs:

Three readings.

An atomic increment costs about 1.9 nanoseconds — roughly seven unprotected increments, and about 2.4× cheaper than a mutex. For a counter, a flag or a pointer, that is the reason to prefer it.

relaxed and seq_cst are indistinguishable here, with the same median and the same range over eleven runs. That is a measurement about this hardware, not a general truth: the instructions genuinely differ, and the next chapter shows the assembly. It is worth internalising anyway, because the usual reason people reach for relaxed is performance, and on this machine there was none to win on a fetch_add.

And an atomic is still about seven times a plain increment, which matters when the counter is hot. The fix is not a weaker ordering; it is fewer atomic operations — a per-thread count summed at the end, which is §9.

For scale, Chapter 26.2 §7 measured starting and joining a thread at about 14.4 µs, which is roughly 7,600 atomic increments.

4. The Flag, and Why a Plain bool Is Not One

Here is the most common concurrency idiom in working code — a worker that stops when told:

bool done = false;                  // WRONG

void worker()  { while (!done) { step(); } }
void stopIt()  { done = true; }

It is a data race by Chapter 26.3 §1's definition, so it is undefined behaviour, and the usual expectation is that it will "probably work" or at worst be slow to notice. Both expectations are wrong, and the measurement is startling. A waiter spins on a flag that another thread sets after 100 ms; seven runs:

A. spinning on a flag another thread sets after 100 ms
   plain bool             noticed after    …  ms
   volatile bool          noticed after  …  ms
   atomic<bool>           noticed after  …  ms

Read the first row again. The flag is written at 100 ms. The plain-bool waiter reported noticing it after 1.5 ms — about a hundred milliseconds before it was set. It did not wait at all, and it did not hang either. It claimed success on a condition that was false.

The assembly settles the mechanism. The three waiters at -O2, with the setters in the same translation unit so the compiler cannot assume nobody writes them:

__Z9waitPlainv:                         ; @_Z9waitPlainv
	ret
__Z7waitVolv:                           ; @_Z7waitVolv
	adrp	x8, _doneVol@PAGE
LBB4_1:                                 ; =>This Inner Loop Header: Depth=1
	ldrb	w9, [x8, _doneVol@PAGEOFF]
	cmp	w9, #1
	b.ne	LBB4_1
	ret
__Z8waitAtomv:                          ; @_Z8waitAtomv
	adrp	x8, _doneAtom@PAGE
	add	x8, x8, _doneAtom@PAGEOFF
LBB5_1:                                 ; =>This Inner Loop Header: Depth=1
	ldarb	w9, [x8]
	cmp	w9, #1
	b.ne	LBB5_1
	ret

waitPlain is one instruction: ret. The loop is not slow, not stale, not unlucky — it is gone. The function never reads memory at all. The compiler is entitled to assume no other thread writes a non-atomic variable, which makes the condition loop-invariant, which makes the loop either infinite or absent, and it chose absent.

waitVol keeps a genuine loop — ldrb, cmp, b.ne — because volatile forbids removing the load. waitAtom keeps a loop too, with ldarb: a load-acquire, which additionally orders everything after it, so the work the other thread did before setting the flag is visible once you see it.

Three conclusions, and the third is the one people miss.

std::atomic<bool> is the correct type for a flag. Not for speed — it measured the same 108 ms as volatile — but because it is defined behaviour and because ldarb gives you the ordering as well as the value.

volatile bool happens to work for this one case and is still the wrong answer. It kept the load, which is all a single boolean on this hardware needs, and it is exactly how volatile earned its undeserved reputation as a threading tool. It gives you no ordering, so the moment the flag is a signal that other data is ready — which is what a done flag almost always is — the data may not be visible even though the flag is. §2's measurement is the same keyword failing the other half of the job.

And a plain bool fails in the direction nobody predicts. Asked what goes wrong, most engineers say "it might not see the update for a while". The measured answer is that the check was deleted and the loop reported success immediately. Chapter 25.1 §2's point exactly: undefined behaviour is not a wrong value, it is the compiler reasoning from a premise you violated.

5. Is It Lock-Free?

std::atomic<T> works for any trivially copyable T, but for large types the library implements it with a hidden lock. You can ask:

D. is it lock-free?
   atomic<int>       (4 bytes) lock_free = 1
   atomic<long long> (8 bytes) lock_free = 1
   atomic<16-byte>   (16 bytes) lock_free = 1
   atomic<32-byte>   (32 bytes) lock_free = 0

On this machine the boundary is between 16 and 32 bytes: up to 16 bytes you get real atomic instructions; at 32 bytes you get a mutex you cannot see.

Why it matters.

It is not signal-safe or async-safe if it locks, and a lock inside what looks like a lock-free operation can deadlock in a signal handler.

The cost is not what you measured on the small one. §3's 1.88 ns is for an 8-byte counter. A 32-byte atomic pays Chapter 26.4 §3's 4.57 ns and more.

And it silently changes the contention story. A 32-byte atomic in a tight loop is a mutex, so §9's collapse applies with the extra cost of pretending not to be one.

Check with is_lock_free() at runtime or is_always_lock_free at compile time — the latter is a constexpr bool, so it can go in a static_assert, which is how you stop a type from quietly crossing the boundary when someone adds a field:

static_assert(std::atomic<Snapshot>::is_always_lock_free,
              "Snapshot grew past the lock-free size");

Chapter 26.3 §6 approached the same boundary from the other side: it looked for tearing in plain 8- and 16-byte structs and found none in millions of reads — which is the same hardware fact, that this machine does 16-byte accesses in one piece, and which is why that chapter was careful to call its negative result a negative result rather than a guarantee.

6. The Operations Worth Knowing

Two notes on syntax. The operators work — ++counter, counter += 1, bool b = flag — and all of them use seq_cst, which is the right default. Write the explicit fetch_add(1, std::memory_order_relaxed) only where you mean it.

And there is no atomic multiply, no atomic "increment if positive", no atomic anything-you-can-think-of. The set above is what the hardware provides. Everything else is built from compare_exchange.

7. compare_exchange and the CAS Loop

compare_exchange_weak(expected, desired) is the general-purpose primitive: if the value is still expected, replace it with desired and return true; otherwise write the current value into expected and return false. That is enough to build any read-modify-write operation:

// atomically raise a high-water mark -- there is no fetch_max
void raiseTo(std::atomic<long>& peak, long candidate)
{
    long current = peak.load(std::memory_order_relaxed);
    while (candidate > current &&
           !peak.compare_exchange_weak(current, candidate, std::memory_order_relaxed))
        { }                       // on failure `current` has been refreshed for us
}

Four things about that loop are worth saying out loud, because every one of them is a common bug.

The failure case updates expected for you. That is why the loop needs no explicit reload, and why current must be a variable rather than a temporary.

_weak may fail spuriously — return false when the value did match — so it must always be used in a loop. _strong does not, and is the right choice when there is no loop to put it in; in a loop, _weak is usually faster.

The loop re-tests candidate > current because the value may have changed to something larger, in which case the right action is to stop, not to overwrite. A CAS loop that recomputes blindly can undo another thread's work.

And it can spin arbitrarily long under contention, because every failure means another thread succeeded. That is progress for the system and not for you, and it is why a CAS loop around an expensive computation is usually a worse design than a mutex.

8. What Atomics Cannot Do

An atomic makes one variable indivisible. It does nothing for an invariant spanning two.

std::atomic<long> count{0};
std::atomic<long> sum{0};

void record(long value)            // WRONG: two atomics, two moments
{
    count.fetch_add(1);
    sum.fetch_add(value);          // a reader between these two sees count without sum
}

Each operation is atomic and the pair is not. A reader computing sum / count can see the new count and the old sum and divide by the wrong thing — and no memory ordering fixes it, because the problem is not visibility but that there are two operations where the invariant needs one.

The same applies to anything compound: a container's size and contents, a pointer and the length it describes, two fields of a record. Chapter 26.3 §6's two-field struct is the same shape. If the invariant spans more than one word, use a mutex — that is Chapter 26.4's job and the reason that chapter is marked Critical and this one High.

A related trap is the atomic that is read twice:

if (!queue.empty()) { auto item = queue.front(); }   // not one operation
if (counter.load() > 0) { counter.fetch_sub(1); }    // another thread may have taken it

Two atomic operations with a gap. The second form is the classic; it needs fetch_sub with a check on the returned value, or a CAS loop, or a mutex.

9. Contention: When an Atomic Is Slower Than No Threads

An atomic removes the race. It does not remove the cost of sharing. Chapter 26.1 §3 measured a single shared atomic counter incremented by every thread:

Eight threads were about 7.7× slower than one. The mechanism is cache coherence: the counter's cache line must be exclusively owned by a core to be modified, so it ping-pongs between cores, and every increment pays a transfer. The arithmetic is correct and the program is defined; it is just slow in a way that gets worse as you add hardware.

Two fixes, in order of preference.

Do not share the counter. Give each thread its own and sum at the end. Chapter 26.4 §4 measured the equivalent for mutexes and found per-operation cost falling from 4.80 ns to 1.20 ns from one thread to eight, because private state actually parallelises.

Pad to a cache line if you must keep them adjacent. An array of per-thread counters packed together suffers false sharing: different counters in one cache line contend as if they were the same variable. alignas(64) — or std::hardware_destructive_interference_size — separates them.

The general rule this module keeps arriving at: the expensive thing is sharing, not the primitive you share with.

10. C++20: wait, notify, and atomic_ref

Three additions worth knowing.

wait / notify_one / notify_all turn an atomic into a thing you can block on, which removes the main reason to pair a flag with a condition variable. Chapter 26.6 §8 measured it: 0.1 ms of CPU for a 200 ms wait, the same as a condition variable and about 2,000× less than spinning. For a single flag it is the simplest correct wait there is, and it is immune to the lost wake-up because wait(false) begins by comparing the current value.

std::atomic_ref<T> applies atomic operations to an object you do not own and cannot change the type of — an element of a large array, or a field in a struct from someone else's API. It is how you make some accesses atomic without declaring the whole array atomic.

std::atomic<std::shared_ptr<T>> is a proper specialisation, which matters because the old free functions on shared_ptr were deprecated. It is the honest way to publish a new immutable snapshot to many readers, and it is not lock-free — see §5.

All of these were confirmed to compile and run on this toolchain, alongside latch, barrier and counting_semaphore, in Chapter 26.6 §8.

11. The Table

12. What to Actually Do

• Use std::atomic when the shared thing is exactly one word — a counter, a flag, a pointer, a sequence number.
• Use a mutex the moment the invariant spans two variables. No ordering argument rescues two atomic operations.
• Never use volatile for threading. It lost most of its increments, and it gives no ordering.
• Never use a plain bool as a flag. The compiler deleted the loop that read it.
• Leave the default seq_cst in place until you have measured a reason to weaken it; here there was none for fetch_add.
• Prefer std::atomic<bool>::wait over a spin, and over a condition variable when the condition really is one flag.
• static_assert(std::atomic<T>::is_always_lock_free) on any type you expect to be lock-free, so a new field cannot cross the boundary quietly.
• Always put compare_exchange_weak in a loop, let it refresh expected for you, and re-test your precondition inside the loop.
• Do not share a hot counter. Count per thread and sum at the end; pad per-thread state to a cache line.
• When reading a disassembly, make the toy match the real program — linkage, visible writers, side effects — and check it against a runtime measurement.

13. Mistakes Worth Making Once

volatile long as a shared counter. Lost a median 326,849 of 800,000 increments.

bool done as a stop flag. The waiter reported success 100 ms before the flag was set.

Concluding the plain flag will "eventually notice". It never reads the variable at all; §4's ret.

volatile bool as a stop flag. It works for the boolean and gives no ordering, so the data the flag announces may not be visible.

Two fetch_adds maintaining one invariant. A reader sees the new count with the old sum.

if (counter.load() > 0) counter.fetch_sub(1); Two operations with a gap.

compare_exchange_weak outside a loop. It is allowed to fail for no reason.

Reloading expected by hand in a CAS loop. The failure path already did it.

A 32-byte atomic in a hot loop. It is a hidden mutex; is_lock_free reported 0.

An array of per-thread counters with no padding. False sharing makes private state behave like shared state.

Eight threads on one atomic counter. 0.13× — slower than a single thread.

Reading assembly from a cut-down file. The static version emitted a trap and deleted the atomic loop too; it described a different program.

14. Interview Relevance

The frozen curriculum marks this High. The reliable opening question is the difference between volatile and std::atomic, and the complete answer has two halves: volatile only prevents the access being optimised away, giving neither indivisibility nor ordering, while std::atomic gives all three and — the clause people omit — makes the program defined rather than merely correct-looking. Quoting the measurement is cheap and convincing: four threads incrementing a volatile long lost a median 326,849 of 800,000 increments, while the atomic was right every run.

The question that separates candidates is the done flag, because almost everyone has written bool done. The strong answer says it is a data race and therefore undefined, and then says what actually happened: the compiler is entitled to assume no other thread writes a non-atomic variable, so the loop condition is invariant, so the loop can be removed — and measured, the waiting thread reported noticing the flag 1.5 ms in when the flag was written at 100 ms, with the assembly for that function being the single instruction ret. That is a much better answer than "it might be slow to see the update", which is the common one and is wrong about the direction of the failure.

Beyond that, three facts carry their weight: an atomic increment at 1.88 ns against 4.57 ns for a mutex and 0.28 ns unprotected; that atomics cannot maintain an invariant across two variables, which is when you go back to a mutex; and that a single shared atomic counter measured 0.13× on eight threads, so the cure for contention is per-thread state rather than a weaker memory order.

15. What You Should Be Able to Say Now

• That std::atomic gives indivisibility, freedom from the data race, and an ordering you choose — and that the middle one makes the program defined, not just lucky.
• That volatile gives none of those for threading: measured, it lost a median 326,849 of 800,000 increments, and ++ on a volatile scalar is deprecated in C++20.
• That an atomic increment costs 1.88 ns [1.88–1.93], 6.71× a plain increment and about 2.4× cheaper than a mutex.
• That relaxed and seq_cst had the same median and the same range here, so weakening the order bought nothing on a fetch_add.
• That a plain bool flag is not slow but absent: the function compiled to a single ret and reported success at 1.5 ms for a flag set at 100 ms.
• That volatile bool keeps the load (ldrb) and still gives no ordering, while atomic<bool> compiles to ldarb — a load-acquire — which is why it is the right type.
• That not every atomic is lock-free: 16 bytes yes, 32 bytes no, checkable with is_always_lock_free in a static_assert.
• That compare_exchange_weak belongs in a loop, refreshes expected on failure, and must re-test the precondition.
• That atomics cannot maintain an invariant over two variables, and that a mutex is the answer when they must.
• That one shared atomic counter measured 0.13× on eight threads, and that the fix is per-thread state padded to a cache line — the expensive thing is sharing, not the primitive.
• That a disassembly is evidence only about the program you actually compiled, and must agree with a runtime measurement before you believe it.

16. Exercises

Exercise 1 — the three counters. Reproduce §2's table with plain, volatile and atomic counters over eleven runs. Confirm the plain one is correct and the volatile one is not.

Exercise 2 — the flag. Reproduce §4's three waiters with a 2-second deadline and a detached waiter. Confirm the plain-bool arm reports noticing before the flag is set.

Exercise 3 — the assembly, done properly. Dump the three waiters at -O2. Then repeat with the variables static and no writer in the file, and explain why the answers differ. Confirm each against the runtime behaviour.

Exercise 4 — the boundary. Print is_lock_free() for atomics of 4, 8, 16 and 32 bytes, then add a field to a 16-byte type and watch a static_assert catch it.

Exercise 5 — fetch_max. Implement §7's raiseTo with compare_exchange_weak. Then break it by recomputing blindly instead of re-testing, and show it losing another thread's larger value.

Exercise 6 — two variables. Build §8's count and sum pair, have a reader compute the mean continuously, and record how often it sees an inconsistent pair. Fix it with a mutex.

Exercise 7 — contention and padding. Measure one shared atomic counter at 1, 2, 4 and 8 threads, then an unpadded array of per-thread counters, then an alignas(64) array. Three different answers.

17. Summary

• std::atomic<T> gives indivisibility, no data race, and a memory ordering. The second clause is the one that matters most: it makes the program defined.
• volatile is not a threading tool. Measured, a volatile long lost a median 326,849 of 800,000 increments while the atomic was correct every run; C++20 deprecated ++ on volatile scalars.
• Cost: 1.88 ns [1.88–1.93] per atomic increment — 6.71× a plain increment, about 2.4× cheaper than a mutex's 4.57 ns, and about 1/7,600 of starting a thread.
• relaxed and seq_cst measured identically here, in median and in range, so there was no performance argument for weakening the order.
• A plain bool is not a flag, and it fails in the unexpected direction. The waiter reported success after 1.5 ms for a flag written at 100 ms, because the function compiled to a single ret — the loop, and the memory read, were deleted.
• volatile bool keeps the load and gives no ordering; atomic<bool> compiles to ldarb, a load-acquire, and is the correct type.
• Not every atomic is lock-free: 4, 8 and 16 bytes were; 32 bytes was not. Pin it with static_assert(std::atomic<T>::is_always_lock_free).
• compare_exchange_weak builds every operation the hardware lacks, must live in a loop, refreshes expected for you, and needs its precondition re-tested inside the loop.
• Atomics cannot protect an invariant across two variables. Two atomic operations are two moments; use a mutex.
• Contention is the real cost: one shared atomic counter measured 0.13× on eight threads, slower than one. Count per thread, pad to a cache line, and remember that the expensive thing is sharing.

Chapter 26.8 is the memory_order argument this chapter kept defaulting — and why relaxed and seq_cst compiled to different instructions while measuring the same.