Memory-Expansion Semantics

Chapter 9.2 got a load to the device and back. That proves the location is reachable. This chapter asks the harder question: what does the access actually guarantee?

1. The Engineering Problem — Reachable Is Not Usable

Software does not merely require that a load returns a value. It requires a set of guarantees so ingrained that nobody states them:

Every one of those is free in a local DRAM controller and none of them is free across a link. A device that answers loads correctly and provides none of these is reachable and unusable — and worse, it will pass every functional test, because functional tests issue one access at a time.

This chapter is about the guarantees, and each one costs something specific.

2. The One-Sentence Model

A location you can reach is not yet memory. Memory is a set of promises — that accesses to the same address appear in order, that a read-modify-write cannot be split, that a write becomes visible at a defined point, and that a stated set of agents agrees about the value — and each promise costs hardware that pure reachability does not.

Call it the promise set. 9.1 and 9.2 built the plumbing; this chapter is what the plumbing has to guarantee.

3. What This Chapter Owns

4. The Four Promises

Architectural. The arrows are obligations, not named messages.

The whole diagram turns on the second and third arrows being different events. If "accepted" were reported as done, the writer could proceed to the flag while the data was still uncommitted, and the reader would find the flag set and the data stale. §8 builds exactly that failure.

5. Teaching-model boundary

6. RTL 1 — Order the Address, Not the Device

The first promise is the one that costs the most if you get it wrong in either direction.

  // Same address: order. Different address: proceed. That is the contract.
  assign must_order = NO_ORDER     ? 1'b0
                    : (GLOBAL_ORDER ? busy_valid : same_addr);

=== EXP1: same address orders, different addresses do not ===
  different address : must_order=0 proceed=1 | global-order proceed=0
  a different address proceeds concurrently          : ok
  the global-order variant serialised it anyway      : ok
  same address      : must_order=1 proceed=0 | no-order proceed=1
  the same address must wait                         : ok
  the no-order variant let it overtake               : ok
  exactly one proceeded and one was ordered         : ok

Three designs, and only the middle one is memory.

The global-order variant is safe and useless: a memory expander that serialises every access delivers one access per round trip, which 9.2 showed is roughly two orders of magnitude below what the link supports. Correctness that costs all the performance is not a viable answer, which is why the contract is scoped to the address rather than the device.

7. RTL 2 — Atomicity Means Holding the Location

A read-modify-write is atomic only if nothing can observe or change the location between the read and the write. Across a link that window is long.

  assign conflict      = held && other_access && (other_addr == hold_addr_q);
  assign other_blocked = conflict && !NO_LOCK;

=== EXP2: atomicity means holding the location ===
  rmw started : held=1
  the location is held for the sequence              : ok
  other access to the same line : correct blocked=1 | unlocked blocked=0
  a conflicting access is blocked, not interleaved   : ok
  the unlocked variant let it through                : ok
  the hold released when the write completed         : ok
  atomicity held across the sequence                 : ok
  the unlocked variant flagged broken atomicity      : ok

Atomicity cannot be built by the requester out of a separate read and a separate write. The two are individually correct and the gap between them is exactly the window an atomic operation must not have — and across a link that gap is hundreds of nanoseconds rather than a few cycles.

So the location must be held at the point of serialisation, which is why atomic support is a property of the memory system rather than something software can synthesise. Note also that holding costs concurrency: n_blocked_q counts accesses delayed by someone else's atomic sequence, and a workload with heavy contention on one line will show it.

8. RTL 3 — Accepted Is Not Visible

This is the promise most often assumed and least often stated.

  assign wr_reported_done = ACCEPT_IS_VISIBLE ? wr_accept : wr_commit;
  // A probe sees the new value only once the write has actually committed.
  assign rd_sees_new = rd_probe && (rd_addr == pend_addr_q) && !pending_q;

=== EXP3: accepted is not visible ===
  at acceptance : correct reported_done=0 | accept-is-visible reported=1
  acceptance alone does not report completion        : ok
  the broken variant reported done immediately       : ok
  probe before commit : sees_new=0
  a reader does not see it before it commits         : ok
  after commit : sees_new=1 reported_done=1
  once committed the value is visible                : ok
  accept-is-visible variant flagged premature done   : ok

Accepted means the device owns the write and it will not be lost. Visible means another observer reading that address will see it. Those are different events with a real gap between them, and conflating them breaks every producer/consumer handshake in software — the flag becomes visible before the data it guards.

A checker that could not see its own violation

The first version of the premature-visibility check read:

if (wr_reported_done && pending_q && !wr_commit) ...

It never fired. On the acceptance cycle pending_q has not yet been set — it is set by that edge — so the condition was false exactly when the violation occurred. The corrected form drops the term that cannot be true yet:

      // Reporting completion on any cycle that is not the commit cycle. On the
      // correct design reported_done IS wr_commit, so this is unreachable; the
      // accept-is-visible variant trips it on the acceptance cycle, when
      // pending_q has not yet been set and cannot be part of the condition.
      if (wr_reported_done && !wr_commit) begin

A checker that samples state the violating edge is still creating cannot see the violation. This is the same state-versus-transition mistake that has appeared throughout this track, in its most compact form yet.

9. RTL 4 — Coherence Has a Scope

  assign in_scope     = access_valid && (ASSUME_ALL_COHERENT ? 1'b1 : via_host_path);
  assign out_of_scope = access_valid && via_side_path && !ASSUME_ALL_COHERENT;

=== EXP4: coherence has a scope ===
  an access through the host is inside the contract  : ok
  side path : correct in_scope=0 out=1 | assume-all in_scope=1
  a path the host does not mediate is out of scope   : ok
  the assume-all variant claimed it anyway           : ok
  assume-all variant flagged an unsafe assumption    : ok

Architectural. The sourced contract says the host manages coherency, which means the guarantee covers agents reaching the memory through the host's coherence machinery. An agent that reaches the same media by another route — a device-internal engine, a management path — is outside that contract.

The failure this prevents is a design that assumes coherence universally and therefore never asks the question. n_out_q exists so the answer is a number rather than an assumption, and a non-zero value on a system that believed it had none is a design conversation worth having early.

10. RTL 5 — A Read Must Not Overtake an Uncommitted Write

  assign hazard    = rd_valid && pend_valid_q && (rd_addr == pend_addr_q) && !NO_HAZARD;
  // Forward the pending value rather than reading stale media.
  assign forwarded = hazard && !NO_FORWARD;
  assign rd_data   = forwarded ? pend_data_q : media_data;

=== EXP5: a read must not overtake an uncommitted write ===
  read same address : hazard=1 forwarded=1 data=0xCAFE0001
  the pending write was forwarded, not media read   : ok
  and it returned the new value, not the old        : ok
  no-forward variant : data=0xDEADBEEF (media)
  the no-forward variant returned stale media       : ok
  the no-forward variant flagged a stale read        : ok

This is §6's ordering promise made concrete. Detecting the hazard is mandatory; forwarding is the optimisation that makes the promise cheap. Without forwarding the read must stall until the write commits, which on a device with real media latency is expensive and shows up as n_stall_q.

The broken variant returns 0xDEADBEEF — the media's old contents — for an address that was just written. Every counter balances and the value is wrong.

11. RTL 6 — What a Fence Actually Waits For

Architectural, modelling the obligation rather than any CXL-defined mechanism.

  assign fence_done = fence_pending_q &&
                      (RETIRE_EARLY ? 1'b1 : (outstanding_q == 8'd0));

=== EXP6: a fence waits for what came before ===
  3 accepted, 0 committed : outstanding=3
  fence issued : correct done=0 | retire-early done=1
  the fence does not pass with work uncommitted     : ok
  the retire-early variant passed immediately        : ok
  the fence passed once everything committed        : ok
  retire-early variant flagged an early fence pass   : ok

A fence is how software asks for an ordering guarantee it does not get for free. Its cost is exactly n_wait_q — cycles spent waiting for prior work to commit — and on a device with long media latency that cost is substantial. This is why per-address ordering matters so much: it is what lets most programs avoid fences entirely.

The counter defect, for the seventh time

=== EXP6b: accept and commit in the same cycle ===
  4 accepted : outstanding=4
  3 cycles of accept+commit together : outstanding=4
  the case-based counter held steady                 : ok
  and drained to zero when commits caught up         : ok

Two non-blocking assignments to one variable; the second wins; accept-and-commit together decrements. Seventh appearance in this track. Here the consequence is a fence that passes early, because the outstanding count reads zero while work is still in flight — a correctness failure produced by a counter bug.

12. RTL 7 — Semantics Counters

=== EXP7: semantics counters, against an oracle ===
  accepted=36 committed=33 max_uncommitted=3
  ordered=9 forwarded=7 rmw=4 rmw-blocked=3 fence-waits=5
  accept/commit matched an independent oracle       : ok
  nothing committed that was not accepted           : ok
  peak uncommitted was exactly 3                    : ok
  same-address orderings counted exactly (9 of 36)  : ok

These are semantics counters, not throughput counters, and each answers a question a performance counter cannot:

Peak uncommitted is the one worth watching. It is the size of the window in which accepted-but-invisible writes exist, and it bounds how wrong an accept-is-visible design would be.

13. Quantitative Reasoning — What Each Promise Costs

Illustrative, using this chapter's measured teaching run.

Of 36 accesses:

Two conclusions.

Forwarding converted most of the ordering cost into nothing. Seven of the nine same-address cases were satisfied by forwarding rather than by stalling. Without it, each would have waited for a media commit — at 9.1's illustrative 100 ns media latency, that is 700 ns of stall avoided in 36 accesses.

Atomics cost more than they appear to. Four atomic sequences blocked three unrelated accesses — nearly one blocked access per atomic. On a contended line that ratio rises sharply, which is why heavy atomic use on far memory is a placement problem rather than a hardware one.

The ordering trade, in throughput terms

Using 9.1's 125-outstanding figure, per-address ordering is worth up to two orders of magnitude over global ordering while providing the same guarantee to any single-address program. That is the entire reason memory expansion is viable.

14. The Four Promises, as Structures

Read the middle row as a bill of materials. The cheapest promise is ordering — one comparator and a forwarding path — and it is worth up to two orders of magnitude of throughput. The most expensive in practice is atomicity, because a hold blocks other agents for a link round trip.

15. Assertions

Icarus Verilog 13.0 does not support concurrent SVA here, so every property is synthesisable checker logic verified in simulation.

Safety

Liveness

The atomic-hold property is the one to watch: a read-modify-write whose write never arrives holds a location forever, blocking every other agent. That is the liveness hazard this chapter's safety mechanism creates, and it is why a hold needs a bound in any real design.

Performance goals

16. Mutation Testing

Twenty mutations. Twenty killed.

The first run scored 18 of 20, and both escapes were shapes now familiar enough to be predictable.

The two-assignment fence counter survived because the testbench never had an accept and a commit in the same cycle. EXP6b added exactly that, and the consequence here is worse than a wrong statistic: an outstanding count that reads zero while work is in flight makes a fence pass early, turning a counter bug into an ordering violation.

The commit conservation law was unreachable on legal stimulus — commits can never exceed accepts on a correct design — so it needed an instance driven with commits and no accepts at all.

Three checkers in this chapter were repaired before mutation testing, by the baseline itself: the premature-visibility check that sampled pending_q too early (§8), and two combinational outputs the testbench sampled after their inputs had deasserted. The baseline catching its own bugs is worth as much as the mutation suite catching mine.

17. Verification Plan

The coverage cross is address relationship × access pair × commit state: same / different address, crossed with read-after-write, write-after-write and atomic, crossed with committed / uncommitted. The uncommitted column is where every guarantee in this chapter is actually tested, and a testbench that commits immediately never enters it.

18. Silicon Observability

Three alarms and six tuning counters. The alarms describe failures with no other symptom — a premature visibility, a stale read and a broken atomic all return plausible values — and each must be identically zero for the life of the product.

The most useful tuning pair is forwards against stalls. A high forward rate means same-address contention exists and is being handled cheaply; a high stall rate means the same contention is being paid for in full, and the fix is usually in software placement rather than hardware.

19. Debug Lab

assign may_proceed = req_valid;      // no ordering at all

  same address      : must_order=1 proceed=0 | no-order proceed=1
  no-order variant flagged a same-address reorder    : ok

assign same_addr   = busy_valid && (req_addr == busy_addr);
assign must_order  = same_addr;
assign may_proceed = req_valid && !must_order;

assign must_order = busy_valid;      // any outstanding access blocks

  different address : must_order=0 proceed=1 | global-order proceed=0
  the global-order variant serialised it anyway      : ok

assign must_order = busy_valid && (req_addr == busy_addr);

// Read, then write. No hold between them.
issue_read(addr); ... issue_write(addr, new_value);

  other access to the same line : correct blocked=1 | unlocked blocked=0
  the unlocked variant flagged broken atomicity      : ok

assign conflict      = held && other_access && (other_addr == hold_addr_q);
assign other_blocked = conflict;

assign wr_reported_done = wr_accept;   // acceptance reported as completion

  at acceptance : correct reported_done=0 | accept-is-visible reported=1
  accept-is-visible variant flagged premature done   : ok

assign wr_reported_done = wr_commit;
if (wr_reported_done && !wr_commit) premature_visible_err <= 1'b1;

if (wr_reported_done && pending_q && !wr_commit) premature_visible_err <= 1'b1;

  accept-is-visible variant flagged premature done   : ok

if (wr_reported_done && !wr_commit) premature_visible_err <= 1'b1;

if (acc_accept) outstanding_q <= outstanding_q + 1;
if (acc_commit) outstanding_q <= outstanding_q - 1;

  3 cycles of accept+commit together : outstanding=4
  the case-based counter held steady                 : ok

case ({acc_accept, acc_commit})
  2'b10: outstanding_q <= outstanding_q + 8'd1;
  2'b01: outstanding_q <= outstanding_q - 8'd1;
  default: ;                                  // both or neither: hold
endcase

assign in_scope = access_valid;      // every path assumed coherent

  side path : correct in_scope=0 out=1 | assume-all in_scope=1
  assume-all variant flagged an unsafe assumption    : ok

assign in_scope     = access_valid && via_host_path;
assign out_of_scope = access_valid && via_side_path;
if (via_side_path && in_scope) unsafe_assumption_err <= 1'b1;

20. Design Review

1. Is ordering scoped to the address or to the device? Both extremes are wrong in different ways.
2. Can a read-modify-write be interrupted, and what holds the location?
3. Is a hold bounded? An atomic whose write never arrives blocks the line forever.
4. Are acceptance and visibility distinct events, and which one reports completion?
5. Which agents are inside the coherence contract, and what reaches the media outside it?
6. Does a read to an address with an uncommitted write forward, stall, or return media?
7. What exactly does a fence wait for, and what counts it?
8. Which three counters must be zero for the life of the product?
9. What is the peak accepted-but-invisible window?
10. What happens on reset with uncommitted writes?

21. How This Appears in Real Engineering

Architecture. The ordering scope decision is worth up to two orders of magnitude of throughput and is usually made once, early, and never revisited.

RTL. Forwarding is a small structure that converts a correctness requirement into a cheap one. Without it the same guarantee costs a media round trip per same-address pair.

DV. Every guarantee here needs two agents or two accesses in flight. A single-access testbench verifies none of them while appearing thorough.

Post-silicon. The three alarms are the difference between a diagnosable corruption and a multi-week investigation, because all three failures return plausible values.

Software. Atomic-heavy data structures on far memory are a placement problem: the measured ratio here was nearly one blocked access per atomic, and that ratio worsens with contention.

22. Common Misconceptions

23. Ordering, Drawn

Teaching model — this is per address, not per device, and that is the whole point. A different address has its own copy of this machine and is unaffected by anything here. The two right-hand states are the two ways a same-address conflict resolves: a read can be forwarded cheaply, while a write must be deferred.

24. Interview Reasoning

25. Exercises

1. Analysis. A device reports ordered=1000, forwarded=0, stalls=1000, and poor throughput on a workload with heavy same-address reuse. Explain what is missing, estimate the cost using 100 ns media latency, and name the structure that would fix it.

2. Design. Add a bound to the atomic hold so a read-modify-write whose write never arrives cannot block a location forever. State what the bound must do to the partially completed operation and why that is harder than aborting a read.

3. RTL task. Extend order_domain to order same-address accesses only when at least one is a write. State the new invariant and name the workload that benefits most.

4. Assertion task. Write the property that catches a fence retiring with outstanding work, then explain why the property must read the outstanding count rather than the fence's own state.

5. Debug task. A producer/consumer handshake fails only when the consumer runs without a delay. Give your investigation order and the single directed test that identifies the cause.

6. Design review. A colleague proposes reporting write completion at acceptance "because the device never loses an accepted write". Give the strongest version of that argument, then name exactly what it breaks and why the argument is true and irrelevant.

26. Summary

A location you can reach is not yet memory.

• Memory is a promise set: same-address ordering, atomicity, a defined visibility point, and a stated coherence scope. All four are free locally and none is free across a link.
• Order the address, not the device. Global ordering is safe and costs up to two orders of magnitude; no ordering breaks programs observing their own writes.
• Atomicity requires holding the location, because the requester cannot close the gap between a read and a write — and across a link that gap is hundreds of nanoseconds.
• The hold is a liveness hazard its own safety mechanism creates, and needs a bound.
• Accepted is not visible. They are distinct events with a real gap, and conflating them breaks every flag-guards-data handshake.
• Coherence has a scope. The sourced contract places it with the host, so paths the host does not mediate fall outside it — and the question is usually unasked rather than answered wrongly.
• Forwarding makes ordering cheap: seven of nine same-address cases avoided a media commit in the measured run.
• Measured: four atomic sequences blocked three unrelated accesses — atomics on contended far memory are a placement problem.
• The two-assignment counter defect appeared for the seventh time, and here it made a fence pass early — a counter bug becoming an ordering violation.
• Verification lesson: a checker that sampled state the violating edge was still creating could not see its own violation, and the baseline caught it before mutation testing did.

Chapter 9.4 walks the read path end to end, and 9.5 takes the write path — which is harder, precisely because of the visibility distinction this chapter drew.