CXL · Module 12
Memory-Pooling Benefits and Challenges
Pooling recovers stranded capacity and charges for it in access cost, availability and allocation delay. The statistical-multiplexing argument measured, the conditions under which it saves nothing, and the five pieces of evidence a decision actually requires.
12.1 through 12.4 built a pool that works. This chapter asks whether it was worth building.
1. The Engineering Problem — Does Any Of This Pay?
The four preceding chapters are mechanism. Every one of them added structure — a ledger, an allocator, an identity, a placement map — and structure is cost. The case for paying it rests on a single observation, and the case against rests on three.
For: hosts do not peak together. Memory is provisioned per server against that server's worst moment, and most servers spend most of their time far below it. Capacity bought for a peak that has passed is capacity nobody can use, and it is owned by a machine that cannot lend it.
Against, first: a pooled access is not a local access. The round trip is longer, and every access pays it, forever.
Against, second: pooled memory fails together. A dedicated DIMM failure takes down one server. A pooled device failure takes down every host with an allocation on it.
Against, third: pooled memory has to be asked for. Dedicated memory is already there. Pooled capacity involves a request, a decision and a delay that varies, and a scheduler experiences the tail rather than the mean.
This chapter measures all four claims and finds the conditions under which the arithmetic reverses.
2. The One-Sentence Model
Pooling trades stranded capacity for shared fate. It converts memory that is owned-and-idle into memory that is available-and-slower, and the exchange rate depends entirely on whether the hosts peak at different times.
Call it saved capacity, shared risk. Both halves are measurable, neither is free, and a pool whose hosts peak together gets the risk without the saving.
3. What This Chapter Owns
| Ground | Owner |
|---|---|
| Mechanism: inventory, allocation, identity, placement | 12.1 – 12.4 |
| Whether the mechanism pays, and what evidence decides it | this chapter |
Deferred:
| Deferred ground | Owner |
|---|---|
| Latency anatomy and performance modelling | Module 18 |
| Fabric and switch architecture | Modules 15 and 16 |
| Disaggregation, composability, fleet economics | Module 23 |
4. Teaching-model boundary
The central risk in a chapter like this is presenting a teaching model's conclusion as an industry finding. The break-even below is real arithmetic on stated weights, and changing the weights changes the answer. What transfers is not the number but the method: state both sides in comparable units, measure both, and identify what would have to be true for the decision to flip.
Nothing here should be read as a claim about what CXL pooling costs or saves in practice.
5. RTL 1 — Why Pooling Can Save Anything At All
The entire economic argument is one inequality.
module capacity_needed #(parameter int HOSTS = 4, parameter int FAULT_INJECT = 0) (
input logic [5:0] d0, d1, d2, d3,
output logic [5:0] peak0, peak1, peak2, peak3,
output logic [7:0] sum_peaks, // capacity needed with dedicated memory
output logic [7:0] peak_sum, // capacity needed with a pool
output logic [7:0] saving,
output logic invariant_err
);
// The peak of a sum can never exceed the sum of the peaks. This is not a
// policy, an assumption or a workload property -- it is arithmetic, and
// it is why pooling can never need MORE capacity than dedicated does.
if (peak_sum > sum_peaks) invariant_err <= 1'b1;Dedicated provisioning must cover each host's own peak. A pool must cover the peak of the total. Those are different numbers whenever the peaks happen at different times.
Measured over five samples of four hosts:
| Value | |
|---|---|
| individual peaks | 14, 12, 15, 13 |
| sum of peaks — dedicated needs | 54 |
| peak of the total — pool needs | 30 |
| saving | 24 units |
That is a 44% reduction in capacity for identical service, and it comes from nothing but the hosts peaking at different moments.
The invariant is arithmetically guaranteed, which makes the monitor unreachable through the module's ports. It carries the same FAULT_INJECT hook as 12.1's conservation check: a second instance of the same source with the hook armed counts only one host toward the dedicated total, which is small enough for the peak of the total to exceed it, and the monitor fires.
6. RTL 2 — Stranded Capacity, Measured
// Stranded capacity is owned, idle, and unlendable. Under dedicated
// provisioning every host strands whatever it is not using.
assign dedicated_stranded = dedicated_total - pooled_used;
assign pooled_stranded = (pool_total > pooled_used) ? (pool_total - pooled_used) : 8'd0;
assign recovered = (dedicated_stranded > pooled_stranded)
? (dedicated_stranded - pooled_stranded) : 8'd0;At the busiest measured instant — 30 units in use across four hosts provisioned at 16 each:
| Mode | Total | Stranded |
|---|---|---|
| dedicated | 64 | 34 |
| pooled | 36 | 6 |
28 units recovered, and they were recovered from nowhere: no host gave anything up and no workload changed.
The recovery is not a constant. A larger pool recovers less — measured, a 48-unit pool recovers 16 while the peak recovery still reports 28, because the peak is latched rather than tracked.
Oversubscription is a real state and a deliberate bet. A pool smaller than current demand is flagged: measured, a 28-unit pool facing 30 units of demand raises oversubscribed_err, while a 30-unit pool facing exactly 30 does not. The boundary is exact and needed its own test.
7. Waveform — Stranding, Instant By Instant
Transcribed from the printed trace.
Four hosts peaking at different times
5 cyclesRead the four host rows against the total row. Host 0 peaks at cycle 0, host 1 at cycle 2, host 2 at cycle 3 and host 3 at cycle 4 — and the total never exceeds 30. Dedicated provisioning has to buy 14 for host 0 and 15 for host 2 and hold both permanently, because it cannot know that host 0 will be quiet when host 2 is busy.
The ded_strand row never falls below 34. That is memory that exists, is powered, is paid for, and is doing nothing — every cycle, in every sample.
8. RTL 3 — Utilisation, Both Ways
// Pooled utilisation below dedicated utilisation means the pool is
// larger than the dedicated fleet it replaced, which defeats the purpose.
if (util_pool_pct < util_ded_pct) util_err <= 1'b1;Measured on identical demand:
| Provisioning | Utilisation |
|---|---|
| dedicated | 46% |
| pooled | 83% |
Nearly double, on the same workload, with no change to any host. That number is the benefit stated as a ratio rather than as a count, and it is the one an operator recognises.
The peak is latched: when demand falls and utilisation drops to 33%, the peak still reports 83. A utilisation graph that tracks the current value cannot tell you the pool was nearly full four hours ago, and nearly full four hours ago is what predicts the next refusal.
9. RTL 4 — What Every Access Pays
module access_cost_mix #(parameter int LOCAL = 4, parameter int POOLED = 11) (
input logic acc_en, input logic is_pooled,
output logic [4:0] cost,
output logic [15:0] n_local, n_pooled, total_cost, avg_x10, pooled_pct
);
assign cost = is_pooled ? POOLED[4:0] : LOCAL[4:0];With illustrative weights of 4 local and 11 pooled — these are teaching values, not latencies, and Module 18 owns the real analysis:
| Pooled fraction | Average cost |
|---|---|
| 0% | 4.0 |
| 25% | 5.7 |
| 50% | 7.5 |
The cost is paid on every access, forever. The capacity saving is a one-time procurement effect; this is a permanent operating tax, and the two are not commensurable without the weights section 13 introduces.
The important structural point is that the blended cost depends on what fraction of the working set is pooled, not on how much capacity was saved. A design that pools rarely-touched capacity pays almost nothing; one that pools hot data pays on every access. The same pool, the same saving, and completely different costs depending on what was placed in it.
10. RTL 5 — Shared Fate
// With dedicated memory a device belongs to exactly one host, so one device
// failure is one host outage -- always, by construction.
assign hosts_hit_dedicated = 3'd1;
assign amplification = hosts_hit_pooled;Measured across three device failures:
| Provisioning | Host outages |
|---|---|
| pooled | 7 |
| dedicated | 3 |
Same three failures. The amplification on the worst of them was 4 — one device carrying allocations for four hosts took all four down, where dedicated memory would have taken down one.
This is the availability price of the capacity saving, and it is structural rather than incidental. Sharing a device between hosts is what makes the capacity fungible; it is also what makes the failure correlated. The mechanisms in 12.4 can spread allocations to reduce how much any one host loses, and they cannot make a shared device unshared.
The amplification factor is exactly the number of hosts on the failed device, which means it is a placement-policy output, not a property of pooling. A pool that packs many hosts onto few devices chooses a high amplification; one that spreads chooses a lower one and pays for it in access cost.
11. RTL 6 — Capacity You Have To Ask For
// The worst case is latched. A mean hides the tail, and the tail is
// what a scheduler actually experiences.
if (cnt_q > worst_wait) worst_wait <= cnt_q;
if (cnt_q < 8'd4) b0 <= b0 + 8'd1;
else if (cnt_q < 8'd8) b1 <= b1 + 8'd1;
else if (cnt_q < 8'd16) b2 <= b2 + 8'd1;
else b3 <= b3 + 8'd1;Dedicated memory has no acquisition latency: it is already attached. Pooled capacity is requested, decided and delivered, and the delay varies.
Measured over five allocations: waits of 2, 6, 11, 1 and 4 cycles, giving a distribution of 2 fast, 2 medium, 1 slow and a worst case of 11.
The distribution is the deliverable, not the mean. A scheduler that must place work within a deadline experiences the tail; an average of 4.8 cycles describes none of the five allocations well and hides the one that took 11.
Two behaviours the model makes explicit. An allocation that never completes raises timeout_err rather than waiting forever — a request with no bound is worse than a refusal, because nothing upstream can react to it. And a second start arriving mid-flight does not restart the measurement: measured, a spurious start three cycles into a seven-cycle allocation still reported a wait of 7, because the wait belongs to the request that was actually made.
12. RTL 7 — One Host's Burst Is Another Host's Wait
// Spread is measured only across hosts that ACTUALLY WANTED service. A host
// that asked for nothing and got nothing is not being treated unfairly, and
// including it makes every window look unfair.
//
// It is also measured on the counts INCLUDING this cycle's service. The
// window closes on the same edge that records its last grant, so evaluating
// the stored counts leaves the window one service short and reports a spread
// of one for a perfectly fair rotation.Measured over 32-cycle windows:
| Window | Worst spread | Flagged |
|---|---|---|
| strict rotation, four active hosts | 0 | no |
| two active hosts served evenly, two idle | 0 | no |
| one host taking 28 of 32 | 28 | yes |
The two comments above are both defects the baseline found, and both produce false alarms rather than missed ones — which is worse, because an alarm that fires on correct behaviour gets disabled.
Idle hosts must be excluded. A host that asked for nothing and received nothing is not a victim, and including it makes every window with an idle host look maximally unfair.
And the window must include the service that closes it. Evaluating the stored counts on the closing edge leaves the window one grant short, so a perfectly fair rotation reports a spread of 1 — small, plausible, and permanently wrong.
The worst case survives later windows: measured, a fair window following the monopoly leaves the worst spread at 28 rather than resetting it to 0.
13. RTL 8 and 9 — The Decision, And The Evidence It Needs
module pooling_tradeoff #(parameter int SAVE_W = 10, parameter int COST_W = 3) (
input logic [7:0] capacity_saved,
input logic [15:0] avg_cost_x10, base_cost_x10,
output logic [15:0] benefit, penalty, margin,
output logic favoured
);
assign excess = (avg_cost_x10 > base_cost_x10) ? (avg_cost_x10 - base_cost_x10) : 16'd0;
assign benefit = {8'b0, capacity_saved} * SAVE_W[15:0];
assign penalty = excess * COST_W[15:0];
assign favoured = (benefit >= penalty);Capacity saved and access cost added are measured in different units, so comparing them requires weights — and the weights are the assumption the whole decision rests on. They are stated in the parameters rather than buried, because they are the part a reader should argue with.
With the measured saving of 24 units and a baseline access cost of 4.0, swept across average costs:
| Cost | Benefit | Penalty | Wins |
|---|---|---|---|
| 4.0 | 240 | 0 | yes |
| 11.0 | 240 | 210 | +30 |
| 13.0 | 240 | 270 | no |
The crossover is measured at an average access cost of 12.0 — roughly three times the local cost. Below it, pooling wins on these weights; above it, the access tax exceeds the capacity saving.
The number is not the point; the structure is. Change SAVE_W or COST_W and the crossover moves. What survives is the shape of the argument: a fixed benefit against a cost that scales with the pooled fraction, crossing at a point that can be computed rather than argued.
14. Quantitative Reasoning
Both sides of the decision, in one place.
The benefit, from statistical multiplexing:
dedicated_capacity = sum over hosts of each host's peak = 54
pooled_capacity = peak over time of the total demand = 30
saving = 54 - 30 = 24 units (44%)Bounded by arithmetic: the peak of a sum can never exceed the sum of the peaks, so pooling can never need more capacity than dedicated. The saving is between zero and the full difference, and it is zero exactly when the hosts peak together.
Recovered stranding:
dedicated_stranded = 64 - 30 = 34
pooled_stranded = 36 - 30 = 6
recovered = 28 unitsUtilisation: 46% dedicated against 83% pooled, on identical demand.
The cost, on every access:
avg = (n_local x 4 + n_pooled x 11) / total
= 4.0 at 0% pooled, 5.7 at 25%, 7.5 at 50%The cost, on availability:
host_outages_pooled = 7 over three device failures
host_outages_dedicated = 3 over the same three
amplification = hosts sharing the failed device (measured worst: 4)The cost, on acquisition: waits of 2, 6, 11, 1 and 4 cycles — a worst case of 11 and a distribution of 2 fast, 2 medium, 1 slow.
The crossover, with the stated weights: pooling is favoured up to an average access cost of 12.0, three times the local cost.
The decision is not universal and cannot be. It depends on the peak correlation between hosts, on what fraction of the working set is pooled, on how many hosts share a device, and on how much the workload tolerates acquisition delay. All four are measurable, and a pool that measures none of them is being justified rather than evaluated.
15. Assertions
Written as SystemVerilog for the reader, executed procedurally — see section 17.
The peak of a sum never exceeds the sum of the peaks.
property p_multiplexing_bound;
@(posedge clk) disable iff (!rst_n) peak_sum <= sum_peaks;
endpropertyDedicated provisioning never strands less than the pool.
property p_stranding_ordered;
@(posedge clk) disable iff (!rst_n) dedicated_stranded >= pooled_stranded;
endpropertyPooled utilisation is never worse than dedicated.
property p_util_improves;
@(posedge clk) disable iff (!rst_n) sample_en |-> (util_pool_pct >= util_ded_pct);
endpropertyA peak register never falls.
property p_peak_latched;
@(posedge clk) disable iff (!rst_n) peak_recovered >= $past(peak_recovered);
endpropertyDedicated memory always fails alone.
property p_dedicated_isolated;
@(posedge clk) disable iff (!rst_n) fail_en |-> (hosts_hit_dedicated == 1);
endpropertyAt most one host is served per cycle.
property p_one_service;
@(posedge clk) disable iff (!rst_n) $countones(served) <= 1;
endpropertyFairness is measured only over hosts that asked.
property p_idle_not_unfair;
@(posedge clk) disable iff (!rst_n)
(active == 0) |-> (spread == 0);
endpropertyA metric is never marked seen while it has never moved.
property p_evidence_honest;
@(posedge clk) disable iff (!rst_n)
(recovered == 0 && !$past(seen_mask[0])) |-> !seen_mask[0];
endproperty92 assertion sites — 85 across the nine models and 7 in the waveform trace. All pass.
16. Mutation Testing
48 mutations injected, 48 killed, 0 surviving.
| Family | Injected |
|---|---|
| capacity requirement and the multiplexing bound | 7 |
| stranding and recovery | 5 |
| utilisation | 3 |
| access-cost mix | 5 |
| shared fate | 4 |
| allocation latency | 6 |
| fairness windows | 6 |
| the break-even comparison | 6 |
| evidence and decidability | 4 |
| integration, run under the waveform trace | 2 |
Representative kills:
| Mutation | Caught by |
|---|---|
| dedicated requirement uses the peak of the total | the 54-against-30 measurement |
| pooled requirement uses the sum of peaks | the same |
| peak of the total not tracked | a sample where demand fell |
| dedicated stranding measured against the pool | the 34-against-6 comparison |
| oversubscription boundary off by one | a pool holding exactly its capacity |
| pooled accesses priced as local | the 4.0 / 5.7 / 7.5 sweep |
| dedicated outages scaled with the pool | one device carrying four hosts |
| worst case tracks the current value | a fast allocation after a slow one |
| spread measured over idle hosts too | two active hosts and two idle |
| window closes one service short | a perfectly fair rotation |
| the break-even comparison inverted | the sweep across average cost |
| one missing metric still decidable | exactly four of five collected |
Fourteen mutations required stimulus work before they died — across two rounds, since fixing the first set exposed a second.
| Category | Count |
|---|---|
| stimulus gaps | 12 |
| checker unreachable through the ports | 1 |
| output never observed by the testbench | 1 |
| missing checkers | 0 |
Two are worth naming. The peak registers — recovery, utilisation, worst wait — all survived their first mutation because every test happened to end at the maximum, where a register that tracks and one that latches are indistinguishable. Each needed a sample where the value fell.
And one survivor was in the waveform trace itself: the demand sequence peaked on its last sample, so the traced peak register could not be shown to latch. Reordering the trace so the peak falls in the middle fixed it — and improved the published figure, which now shows the total rising to 30 and coming back down.
The taxonomy has now held for the ninth consecutive batch.
17. Verification Strategy
Tool reality. Icarus Verilog 13.0 — no concurrent SVA, so every property has an executable procedural counterpart, and every run is bounded by a hard timeout.
A delta-cycle race found here is worth recording. Reading a combinational output in the same simulation delta as changing its driver returns the stale value. The fairness spread appeared wrong until a settle delay was added between assigning served and sampling spread. This is a testbench defect rather than a design one, and it produces a plausible wrong number rather than an error.
Independent oracles.
| Model | Design, then oracle |
|---|---|
| capacity | registered running maxima → plain integer maxima |
| stranding | derived subtractions → expected value per configuration |
| access cost | accumulated weights → expected average per mix |
| shared fate | a popcount over a bitmap → expected outages per failure |
| latency | a counter and bucket chain → wait and bucket per case |
| tradeoff | weighted comparison → a swept crossover |
The last one is the most useful pattern in the chapter. Rather than asserting the break-even, the testbench sweeps average access cost from 4.0 to 20.0 and observes where the verdict flips, then asserts that the crossover is at 12.0. A directly asserted threshold would have reproduced whatever the design computed; the sweep finds it independently.
Coverage. The points that matter are peak correlation between hosts, pooled fraction, hosts per failed device, allocation wait bucket, and evidence completeness. The cross worth driving is peak correlation against pooled fraction — the two variables that jointly decide the outcome, and the pair that produces the zero-saving case. A cross of host against sample index is noise.
18. Synthesis and Implementation Reality
These are measurement structures, not datapath. None of them sits on an access path, and none needs to be fast. That changes what matters about them.
Running maxima are cheap and must be latched. A comparator and a register per tracked value. The recurring defect is writing them as tracking registers, which costs the same and reports the wrong thing.
Percentages need width discipline. Every ratio in this chapter uses a 16-bit or 32-bit intermediate, because Verilog sizes an expression from its operands — the defect class 12.4 documents at length. A percentage computed in eight bits is wrong by a factor that looks plausible.
Averages need a scale factor. The blended access cost is accumulated scaled by ten so the fractional part survives integer division. Reporting 4, 5 and 7 instead of 4.0, 5.7 and 7.5 hides most of the movement the metric exists to show, and the division is the only place the resolution can be lost.
Histograms are cheaper than they look and worth more than means. Four counters and a comparison chain give a distribution; a mean gives one number that describes none of the samples. For allocation latency the tail is the deliverable, and the tail is exactly what a mean removes.
The evidence bank is five flip-flops and is arguably the highest-value structure in the chapter, because it converts we did not measure that from an invisible omission into an explicit output.
19. Silicon Observability
| Counter | Class |
|---|---|
sum_peaks, peak_sum, saving | policy input |
recovered, peak_recovered | telemetry |
util_ded_pct, util_pool_pct | policy input |
peak_pool_util | telemetry |
oversubscribed_err | hard alarm |
avg_x10, pooled_pct | policy input |
n_outages pooled vs dedicated | telemetry |
amplification | policy input |
worst_wait, bucket counts | policy input |
timeout_err | hard alarm |
worst_spread | policy input |
unfair_err | telemetry |
multi_serve_err | hard alarm |
decidable, seen_mask | meta |
| Observation | Reading |
|---|---|
saving near zero with pooling deployed | hosts are peaking together; the pool is all cost |
peak_recovered high, recovered low now | the benefit is real but intermittent |
avg_x10 rising with pooled_pct flat | hot data has moved into the pool |
amplification above 1 and rising | placement is concentrating hosts onto devices |
worst_wait growing with the mean flat | the tail is degrading and averages will not show it |
unfair_err set with worst_spread moderate | one window was bad; check whether it repeats |
oversubscribed_err set | pool smaller than demand — a deliberate bet |
decidable low | the pooling decision is being made on incomplete evidence |
The last row is the one this chapter adds to the curriculum. Every other counter measures the system. decidable measures whether anyone is in a position to judge it, and a pool running with two of five metrics unwired is being justified by the absence of contrary evidence.
20. Debug Lab
Pooling was deployed and saved nothing
PEAK-CORRELATIONA pool replaces dedicated memory across four hosts. Capacity requirements did not fall. Access costs rose as expected and availability got worse as expected. The saving never appeared.
all four hosts peaking together : sum_peaks=54 peak_sum=54 saving=0Compare the sum of individual peaks against the peak of the total. If they are the same number, the hosts peak together and there is no multiplexing to exploit.
A synchronised workload — a fleet-wide batch window, a coordinated training step, a cron schedule that fires everywhere at once; or a sizing study that measured each host separately and never measured the aggregate.
Sample all hosts at the same instants and track the total. The measured favourable case has individual peaks of 14, 12, 15 and 13 with a total never above 30 — a saving of 24. The unfavourable case has the same four peaks occurring in one sample, giving a total of 54 and a saving of zero.
Pooling monetises hosts being busy at different times. When they are busy together, the entire benefit disappears and every cost remains.
assign saving = (sum_peaks > peak_sum) ? (sum_peaks - peak_sum) : 8'd0;Measure peak correlation before deploying a pool, not after. It is the single condition that decides whether any of the mechanism pays.
The utilisation graph says the pool is fine and allocations are failing
TRACKED-PEAKAllocations were refused overnight. The utilisation dashboard shows the pool comfortably below capacity for the entire period. Nobody can find the event.
utilisation : dedicated 46%, pooled 83% (peak 83%)Check whether the peak register latches or tracks. If the reported peak ever falls, it is not a peak.
A peak register written as peak <= current; software polling slower than the excursion; a dashboard displaying instantaneous utilisation and calling it a maximum.
Drive utilisation up and then down and read the peak. In the measured run utilisation reaches 83%, falls to 33%, and the peak still reports 83. A tracking register would report 33 and the excursion would be unrecoverable.
The excursion that caused the refusals was shorter than the sampling interval, and nothing latched it.
if (util_pool_pct > peak_pool_util) peak_pool_util <= util_pool_pct;Latch peaks in hardware and let software read them at leisure. Polling can never be fast enough, and this defect class appeared in three separate registers in this chapter alone.
One device failure took out more hosts than the fleet has failures
SHARED-FATEThree device failures over a quarter produced seven host outages. The dedicated fleet it replaced averaged one outage per failure. Nothing is broken.
one device failure : 7 host outages pooled, 3 dedicated (amplification 4)Read the amplification per failure — the number of hosts holding allocations on the failed device. Dedicated memory is 1 by construction; anything above that is the price of sharing.
A placement policy packing many hosts onto few devices; an availability model that counted device failures rather than host outages; capacity efficiency optimised without an amplification constraint.
Record hosts-per-failed-device on every failure. In the measured run one failure had four hosts on it, producing four outages where dedicated memory would have produced one. That factor is a placement output, not an inherent property of pooling.
Sharing a device between hosts is what makes capacity fungible and what makes failures correlated. They are the same mechanism.
assign hosts_hit_dedicated = 3'd1; // by construction
assign amplification = hosts_hit_pooled; // measured, per failureConstrain hosts-per-device in the placement policy and report the amplification. An availability budget expressed in device failures rather than host outages will be met and still miss.
Allocation latency looks fine and the scheduler misses deadlines
TAILMean allocation latency is well inside budget. A scheduler placing work against deadlines misses them regularly, and the misses do not correlate with load.
allocation latency : worst=11 cycles | buckets 2/2/1Read the distribution rather than the mean. Measured waits of 2, 6, 11, 1 and 4 give a mean of 4.8 that describes none of the five, and hides the one that took 11.
Only a mean recorded; a histogram with buckets too coarse to show the tail; a worst case that tracks rather than latches.
Bucket the waits and read the worst case. Then check that a fast allocation following a slow one does not reset the worst case — measured, the worst still reports 11 after a 1-cycle allocation.
A scheduler experiences the tail, not the mean, and the mean is what was being reported.
if (cnt_q > worst_wait) worst_wait <= cnt_q;
if (cnt_q < 8'd4) b0 <= b0 + 8'd1;
else if (cnt_q < 8'd8) b1 <= b1 + 8'd1;Publish a histogram and a latched worst case. Four counters cost almost nothing and a mean cannot be un-averaged afterwards.
An allocation request never came back
UNBOUNDED-WAITA host requests capacity and blocks. There is no refusal, no error and no completion. The pool reports healthy capacity throughout.
allocation that never completes : timeout_err=1Check whether any bound exists on the request. An unbounded wait produces no counter, no alarm and no evidence — the requester simply stops.
A request queued behind something that never completes; a decision awaiting evidence that never arrives; no timeout on the allocation path at all.
Start an allocation and never complete it. The measured design raises timeout_err after a bounded wait rather than waiting indefinitely. Then confirm a spurious second start does not restart the measurement — the wait belongs to the request that was actually made, measured at 7 cycles despite a start arriving three cycles in.
A request with no bound is worse than a refusal, because nothing upstream can react to it and no counter records it.
if (cnt_q >= 8'd31) timeout_err <= 1'b1;Bound every request and count the bound being hit. A refusal is actionable; silence is not.
The fairness monitor cries wolf on a perfectly fair pool
FALSE-ALARMThe fairness alarm fires on windows where service is visibly even. Operators disable it. Two months later a genuine starvation goes unnoticed.
strict rotation, 4 active : worst spread=0 unfair=0Two separate causes produce this, and both were found here. Check whether idle hosts are included in the spread, and whether the window includes the service that closes it.
Spread computed across all hosts rather than active ones, so any idle host makes the window look maximally unfair; or the window boundary evaluated on the stored counts, leaving it one grant short.
Run a strict rotation across four active hosts: the spread must be exactly 0. Then run two active hosts with two idle: also 0. If the first gives 1, the window is one service short; if the second gives a large number, idle hosts are being counted.
Both defects produce false alarms rather than missed ones, which is worse — an alarm that fires on correct behaviour gets turned off, and takes the real alarm with it.
for (i=0;i<HOSTS;i=i+1) sn[i] = s[i] + {7'b0, served[i]}; // include the closing grant
for (i=0;i<HOSTS;i=i+1) if (active[i]) begin // only hosts that askedVerify monitors against known-good behaviour, not only against known-bad. A monitor is as much a liability as an asset until it has been shown to stay quiet when it should.
The pool was smaller than demand and nobody decided that
OVERSUBSCRIPTIONAllocations fail during a demand peak. Capacity planning says the pool is correctly sized. The failures do not recur outside the peak.
30-unit pool, 30 units demanded : oversubscribed_err=0
28-unit pool, 30 units demanded : oversubscribed_err=1Compare demand against pool capacity at the peak, not on average. A pool sized to the mean is oversubscribed at the peak by construction.
Sizing against average demand; a peak measured before the workload changed; a deliberate oversubscription that was never recorded as one.
Check the boundary precisely: a pool holding exactly its capacity is not oversubscribed, and one unit short is. Both sides are needed — a comparison wrong in either direction passes every test that only exercises the comfortable case.
Oversubscription is a legitimate bet. It stops being legitimate when nobody made it deliberately and no counter records it.
if (pooled_used > pool_total) oversubscribed_err <= 1'b1;Alarm on oversubscription rather than inferring it from failures. The alarm fires at the moment of the bet; the failures arrive later and look like something else.
A pooling decision was made on three of five metrics
MISSING-EVIDENCEA pool is approved on the strength of a capacity saving. Six months later the access-cost penalty and the availability amplification are discovered, and neither was ever on a dashboard.
evidence : 3 of 5 -> decidable=0 ; 4 of 5 -> decidable=0 ; all 5 -> decidable=1Check which metrics have ever been observed to move. A metric that has never been collected reads as zero, and zero cost is exactly what a missing panel looks like.
Telemetry wired for the benefit and not the costs; metrics that exist in RTL and were never plumbed to software; a review that treated an empty panel as a good result.
Track a seen-mask across the five metrics: capacity recovered, access-cost delta, worst allocation wait, failure amplification, fairness spread. Measured, three of five is not decidable and four of five is not decidable either — the threshold is all five, and a design that accepts four accepts exactly this failure.
A metric that was never collected is indistinguishable from a metric that is genuinely zero, and the difference is the whole decision.
if (recovered != 8'd0) seen_mask[0] <= 1'b1; // seen means it MOVED
assign decidable = (&seen_mask);Publish decidability alongside the metrics. It is five flip-flops, and it converts we did not measure that from an invisible omission into an explicit output.
21. Design Review
What a reviewer should attack first.
Peak correlation, before anything else. Ask for the sum of individual peaks and the peak of the total, measured on the same samples. If they are close, the pool is all cost, and no other question matters.
Which data is pooled, not how much. The access tax scales with the pooled fraction of the working set, not with the capacity saved. Pooling cold capacity is nearly free; pooling hot data is not.
Hosts per device. The availability amplification is a placement output. Ask what constrains it, and whether the availability budget is expressed in device failures or host outages — the second is the one that matters and the first is the one usually measured.
The latency distribution, not the mean. Ask for buckets and a latched worst case. A mean cannot be un-averaged later.
The weights in the comparison. Benefit and cost are in different units, so something converts them. Ask what, and ask who chose it. A comparison whose weights are buried is an opinion presented as arithmetic.
Decidability. Ask which of the five metrics have ever moved. Two unwired panels read as zero cost, and a decision made that way is a decision made on the absence of contrary evidence.
What is deliberately not here. No latency anatomy, no bandwidth model, no queueing theory — Module 18. No fleet economics, no currency, no procurement, no composability — Module 23. No new mechanism at all: every structure in this chapter measures the four that came before it.
22. How This Appears in Real Engineering
In architecture review, the pooling case is usually presented as a capacity saving with the costs described qualitatively. This chapter's position is that all four costs are measurable and that a proposal quantifying only the benefit is incomplete rather than optimistic.
In RTL design, the recurring defect here is the tracking register written where a latching one was meant. It appeared in three separate places in these models — recovery, utilisation and worst wait — and it costs the same area, reports the wrong thing, and is invisible in any test that ends at the maximum.
In verification, the lesson is that monitors need to be verified against correct behaviour as well as broken behaviour. Both fairness defects here produce false alarms, and a false alarm is worse than a missing one because it gets the monitor disabled.
In bring-up, the surprise is that a pool can be working perfectly and saving nothing. Everything is correct, every mechanism does its job, and the workload simply peaks together.
In operations, the question is whether to grow the pool or go back to dedicated memory, and it cannot be answered without all five metrics. The most common failure is not measuring badly — it is not measuring at all and reading the empty panel as good news.
23. Common Misconceptions
"Pooling saves capacity." It saves capacity when hosts peak at different times. Measured, the same four peaks give a 24-unit saving spread out and zero when simultaneous.
"The saving is the whole story." It is one of five numbers. The others are access cost, availability amplification, allocation latency and fairness, and all four move the wrong way.
"A pooled access is a bit slower." It is slower on every access, forever, while the capacity saving is a one-time procurement effect. They are not the same kind of quantity and comparing them requires weights somebody has to choose.
"Spreading allocations fixes shared fate." It reduces how much one host loses. It cannot make a shared device unshared, and a device carrying allocations for four hosts takes down four.
"Average allocation latency is inside budget." A scheduler experiences the tail. Waits of 2, 6, 11, 1 and 4 give a mean of 4.8 that describes none of them.
"The utilisation graph shows the pool was fine." Only if the peak is latched. Utilisation reaching 83% and falling to 33% reads as 33% on any register that tracks.
"A metric reading zero means the cost is zero." It means either the cost is zero or nobody measured it, and those are indistinguishable without a decidability check.
"The break-even is a property of pooling." It is a property of the weights. Change them and it moves — which is why they belong in the parameters rather than in the argument.
24. Interview Reasoning
25. Exercises
-
Calculation. Eight hosts have peak demands of 12, 9, 15, 7, 11, 14, 8 and 10. Their maximum simultaneous total is 61. Compute the dedicated requirement, the pooled requirement, the saving and the percentage saved. Then compute what the saving becomes if the maximum simultaneous total is 78.
-
Analysis. A pool saves 30% of capacity and raises blended access cost from 4.0 to 6.5. Using the weights in section 13, determine whether pooling is favoured. Then state what the capacity saving would have to be for the decision to flip at that access cost.
-
RTL task. Extend
capacity_neededto report a rolling peak over a sliding window rather than an all-time peak. State what storage that requires, and explain why an all-time peak eventually over-provisions a pool whose workload has changed. -
Assertion task. Write the property proving a peak register never falls. Then explain why this property alone is insufficient to prove the register is correct, and state the second property required.
-
Design task. Design a placement constraint that bounds failure amplification to at most two hosts per device. State how it restricts the allocator, what it costs in achievable utilisation, and how you would measure whether the constraint was worth its cost.
-
Testbench design. Design the stimulus that distinguishes a latching peak register from a tracking one. Explain why any test ending at the maximum cannot distinguish them, and state the minimum sequence required.
-
Debug task. A pool shows a 40% capacity saving and users report intermittent latency spikes that do not correlate with utilisation. Give your investigation order, name the metric most likely missing, and explain why the mean would have looked healthy throughout.
-
Design review. A colleague proposes oversubscribing the pool by 20% on the grounds that peak demand is rare. Give the strongest version of that argument, name the counter that makes the bet visible, and state what must be true about the workload for the bet to be sound.
26. Summary
Pooling trades stranded capacity for shared fate.
- The benefit is statistical multiplexing. Individual peaks of 14, 12, 15 and 13 sum to 54; the total never exceeded 30. A saving of 24 units — 44% — for identical service.
- It is bounded by arithmetic. The peak of a sum never exceeds the sum of the peaks, so pooling can never need more capacity than dedicated provisioning.
- And it collapses to nothing when hosts peak together. The same four peaks occurring simultaneously give a saving of zero, with every cost intact.
- Stranding recovered: 34 units down to 6, and utilisation from 46% to 83% on identical demand.
- Every access pays. Blended cost 4.0 to 5.7 to 7.5 as the pooled share of accesses rose to half — a permanent tax against a one-time saving.
- Failures correlate. Three device failures produced 7 host outages pooled against 3 dedicated, with a worst-case amplification of 4.
- Capacity must be asked for. Waits of 2, 6, 11, 1 and 4 cycles — a latched worst case of 11 that a mean of 4.8 conceals entirely.
- Fairness monitors must not cry wolf. Both defects found here produced false alarms: idle hosts counted as victims, and a window evaluated one grant short reporting a spread of 1 for a perfect rotation.
- The decision has a computable crossover — favoured up to an average access cost of 12.0 on the stated weights — and the weights are the assumption, not the conclusion.
- A metric that was never collected is not a zero. Three of five is not decidable; four of five is not decidable either.
- Verification: 92 assertion sites, 48 of 48 mutations killed, zero surviving. Fourteen needed stimulus work: twelve stimulus gaps, one unreachable checker, one unobserved output, zero missing checkers. One survivor was in the waveform trace itself, whose demand peaked on its last sample.
Module 12 — Memory Pooling is complete. 12.1 made capacity into inventory, 12.2 turned a grant into a lifecycle, 12.3 gave every allocation an identity that survives reuse, 12.4 gave it a location and a blast radius, and this chapter asked what the whole arrangement is worth — and found that the answer depends on a single measurable property of the workload.
Next: Module 13 — Coherency Fundamentals, which returns to an assumption every chapter in this module relied on: that each allocation has exactly one owner at a time. Removing it is what CXL.cache is for.
Continue learning
Related tutorials
- Related topic
Shared Memory Pools
A pool is not several hosts sharing memory. It is a capacity inventory with correctness obligations: counted, owned, placed, and provably conserved. Why free capacity is not allocatable capacity, and what hardware has to hold to make the distinction.
- Related topic
Resource Allocation
Admission says a request can be served. Allocation decides where, and that decision determines whether the pool can serve the next one. First fit against best fit measured on an identical workload, extent split and merge, and why a grant is a lifecycle rather than a bitmap write.
- Related topic
Multi-Host Systems
An allocation with no owner is just a bit. Host identity, generation counters that stop a late event from corrupting a reused slot, range isolation, per-host quota, and what happens to capacity when the host holding it disappears.
- Related topic
Datacenter Architecture
A pool that spans a rack is the same pool with a geography. Physical placement, which hosts can reach which enclosures, what one device failure actually costs, and why a pool that can survive a loss is a pool forbidden to use all its capacity.
Standards & specifications
- Governing standard
- CXL Specification (CXL Consortium)(opens CXL Consortium in a new tab)
Defines CXL.io, CXL.cache and CXL.mem, and the coherence and memory-pooling behaviour built on them. System design and deployment topology are not mandated.
This page also covers RTL structure, verification approach and debugging technique. Those are engineering practice built on the standard, not requirements the standard itself imposes.
Where this fits
Part of the CXL curriculum.
