Memory-Pooling Question

27.4 established what device memory is. This chapter is the design question that follows — design me a pooled-memory architecture — and it is a whiteboard question, which means the interviewer is watching the decisions rather than the recall.

The memory is shared. That is where most answers start and where a surprising number of them stop, and it leaves five design decisions unmade — each of which has a number attached and each of which somebody has to choose.

1. The Engineering Problem — Assigned Is Not Shared

Pooled memory is capacity a fabric hands to one host at a time. Memory two hosts read simultaneously is a different and much harder thing, and claiming five hundred and twelve gigabytes as simultaneously shared is claiming a coherency story the pool does not have. Section 5.

Reassignment has a period and so does demand. An hour to move capacity against a demand that moves every minute is every period missed — a pool that is pooled in name and static in practice. Section 6.

The saving is bought by assuming peaks do not coincide. Eight peaks landing together on a pool sized for four is half the demand unmet, on the day the assumption fails. Section 7.

One host must not read another's. Half the assigned capacity gated and none of it cleared is five hundred peer reads reaching another tenant's data, at memory speed, with no log anywhere. Section 9.

Capacity is assigned and bandwidth is contended. Eight hosts on one device's sixty-four units, each wanting sixteen, is eight units each and every host starved — and nothing in the capacity assignment said so. Section 10.

This chapter against 27.4, stated precisely. That one owns what a far memory tier is. This one owns what a fleet of them has to be arranged into — which is why every model here is about a decision with a cost rather than a property with a value, and why section 14's weak definition is the opening sentence of an answer rather than a technical claim.

2. The One-Sentence Model

A memory pool does what it was bought for when capacity really is pooled rather than simultaneously shared, when it moves between hosts inside the demand's own timescale, when it is sized against coincident peaks rather than averages, when one tenant cannot read another's, when one host cannot take the pool's bandwidth, and when the assignment chunk does not eat the saving — and "the memory is shared" is one of those six.

3. What This Chapter Owns

The shape of the answer first, because a design question wants a structure rather than a list.

A pool is three things: devices, a fabric, and something that decides. The devices hold the memory. The fabric — one or more CXL switches — lets any host reach any device. The fabric manager decides which capacity belongs to which host, and issues the configuration that makes it so.

Capacity is assigned, not shared. At any moment a given range belongs to one host, which maps it and uses it as ordinary memory. Reassignment means taking it from one host and giving it to another, and the host losing it has to have stopped using it first. This is the default meaning of pooling and it is the one to build, because it needs no coherency between hosts and therefore needs no agreement between operating systems.

Simultaneously shared memory is the other thing and it is harder. Two hosts reading and writing the same range at the same time need a coherency protocol between them, which is a much larger architectural commitment. It exists, it is useful for specific problems, and an answer that offers it casually has not noticed the difference. Section 5.

The saving comes from oversubscription. A pool sized for the sum of every host's peak saves nothing — it is the same memory in a different place. A pool sized for the expected coincidence of peaks saves the difference, and that saving is the entire commercial case. Section 7 is what happens when the expectation is wrong.

The costs are a switch, a hop, and a manager. Every access crosses a fabric it would not have crossed, every port costs money, and something has to be running to make assignment decisions. Sections 8 and 12.

There is one more cost that belongs in the opening summary because it is the one that gets forgotten: the pool has to be filled before it can lend. Capacity that a host has been given and is not currently using is not available to anybody else unless the host gives it back, and giving it back means the host's own memory manager has to have released it. A pool whose hosts never shrink is a pool that only ever grows assignments, which reclaims nothing after the first round and looks, from the fabric manager, like a pool with no spare capacity. That is a host-software requirement rather than a fabric one, and it is the requirement most likely to be outside the designer's control.

The hazards are isolation, fairness and granularity. The same DRAM goes to different tenants in turn; two hosts with separate ranges on one device share its bandwidth; and capacity moves in chunks that round up. Sections 9, 10 and 11.

One more structural choice belongs in the opening answer, because an interviewer will probe for it. A pool can be built with the devices behind a switch, reachable by any host, or with devices directly attached to a small group of hosts that share among themselves. The first is a true pool and pays the fabric's cost; the second is cheaper, has no switch hop, and can only move capacity within its group. The second is not a worse design — it is a different point on the same curve, and which one is right depends on how correlated the demand is between the hosts that would share a group. Naming that as a choice rather than assuming the switch is what separates a design answer from a diagram.

4. Teaching-Model Boundary

Every model in this chapter is a teaching model, not a fabric manager. It computes the one relationship the section is about and nothing else. There is no switch, no assignment algorithm, no telemetry and no allocator anywhere in this file.

Each model is built twice from one source. A parameter selects between the measured build, which computes what a design decision actually costs, and the it-is-shared build, which computes what follows from treating the pool as a solved problem. Every section's headline number is the gap between them.

5. RTL 1 — Assigned Is Not Shared

Start with the distinction that decides how hard the rest of the design is, because getting it wrong makes everything downstream a different problem.

Pooled memory is assigned. A range belongs to one host, which maps it, caches it and uses it as memory. When the fabric manager reassigns it, the losing host unmaps it first and the gaining host maps it after. No two hosts hold it at once, which means no coherency is needed between hosts and the operating systems involved need to agree about nothing.

Simultaneously shared memory is not that. Two hosts reading and writing one range at the same time requires a coherency protocol spanning both of them, which is a far larger commitment — and an answer that says "and the hosts share it" without noticing has promised the harder thing by accident.

// RTL 1 - assigned is not shared. Pooled memory is capacity a fabric can give
// to one host at a time; memory two hosts read simultaneously is a different
// and much harder thing, and conflating them is the commonest design error in
// a whiteboard answer.
module pool_semantics #(parameter int SHARED_MEANS_SIMULTANEOUS = 0) (
  input  logic clk, rst_n,
  input  logic        evaluate,
  input  logic [15:0] pool_gb, assigned_gb, claimed_shared_gb, hosts,
  output logic [15:0] one_at_a_time_gb, simultaneous_gb, overclaimed_gb, pooled_pct,
  output logic        pooled,
  output logic [7:0]  n_evals, n_overclaimed,
  output logic        partition_err
);
  logic [15:0] true_over;
  logic [31:0] p_q;
  logic        truly_over;
  // Capacity assigned to one host at a time is the pool's normal mode.
  assign one_at_a_time_gb = (assigned_gb > pool_gb) ? pool_gb : assigned_gb;
  // Capacity claimed to be readable by several hosts at once is a claim that
  // needs a coherency story the pool does not have by default.
  assign simultaneous_gb = (claimed_shared_gb > pool_gb) ? pool_gb : claimed_shared_gb;
  assign true_over = simultaneous_gb;
  assign overclaimed_gb = (SHARED_MEANS_SIMULTANEOUS != 0) ? 16'd0 : true_over;
  assign p_q = (pool_gb == 16'd0) ? 32'd100
             : (({16'd0, one_at_a_time_gb} * 32'd100) / {16'd0, pool_gb});
  assign pooled_pct = (p_q > 32'd100) ? 16'd100 : p_q[15:0];
  assign pooled = (overclaimed_gb == 16'd0) && (one_at_a_time_gb != 16'd0);
  assign truly_over = (true_over != 16'd0) && (hosts > 16'd1);
  assign partition_err = evaluate && truly_over && pooled;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_evals <= 8'd0; n_overclaimed <= 8'd0;
    end else if (evaluate) begin
      n_evals <= n_evals + 8'd1;
      if (truly_over) n_overclaimed <= n_overclaimed + 8'd1;
    end
  end
endmodule

Five hundred and twelve gigabytes assigned and the same five hundred and twelve claimed as simultaneously shared is all of it overclaimed — a design that says it is a pool and describes something else. The shared-means-simultaneous view overclaims nothing, because it does not distinguish the two.

Figure 1 — the word that does two jobs. "Shared" in ordinary speech covers both a library book, which one person has at a time, and a shared drive, which everybody reads at once. Memory pooling is the first and the answer that sounds like the second has promised a coherency protocol spanning every host in the pool. The distinction costs one clause to make and is the difference between a buildable design and a research project.

The fifth case is worth reading because it removes the problem rather than solving it. On a single-host system there is nobody to share with, and the claim costs nothing — the model reports the overclaim and declines to call it a fault. That is the state a prototype is in, which is why this error survives a demonstration and surfaces at the second host.

It is worth saying what simultaneous sharing would actually require, because "harder" is not an argument on its own. Two hosts writing one range need their caches kept consistent with each other, which means a coherency domain spanning both — either a hardware protocol between the hosts, which does not exist across ordinary servers, or a software discipline in which the application does the synchronisation explicitly and the memory is treated as uncached or carefully flushed. The second is real and is what shared-memory CXL deployments actually do, and it constrains the application rather than the hardware. Offering it as though it were a property of the pool skips that constraint entirely.

The degenerate cases are the two empties. A pool with no capacity is not a pool, and a pool with capacity and nothing assigned is idle rather than wrong — both builds agree on the first and the model declines to alarm on the second.

6. RTL 2 — Reassignment Has A Period, And So Does Demand

The second decision, and the one that decides whether the pool does anything at all.

Pooling moves capacity between hosts. How fast it can move is a property of the implementation; how fast it needs to move is a property of the workload. A pool that reassigns in an hour serves a fleet whose needs change daily and does nothing whatsoever for one whose needs change by the minute — it is, for that workload, a set of statically-assigned ranges with extra hardware in the path.

This is the question an interviewer is most likely to ask second, because it separates an answer that has thought about the mechanism from one that has thought about the use.

// RTL 2 - reassignment has a period and so does demand. A pool that can move
// capacity once an hour serves a workload whose needs change once a day and
// does nothing at all for one that changes every minute.
module reassign_timing #(parameter int CAPACITY_MOVES_INSTANTLY = 0) (
  input  logic clk, rst_n,
  input  logic        evaluate,
  input  logic [15:0] demand_period_s, reassign_s, periods, gb_at_stake,
  output logic [15:0] cycles_served, cycles_missed, missed_gb, served_pct,
  output logic        reassignable,
  output logic [7:0]  n_evals, n_too_slow,
  output logic        timing_err
);
  logic [15:0] true_missed;
  logic [31:0] m_q, s_q;
  logic        truly_slow;
  // A reassignment slower than the demand period misses every period it spans.
  assign true_missed = (reassign_s > demand_period_s) ? periods : 16'd0;
  assign cycles_missed = (CAPACITY_MOVES_INSTANTLY != 0) ? 16'd0 : true_missed;
  assign cycles_served = periods - true_missed;
  assign m_q = {16'd0, cycles_missed} * {16'd0, gb_at_stake};
  assign missed_gb = (m_q > 32'hFFFF) ? 16'hFFFF : m_q[15:0];
  assign s_q = (periods == 16'd0) ? 32'd100
             : (({16'd0, cycles_served} * 32'd100) / {16'd0, periods});
  assign served_pct = (s_q > 32'd100) ? 16'd100 : s_q[15:0];
  assign reassignable = (cycles_missed == 16'd0);
  assign truly_slow = (true_missed != 16'd0) && (gb_at_stake != 16'd0);
  assign timing_err = evaluate && truly_slow && reassignable;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_evals <= 8'd0; n_too_slow <= 8'd0;
    end else if (evaluate) begin
      n_evals <= n_evals + 8'd1;
      if (truly_slow) n_too_slow <= n_too_slow + 8'd1;
    end
  end
endmodule

An hour to reassign against a demand that moves every minute is every one of a hundred periods missed — twenty-five thousand six hundred gigabyte-periods of capacity that was needed somewhere else and stayed where it was. The moves-instantly view misses nothing.

The fourth case is the one that makes a slow pool survive procurement. A reassignment nobody was waiting on costs nothing — the pool is slow, every period is missed, and no capacity was at stake in any of them. A fleet with light and stable demand will not notice, which is precisely the fleet a pool was least needed for.

The section's practical output is a single requirement to gather before anything is designed: how often does this fleet's per-host demand actually change, and by how much? That number decides whether the pool needs to reassign in seconds, minutes or hours, and every subsequent decision — the chunk size in section 11, the wipe-or-gate choice in section 9, the manager's availability requirement in section 12 — is sized against it. It is one measurement and it is frequently not taken, because the pool is bought on the capacity argument and the timing argument arrives afterwards.

The last case is the scaling and it is worth reading as a warning about how these numbers are quoted. A thousand periods with a thousand gigabytes at stake saturates the capacity-lost figure — the model reports sixty-five thousand five hundred and thirty-five and the true number is a million. A saturated loss is still a loss, and a proposal that quotes a saturated figure as though it were the total has understated the problem by a factor it cannot see. This is the same shape as every other saturation in the batch and it is the reason all of them clamp high rather than wrapping.

The boundary is driven deliberately: reassignment exactly as long as the demand period keeps up, and the model does not alarm at the equality. That matters because a design that targets the demand period exactly is a design that has done the arithmetic, and a model that failed it would be punishing the right answer.

The number worth carrying is not the reassignment time itself but the ratio between it and the demand period, for the same reason 27.4 section 6 cares about a latency ratio rather than a latency. A pool that reassigns in ten seconds is fast or slow depending entirely on whether the workload's needs shift every minute or every day, and the same pool is both at once for a fleet running mixed workloads. A design that quotes a reassignment time without naming the demand period it was chosen against has given half a specification.

There is also a practical asymmetry worth knowing: giving capacity is fast and taking it back is slow. A host receiving a new range maps it and uses it; a host losing one has to stop using it first, which means unmapping, flushing and quiescing whatever was touching it. A pool whose measured reassignment time was taken on the giving side has measured the easy half, and section 18 says to test it under load for exactly that reason.

7. RTL 3 — A Pool Sized Below The Sum Of Peaks

The third decision, and the one the commercial case rests on entirely.

A pool sized for the sum of every host's peak demand holds exactly as much memory as buying each host its own peak would have. The saving is the oversubscription — sizing for the expected coincidence rather than the sum — and how much saving there is depends on how independent the peaks are.

That independence is an assumption about the workload, and it is usually true. It is true right up until something correlates the hosts: a scheduled job, a traffic spike, a failover that moves load onto the surviving machines. The failure mode of oversubscription is not gradual; it is fine, fine, fine, and then several hosts want their peak at once.

// RTL 3 - a pool sized below the sum of peaks. The saving is the whole point
// and it is bought by assuming the peaks do not coincide, which is true until
// the day they do.
module oversubscription #(parameter int PEAKS_NEVER_COINCIDE = 0) (
  input  logic clk, rst_n,
  input  logic        evaluate,
  input  logic [15:0] sum_peaks, pool_size, coincident_peaks, peak_gb,
  output logic [15:0] demand_now, served_gb, shortfall_gb, served_pct,
  output logic        pool_sized,
  output logic [7:0]  n_evals, n_short,
  output logic        oversub_err
);
  logic [15:0] true_short;
  logic [31:0] d_q, s_q;
  logic        truly_short;
  // When several peaks land together the demand is their sum.
  assign d_q = {16'd0, coincident_peaks} * {16'd0, peak_gb};
  assign demand_now = (d_q > 32'hFFFF) ? 16'hFFFF : d_q[15:0];
  assign served_gb = (demand_now > pool_size) ? pool_size : demand_now;
  assign true_short = demand_now - served_gb;
  assign shortfall_gb = (PEAKS_NEVER_COINCIDE != 0) ? 16'd0 : true_short;
  assign s_q = (demand_now == 16'd0) ? 32'd100
             : (({16'd0, served_gb} * 32'd100) / {16'd0, demand_now});
  assign served_pct = (s_q > 32'd100) ? 16'd100 : s_q[15:0];
  assign pool_sized = (shortfall_gb == 16'd0);
  assign truly_short = (true_short != 16'd0) && (sum_peaks != 16'd0);
  assign oversub_err = evaluate && truly_short && pool_sized;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_evals <= 8'd0; n_short <= 8'd0;
    end else if (evaluate) begin
      n_evals <= n_evals + 8'd1;
      if (truly_short) n_short <= n_short + 8'd1;
    end
  end
endmodule

Eight peaks of two hundred and fifty-six gigabytes landing together on a thousand-gigabyte pool is two thousand and forty-eight wanted, a thousand and twenty-four served — half the demand unmet.

Figure 2 — the assumption the saving is bought with. The upper path is not a careless assumption; it is the correct one on almost every day, and it is what makes the pool cheaper than buying each host its peak. The lower path is the same pool on the day the peaks correlate. The design question is not whether to oversubscribe — that is the whole point — but by how much, and what happens when the assumption fails, and an answer that does not name a degradation behaviour has only answered half.

The third case is what makes this hard to argue about. One peak at a time is fully served — the oversubscription is correct, the saving is real, and nothing is wrong. The model is explicit that the assumption holds almost all of the time, which is exactly what makes the exception expensive: there is no gradual signal, no period of mild degradation, no warning.

The practical content of the section is therefore not "do not oversubscribe" but "decide what happens when it fails." The options are real and choosable: refuse the allocation, degrade a lower-priority tenant, or fall back to a slower tier. An answer that names one of those has designed something; an answer that stops at the ratio has not.

There is a second question hiding behind the degradation behaviour and it is the one that decides which option is available. Does the pool know whose demand matters more? Refusing an allocation treats every host equally, which is the right answer when they are equal and the wrong one when they are not. Degrading a lower-priority tenant needs priorities to exist, which is a policy input the fabric manager has to be given. Falling back to a slower tier needs a slower tier to exist. Each option is a prerequisite as much as a choice, and the cheapest of the three to add later is refusal, which is why it is what most deployments end up with by default.

The sixth case is worth a note because it makes the arithmetic vivid. A thousand hosts peaking together saturates the demand figure and serves one percent of it — which is not a realistic deployment but is the limit the shape points at, and the shape is what matters. Oversubscription is a bet whose payoff is linear in the ratio and whose loss is bounded only by how many hosts can correlate at once, and the second of those is a property of what the hosts are doing rather than of how many there are.

It is worth separating two things that sound alike. Oversubscription in capacity is not the same as oversubscription in bandwidth, and a pool can be conservative on one and reckless on the other. Section 7 is capacity: more demand than the pool holds. Section 10 is bandwidth: more traffic than the device can serve, at any capacity assignment. A design can size the capacity carefully against coincident peaks and still put eight bandwidth-hungry tenants on one device, and the two mistakes have nothing to do with each other beyond the word they share.

8. RTL 4 — There Is A Fabric In The Middle

The fourth decision, and the one people skip because it is the boring part of the diagram.

Any host reaching any device needs something between them. Directly attaching every host to every device needs a port per pair, which is a product, and products do not scale. A switch needs a port per host plus a port per device, which is a sum, and sums do.

That is the whole argument for a fabric and it is worth being able to say in one line, because the follow-up is the interesting part: the switch costs money, adds a hop to every access, and becomes something else that can fail.

// RTL 4 - there is a fabric in the middle and it is not free. Any host reaching
// any device needs a switch, the switch needs ports, and every access pays a
// hop it would not have paid to a directly attached device.
module fabric_cost #(parameter int DRAW_A_LINE_TO_EVERY_DEVICE = 0) (
  input  logic clk, rst_n,
  input  logic        evaluate,
  input  logic [15:0] hosts, devices, port_cost, hop_ns,
  output logic [15:0] ports_needed, direct_ports, fabric_spend, added_lat_ns,
  output logic        fabric_ok,
  output logic [7:0]  n_evals, n_unbuildable,
  output logic        fabric_err
);
  logic [15:0] true_direct;
  logic [31:0] d_q, s_q;
  logic        truly_unbuildable;
  // A switch needs one port per host and one per device.
  assign ports_needed = hosts + devices;
  // Direct attachment needs a port per pair, which is what does not scale.
  assign d_q = {16'd0, hosts} * {16'd0, devices};
  assign true_direct = (d_q > 32'hFFFF) ? 16'hFFFF : d_q[15:0];
  assign direct_ports = (DRAW_A_LINE_TO_EVERY_DEVICE != 0) ? ports_needed : true_direct;
  assign s_q = {16'd0, ports_needed} * {16'd0, port_cost};
  assign fabric_spend = (s_q > 32'hFFFF) ? 16'hFFFF : s_q[15:0];
  assign added_lat_ns = hop_ns;
  assign fabric_ok = (direct_ports <= ports_needed);
  assign truly_unbuildable = (true_direct > ports_needed) && (hosts != 16'd0);
  assign fabric_err = evaluate && truly_unbuildable && fabric_ok;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_evals <= 8'd0; n_unbuildable <= 8'd0;
    end else if (evaluate) begin
      n_evals <= n_evals + 8'd1;
      if (truly_unbuildable) n_unbuildable <= n_unbuildable + 8'd1;
    end
  end
endmodule

Thirty-two hosts and sixteen devices is forty-eight switch ports against five hundred and twelve direct connections — and a hundred and fifty nanoseconds of hop on every access that crosses it.

The third case is the crossover and it is a nice number to know: at two hosts and two devices the two approaches cost the same. Below that, direct attachment wins and a switch is overhead. That is the honest form of the argument — a fabric is not always right, it is right at scale, and the scale at which it becomes right is small.

The fifth case isolates the real constraint. A fabric whose ports cost nothing still cannot be replaced by direct attachment, because the limit is the port count rather than the port price. A design conversation that argues about switch cost has usually missed that the alternative does not exist at the scale being discussed.

The fourth case is the one that keeps the model honest about what it is counting. Devices with no hosts attached need no fabric at all — sixteen ports for the devices and nothing to connect them to — and the model reports it as buildable rather than as an error. A pool being stood up before its hosts arrive is a normal state and a model that alarmed on it would be reporting a schedule as a defect.

The hop deserves a sentence of its own, because it is the cost that never goes away. Every access through the fabric pays it, forever, on a tier that was already further away than attached memory — so the fabric's latency adds to 27.4 section 6's ratio rather than being a separate budget. A design that is comfortably inside the latency budget directly attached can be outside it through two switch hops.

The failure modes of the fabric itself are a whole chapter and it is 26.5: credit that runs out on one hop while the topology dump says the path is up, routing tables that disagree between switches, cycles that burn capacity, and head-of-line blocking that stops flows with no relationship to the congested one. Adding a fabric adds that entire failure family to a memory system that previously had none of it, and an honest design answer says so rather than treating the switch as a wire with ports.

9. RTL 5 — One Host Must Not Read Another's Memory

The fifth decision, and the one that turns a performance feature into a security property.

A pool is a machine that hands the same physical DRAM to different tenants in turn. When a range moves from host A to host B, whatever host A left in it is still there unless something removed it, and host B can read it at memory speed with no fault, no log and nothing to distinguish it from its own data.

There are two mechanisms and a design needs one of them to work. Gating: the fabric enforces that a host can only address ranges assigned to it, so the old contents are unreachable. Clearing: the contents are wiped on reassignment, so there is nothing to find. Either is sufficient; neither is automatic.

// RTL 5 - one host must not read another's memory. A pool is a machine that
// hands the same DRAM to different tenants in turn, and the thing standing
// between them is a gate somebody has to have built and somebody has to have
// tested.
module tenant_isolation #(parameter int A_REASSIGN_IS_A_WIPE = 0) (
  input  logic clk, rst_n,
  input  logic        audit,
  input  logic [15:0] gb_assigned, gb_gated, gb_wiped, peer_reads,
  output logic [15:0] guarded_gb, exposed_gb, leaked_gb, guarded_pct,
  output logic        isolated,
  output logic [7:0]  n_audits, n_exposed,
  output logic        isolation_err
);
  logic [15:0] gated_ok, wiped_ok, true_exposed;
  logic [31:0] g_q, l_q;
  logic        truly_exposed;
  // A range is guarded if the fabric gates it or the contents were cleared.
  assign gated_ok = (gb_gated > gb_assigned) ? gb_assigned : gb_gated;
  assign wiped_ok = (gb_wiped > gb_assigned) ? gb_assigned : gb_wiped;
  assign guarded_gb = (gated_ok > wiped_ok) ? gated_ok : wiped_ok;
  assign true_exposed = gb_assigned - guarded_gb;
  assign exposed_gb = (A_REASSIGN_IS_A_WIPE != 0) ? 16'd0 : true_exposed;
  assign l_q = (gb_assigned == 16'd0) ? 32'd0
             : (({16'd0, true_exposed} * {16'd0, peer_reads}) / {16'd0, gb_assigned});
  assign leaked_gb = (l_q > {16'd0, peer_reads}) ? peer_reads : l_q[15:0];
  assign g_q = (gb_assigned == 16'd0) ? 32'd100
             : (({16'd0, guarded_gb} * 32'd100) / {16'd0, gb_assigned});
  assign guarded_pct = (g_q > 32'd100) ? 16'd100 : g_q[15:0];
  assign isolated = (exposed_gb == 16'd0);
  assign truly_exposed = (true_exposed != 16'd0) && (peer_reads != 16'd0);
  assign isolation_err = audit && truly_exposed && isolated;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_audits <= 8'd0; n_exposed <= 8'd0;
    end else if (audit) begin
      n_audits <= n_audits + 8'd1;
      if (truly_exposed) n_exposed <= n_exposed + 8'd1;
    end
  end
endmodule

Half the assigned capacity gated and none of it cleared leaves two hundred and fifty-six gigabytes exposed and five hundred peer reads reaching another tenant's data.

The third case is the design freedom and it is worth stating explicitly. Either mechanism suffices on its own — a fully gated range needs no wipe, and a fully wiped range needs no gate. That means a design has a genuine choice, and the choice has a cost profile: gating is enforcement on every access and wiping is a one-off cost at reassignment time that makes reassignment slower, which feeds directly back into section 6.

The fifth case is the one to be careful about. An exposure nobody has exploited is still an exposure — the model reports zero leaked reads because nobody looked, and reports the capacity as unguarded regardless. A design review that accepts "we have never seen a cross-tenant read" has accepted the absence of evidence.

The reason this is a design problem rather than an operational one is in section 20's last line: the failure leaves no trace by construction. A tenant reading a previous tenant's data performs a successful read of memory it legitimately owns. There is no fault, no exception, no log entry and nothing anomalous about the access — the data is simply not what the reader expected to find, and in the common case the reader does not know what it expected to find. Detection is not available, which means the mechanism has to be right rather than monitored.

Two further design notes belong here because they are the follow-ups an interviewer with a security background will reach for.

Wiping is not free and its cost lands on reassignment. Clearing a terabyte takes real time on real memory bandwidth, and that time is added directly to section 6's reassignment latency. A design that chooses wiping over gating has traded a security mechanism for a speed one, which is a legitimate trade and needs to be a deliberate one.

Gating has to survive the reassignment, not merely exist. The dangerous window is not steady state — it is the moment between one host releasing a range and the next receiving it, when the fabric's own tables are being rewritten. A gate that is correct before and after and open during is a gate with a hole in it, and it is exactly the kind of transient that a steady-state test never touches.

There is one more mechanism that is sometimes offered and is worth being sceptical about in an interview. Encryption per tenant — each host's range encrypted with a key it alone holds — makes a stale read return noise rather than data, which is a real mitigation. It is also not a gate: the previous tenant's ciphertext is still readable, it is still a side channel, and the key management is a system of its own. As a defence in depth it is worth having; offered as the answer to section 9 it is a way of not choosing between gating and wiping.

10. RTL 6 — Capacity Is Assigned And Bandwidth Is Contended

The sixth decision, and the one that is invisible in every capacity diagram ever drawn.

Assignment divides capacity. Two hosts given separate two-hundred-and-fifty-six gigabyte ranges on one device have entirely separate memory, cannot see each other's data, and are isolated in every sense the previous section cares about. They share the device's bandwidth completely, and nothing in the capacity assignment mentions it.

This produces the classic noisy-neighbour problem in a place people do not expect it, because the isolation is real and is about the wrong axis.

// RTL 6 - capacity is assigned and bandwidth is contended. Two hosts given
// separate ranges on one device share one device's bandwidth, and nothing in
// the capacity assignment says so.
module bandwidth_share #(parameter int SEPARATE_RANGES_ARE_SEPARATE = 0) (
  input  logic clk, rst_n,
  input  logic        measure,
  input  logic [15:0] pool_bw, hosts, demand_each, quota_bw,
  output logic [15:0] total_demand, share_each, served_each, starved_hosts,
  output logic        bandwidth_fair,
  output logic [7:0]  n_measures, n_starved,
  output logic        fairness_err
);
  logic [15:0] true_share, true_starved;
  logic [31:0] t_q, s_q;
  logic        truly_starved;
  assign t_q = {16'd0, hosts} * {16'd0, demand_each};
  assign total_demand = (t_q > 32'hFFFF) ? 16'hFFFF : t_q[15:0];
  // One device's bandwidth divided among the hosts asking for it.
  assign s_q = (hosts == 16'd0) ? 32'd0 : ({16'd0, pool_bw} / {16'd0, hosts});
  assign true_share = (s_q > 32'hFFFF) ? 16'hFFFF : s_q[15:0];
  assign share_each = (SEPARATE_RANGES_ARE_SEPARATE != 0) ? pool_bw : true_share;
  assign served_each = (demand_each > share_each) ? share_each : demand_each;
  assign true_starved = (demand_each > true_share) ? hosts : 16'd0;
  assign starved_hosts = (SEPARATE_RANGES_ARE_SEPARATE != 0) ? 16'd0 : true_starved;
  assign bandwidth_fair = (starved_hosts == 16'd0) || (quota_bw != 16'd0);
  assign truly_starved = (true_starved != 16'd0) && (quota_bw == 16'd0);
  assign fairness_err = measure && truly_starved && bandwidth_fair;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_measures <= 8'd0; n_starved <= 8'd0;
    end else if (measure) begin
      n_measures <= n_measures + 8'd1;
      if (truly_starved) n_starved <= n_starved + 8'd1;
    end
  end
endmodule

Eight hosts on a sixty-four unit pool, each wanting sixteen, is eight units each — every host starved, on a design where each of them has its own capacity and nothing is oversubscribed.

The third case is the answer to the section and it is a design decision rather than a fix. A declared bandwidth quota makes the limit a policy rather than a fault — the hosts are still limited, the model still reports the limit, and it stops calling it a fairness failure because somebody chose it. That distinction is the whole content: the problem is not that bandwidth is shared, it is that it is shared by accident.

The last case shows the scaling. A thousand hosts each wanting a thousand units get nothing each once the pool is divided, and the separate-ranges view still reports none of them starved. The larger the pool, the further the capacity story and the bandwidth story diverge.

It is worth noting that this is not unique to pooling and is worse in it. Two processes on one machine share its memory bandwidth too, and operating systems have decades of mechanisms for noticing and managing that. What is new is that the contending parties are different machines, with no shared scheduler, no shared view of the contention, and no mechanism by which either could notice the other exists. The isolation that makes pooling safe is exactly what removes the visibility that would let the problem be managed.

11. RTL 7 — Capacity Moves In Chunks

The seventh decision, and the one that quietly eats the saving the whole architecture was bought for.

Memory is not assigned by the byte. A fabric manager hands over capacity in chunks — some quantum determined by the device's decoder granularity and the manager's own bookkeeping — and a host wanting seventy gigabytes with a sixty-four gigabyte chunk gets a hundred and twenty-eight.

The rounding is per host and per assignment, so it multiplies across the fleet, and it is subtracted directly from the stranded capacity the pool was reclaiming.

// RTL 7 - capacity moves in chunks. A pool that assigns in sixty-four gigabyte
// units to a host that wants seventy hands over a hundred and twenty-eight, and
// the rounding is subtracted from the saving the pool was bought for.
module assign_granularity #(parameter int ASSIGN_WHAT_THEY_ASKED_FOR = 0) (
  input  logic clk, rst_n,
  input  logic        evaluate,
  input  logic [15:0] chunk_gb, want_gb, hosts, pool_gb,
  output logic [15:0] chunks_given, given_gb, waste_per_host, fleet_waste,
  output logic        granular,
  output logic [7:0]  n_evals, n_wasteful,
  output logic        granularity_err
);
  logic [15:0] true_waste, given_true;
  logic [31:0] c_q, g_q, f_q;
  logic        truly_wasteful;
  // Rounded up to whole chunks.
  assign c_q = (chunk_gb == 16'd0) ? 32'd0
             : (({16'd0, want_gb} + {16'd0, chunk_gb} - 32'd1) / {16'd0, chunk_gb});
  assign chunks_given = (c_q > 32'hFFFF) ? 16'hFFFF : c_q[15:0];
  assign g_q = {16'd0, chunks_given} * {16'd0, chunk_gb};
  assign given_true = (g_q > 32'hFFFF) ? 16'hFFFF : g_q[15:0];
  assign given_gb = (ASSIGN_WHAT_THEY_ASKED_FOR != 0) ? want_gb : given_true;
  assign true_waste = (given_true > want_gb) ? (given_true - want_gb) : 16'd0;
  assign waste_per_host = (ASSIGN_WHAT_THEY_ASKED_FOR != 0) ? 16'd0 : true_waste;
  assign f_q = {16'd0, true_waste} * {16'd0, hosts};
  assign fleet_waste = (f_q > 32'hFFFF) ? 16'hFFFF : f_q[15:0];
  assign granular = (waste_per_host == 16'd0);
  assign truly_wasteful = (true_waste != 16'd0) && (pool_gb != 16'd0);
  assign granularity_err = evaluate && truly_wasteful && granular;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_evals <= 8'd0; n_wasteful <= 8'd0;
    end else if (evaluate) begin
      n_evals <= n_evals + 8'd1;
      if (truly_wasteful) n_wasteful <= n_wasteful + 8'd1;
    end
  end
endmodule

Sixty-four gigabyte chunks against a host wanting seventy is fifty-eight gigabytes wasted per host — five hundred and eighty across ten hosts, which is most of a device.

The sixth case shows where the parameter matters. A two-hundred-and-fifty-six gigabyte chunk against the same request wastes a hundred and eighty-six — the waste scales with the chunk rather than with the request, which means the design decision is entirely in the chunk size and not at all in the workload.

The tension is with section 6. A smaller chunk wastes less and makes every assignment more work, because the manager has more decisions to make and more bookkeeping to hold. A design that minimises rounding waste by making the chunk tiny has made reassignment slower, and slower reassignment is section 6's failure. The two pull against each other and the answer is a number somebody has to choose, which is exactly the kind of decision a whiteboard question is asking for.

There is a partial escape worth knowing about: the waste depends on the relationship between the chunk and the request, so shaping the requests helps as much as shrinking the chunk. A fleet whose hosts ask for capacity in multiples of the chunk wastes nothing at any chunk size. That turns a hardware parameter into a policy one — the fabric manager rounds the request up before deciding, and the host is told what it got — and it costs nothing except the willingness to have the conversation with whoever is asking.

12. RTL 8 — Something Decides Who Gets What

The eighth piece, and the one that is a component rather than a property.

A pool has a fabric manager. It knows what capacity exists, what each host has, and what each host is asking for, and it issues the configuration that moves capacity between them. It is software, it runs somewhere, and it can be down.

When it is, the pool does not fail in the way memory fails. Every existing assignment keeps working — the ranges are mapped, the decoders are programmed, the hosts carry on. What stops is change. Capacity is frozen wherever it was, and a host that needs more does not get it, which is a failure that looks like nothing at all from every host's point of view.

// RTL 8 - something decides who gets what, and it can be down. A pool with no
// manager is a set of ranges frozen in whatever assignment they had when the
// manager stopped, which is a different failure from the memory failing.
module management_plane #(parameter int THE_FABRIC_DECIDES = 0) (
  input  logic clk, rst_n,
  input  logic        evaluate,
  input  logic [15:0] mgr_up, decisions_wanted, decisions_made, gb_per_decision,
  output logic [15:0] served_decisions, stuck_decisions, stuck_gb, served_pct,
  output logic        mgmt_ok,
  output logic [7:0]  n_evals, n_frozen,
  output logic        mgmt_err
);
  logic [15:0] true_stuck, made_ok;
  logic [31:0] g_q, s_q;
  logic        truly_frozen;
  // Decisions are made only while something is there to make them.
  assign made_ok = (decisions_made > decisions_wanted)
                   ? decisions_wanted : decisions_made;
  assign served_decisions = (mgr_up == 16'd0) ? 16'd0 : made_ok;
  assign true_stuck = decisions_wanted - served_decisions;
  assign stuck_decisions = (THE_FABRIC_DECIDES != 0) ? 16'd0 : true_stuck;
  assign g_q = {16'd0, true_stuck} * {16'd0, gb_per_decision};
  assign stuck_gb = (g_q > 32'hFFFF) ? 16'hFFFF : g_q[15:0];
  assign s_q = (decisions_wanted == 16'd0) ? 32'd100
             : (({16'd0, served_decisions} * 32'd100) / {16'd0, decisions_wanted});
  assign served_pct = (s_q > 32'd100) ? 16'd100 : s_q[15:0];
  assign mgmt_ok = (stuck_decisions == 16'd0);
  assign truly_frozen = (true_stuck != 16'd0) && (gb_per_decision != 16'd0);
  assign mgmt_err = evaluate && truly_frozen && mgmt_ok;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_evals <= 8'd0; n_frozen <= 8'd0;
    end else if (evaluate) begin
      n_evals <= n_evals + 8'd1;
      if (truly_frozen) n_frozen <= n_frozen + 8'd1;
    end
  end
endmodule

Twenty reassignments wanted with the manager down is twelve hundred and eighty gigabytes frozen where they were — not lost, not corrupted, not unavailable to their current owner, and not movable.

The sixth case is the one that is more common than the outage. A manager that is up and falling behind serves five of twenty and freezes fifteen — a partial failure that produces a pool which mostly works, which is harder to notice and harder to attribute than one that has stopped.

The design consequence is a question worth asking out loud: what is the pool's behaviour when the manager is unavailable, and is that acceptable for the time it takes to restore it? For a pool whose demand moves daily, an hour of frozen assignment is nothing. For one whose demand moves by the minute — section 6's fast case — it is the same failure as a pool that cannot reassign at all, arriving intermittently.

The design response is the standard one for a control plane and it is worth saying in those terms, because it signals that the problem has been recognised for what it is. The manager's state must survive its own restart, which means it is persisted rather than reconstructed; two managers must not both believe they are in charge, which means leader election; and a host must not be left with a half-applied assignment when the manager dies mid-decision, which means the assignment sequence is idempotent and resumable. None of that is specific to memory, and all of it is what makes the difference between a pool that recovers and a pool that needs a human.

13. RTL 9 — What The Pool Saves Against What It Costs

Nine sections of design. This one is the balance sheet, and it is the part an answer most often gives only half of.

The credit is the reclaimed capacity, priced at what the DRAM would have cost. 27.4 section 12 computes it: stranded memory, made available, bought once instead of twice.

The debit is the devices, the switch and the ports. A pool is not free capacity; it is capacity plus a fabric, and the fabric is a real purchase with a real number on it.

// RTL 9 - what the pool saves against what the pool costs. The reclaimed
// capacity is a credit and the devices, the switch and the ports are a debit,
// and a design answer that names only the credit has answered half.
module pooling_tco #(parameter int COUNT_THE_SAVING_ONLY = 0) (
  input  logic clk, rst_n,
  input  logic        evaluate,
  input  logic [15:0] reclaimed_gb, dram_per_gb, pool_build, switch_build,
  output logic [15:0] credit, debit, net_saving, saving_pct,
  output logic        pooling_wins,
  output logic [7:0]  n_evals, n_unprofitable,
  output logic        tco_err
);
  logic [15:0] true_debit;
  logic [31:0] c_q, p_q;
  logic        truly_unprofitable;
  assign c_q = {16'd0, reclaimed_gb} * {16'd0, dram_per_gb};
  assign credit = (c_q > 32'hFFFF) ? 16'hFFFF : c_q[15:0];
  assign true_debit = pool_build + switch_build;
  assign debit = (COUNT_THE_SAVING_ONLY != 0) ? 16'd0 : true_debit;
  assign net_saving = (credit > debit) ? (credit - debit) : 16'd0;
  assign p_q = (credit == 16'd0) ? 32'd0
             : (({16'd0, net_saving} * 32'd100) / {16'd0, credit});
  assign saving_pct = (p_q > 32'd100) ? 16'd100 : p_q[15:0];
  assign pooling_wins = (net_saving != 16'd0);
  assign truly_unprofitable = (true_debit >= credit) && (reclaimed_gb != 16'd0);
  assign tco_err = evaluate && truly_unprofitable && pooling_wins;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_evals <= 8'd0; n_unprofitable <= 8'd0;
    end else if (evaluate) begin
      n_evals <= n_evals + 8'd1;
      if (truly_unprofitable) n_unprofitable <= n_unprofitable + 8'd1;
    end
  end
endmodule

A thousand gigabytes reclaimed at ten units each is ten thousand of credit against five thousand of build cost: five thousand net, half the credit surviving. A hundred gigabytes reclaimed against the same build is a thousand of credit against five thousand of cost — and the saving-only view books the thousand as a win.

The third case is the boundary and the model is deliberate about which side it belongs to. Breaking even is not winning — equal credit and debit is a great deal of engineering effort for nothing, and a design that lands there should be a decision rather than an accident. The saving-only view calls it a win, which is how a break-even proposal gets approved.

There is a term missing from both columns that a careful answer names. The pool's credit assumes the reclaimed capacity is actually usable by whoever needs it — which requires section 6's reassignment to be fast enough, section 11's chunk not to have rounded it away, and section 7's peaks not to have wanted it simultaneously. The balance sheet in this model takes the reclaimed figure as an input, and every earlier section in the chapter is a reason that input might be smaller than the proposal assumed. That is the sense in which this section is last: it depends on all the others being right.

The last case is the shape to watch for: fifty gigabytes reclaimed against a sixty-thousand-unit build. A pool can be built at any scale and the fabric cost does not shrink proportionally, so a small pool is a poor pool — and "we will start small and grow it" is a plan that begins with the least favourable arithmetic the architecture has.

14. RTL 10 — A Pooled-Memory Design Assembled

Nine decisions. This one puts them in one place and makes the opening sentence visible as what it is: one bit of six.

// RTL 10 - a pooled-memory design assembled. Everything that must hold before
// pooled memory does what it was bought to do, with "the memory is shared" as
// one of the six rather than the whole design.
module pool_design_signoff #(parameter int THE_MEMORY_IS_SHARED = 0) (
  input  logic clk, rst_n,
  input  logic       evaluate,
  input  logic       pooled,          // capacity is in a pool a fabric can assign
  input  logic       reassignable,    // it moves inside the demand's own timescale
  input  logic       pool_sized,      // sized against coincident peaks, not averages
  input  logic       isolated,        // one tenant cannot read another's
  input  logic       bandwidth_fair,  // one host cannot take the pool's bandwidth
  input  logic       granular,        // the assignment chunk does not eat the saving
  output logic       design_sound,
  output logic [5:0] fail_mask,
  output logic [7:0] n_eval, n_sound,
  output logic       false_pool_err
);
  assign fail_mask[0] = ~pooled;
  assign fail_mask[1] = ~reassignable;
  assign fail_mask[2] = ~pool_sized;
  assign fail_mask[3] = ~isolated;
  assign fail_mask[4] = ~bandwidth_fair;
  assign fail_mask[5] = ~granular;
  // The memory-is-shared build stops at the first bit.
  assign design_sound = (THE_MEMORY_IS_SHARED != 0) ? pooled : (fail_mask == 6'd0);
  assign false_pool_err = evaluate && design_sound && (fail_mask != 6'd0);

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_eval <= 8'd0; n_sound <= 8'd0;
    end else if (evaluate) begin
      n_eval <= n_eval + 8'd1;
      if (design_sound) n_sound <= n_sound + 8'd1;
    end
  end
endmodule

The stimulus walks all six bits one at a time. When the memory is pooled and any one of the other five fails, the assembled model reports that the design is not sound and the shared view reports a pool.

Across the eight evaluations, the assembled model calls one design sound and the shared view calls six of them a pool.

The bit order is by when the decision has to be made. Bit 0 is an architectural commitment made in the first sentence. Bit 1 is a requirement gathered from the workload. Bit 2 is a sizing decision. Bits 3 and 4 are enforcement mechanisms chosen during design. Bit 5 is a parameter chosen at the end — and it is last because it is the only one that can be changed after the thing is built.

Figure 4 — the mask ordered by when the decision has to be made. The first is an architectural commitment in the opening sentence, the middle three are chosen during design, and the last is a parameter — and it is last because it is the only one of the six that can still be changed after the pool is built.

15. Quantitative Reasoning

Five hundred and twelve gigabytes overclaimed by describing assignment as simultaneous sharing — a promise of a coherency protocol spanning every host in the pool, made in one word.

A hundred periods of a hundred missed, from a pool that reassigns in an hour against a demand that moves every minute. Twenty-five thousand six hundred gigabyte-periods.

Half the demand unmet when eight peaks land together on a pool sized for four — with no gradual signal beforehand, because the oversubscription is correct on every other day.

Forty-eight switch ports against five hundred and twelve direct connections. A sum against a product, and the crossover is at two by two.

Five hundred peer reads reaching another tenant's data, at memory speed, on a range that is half gated and not cleared.

Eight hosts, sixty-four units of bandwidth, eight each, all eight starved — on a design where every host has its own capacity and nothing is oversubscribed.

Fifty-eight gigabytes wasted per host by a sixty-four gigabyte chunk against a seventy gigabyte request. Five hundred and eighty across ten hosts, and a hundred and eighty-six each at a two-hundred-and-fifty-six gigabyte chunk.

Twelve hundred and eighty gigabytes frozen by a manager that is down — not lost, not corrupted, and not movable.

Ten thousand of credit against five thousand of cost is half the saving surviving. A hundred gigabytes reclaimed against the same build is a loss the saving-only view reports as a win.

One design of eight sound; the shared view counts six. The assembled model's summary, and the chapter's.

One number that does not appear in the models and belongs in any real balance sheet: the pool's own power. Devices, switch ports and a manager all draw continuously, and in a facility where power rather than capital is the binding constraint that can dominate the DRAM saving entirely. The models here price capital because capital is what a whiteboard answer is usually asked about; a deployment answer that stops there has left out the operating cost of the thing it just built.

16. Assertions

The testbenches carry 517 checks across ten models.

Every output of every model is asserted as a value, in both builds. The output listing step reported twenty-three and two were real gaps, both of the same shape: a total or a percentage derived from the parameter-dependent value while its neighbours were pinned. The reassignment model's capacity-lost figure reports twenty-five thousand six hundred in the measured build and zero in the other; the balance-sheet model's saving percentage reports fifty against a hundred. Both are the section's headline number.

That makes it four chapters running where the outscan gap was a derived figure rather than a primary one. The primary values get asserted because they are what the prose quotes; the figure computed from them one step later does not, and it is the one a mutation can corrupt while every asserted neighbour stays correct.

Both builds are asserted on every degenerate case. An empty pool, a pool with nothing assigned, a workload with no periods, a pool with no capacity, a topology with nothing to connect, an unassigned range, a pool with no hosts, a host asking for nothing, a manager with nothing to decide, a pool that reclaims nothing.

Every clamp that an input can reach is driven past its limit exactly once. More assigned than the pool holds, more claimed than the pool holds, a gate covering more than was assigned, a decision log longer than the decisions wanted, demand and connection counts that saturate.

Every error output is checked in both directions in every case. Section 6's fourth case, section 7's third case, section 9's third case and section 10's third case exist to assert the quiet half — a slow pool nobody needed, one peak at a time, a wiped range with no gate, and a declared quota. Each of those is a case where the design is fine and a naive model would alarm.

17. Mutation Testing

106 mutations, 106 killed. Fifty against the first testbench, fifty-six against the second, both clean on the first run.

Both sides clean on the first run, which is the first chapter in two batches where that happened. The reason is worth recording because it is not luck: the two survivor classes that cost 27.3 and 27.4 a run each — a zero-guard whose degenerate case was under-specified — were pre-empted here by driving every zero-guard with the other inputs non-zero, which is the rule 27.4 section 17 extracted from its own survivor.

That is the pattern the batch keeps producing. A survivor names a class; the class becomes a rule; the rule is applied while drafting rather than after a mutation asks; and the class stops appearing. The split check did the same thing one batch earlier, and the two rules together are why both campaigns here needed no second run.

It is worth being precise about what that does and does not prove. It does not prove the models are correct — the campaign checks the testbench, not the design. What it shows is that the assertions were load-bearing on the first attempt, which is a property of how the stimulus was written rather than of how the models were. The two rules that produced it are both about stimulus: unbalance the counter splits, and drive the zero-guards with the other inputs non-zero. Neither is about the code under test at all.

Three mutations were designed and discarded as equivalent before injection, all of the loosened-comparison shape in models where no stimulus produced the equality.

18. Verification Strategy

Verify reassignment under load, not at rest. Section 6's timing is measured while hosts are using the capacity, and a reassignment benchmarked on an idle pool measures the easy case.

Correlate the hosts deliberately. Section 7's failure needs coincident peaks, which a random workload will not produce often enough to find. Drive them together on purpose.

Test the fabric's port count at the target scale, not the prototype's. Section 8's argument only appears past two by two.

Read a reassigned range before writing it. Section 9's test is one read, and it is the only way to find out whether the gate or the wipe is actually happening.

Measure per-host bandwidth with every host active. Section 10 is invisible with one host and obvious with eight.

Request capacity that is not a whole number of chunks. Section 11's rounding does not appear if every test asks for exactly one chunk.

Kill the fabric manager and check what still works. Section 12. The answer should be "everything except change", and confirming that is a five-minute test nobody runs.

Put the build cost in the same document as the saving. Section 13. The two numbers are usually owned by different people and the comparison is what the decision needs.

19. Synthesis and Implementation Reality

Reassignment time is dominated by what the host has to do, not by the fabric. Unmapping a range, flushing caches, quiescing anything using it — the fabric's own reconfiguration is usually the smallest part, which means section 6's number is a software property rather than a hardware one.

Gating is enforcement per access and wiping is a cost per reassignment. Section 9's choice is between a continuous overhead and a periodic one, and the right answer depends on how often reassignment happens — which is section 6's number again.

The chunk size is a decoder property and a bookkeeping property at once. Section 11's parameter is bounded below by what the device's address decoders can express, which is a hardware limit a design cannot argue with.

A bandwidth quota needs enforcement somewhere. Section 10's fix is easy to state and needs a mechanism — in the device, in the switch, or in a host-side rate limiter — and each of those has a different owner.

The fabric manager is a distributed-systems problem wearing a memory hat. It holds state that must survive its own restart, it must not make two conflicting assignments, and it is the single point whose failure freezes the pool. Section 12 is the consequence and the design work is the usual one.

Two switch hops is a different latency budget from one. Section 8's hop adds to 27.4 section 6's ratio, and a topology chosen for port count can put a host two hops from its memory.

20. Silicon Observability

Free, already there. Current assignment: which ranges belong to which host. Every fabric manager has this and it answers nothing on its own.

Cheap. Reassignment latency, measured end to end. Section 6 is this number against the workload's period.

Cheap. Requested capacity against assigned capacity per host. Section 11's rounding waste is the difference, summed.

Moderate. Per-host bandwidth at the device. Section 10 is invisible without it and it is the counter most often absent, because bandwidth is not what a pooling product is sold on.

Moderate. Coincident-demand history — how often more than N hosts wanted their peak simultaneously. Section 7's sizing decision needs this and it can only be gathered by running.

Expensive. A protocol analyser on a fabric link, which shows the access mix and the contention directly. 26.7 is about using one well, and 26.5 is what it finds.

Unobtainable. Whether a tenant read data left by a previous one. Section 9's failure leaves no trace by construction — it is a successful read of valid memory — which is why the design has to prevent it rather than detect it.

21. Debug Lab

A pooled deployment where hosts periodically fail to get the capacity they ask for.

Step 1 — check whether the request was refused or was never served. Section 12 against section 7: a manager that is down produces no decision, and a pool that is short produces a refusal. Different failures, different fixes, and the distinction is in the manager's own log.

Step 2 — if it was refused, look at coincident demand. Section 7. How many hosts wanted their peak at that moment, and what was the pool sized for?

Step 3 — if it was served but late, measure reassignment latency against the demand period. Section 6.

Step 4 — compare requested capacity against assigned capacity, summed over hosts. Section 11. If the difference is large, the pool is smaller than it looks.

Step 5 — if the complaint is performance rather than capacity, measure per-host bandwidth. Section 10, then 26.6.

Step 6 — if the complaint is a hop's worth of latency, count the hops. Section 8, then 26.5.

Steps 1 to 4 are all reads from the fabric manager, which makes this an unusually cheap investigation — the whole of the capacity side is observable from one component, which is the opposite of 27.3's situation.

The exception is section 10, and it is worth calling out because it breaks the pattern. Per-host bandwidth is not visible from the fabric manager — the manager assigns capacity and has no view of the traffic — so the one failure in this chapter that produces a continuous, reproducible, tenant-visible symptom is the one the pool's own control plane cannot see. That is why it is the ticket that gets misrouted: every component reports health, and the complaint is about a quantity none of them measures.

22. Design Review

Is the memory assigned to one host at a time, or simultaneously shared? Which did the design intend?

How fast can capacity move, and how fast does the workload need it to?

What is the oversubscription ratio, and what happens when the peaks coincide?

Is a reassigned range gated, wiped, or neither? Which, and has it been read-tested?

Is there a bandwidth quota, and what enforces it?

What is the assignment chunk, and what is the expected rounding waste across the fleet?

What happens to the pool when the fabric manager is down, and for how long is that acceptable?

How many switch hops is a host from its memory, worst case?

What is the build cost, and what is the reclaimed capacity worth?

23. How This Appears In Real Engineering

The pool is built, the capacity is reclaimed, the numbers in the proposal were right — and hosts periodically cannot get memory they asked for.

The cause is usually section 7 and the discussion that follows is usually the wrong one. The instinct is to grow the pool, which works and is expensive and treats a correlation problem as a capacity problem. The cheaper answer is almost always a degradation behaviour, which nobody specified because the design stopped at the ratio.

The second shape is section 10, and it is the one that generates the most confused tickets. Two tenants on one device, correctly isolated, separate ranges, no oversubscription — and one of them slows down whenever the other is busy. Everything about the capacity design is correct and the complaint is entirely real, which makes it very hard to route: the pooling team sees a working pool and the tenant sees a machine that is intermittently slow.

The third is section 11 and it is quiet. The pool reclaims less than the proposal said, by twenty or thirty percent, and nobody notices because the proposal's number was never measured against the outcome. The rounding is arithmetic that was available at design time and was not done, and the saving it consumes is the saving the pool exists for.

There is a fourth shape that shows up once a deployment has been running a while, and it belongs to section 6. The pool works, the reassignments happen, and over months the reassignment rate falls — not because the mechanism slowed down but because operators, having been burned once by a reassignment that took longer than expected, stopped requesting them. The pool becomes statically assigned by habit, the capacity stops moving, and the saving quietly reverts to what static provisioning would have achieved. Nothing in the system reports this, because from every component's point of view it is behaving correctly and simply being asked to do less.

The pattern underneath is that five of the six bits are decisions rather than defects. Nothing is broken in any of these stories. Somebody chose a ratio, a chunk, a mechanism or nothing at all, and the consequence arrived later and looked like a fault.

That has a consequence for how the work is reviewed. A defect is found by testing and a decision is found by asking, and a deployment review built around testing will pass a pool with all five decisions made badly. The six questions in section 22 take a few minutes between them and none of them is a measurement; they are a design review conducted after the design, which is unusual and is what this failure class requires.

24. Common Misconceptions

"The memory is shared." Assigned, one host at a time. Simultaneous sharing is a much larger commitment. Section 5.

"It's a pool, so capacity moves." How fast, against a demand that moves how fast? Section 6.

"We sized it for the sum of the peaks." Then it saves nothing. The saving is the oversubscription. Section 7.

"We'll connect the hosts to the devices." That is a product and it does not scale. Section 8.

"Reassignment gives the host a clean range." Only if something gates it or clears it. Section 9.

"Each host has its own capacity, so they're isolated." In capacity. They share the device's bandwidth entirely. Section 10.

"It assigns what the host asks for." In chunks, rounded up, per host, across the fleet. Section 11.

"If the manager dies the memory keeps working." It does. What stops is change. Section 12.

"The saving is the reclaimed DRAM." Minus the devices, the switch and the ports. Section 13.

25. Interview Reasoning

"Design me a pooled-memory architecture." Devices holding the memory, a CXL switch so any host can reach any device, and a fabric manager that assigns capacity. Capacity is assigned to one host at a time, not simultaneously shared. Size it below the sum of peaks — that oversubscription is the saving — and decide what happens when the peaks coincide. Gate or wipe on reassignment. Put a bandwidth quota on it. Choose a chunk size, knowing that smaller wastes less and makes reassignment slower.

"Why not share the memory between hosts?" Because that needs coherency spanning the hosts, which is a much larger commitment than assignment. The pooling case does not need it and buying it accidentally is the commonest design error on this question.

"How do you size the pool?" Below the sum of peaks, by the amount the peaks are expected to be independent. The saving is exactly that difference, which means the sizing decision and the business case are the same decision.

"What happens when several hosts peak together?" Whatever you designed to happen. Refuse the allocation, degrade a lower-priority tenant, or fall back to a slower tier. Not having an answer is the failure.

"Two tenants have separate ranges on one device. Are they isolated?" In capacity, if the fabric gates it. In bandwidth, not at all — they share the device completely, and that needs its own mechanism.

"What if the fabric manager goes down?" Existing assignments keep working. Nothing new can be assigned. Whether that is acceptable depends on how fast the demand moves, which is the same number as the reassignment question.

"Where does the saving go?" Into the fabric, the switch ports, the chunk rounding and whatever redundancy the availability requirement demands. An answer that quotes the reclaimed capacity as the saving has quoted the credit and not the balance.

26. Exercises

1. A pool of 4 TB serves 12 hosts. 2 TB is assigned one-host-at-a-time and 1 TB is claimed as simultaneously readable by three hosts. What has been promised, and what would delivering it require?

2. Reassignment takes 900 s. The workload's demand shifts every 300 s, over 40 periods, with 128 GB at stake each time. How many periods are served, and how much capacity-time is lost? What reassignment time would serve all of them?

3. Sixteen hosts peak at 192 GB each. Compute the sum. If the pool is 1.5 TB and six peak together, what is the shortfall? At what coincidence count does the pool first fall short?

4. Compute switch ports and direct connections for 64 hosts and 24 devices. At what host count does a switch become cheaper for 24 devices?

5. A 1 TB range is reassigned. 600 GB is gated and 200 GB is wiped. How much is guarded? If 5,000 peer reads are issued, how many reach the previous tenant's data?

6. Twelve hosts share a device with 96 units of bandwidth, each wanting 12. Compute the share and the starved count. What quota would make the limit a policy?

7. A chunk is 128 GB. Hosts want 130, 200, 260 and 400 GB. Compute the waste for each and the total. Now halve the chunk and recompute — what does that do to the manager's work?

8. Extend the assembled model with a seventh bit for a decision this chapter does not cover. Justify its position using the rule that the ordering is by when the decision has to be made.

27. Summary

Assigned is not shared. Capacity handed to one host at a time needs no coherency between hosts; memory two hosts read at once is a far larger commitment, and the difference is one word.

Reassignment has a period and so does demand. A pool slower than its workload is a set of static assignments with extra hardware in the path.

The saving is the oversubscription, and its failure mode is not gradual — it is fine until several peaks land together.

A fabric is a sum where direct attachment is a product, and the crossover is at two by two.

The same DRAM goes to a second tenant. Gate it or clear it; neither happens by itself, and the failure leaves no trace.

Capacity is assigned and bandwidth is contended. Eight isolated hosts on one device are eight hosts sharing one device's bandwidth.

Capacity moves in chunks, and the rounding is subtracted from the saving — fifty-eight gigabytes per host, five hundred and eighty across ten.

Something decides who gets what, and it can be down. What stops is not the memory; it is change.

The saving is the credit minus the fabric. Breaking even is not winning, and a small pool is a poor pool.

Six bits, and "the memory is shared" is one of them. One design of eight is sound; the shared view counts six.