CXL 2.0 Memory Pooling

20.1 built the switch. This is what it is for.

Memory pooling is the reason CXL 2.0 exists as a generation rather than an erratum. Every server in a fleet is provisioned for its own peak, every server spends most of its life below that peak, and the difference — memory bought, powered, and never touched — is the largest single line of waste in a modern data centre. Pooling converts that waste into capacity somebody else can use, and the entire argument rests on one arithmetic observation: hosts do not peak together.

Everything else in this chapter is what it costs to collect on that observation.

1. The Engineering Problem — The Saving Is Real And It Is Not Free

The saving is a statistical claim, and the statistics have to hold. Three hosts peaking at 128 GB each need 384 GB dedicated and 192 GB pooled — if their combined peak really is 192. If they peak together, pooling saves nothing and the pool is a switch you paid for. Section 5.

A pool hands out blocks, not bytes. A request that is not a multiple of the block takes the next whole one, and the remainder is capacity removed from the pool that no host can use. Section 7.

Free capacity and allocatable capacity are different numbers. A pool with 100 GB free in three runs of 40, 30 and 30 cannot satisfy a 60 GB request, and the number a capacity report shows is the one that cannot. Section 8.

A rebind is not a control-plane operation with a control-plane cost. Moving 32 GB between hosts means scrubbing 32 GB, and at 20 GB/s that is 1.6 seconds during which the capacity belongs to nobody. Section 9.

Capacity a host owns and never touches is the thing pooling was supposed to eliminate, and reclaiming it needs a definition of idle that is safe to act on. Section 11.

And the fabric manager's view of the pool is not the device's view. What was intended and what was applied diverge, and free capacity computed from the wrong one is a number nobody can allocate against. Section 14.

This chapter against 20.1, stated precisely. That one owns the switch as a component. This one owns the capacity model the switch makes possible — and every property here can fail on a switch that routes perfectly.

2. The One-Sentence Model

A pool is workable when it actually saves capacity, presents no more logical devices than it supports, has free capacity that is allocatable rather than merely present, can move capacity inside its time budget, adds capacity in the right order, and agrees with the fabric manager about what it holds — and every defect below is a pool that saves money on paper and fails one of the other five.

3. What This Chapter Owns

Deferred:

4. Teaching-Model Boundary

Every model is a small synchronous block isolating one property. A real pool is a fabric manager, a placement database, a device with a decoder array and a scrub engine, and a host-side driver that has to be told when its address map changed. None of that is reproduced; the arithmetic each of them has to get right is.

Each model is built twice — a correct build and a broken build selected by a parameter. Every broken build here is a spreadsheet that somebody trusted: the summing capacity model, the exact-allocation report, the total-free number, the control-plane rebind estimate. None of them is a coding error. They are all the right calculation of the wrong quantity.

Figure 1 — The pool holds 192 GB where dedicated provisioning would have bought 384. The dashed edge is section 14: the fabric manager's record of what is free and the device's record of what it released are two numbers, and only one of them can be allocated against.

5. RTL 1 — The Saving Is A Statistical Claim

// RTL 1 - the pooling argument. Capacity sized per host must cover each host's
// peak; capacity pooled must cover their combined peak, which is smaller.
module pooling_saving #(parameter int SUM_PEAKS = 0) (
  input  logic clk, rst_n,
  input  logic        eval,
  input  logic [15:0] h0_peak, h1_peak, h2_peak, combined_peak,
  output logic [15:0] dedicated_gb, pooled_gb, saved_gb, saving_pct,
  output logic        pooling_wins, inputs_consistent,
  output logic [7:0]  n_evals, n_wins,
  output logic        overprovision_err
);
  logic [31:0] s_q;
  assign dedicated_gb = h0_peak + h1_peak + h2_peak;
  // Pooled capacity is sized against the combined peak, which is what the hosts
  // actually need at once. The summing build sizes it against the sum anyway.
  assign pooled_gb = (SUM_PEAKS != 0) ? dedicated_gb : combined_peak;
  assign saved_gb  = (dedicated_gb > pooled_gb) ? (dedicated_gb - pooled_gb) : 16'd0;
  assign s_q = (dedicated_gb == 16'd0) ? 32'd0
                                       : (({16'd0, saved_gb} * 32'd100) / {16'd0, dedicated_gb});
  assign saving_pct  = (s_q > 32'd65535) ? 16'hFFFF : s_q[15:0];
  assign pooling_wins = (pooled_gb < dedicated_gb);
  // Hosts cannot peak to more, together, than the sum of their own peaks. A
  // combined figure above the sum is a measurement error, and the saving must
  // floor at zero rather than wrap.
  assign inputs_consistent = (combined_peak <= dedicated_gb);
  // Buying pooled capacity as if it were dedicated: the pool exists and saves
  // nothing.
  assign overprovision_err = eval && !pooling_wins && (combined_peak < dedicated_gb);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_evals <= 8'd0; n_wins <= 8'd0;
    end else if (eval) begin
      n_evals <= n_evals + 8'd1;
      if (pooling_wins) n_wins <= n_wins + 8'd1;
    end
  end
endmodule

Five evaluations. Three hosts peaking at 128 GB each.

Pooling won two of five, and the summing build overprovisioned twice.

Row two is not a failure. Three hosts that peak together need the full sum, and a pool sized at 192 GB for them is a pool that will run out. overprovision_err is gated on combined_peak < dedicated_gb for exactly that reason: buying 384 GB when the hosts need 384 GB is correct, and only buying 384 GB when 192 would have done is waste.

Row five is the input the model refuses to trust. Hosts cannot peak, together, to more than the sum of their individual peaks — the combined peak is bounded above by the sum by construction. A figure above it comes from a measurement pipeline with a bug, and the response is to floor the saving at zero and flag the inconsistency rather than compute a negative saving that wraps to 65,000 GB. Section 18 shows this guard was unreachable until the model was made to say so.

Why the broken build is not a strawman. Summing per-host peaks is how capacity has always been planned, because with dedicated memory it is the correct calculation. It is also the conservative one, and a capacity planner who sizes a pool by summing peaks will never run out. They will simply have bought a switch and saved nothing, and the finance case for the whole programme evaporates without anybody being wrong about anything.

6. RTL 2 — Sixteen Logical Devices, And The Identifier Has To Name One

// RTL 2 - a multi-logical device presents separate logical devices to separate
// hosts, and the count is bounded.
module logical_devices #(parameter int UNBOUNDED_LD = 0) (
  input  logic clk, rst_n,
  input  logic       assign_req,
  input  logic [7:0] ld_wanted,
  input  logic [3:0] ld_id,
  output logic       within_limit, id_valid, may_assign,
  output logic [7:0] max_ld,
  output logic [7:0] n_assigns, n_refused,
  output logic       over_limit_err
);
  assign max_ld       = 8'd16;
  assign within_limit = (ld_wanted <= max_ld);
  // A logical-device identifier must name one of the supported logical devices.
  assign id_valid = ({4'd0, ld_id} < ld_wanted);
  assign may_assign = (UNBOUNDED_LD != 0) ? assign_req
                                          : (assign_req && within_limit && id_valid);
  // An assignment accepted beyond what the device can present.
  assign over_limit_err = may_assign && !within_limit;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_assigns <= 8'd0; n_refused <= 8'd0;
    end else if (assign_req) begin
      n_assigns <= n_assigns + 8'd1;
      if (!may_assign) n_refused <= n_refused + 8'd1;
    end
  end
endmodule

Three refusals against none, and one over-limit acceptance.

Two independent checks, and the second is the one that gets skipped. Sixteen is the ceiling on how many logical devices a device presents; the identifier check is whether the one being named exists in the configuration currently programmed. A device configured for eight logical devices refuses identifier 9 and identifier 8 alike — the last valid identifier of eight is 7, and off-by-one there assigns capacity to a logical device that has no host bound to it.

Row five is that boundary, and section 18 records that the comparison was untestable until it was driven.

7. RTL 3 — A Pool Hands Out Blocks

// RTL 3 - capacity granularity. A pool hands out blocks, and a request that is
// not a multiple of the block wastes the remainder.
module capacity_granularity #(parameter int EXACT_ALLOC = 0) (
  input  logic clk, rst_n,
  input  logic        req,
  input  logic [15:0] want_gb, block_gb,
  output logic [15:0] blocks_used, allocated_gb, wasted_gb, waste_pct,
  output logic        aligned,
  output logic [7:0]  n_reqs, n_wasteful,
  output logic        waste_unreported_err
);
  logic [15:0] rem;
  logic [31:0] w_q;
  assign rem     = (block_gb == 16'd0) ? 16'd0 : (want_gb % block_gb);
  assign aligned = (rem == 16'd0);
  assign blocks_used = (block_gb == 16'd0) ? 16'd0
                     : (aligned ? (want_gb / block_gb) : ((want_gb / block_gb) + 16'd1));
  // A pool allocates whole blocks. The exact build reports the request instead
  // of what the pool actually took out.
  assign allocated_gb = (EXACT_ALLOC != 0) ? want_gb : (blocks_used * block_gb);
  assign wasted_gb = (allocated_gb > want_gb) ? (allocated_gb - want_gb) : 16'd0;
  assign w_q = (allocated_gb == 16'd0) ? 32'd0
                                       : (({16'd0, wasted_gb} * 32'd100) / {16'd0, allocated_gb});
  assign waste_pct = (w_q > 32'd65535) ? 16'hFFFF : w_q[15:0];
  // Capacity removed from the pool that no report accounts for.
  assign waste_unreported_err = req && !aligned && (wasted_gb == 16'd0);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_reqs <= 8'd0; n_wasteful <= 8'd0;
    end else if (req) begin
      n_reqs <= n_reqs + 8'd1;
      if (!aligned) n_wasteful <= n_wasteful + 8'd1;
    end
  end
endmodule

Four requests against 8 GB blocks.

Two unaligned requests, and the exact build accounted for neither.

The third row is the pathological case and it is not rare. A host asking for a single gigabyte takes a whole 8 GB block, and 87% of what left the pool is unusable by anybody. That is the granularity argument in one number, and it is why block size is a first-order design parameter rather than an implementation detail: halving it halves this waste and doubles the decoder state the device has to hold.

The exact build's report is the failure that matters most in practice, because it is the one a capacity dashboard shows. It reports the request — 33 GB — while 40 GB left the pool. The 7 GB difference is not lost, exactly; it is assigned to a logical device that will never use it, and it is invisible to every report computed from allocation requests rather than from block accounting. Over a thousand allocations, that difference is a device's worth of capacity nobody can find.

8. RTL 4 — Free Is Not The Same As Allocatable

// RTL 4 - fragmentation. A pool with enough total free capacity may still have
// no single run large enough to satisfy a request.
module pool_fragmentation #(parameter int TOTAL_FREE_ONLY = 0) (
  input  logic clk, rst_n,
  input  logic        req,
  input  logic [15:0] free_a, free_b, free_c, want_gb,
  output logic [15:0] total_free, largest_run,
  output logic        contiguous_ok, total_ok, granted,
  output logic [7:0]  n_reqs, n_failed,
  output logic        false_grant_err
);
  logic [15:0] max_ab;
  assign total_free = free_a + free_b + free_c;
  assign max_ab     = (free_a > free_b) ? free_a : free_b;
  assign largest_run = (max_ab > free_c) ? max_ab : free_c;
  assign contiguous_ok = (want_gb <= largest_run);
  assign total_ok      = (want_gb <= total_free);
  // The total-free build grants against the sum, which is the number a capacity
  // report shows and not the number an allocation needs.
  assign granted = (TOTAL_FREE_ONLY != 0) ? (req && total_ok)
                                          : (req && contiguous_ok);
  // An allocation granted that no single run can satisfy.
  assign false_grant_err = granted && !contiguous_ok;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_reqs <= 8'd0; n_failed <= 8'd0;
    end else if (req) begin
      n_reqs <= n_reqs + 8'd1;
      if (!granted) n_failed <= n_failed + 8'd1;
    end
  end
endmodule

Free runs of 40, 30 and 30 GB — 100 GB free in total.

One false grant.

The second and fourth rows have identical totals. 100 GB free, a 60 GB request, and the answer is no in one and yes in the other — the difference is entirely in the shape of the free capacity, which is a number no capacity report carries. "The pool has 100 GB free" is true in both and useful in neither.

The third row is why the total-free model survives review: it is correct whenever the request is larger than the total, which is the case everybody tests. It only fails in the band between the largest run and the total, and that band is exactly where a busy pool spends its time.

9. RTL 5 — What A Rebind Actually Costs

// RTL 5 - the rebind cost. Capacity moving between hosts is not instantaneous,
// and the time it takes is capacity nobody can use.
module rebind_cost #(parameter int IGNORE_SCRUB = 0) (
  input  logic clk, rst_n,
  input  logic        rebind,
  input  logic [15:0] size_gb, scrub_gbps, quiesce_ms,
  output logic [15:0] scrub_ms, total_ms, stranded_gb_ms,
  output logic        within_sla,
  input  logic [15:0] sla_ms,
  output logic [7:0]  n_rebinds, n_over_sla,
  output logic        sla_miss_err
);
  // Scrubbing the capacity is the dominant term and the one a rebind model
  // written from a control-plane view leaves out.
  assign scrub_ms = (scrub_gbps == 16'd0) ? 16'd0
                                          : ((size_gb * 16'd1000) / scrub_gbps);
  assign total_ms = (IGNORE_SCRUB != 0) ? quiesce_ms : (quiesce_ms + scrub_ms);
  assign stranded_gb_ms = size_gb * total_ms;
  assign within_sla = (total_ms <= sla_ms);
  assign sla_miss_err = rebind && !within_sla;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_rebinds <= 8'd0; n_over_sla <= 8'd0;
    end else if (rebind) begin
      n_rebinds <= n_rebinds + 8'd1;
      if (!within_sla) n_over_sla <= n_over_sla + 8'd1;
    end
  end
endmodule

A 50 ms quiesce, a 1000 ms SLA.

One SLA miss against none.

The control-plane model is wrong by a factor of thirty-three. It reports 50 ms for an operation that takes 1650, because from the fabric manager's point of view a rebind is a few register writes and an acknowledgement. Everything else happens inside the device. The rebind is not slow because the control plane is slow; it is slow because 32 GB of DRAM has to be written before anybody else may read it — which is 19.3 section 9's residue requirement, priced.

Row three is the lever: the scrub rate is the only term a device team controls, and doubling it halves the dominant cost. Row five is the shortcut, and it is worth naming clearly — a device with no scrub engine rebinds in 50 ms and hands the next tenant the previous one's memory.

The stranded_gb_ms output is the quantity a fleet operator actually cares about. 32 GB unavailable for 1650 ms is 52,800 GB-milliseconds of capacity that exists, is powered, and belongs to nobody. Summed over a rebalancing cycle, it is the real efficiency cost of pooling, and it is invisible to any metric that samples capacity rather than integrating it.

10. Waveform — A Rebind In Progress

Nothing is broken in that waveform. Every phase is correct, the scrub completes, the new host is bound with clean memory. The only defect is a control plane that reported completion at cycle 2 — and a fleet scheduler acting on that report will issue the next rebalancing decision while 32 GB is still mid-scrub.

11. RTL 6 — Capacity Nobody Is Using

// RTL 6 - stranded capacity. Memory a host owns and is not using is memory the
// pool exists to reclaim, and reclaiming it needs a definition of idle.
module stranded_capacity #(parameter int NEVER_RECLAIM = 0) (
  input  logic clk, rst_n,
  input  logic        scan,
  input  logic [15:0] owned_gb, touched_gb, idle_ms, idle_threshold_ms,
  output logic [15:0] untouched_gb, reclaimable_gb, stranded_pct,
  output logic        idle_long_enough, may_reclaim,
  output logic [7:0]  n_scans, n_reclaimable,
  output logic        strand_err
);
  logic [31:0] p_q;
  assign untouched_gb     = (touched_gb >= owned_gb) ? 16'd0 : (owned_gb - touched_gb);
  assign idle_long_enough = (idle_ms >= idle_threshold_ms);
  // Reclaiming needs both: capacity that is not being used, and long enough to
  // be confident it is not about to be.
  assign may_reclaim = (NEVER_RECLAIM != 0) ? 1'b0
                                            : (idle_long_enough && (untouched_gb != 16'd0));
  assign reclaimable_gb = may_reclaim ? untouched_gb : 16'd0;
  assign p_q = (owned_gb == 16'd0) ? 32'd0
                                   : (({16'd0, untouched_gb} * 32'd100) / {16'd0, owned_gb});
  assign stranded_pct = (p_q > 32'd65535) ? 16'hFFFF : p_q[15:0];
  // Capacity that qualifies for reclaim and is left where it is.
  assign strand_err = scan && idle_long_enough && (untouched_gb != 16'd0)
                      && (reclaimable_gb == 16'd0);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_scans <= 8'd0; n_reclaimable <= 8'd0;
    end else if (scan) begin
      n_scans <= n_scans + 8'd1;
      if (may_reclaim) n_reclaimable <= n_reclaimable + 8'd1;
    end
  end
endmodule

128 GB owned, a 1000 ms idle threshold.

Two reclaimable scans against none, and two stranding events.

Row two is not stranding, it is waiting, and the distinction is the whole model. Capacity untouched for 500 ms against a 1000 ms threshold is capacity that may be about to be used, and reclaiming it is a worse outcome than leaving it. strand_err fires only when the capacity qualifies on both counts and is left anyway.

Row one is the entire pooling thesis restated as a failure. 75% of a host's allocation is untouched and has been for five seconds. That is precisely the waste pooling was built to eliminate, and a pool that never reclaims has reproduced dedicated provisioning with extra hardware in the path.

Row five is the accounting drift: more recorded as touched than the host owns, which arrives from a sampling pipeline that missed a release. The floor keeps it at zero untouched rather than wrapping into a reclaim of 65,000 GB.

12. RTL 7 — Hot-Add Has An Order

// RTL 7 - hot-add. Capacity appearing under a running host is only useful if
// the host is told, and only safe if the decoder is programmed first.
module hot_add_order #(parameter int ANNOUNCE_FIRST = 0) (
  input  logic clk, rst_n,
  input  logic       step,
  input  logic       decoder_programmed, host_notified, capacity_present,
  output logic       usable, order_correct,
  output logic [7:0] n_steps, n_premature,
  output logic       premature_use_err
);
  // A host may use capacity only once it is present and decoded. Announcing it
  // before the decoder is programmed invites a request with no route.
  assign order_correct = !host_notified || (decoder_programmed && capacity_present);
  assign usable = (ANNOUNCE_FIRST != 0) ? host_notified
                                        : (host_notified && decoder_programmed && capacity_present);
  // A host told it may use capacity that is not yet routable.
  assign premature_use_err = step && usable && !order_correct;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_steps <= 8'd0; n_premature <= 8'd0;
    end else if (step) begin
      n_steps <= n_steps + 8'd1;
      if (usable && !order_correct) n_premature <= n_premature + 8'd1;
    end
  end
endmodule

Two premature uses against none.

Row four is the case that keeps order_correct honest. A host that has been told nothing cannot act out of order, whatever the decoder and the capacity are doing. The condition is !host_notified || (...) rather than (...) alone, because the ordering constraint is a statement about the notification, not about the device's internal state.

A request issued to capacity whose decoder is not yet programmed is exactly 20.1 section 5's unmatched address — dropped by a correct switch, delivered to port 0 by a default-routing one. The two chapters meet here: hot-add ordering is what stops a pool from generating unrouted requests as a matter of routine.

Figure 3 — Both pools report a hundred gigabytes free and only one can serve the request. The report at the right is identical in both cases, which is why "the pool has 100 GB free" is true and useless.

13. RTL 8 — The Pool's Own Blast Radius

// RTL 8 - the pool's failure domain, from the host's side. Capacity sourced
// from one device is capacity one device failure takes entirely.
module pool_blast #(parameter int ONE_DEVICE = 0) (
  input  logic clk, rst_n,
  input  logic        fail,
  input  logic [7:0]  devices_in_pool,
  input  logic [15:0] gb_per_host,
  output logic [7:0]  devices_per_host,
  output logic [15:0] gb_lost, gb_kept,
  output logic [7:0]  loss_pct,
  output logic [7:0]  n_failures, n_total_loss,
  output logic        total_loss_err
);
  logic [15:0] share;
  logic [31:0] l_q;
  // Sourcing a host's capacity from every device in the pool means one failure
  // costs a share. Sourcing it from one device costs all of it.
  assign devices_per_host = (ONE_DEVICE != 0) ? 8'd1 : devices_in_pool;
  // Ceiling division: some device carries the remainder, and that is the one
  // whose failure to size against.
  assign share = (devices_per_host == 8'd0) ? 16'd0
               : ((gb_per_host + {8'd0, devices_per_host} - 16'd1) / {8'd0, devices_per_host});
  assign gb_lost = fail ? share : 16'd0;
  assign gb_kept = gb_per_host - gb_lost;
  assign l_q = (gb_per_host == 16'd0) ? 32'd0
                                      : (({16'd0, gb_lost} * 32'd100) / {16'd0, gb_per_host});
  assign loss_pct = (l_q > 32'd255) ? 8'hFF : l_q[7:0];
  // A single device failure costing a host everything it holds.
  assign total_loss_err = fail && (gb_per_host != 16'd0) && (gb_lost == gb_per_host);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_failures <= 8'd0; n_total_loss <= 8'd0;
    end else if (fail) begin
      n_failures <= n_failures + 8'd1;
      if ((gb_per_host != 16'd0) && (gb_lost == gb_per_host)) n_total_loss <= n_total_loss + 8'd1;
    end
  end
endmodule

128 GB per host.

One total loss when spread against three when sourced from one device.

This is 19.4 section 11 read from the other side. That chapter asked how many tenants a device failure costs; this one asks how much of one host's memory it costs. The answers move in opposite directions under the same decision: spreading a host's capacity across devices lowers its loss per failure and raises the number of hosts each failure touches.

The ceiling division is the same discipline as 19.4's. 128 GB over three devices is 42.67, and the blast radius is 43 — some device carries the remainder, and that is the one to size against.

14. RTL 9 — Two Views Of The Same Pool

// RTL 9 - two views of the same pool. The fabric manager records what it
// intended and the device holds what it applied, and free capacity computed
// from the wrong one is a number nobody can allocate against.
module capacity_accounting #(parameter int TRUST_INTENT = 0) (
  input  logic clk, rst_n,
  input  logic        query,
  input  logic [15:0] pool_gb, fm_assigned_gb, device_applied_gb,
  output logic [15:0] fm_free_gb, device_free_gb, divergence_gb,
  output logic [15:0] reported_free_gb,
  output logic        views_agree,
  output logic [7:0]  n_queries, n_diverged,
  output logic        phantom_capacity_err
);
  assign fm_free_gb     = (fm_assigned_gb >= pool_gb) ? 16'd0 : (pool_gb - fm_assigned_gb);
  assign device_free_gb = (device_applied_gb >= pool_gb) ? 16'd0 : (pool_gb - device_applied_gb);
  assign divergence_gb  = (fm_assigned_gb > device_applied_gb)
                          ? (fm_assigned_gb - device_applied_gb)
                          : (device_applied_gb - fm_assigned_gb);
  assign views_agree = (fm_assigned_gb == device_applied_gb);
  // Free capacity must be reported from the smaller of the two views. The
  // trusting build reports the fabric manager's, which is what it meant to do.
  assign reported_free_gb = (TRUST_INTENT != 0) ? fm_free_gb
                          : ((fm_free_gb < device_free_gb) ? fm_free_gb : device_free_gb);
  // Reporting free capacity the device has not actually released.
  assign phantom_capacity_err = query && (reported_free_gb > device_free_gb);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_queries <= 8'd0; n_diverged <= 8'd0;
    end else if (query) begin
      n_queries <= n_queries + 8'd1;
      if (!views_agree) n_diverged <= n_diverged + 8'd1;
    end
  end
endmodule

A 256 GB pool.

Three divergences, three phantom reports.

Divergence happens in both directions and they are different bugs. Row two is a release the fabric manager recorded and the device has not finished applying — the mid-rebind state of section 9, which lasts 1650 ms and during which the fabric manager genuinely believes there is 192 GB free. Row three is the opposite: the device applied something the fabric manager did not record, which is what a retried command produces.

The safe report is the minimum of the two, always, and the reason is asymmetric consequences. Under-reporting free capacity costs an allocation that could have succeeded. Over-reporting it produces a grant against capacity the device will refuse, which surfaces as a failure deep in the allocation path where nobody can attribute it. Phantom capacity is the pool equivalent of section 8's false grant — a number that is true about somebody's records and false about the hardware.

Figure 4 — Every path that is not the left-hand column ends in the same place: capacity that exists, is powered, and belongs to nobody. Section 9 prices how long that state lasts, and the scrub gate is the one a control plane is tempted to skip.

15. RTL 10 — Pooling Assembled

// RTL 10 - pooling assembled. Every property that must hold before capacity can
// move between hosts without moving a cable.
module pooling_model #(parameter int CAPACITY_ONLY = 0) (
  input  logic clk, rst_n,
  input  logic       evaluate,
  input  logic       capacity_saved,     // the pool costs less than dedicated
  input  logic       ld_within_limit,    // the logical-device count is supported
  input  logic       runs_contiguous,    // free capacity is allocatable, not just present
  input  logic       rebind_within_sla,  // moving capacity fits its time budget
  input  logic       add_ordered,        // decoder before announcement
  input  logic       views_reconciled,   // fabric manager and device agree
  output logic       workable,
  output logic [5:0] fail_mask,
  output logic [7:0] n_eval, n_workable,
  output logic       false_workable_err
);
  assign fail_mask[0] = ~capacity_saved;
  assign fail_mask[1] = ~ld_within_limit;
  assign fail_mask[2] = ~runs_contiguous;
  assign fail_mask[3] = ~rebind_within_sla;
  assign fail_mask[4] = ~add_ordered;
  assign fail_mask[5] = ~views_reconciled;
  // The capacity-only build checks that the pool is cheaper than dedicated
  // memory and calls pooling justified, which is the business case with none of
  // the engineering attached.
  assign workable = (CAPACITY_ONLY != 0) ? capacity_saved : (fail_mask == 6'd0);
  assign false_workable_err = evaluate && workable && (fail_mask != 6'd0);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_eval <= 8'd0; n_workable <= 8'd0;
    end else if (evaluate) begin
      n_eval <= n_eval + 8'd1;
      if (workable) n_workable <= n_workable + 8'd1;
    end
  end
endmodule

One workable against four, and three false claims.

The capacity-only definition is the business case with none of the engineering attached, and it is the one that authorises the purchase. The pool saves 50% of the memory bill; therefore pooling. Row two is the same pool six months later, with 100 GB free that cannot satisfy a 60 GB request, and the saving is still exactly as real as it was on the slide.

16. Quantitative Reasoning

The saving. Three hosts at 128 GB each: 384 GB dedicated, 192 GB pooled, a 50% saving. A combined peak of 128 makes it 66%. Hosts that peak together make it zero, and that is the workload rather than a defect.

Logical devices. Sixteen supported. Eight configured means identifier 7 is the last valid one, and identifier 8 is refused — a boundary that was untestable until section 18 drove it.

Granularity. 8 GB blocks. A 33 GB request takes 40 GB — 7 GB, 17% waste. A 1 GB request takes a whole block: 7 GB of 8, 87% waste. The exact-allocation report shows zero waste in both.

Fragmentation. 100 GB free in runs of 40, 30 and 30. A 60 GB request fits the total and no run, and the total-free model grants it. Defragmented into one run of 100, the identical request succeeds — same total, opposite answer.

Rebind. 32 GB at 20 GB/s: 1600 ms of scrub against a 50 ms quiesce, a total of 1650 against a 1000 ms SLA. The control-plane model reports 50 — wrong by a factor of 33. The stranded cost is 52,800 GB-milliseconds of powered memory belonging to nobody.

Stranded capacity. 128 GB owned, 32 touched: 96 GB, 75% of the allocation, idle for five seconds. A pool that never reclaims it has rebuilt dedicated provisioning with a switch in the path.

Hot-add. Two of four steps announced capacity before it was routable — each one a request the switch has no route for.

Blast radius. 128 GB per host over four devices loses 32 GB, 25%, per failure; over three, 43 GB and 33% after the ceiling; from one device, 128 GB and 100%.

Accounting. A 256 GB pool mid-rebind: the fabric manager sees 192 GB free, the device has released 128, and the trusting report offers 64 GB of phantom capacity to whoever allocates next.

The assembled model. Six properties, five configurations, one workable. The capacity-only definition reported four.

17. Assertions

Every check is an explicit comparison against an exact value. Icarus Verilog 13.0 has no concurrent assertion support, so each is a procedural comparison against 1'b1, and every one is an equality.

The saving. Hosts peaking together are asserted as not overprovisioning, which is what separates a correct sizing from a wasteful one.

chk(gPg == 16'd384, "hosts peaking together need the full sum");
chk(gSg == 16'd0,   "so nothing is saved");
chk(gOe == 1'b0,    "which is not overprovisioning, it is the workload");

The impossible input is asserted inconsistent and floored rather than wrapped.

chk(gIc == 1'b0,  "so a combined peak of 500 is inconsistent");
chk(gSg == 16'd0, "and the saving floors at zero rather than wrapping");

Logical devices. Both boundaries are driven: exactly sixteen is within the limit, and an identifier exactly equal to the count is not valid.

chk(lWl == 1'b1, "eight logical devices is within the limit");
chk(lIv == 1'b0, "and identifier 8 is one past the last of eight");

Granularity. The waste is asserted as an exact value in the zero-block case as well, because an unasserted output is an output a mutation can change freely.

chk(cWg == 16'd0, "and wastes nothing rather than wrapping");

Fragmentation. The same total with two different shapes is asserted to two different answers.

Rebind. The exact-SLA boundary is driven.

chk(kSm == 16'd950,  "19 GB at 20 GB/s takes 950 ms");
chk(kTm == 16'd1000, "for exactly 1000 ms with the quiesce");
chk(kWs == 1'b1,     "which meets a 1000 ms SLA");

Stranded capacity. Idle-but-not-long-enough is asserted as waiting, not stranding.

chk(nIl == 1'b0, "but 500 ms is not the 1000 ms threshold");
chk(nSe == 1'b0, "which is not stranding, it is waiting");

Hot-add. A host told nothing is asserted as ordered.

chk(hOc == 1'b1, "a host not told anything cannot act out of order");

Blast radius. The ceiling division is asserted at exactly 43.

Accounting. Both directions of divergence are asserted, and the drift case beyond the pool size is asserted to floor.

The assembled model. Every fail mask is asserted as an exact six-bit value.

Totals: 251 checks across two testbenches, 126 on the front five models and 125 on the back five, all passing on the unmutated sources.

18. Mutation Testing

Forty-nine mutations were injected one at a time.

49 injected, 49 killed, after six survivors were diagnosed.

Survivors 1, 2 and 3 — boundaries the testbench never reached. The identifier comparison ld_id < ld_wanted survived becoming <= because no case drove an identifier exactly equal to the count. The SLA comparison survived becoming exclusive because no rebind totalled exactly 1000 ms. The device-free floor survived because no case recorded more applied than the pool holds. Three guards, all correct, all invisible — and each fixed by one more stimulus.

Survivor 4 — an unobserved output. Removing the floor from wasted_gb survived even though the zero-block-size case was driven, because that case asserted blocks_used, allocated_gb and aligned and never wasted_gb. The stimulus was there and the assertion was not. A driven case with no assertion on the affected output is not coverage.

Survivor 5 — an unreachable guard, resolved by making the invariant explicit. The floor on saved_gb never fired, because combined_peak is bounded above by the sum of peaks and pooled_gb therefore never exceeds dedicated_gb. The first instinct is to delete the guard as redundant. But nothing was checking the invariant — the combined peak arrives from a measurement pipeline, and a pipeline with a bug produces a figure above the sum. The resolution was to add inputs_consistent, drive an impossible input, and assert that the saving floors rather than wrapping to 65,000 GB. The guard was not redundant; it was undocumented.

Survivor 6 — a malformed anchor. The ceiling-division mutation failed to compile because the replacement text left unbalanced parentheses. A mutation that does not compile is not a survivor and not a kill; it is a mutation that did not run, and reporting it as either would be wrong.

19. Verification Strategy

What a testbench for a real pool must cover.

Every threshold at exactly its value. The logical-device limit, the identifier bound, the idle threshold, the SLA, the block boundary, the pool size. Four of section 18's six survivors were a threshold the stimulus stepped over.

An assertion on every output the stimulus touches. The zero-block case was driven and three of its four outputs were checked. The gap was in the assertions, not the stimulus, which is a different failure and needs a different fix.

Impossible inputs, deliberately. A combined peak above the sum of peaks. More touched than owned. More applied than the pool holds. Each arrives from a real measurement or accounting pipeline, and each is where an unfloored subtraction wraps into a number that looks like an enormous quantity of free memory.

Both directions of every divergence. The fabric manager ahead of the device and the device ahead of the fabric manager are different bugs with different causes, and a model that handles one may not handle the other.

The cases that are correct and look like failures. Hosts that peak together, so pooling saves nothing. Capacity idle but not long enough to reclaim. A host that has been told nothing about capacity that is not ready. Each trips a naive checker.

What a real pool needs that these models do not have. Concurrency — two hosts requesting the last free run at once. Ordering — a hot-remove issued while a hot-add for the same blocks is in flight. Persistence — a fabric manager that restarts and must rebuild its view from the device rather than from its own log, which is section 14's divergence with no authoritative copy.

20. Synthesis and Implementation Reality

The decoder state is the cost of small blocks. Section 7's waste halves when the block size halves, and the device pays for it in decoder entries: each assigned run needs a base, a size and a logical-device identifier, held in registers that are read on every access. Block size is a trade between wasted DRAM and decoder area, and it is fixed in silicon.

The scrub engine is the rebind budget. Section 9's 1600 ms is a memory-controller bandwidth number, and the only lever a device team has is how much of the controller's bandwidth the scrub may take. Give it all of it and rebinds are fast and tenant traffic stalls; give it a slice and rebinds take proportionally longer. There is no third option, and the choice is a policy register somebody has to set with the SLA in front of them.

Fragmentation cannot be fixed by defragmenting. Moving an allocated run to coalesce free space means changing the address a running host is using, and CXL 2.0 has no mechanism for that — the host's decoder points where it points. Section 8's fragmentation is permanent until the allocations holding it are released, which makes allocation order a durable property of the pool rather than a transient one.

Hot-add ordering is a protocol sequence, not a software convention. The decoder write, the capacity presence and the host notification are three separate transactions on two different paths, and the ordering between them is enforced by the fabric manager's sequencing. There is nothing in the device that refuses a notification issued early.

Sixteen logical devices is a decoder-array size. It bounds not just how many hosts can share a device but how finely capacity can be subdivided, and it is why pooling at rack scale needs many devices rather than one large one.

21. Silicon Observability

The last row is the one that requires arguing for. The fabric manager already knows what it intended; the whole value of the device's counter is that it is a second, independent record. A device that simply echoes back what the fabric manager wrote provides no reconciliation at all, and section 14's divergence becomes undetectable by construction.

22. Debug Lab

Symptom. A pool reports 340 GB free across a 2 TB fleet. A scheduler tries to place a workload needing 96 GB and the allocation fails. It retries on a second pool, and a third, and eventually places at 48 GB with degraded performance. The capacity dashboard shows the fleet at 83% utilisation with plenty of headroom.

Step 1 — is the free-capacity number wrong? Query the device directly rather than the fabric manager. Both report 340 GB. The number is right.

Step 2 — is it fragmentation? Read the free-run distribution. The 340 GB is in eleven runs, the largest of which is 56 GB. A 96 GB request fits the total and no run, which is section 8 exactly. The dashboard number was never the allocatable number.

Step 3 — why is it so fragmented? Read the allocation history. The pool has served 400 allocations over three weeks, averaging 5 GB, against an 8 GB block size. Every one of those requests took a whole block, and the releases have come back in the order the workloads finished rather than the order they were allocated.

Step 4 — how much capacity is that costing? Compare blocks allocated against bytes requested: 400 allocations, 2,180 GB requested, 3,200 GB of blocks. A little over a terabyte of the fleet is block remainder — assigned, powered, and unusable by anyone.

Step 5 — is anything reclaimable? Read untouched capacity per logical device. Four logical devices hold 62 GB, 71 GB, 44 GB and 88 GB untouched, all idle for over an hour. That is 265 GB the pool could take back, none of which the fabric manager has been asked to reclaim.

The finding. Three separate defects compounding. Fragmentation makes the free capacity unallocatable; block granularity against small requests generates the fragmentation and a terabyte of remainder; and no reclaim policy leaves 265 GB stranded on hosts that stopped using it. None of them is a fault. Every one of them is a design parameter nobody set.

The fix, in order. Turn on reclaim first — it is policy, it is reversible, and it returns 265 GB today. Then change the allocator to round requests up to the block size at the scheduler, so the waste is visible in the request rather than hidden in the pool. Then, on a longer timescale, argue for a smaller block size on the next device generation, with section 20's decoder-area cost attached.

What made this hard. Every number on the dashboard was correct. 340 GB really was free, the fleet really was at 83%, and the allocation really did fail. The reported quantity and the useful quantity were different quantities, and nothing in the reporting said so.

23. Design Review

1. What is the combined peak, measured, against the sum of per-host peaks? If nobody has measured it, the saving is a hypothesis. Section 5.

2. What is the block size, and what is the mean allocation size? The ratio is the waste. Section 7.

3. Does the capacity report show the largest free run, or only the total? Section 8, and section 22 is what it costs to show only the total.

4. What is the rebind SLA, and does it include the scrub? Section 9, and the honest version asks for the number at the largest region size the pool supports.

5. What fraction of the scrub engine's bandwidth may the scrub take? There is no answer that is fast and non-disruptive. Section 20.

6. Is there a reclaim policy, and what is the idle threshold? No policy means dedicated provisioning with extra hardware. Section 11.

7. Is the decoder programmed before the host is notified? Section 12, and the failure surfaces as 20.1's unrouted request.

8. How many devices does one host's capacity come from? Section 13, and the answer trades against 19.4's tenant blast radius in the opposite direction.

9. Can the device's assignment state be read independently of the fabric manager's record? If it only echoes, there is no reconciliation. Section 21.

10. Which of the six properties does the team believe pooling means? Ask separately. Section 15 exists because the answers differ, and the capacity-only answer is the one on the slide that funded the project.

24. How This Appears In Real Engineering

A capacity planning team building the business case does section 5 and stops there, because that is the section with the money in it. The number they need and usually do not have is the measured combined peak — not the sum of peaks, and not a modelled one. Getting it requires per-host memory telemetry at a resolution most fleets do not collect.

A device team choosing a block size is making section 7's trade permanently. The waste is a function of the block size and the workload's allocation distribution, and the second half is not knowable at design time. The defensible position is to size for the smallest allocation the pool is expected to serve and pay the decoder area, because the alternative is discovered three years later as section 22.

A fleet operations team owns sections 8 and 11, and both are policy rather than hardware. Fragmentation is managed by allocation policy — rounding at the scheduler, sizing classes, refusing to place small workloads on pools that are already fragmented. Reclaim is managed by an idle threshold that somebody has to be willing to be wrong about.

A fabric-manager team owns section 14 and will be tempted to trust its own record, because its own record is available synchronously and the device's is not. The argument against is that the divergence window is section 9's 1650 ms, which on a pool rebalancing continuously is not a rare state.

25. Common Misconceptions

"Pooling saves memory." Pooling saves memory if hosts do not peak together. If they do, it saves nothing and costs a switch. Section 5.

"The pool has 340 GB free, so a 96 GB request will succeed." Only if one run holds 96 GB. Sections 8 and 22.

"A 33 GB allocation takes 33 GB." It takes 40, on 8 GB blocks, and the report will still say 33. Section 7.

"Rebinding is a control-plane operation." It is a control-plane operation followed by a scrub that is thirty times longer. Section 9.

"Small allocations are cheap." A 1 GB allocation takes a whole 8 GB block. 87% waste, section 7.

"Fragmentation resolves itself." Not without defragmentation, and CXL 2.0 cannot move an allocated run under a running host. Section 20.

"Memory a host is not using is available to the pool." Only if a reclaim policy exists and something acts on it. Section 11.

"The fabric manager knows what the pool holds." It knows what it intended. The device knows what it applied, and mid-rebind those differ. Section 14.

"Hot-add is just a notification." The order matters, and announcing first produces requests the switch has no route for. Section 12.

"The pool is one big device, so a failure is one device's worth." It is one device's worth of every host that draws from it. Section 13.

26. Interview Reasoning

Q. Three hosts each peak at 128 GB. How much memory does a pool need?

Not 384, and not necessarily 192 either — it needs their combined peak, which has to be measured rather than assumed. The follow-up worth reaching: if they peak together, pooling saves nothing, and that is a workload property rather than a design failure. A candidate who asks for the combined peak instead of computing one is thinking correctly.

Q. A pool reports 340 GB free and a 96 GB allocation fails. What happened?

Fragmentation. The free capacity is in runs and none of them holds 96 GB. The follow-up: what number should the report have shown — largest free run, alongside the total. And the deeper one: why is it fragmented, which is usually block granularity against small allocations plus release order.

Q. How long does it take to move 32 GB from one host to another?

Long enough to scrub 32 GB, which at 20 GB/s is 1.6 seconds — the control-plane part is tens of milliseconds and is not the answer. The good follow-up is what that capacity is doing during those 1.6 seconds: it is powered, it exists, and it belongs to nobody.

Q. Why can a pool not defragment itself?

Because defragmenting means moving an allocated run, which means changing the address a running host is already using, and CXL 2.0 has no mechanism to do that under a live host. Fragmentation is therefore durable until the allocations holding it release, which makes allocation policy the only lever.

Q. The fabric manager says 192 GB is free and the device says 128. Which do you report?

128 — the minimum, always. The reasoning is asymmetric consequences: under-reporting costs an allocation that might have succeeded, and over-reporting produces a grant the device will refuse somewhere deep in the path where nobody can attribute it. The follow-up is why they differ, and the usual answer is a rebind mid-scrub.

Q. A host's 128 GB comes from four devices instead of one. Better or worse?

Better for that host — one device failure costs 32 GB instead of 128. Worse for the fleet, because every device failure now touches every host. It is the same decision as tenant placement seen from the other end, and the two metrics move in opposite directions under it.

27. Exercises

1. Extend RTL 1 to N hosts and compute the combined peak from a per-host time series rather than taking it as an input. Show how the saving changes as the hosts' peaks become correlated.

2. Add a per-logical-device capacity limit to RTL 2 and show that sixteen logical devices with a fixed maximum each bounds the pool differently from sixteen with a shared budget.

3. Modify RTL 3 so the scheduler rounds the request to the block size before it reaches the pool, and show that the total waste is unchanged and the reporting is now correct. This is section 22's second fix.

4. Extend RTL 4 to maintain a run list across allocations and releases, and measure how the largest free run degrades over 400 allocations with a realistic size distribution.

5. Gate RTL 5's rebind completion on the scrub, and add a scrub-bandwidth parameter that trades rebind time against tenant traffic. Find the setting that meets a 1000 ms SLA at 64 GB.

6. Give RTL 6 a per-region age rather than a single idle counter, and show that reclaiming the oldest region first returns more capacity per disruption than reclaiming the largest.

7. Turn RTL 7 into an explicit sequence and add the hot-remove ordering, which is the reverse. Show which of the two is more dangerous to get wrong and why.

8. Combine RTL 8 with 19.4's failure-domain model and find the devices-per-host value that minimises the product of host loss and hosts affected.

9. Add a fabric-manager restart to RTL 9: the record is lost and must be rebuilt from the device. Show what is recoverable and what is not.

10. Add a seventh property to RTL 10. If it is implied by one of the six, say which; if not, give the configuration it catches that the current mask calls workable.

28. Summary

Pooling is the feature CXL 2.0 exists for, and it rests on one observation with five bills attached.

The saving is statistical. Three hosts at 128 GB each need 384 GB dedicated and 192 pooled — a 50% saving — and only if their combined peak really is 192. Hosts that peak together save nothing, and that is the workload, not a defect.

A pool hands out blocks. A 33 GB request takes 40 GB on 8 GB blocks; a 1 GB request takes a whole block, 87% waste. The report that shows the request instead of the blocks hides a terabyte across four hundred allocations.

Free is not allocatable. 100 GB free in runs of 40, 30 and 30 cannot serve a 60 GB request, and the same 100 GB in one run can. No capacity report carries the difference.

A rebind costs a scrub. 32 GB at 20 GB/s is 1650 ms against a control-plane estimate of 50 — wrong by a factor of 33, and 52,800 GB-milliseconds of powered memory belonging to nobody.

Capacity nobody touches is the waste pooling was built to remove. 96 GB of 128, idle for five seconds — and a pool with no reclaim policy has rebuilt dedicated provisioning with a switch in the path.

Hot-add has an order, and announcing before the decoder is programmed produces exactly the unrouted request 20.1 had to decide what to do with.

The pool's blast radius runs the other way from the fleet's. 128 GB from four devices loses 32 GB per failure; from one, all of it — and the same decision that improves this makes 19.4's number worse.

The fabric manager's view is not the device's. Mid-rebind it sees 192 GB free where 128 is released, and the safe report is always the minimum, because over-reporting fails deep in the allocation path where nobody can attribute it.

Impossible inputs arrive. A combined peak above the sum of peaks, more touched than owned, more applied than the pool holds — each from a real pipeline, each a place an unfloored subtraction wraps into 65,000 GB of imaginary memory.

The saving is one property of six, and the capacity-only definition — the one on the slide that funded the programme — called four of five pools workable when one was.

20.3 — CXL 2.0 Architectural Changes steps back from the two marquee features to what actually changed in the adapter, the decoders and the fabric-manager interface between 1.1 and 2.0 — and what a device built for 1.1 has to become.