The CXL 3.x Future Vision

Module 21 has built a fabric that routes past one switch level, transfers between devices without a host, and survives leaving the rack it was drawn in. None of that is the point.

The point is a data centre where a machine is assembled — this many cores, this much memory, these accelerators — rather than bought in a fixed ratio and then made to fit whatever runs on it. Every chapter of Module 20 and 21 is infrastructure for that idea, and this chapter is the idea itself: what it saves, what it costs, and the six things that have to be true before a fabric of resources is a machine somebody can run a workload on.

This is a closing chapter and it is not a speculative one. Every model here is arithmetic on quantities a real deployment already measures, and the honest finding is section 15's: the fabric is the easiest of the six.

1. The Engineering Problem — The Fabric Was The Easy Part

A fixed machine gives memory in proportion to cores. A composed one gives what was asked for, and the difference is a workload that fits against one that does not. Section 5.

Stranded capacity is the entire financial case. 48 GB idle in each of four machines is 192 GB that exists, is powered, and cannot be lent — and a report that counts it as available has hidden the argument for composition. Section 6.

Composition takes time. A machine assembled on demand is not available until it is assembled, and binding plus scrubbing plus booting is not a rounding error. Section 7.

A composed machine's failure domain is every resource it borrowed. Three local parts and five borrowed is eight things whose failure takes it down, not three. Section 8.

The saving depends entirely on demand being uncorrelated. Workloads that peak together need the sum, and pooling them saves nothing at all. Section 9.

And the software has to want it. A pool nothing schedules against is capacity that has been moved from stranded-inside-machines to stranded-inside-a-pool. Section 14.

This chapter against 21.3, stated precisely. That one owns what a fabric needs to work at scale. This one owns what the scale is for — and section 15 shows a data centre whose fabric reaches everything and composes nothing.

2. The One-Sentence Model

A data centre is composable when the fabric reaches its resources, they come in the amounts asked for rather than in fixed ratios, assembly fits its time budget, every borrowed part is in the failure model, released machines return everything, and schedulers actually ask for composition — and every defect below is a fabric that reaches everything and fails one of the other five.

3. What This Chapter Owns

Deferred:

4. Teaching-Model Boundary

Every model is a small synchronous block isolating one property. A real composable deployment is a scheduler, a resource manager, a fabric manager, a firmware stack and a fleet of workloads, and none of that is reproduced. What is reproduced is the arithmetic each of them has to get right.

Each model is built twice — a correct build and a broken build selected by a parameter. The broken builds in this chapter are almost all the composability pitch, taken at face value. Resources are fungible. Idle capacity is available. Assembly is instantaneous. Demand is uncorrelated. Everyone will use it. Each is the version that appears on a slide, and each is the version that decides whether the programme succeeds.

Figure 1 — The same workload against two provisioning models. The fixed machine's ratio is not wrong; it is simply not this workload's ratio, and the memory it did not give is sitting idle in a machine that cannot lend it.

5. RTL 1 — Ratios Are What Composition Removes

// RTL 1 - composition. A machine assembled per workload needs each resource
// class present in the amount that workload asks for, not in a fixed ratio.
module composed_machine #(parameter int FIXED_RATIO = 0) (
  input  logic clk, rst_n,
  input  logic       compose,
  input  logic [7:0] want_cores, want_gb, want_accel,
  input  logic [7:0] pool_cores, pool_gb, pool_accel,
  input  logic [7:0] ratio_gb_per_core,
  output logic [7:0] given_cores, given_gb, given_accel,
  output logic       satisfied,
  output logic [7:0] n_composes, n_refused,
  output logic       mismatch_err
);
  logic [15:0] r_q;
  // A fixed-ratio machine gives memory in proportion to cores, which is what a
  // server SKU does. A composed one gives what was asked for.
  assign r_q = {8'd0, want_cores} * {8'd0, ratio_gb_per_core};
  assign given_cores = (want_cores > pool_cores) ? pool_cores : want_cores;
  assign given_gb = (FIXED_RATIO != 0)
                  ? ((r_q > 16'd255) ? 8'hFF : r_q[7:0])
                  : ((want_gb > pool_gb) ? pool_gb : want_gb);
  assign given_accel = (want_accel > pool_accel) ? pool_accel : want_accel;
  assign satisfied = (given_cores == want_cores) && (given_gb == want_gb)
                     && (given_accel == want_accel);
  // A machine composed with a resource amount the workload did not ask for, and
  // that the pool could have supplied. Gating on what was WANTED rather than on
  // what was GIVEN is what separates a ratio mismatch from an empty pool.
  assign mismatch_err = compose && (given_gb != want_gb) && (want_gb <= pool_gb);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_composes <= 8'd0; n_refused <= 8'd0;
    end else if (compose) begin
      n_composes <= n_composes + 8'd1;
      if (!satisfied) n_refused <= n_refused + 8'd1;
    end
  end
endmodule

Six compositions. A 4 GB-per-core ratio, a pool of 64 cores and 16 accelerators.

Three unsatisfied compositions against five, and three ratio mismatches.

Row four is why the fixed machine is not a strawman. When the workload's ratio happens to match the SKU's, the fixed machine is exactly right and costs nothing extra. Server SKUs exist because that is true often enough to be worth standardising on — and composition is worth building for the workloads where it is not.

Row five is a distinction the model had to be corrected to make. The composed machine gives 100 GB against a 200 GB request because the pool holds 100 — that is a capacity shortfall, not a ratio mismatch, and lumping the two together tells an operator to fix a provisioning model when they need to buy memory. Section 18 records that the guard was gated on the wrong quantity until a new stimulus exposed it.

Row six is the third resource class doing its job. Cores and memory are satisfied exactly and the machine is still refused, because twenty accelerators do not exist. Composition is only as good as the scarcest class, which is section 12's argument arriving early.

6. RTL 2 — Stranded Capacity Is The Whole Case

// RTL 2 - stranded resources. The case for composition is the capacity a fixed
// machine has and cannot lend.
module stranded_ratio #(parameter int IGNORE_STRANDING = 0) (
  input  logic clk, rst_n,
  input  logic       measure,
  input  logic [7:0] installed_gb, used_gb, machines,
  input  logic       is_composed,
  output logic [7:0] idle_per_machine, idle_total, idle_pct,
  output logic       lendable,
  output logic [7:0] n_measures, n_wasteful,
  output logic       waste_hidden_err
);
  logic [15:0] t_q, p_q;
  assign idle_per_machine = (used_gb >= installed_gb) ? 8'd0 : (installed_gb - used_gb);
  assign t_q = {8'd0, idle_per_machine} * {8'd0, machines};
  assign idle_total = (t_q > 16'd255) ? 8'hFF : t_q[7:0];
  assign p_q = (installed_gb == 8'd0) ? 16'd0
             : (({8'd0, idle_per_machine} * 16'd100) / {8'd0, installed_gb});
  assign idle_pct = (p_q > 16'd255) ? 8'hFF : p_q[7:0];
  // Idle memory inside a fixed machine cannot be lent to another machine; idle
  // memory in a pool can. The ignoring model counts both as available.
  assign lendable = (IGNORE_STRANDING != 0) ? 1'b1 : is_composed;
  // Counting idle capacity as available when nothing can reach it.
  assign waste_hidden_err = measure && lendable && !is_composed && (idle_total != 8'd0);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_measures <= 8'd0; n_wasteful <= 8'd0;
    end else if (measure) begin
      n_measures <= n_measures + 8'd1;
      if (idle_total != 8'd0) n_wasteful <= n_wasteful + 8'd1;
    end
  end
endmodule

Four measurements. 64 GB installed per machine, four machines.

Two measurements found idle capacity, and the ignoring model hid it once.

192 GB across four machines is the entire financial argument, and it is invisible from inside any one machine. Each machine reports 48 GB free — unremarkable, healthy headroom — and only the fleet view shows that three machines' worth of memory is sitting somewhere nothing can reach.

Row two is the fix and it is the same 192 GB. Composition does not create capacity; it makes existing capacity reachable. The measurement is identical and the disposition is opposite, which is why lendable depends on the topology rather than on a parameter — the same idle memory is waste in a fixed fleet and inventory in a pooled one.

Why the broken build is not a strawman. Every capacity dashboard sums free memory across a fleet, because that is the natural thing to sum. The number it produces is correct and describes memory that no workload except the one already on that machine can use. Presenting it as available is how a composability programme fails to get funded, because the waste it exists to recover is reported as headroom.

The counter that fixes it is one boolean wider. Section 21 asks for idle capacity and whether it is lendable, and the second field costs nothing to compute — a machine knows whether its memory is local or pooled. The measurement was never the hard part; distinguishing two dispositions of the same number was, and a dashboard that reports only the first will keep describing 192 GB of waste as healthy headroom.

7. RTL 3 — Assembly Takes Time

// RTL 3 - composition has a time cost. A machine assembled on demand is not
// available until it is assembled.
module compose_latency #(parameter int IGNORE_ASSEMBLY = 0) (
  input  logic clk, rst_n,
  input  logic        request,
  input  logic [15:0] bind_ms, scrub_ms, boot_ms, sla_ms,
  output logic [15:0] assemble_ms, total_ms,
  output logic        within_sla,
  output logic [7:0]  n_requests, n_late,
  output logic        sla_miss_err
);
  // Assembly is binding the resources, scrubbing what was somebody else's, and
  // booting what was composed.
  assign assemble_ms = (IGNORE_ASSEMBLY != 0) ? 16'd0 : (bind_ms + scrub_ms);
  assign total_ms = assemble_ms + boot_ms;
  assign within_sla = (total_ms <= sla_ms);
  assign sla_miss_err = request && !within_sla;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_requests <= 8'd0; n_late <= 8'd0;
    end else if (request) begin
      n_requests <= n_requests + 8'd1;
      if (!within_sla) n_late <= n_late + 8'd1;
    end
  end
endmodule

One SLA miss; the ignoring model saw none.

The scrub dominates and it is 19.3 §9's requirement arriving as a latency. Memory that belonged to another workload has to be cleared before this one sees it, and at realistic rates that is the largest term in the assembly. Row four is the exception that proves it: resources that were never used need no scrub, and assembly collapses to a bind.

A composed machine is slower to start than a fixed one, always. That is not a defect to engineer away — it is the price of the flexibility, and the question a platform team has to answer is whether the workload's start-up SLA can absorb it. For a long-running training job, easily. For a request-scoped function, not at all.

8. RTL 4 — Borrowed Resources Are In The Failure Domain

// RTL 4 - a composed machine's failure domain is every resource it borrowed.
module composed_blast #(parameter int COUNT_LOCAL_ONLY = 0) (
  input  logic clk, rst_n,
  input  logic       fail,
  input  logic [7:0] local_parts, borrowed_parts,
  input  logic [7:0] failed_part_is_borrowed,
  output logic [7:0] exposed_parts,
  output logic       machine_down, counted_right,
  output logic [7:0] n_failures, n_down,
  output logic       understated_err
);
  // Every borrowed resource is another thing whose failure takes the machine.
  assign exposed_parts = (COUNT_LOCAL_ONLY != 0) ? local_parts
                                                 : (local_parts + borrowed_parts);
  assign machine_down = fail;
  assign counted_right = (exposed_parts == (local_parts + borrowed_parts));
  // A failure in a borrowed resource that the local-only count never modelled.
  assign understated_err = fail && (failed_part_is_borrowed != 8'd0) && !counted_right;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_failures <= 8'd0; n_down <= 8'd0;
    end else if (fail) begin
      n_failures <= n_failures + 8'd1;
      if (machine_down) n_down <= n_down + 8'd1;
    end
  end
endmodule

Two understatements.

Composition trades a reliability property for a utilisation one, and the trade is rarely written down. A fixed machine depends on its own parts; a composed one depends on its own parts and every pool it drew from and the fabric between them. Row four is the limit: a machine that is 15/16ths borrowed has a failure domain sixteen times its local one.

Row two is what keeps the model honest. A failure in a local part is inside the local-only count, so the understatement is zero — the local model is not wrong about everything, it is wrong about exactly the failures composition introduced. That is why the error is gated on where the failure was, and it is what makes the counter in section 21 worth having: the understatement is only observable when a borrowed part is what failed.

9. RTL 5 — The Saving Needs Uncorrelated Demand

// RTL 5 - the utilisation case. Composition wins when demand is uncorrelated
// and wins nothing when every workload peaks together.
module pool_utilisation #(parameter int ASSUME_UNCORRELATED = 0) (
  input  logic clk, rst_n,
  input  logic        eval,
  input  logic [15:0] sum_peaks, observed_peak, pool_size,
  output logic [15:0] fixed_need, composed_need, saved,
  output logic [15:0] saving_pct,
  output logic        pool_sufficient,
  output logic [7:0]  n_evals, n_wins,
  output logic        overclaim_err
);
  logic [31:0] s_q;
  assign fixed_need = sum_peaks;
  // The uncorrelated assumption substitutes an ideal combined peak for the
  // measured one.
  assign composed_need = (ASSUME_UNCORRELATED != 0) ? (sum_peaks >> 1) : observed_peak;
  assign saved = (fixed_need > composed_need) ? (fixed_need - composed_need) : 16'd0;
  assign s_q = (fixed_need == 16'd0) ? 32'd0
             : (({16'd0, saved} * 32'd100) / {16'd0, fixed_need});
  assign saving_pct = (s_q > 32'd65535) ? 16'hFFFF : s_q[15:0];
  assign pool_sufficient = (composed_need <= pool_size);
  // Claiming a saving on a workload whose measured peak is the sum.
  assign overclaim_err = eval && (saving_pct != 16'd0) && (observed_peak >= sum_peaks);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_evals <= 8'd0; n_wins <= 8'd0;
    end else if (eval) begin
      n_evals <= n_evals + 8'd1;
      if (saving_pct != 16'd0) n_wins <= n_wins + 8'd1;
    end
  end
endmodule

Six evaluations. Peaks summing to 400, a 256 pool.

Three evaluations showed a saving, and the uncorrelated model overclaimed twice.

Row two is the case that decides whether the programme is worth doing. Workloads that peak together need the sum, and pooling them saves nothing while adding assembly latency, a larger failure domain and an access-latency penalty. Composition is not a general efficiency win; it is a bet on the shape of the demand, and that shape is measurable before anything is built.

This is 20.2 §5's argument at the machine level rather than the device level, and it has the same failure mode: substituting an assumed combined peak for a measured one. Row five is the measurement artefact — an observed peak above the sum, which sampling produces and which floors rather than wrapping.

10. Waveform — Assembling A Machine On Demand

The fixed machine in that waveform is not faster at anything. It boots in exactly the same time; it simply had nothing to assemble, because its resources were bolted to it when it was built. The four cycles are the price of the flexibility, and whether they matter is a property of the workload rather than of the fabric.

11. RTL 6 — Taking A Machine Apart

// RTL 6 - a composed machine has to be taken apart as reliably as it was put
// together, or the pool leaks resources it can never reclaim.
module decomposition #(parameter int NO_TEARDOWN_CHECK = 0) (
  input  logic clk, rst_n,
  input  logic       release_req,
  input  logic [7:0] held_parts, returned_parts,
  input  logic       scrub_done,
  output logic [7:0] leaked_parts,
  output logic       fully_returned, reusable,
  output logic [7:0] n_releases, n_leaks,
  output logic       leak_err
);
  assign leaked_parts = (returned_parts >= held_parts) ? 8'd0
                                                       : (held_parts - returned_parts);
  assign fully_returned = (leaked_parts == 8'd0);
  // Resources are reusable only when every part came back and was scrubbed.
  assign reusable = (NO_TEARDOWN_CHECK != 0) ? 1'b1 : (fully_returned && scrub_done);
  // Resources declared reusable that the pool has not actually recovered.
  assign leak_err = release_req && reusable && !fully_returned;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_releases <= 8'd0; n_leaks <= 8'd0;
    end else if (release_req) begin
      n_releases <= n_releases + 8'd1;
      if (!fully_returned) n_leaks <= n_leaks + 8'd1;
    end
  end
endmodule

One leak; the no-check model reused it.

A leak here is permanent in a way a leak elsewhere is not. Two parts the pool believes are in use, held by a machine that no longer exists — nothing will ever release them, because the thing that would have is gone. The pool's usable capacity ratchets down, one abandoned teardown at a time, and the only recovery is an operator noticing the discrepancy.

Row three is the distinction between a leak and unfinished work. Every part came back and none is scrubbed: the pool has its resources, they are simply not yet safe to hand to anybody. That is a queue, not a loss, and treating it as one would have an operator hunting for parts that are sitting in front of them.

Figure 3 — The dashed path is an availability model that survived from a world where a machine's parts were bolted to it. Eight things can take this machine down, and five of them are somewhere else — plus the fabric between, which the diagram shows and no bill of materials lists.

12. RTL 7 — A Pool Of Several Types Is Not A Pool

// RTL 7 - heterogeneity. A pool of one part type is a pool; a pool of several
// is an allocation problem where the scarcest type binds.
module heterogeneous_pool #(parameter int TREAT_AS_FUNGIBLE = 0) (
  input  logic clk, rst_n,
  input  logic       alloc,
  input  logic [7:0] want_a, want_b,
  input  logic [7:0] free_a, free_b,
  output logic [7:0] total_free, total_want, granted_a, granted_b,
  output logic       satisfiable,
  output logic [7:0] n_allocs, n_refused,
  output logic       false_grant_err
);
  assign total_free = free_a + free_b;
  assign total_want = want_a + want_b;
  assign granted_a = (want_a > free_a) ? free_a : want_a;
  assign granted_b = (want_b > free_b) ? free_b : want_b;
  // Treating types as fungible allocates against the total, which is the number
  // a capacity report shows and not the one an allocation needs.
  assign satisfiable = (TREAT_AS_FUNGIBLE != 0) ? (total_want <= total_free)
                     : ((want_a <= free_a) && (want_b <= free_b));
  // An allocation granted that no single type can satisfy.
  assign false_grant_err = alloc && satisfiable
                           && ((want_a > free_a) || (want_b > free_b));
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_allocs <= 8'd0; n_refused <= 8'd0;
    end else if (alloc) begin
      n_allocs <= n_allocs + 8'd1;
      if (!satisfiable) n_refused <= n_refused + 8'd1;
    end
  end
endmodule

Five allocations. Eight of each type free.

Three refusals against one, and two false grants.

This is 20.2 §8's fragmentation argument in a different dimension. There, 100 GB free in three runs could not satisfy a 60 GB request because the free space had a shape. Here, 16 parts free cannot satisfy a 14-part request because the free parts have a type. In both cases the total is a real number that no allocation can use, and in both cases it is the number the capacity report shows.

Both scarcity directions are driven deliberately. Rows two and five are the same shortfall in different types, and section 18 records that testing only one of them left half the satisfiability check unexercised — a conjunction needs each half to fail alone, and with symmetric types it is easy to test one and believe you tested both.

13. RTL 8 — Composed Memory Is Further Away

// RTL 8 - what a composed machine is worth against a fixed one. The saving has
// to survive the latency composition adds to every access.
module composition_value #(parameter int IGNORE_ACCESS_COST = 0) (
  input  logic clk, rst_n,
  input  logic        assess,
  input  logic [15:0] local_ns, pooled_ns,
  input  logic [7:0]  pooled_share_pct,
  input  logic [15:0] budget_ns,
  output logic [15:0] blended_ns, true_blended_ns, penalty_pct,
  output logic        meets_budget, worth_composing,
  output logic [7:0]  n_assess, n_rejected,
  output logic        false_value_err
);
  logic [31:0] b_q, p_q, t_q;
  logic [7:0] share;
  // A parameter that gates the cost must gate the share it applies to.
  assign share = (IGNORE_ACCESS_COST != 0) ? 8'd0 : pooled_share_pct;
  assign b_q = (({16'd0, local_ns} * (32'd100 - {24'd0, share}))
              + ({16'd0, pooled_ns} * {24'd0, share})) / 32'd100;
  assign blended_ns = (b_q > 32'd65535) ? 16'hFFFF : b_q[15:0];
  // What the blend really is, against what the model chose to compute.
  assign t_q = (({16'd0, local_ns} * (32'd100 - {24'd0, pooled_share_pct}))
              + ({16'd0, pooled_ns} * {24'd0, pooled_share_pct})) / 32'd100;
  assign true_blended_ns = (t_q > 32'd65535) ? 16'hFFFF : t_q[15:0];
  assign p_q = ((local_ns == 16'd0) || (blended_ns <= local_ns)) ? 32'd0
             : ((({16'd0, blended_ns} - {16'd0, local_ns}) * 32'd100) / {16'd0, local_ns});
  assign penalty_pct = (p_q > 32'd65535) ? 16'hFFFF : p_q[15:0];
  assign meets_budget = (blended_ns <= budget_ns);
  assign worth_composing = meets_budget;
  // A composition called worthwhile whose blended access misses the budget.
  assign false_value_err = assess && worth_composing && (true_blended_ns > budget_ns);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_assess <= 8'd0; n_rejected <= 8'd0;
    end else if (assess) begin
      n_assess <= n_assess + 8'd1;
      if (!worth_composing) n_rejected <= n_rejected + 8'd1;
    end
  end
endmodule

Five assessments. 100 ns local, 400 ns pooled, a 250 ns budget.

One assessment not worth composing; the cost-ignoring model claimed value once.

Half the working set in the pool is exactly the budget, which makes the pooled share the number that decides the answer. And the penalty is severe: even at 30% pooled, accesses cost 90% more than local, which a workload has to be able to absorb.

Row five is the case that will matter more over time. A pooled tier faster than a local one is not hypothetical — a machine with slow local memory and a fast pooled tier inverts the assumption — and the penalty floors at zero rather than wrapping. Section 18 records that the floor was untested until that case was driven.

Section 22 is what happens when this model and section 9's are both right and the deployment still fails.

14. RTL 9 — The Software Has To Want It

// RTL 9 - the software has to want it. A composable fabric that no scheduler
// asks anything of composes nothing.
module scheduler_uptake #(parameter int ASSUME_ADOPTION = 0) (
  input  logic clk, rst_n,
  input  logic       window,
  input  logic [7:0] jobs_total, jobs_composing, pool_capacity_pct,
  output logic [7:0] uptake_pct, pool_used_pct,
  output logic       pool_justified,
  output logic [7:0] n_windows, n_idle,
  output logic       stranded_pool_err
);
  logic [15:0] u_q;
  assign u_q = (jobs_total == 8'd0) ? 16'd0
             : (({8'd0, jobs_composing} * 16'd100) / {8'd0, jobs_total});
  assign uptake_pct = (ASSUME_ADOPTION != 0) ? 8'd100
                    : ((u_q > 16'd255) ? 8'hFF : u_q[7:0]);
  assign pool_used_pct = (uptake_pct > pool_capacity_pct) ? pool_capacity_pct : uptake_pct;
  // A pool is justified only if enough jobs actually ask for composition.
  assign pool_justified = (uptake_pct >= 8'd25);
  // A pool built and left idle because nothing schedules against it.
  assign stranded_pool_err = window && pool_justified && (u_q < 16'd25);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_windows <= 8'd0; n_idle <= 8'd0;
    end else if (window) begin
      n_windows <= n_windows + 8'd1;
      if (u_q < 16'd25) n_idle <= n_idle + 8'd1;
    end
  end
endmodule

Five observation windows against 100 jobs.

Two windows found the pool under-used, and the assuming model stranded it twice.

This is the property with no hardware in it at all, and it is the one that decides the outcome. A composable fabric is an offer; a scheduler has to accept it. Row two is a pool built, powered, connected and asked for nothing — capacity moved from stranded-inside-machines to stranded-inside-a-pool, which is a worse outcome than not building it, because the machines were at least running workloads.

Row four is the good problem. Ninety percent of jobs want composition and the pool covers half of them — a capacity shortfall, which is a purchase order rather than a programme failure. Under-used and over-subscribed look nothing alike and both are "the pool is not the right size".

Figure 4 — Four chapters of Modules 20 and 21 build the first gate. The last one has no hardware in it at all, and section 22 is a deployment that passed five and failed it.

15. RTL 10 — Composability Assembled

// RTL 10 - composability assembled. Everything that must hold before a fabric of
// resources is a machine somebody can run a workload on.
module composability_model #(parameter int FABRIC_EXISTS_ONLY = 0) (
  input  logic clk, rst_n,
  input  logic       evaluate,
  input  logic       fabric_reaches,     // the resources are reachable
  input  logic       ratios_free,        // resources come in the amounts asked for
  input  logic       assembly_budgeted,  // composing fits its time budget
  input  logic       blast_accounted,    // every borrowed part is in the failure model
  input  logic       teardown_complete,  // released machines return everything
  input  logic       schedulers_use_it,  // software actually asks for composition
  output logic       composable,
  output logic [5:0] fail_mask,
  output logic [7:0] n_eval, n_composable,
  output logic       false_claim_err
);
  assign fail_mask[0] = ~fabric_reaches;
  assign fail_mask[1] = ~ratios_free;
  assign fail_mask[2] = ~assembly_budgeted;
  assign fail_mask[3] = ~blast_accounted;
  assign fail_mask[4] = ~teardown_complete;
  assign fail_mask[5] = ~schedulers_use_it;
  // The fabric-exists build checks that the resources are reachable and calls
  // the data centre composable, which is what a roadmap slide shows.
  assign composable = (FABRIC_EXISTS_ONLY != 0) ? fabric_reaches : (fail_mask == 6'd0);
  assign false_claim_err = evaluate && composable && (fail_mask != 6'd0);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_eval <= 8'd0; n_composable <= 8'd0;
    end else if (evaluate) begin
      n_eval <= n_eval + 8'd1;
      if (composable) n_composable <= n_composable + 8'd1;
    end
  end
endmodule

One composable of six, and four false claims.

The fabric-exists definition is what a roadmap slide shows, and it is right about exactly one of the six. Row five is the honest ending to Module 21: a fabric that reaches everything, resources that come in any amount, assembly inside its budget, a correct failure model and complete teardown — and no scheduler asking for any of it.

Everything Modules 20 and 21 built is fail_mask[0]. Four chapters of switching, pooling, routing, peer transfers and scale, and they establish the first bit of six. That is not a criticism of the hardware; it is the shape of the problem. The fabric was the easy part, and it is the part with a specification.

16. Quantitative Reasoning

Ratios. A workload wanting 8 cores and 64 GB gets 32 GB from a 4-GB-per-core machine — half. At 200 GB it gets 32, a 168 GB mismatch. Three of six compositions mismatched, and one was a pool shortfall the model had to be corrected to distinguish.

Stranded capacity. 48 GB idle in each of four machines: 192 GB, 75% of what was installed, and none of it lendable. The same 192 GB in a pool is inventory. The measurement is identical; the disposition is opposite.

Assembly. 200 ms bind, 800 ms scrub, 3000 ms boot: 4000 ms. A larger scrub takes it to 5700 against a 5000 ms SLA, and the scrub is the dominant term.

Failure domain. Three local parts and five borrowed is eight, not three. A machine that is 15/16ths borrowed is sixteen times its local exposure.

Utilisation. Peaks summing to 400 with a measured combined peak of 200 saves 50%. Workloads that peak together save nothing, and the uncorrelated model claims 50% regardless.

Teardown. Eight parts held, six returned: two leaked permanently, because the machine that would have released them no longer exists.

Heterogeneity. Sixteen parts free cannot satisfy a fourteen-part request when twelve of them are one type. Two false grants, one in each scarcity direction.

Access latency. 30% of the working set pooled costs 90% more than local; 50% pooled is exactly a 250 ns budget; 60% misses it.

Uptake. Five percent of jobs composing does not justify a pool. Capacity moved from stranded-inside-machines to stranded-inside-a-pool.

The assembled model. Six properties, six configurations, one composable. The fabric-exists definition reported five.

17. Assertions

Every check is an explicit comparison against an exact value. Icarus Verilog 13.0 has no concurrent assertion support, so each is a procedural comparison against 1'b1, and every one is an equality.

Composition. The pool shortfall is asserted as not a mismatch, which is the distinction section 18 records the model being corrected to make.

chk(cGg == 8'd100, "the pool holds 100 GB, so 100 is given");
chk(cSa == 1'b0,   "which does not satisfy a 200 GB request");
chk(cMe == 1'b0,   "and is not a mismatch, it is an empty pool");

The accelerator class is failed alone, with cores and memory exact.

Stranding. The same idle figure is asserted lendable in a pool and not in a fixed machine.

chk(sIt == 8'd192, "the same 192 GB idle");
chk(sLe == 1'b1,   "which in a pool is lendable");

Assembly. The exact SLA boundary is driven, and the no-scrub case isolates the bind.

Blast radius. A local failure is asserted as not an understatement.

Utilisation. The peak-together case is asserted to save nothing, and the measurement artefact is asserted to floor.

chk(uCn == 16'd500, "the measured peak exceeds the sum");
chk(uSv == 16'd0,   "so the saving floors at zero rather than wrapping");

Teardown. Unscrubbed-but-returned is asserted as unfinished work rather than a leak.

chk(dFr == 1'b1, "so nothing leaked");
chk(dRu == 1'b0, "but an unscrubbed part is not reusable");
chk(dLe == 1'b0, "which is not a leak, it is unfinished work");

Heterogeneity. Both scarcity directions are driven, and the granted count is asserted in each.

Access latency. The exact budget is driven, and a faster pool is asserted to floor the penalty.

Uptake. The exact justification threshold is driven, and the pool-limited case is asserted separately from the uptake-limited one.

The assembled model. Every fail mask is asserted as an exact six-bit value, and each of the six bits is driven false alone.

Totals: 246 checks across two testbenches, 123 on the front five models and 123 on the back five, all passing on the unmutated sources.

18. Mutation Testing

Forty-eight mutations were injected one at a time.

48 injected, 48 killed, after nine survivors were diagnosed across two rounds — and one of them found a defect in the design.

A design defect, found by a stimulus a survivor asked for. Removing the pool cap from given_gb survived because no composition had ever requested more memory than the pool held. Driving it — 200 GB wanted against a 100 GB pool — killed the mutation and immediately failed three assertions, because mismatch_err was gated on given_gb <= pool_gb rather than on want_gb <= pool_gb. With a shortfall, given_gb equals the pool size, so the guard passed and the model reported a ratio mismatch for what was actually an empty pool. The fix changes which quantity the guard tests, and section 5's row five exists to document the distinction.

Survivors that appeared only after the testbench changed. Three m1 mutations were killed in the first round and survived in the second, because the added stimulus altered the counter totals the earlier kills had depended on. A mutation set is only valid against the testbench it was run with, and re-running the whole set after every stimulus change is what caught it — a partial re-run would have reported them still dead.

Two symmetric halves, one tested. The heterogeneous-pool satisfiability check survived dropping type B because every scarcity case in the testbench was scarce in type A. With symmetric types it is easy to test one direction and believe both are covered. A conjunction over symmetric operands needs each operand to fail alone, and the symmetry is what makes the gap invisible.

Four floors and thresholds never driven. The saving floor needed a measured peak above the sum. The penalty floor needed a pool faster than local. The sufficiency comparison needed a demand exactly the pool size. The uncapped grant needed a request exceeding one type. All four are guards the correct code was written to have and no natural stimulus reaches.

19. Verification Strategy

What a testbench for a composable platform must cover.

Re-run the entire mutation set after every stimulus change. Section 18's second finding: three mutations killed in one round survived in the next, because the counters their kills depended on had moved. A partial re-run reports stale results confidently.

Each operand of a symmetric conjunction, failing alone. Type A scarce and type B scarce are different tests, and symmetry is exactly what makes it feel like they are not.

Every floor, with an input that reaches it. A measured peak above the sum of peaks. A pooled tier faster than a local one. More returned than was held. None of these is a healthy state and all of them are states a real pipeline reports.

The cases that are correct and look like failures. A pool shortfall that is not a ratio mismatch. Unscrubbed parts that are not leaked. A local failure that the local-only model correctly covers. A fixed machine whose ratio happens to fit.

Each mask bit driven false alone. Six properties, six single-bit configurations plus the all-clear.

What a real platform needs that these models do not have. Concurrency — two compositions competing for the last accelerator. Preemption — reclaiming a resource from a running composed machine, which is 19.4 §9's eviction problem with a machine attached. Partial failure — a borrowed resource that degrades rather than disappearing, where the machine is neither up nor down.

20. Synthesis and Implementation Reality

Nothing in this chapter is silicon, and that is the point. 21.1 and 21.3 described switches, channels, PHYs and link budgets. This chapter describes a scheduler, a resource manager and a capacity model. The hardware to do composition exists; the software to want it is the open problem.

Assembly latency is dominated by a memory-controller operation. Section 7's scrub is 19.3 §9's residue requirement priced in milliseconds, and the only lever is scrub bandwidth — which is 20.2 §20's trade between rebind speed and tenant traffic, reappearing at machine granularity.

The failure model has to be rebuilt, not extended. Section 8's borrowed parts are not additional components of a known machine; they are a machine whose composition changes per workload. Availability models that assume a fixed bill of materials do not have a term for this, and producing one is a modelling project rather than a configuration change.

Teardown reliability is a distributed-systems problem. Section 11's leak happens when the machine that should release resources is gone. The fix is a lease with an expiry rather than a release protocol — the pool reclaims on timeout rather than waiting to be told — which is a different architecture from the one a release-based design produces.

Heterogeneous allocation is bin-packing. Section 12's two types generalise to cores, memory, accelerators, network and storage tiers, and the allocator is choosing across all of them simultaneously. That is the scheduler problem section 14 says has to be solved, and it is the piece of the vision with the least hardware in it and the most work.

21. Silicon Observability

The last row is the one nobody builds because it is not a hardware counter. Every other line here is a fabric or device statistic; uptake is a scheduler statistic, and a platform team instrumenting the fabric thoroughly can end up with perfect visibility into a pool and no idea how many workloads asked for it. Section 22 is that gap.

22. Debug Lab

Symptom. A composable memory pool is deployed alongside 200 fixed machines. Six months in, the finance review asks what it saved. The answer is nothing measurable. The fabric is healthy, the pool is at 12% utilisation, and no incident has ever been attributed to it.

Step 1 — is the pool working? Compositions attempted: 4,100 over six months. Satisfied: 4,050. Refused: 50, all pool shortfalls at peak. The mechanism works.

Step 2 — is the saving real for those that used it? Compare the composed machines' memory against what a fixed SKU would have given. The composed machines took an average of 96 GB against a 32 GB SKU allocation — exactly section 5's argument, and a real saving for those workloads.

Step 3 — then why is the total zero? 4,100 compositions over six months against a fleet running roughly 400,000 jobs. Uptake is 1%. Section 14's row two, at production scale.

Step 4 — why is uptake 1%? The scheduler requests composition only when a job explicitly asks for a non-standard memory ratio, and the job submission template does not expose that field. Almost no job asks because almost no job can.

Step 5 — would uptake help? Read measured combined peak against sum of peaks for the fixed fleet: 0.55. Section 9's bet is good — demand genuinely is uncorrelated, and a pool serving the whole fleet would save roughly 45% of installed memory.

The finding. Every hardware property held. The fabric reached, the ratios were free, assembly fit, the failure model was correct and teardown was clean — five of six. The sixth was a field missing from a job submission template, and it made the other five worth nothing.

The fix. Expose the ratio in the submission path and default it from observed usage rather than from a SKU. That is a change to a scheduler and a form, and it is the entire difference between a 1% uptake and a 45% memory saving.

What made this hard. Nothing failed. There was no incident to investigate, no counter out of range, and no component behaving incorrectly. The only observable was an absence — jobs that never asked — and the fleet had no counter for it because it is not a hardware quantity.

23. Design Review

1. What is the measured combined peak against the sum of per-machine peaks? Below one, composition saves. At one, it saves nothing. Section 9, and it is measurable before anything is built.

2. How much capacity is idle and unlendable today? Section 6, and the second half of the question is the whole argument.

3. What is the assembly time, split into bind, scrub and boot? The scrub dominates. Section 7.

4. Does the availability model include borrowed parts? Section 8, and if the model assumes a fixed bill of materials it has no term for them.

5. What happens to a composed machine's resources if the machine dies without releasing them? If the answer is "we wait for the release", section 11.

6. Is free capacity reported by type or in total? Section 12, and the total is the misleading one.

7. What fraction of the working set will be pooled, and what does that blend to? 30% pooled is a 90% access penalty. Section 13.

8. Can a job ask for a non-standard ratio, and how? Section 22 turns entirely on this question.

9. Is there an uptake counter, and who owns it? It is a scheduler statistic, which is why a fabric team can instrument everything and still not have it. Section 21.

10. Which of the six properties does the team believe "composable" means? Section 15 exists because the answer is the fabric — the one part that already has a specification.

24. How This Appears In Real Engineering

A platform team building a composability business case should do sections 9 and 6 first and in that order: measure the combined peak against the sum, then measure idle-and-unlendable capacity. Both are measurable on the existing fleet with no new hardware, and together they bound the saving before a single part is bought.

A scheduler team owns sections 12 and 14, which are the two properties with no hardware in them and the two that section 22 shows deciding the outcome. Heterogeneous bin-packing is a genuinely hard allocation problem, and exposing composition in a submission path is a genuinely easy one — and the easy one is worth more.

An availability team meets section 8 and finds that their model has no shape for it. A composed machine's bill of materials changes per workload, and an availability model built around fixed configurations cannot express "this machine depends on those five pool devices this week". That is a modelling project, and it is usually discovered after the pool is deployed.

A finance reviewer asks section 22's question, and the honest answer needs sections 6, 9 and 14 together: how much was stranded, how uncorrelated the demand is, and how much of the fleet actually asked. Two of those three are hardware measurements and the third decides the answer.

25. Common Misconceptions

"CXL 3.x delivers composable infrastructure." It delivers the fabric — one property of six. Section 15.

"Idle memory across the fleet is available capacity." Not from any machine but the one it is bolted to. 192 GB across four machines, unreachable. Section 6.

"Composition saves memory." Only if demand is uncorrelated. Workloads that peak together need the sum and save nothing. Section 9.

"A composed machine is like a fixed one." It starts slower — 4000 ms against 3000 — and its failure domain is every pool it borrowed from. Sections 7 and 8.

"The pool has 16 parts free, so a 14-part request will succeed." Not if twelve of them are the wrong type. Section 12.

"Pooled memory is a bit slower." 30% of the working set pooled is a 90% access penalty over local. Section 13.

"Releasing a machine returns its resources." Unless the machine died first, in which case the parts are held forever. Section 11.

"Unscrubbed parts have leaked." They are in the pool and not yet safe to hand out. A queue, not a loss. Section 11.

"We built the pool, so it will be used." Five percent uptake is capacity moved from stranded-in-machines to stranded-in-a-pool. Sections 14 and 22.

"The hard part is the hardware." Four chapters of Modules 20 and 21 establish fail_mask[0]. Section 15.

26. Interview Reasoning

Q. What does composable infrastructure actually save?

Stranded capacity — memory installed in a machine, idle, and unreachable by any other machine. 48 GB in each of four machines is 192 GB. The follow-up that matters: it only saves if demand is uncorrelated, because workloads that peak together need the sum, and that ratio is measurable on the existing fleet before anything is bought.

Q. What does a composed machine cost that a fixed one does not?

Three things: assembly time, dominated by scrubbing memory that belonged to somebody else; a larger failure domain, because every borrowed part can take it down; and an access-latency penalty on the pooled fraction of its working set — 90% over local at a 30% pooled share.

Q. A pool reports 16 parts free and a 14-part allocation fails. Why?

Twelve of the fourteen were one type and only eight of that type exist. Free capacity by type is the number an allocation needs; the total is the number the report shows. The follow-up: this is the same shape as memory fragmentation — a real total that no allocation can use, differing in shape rather than in type.

Q. A composed machine crashes without releasing its resources. What happens?

They are held forever, because the thing that would have released them is gone — and the pool's usable capacity ratchets down. The right answer is an architectural one: leases with expiry rather than a release protocol, so the pool reclaims on timeout rather than waiting to be told.

Q. Your composable pool has been deployed for six months and saved nothing. Where do you look?

Uptake, before anything else. If the mechanism works and the per-composition saving is real, the only remaining variable is how many workloads asked — and that is a scheduler counter, not a fabric one, which is why a well-instrumented fabric team can miss it entirely. Section 22's answer was a missing field in a submission template.

Q. How much of CXL 3.x's work does composability need?

All of it, and it is one of six things. The fabric, the pooling, the peer transfers and the scale are fail_mask[0] — the resources being reachable. The other five are ratios, assembly time, failure modelling, teardown and software uptake, and only the first of those has a specification to implement against.

27. Exercises

1. Extend RTL 1 to five resource classes and show that satisfiability is bounded by the scarcest, whichever it is.

2. Combine RTL 2 and RTL 5 into one business case: measure stranded capacity and combined peak, and derive the saving as a single number.

3. Make RTL 3's scrub time a function of the memory size and show the size at which assembly exceeds a given start-up SLA.

4. Extend RTL 4 to compute an availability figure from per-part MTBFs and show how much a composed machine's availability falls against a fixed one.

5. Replace RTL 6's release protocol with a lease and expiry, and show that the permanent-leak failure mode disappears.

6. Turn RTL 7 into a bin-packer over three classes and measure how much capacity is unallocatable at a given request-size distribution.

7. Give RTL 8 a working-set model rather than a fixed share, and find the working-set size at which the pooled fraction crosses the budget.

8. Model section 22 end to end: a working pool, a real per-composition saving, and 1% uptake. Show that the aggregate saving is the product.

9. Combine RTL 5 and RTL 9 and find the uptake at which a pool sized for 45% of the fleet becomes over-subscribed.

10. Add a seventh property to RTL 10. If it is implied by one of the six, say which; if not, give the data centre it catches that the current mask calls composable.

28. Summary

Module 20 built the switch and the pool. Module 21 built the fabric, the peer transfer and the scale. All of it is one bit of six.

Ratios are what composition removes. A workload wanting 64 GB gets 32 from a 4-GB-per-core machine — and a pool that holds only 100 GB against a 200 GB request is a shortfall, not a mismatch, a distinction the model had to be corrected to make.

Stranded capacity is the entire case. 192 GB across four machines, 75% of what was installed, invisible from inside any one of them — and the same 192 GB in a pool is inventory. The measurement is identical and the disposition is opposite.

Assembly takes time and the scrub dominates it. 4000 ms against a fixed machine's 3000, and a bigger scrub misses a 5000 ms SLA — 19.3's residue requirement arriving as a start-up latency.

A composed machine's failure domain is everything it borrowed. Three local parts and five borrowed is eight; 1 and 15 is sixteen times the local exposure, and the understatement is only visible when a borrowed part is what failed.

The saving is a bet on uncorrelated demand. Workloads that peak together save nothing, and the assumption is measurable on the existing fleet before anything is built.

A machine that dies without releasing leaks permanently, because the thing that would have released is gone — and unscrubbed parts are a queue rather than a loss.

A pool of several types is a bin-packing problem. Sixteen parts free cannot serve fourteen when twelve are one type — the same shape as fragmentation, differing in type rather than in shape.

Pooled memory is further away. 30% of the working set pooled is a 90% access penalty; half is exactly a 250 ns budget; 60% misses it.

And the software has to ask. Section 22's deployment had five of six properties and 1% uptake, because a job submission template had no field for a memory ratio — which made a 45% fleet-wide saving worth nothing.

A mutation survivor found a real defect again, as it did in 21.2: a request larger than the pool exposed a guard testing what was given instead of what was wanted, reporting a ratio mismatch for an empty pool.

Reaching the resources is one property of six. The definition a roadmap slide shows called five of six data centres composable when one was — and the five it got wrong are ratios, time, failure modelling, teardown and adoption, of which only the first has a specification.

The fabric was the easy part. That is not a disappointment; it is where Module 21 ends and the work begins.