Isolation

19.1 established who is on the link and 19.2 protected what crosses it. Both are properties of a link between two parties.

A pooled CXL device has more than two parties. Four hosts attach to one device through a switch, each legitimately, each authenticated, each on a protected link — and every one of them is now reaching into the same silicon. Nothing in the previous two chapters says a word about what stops host A reading host B's memory.

That is this chapter.

1. The Engineering Problem — Sharing A Device Is Not Isolation

Isolation is the property everybody assumes and almost nobody specifies. Six things make it hard.

Disjoint address ranges are the easy part and the smallest part. Two regions that do not overlap are the first requirement, not the definition, and section 17 shows a device that checks only this calling four of five insecure configurations isolated.

A check the host performs is not enforcement. The host that would violate the boundary is the host you are asking to police it. Section 6.

A fault has a radius. An uncorrectable error in one tenant's region either stops there or takes down every host attached to the device, and that is a design decision made in the error-handling path long before the fault occurs. Section 8.

Released memory still holds what the last tenant wrote. Capacity that moves from tenant A to tenant B is a disclosure unless something clears it in between, and the scrub costs time the reallocation path has to pay for. Section 9.

Reset has a blast radius too. A host that resets its own link and takes another host's regions down with it has broken isolation using a mechanism nobody thinks of as a security surface. Section 10.

And a shared structure is an observable one. Two tenants sharing one queue can each infer the other's occupancy from their own latency, without reading a single byte of memory. Section 12.

This chapter against 19.2, stated precisely. That one owns protecting traffic between two authenticated endpoints. This one owns what happens inside a device that several authenticated endpoints share — a place where every link is protected and isolation can still fail completely.

2. The One-Sentence Model

Two tenants are isolated when their regions are disjoint, the device itself enforces that, one tenant's fault and one tenant's reset stay inside its own regions, released capacity is scrubbed before reuse, and no shared structure lets one infer the other — and every defect below is a device satisfying some of those and being described as isolated.

3. What This Chapter Owns

Deferred:

4. Teaching-Model Boundary

Every model here is a small synchronous block that isolates one property. Real isolation lives across a decoder, an error-handling path, a fabric-manager interface and a memory controller, and none of those is reproduced. The models are deliberately small enough that the failure is visible in the source.

Each model is built twice: a correct build and a broken build selected by a parameter. The broken build is never a strawman — every one of them is a real design that shipped somewhere, or a real shortcut somebody argued for in a review.

Figure 1 — Both links are authenticated and both are protected, and the dashed path is still open. Everything 19.1 and 19.2 established happens to the left of the switch; every property this chapter owns is to the right of it.

5. RTL 1 — Two Regions That Must Not Overlap

The first requirement is arithmetic. Two regions overlap when each begins before the other ends, and the comparison must be half-open — a region running from base to base plus size occupies the base and not the end, so two regions that touch exactly do not overlap.

// RTL 1 - the first requirement: two tenants' regions must not overlap.
// Half-open ranges: a region owns [base, base+size), so touching is not
// overlapping.
module region_overlap #(parameter int TRUST_HOST = 0) (
  input  logic clk, rst_n,
  input  logic        allocate,
  input  logic [15:0] a_base, a_size, b_base, b_size,
  output logic        overlaps, granted,
  output logic [15:0] a_end, b_end,
  output logic [7:0]  n_alloc, n_refused,
  output logic        overlap_granted_err
);
  assign a_end = a_base + a_size;
  assign b_end = b_base + b_size;
  // A zero-size region occupies no addresses and can overlap nothing. Without
  // the size guard the half-open comparison reports an overlap for a region that
  // does not exist.
  assign overlaps = (a_size != 16'd0) && (b_size != 16'd0)
                    && (a_base < b_end) && (b_base < a_end);
  // The trusting build assumes the fabric manager already checked.
  assign granted = (TRUST_HOST != 0) ? allocate : (allocate && !overlaps);
  // An allocation accepted that puts two tenants on the same bytes.
  assign overlap_granted_err = granted && overlaps;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_alloc <= 8'd0; n_refused <= 8'd0;
    end else if (allocate) begin
      n_alloc <= n_alloc + 8'd1;
      if (!granted) n_refused <= n_refused + 8'd1;
    end
  end
endmodule

What the broken build gets wrong. TRUST_HOST performs no overlap check at all, on the grounds that the fabric manager computed the layout and would not send an overlapping one. That is true right up until a fabric manager with a bug, a stale view of the device, or a race between two concurrent allocations sends one anyway.

Seven allocations, three of them overlapping.

Three refusals against none, and the trusting build granted every overlap it was given.

The two adjacency rows are the ones worth reading twice. [0,100) and [100,200) touch and do not overlap; move the second region one byte earlier and they do. Both directions matter: the last row is the mirror case, a region ending exactly where another begins, and a comparison that is half-open on one side and inclusive on the other passes the first and fails the second.

The zero-size row is the degenerate case. A region of no bytes overlaps nothing, and a half-open comparison without a size guard reports that it overlaps everything containing its base — a refusal for an allocation that touches no memory at all.

6. RTL 2 — The Device Must Enforce, Not The Host

The overlap check runs when capacity is assigned. Enforcement runs on every access afterwards, and the question is which side of the link performs it.

// RTL 2 - a check the host performs is not enforcement. The device decides
// whether a request it received is one the requester was entitled to make.
module device_enforcement #(parameter int HOST_SIDE_ONLY = 0) (
  input  logic clk, rst_n,
  input  logic        access,
  input  logic [15:0] addr, owner_base, owner_size,
  input  logic [3:0]  requester, owner_id,
  output logic        in_owner_range, right_owner, permitted,
  output logic [7:0]  n_access, n_blocked,
  output logic        escape_err
);
  logic [15:0] owner_end;
  assign owner_end     = owner_base + owner_size;
  assign in_owner_range = (addr >= owner_base) && (addr < owner_end);
  assign right_owner   = (requester == owner_id);
  // The host-side build performs no device check at all: whatever arrives is
  // assumed to have been validated before it was sent.
  assign permitted = (HOST_SIDE_ONLY != 0) ? 1'b1
                                           : (in_owner_range && right_owner);
  // A request served for an address or a requester it does not own.
  assign escape_err = access && permitted && !(in_owner_range && right_owner);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_access <= 8'd0; n_blocked <= 8'd0;
    end else if (access) begin
      n_access <= n_access + 8'd1;
      if (!permitted) n_blocked <= n_blocked + 8'd1;
    end
  end
endmodule

Two conditions, and they fail independently. An address inside the range from the wrong requester is one failure; the right requester at an address outside the range is another. A model that checks only the address serves every cross-tenant request that happens to land in bounds — which, when a fabric manager has just moved a region, is most of them.

Three blocks against none. The last two rows are the range boundary: address 0 is the base and is inside, address 100 is the end and is not.

Why the broken build is not a strawman. The host-side model is how a single-host CXL 1.1 attachment works, and it is correct there — one host owns the whole device and there is nothing to enforce against. The failure is that the model survives into a pooled deployment where the premise no longer holds. The device did not change. The number of hosts did.

7. Waveform — An Access That Escapes

Three escapes in eight cycles, and nothing on the link was wrong. Every one of those requests was correctly formed, correctly authenticated and correctly protected. The failure is entirely inside the device, in a check that was not performed.

8. RTL 3 — A Fault Must Stay In Its Region

An uncorrectable error in pooled memory belongs to whichever tenant owned the address. What the device does about it decides whether the fault is one tenant's problem or everybody's.

// RTL 3 - a fault has a radius. An error in one tenant's region either stops
// there or takes every attached host down with it.
module error_containment #(parameter int GLOBAL_FAULT = 0) (
  input  logic clk, rst_n,
  input  logic       fault,
  input  logic [3:0] faulting_region, observer_region,
  output logic       same_region, observer_affected, contained,
  output logic [7:0] n_faults, n_spread,
  output logic       spread_err
);
  assign same_region = (faulting_region == observer_region);
  // The global build escalates every uncorrectable error to a device-level
  // event, which every attached host sees.
  assign observer_affected = (GLOBAL_FAULT != 0) ? 1'b1 : same_region;
  assign contained = !observer_affected || same_region;
  // A tenant disturbed by a fault in a region that is not theirs.
  assign spread_err = fault && observer_affected && !same_region;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_faults <= 8'd0; n_spread <= 8'd0;
    end else if (fault) begin
      n_faults <= n_faults + 8'd1;
      if (observer_affected && !same_region) n_spread <= n_spread + 8'd1;
    end
  end
endmodule

Four faults, one of them in the observer's own region.

Three spreads against none. The first row is the case that must not be mistaken for containment: the observer is affected, and that is correct, because the fault is in memory the observer owns. Containment is not "nobody is affected"; it is "only the owner is".

Why the broken build is not a strawman. Escalating an uncorrectable memory error to a device-level fatal event is the conservative choice, and on a single-host device it is the right one — there is no distinction between the device failing and the tenant failing. On a pooled device the same policy converts one tenant's bad DRAM into an outage for every host attached to the switch port, which is a much larger event than the one that occurred.

9. RTL 4 — Released Memory Carries The Previous Tenant's Data

Capacity that moves from one tenant to another is a disclosure channel with no protocol involved at all. The bytes are simply still there.

// RTL 4 - released capacity still holds what the previous tenant wrote.
// Reallocating it to a different tenant without scrubbing is a disclosure.
module residue_scrub #(parameter int NO_SCRUB = 0) (
  input  logic clk, rst_n,
  input  logic        reallocate,
  input  logic [15:0] released_bytes, scrubbed_bytes,
  input  logic [3:0]  prev_owner, new_owner,
  output logic        fully_scrubbed, different_owner, safe_to_reissue,
  output logic [15:0] residue_bytes,
  output logic [7:0]  n_realloc, n_leaked,
  output logic        residue_err
);
  assign different_owner = (prev_owner != new_owner);
  assign residue_bytes   = (scrubbed_bytes >= released_bytes) ? 16'd0
                                                              : (released_bytes - scrubbed_bytes);
  // The no-scrub build reallocates immediately, on the grounds that the new
  // tenant will overwrite what it needs.
  assign fully_scrubbed  = (NO_SCRUB != 0) ? 1'b0 : (residue_bytes == 16'd0);
  // Reissuing to the same owner needs no scrub: the data is already theirs.
  assign safe_to_reissue = !different_owner || fully_scrubbed;
  // Handing a new owner bytes the previous one wrote.
  assign residue_err = reallocate && different_owner && !fully_scrubbed;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_realloc <= 8'd0; n_leaked <= 8'd0;
    end else if (reallocate) begin
      n_realloc <= n_realloc + 8'd1;
      if (different_owner && !fully_scrubbed) n_leaked <= n_leaked + 8'd1;
    end
  end
endmodule

Four reallocations.

One leak in the correct build against three, and the third row is the reason the condition is different_owner and not simply "reallocated". Handing capacity back to the tenant that just released it needs no scrub: the data being reissued is that tenant's own. A model that scrubs unconditionally is not wrong, it is slow — and on a pool where most reallocations are the same tenant growing and shrinking, it is slow for no benefit.

The second row is the partial scrub, which is the one that actually happens. A scrub interrupted by a power event, a fabric-manager timeout or a reset leaves a fraction of the region carrying the previous tenant's data, and 512 bytes of somebody else's memory is a full disclosure of 512 bytes.

10. RTL 5 — Reset Has A Blast Radius

Reset is not usually thought of as a security surface. On a pooled device it is one, because the scope of a reset and the scope of a tenant are two different things that a designer has to deliberately make agree.

// RTL 5 - a reset requested by one host must clear only that host's regions.
// A device-wide reset is a denial-of-service one tenant can trigger.
module reset_isolation #(parameter int DEVICE_WIDE_RESET = 0) (
  input  logic clk, rst_n,
  input  logic       reset_req,
  input  logic [3:0] resetting_host, other_host,
  input  logic [3:0] regions_of_resetter, regions_of_other,
  output logic       other_disturbed, scope_correct,
  output logic [3:0] regions_cleared,
  output logic [7:0] n_resets, n_collateral,
  output logic       collateral_err
);
  // A correct reset clears only the resetting host's own regions.
  assign regions_cleared = (DEVICE_WIDE_RESET != 0)
                         ? (regions_of_resetter | regions_of_other)
                         : regions_of_resetter;
  assign other_disturbed = |(regions_cleared & regions_of_other);
  assign scope_correct   = (regions_cleared == regions_of_resetter);
  // Clearing regions belonging to a host that did not ask for a reset.
  assign collateral_err = reset_req && other_disturbed
                          && (resetting_host != other_host);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_resets <= 8'd0; n_collateral <= 8'd0;
    end else if (reset_req) begin
      n_resets <= n_resets + 8'd1;
      if (other_disturbed && (resetting_host != other_host)) n_collateral <= n_collateral + 8'd1;
    end
  end
endmodule

The regions are a four-bit mask. The resetting host owns regions 0 and 1 (4'b0011); the other host owns 2 and 3 (4'b1100).

Two collateral events against none. The middle row is the case that keeps the model honest: a device-wide reset still clears every region, but when the host that owns those regions is the host that asked, there is nobody to have wronged. Collateral damage requires a victim, and other_disturbed alone does not establish one.

The third row is the degenerate case. A host that owns no regions clears none of its own, and the correct build's blast radius is empty — while the device-wide build still takes the other tenant down on behalf of a host with nothing on the device at all.

Figure 3 — The same request, two scopes. Nothing about the reset itself distinguishes them: the difference is entirely in which regions the device decided a link reset owns, which is a line of RTL written years before the device was pooled.

11. RTL 6 — A Shared Structure Is An Observable One

The four properties so far are about memory. This one is not, and it is the one that survives every address check in the chapter.

// RTL 6 - a shared structure is an observable one. Sharing a queue between
// tenants makes one tenant's occupancy visible to the other.
module shared_structure #(parameter int SHARED_QUEUE = 0) (
  input  logic clk, rst_n,
  input  logic        probe,
  input  logic [7:0]  a_occupancy, b_occupancy, depth,
  output logic [7:0]  observed_by_b, a_share_pct,
  output logic        b_sees_a, partitioned,
  output logic [7:0]  n_probes, n_observable,
  output logic        observable_err
);
  logic [15:0] sp_q;
  // A partitioned queue shows a tenant only its own occupancy. A shared one
  // shows the total, from which the other tenant's share is a subtraction.
  assign observed_by_b = (SHARED_QUEUE != 0) ? (a_occupancy + b_occupancy)
                                             : b_occupancy;
  assign partitioned = (SHARED_QUEUE == 0);
  assign b_sees_a    = (observed_by_b != b_occupancy);
  assign sp_q = (depth == 8'd0) ? 16'd0
                                : (({8'd0, a_occupancy} * 16'd100) / {8'd0, depth});
  assign a_share_pct = (sp_q > 16'd255) ? 8'hFF : sp_q[7:0];
  // One tenant able to infer another's occupancy from what it can observe. No
  // separate "A is non-empty" term is needed: b_sees_a is already the statement
  // that the observed total differs from B's own, which happens only when A
  // holds something.
  assign observable_err = probe && b_sees_a;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_probes <= 8'd0; n_observable <= 8'd0;
    end else if (probe) begin
      n_probes <= n_probes + 8'd1;
      if (b_sees_a) n_observable <= n_observable + 8'd1;
    end
  end
endmodule

Four probes against a 64-entry queue.

Three observable probes against none. The second row is worth sitting with: when A is empty, the shared queue and the partitioned queue report the same number, and there is no leak because there is nothing to leak. A channel that carries no information when the secret is absent is still a channel — it is carrying the information that the secret is absent.

This is the property that survives everything else in the chapter. The regions are disjoint, the device enforces them, faults are contained, capacity is scrubbed and resets are scoped — and B can still watch its own queue depth and learn when A got busy. Nothing about the address path is involved.

The zero-depth row is the divide-by-zero guard. A queue with no depth has no share to report, and a percentage computed without the guard is an X propagating into a telemetry register that some fabric manager will eventually read.

12. RTL 7 — The Fabric Manager Is A Separate Trust Domain

Everything so far treats the region layout as given. Somebody programs it, and whoever that is holds power over every tenant on the device.

// RTL 7 - the fabric manager is a separate trust domain. A host that can reach
// the management path can reconfigure every other host's binding.
module management_isolation #(parameter int HOST_CAN_MANAGE = 0) (
  input  logic clk, rst_n,
  input  logic       request,
  input  logic [1:0] origin,        // 0 host data path, 1 host mgmt path, 2 fabric manager
  input  logic       is_rebind,
  output logic       permitted, from_fm,
  output logic [7:0] n_requests, n_denied,
  output logic       privilege_err
);
  assign from_fm = (origin == 2'd2);
  // Rebinding is a fabric-manager operation. The permissive build lets a host
  // issue it over its own management path.
  assign permitted = (HOST_CAN_MANAGE != 0) ? (from_fm || (origin == 2'd1))
                                            : from_fm;
  // A rebind accepted from something that is not the fabric manager.
  assign privilege_err = request && is_rebind && permitted && !from_fm;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_requests <= 8'd0; n_denied <= 8'd0;
    end else if (request) begin
      n_requests <= n_requests + 8'd1;
      if (!permitted) n_denied <= n_denied + 8'd1;
    end
  end
endmodule

Four management requests.

Three denials against one, and exactly one escalation.

The last row is the distinction the model exists to make. The permissive build allows a host's management path, and for a read that is a policy choice somebody can defend. It becomes an escalation when the operation carries privilege — a rebind moves capacity from one tenant to another, and a host that can issue one can take memory from a tenant it has no relationship with. privilege_err is gated on is_rebind for that reason: the origin alone does not establish harm, and the operation alone does not either.

Why the broken build is not a strawman. Exposing management operations over a host's own management path is genuinely convenient — it removes a round trip to an out-of-band fabric manager on every capacity change, which on a pool that rebalances frequently is a real latency saving. It is also how a host would do it if the device were not shared. The cost is that the trust boundary between tenant and administrator now runs through a path a tenant controls.

13. RTL 8 — A Quota Is A Ceiling, Not A Reservation

Address isolation says where a tenant may go. Capacity isolation says how much it may take, and the two failures look nothing alike.

// RTL 8 - a capacity quota per tenant, and what a device does when one asks for
// more than its share.
module capacity_quota #(parameter int NO_QUOTA = 0) (
  input  logic clk, rst_n,
  input  logic        request,
  input  logic [15:0] used_gb, quota_gb, want_gb, pool_free_gb,
  output logic        within_quota, pool_has_room, granted,
  output logic [15:0] headroom_gb,
  output logic [7:0]  n_requests, n_refused,
  output logic        quota_breach_err
);
  assign within_quota  = ((used_gb + want_gb) <= quota_gb);
  assign pool_has_room = (want_gb <= pool_free_gb);
  assign headroom_gb   = (used_gb >= quota_gb) ? 16'd0 : (quota_gb - used_gb);
  // Without a quota a tenant is limited only by the pool, so the first tenant to
  // ask can take everything.
  assign granted = (NO_QUOTA != 0) ? (request && pool_has_room)
                                   : (request && within_quota && pool_has_room);
  // Granting past a tenant's quota.
  assign quota_breach_err = granted && !within_quota;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_requests <= 8'd0; n_refused <= 8'd0;
    end else if (request) begin
      n_requests <= n_requests + 8'd1;
      if (!granted) n_refused <= n_refused + 8'd1;
    end
  end
endmodule

A 64 GB quota against a 256 GB pool, six requests.

Four refusals against two, and two quota breaches.

The two conditions are independent and both are required. Row five is outside the quota and outside the pool; row six is inside the quota and outside the pool. A device that checks only the quota grants row six and then fails the allocation somewhere deeper, where the error is much harder to attribute to a tenant. A device that checks only the pool is the unquota'd build, where the first tenant to ask takes everything.

Row four is the tenant already past its quota — 80 GB used against a 64 GB ceiling, which happens after a quota is lowered on a tenant that already had capacity. Headroom floors at zero rather than wrapping, and within_quota is false for a request of a single gigabyte. The tenant is not evicted; it simply cannot grow.

14. RTL 9 — Ownership And Capability Are Two Questions

The last property before assembly. Owning a region establishes that it is yours. It does not establish what you are allowed to do to it.

// RTL 9 - what a tenant may do to a region, as distinct from whether the region
// is theirs. Ownership and capability are two questions.
module capability_scope #(parameter int OWNER_MAY_ALL = 0) (
  input  logic clk, rst_n,
  input  logic       attempt,
  input  logic       is_owner,
  input  logic [1:0] operation,     // 0 read, 1 write, 2 remap, 3 release
  input  logic [3:0] granted_caps,
  output logic       has_cap, permitted,
  output logic [7:0] n_attempts, n_denied,
  output logic       overreach_err
);
  logic [3:0] needed;
  assign needed = 4'b0001 << operation;
  assign has_cap = |(granted_caps & needed);
  // Owning a region is not the same as being allowed to remap it. The permissive
  // build collapses the two questions into one.
  assign permitted = (OWNER_MAY_ALL != 0) ? is_owner : (is_owner && has_cap);
  // Performing an operation the tenant holds no capability for.
  assign overreach_err = attempt && permitted && !has_cap;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_attempts <= 8'd0; n_denied <= 8'd0;
    end else if (attempt) begin
      n_attempts <= n_attempts + 8'd1;
      if (!permitted) n_denied <= n_denied + 8'd1;
    end
  end
endmodule

The tenant owns the region and holds read and write capabilities (4'b0011), not remap or release.

Three denials against one, and two overreaches. The last row is the mirror: a tenant holding the read capability for a region it does not own is denied by both builds, because capability without ownership is not authority either. Both questions must be answered yes, and the two failures are separable.

Why this matters on a pool specifically. Remap and release are the operations that move capacity between tenants. A tenant that may release its own region can hand capacity back to the pool at a moment of its choosing — which is a legitimate operation and also a way to force a reallocation, which is the operation section 9 showed carries residue. Capabilities are how the two chapters connect: the residue path is only reachable by a tenant that holds release.

Figure 4 — Four gates, and a device that skips any one of them is a device that serves a request somebody was not entitled to make. The right-hand path is not an error path — on a healthy pool it is taken constantly, by tenants probing the edges of their own allocations.

15. RTL 10 — Isolation Assembled

Five boundaries. A tenant pair is isolated when every one of them holds.

// RTL 10 - isolation assembled: every boundary that must hold before two tenants
// may share one device.
module isolation_model #(parameter int ADDRESS_ONLY = 0) (
  input  logic clk, rst_n,
  input  logic       evaluate,
  input  logic       ranges_disjoint,   // no two regions overlap
  input  logic       device_enforces,   // the device checks, not just the host
  input  logic       faults_contained,  // one region's fault stays there
  input  logic       residue_scrubbed,  // released memory is cleared before reuse
  input  logic       structures_split,  // no shared queue leaks occupancy
  output logic       isolated,
  output logic [4:0] fail_mask,
  output logic [7:0] n_eval, n_isolated,
  output logic       false_isolation_err
);
  assign fail_mask[0] = ~ranges_disjoint;
  assign fail_mask[1] = ~device_enforces;
  assign fail_mask[2] = ~faults_contained;
  assign fail_mask[3] = ~residue_scrubbed;
  assign fail_mask[4] = ~structures_split;
  // The address-only build checks that the ranges do not overlap and calls that
  // isolation, which is the most common definition and the weakest.
  assign isolated = (ADDRESS_ONLY != 0) ? ranges_disjoint : (fail_mask == 5'd0);
  assign false_isolation_err = evaluate && isolated && (fail_mask != 5'd0);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_eval <= 8'd0; n_isolated <= 8'd0;
    end else if (evaluate) begin
      n_eval <= n_eval + 8'd1;
      if (isolated) n_isolated <= n_isolated + 8'd1;
    end
  end
endmodule

Five configurations.

One isolated against four, and three false claims.

That is the argument of the chapter in a single table. The address-only model — two hosts, non-overlapping ranges, therefore isolated — is the definition almost everybody uses, and it is right about exactly one of the five configurations. The third row is the worst of them: three boundaries down, a device with no enforcement, unscrubbed capacity and a shared queue, and the weak model still reports isolation because the arithmetic on the region bases happens to work out.

And the weak model only notices when the ranges themselves overlap, which is the failure a fabric manager is least likely to produce, because computing disjoint ranges is the one part of the problem that is pure arithmetic.

16. Quantitative Reasoning

Numbers from the models, all of them teaching values.

Enforcement. Eight accesses arrive, five are legitimate, three are not. The correct device blocks three; the host-side device blocks zero. Three escapes in eight requests is a 37.5% escape rate on a link where nothing was wrong — the failure is not on the wire and no link-level counter will show it.

Residue. A 1024-byte region, half-scrubbed. 512 bytes of the previous tenant's data reach the next one. Scale that to a realistic reallocation: a 16 GB region scrubbed at 20 GB/s takes 800 ms, which is why the shortcut is tempting and why an interrupted scrub is a partial one.

Reset blast radius. Two hosts, two regions each. A device-wide reset triggered by one host clears four regions of four — a 100% blast radius for a request that legitimately owns 50% of the device. On an eight-host pool the same policy gives one host the ability to reset seven others.

Shared structure. A 64-entry queue, A holding 30. B observes 40 instead of 10, and A's occupancy is a subtraction: 30 entries, 46% of depth, recovered exactly. Not statistically inferred — computed, from a number B is entitled to read.

Quotas. A 64 GB quota against a 256 GB pool. Four tenants at quota fill the pool exactly. Remove the quota and the first tenant to ask takes 256 GB of 256 — a 4x overshoot of its share, and the other three tenants receive a refusal from a pool that was sized for them.

The assembled model. Five boundaries, five configurations, one genuinely isolated. The address-only definition reports four. Its error rate is not a rounding difference; it is three false claims in five, and every one of them is a deployment somebody would have signed off.

17. Assertions

Every check is written as an explicit comparison against an exact value. Icarus Verilog 13.0 has no concurrent assertion support, so each check is a procedural comparison against 1'b1 inside the testbench, and every one of them is an equality — never a bound, never a relation.

Region overlap.

chk(oAe == 16'd100, "region A ends at 100");
chk(oOv == 1'b0,    "which do not overlap");
chk(oGr == 1'b1,    "so the allocation is granted");
chk(oE  == 1'b0,    "with no overlap granted");

Both adjacency directions are asserted, and the mirror case — a region ending exactly where another begins — is asserted separately from the case where one begins exactly where another ends. A half-open comparison that is correct on one side and inclusive on the other passes one and fails the other, so both must be driven.

chk(oBe == 16'd200, "B ends exactly where A begins");
chk(oOv == 1'b0,    "which is still not an overlap");

Device enforcement. The two conditions are asserted independently, because a model checking only one of them serves the other's violations.

chk(eIr == 1'b1, "address 50 is in range");
chk(eRo == 1'b0, "but requester 7 does not own it");
chk(ePm == 1'b0, "so it is blocked");
chk(hPm == 1'b1, "and served by the host-side build");

The range boundary is driven at both ends: addr == owner_base is inside, addr == owner_base + owner_size is not.

Fault containment. The critical assertion is the one that is not an error.

chk(cSr == 1'b1, "the fault is in the observer's own region");
chk(cOa == 1'b1, "so the observer is affected");
chk(cSp == 1'b0, "which is not a spread");

An observer affected by a fault in its own memory is containment working, not containment failing, and a checker that flags every affected observer would flag correct behaviour.

Residue. The same-owner reissue is asserted safe without a scrub.

chk(sDo == 1'b0, "the same owner is not a different one");
chk(sSr == 1'b1, "so it is safe to reissue unscrubbed");
chk(sRe == 1'b0, "and is not residue");

Reset scope. Both the mask and the victim are asserted, and the case where the disturbed host is the requesting host is asserted as not collateral.

chk(wOd == 1'b1, "the device-wide reset still clears the other regions");
chk(wCe == 1'b0, "but the same host asked, so it is not collateral");

Shared structure. The empty-A case is asserted to leak nothing, which is the case that distinguishes a channel carrying a secret from a channel carrying its absence.

chk(qOb == 8'd10, "an empty A makes the totals agree");
chk(qBs == 1'b0,  "so B sees no difference");
chk(qOe == 1'b0,  "and there is nothing to leak");

Management privilege. The non-rebind case over the permitted path is asserted as permitted and not an escalation, which is what separates origin from operation.

chk(kPm == 1'b1, "the permissive build permits the management path");
chk(kPe == 1'b0, "but a read is not a rebind, so it does not escalate");

Quotas. Both refusal reasons are driven separately, and the within-quota-but-pool-empty case is asserted refused by both builds.

chk(uWq == 1'b1, "32 GB is within the quota");
chk(uPh == 1'b0, "but the pool holds only 16 GB");
chk(uGr == 1'b0, "so the request is refused");

Capability. Ownership without capability and capability without ownership are asserted separately, both denied.

The assembled model. Every fail mask is asserted as an exact five-bit value, and the address-only build's false claims are counted rather than merely observed.

chk(iFm == 5'b11010, "three boundaries fail at once");
chk(jIs == 1'b1,     "and the address-only model still says isolated");
chk(fi_b == 3, "the address-only model did three times");

Totals: 211 checks across two testbenches, 96 on the front five models and 115 on the back five, all passing on the unmutated sources.

18. Mutation Testing

Thirty-eight mutations were injected, one at a time, into the ten models. A mutation that leaves the testbench passing is a survivor, and a survivor is a statement about the testbench or the design, never something to patch away.

38 injected, 38 killed. That number was reached after three survivors were diagnosed, and the diagnoses are the useful part of the section.

Survivor 1 — a stimulus gap. Changing a_base < b_end to a_base <= b_end survived. The testbench drove one adjacency — A ending exactly where B begins — and never the mirror, B ending exactly where A begins. With A's base at zero the mirror case is unreachable, because B would need to end at address zero with a non-zero size. Moving A to base 200 and B to [100,200) drives it, and the mutation dies. The gap was in the stimulus, not the design: the model was always right, and the testbench could not tell.

Survivor 2 — a provably equivalent mutation, and a redundant term. Removing a_occupancy != 8'd0 from observable_err survived every case. It survived because it is implied: b_sees_a is (a + b) != b, which in eight-bit arithmetic is true exactly when a is non-zero. The term was documentation written as logic. The resolution was to delete it and say so in a comment, and to replace the mutation with one that is not equivalent — reporting the leak from the topology alone, probe && !partitioned, which the empty-A case kills immediately.

Survivor 3 — a second stimulus gap. Dropping pool_has_room from the quota grant survived, because the only pool-exhaustion case in the testbench asked for 300 GB against a 64 GB quota — already outside the quota, so the pool check was never the deciding condition. A request of 32 GB within a 64 GB quota against a pool holding 16 GB makes the pool check the only thing standing between the tenant and a grant, and the mutation dies. A compound condition needs each half driven alone, and "outside both" does not exercise either.

19. Verification Strategy

What a testbench for a real isolation block must cover, beyond what these models reach.

Both sides of every boundary. Every range comparison in this chapter is half-open, and every one of them was checked at both ends and in both directions. The mutation survivor in section 18 exists because one of those four cases was missing, and it was missing in the direction that a base address of zero makes unreachable. A boundary test that only ever runs against region zero is a boundary test with half its cases quietly deleted.

Each half of every compound condition, driven alone. Enforcement is range and owner; granting is quota and pool; permission is ownership and capability. Six halves, and a stimulus that only ever violates both at once proves nothing about either.

The cases that are correct and look like failures. An observer affected by a fault in its own region. A host disturbed by its own reset. A reissue to the same owner with no scrub. Each of those trips a naive checker, and a testbench that does not assert them as correct will accept a design that refuses them.

The degenerate cases. A zero-size region. A zero-depth queue. A host owning no regions. A tenant already past a lowered quota. Every one of them is reachable in a real pool during a reconfiguration, and every one of them is where an unguarded expression divides by zero or wraps.

What a real block needs that these models do not have. Concurrency — two hosts issuing overlapping requests to the same decoder in the same cycle. Ordering — a rebind arriving between an access being decoded and being served. Persistence — a region whose ownership changed while a scrub was in flight. Each of those is where isolation actually fails in silicon, and none of them is visible in a combinational model.

20. Synthesis and Implementation Reality

The comparisons are cheap and the timing is not.

Range checks are subtractors on the critical path. addr >= base && addr < base + size is two 64-bit comparisons against a value that must be computed. On a real decoder the end address is registered rather than recomputed, because the adder in front of the comparator is the difference between hitting the frequency target and missing it.

The decoder is a lookup, not a pair of comparators. A device with sixteen logical devices does sixteen range comparisons in parallel and priority-encodes the result — which is sixteen adders and sixteen comparator pairs, all of them switching on every access. That is a real power number on a device serving hundreds of millions of requests per second.

The scrub is a memory-controller operation, not a decoder one. Scrubbing 16 GB is a background traffic pattern competing with tenant traffic for the same DRAM banks, and the reallocation is not complete until it finishes. This is why the no-scrub shortcut gets proposed: the correct behaviour holds capacity out of the pool for the better part of a second per reallocation.

The reset scope is structural. Making a reset per-logical-device rather than per-device is not a policy register; it is separate reset trees, separate state to clear, and separate assertions that nothing crosses between them. It is the kind of change that cannot be made after tapeout, and it is the reason section 10's failure mode persists in shipped silicon.

The shared queue is an area decision. Partitioning a 64-entry queue into two 32-entry queues costs nothing in flops and everything in utilisation — each tenant now stalls at 32 entries while the other half sits empty. The observability in section 12 is bought with real throughput, which is why the shared structure keeps being chosen.

21. Silicon Observability

What a device must expose for an operator to know isolation is holding.

The attribution is the hard part. A device-wide counter of blocked accesses tells an operator that something is wrong and nothing about which tenant. Every counter in that table is per logical device for that reason, and a device that aggregates them has built telemetry that cannot answer the only question an operator has.

22. Debug Lab

Symptom. A tenant on a pooled device reports memory corruption. Its own writes read back correctly; a region it has not touched in an hour returns data it does not recognise. The link is clean — no CRC errors, no retries, no integrity failures. Both hosts authenticated successfully at attach.

What the link counters say. Nothing. Every counter in 19.2 is zero, because nothing on the wire was wrong.

Step 1 — is it an overlap? Read back the decoder configuration for both logical devices and compare the ranges. If they overlap, the fabric manager produced a bad layout and section 5 is the chapter. In this case they do not: the ranges are disjoint and the arithmetic is correct.

Step 2 — is it an escape? Check the blocked-access counter for both logical devices. It reads zero for both — which, given that a tenant is seeing another's data, is the finding. A zero blocked-access count on a device that is demonstrably serving cross-tenant requests means the device is not checking, which is section 6.

Step 3 — or is it residue? Before concluding, check whether the corrupted region was recently reallocated. The fabric-manager log shows it was released by another tenant nine minutes ago and rebound to this one. Compare bytes released against bytes scrubbed: 16 GB released, 11.2 GB scrubbed, and the reallocation completed anyway.

The finding. Not an escape at all. The scrub was still running when the region was rebound, and the tenant is reading 4.8 GB of the previous tenant's memory through a decoder that is working perfectly. Every range check passed. Every access was permitted. The isolation failure is in the ordering between the scrub and the rebind, which no address check can see.

The fix. The rebind must not complete until scrubbed_bytes >= released_bytes, and that condition must be a gate on the reallocation rather than a counter somebody reads afterwards. The cost is the 800 ms from section 16, paid on every cross-tenant reallocation.

What made this hard. Three of the five boundaries were intact, the two link chapters had nothing to say, and the symptom pointed at the decoder — the one part of the system that was behaving correctly.

23. Design Review

Ten questions for a pooled-device design review.

1. Does the device check, or does it trust the requester? If the answer involves the word "the host already validated", the answer is trust. Section 6.

2. Are the range comparisons half-open, and is that tested in both directions? Section 5, and the mutation survivor in section 18.

3. What is the blast radius of an uncorrectable error? If the answer is "the device", section 8 applies and it is a design decision, not a protocol requirement.

4. What is the blast radius of a link reset? If nobody knows, it is device-wide. Section 10.

5. Is capacity scrubbed before reissue, and is the rebind gated on the scrub completing? The second half is the one that fails. Section 22.

6. Which structures are shared between logical devices? Queues, buffers, retry state, credit pools. Each one is an observation channel. Section 12.

7. Can a host reach a management operation over its own path? And if so, which operations. Section 13.

8. Are quotas enforced separately from pool capacity? Two conditions, two refusal reasons, two counters. Section 13.

9. Is every counter per logical device? Section 21, and the answer decides whether an operator can debug anything.

10. Which of the five boundaries does the team believe isolation means? Ask everyone separately. Section 15 exists because the answers differ.

24. How This Appears In Real Engineering

A hyperscaler qualifying a pooled device does not accept a datasheet claim of isolation. It builds a tenant that deliberately misbehaves — accesses outside its range, releases and immediately re-requests capacity, resets its link at high frequency, floods a shared queue — and instruments a second tenant to detect any effect. Every one of the five boundaries in this chapter is a separate test in that suite.

A device team implementing multi-logical-device support discovers that the decoder was the easy part. The error-handling path, the reset tree and the scrub engine were all written when the device had one host, and each of them encodes the assumption that device scope and tenant scope are the same thing. That assumption is in structural RTL, not in a configuration register.

A fabric-manager team owns the overlap check in section 5 and, in most deployments, is the only place it happens. That is the argument for the device checking too: the fabric manager is software, it is updated more often than the device, and a device that trusts it has no defence against a regression in it.

A security review of a pooled deployment will ask for the residue behaviour first, because it is the boundary that leaks data with no attacker sophistication at all — the previous tenant's bytes are simply present. It will ask about shared structures last, and get the vaguest answers, because that boundary is the one nobody owns.

25. Common Misconceptions

"The ranges do not overlap, so the tenants are isolated." One of five boundaries. Section 15 shows the address-only definition calling four of five configurations isolated, including one with three boundaries down.

"The host validates the address before sending it." The host that would violate the boundary is the host being asked to enforce it. Section 6.

"The links are authenticated and encrypted, so the device is secure." Both are properties of a link between two parties. Every failure in this chapter happens on a device where every link is perfect. Figure 1.

"An uncorrectable error is a device-level event." It is on a single-host device. On a pool it is a tenant-level event that a design decision escalates. Section 8.

"The new tenant will overwrite the memory before reading it." Some of it, eventually, in an order it chooses. The bytes it does not write first are readable. Section 9.

"A reset only affects the host that issued it." Only if somebody built separate reset trees. Section 10.

"Sharing a queue is a performance decision, not a security one." It is both, and section 12 recovers a tenant's exact occupancy from a number the other tenant is entitled to read.

"A quota guarantees a tenant its capacity." A quota is a ceiling. Section 13's sixth row is a tenant inside its quota being refused by an empty pool.

"Owning the region means being allowed to remap it." Two questions. Section 14, and the permissive build overreaches on two of five attempts.

"Scrubbing is fast." 16 GB at 20 GB/s is 800 ms of a memory controller that also has tenants to serve. Section 20.

26. Interview Reasoning

Q. Two hosts share a pooled CXL device. Their memory ranges do not overlap. Are they isolated?

No, and the question is testing whether the candidate stops at the arithmetic. Disjoint ranges is one of five boundaries. Ask in turn: does the device enforce them, or only the host? Does one tenant's uncorrectable error take the other down? Does a link reset from one clear the other's regions? Is capacity scrubbed between owners? Do they share a queue? A candidate who names two or three of those is thinking about the right system.

Q. A tenant reads another tenant's data. The decoder configuration is correct and no access was blocked. Where do you look?

Residue, before anything else. A correct decoder plus a cross-tenant read plus a zero block count is the signature of memory reallocated without a completed scrub — the reads are legitimate, the data is stale. Section 22 walks it. The second candidate is a decoder that is configured correctly and not consulted, which the same zero block count also fits, and the two are distinguished by whether the region was recently rebound.

Q. Why is a shared queue a security problem when it carries no tenant data?

Because occupancy is data. A tenant observing its own queue depth on a shared structure observes the total, and its own contribution is known, so the other tenant's is a subtraction. Section 12 recovers 30 entries exactly. The follow-up worth asking is what partitioning costs: each tenant now stalls at half the depth while the other half is idle, which is why the shared structure keeps winning the argument.

Q. A host requests a link reset. What must the device clear?

Only the regions belonging to that host's logical device. The follow-up is what makes this hard: reset scope is structural — separate reset trees and separate state — not a policy register, so a device that got it wrong got it wrong at tapeout.

Q. Where does the overlap check belong, the fabric manager or the device?

Both, and the reason is the interesting half. The fabric manager computes the layout, so it must check. The device must check too because the fabric manager is software that changes more often than the device, and a device that trusts it has no defence against a regression. Section 5's broken build is exactly the trust argument, stated by someone reasonable.

Q. What does a quota guarantee?

A ceiling, not a floor. A tenant inside its quota can still be refused by a pool that is full, and that refusal must be distinguishable in telemetry from a quota refusal — one means the tenant is at its limit and the other means the pool is oversubscribed, and they call for opposite responses.

27. Exercises

1. Extend RTL 1 to sixteen regions and return the identity of the region an overlapping allocation collides with, not merely that it collides. Assert the identity, not the flag.

2. Modify RTL 2 so the range end is registered rather than recomputed, and add a test that the registered end is correct on the cycle after a rebind. This is section 20's timing argument as a functional bug.

3. Add a partial-containment mode to RTL 3 in which a fault degrades every tenant's bandwidth without taking any of them down. Decide whether that is contained, and write the assertion that encodes your decision.

4. Gate the reallocation in RTL 4 on scrub completion rather than reporting residue afterwards, and add a counter for reallocations delayed by an incomplete scrub. Assert the delay count exactly.

5. Extend RTL 5 to eight hosts and report the blast radius as a fraction of the device. Verify that a host owning no regions has a blast radius of zero in the correct build and one in the device-wide build.

6. Add a third tenant to RTL 6 and show that B can no longer recover A's occupancy exactly. Quantify what B can still recover, and assert it.

7. Extend RTL 7 so that management operations carry a capability mask rather than a single is_rebind flag, and show which operations a host may safely be given over its own path.

8. Add oversubscription to RTL 8: quotas summing to more than the pool. Assert the exact request at which the pool is exhausted while every tenant is inside its quota.

9. Combine RTL 9 and RTL 4 — make the release capability the only path to a reallocation, and show that a tenant without it cannot reach the residue path at all.

10. Add a sixth boundary to RTL 10 of your choosing, and justify it against the five. If it is implied by one of the existing five, say which — and if it is not, explain what configuration it catches that the current mask calls isolated.

28. Summary

Authentication established who is on the link and secure communication protected what crosses it. Both stop at the switch.

Disjoint ranges are one boundary of five, and the address-only definition — the one almost everybody uses — called four of five configurations isolated when only one was.

A check the host performs is not enforcement. Three escapes in eight accesses, on a link where nothing was wrong and no link counter moved.

Half-open comparisons need both directions tested. A mutation survived until the mirror adjacency was driven, and it survived because region zero makes that case unreachable.

A fault has a radius, and escalating it device-wide converts one tenant's bad DRAM into an outage for every host on the port.

Released capacity carries the previous tenant's bytes. A half-finished scrub leaked 512 of 1024 bytes through a decoder behaving perfectly — and the real failure is the rebind not waiting for the scrub.

Reset has a blast radius too: one host's reset clearing four regions of four, using a mechanism nobody files under security.

A shared structure is an observable one. B recovered A's occupancy exactly — 30 entries, 46% of depth — from a number it was entitled to read, with the address path perfect throughout.

The fabric manager is a separate trust domain, and a host reaching it can move capacity belonging to tenants it has no relationship with.

A quota is a ceiling, not a reservation, and pool exhaustion and quota exhaustion are different refusals that call for opposite responses.

Ownership and capability are two questions, and collapsing them let a tenant remap and release regions it merely owned.

Isolation is all five, or it is a word. One configuration of five was genuinely isolated, and every one of the three false claims is a deployment somebody would have signed off.

19.4 — Multi-Tenant Environments takes these five boundaries and asks what changes when the tenants number in the hundreds and the policy that places them is itself a system.