AI Accelerator Packages

26.1 read a server partition; 26.2 read a client one. This chapter reads the packages where the partition is driven by something neither of them faced: sustained bandwidth on every edge at once.

1. The One-Sentence Model

An AI accelerator package is a resource graph — compute, local memory, die-to-die transport, HBM, the scheduler and synchronisation are all nodes, and achieved throughput is set by the minimum sustainable service along the path a phase actually uses.

Peak compute is one node in that graph. A package can double its FLOPS and gain nothing, because the constraint was never at the compute node — and §9 shows the constraint moving as die count grows, which is why an architecture that was balanced at two dies is not balanced at eight.

And the second idea is the one this chapter is built around: in a multi-die accelerator, a single degraded participant sets the phase time for every other die (§17–§18). That is a performance failure with no corrupted data, no error bit, and no protocol violation — which makes it invisible to every functional check in the system.

2. What This Chapter Owns

22.3 owned the opportunity — why AI workloads suit chiplets and what UCIe evidence exists. This chapter owns the evidence and the engineering lesson:

Two documented packages, read as graphs (§5–§7), from vendor architecture documentation rather than characterised.

The quantitative model, worked end to end (§8–§10) — including why scaling compute stops helping.

What AMD's partitioning modes reveal (§11) — a shipping product that lets software choose which resource graph it sees, which is the most instructive documented fact in this chapter.

And the scheduler consequence (§12–§18): even distribution is the wrong default the moment participants can differ.

3. Sourcing

4. Claim-vs-Evidence

Five readings.

Rows 1–9 are Class A — the strongest evidence in Module 26 so far, stronger even than 26.1's, because AMD documents partitioning behaviour that software actually depends on.

Row 5 is the row to handle carefully, and it is the chapter's discipline in miniature. "Peak memory bandwidth 5.3 TB/s" is a peak. It proves the memory system's ceiling; it proves nothing about what any workload achieves, and §7–§8 exist because the gap between those two numbers is where all the engineering is.

Row 7 is the most interesting single sentence I found. "The number of memory partitions must be less than or equal to the number of compute partitions" is a documented architectural invariant with a real reason behind it (§11) — the kind of constraint that reveals structure.

Rows 10–13 are Class B and read differently. A vendor engineering blog is authoritative about its own product's architecture and is written to persuade as well as inform, so I quote its architectural statements and do not build arguments on its performance framing.

And the last two rows stay empty. Both packages document proprietary die-to-die interconnects, and §22 argues that is exactly why they are useful here.

5. Two Documented Packages

Four readings, and the last is the chapter's most useful comparison.

Both are multi-die, and their compute die counts differ by 4×. That difference is not a quality judgement — it is a different answer to how finely should compute be cut, and §6 is the trade.

The cache rows describe genuinely different structures. MI300X documents an L2 per XCD that "coalesce[s] all memory traffic for the die"; the Blackwell post documents a shared L2 that is "fully coherent" across the two dies. Those imply different things about what the die-to-die edge carries (§7).

I do not compare 5.3 TB/s with 8 TB/s. They are different memory generations, different capacities, different products and — critically — both are peak figures. Placing them side by side as a ranking would be the exact numeric error 23.1 §13's comparison conditions forbid. They appear in one table because each is evidence about its own package's ceiling.

And row 7 is the deepest architectural difference between them. One package exposes its partition to software and lets you choose the granularity; the other hides it and presents one device. §11 is why that is a real design decision rather than a marketing one, and 26.5 makes it a central theme.

6. Why Cut Compute at All

The scorecard from 25.7 §6, applied to an accelerator.

Three readings.

Row 1 is the reason that admits no trade. NVIDIA's post describes two reticle-sized dies; if you want more than a reticle of compute, you are building a multi-die package or you are not building it. Everything else on this list is an optimisation; this one is a physical wall.

Row 3 is quietly the most economically important, and it is documented rather than inferred: 2 disabled CUs per XCD "for yield management" means the architecture plans to ship dies with defects. Harvest is not damage control — it is a designed-in property of the partition, and it works better with more, smaller dies.

And rows 5–6 are why this is a graph problem rather than a division problem. Cutting compute into n pieces does not divide the work into n independent pieces: it creates edges that must carry what wires used to carry, and a synchronisation cost that does not shrink (§9).

7. The Bandwidth Graph

Each edge needs six properties before it can be reasoned about, and a headline number gives you one of them.

Three readings.

Row 4 is the one that quietly invalidates most comparisons. NVIDIA documents NV-HBI at 10 TB/s; the cited post does not state whether that is aggregate or per direction. A workload whose traffic is asymmetric cares enormously, and I therefore use the figure as "a documented D2D edge exists with a stated rate" and not as an input to arithmetic (§4 row 10).

Row 3 is where real systems lose bandwidth without anything being broken. An edge with ample bandwidth and insufficient outstanding capacity idles — 21.5 §26's bandwidth-delay product, and 26.4 §14 works it through with numbers.

And row 6 is what makes the coherence row of §5 architecturally loud. A fully coherent shared L2 across a die boundary means the D2D edge carries traffic that a compute unit stalls on — not bulk transfers that can be prefetched. A dependency edge and a bandwidth edge fail in completely different ways (26.1 §10).

8. Worked Example — the Minimum Sustainable Service

Setup. Four compute dies. One phase of work = 1200 units. Per die per cycle:

Step 1 — compute node. 4 dies × 4 = 16 units/cycle.

Step 2 — HBM node. 10 units/cycle, package-wide, and it does not scale with die count.

Step 3 — D2D node. Half the work crosses, so to sustain R units/cycle the package needs 0.5 × R crossing service. Available crossing service is 4 × 1.5 = 6 units/cycle, which supports R = 12 units/cycle.

Step 4 — the minimum. min(16, 10, 12) = 10 units/cycle. HBM binds.

Step 5 — phase time. 1200 / 10 = 120 cycles of streaming, plus the 200-cycle barrier = 320 cycles.

Step 6 — the useful ratio. Compute could have done 1200 units in 1200/16 = 75 cycles. Achieved phase time is 320. Compute utilisation ≈ 23 % — and the barrier alone accounts for 200 of the 320 cycles.

Three readings, and they are the reason to do the arithmetic rather than assert the principle.

The binding node is HBM, so adding compute changes nothing. Doubling every die's compute to 8 units/cycle makes step 1 read 32 and leaves min(32, 10, 12) = 10. Phase time is still 320 cycles. This is the whole of §1 in one substitution.

The barrier is 62 % of the phase, and it is fixed. Any change that shortens only the streaming part is attacking 120 of 320 cycles, which caps the best possible speed-up at about 1.6× no matter how good the change is — Amdahl's structure arriving through packaging rather than through software.

And the D2D node is second, not last. At R = 12 it is closer to binding than the compute node is. A modest HBM improvement makes D2D the constraint, which is §10: fixing the bottleneck does not remove the bottleneck, it moves it.

9. Why Compute Scaling Stops Working

Same illustrative setup, varying die count. Fixed 1200-unit phase, 200-cycle barrier, HBM service 10 units/cycle package-wide, per-die D2D 1.5 units/cycle with 50 % of work crossing.

Four readings.

Two dies gives 1.25×, not 2× — because the barrier was already 40 % of the single-die phase, and doubling compute cannot touch it.

Four dies gives 1.56×, and the constraint has moved to HBM. Between 2 and 4 dies the binding node changed from D2D to HBM without anyone changing the memory system.

Eight dies gives exactly nothing. Compute is 32, D2D supports 24, HBM still serves 10. The package doubled its most expensive resource for zero throughput — and every functional test still passes, which is precisely why this failure survives to silicon.

And the useful conclusion is a design rule. Scaling one node of a resource graph is only an improvement while that node is the minimum. 25.7 §10 argued partitions are decided by dependency rather than bandwidth; this table is the same argument in throughput terms — you must know which node binds at the die count you are building, not at the one you prototyped.

10. Bottleneck Migration

Two readings.

Row 4 is the one an architecture review usually reaches last and should reach first. At four dies the barrier is 200 of 320 cycles. Halving it beats any bandwidth improvement in the table — and it is a synchronisation and scheduling problem (§17), not an interconnect problem.

And rows 2 and 3 are why single-resource roadmaps disappoint. Improving HBM alone converts an HBM bottleneck into a D2D bottleneck and delivers a fraction of the projected gain. A resource graph is improved in matched steps or not at all.

11. What Partitioning Modes Reveal

AMD's partitioning documentation is the most instructive material in this chapter, because it shows a shipping product letting software choose which resource graph it sees.

And one documented invariant: "the number of memory partitions must be less than or equal to the number of compute partitions."

Four readings, and this section is why the chapter uses MI300X rather than a more famous name.

The invariant is real structure, not a software limitation. A memory partition must be reachable and owned by some compute partition. More memory partitions than compute partitions would leave a pool with no owner — a resource published with nobody responsible for it, which is 26.2 §10's ownership problem in a different domain. The constraint falls out of ownership, and that is why it is a hard rule rather than a tuning knob.

NPS4 requiring CPX is the same argument at its limit. Four memory pools need at least four compute partitions; the finest memory split is only legal with the finest compute split.

Locality becomes visible exactly when you ask for it. The documentation notes that latency "is minimized because it remains on the same AID" — so partitioning is not just carving up capacity, it is choosing whether software must reason about placement. SPX hides the graph; CPX+NPS4 hands it to you.

And the trade is a genuine one with no default answer. One big device is simplest, gets full bandwidth to a single consumer, and hides the topology — but a small job cannot use it efficiently and one straggler affects everything. Eight small devices give isolation and explicit locality, at the cost of software now owning placement. The same silicon serves both, which is the strongest possible evidence that the resource graph is a choice rather than a fixed property of the package.

12. Illustrative RTL — a Work Distributor

// ILLUSTRATIVE ONLY. A generic accelerator work distributor. The point of the
// structure is the SEPARATION: semantic job identity is owned by the scheduler
// and is stable for the job's whole life; the target and the route epoch are
// placement facts that must never be re-derived from a transport event.
typedef struct packed {
  logic [JOB_W-1:0]   job_id;        // SEMANTIC — stable for the job's life
  logic [GEN_W-1:0]   generation;    // which use of that id (25.4)
  logic [TGT_W-1:0]   target;        // which compute die this was dispatched to
  logic [EPOCH_W-1:0] route_epoch;   // the mapping it was dispatched under
  logic [WORK_W-1:0]  work_units;    // how much work — used for service rate
  logic               live;
} job_entry_t;
 
job_entry_t  jobs_q [N_JOB];
logic [15:0] outstanding_q [N_TGT];
logic        target_enabled_q [N_TGT];
 
always_ff @(posedge clk or negedge rst_n) begin
  if (!rst_n) begin
    for (int j = 0; j < N_JOB; j++) jobs_q[j].live <= 1'b0;
    for (int t = 0; t < N_TGT; t++) outstanding_q[t] <= '0;
  end else begin
    // DISPATCH on ACCEPTANCE (valid && ready), never on offering. Offering a
    // job to a full target and counting it is 21.5 §11's classic overcount.
    if (dispatch_fire) begin
      jobs_q[alloc_idx].job_id      <= next_job_id;
      jobs_q[alloc_idx].generation  <= next_gen;
      jobs_q[alloc_idx].target      <= sel_target;
      jobs_q[alloc_idx].route_epoch <= route_epoch_q;
      jobs_q[alloc_idx].work_units  <= sel_work;
      jobs_q[alloc_idx].live        <= 1'b1;
    end
 
    // RETIRE only on a completion that matches a LIVE entry AND its generation.
    // A stale completion from a previous use of this id must not retire it.
    if (cmpl_fire && jobs_q[cmpl_idx].live &&
        (jobs_q[cmpl_idx].generation == cmpl_gen))
      jobs_q[cmpl_idx].live <= 1'b0;
 
    // One signed next-state expression per target, so a dispatch and a
    // completion in the same cycle net correctly instead of losing one.
    for (int t = 0; t < N_TGT; t++)
      outstanding_q[t] <= outstanding_q[t]
                        + 16'(dispatch_fire && (sel_target  == t[TGT_W-1:0]))
                        - 16'(cmpl_fire && jobs_q[cmpl_idx].live
                              && (jobs_q[cmpl_idx].target == t[TGT_W-1:0]));
  end
end

Architecture. A scheduler needs to know, per target, how much is outstanding, and per job, which target and which mapping it was dispatched under — that last field is what makes a route change safe (§20).

State. One entry per job slot plus one outstanding count per target. route_epoch is the field most designs omit and then need.

Event. Allocation on dispatch_fire (valid && ready); retirement on a completion that matches both a live entry and its generation. Two guards, because they catch two different bugs: live catches completions for a retired job, generation catches completions for a previous use of the same slot (25.8 §14).

Contract. The consumer assumes job_id is stable for the job's whole life regardless of how many transport attempts occur underneath. The scheduler must never re-derive identity from a transport event (25.5 §11).

Failure. Drop the generation check and a late completion from a recycled slot retires a live job. The scheduler then believes work finished that is still running — and the barrier releases early, which is a correctness bug produced by a performance structure.

DV/debug. outstanding_q per target is the single most useful number in the design. All targets non-zero and one much larger is the degradation signature of §18, visible without a single waveform.

13. Wrong RTL — Even Distribution With Unequal Targets

// WRONG. ILLUSTRATIVE. Plain round-robin. Written by someone whose previous
// design had one compute engine, where "next target" was a correct and complete
// scheduling policy. It is still correct — it is just no longer sufficient.
always_ff @(posedge clk or negedge rst_n) begin
  if (!rst_n) begin
    rr_ptr_q <= '0;
  end else if (dispatch_fire) begin
    // BUG 1: selection ignores how much is already outstanding at the target.
    // BUG 2: selection ignores how fast the target is actually retiring work.
    // BUG 3: a disabled or degraded target is still in the rotation.
    rr_ptr_q <= (rr_ptr_q == N_TGT-1) ? '0 : rr_ptr_q + 1'b1;
  end
end
assign sel_target = rr_ptr_q;

Architecture. One pointer. No feedback from the targets at all — the defining property, and the bug.

State. rr_ptr_q. The missing state is every target's service rate.

Event. Advances on dispatch, unconditionally.

Contract. It silently assumes all targets are identical and remain identical — which was true on a single-die part and is not true the moment a package has four dies that can differ in clock, thermal headroom, link health or harvested unit count (§6 row 3: MI300X documents that dies ship with units disabled, so "identical" is not even true at time zero).

Failure — the timeline. Four dies, 1200-unit phase, 300 units each. Die 2's effective service degrades to half.

Root cause. Work was distributed by count when the resource being shared is time. An even split of units is only an even split of time if service rates are equal — and nothing in the design checks that they are.

And the reason it survives every test is that it is functionally perfect. Every job completes, exactly once, with correct data. There is no assertion to write against "correct but slow" unless someone measures service rate, which is §15.

DV/debug. The signature is unmistakable once you look at the right counter: outstanding_q near zero for three targets and large for one, with the barrier pending. Package throughput halves while every functional check passes — and if you only watch aggregate utilisation, you see 25 % and no reason for it.

14. Corrected — Service-Rate-Aware Distribution

// CORRECT. ILLUSTRATIVE (§12). Choose the target expected to FINISH soonest,
// not the one whose turn it is. The estimator is §15; this is the policy that
// consumes it. Ties break round-robin so behaviour stays deterministic.
logic [31:0] est_drain_cyc [N_TGT];
 
always_comb begin
  // Expected drain = outstanding work / measured service rate. Guard the
  // divide: a target with no measured rate is treated as MAXIMALLY loaded
  // rather than infinitely fast — the safe direction for an unknown.
  for (int t = 0; t < N_TGT; t++) begin
    if (!target_enabled_q[t] || (svc_rate_q[t] == '0))
      est_drain_cyc[t] = 32'hFFFF_FFFF;
    else
      est_drain_cyc[t] = outstanding_work_q[t] / svc_rate_q[t];
  end
end
 
always_comb begin
  sel_target = '0;
  for (int t = 1; t < N_TGT; t++)
    if (est_drain_cyc[t] < est_drain_cyc[sel_target]) sel_target = t[TGT_W-1:0];
end
 
// MANDATORY. English: work is never dispatched to a target that is not enabled.
// Catches the degradation case where a target is taken out of service but the
// scheduler's rotation was not told.
a_no_dispatch_to_disabled: assert property (
  @(posedge clk) disable iff (!rst_n)
    dispatch_fire |-> target_enabled_q[sel_target]
);
 
// MANDATORY. English: while all targets are enabled and healthy, no target's
// outstanding work exceeds the minimum by more than the imbalance budget.
// This is the property §13's round-robin cannot satisfy under degradation.
a_bounded_imbalance: assert property (
  @(posedge clk) disable iff (!rst_n || any_target_degraded)
    (max_outstanding_work - min_outstanding_work) <= IMBALANCE_BUDGET
);

Architecture. Selection by expected completion time rather than by turn. The policy is small; the measurement it depends on is the real work (§15).

State. A per-target drain estimate, recomputed combinationally from state that §15 maintains.

Event. Evaluated at each dispatch. Nothing here is triggered by a transport event — service rate is a property of a target, not of a link.

Contract. svc_rate_q == 0 must mean "unknown" and be treated as maximally loaded. Treating an unmeasured target as infinitely fast sends the whole phase to the one target you know least about — a plausible and very destructive sign error.

Failure. The residual risk is estimator lag: if the rate window (§15) is long relative to a phase, the scheduler balances against stale information and can oscillate. The estimate must be faster than the thing it is estimating, and a phase-length window is the wrong choice.

DV/debug. a_bounded_imbalance is disabled while a target is degraded, because imbalance is then the correct behaviour. An assertion that fires when the design is doing the right thing gets waived, and a waived assertion protects nothing (21.6 §29).

15. Illustrative RTL — the Service-Rate Estimator

// ILLUSTRATIVE (§12). Measurement is the load-bearing part. Counters are
// snapshotted per epoch so a rate is computed over a BOUNDED window rather than
// over all time — an all-time average cannot see a target that degraded a
// millisecond ago, which is the only case that matters.
localparam int RATE_EPOCH_CYC = 4096;
 
logic [31:0] accepted_work_q  [N_TGT];   // work ACCEPTED (valid && ready)
logic [31:0] completed_work_q [N_TGT];   // work COMPLETED and RETIRED
logic [31:0] active_cyc_q     [N_TGT];   // cycles with work outstanding
logic [31:0] stall_cyc_q      [N_TGT];   // outstanding, but nothing retiring
logic [31:0] svc_rate_q       [N_TGT];   // completed work per 1k active cycles
logic [15:0] epoch_cyc_q;
 
function automatic logic [31:0] sat_add32(logic [31:0] a, logic [31:0] b);
  sat_add32 = ((a + b) < a) ? 32'hFFFF_FFFF : (a + b);   // saturate, never wrap
endfunction
 
always_ff @(posedge clk or negedge rst_n) begin
  if (!rst_n) begin
    epoch_cyc_q <= '0;
    for (int t = 0; t < N_TGT; t++) begin
      accepted_work_q[t]  <= '0; completed_work_q[t] <= '0;
      active_cyc_q[t]     <= '0; stall_cyc_q[t]      <= '0;
      svc_rate_q[t]       <= '0;
    end
  end else begin
    for (int t = 0; t < N_TGT; t++) begin
      if (dispatch_fire && (sel_target == t[TGT_W-1:0]))
        accepted_work_q[t] <= sat_add32(accepted_work_q[t], 32'(sel_work));
 
      // COMPLETED counts only RETIRED semantic work. A retried transport
      // attempt is not work (§16) and must never appear here.
      if (retire_fire && (jobs_q[cmpl_idx].target == t[TGT_W-1:0]))
        completed_work_q[t] <= sat_add32(completed_work_q[t],
                                         32'(jobs_q[cmpl_idx].work_units));
 
      if (outstanding_q[t] != '0) begin
        active_cyc_q[t] <= sat_add32(active_cyc_q[t], 32'd1);
        if (!retire_any[t]) stall_cyc_q[t] <= sat_add32(stall_cyc_q[t], 32'd1);
      end
    end
 
    // Epoch boundary: publish a rate over the window, then clear the window.
    if (epoch_cyc_q == RATE_EPOCH_CYC-1) begin
      epoch_cyc_q <= '0;
      for (int t = 0; t < N_TGT; t++) begin
        svc_rate_q[t] <= (active_cyc_q[t] == '0) ? '0
                       : ((completed_work_q[t] * 32'd1000) / active_cyc_q[t]);
        completed_work_q[t] <= '0;
        active_cyc_q[t]     <= '0;
      end
    end else begin
      epoch_cyc_q <= epoch_cyc_q + 16'd1;
    end
  end
end

Architecture. Five counters per target and a windowed publish. The window is the design decision — everything else is bookkeeping.

State. Per-target accumulators plus a published rate that survives the window clear, so consumers always read a complete measurement rather than a partial one.

Event. accepted_work increments on acceptance; completed_work on semantic retirement. These are deliberately different events — the gap between them is the outstanding work (21.5 §11).

Contract. svc_rate is defined per active cycle, not per elapsed cycle. A target with no work is not slow; it is idle. Dividing by elapsed time makes an idle target look degraded and sends it even less work — a self-reinforcing error that looks exactly like a real fault.

Failure. Two realistic mistakes. An all-time average cannot see recent degradation, because thousands of good epochs drown one bad one. And a window shorter than a job's latency measures noise, and the scheduler chases it.

DV/debug. stall_cyc versus active_cyc is the discriminator that separates "this target is slow" from "this target is starved". High active with high stall is a target waiting on something downstream — usually memory, which is 26.4's subject — while high active with low stall is genuinely saturated compute. Those need opposite responses, and one counter tells them apart.

16. Retry Traffic Is Not Useful Work

A physical attempt is not a unit of AI work, and a performance counter that conflates them reports a busy link as a productive one (21.5 §14).

Three readings.

A link at 95 % utilisation and falling throughput is the classic signature, and the only way to see it is to have counted the second row separately. If retries are counted as work, degradation makes the link look better — the worst possible property in a diagnostic.

§15 encodes this in one line: completed_work increments on retirement, not on transport success. A transport-level completion is not a semantic completion (25.3 §14).

And the scheduler consequence is severe. A service-rate estimator fed retry bytes will conclude that a degrading target is speeding up and send it more work — turning a mild degradation into the §13 collapse, with the adaptive mechanism actively making it worse.

17. Synchronisation

Multi-die phases usually end in a barrier: all dies complete, then the next phase begins. That structure makes the slowest participant the phase's clock.

Three readings.

Row 1 is the sentence to keep. Averages are the wrong statistic for a barrier. A package whose dies average 95 % of nominal rate but where one runs at 50 % performs like a package of 50 % dies — and an average-based dashboard reports it as healthy.

Row 3 is why this gets worse exactly as you scale. Splitting a phase across more dies shrinks per-die work while leaving the barrier alone. §9's table shows the fixed 200 cycles going from 40 % of the phase at one die to 62 % at four.

And row 4 is the cost that turns a local problem into a package problem. Idle time on three dies is not merely "unused" — it is capacity that was purchased, powered and cooled, and §18 is what it looks like when the cause is a fault nobody detected.

18. Failure Case — One Die Degrades and Nothing Reports It

The scenario that motivates every counter in this chapter. No data is corrupted, no protocol is violated, and package throughput halves.

Four readings — this is the diagnosis path, and it is short if the counters exist.

Step 1 is the trap. The link is up; it trained; it is in ACTIVE. ACTIVE proves the link trained and nothing about the rate at which it moves useful work (25.9 §5). An engineer who checks link state concludes the interconnect is fine and looks at software.

Step 4 is the first divergence, and it is one register read. Per-target outstanding_q growing on exactly one target while the others drain localises this to one die immediately (25.9 §12).

Step 2 versus step 3 is the discriminator between two very different bugs. A falling svc_rate with rising stall_cyc on one target is a degraded participant. A falling rate on all targets with a full upstream queue is a shared-resource problem — HBM or D2D, which is 26.4 §16's subject. The counters distinguish them; a throughput number alone never can.

And the fix is architectural rather than a repair. §14's service-rate-aware distribution absorbs a degraded participant instead of being destroyed by it: die 2 at half rate receives roughly half the work, everyone finishes together, and the package loses about 12 % rather than 50 %. The degradation is not prevented — it is prevented from being amplified, which is the achievable goal.

19. Where HBM Enters

§8 made HBM the binding node, and §15's stall_cyc diagnostic points downstream. Both hand off to the next chapter rather than reteaching memory here.

And one documented figure sets up the next chapter precisely. NVIDIA's post states eight HBM3E stacks and 16 × 512-bit controllers — two controllers per stack (§4 row 13). That ratio is the first hint that "bandwidth per stack" is not one number but a structure, and 26.4 is about what happens when a workload lands unevenly across it.

20. Assertions Worth Writing

// ILLUSTRATIVE (§12). Three properties that catch the specific bugs this
// chapter's structures are exposed to. English statement first, every time.
 
// English: an accepted job is retired at most once. Catches a duplicate
// completion retiring a slot that has already been reallocated — the bug
// that silently releases a barrier early (§12 Failure lens).
a_retire_once: assert property (
  @(posedge clk) disable iff (!rst_n)
    retire_fire |-> jobs_q[cmpl_idx].live
);
 
// English: a transport retry never creates a second semantic job. Catches
// §16's conflation at its source, where it would corrupt both the work count
// and the service-rate estimate.
a_retry_creates_no_job: assert property (
  @(posedge clk) disable iff (!rst_n)
    transport_retry_fire |-> !dispatch_fire
);
 
// English: a live job's route epoch never changes underneath it. Catches a
// route table updated entry-by-entry while work is outstanding — the
// correctness failure behind 26.1 §15's misroute, in a scheduler.
a_route_epoch_stable: assert property (
  @(posedge clk) disable iff (!rst_n)
    (jobs_q[chk_idx].live && $past(jobs_q[chk_idx].live))
      |-> (jobs_q[chk_idx].route_epoch == $past(jobs_q[chk_idx].route_epoch))
);

Architecture. Three properties covering the three distinct hazards: duplicate retirement, retry-as-work, and placement changing under a live object.

State. All three read state the design already maintains — no assertion here requires a shadow model, which is what makes them viable in a large simulation.

Event. Each is triggered by a specific accepted event, so none is vacuous while traffic flows. a_retire_once is meaningful only if retirements actually occur, which coverage must confirm rather than assume (25.8 §22).

Contract. a_retry_creates_no_job encodes the layer separation of §16 as a checkable property rather than a convention — the reason it belongs in RTL and not only in a review checklist.

Failure. a_route_epoch_stable fires exactly when someone adds a "quick" runtime remap without quiescing. That change looks harmless in review and is not, which is why the property is worth its cost.

DV/debug. These are simulation-suitable as written. a_route_epoch_stable over a large job array is expensive to prove formally; a formal effort should bind it to a single symbolic index rather than the whole array — a scoping decision worth making deliberately instead of discovering when the proof does not converge.

21. What Public Sources Do Not Disclose

Mandatory section. This is what I looked for and did not find, and what I therefore refuse to draw or claim.

And the reason this section is mandatory rather than decorative. Every item here is something a chapter could invent and sound more authoritative for inventing. A drawn queue with a depth, a named credit scheme, a plausible arbitration policy — none of it is in any source I could reach, and presenting it as product architecture would be fabrication regardless of how reasonable the guess was.

Public material reviewed does not establish any of the above for either package. What the framework still gives you is the list of questions to ask — which is why §7's six edge properties are more useful than any number a datasheet prints.

22. Not a UCIe Deployment

Three bounded statements.

Neither package's die-to-die links are established as UCIe. AMD documents Infinity Fabric as the interconnect between XCDs, HBM and I/O dies. NVIDIA documents NV-HBI, described as "a custom, power-efficient die-to-die interconnect technology" — the word "custom" is the vendor's own (§4 row 10).

The standard is meanwhile advancing on its own schedule. UCIe 3.0 (Aug 2025) adds 48 GT/s and 64 GT/s rates (Class E). That is evidence about the specification, not about adoption — and conflating the two is 22.1 §12's error.

And these boundaries are exactly where a standard's advantages are weakest (23.2 §8). XCD-to-IOD and die-to-die inside one GPU are internal boundaries between dies designed together, by one company, on one schedule, in one package. Co-design wins there. UCIe's decisive advantage — cross-vendor, cross-node interoperability — is not being asked for at these boundaries at all.

Which makes them better teaching material, for the same reason 26.1 §17 gave: they are documented at the architectural level, and a UCIe-based accelerator package currently is not.

23. Common Misconceptions

"More compute means more performance." §8–§9: at four dies HBM binds at 10 units/cycle; doubling compute changes the phase time by zero cycles.

"Peak bandwidth tells you what you will get." §4 row 5, §7: a peak omits sustained service, latency, concurrency, direction and burstiness. It is one of six numbers you need.

"Fix the bottleneck and you are done." §10: fixing HBM makes D2D bind. Resource graphs improve in matched steps.

"Even distribution is fair." §13: an even split of units is only an even split of time if service rates are equal — and MI300X's documented harvesting means dies are not identical even when new.

"A high-utilisation link is a productive link." §16: retries make utilisation rise while throughput falls. Degradation can make the link look better.

"The link is ACTIVE, so the interconnect is fine." §18 step 1: ACTIVE proves training succeeded, not that useful work is moving (25.9 §5).

"Partitioning modes are a software feature." §11: NPS4 requires CPX, and memory partitions cannot exceed compute partitions — the invariant comes from ownership, and it is architectural.

"5.3 TB/s and 8 TB/s can be compared." §5: different memory generations, capacities and products, both peaks. Comparing them as stated is a category error.

"These are UCIe products." §22: Infinity Fabric and NV-HBI, the latter described by NVIDIA as "custom".

24. Understanding Check

25. Summary

Six things.

An accelerator package is a resource graph (§1, §7). Compute, cache, D2D, HBM, scheduler and barrier are all nodes, and achieved throughput is the minimum sustainable service along the path a phase uses.

Peak compute is one node, and usually not the binding one (§8). Four dies, HBM binding at 10 units/cycle: doubling compute changes the phase time by zero.

The constraint migrates as you scale (§9–§10). Between two and four dies it moved from D2D to HBM with nobody touching memory; at eight dies the package gains nothing at all.

Partitioning modes show the graph is a choice (§11). SPX / DPX / CPX and NPS1 / NPS2 / NPS4, with a documented invariant — memory partitions cannot exceed compute partitions — that comes straight from ownership.

A barrier makes the slowest participant the phase clock (§17–§18). One die at half rate halves package throughput, with no error, no corruption and every functional test passing — and only service-rate-aware distribution (§14) absorbs it.

And neither package is a UCIe deployment (§22) — Infinity Fabric and a "custom" NV-HBI. That has not made either less instructive, because the lessons above are interconnect-agnostic and these are the packages whose architecture is actually published.