Scalability

Chapters 15.1 and 15.2 quantified one link: how much it carries, and where one transaction's time goes. Both treated the link as the system.

A real package has several links, several dies, and one shared middle.

1. The One-Sentence Model

Scalability is the ability to add resources without creating a new shared bottleneck faster than capacity grows. Adding links adds edge bandwidth; whether it adds throughput depends entirely on what those links share.

The failure mode is not that scaling stops working. It is that scaling works arithmetically and not physically — a spreadsheet sums four links and reports 256 GB/s, and the silicon delivers 128, with no bug anywhere and no counter reporting an error.

2. What This Chapter Owns

Specifically new here: the six scale dimensions and why they are independent; the verified cluster-to-link aggregation; the min-cut bound, which is the chapter's central tool; oversubscription and the conditions under which it is correct engineering rather than under-provisioning; load imbalance and its address-interleaving cause; arbitration that stops scaling because a central grant network runs out of timing rather than out of logic; response-path and full-duplex asymmetry, where the forward direction scales and reads do not; scaling efficiency as a measured quantity; and per-link first-binding-resource attribution, which is what lets you say "link 2 is target-limited and link 3 is arbitration-limited" rather than "scaling is poor".

3. Sourcing

4. Six Scale Dimensions

These are independent axes, and conflating them is the first modelling error.

Three observations.

The first two dimensions scale cleanly and the last four do not. Adding lanes or clusters increases capacity on a path that is otherwise unchanged; adding links, dies, traffic sources or protocols introduces contention for something. So "we scaled" needs to name the axis, because two of the six are essentially free and four are not.

They have different units of cost. Lanes cost shoreline. Links cost shoreline and fabric ports. Dies cost package area and routing. Traffic costs nothing physical and can still destroy throughput by creating contention. A cost model that only counts silicon misses the fourth.

And they interact. Doubling links while halving clusters per link keeps raw capacity constant and changes the contention structure completely — more independent paths into a shared fabric, each narrower. Whether that helps depends on §18's balance question.

5. The Verified Aggregation Step

One place in this chapter has an exact, enumerated, verified answer.

UCIe 1.0, Figure 6: "Cluster Width; 1, 2, or 4 Clusters can be combined in each packaging option to deliver higher bandwidth."

raw_link_bw = raw_cluster_bw × cluster_count,   cluster_count ∈ {1, 2, 4}

Worked, using the verified inputs of 15.1 §12:

one standard-package cluster, 16 data lanes, 32 GT/s, 1 bit/transfer:
  raw_cluster_bw = 32e9 × 16 × 1 = 512e9 bit/s = 512 Gb/s = 64 GB/s per direction
 
  1 cluster:  64 GB/s per direction
  2 clusters: 128 GB/s per direction
  4 clusters: 256 GB/s per direction

Two things to note about the enumerated set.

It is {1, 2, 4}, not "any number". A model that assumes three clusters, or seven, is describing something the material does not. This is the kind of detail that separates a model built from sources from one built from intuition — and it is verified, so there is no reason to guess.

And aggregation multiplies the link's ceiling, not the package's throughput. Four clusters give one link 256 GB/s per direction. Whether the die behind it can source or sink 256 GB/s is a different question entirely, and it is §9's.

6. Topologies, Illustratively

No official source I could reach defines or prefers a package topology (§3). These are generic organisations, listed because the contention structure differs and that is what §9 measures.

Three consequences, and the third is the design lesson.

Direct links have no shared middle and therefore scale best — and cost most. Every pair that must communicate needs shoreline on both dies. For N dies fully connected that is N(N−1)/2 links, which is why real packages do not do it beyond small N.

Chains make transit traffic invisible in a per-die budget. A die in the middle carries traffic that neither originates nor terminates there, so its link and fabric budgets must include flows it has no stake in. A budget built per-die from that die's own workload will under-provision it.

And the topology determines which min-cut matters. In a star it is the central die. In a chain it is the busiest intermediate hop. In a fabric it is the bisection. §9's tool is the same in all four; the cut you have to find is different, and finding the wrong one produces a model that predicts the wrong plateau.

7. Where the Sharing Is

The figure exists to make one thing visually unavoidable: every flow crosses the fabric. Adding a fourth link adds a fourth arrow into the same box. The edge capacity grows; the box does not.

And note the right-hand column. Even a generously provisioned fabric does not remove the limit — it moves it to the targets. There is always a next cut, which is why §9 is a minimum over a set rather than a single check.

8. Aggregate Raw Bandwidth Is Not a Model

aggregate_raw_bw = Σ raw_bw(link i)   over active links

This is arithmetic, and it is true. It is also not a prediction.

The equation is valid only under an assumption it does not state: that traffic can use those links independently and simultaneously. That assumption fails whenever the links share anything downstream — which, per §6, they almost always do.

Summing link capacities gives you an upper bound on edge bandwidth. It gives you no information at all about achievable package throughput.

Three ways the assumption fails in practice, each developed later:

9. The Min-Cut Bound

The central tool of the chapter, and it is worth more than any equation in it.

The throughput of a flow is bounded by the narrowest cut it must cross. The throughput of a set of flows is bounded by the narrowest cut that all of them cross.

For a single flow, the bound is a minimum over the path:

throughput(flow) ≤ min(
    source_capacity,          // can the producer generate it?
    ucie_link_capacity,       // Chapter 15.1
    shared_fabric_capacity,   // the SoC's, not UCIe's
    target_capacity           // memory, accelerator, controller
)
 
Units: every term in bytes/s at a stated measurement boundary (Section 17)

For multiple flows the interesting cuts are the shared ones, and this is where a naive model goes wrong:

Σ throughput(flows crossing cut C) ≤ capacity(C)     for every cut C

So the package's throughput is bounded by whichever constraint binds first, over both per-flow paths and shared cuts. Finding it is the whole exercise, and §6 established that the topology determines where to look.

Two properties of the bound.

It is a bound, not a prediction. A flow can be far below its min-cut for reasons this chapter does not model — insufficient concurrency (13.5 §13), latency-bound behaviour, or a producer that simply cannot generate the work. Min-cut tells you the ceiling; 15.5 tells you why you are below it.

And the cuts must be measured at consistent boundaries. A link capacity in transmitted bytes and a memory capacity in payload bytes are not comparable, and comparing them produces a false conclusion. §17 is that trap.

10. Worked Min-Cut

Illustrative capacities, except the link figure which uses the verified inputs of §5.

CONFIGURATION
  4 UCIe links, each 1 standard-package cluster at 32 GT/s   [VERIFIED inputs]
  shared internal fabric                                     [ILLUSTRATIVE]
  memory subsystem behind the fabric                         [ILLUSTRATIVE]
 
STEP 1 — link capacity, per direction (Section 5)
  per link raw   = 32e9 × 16 × 1 = 512 Gb/s = 64 GB/s
  4 links        = 256 GB/s edge capacity, per direction
 
STEP 2 — useful capacity after framing (15.1 Section 19)
  framing_efficiency = 64/68 = 0.9412              [VERIFIED 68B flit, 4B overhead]
  useful edge        = 256 × 0.9412 = 241.0 GB/s
 
STEP 3 — the other cuts
  shared fabric capacity   = 128 GB/s              [ILLUSTRATIVE]
  memory subsystem         = 180 GB/s              [ILLUSTRATIVE]
  compute die egress       = 200 GB/s              [ILLUSTRATIVE]
 
STEP 4 — the minimum
  throughput ≤ min(200, 241.0, 128, 180) = 128 GB/s      ← the FABRIC binds
 
RESULT
  edge capacity        = 241.0 GB/s useful
  achievable bound     = 128 GB/s
  stranded link capacity = 241.0 − 128 = 113.0 GB/s  (47% of the useful edge)

Three readings.

Nearly half the link capacity cannot be used, and no counter on any link reports a problem. Each link sees backpressure, which is correct behaviour — 13.3 established that a fully backed-up path is a legal steady state.

Adding a fifth link changes nothing. The bound is 128 either way. This is the single most common wasted architectural decision in multi-die design, and the min-cut calculation costs ten minutes.

And the fix is named by the calculation. The fabric is the cut, so the fabric is what to widen. Widening it to 200 moves the bound to 180 — the memory — which is §36's bottleneck migration seen in advance rather than discovered afterwards.

11. Oversubscription

oversubscription_ratio = Σ input_peak_capacity / shared_output_capacity

Worked, illustrative:

4 inputs, each peaking at 50 GB/s      → Σ = 200 GB/s
shared output capacity                  = 100 GB/s
 
oversubscription = 200 / 100 = 2.0     → "2:1 oversubscribed"
 
Units: (bytes/s) / (bytes/s) = dimensionless          ✓

Interpretation, and the sign convention matters: a ratio greater than 1 means the inputs can collectively demand more than the output can deliver. It does not mean they will.

12. Oversubscription Is Not Automatically Wrong

This is the section that stops the min-cut tool from being used as a blunt instrument.

Oversubscription is correct engineering when the inputs do not peak simultaneously. Provisioning a shared resource for the sum of all peaks means provisioning for a case that never occurs — and paying for it in area, power and routing on every part shipped.

Two design consequences.

The acceptable ratio is a property of the workload, not of the hardware. So an oversubscription figure quoted without a traffic assumption is meaningless. "2:1" is a statement about provisioning; whether it is safe is a statement about correlation.

And correlation is the thing that catches teams out. Four accelerators running the same kernel on different data will peak together — they are not independent, they are four copies of one behaviour. The statistical argument for oversubscription requires independence, and identical engines are the least independent things in a package.

13. Wrong Assumption — Every Link Peaks Simultaneously

The archetypal modelling failure, and it lives in a spreadsheet rather than in RTL.

THE SPREADSHEET
  link 0: 64 GB/s
  link 1: 64 GB/s
  link 2: 64 GB/s
  link 3: 64 GB/s
  ──────────────────
  "package bandwidth" = 256 GB/s
 
THE MEASUREMENT
  achieved = 128 GB/s        → half the estimate
 
THE INVESTIGATION
  every link: correct, no errors, no dropped packets, all assertions pass
  every queue: legal occupancy
  every credit counter: conserved
  RTL: no bug found, because there is no bug

Four properties of this failure.

There is nothing to fix in the design. The model was wrong, not the silicon. And the model is the artefact that drove the floorplan, the shoreline allocation and the power budget — so the cost of the error is already committed by the time the measurement exists.

It survives review because the arithmetic is correct. Four times 64 is 256. The defect is an unstated assumption, and unstated assumptions do not appear in the cells of a spreadsheet.

It is discovered late, because it needs a real multi-link workload — which typically arrives after the architecture is frozen.

And the diagnosis feels like a hardware problem. Half the expected performance with no errors looks like a bug, so the search happens in RTL and DV. The min-cut calculation of §10 would have predicted 128 before any RTL existed, which is the argument for doing it at architecture time.

A capacity sum is not a throughput model. Every capacity sum needs the sentence "assuming traffic can use these independently" written next to it — and then that sentence needs to be checked.

14. Per-Link Instrumentation

// ILLUSTRATIVE per-link performance counters. No UCIe counter definition is
// claimed (Section 3). Event-qualified per 15.1 Section 27 — an actual
// transfer, never an offer.
localparam int NUM_LINKS = 4;
 
logic [63:0] link_req_count_q       [NUM_LINKS];  // transactions accepted
logic [63:0] link_byte_count_q      [NUM_LINKS];  // ALL transmitted bytes
logic [63:0] link_payload_bytes_q   [NUM_LINKS];  // NEW payload only
logic [63:0] link_transfer_cycles_q [NUM_LINKS];
logic [63:0] link_observation_q     [NUM_LINKS];
 
always_ff @(posedge clk or negedge rst_n) begin
  if (!rst_n || perf_clear) begin
    for (int l = 0; l < NUM_LINKS; l++) begin
      link_req_count_q[l]       <= '0; link_byte_count_q[l]      <= '0;
      link_payload_bytes_q[l]   <= '0; link_transfer_cycles_q[l] <= '0;
      link_observation_q[l]     <= '0;
    end
  end else begin
    for (int l = 0; l < NUM_LINKS; l++) begin
      link_observation_q[l] <= link_observation_q[l] + 64'd1;
 
      // beat_transferred[l] is valid && ready on THAT link (15.1 Section 27).
      if (beat_transferred[l]) begin
        link_transfer_cycles_q[l] <= link_transfer_cycles_q[l] + 64'd1;
        link_byte_count_q[l]      <= link_byte_count_q[l] + 64'(beat_bytes[l]);
        if (!beat_is_replay[l])
          link_payload_bytes_q[l] <= link_payload_bytes_q[l] + 64'(beat_payload_bytes[l]);
      end
      if (req_accepted[l]) link_req_count_q[l] <= link_req_count_q[l] + 64'd1;
    end
  end
end

Architecture. Five counters per link, because the questions this chapter asks — is the load balanced, which link is saturated, which link is starved — cannot be answered from an aggregate. An aggregate counter can tell you the package is at half capacity; only per-link counters can tell you it is because one link carries 90% of the traffic.

State. All observation-window lifetime, cleared together across every link. Clearing one link's counters independently breaks every cross-link comparison, because the ratios would span different windows.

Cycle behaviour. Every counter is qualified by beat_transferred[l] — 15.1 §27's rule applied per link. The per-link version of that bug is worse than the single-link one, because a design that counts offers on a backpressured link will report the most congested link as the busiest, which is true, and as the highest-throughput link, which is false — and that inverts the load-balance conclusion.

Contract. These feed §19's imbalance metric, §34's scaling efficiency and §16's aggregate reconciliation. All three assume a common window and a common measurement boundary (§17).

Failure. Counting offers rather than transfers, per above. Or sizing the arrays by a configured link count while the active count is lower — an inactive link's zero counters then drag the mean down and make balanced traffic look imbalanced.

DV. Drive a known per-link workload and check each counter independently. Then drive all traffic to one link and confirm the others read zero — which sounds trivial and catches an indexing error that otherwise silently smears traffic across links in the report.

15. The Aggregate Counter, Derived

// ILLUSTRATIVE package aggregate. DERIVED from per-link accounting, never
// counted independently — Section 16 is why.
logic [63:0] package_payload_bytes_q;
 
// Combinational sum for the current window. For wide NUM_LINKS this is a
// pipelined adder tree in practice; the point is the SOURCE of the value.
always_comb begin
  package_payload_bytes_q = '0;
  for (int l = 0; l < NUM_LINKS; l++)
    package_payload_bytes_q += link_payload_bytes_q[l];
end

Architecture. A derived value rather than an independently maintained counter. That choice is the section, and it is not merely economy.

Why it must be derived. An independently incremented package counter has its own event definition, its own qualification and its own bugs. When it disagrees with the sum of the per-link counters, there is no way to tell which is right — the design has two sources of truth for one quantity. Deriving it makes disagreement structurally impossible and turns §16's property into a tautology that is worth asserting anyway, because someone will eventually "optimise" the derivation into a separate counter.

State. None of its own — that is the point.

Cycle behaviour. Combinational from the per-link registers. In a real design with many links this is registered and pipelined, and the moment it is, the aggregate lags the per-link values by the pipeline depth. That is fine for a window-boundary read and wrong for a same-cycle comparison — which is why §16's property is evaluated at window_end rather than every cycle.

Contract. Whoever reads the aggregate must read it from the same snapshot as the per-link values (15.1 §30).

Failure. Maintaining it independently. The characteristic symptom is a package figure that disagrees with the sum by a small, load-dependent amount — small enough to be dismissed as rounding, and it is not rounding.

16. SVA — the Aggregate Reconciles

// Verification-only. Trivially true if the aggregate is DERIVED (Section 15),
// and the whole point is to catch someone changing that.
property p_aggregate_equals_sum;
  @(posedge clk) disable iff (!rst_n)
    window_end |-> (package_payload_bytes_q == sum_of_link_payload_bytes());
endproperty
a_aggregate_equals_sum: assert property (p_aggregate_equals_sum);
 
// The scoreboard-level version, at a matched SEMANTIC boundary (Section 17).
property p_package_payload_matches_monitor;
  @(posedge clk) disable iff (!rst_n)
    window_end |-> (package_payload_bytes_q == tb_observed_payload_bytes);
endproperty
a_package_payload_matches_monitor: assert property (p_package_payload_matches_monitor);

Architecture. Two properties: an internal consistency check and an external correctness check.

Why the first is worth writing even though §15 makes it tautological. It is a guard against a future refactor. The derivation is the kind of code that gets replaced by a dedicated counter for timing reasons, and when it does, this property is what notices. A tautological assertion over a structure that is designed to make it tautological is documentation with teeth.

Why the second needs a monitor. Internal consistency does not establish that either number is right. Only an independently observed count at a matched boundary does — and that boundary must be stated, which is §17.

DV. Both need a multi-link workload with unequal loads. With traffic on one link only, the sum equals the single link's value and the property passes without exercising the summation at all.

17. Measurement Boundaries Must Match

The trap that invalidates more cross-resource comparisons than any other.

The same traffic has different byte counts at different boundaries:

Worked, illustrative. 100 GB/s of semantic payload, a 68 B flit and a 2% replay fraction:

semantic payload      = 100.0 GB/s
transmitted bytes     = 100.0 / 0.9412  = 106.2 GB/s      (framing added)
lane traffic          = 106.2 / 0.98    = 108.4 GB/s      (replay added)
 
So the SAME traffic is 100, 106.2 or 108.4 GB/s depending on where you measure.

Two false conclusions this produces.

"Bytes disappeared." Comparing a link counter measured in transmitted bytes against a memory counter measured in payload bytes shows the link carrying more than memory received. Nothing was lost — 8.4% of it was framing and replay, and the comparison was invalid.

And a false min-cut. §9's minimum is only meaningful if every term is at the same boundary. Comparing a link's transmitted-byte capacity against a memory subsystem's payload-byte capacity understates the link's effective payload capacity by 8.4% and can name the wrong bottleneck.

State the boundary with every capacity and every measurement, and convert explicitly before comparing. A min-cut over mixed boundaries is not a bound — it is a coincidence.

18. Load Balance

Equal link capacity does not produce equal link load.

ILLUSTRATIVE measurement, 2 links of 64 GB/s each:
 
  link 0 delivered:  57.6 GB/s   (90% of the traffic)
  link 1 delivered:   6.4 GB/s   (10%)
  ────────────────────────────
  total delivered:   64.0 GB/s
 
  aggregate capacity: 128 GB/s
  achieved:            64 GB/s   → 50% of capacity
 
  and link 0 is at 90% of ITS capacity while link 1 is at 10% of its own

The system is simultaneously saturated and half-idle. Link 0 cannot go faster; link 1 has 57.6 GB/s of unused capacity that no flow can reach. Adding a third link makes this worse in ratio terms, because the imbalance mechanism (§20) does not improve with more links.

And the diagnosis from an aggregate view is wrong. Total delivered is 64 of 128, so an aggregate-only measurement says "we are at 50% utilisation, so we are not link-limited." We are link-limited — on one link. Only per-link counters distinguish these.

19. The Imbalance Metric

// VERIFICATION-ONLY. Simple metrics, deliberately — a complicated fairness
// index is not needed to see a 90/10 split.
function automatic real link_utilisation(int l);
  return real'(link_payload_bytes_q[l]) / real'(link_capacity_bytes[l]);
endfunction
 
function automatic real imbalance_spread();
  real mx = 0.0, mn = 1.0e30;
  for (int l = 0; l < NUM_ACTIVE_LINKS; l++) begin
    real u = link_utilisation(l);
    if (u > mx) mx = u;
    if (u < mn) mn = u;
  end
  return mx - mn;                     // 0.0 = perfectly balanced
endfunction
 
function automatic real balance_ratio();
  // delivered total ÷ (worst link's utilisation × total capacity).
  // 1.0 = perfectly balanced; 0.5 = one link doing twice its share.
  real mx = 0.0, tot = 0.0;
  for (int l = 0; l < NUM_ACTIVE_LINKS; l++) begin
    real u = link_utilisation(l);
    if (u > mx) mx = u;
    tot += real'(link_payload_bytes_q[l]);
  end
  return (mx == 0.0) ? 1.0
                     : tot / (mx * real'(total_capacity_bytes));
endfunction

Architecture. Two scalar metrics over the per-link counters. The second is the more actionable, because it directly answers "how much of my capacity is stranded by imbalance?"

Worked on §18's numbers:

link 0 utilisation = 57.6/64 = 0.90
link 1 utilisation =  6.4/64 = 0.10
 
imbalance_spread = 0.90 − 0.10 = 0.80        → severe
balance_ratio    = 64 / (0.90 × 128) = 0.556 → 44% of capacity stranded

Contract. Both use active links only, and both use the payload boundary consistently (§17). Mixing a payload numerator with a transmitted-byte capacity would understate utilisation by the framing factor and make a saturated link look like it has headroom.

Failure. Including inactive links, which drags the minimum to zero and makes imbalance_spread report 1.0 for perfectly balanced traffic. Or using request counts rather than bytes, which is misleading whenever transaction sizes differ across links.

DV. Drive perfectly balanced traffic and check both metrics report their ideal values; drive a 90/10 split and check they report the numbers above. Two directed tests, and they establish the metric is measuring what its name claims.

20. Address Interleaving and Link Selection

The usual mechanical cause of §18's imbalance.

If the target link is chosen from address bits, then the traffic distribution across links is the workload's address distribution, transformed by the selection function. That is a property of the software, not of the interconnect.

This is 11.2's interleaving discussion arriving in the performance domain. There it determined which device holds an address; here it determines which link carries the traffic, and therefore whether §18's imbalance exists.

And the granularity choice is a genuine trade rather than a tuning parameter. Fine interleaving maximises spread and minimises locality; coarse does the reverse. A workload with strong spatial locality and coarse interleaving is the exact configuration that produces a 90/10 split — and both halves of that sentence are design decisions made by different teams.

21. Illustrative Link Selection

// ILLUSTRATIVE link selection from address bits. NO UCIe address-mapping or
// link-selection rule is published (Section 3) — this is SoC design.
localparam int LINK_SEL_LSB = 12;     // ILLUSTRATIVE: 4 KiB granularity
localparam int LINK_SEL_MSB = LINK_SEL_LSB + $clog2(NUM_LINKS) - 1;
 
assign target_link = addr[LINK_SEL_MSB:LINK_SEL_LSB];
 
// The granularity this implies, made explicit rather than left in the comment.
localparam longint INTERLEAVE_GRANULE_BYTES = 64'd1 << LINK_SEL_LSB;
 
initial assert (NUM_LINKS == (1 << $clog2(NUM_LINKS)))
  else $fatal(1, "bit-slice link selection requires a power-of-two link count");

Architecture. A bit-slice selector — the cheapest possible mapping, and the one whose behaviour is most directly a function of the workload's address pattern.

State. None; combinational from the address.

Cycle behaviour. None. Note it must be stable for the whole of a transaction — a selector re-evaluated on a later beat of the same object could send beats of one transaction to two links, which is a correctness failure rather than a performance one.

Contract, and the elaboration assertion is doing real work. A bit slice can only select among a power-of-two number of links. With three active links — after one has been isolated (14.5 §17) — a bit slice selects among four and one selection value points at a link that is not there. That is the interaction between this chapter and Module 14 that a purely performance-focused design misses: degradation changes the active link count, and a power-of-two selector cannot follow it.

Failure. The non-power-of-two case above. And a granularity mismatch with the memory interleaving, which can produce a systematic pattern where certain address strides always land on one link.

DV. Sweep a contiguous address range and histogram the selected link — the histogram is the load balance, predicted before any traffic runs. Then sweep with a stride equal to NUM_LINKS × INTERLEAVE_GRANULE_BYTES and confirm the pathological single-link case is reproduced, because that is the stride a real workload can accidentally have.

22. Wrong Interleave Assumption — the Benchmark That Lies

THE BENCHMARK
  synthetic streaming test, addresses swept linearly across the whole space
  → the bit-slice selector spreads perfectly across 4 links
  → measured: 4 × 57 = 228 GB/s     "scaling efficiency 0.95, excellent"
 
THE PRODUCTION WORKLOAD
  a single large contiguous working set, 256 KiB, hot in a loop
  with 4 KiB granularity and 4 links, 256 KiB spans 64 granules → spread is fine
  BUT the hot inner loop touches one 4 KiB page repeatedly
  → nearly all traffic selects ONE link
  → measured: ~57 GB/s              "scaling efficiency 0.25"

Four properties of this failure.

The benchmark is not wrong — it is measuring something else. A linear sweep measures the interconnect's scaling. A hot loop measures the mapping's fit to a workload. Both are legitimate measurements and only one of them predicts production.

The hardware is identical in both cases. No configuration changed, no fault occurred, no counter reports an error. The only difference is the address distribution, which is not a hardware property at all.

And the failure is worse with coarse granularity. Finer interleaving would have spread even the hot page — at the cost of splitting every burst across links, which harms locality and increases per-transaction latency. There is no granularity that is right for both workloads, which is why the choice needs the workload stated.

The defence is §21's histogram, run against the real workload's address trace rather than a synthetic sweep. That is a cheap, static, pre-silicon experiment and it predicts the imbalance exactly.

23. Arbitration Scaling

More requesters is not merely more logic — it is a different timing problem.

The consequence that surprises people: adding links can reduce the achievable clock frequency, and therefore reduce the throughput of every link including the ones that already existed.

A design whose arbitration is a single flat priority network does not scale in links — it scales until timing closure fails, and then it stops.

And the cost is not visible in functional simulation at all. 13.4 §17's arbiter is functionally correct at any N. The failure is in synthesis and in silicon, which is the wrong place to discover an architectural limit.

24. Hierarchical Arbitration

// ILLUSTRATIVE two-level arbitration. The structural answer to Section 23:
// replace one N-way network with G groups of N/G plus one G-way.
localparam int NUM_GROUPS   = 4;
localparam int PER_GROUP    = NUM_LINKS / NUM_GROUPS;
 
// Level 1: one rotating-priority arbiter per group (13.4 Section 17).
logic [PER_GROUP-1:0] group_req   [NUM_GROUPS];
logic [PER_GROUP-1:0] group_grant [NUM_GROUPS];
logic [NUM_GROUPS-1:0] group_has_req;
 
generate for (genvar g = 0; g < NUM_GROUPS; g++) begin : g_local
  rr_arbiter #(.N(PER_GROUP)) u_local (
    .clk(clk), .rst_n(rst_n),
    .request(group_req[g]),
    .transfer_done(group_xfer_done[g]),     // 13.4 Section 17: on TRANSFER
    .grant(group_grant[g])
  );
  assign group_has_req[g] = (group_req[g] != '0);
end endgenerate
 
// Level 2: one rotating-priority arbiter across groups.
logic [NUM_GROUPS-1:0] top_grant;
rr_arbiter #(.N(NUM_GROUPS)) u_top (
  .clk(clk), .rst_n(rst_n),
  .request(group_has_req),
  .transfer_done(any_xfer_done),
  .grant(top_grant)
);
 
// The final grant is the conjunction: the group won, and the requester won
// within its group.
generate for (genvar g = 0; g < NUM_GROUPS; g++) begin : g_final
  for (genvar i = 0; i < PER_GROUP; i++) begin : g_bit
    assign final_grant[g*PER_GROUP + i] = top_grant[g] && group_grant[g][i];
  end
end endgenerate
 
initial assert (NUM_LINKS == NUM_GROUPS * PER_GROUP)
  else $fatal(1, "hierarchical arbiter requires NUM_LINKS divisible by NUM_GROUPS");

Architecture. Replace one N-way decision with G parallel N/G-way decisions plus one G-way decision. For 64 requesters that is 8 eight-way arbiters plus one eight-way arbiter — each network is a fraction of the depth of a flat 64-way one, and the two levels are separated by a register in a pipelined variant.

State. One rotating pointer per group plus one across groups — (G + 1) pointers of log₂ width each, rather than one wide pointer. Total fairness state is comparable; the path through it is much shorter.

Cycle behaviour. Both levels evaluate combinationally in the form shown, so the depth is the sum of the two networks rather than one flat network — already better, and it becomes much better when a register is inserted between the levels, at the cost of one cycle of arbitration latency.

Contract. Both levels must use the transfer-qualified pointer update of 13.4 §17, and the top level's transfer_done must be the OR of the groups'. A top-level pointer advancing on a group's request rather than on an actual transfer reintroduces 13.4 §18's bias at the group granularity — which starves whole groups rather than individual links, and is correspondingly harder to notice.

Failure — and it is specific to hierarchy. Fairness is now fair between groups and fair within groups, which is not the same as fair between requesters. A group of one requester and a group of seven each get half the grants, so the lone requester gets 50% and each of the seven gets about 7%. That is a real and often unintended consequence, and the fix is either equal group sizes or weights at the top level.

DV. Assert grant-implies-request and one-hot at the final grant, not only per level. Then measure the per-requester grant distribution with unequal group populations and confirm it matches the intended policy — which is the test that exposes the fairness asymmetry above.

25. Wrong Design — One Flat Grant Network

// WRONG at scale — a single combinational priority network across every
// requester. Functionally perfect, and it will not close timing.
always_comb begin
  final_grant = '0;
  for (int i = 0; i < 64; i++)                 // 64 requesters
    if (request[i] && !granted_yet) begin
      final_grant[i] = 1'b1;
      granted_yet = 1'b1;
    end
end

Why it is a scaling failure rather than a coding failure.

Functional simulation is perfect at any width. The arbiter grants exactly one requester, always the lowest-indexed one requesting. Every assertion in 13.4 §19 passes.

Synthesis produces a network whose depth grows with the requester count, and that network sits between the request registers and the grant registers — on the critical path of the resource being shared. The consequence is a lower achievable clock, which reduces the throughput of every link.

Three properties that make it an architecture problem rather than an RTL problem.

The cost is paid by everything, not by the new links. Adding links 33 to 64 lowers Fmax, which lowers the throughput of links 1 to 32 as well. So the marginal link has negative value beyond some point — the arithmetic of §8 says capacity rose, and measured throughput fell.

It is discovered at synthesis, after the architecture is committed. The link count came from a bandwidth model; the timing failure arrives months later; and by then the shoreline, the bumps and the floorplan assume that link count.

And the priority encoding compounds it with unfairness. This particular form is also strict priority, so 13.4 §16's starvation applies: requester 63 is served only when none of 0–62 is requesting. Two independent defects in one block, and only one of them shows up in simulation.

26. Queue Scaling

The hybrid is almost always the right answer at scale, and the reason is the same one 13.1 §13 gave for credit pools: a reserved minimum bounds the damage one link can do, and a shared remainder recovers most of the utilisation.

And the shared-queue failure gets worse with more links, not better. With one shared queue and N links, a single blocked head blocks traffic for all N. The blast radius of head-of-line blocking scales with the thing you were trying to scale, which is the specific reason a design that worked at two links can fall over at eight.

27. Credit and Resource Pools Do Not Merge

Each link has its own remote receiver, its own replay obligations and its own queues. Those resources are per-link because the storage they reserve is per-link, and merging them into a package-wide pool is 13.1 §6's error at package scale.

The failure a merged pool produces. A package-wide credit counter of 400 with four links of 100 each would permit 400 objects to be sent to one link — which has room for 100. The other 300 overflow a receiver that never advertised capacity for them, which is 13.1 §2's catastrophic over-advertisement direction, reached by a package-level abstraction rather than by a counting bug.

And the tempting version is subtler than that. Not a single merged counter, but a shared spare pool on top of per-link reservations — which is fine only if the spare can be bound to a specific link before it is spent, and is a bug if it can be spent generically. The rule is that a credit must name its destination at the moment it is consumed.

28. Response-Path Scaling

The forward path can scale perfectly while reads do not.

ILLUSTRATIVE structure:
  4 request links, each 64 GB/s        → request capacity 256 GB/s
  1 shared response fabric, 100 GB/s   → response capacity 100 GB/s
 
A read workload:
  request out:  small — an address and a header, say 32 B
  response in:  large — 64 B of data plus framing
 
So a read's bytes are asymmetric: tiny out, large in.

What the measurement looks like:

Read throughput plateaus at the response fabric, and the request direction looks almost idle. An engineer looking at TX utilisation concludes the links are over-provisioned and the interconnect is not the problem. An engineer looking at RX utilisation concludes the opposite. Both are reading real numbers.

Three consequences.

Reads and writes scale differently on the same hardware. A write-heavy workload loads TX; a read-heavy one loads RX. A package budgeted on an aggregate figure is budgeted for neither — 15.4 §14 develops this into a directional budget.

Adding request links does nothing for reads. The min-cut is the response path (§9), and it is shared, so it does not grow when links are added.

And this is the most common "scaling stopped working" pattern in read-dominated systems. The forward-path arithmetic is correct and irrelevant. §39's taxonomy lists it: reads scale poorly while writes scale well means the response path is the cut.

29. Full-Duplex Accounting

The lanes are described as "unidirectional, full-duplex" (§3), so TX and RX are separate capacities — and 15.1 §10 established that summing them is only meaningful when the workload uses both.

At package scale the rule sharpens:

For each link i:
  TX_utilisation(i) = tx_payload_bytes(i) / tx_capacity(i)
  RX_utilisation(i) = rx_payload_bytes(i) / rx_capacity(i)
 
Report BOTH. Never report (tx + rx) / (tx_cap + rx_cap).

Why the combined figure is actively misleading. A read-heavy link at 5% TX and 100% RX reports a combined utilisation of about 52% — which reads as comfortable headroom on a link that is completely saturated in the direction that matters. The averaging hides the saturation, and the more asymmetric the workload, the more it hides.

And the asymmetry is not a corner case. Read-dominated and write-dominated phases are normal, and 13.1 §11 established that the quiet direction still has to carry credit returns and acknowledgements — so the directions are separate capacities that are not independent, and a design that starves the quiet direction stalls the busy one.

30. Scalability Against Latency

A more scalable architecture can have worse per-transaction latency, and that is not automatically a failure.

Every row trades one for the other, and which is right depends on the workload. A throughput-oriented streaming workload should take all four. A latency-sensitive coherence workload should take none of them lightly.

The design error is to treat added latency as a defect by default. 15.2 §22 made the same point for pipeline stages: latency in cycles is not the metric; the metric is whether the system meets its requirement. A package that doubles throughput for one extra cycle of arbitration has almost certainly made the right trade — and one that adds three routing hops to a coherence path has probably not.

31. Scalability Against Fairness

Total package throughput can improve while a specific die gets worse, and an aggregate-only measurement cannot see it.

ILLUSTRATIVE, before and after adding two links and a shared fabric:
 
              before        after       change
  die A:      40 GB/s       70 GB/s     +75%
  die B:      40 GB/s       55 GB/s     +38%
  die C:      40 GB/s       12 GB/s     −70%     ← starved
  ──────────────────────────────────────────
  total:     120 GB/s      137 GB/s     +14%     "scaling worked"

The aggregate improved by 14% and one die lost 70% of its bandwidth. If die C has a latency or throughput requirement, the package has regressed while the headline number improved.

So a scaling measurement needs four quantities, not one:

And the mechanism behind die C's loss is usually arbitration or mapping, not capacity — which is why §32's per-link stall attribution is what identifies it. A die that lost bandwidth while capacity increased did not lose it to physics.

32. Per-Link Stall Attribution

// ILLUSTRATIVE per-link, per-reason stall counters. The same partition
// discipline as 13.5 Section 34 and 15.2 Section 23 — EXACTLY ONE bucket per
// stalled cycle per link, so the buckets partition each link's stalled time.
logic [63:0] stall_credit_q [NUM_LINKS];
logic [63:0] stall_queue_q  [NUM_LINKS];
logic [63:0] stall_arb_q    [NUM_LINKS];
logic [63:0] stall_target_q [NUM_LINKS];
logic [63:0] stall_replay_q [NUM_LINKS];
logic [63:0] idle_q         [NUM_LINKS];   // nothing offered — NOT a stall
 
always_ff @(posedge clk) begin
  for (int l = 0; l < NUM_LINKS; l++) begin
    if (!beat_transferred[l]) begin
      // Priority order = causal order (Section 33).
      unique case (1'b1)
        !work_offered[l]     : idle_q[l]         <= idle_q[l]         + 64'd1;
        !link_operational[l] : /* link down — counted elsewhere */ ;
        !credit_avail[l]     : stall_credit_q[l] <= stall_credit_q[l] + 64'd1;
        !replay_space[l]     : stall_replay_q[l] <= stall_replay_q[l] + 64'd1;
        !queue_space[l]      : stall_queue_q[l]  <= stall_queue_q[l]  + 64'd1;
        !arb_grant[l]        : stall_arb_q[l]    <= stall_arb_q[l]    + 64'd1;
        !target_ready[l]     : stall_target_q[l] <= stall_target_q[l] + 64'd1;
        default              : /* unattributed — see the identity below */ ;
      endcase
    end
  end
end

Architecture. Six buckets per link, with a priority-encoded single-bucket rule. The per-link dimension is what makes this chapter's diagnoses possible: an aggregate stall profile says "we are credit-limited", while this says "link 0 is target-limited, link 2 is arbitration-limited, link 3 is idle" — three different problems that an aggregate averages into one wrong answer.

State. All observation-window lifetime, cleared with the counters of §14.

Cycle behaviour. unique case (1'b1) takes the first true condition, so exactly one bucket per link per non-transferring cycle.

Contract — and it is a checkable identity, per link:

for each link l:
  observation[l] = transfer_cycles[l] + idle[l]
                 + stall_credit[l] + stall_replay[l] + stall_queue[l]
                 + stall_arb[l] + stall_target[l] + link_down[l] + unattributed[l]

Every link's identity must hold independently. A violation on one link means that link's priority logic has a hole, and checking only the aggregate would let one link's hole be masked by another link's surplus — which is a real hazard of aggregating before checking.

Failure. Incrementing every true condition rather than the first, which makes the fractions sum above 100% (15.5 §12 develops that). Or sharing one bucket set across links, which loses the entire per-link dimension.

DV. Stall one link on exactly one resource and confirm only that link's corresponding bucket accrues. Then verify each link's identity at every window boundary.

33. First Binding Resource, Per Link

The priority order in §32 is a design decision, and the same reasoning as 13.5 §35 applies with one addition.

Two properties, and the second is specific to multi-link.

Arbitration is deliberately late. A link that lacks credit and lost arbitration is credit-limited: giving it the grant would not have helped. Reporting it as arbitration-limited would send the investigation to the scheduler, which is working correctly.

And the per-link profile is what distinguishes the diagnoses of §39. "All links show high target" is a shared-target bottleneck. "One link shows high arb while others are idle" is an imbalance or a policy problem. "One link shows high idle while others are saturated" is a mapping problem (§20). The same three counters, read per link, name three different root causes — and read in aggregate they name none.

34. Scaling Efficiency

scaling_efficiency(N) = throughput(N links) / (N × throughput(1 link))
 
Units: (bytes/s) / (bytes/s) = dimensionless        ✓
Ideal = 1.0

Interpretation, including the case that confuses people:

Why greater than 1 is possible and is not a paradox. The denominator uses throughput(1 link), and a single link may be limited by something that is not per-link — insufficient outstanding transactions (13.5 §13), a producer that cannot saturate one link, or a fixed per-transaction overhead amortised better with more links in flight. Then adding links adds concurrency as well as capacity, and the ratio can exceed 1.

So a super-unity result is a finding about the baseline, not about the scaling. It says "my one-link measurement was not link-limited" — which is useful information and means the efficiency figure should be recomputed against a properly saturated baseline before it is trusted.

35. Worked Scaling Example

Illustrative measurements. Per-link capacity 64 GB/s useful.

  N links   throughput   N × T(1)   efficiency   marginal gain
  ───────────────────────────────────────────────────────────
      1       50 GB/s      50         1.000            —
      2       95 GB/s     100         0.950        +45 GB/s
      4      150 GB/s     200         0.750        +55 GB/s  (over 2 links)
      8      160 GB/s     400         0.400        +10 GB/s  (over 4 links)
 
ARITHMETIC CHECK
  efficiency(2) =  95 / (2 × 50) =  95/100 = 0.950   ✓
  efficiency(4) = 150 / (4 × 50) = 150/200 = 0.750   ✓
  efficiency(8) = 160 / (8 × 50) = 160/400 = 0.400   ✓

Four readings, and the marginal column is the most informative.

The bend is between 4 and 8 links. Marginal gain collapses from 55 GB/s to 10 GB/s. That is where a shared resource started binding, and it is visible in the marginal column long before the efficiency figure looks alarming.

160 GB/s is almost certainly a shared-resource ceiling. Throughput is nearly flat from 4 to 8 links, which is the signature of a hard limit rather than of gradual inefficiency. §9's calculation should be run to find which cut sits at ~160 — and if none does, the limit is concurrency rather than capacity.

Efficiency at 8 links is 0.40, and quoting that alone is misleading. It compares against 8 × 50 = 400 GB/s, which was never achievable given a 160 GB/s cut. The honest statement is "throughput saturates at ~160 GB/s, which is 4-link capacity" — and the efficiency figure is a way of noticing that, not a grade.

And the right next action is named by §33's per-link profile. If all eight links show high target, the shared target is the cut. If four show high arb, the arbiter is. The scaling table finds that there is a bottleneck; the stall profile finds which.

36. The Saturation Curve, and Bottleneck Migration

Throughput against offered load has a characteristic shape, and each region means something different:

The bend locates the bottleneck's capacity, and that is its practical value: the offered load at which the curve departs from linear is approximately the binding resource's capacity. No plotting framework is needed — a table of five offered loads and five measured throughputs shows it.

And relieving one bottleneck moves the curve's bend rather than removing it. In §10's example, widening the fabric from 128 to 200 GB/s moves the bound to the memory subsystem at 180. The plateau moves from 128 to 180 and does not disappear, because §9's minimum is over a set and there is always a next member. 15.5 §23 develops this as a methodology.

37. The Scalability Scoreboard

// Verification-only reference model. Not synthesisable.
class scalability_scoreboard;
 
  typedef struct {
    longint accepted_payload;     // entered this link
    longint delivered_payload;    // completed via this link
    longint stall[NUM_REASONS];
    longint idle;
    longint transfer_cycles;
    longint observation;
  } link_stats_t;
 
  link_stats_t link[NUM_LINKS];
  longint      fabric_payload, target_payload;
  longint      capacity[NUM_CUTS];
 
  // ---- Check 1: per-link conservation at a matched semantic boundary.
  function void check_link_conservation(int l, longint outstanding);
    if (link[l].accepted_payload != link[l].delivered_payload + outstanding)
      $error("LINK %0d CONSERVATION: accepted %0d != delivered %0d + outstanding %0d",
             l, link[l].accepted_payload, link[l].delivered_payload, outstanding);
  endfunction
 
  // ---- Check 2: the aggregate is the sum (Section 16), same boundary.
  function void check_aggregate(longint dut_package_payload);
    longint sum = 0;
    for (int l = 0; l < NUM_LINKS; l++) sum += link[l].delivered_payload;
    if (dut_package_payload != sum)
      $error("AGGREGATE: dut %0d != sum of per-link %0d", dut_package_payload, sum);
  endfunction
 
  // ---- Check 3: each link's cycle identity (Section 32).
  function void check_cycle_identity(int l);
    longint s = link[l].transfer_cycles + link[l].idle;
    for (int r = 0; r < NUM_REASONS; r++) s += link[l].stall[r];
    if (s != link[l].observation)
      $error("LINK %0d CYCLES: accounted %0d != observed %0d (%0d unattributed)",
             l, s, link[l].observation, link[l].observation - s);
  endfunction
 
  // ---- Check 4: no cut was exceeded. Catches a units or boundary error
  //      (Section 17) as well as a genuinely impossible measurement.
  function void check_cuts(longint window_cycles);
    for (int c = 0; c < NUM_CUTS; c++)
      if (cut_bytes(c) > capacity[c] * window_cycles)
        $error("CUT %0d EXCEEDED: %0d bytes over capacity %0d — units or boundary bug",
               c, cut_bytes(c), capacity[c]);
  endfunction
 
  // ---- Check 5: the predicted bottleneck matches the observed stall profile.
  //      This is what validates the MODEL rather than the counters.
  function void check_bottleneck_prediction();
    int predicted = argmin_cut();                 // from capacities and demand
    int observed   = dominant_stall_reason_across_links();
    if (!consistent(predicted, observed))
      $error("MODEL MISMATCH: min-cut predicts %0d, stall profile says %0d",
             predicted, observed);
  endfunction
 
endclass

Architecture. Five checks in three categories: conservation (1, 2), accounting integrity (3, 4), and model validity (5).

Check 1 is per link and per boundary. Payload accepted on a link equals payload delivered plus payload still outstanding — the same census discipline 14.2 §12 used, applied per link so a loss on one link cannot be masked by another's surplus.

Check 3 must be per link, and that is the section's point. An aggregate cycle identity can hold while two links individually violate it in opposite directions. Checking the aggregate first is how a real accounting hole survives a regression.

Check 4 is crude and catches the most consequential class of error. A cut carrying more bytes than its capacity allows is impossible, so the finding is either a units error, a boundary mismatch (§17), or a counter bug — never a discovery.

Check 5 is the one most scaling environments lack. It compares the predicted bottleneck from the min-cut model against the observed dominant stall reason. A mismatch means the architectural model and the silicon disagree about what is limiting the system, and that is exactly the situation where the model is about to be used to justify a decision.

38. Coverage

covergroup cg_scalability @(posedge clk);
  option.per_instance = 1;
 
  // --- How many links are active. 1 and max are both essential.
  cp_active_links : coverpoint num_active_links {
    bins one  = {1};
    bins two  = {2};
    bins some = {[3:NUM_LINKS-1]};
    bins all  = {NUM_LINKS};
  }
 
  // --- Load distribution shape (Sections 18 to 22).
  cp_balance : coverpoint balance_bucket {
    bins balanced  = {0};                  // spread within a few percent
    bins skewed    = {1};                  // 60/40-ish
    bins hotspot   = {2};                  // one link dominant
    bins single    = {3};                  // all traffic on one link
  }
 
  // --- Which resource bound each link (Sections 32, 33).
  cp_bind_reason : coverpoint per_link_dominant_stall {
    bins credit = {SR_CREDIT}; bins replay = {SR_REPLAY};
    bins queue  = {SR_QUEUE};  bins arb    = {SR_ARB};
    bins target = {SR_TARGET}; bins idle   = {SR_IDLE};
    bins none   = {SR_NONE};               // full rate, nothing binding
  }
 
  // --- Direction asymmetry (Sections 28, 29).
  cp_direction : coverpoint traffic_direction_mix {
    bins read_heavy  = {0};
    bins write_heavy = {1};
    bins balanced_rw = {2};
  }
 
  // --- Degradation and recovery interacting with scaling.
  cp_degraded_link : coverpoint any_link_degraded;
  cp_link_isolated : coverpoint any_link_isolated;     // 14.5 Section 17
  cp_link_recovering : coverpoint any_link_in_recovery;
 
  // --- Address-interleave boundaries (Section 21).
  cp_interleave_boundary : coverpoint addr_crosses_interleave_granule;
  cp_pathological_stride : coverpoint stride_equals_granule_times_links;
 
  // --- Arbitration pressure (Sections 23 to 25).
  cp_concurrent_requesters : coverpoint concurrent_link_requests {
    bins one = {1}; bins few = {[2:3]}; bins many = {[4:$]};
  }
 
  // --- Crosses that carry the information.
  x_links_balance   : cross cp_active_links, cp_balance;
  x_links_bind      : cross cp_active_links, cp_bind_reason;   // Section 35
  x_dir_bind        : cross cp_direction, cp_bind_reason;      // Section 28
  x_degraded_balance: cross cp_degraded_link, cp_balance;
endcovergroup

Six bins whose value is being non-zero:

cp_active_links.one and .all. §34's scaling efficiency needs a single-link baseline and a full-configuration measurement. A regression that only runs the full configuration cannot compute efficiency at all.

cp_balance.hotspot and .single. §18's imbalance and §22's benchmark-versus-workload failure both require deliberately skewed traffic. Random uniform addressing never produces them.

cp_pathological_stride. The stride equal to granule × links, which sends everything to one link (§21). This is the case a real workload can have accidentally and a synthetic test never generates.

cp_bind_reason — every bin, including none. Each resource must be the binding one on some link at some point, and none establishes the full-rate baseline that §34's efficiency figure is measured against.

x_dir_bind with read_heavy and target. §28's response-path bottleneck — the case where TX looks idle and reads plateau.

And x_degraded_balance. A degraded or isolated link changes the active link count, which §21 showed can break a power-of-two selector. Scaling and Module 14 interact here, and only this cross exercises it.

39. Debug Taxonomy

40. Debug Checklist

1. How many links are active right now? Not configured — active, after any isolation (14.5 §17).
2. What is per-link raw and useful capacity? With the cluster count, which is 1, 2 or 4 (§5).
3. What is aggregate capacity, and at which boundary? (§17).
4. What shared resources do the flows cross? Fabric, controller, response path, target.
5. Which cut is smallest? Compute it — do not guess (§9, §10).
6. What is the oversubscription ratio, and are the inputs correlated? (§11, §12).
7. Is the load balanced? Per-link utilisation and the imbalance metric (§19).
8. How are addresses mapped to links, and what does the workload's histogram look like? (§20, §21).
9. Do responses use a shared path? And is RX measured separately from TX (§28, §29)?
10. Which arbiter is binding, and per which link? (§32, §33).
11. Are queues isolated, shared, or hybrid? And does the blast radius scale with links (§26)?
12. Are credits and replay pools per-link? A merged pool is an over-advertisement bug (§27).
13. Is any link degraded, isolated, or recovering? All three change the active capacity.
14. What is scaling efficiency, and against what baseline? A super-unity result indicts the baseline (§34).
15. Is per-device fairness still acceptable? Aggregate improvement can hide a regression (§31).
16. Does the reference model predict the observed plateau? If not, the model or the instrumentation is wrong (§37).

41. Common Misconceptions

"Twice the links means twice the throughput." It means twice the edge capacity. Whether throughput follows depends entirely on what the links share — and in §10's worked case, half the useful edge capacity is unreachable behind a fabric (§8, §9).

"Aggregate raw bandwidth is automatically usable." Summing capacities is arithmetic that silently assumes traffic can use the links independently and simultaneously. That assumption fails through a shared resource, an address mapping, or a shared response path (§8).

"More links only affect throughput, not timing." A flat central grant network's depth grows with requester count, on the critical path of the shared resource — so adding links can lower Fmax and reduce the throughput of the links that already existed (§23, §25).

"Balanced link widths guarantee balanced traffic." Traffic distribution is the workload's address distribution transformed by the selection function. Equal capacity with a hot page produces a 90/10 split and strands 44% of capacity (§18, §20).

"A central arbiter scales indefinitely." It scales until timing closure fails, which is a synthesis discovery after the architecture is committed. Hierarchy is the structural answer, at the cost of one cycle and a fairness asymmetry that must be sized (§24, §25).

"One shared queue always maximises utilisation." It maximises utilisation and makes head-of-line blocking's blast radius scale with the link count — so a design that worked at two links can fall over at eight (§26).

"Forward-path scaling guarantees read scaling." A read is small out and large in. With a shared response fabric, reads plateau while TX utilisation reads about 5% and looks massively over-provisioned (§28).

"Package throughput and per-device fairness are the same thing." Aggregate can improve 14% while one die loses 70% of its bandwidth. A scaling measurement needs aggregate, per-link, per-device bandwidth and per-device worst case (§31).

"Oversubscription is always bad." It is correct engineering when inputs are bursty and statistically independent, and wrong when they are correlated — and four identical accelerators running the same kernel are the least independent things in a package (§12).

"Synthetic striping proves workload scalability." A linear address sweep measures the interconnect; a hot loop measures the mapping's fit to a workload. Both are real, and only the second predicts production (§22).

42. Understanding Check

43. Summary and What Comes Next

Scalability is the ability to add resources without creating a new shared bottleneck faster than capacity grows. The failure mode is that scaling works arithmetically and not physically.

Six dimensions, and only two are free. Lanes and clusters — the latter in the verified set of 1, 2 or 4 per link — scale without new sharing. Links, dies, traffic and protocols each introduce contention, and traffic scale costs nothing physical while being able to destroy throughput.

Summing link capacities is not a model. It silently assumes traffic can use the links independently and simultaneously, and that fails through a shared resource, an address mapping, or a shared response path.

The min-cut is the tool. Throughput is bounded by the narrowest cut a flow must cross, and for a set of flows by the narrowest cut they all cross. In the worked case a 128 GB/s fabric strands 47% of a 241 GB/s useful edge — and adding a fifth link changes nothing. Every term must be measured at the same boundary, or the minimum is a coincidence rather than a bound.

Oversubscription is correct engineering when inputs are bursty and independent — and four identical engines running one kernel are the least independent things in a package.

Equal capacity does not give equal load. A 90/10 split strands 44% of capacity while an aggregate view reports a comfortable 50% utilisation. The cause is usually the interleave granularity meeting the workload's locality, and the prediction is a static address histogram.

Arbitration stops scaling at the timing boundary, not the logic boundary. A flat grant network lowers Fmax and therefore lowers the throughput of links that already existed. Hierarchy fixes it and introduces a fairness asymmetry between unequal groups that must be sized deliberately.

Credits, replay entries and sequence identities are per link, because the storage they reserve is per link — a merged pool is an over-advertisement bug reached through a package-level abstraction.

And measure four things, not one. Aggregate throughput, per-link throughput, per-device delivered bandwidth, and per-device worst case — because aggregate can improve 14% while one die loses 70%. Per-link, per-reason first-binding-resource attribution is what turns "scaling is poor" into "link 2 is target-limited and link 3 is arbitration-limited."

Scalability tells us how individual links interact. The next chapter stops treating links as independent objects and builds one budget for the entire package: where bandwidth is generated, shared, consumed, and lost.

• 15.4 — Package-Level Performance — the resource graph, the flow matrix, demand against capacity, and the bottleneck the budget predicts.

Browse the full path on the UCIe tutorials index.