Package-Level Performance

Chapter 15.3 established that links share things, and that the narrowest shared cut bounds the total. It found bottlenecks one at a time.

This chapter builds the whole budget at once — and then spends it.

1. The One-Sentence Model

A package is a network of capacity budgets, not a collection of peak numbers. Every edge has a capacity, every node has an ingress, egress and service limit, and a workload is a set of flows that consume several of them at once.

The discipline is bookkeeping, and it is unglamorous and decisive. Most package performance surprises are not discovered in silicon — they are latent in a spreadsheet that summed peaks instead of tracking flows.

2. What This Chapter Owns

The distinction from 15.3 is worth stating, because the two could collapse into one chapter and must not. 15.3 asks "what is the narrowest cut?" and answers with a bound. 15.4 asks "given every flow and every resource, what is the sustainable allocation, is it feasible, and who gets what?" — which needs a matrix, a policy, and a headroom decision that a min-cut does not supply.

Specifically new here: the resource graph with ingress, egress and service limits distinguished; the flow matrix, which is the chapter's most important artefact; utilisation per resource and what a value above 1 means; sustainable allocation and weighted fair share; work-conserving policy and the SVA for it; stranded reservations; the memory and compute bottlenecks that no amount of link bandwidth addresses; directional read and write budgeting; the replay reserve and headroom policy; four worked scenarios; binding-resource attribution at package scope; and the trap that the busiest queue is usually not the bottleneck.

3. Sourcing

4. One Illustrative Package, Used Throughout

The same model for the whole chapter, so the arithmetic accumulates instead of restarting.

Not a canonical topology. No official source defines one (§3). It is a star in 15.3 §6's terms — chosen because a star makes the shared middle explicit, which is what a budget has to capture.

5. Ingress, Egress and Service Are Three Different Limits

A node is not one number, and treating it as one is the first modelling error.

These are independent, and the smallest binds. A memory chiplet with a 64 GB/s ingress, a 64 GB/s egress and a 40 GB/s service limit delivers 40 GB/s — and its link utilisation will read about 63%, which looks like healthy headroom on a device that is completely saturated.

And for a read, the three limits are loaded asymmetrically:

a read at the memory chiplet:
  ingress  carries the request  — small
  service  performs the access  — the real work
  egress   carries the data     — large

So a read-heavy workload can saturate service and egress while ingress reads nearly idle — which is 15.3 §28's response-path asymmetry seen from the target's side rather than the link's.

6. The Package Resource Graph

Read the figure for the two places demand is summed. Three sources converge on the fabric; two memory links converge on one memory service limit. Those are the two resources where feasibility has to be checked, and everything else in the figure has slack by construction in this example.

7. The Resource Table

Capacities. UCIe link figures are derived from verified inputs; everything else is illustrative and marked.

The link capacity, derived once:

per link: 1 standard-package cluster, 16 data lanes, 32 GT/s, 1 bit/transfer   [VERIFIED inputs]
  raw = 32e9 × 16 × 1 = 512e9 bit/s = 512 Gb/s = 64 GB/s per direction

Note the two rows that are not links. The fabric at 160 and memory service at 180 are the resources §6 identified as summing points. Four links at 64 give 256 GB/s of link capacity into a 160 GB/s fabric — the package is already oversubscribed at the fabric before any workload is named, which §10 quantifies.

8. The Flow Matrix

The chapter's most important artefact. Rows are flows; columns are resources; a mark means the flow consumes that resource.

Three things the matrix makes visible that a resource list does not.

The fabric column is full. Every flow crosses it. So the fabric's capacity constrains the sum of all five flows, and it is the first place to check feasibility.

Memory service is loaded by three flows — F1, F2, F3 — none of which knows about the others. A per-flow budget built independently for CPU, A and B will each conclude memory is adequate, and collectively they will oversubscribe it.

And the accelerator links carry two flows each in opposite directions. L_A carries F2's requests outbound and its read data inbound, plus F4's writes outbound. That is a directional budget question, which §14 makes explicit and which a single ✓ per cell cannot express.

Build the matrix before computing anything. It is the artefact that turns "we have four links and a fabric" into a set of checkable inequalities — and it is where an unassigned flow becomes visible.

9. Utilisation Per Resource

utilisation(R) = Σ demand(flow f on R) / capacity(R)
 
Units: (bytes/s) / (bytes/s) = dimensionless           ✓

The > 1.0 case is the one that matters and it is not an error in the model. It is the model correctly reporting that the workload as specified cannot run at full rate. What happens instead is a policy question (§11), and if no policy is stated, the answer is whatever the arbiters happen to do — which is §15's stranded-bandwidth failure.

And the boundary must be consistent across every term (15.1 §17, 15.3 §17). Demand in payload bytes against a capacity in transmitted bytes understates utilisation by the framing factor — about 6% for the verified 68 B flit, which is enough to move a resource from "0.95, tight" to "1.01, infeasible" or back.

10. Worked Feasibility

Demands are illustrative; capacities from §7.

OFFERED DEMAND (payload bytes/s, illustrative)
  F1  CPU → memory read       =  80 GB/s
  F2  A   → memory read       =  40 GB/s
  F3  B   → memory read       =  40 GB/s
  F4  CPU → A  write          =   8 GB/s
  F5  CPU → B  write          =   8 GB/s
 
CONVERT TO TRANSPORT BYTES at the link/fabric boundary (15.1 Section 19)
  framing_efficiency = 64/68 = 0.9412        [VERIFIED 68B flit, 4B overhead]
  transport = payload / 0.9412 = payload × 1.0625
 
  F1 → 85.0    F2 → 42.5    F3 → 42.5    F4 → 8.5    F5 → 8.5   GB/s
 
RESOURCE BY RESOURCE
 
  Fabric (capacity 160, crossed by ALL flows, transport bytes)
    demand = 85.0 + 42.5 + 42.5 + 8.5 + 8.5 = 187.0 GB/s
    utilisation = 187.0 / 160 = 1.169                  → INFEASIBLE (117%)
 
  Memory service (capacity 180, crossed by F1+F2+F3, PAYLOAD bytes)
    demand = 80 + 40 + 40 = 160 GB/s
    utilisation = 160 / 180 = 0.889                    → 89%, tight
 
  L_M0 + L_M1 (capacity 128 total, memory traffic, transport bytes)
    demand = 85.0 + 42.5 + 42.5 = 170.0 GB/s
    utilisation = 170.0 / 128 = 1.328                  → INFEASIBLE (133%)
 
  L_A (capacity 64, carries F2 + F4, transport bytes)
    demand = 42.5 + 8.5 = 51.0 GB/s
    utilisation = 51.0 / 64 = 0.797                    → 80%
 
  A service (capacity 45, PAYLOAD)
    demand = 40 GB/s → 0.889                           → 89%
 
BINDING RESOURCE
  the memory links, at 133% — worse than the fabric at 117%

Four readings, and the ordering is the point.

Two resources are infeasible and the memory links are worse than the fabric. A model that only checked the fabric — the obvious shared resource from Figure 1 — would have named the wrong bottleneck and widened the wrong thing.

Memory service at 89% is not the binding resource here, even though it is the resource most people would guess. The links in front of it run out first. So adding memory devices would not help until the links are widened, which is exactly the kind of ordering error that costs a design cycle.

Note the boundary discipline. Fabric and link demands are in transport bytes; memory service and accelerator service are in payload bytes. Mixing them would have made the memory-service utilisation read 0.944 instead of 0.889 — still not binding, but the error compounds when a resource is near 1.

And the 6.25% framing conversion moved a resource across the line. The fabric at payload demand would be 176/160 = 1.10; at transport demand it is 1.169. Both are infeasible, but the magnitude of the shortfall — and therefore how much widening is needed — differs by the framing factor.

11. Sustainable Allocation Needs a Policy

When demand exceeds capacity, something is throttled. What is a policy decision, and if it is not stated the arbiters decide by accident.

Not a UCIe requirement. No official source I could reach defines a package-level allocation policy (§3). The 8 QoS virtual channels of the management transport are a management mechanism and are not extended here.

Worked weighted share, illustrative weights, on §10's memory-link shortfall:

capacity available (L_M0 + L_M1, transport bytes) = 128 GB/s
weights: CPU 2, A 1, B 1                                    [ILLUSTRATIVE]
total weight = 4
 
ideal weighted shares:
  CPU = 128 × 2/4 = 64.0 GB/s transport   → 64.0 × 0.9412 = 60.2 GB/s payload
  A   = 128 × 1/4 = 32.0 GB/s transport   → 30.1 GB/s payload
  B   = 128 × 1/4 = 32.0 GB/s transport   → 30.1 GB/s payload
 
against demand (payload): CPU wanted 80, A wanted 40, B wanted 40
  CPU gets 60.2 of 80   → 75% of its request
  A   gets 30.1 of 40   → 75%
  B   gets 30.1 of 40   → 75%
 
CHECK: 60.2 + 30.1 + 30.1 = 120.4 payload → /0.9412 = 127.9 ≈ 128 transport  ✓

Every client gets 75% of its request because the weights happen to be proportional to the demands. That is a coincidence of this example and not a property of weighted sharing — with CPU demand at 40 instead of 80, CPU would be capped at 40 by its own demand and its unused 20 GB/s of share becomes the subject of §12.

12. Work-Conserving Allocation

A resource is work-conserving if it never idles while an eligible requester is waiting. Reserved-but-unused capacity should be reclaimable by whoever can use it.

Worked, continuing §11 with a lower CPU demand:

capacity = 128 GB/s transport
weights: CPU 2, A 1, B 1  → shares 64 / 32 / 32
 
demand (transport): CPU 42.5, A 42.5, B 42.5
 
NON-work-conserving (fixed shares):
  CPU takes min(42.5, 64) = 42.5   → 21.5 of its share UNUSED
  A   takes min(42.5, 32) = 32.0   → capped, wants 10.5 more
  B   takes min(42.5, 32) = 32.0   → capped, wants 10.5 more
  delivered = 106.5 GB/s   → 21.5 GB/s STRANDED (17% of capacity)
 
WORK-CONSERVING (redistribute the surplus by weight among the unsatisfied):
  CPU takes 42.5, leaving 21.5 surplus
  A and B are unsatisfied with equal weights → 10.75 each
  A gets 32.0 + 10.75 = 42.75, capped by its demand of 42.5 → 42.5
  B likewise → 42.5
  delivered = 42.5 + 42.5 + 42.5 = 127.5 GB/s   → ~0 stranded

Work conservation recovered 21 GB/s — 17% of the resource — with no additional hardware capacity. That is the largest single improvement available in this chapter, and it is a policy change rather than a silicon one.

And the trade is real, not free. A strict reservation guarantees a client its share even when it is idle, which is what a hard QoS commitment sometimes requires. Work conservation gives that capacity away and must be able to take it back promptly when the reserved client returns — which is a latency requirement on reclamation, and if reclamation is slow the guarantee is soft in practice.

13. SVA — No Idle Cycle With an Eligible Request

// ILLUSTRATIVE work-conservation property. This is a PERFORMANCE property and
// it must be conditional — see the assumptions.
//
//   A1: the sink can accept this cycle
//   A2: at least one requester is eligible (has work AND has its resources)
//   A3: the design's arbitration latency is ARB_LATENCY cycles
//
// Without A3 the property fires on a correct pipelined arbiter.
localparam int ARB_LATENCY = 2;        // ILLUSTRATIVE
 
property p_work_conserving;
  @(posedge clk) disable iff (!rst_n)
    (eligible_requests != '0) && sink_ready
      |-> ##[1:ARB_LATENCY] (grant != '0);
endproperty
a_work_conserving: assert property (p_work_conserving);
 
// The safety companion, unconditional: a grant only ever goes to an eligible
// requester. Catches a reservation that grants an idle client its slot.
property p_grant_only_to_eligible;
  @(posedge clk) disable iff (!rst_n)
    (grant != '0) |-> ((grant & ~eligible_requests) == '0);
endproperty
a_grant_only_to_eligible: assert property (p_grant_only_to_eligible);

Architecture. One conditional performance property and one unconditional safety property.

Why the performance property must carry ARB_LATENCY. A pipelined arbiter cannot grant in the same cycle it observes a request. A property written without the delay window fires on every correct hierarchical arbiter (15.3 §24) and will be disabled — after which it protects nothing, which is 15.2 §36's argument about conditional performance properties.

Why the safety companion is the more valuable of the two. p_grant_only_to_eligible catches §14's non-work-conserving bug directly: a fixed timeslot handed to a client with no work shows up as a grant to a non-eligible requester, immediately, with no assumptions and no tuning. The performance property tells you the resource idled; the safety property tells you why.

Contract. eligible_requests must mean has work and has its resources — a requester with work but no credit is not eligible, and counting it as eligible would make the property fire on a correct design that had nothing it could legally grant.

DV. The performance property needs a stimulus where a resource is idle while someone waits — which a well-behaved arbiter never produces, so it must be created by forcing a non-work-conserving policy and confirming the property fires. A property never demonstrated to fire has not been validated.

14. Wrong Policy — a Reserved Slot for an Idle Client

// WRONG — a fixed rotating timeslot, granted whether or not the client has work.
always_ff @(posedge clk)
  slot_q <= (slot_q == NUM_CLIENTS-1) ? '0 : slot_q + 1'b1;
 
assign grant = (1 << slot_q);          // granted unconditionally

What it costs, illustratively. Four clients, one of them idle, a 128 GB/s resource:

each client gets 1 slot in 4 → 32 GB/s of opportunity each
client D is idle → its 32 GB/s of slots are spent granting nothing
 
delivered = 3 × 32 = 96 GB/s of a 128 GB/s resource
stranded  = 32 GB/s = 25% of capacity, permanently, while clients wait

Four properties of this failure.

It is functionally perfect. Nothing is lost, nothing is corrupted, ordering holds, and every safety assertion in 13.4 §19 passes. The only symptom is that the package is 25% slower than its silicon permits.

It is deterministic and therefore easy to mistake for a capacity limit. The measured ceiling is stable at 96 GB/s, which looks exactly like a resource whose capacity is 96 GB/s. A team without a capacity model will conclude the fabric is smaller than it is.

The waste scales with the number of idle clients. With two of four idle, half the capacity is spent granting nothing. So the failure gets worse in exactly the lightly-loaded conditions where the design should look best.

And p_grant_only_to_eligible catches it on the first idle slot (§13) — a two-line property against a bug that otherwise costs a quarter of a shared resource.

15. Reservation Against Utilisation

The third column is the usual answer, and it is the same structure 13.1 §13 recommended for credit pools and 15.3 §26 for queues. A reserved minimum bounds the damage any one client can do; a shared remainder recovers most of the utilisation.

And the parameter that makes or breaks it is reclaim latency. A shared pool is only useful if a reserved client can get its guaranteed share back promptly when it becomes active again. If reclamation takes longer than the client's tolerance, the guarantee is nominal — the client is told it has a reservation and experiences a delay, which is the worst of both designs.

16. The Memory Bottleneck No Link Can Fix

Illustrative, using §7's capacities.

link capacity into memory (L_M0 + L_M1) = 128 GB/s transport
                                        = 128 × 0.9412 = 120.5 GB/s payload
 
memory SERVICE capacity                 = 180 GB/s payload
 
→ here the LINKS bind, and adding memory devices does nothing.
 
Now widen the memory links to 4 clusters each (verified {1,2,4}, Section 3):
  per link raw = 64 GB/s × 4 = 256 GB/s
  two links    = 512 GB/s transport = 481.9 GB/s payload
 
→ now MEMORY SERVICE binds at 180 GB/s payload
  and 301.9 GB/s of link payload capacity is stranded (63%)

The lesson is the transition, not either endpoint. Widening the links moved the bottleneck from 120.5 to 180 GB/s — a real 49% improvement — and then further widening buys exactly nothing. 15.3 §36's bottleneck migration, with the numbers.

And the cluster count is the verified lever. {1, 2, 4} clusters per link is an enumerated set, so the available widening steps are 64, 128 and 256 GB/s per link and nothing between them. A budget that asks for 1.5× is asking for a configuration the material does not describe.

17. The Compute Bottleneck

The mirror case, and it prevents link-centric thinking in the other direction.

accelerator A service capacity = 45 GB/s payload      [ILLUSTRATIVE]
link L_A capacity              = 64 GB/s transport = 60.2 GB/s payload
 
utilisation of L_A by A's flow at full engine rate:
  45 payload → 45 / 0.9412 = 47.8 transport
  47.8 / 64 = 0.747                        → 75%
 
Doubling L_A to 2 clusters (128 GB/s transport):
  47.8 / 128 = 0.373                       → 37%
  workload throughput: UNCHANGED at 45 GB/s

An accelerator that can consume 45 GB/s does not go faster on a 128 GB/s link. The link was never the constraint, and the extra capacity is shoreline and power spent on nothing.

Two consequences.

Link sizing must be driven by the consumer's service rate, not by what the link technology can offer. The cluster count is a budget decision informed by the die behind it, and {1, 2, 4} means the choice is coarse — so a 45 GB/s engine on a 64 GB/s single-cluster link is a good fit and the next step up is waste.

And this is the most common source of "UCIe performance is poor" reports that are not about UCIe. Low link utilisation with low application throughput means the constraint is the source or the sink (15.1 §38) — and the per-resource utilisation table names which.

18. Directional Budgets

Reads and writes load the package asymmetrically, and one aggregate figure budgets for neither.

Worked on a 70/30 read/write mix, illustrative, 100 GB/s of total payload demand to memory:

70 GB/s of reads, 30 GB/s of writes (payload, at the memory boundary)
 
assume a read = 32 B request out + 64 B data in
       a write = 64 B data out + 8 B completion in
 
READS (70 GB/s of data in → 70/64 = 1.094e9 reads/s):
  outbound = 1.094e9 × 32 B = 35.0 GB/s
  inbound  = 1.094e9 × 64 B = 70.0 GB/s
 
WRITES (30 GB/s of data out → 30/64 = 0.469e9 writes/s):
  outbound = 0.469e9 × 64 B = 30.0 GB/s
  inbound  = 0.469e9 ×  8 B =  3.8 GB/s
 
DIRECTIONAL TOTALS (payload, before framing)
  outbound (TX) = 35.0 + 30.0 = 65.0 GB/s
  inbound  (RX) = 70.0 +  3.8 = 73.8 GB/s
 
after framing (÷0.9412):
  TX transport = 69.1 GB/s
  RX transport = 78.4 GB/s
 
against 128 GB/s per direction on the memory links:
  TX utilisation = 69.1 / 128 = 0.540
  RX utilisation = 78.4 / 128 = 0.613

Two readings.

The directions differ by 13% here, and with a read-only workload they would differ by roughly 2× — 15.3 §29's asymmetry. Reporting a combined 57.7% utilisation would hide which direction is closer to its limit, and under a more read-heavy mix that difference decides which resource binds.

And the mix is a workload property that changes over time. A phase change from 70/30 to 30/70 moves load between directions without changing the total. So a budget must be checked at the extremes of the expected mix, not at its mean.

19. Replay Reserve

Retry traffic consumes capacity and delivers no new payload (15.1 §23), so a budget must either include it or explicitly carry headroom for it.

What the verified BER implies, computed. At the UCIe 3.0 target of 10⁻¹² at 64 GT/s (§3), following 15.1 §23's calculation:

per lane:  64e9 bit/s × 1e-12 = 0.064 errored bits/s
16 lanes:  1.024 errored bits/s
 
if one bit error costs one retransmitted 68 B unit:
  replay bytes/s ≈ 1.024 × 68 = 69.6 B/s
 
as a fraction of a 64 GB/s link:
  69.6 / 64e9 ≈ 1.1e-9         → about 0.0000001%

At the specified BER the replay reserve is numerically negligible. That is the point of specifying a BER — and it is also why the reserve must be a policy rather than a calculation.

Because the number to budget is not the specified rate. It is the rate at which the package must still function:

And the reserve interacts with headroom rather than adding to it. A package at 95% utilisation has 5% of headroom, and if a replay storm consumes 3% of capacity the remaining margin is 2%. §20 treats them as one budget for exactly this reason.

20. Headroom

A package designed to 100% nominal utilisation has no room for anything that actually happens.

Illustrative headroom policy — and the numbers are illustrative because no official source defines a utilisation threshold (§3):

target steady-state utilisation of any shared resource ≤ 0.80   [ILLUSTRATIVE]
  → 20% headroom, allocated roughly as:
      bursts and phase changes      ~10%
      replay reserve                 ~3%      (Section 19)
      degraded-link margin           ~5%      (Section 21)
      background/management          ~2%

Two properties of a headroom policy.

It is a design choice that must be written down. An undocumented headroom target becomes a series of independent judgements by different engineers, and they will not agree. Writing "≤0.80 on shared resources" makes every subsequent capacity argument checkable.

And it is not uniform across resources. A resource with a hard QoS client needs more; a resource carrying only best-effort bulk traffic needs less. A single global percentage is a starting point, not an answer.

21. Scenario A — Balanced

All feeders balanced, no resource near its limit.

demand: F1 60, F2 30, F3 30, F4 8, F5 8 GB/s payload
transport (×1.0625): 63.8, 31.9, 31.9, 8.5, 8.5
 
  Fabric        = 144.6 / 160 = 0.904        → NEAR-LIMIT
  Memory links  = 127.6 / 128 = 0.997        → AT LIMIT
  Memory service= 120.0 / 180 = 0.667        → GREEN
  L_A           =  40.4 /  64 = 0.631        → GREEN
  A service     =  30.0 /  45 = 0.667        → GREEN
 
STATUS: feasible, but the memory links are at 99.7% — no headroom at all.

Feasible and badly provisioned. Nothing is over 1.0, so the workload runs — and any burst, any replay, or any link degradation immediately makes it infeasible. Against §20's 0.80 target, the memory links fail the headroom policy by a wide margin even though they pass the feasibility test.

22. Scenario B — Memory Service Binds

memory links widened to 2 clusters each: 128 GB/s each → 256 GB/s total transport
demand: F1 90, F2 45, F3 45 GB/s payload (reads)
 
  Memory links   = 191.3 / 256 = 0.747       → GREEN
  Fabric         = 191.3 / 160 = 1.196       → INFEASIBLE
  Memory service = 180.0 / 180 = 1.000       → AT LIMIT
 
Widen the fabric to 240 GB/s:
  Fabric         = 191.3 / 240 = 0.797       → GREEN
  Memory service = 180.0 / 180 = 1.000       → BINDING
 
Widen memory links again to 4 clusters (512 GB/s total):
  Memory links   = 191.3 / 512 = 0.374       → GREEN, and irrelevant
  Memory service = 180.0 / 180 = 1.000       → STILL BINDING

The final step is the lesson: adding link capacity to a service-bound system changes nothing. Memory service is at 1.000 in all three configurations. The only remedies are more memory devices, a faster memory technology, or less memory traffic — and none of them is an interconnect decision.

23. Scenario C — One Degraded Link

Verified mechanism: on a standard package, failure handling is width degradation (§3), so a lane failure reduces that link's capacity.

Start from Scenario A (feasible, memory links at 99.7%).
L_M0 degrades from 16 to 8 active data lanes → capacity halves: 64 → 32 GB/s
 
memory link capacity = 32 (L_M0) + 64 (L_M1) = 96 GB/s transport
memory demand        = 127.6 GB/s transport
 
  utilisation = 127.6 / 96 = 1.329           → INFEASIBLE (133%)

What happens next depends entirely on traffic placement, not on the lost capacity.

Two readings, and the second is the one budgets get wrong.

Package throughput loss is not proportional to lost link capacity. 32 GB/s of 128 was lost — 25% of the memory link capacity — and the achievable memory throughput falls from 120.5 GB/s payload to 90.4 GB/s, which is also 25%. But only because memory links were the binding resource. Had memory service been binding at 180, losing 32 GB/s of link capacity would have taken the links from 256 to 224, still above 191, and package throughput would not have changed at all.

So "a degraded link costs the package proportionally" is false in general. It costs nothing if the link was not binding, and it costs more than proportionally if the loss pushes a resource past 1.0 and triggers congestion (13.4 §26).

And on an advanced package the arithmetic differs because spare lanes replace the failed lane rather than removing it (§3) — so capacity is preserved and this scenario does not arise in the same form. The budget must know which package class it is describing.

24. Scenario D — Replay Storm on One Link

L_A's channel degrades badly: 20% of its transmitted bytes become replay
                              [ILLUSTRATIVE — far outside the specified BER]
 
A's demand: 40 GB/s payload → 42.5 GB/s transport of NEW data
with 20% replay: lane traffic = 42.5 / 0.80 = 53.1 GB/s
 
  L_A utilisation = 53.1 / 64 = 0.830        → 83%, up from 66%
  A's DELIVERED payload: unchanged at 40 GB/s IF the link keeps up
 
Push A's demand to 50 GB/s payload:
  transport of new data = 53.1
  lane traffic          = 53.1 / 0.80 = 66.4 GB/s > 64 capacity
  → L_A saturates; A's delivered payload caps at
      64 × 0.80 × 0.9412 = 48.2 GB/s payload

The diagnostic signature is the interesting part.

One link is saturated while every shared resource is half idle. A package-level aggregate would report comfortable utilisation and miss it entirely. And the cause is not capacity at all — it is 14.5's fault domain, so the correct response is to investigate the channel, not to widen anything.

This is the clearest case in the chapter where a performance symptom has a reliability root cause. The budget correctly reports L_A as binding, and the fix is not in the budget.

25. Package Performance State

// ILLUSTRATIVE per-resource performance record. Verification and diagnostic
// structure — NOT a software-visible register map (Section 3).
typedef struct packed {
  logic [63:0] accepted_bytes;    // entered this resource
  logic [63:0] delivered_bytes;   // left it
  logic [63:0] busy_cycles;       // actively transferring
  logic [63:0] blocked_cycles;    // had work, could not proceed
  logic [63:0] idle_cycles;       // no work offered — NOT blocked
  logic [63:0] observation;
} perf_resource_t;
 
perf_resource_t resource_q [NUM_RESOURCES];

Architecture. One record per monitored resource — link, fabric port, memory interface, engine interface — rather than per link. That is the difference from 15.3 §14: this chapter's unit of accounting is the resource, because the flow matrix's columns are resources and the budget is per resource.

State. All observation-window lifetime, cleared together across every resource. A per-resource clear breaks every cross-resource comparison, which is the whole purpose of the array.

Cycle behaviour. Three mutually exclusive cycle classes per resource — busy, blocked, idle — so they partition the observation window:

for each resource r:
  observation[r] = busy[r] + blocked[r] + idle[r]

Contract, and blocked against idle is the distinction that matters. A resource that is idle has nothing offered; one that is blocked has work and cannot proceed. §27's binding-resource attribution depends entirely on that separation — a resource with high busy is working hard, and a resource with high blocked is waiting on something downstream, and only the first is a candidate bottleneck.

Failure. Merging blocked and idle, which makes an under-fed resource indistinguishable from a stalled one — and then §28's misdiagnosis becomes unavoidable.

DV. Assert the partition identity per resource at every window boundary. Then drive a workload that idles one resource and blocks another, and confirm the two are reported differently.

26. Flow Tagging

// ILLUSTRATIVE flow attribution. perf_flow_id is a VERIFICATION AND
// INSTRUMENTATION tag — it is NOT a protocol identifier and must not be
// confused with a transaction ID or a UCIe field.
localparam int NUM_FLOWS = 5;                 // F1..F5 of Section 8
logic [FLOW_W-1:0] perf_flow_id;
 
// Per-resource, per-flow byte accounting: the flow matrix, measured.
logic [63:0] flow_bytes_q [NUM_RESOURCES][NUM_FLOWS];
 
always_ff @(posedge clk)
  if (resource_transfer[r_idx])
    flow_bytes_q[r_idx][perf_flow_id] <= flow_bytes_q[r_idx][perf_flow_id]
                                       + 64'(transfer_bytes);

Architecture. A two-dimensional array indexed by resource and flow — which is literally §8's flow matrix with measurements in the cells instead of ticks.

Why that matters more than it looks. The matrix was built at design time from assumptions about which flows use which resources. This structure measures it. A non-zero cell where the matrix has no tick means a flow is consuming a resource nobody budgeted for — and an unassigned flow is the most common reason a budget under-predicts demand.

State. NUM_RESOURCES × NUM_FLOWS counters, observation-window lifetime. This is expensive — for 8 resources and 5 flows, 40 64-bit counters — so in practice it is a verification structure, or a silicon structure with a narrower flow space or time-multiplexed collection.

Contract. perf_flow_id must travel with the transaction from its origin. A tag assigned at the resource rather than at the source attributes traffic to whoever last touched it, which for a flow crossing four resources gives four different answers.

Failure. Conflating the flow tag with a protocol ID. Protocol IDs are reused and reallocated (12.4); a flow tag must be stable for the flow's lifetime, and using a recycled identity smears one flow's bytes across several rows.

DV. Drive one flow in isolation and confirm only its column accumulates, and only in the resources the matrix marks. A tick in the matrix with a zero cell means the flow does not actually use that resource — the model is wrong; a zero tick with a non-zero cell means an unbudgeted consumer.

27. Binding-Resource Attribution

// ILLUSTRATIVE first-bind attribution at package scope. For each transaction,
// which resource FIRST prevented progress? Aggregated per resource.
logic [63:0] first_bind_count_q [NUM_RESOURCES];
 
always_ff @(posedge clk)
  if (txn_blocked_this_cycle && !txn_already_attributed)
    first_bind_count_q[first_blocking_resource] <=
      first_bind_count_q[first_blocking_resource] + 64'd1;

Architecture. A count, per resource, of how often it was the first thing to block a transaction. This is the package-scope version of 13.5 §35's stall attribution and 15.3 §33's per-link version.

Why first-bind beats occupancy as a bottleneck indicator. Occupancy tells you where work is waiting; first-bind tells you what it is waiting for. Those are different resources, and §28 is the failure that follows from confusing them.

Contract. first_blocking_resource must be evaluated in causal order and attributed once per transaction per blocking episode — not once per cycle, or a long block on one resource dominates the count purely by duration rather than by frequency. Both quantities are useful and they answer different questions, so a design wanting both needs two counters with different names.

Failure. Attributing per cycle rather than per episode, which conflates how often a resource binds with how long it binds. A resource that blocks rarely but for a long time then looks like the dominant bottleneck.

DV. Construct a workload where a known resource is the bottleneck and confirm its first-bind count dominates. Then construct one where two resources alternate and confirm the counts split as expected.

28. Wrong Diagnosis — the Busiest Queue Is the Bottleneck

The most common package-level misdiagnosis.

OBSERVED
  fabric ingress queue: occupancy 95% of depth, almost always full
  memory interface queue: occupancy 12%
 
CONCLUSION DRAWN
  "the fabric ingress queue is the bottleneck — make it deeper"
 
WHAT IS ACTUALLY HAPPENING
  memory service is saturated
  → the memory interface cannot accept
  → the fabric cannot drain into it
  → the fabric's ingress queue backs up
 
  The full queue is DOWNSTREAM-blocked. It is a symptom.

Three properties of this error.

Deepening the full queue makes it worse. More depth means more residence, so latency rises and nothing else changes — 13.4 §28's bufferbloat, arriving through a misdiagnosis rather than through a sizing decision.

The empty queue is the more informative one. The memory interface queue is at 12% because it is being drained as fast as memory can serve — an empty queue in front of a saturated resource is the signature of that resource being the bottleneck. Full queues accumulate behind the bottleneck; the bottleneck itself is starved of nothing.

And the fix is named by first-bind attribution, not by occupancy. §27's counter attributes those blocked cycles to memory service, because that is what the transactions were waiting for. Occupancy would have named the fabric; first-bind names memory — and only one of those two leads to a change that helps.

Queue occupancy locates the queue behind the bottleneck. To find the bottleneck, ask what the blocked transactions were waiting for.

29. Little's Law at Package Scope

in_flight ≈ throughput × latency

Used to size the total outstanding state a package needs, rather than one link's.

Worked, illustrative:

target package read throughput = 120 GB/s payload
average read size              = 64 B
→ read rate = 120e9 / 64 = 1.875e9 reads/s
 
average read latency (15.2 Section 32's decomposition) ≈ 96.5 ns
 
in_flight ≈ 1.875e9 × 96.5e-9 = 181 reads outstanding, package-wide
 
Units: (reads/s) × s = reads              ✓

Two consequences.

181 outstanding reads must be trackable somewhere. Distributed across four links that is about 45 per link, which must fit each link's outstanding-transaction pool (13.5 §16) — and if it does not, the package cannot reach 120 GB/s regardless of any capacity in §7's table. That is a concurrency limit masquerading as a bandwidth question.

And it interacts with the sequence-space bound. 14.3 §25 established that a modulo-N identity scheme constrains the outstanding window to less than half the identity space under that design rule. A package needing 45 outstanding per link needs an identity space comfortably above 90 per link — which is a budget item that lives in the reliability chapter and binds the performance chapter.

Do not over-formalise it. It gives an average requirement; variance means the real pool must exceed it, exactly as 13.1 §4's headroom argument requires.

30. Throughput Fairness

Package throughput can be optimal while one client starves — 15.3 §31 showed the mechanism; here is the metric.

// VERIFICATION-ONLY. Delivered against entitled share, per client.
function automatic real fairness_ratio(int client);
  real entitled = real'(weight[client]) / real'(total_weight)
                * real'(shared_capacity_bytes);
  real delivered = real'(client_delivered_bytes[client]);
  return (entitled == 0.0) ? 1.0 : delivered / entitled;
endfunction

Interpretation:

The fourth row matters. Under oversubscription every client is below its entitlement, and that is the system working correctly — a fairness metric must not report oversubscription as unfairness. The signal to look for is dispersion: some clients at 1.0 and one at 0.3.

Deliberately simple. A normalised share ratio answers the question; a Jain index or Gini coefficient adds mathematics without adding an action. The action is either "adjust the weights" or "the policy is not work-conserving", and the ratio distinguishes them.

31. The Package Budget Worksheet

The deliverable of the chapter. Scenario A's numbers (§21), with status against §20's ≤0.80 target.

All figures in GB/s. Fabric and link rows are transport bytes; service rows are payload bytes (§9's boundary rule).

Three readings.

The package is feasible and fails its headroom policy on two resources. Nothing exceeds 1.0, so the workload runs — and the memory links have 0.3% margin, which is no margin. Feasibility and adequacy are different tests, and a worksheet that only reports the first will ship a design that falls over on the first burst.

The binding resource is the memory links, not the fabric. Despite the fabric being the obvious shared resource in Figure 1. This is why the worksheet is computed rather than reasoned about.

And the remedy is named and bounded. Memory links to 2 clusters each doubles that row's capacity to 256, taking utilisation to 0.498 — and then the fabric at 0.904 becomes the binding resource. The worksheet predicts the next bottleneck before the change is made, which is 15.3 §36's migration used as a planning tool.

32. The Package Performance Scoreboard

// Verification-only reference model. Not synthesisable.
class package_scoreboard;
 
  real capacity[NUM_RESOURCES];         // from the resource table
  real demand[NUM_RESOURCES];           // from the flow matrix × workload
  real weights[NUM_FLOWS];
  longint delivered[NUM_FLOWS];
  longint resource_bytes[NUM_RESOURCES];
 
  // ---- Check 1: byte conservation at a matched SEMANTIC boundary.
  function void check_conservation(int f, longint accepted, longint delivered_f,
                                   longint outstanding, longint failed);
    if (accepted != delivered_f + outstanding + failed)
      $error("FLOW %0d CONSERVATION: accepted %0d != delivered %0d + outstanding %0d + failed %0d",
             f, accepted, delivered_f, outstanding, failed);
  endfunction
 
  // ---- Check 2: no resource carried more than its capacity.
  //      Catches a boundary or units error as well as an impossibility.
  function void check_capacity(int r, longint window_cycles, real bytes_per_cycle);
    real carried = real'(resource_bytes[r]) / real'(window_cycles);
    if (carried > capacity[r] * 1.001)                  // 0.1% tolerance
      $error("RESOURCE %0d EXCEEDED: carried %0.2f > capacity %0.2f — boundary or units bug",
             r, carried, capacity[r]);
  endfunction
 
  // ---- Check 3: the PREDICTED bottleneck matches the OBSERVED first-bind.
  //      This validates the MODEL, which is the chapter's real deliverable.
  function void check_bottleneck();
    int predicted = argmax_utilisation();               // from demand/capacity
    int observed  = argmax_first_bind();                // from Section 27
    if (predicted != observed)
      $error("MODEL MISMATCH: budget predicts resource %0d binds, measurement says %0d",
             predicted, observed);
  endfunction
 
  // ---- Check 4: work conservation — a resource must not idle with demand.
  function void check_work_conserving(int r, longint idle, longint blocked_elsewhere);
    if (idle > 0 && blocked_elsewhere > 0)
      $error("RESOURCE %0d STRANDED: %0d idle cycles while %0d transactions waited",
             r, idle, blocked_elsewhere);
  endfunction
 
  // ---- Check 5: fairness dispersion, not absolute level (Section 30).
  function void check_fairness();
    real mn = 1.0e30, mx = 0.0;
    foreach (delivered[f]) begin
      real ratio = fairness_ratio(f);
      if (ratio < mn) mn = ratio;
      if (ratio > mx) mx = ratio;
    end
    if ((mx - mn) > FAIRNESS_SPREAD_LIMIT)
      $error("FAIRNESS DISPERSION %0.2f exceeds limit — check policy and weights", mx - mn);
  endfunction
 
endclass

Architecture. Five checks in three categories: conservation (1), physical sanity (2), and model validity (3, 4, 5).

Check 3 is the chapter's real deliverable. The budget's whole purpose is to predict which resource binds. A scoreboard that compares the prediction against measured first-bind attribution is what establishes the budget is a model rather than a guess — and a mismatch means the flow matrix is missing a flow, a capacity is wrong, or a boundary was mixed.

Check 4 catches §14's stranded reservation by looking for the specific contradiction: a resource idling while transactions wait for it. That is not detectable from utilisation alone, because a non-work-conserving resource shows a plausible utilisation figure.

Check 5 measures dispersion rather than level, for §30's reason: under oversubscription everyone is below entitlement and that is correct.

33. Coverage

covergroup cg_package @(posedge clk);
  option.per_instance = 1;
 
  // --- Which resource is binding. Every one must bind at least once.
  cp_binding : coverpoint binding_resource {
    bins fabric   = {R_FABRIC};
    bins mem_link = {R_MEM_LINK};
    bins mem_svc  = {R_MEM_SERVICE};
    bins acc_link = {R_ACC_LINK};
    bins acc_svc  = {R_ACC_SERVICE};
    bins none     = {R_NONE};          // nothing binding — the GREEN case
  }
 
  // --- Utilisation regions, including infeasible.
  cp_utilisation : coverpoint util_bucket {
    bins green      = {0};             // < 0.7
    bins working    = {1};             // 0.7 - 0.9
    bins near_limit = {2};             // 0.9 - 1.0
    bins over       = {3};             // > 1.0 — must be reached
  }
 
  // --- Flow activity combinations. The matrix's rows.
  cp_active_flows : coverpoint active_flow_mask {
    bins single   = {[1:1], [2:2], [4:4], [8:8], [16:16]};
    bins pairs    = {[3:3], [5:5], [6:6], [9:9], [10:10], [12:12]};
    bins all_five = {31};
  }
 
  // --- Read/write mix (Section 18).
  cp_rw_mix : coverpoint rw_mix_bucket {
    bins read_only  = {0};
    bins read_heavy = {1};             // ~70/30
    bins balanced   = {2};
    bins write_heavy= {3};
    bins write_only = {4};
  }
 
  // --- Policy behaviour (Sections 12 to 15).
  cp_work_conserving : coverpoint surplus_was_redistributed;
  cp_stranded        : coverpoint capacity_stranded_this_window;
  cp_policy          : coverpoint allocation_policy_sel;
 
  // --- Scenarios C and D.
  cp_degraded_link : coverpoint any_link_width_reduced;
  cp_replay_heavy  : coverpoint replay_fraction_bucket {
    bins spec = {0}; bins elevated = {1}; bins storm = {2};
  }
 
  // --- Fairness.
  cp_client_starved : coverpoint any_client_below_half_entitlement;
 
  // --- Crosses.
  x_util_binding    : cross cp_utilisation, cp_binding;
  x_mix_binding     : cross cp_rw_mix, cp_binding;        // Section 18
  x_degraded_binding: cross cp_degraded_link, cp_binding; // Section 23
  x_replay_binding  : cross cp_replay_heavy, cp_binding;  // Section 24
  x_policy_stranded : cross cp_policy, cp_stranded;       // Section 14
endgroup

Six bins whose value is being non-zero:

cp_binding — every bin including none. Each resource must be the bottleneck at least once, or its capacity figure is never tested against reality. none establishes the all-green baseline that §31's worksheet is compared against.

cp_utilisation.over. The infeasible case, which is where §11's allocation policy actually operates. A regression that never oversubscribes anything never exercises the policy at all.

x_mix_binding with read_only and write_only. §18's directional asymmetry moves which resource binds, and only the extremes of the mix reveal it.

x_policy_stranded with a non-work-conserving policy and stranded capacity. §14's bug, deliberately induced.

x_replay_binding with storm. §24's case, where one link binds while every shared resource is half idle.

And cp_client_starved. §30's dispersion case — which must be reachable, because a policy that cannot starve anyone is either work-conserving with adequate capacity or not being tested under oversubscription.

34. Debug Taxonomy

35. Debug Checklist

1. What flows exist, and is the list complete? An unbudgeted flow is the commonest under-prediction (§26).
2. Which resources does each flow traverse? The matrix — measured, not assumed (§8, §26).
3. What is each resource's capacity, and is it ingress, egress or service? (§5, §7).
4. Are all figures at the same boundary? Payload against transport differs by ~6% for the verified flit (§9).
5. What is the offered demand per flow? And at which extreme of the read/write mix (§18)?
6. What is utilisation per resource? Compute it; do not reason about it (§10).
7. Which resource is over 1.0, and which is highest? The two questions have different answers (§10).
8. Is the allocation work-conserving? Or is capacity stranded on idle clients (§12, §14)?
9. How much headroom does the policy require, and which resources fail it? (§20, §31).
10. Are reads and writes budgeted separately, per direction? (§18).
11. Is a replay reserve included, and sized for a channel below specification? (§19).
12. Is any client starved relative to its entitlement — and is dispersion the signal? (§30).
13. Has any link degraded, and was it binding before it degraded? (§23).
14. Does the budget predict the measured bottleneck? If not, the model is wrong (§32).
15. Is the busiest queue actually the bottleneck, or is it behind it? (§28).
16. What change removes the current bottleneck, and what becomes binding next? (§31).

36. Common Misconceptions

"Package bandwidth is the sum of the UCIe links." It is bounded by the narrowest shared cut. In §10's worked case, four 64 GB/s links face a 160 GB/s fabric and a 128 GB/s memory-link pair — and the binding resource is neither the sum nor the obvious shared resource (§8, §10).

"The busiest queue is the bottleneck." A full queue accumulates behind the bottleneck. The bottleneck's own input queue is typically empty, because it is drained as fast as the resource can serve. Deepening the full queue adds latency and nothing else (§28).

"100% nominal utilisation is good design." It leaves nothing for bursts, replay, arbitration variation, a degraded link, background management traffic, or a read/write phase change. Feasibility and adequacy are different tests (§20, §21).

"Unused reserved bandwidth is always reusable." Only under a work-conserving policy. A fixed timeslot handed to an idle client strands its share permanently — 25% of a resource with one idle client in four — and it is functionally perfect while doing so (§12, §14).

"Memory bandwidth and UCIe bandwidth can be compared directly." Not without a boundary conversion. Link capacity is in transmitted bytes and memory service in payload bytes, and the verified 68 B flit's framing is about 6% — enough to move a resource across the feasibility line (§9).

"Read and write traffic consume the link symmetrically." A read is small out and large in; a write the reverse. A 70/30 mix loads the two directions differently, and a read-only workload differs by roughly 2× (§18).

"Link degradation causes proportional package slowdown." It costs nothing if the link was not binding, exactly proportionally if it was the binding resource, and more than proportionally if the loss pushes a resource past 1.0 and triggers congestion (§23).

"More buffering increases package bandwidth." Buffering absorbs bursts and converts a throughput problem into a latency problem. It does not increase any resource's service rate, and deepening a downstream-blocked queue is the specific version of this that wastes a design cycle (§28).

"High total throughput proves fair performance." Aggregate can be optimal while one client gets a third of its entitlement. Fairness needs per-client delivered bandwidth against entitlement, read as dispersion rather than as level (§30).

"If the links are underutilised, UCIe performance is poor." Low link utilisation with low application throughput means the constraint is the source, the sink, or concurrency. An accelerator that can consume 45 GB/s does not go faster on a 128 GB/s link (§17, §29).

37. Understanding Check

38. Summary and What Comes Next

A package is a network of capacity budgets, not a collection of peak numbers. Every node has an ingress, egress and service limit, and the smallest binds — a device at 63% link utilisation can be completely saturated in service.

Build the flow matrix first. Rows are flows, columns are resources, and it turns an informal architecture into checkable inequalities. It reveals the column that every flow crosses, the resource loaded by flows that do not know about each other, and — when instrumented as a measured per-resource-per-flow array — the unbudgeted flow that makes a budget under-predict.

Compute utilisation per resource, at a consistent boundary. Payload against transport differs by about 6% for the verified 68 B flit, which is enough to move a resource across the feasibility line. In the worked case two resources were infeasible and the binding one was not the obvious shared resource — which is why the worksheet is computed rather than reasoned about.

Above 1.0 is not a model error; it is a policy question. And work conservation is the largest single improvement in the chapter — 17% of a resource recovered by redistributing surplus, with no additional silicon. A fixed slot for an idle client strands 25% of a resource while passing every functional assertion.

Service limits bound what no link can fix. Widening memory links moved the bottleneck from 120.5 to 180 GB/s and then bought nothing; an engine that consumes 45 GB/s does not go faster on a 128 GB/s link. The verified cluster set {1, 2, 4} means the widening steps are coarse, so link sizing must be driven by the die behind it.

Budget by direction, include a replay reserve sized for a channel below specification, and write the headroom policy down. Feasibility and adequacy are different tests: 0.997 utilisation is feasible and has no margin at all.

And the busiest queue is usually not the bottleneck. Full queues accumulate behind it; the bottleneck's own input queue is empty. First-bind attribution names what blocked transactions were waiting for; occupancy names where they were waiting. Only the first leads to a change that helps.

We now know the package's theoretical capacity and its shared bottlenecks. The final chapter asks the most practical question: when the silicon or the simulation delivers less than this model predicts, how do we account for every missing unit of throughput?

• 15.5 — Throughput Analysis — cycle-state classification, one primary stall cause, workload sweeps, bottleneck migration, and driving the unexplained residual to zero.

Browse the full path on the UCIe tutorials index.