Throughput Optimisation

Chapter 13.4 kept the system stable when pressure exists. This chapter asks the complementary question, and it is the one a performance architect is actually paid to answer.

1. The One-Sentence Model

Throughput is determined by the slowest recurring resource cycle in the path, not by the fastest individual block — and almost never by the physical link rate.

A datapath can be functionally perfect, pass every assertion in Modules 9 through 13, never overflow, never starve, never deadlock, and deliver a fraction of the work per second that the same silicon is capable of. There is no correctness failure to find, which is what makes this the hardest class of problem in the module and the reason it needs its own chapter rather than a section.

2. What This Chapter Owns

New here: the three quantities people routinely conflate; the useful-throughput equation and each of its terms; initiation interval as distinct from latency, which is the single most valuable idea in the chapter; elastic versus globally enabled pipelines; outstanding-transaction depth against round-trip latency; the concurrency conjunction that unifies credits, replay entries, tracking slots and queue capacity into one bound; batching and the latency it buys throughput with; arbitration bubbles; the Fmax against pipeline depth trade that makes a one-cycle design slower than a four-cycle one; width against frequency; and stall attribution, which is the instrumentation that turns all of this from guesswork into measurement.

Safety, liveness, and performance are three separate categories, and this chapter is about the third. 13.3 §19 established that a design can satisfy every safety property while deadlocked. This chapter's version is weaker and far more common: a design can satisfy every safety property and every liveness property and still be slow. Nothing is incorrect; nothing is stuck; progress simply happens at a fraction of the achievable rate. No assertion catches that, because it is not a violation of anything.

3. Sourcing

4. Three Quantities, Routinely Conflated

The three move independently, and two of the possible interactions are counter-intuitive.

Throughput can improve while latency gets worse. Add a pipeline stage: each object now takes one more cycle end to end, but the clock can run faster or the stage can accept one object per cycle where it previously could not. More work per second, more time per item. §25 and §26 are this trade in detail.

Latency can improve while throughput gets worse. Reduce batching to send each object immediately: each object arrives sooner and the per-object overhead is paid every time, so fewer objects fit in the same bandwidth. §18 is this trade.

And raw bandwidth can improve with no effect on either. If the binding constraint is outstanding-transaction depth (§13), doubling the link rate changes nothing at all — the design was not waiting on the link.

"Make it faster" is not a specification. Which of the three you mean determines which of a dozen mechanisms is the right one to touch, and several of them make the other two worse.

5. The Useful-Throughput Equation

useful_throughput
  = raw_link_rate
  × encoding_efficiency      // physical-layer overhead: framing, sync, FEC if present
  × payload_efficiency       // semantic bytes per transported byte
  × utilization              // fraction of cycles the path is actually transferring

Keep it symbolic, because only one term has an officially available value.

raw_link_rate — lanes × rate. UCIe 3.0 supports 48 and 64 GT/s per lane with quarter-rate clocking mandated (§3).

encoding_efficiency — physical-layer overhead. Not published in any source I could reach, and not invented here.

payload_efficiency — this one is computable from verified facts, for one case. The Adapter adds a 2-byte header and a 2-byte CRC, and the Consortium describes flits of 68B or 256B. For the 68-byte case that is 64 bytes of payload in 68 bytes transported:

payload_efficiency(68B flit) = 64 / 68 = 0.941        // 94.1%
overhead                     =  4 / 68 = 0.059        //  5.9%

That is arithmetic on quoted numbers, not a specification claim. The overhead breakdown of the 256-byte flit format is not assumed to be the same four bytes — official material does not enumerate it, and a larger flit with different protection may well allocate differently. What the two sizes do establish is the shape of the trade: a larger transport unit amortises a fixed header over more payload, which is §18's argument arriving from the format side.

utilization — the fraction of cycles the path actually moves useful work. This is the term engineers control and the term that is usually worst. Everything from §8 to §31 is about this one factor, and it is the only term in the equation that a bad design can drive to 0.1 while every other term is at its maximum.

The equation's first three terms are largely given to you. The fourth is the design.

6. Physical Bandwidth Is a Ceiling, Not a Prediction

The bandwidth-density figures UCIe publishes are genuinely large — "1300+" at a 45 µm bump pitch, "about 20X what we can achieve with the most efficient PCIe SERDES", rising "by up to 3.24X" at 25 µm (§3). None of that is delivered throughput.

Five mechanisms stand between the ceiling and the floor, and each is a later section:

The trap is that each of the five is invisible in a bandwidth measurement. A link transmitting retransmissions is fully utilised at the physical layer and delivering nothing new; a link idle on outstanding depth shows low utilisation and a perfectly healthy PHY. The two look nothing alike and both read as "the link is not the problem."

7. The Path and Its Binding Resources

Read the figure for one structural distinction, which the rest of the chapter depends on.

The top row sets the initiation interval. The bottom row sets the concurrency. They are different limits with different remedies. If a stage cannot accept one object per cycle, no amount of extra credit helps — the pipeline is the bottleneck. If every stage can accept one per cycle but only one transaction may be outstanding, no amount of pipelining helps — the pool is the bottleneck. Diagnosing which is the entire content of §35.

8. Pipeline Occupancy, and What Fill Costs

Four stages, each able to accept one object per cycle. Feed it continuously:

Latency is four cycles. Throughput is one object per cycle. Both numbers are correct simultaneously, and the fact that they can be is the most useful thing in this chapter.

The fill cost is paid once and it is stages − 1 cycles of lost output. Over a burst of N objects the pipeline delivers N objects in N + stages − 1 cycles, so efficiency is N / (N + stages − 1):

Two conclusions, and the second is the one that gets missed.

Deep pipelines are excellent for long bursts and terrible for short ones. The crossover is entirely a function of the traffic profile, so "how many stages?" cannot be answered without knowing the burst-length distribution.

And the fill cost is paid again after every stall. 13.3 §10 established that a restart wave costs a bubble equal to the propagation latency. So a workload that stalls every 8 objects in a 4-stage pipeline is permanently running at roughly 8 / 11 — 73% — and the fix is not more stages but fewer stalls. The fill cost is a one-time cost only if the pipeline never empties.

9. Latency Is Not Initiation Interval

The distinction §8 demonstrated, stated as the principle it is.

Latency is how long one object takes to traverse. Initiation interval is how often a new object may enter. They are independent, and the second is what determines throughput.

Row three is the common structural bug. Five perfectly pipelined stages and one stage that needs three cycles per object gives an initiation interval of three. The throughput of a pipeline is set by its worst stage's initiation interval, and every other stage's excellence is invisible. This is the "slowest recurring resource cycle" of §1 in its simplest form.

Row four is the same arithmetic applied to a resource rather than a stage, and it is §13. The response-latency loop is a stage, structurally — it just happens to be 50 cycles long and located on the other die.

Why this matters beyond throughput. It is the reason the correct answer to "our pipeline has 6 cycles of latency, is that too slow?" is "for what?" If the design issues one transaction and waits, latency is everything. If it issues thousands of independent transactions, latency is almost irrelevant and only the initiation interval matters. The same silicon is fast or slow depending entirely on the concurrency available in the workload.

10. An Elastic Pipeline Stage

// ILLUSTRATIVE elastic pipeline stage — per-stage valid/ready, so a stall
// affects only the stages behind it. This is the standard skid-free form:
// the stage holds one object and accepts a new one when it is either empty
// or being emptied this cycle.
module elastic_stage #(parameter int W = 64) (
  input  logic          clk,
  input  logic          rst_n,
  // upstream
  input  logic          in_valid,
  output logic          in_ready,
  input  logic [W-1:0]  in_data,
  // downstream
  output logic          out_valid,
  input  logic          out_ready,
  output logic [W-1:0]  out_data
);
  logic         full_q;
  logic [W-1:0] data_q;
 
  // Accept when empty, or when the held object departs this cycle.
  assign in_ready  = !full_q || out_ready;
  assign out_valid = full_q;
  assign out_data  = data_q;
 
  wire load    = in_valid  && in_ready;      // an object enters
  wire unload  = out_valid && out_ready;     // an object leaves
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      full_q <= 1'b0;
    end else begin
      // ONE writer, all four combinations. The 2'b11 arm is a simultaneous
      // load and unload — occupancy is unchanged and the data MUST update.
      unique case ({load, unload})
        2'b10  : full_q <= 1'b1;
        2'b01  : full_q <= 1'b0;
        2'b11  : full_q <= 1'b1;             // stays full, new object
        default: full_q <= full_q;
      endcase
    end
  end
 
  always_ff @(posedge clk)
    if (load) data_q <= in_data;             // qualified by load, always
endmodule

Architecture. One object of storage per stage plus a valid bit, with in_ready depending on out_ready. The stage is elastic in that it decouples the upstream's decision from the downstream's — but note it does not decouple the combinational path: in_ready is combinational from out_ready, which is exactly 13.3 §5's ready-chain problem. Breaking that requires a two-entry skid buffer, which costs one more object of storage and is the standard remedy.

State. full_q and data_q, per-object lifetime — valid only while full_q.

Cycle behaviour. The 2'b11 arm is the one that matters and the one most often wrong. When an object arrives in the same cycle another departs, occupancy is unchanged but the data must update. A design that writes full_q <= 1'b1 correctly but forgets to qualify the data register on load will hold the old object and drop the new one — a silent data loss with a perfectly legal valid bit, which is the same shape of bug 13.2 §6 found in occupancy updates.

Contract. in_ready must not depend on in_valid, or the handshake combinationally self-references and can deadlock in simulation and glitch in synthesis. This is the standard valid/ready rule and it is worth restating because the "accept when empty or emptying" form makes it easy to accidentally include in_valid in the expression.

Failure. Two. Omit || out_ready from in_ready and the stage accepts only when empty, so the initiation interval becomes 2 rather than 1 — exactly half throughput, no correctness symptom. Forget to qualify data_q on load and objects are silently overwritten during full-rate operation.

DV. Sustain in_valid and out_ready and check one transfer per cycle indefinitely (§12). Then check the 2'b11 case explicitly by driving a full stage with both signals high and confirming the new object emerges next cycle. Then stall out_ready for many cycles and confirm the held object is unchanged.

11. Wrong RTL — The Global Enable

// WRONG — one enable for the whole pipeline. Every stage advances only when
// EVERY stage can advance, so any single stall freezes the entire pipeline
// including stages with slack.
wire pipe_advance = s1_ready && s2_ready && s3_ready && s4_ready && sink_ready;
 
always_ff @(posedge clk) begin
  if (pipe_advance) begin
    s2_data <= s1_data;
    s3_data <= s2_data;
    s4_data <= s3_data;
  end
end

Three problems, in increasing order of severity.

It throws away slack that exists. If stage 4 stalls for 5 cycles, stages 1–3 stop too — even though stage 2 is empty and could have accepted an object from stage 1. The pipeline had capacity for three more objects and used none of it.

It makes every stall cost the full fill again. A globally enabled pipeline empties nothing and fills nothing during a stall — it freezes. On release, throughput resumes immediately, which sounds good, but the pipeline never absorbed anything during the stall, so the upstream backed up for the entire duration. An elastic pipeline would have absorbed stages objects before propagating the stall at all, which is 13.3 §10's wave: the wave is a feature, and a global enable eliminates it.

And it creates a combinational path across the entire pipeline. pipe_advance is the AND of every stage's ready, so the critical path runs from the sink's ready through five terms to every stage's clock enable. Fmax degrades with pipeline depth — the design gets slower as you add stages, which is the exact opposite of the intent. This is 13.3 §5's global ready chain in its purest form.

Quantified. Sink stalls one cycle in four. Global enable: throughput is 0.75 objects/cycle and the upstream sees backpressure 25% of the time. Elastic, 4 stages: the pipeline absorbs the stalls into its own occupancy and the upstream sees backpressure only when the pipeline is genuinely full — throughput is still 0.75 (the sink is the limit and nothing changes that), but the upstream is decoupled, so a bursty producer with its own buffer sees no stall at all. Elasticity does not raise the ceiling; it stops the ceiling from propagating.

12. SVA — No Bubble Under Sustained Supply and Demand

// The throughput contract: with a source that always has an object and a sink
// that always accepts, the pipeline must reach and hold its target initiation
// interval after the fill.
//
// Note this is a PERFORMANCE property expressed as a bounded safety property.
// It is not liveness — it makes a claim about RATE, under assumptions.
localparam int STAGES  = 4;
localparam int TARGET_II = 1;
 
// Environment assumptions, stated explicitly.
assume property (@(posedge clk) disable iff (!rst_n) src_always_ready |-> in_valid);
assume property (@(posedge clk) disable iff (!rst_n) sink_always_ready |-> out_ready);
 
// After the fill window, every cycle must produce an output.
property p_no_bubble_at_full_rate;
  @(posedge clk) disable iff (!rst_n)
    (src_always_ready && sink_always_ready)[*STAGES]
      |=> (out_valid && out_ready);
endproperty
a_no_bubble_at_full_rate: assert property (p_no_bubble_at_full_rate);
 
// A weaker, more portable form: a windowed throughput monitor. Over any
// WINDOW cycles of sustained supply and demand, at least WINDOW-STAGES+1
// transfers must complete. Survives II > 1 designs by reparameterisation.
localparam int WINDOW = 64;
int unsigned xfer_in_window;
always_ff @(posedge clk or negedge rst_n) begin
  if (!rst_n)                       xfer_in_window <= 0;
  else if (window_start)            xfer_in_window <= 0;
  else if (out_valid && out_ready)  xfer_in_window <= xfer_in_window + 1;
end
 
property p_windowed_throughput;
  @(posedge clk) disable iff (!rst_n)
    window_end && sustained_supply_and_demand
      |-> (xfer_in_window >= (WINDOW - STAGES + 1) / TARGET_II);
endproperty
a_windowed_throughput: assert property (p_windowed_throughput);

Architecture. Two forms of the same claim, because they fail differently and both are worth having. The first is a strict per-cycle property that catches §10's missing || out_ready immediately. The second is a windowed monitor that tolerates a design whose target initiation interval is greater than one and still catches a systematic bubble.

Why the assumptions must be explicit. Without src_always_ready and sink_always_ready, this property is simply false — a pipeline with nothing to do produces no output, correctly. A performance property with unstated environment assumptions will fail on correct behaviour, get weakened, and then catch nothing. That is the same argument 13.4 §20 made for bounded fairness, and it applies with more force here because performance properties are the first thing disabled when a regression gets noisy.

Contract. The property constrains the pipeline in isolation and needs no reference model. It is the cheapest possible guard against the single most common throughput bug — an initiation interval of two in a design intended to have one.

Failure it catches. §10's in_ready = !full_q without the emptying term: the strict property fails on the second cycle of sustained traffic, with a counterexample two cycles long.

DV. The antecedent needs directed stimulus. Random traffic almost never sustains both supply and demand for STAGES consecutive cycles, so this property can pass vacuously through an entire random regression while the design runs at half rate. Cover the antecedent, not the property.

13. Outstanding Transactions

The second of §7's two limits, and the one that produces the most dramatic gap between capability and delivery.

A transaction with a response has a round trip. If the response takes L cycles and only one transaction may be outstanding, then the issue rate is one per L cycles regardless of how wide the link is, how deep the queues are, or how many credits exist.

throughput ≤ outstanding_depth / response_latency

With numbers. A 50-cycle response latency:

Idealised, and deliberately so: it assumes independent transactions, a fixed latency, and no other binding resource. Real latency varies, which means the depth needed to reliably stay busy exceeds the depth needed to stay busy on average — the same headroom argument 13.1 §4 made for credits, for the same reason.

The two rows worth dwelling on. At depth 1, 98% of the link is idle and every assertion passes. At depth 64 the extra 14 slots buy nothing, because the constraint moved to the pipeline — which is what "optimisation changes the bottleneck rather than removing it" means in practice, and the reason §35's stall attribution has to be re-read after every change.

There is a published latency anchor worth noting. The Consortium states a latency target of "2ns for round-trip from FDI to bump" on a 16 GT/s link, described as "an estimate based on 16G frequency and a 2G internal clock" (§3). That is one segment of the path, not the transaction round trip — it does not include the far die's Adapter, its Protocol Layer, or any memory or coherence lookup. Quoting it as the response latency would be badly wrong. What it does tell you is that the UCIe portion of the round trip is engineered to be small relative to the semantic work at either end, which means the outstanding depth a design needs is usually set by the far side's service time rather than by the interconnect.

14. Little's Law, Just Enough of It

required_concurrency ≈ throughput × latency

That is the whole tool. It is the same equation as the bandwidth-delay product 13.1 §4 used for credits, with different units — there, permissions in flight; here, transactions in flight. Recognising them as one relationship is worth more than either individually:

Every row is rate × latency, and every row is a separate pool that can bind independently. That is §17.

Two cautions on using it. It gives an average, so it is a floor rather than a target — variance requires headroom on top. And it says nothing about which resource is scarce; it only tells you how much of each you would need if that one were the only limit.

15. Wrong Design — A Single Outstanding Transaction

// WRONG for any path with meaningful response latency — and note that
// NOTHING here is incorrect. It is a pure architectural throughput failure.
localparam int MAX_OUTSTANDING = 1;
 
logic busy_q;
 
assign can_issue = !busy_q;
 
always_ff @(posedge clk or negedge rst_n) begin
  if (!rst_n)             busy_q <= 1'b0;
  else if (issue)         busy_q <= 1'b1;
  else if (response_seen) busy_q <= 1'b0;
end

What is right about it. Ordering is trivially maintained. No identity space is needed, so no tag allocator, no reflection checks, no 12.4 lifetime table. There is exactly one outstanding transaction, so it is impossible to mismatch a response to a request. It is the simplest correct design, and correctness is not the problem.

What it costs. With a 50-cycle response latency, 49 of every 50 cycles are idle. The scoreboard passes. Every assertion in Modules 9 to 13 passes. Occupancy never exceeds 1. Credits never run out — the design cannot consume enough credits to matter, which is itself a diagnostic signature: a credit counter that never drops below its maximum means concurrency is limited by something else entirely.

And it is invisible to functional verification by construction, because functional verification asks whether the response matched the request, and it always does.

This is the archetype of the chapter: a design with no bugs, no stalls it did not cause, no illegal states, and 2% of the achievable throughput.

The fix is not "increase the constant." Supporting N outstanding transactions requires the identity space, the tracking table, and the response-matching discipline Module 12 built — the tag allocator, the reflection check, the lifetime accounting. The throughput is bought with complexity, and the complexity is where the correctness bugs live. §16 is the sizing question; Module 12 is the correctness cost.

16. Sizing the Outstanding Pool

// ILLUSTRATIVE outstanding-transaction occupancy. This reuses Module 12's
// tracking table and Chapter 13.2 Section 5's counter width discipline — the
// point here is the PERFORMANCE consequence, not a second implementation.
localparam int MAX_OUTSTANDING = 32;
localparam int OS_W = $clog2(MAX_OUTSTANDING + 1);   // +1: MAX must be held
 
logic [OS_W-1:0] outstanding_q;
logic [OS_W-1:0] min_slack_q;      // low-water mark of remaining slots
 
assign os_slots_free = (outstanding_q < OS_W'(MAX_OUTSTANDING));
 
always_ff @(posedge clk or negedge rst_n) begin
  if (!rst_n) begin
    outstanding_q <= '0;
    min_slack_q   <= OS_W'(MAX_OUTSTANDING);
  end else begin
    unique case ({issue_accepted, transaction_retired})
      2'b10  : outstanding_q <= outstanding_q + OS_W'(1);
      2'b01  : outstanding_q <= outstanding_q - OS_W'(1);
      default: outstanding_q <= outstanding_q;     // includes the 2'b11 case
    endcase
 
    // The sizing instrument: how close did we ever come to running out?
    if ((OS_W'(MAX_OUTSTANDING) - outstanding_q) < min_slack_q)
      min_slack_q <= OS_W'(MAX_OUTSTANDING) - outstanding_q;
  end
end

Architecture. An occupancy counter and, more importantly, a low-water mark on the remaining slack. The counter tells you the current state; the low-water mark tells you whether the pool is sized correctly, and it is the only instrument that answers that question without a sweep.

State. outstanding_q has per-cycle meaning and transaction-set lifetime. min_slack_q is sticky for the observation window and is a pure diagnostic.

Cycle behaviour. The 2'b11 case falls into default deliberately: issue and retire in the same cycle is a net-zero change. Writing that as two separate statements loses one of the two events — the identical hazard 13.1 §9 and 13.2 §6 both call out, appearing a third time because it is the most repeated bug in resource counters.

Contract. os_slots_free is one term in §17's conjunction, and it must be named separately there rather than folded into a generic "cannot issue" — otherwise §35 cannot attribute a stall to it.

Reading the low-water mark, which is the actionable part:

Failure. Sizing from average latency rather than from the tail. A pool sized for mean latency spends a substantial fraction of time at zero whenever latency exceeds the mean, which is half the time by definition.

DV. Sweep the pool depth against a fixed response latency and plot achieved throughput — the curve should rise linearly and then flatten, and the flattening point is where the bottleneck moved. If it flattens well below the pipeline's capability, something else in §17's conjunction is binding.

17. The Concurrency Conjunction

This is the synthesis of Modules 9, 12 and 13, and it is the single most useful expression in the chapter.

effective_concurrency ≤ min(
    available_remote_credits,     // Ch 13.1 — remote storage reserved
    outstanding_slots,            // Ch 12   — local tracking records
    replay_entries,               // Ch 9.4  — retention until confirmed
    local_queue_capacity          // Ch 13.2 — somewhere to put it now
)

Four independent pools, four different owners, four different rate × latency products (§14) — and the effective concurrency is the minimum, not the average and certainly not the sum.

Three consequences, and they are why this expression is worth memorising.

Provisioning three pools generously and one poorly gives you the poor one's throughput. Concurrency is a minimum. Doubling credits when the binding pool is the outstanding table changes nothing measurable — and this is the most common wasted optimisation in interconnect design.

The binding pool changes with the traffic shape, so there is no single answer. 13.1 §14 established the mechanism: a large write consumes many credits and one tracking entry, while a small read consumes one credit and one tracking entry. So a large-write workload binds on credits and a small-read workload binds on the table, in the same design, with the same parameters. Any sizing exercise that uses one workload has sized for one workload.

And the symptom is identical in all four cases: the link is not busy and the local side cannot issue. Only per-pool instrumentation separates them, which is exactly why §34's stall counters are per-reason rather than a single stall count.

// The gate, with every term named. Named terms are what make Section 35's
// attribution possible; a collapsed "can_issue" makes it impossible.
assign can_issue = credit_available      // 13.1
                && os_slots_free         // Section 16
                && replay_space          // 9.4
                && queue_space;          // 13.2

18. Batching — What It Amortises and What It Costs

The registry names "batched FLITs" for this chapter, so the framing has to be precise about what is known.

What batching amortises:

What batching costs:

The core asymmetry, and it is what makes a fixed batch size wrong:

Batching costs latency in inverse proportion to the arrival rate — exactly when the throughput benefit is smallest. At high load, batches fill instantly and cost almost nothing. At low load, batches fill slowly and cost the most, while the bandwidth they save is not needed.

That sentence is the argument for §21's adaptive policy and for §20's flush condition, and it is why "just use a bigger batch" is wrong in both directions.

19. A Batch Assembler

// ILLUSTRATIVE batch assembler. BATCH_MAX and FLUSH_TIMEOUT are DESIGN
// CHOICES and are NOT UCIe-defined values — UCIe publishes no batching
// semantics (see the callout in Section 18).
localparam int BATCH_MAX     = 8;    // ILLUSTRATIVE
localparam int FLUSH_TICKS   = 24;   // ILLUSTRATIVE
localparam int BATCH_W       = $clog2(BATCH_MAX + 1);
localparam int TIMER_W       = $clog2(FLUSH_TICKS + 1);
 
logic [BATCH_W-1:0] batch_count_q;
logic [TIMER_W-1:0] flush_timer_q;
logic               flush_pending_q;
 
// Three independent reasons to send. ANY of them is sufficient; having none
// of them is Section 20's bug.
wire batch_full   = (batch_count_q == BATCH_W'(BATCH_MAX));
wire timed_out    = (flush_timer_q == TIMER_W'(FLUSH_TICKS));
wire urgent       = append_valid && append_is_urgent;   // latency-critical class
 
assign flush_now = (batch_count_q != '0)
                && (batch_full || timed_out || urgent || drain_requested);
 
wire sent = flush_now && downstream_ready;
 
always_ff @(posedge clk or negedge rst_n) begin
  if (!rst_n) begin
    batch_count_q   <= '0;
    flush_timer_q   <= '0;
    flush_pending_q <= 1'b0;
  end else begin
    // Count: one writer, all four combinations of append and send.
    unique case ({append_accepted, sent})
      2'b10  : batch_count_q <= batch_count_q + BATCH_W'(1);
      2'b01  : batch_count_q <= '0;
      2'b11  : batch_count_q <= BATCH_W'(1);   // sent, then this one starts
      default: batch_count_q <= batch_count_q;
    endcase
 
    // Timer measures how long the OLDEST object has waited. It therefore
    // starts on the first append into an empty batch and clears on send.
    if (sent)                                   flush_timer_q <= '0;
    else if (batch_count_q == '0 && !append_accepted) flush_timer_q <= '0;
    else if (flush_timer_q != TIMER_W'(FLUSH_TICKS))  flush_timer_q <= flush_timer_q + TIMER_W'(1);
 
    flush_pending_q <= flush_now && !downstream_ready;
  end
end

Architecture. An accumulator with three independent flush reasons, and the plurality is the design. A single reason — "full" — is §20's bug. Two reasons, full and timeout, is the minimum viable design. The third, urgency, is what lets a latency-critical class bypass the batching entirely, which is the software-visible difference between a link that is fast on benchmarks and one that is fast on interactive work.

State. batch_count_q and flush_timer_q have per-batch lifetime; flush_pending_q is per-cycle and exists only to hold the decision across a downstream stall.

Cycle behaviour. Two subtleties. The 2'b11 arm: sending and appending in the same cycle means the batch departs and the new object starts the next batch, so the count becomes 1 and not 0 — writing '0 there loses an object. And the timer's start condition: it must measure the wait of the oldest object, so it starts when the batch becomes non-empty rather than on every append. A timer that restarts on each append can be held off indefinitely by a steady trickle of arrivals, which reintroduces §20's unbounded wait through a different door.

Contract. Downstream must accept a partial batch. If the receiver requires exactly BATCH_MAX, the timeout cannot flush and the design is back to §20 regardless of the RTL here. The framing must permit partial units, and this is precisely the kind of thing to verify against your revision rather than assume.

Failure. Three, all latency failures with no correctness symptom. No timeout: unbounded wait at low load (§20). Timer restarting on append: unbounded wait under a trickle. Timeout too long: bounded but bad tail latency, which is the one that survives to silicon because it is not obviously broken.

DV. Drive exactly one object and check it departs within FLUSH_TICKS. Drive a trickle at a rate slower than FLUSH_TICKS and check every object departs. Drive one object per FLUSH_TICKS - 1 cycles and check the timer does not reset. Drive BATCH_MAX objects in consecutive cycles and check the batch departs on the count rather than the timer. Cover partial-batch sends at every count from 1 to BATCH_MAX − 1.

20. Wrong Batching — Waiting for a Full Batch

// WRONG — send only when the batch is full. At low arrival rates, objects
// wait for arrivals that may never come.
assign flush_now = (batch_count_q == BATCH_W'(BATCH_MAX));

The failure, with numbers. BATCH_MAX = 8. Traffic arrives at one object per 1,000 cycles.

And if traffic stops after object 5, the first five never depart at all. That is not a latency problem; it is a hang, produced by an optimisation.

Four properties make this bug characteristic of throughput work.

It is invisible at high load. Run a bandwidth benchmark and batches fill in 8 cycles; latency is excellent and throughput is at the ceiling. The bug requires low load to appear, and low load is what performance testing tends to skip.

It is a correctness failure at the system level while being a latency failure locally. The object is not lost — it is in a buffer, valid, waiting. Every assertion passes. The conservation scoreboard balances: nothing is missing, it just has not arrived. And the transaction that generated it times out, so the failure surfaces at 12.4's timeout, several layers away from the batch assembler.

It has the same shape as 13.4 §21's drain hazard and a different cause. Both are "waiting for a condition that the workload may not supply." The general lesson: any accumulation whose release depends only on more input needs a second release condition that does not.

And the fix is not "make the batch smaller." A batch of 2 has the same bug — object 1 waits 1,000 cycles instead of 7,000. Only an end condition independent of arrivals fixes it: a timer, an idle indication, an explicit flush, or an urgency bypass. §19 has three.

21. Adaptive Batching

Since the cost of batching is highest exactly where its benefit is lowest (§18), the batch size should be a function of load.

// ILLUSTRATIVE load-adaptive batch target. Uses the pressure signal the
// congestion policy already computes (13.4 Section 9) rather than measuring
// arrival rate again — one observation, two consumers.
always_comb begin
  unique case (cg_state_q)
    CG_NORMAL   : batch_target = BATCH_W'(1);           // sparse: send at once
    CG_PRESSURED: batch_target = BATCH_W'(BATCH_MAX/2); // load rising: amortise
    default     : batch_target = BATCH_W'(BATCH_MAX);   // loaded: maximise
  endcase
end
 
wire batch_ready = (batch_count_q >= batch_target);

Architecture. Reuse of the congestion state as a load proxy. This is a genuine architectural economy: the design already computes a hysteretic, debounced measure of load pressure, and re-deriving arrival rate with a second filter would cost state and could disagree with the first.

Why the congestion state is the right proxy. Batching helps when there is queueing to amortise over — and queueing is precisely what the congestion FSM measures. At CG_NORMAL there is nothing to batch, so batch of 1 is optimal: minimum latency at no throughput cost, because the bandwidth was not needed. At CG_CONGESTED the queue is full of objects that are waiting anyway, so batching them costs almost no additional latency and buys real efficiency.

Cycle behaviour. batch_target changes only when the congestion state changes, which is already debounced by the hysteresis of 13.4 §10. That matters: a batch target derived from a raw occupancy comparison would oscillate exactly as 13.4 §11's policy did, and an oscillating batch target produces objects that are sometimes batched and sometimes not, which is the worst case for tail latency.

Contract. batch_target must never exceed BATCH_MAX and never be zero. Zero would flush an empty batch; exceeding BATCH_MAX would make batch_ready unreachable and reintroduce §20.

Failure. Deriving the target from an undebounced signal (oscillation, per above). Or setting the CG_NORMAL target above 1, which reintroduces low-load latency for a throughput benefit that is unmeasurable at low load.

DV. Sweep offered load across the full range and check that measured latency at low load matches the unbatched design, while measured efficiency at high load matches the fully batched one. This is a performance regression rather than an assertion, and §37 is where it belongs.

22. Arbitration Bubbles

Arbitration costs cycles, and whether that cost matters is a ratio.

If selection takes one cycle and a granted object occupies D cycles, the arbitration overhead is 1 / (D + 1):

Short objects make arbitration the dominant cost. A design transferring single-cycle objects through a one-cycle arbiter delivers half the throughput of the same datapath with the arbitration overlapped. This is one of the few places where a genuinely large factor is available.

Two remedies, with different risk.

Overlap the decision with the current transfer. Compute the next grant during the final beat of the current object, so the next transfer starts in the following cycle. Safe when the decision does not depend on the outcome of the current object.

Pipeline the arbiter itself. A registered grant adds a cycle of latency and permits a higher clock, which is §25's trade appearing inside the arbiter.

And the hazard that both remedies share is §23: a decision made early is a decision made with incomplete information.

23. Wrong Lookahead Arbitration

// WRONG — the next grant is computed and committed before the current object
// is known to be retirable, and before ordering dependencies are resolved.
always_ff @(posedge clk) begin
  next_grant_q <= arbitrate(request);        // decided during current transfer
  if (last_beat) begin
    active_grant_q <= next_grant_q;          // committed unconditionally
  end
end

Two distinct failures, and the second is worse.

The current object may not retire. If the transfer being overlapped fails its final check — a CRC mismatch (Module 14), a resource that disappears, a stall that extends past the last beat — then the design has already committed to the next grant. The next object launches while the previous one is still occupying the resource, and depending on the datapath, they interleave, one is dropped, or the framing is corrupted.

Ordering can be violated. If the granted class carries a transaction with an ordering dependency on something still in flight, deciding early means deciding before the dependency is resolved. 12.2 established which orderings must be preserved; a lookahead arbiter that does not consult the dependency state can reorder two operations that were required to complete in order — and the resulting bug appears as a coherence or consistency failure many cycles later, with no evidence pointing at the arbiter.

The safe form separates computing from committing:

// The decision may be computed early; it may be COMMITTED only when the
// preconditions actually hold.
wire commit_ok = last_beat && current_retirable && ordering_clear;
 
always_ff @(posedge clk) begin
  next_grant_q <= arbitrate(request);        // speculative — cheap, discardable
  if (commit_ok) active_grant_q <= next_grant_q;
end

Speculation is free until it becomes a commitment. The bug is never computing early; it is acting on the computation before its preconditions are known.

24. Precomputation, and Where Speculation Must Stop

Throughput work systematically moves work earlier: address decode, metadata generation, next-grant selection, partial CRC accumulation.

And UCIe's own format design is evidence that this is the right instinct. The Consortium states that "performance critical bits (such as Flit Header) appear early on in the Flit to enable pipelined operations" (§3). That is a protocol-format decision made specifically so that implementations can begin work before the whole unit has arrived — routing on a Protocol ID, starting a CRC, selecting a destination queue. The format is cooperating with the pipeline.

What precomputation costs:

And the boundary that must not be crossed:

Precomputation may produce a result early. It must never produce a commitment early. The moment a speculative result becomes visible to a consumer that will act on it, it has stopped being speculation.

Three concrete instances of that boundary, which recur throughout the stack:

• Routing may be computed early; the object must not be delivered early. Module 14 makes this precise: an object whose integrity check has not completed must not reach a semantic consumer.
• A grant may be computed early; it must not be committed early (§23).
• A credit may be predicted early; it must not be consumed early. 13.1 §7's conservation law is exact, and a speculative consume that is not backed out on a discard is a permanent leak.

25. Fmax Against Pipeline Latency

The trade that makes a one-cycle design slower than a four-cycle one.

A single-cycle implementation has the best possible latency in cycles and may have a terrible clock period. If decode, route, mux, CRC and enqueue all sit in one combinational path, the period must cover all five. The design has latency 1 and Fmax set by the sum.

Splitting the path into four stages:

Two conclusions, and they point in different directions, which is the point.

Throughput improves unambiguously. The initiation interval stays at one object per cycle and the cycle is shorter, so objects per second rises roughly in proportion to the frequency gain. This is the clearest win in the chapter and it is why deep pipelines exist.

Latency in time may improve or worsen, and which one depends on how evenly the path splits. Four stages at a quarter of the delay each gives identical latency in time and four times the throughput — the ideal. Four stages where one holds 70% of the delay gives 4 × 0.7 = 2.8 times the original latency in time, and only a 1.43× frequency gain. The quality of the split is the entire result, which is why "add a pipeline stage" is not a plan and "find where the path actually is" is.

No timing numbers are claimed here, deliberately. Real periods depend on process, library, floorplan and constraint set, and a fabricated number would be worse than none.

26. The Pipeline-Register Tradeoff, Concretely

// ILLUSTRATIVE single-cycle form. All five operations in one path. Best
// latency in cycles, worst achievable clock period.
always_comb begin
  decoded   = decode(in_flit);                    //  ─┐
  route_sel = route(decoded.protocol_id);         //   │ one combinational
  muxed     = mux_select(route_sel, in_flit);     //   │ path — the period
  crc_val   = crc_next(crc_q, muxed);             //   │ must cover all five
  enq_data  = pack(muxed, crc_val);               //  ─┘
end

// ILLUSTRATIVE four-stage form. Same function, latency 4, and the period
// need only cover the largest single operation.
always_ff @(posedge clk) begin
  if (s1_en) begin decoded_q   <= decode(in_flit);                    s1_v_q <= in_valid;  end
  if (s2_en) begin route_sel_q <= route(decoded_q.protocol_id);       s2_v_q <= s1_v_q;    end
  if (s3_en) begin crc_val_q   <= crc_next(crc_q, muxed_q);           s3_v_q <= s2_v_q;    end
  if (s4_en) begin enq_data_q  <= pack(muxed_q, crc_val_q);           s4_v_q <= s3_v_q;    end
end

Architecture. Identical function, two structures. The second costs four sets of registers and gains a period bounded by the largest stage rather than the sum.

State. The four-stage form carries the payload and a valid bit per stage. The valid pipeline is not optional and it is not free — it is what lets the pipeline hold bubbles, and a design that pipelines data without pipelining valid will process stale data as though it were new.

Cycle behaviour. Each s*_en is the stage's own enable, from §10's elastic logic. Note the enables must be per stage: a single pipe_en here is §11's global enable and gives back the Fmax gain by creating a combinational path across all four.

Contract. The split points must be chosen where the timing is, not where the code is. Splitting after decode because it reads nicely, when the real delay is in crc_next, produces four stages and almost no frequency gain — the classic result of pipelining by structure rather than by measurement.

Failure. Two, and both are common. Unbalanced stages: the period is still set by the worst stage, so latency quadrupled for a small frequency gain. And forgetting the valid pipeline: bubbles are processed as data.

DV. Assert the valid pipeline's shape — an object entering at cycle n must appear at the output at cycle n + 4 under sustained ready, and no output may be valid without a corresponding input four cycles earlier. That is a per-stage conservation property and it catches the missing-valid bug immediately.

27. Width Against Frequency

The other axis, and UCIe's own history is the clearest available illustration.

throughput ≈ datapath_width × clock_frequency × utilization

Two ways to double the first product, with completely different consequences:

And UCIe chose width, explicitly, for its densest variant. The UCIe 2.0 white paper on UCIe-3D: "even at 4 GT/s frequency, there are orders of magnitude improvement in bandwidth density over UCIe 2.5D at 32 GT/s (e.g., 300 TB/s/mm² with UCIe-3D at 1 µm bump pitch vs 1.35 TB/s/mm² with UCIe-2.5D at 25 µm bump pitch)." An eight-fold lower per-lane rate delivering roughly 220 times the bandwidth density, because the bump pitch shrank 25-fold and the width went up as the square.

Three engineering lessons, and the third is the one worth carrying.

Width scales with area; frequency does not. Halving the bump pitch quadruples the lanes in the same shoreline. No frequency increase available at a fixed process gives 4×.

Lower frequency simplifies everything downstream. The same white paper notes that at 3D pitches "it is essential to keep the associated circuitry simple" and that with SoC frequencies the circuits are "uncomplicated — consisting of simple inverters", and — most striking — that the design "eliminated the need for (de)serialization, CRC, replay, etc., by choosing the appropriate bit error rate." Going wider and slower removed entire mechanisms, including the reliability machinery Module 14 exists to describe.

And that is the general principle. Width buys bandwidth at the cost of area and partial-object complexity; frequency buys bandwidth at the cost of timing closure, power, and — at high rates — the error rate that forces retry machinery into existence. The UCIe 3.0 figures make that last cost explicit: 10⁻¹⁵ BER at 48 GT/s and 10⁻¹² at 64 GT/s (§3), so the final rate step traded three orders of magnitude of error rate for a 33% rate gain, and paid for it with the CRC and replay path that §29 then taxes.

28. The Replay Window as a Throughput Limit

An easily overlooked term in §17's conjunction.

Chapter 9.4 established that a sender must retain a transmitted object until delivery is confirmed. The retention capacity is therefore a concurrency pool with a rate × latency requirement of its own — the latency being the confirmation round trip, not the data round trip.

required_replay_entries ≈ transmission_rate × confirmation_latency

If the window is smaller than that, the sender stalls with credits available. The remote receiver has space, permission exists, the queue has objects ready — and the sender cannot transmit because it has nowhere to record what it sent. The signature is distinctive and easily misread: credits abundant, remote occupancy low, local queue full, link idle.

Two design consequences.

Confirmation latency is not data latency. Confirmation depends on how often the far side reports, which may be batched or piggybacked — 13.1 §11's piggyback analysis applies directly, and the same hazard exists: if confirmations ride on reverse traffic and there is none, the confirmation latency is unbounded and no replay window is large enough.

And a raised error rate lengthens it. A retransmitted object's entry cannot retire until its delivery is confirmed, so the effective confirmation latency rises with the error rate, and the required window rises with it. A replay window sized at a clean error rate is undersized at a dirty one — which is 13.4 §26's amplification chain, entered from the sizing side.

29. The Retry Tax

useful_rate ≈ offered_rate × (1 − retransmission_fraction)

First-order only, and it understates the cost. Retransmission also consumes replay residency (§28), arbitration opportunities (§22), and queue time — so the real coefficient is worse than the linear term suggests. 13.4 §25 is the mechanism.

What the verified BER figures let us compute. UCIe 3.0 targets 10⁻¹⁵ at 48 GT/s and 10⁻¹² at 64 GT/s (§3). Per lane:

48 GT/s × 1e-15 = 4.8e-5 errored bits per second per lane
                → one bit error every ~20,800 seconds per lane  (~5.8 hours)
 
64 GT/s × 1e-12 = 6.4e-2 errored bits per second per lane
                → one bit error every ~15.6 seconds per lane

Derived arithmetic on quoted figures, not a specification claim — and the qualifications matter. A bit error is not necessarily one retransmission: an error may fall in an idle period, and a burst may corrupt several units at once, which Module 14 develops. The point is not the exact rate but the ratio: roughly 1,300× more error events per lane per second at the higher rate, from a 33% rate increase.

Three consequences for a performance model.

The retry tax at either rate is negligible for bandwidth. One event per 15.6 seconds per lane costs a vanishing fraction of throughput. Retransmission is not a bandwidth problem at the specified BER — which is exactly the point of specifying a BER.

But it is a design problem, because the machinery must exist and be sized. The replay window, the confirmation path, and the detection logic all cost area and latency continuously, in exchange for handling an event that is rare. That is the trade UCIe-3D declined by choosing a BER good enough to delete the mechanism entirely (§27).

And the tax is catastrophic when the BER is not met. A marginal channel running orders of magnitude worse than specification enters 13.4 §26's amplification chain, and the collapse is non-linear. So the retry path is sized not for the specified error rate but for the error rate at which the design must still function — and those are very different numbers.

30. Performance Collapse From Errors

The chain from 13.4 §26, stated from the throughput side because the instrumentation differs:

1. Error rate rises above specification.
2. Retransmissions rise; transmitted flits per second is unchanged or higher.
3. Replay entries retire more slowly; the window (§28) becomes the binding pool in §17's conjunction.
4. Admission stalls with credits available.
5. Useful flits per second falls while the link stays busy.

The distinguishing measurement is useful_flits / transmitted_flits. A link at 100% transmitted utilisation and 40% useful throughput is retransmitting, and no other measurement in this chapter distinguishes that from being at capacity.

That ratio is the single most valuable counter a link can have, and it costs two registers (§34). Without it, "the link is saturated" and "the link is thrashing" are the same reading.

31. Queue Depth, Tuned Rather Than Guessed

The two inputs, from 13.2 §13: the burst length to absorb, and the round-trip latency to cover. Depth beyond max(burst, rate × latency) buys latency, not throughput.

Deeper is not faster, and the reason is worth stating precisely: a queue improves throughput only by preventing stalls it can absorb. Once it is deep enough to absorb every stall the workload produces, additional depth is never exercised as absorption — it is exercised only as residency, which is latency.

32. Flagship Trace — Full Rate After Fill

Illustrative. Five stages: map, queue, adapter, PHY-in, PHY-out. Source always has an object; sink always ready. Sufficient credits, outstanding slots and replay entries throughout.

Four readings.

First-object latency is 5 cycles; steady-state initiation interval is 1. Both true at once, and the whole of §9 in one table.

The fill cost is 4 cycles of no output, which is stages − 1, paid once. Over the 13 cycles shown the pipeline delivered 8 objects — 62% — and over 1,000 cycles it would deliver 996, or 99.6%. The efficiency figure depends entirely on the observation window, which is why short benchmarks understate pipelined designs.

Every stage is occupied from cycle 4 onward. That is the definition of a full pipeline, and it means every stage's logic is doing useful work every cycle. A stage that is ever empty during sustained supply and demand is a bug — §12's assertion.

And nothing here is limited by any resource pool. Credits, outstanding slots and replay entries are all abundant, so §17's conjunction is not binding and the pipeline's initiation interval is the sole determinant. §33 removes that assumption.

33. Second Trace — A Credit Bubble, and Its Removal

Same pipeline. Now credits are limited: 4 credits, a return latency of 6 cycles, and returns arriving one per cycle once they start.

Read cycles 4 and 5. The pipeline is not full, the sink is ready, every stage could accept — and nothing is issued, because there is no permission. Two cycles of the link's capacity are gone and no counter in the pipeline records why. Only a per-reason stall counter (§34) attributes this to credits rather than to the pipeline.

Why exactly two cycles. Four credits were spent by cycle 3. The first return arrives at cycle 6, which is the issue at cycle 0 plus the 6-cycle return latency. So the gap is return_latency − credits = 6 − 4 = 2. That is 13.1 §4's bandwidth-delay product as a subtraction: the credits cover 4 of the 6 cycles of the loop, and the shortfall is the bubble.

With 6 credits instead of 4:

No bubble. The credit count reaches 1 at cycle 5 and the first return arrives at cycle 6, so the pool never empties. The requirement was rate × latency = 1 × 6 = 6, and 6 credits is exactly enough for a perfectly smooth producer — which is why 13.1 §4 insists that 6 is a floor and real designs need headroom on top of it for burstiness and jitter.

And the general form of the lesson: the same two bubble cycles could have been removed by shortening the return latency instead of adding credits. rate × latency has two factors, and both are design variables.

34. The Performance Monitor

// ILLUSTRATIVE performance counters. Synthesisable, cheap, and the difference
// between "the link is slow" and a diagnosis. Counters only — no division in
// hardware (Section 36).
logic [31:0] cycles_q;            // observation window
logic [31:0] useful_flits_q;      // flits carrying NEW payload
logic [31:0] tx_flits_q;          // ALL flits transmitted, including replays
logic [31:0] stall_credit_q;      // could not issue: no credit          (13.1)
logic [31:0] stall_replay_q;      // could not issue: no replay entry    (9.4)
logic [31:0] stall_os_q;          // could not issue: no tracking slot   (Sec 16)
logic [31:0] stall_queue_q;       // could not issue: no queue space     (13.2)
logic [31:0] stall_arb_q;         // requested and lost arbitration      (Sec 22)
logic [31:0] stall_batch_q;       // held in a batch awaiting flush      (Sec 19)
logic [31:0] stall_down_q;        // downstream not ready                (13.3)
logic [31:0] idle_q;              // nothing offered — NOT a stall
 
always_ff @(posedge clk or negedge rst_n) begin
  if (!rst_n || window_clear) begin
    cycles_q <= '0; useful_flits_q <= '0; tx_flits_q <= '0;
    stall_credit_q <= '0; stall_replay_q <= '0; stall_os_q <= '0;
    stall_queue_q  <= '0; stall_arb_q    <= '0; stall_batch_q <= '0;
    stall_down_q   <= '0; idle_q         <= '0;
  end else begin
    cycles_q <= cycles_q + 32'd1;
 
    if (flit_transmitted) begin
      tx_flits_q <= tx_flits_q + 32'd1;
      if (!is_replay) useful_flits_q <= useful_flits_q + 32'd1;
    end
 
    // EXACTLY ONE bucket per non-transferring cycle — see Section 35.
    if (!flit_transmitted) begin
      unique case (1'b1)
        !have_work        : idle_q         <= idle_q         + 32'd1;
        !credit_available : stall_credit_q <= stall_credit_q + 32'd1;
        !replay_space     : stall_replay_q <= stall_replay_q + 32'd1;
        !os_slots_free    : stall_os_q     <= stall_os_q     + 32'd1;
        !queue_space      : stall_queue_q  <= stall_queue_q  + 32'd1;
        !downstream_ready : stall_down_q   <= stall_down_q   + 32'd1;
        held_in_batch     : stall_batch_q  <= stall_batch_q  + 32'd1;
        lost_arbitration  : stall_arb_q    <= stall_arb_q    + 32'd1;
        default           : /* unattributed — see Section 35 */ ;
      endcase
    end
  end
end

Architecture. Eleven counters, and the design is entirely in the last block: exactly one bucket increments per non-transferring cycle, chosen by a priority unique case (1'b1). That makes the counters partition the window rather than overlap it.

State. All eleven have observation-window lifetime, cleared together by window_clear. Clearing them individually would break the partition invariant below and make every ratio meaningless.

Cycle behaviour. unique case (1'b1) evaluates conditions in order and takes the first true one — a priority selection with unique's checking. The order encodes the priority policy of §35, and it is a real design decision, not an implementation detail.

Contract — and it is checkable, which is the point:

cycles = useful_transfer_cycles + replay_transfer_cycles + idle
       + stall_credit + stall_replay + stall_os + stall_queue
       + stall_down + stall_batch + stall_arb + unattributed

Every cycle is in exactly one bucket, and the sum must equal cycles_q. A discrepancy means some cycle was attributed to nothing, which is a hole in the priority logic — and that is a bug in the instrumentation, not a performance finding. Check this identity in the testbench at the end of every window; it is the cheapest possible guard against believing a wrong diagnosis.

Note idle_q is separated from every stall. A cycle with nothing offered is not a stall, and merging the two is the most common instrumentation error in performance work: it makes a lightly loaded link look catastrophically stalled, and it hides real stalls inside a large idle count.

Failure. Two. Overlapping counters — incrementing every true condition — produces a sum exceeding cycles_q and ratios above 1, at which point nobody trusts any of them. And a missing default for the unattributed case hides the hole rather than exposing it.

DV. Assert the partition identity every window. Then verify each counter independently by constructing a stimulus that stalls on exactly one resource and checking only that counter moves.

35. Stall Attribution — First Blocking Resource

The priority order in §34 is a policy, and this is the argument for it.

Multiple resources are frequently unavailable at once. Under sustained load, credits may be at zero and the queue full and the downstream not ready. Counting all three overstates every one of them and the fractions no longer sum to anything meaningful.

Attribute each stalled cycle to the first blocking resource in a fixed priority order. The resulting fractions partition the window and can be compared against each other.

The order used in §34, and why:

Two properties of this ordering deserve stating.

It is deliberately ordered by remoteness and cost of remedy, not by frequency. When credits and queue space are both unavailable, reporting "credit-limited" is more actionable, because the credit loop is longer and its fix — more credits, faster return, or reduced latency — takes longer to implement and validate.

And it means the fractions are not independent measurements. Fixing the top-priority stall reveals the next one, so the attribution must be re-measured after every change. A design that is 40% credit-limited does not become 40% faster when credits are added; it becomes queue-limited, or outstanding-limited, at some new fraction. That is the honest meaning of "optimisation moves the bottleneck", and it is why §16's sweep flattens rather than continuing to rise.

When overlap measurement is genuinely wanted — "how often were credits and the queue both binding?" — that is a separate, deliberately overlapping set of counters with a different name, not a modification of the partitioning set. Mixing the two purposes in one group of registers destroys both.

36. Utilisation, Measured Where Division Is Free

// VERIFICATION-ONLY. Ratios belong in the testbench, not the datapath.
function automatic real utilisation();
  return real'(useful_flits_q) / real'(cycles_q);
endfunction
 
function automatic real useful_fraction_of_transmitted();
  return (tx_flits_q == 0) ? 0.0
       : real'(useful_flits_q) / real'(tx_flits_q);     // Section 30's ratio
endfunction
 
function automatic void report_window();
  $display("--- performance window: %0d cycles ---", cycles_q);
  $display("  utilisation (useful/cycles)     : %0.4f", utilisation());
  $display("  useful / transmitted            : %0.4f", useful_fraction_of_transmitted());
  $display("  stall credit                    : %0.4f", real'(stall_credit_q) / real'(cycles_q));
  $display("  stall replay                    : %0.4f", real'(stall_replay_q) / real'(cycles_q));
  $display("  stall outstanding               : %0.4f", real'(stall_os_q)     / real'(cycles_q));
  $display("  stall queue                     : %0.4f", real'(stall_queue_q)  / real'(cycles_q));
  $display("  stall downstream                : %0.4f", real'(stall_down_q)   / real'(cycles_q));
  $display("  stall batch                     : %0.4f", real'(stall_batch_q)  / real'(cycles_q));
  $display("  stall arbitration               : %0.4f", real'(stall_arb_q)    / real'(cycles_q));
  $display("  idle (not a stall)              : %0.4f", real'(idle_q)         / real'(cycles_q));
endfunction
 
// The partition invariant from Section 34 — checked, not assumed.
function automatic void check_partition();
  int unsigned total = useful_flits_q + replay_flits_q + idle_q
                     + stall_credit_q + stall_replay_q + stall_os_q
                     + stall_queue_q  + stall_down_q   + stall_batch_q
                     + stall_arb_q    + unattributed_q;
  if (total != cycles_q)
    $error("stall partition broken: buckets sum to %0d, window is %0d — %0d cycles unattributed",
           total, cycles_q, cycles_q - total);
endfunction

Architecture. All division and all reporting on the verification side. The hardware counts; the testbench divides.

Why division must not be in the datapath. A divider is large, slow, and buys nothing — the ratio is needed by a human or a script reading a register, not by the datapath making a decision. If a design genuinely needs a ratio at runtime for a policy decision, compare products instead of computing a quotient: a × d > c × b rather than a/b > c/d, which is a multiply and a compare rather than a divide.

Contract. check_partition must run at every window boundary. It is the guard that the numbers mean what the report says they mean, and it is the difference between measurement and decoration.

DV. Verify the reporting against a known-answer stimulus: a workload constructed to stall exactly 25% of cycles on credits and nothing else should report 0.25 in that bucket and 0.00 in every other.

37. Performance Regression Methodology

Throughput is not verified by assertions; it is measured, repeatedly, across a matrix. Three axes.

Plus two overlays that change the answer rather than adding a point:

• Error injection at zero, specification, and above-specification rates — §29's tax and §30's collapse;
• Downstream duty cycle swept, including periods coprime and non-coprime with the class count — 13.4 §18's arbiter bias only appears here.

Compare four things per cell, not one:

Achieved utilisation — the headline number, and useless alone.

Stall-cause fractions — the diagnosis. A cell whose utilisation regressed by 5% with the same stall profile is a different problem from one whose profile shifted from queue-limited to credit-limited, even at identical utilisation.

Tail latency — because §18's batching and §31's depth both trade it for throughput, so a "win" that ignores it may be a loss.

Useful over transmitted (§30) — the only figure that distinguishes saturation from thrashing.

A regression that tracks only utilisation will accept a change that improved throughput by 2% and doubled tail latency, and will report a bufferbloat regression as a win.

And the trickle case is the one to keep in the matrix permanently, because it is where §20's unbounded-wait bug lives and where every batching and aggregation optimisation is most likely to have broken something. High-load testing cannot find it.

38. Coverage

covergroup cg_throughput @(posedge clk);
  option.per_instance = 1;
 
  // --- Pipeline occupancy. "full" is the bin that proves Section 12 ran.
  cp_pipe_occ : coverpoint pipeline_occupancy {
    bins empty   = {0};
    bins partial = {[1:STAGES-1]};
    bins full    = {STAGES};
  }
  cp_ii : coverpoint measured_initiation_interval {
    bins one    = {1};                 // target
    bins two    = {2};
    bins worse  = {[3:$]};
  }
 
  // --- Which pool was binding (Section 17). Each must bind at least once.
  cp_binding : coverpoint binding_resource {
    bins credit      = {BIND_CREDIT};
    bins outstanding = {BIND_OS};
    bins replay      = {BIND_REPLAY};
    bins queue       = {BIND_QUEUE};
    bins pipeline    = {BIND_PIPE};
    bins none        = {BIND_NONE};    // unconstrained full rate
  }
 
  // --- Outstanding depth actually reached, not just permitted.
  cp_os_occ : coverpoint outstanding_q {
    bins one     = {1};
    bins low     = {[2 : MAX_OUTSTANDING/4]};
    bins mid     = {[MAX_OUTSTANDING/4+1 : 3*MAX_OUTSTANDING/4]};
    bins near_max= {[3*MAX_OUTSTANDING/4+1 : MAX_OUTSTANDING-1]};
    bins at_max  = {MAX_OUTSTANDING};  // proves the pool was exercised
  }
 
  // --- Batching (Sections 19 to 21). Partial flushes are the risky path.
  cp_batch_size : coverpoint batch_count_at_send {
    bins single  = {1};
    bins partial = {[2 : BATCH_MAX-1]};
    bins full    = {BATCH_MAX};
  }
  cp_flush_reason : coverpoint flush_reason {
    bins on_full   = {FLUSH_FULL};
    bins on_timer  = {FLUSH_TIMER};    // proves Section 20 cannot happen
    bins on_urgent = {FLUSH_URGENT};
    bins on_drain  = {FLUSH_DRAIN};
  }
 
  // --- Load, because several mechanisms only misbehave at one end.
  cp_load : coverpoint offered_load_bucket {
    bins trickle = {0};                // Section 20's habitat
    bins low     = {1};
    bins high    = {2};
    bins over    = {3};
  }
 
  // --- Arbitration and errors.
  cp_arb_contend : coverpoint concurrent_requesters { bins one={1}; bins some={[2:3]}; bins many={[4:$]}; }
  cp_retry_rate  : coverpoint retry_rate_bucket { bins none={0}; bins spec={1}; bins over_spec={[2:$]}; }
 
  // --- Crosses that carry the information.
  x_load_batch    : cross cp_load, cp_batch_size;      // trickle × partial flush
  x_binding_load  : cross cp_binding, cp_load;         // the pool changes with load
  x_retry_binding : cross cp_retry_rate, cp_binding;   // Section 30's shift to replay
  x_ii_occ        : cross cp_ii, cp_pipe_occ;          // II=1 must occur at full occupancy
endcovergroup

Five bins whose value is being non-zero, each proving a mechanism was exercised rather than merely present:

cp_pipe_occ.full crossed with cp_ii.one. Without this cross, §12's no-bubble assertion has passed vacuously. A regression showing x_ii_occ with no (one, full) hit has never observed the pipeline at full rate, which is indistinguishable from the pipeline being incapable of it.

cp_os_occ.at_max. If the outstanding pool never fills, its depth is untested and the sizing conclusion of §16 is unsupported by evidence.

cp_flush_reason.on_timer. Proves the timeout path works, which is the only thing standing between the design and §20's unbounded wait.

cp_load.trickle crossed with cp_batch_size. The trickle-load column is where every batching bug lives, and high-load testing never enters it.

cp_binding — every bin. Each pool must be the binding constraint at least once, or that term of §17's conjunction has never been the limit and its sizing is unverified.

And one whose value is being zero in a clean run: cp_ii.worse. An initiation interval of 3 or more under sustained supply and demand is a structural defect, and it should appear only in a directed test that deliberately creates it.

39. Debug Checklist

1. What is the raw physical bandwidth? Lanes × rate — the ceiling, and nothing more (§6).
2. What useful throughput is actually measured? Useful flits per second, not transmitted (§30).
3. What is useful / transmitted? Below 1 means retransmission; this single ratio separates saturation from thrashing.
4. Which stall bucket is largest? And does the partition identity hold, so the buckets can be trusted (§34)?
5. Is the measured initiation interval actually 1? Or is one stage taking longer and capping the pipeline (§9)?
6. Is idle being counted as a stall? If so, every fraction is wrong (§34).
7. Is outstanding depth sufficient for the response latency? throughput × latency, and read the low-water mark rather than guessing (§16).
8. Are credits sufficient for the credit round trip? Note this is a different latency from the response latency (§33).
9. Is the replay window limiting? Credits abundant, table free, link idle is its signature (§28).
10. Are batches waiting too long? Check the flush-reason distribution: all-timer at low load is expected, all-timer at high load means the batch never fills and the target is wrong (§19).
11. Is arbitration adding bubbles? Compare object duration against selection cost — the ratio is the overhead (§22).
12. Is queue depth too small — or too large? Bubbles indicate too small; latency with clean stall counters indicates too large (13.4 §28).
13. Is a long combinational path capping Fmax? A design that misses timing is slow in seconds even at initiation interval 1 (§25).
14. Are the pipeline stages balanced? An unbalanced split quadruples latency for a small frequency gain (§26).
15. Are retries consuming meaningful bandwidth? At the specified BER they should not; if they do, the channel is not meeting specification (§29).
16. Is this a bandwidth, a throughput, or a latency problem? They have different fixes and several fixes make the other two worse (§4).
17. Which optimisation changes the bottleneck rather than removing it? Re-measure the attribution after every change, because the answer moves (§35).

40. Common Misconceptions

"Low latency means high throughput." They are independent. A six-stage pipeline with an initiation interval of 1 has six times the latency and the same throughput as a one-stage version, at a much higher clock (§9, §25).

"Physical bandwidth equals useful throughput." Bandwidth is a ceiling. Bubbles, insufficient concurrency, credit round trips, arbitration and retransmission all sit between it and delivered work, and none of them is visible in a bandwidth measurement (§6).

"A deeper FIFO always improves throughput." Only up to max(burst, rate × latency). Beyond that, depth is exercised as residency rather than as absorption, which is latency (13.4 §28, §31).

"One outstanding transaction is enough." With a 50-cycle response latency it caps throughput at 2% of a one-per-cycle path, with no bug anywhere and every assertion passing (§15).

"Pipeline latency and initiation interval are the same thing." Latency 6 with an initiation interval of 1 is normal and desirable. The initiation interval determines throughput; latency does not (§9).

"More outstanding requests always help." Only until another pool in §17's conjunction becomes the minimum. Past that, extra slots are area (§16, §17).

"Bigger batches are always better." Batching costs latency in inverse proportion to arrival rate — most where the benefit is least. And a batch with no flush condition other than "full" can wait forever (§18, §20).

"More credits always improve throughput." Only if credits are the binding pool. Add credits when the tracking table is the constraint and nothing measurable changes (§17).

"Retry only costs retransmission bandwidth." It also holds replay entries, consumes arbitration opportunities and extends queue residency, so the linear tax understates it — and above specification it amplifies non-linearly (§29, §30).

"Optimising one stage guarantees system throughput." Throughput is set by the worst stage's initiation interval and the scarcest pool. Improving anything else is invisible (§1, §9).

"Higher Fmax always means lower latency." Fmax raises throughput reliably; latency in time improves only if the pipeline split is balanced. A badly split four-stage design has worse latency in nanoseconds than the single-cycle version it replaced (§25, §26).

"Utilisation of 100% means we are at capacity." A link retransmitting is 100% utilised and delivering less. Only useful / transmitted distinguishes them (§30).

41. Understanding Check

42. Summary — and Module 13 Closed

Throughput is set by the slowest recurring resource cycle, not the fastest block, and almost never by the physical link rate. Bandwidth is a ceiling; useful throughput is raw_rate × encoding_efficiency × payload_efficiency × utilisation, and utilisation is the only term the design controls — and the only one a bad design can drive to 0.1 with everything else at maximum.

Latency and initiation interval are independent, and the second determines throughput. Latency 6 with an initiation interval of 1 is normal and good. The initiation interval of a pipeline is that of its worst stage, so every other stage's excellence is invisible.

Elasticity does not raise the ceiling; it stops the ceiling from propagating. Per-stage valid/ready lets a stall be absorbed into pipeline occupancy instead of freezing stages with slack — and a global enable both discards that slack and creates a combinational path whose length grows with the pipeline, giving back the Fmax that pipelining was for.

Concurrency is a minimum over four independent pools — remote credits, outstanding slots, replay entries, local queue capacity — each with its own rate × latency requirement and its own owner. Provisioning three generously and one poorly delivers the poor one's throughput, the binding pool changes with the traffic shape, and all four produce the identical symptom of an idle link that cannot issue.

Batching costs latency in inverse proportion to arrival rate — most where the throughput benefit is least. Any accumulation whose release depends only on more input needs a second release condition that does not, or it waits unboundedly at low load and hangs when traffic stops.

Speculation is free until it becomes a commitment. Compute the next grant early; commit it only when the current object is known retirable and ordering is resolved. The same boundary governs routing versus delivery and predicting versus consuming a credit.

Pipelining buys throughput reliably and latency-in-time only if the split is balanced, and width buys bandwidth where area allows while frequency buys it at the cost of timing, power, and eventually the error rate that forces reliability machinery into existence. UCIe's own history is the demonstration in both directions — 4 GT/s and 300 TB/s/mm² in 3D with CRC and replay deleted, against 64 GT/s with a target BER three orders of magnitude worse than 48 GT/s and the retry path absorbing the difference.

And measurement is the discipline that makes all of it real. Per-reason stall counters that partition the window rather than overlap it, with the partition identity checked; idle separated from every stall; attribution to the first blocking resource in a fixed priority order; and useful / transmitted as the one ratio that distinguishes a saturated link from a thrashing one. Optimisation moves the bottleneck rather than removing it, so every number must be re-measured after every change.

Module 13 — Flow Control, complete

Flow control is now complete: permission, storage, propagation, response, and efficient steady-state operation. Every trace in all five chapters has assumed one thing without ever saying so — that the bits which arrive are the bits that were sent.

Module 14 removes that assumption.

• 14.1 — Error Detection — CRC, parity and sequence checks per layer, and how each layer decides that what arrived is not what should have arrived.

Browse the full path on the UCIe tutorials index.