Streaming Flow Control

Chapter 9.4 ended with a transmitter that cannot accept a packet unless it has somewhere to retain it. That was a local resource constraint — the sender checking its own replay buffer.

This chapter is the harder half of the same question. The sender must also know whether the receiver, on the other die, still has room. And it must know this without being able to see the receiver's buffers, across a link with real latency, while packets are already in flight whose fate is not yet known.

That problem has one good solution, and it is old, elegant, and easy to implement almost correctly.

1. The One-Sentence Model

A credit is permission backed by real storage. A receiver may advertise a credit only when a unit of receive capacity actually exists; spending it reserves that unit; and the credit returns only when the unit is genuinely free again.

The word to hold is backed. A credit is not a hint, not a rate limit, and not a ready signal — it is a claim on a specific physical resource on another die. Every bug in this chapter is a case where the accounting drifts away from the storage it is supposed to describe: credits that outnumber slots, credits that vanish, credits returned before the slot is free.

2. The Conservation Invariant

Credit flow control is distributed accounting, and like all accounting it has a conservation law. In the simplest form, for one receive buffer:

credits_held_by_sender  +  packets_in_flight  +  slots_occupied_at_receiver
        =  total_receiver_capacity

Every packet is in exactly one of three places: not yet sent (its permission sits with the sender), on the wire, or occupying a receive slot. Nothing may be in two, and nothing may be in none.

Credits must never create storage that does not exist. The invariant is the whole subject; every mechanism below exists to maintain it, and every bug below violates it.

Two consequences worth extracting immediately.

The sender's credit count is a replica of a fact that lives elsewhere. It is the sender's belief about the receiver's free capacity, updated by messages that take time to arrive. Like any replica it can be stale — safely stale in one direction only, which §6 develops.

Being stale-low is safe; being stale-high is catastrophic. If the sender believes it has fewer credits than it really does, it sends less than it could — a performance loss. If it believes it has more, it sends into storage that does not exist, and the receiver overflows. That asymmetry is why every design decision in this chapter breaks toward conservatism.

3. Sender State

// Illustrative flow-control RTL — not UCIe normative signal naming or encoding.
localparam int CREDIT_W = 6;
 
logic [CREDIT_W-1:0] tx_credit_q;    // permission currently held
logic [CREDIT_W-1:0] max_credit_q;   // capacity advertised by the peer
logic                credit_valid_q; // advertisement received; may send

Architecture. The sender must gate transmission on a quantity it cannot observe directly, so it maintains a local replica updated by advertisement and return.

State. A count, an advertised maximum, and a validity bit — per-link-epoch lifetime, re-established whenever the link re-initialises (§13).

Cycle behaviour. Initialised from the advertisement, decremented on consumption, incremented on return.

Contract. The receiver relies on the sender never exceeding what it advertised. That is the entire safety property.

Failure. Without credit_valid_q, the counter's reset value — usually zero, occasionally something worse — is treated as a real credit count, and the sender either stalls forever or transmits with no authority at all.

Note max_credit_q is a register, not a parameter. §5 explains why the receiver's depth must not be a sender-side constant.

4. Receiver State

// Illustrative — the receiver tracks its own occupancy; the sender never sees this.
localparam int RX_DEPTH = 16;
localparam int OCC_W    = $clog2(RX_DEPTH + 1);
 
logic [OCC_W-1:0] rx_occ_q;    // slots currently holding undrained packets
logic             rx_full;
logic             rx_empty;
 
assign rx_full  = (rx_occ_q == OCC_W'(RX_DEPTH));
assign rx_empty = (rx_occ_q == '0);

Architecture. The receiver owns the storage and therefore owns the truth. Everything the sender knows is derived from what the receiver chose to tell it.

State. One occupancy counter. Track occupancy or free slots, but not both — two counters that must agree is two sources of truth, and their disagreement is a bug you have created rather than prevented. Occupancy is usually the better choice because it is what the buffer's own control logic already needs.

Cycle behaviour. Increments on arrival, decrements on drain.

Contract. The receiver must never be asked to accept when full — and if it ever is, the credit accounting has already failed upstream, so this is an assertion point rather than a condition to handle gracefully.

Failure. Deriving free slots as RX_DEPTH - rx_occ_q in several places invites one of them to use a stale copy.

// Illustrative — the receiver must never overflow. If it can, credits are wrong.
property p_rx_never_overflows;
  @(posedge clk) disable iff (!rst_n)
    rx_push |-> !rx_full;
endproperty

5. The Sender Does Not Own the Receiver's Depth

// WRONG — the sender assumes it knows the far side's buffer.
initial tx_credit_q = RX_DEPTH;      // RX_DEPTH is a LOCAL parameter

The receiver is on another die, quite possibly from another vendor, quite possibly a different revision of the same design. Its buffer depth is its own business, and hard-coding it at the sender creates a silent coupling that holds right up until the two sides are built from different parameter files.

The failure mode when they diverge is asymmetric and instructive. If the sender's constant is smaller than the real depth, throughput is quietly below capability — nobody notices. If it is larger, the sender over-sends and the receiver overflows, dropping packets with no error from the flow-control mechanism, because from its own point of view the sender never exceeded its credits.

// Illustrative — capacity is learned, not assumed.
always_ff @(posedge clk or negedge rst_n) begin
  if (!rst_n) begin
    tx_credit_q    <= '0;
    max_credit_q   <= '0;
    credit_valid_q <= 1'b0;
  end else if (credit_advert_valid) begin
    tx_credit_q    <= advertised_credits;
    max_credit_q   <= advertised_credits;
    credit_valid_q <= 1'b1;
  end
  // ... consume/return in §6
end

Architecture. Capacity is a property of the receiver, communicated during initialisation. UCIe places this in parameter exchange for the D2D Adapter — the advertisement is part of bring-up, which is why Chapter 8.5's evidence-lifetime reasoning applies to it.

Contract. The sender may not transmit before credit_valid_q. That makes credit advertisement one more prerequisite in Chapter 8.5's dependency graph, and one more piece of evidence invalidated when the link restarts.

6. Consume and Return in the Same Cycle

The bug that defines this chapter.

// WRONG — two independent statements assigning the same register.
always_ff @(posedge clk) begin
  if (credit_consume) tx_credit_q <= tx_credit_q - 1'b1;
  if (credit_return)  tx_credit_q <= tx_credit_q + 1'b1;
end

When both are true in the same cycle — which happens constantly on a busy link — the last statement wins. The decrement is lost, so the sender keeps a credit it has spent. Repeat that a few thousand times and the sender believes it has capacity the receiver does not have.

What makes it vicious is the delay between cause and symptom. The counter drifts upward slowly, entirely silently, and the receiver overflows minutes later under a traffic pattern that happens to produce many coincident events. Nothing points back at the counter.

// Illustrative — one assignment, all four cases explicit, conservation visible.
always_ff @(posedge clk or negedge rst_n) begin
  if (!rst_n) begin
    tx_credit_q <= '0;
  end else if (credit_advert_valid) begin
    tx_credit_q <= advertised_credits;
  end else begin
    unique case ({credit_consume, credit_return})
      2'b10 : tx_credit_q <= tx_credit_q - 1'b1;   // spent one
      2'b01 : tx_credit_q <= tx_credit_q + 1'b1;   // regained one
      2'b11 : tx_credit_q <= tx_credit_q;          // spent and regained — net zero
      default: tx_credit_q <= tx_credit_q;         // 2'b00 — idle
    endcase
  end
end

Architecture. The counter is the sender's model of remote capacity, and every change to it must be a net effect of all simultaneous events.

State. One counter, one always block, one assignment path.

Cycle behaviour. Exactly the four cases, with 2'b11 explicitly a no-op rather than an accident of ordering.

Contract. Everything gating on the credit count relies on it being an accurate net.

Failure. The version above; also note that writing it as tx_credit_q <= tx_credit_q - consume + return is arithmetically correct but hides the reasoning, and — with unsigned operands — makes the underflow analysis of §8 much harder to see. Make the conservation visible in the code structure, because this is a register whose correctness a reviewer must be able to confirm by reading.

// Illustrative — the net effect is exactly right in every case.
property p_credit_net_arithmetic;
  @(posedge clk) disable iff (!rst_n || credit_advert_valid)
    tx_credit_q == ($past(tx_credit_q)
                    - CREDIT_W'($past(credit_consume) && !$past(credit_return))
                    + CREDIT_W'($past(credit_return)  && !$past(credit_consume)));
endproperty

7. Gating

// WRONG — physical readiness treated as permission.
assign stream_ready = phy_ready;

Chapter 9.4 §11 already rejected this for the replay buffer. Credits add a second, independent reason it is wrong: the PHY can be ready, the replay buffer can have space, and the receiver can still be full.

// Illustrative — three independent resources, all required.
assign can_send = link_operational        // Chapter 8.5's derived conclusion
               && credit_valid_q          // capacity has been advertised
               && (tx_credit_q != '0)     // and some remains
               && replay_space;           // and we can retain it (Ch 9.4 §11)

Architecture. Sending a packet incurs three obligations simultaneously: the link must carry it, the receiver must store it, and the sender must retain it for possible replay. Any one missing makes the send unsafe, and each is owned by a different mechanism.

State. None of its own; a conjunction over four registered facts.

Contract. The producer relies on can_send meaning the packet will be handled safely end to end.

Failure. Each omission fails differently, which is worth knowing for debug: without link_operational, packets go into a link that cannot carry them; without tx_credit_q, the receiver overflows; without replay_space, the sender loses recoverability. Three different symptoms on three different dies.

One resource being available does not make the transaction safe. This is the cross-mechanism lesson of Module 9 — the packetizer, the replay buffer, and the credit counter each hold a veto.

// Illustrative — nothing is sent without permission.
property p_no_send_without_credit;
  @(posedge clk) disable iff (!rst_n)
    tx_fire |-> ($past(tx_credit_q) != '0) && $past(credit_valid_q);
endproperty

Note the $past. The decision was made with the credit value as it stood before this cycle's update — asserting against the current value would compare against a count that already reflects the very consumption being checked.

8. Underflow Is a Fault, Not a Saturation

// WRONG — silently clamping a correctness counter.
if (credit_consume)
  tx_credit_q <= (tx_credit_q == '0) ? '0 : tx_credit_q - 1'b1;

This looks defensive and is the opposite. If a consume occurs at zero credits, the design has already sent a packet it had no permission to send — the receiver may already be overflowing. Clamping hides that, lets the machine continue, and destroys the evidence.

The unsigned-wrap version is worse still. 0 - 1 on a 6-bit counter is 63, so a single illegal send converts an empty credit pool into an apparently enormous one, and the sender floods the receiver.

Saturate diagnostic counters; assert on resource counters. Chapter 7.6's error counters saturate because losing history is the only harm. A credit counter that saturates is concealing a protocol violation that has already occurred.

// Illustrative — the violation is caught, not absorbed.
property p_no_consume_at_zero;
  @(posedge clk) disable iff (!rst_n)
    credit_consume |-> (tx_credit_q != '0);
endproperty
 
property p_credit_never_exceeds_advertised;
  @(posedge clk) disable iff (!rst_n)
    credit_valid_q |-> (tx_credit_q <= max_credit_q);
endproperty

The second catches credit duplication — the same freed slot generating two returns — and catches it at the moment it happens rather than at the overflow it eventually causes. §16 explains why that immediacy matters so much.

9. Return on Release, Not on Arrival

The conservation error that is easiest to write and hardest to see.

// WRONG — credit returned when the packet lands in the buffer.
assign credit_return = rx_push;

Trace it. A packet arrives and occupies a slot; a credit returns immediately; the sender spends it and sends another packet — while the first is still occupying its slot, because the consumer has not drained it. The buffer now holds two packets against one slot's worth of permission, and with a slow consumer this compounds until it overflows.

The credit describes the slot, not the packet. It may return only when the slot is genuinely free.

// Illustrative — return follows the consumer, not the arrival.
always_ff @(posedge clk or negedge rst_n) begin
  if (!rst_n) begin
    rx_occ_q      <= '0;
    credit_return <= 1'b0;
  end else begin
    credit_return <= 1'b0;                       // default: no return this cycle
 
    unique case ({rx_push, rx_drain})
      2'b10 : rx_occ_q <= rx_occ_q + 1'b1;
      2'b01 : begin
                rx_occ_q      <= rx_occ_q - 1'b1;
                credit_return <= 1'b1;           // a slot became free
              end
      2'b11 : begin
                // Simultaneous: occupancy unchanged, but a slot DID free and
                // was immediately refilled — the credit is still owed back.
                credit_return <= 1'b1;
              end
      default: ;
    endcase
  end
end

Architecture. Permission is regenerated by release, and only the consumer knows when release happens.

State. Occupancy plus a return pulse. Per-buffer-slot lifetime — each return corresponds to exactly one slot becoming available.

Cycle behaviour. The 2'b11 case is the subtle one and is worth reading twice: occupancy is unchanged because one packet left and another arrived, but a slot genuinely freed, so the credit is owed. Omitting the return here leaks one credit per simultaneous push-and-drain — which on a steadily flowing link is most cycles, so throughput collapses within moments and the bug is at least loud.

Contract. Exactly one return per released slot. Not zero, not two.

Failure. Returning on arrival overflows the buffer, as above. Returning twice for one slot inflates the sender's count past reality, caught by §8's maximum assertion.

// Illustrative — every return corresponds to a real release.
property p_return_implies_release;
  @(posedge clk) disable iff (!rst_n)
    credit_return |-> $past(rx_drain);
endproperty

10. A Worked Trace

Depth 4, four credits advertised, illustrative round-trip latency of three cycles for a return. Real return latency is architecture-specific; the numbers here exist only to make the mechanism visible.

Four readings.

Cycles 5 and 6 are correct behaviour, not a fault. The sender has packets and no permission; the mechanism is doing its job.

The return from cycle 7 does not arrive until cycle 10 — three cycles of round trip. During that window the receiver had free space that the sender could not use, which is §11's entire subject.

At cycle 6 every credit is accounted for: 0 held + 0 in flight + 4 occupied = 4. The invariant holds at every row, and checking it row by row is exactly what §18's reference model does continuously.

The sender never learns the buffer is full. It only ever knows it has no credits. That is the design working as intended — the sender does not need to model the receiver, only to hold permission.

11. Bandwidth-Delay Product

The performance insight, and the reason a correct credit machine can still be slow.

Between spending a credit and getting one back, there is a round trip: the packet travels, the receiver stores and eventually drains it, and the return travels back. Call that credit round-trip latency, T_rt. If the sender can transmit one packet per cycle, then during T_rt cycles it can spend T_rt credits.

credits_needed_for_full_rate  ≳  transmit_rate × credit_round_trip_latency

with transmit_rate in packets per cycle and T_rt in cycles, so the product is in packets — the number of credits that must be outstanding simultaneously to keep the link busy.

A credit scheme with too few credits is correct and slow. No error, no overflow, no assertion fires — the sender simply spends its last credit and waits for the first return, every round trip, forever.

Concretely: at one packet per cycle with a 20-cycle round trip, fewer than about 20 credits produces bubbles even when the consumer drains instantly. With 4 credits you get 4 packets and then a 16-cycle stall, repeating — roughly 20 % of achievable throughput, with everything reporting healthy.

Three practical consequences:

• Buffer depth is a latency decision, not just a capacity one. The receiver's buffer must be deep enough to hold the round-trip's worth of traffic, or the credits cannot exist to be advertised.
• Return latency is worth optimising. UCIe implementations have reason to make returns cheap — one described approach encodes credit returns on the Valid framing signal rather than as separate traffic.
• This is a design-time calculation, and getting it wrong produces a link that passes every functional test at a fraction of its rated bandwidth.

// Illustrative diagnostic — is the link starved, and for how long?
logic                 credit_starved;
logic [STARVE_W-1:0]  starved_cycles_q;    // saturating
logic                 starved_seen_q;      // sticky
 
assign credit_starved = packet_pending && (tx_credit_q == '0) && link_operational;
 
always_ff @(posedge clk or negedge rst_n) begin
  if (!rst_n) begin
    starved_cycles_q <= '0;
    starved_seen_q   <= 1'b0;
  end else begin
    if (credit_starved && !(&starved_cycles_q))
      starved_cycles_q <= starved_cycles_q + 1'b1;
    if (credit_starved) starved_seen_q <= 1'b1;
  end
end

Architecture. Under-credited is invisible without instrumentation, because it is not an error.

State. A saturating cycle counter and a sticky flag — diagnostic lifetime, surviving everything short of a broad reset (Chapter 8.1 §18).

Contract. Diagnostics only; this must never gate correctness.

Failure. Without it, "the link is slower than the datasheet" has no local evidence, and the investigation starts by opening the PHY.

12. Backpressure Is State, Not a Wire

Chapter 5.5 established that backpressure propagates through time. Credits are the sharpest form of that idea.

There is no signal travelling backwards to stop the sender. There is a number in a register on the sending die, updated by messages that already crossed the link. When the receiver fills up, the sender does not find out — it simply stops receiving returns, and its counter drains to zero.

Two consequences that catch people out:

Backpressure arrives late by exactly the return latency. A receiver that fills at cycle N affects the sender's behaviour around cycle N + T_rt. The system is always operating on information that is one round trip old.

The sender cannot distinguish causes. Zero credits looks identical whether the consumer is slow, the return path is broken, or the link went down. Only §16's separate diagnostics tell them apart.

13. Credits Across Reset and Recovery

Credit state is per-link-epoch, and both ends must re-establish it together.

The failure is a straightforward divergence: the receiver resets, clears its buffer, and now has all slots free — while the sender retains a credit count reflecting packets that no longer exist anywhere. Whichever side kept the stale view is now wrong, and the accounting is broken until something re-initialises it.

The middle two rows are Chapter 8.1 §17's partial-reset problem in its flow-control form: a local reset is local in its wiring and never in its effects.

// Illustrative — credit validity is evidence, and evidence has a lifetime.
always_ff @(posedge clk or negedge rst_n) begin
  if (!rst_n)                                credit_valid_q <= 1'b0;
  else if (credit_advert_valid)              credit_valid_q <= 1'b1;
  else if (link_left_operational || peer_restart_seen)
                                             credit_valid_q <= 1'b0;
end

Architecture. Credit state describes a relationship with a specific peer in a specific link epoch. When that epoch ends, the state is not merely stale — it is meaningless.

Contract. Both ends must re-advertise before traffic resumes. The exact reset and recovery semantics are a specification question for your revision; what is universal is that the two sides must re-baseline together, which is Chapter 8.5's cascading-invalidation rule applied to credits.

Failure. A sender that keeps credits across a peer restart sends into a buffer whose state it cannot possibly know.

14. Two Credit Layers Can Coexist

An architectural point that becomes important the moment Module 10 begins.

UCIe's link-level credits manage UCIe's own transport storage. The protocol being carried may have its own flow control with its own credits managing its buffers — and secondary technical material notes exactly this, giving PCIe's posted-header and posted-data credits as the example of credits that exist in addition to the link's.

Two layers, two purposes:

Having a link credit does not mean the protocol above will accept the packet, and having a protocol credit does not mean the link can carry it. They are independent vetoes, exactly like the three-way conjunction in §7.

The practical consequence for debug: a stall at the protocol layer and a stall at the link layer look identical from a distance — traffic stops, nothing errors. Distinguishing them means reading both credit counters, which means both must be observable.

15. Deadlock Is Not Starvation

Credit accounting can be perfectly legal while nothing progresses. The distinction matters because the two have different fixes.

Starvation is an arbitration failure: a stream never gets selected, but the system as a whole is making progress. Chapter 9.3 §11 covered it; fixing it means changing the scheduler.

Deadlock is a cyclic resource dependency in which nothing can progress. The classic shape:

Die A holds a request in its buffer, and cannot free the slot
   until it sends a response.
Die A cannot send the response because it has no credits.
Credits return to A only when Die B drains its receive buffer.
Die B cannot drain, because processing that entry requires
   the response A is trying to send.
   → cycle: A waits on B waits on A

Nothing is broken. Every counter is legal, every buffer is within bounds, and no assertion fires — because deadlock is a liveness failure and all the safety properties still hold.

The architectural defences are the standard ones, and the reason to name them is that they are decisions to be made deliberately rather than properties to hope for:

• Separate resources for dependent traffic classes. If responses cannot be blocked by requests, the cycle cannot form. This is why protocols distinguish request and response channels and why §14's protocol-level credits are typically per class.
• Guaranteed drain. A receiver that can always eventually consume, independent of anything it must transmit, breaks the dependency.
• Never let a response depend on acquiring a request-class resource.

Do not assume UCIe defines a deadlock-freedom property for arbitrary carried protocols. The transport can only guarantee things about its own resources; a deadlock created by your protocol's dependency structure is yours to prevent — which is exactly the payload-opacity consequence from Chapter 9.1.

// Illustrative — bounded progress, under explicitly stated assumptions.
// Assumes: link operational, peer eventually drains, returns are not lost.
// Without those, no progress property can hold, and saying so is the point.
property p_pending_work_eventually_sends;
  @(posedge clk) disable iff (!rst_n)
    (packet_pending && link_operational && peer_draining)
      |-> ##[1:MAX_CREDIT_WAIT] tx_fire;
endproperty

Bounded, and with the assumptions written into the antecedent. An unbounded eventuality is vacuous in simulation, and without peer_draining the property is simply false — a genuinely blocked sender is behaving correctly.

16. Leakage and Duplication

Two failure modes with opposite arithmetic and opposite signatures.

Credit leakage — consumed but never returned. The count drifts down, throughput falls gradually, and eventually the sender stalls permanently. No errors, no CRC failures, and often an empty receive buffer while the sender reports zero credits — that combination is close to diagnostic on its own. Common causes: the 2'b11 case of §9 omitted, a return dropped during recovery, or a return generated but lost.

Credit duplication — one freed slot generating two returns. The count drifts up past real capacity, and the eventual symptom is a receiver overflow that occurs long after and far from the bug. This is why §8's p_credit_never_exceeds_advertised matters so much: it converts a delayed, displaced overflow into an immediate assertion failure at the moment the extra credit appears.

The retry path is a common source of both, which is why Chapter 9.4's separation matters: a replay re-transmits a packet that already consumed a credit, so a retry must not consume a second credit unless the contract says the receiver's slot was genuinely released and re-occupied. Getting that wrong leaks a credit per retry.

17. Coverage

// Illustrative flow-control coverage — not UCIe-defined.
covergroup cg_credit @(posedge clk);
 
  cp_credit : coverpoint tx_credit_q {
    bins zero    = {0};
    bins one     = {1};
    bins mid     = {[2 : MAX_CREDIT-1]};
    bins full    = {MAX_CREDIT};
  }
  cp_event : coverpoint {credit_consume, credit_return} {
    bins idle        = {2'b00};
    bins consume     = {2'b10};
    bins ret         = {2'b01};
    bins simultaneous= {2'b11};        // the §6 case
  }
  cp_rx_occ : coverpoint rx_occ_q {
    bins empty = {0}; bins partial = {[1:RX_DEPTH-1]}; bins full = {RX_DEPTH};
  }
  cp_starved : coverpoint credit_starved;
  cp_replay  : coverpoint replay_space;
 
  // Was the simultaneous case exercised at the counter boundaries?
  x_event_by_credit  : cross cp_event, cp_credit;
  // Did credits ever run out while the replay buffer still had room, and vice versa?
  x_credit_by_replay : cross cp_credit, cp_replay;
  // Was the receiver ever full while the sender still believed it had credit?
  x_occ_by_credit    : cross cp_rx_occ, cp_credit;
 
endgroup

Why x_event_by_credit. Simultaneous consume-and-return at a mid-range count is benign even with the §6 bug present, because the drift takes time to matter. Simultaneous events at zero and at maximum are where the arithmetic error becomes an immediate violation, and the cross is what proves those corners were reached.

Why x_credit_by_replay. Chapter 9.4 and this chapter each hold a veto, and a regression that only ever exhausts one of them has never tested the interaction — including the case where credits are available and replay is not, which is §7's third failure mode.

18. The Reference Credit Machine

The scoreboard is unusually simple here and unusually powerful, because credit flow control has a closed-form invariant that a model can check every cycle.

What it tracks:

advertised        — capacity the receiver announced this epoch
consumes          — count of packets sent
returns           — count of credits returned
expected_credit   = advertised - consumes + returns
rx_occupancy      — model of the receiver's buffer

What it checks, every cycle:

• expected_credit == tx_credit_q — the DUT's counter matches the model exactly.
• expected_credit >= 0 and expected_credit <= advertised — the accounting stays legal.
• expected_credit + in_flight + rx_occupancy == advertised — the conservation law of §2, checked continuously.
• On re-advertisement, both model and DUT re-baseline together (§13).

Why the third check is the valuable one. The first catches a counter that has drifted; the third catches where the drift came from, because it fails at the exact cycle a packet or credit stops being accounted for. A leaked credit fails it the moment the missing return should have occurred, not thousands of cycles later when throughput has visibly decayed.

What to inject: a receiver that drains slowly, and one that drains in bursts; simultaneous consume-and-return sustained across the counter's full range; a retry while credits are outstanding; a peer restart with a non-zero credit count; and re-advertisement with a different capacity than before — which is the case that catches a model or a DUT that treats the advertised maximum as a constant.

19. Debug Checklist

1. Is the link operational? Chapter 8.5 — credits are meaningless below that.
2. Was a credit advertisement received? If credit_valid_q is low, this is a bring-up problem, not a flow-control one.
3. What is the current credit count, and what was advertised? The two numbers together frame everything below.
4. Is the receive buffer empty while the sender has zero credits? That combination is close to conclusive for leakage (§16).
5. Is the receive buffer full? Then the consumer is the bottleneck, and the credit machine is working.
6. Does the conservation sum balance? Held + in flight + occupied should equal advertised (§18).
7. Were consume and return ever coincident? §6 — check the counter across such a cycle.
8. Does every release generate exactly one return? Not zero (leak), not two (duplication).
9. Did a retry consume a second credit? §16 — a common leak.
10. Is replay space, rather than credits, the blocker? §7 — three different vetoes, three different fixes.
11. Did a reset or recovery occur without re-advertisement? §13.
12. Is this starvation or deadlock? Starvation means someone else is progressing; deadlock means nobody is (§15).
13. Is throughput low with no stall at all? Then it is bandwidth-delay product, not a bug (§11).

Steps 3 to 5 resolve most cases from two register reads. Step 13 is worth keeping in mind, because a correct, under-credited link produces a performance complaint with no defect to find.

20. Common Misconceptions

"A credit is just another ready signal." A ready signal is a live wire from the block that will accept the transfer. A credit is permission to use storage on a different die, held as a number, updated by messages that already crossed the link (§12).

"Credit can return when the packet arrives." It describes the slot, not the packet. Returning on arrival lets the sender refill a slot that is still occupied (§9).

"Unsigned underflow will obviously fail loudly." 0 − 1 becomes the maximum, so a single illegal send converts an empty pool into an apparently huge one (§8).

"Two independent if-statements are fine for increment and decrement." When both fire the last wins, and the counter drifts silently until the receiver overflows much later (§6).

"Receiver depth can be hard-coded at the sender." It belongs to the other die and is learned by advertisement; a too-large constant overflows the receiver with no flow-control error (§5).

"If credits are correct, deadlock is impossible." Every counter can be legal while a cyclic dependency stops all progress — safety holds, liveness does not (§15).

"Credit exhaustion propagates instantly." It arrives one round trip late, and the system always acts on information that old (§12).

"A credit guarantees replay-buffer space." Three independent resources each hold a veto, and they fail on different dies with different symptoms (§7).

"More credits always increase throughput." Beyond the bandwidth-delay product they add nothing; below it the link is correct and slow (§11).

"Shared credits are always more efficient." Sharing improves utilisation and permits one stream to starve another; partitioning isolates and can strand capacity (§14, §15).

"Each side can manage its credit state through reset independently." They describe one relationship; if one clears and the other does not, the accounting diverges silently (§13).

21. Understanding Check

22. Summary and What Comes Next

A credit is permission backed by real storage, and credit flow control is distributed accounting for a finite resource on another die. The conservation law — credits held, plus packets in flight, plus slots occupied, equals advertised capacity — is the whole subject, and every bug is a drift between the accounting and the storage it describes.

The mechanisms: capacity is learned by advertisement, never assumed, because the buffer belongs to the other die. The counter takes one assignment path with all four consume/return cases explicit, because two independent statements silently lose the decrement. Underflow is asserted, not saturated, because a consume at zero means a packet has already been sent without permission — and the unsigned wrap turns one violation into an apparently enormous credit pool. Returns follow release, not arrival, and the simultaneous push-and-drain case still owes a credit. And sending requires three independent permissions — the link, the receiver's credit, and local replay space — each owned by a different mechanism and each failing on a different die.

Two things that are not bugs: credit exhaustion arrives one round trip late, because backpressure here is state rather than a wire; and a link with fewer credits than its bandwidth-delay product is correct and slow, which needs instrumentation because it produces a performance complaint with no defect to find.

And the failure taxonomy worth memorising: leakage drifts down and stalls with an empty receiver; duplication drifts up and overflows much later; starvation means someone else is progressing; deadlock means nobody is, with every counter legal and every safety property holding.

Module 9 is complete. Streaming showed how UCIe carries a protocol it does not interpret — framing, ordering, reliability, and now flow control, with the payload opaque throughout. Module 10 takes the opposite approach: rather than carrying an opaque payload, UCIe maps a protocol it does know, natively:

• 10.1 — PCIe Tunneling Over UCIe — what changes and what deliberately does not when a PCIe protocol engine talks through UCIe instead of a conventional PCIe PHY.

Browse the full path on the UCIe tutorials index.