UCIe · Module 8
End-to-End Bring-Up Flow
How reset, discovery, training, and calibration compose into one bring-up — bring-up as a dependency graph, local versus bilateral completion, evidence lifetime and cascading invalidation, attempt epochs, phase versus system timeouts, and first-error preservation.
Four chapters have each taken one mechanism apart. Reset established known local state across independently clocked domains. Discovery established that a peer exists and that the sideband works. Mainband initialisation and training qualified the physical resources and aligned them. Calibration chose an operating point inside the margin that remained.
Each was studied in isolation, and each is stateful, retryable, and independently capable of failing. This chapter asks the question none of them could: what happens when you run them together?
The answer is more interesting than "in order". A phase does not merely follow its predecessor — it depends on it, consumes evidence it produced, and becomes invalid if that evidence stops being true. Most of the hardest bring-up bugs in real systems are not inside any one phase. They are in the joints.
1. The One-Sentence Model
Bring-up is a dependency graph, not a checklist. Each phase consumes evidence produced by earlier phases, and evidence that stops being true invalidates everything derived from it.
A checklist has one failure mode: an item not done. A dependency graph has two, and the second is the one that produces field escapes — an item that was done and has since become false, while everything downstream still believes it.
That distinction drives the whole chapter. §5 is about representing evidence explicitly, §7 is about its lifetime, §8 is about propagating invalidation, and §14 is about the debug method that falls out.
2. The Dependency Table
| Phase | What must already be true | Evidence it produces | Who consumes it |
|---|---|---|---|
| Reset exit | supplies stable; required clocks running; per-domain resets released | known local state | everything |
| Sideband / discovery | local state known; sideband clock alive | peer exists; control path works; both ends agree an attempt is running | mainband init |
| Mainband init / training | peer reachable over sideband | qualified lanes, identity map, deskew, negotiated rate | operational logic |
| Calibration-related work | required lanes qualified; rate settled | committed operating settings | the active PHY |
| Operational | all of the above, still true | a link the Adapter may use | Adapter, then Protocol |
Read the last row carefully. It does not say "the previous phase completed". It says all of the above, still true — and the two are different claims, which §6 makes concrete.
Note also the second column of the discovery row. Reset produces local state; discovery is the first phase whose output says anything about the other die. That asymmetry is the subject of §4.
3. Evidence, Not Progress
The first design decision is how to represent where bring-up has got to. The tempting answer is a single state variable, and it is wrong for a reason worth naming.
// Illustrative bring-up RTL — not a UCIe-defined encoding.
// Each bit is EVIDENCE that a prerequisite currently holds — not a record
// that a phase once ran.
logic reset_done_q; // local domains released, clocks qualified
logic discovery_done_q; // peer reachable, bilateral agreement reached
logic training_done_q; // required lanes qualified, map and deskew committed
logic calibration_valid_q; // operating settings committed and in force
logic peer_operational_q; // the far end reports itself usable
logic fatal_fault_q; // something unrecoverable was observedArchitecture. Bring-up has several independent prerequisites that can each become false independently. A single position marker — "we are in phase 3" — cannot express which prerequisite failed, and cannot express that an earlier one stopped holding while a later one still reads complete.
State. One registered bit per prerequisite. Registered rather than combinational because these are conclusions drawn from qualified observations, several of which cross clock domains (Chapter 7.5 §9) and must be synchronised before they are believed.
Cycle behaviour. Each bit sets when its phase produces valid evidence and clears when that evidence is invalidated (§7). They do not clear merely because a later phase started.
Contract. The operational qualification in §5 reads all of them. Debug reads them individually — which is the point.
Failure. With one state variable, a debugger who finds the machine parked before operational learns only where it stopped, not why, and cannot distinguish "never got there" from "got there and lost it".
A single
link_upbit is a summary that destroys its own evidence. Keep the terms; derive the summary.
4. Local Completion Is Not Bilateral Completion
Chapter 8.2 §9 established this for discovery, where UCIe's own exit condition requires having both sent and received the done response. It generalises to every phase, and it is the single most common architectural mistake in bring-up.
At each phase, three distinct questions:
- What has this endpoint finished? Local, cheap to know, and insufficient.
- What evidence exists that the peer finished? Requires something to have come back.
- What is safe to expose upward? Only the intersection.
The failure when this is collapsed is asymmetric and nasty. Endpoint A completes, declares the link operational to its Adapter, and begins transmitting. Endpoint B is still training, so it is not observing. Data goes into a receiver that is not listening — and no layer reports an error, because each behaved correctly according to its own local view. Eventually B times out, restarts, and now the two ends are in phases that are not merely different but incompatible.
// Illustrative — the local view and the peer's view are different facts.
assign local_ready = reset_done_q && discovery_done_q &&
training_done_q && calibration_valid_q;
// Operational requires BOTH, because "I am ready" says nothing about the peer.
assign link_operational = local_ready && peer_operational_q && !fatal_fault_q;5. Operational Is Derived, Never Asserted
// WRONG — the last phase treated as proof of all the earlier ones.
assign link_operational = training_done;Training completing says training completed. It does not say reset is still released, that discovery's evidence is still valid, that the peer has not restarted since, or that calibration produced a usable setting. Each of those can become false after training finishes, and this expression cannot see any of it.
The correct form is a conjunction over current evidence:
// Illustrative bring-up RTL — not a UCIe-defined signal or encoding.
assign link_operational =
reset_done_q && // local domains still out of reset
discovery_done_q && // peer still the one we discovered
training_done_q && // lanes still qualified, config still committed
calibration_valid_q && // operating point still in force
peer_operational_q && // far end still agrees
!fatal_fault_q; // nothing unrecoverable observedArchitecture. Operational is a conclusion about the present, not a milestone that was passed.
State. None of its own — deliberately. Every term is registered elsewhere, and the conjunction is combinational so that the loss of any prerequisite withdraws operational status in the same cycle it is observed.
Cycle behaviour. Combinational over registered inputs. A prerequisite clearing deasserts link_operational immediately.
Contract. The Adapter gates on this. Chapter 7.1 §10's rule applies: acceptance is conditioned on it, and it must be the synchronised version in the Adapter's clock domain.
Failure. Any term omitted becomes a way for the link to appear usable while a prerequisite is false — and each omission has a different silicon symptom, which is why §14's checklist walks them individually.
A caution against reading this as a template. A raw AND is the right teaching form and rarely the whole implementation. Real designs typically add hysteresis so a momentary glitch does not flap the link, a defined path for withdrawing operational status in an orderly way rather than instantaneously, and separate reporting of why it was withdrawn. The invariant to keep is that operational is derived from current evidence; the exact conditioning is architecture.
// Illustrative — operational implies every prerequisite currently holds.
property p_operational_requires_all_prerequisites;
@(posedge clk) disable iff (!rst_n)
link_operational |-> (reset_done_q && discovery_done_q && training_done_q
&& calibration_valid_q && peer_operational_q
&& !fatal_fault_q);
endproperty
// Illustrative — nothing crosses to the peer before the link is operational.
property p_no_traffic_before_operational;
@(posedge clk) disable iff (!rst_n)
payload_accepted |-> link_operational_sync;
endpropertyThe second property deliberately references the synchronised operational status. Chapter 7.5 §16 established that a status bit generated in one domain and consumed in another is a crossing; asserting against the raw signal proves something about a domain the Adapter does not live in.
6. Evidence Has a Lifetime
For every evidence bit, four questions — and the answers are not the same:
| Evidence | Invalidated by local reset | Invalidated by peer restart | Survives a training retry | Survives runtime recovery |
|---|---|---|---|---|
reset_done_q | yes | no | yes | yes |
discovery_done_q | yes | yes | yes | usually |
training_done_q | yes | yes | no | depends on the recovery |
calibration_valid_q | yes | yes | no | no, if settings were disturbed |
peer_operational_q | yes | yes | no | no |
| retry counts, first-error cause | no — deliberately | no | no — deliberately | no |
| per-lane historical error counts | no | no | no | no |
Three rows repay attention.
discovery_done_q dies on peer restart. The peer you discovered and the peer that is there now may not be in the same state. This is the row people forget, and forgetting it is §8's bug.
calibration_valid_q dies on a training retry. Retraining can change the lane set, the map, or the rate, and an operating point chosen for the previous configuration is not valid for the new one — Chapter 8.4 §2's point that calibration's output is an optimum under conditions.
Diagnostic state deliberately survives everything short of a broader reset. Chapter 8.1 §18: a counter cleared by the event it counts counts to one. A link that succeeds on the third attempt every time must be distinguishable from one that succeeds first time.
7. The Dependency Cone
The structural consequence, and the chapter's most transferable idea:
Evidence has a dependency cone. Invalidating a prerequisite must invalidate everything derived from it — transitively, and in the same cycle.
discovery evidence invalid
→ training result no longer trustworthy (it was trained against that peer)
→ calibration result no longer trustworthy (chosen for that configuration)
→ operational no longer true// Illustrative bring-up RTL — cascading invalidation, ordered by dependency.
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
discovery_done_q <= 1'b0;
training_done_q <= 1'b0;
calibration_valid_q<= 1'b0;
peer_operational_q <= 1'b0;
end else begin
// --- set on genuine completion -------------------------------------
if (discovery_complete_evt) discovery_done_q <= 1'b1;
if (training_complete_evt) training_done_q <= 1'b1;
if (calibration_commit_evt) calibration_valid_q <= 1'b1;
if (peer_reports_operational)peer_operational_q <= 1'b1;
// --- invalidate, and let the cone collapse in ONE cycle -------------
// Written after the sets above so invalidation always wins on a tie.
if (peer_restart_seen || sideband_lost) begin
discovery_done_q <= 1'b0;
training_done_q <= 1'b0; // trained against a peer that has gone
calibration_valid_q <= 1'b0;
peer_operational_q <= 1'b0;
end else if (retrain_started) begin
training_done_q <= 1'b0;
calibration_valid_q <= 1'b0; // settings were for the old configuration
peer_operational_q <= 1'b0;
end else if (recalibration_started) begin
calibration_valid_q <= 1'b0;
peer_operational_q <= 1'b0;
end
end
endArchitecture. Derived evidence outlives its basis unless something explicitly kills it. Ordering the invalidation branches after the set branches, and cascading each one down the cone, makes that impossible to forget.
State. The four evidence bits from §3.
Cycle behaviour. The whole cone collapses on one edge. That matters: staging invalidation across cycles creates a window in which link_operational is computed from a mixture of valid and stale terms — the same class of hazard as Chapter 8.3 §11's lane-by-lane commit.
Contract. Everything reading link_operational may assume that when a prerequisite dies, the conclusion dies with it, immediately.
Failure. See §8.
DV. Assert the cone directly rather than trusting the branch structure to stay correct:
// Illustrative — killing a prerequisite kills everything derived from it.
property p_peer_restart_collapses_cone;
@(posedge clk) disable iff (!rst_n)
peer_restart_seen |=> (!discovery_done_q && !training_done_q
&& !calibration_valid_q && !peer_operational_q);
endproperty
property p_retrain_invalidates_calibration;
@(posedge clk) disable iff (!rst_n)
retrain_started |=> (!training_done_q && !calibration_valid_q);
endproperty8. The Stale-Downstream Bug
The failure the cone exists to prevent, in the form it actually appears:
// WRONG — the trigger is handled, but only at the level it was observed.
if (peer_restart_seen)
discovery_done_q <= 1'b0; // and nothing elseCorrect as far as it goes, and catastrophic. training_done_q, calibration_valid_q, and peer_operational_q all remain set. If link_operational is derived from a subset — or is itself registered rather than recomputed — the link continues to report itself usable.
The cycle-level consequence:
| Cycle | Event | discovery_done_q | training_done_q | link_operational | Adapter |
|---|---|---|---|---|---|
| n | steady operation | 1 | 1 | 1 | transmitting |
| n+1 | peer resets; A has not noticed | 1 | 1 | 1 | transmitting into a resetting die |
| n+2 | restart detected | 0 | 1 | 1 ← the bug | still transmitting |
| n+3 | Adapter accepts another item | 0 | 1 | 1 | item accepted, owned, undeliverable |
| … | peer completes its own reset and begins discovery | 0 | 1 | 1 | A is mid-transmission into SBINIT |
Row n+3 is where it becomes a data-integrity problem rather than a performance one. Chapter 7.1 §12's rule — once accepted, an item's fate must remain knowable — is violated by a link that accepts work it can no longer deliver and does not know it.
And the debug signature is misleading in a specific way: the symptom appears at the Adapter, as transactions that never complete, so the investigation starts several layers above the actual defect.
9. Attempts and Epochs
Chapter 8.2 §8 introduced correlation for a single discovery exchange. At the composition level the same hazard exists across phases, and it is larger.
A bring-up attempt spans reset through operational. If attempt n fails partway and attempt n+1 starts, then in flight there may still be a discovery response from n, a training result from n, or a calibration measurement from n. Any of them arriving during n+1 can complete a phase on evidence generated under conditions that no longer exist.
// Illustrative implementation technique — NOT a UCIe-defined field.
// A bring-up generation, advanced once per attempt, so that results produced
// by an abandoned attempt cannot satisfy the current one.
logic [EPOCH_W-1:0] bringup_epoch_q;
always_ff @(posedge clk or negedge global_rst_n) begin
if (!global_rst_n) bringup_epoch_q <= '0;
else if (attempt_restart) bringup_epoch_q <= bringup_epoch_q + 1'b1;
end
// A phase result is only accepted if it belongs to the attempt now running.
assign accept_phase_result = phase_result_valid &&
(phase_result_epoch == bringup_epoch_q);Architecture. Phases are long relative to the events that abandon them, so results outlive their attempts.
State. A small counter, reset by the global reset rather than by the attempt restart — otherwise it could not distinguish attempts, which is Chapter 8.1 §18's observability-domain rule appearing again.
Cycle behaviour. Advanced once per restart; compared combinationally against arriving results.
Contract. Everything that can complete a phase must carry or be tagged with the epoch it was produced under.
Failure. Without it, attempt n+1 can be completed by attempt n's evidence — and because the resulting link is "operational" on stale grounds, the failure surfaces much later as unexplained traffic loss.
Important boundary. This is an internal implementation technique. Do not assume UCIe messages carry such a generation field; where the protocol provides its own correlation, use that. The epoch here correlates local phase machinery across a restart, which is a problem your own design owns regardless of what the wire protocol offers.
10. Timeouts at Two Levels
Chapters 8.2 through 8.4 each built a phase-local timer. Composition adds a second kind, and the two are not redundant.
| Phase timeout | Overall bring-up budget | |
|---|---|---|
| Scope | one phase | the whole attempt |
| Answers | which mechanism is stuck | is the system still waiting? |
| Duration | tuned to that phase's worst case | the sum, plus retries, plus margin |
| On expiry | fail that phase with its own cause | abandon the attempt; report to the platform |
| If it is the only one | cannot bound total wait | cannot localise the failure |
Phase timeouts exist for diagnosis. The overall budget exists for system liveness. A design with only one of them is either undiagnosable or unbounded.
The overall budget matters more than it looks, because retries multiply. Three phases that each retry three times can, with generous per-phase timeouts, keep a platform waiting far longer than any single timeout suggests — and platform software usually has its own patience limit, so the choice is between reporting a bounded failure yourself or having something above you declare the device broken.
// Illustrative — the overall budget is bounded and its expiry is observable.
property p_bringup_budget_bounded;
@(posedge clk) disable iff (!rst_n)
(bringup_active && bringup_budget_expired) |=> (bu_state_q == BU_FAILED);
endproperty11. First-Error Preservation
A subtle RTL principle with a large debug payoff.
When bring-up fails, several errors usually occur in sequence: the root cause, then the consequences of abandoning the phase, then a timeout somewhere during cleanup. A naive cause register records the last one.
// WRONG — the last error overwrites the root cause.
if (error_event)
failure_cause_q <= current_error;By the time software reads it, failure_cause_q holds whatever happened most recently — typically a cleanup-path timeout, which points nowhere useful. The genuine failure, several events earlier, has been erased by its own consequences.
// Illustrative bring-up RTL — first meaningful error wins; later ones are noted.
typedef enum logic [2:0] {
BU_FAIL_NONE = 3'd0,
BU_FAIL_RESET = 3'd1,
BU_FAIL_DISCOVERY = 3'd2,
BU_FAIL_TRAINING = 3'd3,
BU_FAIL_CALIBRATION = 3'd4,
BU_FAIL_PEER_RESTART = 3'd5,
BU_FAIL_BUDGET = 3'd6
} bringup_fail_t;
bringup_fail_t first_fail_q; // root cause — written once per attempt
bringup_fail_t last_fail_q; // most recent — useful for sequence
logic [FAIL_CNT_W-1:0] fail_events_q; // saturating count this attempt
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
first_fail_q <= BU_FAIL_NONE;
last_fail_q <= BU_FAIL_NONE;
fail_events_q <= '0;
end else if (attempt_restart) begin
first_fail_q <= BU_FAIL_NONE; // per-attempt: a new attempt, new root
last_fail_q <= BU_FAIL_NONE;
fail_events_q <= '0;
end else if (error_event) begin
if (first_fail_q == BU_FAIL_NONE)
first_fail_q <= current_error; // sticky within the attempt
last_fail_q <= current_error;
if (!(&fail_events_q)) fail_events_q <= fail_events_q + 1'b1;
end
endArchitecture. Root cause and most-recent symptom are different facts, and debugging needs the first while sequence reconstruction needs the second.
State. Two cause registers and a saturating event count. The count is what tells you whether you are looking at one clean failure or a cascade.
Cycle behaviour. first_fail_q is written on the first error of an attempt and then locked; last_fail_q tracks every error; both clear at the attempt boundary.
Contract. Diagnostics read all three. Nothing functional depends on them.
Failure. With last-wins only, the recorded cause is the cleanup symptom — and teams chase timeouts that are consequences, not causes.
Note the interaction with §14's per-phase causes. The top-level cause locates the phase; each phase keeps its own detailed cause register (Chapter 8.3 §15, Chapter 8.4 §11). The top-level register must not overwrite the phase-level detail — they are a hierarchy, and collapsing them loses exactly the specificity that makes the hierarchy worth having.
// Illustrative — the root cause survives subsequent errors within an attempt.
property p_first_error_preserved;
@(posedge clk) disable iff (!rst_n)
((first_fail_q != BU_FAIL_NONE) && !attempt_restart) |=> $stable(first_fail_q);
endproperty12. The Composition Controller
// Illustrative bring-up wrapper FSM — NOT UCIe's link state machine, whose
// states Chapter 8.6 covers. This exists to teach dependency composition.
typedef enum logic [2:0] {
BU_RESET = 3'd0,
BU_DISCOVERY = 3'd1,
BU_MB_INIT = 3'd2,
BU_TRAIN = 3'd3,
BU_QUALIFY = 3'd4, // calibration-related work and commit
BU_LINK_UP = 3'd5,
BU_RESTART = 3'd6,
BU_FAILED = 3'd7
} bringup_state_t;
bringup_state_t bu_state_q, bu_state_d;
always_comb begin
bu_state_d = bu_state_q; // explicit default: hold
unique case (bu_state_q)
BU_RESET : if (reset_done_q) bu_state_d = BU_DISCOVERY;
BU_DISCOVERY : if (phase_failed) bu_state_d = BU_RESTART;
else if (discovery_done_q) bu_state_d = BU_MB_INIT;
BU_MB_INIT : if (phase_failed) bu_state_d = BU_RESTART;
else if (mb_init_done) bu_state_d = BU_TRAIN;
BU_TRAIN : if (phase_failed) bu_state_d = BU_RESTART;
else if (training_done_q) bu_state_d = BU_QUALIFY;
BU_QUALIFY : if (phase_failed) bu_state_d = BU_RESTART;
else if (calibration_valid_q) bu_state_d = BU_LINK_UP;
BU_LINK_UP : if (!link_operational) bu_state_d = BU_RESTART;
BU_RESTART : bu_state_d = (attempts_q >= MAX_ATTEMPTS) ? BU_FAILED : BU_RESET;
BU_FAILED : ; // terminal until reset
default : bu_state_d = BU_RESET; // illegal encoding recovers
endcase
endArchitecture. Something must sequence the phases, notice when a prerequisite dies, and decide whether to retry or give up. The wrapper is that something.
State. An eight-value state register plus the attempt counter.
Cycle behaviour. One transition per clock. Note BU_LINK_UP's exit: it leaves on !link_operational, so the derived conjunction of §5 is what pulls the machine out of operation. The FSM does not separately re-check prerequisites — there is one definition of operational and everything uses it.
Contract. The wrapper owns retry policy; each phase owns its own mechanism and its own detailed failure cause.
Failure. Without the BU_LINK_UP → BU_RESTART edge, a link that loses a prerequisite stays in the operational state with link_operational low — reporting "up" by state while reporting "not usable" by signal, which is exactly the kind of internal disagreement that makes silicon debug miserable.
13. Two Traces
A successful bring-up. Durations are illustrative; real phases differ by orders of magnitude, and Chapter 8.1 §3 noted that RESET alone carries a floor on the order of milliseconds for PLL stabilisation.
| Step | State | Evidence set | Note |
|---|---|---|---|
| 1 | BU_RESET | — | supplies stable; per-domain resets releasing |
| 2 | BU_RESET | reset_done_q | clocks qualified (Ch 7.5 §11) |
| 3 | BU_DISCOVERY | — | sideband up; out-of-reset transmitted |
| 4 | BU_DISCOVERY | — | peer's message received — local half done |
| 5 | BU_DISCOVERY | discovery_done_q | both directions observed (Ch 8.2 §9) |
| 6 | BU_MB_INIT | — | mainband initialisation; repair and reversal resolved |
| 7 | BU_TRAIN | — | per-lane evidence accumulating |
| 8 | BU_TRAIN | — | required lanes qualified; deskew measured |
| 9 | BU_TRAIN | training_done_q | map and deskew committed atomically (Ch 8.3 §11) |
| 10 | BU_QUALIFY | — | centring and calibration-related work |
| 11 | BU_QUALIFY | calibration_valid_q | operating settings committed |
| 12 | BU_QUALIFY | peer_operational_q | far end reports itself usable |
| 13 | BU_LINK_UP | link_operational | conjunction true; Adapter enabled |
Steps 4 and 5 are one phase and two different facts. Step 12 is the one designs omit.
A bring-up that retries. Same start; a lane fails during training.
| Step | State | Event | Evidence after | Diagnostics |
|---|---|---|---|---|
| 7 | BU_TRAIN | lane 9 never qualifies | — | — |
| 8 | BU_TRAIN | phase timeout | — | first_fail_q = TRAINING; phase cause LANE |
| 9 | BU_RESTART | attempt state cleared | discovery_done_q cleared too | attempts_q = 1; lanes_failed_ever_q[9] set |
| 10 | BU_RESET | epoch advanced | — | first_fail_q cleared for the new attempt |
| 11 | BU_DISCOVERY | re-run | discovery_done_q | — |
| 12 | BU_TRAIN | lane 9 qualifies; repair used | training_done_q | lanes_failed_ever_q[9] still set |
| 13 | BU_LINK_UP | success | link_operational | attempts_q = 1 retained |
Two things worth extracting. Step 9 clears discovery evidence as well as training evidence, because the restart re-runs from reset and the peer may have moved. And step 13 ships a working link that records it needed two attempts and which lane was implicated — a link that works and quietly took three tries is a field problem waiting to be discovered by a customer.
14. Peer Restart, Phase by Phase
The corner case that composition uniquely exposes.
| Peer restarts during | Local endpoint sees | Correct response |
|---|---|---|
| Discovery | timeout, or a response from the peer's previous attempt | abandon attempt; epoch prevents stale completion (§9) |
| Mainband init / training | patterns stop, or results stop arriving | invalidate training evidence; do not commit a partial configuration |
| Calibration work | measurements stall or become nonsense | do not commit; a measurement taken against a resetting peer is not a measurement |
| Operational | link status from the peer drops | collapse the whole cone (§7) and stop accepting traffic |
The last row is the one with data-integrity consequences, and it is §8's bug. The others cost time; that one can lose accepted work.
What must be true in all four cases: the local endpoint must detect the restart, invalidate the dependent evidence, and not accept stale completion from before it. The exact state the specification returns the link to is a normative question for the revision you implement — Chapter 8.6 examines the state machine — and this chapter deliberately does not invent a transition.
15. Verifying Composition
The scoreboard for this chapter is not a data checker. It is a bring-up reference model that tracks:
- the expected phase, given what the testbench has permitted to succeed;
- the current attempt number and epoch;
- which evidence bits should currently be valid;
- the legal next transitions from the current wrapper state;
- whether operational should currently be true;
- the expected first-failure cause, given the injected fault.
It then compares against the DUT's observable state. That last item is the one that catches the most: a testbench that only checks "did the link come up" cannot distinguish a link that came up correctly from one that came up on stale evidence.
// Illustrative bring-up coverage — not UCIe-defined.
covergroup cg_bringup @(posedge clk iff bu_state_change);
cp_fail_phase : coverpoint first_fail_q;
cp_result : coverpoint bu_state_q {
bins up = {BU_LINK_UP};
bins retry = {BU_RESTART};
bins failed = {BU_FAILED};
}
cp_attempts : coverpoint attempts_q {
bins first = {0}; bins retried = {[1:MAX_ATTEMPTS-1]}; bins exhausted = {MAX_ATTEMPTS};
}
cp_restart_phase : coverpoint peer_restart_phase; // which phase the peer died in
// The valuable one: did each failure phase ever RECOVER, or only ever fail?
x_phase_by_result : cross cp_fail_phase, cp_result;
// And was a peer restart survived from every phase?
x_restart_by_result : cross cp_restart_phase, cp_result;
endgroupWhy the first cross. Every regression reaches "no failure, link up". The cases that matter are each failure phase followed by each outcome — because a phase whose failure path has only ever been observed to end in BU_FAILED has never had its recovery path exercised, and recovery paths are where the state-lifetime bugs of §6 live.
16. Debug Checklist — Link Never Becomes Operational
Work the dependency graph, not the timeline. Each step is answerable from state.
- Is reset released locally, in every domain? Chapter 8.1 §20 — and a stopped clock legitimately holds reset.
- Is the sideband alive? No sideband, no discovery, no anything.
- Did discovery complete — bilaterally? Local completion is the trap (§4).
- Did mainband initialisation start? If not, discovery evidence is the prerequisite to check.
- Did the required lanes qualify? Read the per-lane counters, not the summary mask.
- Are the map and deskew valid and committed? Committed is distinct from computed.
- Did calibration-related work complete and commit?
- Which phase owns the first failure? Read
first_fail_q, notlast_fail_q(§11). - Did a peer restart invalidate later evidence? Check whether the cone collapsed or only its trigger did (§8).
- Did the wrapper retry, and how many times? A link that eventually works after retries is still a finding.
- Was per-attempt state actually cleared? Evidence inherited across attempts produces failures that depend on history.
- Could stale completion have satisfied the current attempt? Check the epoch (§9).
- Is
link_operationalderived from all current prerequisites, or from a convenient subset (§5)? - Which prerequisite is currently false? With §3's separate bits this is one register read; with a single
link_upbit it is a week.
Steps 1 to 3 resolve a large fraction. Steps 8, 9, and 14 are the ones that separate a design with evidence bits from one without — and step 14 is, in the end, the whole argument for §3.
17. Common Misconceptions
"Bring-up is a linear checklist." It is a dependency graph, whose distinctive failure is an item that was done and has since become false (§1).
"The last phase being done means the earlier ones are still valid." Each prerequisite can become false independently and after the fact (§5).
"One global timeout is enough." A phase timeout localises the failure; an overall budget bounds the wait. Each alone leaves the other problem unsolved (§10).
"A retry can preserve phase state." Per-attempt evidence must clear or a lane qualifies on data collected before it broke; diagnostic state must not clear or history is destroyed (§6, §13).
"The most recent error is the root cause." It is usually the cleanup symptom. Preserve the first (§11).
"A peer reset affects only discovery." It invalidates everything trained, calibrated, and agreed against that peer — the entire cone (§7, §8).
"Local completion means the peer is ready." It means one endpoint finished its half. Advancing alone transmits into a die that is not listening (§4).
"Operational can be reconstructed from any convenient done bit." Every omitted term is a way for the link to look usable while a prerequisite is false (§5).
"A retraining attempt can accept the previous attempt's completion." That is exactly what the epoch exists to prevent (§9).
"Reset → discovery → training → calibration is the normative UCIe ordering." It is a useful cognitive decomposition. The specification's state machine interleaves calibration-related work inside mainband training (§1).
18. Understanding Check
19. Summary and What Comes Next
Bring-up is a dependency graph, not a checklist. Each phase consumes evidence produced by earlier ones, and evidence that stops being true invalidates everything derived from it.
Represent that structure explicitly. Separate evidence bits, not one link_up — a summary destroys the information a debugger needs, and with separate bits "which prerequisite is false?" is one register read instead of a week. Operational is derived from current evidence, never asserted by the last phase that finished; every omitted term is a way for the link to look usable while a prerequisite is false.
Local completion is not bilateral completion. Each endpoint sees only its own half, and advancing alone means transmitting into a die that is not listening — with no layer reporting an error, because each behaved correctly locally.
Evidence has a lifetime and a cone. Discovery evidence dies on peer restart, calibration evidence dies on a training retry, and diagnostic state deliberately dies on nothing short of a broader reset. Invalidation must cascade in one cycle, or operational is briefly computed from a mixture of fresh and stale terms. The stale-downstream bug — clearing only the trigger — turns a peer restart into accepted work that can never be delivered.
Attempts need epochs, because phases are long relative to the events that abandon them. Timeouts come in two kinds: phase-local for diagnosis, overall for liveness. And the first error must survive its own consequences, with the top-level cause locating the phase and each phase keeping its own detail.
Bring-up ends with a link reporting itself operational. That is not the end of its state machine — it is the beginning of the part that runs for the lifetime of the system, with low-power transitions, error recovery, retraining, and the question of what state survives each of them:
- 8.6 — Link States — the link state machine after bring-up: what each state guarantees, what may be retained across transitions, quiescence before low power, recovery ownership, and the invariants that make transitions safe.
Browse the full path on the UCIe tutorials index.