Skip to content

PCIe · Module 18

Recovery — Repairing and Changing a Link That Already Exists

Recovery is not error retry and not replay. It is how an operating Link temporarily withdraws normal traffic, re-establishes physical certainty or changes its speed, and returns with a coherent configuration — or does not return at all.

Chapters 18.218.4 built a Link from nothing: presence, then communication, then a committed configuration.

Recovery is the opposite problem. A Link already exists. It has a negotiated speed, a lane map, a width, and traffic flowing over it. Then something happens — the receiver loses lock, software asks for a faster rate, the partner starts sending training sets, a burst of PHY errors arrives — and the Link must stop being a transport for a while, fix or change itself, and come back.

How does an operating Link withdraw normal traffic, re-establish physical certainty or change its operating rate, and return to a coherent configuration — without ever letting anything above the Physical Layer see a Link that is half-repaired?

1. The Verified Substates and the Verified Causes

3. Four Things Called "Retry"

A terminology table, because all four appear in the same bug reports.

TermLayerWho does itWhat is repeatedWhere taught
Software retrydriver / applicationsoftwarea whole operationoutside this curriculum
Completion error handlingTransactionrequester logica new Request with a new Tag13.1
Data Link replayData Linkhardwarethe same TLP, same sequence number15.1
LTSSM RecoveryPhysicalhardwarenothing — the Link is re-establishedthis chapter

The distinction that matters most for silicon debug is the third versus the fourth. Replay repeats a packet that still exists and still has its identity (Chapter 14.5). Recovery repeats no packet at all — it repairs the pipe, and whatever was in the replay buffer is still sitting there, unchanged, when the Link comes back (§18).

§1's cause register groups naturally into four kinds of reason, and each implies a different investigation.

Something was lost. Receive lock, alignment, channel bond, or the periodic evidence that the partner is still transmitting — direct_to_recovery_phy, direct_to_recovery_ch_bond, l0_to_rec_rcvr_lock_rx_infer. The Link is broken and Recovery tries to fix it.

Something is being changed on purpose. directed_speed_change, autonomous_width_change, directed_retrain_link. Nothing is wrong; the Link is being reconfigured, and Recovery is the only state in which that can happen safely.

The partner asked. l0_to_rec_rcvr_lock_rx_ts12, l0_to_rec_rcvr_lock_rx_8g_eie"The link partner is directing Recovery entry." Recovery is bilateral: one end deciding to retrain drags the other with it, and from the local point of view this is indistinguishable from a locally-caused entry except by the cause bits.

Another state is being routed through. direct_to_hot_reset, direct_to_disable, direct_to_loopback_entry. Recovery is on the path, not the destination — and this is why "the Link is in Recovery" is not by itself a fault report.

5. The Four Substates Are Four Different Problems

Recovery.RcvrLock — re-establish receive lock and training communication. The same class of problem as Chapter 18.3's Polling, on a Link that already worked. §1 confirms this is where L0 lands: idle_infer_l0_to_rec_rcvr_lock is described as "Electrical Idle inference status in L0 — event causes entry into Recovery.RcvrLock."

Recovery.RcvrCfg — re-establish a compatible configuration with the partner. §1 shows this substate has its own Electrical Idle inference status bit (idle_infer_rec_rcvr_cfg), so it is a substate in which the partner can go quiet and be detected doing so.

Recovery.Idle — a stable point from which Recovery leaves.

Recovery.Speed — coordinate a signalling-rate change, and §6 gives it its own section because it is not one step.

Do not assume every Recovery entry visits every substate. A retrain caused by lost lock has no reason to enter Recovery.Speed at all, and §1's evidence does not state a single mandatory path. §16's controller is therefore written so that the speed phase is entered only when the captured cause says so.

6. Recovery.Speed Is Four Phases, and None of Them Is a Register Write

The single most instructive thing in §1's table is that changing speed takes four distinct implementation phases, and their verbatim descriptions read like a protocol:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
SPEED0   "Transmit Electrical Idle Ordered Sets to notify the
          link partner that we are going idle"
 
SPEED1   "Determine to which speed to change"
 
SPEED2   "Wait for remote device to enter electrical idle and
          remain there for required minimum time"
 
SPEED3   "Request PHY change speed and wait for PHY to finish
          changing speed"

7. Traffic Stops, the Transmitter Does Not Go Silent

8. Where Generation Matters

Recovery's architecture is stable across generations. Several of its mechanisms are not, and §1's own evidence is explicitly generation-scoped in three places.

GenerationWhat differs, per §1's sourced evidence
Gen1 / Gen2 (2.5G / 5G)direct_to_recovery_phy is scoped "at 2.5G or 5G" — the PHY-error-burst trigger is described only for these rates
Gen3–Gen5 (8G+)l0_to_rec_rcvr_lock_rx_8g_eie — EIE ordered sets direct Recovery entry "at >= 8G"; and ltssm_equalization exists "For >= 8G capable cores only"
Gen6+flit-based operation with PAM4 and FEC changes physical behaviour materially; this chapter's sourced evidence does not extend there, and it does not claim to

Two rules follow. Do not universalize a trigger — an EIE-based entry is an 8G-and-above mechanism in this evidence, and a Gen2 design looking for it is looking for something that does not arrive. And do not assume equalization is always in the picture: §1's equalization status register does not exist below 8G, which is exactly why §17 consumes a normalized, optional equalization result rather than making it mandatory.

9. The Recovery Subtree

The Recovery subtree. Entry from L0 leads to Recovery RcvrLock, which re-establishes receiver lock. From there the machine reaches Recovery RcvrCfg to re-establish a compatible configuration, and then Recovery Idle. When a speed change is required, Recovery Speed coordinates the rate change with the link partner and the PHY, and returns to Recovery RcvrLock to retrain at the new rate. From Recovery Idle the link returns to L0. A failure falls back toward Detect.from L0Recovery.RcvrLockRecovery.RcvrCfgRecovery.IdleRecovery.Speedback toL0towardDetectcause capturedcausecapturedlock re-establishedlock re-establishedconfiguration agreedconfiguration agreedCOMMIT - traffic re-enabledCOMMIT - traffic re-enabledCOMMIT -traffic…speed change requiredspeed change requiredrate changed - retrainrate changed - retrainratechanged -…cannot re-lockcannotre-lockspeed change failedspeed change failed
Figure 1 — the Recovery subtree, following the canonical substates verified in §1. Entry from L0 lands in Recovery.RcvrLock, per the sourced description of the Electrical Idle inference event in L0. The Recovery.Speed branch is taken only when the captured cause calls for a rate change; its four phases are drawn as one node and expanded in §6. A Recovery that cannot re-establish what it needs falls back toward Detect. Exact transition criteria and timeouts are not published here.

Four things to read out of the figure.

Entry lands in Recovery.RcvrLock, and §1 sources it: the Electrical Idle inference event in L0 "causes entry into Recovery.RcvrLock." Recovery starts by re-establishing the most basic thing.

The speed branch loops back to RcvrLock. A rate change invalidates lock by construction — the receiver's timing recovery was tuned for the old rate — so changing speed necessarily means retraining afterwards, and the arrow from Recovery.Speed back to Recovery.RcvrLock is that fact drawn.

The commit is on the way out, at the return to L0 — the same boundary discipline as Chapter 18.4 §8, for the same reason (§15).

And failure goes toward Detect, not back to L0. A Recovery that cannot re-lock has no configuration worth returning with.

What is not drawn: every entry and exit criterion, all timeouts, Recovery.Speed's four internal phases (§6), and the paths toward Hot Reset, Disable and Loopback that §1's cause register names. Chapter 18.1 §6 owns the full topology.

10. Two Traces

Internal teaching signals, not PCIe pins. No cycle counts are implied — the point is ordering and ownership.

Trace A — retrain, no speed change

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
step               1     2     3     4     5     6     7     8     9    10
ltssm_recovery     0     1     1     1     1     1     1     1     0     0
rec_causes        --    04    04    04    04    04    04    04    04    04
traffic_enable     1     0     0     0     0     0     0     0     1     1
 
rx_lock            1     0     0     1     1     1     1     1     1     1
rec_qual_count     -     0     0     0     1     2     3     4     4     4
speed_change_req   0     0     0     0     0     0     0     0     0     0
 
cand_valid         0     0     0     0     0     0     0     1     1     0
commit_req         0     0     0     0     0     0     0     0     1     0
active_speed      G3    G3    G3    G3    G3    G3    G3    G3    G3    G3
active_width      x8    x8    x8    x8    x8    x8    x8    x8    x8    x8

Read step 2. Lock is lost, Recovery is entered, traffic_enable drops in the same cycle — and rec_causes captures 0x04 (lock_loss) and never changes again for this episode (§11).

Read steps 3–8. Lock returns, qualification accumulates, a candidate forms.

Now read active_speed and active_width across the entire trace. They never move. The Link was broken for six cycles and its committed configuration was correct the whole time — because nothing was wrong with it, and re-deriving it changed nothing.

And read step 9. Commit, then traffic re-enabled. In that order.

Trace B — speed change

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
step               1     2     3     4     5     6     7     8     9    10    11
ltssm_recovery     0     1     1     1     1     1     1     1     1     1     0
rec_causes        --    08    08    08    08    08    08    08    08    08    08
traffic_enable     1     0     0     0     0     0     0     0     0     0     1
 
eios_tx            0     1     1     0     0     0     0     0     0     0     0
partner_idle       0     0     0     1     1     1     1     1     1     1     0
rate_req_valid     0     0     0     0     1     1     0     0     0     0     0
rate_req_ready     0     0     0     0     0     1     0     0     0     0     0
rate_done          0     0     0     0     0     0     0     1     1     1     1
rate_ok            -     -     -     -     -     -     -     1     1     1     1
 
rx_lock            1     0     0     0     0     0     0     0     1     1     1
cand_speed        --    --    --    --    G4    G4    G4    G4    G4    G4    G4
cand_valid         0     0     0     0     0     0     0     0     0     1     1
commit_req         0     0     0     0     0     0     0     0     0     0     1
active_speed      G3    G3    G3    G3    G3    G3    G3    G3    G3    G3    G4

Read steps 2–3 — SPEED0. Electrical Idle Ordered Sets are transmitted: the partner is told before anything changes (§6).

Read step 4 — SPEED2. The partner has gone idle. Only now is a rate change safe.

Read steps 5–6 — SPEED3, and this is §12's whole contract. rate_req_valid rises at step 5 and the macro is not ready. The request is held. At step 6 rate_req_ready arrives and the request is accepted — and rate_req_valid drops immediately afterwards, on the handshake, not on rate_done. §20 measured what happens if it waits for rate_done instead.

Read step 8. rate_done with rate_ok. The PHY took two cycles here and could have taken two hundred — the digital side does not control it.

Read active_speed at step 10 versus step 11. At step 10 the candidate is valid, lock is back, and the active speed is still G3. The Link is fully repaired at the new rate and has not yet said so. At step 11 it commits — speed, width and lane map together — and traffic is re-enabled.

That one-cycle gap between "ready" and "committed" is the entire subject of §15.

11. RTL — Recovery Cause Latch

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SYNTHESIZABLE. Normalized Recovery cause capture.
// THAT CAUSES ARE A STICKY BITMASK rather than an enum follows the
// vendor's own l0_to_rec register (section 1). THE SPECIFIC BITS are this
// chapter's normalization -- exact triggers are generation- and
// implementation-dependent (section 8).
package recovery_pkg;
 
  typedef struct packed {
    logic retrain_req;      // software/management directed
    logic speed_req;        // directed speed change
    logic width_req;        // autonomous width change
    logic lock_loss;        // receiver lost lock / alignment
    logic phy_error;        // PHY error burst
    logic partner_ts;       // partner is directing entry via training sets
    logic replay_exhausted; // repeated replay failure (section 2)
  } rec_cause_t;
 
  localparam int CAUSE_W = $bits(rec_cause_t);
 
  function automatic bit needs_speed_change(input rec_cause_t c);
    return c.speed_req;
  endfunction
 
  // A cause set with NO bits is not a legal Recovery episode. Something
  // asked for this; if nothing did, the entry itself is the bug.
  function automatic bit cause_valid(input rec_cause_t c);
    return (|c);
  endfunction
 
endpackage
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import recovery_pkg::*;
 
// SYNTHESIZABLE. Capture ALL causes present at Recovery entry, then freeze.
// THE CAUSE IS THE MOST VALUABLE DEBUG STATE THIS BLOCK OWNS (section 4).
// It must describe why THIS episode started, which means it cannot track
// live events -- section 20 measured what happens when it does.
module recovery_cause_latch (
  input  logic clk,
  input  logic rst_n,
 
  input  logic       rec_entry,      // single-cycle: entering Recovery
  input  rec_cause_t live_causes,    // whatever is asserting right now
 
  // A directed retrain can arrive while Recovery is already running, or
  // while entry is blocked. It must not evaporate.
  input  logic       retrain_pulse,
 
  output rec_cause_t causes,
  output logic       causes_valid,
  output logic       retrain_pending
);
 
  rec_cause_t causes_q;
  logic       valid_q, retrain_q;
 
  assign causes         = causes_q;
  assign causes_valid   = valid_q;
  assign retrain_pending = retrain_q;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      causes_q <= '0; valid_q <= 1'b0; retrain_q <= 1'b0;
    end else begin
      // ==============================================================
      // A PENDING RETRAIN REQUEST IS A LEVEL, NOT A PULSE.
      //
      // Entry into Recovery can be blocked for a cycle -- the machine may
      // be finishing a previous episode. A one-cycle request wired
      // straight into the entry condition is simply lost, and the
      // software that wrote the Retrain Link register waits forever
      // (section 20, mutation 12).
      // ==============================================================
      if (retrain_pulse)   retrain_q <= 1'b1;
      else if (rec_entry)  retrain_q <= 1'b0;   // consumed EXACTLY ONCE
 
      if (rec_entry) begin
        // CAPTURE EVERYTHING TRUE AT ENTRY, in one assignment. The
        // retrain bit is OR-ed in from the pending flag so a request that
        // arrived while entry was blocked still appears as a cause.
        causes_q             <= live_causes;
        causes_q.retrain_req <= live_causes.retrain_req | retrain_q;
        valid_q              <= 1'b1;
      end
      // ==============================================================
      // AND THEN NOTHING. No arm updates causes_q while the episode runs.
      //
      // This is the point of the block. Live events keep happening during
      // Recovery -- lock drops again, the partner sends more training
      // sets -- and every one of them would overwrite the record of why
      // this episode began. Section 20 measured the damage: 62 of 63
      // cause combinations end up wrong.
      // ==============================================================
    end
  end
 
endmodule

Classification: synthesizable.

Architecture. Capture-once-and-freeze, plus a pending flag for the one cause that can arrive asynchronously to entry.

Two decisions, both measured (§20). Freezing at entry is correct for all 63 non-empty cause combinations; the live-tracking version is wrong for 62 of 63. And the retrain pending flag is consumed exactly once — set by the pulse, cleared by entry.

Failure — four. Recomputing from live signals destroys the diagnosis. An enum instead of a bitmask discards every simultaneous cause but one. Wiring retrain_pulse directly to the entry condition loses requests whenever entry is blocked. And clearing the pending flag on anything other than entry either drops the request or replays it into a second episode.

12. RTL — PHY Rate-Change Owner

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import recovery_pkg::*;
 
// SYNTHESIZABLE. Own exactly one rate-change transaction with the PHY.
// THIS IS SECTION 1'S RECOVERY_SPEED3 MADE CONCRETE: "Request PHY change
// speed and wait for PHY to finish changing speed."
// THE PHY IS A MACRO, not a register. It may stall the request, and it
// takes an unbounded and unknown time to complete.
module phy_rate_owner #(
  parameter int RATE_W = 3
) (
  input  logic clk,
  input  logic rst_n,
 
  // ---- From the Recovery controller --------------------------------------
  input  logic              start,
  input  logic [RATE_W-1:0] requested_rate,
 
  // ---- To the PHY macro ---------------------------------------------------
  output logic              req_valid,
  input  logic              req_ready,
  output logic [RATE_W-1:0] req_rate,
 
  // ---- From the PHY macro. CANNOT BE BACKPRESSURED. -----------------------
  input  logic              done,
  input  logic              success,
 
  // ---- Result -------------------------------------------------------------
  output logic              result_valid,
  output logic [RATE_W-1:0] result_rate,
  output logic              result_success,
  output logic              unowned_done
);
 
  typedef enum logic [1:0] { S_IDLE, S_ISSUE, S_WAIT, S_REPORT } st_e;
  st_e st_q;
 
  logic [RATE_W-1:0] rate_q;
  logic              ok_q, unowned_q;
 
  // ==================================================================
  // req_valid IS TRUE ONLY IN S_ISSUE.
  //
  // The state is left on the HANDSHAKE (req_valid && req_ready), not on
  // `done`. Holding valid until `done` is the classic bug from Chapter
  // 18.2 section 14, and section 20 measured it here: over 2,396,744
  // sequences it produces 452,512 double-accepts and 588,768 results
  // with no accepted request at all.
  // ==================================================================
  assign req_valid      = (st_q == S_ISSUE);
  assign req_rate       = rate_q;              // CAPTURED, never re-read
  assign result_valid   = (st_q == S_REPORT);
  assign result_rate    = rate_q;
  assign result_success = ok_q;
  assign unowned_done   = unowned_q;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      st_q <= S_IDLE; rate_q <= '0; ok_q <= 1'b0; unowned_q <= 1'b0;
    end else begin
      unique case (st_q)
        S_IDLE : begin
          if (start) begin
            // THE RATE IS CAPTURED AT START. The controller's requested
            // rate may change while the macro stalls -- re-reading it
            // would send the PHY to a rate nobody asked for.
            rate_q <= requested_rate;
            st_q   <= S_ISSUE;
          end
          // A `done` with no outstanding request is REPORTED, never
          // silently consumed. It means a previous transaction completed
          // late, or the macro is signalling something unowned.
          if (done) unowned_q <= 1'b1;
        end
 
        S_ISSUE :
          // LEAVE ON THE HANDSHAKE. One start, one accepted request.
          if (req_ready) st_q <= S_WAIT;
 
        S_WAIT :
          if (done) begin
            ok_q <= success;
            st_q <= S_REPORT;
          end
 
        S_REPORT : st_q <= S_IDLE;
 
        default : st_q <= S_IDLE;
      endcase
    end
  end
 
endmodule

Classification: synthesizable.

Architecture. Four states, request held under stall, rate captured at start, state left on the handshake.

Verified exhaustively (§20): over 2,396,744 (start, ready, done) sequences of length ≤ 7, the contract accepts ≤ starts and results ≤ accepts holds with zero violations.

And the mutation was measured, not imagined. Holding req_valid until done violates the contract on 1,041,280 of those sequences, splitting into two distinct failures: 452,512 double-accepts (smallest witness: start with the macro stalled, then two ready cycles → one start, two accepted rate changes) and 588,768 results with no accepted request at all (smallest witness: start stalled, then done → the machine reports a completed rate change the PHY never accepted).

Failure — five. Valid held until done produces both failures above. Pulsing the request loses it whenever the macro stalls. Re-reading requested_rate in S_ISSUE sends the PHY somewhere nobody asked for. Consuming an unowned done attributes a stale completion to a current request. And no success capture treats a failed rate change as a successful one — §1's speed_change_fail bit exists precisely because that outcome is real.

13. RTL — Recovery Training Qualifier

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SYNTHESIZABLE. Qualify re-established training during Recovery.
// SAME CONTRACT AS CHAPTER 18.3's qualifier -- consecutive qualifying
// observations, reset to zero on a bad one, saturating -- because the
// underlying question is the same. THE THRESHOLD IS SEPARATE AND
// PARAMETERISED: this chapter has no source stating that Recovery
// qualification equals Polling qualification, so it does not assume it.
module recovery_qualifier #(
  parameter int THRESH = 4,
  parameter int CNT_W  = (THRESH <= 1) ? 1 : $clog2(THRESH + 1)
) (
  input  logic clk,
  input  logic rst_n,
 
  input  logic restart,          // Recovery entry, or lock lost again
  input  logic qualifying,
  input  logic disqualifying,
 
  output logic [CNT_W-1:0] count,
  output logic             threshold_met
);
 
  generate
    if (THRESH < 1) $error("THRESH must be at least 1");
    if ((1 << CNT_W) < (THRESH + 1)) $error("CNT_W too narrow for THRESH");
  endgenerate
 
  logic [CNT_W-1:0] cnt_q;
  assign count         = cnt_q;
  assign threshold_met = (cnt_q >= CNT_W'(THRESH));
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n)             cnt_q <= '0;
    else if (restart)       cnt_q <= '0;   // priority, declared
    else if (disqualifying) cnt_q <= '0;   // CONSECUTIVE, not cumulative
    else if (qualifying)
      if (cnt_q < CNT_W'(THRESH)) cnt_q <= cnt_q + CNT_W'(1);
  end
 
endmodule

Classification: synthesizable.

Architecture. Chapter 18.3 §11's contract, reused identically — the same priority order, the same saturation, the same reset-to-zero.

And the same off-by-one hazard applies here, including the part that makes it dangerous: at THRESH = 1 the >= THRESH-1 mutation is observationally identical to the correct form, so a verification plan that only exercises the minimum threshold cannot detect it (Chapter 18.3 §14, measured).

14. RTL — Candidate Snapshot

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import recovery_pkg::*;
 
// SYNTHESIZABLE. Build the NEW configuration beside the operating one.
// NOTHING HERE IS VISIBLE ABOVE THE PHYSICAL LAYER. This is Chapter 18.4
// section 8's candidate/committed split, applied to a Link that already
// has a committed configuration worth protecting.
module recovery_candidate #(
  parameter int MAX_LANES = 16,
  parameter int LANE_W    = (MAX_LANES <= 1) ? 1 : $clog2(MAX_LANES),
  parameter int WIDTH_W   = 5,
  parameter int RATE_W    = 3
) (
  input  logic clk,
  input  logic rst_n,
 
  input  logic rec_entry,
 
  // ---- The configuration currently in operation --------------------------
  input  logic [RATE_W-1:0]    active_speed,
  input  logic [WIDTH_W-1:0]   active_width,
  input  logic [MAX_LANES-1:0] active_lane_map,
 
  // ---- Results arriving during Recovery ----------------------------------
  input  logic                 rate_result_valid,
  input  logic [RATE_W-1:0]    rate_result,
  input  logic                 rate_result_ok,
 
  input  logic                 width_result_valid,
  input  logic [WIDTH_W-1:0]   width_result,
  input  logic [MAX_LANES-1:0] lane_map_result,
 
  input  logic                 train_ok,
 
  output logic [RATE_W-1:0]    cand_speed,
  output logic [WIDTH_W-1:0]   cand_width,
  output logic [MAX_LANES-1:0] cand_lane_map,
  output logic                 cand_valid
);
 
  logic [RATE_W-1:0]    spd_q;
  logic [WIDTH_W-1:0]   wid_q;
  logic [MAX_LANES-1:0] map_q;
  logic                 spd_bad_q;
 
  assign cand_speed    = spd_q;
  assign cand_width    = wid_q;
  assign cand_lane_map = map_q;
 
  // ==================================================================
  // A CANDIDATE IS VALID ONLY IF TRAINING SUCCEEDED AND NOTHING FAILED.
  //
  // A failed rate change poisons the episode: section 1's speed_change_fail
  // bit exists because the PHY can decline, and committing a configuration
  // that names a rate the PHY never reached is worse than not recovering.
  // ==================================================================
  assign cand_valid = train_ok && !spd_bad_q
                   && (wid_q != '0) && (map_q != '0);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      spd_q <= '0; wid_q <= '0; map_q <= '0; spd_bad_q <= 1'b0;
    end else if (rec_entry) begin
      // ==============================================================
      // SEEDED FROM THE ACTIVE CONFIGURATION, not from zero.
      //
      // Most Recovery episodes change NOTHING -- a retrain after lost
      // lock returns the identical speed, width and lane map. Starting
      // from the operating values means an episode that renegotiates
      // nothing still commits something coherent, and only the fields
      // that were actually re-derived differ.
      //
      // Starting from zero would make every retrain look like a
      // configuration change (section 20, mutation 7).
      // ==============================================================
      spd_q     <= active_speed;
      wid_q     <= active_width;
      map_q     <= active_lane_map;
      spd_bad_q <= 1'b0;
    end else begin
      if (rate_result_valid) begin
        if (rate_result_ok) spd_q     <= rate_result;
        else                spd_bad_q <= 1'b1;   // sticky for the episode
      end
      if (width_result_valid) begin
        wid_q <= width_result;
        map_q <= lane_map_result;   // WIDTH AND MAP TOGETHER, always
      end
    end
  end
 
endmodule

Classification: synthesizable.

Architecture. A candidate seeded from the active configuration, with a sticky speed-failure flag.

Seeding is the decision worth defending. Most Recovery episodes change nothing — §4 showed that lost lock, partner-directed entry and PHY error bursts are all repairs, not changes. Seeding from the active values means those episodes commit the configuration they already had, and only genuinely re-derived fields differ.

Width and lane map move together, in one arm, because Chapter 18.4 §14 established that a width and a mask that disagree hand the lane distributor an impossible instruction.

Failure — four. Seeding from zero makes every retrain look like a reconfiguration. Updating width without the map reintroduces 18.4's worst bug. A non-sticky speed-failure flag lets a later valid result mask an earlier failure. And exposing these outputs upward defeats the entire point (§15).

15. RTL — Atomic Recovery Commit

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import recovery_pkg::*;
 
// SYNTHESIZABLE. The flagship block: replace the operating configuration
// in ONE cycle or not at all.
// EVERYTHING ABOVE THE PHYSICAL LAYER READS ONLY THESE OUTPUTS. Chapter
// 17.1's lane distributor and Chapter 17.2's recombiner consume them, and
// a Link whose transmitter and receiver disagree about speed or lane map
// corrupts data that no error check can attribute to its cause.
module recovery_commit #(
  parameter int MAX_LANES = 16,
  parameter int WIDTH_W   = 5,
  parameter int RATE_W    = 3
) (
  input  logic clk,
  input  logic rst_n,
 
  input  logic                 cand_valid,
  input  logic [RATE_W-1:0]    cand_speed,
  input  logic [WIDTH_W-1:0]   cand_width,
  input  logic [MAX_LANES-1:0] cand_lane_map,
 
  input  logic recovery_success,   // the Recovery.Idle -> L0 edge
  input  logic link_down,
 
  output logic [RATE_W-1:0]    active_speed,
  output logic [WIDTH_W-1:0]   active_width,
  output logic [MAX_LANES-1:0] active_lane_map,
  output logic                 active_config_valid,
  output logic                 commit_rejected
);
 
  // ONE PACKED STRUCT. Partial commit is not a bug to be avoided here;
  // it is UNCONSTRUCTABLE, because there is no sequence to get wrong.
  typedef struct packed {
    logic                 valid;
    logic [RATE_W-1:0]    speed;
    logic [WIDTH_W-1:0]   width;
    logic [MAX_LANES-1:0] lane_map;
  } link_cfg_t;
 
  link_cfg_t cfg_q;
  logic      rej_q;
 
  assign active_speed        = cfg_q.speed;
  assign active_width        = cfg_q.width;
  assign active_lane_map     = cfg_q.lane_map;
  assign active_config_valid = cfg_q.valid;
  assign commit_rejected     = rej_q;
 
  // Width and map must agree, or the distributor gets an impossible
  // instruction (Chapter 18.4 P6).
  wire commit_ok = cand_valid
                && (cand_width != '0)
                && ($countones(cand_lane_map) == cand_width);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      cfg_q <= '0; rej_q <= 1'b0;
    end else if (link_down) begin
      // ==============================================================
      // LINK-DOWN DOMINATES THE COMMIT. Declared priority.
      //
      // If the Link went down in the same cycle a commit was ready, the
      // configuration being committed describes a Link that no longer
      // exists. Committing it would leave active_config_valid high on a
      // dead Link, and everything above would keep transmitting into it
      // (section 20, mutation 13).
      // ==============================================================
      cfg_q <= '0; rej_q <= 1'b0;
    end else if (recovery_success) begin
      if (commit_ok) begin
        // ============================================================
        // ONE NONBLOCKING ASSIGNMENT. speed, width, lane_map and valid
        // become visible in the SAME cycle.
        //
        // Section 20 measured the alternative across 100,000 random
        // episodes: split registers produced 98,251 cycles in which the
        // active configuration was a combination that was never a
        // coherent candidate. The atomic version produced 0.
        // ============================================================
        cfg_q <= '{ valid    : 1'b1,
                    speed    : cand_speed,
                    width    : cand_width,
                    lane_map : cand_lane_map };
      end else begin
        // A Recovery that "succeeded" without a coherent candidate is a
        // control bug. Expose nothing and say so.
        cfg_q <= '0;
        rej_q <= 1'b1;
      end
    end
    // NOTE THE ABSENCE OF AN ENTRY ARM.
    // Chapter 18.4's cfg_commit clears on state entry because
    // Configuration is reached from Polling with no operational Link to
    // preserve. RECOVERY IS THE OPPOSITE CASE: the old configuration
    // stays active and intact throughout the episode. Copying 18.4's
    // entry arm here would tear down a working configuration in order to
    // renegotiate one that is usually identical (section 23).
  end
 
endmodule

Classification: synthesizable.

Architecture. One packed struct, one assignment, link-down dominating, and no entry arm.

The missing entry arm is the whole difference between this chapter and 18.4. Initial Configuration has nothing to protect. Recovery does, and the configuration it is protecting is usually the one it will end up re-committing unchanged.

Measured (§20): atomic commit produced 0 incoherent active states across 100,000 random episodes; the split-register version produced 98,251.

Failure — five. Separate registers (§20's counterexample). An entry arm copied from 18.4 destroys a working configuration. Commit dominating link-down leaves a valid configuration on a dead Link. Committing without checking width against the map hands the distributor an impossible instruction. And committing on cand_valid alone, without the Recovery-success edge, commits mid-episode.

16. RTL — Recovery Progress Controller

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import recovery_pkg::*;
 
// SYNTHESIZABLE. Progress within Recovery only.
// THIS IS NOT THE LTSSM (Chapter 18.1 section 9). It is an
// implementation-facing coordinator whose phases map onto the canonical
// substates verified in section 1 -- it is not their normative encoding,
// and section 1 does not publish the transition criteria it would need
// to be one.
module recovery_progress #(
  parameter int RATE_W = 3
) (
  input  logic clk,
  input  logic rst_n,
 
  input  logic       rec_entry,
  input  rec_cause_t causes,
  input  logic       link_down,
 
  input  logic       rx_lock,
  input  logic       train_threshold_met,
  input  logic       cfg_agreed,
 
  // ---- Rate-change owner (section 12) ------------------------------------
  input  logic       partner_idle,
  output logic       rate_start,
  output logic [RATE_W-1:0] rate_target,
  input  logic       rate_result_valid,
  input  logic       rate_result_ok,
  input  logic [RATE_W-1:0] negotiated_rate,
 
  // ---- Equalization, where applicable (section 17) ------------------------
  input  logic       eq_required,
  input  logic       eq_done,
  input  logic       eq_ok,
 
  input  logic       cand_valid,
  input  logic       timeout_elapsed,
 
  output logic       recovery_success,
  output logic       recovery_fail,
  output logic       traffic_enable,
  output logic       in_speed_phase
);
 
  typedef enum logic [2:0] {
    S_LOCK, S_CFG, S_SPEED_ANNOUNCE, S_SPEED_WAIT, S_SPEED_REQ, S_IDLE,
    S_DONE, S_FAIL
  } st_e;
  st_e st_q;
 
  logic spd_done_q;
 
  assign recovery_success = (st_q == S_DONE);
  assign recovery_fail    = (st_q == S_FAIL);
  assign in_speed_phase   = (st_q inside {S_SPEED_ANNOUNCE, S_SPEED_WAIT,
                                          S_SPEED_REQ});
 
  // ==================================================================
  // NORMAL TRAFFIC IS DISABLED THROUGHOUT RECOVERY, INCLUDING S_DONE.
  //
  // It re-enables only once the commit has happened -- which is the cycle
  // AFTER S_DONE asserts (section 15 registers it). Enabling in S_DONE
  // would let packets out one cycle before the configuration describing
  // how to transmit them became active (section 20, mutation 8).
  // ==================================================================
  assign traffic_enable = 1'b0;   // driven externally from active_config_valid
 
  assign rate_start  = (st_q == S_SPEED_REQ) && !spd_done_q;
  assign rate_target = negotiated_rate;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      st_q <= S_LOCK; spd_done_q <= 1'b0;
    end else if (link_down) begin
      st_q <= S_FAIL;                        // priority over everything
    end else if (rec_entry) begin
      st_q <= S_LOCK; spd_done_q <= 1'b0;
    end else begin
      unique case (st_q)
        S_LOCK :
          if (timeout_elapsed)                    st_q <= S_FAIL;
          else if (rx_lock && train_threshold_met) st_q <= S_CFG;
 
        S_CFG :
          if (timeout_elapsed)      st_q <= S_FAIL;
          else if (!rx_lock)        st_q <= S_LOCK;   // lost it again
          else if (cfg_agreed) begin
            // THE SPEED BRANCH IS TAKEN FROM THE CAPTURED CAUSE, not from
            // a live signal -- and only once per episode.
            if (needs_speed_change(causes) && !spd_done_q)
                                    st_q <= S_SPEED_ANNOUNCE;
            else                    st_q <= S_IDLE;
          end
 
        // ---- Section 1's SPEED0: tell the partner first ------------------
        S_SPEED_ANNOUNCE :
          if (timeout_elapsed) st_q <= S_FAIL;
          else                 st_q <= S_SPEED_WAIT;
 
        // ---- Section 1's SPEED2: BOTH ends must be quiet ----------------
        S_SPEED_WAIT :
          if (timeout_elapsed)   st_q <= S_FAIL;
          else if (partner_idle) st_q <= S_SPEED_REQ;
 
        // ---- Section 1's SPEED3: request the PHY and wait --------------
        S_SPEED_REQ :
          if (rate_result_valid) begin
            spd_done_q <= 1'b1;               // one speed change per episode
            if (rate_result_ok) st_q <= S_LOCK;  // RETRAIN AT THE NEW RATE
            else                st_q <= S_FAIL;
          end else if (timeout_elapsed) st_q <= S_FAIL;
 
        S_IDLE :
          // Equalization, where the generation requires it (section 17).
          if (eq_required && !eq_done)      st_q <= S_IDLE;
          else if (eq_required && !eq_ok)   st_q <= S_FAIL;
          else if (cand_valid)              st_q <= S_DONE;
          else if (timeout_elapsed)         st_q <= S_FAIL;
 
        S_DONE, S_FAIL : ;
        default : st_q <= S_FAIL;
      endcase
    end
  end
 
endmodule

Classification: synthesizable (Recovery only — not the LTSSM).

Architecture. Lock, configure, optionally change speed, settle, commit — with the speed branch driven by the captured cause and taken at most once.

Three decisions worth naming. S_SPEED_REQ returns to S_LOCK on success, because a rate change invalidates lock by construction (§9). spd_done_q bounds the episode to one speed change, so a cause bit that stays set cannot produce an endless announce-wait-request-retrain loop. And link-down beats everything, including a completed episode.

Failure — four. Branching on a live speed-request signal rather than the captured cause re-enters the speed phase whenever the request is still asserted. Returning to S_IDLE instead of S_LOCK after a rate change skips retraining at the new rate — and the Link returns to L0 with a receiver locked for the old one. No spd_done_q loops forever. And traffic enabled in S_DONE opens the gate one cycle early (§20, mutation 8).

17. Equalization Belongs to a Controller, Not to This State Machine

Chapter 17.4 owns equalization. Recovery consumes it, and §16 shows the entire interface: eq_required, eq_done, eq_ok.

Three facts justify that narrow surface, and §1 sources the first.

It is generation-scoped. The ltssm_equalization register exists "For >= 8G capable cores only" — so below 8G there is nothing for Recovery to wait on, and a controller that always waited would hang on a Gen2 Link.

It can fail. That register's only field is fail, "Equalization Failure error indicator." §16 therefore treats eq_required && !eq_ok as a Recovery failure, not as a condition to proceed past — because a Link operating at a rate whose equalization failed will produce errors that look like anything except an equalization problem.

And its state is not LTSSM state. Transmitter coefficients, CTLE gain and DFE taps live in the PHY and its adaptation logic (Chapter 17.4 §§6–8). The LTSSM never holds a tap value, and a design that put coefficients into the state machine would have merged a control problem with a signal-processing one.

18. What Happens to the Replay Buffer

A Link entering Recovery may have unacknowledged TLPs sitting in its replay buffer (Chapter 15.1), outstanding Requests awaiting Completions, and pending ACK state.

19. Assertions

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SVA over the Recovery blocks. These assert the LOCAL contract: what may
// be committed, what the cause record must satisfy, and what the PHY
// transaction owner may do. They assert NOTHING about Recovery succeeding,
// about the partner cooperating, or about any rate being achievable.
 
// ---- ENVIRONMENT ------------------------------------------------------
assume property (@(posedge clk) disable iff (!rst_n)
  rec_entry |=> !rec_entry);
assume property (@(posedge clk) disable iff (!rst_n)
  req_valid |-> !$isunknown(req_rate));
 
// ---- CAUSE OWNERSHIP --------------------------------------------------
 
// P1: the cause is captured on entry and is NEVER recomputed. Section 20
// measured the alternative: wrong for 62 of 63 cause combinations.
property p_cause_frozen;
  @(posedge clk) disable iff (!rst_n)
  (causes_valid && !rec_entry) |=> $stable(causes);
endproperty
a_frozen : assert property (p_cause_frozen);
 
// P2: a Recovery episode always has at least one cause. An entry with no
// cause means something entered Recovery for no represented reason.
property p_cause_nonempty;
  @(posedge clk) disable iff (!rst_n)
  $rose(causes_valid) |-> cause_valid(causes);
endproperty
a_nonempty : assert property (p_cause_nonempty);
 
// P3: A DIRECTED RETRAIN REQUEST IS NEVER LOST. It is held until an entry
// consumes it.
property p_retrain_held;
  @(posedge clk) disable iff (!rst_n)
  (retrain_pending && !rec_entry) |=> retrain_pending;
endproperty
a_retain : assert property (p_retrain_held);
 
// P4: and it is consumed EXACTLY ONCE.
property p_retrain_consumed_once;
  @(posedge clk) disable iff (!rst_n)
  (retrain_pending && rec_entry) |=> !retrain_pending;
endproperty
a_consumed : assert property (p_retrain_consumed_once);
 
// ---- THE PHY RATE TRANSACTION -----------------------------------------
 
// P5: THE REQUEST IS HELD UNDER STALL. The PHY may take any number of
// cycles to become ready.
property p_req_held;
  @(posedge clk) disable iff (!rst_n)
  (req_valid && !req_ready) |=> req_valid;
endproperty
a_held : assert property (p_req_held);
 
// P6: and the requested rate is STABLE while it stalls -- section 12
// captured it at start precisely so this holds.
property p_rate_stable;
  @(posedge clk) disable iff (!rst_n)
  (req_valid && !req_ready) |=> $stable(req_rate);
endproperty
a_rate_stable : assert property (p_rate_stable);
 
// P7: ONE START, AT MOST ONE ACCEPTED REQUEST. This is the property the
// "valid held until done" mutation violates on 452,512 of 2,396,744
// sequences (section 20).
property p_no_double_issue;
  @(posedge clk) disable iff (!rst_n)
  (req_valid && req_ready) |=> !req_valid until_with start;
endproperty
a_single : assert property (p_no_double_issue);
 
// P8: a completion with no outstanding request is REPORTED, never
// consumed as if it belonged to a current transaction.
property p_unowned_done;
  @(posedge clk) disable iff (!rst_n)
  (done && (st_q == S_IDLE)) |=> unowned_done;
endproperty
a_unowned : assert property (p_unowned_done);
 
// P9: A FAILED RATE CHANGE CANNOT BE REPORTED AS SUCCESS. Section 1's
// speed_change_fail bit exists because this outcome is real.
property p_fail_not_success;
  @(posedge clk) disable iff (!rst_n)
  (result_valid && !result_success) |-> !recovery_success;
endproperty
a_spdfail : assert property (p_fail_not_success);
 
// ---- THE COMMIT -------------------------------------------------------
 
// P10: NO COMMIT WITHOUT A VALID CANDIDATE.
property p_no_blind_commit;
  @(posedge clk) disable iff (!rst_n)
  $changed(active_config_valid) && active_config_valid |-> $past(cand_valid);
endproperty
a_guard : assert property (p_no_blind_commit);
 
// P11: ATOMICITY. No field of the active configuration may move alone.
// The property the split-register mutation violates (section 20: 98,251
// incoherent cycles across 100,000 episodes).
property p_atomic;
  @(posedge clk) disable iff (!rst_n)
  ($changed(active_speed) || $changed(active_width)
                          || $changed(active_lane_map))
    |-> ($past(recovery_success) || $past(link_down) || $past(!rst_n));
endproperty
a_atomic : assert property (p_atomic);
 
// P12: THE ACTIVE CONFIGURATION DOES NOT MOVE DURING RECOVERY. The old
// configuration survives the episode intact until the commit.
property p_active_stable_in_recovery;
  @(posedge clk) disable iff (!rst_n)
  (in_recovery && !recovery_success && !link_down)
    |=> ($stable(active_speed) && $stable(active_width)
                               && $stable(active_lane_map));
endproperty
a_stable : assert property (p_active_stable_in_recovery);
 
// P13: candidate state never reaches a consumer. Structurally, the
// distributor's only configuration source is the active outputs.
property p_no_candidate_leak;
  @(posedge clk) disable iff (!rst_n)
  dut_distributor.active_mask ==
    (active_config_valid ? active_lane_map : '0);
endproperty
a_leak : assert property (p_no_candidate_leak);
 
// P14: LINK-DOWN DOMINATES THE COMMIT.
property p_linkdown_priority;
  @(posedge clk) disable iff (!rst_n)
  (link_down && recovery_success) |=> !active_config_valid;
endproperty
a_down : assert property (p_linkdown_priority);
 
// P15: width and lane map always agree in the committed configuration.
property p_width_map_agree;
  @(posedge clk) disable iff (!rst_n)
  active_config_valid |-> ($countones(active_lane_map) == active_width);
endproperty
a_agree : assert property (p_width_map_agree);
 
// ---- TRAFFIC ----------------------------------------------------------
 
// P16: NO NORMAL PACKET TRAFFIC DURING RECOVERY. Note this says nothing
// about the transmitter being silent (section 7).
property p_no_traffic;
  @(posedge clk) disable iff (!rst_n)
  (dut_ltssm.cur_state == ST_RECOVERY) |-> !dut_gate.traffic_enabled;
endproperty
a_no_traffic : assert property (p_no_traffic);
 
// P17: traffic cannot be re-enabled before the configuration is active.
property p_traffic_after_commit;
  @(posedge clk) disable iff (!rst_n)
  dut_gate.traffic_enabled |-> active_config_valid;
endproperty
a_order : assert property (p_traffic_after_commit);
 
// P18: stale training evidence from before this episode cannot qualify it.
property p_no_stale_qual;
  @(posedge clk) disable iff (!rst_n)
  rec_entry |=> (count == '0);
endproperty
a_stale : assert property (p_no_stale_qual);
 
// P19: at most one speed change per Recovery episode.
property p_one_speed_change;
  @(posedge clk) disable iff (!rst_n)
  (result_valid && !rec_entry) |=> !rate_start until_with rec_entry;
endproperty
a_one_speed : assert property (p_one_speed_change);
 
// P20: reset.
property p_reset;
  @(posedge clk)
  !rst_n |=> (!active_config_valid && !causes_valid && !req_valid);
endproperty
a_reset : assert property (p_reset);

P11 and P12 are the pair that defines this chapter. P11 forbids a partial commit; P12 forbids the active configuration moving at all during the episode. Together they say: the operating configuration is untouched until one cycle replaces it entirely.

P1 is the debugging property. A cause record that drifts is worse than no cause record, because it looks authoritative.

And P7 is the exact-once property whose violation §20 measured in both directions.

No liveness. "Recovery eventually succeeds" depends on a partner and a channel; "the target rate eventually locks" depends on the physical link. Asserting either would assume away the failures this state exists to handle.

20. Verification, Fault Injection, and Model Verification

Every number in this chapter was executed before publication.

The PHY rate owner — exhaustive

All (start, req_ready, done) sequences of length ≤ 7 — 2,396,744 sequences — checked against the ownership contract accepts ≤ starts and results ≤ accepts:

ImplementationContract violations
§12 as written (leave S_ISSUE on the handshake)0
req_valid held until done1,041,280

And the broken version fails in two distinct ways, which is worth separating because they present completely differently in a lab:

FailureSequencesSmallest witness (start, ready, done)
double accept (accepts > starts)452,512(1,0,0) (0,1,0) (0,1,0)1 start, 2 accepted rate changes
result with no accepted request588,768(1,0,0) (0,0,1) (0,0,1)1 start, 0 accepts, 1 reported result

The second is the more dangerous of the two. The machine reports a completed, successful rate change that the PHY never accepted — so the digital side believes the Link is at Gen4 and the PHY is still running Gen3. Every subsequent symptom is a mismatch nobody can trace to a rate change that did not happen.

The atomic commit — 100,000 random episodes

Random candidate updates interleaved with commits and link-down events, counting cycles in which the active configuration held a combination that was never a coherent candidate:

ImplementationIncoherent active-configuration cycles
§15's single packed struct0
separate speed / width registers98,251

The cause latch — exhaustive over simultaneous causes

All 63 non-empty combinations of the seven modelled causes, each presented at entry and then contradicted by live events for the rest of the episode:

ImplementationCombinations recorded correctly
§11 as written (capture and freeze)63 / 63
recomputed from live signals1 / 63

The single case the broken version gets right is the one where the live events happen to end up matching — which is exactly why the bug survives casual testing.

Directed tests

  • Simple retrain, no speed change — verify the active configuration never moves (P12) and the committed result equals the previous one (§14's seeding).
  • Loss of lock mid-episode — verify qualification restarts (P18).
  • Speed-change request — full SPEED0→SPEED2→SPEED3 walk.
  • Same-speed Recovery — verify no rate transaction is issued at all.
  • Rate macro stalls for 1, 2 and 50 cycles — verify hold and rate stability (P5, P6).
  • done before any start — verify unowned_done (P8). Required.
  • Rate change fails (success low) — verify Recovery fails and nothing commits (P9).
  • Candidate width/map/speed changes during the episode — verify no active movement (P12).
  • Invalid candidate at commit time — verify refusal (P10).
  • Link-down in the same cycle as commit — verify link-down wins (P14). Required.
  • Reset mid-episode, in each phase (P20).
  • All 63 simultaneous cause combinations (P2).
  • Retrain request while Recovery is already active — verify it is held for the next episode, not merged into this one (P3, P4).
  • Replay-buffer occupancy maintained across an episode — verify sequence numbers and Tags are unchanged (§18).

Mutations

#MutationCaught byLab symptom
1cause recomputed from live signalsP1cause register lies; every Recovery blames the last thing that happened
2speed request pulsed for one cycleP5rate change never starts when the PHY is busy; Link stuck in Recovery
3req_valid held until doneP7two rate changes for one request, or a result with no request (measured above)
4requested rate re-read in S_ISSUEP6PHY sent to a rate nobody asked for
5done from a previous request acceptedP8stale completion attributed to the current transaction
6speed committed before width and mapP11new speed + old lane map; LCRC errors on a clean channel
7candidate seeded from zero instead of the active configP15every retrain looks like a width change; x0 committed
8traffic enabled in S_DONEP17packets transmitted one cycle before the configuration is active
9qualifier not reset on a bad observation18.3's P6Recovery completes on scattered evidence; recovers repeatedly
10stale pre-entry observation countedP18episode qualifies on evidence from before it began
11equalization failure treated as successP9Link runs at a rate whose equalization failed; unexplained errors
12retrain pulse wired directly to the entry conditionP3software writes Retrain Link and nothing happens
13commit dominates link-downP14active_config_valid high on a dead Link; transmission continues
14active lane map changes during RecoveryP12distributor reconfigured mid-episode
15spd_done_q omittedP19endless announce→wait→request→retrain loop while the cause bit is set
16replay treated as a new Transaction Layer requestscoreboard Tag checkduplicate operation at the completer; Completion with a reassigned Tag
17speed branch taken from a live signal, not the captured causeP19re-enters the speed phase whenever the request is still asserted
18S_SPEED_REQ returns to S_IDLE instead of S_LOCKP10returns to L0 with the receiver locked for the old rate

Same-cycle audit

CaseDeclared resolution
rate_req_ready and done in the same cyclehandshake first: the request is accepted, then S_WAIT observes done
training qualification met and timeout elapsedsuccess wins — the evidence exists (18.3 §12)
commit ready and link_downlink-down wins (P14)
retrain request while the episode is completingheld for the next episode (P3), never merged
candidate updated in the commit cyclethe commit takes the value presented that cycle; §14 assigns width and map together so no mixed pair exists
rec_entry and a live cause changing togetherentry captures the value at entry; nothing after it matters (P1)

21. Debugging

Symptom → captured cause → local signal → distinguishing experiment.

The first read is always the cause register (§11). "In Recovery" is not a diagnosis — §4 showed two of the four cause groups are perfectly healthy.

The most important LTSSM symptom in PCIe, and the cause bits split it into three completely different investigations.

If speed_req dominates: something is repeatedly asking for a rate change — and if the rate keeps changing back, the Link is failing at the higher rate and falling back. Go to the next scenario.

If lock_loss or phy_error dominates: a physical problem. Read lock, alignment and per-lane error counts (Chapter 17.4 §2, Chapter 18.3 §16). The distinguishing experiment is to change the channel — a shorter cable or a different slot — because if the oscillation stops, the logic was never the problem.

If partner_ts dominates: the far end is directing entry (§1). The local port is healthy and following. Investigating the local design here is investigating the wrong device, and this is the single most common misdirection in this symptom.

And if replay_exhausted appears: read it as one story, not two (§2). The replays were the evidence; Recovery was the consequence.

Gen5 repeatedly falls back to Gen4

Do not conclude "software is not requesting Gen5" merely because Device Capability advertises it.

Read three things in order. speed_change_fail (§1) — if it is set, the change was attempted and did not work. The equalization fail bit — available "for >= 8G capable cores only" (§1), and if equalization failed at the higher rate, the rate is not usable no matter what is requested. Per-lane error counts at each rate.

The distinguishing experiment is a channel change, exactly as in Chapter 17.4 §2: if Gen5 holds on a shorter channel, the design is fine and the margin was not there. Falling back is the mechanism working, not failing.

Recovery finishes and data corruption starts immediately

Suspect the commit before suspecting the channel — this is §20's counterexample.

The signature: LCRC errors with no PHY errors and no bit errors, starting within a few cycles of recovery_done, and usually recovering via replay so they read as transient.

The distinguishing experiment: sample active_speed, active_width and active_lane_map in the same cycle, continuously, and check $countones(lane_map) == width and that all three last changed in the same cycle. One cycle of disagreement is the whole bug, and P11 with P15 is that check made permanent.

A second candidate with the same symptom: §16's S_SPEED_REQ returning to S_IDLE instead of S_LOCKthe Link returns to L0 with the receiver locked for the previous rate.

Recovery never finishes after a rate change

Walk the transaction in order — each stage has a different owner:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
rate_start        did the controller ask?          -> §16, cause bits
req_valid/ready   did the macro ACCEPT it?         -> §12, P5
done              did the PHY finish?              -> the macro
success           did it finish SUCCESSFULLY?      -> §1's speed_change_fail
rx_lock           did lock return at the new rate? -> §16 returns to S_LOCK
qual_count        is training being recognised?    -> §13
cand_valid        is there a coherent candidate?   -> §14
commit            did it commit?                   -> §15

The first stage that never advances names the owner. req_valid high forever means the macro never became ready — a PHY or integration problem, not an LTSSM one. done never arriving after acceptance is the macro. rx_lock never returning means the new rate is not viable on this channel.

Read retrain_pending (§11). If the pulse was wired directly into the entry condition and entry was blocked that cycle, the request is simply gone — mutation 12, and it presents as software being ignored rather than as a hardware fault.

22. Common Misconceptions

  • "Recovery means TLP replay." Different layer, different loop — though repeated replay failure causes Recovery (§2).
  • "Recovery is software retry." Hardware, physical layer, no packet repeated (§3).
  • "Recovery only happens after fatal errors." §1's cause register lists directed speed changes and retrain requests; the first one fires during normal start-up (§4).
  • "Recovery always changes speed." Recovery.Speed is a branch, not a step (§5).
  • "Advertising Gen5 guarantees Gen5." §1's speed_change_fail bit exists because it does not.
  • "Normal packets can keep flowing through Recovery." They cannot (P16).
  • "Recovery means the transmitter goes silent." It is transmitting training and control ordered sets throughout (§7).
  • "Electrical Idle means Recovery." Many states use it; an EIOS is an announcement of idle (§7).
  • "The new speed can be published before the lane configuration catches up." §20's counterexample, measured at 98,251 incoherent cycles per 100,000 episodes.
  • "Retraining gives replayed TLPs new identities." It does not, and treating them as new Requests is a protocol error (§18).
  • "Equalization and Recovery are the same state." Recovery consumes a normalized equalization result; 17.4 owns the mechanism (§17).
  • "Recovery qualification must equal Polling qualification." No source here states that; §13 parameterises it.
  • "One lost CDR lock means the device disappeared." It means Recovery — which is the mechanism for exactly this (§4).
  • "Recovery always returns to L0." Failure falls toward Detect, and §1's cause register names paths toward Hot Reset, Disable and Loopback (§9).
  • "A Link in Recovery is unhealthy." Two of §4's four cause groups are entirely healthy.
  • "Frequent Recovery is a Transaction Layer bug." It is a physical-layer state; the cause bits say which physical thing (§21).
  • "The LTSSM holds the equalization coefficients." They live in the PHY and its adaptation logic (§17).

23. Understanding Check

24. What's Next

Recovery is how a Link that already exists gets repaired or changed.

It is not replay and it is not retry. It is the Physical Layer withdrawing normal traffic, re-establishing certainty, and returning — and §1's cause register shows that half of its reasons are entirely healthy, including the speed changes that happen during ordinary start-up.

Changing speed is a four-phase bilateral protocol — announce, decide, wait for the partner, request the PHY and wait — and the PHY is a macro that may stall and may decline. §20 measured what the wrong handshake produces: 452,512 double-accepts and 588,768 phantom results across 2.4 million sequences.

And the operating configuration survives the whole episode intact. It is not cleared on entry, it does not move while the candidate is built, and it is replaced in one cycle or not at all — because the alternative produced 98,251 incoherent cycles per 100,000 episodes, each one a window in which the two ends of a Link disagree about its shape.

Chapter 18.6 — L0 takes the state everything so far has been building toward. L0 is where the Link is finally allowed to do its job — and the interesting question is not what L0 is, but exactly what becomes legal there, what must stay perfectly still while traffic flows, and why "the LTSSM says L0" does not mean a single packet can move.

The idea to carry forward: a configuration under repair is not a configuration, and the operating one is not disposable.