Skip to content

PCIe · Module 18

Configuration — Lanes Become a Link

Trained lanes are not yet a Link. Configuration decides which of them belong together, what their logical identities are, and how wide the result is — and nothing above may see any of it until it is committed atomically.

Chapter 18.2 established that somebody is there. Chapter 18.3 established that the two ports can read each other.

Neither established a Link.

What exists at this point is a set of lanes that individually work. A Link is more than that: it is an agreement about which lanes belong together, what each one is called, in what order they carry data, and how wide the result is — agreed by both ends, because a receiver that recombines lanes in a different order than the transmitter split them produces data that was never sent (Chapter 17.2 §7).

How do two adjacent ports turn trained physical lanes into one coherent logical Link, and when does a discovery become a commitment?

1. The Verified Substates

2. Configuration Is Not Configuration Space

A second collision, and the same shape.

The Link Number is negotiated during Configuration between two adjacent ports (§1), lives in a 9-bit field, and identifies the Link's configuration context at the Physical Layer.

A Bus Number is assigned by software during enumeration (Chapter 7.4), identifies a bus in the hierarchy, and is written into Configuration Space registers.

Different layers, different agents, different lifetimes, and no relationship. A Link Number is settled by hardware before any software runs; a Bus Number is assigned by software long afterwards, over the Link that the Link Number helped establish.

The practical consequence: a training-time Link Number mismatch is a physical-layer problem between two ports. Reassigning bus numbers cannot affect it, and looking at the enumeration code is looking at software that has not started.

4. The Two Ends Are Not Symmetric

§1's substate table shows separate Upstream and Downstream paths through the same canonical substates — and that asymmetry is real.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
Downstream Port  (toward the Endpoint)      Upstream Port  (toward the Root)
  CONFIGURATION_DS_LW_START                   CONFIGURATION_US_LW_START
  CONFIGURATION_DS_LW_ACCEPT                  CONFIGURATION_US_LW_ACCEPT
  CONFIGURATION_DS_LN_WAIT                    CONFIGURATION_US_LN_WAIT
  CONFIGURATION_DS_LN_ACCEPT                  CONFIGURATION_US_LN_ACCEPT
                    both converge at
              CONFIGURATION_COMPLETE → CONFIGURATION_IDLE

5. Lane Number Is Not Pad Index

A physical lane is a pair of pins. A logical lane is a position in the Link's ordering. They are not the same, and assuming they are produces a design that works on some boards.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
physical lane (pins, routing)
    → training
        → logical lane number (position within this Link)

Why they can differ. Board layout does not always permit routing physical lane 0 to the position that becomes logical lane 0. Crossing traces to make the numbering match costs layers, area and signal integrity — so the protocol allows the mapping to be established during training instead.

Lane reversal is the case where the ordering is inverted end to end — physical lane 7 becoming logical lane 0, and so on. §1's source lists Polarity Inversion among its features and tracks lane numbering separately, and this chapter teaches the concept while declining the detection algorithm (§1).

6. Three Different Widths

The word "width" means three different things at three stages, and conflating them is where most Configuration confusion comes from.

WidthMeaningEstablished by
Connector / routedhow many lanes physically existthe board
Candidatehow many have a receiver, and train18.2, 18.3
Configuredhow many form the operating Linkthis chapter

Each is bounded by the one above and can be smaller. An x16 slot with an x8 card routed to eight lanes has a routed width of 8. If two of those lanes fail to train, the candidate set is 6. And 6 is not a Link width — §7.

A worked example, computed (§18):

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
routed         x8      lanes 0-7 present on the board
candidate    0x3F      lanes 0-5 detected and trained  (6 lanes)
configured     x4      lanes 0-3

Two lanes that work perfectly are not used. Not because anything is wrong with them, but because x6 is not a Link width, and the largest legal width the usable set supports is x4.

7. Width Is Not Popcount

One scoping note. The contiguous-from-zero policy above is this chapter's illustrative normalization, consistent with Chapter 17.3 §11. The real grouping rules are specification detail (§1), and §14's classifier therefore takes an already-normalized logical lane mask as input — its job is to recognise legal width classes, not to invent the normalization.

8. Provisional Is Not Committed

The chapter's flagship idea, and the one with the highest cost of failure.

During Configuration, everything discovered is provisional. Width can shrink as lanes are rejected. A lane map under construction is incomplete. A Link Number is proposed before it is agreed.

§1's source pins the boundary exactly: the Link Number "changes to the negotiated Link Number… when the LTSSM changes from CFG_COMPLETE to CFG_IDLE." The result becomes real at a specific transition, not gradually as it is discovered.

9. The Configuration Subtree

The Configuration subtree. Entry from Polling leads to Configuration Linkwidth Start, then Configuration Linkwidth Accept, then Configuration Lanenum Wait, then Configuration Lanenum Accept, then Configuration Complete, and finally Configuration Idle, from which the link proceeds toward normal operation. A failure at any phase returns toward Detect.fromPollingConfiguration.Linkwidth.StartConfiguration.Linkwidth.AcceptConfiguration.Lanenum.WaitConfiguration.Lanenum.AcceptConfiguration.CompleteConfiguration.Idleback toDetecttrained lanestrained laneswidth proposedwidthproposedwidth acceptedwidthacceptedlane numbers seenlane numbers seenlane numbers acceptedlane numbers acceptedCOMMIT - result becomes finalCOMMIT - result becomes finalCOMMIT -result become…failurefailurefailurefailure
Figure 1 — the Configuration subtree, following the canonical substate names verified in §1. Link width is settled first, then lane numbers, then the configuration is completed and finally becomes effective at the transition into Configuration.Idle — which §1's source identifies as the point at which the negotiated Link Number takes its final value. The Upstream and Downstream roles traverse the same named phases by different paths (§4); that split is not drawn here. Acceptance criteria and timeouts are not published in this chapter.

Three things to read out of the figure.

The order is width, then lane numbers. The canonical substate names say so — Linkwidth before Lanenum — and it makes sense: how many lanes participate is a smaller question than which one is which, and the second is only meaningful once the first is settled.

Complete and Idle are two states, not one. §1's source shows the negotiated Link Number taking its final value at the transition between themthat edge is the commit (§8), and the diagram labels it as such.

And failure returns toward Detect. Same principle as Chapters 18.2 §6 and 18.3 §8: a state that cannot establish what it needs goes back rather than proceeding on an unfounded configuration.

What is not drawn: the Upstream/Downstream split (§4), every acceptance criterion, and all timeouts. This is the Configuration subtree's phase topologyChapter 18.1 §6 owns the full map.

10. A Trace

Internal teaching signals, not PCIe pins. Four lanes, one duplicate arriving late.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
step             1     2     3     4     5     6     7     8     9
obs_valid        0     1     1     1     1     1     0     0     0
obs_lane         -     0     1     2     1     3     -     -     -
obs_link         -    0A    0A    0A    0A    0A     -     -     -
 
cand_mask       00    01    03    07    07    0F    0F    0F    0F
dup_error        0     0     0     0     1     1     1     1     1
cand_consistent  1     1     1     1     0     0     0     0     0
 
commit_req       0     0     0     0     0     0     0     1     0
config_valid     0     0     0     0     0     0     0     0     0
active_width     -     -     -     -     -     -     -     -     -
l0_handoff       0     0     0     0     0     0     0     0     0

Read steps 2–4. Three lane observations, all distinct, all with the same Link number. cand_mask accumulates one bit at a time and cand_consistent stays high.

Read step 5 — the duplicate. Lane 1 is claimed a second time. dup_error sets and cand_consistent goes low, and it stays low: consistency is not a per-observation verdict but a property of the whole set (§13).

Read step 6. Lane 3 arrives and the mask reaches 0x0Fwhich would be a perfectly good x4 if the set were consistent. It is not.

Read step 8 — the commit that does not happen. commit_req asserts and config_valid stays low. An inconsistent candidate cannot be committed (§15, P7), so nothing is exposed upward and l0_handoff never rises.

And note active_width across the whole trace. It never takes a value. A candidate is not a configuration (§8), and a design that had exposed the provisional width would have shown x1, then x2, then x4 as observations arrived — three configurations that were never true.

11. RTL — Normalized Configuration Types

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SYNTHESIZABLE. Normalized Configuration types.
// LEGAL LINK WIDTHS are canonical: x1, x2, x4, x8, x16. Everything else
// here -- field widths, the contiguity policy, the encoding -- is this
// chapter's internal normalization (section 1).
package config_pkg;
 
  parameter int MAX_LANES = 16;
  // GUARDED. Never zero-width, even at MAX_LANES == 1.
  parameter int LANE_W    = (MAX_LANES <= 1) ? 1 : $clog2(MAX_LANES);
  parameter int LINK_W    = 9;   // section 1: the vendor link_num field is 9 bits
  parameter int WIDTH_W   = 5;   // holds 0..16
 
  typedef struct packed {
    logic                valid;
    logic [LANE_W-1:0]   lane_num;   // LOGICAL lane number, not a pad index
    logic [LINK_W-1:0]   link_num;   // NOT a bus number (section 3)
  } cfg_obs_t;
 
  // THE COMMITTED CONFIGURATION. One struct, so section 15 can assign it
  // in a single nonblocking statement and partial commit is unconstructable.
  typedef struct packed {
    logic                   valid;
    logic [WIDTH_W-1:0]     width;       // 0, 1, 2, 4, 8 or 16
    logic [MAX_LANES-1:0]   lane_mask;   // exactly `width` bits set
    logic [LINK_W-1:0]      link_num;
  } link_config_t;
 
  function automatic bit is_legal_width(input logic [WIDTH_W-1:0] w);
    return (w inside {5'd1, 5'd2, 5'd4, 5'd8, 5'd16});
  endfunction
 
endpackage

12. RTL — Candidate Lane Collector

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import config_pkg::*;
 
// SYNTHESIZABLE. Accumulate the set of logical lanes claimed during this
// Configuration attempt.
// THIS BUILDS A CANDIDATE, NOT A CONFIGURATION (section 8). Nothing here
// is visible above the Physical Layer.
module cfg_candidate_collect (
  input  logic clk,
  input  logic rst_n,
 
  input  logic     state_entry,   // entering Configuration: start over
  input  cfg_obs_t obs,
 
  output logic [MAX_LANES-1:0] cand_mask,
  output logic [LINK_W-1:0]    cand_link,
  output logic                 cand_link_seen
);
 
  logic [MAX_LANES-1:0] mask_q;
  logic [LINK_W-1:0]    link_q;
  logic                 seen_q;
 
  assign cand_mask      = mask_q;
  assign cand_link      = link_q;
  assign cand_link_seen = seen_q;
 
  wire in_range = (MAX_LANES == 1) ? (obs.lane_num == '0)
                                   : (obs.lane_num < LANE_W'(MAX_LANES));
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      mask_q <= '0; link_q <= '0; seen_q <= 1'b0;
    end else if (state_entry) begin
      // ==============================================================
      // CHECKED FIRST. A candidate set inherited from a previous attempt
      // would let lanes that trained under different conditions join this
      // Link -- Chapter 18.3 section 12's rule, one layer up.
      // ==============================================================
      mask_q <= '0; link_q <= '0; seen_q <= 1'b0;
    end else if (obs.valid && in_range) begin
      // ONE BIT PER OBSERVATION. Setting a range of bits from a width
      // would assume contiguity that has not been established yet --
      // that is section 14's job, on the finished set.
      mask_q[obs.lane_num] <= 1'b1;
      if (!seen_q) begin
        link_q <= obs.link_num;
        seen_q <= 1'b1;
      end
    end
  end
 
endmodule

Classification: synthesizable.

Architecture. A set accumulator with entry clearing checked first and one bit written per observation.

The first Link number is latched, not overwritten. Later observations are compared against it by §13 rather than replacing it — a design that overwrote would silently adopt whichever partner spoke last and never notice a disagreement.

Failure — four. No entry clear carries a previous attempt's lanes into this one. Setting a range of bits assumes contiguity before it is checked. Overwriting the Link number hides a mismatch. And omitting the range check writes outside the mask.

13. RTL — Duplicate and Range Checker

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import config_pkg::*;
 
// SYNTHESIZABLE. Detect a candidate set that cannot describe a Link.
// TWO LANES CLAIMING THE SAME LOGICAL NUMBER is not a width problem and
// not recoverable by choosing a smaller width -- the set is incoherent,
// and section 15 refuses to commit it at all.
module cfg_consistency (
  input  logic clk,
  input  logic rst_n,
 
  input  logic     state_entry,
  input  cfg_obs_t obs,
 
  input  logic [MAX_LANES-1:0] cand_mask,      // set BEFORE this observation
  input  logic [LINK_W-1:0]    cand_link,
  input  logic                 cand_link_seen,
 
  output logic dup_error,
  output logic range_error,
  output logic link_mismatch,
  output logic cand_consistent
);
 
  logic dup_q, rng_q, lnk_q;
 
  assign dup_error     = dup_q;
  assign range_error   = rng_q;
  assign link_mismatch = lnk_q;
 
  // CONSISTENCY IS A PROPERTY OF THE WHOLE SET, not a verdict on the
  // latest observation -- so these are STICKY until the next attempt.
  // A design that recomputed it per observation would report the set
  // consistent again as soon as a clean observation arrived after a bad
  // one (section 18, mutation 5).
  assign cand_consistent = !dup_q && !rng_q && !lnk_q;
 
  wire in_range = (MAX_LANES == 1) ? (obs.lane_num == '0)
                                   : (obs.lane_num < LANE_W'(MAX_LANES));
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      dup_q <= 1'b0; rng_q <= 1'b0; lnk_q <= 1'b0;
    end else if (state_entry) begin
      // Cleared only on a NEW ATTEMPT. Checked first.
      dup_q <= 1'b0; rng_q <= 1'b0; lnk_q <= 1'b0;
    end else if (obs.valid) begin
      if (!in_range) begin
        rng_q <= 1'b1;
      end else begin
        // THE DUPLICATE TEST. Note it reads cand_mask BEFORE this
        // observation is applied -- comparing against the post-update
        // mask would find every observation to be a duplicate of itself.
        if (cand_mask[obs.lane_num]) dup_q <= 1'b1;
      end
      // A SECOND LINK NUMBER means two different negotiations are being
      // observed on one candidate set. Not a width problem either.
      if (cand_link_seen && (obs.link_num != cand_link)) lnk_q <= 1'b1;
    end
  end
 
endmodule

Classification: synthesizable.

Architecture. Three sticky error flags, cleared only on a new attempt.

Three distinct errors kept separate, because they mean different things: a duplicate means two lanes claim one position, a range error means a malformed identity, and a Link-number mismatch means two negotiations are being conflated. Merged into one "bad" bit, §19 loses three diagnoses.

Why sticky. Consistency describes the set, so a clean observation after a bad one does not repair it — a recomputing implementation would declare the set consistent again and commit an incoherent lane map.

And the duplicate test reads the pre-update mask. Against the post-update mask every observation is its own duplicate, and no configuration ever commits.

Executed (§18): the collector and checker were run over lane sequences — [0,1,2,3] gives mask 0x0F with no error; [0,1,1,3] sets dup_error with mask 0x0B; [0,1,2,9] sets range_error; and [3,2,1,0] — arriving in reverse order — is clean, mask 0x0F. That last case is the point of §5: reversed order is not an error, it is a lane map.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import config_pkg::*;
 
// SYNTHESIZABLE. Reduce a candidate lane set to a LEGAL Link width.
// WIDTH IS NOT POPCOUNT (section 7). This is combinational and pure --
// it holds no state, so it can be reasoned about and exhaustively checked
// as a function, which section 18 did over all 65,536 masks.
module cfg_width_class (
  input  logic [MAX_LANES-1:0] cand_mask,
  output logic [WIDTH_W-1:0]   width,
  output logic [MAX_LANES-1:0] width_mask
);
 
  // Step 1: the CONTIGUOUS RUN FROM LANE 0.
  // A gap terminates the run. Lanes beyond a gap are good lanes that this
  // Link cannot use (section 7) -- 0x000D has three good lanes and a run
  // of one.
  logic [MAX_LANES:0] run_ok;
  logic [WIDTH_W-1:0] run_len;
 
  always_comb begin
    run_ok[0] = 1'b1;
    for (int i = 0; i < MAX_LANES; i++)
      run_ok[i+1] = run_ok[i] && cand_mask[i];
 
    run_len = '0;
    for (int i = 0; i < MAX_LANES; i++)
      if (run_ok[i+1]) run_len = WIDTH_W'(i + 1);
  end
 
  // Step 2: REDUCE THE RUN TO A LEGAL WIDTH, largest first.
  // A run of 6 is not x6. It is x4, and two working lanes go unused.
  always_comb begin
    if      (run_len >= 5'd16) width = 5'd16;
    else if (run_len >= 5'd8)  width = 5'd8;
    else if (run_len >= 5'd4)  width = 5'd4;
    else if (run_len >= 5'd2)  width = 5'd2;
    else if (run_len >= 5'd1)  width = 5'd1;
    else                       width = 5'd0;   // no usable width
  end
 
  // Step 3: the mask of exactly the lanes the Link will use.
  // BUILT FROM THE WIDTH, never passed through from cand_mask -- so the
  // two can never disagree, and Chapter 17.1's distributor cannot be
  // handed a lane outside the configured width.
  always_comb begin
    width_mask = '0;
    for (int i = 0; i < MAX_LANES; i++)
      if (WIDTH_W'(i) < width) width_mask[i] = 1'b1;
  end
 
endmodule

Classification: synthesizable, purely combinational.

Architecture. Run, then reduce, then rebuild the mask from the width.

Step 3 is the one that prevents a class of bug outright. Passing cand_mask through would let the mask contain lanes outside the configured width — exactly the input Chapter 17.1 §10's distributor must never receive. Rebuilding from the width makes width and mask agree by construction, and §18 verified it: across all 65,536 masks, every width_mask is a subset of cand_mask with a popcount equal to width.

Failure — four. $countones is wrong for 65,530 of 65,536 masks (§7). Not requiring lane 0 accepts 0xFF00 as x8 when the run is empty. Passing the candidate mask through decouples mask from width. And reducing smallest-first returns x1 for a perfectly good x16.

15. RTL — Provisional Register and Atomic Commit

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import config_pkg::*;
 
// SYNTHESIZABLE. The commit boundary (section 8).
// EVERYTHING ABOVE THE PHYSICAL LAYER READS ONLY `cfg` -- the provisional
// values are not brought out. Chapter 17.1's lane distributor and Chapter
// 17.2's recombiner consume this struct, and a half-updated version makes
// them disagree about the shape of the Link.
module cfg_commit (
  input  logic clk,
  input  logic rst_n,
 
  input  logic state_entry,
 
  // ---- Provisional, from sections 12-14 ---------------------------------
  input  logic [WIDTH_W-1:0]   prov_width,
  input  logic [MAX_LANES-1:0] prov_mask,
  input  logic [LINK_W-1:0]    prov_link,
  input  logic                 prov_link_seen,
  input  logic                 cand_consistent,
 
  input  logic commit_req,       // the Complete -> Idle edge (section 1)
 
  // ---- Committed. The ONLY outputs anything above may use. --------------
  output link_config_t cfg,
  output logic         commit_rejected
);
 
  link_config_t cfg_q;
  logic         rej_q;
 
  assign cfg             = cfg_q;
  assign commit_rejected = rej_q;
 
  // ==================================================================
  // THE COMMIT GUARD. Every condition must hold; there is no arm that
  // commits a partial or incoherent result.
  //
  //   consistent   -- no duplicate, no range error, no link mismatch (13)
  //   legal width  -- x0 is not a Link (section 14 returns 0 for an
  //                   unusable set, and that must be refused, not
  //                   committed as a zero-wide Link)
  //   link seen    -- a Link number was actually observed
  // ==================================================================
  wire commit_ok = cand_consistent
                && is_legal_width(prov_width)
                && prov_link_seen;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      cfg_q <= '0; rej_q <= 1'b0;
    end else if (state_entry) begin
      // ENTERING CONFIGURATION INVALIDATES THE COMMITTED CONFIGURATION.
      // Checked first. Configuration is reached from Polling, so there is
      // no operational Link to preserve -- there is nothing above that may
      // legally act on a configuration right now, and a stale width left
      // exposed could only mislead.
      //
      // RECOVERY IS THE OPPOSITE CASE and must not copy this arm. A Link
      // entering Recovery already HAS a coherent operational configuration
      // that stays active until a new one commits (Chapter 18.5). Clearing
      // on entry there would tear down a working configuration to
      // renegotiate one that may end up identical.
      cfg_q <= '0; rej_q <= 1'b0;
    end else if (commit_req) begin
      if (commit_ok) begin
        // ============================================================
        // ONE NONBLOCKING ASSIGNMENT TO ONE PACKED STRUCT.
        //
        // width, lane_mask, link_num and valid become visible in the SAME
        // CYCLE. There is no ordering between them because there is no
        // sequence -- partial commit is not a bug to be avoided here, it
        // is UNCONSTRUCTABLE (section 8).
        // ============================================================
        cfg_q <= '{ valid     : 1'b1,
                    width     : prov_width,
                    lane_mask : prov_mask,
                    link_num  : prov_link };
      end else begin
        // A REFUSED COMMIT LEAVES NOTHING EXPOSED and says so. Committing
        // a "best effort" configuration would hand the distributor a lane
        // map that the receiver is not using.
        cfg_q <= '0;
        rej_q <= 1'b1;
      end
    end
  end
 
endmodule

Classification: synthesizable.

Architecture. One packed struct, one assignment, guarded.

The atomicity is structural, not procedural. Nothing sequences the fields because they are one object — so no future edit can introduce an ordering, and no review has to check for one.

And an unusable set is refused, not committed as x0. cfg_width_class returns width 0 when there is no run from lane 0; is_legal_width rejects it. A zero-wide committed Link would satisfy "valid" and carry nothing — the worst possible outcome, because everything above would proceed.

Failure — five. Separate registers for width and mask permit the §18 counterexample. Committing on commit_req unconditionally exposes an incoherent map. Accepting width 0 produces a valid zero-wide Link. Exposing the provisional values lets consumers act on a configuration under construction. And no entry invalidation presents a stale configuration during renegotiation.

16. RTL — Configuration Progress Controller

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import config_pkg::*;
 
// SYNTHESIZABLE. Progress within Configuration only.
// THIS IS NOT THE LTSSM (Chapter 18.1 section 9), and it is not Chapter
// 18.5's Recovery -- speed changes, Up/Down-configure and autonomous width
// changes are NOT modelled here (section 1). It sequences one thing:
// collect, check, classify, commit, hand off.
module cfg_progress #(
  parameter int CNT_W = 24
) (
  input  logic clk,
  input  logic rst_n,
 
  input  logic state_entry,
 
  input  logic cand_consistent,
  input  logic prov_link_seen,
  input  logic [WIDTH_W-1:0] prov_width,
  input  logic settle_done,        // enough observation to stop collecting
  input  logic timeout_elapsed,
 
  input  link_config_t cfg,
 
  output logic commit_req,
  output logic config_done,
  output logic config_fail,
  output logic l0_handoff
);
 
  typedef enum logic [2:0] {
    S_WIDTH, S_LANENUM, S_COMPLETE, S_IDLE, S_FAIL
  } st_e;
  st_e st_q;
 
  // COMMIT IS AN EDGE, not a level -- section 1 places it at the
  // Complete -> Idle transition. A level would re-commit every cycle.
  assign commit_req  = (st_q == S_COMPLETE) && settle_done;
  assign config_done = (st_q == S_IDLE);
  assign config_fail = (st_q == S_FAIL);
 
  // ==================================================================
  // THE HANDOFF IS GATED ON THE COMMITTED STRUCT, not on reaching Idle.
  //
  // Reading `cfg.valid` rather than the state means the Link cannot be
  // handed to normal operation unless a configuration was actually
  // committed -- so a refused commit (section 15) cannot be followed by
  // traffic (section 18, mutation 12).
  // ==================================================================
  assign l0_handoff = (st_q == S_IDLE) && cfg.valid;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      st_q <= S_WIDTH;
    end else if (state_entry) begin
      st_q <= S_WIDTH;
    end else begin
      unique case (st_q)
        S_WIDTH :
          // A CANDIDATE THAT IS ALREADY INCOHERENT FAILS IMMEDIATELY.
          // Continuing to collect lanes onto a set with a duplicate wastes
          // the rest of the timeout for a result that cannot commit.
          if (!cand_consistent)          st_q <= S_FAIL;
          else if (timeout_elapsed)      st_q <= S_FAIL;
          else if (settle_done && is_legal_width(prov_width))
                                         st_q <= S_LANENUM;
 
        S_LANENUM :
          if (!cand_consistent)          st_q <= S_FAIL;
          else if (timeout_elapsed)      st_q <= S_FAIL;
          else if (prov_link_seen)       st_q <= S_COMPLETE;
 
        S_COMPLETE :
          // The commit fires on the way out, and the result is checked on
          // arrival: if it was refused, Idle exposes nothing and the
          // handoff does not happen.
          if (!cand_consistent)          st_q <= S_FAIL;
          else if (settle_done)          st_q <= S_IDLE;
          else if (timeout_elapsed)      st_q <= S_FAIL;
 
        S_IDLE, S_FAIL : ;
 
        default : st_q <= S_WIDTH;
      endcase
    end
  end
 
endmodule

Classification: synthesizable (Configuration only — not the LTSSM, and not Recovery).

Architecture. Width, then lane numbers, then complete, then idle — §9's phase order, with a fail path.

Two decisions worth naming. commit_req is an edge at the Complete → Idle boundary, matching §1's evidence; a level would re-commit every cycle and defeat the point of a commit. And l0_handoff reads cfg.valid, not the state — so a refused commit cannot be followed by traffic, which a state-only gate would permit.

Failure — four. Committing on a level. Gating the handoff on S_IDLE alone. Continuing to collect onto an already-incoherent set. And treating a width the classifier rejected as progress.

17. Assertions

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SVA over the Configuration blocks. These assert the LOCAL contract:
// what may be committed, and what the committed result must satisfy.
// They assert NOTHING about the partner agreeing, about any particular
// width being achievable, or about Configuration succeeding.
 
// ---- ENVIRONMENT ------------------------------------------------------
assume property (@(posedge clk) disable iff (!rst_n)
  obs.valid |-> !$isunknown({obs.lane_num, obs.link_num}));
assume property (@(posedge clk) disable iff (!rst_n)
  state_entry |=> !state_entry);
 
// ---- THE CANDIDATE ----------------------------------------------------
 
// P1: entering Configuration starts from an empty candidate -- lanes from
// a previous attempt must not join this Link.
property p_entry_clears;
  @(posedge clk) disable iff (!rst_n)
  state_entry |=> ((cand_mask == '0) && cand_consistent && !cfg.valid);
endproperty
a_entry : assert property (p_entry_clears);
 
// P2: a lane bit is set ONLY by an in-range observation of that lane.
property p_mask_growth;
  @(posedge clk) disable iff (!rst_n)
  ((cand_mask & ~$past(cand_mask)) != '0)
    |-> ($past(obs.valid) && (cand_mask & ~$past(cand_mask))
                          == (MAX_LANES'(1) << $past(obs.lane_num)));
endproperty
a_growth : assert property (p_mask_growth);
 
// P3: A DUPLICATE LOGICAL LANE IS DETECTED. Two lanes cannot occupy one
// position (section 13).
property p_dup_detected;
  @(posedge clk) disable iff (!rst_n)
  (obs.valid && cand_mask[obs.lane_num]) |=> dup_error;
endproperty
a_dup : assert property (p_dup_detected);
 
// P4: inconsistency is STICKY until the next attempt -- a clean
// observation after a bad one does not repair the set.
property p_sticky;
  @(posedge clk) disable iff (!rst_n)
  (!cand_consistent && !state_entry) |=> !cand_consistent;
endproperty
a_sticky : assert property (p_sticky);
 
// ---- THE WIDTH --------------------------------------------------------
 
// P5: THE COMMITTED WIDTH IS ALWAYS LEGAL. x6 is not a Link.
property p_legal_width;
  @(posedge clk) disable iff (!rst_n)
  cfg.valid |-> is_legal_width(cfg.width);
endproperty
a_legal : assert property (p_legal_width);
 
// P6: WIDTH AND MASK AGREE, and the mask uses only candidate lanes.
// The property that keeps Chapter 17.1's distributor safe.
property p_mask_matches_width;
  @(posedge clk) disable iff (!rst_n)
  cfg.valid |-> (($countones(cfg.lane_mask) == cfg.width)
              && ((cfg.lane_mask & ~cand_mask) == '0));
endproperty
a_match : assert property (p_mask_matches_width);
 
// ---- THE COMMIT -------------------------------------------------------
 
// P7: AN INCONSISTENT CANDIDATE IS NEVER COMMITTED.
property p_no_bad_commit;
  @(posedge clk) disable iff (!rst_n)
  $rose(cfg.valid) |-> $past(cand_consistent);
endproperty
a_guard : assert property (p_no_bad_commit);
 
// P8: ATOMICITY. Every field of the committed configuration changes in the
// SAME cycle -- no field may move while the others are stable.
// The counterexample in section 18 is exactly the violation of this.
property p_atomic;
  @(posedge clk) disable iff (!rst_n)
  ($changed(cfg.width) || $changed(cfg.lane_mask) || $changed(cfg.link_num))
    |-> $changed(cfg.valid) || $rose(state_entry) || $rose(commit_req);
endproperty
a_atomic : assert property (p_atomic);
 
// P9: the committed configuration is STABLE between commits -- consumers
// may not see it drift.
property p_stable;
  @(posedge clk) disable iff (!rst_n)
  (cfg.valid && !commit_req && !state_entry) |=> $stable(cfg);
endproperty
a_stable : assert property (p_stable);
 
// P10: A REFUSED COMMIT EXPOSES NOTHING and reports itself.
property p_reject;
  @(posedge clk) disable iff (!rst_n)
  (commit_req && !cand_consistent) |=> (!cfg.valid && commit_rejected);
endproperty
a_reject : assert property (p_reject);
 
// P11: a zero width is never committed as a valid Link.
property p_no_zero_link;
  @(posedge clk) disable iff (!rst_n)
  cfg.valid |-> (cfg.width != '0);
endproperty
a_nonzero : assert property (p_no_zero_link);
 
// ---- THE HANDOFF ------------------------------------------------------
 
// P12: NO HANDOFF TO NORMAL OPERATION WITHOUT A COMMITTED CONFIGURATION.
// The property that connects this chapter to Chapter 18.6.
property p_handoff_needs_commit;
  @(posedge clk) disable iff (!rst_n)
  l0_handoff |-> cfg.valid;
endproperty
a_handoff : assert property (p_handoff_needs_commit);
 
// P13: NO NORMAL TRAFFIC DURING CONFIGURATION. Bound against Chapter
// 18.1's gate -- the Link is not a Link yet.
property p_no_traffic;
  @(posedge clk) disable iff (!rst_n)
  (dut_ltssm.cur_state == ST_CONFIGURATION) |-> !dut_gate.traffic_enabled;
endproperty
a_no_traffic : assert property (p_no_traffic);
 
// P14: the provisional values never reach a consumer -- structurally, the
// only configuration port is `cfg`.
property p_consumer_uses_committed;
  @(posedge clk) disable iff (!rst_n)
  dut_distributor.active_mask == (cfg.valid ? cfg.lane_mask : '0);
endproperty
a_consumer : assert property (p_consumer_uses_committed);
 
// P15: reset.
property p_reset;
  @(posedge clk)
  !rst_n |=> (!cfg.valid && (cand_mask == '0) && !l0_handoff);
endproperty
a_reset : assert property (p_reset);

P6 and P8 are the two that matter most. P6 is what makes Chapter 17.1's distributor safe — width and mask cannot disagree, and the mask cannot contain a lane that was never a candidate. P8 forbids any field moving alone, which is the partial-commit failure stated as a property.

P11 exists because "valid and zero-wide" is the worst outcome. Everything above proceeds on a Link that carries nothing, and no error is reported anywhere.

No liveness. "Configuration eventually completes" depends on a partner, and asserting it would assume away the failure this state exists to detect.

18. Verification, Fault Injection, and Model Verification

Every number in this chapter was executed before publication.

The width classifier — exhaustive

All 65,536 lane masks at MAX_LANES = 16, checked against three independent invariants:

  • every width is in 16
  • every width_mask is a subset of cand_mask
  • every width_mask has a popcount equal to width

Zero violations.

And the distribution, which is worth seeing:

ResultMasksOf 65,536
x1610.002%
x82550.4%
x43,8405.9%
x212,28818.8%
x116,38425.0%
no usable width32,76850.0%

Exactly half of all masks yield nothing, because exactly half have lane 0 absent — which is the contiguity policy made visible.

The popcount comparison (§7): popcount equals the legal width for 6 masks out of 65,5360x0000, 0x0001, 0x0003, 0x000F, 0x00FF, 0xFFFF, and nothing else.

The quoted cases, computed:

cand_maskrunwidthwidth_mask
0x00FF8x80x00FFcontiguous
0x001F5x40x000F5 good lanes, one unused
0x000D1x10x0001gap at lane 1
0xFF000none0x00008 good lanes, no usable Link
0xFFFF16x160xFFFF

The collector and checker — executed

Observation orderResult
[0,1,2,3]mask 0x0F, no error — clean x4
[0,1,1,3]dup_error, mask 0x0B
[0,1,2,9]range_error
[3,2,1,0]mask 0x0F, no error — reversed order is a lane map, not a fault

Directed tests

  • Every legal width — x1, x2, x4, x8, x16 — committed and read back (P5, P6).
  • MAX_LANES = 1, 2, 4, 8, 16. The LANE_W guard is exercised at 1.
  • Candidate with a gap (0x000D) — verify x1, not x3.
  • Candidate without lane 0 (0xFF00) — verify no commit (P11).
  • Duplicate at the last observation — verify the whole set is refused (P4, P7).
  • Two different Link numbers — verify link_mismatch and refusal.
  • commit_req with an inconsistent candidate — verify nothing is exposed (P10). Required.
  • Re-entry mid-negotiation — verify the committed configuration is invalidated (P1).
  • Reset during Complete (P15).

Mutations

#MutationCaught byLab symptom
1width = $countones(cand_mask)P5x6 committed; distributor drives six lanes the receiver is not recombining
2width_mask passed through from cand_maskP6mask and width disagree; lanes outside the width driven
3run not anchored at lane 0P60xFF00 committed as x8; no data ever arrives
4duplicate test against the post-update maskP3every observation is a duplicate; nothing ever commits
5consistency recomputed per observationP4a clean observation after a duplicate commits an incoherent map
6no entry clear on the candidateP1lanes from a previous attempt join this Link
7width and mask in separate registersP8the counterexample below
8commit on a level rather than the edgeP9configuration re-commits every cycle; consumers see churn
9width 0 committed as validP11a valid zero-wide Link; everything above proceeds and nothing moves
10Link number overwritten by each observationmismatch checktwo negotiations conflated silently
11provisional width exposed to the distributorP14width changes as observations arrive
12handoff gated on S_IDLE aloneP12traffic after a refused commit
13traffic enabled during ConfigurationP13packets into a Link that does not exist
14reset leaves the committed configurationP15a stale width survives reset
15lane reversal treated as a duplicatecollector testvalid reversed maps rejected; works on one board, fails on another

19. Debugging

Symptom → which quantity is wrong → signal → distinguishing experiment.

The most common Configuration observation, and it is usually not a fault.

Read the three widths in order (§6):

  1. Routed width — the board. An x16 slot does not mean sixteen routed lanes.
  2. Candidate mask — which lanes detected and trained (18.2, 18.3).
  3. Configured width — what §14 made of the candidate set.

The single most informative reading is cand_mask against cfg.width. A candidate of 0x001F — five good lanes — configuring as x4 is correct behaviour: x5 is not a Link width, and one working lane goes unused (§7).

But a candidate of 0x000D configuring as x1 points at lane 1, which trained on neither end. One lane's problem, costing three lanes of bandwidth — because a gap truncates the run.

And a candidate of 0xFF00 configuring as nothing is the striking one: eight good lanes and no Link at all, because lane 0 is absent.

It reaches Configuration and returns to Detect

Read the three consistency flags separately (§13) — this is why they are not merged:

FlagMeaningWhere to look
dup_errortwo lanes claim one logical numberlane mapping, board routing, reversal handling
range_errora malformed lane identitythe parser, or the partner
link_mismatchtwo Link numbers on one candidate settwo negotiations conflated — check topology

One bit would have said "Configuration failed" and left all three open.

Suspect the commit, not the channel (§18's counterexample).

The signature is specific: LCRC errors (Chapter 14.3) with no PHY errors and no bit errors — a channel that is measurably clean and a Data Link Layer that keeps rejecting packets. Replay usually succeeds, which is what makes it look transient.

The distinguishing experiment: capture cfg.width and cfg.lane_mask in the same cycle and check $countones(lane_mask) == width continuously, not once. A single cycle of disagreement is the whole bug — and P6 with P8 is that check made permanent.

Works on one board, fails on another, same design

Suspect lane mapping (§5).

The distinguishing experiment: transmit a distinct pattern per lane and observe which lane receives which.

  • All lanes correct but in reversed orderlane reversal, and the mapping logic is the target.
  • One lane's data inverted, order correctpolarity inversion on that lane, a PHY matter (§5).
  • Correct on some lanes, absent on others → a training or routing problem, not a mapping one.

Three different investigations, separated in one run — and a merged "data is wrong" observation separates none of them.

It is stuck and someone is reading enumeration code

Stop (§2).

If the LTSSM has not reached normal operation, no software explanation is possible, because no TLP can have been issued. Configuration Space is unreachable by definition, and time spent on the enumeration path is time not spent on the Link.

20. Common Misconceptions

  • "Configuration means Configuration Space." The most expensive collision in PCIe (§2).
  • "Configuration is done by software." Hardware, two adjacent ports, before software runs.
  • "Link Number is Bus Number." Different layers, agents and lifetimes (§3).
  • "Lane number equals the physical pad index." Logical position, established during training (§5).
  • "Lane reversal and polarity inversion are the same." Across lanes versus within a pair (§5).
  • "Width equals the number of working lanes." Correct for 6 masks out of 65,536 (§7).
  • "Six good lanes give x6." x6 is not a Link width; x4, with two unused (§6).
  • "Eight good lanes always give x8." Not if lane 0 is absent — 0xFF00 gives nothing (§18).
  • "The width is known as soon as lanes train." Provisional until committed (§8).
  • "Committing the width and the map one after the other is fine." §18's counterexample.
  • "An x16 slot means an x16 Link." Routed, candidate and configured are three different widths (§6).
  • "Both ends play the same part." The Upstream/Downstream split is structural (§4).
  • "Configuration negotiates the link speed." Speed changes belong to Chapter 18.5 — Recovery.
  • "A narrower-than-expected Link is a bug." Usually the correct reduction of an imperfect candidate set (§19).
  • "Reaching Configuration.Idle means traffic can flow." Only if a configuration was actually committed (P12).

21. Understanding Check

22. What's Next

Configuration is where lanes become a Link.

Three quantities are established — which lanes belong, what their logical identities are, and how wide the result is — plus a Link identity that both ends agree on, and §1's source pins where that becomes final: the Configuration.CompleteConfiguration.Idle transition.

The arithmetic is not what it looks like. Width is not popcount — that equivalence holds for 6 of 65,536 lane masks. A gap truncates the run; a missing lane 0 makes eight good lanes worth nothing; five usable lanes give x4 and one lane goes unused.

And the commit boundary is the idea to carry. Everything discovered is provisional; the committed configuration changes all at once or not at all, because the two blocks that consume it are the two halves of a Link, and one cycle of disagreement between them produces corruption that surfaces as a CRC error on a channel with no errors in it.

Chapter 18.5 — Recovery takes the question this chapter deliberately left alone. A configured Link does not stay configured forever. Speed changes, equalization at a new rate, width changes, and re-synchronization after a disturbance all pass through Recovery — and it is the state a working Link spends its life re-entering, which is a very different problem from establishing one.

The idea to carry forward: a discovery is not a commitment, and the difference has to be a boundary rather than a habit.