Skip to content

PCIe · Module 6

x2 Links — Two Lanes, One Logical Connection

Adding a second lane does not simply add capacity — it introduces coordination. Why one logical stream spread across two physical paths creates distribution, reassembly, relative-arrival, and lane-status problems that do not exist at x1, and how to verify and debug them.

Chapter 6.1 established a lane as one unit of physical Link width and an x1 Link as the case where the Link is one lane wide. At x1 there was nothing to coordinate — one lane, one path per direction, no possibility of two parts of the connection disagreeing.

Add one lane and that changes. Not gradually — immediately, and permanently.

What new hardware and verification problems appear when a PCIe Link grows from one lane to two?

1. What Actually Changes

Put the two side by side, because the delta is the subject.

x1x2
Physical lanes12
Paths per direction12
Distribution decisionnone — there is one pathinformation must be spread across lanes
Reassemblynonelane streams must be recombined into one
Relative arrivalnot a conceptthe two lanes need not arrive together
Lane statusone lane's status is the Link'stwo lane statuses must be reconciled
Failure modesdirection-specificdirection-specific and lane-specific and cross-lane

Only the first row is about capacity. Every other row is a new problem, and none of them has an analogue at x1.

An x2 Link between component A and component B. Two physically distinct lanes, lane 0 and lane 1, each carrying both directions, connect the same pair of components. Both lanes together form one logical Link rather than two separate Links.Component Aone endpoint of oneLinkLane 0two directional pairsLane 1two directional pairsComponent Bthe other endpoint12
Figure 1 — an x2 Link. Two physically distinct lanes connect the same pair of components, each lane carrying both directions as established in Chapter 6.1. They are not two Links and they do not carry independent traffic: both lanes together form one logical Link carrying one connection's traffic.

The figure is deliberately plain, because the important content is what it is not showing:

Not two Links. There is one connection between A and B. The two lanes are the width of that connection.

Not two independent traffic streams. Lane 0 does not carry one set of transactions while lane 1 carries another. One logical stream is spread across both.

Not a pair of x1 Links side by side. Two x1 Links would each be complete connections, independently established, independently configured, independently able to work or fail. An x2 Link is one connection whose lanes must behave coherently.

That last distinction is the source of everything difficult in this chapter.

3. Distribution and Reconstruction

If one logical stream is carried by two physical lanes, then something must decide which information goes on which lane, and something at the other end must put it back together in the right order.

That is the new problem, stated in full. It has two halves:

Distribution. At the transmitter, a logical stream is divided so that each lane carries part of it.

Reconstruction. At the receiver, the parts arriving on each lane are recombined into the original logical stream, in the original order, with nothing lost, duplicated, or misplaced.

The two halves must agree exactly. A receiver reconstructing by a different rule than the transmitter distributed by produces data that is not corrupted in any single position — every byte arrives intact — but is assembled wrongly. That failure mode is genuinely nasty, because it looks like data corruption while every individual lane is behaving perfectly.

4. Relative Arrival Between Lanes

The second new problem is subtler, and it is the one that most surprises engineers coming from single-lane thinking.

The two lanes need not deliver their information at exactly the same instant.

The lanes are physically distinct paths. Their routing lengths can differ, they traverse different regions of package and board, and they are subject to independent variation. Information launched simultaneously on both lanes need not arrive simultaneously.

This relative arrival difference between lanes of the same Link is called skew.

The consequence for the receiver is direct: it cannot assume that the parts of one logical unit are simultaneously available. It must be able to hold what has arrived on one lane while waiting for the corresponding part on another, and then combine them.

5. What x2 Does and Does Not Give You

What x2 gives you: twice the physical lane count of x1.

That is a precise statement about the transport, and it is the only unqualified one available.

Why "x2 is twice as fast" is not acceptable: delivered throughput depends on the Link and on everything else in the path. From Chapter 5.1 and Chapter 5.2: additional Link capacity improves delivered throughput only to the extent the Link was the limiting resource.

Concretely, an x2 Link fails to deliver twice an x1 Link's throughput whenever:

  • the traffic source cannot generate demand at the higher rate;
  • the destination cannot absorb it;
  • another segment of the path is narrower or slower;
  • the workload's transaction pattern makes per-transaction overhead dominant, so the payload fraction is low regardless of width;
  • or the Link is substantially idle, in which case width was never the constraint.

The disciplined statement:

x2 doubles the physical lane count relative to x1. Delivered throughput approaches a corresponding increase only when the Link was the limiting resource and the rest of the path can sustain the higher demand.

Chapter 6.7 owns the actual arithmetic across rate and width. This chapter deliberately produces no numbers and no tables — the proportionality from Chapter 6.1, aggregate capacity ∝ lane count, remains the full extent of the quantitative claim.

6. Microarchitecture

An illustrative two-lane digital organisation. On transmit, a logical transmit stream feeds a distributor, which feeds the lane 0 and lane 1 transmit paths toward the far component. On receive, the lane 0 and lane 1 receive paths from the far component feed a collector, which produces the logical receive stream.Logical TXstreamone stream, aggregatewidthDistributordivides a beat intofragmentsLane 0 and lane1 TXtwo independent pathsTo the farcomponentone Link, two lanesLogical RXstreamreconstructed, inorderCollectorwaits for bothfragmentsLane 0 and lane1 RXmay arrive atdifferent timesFrom the farcomponentone Link, two lanes12
Figure 2 — an illustrative two-lane digital organisation. On transmit, a distributor divides one logical stream into per-lane fragments. On receive, a collector holds fragments as they arrive on each lane and combines them once the corresponding parts are present. Illustrative structure — not PCIe's normative aggregation architecture.

Illustrative architecture — not PCIe's normative lane-aggregation partitioning. The distributor and collector are named for what they do in this teaching model. Their real counterparts are shaped by the aggregation rules Chapter 6.6 covers.

The structural point stands regardless: x2 hardware contains a distribution stage and a collection stage that x1 hardware does not. Those stages are where the new bugs live.

7. RTL — Two-Lane Distribution

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SYNTHESIZABLE. Generic two-lane distributor.
// The distribution RULE here is a teaching placeholder. The COORDINATION
// behaviour around it is the point: an aggregate beat is accepted atomically,
// and its two fragments never separate.
module x2_stripe_tx #(
  parameter int LANE_W = 32                 // bits per lane fragment
) (
  input  logic                clk,
  input  logic                rst_n,
 
  // Abstract lane availability. Not PCIe signals.
  input  logic [1:0]          lane_active,
 
  // Aggregate ingress: one logical beat spans both lanes.
  input  logic                in_valid,
  output logic                in_ready,
  input  logic [2*LANE_W-1:0] in_data,
  input  logic                in_last,
 
  // Per-lane egress.
  output logic                l0_valid,
  input  logic                l0_ready,
  output logic [LANE_W-1:0]   l0_data,
  output logic                l0_last,
 
  output logic                l1_valid,
  input  logic                l1_ready,
  output logic [LANE_W-1:0]   l1_data,
  output logic                l1_last
);
 
  logic [LANE_W-1:0] d0_q, d1_q;
  logic              last0_q, last1_q;
  logic              v0_q,  v1_q;
 
  // This simplified model operates at x2 only when both lanes are available.
  // Reduced-width operation is a negotiated behaviour and is NOT modelled here.
  wire x2_available = (lane_active == 2'b11);
 
  // Atomic acceptance: the aggregate beat is taken only when BOTH lane slots
  // can receive their fragment. in_ready never depends on in_valid.
  assign in_ready = x2_available
                 && (!v0_q || l0_ready)
                 && (!v1_q || l1_ready);
 
  wire accept = in_valid && in_ready;
 
  // Both lane valids are REGISTERED and never observe their own ready.
  assign l0_valid = v0_q;  assign l0_data = d0_q;  assign l0_last = last0_q;
  assign l1_valid = v1_q;  assign l1_data = d1_q;  assign l1_last = last1_q;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      v0_q <= 1'b0; v1_q <= 1'b0;
      d0_q <= '0;   d1_q <= '0;
      last0_q <= 1'b0; last1_q <= 1'b0;
    end else begin
      // Lanes release independently — a fragment already accepted by lane 0
      // is gone even if lane 1 is still holding. Association was established
      // at load time, so independent release cannot mix items.
      if (v0_q && l0_ready) v0_q <= 1'b0;
      if (v1_q && l1_ready) v1_q <= 1'b0;
 
      // Atomic load. Both fragments of one beat are written in the SAME cycle,
      // so no half of a logical beat can ever exist in the lane registers.
      if (accept) begin
        d0_q    <= in_data[LANE_W-1:0];        // teaching rule: low half
        d1_q    <= in_data[2*LANE_W-1:LANE_W]; // teaching rule: high half
        last0_q <= in_last;
        last1_q <= in_last;                    // metadata replicated, not split
        v0_q    <= 1'b1;
        v1_q    <= 1'b1;
      end
    end
  end
 
endmodule

Classification: synthesizable.

What it teaches — four things, none of which is the striping rule:

  1. Acceptance is atomic. in_ready requires both lane slots to be free or freeing. A design that accepted an aggregate beat when only one lane could take it would have to store half a beat somewhere, and that somewhere is where data gets lost.
  2. One stalled lane stalls the aggregate stream. If l1_ready stays low, in_ready goes low and ingress stops — even though lane 0 is perfectly capable of accepting more. This is a direct, unavoidable consequence of the two lanes carrying one stream, and it is the first performance surprise engineers meet at x2.
  3. Metadata is replicated, not divided. in_last describes the logical beat, so both fragments carry it. Splitting a per-beat attribute across lanes would make each lane's view of the stream incoherent.
  4. Association is established at load time. Because both fragments are written in the same cycle, they belong together by construction — independent release afterwards cannot pair the wrong fragments.

Deliberately simplified: one beat of holding per lane, so a stalled lane immediately stalls ingress; x2 operation requires both lanes available, with no reduced-width behaviour; the distribution rule is arbitrary; and there is no per-lane error path.

Production implication: a real transmit path needs per-lane skid buffering so a briefly stalled lane does not immediately stall the aggregate stream, the specified distribution rule rather than an arbitrary one, defined behaviour when a lane becomes unavailable mid-beat, and coordination with the width the Link actually negotiated.

8. RTL — Two-Lane Collection

The receive side is where skew becomes a hardware requirement rather than a concept.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SYNTHESIZABLE. Generic two-lane collector — the inverse of x2_stripe_tx.
// Accepts each lane's fragment INDEPENDENTLY, which is what allows the lanes
// to arrive at different times, then produces an aggregate beat once both
// fragments are present.
module x2_collect_rx #(
  parameter int LANE_W = 32
) (
  input  logic                clk,
  input  logic                rst_n,
 
  input  logic                l0_valid,
  output logic                l0_ready,
  input  logic [LANE_W-1:0]   l0_data,
  input  logic                l0_last,
 
  input  logic                l1_valid,
  output logic                l1_ready,
  input  logic [LANE_W-1:0]   l1_data,
  input  logic                l1_last,
 
  output logic                out_valid,
  input  logic                out_ready,
  output logic [2*LANE_W-1:0] out_data,
  output logic                out_last,
 
  // Sticky: the two fragments of a pair disagreed about beat metadata.
  output logic                meta_mismatch
);
 
  logic [LANE_W-1:0] f0_q, f1_q;
  logic              last0_q, last1_q;
  logic              v0_q,   v1_q;
  logic              mism_q;
 
  // A pair is complete when BOTH fragments are held. out_valid is derived
  // only from registers — it never observes out_ready.
  wire pair_complete = v0_q && v1_q;
  wire pair_taken    = pair_complete && out_ready;
 
  assign out_valid = pair_complete;
  assign out_data  = {f1_q, f0_q};   // inverse of the teaching rule in §7
  assign out_last  = last0_q;
 
  // Each lane accepts independently while its own slot is empty. THIS is the
  // skew tolerance: lane 0 may take its fragment many cycles before lane 1.
  // Depth is one fragment, so a lane that runs ahead is then backpressured
  // until its partner catches up and the pair is consumed.
  assign l0_ready = !v0_q || pair_taken;
  assign l1_ready = !v1_q || pair_taken;
 
  assign meta_mismatch = mism_q;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      v0_q <= 1'b0; v1_q <= 1'b0;
      f0_q <= '0;   f1_q <= '0;
      last0_q <= 1'b0; last1_q <= 1'b0;
      mism_q  <= 1'b0;
    end else begin
      // Consume the completed pair first; the per-lane loads below may refill
      // in the same cycle, giving back-to-back aggregate beats.
      if (pair_taken) begin
        v0_q <= 1'b0;
        v1_q <= 1'b0;
      end
 
      if (l0_valid && l0_ready) begin
        f0_q <= l0_data;  last0_q <= l0_last;  v0_q <= 1'b1;
      end
      if (l1_valid && l1_ready) begin
        f1_q <= l1_data;  last1_q <= l1_last;  v1_q <= 1'b1;
      end
 
      // A completed pair whose fragments disagree about the beat's metadata
      // indicates the lanes have lost alignment. Sticky: a later matching
      // pair must not erase the evidence.
      if (pair_complete && (last0_q != last1_q))
        mism_q <= 1'b1;
    end
  end
 
endmodule

Classification: synthesizable.

What it teaches:

  1. Independent per-lane acceptance is the skew tolerance. Because each lane loads into its own slot without waiting for the other, an arbitrary arrival difference is absorbed — up to the depth provided, which here is one fragment.
  2. Reconstruction waits for the complete pair. out_valid requires both fragments. A collector that emitted an aggregate beat with one fragment present would be emitting half-real data, and the half that was missing would be whatever the register held from before.
  3. Misalignment is detectable and must be sticky. meta_mismatch catches fragments that were paired but do not belong together. Without stickiness, a single misalignment event in a long run leaves no trace.
  4. Depth bounds skew tolerance. One fragment of storage means a lane running more than one fragment ahead is backpressured. Real receivers need more depth, and how much is a physical-layer question.

Deliberately simplified: one fragment of depth per lane; meta_mismatch detects disagreement but takes no recovery action; no alignment or resynchronisation mechanism; and no per-lane error reporting.

Production implication: a real collector needs depth sized to the skew that must actually be tolerated, a defined recovery path when alignment is lost rather than a status bit, an alignment mechanism to establish the pairing in the first place — which is training and aggregation territory, Modules 17.3 and Chapter 6.6 — and per-lane error attribution.

9. Active-Lane State

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SYNTHESIZABLE fragment. Abstract lane availability, not a PCIe encoding.
logic [1:0] lane_active;
 
// This simplified model treats x2 as available only when both lanes are.
wire x2_available = (lane_active == 2'b11);

Classification: synthesizable fragment.

What it teaches: that a multi-lane Link has a per-lane availability vector and a Link-level conclusion derived from it, and that the two are not the same thing. At x1 they were — one lane's status was the Link's status. At x2 a reduction function appears, and every wider Link has one.

Deliberately simplified — and this is a significant simplification worth naming. This model recognises exactly two outcomes: both lanes available, or not operating at x2. What a real Link does when fewer lanes are usable — whether it can operate at a reduced width, how that is established, and what happens to traffic in flight — is negotiated behaviour. Link training and width negotiation are Module 17.3's subject, and degraded-width operation depends on mechanisms this chapter has deliberately not introduced.

Stating the simplification explicitly matters here, because the natural reading of lane_active == 2'b11 is that any lane loss kills the Link. That is not a claim this chapter makes.

10. Assertions

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SVA over x2_stripe_tx and x2_collect_rx. Implementation invariants for
// THESE designs — not PCIe protocol requirements.
 
// ── Distributor ──────────────────────────────────────────────────────────
 
// COHERENCE — P1: the two fragments of a logical beat load together. No half
// of a beat can ever exist in the lane registers.
property p_atomic_fragment_load;
  @(posedge clk) disable iff (!rst_n)
  accept |=> (v0_q && v1_q);
endproperty
a_atomic_load : assert property (p_atomic_fragment_load);
 
// COHERENCE — P2: a lane valid never rises without an accepted aggregate beat.
// Catches a lane fabricating a fragment its partner knows nothing about.
property p_no_lone_fragment;
  @(posedge clk) disable iff (!rst_n)
  ((!v0_q ##1 v0_q) or (!v1_q ##1 v1_q)) |-> $past(accept);
endproperty
a_no_lone_fragment : assert property (p_no_lone_fragment);
 
// COHERENCE — P3: paired fragments agree on beat metadata. Replicated, not
// split — a mismatch means the pair does not describe one logical beat.
property p_metadata_agrees;
  @(posedge clk) disable iff (!rst_n)
  (v0_q && v1_q) |-> (last0_q == last1_q);
endproperty
a_metadata_agrees : assert property (p_metadata_agrees);
 
// STABILITY — P4: a fragment held for a stalled lane is stable. Each lane
// independently owes the handshake contract to its own sink.
property p_lane0_stable_under_stall;
  @(posedge clk) disable iff (!rst_n)
  (l0_valid && !l0_ready) |=> (l0_valid && $stable(l0_data) && $stable(l0_last));
endproperty
a_l0_stable : assert property (p_lane0_stable_under_stall);
 
property p_lane1_stable_under_stall;
  @(posedge clk) disable iff (!rst_n)
  (l1_valid && !l1_ready) |=> (l1_valid && $stable(l1_data) && $stable(l1_last));
endproperty
a_l1_stable : assert property (p_lane1_stable_under_stall);
 
// SAFETY — P5: no aggregate beat is accepted unless x2 is available under
// this model's contract. Accepting here takes ownership with no delivery path.
property p_accept_requires_both_lanes;
  @(posedge clk) disable iff (!rst_n)
  accept |-> (lane_active == 2'b11);
endproperty
a_accept_needs_x2 : assert property (p_accept_requires_both_lanes);
 
// ── Collector ────────────────────────────────────────────────────────────
 
// COHERENCE — P6: no aggregate beat is produced from one lane's fragment.
// The single most important invariant on the receive side: emitting with one
// fragment present publishes whatever the other register happened to hold.
property p_no_half_beat_output;
  @(posedge clk) disable iff (!rst_n)
  out_valid |-> (v0_q && v1_q);
endproperty
a_no_half_beat : assert property (p_no_half_beat_output);
 
// SAFETY — P7: a held fragment is immutable until its pair is consumed.
// Catches a lane overwriting a waiting fragment with the NEXT one, which
// silently mixes two logical beats.
property p_held_fragment_immutable;
  @(posedge clk) disable iff (!rst_n)
  (v0_q && !pair_taken) |=> ($stable(f0_q) && $stable(last0_q));
endproperty
a_frag_immutable : assert property (p_held_fragment_immutable);
 
// STABILITY — P8: the reconstructed beat is stable while the sink stalls.
property p_aggregate_stable_under_stall;
  @(posedge clk) disable iff (!rst_n)
  (out_valid && !out_ready) |=> (out_valid && $stable(out_data) && $stable(out_last));
endproperty
a_agg_stable : assert property (p_aggregate_stable_under_stall);
 
// SAFETY — P9: a detected misalignment is sticky. One bad pair in a long run
// must remain visible after the run recovers.
property p_mismatch_sticky;
  @(posedge clk) disable iff (!rst_n)
  meta_mismatch |=> meta_mismatch;
endproperty
a_mismatch_sticky : assert property (p_mismatch_sticky);

P1, P2, and P6 are the multi-lane properties. Nothing resembling them exists at x1, because at x1 there is no such thing as half of a beat. They are the assertions that encode "two lanes, one Link" as a checkable invariant.

P6 is the highest-value property in the chapter. A collector that emits with one fragment present produces an aggregate beat in which half the bits are stale. There is no error anywhere, no lane reports a problem, and the data is wrong. Without this assertion the failure is found in system integration, if at all.

P7 is its transmit-side twin. A lane overwriting a waiting fragment mixes bits of two different logical beats into one reconstructed beat — again with no error indication.

11. Verification-Only Skew Model

RTL simulation cannot model the physical causes of skew. It can model the consequence — fragments of one logical beat arriving on different lanes at different times — and that consequence is exactly what the collector must survive.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// VERIFICATION-ONLY. NOT synthesizable, NOT PCIe protocol state.
//
// This does NOT model PCIe electrical skew, its limits, or its causes. It is
// a digital delay abstraction that reproduces the OBSERVABLE consequence:
// fragments of one logical beat becoming available to the collector at
// different times. Its unit is simulation cycles, not physical time.
class lane_delay_model #(int LANE_W = 32);
 
  typedef struct {
    bit [LANE_W-1:0] data;
    bit              last;
    int unsigned     item_id;   // TESTBENCH correlation only — see §12
  } frag_t;
 
  protected frag_t     q[$];
  protected int        delay_cycles;
 
  function new(int delay_cycles = 0);
    this.delay_cycles = delay_cycles;
  endfunction
 
  // Called once per simulation cycle with the fragment offered this cycle
  // (or a null-valid indication). Returns the fragment that should become
  // visible to the collector this cycle, delayed by `delay_cycles`.
  function bit push_pop(input bit in_valid, input frag_t in_frag,
                        output frag_t out_frag);
    if (in_valid) q.push_back(in_frag);
    if (q.size() > delay_cycles) begin
      out_frag = q.pop_front();
      return 1'b1;
    end
    return 1'b0;
  endfunction
 
endclass
 
// Instantiate one per lane with DIFFERENT delays to create relative skew.
// A test that gives both lanes the same delay exercises no skew at all and
// will pass against a collector with no skew tolerance whatsoever.

Classification: verification-only.

What it teaches: that multi-lane reconstruction must be verified against unequal lane timing, and that the interesting parameter is the difference between lane delays rather than either delay on its own.

Deliberately simplified: a fixed integer cycle delay per lane, with no variation over time, no jitter, and no relationship to any physical quantity.

Production implication: a real environment would sweep the relative delay across the range the design claims to tolerate, vary it within a run, verify behaviour at and beyond the claimed limit, and — critically — establish the actual tolerable skew by physical characterisation rather than by simulation. This model verifies the digital response to skew, never the skew budget itself.

12. Scoreboard

Reconstruction correctness cannot be checked by looking at either lane alone. It requires an independent model that knows what each logical beat should become.

The correlation ID. The environment tags each logical beat entering the transmitter with a monotonically increasing item_id. This is testbench state only — it is not transmitted, not part of any protocol, and has no hardware counterpart. Its sole purpose is to let the checker say which beat a fragment belonged to, which the design itself never needs to know.

What the scoreboard tracks, per logical beat:

  • the aggregate beat as it was offered at ingress, with its item_id;
  • the fragment expected on lane 0, derived from the scoreboard's own model of the distribution rule;
  • the fragment expected on lane 1, likewise;
  • the reconstructed aggregate beat observed at the collector's output.

What it verifies:

  • No fragment loss. Every fragment expected on each lane is observed on that lane.
  • No fragment swap. The fragment observed on lane 0 is the one expected on lane 0 — not lane 1's. A distributor and collector that both got the rule wrong in the same way would produce correct end-to-end data; one that got it wrong asymmetrically produces reconstructed beats with the halves exchanged.
  • No cross-item mixing. Every reconstructed beat is assembled from two fragments carrying the same item_id. This is the check that catches P7's failure mode from the outside.
  • Correct order. Reconstructed beats emerge in the order they were offered.
  • Correct reconstruction under skew. All of the above hold across the full range of relative lane delays under test.

13. Verification Scenarios

Both lanes healthy. Continuous aggregate traffic, both lane sinks always ready. Verify one reconstructed beat per offered beat, in order, at full rate.

One lane stalls. Hold l1_ready low for varying durations. Verify: ingress stalls (a direct consequence of §7's atomic acceptance); lane 0's held fragment is stable (P4); nothing is lost when flow resumes; and — the performance lesson — that aggregate throughput is limited by the slower lane, not the average of the two.

Unequal lane delay. The core skew scenario. Sweep relative delay from 0 up to and beyond the collector's one-fragment depth. Verify correct reconstruction throughout, and verify that beyond the depth the design backpressures rather than mispairing.

Lane-local corruption. Inject a corrupted fragment on one lane only. Verify the corruption appears in the correct half of the reconstructed beat and only there — a design that spreads a single-lane error across both halves has a reconstruction bug independent of the injected fault.

Metadata disagreement. Force last to differ between paired fragments. Verify meta_mismatch sets and remains set (P9). This is the alignment-loss signature.

Reset mid-beat. Reset with one lane's fragment held and the other not yet arrived. Verify the partial pair is discarded cleanly and that the first beat after reset is correctly formed — a collector retaining a stale v0_q will pair a pre-reset fragment with a post-reset one, producing a plausible-looking corrupt beat.

Concurrent transmit and receive. Independent aggregate traffic in both directions at once, on the basis established in Chapter 6.1: verify independently first, then concurrently.

Coverage should include: relative lane delay across its range, including zero; each lane being the leading lane; stall lengths on each lane including one cycle; in_last at each position; reset with each combination of v0_q/v1_q; and the cross of relative delay against sink backpressure.

14. Debugging — The x1 Versus x2 Comparison

This is the most valuable diagnostic technique in Module 6, and it is available precisely because x1 and x2 differ in a controlled way.

Symptom: x1 works, x2 fails, same devices, same generation, same workload.

That comparison changes one variable. What it implicates is everything x2 has that x1 does not:

  • Cross-lane coordination — distribution, reconstruction, and the pairing decision.
  • The additional lane itself — lane 1's datapath, its physical path, its status.
  • Skew handling — insufficient depth, or a pairing decision that assumes simultaneity.
  • Lane-status reduction — the logic combining per-lane availability into a Link-level conclusion.

What it argues against: transaction semantics. A malformed request is malformed at any width. The transaction layer is identical in both cases, so a fault appearing only at x2 is unlikely to originate there.

Symptom: only one lane reports errors.

The asymmetry is the information, exactly as direction asymmetry was at x1. Whatever is shared between the lanes — the distributor, the collector, the aggregate datapath, the Link's configuration — would be expected to affect both. One lane reporting errors while the other does not points at what is specific to that lane: its datapath, its physical path, its own PHY resources.

This is where per-lane error attribution earns its cost. A design reporting one aggregated "link error" bit cannot distinguish this case from the next one, and they lead to entirely different investigations.

Symptom: each lane passes individually, but x2 fails.

This is the strongest coordination signature there is.

If lane 0 carries traffic correctly on its own, and lane 1 carries traffic correctly on its own, then both physical paths work, both lane datapaths work, and both PHY resources work. Every lane-local hypothesis has been tested and passed.

What remains untested by those two experiments is precisely the logic that only runs when both lanes carry one stream: distribution, reconstruction, pairing, skew handling, and status reduction. Suspect coordination, not signalling.

Concretely, in rough order of likelihood: a collector emitting before both fragments are present (P6); a fragment overwritten while waiting for its partner (P7); insufficient skew depth for the actual arrival difference; a distribution rule and reconstruction rule that do not match; or metadata split across lanes rather than replicated.

Symptom: x2 works but delivers little more than x1.

Not necessarily a fault. Work through it in order: is the Link the limiting resource at all, or is it substantially idle (Chapter 5.1)? Can the source generate the demand and the destination absorb it? Is another segment of the path narrower? Is one lane persistently backpressuring, which under §7's atomic acceptance caps the aggregate at the slower lane's rate?

Only after those are excluded is a coordination defect the likely explanation.

15. Common Misconceptions

  • "x2 is x1 with twice the clock." Width and rate are independent dimensions (Chapter 6.1). x2 adds a second lane; the per-lane signalling rate is the generation's business and is unchanged.
  • "x2 means two independent PCIe Links." It is one Link, one connection, one logical traffic stream, carried across two lanes. Two x1 Links would be independently established, independently configured, and independently able to work — an x2 Link's lanes must behave coherently.
  • "Each lane carries unrelated transactions." One logical stream is distributed across both lanes. A given transaction's information can occupy both, which is why reconstruction exists at all.
  • "x2 always gives exactly 2× application throughput." It gives twice the physical lane count. Delivered throughput approaches a corresponding increase only when the Link was the limiting resource and the rest of the path can sustain the demand — and under this chapter's model, a persistently slow lane caps the aggregate.
  • "If both lanes work individually, x2 must work." This is the misconception the chapter most wants to kill. Individual lane tests exercise no distribution, no reconstruction, no pairing, and no skew handling. Both lanes passing alone is entirely consistent with x2 failing, and it is the signature that points hardest at coordination.
  • "Multi-lane Links need no extra coordination." Distribution, reconstruction, relative arrival, and status reduction are all new at x2 and all absent at x1. They are new hardware and new failure modes.
  • "Lane skew is the same as clock skew." Different phenomena at different levels. Lane skew is relative arrival between physically separate paths of one Link; clock skew is edge arrival within a design.
  • "One broken lane makes PCIe impossible." This chapter's simplified model requires both lanes for x2 operation, and that is a property of this teaching model, not a claim about PCIe. What a real Link does with fewer usable lanes involves negotiated behaviour that Module 17.3 covers.
  • "The striping in this chapter's RTL is how PCIe does it." The low-half/high-half rule is an arbitrary placeholder chosen to make coordination visible. Chapter 6.6 owns the actual mechanism.

16. Understanding Check

17. What's Next

x2 established the complete set of coordination problems: distribution, reconstruction, relative arrival, and status reduction. Wider Links do not add new categories — they add scale and pressure.

Chapter 6.3 (x4), 6.4 (x8), and 6.5 (x16) take these ideas through the widths used in real systems, where the questions become which functions justify which width, what wider Links cost in resources and routing, and how the coordination pressure grows.

Chapter 6.6 — Lane Aggregation is where the placeholder in this chapter's RTL is replaced by the real mechanism: how PCIe actually distributes information across lanes, how reconstruction is specified, and how lanes are coordinated to agree on it.

Chapter 6.7 — Throughput Calculations finally combines Module 5's per-lane arithmetic with Module 6's width, producing the rate × width × encoding figures that both modules have deliberately withheld.

Chapter 6.8 — Real System Trade-offs closes the module with the question every design eventually faces: given what width costs in pins, power, board area, and system resources, how wide should this Link actually be?