Skip to content
VLSI Mentor

Ethernet · Module 3

Link Establishment

A link climbs a ladder of six gates, each attemptable only once the one below it has succeeded. Every gate is a search whose duration depends on the evidence available to it, which is why bring-up time is a distribution and why the only useful question is which gate it stalled at.

Every chapter in Module 3 built a piece of what happens when a link comes up, and each one stopped at its own boundary.

Chapter 3.2 gave signal detect. Chapter 2.6 gave clock recovery and block synchronisation. Chapter 3.4 built the aligner that searches for a block boundary, and deferred lane deskew. Chapter 3.5 explained where the aligner's evidence comes from and — this is the part that matters here — quantified how strong that evidence is. Chapter 3.7 added a codeword structure that must itself be located.

Assembling them is not merely a matter of listing them in order. The interesting structure is why the order is forced, and why the duration of each stage is so different.

The order is forced because each stage can only be attempted once the one below it has succeeded. You cannot recover a clock from a signal that is not there. You cannot find a block boundary in a bit stream whose bit boundaries are unknown. You cannot deskew lanes that have not individually locked. Each stage's precondition is the previous stage's result, and the ladder has no shortcuts.

And every stage is a discovery. The transmitter knows where everything is; the receiver must find out. Chapter 2.2 named that asymmetry for frames and Chapter 3.4 for blocks — here it applies to the entire link, and it has a consequence nobody expects: bring-up time is a sum of search times, and each search takes as long as its evidence is weak.

Which gate does a stalled link stop at, why does that gate take the time it does, and what does the answer tell you to do next?

1. Scope — What This Chapter Owns

This chapter owns: the ladder of assumptions and why it cannot be reordered; alignment markers and what they are for; lane deskew including how skew is measured rather than merely absorbed; auto-negotiation at the depth an RTL engineer needs; the bring-up sequencer as a state machine with real fallback paths; and the timeline instrumentation that makes "which gate, for how long" answerable.

This chapter does not own: the mechanisms it sequences. Signal detect is Chapter 3.2's, clock recovery and elastic buffering are Chapter 2.6's, the block aligner is Chapter 3.4's, the invalid-pattern evidence is Chapter 3.5's, and FEC is Chapter 3.7's. This chapter cites them rather than restating them — its subject is the sequence, the dependencies, and the diagnosis.

It also deliberately does not encode any auto-negotiation page format. The link codeword structure is clause-specific and reproducing a partial version would look complete while being wrong. Section 8 models the negotiation state machine — propose, detect agreement, commit — which is what an integrator designs against and where the bugs live.

The debts it repays: Chapter 3.1 §7 deferred auto-negotiation. Chapter 3.1 §6, Chapter 3.2 §8 and Chapter 3.4 §10 all deferred lane deskew. Chapter 3.4 §13 built a status vector and left the sequence it summarises to here.

2. The Ladder, and Why It Cannot Be Reordered

Link bring-up proceeds through six ordered gates. Signal detect confirms energy is arriving. Clock lock recovers symbol timing from it. Block lock finds where coded blocks begin. Alignment marker lock identifies the lane and its position. Lane deskew measures and removes the delay differences between lanes. FEC codeword lock finds where codewords begin. Only then is the link usable.The bring-up ladder1Signal detectis anything arriving at all2Clock lockrecover the symbol timing3Block lockfind where blocks begin4Marker lockwhich lane, and where in it5Lane deskewmeasure and remove the lag6Codeword lockfind where codewords begin
Figure 1 — six gates, each attemptable only after the one below it has succeeded.

Take each dependency in turn, because the reason each is forced is different.

Clock lock needs signal. Chapter 2.6 established that no clock is transmitted — timing is extracted from transitions in the data. No signal, no transitions, nothing to lock to. This one is physical.

Block lock needs clock. Chapter 3.4's aligner slices a bit stream at candidate offsets, which presupposes that bit boundaries are known. Without recovered timing there are no bits, only a waveform. This one is definitional.

Marker lock needs block lock. An alignment marker is a block with a recognisable content, so recognising one requires already being able to parse blocks. This one is structural.

Deskew needs marker lock on every lane. Skew is measured by comparing when the same marker arrives on different lanes, so each lane must first be able to recognise markers. This one is a measurement dependency, and Section 7 makes it concrete.

Codeword lock needs deskew. A FEC codeword spans lanes, so its symbols are only in the right order once the lanes are aligned. This one follows from Chapter 3.7's codeword being a structure over an ordered symbol sequence.

Six gates, five dependencies, five different kinds of reason — physical, definitional, structural, measurement, and structural again. That variety is why the ladder is worth understanding rather than memorising: each dependency fails differently, and Section 15's diagnosis uses that.

3. Alignment Markers — What They Actually Solve

A multi-lane PCS distributes blocks across lanes in a fixed rotation. The receiver must undo that, and to undo it, it must know which physical lane carries which position in the rotation.

That is not knowable from the data. Every lane carries valid, legal, well-formed blocks. Nothing in a block says which lane it came from, and the lanes may arrive in any physical order — a cable with its fibres in a different sequence, a connector wired differently, a board routed for convenience.

So the transmitter periodically inserts a marker into each lane, and the marker identifies the lane. The receiver finds markers, reads the identity, and now knows the mapping from physical lane to logical position.

The markers solve a second problem at the same time, and it is the one Section 7 depends on: because the same marker is inserted into every lane at the same moment, the difference in arrival times of that marker across lanes is the skew. The marker is simultaneously an identity label and a timing reference.

On the marker's contents, this chapter is deliberately unspecific. The exact structure is clause-specific, and a partial reproduction would look authoritative while being wrong. What is common across families and what matters architecturally is that a marker is recognisable, lane-identifying, and periodic — and Section 6's RTL models exactly those three properties.

4. Auto-Negotiation — Agreeing What to Try

Everything above assumes the two ends are attempting the same thing. Auto-negotiation is how they come to agree, and it happens before the ladder rather than as part of it.

The problem it solves: a port may support several speeds, several duplex modes, and several optional features. The far end supports its own set. Trying the wrong combination fails at some gate on the ladder — often at clock lock, with a perfectly healthy channel — and the failure is indistinguishable from a hardware fault.

The shape of the solution is the same in every family that has it, even though the encodings differ completely:

  1. Advertise. Each end transmits what it can do, in a form the other can receive without having agreed on anything yet. This is the hard part: the advertisement must be readable at a common denominator both ends support unconditionally.
  2. Receive and acknowledge. Each end reads the other's advertisement and signals that it did.
  3. Resolve. Both ends independently apply the same priority rule to the intersection of the two capability sets. Because the rule is deterministic and both see the same intersection, both reach the same answer without further exchange.
  4. Commit. Both reconfigure and begin the ladder.

Step 3 is the elegant part and the one worth understanding. There is no negotiation in the bargaining sense — no proposal and counter-proposal. Both ends compute the same function of the same inputs and therefore agree by construction. That is what makes the protocol robust: there is no state in which the two ends have agreed on different answers, because agreement is not communicated, it is derived.

5. RTL 1 — The Bring-Up Sequencer

A bring-up state machine. From DOWN, detecting a signal moves to SIGNAL. From SIGNAL, recovering timing moves to CLOCKED. From CLOCKED, finding block boundaries moves to BLOCKED. From BLOCKED, recognising alignment markers moves to MARKED. From MARKED, removing lane skew moves to UP. Losing signal from any state returns to DOWN. Losing block lock returns to CLOCKED, and losing marker lock or exceeding the deskew capability returns to BLOCKED.DOWNSIGNALCLOCKEDBLOCKEDUPMARKEDsignal detectedsignal detectedtiming recoveredtiming recoveredblock lockblock lockmarkers foundmarkers foundskew removedskew removedblock lock lostblock lock lostmarker lock lostmarker lock lostsignal lostsignal lost
Figure 2 — every gate can be lost, and each loss falls back only as far as its own precondition.

Two features of this topology are load-bearing and neither is arbitrary.

Losing block lock falls back to CLOCKED, not to DOWN. Timing is still recovered — nothing about the signal changed — so restarting from the bottom would throw away work that is still valid and turn a brief search into a full outage. Each loss falls back exactly as far as its own precondition and no further, which is the same warm-restart argument Chapter 3.1 §7 made for pair training.

Losing marker lock falls back to BLOCKED, not to CLOCKED. Blocks still parse; only the lane identity is in doubt. And this is why Section 3's markers are periodic — the recovery is bounded by one marker interval rather than requiring a new block-lock search.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SYNTHESIZABLE. The bring-up sequencer.
//
// It sequences; it does not implement. Every input here is produced by a
// block an earlier chapter owns:
//
//   signal_ok      <- PMD             (Chapter 3.2)
//   clock_locked   <- PMA             (Chapter 2.6)
//   block_locked   <- PCS aligner     (Chapters 3.4, 3.5)
//   markers_locked <- Section 6
//   deskew_ok      <- Section 7
//
// The design rule this module exists to enforce: EACH LOSS FALLS BACK
// EXACTLY AS FAR AS ITS OWN PRECONDITION. A sequencer that returns to DOWN
// on any loss converts every brief search into a full outage, and upper
// layers see a link flap where the PHY saw a two-microsecond re-lock.
package bringup_pkg;
  typedef enum logic [2:0] {
    LS_DOWN    = 3'd0,   // nothing arriving
    LS_SIGNAL  = 3'd1,   // energy present, no timing
    LS_CLOCKED = 3'd2,   // timing recovered, no block boundaries
    LS_BLOCKED = 3'd3,   // blocks parse, lane identity unknown
    LS_MARKED  = 3'd4,   // lanes identified, skew not yet removed
    LS_UP      = 3'd5
  } link_state_e;
endpackage
 
module bringup_sequencer
  import bringup_pkg::*;
#(
  // Per-stage watchdogs. A stage that has not completed within its budget
  // is STALLED, which is a different report from "still progressing" and
  // must be distinguishable -- Section 11 depends on it.
  parameter int unsigned T_SIGNAL = 1_000_000,
  parameter int unsigned T_CLOCK  = 2_000_000,
  parameter int unsigned T_BLOCK  = 8_000_000,
  parameter int unsigned T_MARKER = 4_000_000,
  parameter int unsigned T_DESKEW = 1_000_000,
  parameter int unsigned TMR_W    = 24
) (
  input  logic clk,
  input  logic rst_n,
 
  input  logic negotiated,     // Section 8: both ends agreed what to try
  input  logic signal_ok,
  input  logic clock_locked,
  input  logic block_locked,
  input  logic markers_locked,
  input  logic deskew_ok,
 
  output link_state_e state,
  output logic        link_up,
 
  // How long the current stage has been running. Exposed continuously, not
  // only on failure -- a stage that usually takes 2 ms and today takes 40
  // is a finding even though it eventually succeeded.
  output logic [TMR_W-1:0] stage_elapsed,
  output logic             stage_stalled,
 
  // Pulses, for the timeline recorder of Section 9.
  output logic advanced,
  output logic regressed
);
 
  link_state_e       state_q, state_d;
  logic [TMR_W-1:0]  timer_q;
  logic [TMR_W-1:0]  budget_c;
 
  always_comb begin
    unique case (state_q)
      LS_DOWN:    budget_c = TMR_W'(T_SIGNAL);
      LS_SIGNAL:  budget_c = TMR_W'(T_CLOCK);
      LS_CLOCKED: budget_c = TMR_W'(T_BLOCK);
      LS_BLOCKED: budget_c = TMR_W'(T_MARKER);
      LS_MARKED:  budget_c = TMR_W'(T_DESKEW);
      default:    budget_c = '1;      // LS_UP has no budget
    endcase
  end
 
  always_comb begin
    state_d = state_q;
 
    // Signal loss beats everything. Evaluated FIRST so no later branch can
    // advance a stage on stale evidence after the signal has gone -- the
    // same priority rule Chapter 3.1 §7 asserted for pair training.
    if (!signal_ok && (state_q != LS_DOWN)) begin
      state_d = LS_DOWN;
    end else begin
      unique case (state_q)
        LS_DOWN:
          // Negotiation must complete before the ladder starts. Without
          // this term the ladder can begin against a mode the far end is
          // not using, which fails at clock lock with a healthy channel.
          if (signal_ok && negotiated) state_d = LS_SIGNAL;
 
        LS_SIGNAL:
          if (clock_locked) state_d = LS_CLOCKED;
 
        LS_CLOCKED:
          if (block_locked) state_d = LS_BLOCKED;
 
        LS_BLOCKED: begin
          if      (!block_locked)  state_d = LS_CLOCKED;  // fall back one rung
          else if (markers_locked) state_d = LS_MARKED;
        end
 
        LS_MARKED: begin
          if      (!block_locked)   state_d = LS_CLOCKED;
          else if (!markers_locked) state_d = LS_BLOCKED; // one rung, not two
          else if (deskew_ok)       state_d = LS_UP;
        end
 
        LS_UP: begin
          if      (!block_locked)   state_d = LS_CLOCKED;
          else if (!markers_locked) state_d = LS_BLOCKED;
          else if (!deskew_ok)      state_d = LS_MARKED;
        end
 
        default: state_d = LS_DOWN;
      endcase
    end
  end
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      state_q   <= LS_DOWN;
      timer_q   <= '0;
      advanced  <= 1'b0;
      regressed <= 1'b0;
    end else begin
      state_q <= state_d;
 
      // The timer measures time IN A STAGE, so it resets on any change.
      if (state_d != state_q) timer_q <= '0;
      else if (timer_q != '1) timer_q <= timer_q + 1'b1;
 
      advanced  <= (state_d > state_q);
      regressed <= (state_d < state_q);
    end
  end
 
  assign state         = state_q;
  assign link_up       = (state_q == LS_UP);
  assign stage_elapsed = timer_q;
  assign stage_stalled = (state_q != LS_UP) && (timer_q >= budget_c);
 
endmodule

Classification: synthesizable.

What it teaches: that fallback depth is a design decision with a large operational consequence. A sequencer returning to LS_DOWN on any loss converts a two-microsecond block re-lock into a full outage that every layer above must absorb. Falling back exactly one rung — to the state whose precondition still holds — keeps a brief search brief.

Deliberately simplified: no per-lane state. The lock inputs here are already-combined across lanes, and Section 7 shows what that combining costs in visibility. A production sequencer usually keeps per-lane block and marker lock and combines them explicitly.

Production implication: stage_stalled and stage_elapsed are separate outputs on purpose. A stage that eventually succeeded after forty times its usual duration is a finding — a marginal channel, a slow far end, a search running with less evidence than expected — and it is completely invisible to any check that only looks at final state. Expose the elapsed time continuously, not only when a watchdog fires.

And negotiated gating the first transition matters. Without it the ladder can begin against a mode the far end is not using, which fails at clock lock with a perfectly healthy channel — Section 4's case two, and one of the hardest bring-up failures to diagnose without this term making the dependency explicit.

6. RTL 2 — Alignment Marker Insertion and Detection

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SYNTHESIZABLE. Alignment marker insertion and detection.
//
// NO CLAUSE'S MARKER FORMAT IS ENCODED HERE. The three properties that
// matter architecturally are:
//
//   RECOGNISABLE    -- distinguishable from data, so a receiver can find one
//   LANE-IDENTIFYING-- carries which logical lane this physical lane is
//   PERIODIC        -- inserted on a fixed interval, forever
//
// The third is what bounds marker-lock time and what makes Section 7's
// skew measurement possible: the SAME marker goes into every lane at the
// SAME moment, so arrival-time differences ARE the skew.
package marker_pkg;
  parameter int unsigned LANES = 4;
  parameter int unsigned LANE_W = $clog2(LANES);
endpackage
 
module marker_inserter
  import marker_pkg::*;
#(
  // Blocks between markers. The worst-case marker-lock time is exactly this
  // many block times, which is a DESIGNED bound rather than a search that
  // depends on data content.
  parameter int unsigned MARKER_PERIOD = 16384,
  parameter int unsigned PER_W = $clog2(MARKER_PERIOD)
) (
  input  logic clk,
  input  logic rst_n,
 
  input  logic [LANE_W-1:0] lane_id,
 
  input  logic       blk_valid,
  input  logic [65:0] blk_in,
  output logic       blk_ready,
 
  output logic       out_valid,
  output logic [65:0] out_blk,
  output logic       out_is_marker,
 
  // Pulses when a marker is inserted. All lanes must pulse on the SAME
  // cycle -- Section 7's measurement is meaningless otherwise, and
  // Section 13 asserts it.
  output logic       marker_inserted
);
 
  logic [PER_W-1:0] count_q;
  logic             insert_c;
 
  assign insert_c  = (count_q == PER_W'(MARKER_PERIOD - 1));
  // A marker displaces a data block rather than being squeezed between
  // them, so the upstream must be held for exactly one cycle.
  assign blk_ready = !insert_c;
 
  // The marker carries the lane identity and a recognisable body. The body
  // pattern here is illustrative; what matters is that it is distinguishable
  // from data and that the identity field exists.
  function automatic logic [65:0] make_marker(input logic [LANE_W-1:0] id);
    make_marker = {2'b10, 8'hA5, 46'h2A2A_2A2A_2A2A, {(10-LANE_W){1'b0}}, id};
  endfunction
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      count_q         <= '0;
      out_valid       <= 1'b0;
      out_blk         <= '0;
      out_is_marker   <= 1'b0;
      marker_inserted <= 1'b0;
    end else begin
      out_valid       <= 1'b1;      // the PCS never goes silent
      marker_inserted <= insert_c;
 
      if (insert_c) begin
        out_blk       <= make_marker(lane_id);
        out_is_marker <= 1'b1;
        count_q       <= '0;
      end else begin
        out_blk       <= blk_valid ? blk_in : 66'0;
        out_is_marker <= 1'b0;
        count_q       <= count_q + 1'b1;
      end
    end
  end
 
endmodule
 
 
// SYNTHESIZABLE. Detection, on the receive side of one lane.
//
// Marker lock is UNLIKE every other gate on the ladder: its duration is
// bounded by when the transmitter next sends a marker, not by how quickly
// evidence accumulates. So a marker lock taking much longer than one
// marker period means markers are NOT ARRIVING -- a different fault from a
// slow search, and one this module makes distinguishable.
module marker_detector
  import marker_pkg::*;
#(
  parameter int unsigned MARKER_PERIOD = 16384,
  // Consecutive markers at the expected spacing before lock is declared.
  // More than one, because a data block can coincidentally resemble a
  // marker; the SPACING is the strong evidence, not the pattern alone.
  parameter int unsigned LOCK_COUNT = 4,
  parameter int unsigned CNT_W = 24
) (
  input  logic clk,
  input  logic rst_n,
 
  input  logic        blk_valid,
  input  logic [65:0] blk_in,
 
  output logic             marker_seen,
  output logic [LANE_W-1:0] detected_lane_id,
  output logic             marker_locked,
 
  // Cycles since the last marker. If this exceeds the period by a margin,
  // markers have stopped -- which is a transmitter or upstream fault, not
  // a receive search problem.
  output logic [CNT_W-1:0] since_last_marker,
  output logic             markers_missing,
 
  output logic [CNT_W-1:0] c_markers_seen,
  output logic [CNT_W-1:0] c_markers_missed
);
 
  logic [CNT_W-1:0]        gap_q;
  logic [$clog2(LOCK_COUNT+1)-1:0] good_q;
  logic                    is_marker_c;
  logic                    spacing_ok_c;
 
  // Recognition: the control sync header plus the marker body pattern.
  assign is_marker_c = blk_valid
                    && (blk_in[65:64] == 2'b10)
                    && (blk_in[63:56] == 8'hA5);
 
  // The spacing check is the real evidence. A lone matching block proves
  // little; one arriving exactly a period after the last is convincing.
  assign spacing_ok_c = (gap_q >= CNT_W'(MARKER_PERIOD - 2))
                     && (gap_q <= CNT_W'(MARKER_PERIOD + 2));
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      gap_q            <= '0;
      good_q           <= '0;
      marker_seen      <= 1'b0;
      marker_locked    <= 1'b0;
      detected_lane_id <= '0;
      c_markers_seen   <= '0;
      c_markers_missed <= '0;
    end else begin
      marker_seen <= is_marker_c;
 
      if (is_marker_c) begin
        detected_lane_id <= blk_in[LANE_W-1:0];
        gap_q            <= '0;
        if (!(&c_markers_seen)) c_markers_seen <= c_markers_seen + 1'b1;
 
        if (spacing_ok_c) begin
          if (good_q < ($clog2(LOCK_COUNT+1))'(LOCK_COUNT)) good_q <= good_q + 1'b1;
          if (good_q + 1'b1 >= ($clog2(LOCK_COUNT+1))'(LOCK_COUNT)) marker_locked <= 1'b1;
        end else begin
          // Right pattern, wrong spacing: a coincidental match. Restart the
          // evidence rather than counting it.
          good_q <= '0;
        end
      end else begin
        if (blk_valid && (gap_q != '1)) gap_q <= gap_q + 1'b1;
 
        // A marker period passed with no marker. Lock is lost, and the
        // cause is upstream rather than here.
        if (gap_q > CNT_W'(MARKER_PERIOD + (MARKER_PERIOD >> 2))) begin
          marker_locked <= 1'b0;
          good_q        <= '0;
          if (!(&c_markers_missed)) c_markers_missed <= c_markers_missed + 1'b1;
        end
      end
    end
  end
 
  assign since_last_marker = gap_q;
  assign markers_missing   = (gap_q > CNT_W'(MARKER_PERIOD + (MARKER_PERIOD >> 2)));
 
endmodule

Classification: synthesizable, with the marker body illustrative and the structure exact.

What it teaches: that the spacing is the evidence, not the pattern. A data block can coincidentally match a marker's body — the body is only so many bits — but a block matching and arriving exactly one period after the last one is convincing. That is the same evidence-accumulation reasoning Chapter 3.4 §11 used for block lock, applied to a different quantity, and it is why LOCK_COUNT is greater than one.

Deliberately simplified: the marker body. No clause's format is reproduced, because a partial reproduction would be worse than an obvious placeholder.

Production implication: markers_missing distinguishes two faults that look identical from the outside. Markers arriving but not locking means a receive-side search problem — spacing tolerance, or a lane identity field being misread. Markers not arriving at all means the far end is not inserting them, or block lock on this lane is failing so they cannot be recognised. Same symptom, opposite ends of the link, and only this output separates them.

Later ownership: the marker's normative structure belongs to the clause.

7. RTL 3 — Lane Deskew, and Measuring the Skew

Three chapters deferred this. Here it is, and the important part is that the buffer measures skew rather than merely absorbing it.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SYNTHESIZABLE. Lane deskew with measurement.
//
// HOW SKEW IS MEASURED: the transmitter inserts the SAME marker into every
// lane at the SAME moment. Any difference in arrival time at the receiver
// is skew accumulated along the physical path -- different fibre lengths,
// different trace lengths, different retimer delays.
//
// So the measurement is: note when each lane's marker arrives, take the
// earliest as the reference, and each lane's lag is its delay.
//
// WHY MEASURING MATTERS MORE THAN ABSORBING: skew drifts with temperature.
// A design that absorbs skew silently works until the day the drift exceeds
// the buffer, and then fails with no warning. A design that REPORTS the
// margin gives months of it.
module lane_deskew
  import marker_pkg::*;
#(
  parameter int unsigned DEPTH  = 64,          // blocks of skew absorbable
  parameter int unsigned PTR_W  = $clog2(DEPTH),
  parameter int unsigned BLK_W  = 66
) (
  input  logic clk,
  input  logic rst_n,
 
  input  logic [LANES-1:0]  lane_valid,
  input  logic [BLK_W-1:0]  lane_blk    [LANES],
  input  logic [LANES-1:0]  lane_marker,     // this lane's marker arrived now
  input  logic [LANES-1:0]  lane_locked,     // this lane has marker lock
 
  output logic              aligned_valid,
  output logic [BLK_W-1:0]  aligned_blk [LANES],
 
  output logic              deskew_ok,
 
  // Measured skew per lane, in blocks, relative to the earliest lane.
  output logic [PTR_W:0]    lane_skew [LANES],
  output logic [PTR_W:0]    worst_skew,
  output logic [LANE_W-1:0] worst_lane,
 
  // How much of the buffer is unused at the current worst skew. This is the
  // number to trend: it shrinks as the system warms, and it reaches zero
  // before anything fails.
  output logic [PTR_W:0]    skew_margin,
 
  // Skew exceeded what the buffer can absorb. Distinct from "not yet
  // aligned" -- this one cannot be fixed by waiting.
  output logic              skew_overflow,
  output logic [15:0]       c_skew_overflow
);
 
  // Per-lane elastic storage. Each lane writes on arrival and all lanes are
  // read together once every lane's marker has been seen.
  logic [BLK_W-1:0] fifo [LANES][DEPTH];
  logic [PTR_W-1:0] wptr [LANES];
  logic [PTR_W-1:0] rptr;
 
  // Marker arrival capture. The first lane to present its marker sets the
  // reference; every other lane's lag is counted from there.
  logic [LANES-1:0] marker_seen_q;
  logic [PTR_W:0]   marker_at   [LANES];
  logic [PTR_W:0]   ref_time_q;
  logic             measuring_q;
  logic [PTR_W:0]   tick_q;
 
  logic all_markers_c;
  assign all_markers_c = &marker_seen_q;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      for (int unsigned l = 0; l < LANES; l++) begin
        wptr[l]       <= '0;
        marker_at[l]  <= '0;
        lane_skew[l]  <= '0;
      end
      rptr            <= '0;
      marker_seen_q   <= '0;
      measuring_q     <= 1'b0;
      tick_q          <= '0;
      ref_time_q      <= '0;
      deskew_ok       <= 1'b0;
      skew_overflow   <= 1'b0;
      c_skew_overflow <= '0;
      aligned_valid   <= 1'b0;
    end else begin
      // Write side: each lane stores independently, as it arrives.
      for (int unsigned l = 0; l < LANES; l++) begin
        if (lane_valid[l]) begin
          fifo[l][wptr[l]] <= lane_blk[l];
          wptr[l]          <= wptr[l] + 1'b1;
        end
      end
 
      // Measurement window: opens on the first marker of a round.
      if (|lane_marker && !measuring_q) begin
        measuring_q   <= 1'b1;
        tick_q        <= '0;
        ref_time_q    <= '0;
        marker_seen_q <= lane_marker;
        for (int unsigned l = 0; l < LANES; l++)
          if (lane_marker[l]) marker_at[l] <= '0;
      end else if (measuring_q) begin
        tick_q <= tick_q + 1'b1;
 
        for (int unsigned l = 0; l < LANES; l++) begin
          if (lane_marker[l] && !marker_seen_q[l]) begin
            marker_seen_q[l] <= 1'b1;
            marker_at[l]     <= tick_q;
            lane_skew[l]     <= tick_q;      // lag behind the earliest lane
          end
        end
 
        // Every lane reported: the round is complete.
        if (all_markers_c) begin
          measuring_q   <= 1'b0;
          marker_seen_q <= '0;
          deskew_ok     <= (&lane_locked) && (tick_q < (PTR_W+1)'(DEPTH));
          if (tick_q >= (PTR_W+1)'(DEPTH)) begin
            skew_overflow <= 1'b1;
            if (!(&c_skew_overflow)) c_skew_overflow <= c_skew_overflow + 1'b1;
          end else begin
            skew_overflow <= 1'b0;
          end
        end else if (tick_q >= (PTR_W+1)'(DEPTH)) begin
          // A lane's marker has not arrived within the buffer's reach. It
          // cannot be aligned no matter how long we wait.
          measuring_q   <= 1'b0;
          marker_seen_q <= '0;
          deskew_ok     <= 1'b0;
          skew_overflow <= 1'b1;
          if (!(&c_skew_overflow)) c_skew_overflow <= c_skew_overflow + 1'b1;
        end
      end
 
      // Read side: all lanes advance together once aligned.
      aligned_valid <= deskew_ok;
      if (deskew_ok) rptr <= rptr + 1'b1;
    end
  end
 
  always_comb begin
    for (int unsigned l = 0; l < LANES; l++)
      aligned_blk[l] = fifo[l][(rptr - PTR_W'(lane_skew[l]))];
  end
 
  // Worst lane and the remaining headroom.
  logic [PTR_W:0]    max_c;
  logic [LANE_W-1:0] maxl_c;
  always_comb begin
    max_c  = lane_skew[0];
    maxl_c = '0;
    for (int unsigned l = 1; l < LANES; l++) begin
      if (lane_skew[l] > max_c) begin
        max_c  = lane_skew[l];
        maxl_c = LANE_W'(l);
      end
    end
    worst_skew  = max_c;
    worst_lane  = maxl_c;
    skew_margin = (max_c < (PTR_W+1)'(DEPTH)) ? ((PTR_W+1)'(DEPTH) - max_c)
                                              : '0;
  end
 
endmodule

Classification: synthesizable.

What it teaches: that skew is measurable, and the measurement is free because the markers of Section 6 are already being transmitted for a different reason. One mechanism, two purposes — lane identity and a timing reference — which is why periodic markers earn their bandwidth.

Deliberately simplified: a simple indexed buffer rather than a per-lane elastic structure with independent read and write clocks. Production designs need the clock-domain handling of Chapter 2.6's elastic buffer here too.

Production implication: skew_margin is the number to trend, and it is the module's reason for existing. Skew drifts with temperature — different fibre lengths and different trace lengths expand differently, and retimer delays vary. A design that absorbs skew silently works perfectly until the drift exceeds DEPTH, then fails abruptly with no warning. A design that reports the margin shows it shrinking across a warm-up cycle and across seasons, giving months of notice. This is Chapter 3.3's slope-against-cliff argument at a third layer.

And skew_overflow is distinct from "not yet aligned" for a reason. Not-yet-aligned resolves by waiting. Overflow does not — the skew exceeds what the hardware can absorb, and waiting forever will not fix it. Merging them produces a link that appears to be still trying when it has actually failed.

Later ownership: the normative maximum skew a clause permits is clause-specific, and DEPTH must be chosen against it.

8. RTL 4 — Auto-Negotiation State

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SYNTHESIZABLE. Auto-negotiation as a state machine, without a page format.
//
// NO CLAUSE'S LINK CODEWORD IS ENCODED HERE. What is modelled is the shape
// common to every family that negotiates:
//
//   ADVERTISE -> ACKNOWLEDGE -> RESOLVE -> COMMIT
//
// The elegant part is RESOLVE. There is no bargaining -- both ends apply the
// SAME deterministic priority rule to the SAME capability intersection, so
// they agree by construction rather than by exchange. There is no state in
// which the two ends have committed to different answers.
module autoneg
#(
  parameter int unsigned CAP_W       = 8,     // one bit per capability
  parameter int unsigned T_ADVERTISE = 500_000,
  parameter int unsigned T_ACK       = 500_000,
  parameter int unsigned TMR_W       = 24
) (
  input  logic clk,
  input  logic rst_n,
 
  input  logic             an_enable,      // some ports are forced, not negotiated
  input  logic [CAP_W-1:0] local_caps,
 
  // From the far end. Not modelled: how these are carried on the wire.
  input  logic             remote_valid,
  input  logic [CAP_W-1:0] remote_caps,
  input  logic             remote_ack,
 
  output logic             advertising,
  output logic             local_ack,
 
  output logic             negotiated,
  output logic [CAP_W-1:0] selected_cap,   // one-hot: the agreed mode
 
  // Why negotiation did not complete. Three causes, three owners -- and the
  // second is the one that produces the hardest field failures.
  output logic             no_remote,          // far end never advertised
  output logic             no_common_cap,      // both advertised, nothing shared
  output logic             an_timeout,
 
  // Sticky: negotiation completed without the far end participating, so a
  // default was assumed. Section 4's case one, made visible -- this is the
  // duplex-mismatch signature.
  output logic             defaulted
);
 
  typedef enum logic [2:0] {
    AN_IDLE, AN_ADVERTISE, AN_ACK, AN_RESOLVE, AN_DONE, AN_FAIL
  } an_state_e;
 
  an_state_e        state_q, state_d;
  logic [TMR_W-1:0] timer_q;
  logic [CAP_W-1:0] common_c;
 
  assign common_c = local_caps & remote_caps;
 
  // The deterministic priority rule. Highest set bit wins, and BOTH ENDS
  // COMPUTE THE SAME FUNCTION -- which is what makes agreement automatic.
  // The rule itself must be identical at both ends; a design whose priority
  // differs from its peer's produces two ends committed to different modes,
  // each believing negotiation succeeded.
  function automatic logic [CAP_W-1:0] resolve(input logic [CAP_W-1:0] c);
    resolve = '0;
    for (int unsigned i = 0; i < CAP_W; i++)
      if (c[i]) resolve = (CAP_W)'(1) << i;   // highest set bit survives
  endfunction
 
  always_comb begin
    state_d = state_q;
    unique case (state_q)
      AN_IDLE:      if (an_enable) state_d = AN_ADVERTISE;
 
      AN_ADVERTISE: begin
        if      (remote_valid)                       state_d = AN_ACK;
        else if (timer_q >= TMR_W'(T_ADVERTISE))     state_d = AN_FAIL;
      end
 
      AN_ACK: begin
        if      (remote_ack)                   state_d = AN_RESOLVE;
        else if (timer_q >= TMR_W'(T_ACK))     state_d = AN_FAIL;
      end
 
      AN_RESOLVE:   state_d = (common_c != '0) ? AN_DONE : AN_FAIL;
 
      AN_DONE:      if (!an_enable || !remote_valid) state_d = AN_IDLE;
 
      AN_FAIL:      if (!an_enable)                  state_d = AN_IDLE;
 
      default:      state_d = AN_IDLE;
    endcase
  end
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      state_q      <= AN_IDLE;
      timer_q      <= '0;
      selected_cap <= '0;
      defaulted    <= 1'b0;
    end else begin
      state_q <= state_d;
      if (state_d != state_q) timer_q <= '0;
      else if (timer_q != '1) timer_q <= timer_q + 1'b1;
 
      if (state_d == AN_DONE) selected_cap <= resolve(common_c);
 
      // Negotiation was enabled, the far end never spoke, and we proceeded
      // on a default. This is the duplex-mismatch signature, and it is
      // sticky because by the time anyone looks the link may be "up".
      if ((state_q == AN_ADVERTISE) && (state_d == AN_FAIL) && !remote_valid)
        defaulted <= 1'b1;
    end
  end
 
  assign advertising   = (state_q == AN_ADVERTISE);
  assign local_ack     = (state_q == AN_ACK) || (state_q == AN_RESOLVE);
  assign negotiated    = (state_q == AN_DONE) || !an_enable;
  assign no_remote     = (state_q == AN_FAIL) && !remote_valid;
  assign no_common_cap = (state_q == AN_FAIL) && remote_valid && (common_c == '0);
  assign an_timeout    = (state_q == AN_FAIL) && (timer_q >= TMR_W'(T_ACK));
 
endmodule

Classification: synthesizable, with the wire encoding deliberately absent.

What it teaches: that resolution is computed, not exchanged. Both ends apply the same deterministic priority rule to the same capability intersection, so they agree by construction — and there is no state in which they have committed to different answers. That is what makes the protocol robust, and it is why the priority rule must be identical at both ends rather than merely compatible.

Deliberately simplified: everything on the wire. How capabilities are carried, how many pages there are, and how extensions are signalled are all clause-specific.

Production implication: defaulted is sticky and it is the single most valuable output in the module. Section 4's case one — one end fixed, one end negotiating — produces a link that comes up, works at low traffic, and collapses under load. By the time anyone investigates, the link reports up, and the only surviving evidence that a default was assumed rather than negotiated is this bit. Without it the diagnosis requires access to both ends' configuration, which is often not available.

And negotiated is asserted when an_enable is low, so a deliberately forced port does not stall Section 5's sequencer forever. That is a small line with a large consequence — a forced port that never asserts negotiated never leaves LS_DOWN, and the symptom is a link that will not come up with no fault reported anywhere.

9. RTL 5 — The Bring-Up Timeline

This is the module that makes the chapter's central diagnostic question answerable.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SYNTHESIZABLE INSTRUMENTATION.
//
// Current state answers "is the link up". It cannot answer:
//
//   - which gate did it stall at?
//   - how long did each gate take?
//   - did it get further last time than this time?
//   - is bring-up getting slower over weeks?
//
// Those are the questions with owners, and all four need a TIMELINE.
module bringup_timeline
  import bringup_pkg::*;
#(
  parameter int unsigned TS_W   = 32,
  parameter int unsigned NSTAGE = 6
) (
  input  logic clk,
  input  logic rst_n,
  input  logic clear,
 
  input  logic        attempt_start,     // a bring-up attempt begins
  input  link_state_e state,
  input  logic        advanced,
  input  logic        regressed,
  input  logic        stage_stalled,
 
  // Time spent in each stage during the attempt that just finished.
  output logic [TS_W-1:0] stage_duration [NSTAGE],
  output logic            timeline_valid,
 
  // The furthest state reached this attempt, and ever. "Ever" matters: a
  // link that once reached UP and now stalls at BLOCKED is a regression,
  // and one that has NEVER reached UP was never configured correctly --
  // completely different investigations.
  output link_state_e furthest_this_attempt,
  output link_state_e furthest_ever,
 
  // Where it stalled, if it did.
  output link_state_e stall_state,
  output logic        stall_valid,
 
  output logic [15:0] c_attempts,
  output logic [15:0] c_successes,
  output logic [15:0] c_regressions,
 
  // Total time of the last successful bring-up. Trend it: a bring-up that
  // took 3 ms in January and 300 ms in July is a marginal link, even though
  // both succeeded and both report UP.
  output logic [TS_W-1:0] last_bringup_time
);
 
  logic [TS_W-1:0] stage_timer_q;
  logic [TS_W-1:0] total_timer_q;
  link_state_e     prev_state_q;
  logic            in_attempt_q;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      for (int unsigned i = 0; i < NSTAGE; i++) stage_duration[i] <= '0;
      stage_timer_q         <= '0;
      total_timer_q         <= '0;
      prev_state_q          <= LS_DOWN;
      in_attempt_q          <= 1'b0;
      timeline_valid        <= 1'b0;
      furthest_this_attempt <= LS_DOWN;
      furthest_ever         <= LS_DOWN;
      stall_state           <= LS_DOWN;
      stall_valid           <= 1'b0;
      c_attempts            <= '0;
      c_successes           <= '0;
      c_regressions         <= '0;
      last_bringup_time     <= '0;
    end else begin
      if (clear) begin
        c_attempts    <= '0;
        c_successes   <= '0;
        c_regressions <= '0;
        // furthest_ever deliberately NOT cleared: it distinguishes "never
        // worked" from "worked and regressed", which no counter can.
      end
 
      if (attempt_start) begin
        for (int unsigned i = 0; i < NSTAGE; i++) stage_duration[i] <= '0;
        stage_timer_q         <= '0;
        total_timer_q         <= '0;
        in_attempt_q          <= 1'b1;
        timeline_valid        <= 1'b0;
        furthest_this_attempt <= LS_DOWN;
        stall_valid           <= 1'b0;
        if (!(&c_attempts)) c_attempts <= c_attempts + 1'b1;
      end else if (in_attempt_q) begin
        if (stage_timer_q != '1) stage_timer_q <= stage_timer_q + 1'b1;
        if (total_timer_q != '1) total_timer_q <= total_timer_q + 1'b1;
 
        // A state change closes the stage that just ended.
        if (state != prev_state_q) begin
          stage_duration[prev_state_q] <= stage_timer_q;
          stage_timer_q                <= '0;
        end
 
        if (state > furthest_this_attempt) furthest_this_attempt <= state;
        if (state > furthest_ever)         furthest_ever         <= state;
 
        if (regressed && !(&c_regressions)) c_regressions <= c_regressions + 1'b1;
 
        // Stalled: record WHERE, once, and hold it.
        if (stage_stalled && !stall_valid) begin
          stall_state <= state;
          stall_valid <= 1'b1;
        end
 
        if (state == LS_UP) begin
          in_attempt_q      <= 1'b0;
          timeline_valid    <= 1'b1;
          last_bringup_time <= total_timer_q;
          if (!(&c_successes)) c_successes <= c_successes + 1'b1;
        end
      end
 
      prev_state_q <= state;
    end
  end
 
endmodule

Classification: synthesizable instrumentation.

What it teaches: that furthest_ever and furthest_this_attempt answer different questions, and both are needed. A link stalling at LS_BLOCKED that has previously reached LS_UP is a regression — something changed, and the configuration was once right. One that has never reached LS_UP was never configured correctly, and the investigation starts from the far end's settings rather than from the channel. Same current state, opposite starting points.

Deliberately simplified: durations in raw clock cycles rather than a real-time base, and no per-lane breakdown of the stages that are per-lane.

Production implication: last_bringup_time, trended, catches a class of fault nothing else sees. A bring-up that took 3 ms in January and 300 ms in July is a marginal link — the searches are taking longer because the evidence is degrading — and both attempts succeeded and both report LS_UP. This is Chapter 3.1 §15's retrain-rate argument and Chapter 3.3's margin-trend argument arriving at a third quantity: the duration of a successful operation is itself a health metric.

And furthest_ever deliberately survives clear, for the same reason Chapter 3.1's ever_ready does: an operator clearing counters for a clean measurement window would otherwise destroy the one fact that separates "never worked" from "worked and degraded".

10. RTL 6 — Per-Lane Lock, and What Combining It Early Costs

Section 5's sequencer takes block_locked and markers_locked as single bits. On a multi-lane link those bits are already an AND across lanes, and that reduction happened somewhere. Where it happens decides whether the link is diagnosable.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SYNTHESIZABLE. Per-lane lock, combined late and preserved.
//
// The sequencer needs one bit. An engineer needs LANES bits. Both are cheap;
// only the second is ever omitted, and it is the one that answers the
// question every multi-lane investigation starts with: WHICH LANE?
//
// The asymmetry that makes this worth a module: a link is up only when
// EVERY lane is locked, so the combined bit is dominated by the WORST lane.
// One lane cycling in and out of lock makes the whole link flap, and from
// the combined bit alone that is indistinguishable from all lanes being
// marginal -- a completely different fault with a completely different fix.
module lane_lock_aggregator
  import marker_pkg::*;
#(
  parameter int unsigned CNT_W = 20
) (
  input  logic clk,
  input  logic rst_n,
  input  logic clear,
 
  input  logic [LANES-1:0] lane_block_locked,
  input  logic [LANES-1:0] lane_marker_locked,
  input  logic [LANES-1:0] lane_markers_missing,
 
  // What the sequencer of Section 5 consumes.
  output logic block_locked,
  output logic markers_locked,
 
  // What an engineer consumes. Never reduced.
  output logic [LANES-1:0] block_mask,
  output logic [LANES-1:0] marker_mask,
 
  // Lock losses per lane. One lane dominating this is a physical claim
  // about one fibre or one trace; a uniform spread points at something
  // shared -- the reference clock, the supply, the far end.
  output logic [CNT_W-1:0] c_block_loss  [LANES],
  output logic [CNT_W-1:0] c_marker_loss [LANES],
 
  // The lane that has lost lock most often. The single most useful output
  // in the module, and the one an AND gate destroys.
  output logic [LANE_W-1:0] worst_lane,
  output logic              worst_lane_valid,
 
  // Which lanes were unlocked at the moment the combined bit last fell.
  // By the time anyone reads it the lane has usually recovered, so live
  // state says nothing and this is the only surviving evidence.
  output logic [LANES-1:0] down_cause_mask,
  output logic             down_cause_valid,
 
  // Sticky per lane: has this lane EVER locked? Distinguishes a lane that
  // has never worked -- miswired, wrong fibre, dead transceiver -- from one
  // that works and drops. Different faults, identical combined bit.
  output logic [LANES-1:0] ever_locked
);
 
  logic [LANES-1:0] blk_q, mrk_q;
  logic             combined_q;
 
  assign block_locked   = &lane_block_locked;
  assign markers_locked = &lane_marker_locked;
  assign block_mask     = lane_block_locked;
  assign marker_mask    = lane_marker_locked;
 
  wire combined_c = (&lane_block_locked) && (&lane_marker_locked);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      for (int unsigned l = 0; l < LANES; l++) begin
        c_block_loss[l]  <= '0;
        c_marker_loss[l] <= '0;
      end
      blk_q            <= '0;
      mrk_q            <= '0;
      combined_q       <= 1'b0;
      down_cause_mask  <= '0;
      down_cause_valid <= 1'b0;
      ever_locked      <= '0;
    end else begin
      if (clear) begin
        for (int unsigned l = 0; l < LANES; l++) begin
          c_block_loss[l]  <= '0;
          c_marker_loss[l] <= '0;
        end
        // ever_locked deliberately NOT cleared -- it is a property of the
        // installation, not of a window an operator happened to choose.
      end else begin
        for (int unsigned l = 0; l < LANES; l++) begin
          if (blk_q[l] && !lane_block_locked[l] && !(&c_block_loss[l]))
            c_block_loss[l] <= c_block_loss[l] + 1'b1;
          if (mrk_q[l] && !lane_marker_locked[l] && !(&c_marker_loss[l]))
            c_marker_loss[l] <= c_marker_loss[l] + 1'b1;
        end
      end
 
      blk_q      <= lane_block_locked;
      mrk_q      <= lane_marker_locked;
      combined_q <= combined_c;
 
      for (int unsigned l = 0; l < LANES; l++)
        if (lane_block_locked[l] && lane_marker_locked[l]) ever_locked[l] <= 1'b1;
 
      // Capture on the DOWN edge only. Sampling throughout the outage would
      // record recovery rather than failure.
      if (combined_q && !combined_c) begin
        down_cause_mask  <= ~(lane_block_locked & lane_marker_locked);
        down_cause_valid <= 1'b1;
      end
    end
  end
 
  // Worst lane by combined loss count.
  logic [CNT_W-1:0]  max_c;
  logic [LANE_W-1:0] maxl_c;
  logic              any_c;
 
  always_comb begin
    max_c  = c_block_loss[0] + c_marker_loss[0];
    maxl_c = '0;
    any_c  = (max_c != '0);
    for (int unsigned l = 1; l < LANES; l++) begin
      automatic logic [CNT_W-1:0] tot = c_block_loss[l] + c_marker_loss[l];
      if (tot != '0) any_c = 1'b1;
      if (tot > max_c) begin
        max_c  = tot;
        maxl_c = LANE_W'(l);
      end
    end
    worst_lane       = maxl_c;
    worst_lane_valid = any_c;
  end
 
endmodule

Classification: synthesizable.

What it teaches: that combining is not the problem — combining early is. The sequencer genuinely needs one bit, and producing it costs an AND gate. What costs nothing extra and is nearly always omitted is keeping the inputs. Once reduced, the question every multi-lane investigation opens with — which lane? — has no answer anywhere in the design.

Deliberately simplified: no per-lane timeline. A production design often keeps Section 9's stage durations per lane too, which answers "which lane was slow" as well as "which lane failed".

Production implication: the two masks separate two faults that produce an identical combined bit. One lane dominating c_block_loss is a physical claim about one fibre or one trace — check that path. A uniform spread across lanes points at something shared: the reference clock, the supply, the environment, or the far end. Same symptom, opposite investigations, and only per-lane counts choose between them.

And ever_locked carries the same distinction Chapter 3.1 §9 drew for pairs. A lane that has never locked is miswired, connected to the wrong fibre, or facing a dead transceiver. A lane that locks and drops is marginal. The combined bit reads identically for both, and only this mask separates them — which is why it survives clear.

11. Why Bring-Up Time Is a Sum of Search Times

The ladder's stages take wildly different times, and until Chapter 3.5 it was not possible to say why. Now it is.

GateWhat bounds its durationTypical character
signal detectthe detector's own response timefast, and fixed
clock lockthe recovery loop's settling timefast, and fixed
block lockevidence quality — how fast a wrong offset is rejectedvariable, and code-dependent
marker lockthe marker interval — when one is next sentbounded, and by design
deskewone marker round across all lanesbounded, and by design
codeword lockcodeword structure and its own evidencevariable

Row three is the one that surprises people, and Chapter 3.5 §11 quantified it. A 4B/5B stream has seven illegal patterns in thirty-two, so a wrong offset is rejected quickly and the search converges fast. A 64b/66b stream has two illegal headers in four, so a wrong offset survives roughly one block in two — the search needs far more blocks to reach the same confidence, and block lock on a 64b/66b link is genuinely, structurally slower.

That is not a defect. It is the price Chapter 3.5 §13 identified for the overhead falling from 25% to 3.125% — the alignment evidence budget shrank along with the overhead, and slower lock is what that costs.

Rows four and five are different in kind. They do not depend on evidence at all; they depend on when the transmitter next acts. Marker lock cannot happen faster than a marker arrives, so the interval is the bound — which makes them the only stages whose duration is predictable from configuration alone.

12. Assertions

Every property below is a property of these teaching models. IEEE 802.3 specifies auto-negotiation behaviour, marker structure and maximum skew per clause; how a particular design sequences the gates, chooses fallback depth and instruments the timeline is an implementation choice.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// ─── Ordering: the ladder cannot be skipped ────────────────────────────────
// The chapter's central claim, asserted. Catches: a sequencer that advances
// on a stale lock indication, declaring a link up on preconditions that no
// longer hold.
property p_ladder_is_monotone_upward;
  @(posedge clk) disable iff (!rst_n)
  advanced |-> (state == link_state_e'($past(state) + 1));
endproperty
 
// ─── Safety: signal loss dominates every other transition ──────────────────
// Catches a priority inversion where a stale block_locked lets the machine
// advance after the cable was unplugged. Same rule Chapter 3.1 §13 asserted
// for pair training, at link scope.
property p_signal_loss_wins;
  @(posedge clk) disable iff (!rst_n)
  (!signal_ok && (state != LS_DOWN)) |=> (state == LS_DOWN);
endproperty
 
// ─── Ordering: fallback goes exactly one rung ──────────────────────────────
// Catches a sequencer returning to LS_DOWN on any loss, which turns a
// microsecond block re-lock into a full outage every layer above absorbs.
property p_block_loss_falls_back_one;
  @(posedge clk) disable iff (!rst_n)
  ((state == LS_BLOCKED) && signal_ok && !block_locked) |=> (state == LS_CLOCKED);
endproperty
 
property p_marker_loss_falls_back_one;
  @(posedge clk) disable iff (!rst_n)
  ((state == LS_MARKED) && signal_ok && block_locked && !markers_locked)
    |=> (state == LS_BLOCKED);
endproperty
 
// ─── Causation: negotiation gates the ladder ───────────────────────────────
// Catches a ladder starting before both ends agree what to attempt, which
// fails at clock lock on a healthy channel -- Section 4's hardest case.
property p_ladder_needs_negotiation;
  @(posedge clk) disable iff (!rst_n)
  ((state == LS_DOWN) && !negotiated) |=> (state == LS_DOWN);
endproperty
 
// ─── Safety: link_up requires every gate ───────────────────────────────────
// Catches a design declaring the link usable on a subset of preconditions.
property p_up_requires_all_gates;
  @(posedge clk) disable iff (!rst_n)
  link_up |-> (signal_ok && clock_locked && block_locked
               && markers_locked && deskew_ok);
endproperty
 
// ─── MARKER PROPERTY: all lanes insert simultaneously ──────────────────────
// Section 7's skew measurement is meaningless unless this holds. Catches a
// per-lane counter that can drift, after which measured "skew" is actually
// insertion phase error and the deskew buffer absorbs a fiction.
property p_markers_inserted_together;
  @(posedge clk) disable iff (!rst_n)
  |marker_inserted_vec |-> &marker_inserted_vec;
endproperty
 
// ─── Causation: marker lock needs spacing, not just a pattern ──────────────
// Catches lock declared on a single coincidental pattern match, which locks
// onto a data block that happened to resemble a marker.
property p_marker_lock_needs_spacing;
  @(posedge clk) disable iff (!rst_n)
  $rose(marker_locked) |-> $past(spacing_ok_c);
endproperty
 
// ─── Safety: deskew requires every lane locked ─────────────────────────────
// Catches alignment declared while a lane is still searching, which reads
// that lane's buffer at an arbitrary offset and corrupts every word.
property p_deskew_needs_all_lanes;
  @(posedge clk) disable iff (!rst_n)
  deskew_ok |-> (&lane_locked);
endproperty
 
// ─── Safety: measured skew never exceeds the buffer ────────────────────────
// Catches a read pointer computed from a skew larger than DEPTH, which
// wraps and silently reads the wrong blocks -- data corruption on a link
// reporting alignment.
property p_skew_within_depth;
  @(posedge clk) disable iff (!rst_n)
  deskew_ok |-> (worst_skew < DEPTH);
endproperty
 
// ─── Mutual exclusion: overflow is not "still trying" ──────────────────────
// Catches merging the two, producing a link that appears to be progressing
// when it has actually failed and cannot recover by waiting.
property p_overflow_excludes_ok;
  @(posedge clk) disable iff (!rst_n)
  skew_overflow |-> !deskew_ok;
endproperty
 
// ─── Conservation: the timeline accounts for the whole attempt ─────────────
// Catches stage durations that drop time, making the timeline unusable for
// exactly the diagnosis it exists for.
property p_stage_durations_sum;
  @(posedge clk) disable iff (!rst_n)
  timeline_valid |-> (last_bringup_time == stage_duration_sum);
endproperty
 
// ─── Stability: furthest_ever never decreases ──────────────────────────────
// Catches furthest_ever folded into the clear branch, destroying the
// distinction between "never worked" and "worked and regressed".
property p_furthest_ever_monotone;
  @(posedge clk) disable iff (!rst_n)
  1'b1 |=> (furthest_ever >= $past(furthest_ever));
endproperty
 
// ─── Bounded response: a stalled stage is reported ─────────────────────────
// Catches a watchdog that never fires, leaving a link stuck forever with
// nothing recording where.
property p_stall_is_reported;
  @(posedge clk) disable iff (!rst_n)
  (stage_stalled && (state != LS_UP)) |-> ##[0:2] stall_valid;
endproperty
 
// ─── Conservation: the combined bit is exactly the AND of the lanes ────────
// Catches a combined signal that drifts from its inputs -- typically a
// registered copy that misses a one-cycle drop, after which the sequencer
// and the per-lane masks disagree about whether the link is locked.
property p_combined_matches_lanes;
  @(posedge clk) disable iff (!rst_n)
  (block_locked == (&lane_block_locked))
    && (markers_locked == (&lane_marker_locked));
endproperty
 
// ─── Stability: ever_locked never clears ───────────────────────────────────
// Catches ever_locked folded into the clear branch, which destroys the
// distinction between a lane that has never worked and one that drops.
property p_ever_locked_monotone;
  @(posedge clk) disable iff (!rst_n)
  1'b1 |=> (ever_locked == ($past(ever_locked) | ever_locked));
endproperty

13. Verification

A transmitter inserts the same alignment marker into four lanes simultaneously. The lanes traverse physical paths of different lengths, so the markers arrive at different times. The deskew buffer takes the earliest arrival as its reference, measures each lane's lag, and delays the earlier lanes so all four are read out together.Insert togethersame marker, same cycleDifferent pathslengths and delays differArrive apartthe spread is the skewRead out togetherlanes alignedMeasure the lagearliest lane is thereferenceReport the marginhow much buffer is left12
Figure 3 — the same marker in every lane, and the arrival spread is the skew.

Scenarios

  1. A clean bring-up, all gates in order. Verify the sequencer passes LS_DOWN through LS_UP with no regression, that advanced pulses five times, and that the timeline records five non-zero stage durations summing to last_bringup_time.
  2. Signal never arrives. Verify the machine stays in LS_DOWN, stage_stalled asserts after T_SIGNAL, and stall_state records LS_DOWN rather than being left invalid.
  3. Signal lost at each of the five higher states. Five runs. Verify every one returns to LS_DOWN immediately, including from LS_UP, and that no intermediate state is passed through.
  4. Block lock lost while LS_MARKED. Verify fallback to LS_CLOCKEDtwo rungs, because marker lock's precondition is also gone — and that this is distinguishable in the timeline from a single-rung fallback.
  5. Marker lock lost while LS_UP. Verify fallback to exactly LS_BLOCKED, not to LS_CLOCKED and not to LS_DOWN.
  6. Negotiation never completes. Verify the ladder does not start, no_remote asserts, and defaulted sticks. This is Section 4's case one.
  7. Negotiation with no common capability. Verify no_common_cap rather than no_remote, since they have different owners.
  8. A forced port with an_enable low. Verify negotiated is asserted anyway so the ladder proceeds. A forced port that stalls in LS_DOWN forever is a real and easily-introduced bug.
  9. Markers on all lanes, zero skew. Verify deskew_ok, worst_skew of zero, and skew_margin equal to the full DEPTH.
  10. Markers with skew just under DEPTH. Verify alignment succeeds, worst_lane names the right lane, and skew_margin reads one.
  11. Markers with skew exactly at DEPTH. The boundary. Verify skew_overflow rather than a silent wrap of the read pointer.
  12. One lane's marker never arrives. Verify markers_missing on that lane, deskew_ok low, and that the failure is attributed to the lane rather than reported as a generic deskew failure.
  13. A data block coincidentally matching a marker pattern. Verify marker lock is not declared, because the spacing check fails. Then send genuine markers and verify lock is achieved.
  14. Markers stop after lock. Verify marker_locked deasserts after the timeout, c_markers_missed advances, and the sequencer falls back one rung.
  15. Repeated bring-up attempts. Verify c_attempts and c_successes diverge correctly and that furthest_this_attempt resets each time while furthest_ever does not.
  16. clear during operation. Verify counters zero and furthest_ever does not — the distinction between "never worked" and "worked and regressed" must survive.
  17. Reset at each state. Verify the machine returns to LS_DOWN, no partial timeline is reported valid, and no stale stall state persists.
  18. One lane losing lock repeatedly, three healthy. Verify worst_lane names it, down_cause_mask captures it on the down edge, and the combined block_locked bit alone gives no indication which lane it was.
  19. A lane that never locks. Verify its ever_locked bit stays low while the others set theirs, and that it survives a clear. This is the miswired-lane case, and it is indistinguishable from a marginal lane on any combined signal.

What the checker must own

  • A sequencer reference model implementing the eight transitions independently, including the fallback depths. Fallback depth is the property most likely to be broken by a refactor and the least likely to be noticed.
  • A skew injector with independent per-lane delay, configurable up to and beyond DEPTH. Without it scenarios 10 through 12 cannot be constructed at all.
  • A structured-payload generator, not merely a random one. Scenario 18 needs it, and so does any honest bring-up-time characterisation.
  • Timeline closure checking: stage durations must sum to the total at every completed attempt, or the timeline cannot be used for the diagnosis it exists for.
  • Coverage crosses of link state against stall against regression. The bin (LS_CLOCKED, regressed, eventually LS_UP) must be well populated — that is a false lock recovered from, the normal healthy behaviour Section 12's rejected property would forbid.

14. Debugging — Which Gate, and For How Long

The symptom: a link that will not come up, or one that takes far longer than expected.

Step 1 — read stall_state, or the current state if it is still progressing. One register, and it names the gate. Every gate names a different owner:

Stalled atMeaningWhere to go
LS_DOWNnothing arriving, or negotiation incompletecheck no_remote first, then Chapter 3.2 or Chapter 3.1
LS_SIGNALsignal present, timing not recoveredsignal quality, or a rate mismatch between the ends
LS_CLOCKEDtiming recovered, blocks do not parsea coding mismatch, or a slow search — check the duration
LS_BLOCKEDblocks parse, markers not foundcheck markers_missing — arriving-but-not-locking is a different fault from not-arriving
LS_MARKEDmarkers found, deskew failingcheck skew_overflow — physical path lengths, not electronics

Step 2 — read stage_elapsed alongside it, because the gate alone is not enough. A stage running for twice its usual duration is searching; one running for a thousand times is stuck. Same state, different problems, and stage_stalled is the design's own opinion about which.

Step 3 — read furthest_ever against furthest_this_attempt. This is the question everyone forgets to ask and it changes where the investigation starts:

  • furthest_ever is LS_UP — this link has worked. Something changed: a cable moved, a module swapped, the temperature rose, the far end reconfigured. Look for the change.
  • furthest_ever has never reached LS_UP — this link has never worked. It was never configured correctly, and no amount of channel investigation will help. Start at the far end's settings.

Step 4 — if it stalls at LS_DOWN, separate the two causes before anything else. no_remote means the far end never advertised, and defaulted sticking means we proceeded on an assumption. Section 4's case one produces a link that comes up and then misbehaves under load, so defaulted should be checked even on links that appear to be working.

Step 5 — if it stalls at LS_MARKED, read worst_lane and skew_margin. Skew is a claim about physical path lengths, so worst_lane names a specific fibre or a specific trace. And skew_margin shrinking across a warm-up cycle is the early warning: a link that aligns when cold and fails when warm is a skew-margin problem, and the margin was visible for months before it failed.

Step 6 — the link comes up but takes far longer than it used to. Trend last_bringup_time. There is no fault to find at any single instant, and the finding exists only in the sequence. Then read stage_duration to see which gate got slower — a block-lock stage that has doubled means the code's evidence is being degraded by errors, which sends you to Chapter 3.7's pre-correction rate and then to Chapter 3.3's margin.

The method stated once: the gate names the owner, the elapsed time separates searching from stuck, furthest_ever decides whether to look for a change or for a misconfiguration, and the stage durations trended over weeks catch the degradation that every instantaneous reading reports as healthy.

15. Common Misconceptions

"A link either comes up or it does not."

The wrong model: bring-up is binary, so "link down" is a complete description of the problem.

What it costs: every investigation starts from scratch with no owner. You cannot tell a channel problem from a configuration problem from a physical path-length problem, and you check them in an arbitrary order.

The corrected model: bring-up is a ladder of six gates, each attemptable only once the one below it has succeeded. The stalled gate names the owner — LS_SIGNAL means a rate mismatch or signal quality, LS_CLOCKED means a coding mismatch, LS_MARKED means physical path lengths. One register read replaces a search.

"Bring-up time is a fixed property of the PHY."

The wrong model: a link takes as long as its datasheet says.

What it costs: you treat a slower bring-up as a defect and chase a fault that is not there. Or worse, you treat a bring-up that has gradually slowed from 3 ms to 300 ms as normal variation, and miss a genuinely degrading link.

The corrected model: bring-up time is a sum of search times, and searches depend on evidence quality and on the data being carried. Block lock on a 64b/66b link is structurally slower than on 4B/5B, because two illegal headers in four is far weaker evidence than seven illegal groups in thirty-two. And structured payload data makes wrong offsets survive longer than random data does.

"Regression during bring-up means something is broken."

The wrong model: a link that falls back a stage is failing.

What it costs: this is Section 12's rejected property. You assert against regression, it fires on working hardware, and either you distrust the assertion suite or you "fix" the design by preventing lock release — producing a link that false-locks permanently and delivers corrupted data with every gate reporting passed.

The corrected model: every gate is a search, and searches can commit wrongly and must recover. Regression is the recovery mechanism working. The finding is a rate — many regressions without reaching LS_UP, or regressions after the link has been up — never a single occurrence.

"Alignment markers are just a start-up preamble."

The wrong model: markers identify lanes once and are then overhead.

What it costs: you remove them after lock as a bandwidth optimisation, and lose continuous skew verification and bounded re-acquisition. A lane that loses block lock then forces the whole link back down the ladder instead of rejoining at the next marker.

The corrected model: markers are periodic and permanent, and they do three jobs — lane identity, a continuously re-checked timing reference for skew, and a bounded re-acquisition path. Skew drifts with temperature, so a one-time measurement made on cold equipment is worthless by afternoon.

"Link up means link ready."

The wrong model: once link_up asserts, the PHY's job is done and its state is uninteresting.

What it costs: you miss every gradual failure in this module. A link whose bring-up time has increased tenfold, or whose skew margin has halved, or which regresses forty times before succeeding, reports link_up at every instant anyone checks — and is weeks from failing.

The corrected model: link_up is instantaneous state. Health is in the durations and the countslast_bringup_time trended, skew_margin trended, c_regressions per attempt. This is Chapter 3.3's slope-against-cliff argument at a third layer, and it applies to every gate on the ladder.

16. Interview Reasoning

"A link will not come up. Walk me through your diagnosis."

The weak answer lists layers. The answer that ends the topic names a first register read — the stalled gate — and then says what each possible answer would mean, because each names a different owner. The follow-up that separates real experience is furthest_ever: has this link ever worked? A link that once did has had something change; one that never has was never configured correctly, and those are opposite investigations from the same current state.

"Why would a link come up in 2 ms on the bench and 40 ms in production?"

Because block lock is a search, and search duration depends on how fast a wrong offset is rejected. Random bench data makes wrong offsets fail almost immediately. Production traffic is structured — repeating patterns, idles, compressed payloads — and structured data lets a wrong offset survive longer, sometimes long enough to false-lock and require recovery. No fault exists. Naming this unprompted signals someone who has brought up real links.

"What are alignment markers actually for?"

Three things, and naming only the first is the common incomplete answer: lane identity, so the receiver knows which physical lane carries which logical position; a timing reference, because the same marker enters every lane simultaneously so arrival differences are the skew; and bounded re-acquisition, because a lane that loses lock rejoins at the next marker instead of restarting the link. The strong follow-up is why they are periodic rather than a preamble — skew drifts with temperature, so the measurement must be continuous.

17. Understanding Check

Because each gate's precondition is the previous gate's result, and the reasons differ by rung:

GateAssumesWhy the dependency is forced
clock locka signal existsphysical — timing is extracted from transitions, and no signal means no transitions
block locktiming is knowndefinitional — slicing at candidate offsets presupposes bit boundaries
marker lockblocks parsestructural — a marker is a block, so recognising one requires parsing blocks
deskewevery lane has marker lockmeasurement — skew is the arrival-time difference of the same marker
codeword locklanes are alignedstructural — a codeword spans lanes and needs them in order

Five dependencies, five different kinds of reason. That variety is why the ladder is worth understanding rather than memorising — each fails differently, and the diagnosis in Section 14 uses that.

The follow-up to be ready for: does every link have all six? No. Marker lock and deskew exist only because there are multiple lanes; a single-lane 10GBASE-R link has three gates. That is a real source of confusion — an engineer whose experience is single-lane expects the link up at block lock, and on a multi-lane link there are two more searches, the second of which cannot begin until every lane has finished the first.

18. What's Next

The claim this chapter defended: a link does not come up; it climbs a ladder of assumptions, and every rung is a search whose duration depends on the evidence available to it.

The ladder cannot be reordered, because each gate's precondition is the previous gate's result — and the five dependencies are forced for five different reasons: physical, definitional, structural, measurement, structural. Each loss falls back exactly one rung, to the state whose precondition still holds, because a sequencer returning to the bottom turns a microsecond re-lock into an outage.

And because every rung is a search, bring-up time is a sum of search times rather than a constant. Chapter 3.5 quantified why: block lock on a code with two illegal headers in four is structurally slower than on one with seven illegal patterns in thirty-two, and structured payload data makes every such search longer than random data does. That is the whole explanation for a link that comes up in 2 ms on the bench and 40 ms in production, and no fault is involved.

Module 3 is complete. Copper and fibre as channels, the analog channel and its two-axis margin, the sublayer split, the codes, the modulation, the correction, and now the sequence that brings them all up. One argument recurred at every layer and is worth carrying out of the module: measure what degrades continuously — margin, pre-correction error rate, skew margin, bring-up duration — and treat what degrades discontinuously as an alarm of last resort.

Chapter 4.1 — Where the MAC Ends and the PHY Begins opens Module 4 by returning to a boundary this module kept crossing. Chapter 3.4 approached it from below, as the top of the PHY, and gave the reconciliation sublayer and the media-independent interface. Module 4 approaches it from above, as the contract a MAC is designed against — and the question it has to answer is the one this module has been implicitly relying on throughout: given any responsibility, which side owns it, and what decides?

The full path is on the Ethernet curriculum index.

Continue learning

Standards & specifications

Governing standard
IEEE Std 802.3 (Ethernet)(opens IEEE in a new tab)

Defines the Ethernet MAC, the media-independent interfaces and the physical-layer sublayers, including framing, access control, auto-negotiation and per-rate PHY specifications. VLAN tagging, priority and time-sensitive shaping are defined by IEEE 802.1, not by 802.3.

This page also covers RTL structure, verification approach and debugging technique. Those are engineering practice built on the standard, not requirements the standard itself imposes.

Where this fits

Part of the Ethernet curriculum.