One Frame, End to End

Chapter 2.1 named the blocks and the contracts between them. That is a static picture: what exists, and what each part is forbidden to know.

A frame is not static. It is a sequence of events across all six blocks, and then the same sequence in reverse at the far end — except that the reverse is not the same at all.

What happens to one frame between a client that has octets and a client that receives them, and why is the return journey structurally harder?

The second half of that question is the chapter. Every stage of the transmit path decides something; every stage of the receive path has to discover what was decided, from a signal that arrived with no explanation attached.

1. The Journey, Named

Twelve steps down, twelve back. Each is a stage that holds state, and each can stall.

TRANSMIT                                    RECEIVE (far end)
  1  client offers octets                    12  client receives octets
  2  MAC accepts, takes ownership            11  MAC delivers, or discards
  3  MAC frames: preamble, SFD, addresses    10  MAC checks the FCS
  4  MAC pads to the minimum                  9  MAC extracts fields, filters
  5  MAC appends the FCS                      8  MAC finds the frame start
  6  MAC waits out the interframe gap         7  RS presents octets
  7  RS slices onto the interface             6  PCS decodes, deletes idle
  8  PCS codes, inserts idle                  5  PCS achieves block lock
  9  PMA serialises                           4  PMA deserialises
 10  PMD drives the medium                    3  PMA recovers the clock
 11  ── the medium ──────────────────────     2  PMD detects a signal
 12                                           1  ── the medium ─────────────

Read the two columns against each other and the asymmetry is immediate. Transmit step 3 places the addresses; receive step 9 has to find them. Transmit step 8 inserts idle; receive step 6 has to recognise and remove it. Transmit step 10 drives a signal; receive steps 2 and 3 have to detect one and recover a clock from it that was never sent as a separate thing.

And notice the ordering. The receive column is numbered from the bottom up, because that is the order in which it happens and the order in which it can fail. Chapter 2.1's link_up conjunction is this column: signal detect, then clock lock, then block lock, then lane alignment — each a discovery that the one above it depends on.

2. Identity — The One Thing That Must Survive

Every layer adds something and removes it again. What must cross unaltered is the client's octets, in order, with nothing inserted and nothing lost.

Two rows are worth reading carefully.

The RS adds nothing. It converts one width into another. That is why Chapter 2.1 §5's conservation property is a ratio rather than a difference — the same octets leave, in a different shape.

The PCS adds a great deal and none of it is addressed to anything. Coding expansion, idle between frames, alignment markers across lanes: none is a header, none has a destination, and none survives past the peer PCS. Confusing this with encapsulation is the misconception Chapter 2.1 §15 names, and this table is why.

The invariant, stated precisely: the octet sequence the client offers appears, in order and unaltered, inside what is transmitted; and the receiving client is delivered exactly that sequence after every layer has removed what its peer added. Section 9's checker is that sentence in RTL.

3. RTL 1 — The Transmit Pipeline

// SYNTHESIZABLE. Four transmit stages with a valid/ready handshake between
// each, carrying an octet and a tag that identifies it end to end.
//
// NOT a MAC or PHY. Stages are placeholders; the point is the handshake
// discipline, the identity tag, and per-stage occupancy.
module tx_journey #(
  parameter int unsigned WIDTH = 8,
  parameter int unsigned TAG_W = 8
) (
  input  logic clk,
  input  logic rst_n,
 
  input  logic             in_valid,
  input  logic [WIDTH-1:0] in_data,
  input  logic [TAG_W-1:0] in_tag,     // identity, injected once at the top
  input  logic             in_last,
  output logic             in_ready,
 
  output logic             out_valid,
  output logic [WIDTH-1:0] out_data,
  output logic [TAG_W-1:0] out_tag,
  output logic             out_last,
  input  logic             out_ready,
 
  // Per-stage occupancy — the instrument Section 7 reads.
  output logic [3:0]       stage_busy
);
 
  localparam int unsigned STAGES = 4;   // MAC, RS, PCS, PMA
 
  logic [WIDTH-1:0] d_q   [STAGES];
  logic [TAG_W-1:0] tag_q [STAGES];
  logic             last_q[STAGES];
  logic             v_q   [STAGES];
 
  // Ready propagates BACKWARD, combinationally. A stage can accept when it
  // is empty, or when it is emptying this cycle because the stage after it
  // is accepting. This is the standard skid-free chain; the important part
  // is what it makes true, which is stated as property P2 in Section 11.
  logic [STAGES:0] rdy;
  assign rdy[STAGES] = out_ready;
  for (genvar s = STAGES - 1; s >= 0; s--) begin : g_ready
    assign rdy[s] = !v_q[s] || rdy[s+1];
  end
  assign in_ready = rdy[0];
 
  for (genvar s = 0; s < STAGES; s++) begin : g_stage
    // Each stage's input is the previous stage's output, except stage 0
    // which takes the port. Written as a conditional rather than a shifted
    // array so the boundary is visible.
    wire             s_valid = (s == 0) ? in_valid : v_q[s-1];
    wire [WIDTH-1:0] s_data  = (s == 0) ? in_data  : d_q[s-1];
    wire [TAG_W-1:0] s_tag   = (s == 0) ? in_tag   : tag_q[s-1];
    wire             s_last  = (s == 0) ? in_last  : last_q[s-1];
 
    always_ff @(posedge clk or negedge rst_n) begin
      if (!rst_n) begin
        v_q[s] <= 1'b0; d_q[s] <= '0; tag_q[s] <= '0; last_q[s] <= 1'b0;
      end else if (rdy[s]) begin
        v_q[s]    <= s_valid;
        if (s_valid) begin
          // THE IDENTITY RULE. Every stage carries the tag through
          // unaltered. No stage may generate, modify or reorder it — which
          // is what makes an end-to-end identity check possible at all, and
          // is exactly the discipline a real datapath must maintain for
          // whatever field it uses to correlate a frame with its completion.
          d_q[s]    <= s_data;
          tag_q[s]  <= s_tag;
          last_q[s] <= s_last;
        end
      end
    end
  end
 
  assign out_valid  = v_q[STAGES-1];
  assign out_data   = d_q[STAGES-1];
  assign out_tag    = tag_q[STAGES-1];
  assign out_last   = last_q[STAGES-1];
  assign stage_busy = {v_q[3], v_q[2], v_q[1], v_q[0]};
 
endmodule

Classification: synthesizable.

What it teaches: that a pipeline of layers is a chain of handshakes, and the property that matters across all of them is identity preservation. Every stage carries the tag through without touching it. A stage that regenerated, reordered or dropped a tag would break the ability to correlate anything end to end — which is precisely what a real MAC must not do with the field it uses to match a frame to its completion.

The backward rdy chain is worth studying. It is combinational through all four stages, which is a real timing cost in a deep pipeline and the reason production designs insert skid buffers. It is written this way here because the alternative — a registered ready — introduces a bubble per stage and obscures the conservation property. The trade is stated rather than hidden, and Section 13's production note says which way a real design should go.

Deliberately simplified: stages do no work; one beat of storage each, so throughput halves under any backpressure; no width conversion between stages, which the RS actually performs; no error path.

Production implication: a real pipeline needs skid buffers to sustain full rate under intermittent backpressure, width conversion at the RS boundary, a clock-domain crossing somewhere in the chain, and a defined abort path so a frame that must be discarded mid-flight does not emerge partially.

4. RTL 2 — The Receive Pipeline, Which Must Search

The same journey backwards, and it is a different kind of machine.

// SYNTHESIZABLE. The receive journey: find a frame start in an idle stream,
// extract a bounded number of octets, then decide whether to deliver.
//
// NOT a MAC receive path. No real SFD value, no FCS, no address filtering.
// Chapter 7.2 owns the real thing.
module rx_journey #(
  parameter int unsigned WIDTH     = 8,
  parameter int unsigned SFD       = 8'hD5,  // ILLUSTRATIVE delimiter value
  parameter int unsigned MAX_BEATS = 64,
  parameter int unsigned MIN_BEATS = 8,      // ILLUSTRATIVE minimum
  localparam int unsigned CNT_W    = $clog2(MAX_BEATS + 1)
) (
  input  logic clk,
  input  logic rst_n,
 
  // From the layer below. NO framing information whatsoever — just octets
  // and a flag saying the physical layer believes them.
  input  logic             phy_valid,
  input  logic [WIDTH-1:0] phy_data,
  input  logic             phy_error,    // the layer below lost confidence
 
  output logic             cli_valid,
  output logic [WIDTH-1:0] cli_data,
  output logic             cli_last,
 
  output logic             rx_ok,
  output logic             rx_runt,      // ended before the minimum
  output logic             rx_giant,     // never ended
  output logic             rx_phy_error, // the layer below reported trouble
  output logic [CNT_W-1:0] rx_length
);
 
  typedef enum logic [1:0] {
    R_HUNT,     // searching an idle stream for a delimiter
    R_RECEIVE,  // extracting, and counting
    R_DECIDE    // one cycle to judge what was received
  } r_state_e;
 
  r_state_e         state_q, state_d;
  logic [CNT_W-1:0] len_q;
  logic             ended_q, err_q, giant_q;
 
  // THE STATE WITH NO TRANSMIT COUNTERPART. A transmitter never hunts; it
  // simply begins. This state can be occupied indefinitely, and everything
  // downstream is idle while it is — which is why a receiver that hunts
  // forever looks identical to a link with no traffic.
  wire found_start = (state_q == R_HUNT) && phy_valid && (phy_data == WIDTH'(SFD));
 
  // A frame ends when the layer below stops presenting octets. That is a
  // DISCOVERY too: the transmitter knew where the end was, and this side
  // infers it from an absence.
  wire ends_now = (state_q == R_RECEIVE) && !phy_valid;
 
  wire at_max = (len_q == CNT_W'(MAX_BEATS));
 
  always_comb begin
    state_d = state_q;
    case (state_q)
      R_HUNT:    if (found_start) state_d = R_RECEIVE;
      R_RECEIVE: if (ends_now || at_max) state_d = R_DECIDE;
      R_DECIDE:  state_d = R_HUNT;   // always return to hunting
      default:   state_d = R_HUNT;
    endcase
  end
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      state_q <= R_HUNT; len_q <= '0;
      ended_q <= 1'b0; err_q <= 1'b0; giant_q <= 1'b0;
    end else begin
      state_q <= state_d;
 
      if (state_q == R_HUNT) begin
        len_q <= '0; ended_q <= 1'b0; err_q <= 1'b0; giant_q <= 1'b0;
      end else if (state_q == R_RECEIVE) begin
        if (phy_valid && !at_max) len_q <= len_q + 1'b1;
        if (phy_error)            err_q <= 1'b1;
        if (at_max)               giant_q <= 1'b1;
        if (ends_now)             ended_q <= 1'b1;
      end
    end
  end
 
  // Delivery is UNCONDITIONAL during receive and the verdict comes after.
  // That is deliberate and it is what a real receive path does: octets are
  // handed on as they arrive because buffering the whole frame would cost a
  // frame time of latency, and the client is told afterwards whether to
  // believe them. Chapter 12.6 develops the same trade for a switch.
  assign cli_valid = (state_q == R_RECEIVE) && phy_valid;
  assign cli_data  = phy_data;
  assign cli_last  = (state_q == R_RECEIVE) && (ends_now || at_max);
 
  // Four verdicts, mutually exclusive by construction. A single "bad frame"
  // output would merge a short frame, a long one and a physical-layer fault
  // — three different causes and three different next steps.
  assign rx_phy_error = (state_q == R_DECIDE) && err_q;
  assign rx_giant     = (state_q == R_DECIDE) && !err_q && giant_q;
  assign rx_runt      = (state_q == R_DECIDE) && !err_q && !giant_q
                        && (len_q < CNT_W'(MIN_BEATS));
  assign rx_ok        = (state_q == R_DECIDE) && !err_q && !giant_q
                        && (len_q >= CNT_W'(MIN_BEATS));
  assign rx_length    = len_q;
 
endmodule

Classification: synthesizable.

What it teaches: compare the two state machines. tx_journey has no states at all — it is a pipeline. rx_journey has three, one of which (R_HUNT) can be occupied indefinitely and has no transmit counterpart. That structural difference is the chapter's thesis in two modules.

R_HUNT is the state that makes receive hard. A receiver in it is doing something — searching — but produces no output and cannot distinguish "no traffic" from "traffic I cannot recognise". Those two situations have completely different causes and identical appearances, which is why Chapter 21.4's link debugging has to descend the stack rather than observing this state.

Delivery before validation is the design decision worth arguing about. This module hands octets to the client during R_RECEIVE and only reports the verdict at R_DECIDE — so a client can receive most of a frame that turns out to be bad. The alternative is to buffer the whole frame and deliver only good ones, which costs a full frame time of latency on every frame. Real MACs do both, selectably; the trade is the same one Chapter 12.6 examines for cut-through switching.

Deliberately simplified: an illustrative delimiter compared as a whole octet, where a real receiver hunts a bit-aligned pattern; no FCS, so rx_ok means "plausible", not "correct"; end-of-frame inferred from phy_valid falling; no address filtering.

Production implication: a real receive path detects the delimiter at bit alignment and recovers octet alignment from it, validates with the FCS before asserting a success indication, filters on the destination address before delivering, and counts each verdict in a saturating counter for the taxonomy Chapter 21.2 builds.

The figure has no self-loop on HUNT and that omission is deliberate: staying in HUNT is not a transition, it is the absence of one, which is exactly why a receiver stuck there produces no event anything can trigger on.

5. Where Each Layer's Discovery Happens

The asymmetry is not confined to the MAC. Each layer has its own search, and each can fail independently.

Read the failure column top to bottom and it is Chapter 2.1's link_up conjunction plus two. The first four terms are the ones aggregated into link status; the last two are frame-level and appear only once the link is up. That is why a link can be up and carry nothing, and why the debugging method descends and then switches to a frame taxonomy.

The clock-recovery row is the one worth pausing on, because it is the least intuitive. No clock is transmitted alongside the data on a serial link; the receiver extracts timing from the data's own transitions. That is why line coding must guarantee transitions — a long run of identical bits starves the recovery — and it is the reason Chapter 3.5's block codes care about transition density at all. A coding property exists because of a receive-side discovery problem.

6. RTL 3 — Instrumenting the Journey

A journey described is an anecdote. This block measures it.

// SYNTHESIZABLE (counters only). Measures per-stage occupancy and end-to-end
// latency across the journey.
//
// NOT a datapath. Instrumentation for the Section 8 localisation method.
module journey_instrument #(
  parameter int unsigned STAGES = 4,
  parameter int unsigned W      = 20
) (
  input  logic clk,
  input  logic rst_n,
  input  logic clear,
 
  input  logic [STAGES-1:0] stage_busy,
  input  logic [STAGES-1:0] stage_stalled,  // busy AND unable to advance
 
  input  logic              frame_in,       // a frame entered the pipeline
  input  logic              frame_out,      // it left
 
  output logic [W-1:0]      occupancy_acc [STAGES],
  output logic [W-1:0]      stall_cnt     [STAGES],
  output logic [W-1:0]      samples,
  output logic [W-1:0]      in_flight_acc,
  output logic [W-1:0]      completed
);
 
  logic [W-1:0] inflight_q;
 
  // Little's Law, computed from two counters and no timestamps: the mean
  // number in flight is the accumulated occupancy divided by the sample
  // count, and mean latency follows from throughput. Cheap, continuous, and
  // it needs no per-frame tagging at all.
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      inflight_q <= '0; samples <= '0; in_flight_acc <= '0; completed <= '0;
      for (int unsigned s = 0; s < STAGES; s++) begin
        occupancy_acc[s] <= '0; stall_cnt[s] <= '0;
      end
    end else if (clear) begin
      inflight_q <= '0; samples <= '0; in_flight_acc <= '0; completed <= '0;
      for (int unsigned s = 0; s < STAGES; s++) begin
        occupancy_acc[s] <= '0; stall_cnt[s] <= '0;
      end
    end else begin
      case ({frame_in, frame_out})
        2'b10:   inflight_q <= inflight_q + 1'b1;
        2'b01:   inflight_q <= inflight_q - 1'b1;
        default: inflight_q <= inflight_q;
      endcase
 
      if (samples != {W{1'b1}}) begin
        samples       <= samples + 1'b1;
        in_flight_acc <= in_flight_acc + inflight_q;
      end
      if (frame_out && completed != {W{1'b1}}) completed <= completed + 1'b1;
 
      for (int unsigned s = 0; s < STAGES; s++) begin
        if (stage_busy[s]    && occupancy_acc[s] != {W{1'b1}})
          occupancy_acc[s] <= occupancy_acc[s] + 1'b1;
        // A stage is STALLED when it is holding something it cannot hand on.
        // This is the counter Section 8 reads as a gradient, and the reason
        // it must be distinguished from merely busy.
        if (stage_stalled[s] && stall_cnt[s] != {W{1'b1}})
          stall_cnt[s] <= stall_cnt[s] + 1'b1;
      end
    end
  end
 
endmodule

Classification: synthesizable; instrumentation only.

What it teaches: that end-to-end latency can be measured continuously from two counters and no timestamps. Accumulate occupancy every cycle, count samples, and the mean in flight is the ratio. Combined with completion throughput that gives mean latency, and it costs two adders rather than a timestamp field on every frame.

Busy and stalled must be separate signals. A stage that is busy is doing its job. A stage that is stalled is holding something it cannot pass on, which is a different fact and the only one that localises a bottleneck. Merging them produces a counter that rises under healthy load and says nothing.

Deliberately simplified: no per-frame latency distribution, only a mean — and a mean hides a bimodal path, which Chapter 8.4 warns about; no percentile tracking; a single clear rather than atomic snapshotting.

Production implication: a real instrument snapshots all counters atomically so one read describes one instant, tracks a maximum alongside the mean, and exposes enough resolution to distinguish a rare long stall from a persistent short one — because those have different causes and the mean is identical.

7. RTL 4 — The End-to-End Identity Checker

Section 2 stated the invariant. This is it as hardware.

// SYNTHESIZABLE (as a checker). Confirms that the octet sequence offered at
// the top emerges at the bottom, in order, unaltered, with nothing inserted
// or lost.
//
// NOT for production: carries a reference copy. This is what a scoreboard
// does in simulation, written as hardware so the invariant is explicit.
module identity_checker #(
  parameter int unsigned WIDTH = 8,
  parameter int unsigned DEPTH = 64,
  localparam int unsigned PTR_W = $clog2(DEPTH)
) (
  input  logic clk,
  input  logic rst_n,
 
  input  logic             tx_valid,
  input  logic [WIDTH-1:0] tx_data,
 
  input  logic             rx_valid,
  input  logic [WIDTH-1:0] rx_data,
 
  output logic             mismatch,     // wrong value at the right position
  output logic             overrun,      // more arrived than was sent
  output logic             underrun,     // fewer arrived, and the stream ended
  output logic [PTR_W:0]   outstanding
);
 
  logic [WIDTH-1:0] ref_q [DEPTH];
  logic [PTR_W-1:0] wr_q, rd_q;
  logic [PTR_W:0]   cnt_q;
 
  wire empty = (cnt_q == '0);
  wire full  = (cnt_q == (PTR_W+1)'(DEPTH));
 
  // THREE FAILURE MODES, KEPT SEPARATE. A single "identity broken" output
  // would merge corruption, duplication and loss — three different bugs in
  // three different places. Section 12's P5, P6 and P7 assert them apart.
  assign mismatch = rx_valid && !empty && (rx_data != ref_q[rd_q]);
  assign overrun  = rx_valid && empty;
  assign underrun = 1'b0;   // asserted by the environment at end of stream
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      wr_q <= '0; rd_q <= '0; cnt_q <= '0;
    end else begin
      if (tx_valid && !full) begin
        ref_q[wr_q] <= tx_data;
        wr_q <= (wr_q == PTR_W'(DEPTH-1)) ? '0 : wr_q + 1'b1;
      end
      if (rx_valid && !empty)
        rd_q <= (rd_q == PTR_W'(DEPTH-1)) ? '0 : rd_q + 1'b1;
 
      case ({tx_valid && !full, rx_valid && !empty})
        2'b10:   cnt_q <= cnt_q + 1'b1;
        2'b01:   cnt_q <= cnt_q - 1'b1;
        default: cnt_q <= cnt_q;
      endcase
    end
  end
 
  assign outstanding = cnt_q;
 
endmodule

Classification: synthesizable as a checker; not a production datapath.

What it teaches: that "the frame arrived correctly" decomposes into three independent claims — right values, no extras, none missing — and that a checker which merges them cannot say which failed. Corruption points at a datapath; duplication points at a handshake accepting twice; loss points at a stage dropping. Three different investigations from three different outputs.

outstanding is the useful diagnostic, not the error flags. It is the number of octets that have entered and not yet emerged, so it is the pipeline's occupancy seen from the outside. A value that grows without bound means a stage has stopped forwarding; a value that goes to zero mid-frame means something was consumed without appearing.

Deliberately simplified: a single stream with no frame boundaries, so it checks octets rather than frames; underrun is left to the environment because a hardware checker cannot know a stream has ended; fixed depth, so a deep pipeline overruns the reference.

Production implication: in silicon this becomes a loopback mode plus a comparison against a known pattern, or a per-frame check value carried alongside the data. The reference-copy form belongs in simulation, where it is the scoreboard — which is why Section 13 says the scoreboard must model the expected sequence independently rather than reading the design's own tags.

8. RTL 5 — Localising a Stall

A stalled pipeline reports the wrong stage if you read the counters naively, and knowing why is one of the more transferable ideas in this track.

// SYNTHESIZABLE. Reads per-stage stall counters and names the stage that is
// actually the bottleneck — which is NOT the stage with the highest count.
//
// The method generalises to any pipeline with backpressure.
module stall_localiser #(
  parameter int unsigned STAGES = 4,
  parameter int unsigned W      = 20
) (
  input  logic clk,
  input  logic rst_n,
  input  logic [W-1:0] stall_cnt [STAGES],
  output logic [$clog2(STAGES)-1:0] blame_stage,
  output logic                      blame_valid
);
 
  // THE COUNTER-INTUITIVE PART, and the reason this block exists.
  //
  // A stage stalls when it is HOLDING something it cannot hand onward. So
  // the stage that is actually the bottleneck does NOT stall — it is busy,
  // and it is the stages BEHIND it that pile up. Stalls accumulate UPSTREAM
  // of their cause and propagate further upstream as the queue fills.
  //
  // An argmax over the stall counters therefore names a stage near the TOP
  // of the pipeline, which is almost always innocent. The bottleneck is
  // immediately BELOW the last stage with a high count.
  logic [$clog2(STAGES)-1:0] last_high;
  logic                      any_high;
  logic [W-1:0]              threshold;
 
  // A stage counts as high if it stalled meaningfully more than the stage
  // below it — the gradient, not the absolute value.
  always_comb begin
    any_high  = 1'b0;
    last_high = '0;
    threshold = '0;
    for (int unsigned s = 0; s < STAGES; s++) begin
      if (s + 1 < STAGES) threshold = stall_cnt[s+1];
      else                threshold = '0;
      // Meaningful means "at least twice the stage below". The factor is
      // ILLUSTRATIVE; a real threshold comes from measured noise.
      if (stall_cnt[s] > (threshold << 1)) begin
        any_high  = 1'b1;
        last_high = ($clog2(STAGES))'(s);
      end
    end
  end
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      blame_stage <= '0; blame_valid <= 1'b0;
    end else begin
      blame_valid <= any_high;
      // Blame the stage BELOW the last one showing an elevated stall count.
      blame_stage <= (last_high + 1 < ($clog2(STAGES))'(STAGES))
                       ? last_high + 1'b1 : last_high;
    end
  end
 
endmodule

Classification: synthesizable.

What it teaches: the single most useful debugging idea in a layered datapath — a bottleneck does not stall, it makes everything behind it stall. Reading stall counters as absolute values and taking the maximum names an innocent stage near the ingress. Reading them as a gradient and blaming the stage below the last elevated one names the culprit.

Why the naive version is so tempting. "Which stage stalled most" is the obvious question, the counters answer it directly, and the answer is wrong. It also fails confidently: the argmax always produces a stage, and that stage is always doing exactly what a healthy stage does when the one after it is slow.

The same reasoning appears elsewhere in the corpus — CXL's system-view analysis reaches it for a different fabric — which is a reason to trust the shape rather than the particular pipeline.

Deliberately simplified: a fixed factor-of-two threshold where a real one comes from measured noise; no hysteresis, so the blame can flap near the threshold; no account of a stage that is slow but never full.

Production implication: a real localiser uses a threshold derived from a quiet-system baseline, holds a verdict for a minimum interval so it can be read, and reports the whole profile alongside the verdict — because "stage 2" without the gradient that produced it is an assertion the reader cannot check.

9. RTL 6 — Loopback, Which Turns a One-Sided Fault Into a Local One

Section 17's last answer names loopback as the technique that makes an asymmetric fault diagnosable. It is worth building, because where the loop is closed determines what the test proves.

// SYNTHESIZABLE. Closes the transmit path back onto the receive path at a
// selectable depth, so a failure can be bisected by layer.
//
// NOT a device's loopback implementation. Real depths and their control are
// device-specific; the structure and what each depth proves are not.
module loopback_mux #(
  parameter int unsigned WIDTH = 8
) (
  input  logic clk,
  input  logic rst_n,
 
  // 0 = none (normal operation), 1 = at the MAC, 2 = at the PCS,
  // 3 = at the PMA. Higher numbers close the loop FURTHER DOWN, so each
  // level includes everything the level above it exercised.
  input  logic [1:0]       loop_depth,
 
  // Transmit taps, one per depth.
  input  logic             mac_tx_valid,
  input  logic [WIDTH-1:0] mac_tx_data,
  input  logic             pcs_tx_valid,
  input  logic [WIDTH-1:0] pcs_tx_data,
  input  logic             pma_tx_valid,
  input  logic [WIDTH-1:0] pma_tx_data,
 
  // The real medium.
  input  logic             wire_rx_valid,
  input  logic [WIDTH-1:0] wire_rx_data,
 
  // Into the receive path.
  output logic             rx_valid,
  output logic [WIDTH-1:0] rx_data,
  output logic             looped        // status: this is not real traffic
);
 
  // The depth must be STABLE while traffic is in flight. Changing it
  // mid-frame splices two different sources together and produces a frame
  // that never existed, which is a spectacularly confusing test result.
  logic [1:0] depth_q;
  logic       inflight_q;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      depth_q <= 2'd0; inflight_q <= 1'b0;
    end else begin
      inflight_q <= mac_tx_valid || pcs_tx_valid || pma_tx_valid;
      if (!inflight_q) depth_q <= loop_depth;   // change only when quiet
    end
  end
 
  always_comb begin
    case (depth_q)
      2'd1: begin rx_valid = mac_tx_valid; rx_data = mac_tx_data; end
      2'd2: begin rx_valid = pcs_tx_valid; rx_data = pcs_tx_data; end
      2'd3: begin rx_valid = pma_tx_valid; rx_data = pma_tx_data; end
      default: begin rx_valid = wire_rx_valid; rx_data = wire_rx_data; end
    endcase
  end
 
  // A loopback that is not obviously a loopback is a trap: a system that
  // reports healthy traffic while its cable is disconnected has been left
  // in loopback, and this bit is the only evidence.
  assign looped = (depth_q != 2'd0);
 
endmodule

Classification: synthesizable.

What it teaches: that loopback is a bisection instrument, and the depth is the bisection point. Each level includes everything above it, so the result is read as a boundary rather than as a pass or fail:

The method is to work down until it fails. A MAC loopback that passes and a PMA loopback that fails localises the fault to the PCS or PMA — two layers instead of six, from two tests.

The depth_q stability rule is not defensive clutter. Changing depth while a frame is in flight splices the first half of one source onto the second half of another, producing a frame that was never transmitted by anything. The result is a failure that reproduces only when the test script changes the register at the wrong moment, which is close to the hardest thing to debug.

And looped earns its status bit. A device left in loopback reports healthy traffic with its cable disconnected. Without an explicit indication, the only symptom is that the far end sees nothing while this end sees everything working — and the natural conclusion is that the far end is broken.

Deliberately simplified: one direction; no separate near-end and far-end loopback, which are different tests; no timing effect, where a real loopback at the PMA changes latency substantially; no automatic exit.

Production implication: a real device distinguishes near-end loopback (this station's transmit into its own receive) from far-end loopback (the peer's receive back onto its transmit), because they exercise opposite halves of the link; exposes the active depth in a status register; and often implements a timeout so a device cannot be left looped indefinitely by a script that crashed.

10. Waveform — One Octet Down and Back

Three readings.

The tag is 07 at cycle 1 and 07 at cycle 7. Four transmit stages and a receive path in between, and no stage altered it. That is Section 2's invariant, observed.

rx_hunt is high for the first six cycles and the receiver produces nothing. It is working — searching — and from outside it is indistinguishable from a receiver with nothing to do. Nothing in the trace before cycle 6 tells you whether this link has traffic it cannot recognise or no traffic at all.

The busy signals move one stage per cycle and never overlap. With one octet in flight the pipeline is almost entirely empty, which is the normal state of a lightly loaded path and the reason latency and throughput must be measured separately — Chapter 8.4 develops that.

11. What Goes Wrong, By Direction

The two directions fail in different ways, and separating them is most of a diagnosis.

Transmit failures are bookkeeping failures. A field placed wrongly, a length miscounted, a gap too short, a stage that dropped a beat. They are deterministic, reproducible in simulation, and usually visible as a malformed frame at the far end.

Receive failures are discovery failures, and they have a signature transmit faults do not: they can produce silence. A receiver that never finds a frame start reports nothing at all — no error, no counter, no event. It looks exactly like a link with no traffic on it.

The third row is why receive-side observability has to be designed in. A hunt state with no counter is a fault with no evidence. The fix is cheap — count entries into the hunt state, and count how long it has been occupied — and without it the only diagnostic is "nothing is happening", which is also what a healthy idle link looks like.

12. Assertions

Invariants of these models. None is an IEEE requirement.

// SVA over tx_journey, rx_journey and identity_checker.
 
// SAFETY — P1: identity is preserved at every stage. The chapter's central
// invariant; a failure means a stage modified something it must only carry.
property p_tag_preserved;
  @(posedge clk) disable iff (!rst_n)
  (v_q[0] && rdy[1]) |=> (tag_q[1] == $past(tag_q[0]));
endproperty
a_tag_preserved : assert property (p_tag_preserved);
 
// SAFETY — P2: no stage accepts while it is full and cannot empty. Catches
// the ready chain being broken, which silently overwrites a held beat.
property p_no_overwrite;
  @(posedge clk) disable iff (!rst_n)
  (v_q[1] && !rdy[2]) |=> $stable(d_q[1]);
endproperty
a_no_overwrite : assert property (p_no_overwrite);
 
// CONSERVATION — P3: one beat in produces exactly one beat out. The
// pipeline neither duplicates nor drops.
property p_beat_conserved;
  @(posedge clk) disable iff (!rst_n)
  (in_valid && in_ready) |-> ##[1:$] (out_valid && out_ready);
endproperty
a_beat_conserved : assert property (p_beat_conserved);
 
// SAFETY — P4: the receiver never delivers while hunting. Delivering from
// HUNT would mean emitting octets that were never recognised as a frame.
property p_no_deliver_while_hunting;
  @(posedge clk) disable iff (!rst_n)
  (state_q == R_HUNT) |-> !cli_valid;
endproperty
a_no_deliver_hunting : assert property (p_no_deliver_while_hunting);
 
// SAFETY — P5: exactly one verdict per received frame. Merged verdicts make
// the counters undiagnosable, which Section 11's table depends on.
property p_one_verdict;
  @(posedge clk) disable iff (!rst_n)
  $onehot0({rx_ok, rx_runt, rx_giant, rx_phy_error});
endproperty
a_one_verdict : assert property (p_one_verdict);
 
// SAFETY — P6: a giant is detected rather than allowed to run forever. The
// counter must saturate and the state machine must leave RECEIVE.
property p_giant_terminates;
  @(posedge clk) disable iff (!rst_n)
  (state_q == R_RECEIVE && len_q == MAX_BEATS) |=> (state_q == R_DECIDE);
endproperty
a_giant_terminates : assert property (p_giant_terminates);
 
// SAFETY — P7: identity failures are reported separately. Corruption,
// duplication and loss point at three different bugs.
property p_identity_failures_distinct;
  @(posedge clk) disable iff (!rst_n)
  $onehot0({mismatch, overrun});
endproperty
a_identity_distinct : assert property (p_identity_failures_distinct);
 
// CAUSATION — P8: a stall counter only increments when a stage is holding
// something it cannot pass. Catches busy being counted as stalled, which
// makes the Section 8 gradient meaningless.
property p_stall_needs_backpressure;
  @(posedge clk) disable iff (!rst_n)
  stage_stalled[1] |-> (stage_busy[1] && !rdy[2]);
endproperty
a_stall_needs_bp : assert property (p_stall_needs_backpressure);
 
// LIVENESS — P9: a receiver eventually leaves the hunt state. ASSUMPTION,
// stated: a delimiter eventually arrives. Without it this is false for a
// correct design on an idle link, which is the whole point of Section 10's
// third row.
assume property (@(posedge clk) s_eventually (phy_valid && phy_data == SFD));
property p_hunt_terminates;
  @(posedge clk) disable iff (!rst_n)
  (state_q == R_HUNT) |-> s_eventually (state_q == R_RECEIVE);
endproperty
a_hunt_terminates : assert property (p_hunt_terminates);

The property that must not be written

// FALSE for a correct design. Included as a warning, not as a check.
// property p_receiver_always_makes_progress;
//   @(posedge clk) disable iff (!rst_n)
//   !cli_valid |-> ##[1:100] cli_valid;
// endproperty

It reads like a liveness requirement on a working receiver, and it asserts something the design cannot control.

A receiver in R_HUNT produces nothing because nothing has arrived. That is correct behaviour on an idle link, and it is indistinguishable — from inside the design — from a link whose traffic cannot be recognised. Asserting progress makes the property fail on every quiet interval.

The correct form is P9, with its assumption written down: given that a delimiter eventually arrives, the receiver eventually leaves the hunt state. That is a statement about the design, and it fails only if the detection logic is broken.

The instructive part is what the wrong version reveals. An engineer who writes it has not internalised that a receiver's silence is ambiguous, and that ambiguity is exactly what Section 11's third row and this chapter's thesis are about. Waiving the property when it fires on idle traffic also removes P9, which is the one that would catch a delimiter comparison that never matches.

13. Verification

Monitors observe: every stage handshake and its occupancy; the tag at ingress and egress; the receive state, length counter and all four verdicts; and the instrument's stall counters against the ready chain.

The scoreboard independently predicts the egress octet sequence from the ingress sequence, and the expected verdict from the offered frame's length and error injection. It must compute the expected sequence itself rather than reading tag_q — a checker reading the design's own tag agrees with it about every tag bug.

Scenarios

1. Single octet, no backpressure. Verify it traverses in exactly STAGES cycles and the tag is intact.
2. Continuous stream, no backpressure. Verify full rate with no bubbles and no dropped beats (P3).
3. Backpressure at the output. Hold out_ready low for one cycle, then many. Verify the ready chain propagates, held beats are stable (P2), and ingress stalls rather than overwriting.
4. Backpressure released mid-stream. Verify the pipeline resumes at full rate without a lost beat.
5. Receiver hunting with no traffic. Verify it stays in R_HUNT, produces nothing, and asserts no verdict — the case P9's assumption excludes.
6. Delimiter arrives. Verify the transition to R_RECEIVE and that the delimiter itself is not delivered as payload.
7. Delimiter-valued octet inside a payload. Offer the delimiter value mid-frame. Verify the receiver does not restart — it is only hunting when it is hunting. The adversarial case that matters most.
8. Frame ends below the minimum. Verify rx_runt, exactly one verdict (P5), and clean return to hunting.
9. Frame never ends. Verify rx_giant, that the length counter saturates rather than wrapping, and that R_RECEIVE is left (P6).
10. Physical-layer error mid-frame. Verify rx_phy_error wins over length-based verdicts, and that the priority is deterministic.
11. Back-to-back frames with a minimal gap. Verify the receiver returns to hunting in time and the second frame is not missed.
12. Identity corruption injected. Force one stage to alter a beat. Verify mismatch and not overrun (P7).
13. Identity duplication injected. Force a stage to emit twice. Verify overrun and not mismatch.
14. Stall localisation. Stall the last stage and verify the localiser blames it rather than the first stage, whose counter is highest.
15. Reset in each receive state and mid-pipeline. Verify no stale tag, no partial frame delivered, and a clean first frame afterwards.

Coverage

Cross backpressure duration against pipeline occupancy at every depth from empty to full. Cover received lengths at MIN-1, MIN, MAX-1, MAX and beyond. Cover the delimiter value appearing at each position in a payload. Cover each verdict, and each pair of conditions that could produce two verdicts, so P5's one-hot property is meaningful.

A directed stimulus for the delimiter-in-payload case

// NON-SYNTHESIZABLE — directed stimulus. Proves the receiver's state gating
// in both directions: it must NOT restart mid-frame, and it MUST start when
// genuinely hunting.
task automatic delimiter_in_payload_then_hunting();
  int unsigned len_before;
 
  // 1. A valid frame whose payload contains the delimiter value.
  send_delimiter();
  send_octet(8'h11);
  send_octet(WIDTH'(SFD));      // the trap: valid payload, delimiter value
  send_octet(8'h22);
  len_before = dut.len_q;
  assert (dut.state_q == dut.R_RECEIVE)
    else $error("receiver restarted on a delimiter value inside a payload");
  assert (dut.len_q > 0)
    else $error("length counter was reset mid-frame");
  end_frame();
  @(posedge clk);
  assert (dut.rx_ok || dut.rx_runt)
    else $error("frame did not reach a verdict");
 
  // 2. Now the receiver IS hunting. The same value must be recognised.
  wait (dut.state_q == dut.R_HUNT);
  send_octet(WIDTH'(SFD));
  @(posedge clk);
  assert (dut.state_q == dut.R_RECEIVE)
    else $error("receiver failed to start on a genuine delimiter");
endtask

Both halves are needed and neither is sufficient. A design that never restarts passes part one and fails part two — it has broken its detection. A design that always restarts passes part two and fails part one. Only the pair distinguishes correct gating from either failure, which is why the task does both rather than being split.

14. Debugging — Which Half of the Journey

The direction split from Section 11, as a method.

Ask first whether anything is arriving at all. A receiver with no counters moving is either idle or hunting, and those are different. If the receive path exposes a hunt-state counter, read it: entries into hunt with no completions means traffic is arriving and cannot be recognised. If it does not, that is the observability gap Section 11 named, and the next check is one layer down — is the link even up.

Then ask which direction is failing. Transmit and receive share almost nothing, so a fault in one is strong evidence against the other. If the far end reports damaged frames while this end receives cleanly, the fault is in this transmit path; if the reverse, in this receive path.

Then localise by gradient, not by maximum. Section 8: stalls accumulate upstream of their cause. The stage with the highest stall count is almost always innocent; the bottleneck is immediately below the last stage with an elevated count.

And when the identity check fails, read which of the three flags fired. A mismatch is a datapath corruption. An overrun is a handshake accepting twice. A shortfall is a stage dropping. Three flags, three investigations, and merging them into one "bad" signal costs all of that.

15. Common Misconceptions

"Receive is just transmit backwards."

The wrong model: the stack is symmetric, so the two paths are mirror images and roughly equal work.

What it costs: the receive path is under-resourced in schedule and in verification, and its discovery states are under-tested. It is also why receive-side observability is so often missing — nobody planned for a state that produces no output.

The corrected model: every layer's transmit side decides and its receive side discovers. Discovery can fail, can succeed wrongly, and takes a variable amount of time — three properties a decision does not have. Section 5's table shows it happening at four layers, not one.

"A receiver producing nothing means the link is idle."

The wrong model: no output means no input.

What it costs: the most expensive misdiagnosis in this chapter. A receiver stuck hunting looks exactly like an idle link, so the investigation goes to the far end or the traffic generator while the fault is local — and there is no counter contradicting the assumption because nobody added one.

The corrected model: silence is ambiguous. It means either nothing arrived or nothing arriving could be recognised, and distinguishing them requires observability that has to be designed in: count entries into the hunt state and how long it has been occupied. Without that, "nothing is happening" is the only available reading and it is compatible with both.

"The stage with the most stalls is the bottleneck."

The wrong model: stall counters point at the problem; take the maximum.

What it costs: the wrong stage is optimised, the bottleneck is untouched, and the measurement appears to confirm the change because relieving an innocent stage moves the counters around. It fails confidently, which is worse than failing loudly.

The corrected model: a stage stalls when it is holding something it cannot hand onward, so the bottleneck itself does not stall — everything behind it does. Read the profile as a gradient and blame the stage immediately below the last one with an elevated count. Section 8's module is that rule; CXL's system-view chapter reaches the same conclusion for a different fabric.

"Each layer wraps the frame in a header, so the journey is encapsulation."

The wrong model: Ethernet's stack nests headers the way a protocol stack does.

What it costs: the PCS is expected to be frame-aware, coding expansion is mistaken for framing, and the length arithmetic at every boundary comes out wrong.

The corrected model: only the MAC adds anything frame-shaped. Below it the PCS codes — it expands the stream and inserts idle and alignment markers that are not headers, are addressed to nothing, and do not survive past the peer PCS. The PMA and PMD add no data at all; they change representation. Section 2's table separates the three operations, and Chapter 2.1 §15 develops why conflating them matters.

16. Interview Reasoning

17. Understanding Check

18. What's Next

A frame's journey is twelve steps down and twelve back, and the two halves are not mirror images. Transmit decides at every layer; receive discovers at every layer, and discovery can fail, can succeed wrongly, and takes a variable time. What survives the round trip is identity: the same octets, in order, after each layer removes what its peer added.

That journey only works because each layer refuses to know what its neighbours are doing. Chapter 2.3 — Layering as an Engineering Contract takes that refusal seriously: what the contract costs, what it buys in reuse and verification, and what happens to a design that quietly violates it for a local optimisation.

Chapter 2.4 then pins down where Ethernet stops altogether — what a MAC deliberately does not do, and why the payload is opaque to it.

The full path is on the Ethernet curriculum index.