Ethernet System Architecture

Module 1 answered why Ethernet exists and why it is shaped the way it is. Every mechanism it examined — contention, slot time, packet switching, switching, full duplex — was about sharing a medium, and Chapter 1.5 ended with most of that machinery deleted.

What survived is what Ethernet actually is: a way to put a frame on a wire and get it off again. That is not one block. It is a stack of them, each with a job so narrowly defined that it can be replaced without touching its neighbours — which is the property Chapter 1.6 argued was worth more than any single technical feature.

What are the blocks between a client with data and a physical medium, and what exactly does each one promise the others?

The promises are the content. A block diagram anyone can draw; the port lists and the forbidden knowledge are what make the architecture work.

1. The Stack

   CLIENT                          octets, and an intent to send them
     │  MAC service interface      "here is a frame; tell me when it is gone"
     ▼
   MAC                             framing, addressing, FCS, transmit access
     │  MAC / PLS interface        octets and control, in bit times
     ▼
   RECONCILIATION SUBLAYER (RS)    maps that onto the physical interface
     │  xMII                       a defined width, clock and control encoding
     ▼
   PCS                             coding, block boundaries, lane alignment
     │  PCS / PMA interface        a coded symbol stream
     ▼
   PMA                             serialisation, clock recovery
     │  PMA / PMD interface        a serial stream
     ▼
   PMD                             drivers, receivers, the connector
     │
     ▼
   MEDIUM                          copper pairs, fibre, a backplane trace

Six blocks and five interfaces. The interfaces matter more than the blocks, and Chapter 1.6 showed why: the blocks have been replaced repeatedly and the interfaces have not.

Two things to fix before any detail.

Not every layer is a separate chip, and the stack is a specification, not a floorplan. A modern integrated device may put the MAC, RS, PCS and PMA on one die with only the PMD outside. The layering still holds, because the interfaces are what the specification defines — where the silicon boundary falls is an implementation decision.

The stack is symmetric and the two directions are not. Everything above happens in reverse on receive, but the receive path is harder in a way that recurs at every layer: transmit decides, receive must discover. A transmitter knows when a frame starts because it started it; a receiver must find out. Chapter 2.2 traces both directions in full.

2. The MAC — What It Owns

The MAC's job is everything about the frame that does not depend on the medium.

And what it is forbidden to know, which is the more useful half:

• The medium. Nothing in the MAC names copper, fibre, a connector or a wavelength.
• The rate, except as a count of bit times. Chapter 1.6 §6 demonstrated one MAC source across three physical layers precisely because of this.
• The line code. Whether the bits become 4B/5B, 64B/66B or PAM4 symbols is entirely below it.
• How many lanes there are. Lane striping is the PCS's business.

That list of prohibitions is the architecture. Every item on it is something that changed several times while the MAC did not.

3. RTL 1 — The MAC's Client Interface

The topmost contract is the one most often got wrong, because it looks like a simple handshake and carries an ownership rule.

// SYNTHESIZABLE. The client-to-MAC contract: offer a frame, transfer
// ownership of it, learn what happened to it.
//
// NOT a MAC. No framing, FCS, addressing or padding.
module mac_client_port #(
  parameter int unsigned WIDTH = 8
) (
  input  logic clk,
  input  logic rst_n,
 
  // Client offers a frame, one beat at a time, with an explicit end.
  input  logic             cli_valid,
  input  logic [WIDTH-1:0] cli_data,
  input  logic             cli_last,
  output logic             cli_ready,
 
  // Completion, returned LATER and out of band with the data. A frame's
  // fate is not known when its last beat is accepted — Chapter 1.2 showed
  // it may still take sixteen attempts and then fail.
  output logic             cmp_valid,
  output logic             cmp_ok,
  output logic             cmp_excessive_collision,
  output logic             cmp_underrun,
 
  // Toward the MAC core.
  output logic             core_valid,
  output logic [WIDTH-1:0] core_data,
  output logic             core_last,
  input  logic             core_ready,
  input  logic             core_done,
  input  logic             core_ok,
  input  logic             core_excess,
  input  logic             core_underrun
);
 
  typedef enum logic [1:0] { C_IDLE, C_XFER, C_WAIT } c_state_e;
  c_state_e state_q, state_d;
 
  // cli_ready does not depend on cli_valid — no combinational path from the
  // client's valid back to its own ready. The same discipline the whole
  // track uses, and the one that makes two blocks composable without
  // creating a loop.
  assign cli_ready = (state_q == C_XFER || state_q == C_IDLE) && core_ready;
 
  wire beat   = cli_valid && cli_ready;
  wire finish = beat && cli_last;
 
  always_comb begin
    state_d = state_q;
    case (state_q)
      C_IDLE: if (beat) state_d = finish ? C_WAIT : C_XFER;
      C_XFER: if (finish) state_d = C_WAIT;
      // THE OWNERSHIP RULE. The client may not offer another frame until
      // this one's fate is known. A MAC that accepted a second frame while
      // the first was still being retried would have two frames in flight
      // and one completion channel — and the client could not tell which
      // frame a completion referred to.
      C_WAIT: if (core_done) state_d = C_IDLE;
      default: state_d = C_IDLE;
    endcase
  end
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) state_q <= C_IDLE;
    else        state_q <= state_d;
  end
 
  assign core_valid = cli_valid && (state_q != C_WAIT);
  assign core_data  = cli_data;
  assign core_last  = cli_last;
 
  // Completion is a PULSE with a reason, not a level with a flag. A client
  // that must poll a status register cannot tell two consecutive identical
  // outcomes apart.
  assign cmp_valid               = (state_q == C_WAIT) && core_done;
  assign cmp_ok                  = cmp_valid && core_ok;
  assign cmp_excessive_collision = cmp_valid && core_excess;
  assign cmp_underrun            = cmp_valid && core_underrun;
 
endmodule

Classification: synthesizable.

What it teaches: that the client interface is not a data path with a completion bolted on — it is an ownership transfer with a deferred result. The client hands over a frame, loses the right to touch it, and finds out later whether it was delivered. That shape is forced by Chapter 1.2: a frame's fate is genuinely unknown when its last octet is accepted.

cmp_underrun is the output worth explaining. It reports that the client failed to supply data fast enough once transmission had begun. This is a client fault reported by the MAC, and it exists because transmission cannot be paused mid-frame — the medium does not tolerate a gap in the middle of a frame. A client that starves the MAC produces a truncated frame on the wire, which the far end discards as damaged, and without this output the client would have no idea it was the cause.

Deliberately simplified: one frame in flight, where a real MAC pipelines several with per-frame completion identifiers; no priority or class; the receive direction is absent; no descriptor model, which Module 18 owns.

Production implication: a real interface carries a frame identifier through to completion so several frames can be outstanding; separates the completion channel so a slow client cannot stall the transmit path; and defines precisely what the client may do with the buffer after the last beat is accepted but before completion arrives — which is the source of a whole class of driver bugs.

4. The Reconciliation Sublayer — The Adapter Nobody Names

The RS is the least-known block in the stack and it is why the stack works.

The MAC's service is defined in abstract terms: octets, control indications, bit times. The physical interface is concrete: a specific width, a specific clock, a specific encoding of control. The reconciliation sublayer maps one onto the other, and it exists so that neither has to change when the other does.

Concretely, it is the block that turns "the MAC is transmitting these octets" into "these data lines carry these values with this enable asserted at this clock rate", in whichever xMII variant is attached.

Why this matters more than it looks. Without the RS, the MAC's definition would have to name a width and a clocking scheme — and there would then be a different MAC for MII, for GMII, for XGMII. With it, there is one MAC and several reconciliation mappings. Module 10 covers the six interface variants; the RS is why they are variants of an interface rather than variants of a MAC.

5. RTL 2 — The MAC-to-RS Boundary

// SYNTHESIZABLE. Maps the MAC's abstract octet service onto a concrete
// parallel interface of a given width.
//
// NOT an xMII implementation. No real control encoding, no DDR, no
// source-synchronous clocking. Module 10 owns those.
module reconciliation_sublayer #(
  parameter int unsigned MAC_W = 8,   // the MAC's service width
  parameter int unsigned PHY_W = 4    // the attached interface's width
) (
  input  logic clk,
  input  logic rst_n,
 
  // From the MAC: octets and control, no medium knowledge.
  input  logic             mac_tx_en,
  input  logic [MAC_W-1:0] mac_tx_data,
  input  logic             mac_tx_er,     // transmit an error indication
  output logic             mac_tx_ready,
 
  // Toward the PHY: a defined width with a defined enable.
  output logic             phy_tx_en,
  output logic [PHY_W-1:0] phy_tx_data,
  output logic             phy_tx_er
);
 
  localparam int unsigned RATIO = MAC_W / PHY_W;
  localparam int unsigned SEL_W = (RATIO <= 1) ? 1 : $clog2(RATIO);
 
  // Width conversion is the RS's whole job in this direction, and the reason
  // the MAC does not have to know the interface width. Four bits at a time
  // for MII, eight for GMII, thirty-two for XGMII — the MAC's service is
  // unchanged and only this ratio moves.
  logic [MAC_W-1:0] hold_q;
  logic [SEL_W-1:0] sel_q;
  logic             busy_q, er_q;
 
  wire last_slice = (sel_q == SEL_W'(RATIO - 1));
 
  // The MAC is held off while a previous octet is still being emitted. This
  // is the ONLY backpressure in the transmit direction, and it exists
  // because the MAC's service is octet-granular while the interface is not.
  assign mac_tx_ready = !busy_q || last_slice;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      hold_q <= '0; sel_q <= '0; busy_q <= 1'b0; er_q <= 1'b0;
    end else begin
      if (mac_tx_en && mac_tx_ready) begin
        hold_q <= mac_tx_data;
        er_q   <= mac_tx_er;
        sel_q  <= '0;
        busy_q <= 1'b1;
      end else if (busy_q) begin
        if (last_slice) busy_q <= 1'b0;
        else            sel_q  <= sel_q + 1'b1;
      end
    end
  end
 
  // Least-significant slice first. The ORDER is a specification decision,
  // not a free choice: both ends must agree, and getting it backwards
  // produces a link that passes every local test and garbles every frame.
  assign phy_tx_en   = busy_q;
  assign phy_tx_data = hold_q[sel_q*PHY_W +: PHY_W];
  assign phy_tx_er   = busy_q && er_q;
 
  // Elaboration-time check: an interface wider than the service, or a ratio
  // that is not an integer, is a configuration error rather than a design
  // one — and it passes every functional test if not caught here.
  if (MAC_W % PHY_W != 0) begin : g_bad_ratio
    $error("MAC_W (%0d) must be an integer multiple of PHY_W (%0d).", MAC_W, PHY_W);
  end
endmodule

Classification: synthesizable, with an elaboration-time parameter check.

What it teaches: that the reconciliation sublayer is where a rate and width negotiation happens without either neighbour knowing it occurred. The MAC offers octets and is told when to wait; the interface receives slices at its own width. Neither has a parameter naming the other's.

The slice-order comment is not pedantry. Bit and octet ordering across an interface is a specification decision that both ends must share, and getting it wrong produces the most frustrating class of bring-up failure: the link comes up, both sides report health, every local test passes, and every frame is garbled. It is a specification-reading problem rather than a logic problem, which is why it survives design review.

mac_tx_er is worth noticing. The MAC can ask the physical layer to transmit an error indication — a deliberate corruption. This exists so a repeater or a store-and-forward device that discovers a frame is bad mid-transmission can mark it rather than emitting a plausible-looking truncated frame. It is a small port on a small block that encodes a real architectural decision: it is better to be visibly wrong than invisibly short.

Deliberately simplified: transmit only; no DDR or source-synchronous clocking; no real control-character encoding; no clock-domain crossing, which a real RS very often has.

Production implication: a real RS handles both directions, crosses clock domains between the MAC and the interface, encodes control characters per the specific xMII variant, and has its width and clocking selected by the same configuration that selects the rate — because a width and a rate that disagree produce exactly the garbled-frame failure above.

6. Below the Interface — PCS, PMA, PMD

Module 3 owns these in depth. What belongs here is the division of labour, because the three are routinely collapsed into "the PHY" and they solve different problems.

The PCS is the one worth knowing about now, because two of its jobs surface repeatedly in later modules.

Block synchronisation. A coded stream has block boundaries that a receiver must find, having been given no external marker. This is a discovery problem of exactly the kind Section 1 noted the receive path is full of, and Chapter 3.5 owns it.

Idle insertion and deletion. The two ends of a link run on independent clocks that are nominally equal and are not exactly equal. The PCS absorbs the difference by adding or removing idle between frames — which is why the interframe gap, Chapter 1.5 showed surviving into full duplex, is a minimum rather than a fixed value. Chapter 4.4 owns the elastic buffer that does it.

And the reason to name them separately at all is that a modern link replaces them independently. The same PCS serves several PMDs; a PMD change for a different reach leaves the PCS untouched. Collapsing them into "the PHY" hides which one a given change affects.

7. The Management Path, Which Is Not the Datapath

Everything above describes how a frame travels. There is a second path through the same stack that carries no frames at all, and leaving it out of the picture is one of the more common ways to misunderstand an Ethernet port.

The physical layer has to be configured and interrogated. Which speed to operate at, whether to auto-negotiate, what the link status is, how many symbol errors have been seen, whether the far end is present. None of that is data, and none of it can travel over the datapath — the datapath may not be working, which is precisely when the answers are needed most.

So there is a separate management interface: MDC and MDIO, a clock and a bidirectional data line, running at a rate unrelated to the link rate and reaching a register space inside the PHY. Chapter 4.5 owns it in detail.

Three architectural consequences belong here.

It is out of band, and that is the point. A management path that shared the datapath would be unusable exactly when it mattered: a link that will not come up cannot be interrogated over itself. The separation is not tidiness, it is the requirement.

It crosses layers that the datapath does not. Software, or a hardware state machine, reaches past the MAC and the RS directly into the PHY's registers. That is a deliberate exception to Section 11's prohibition table — the management path is allowed to know things the datapath is not, because its job is to configure exactly those things.

It is where configuration and reality can disagree. The mode register in Chapter 1.5 §9, the negotiated speed and duplex in Chapter 11.2, and the width configuration in Section 6 all arrive through this path — and every one of them is a value that must match something at the far end or inside a neighbouring block. Most of the failures in Section 14's debugging table are configuration failures, and configuration arrives here.

8. RTL 3 — The Stack as a Hierarchy

The clearest statement of an architecture is a hierarchy whose port lists are correct, even where the bodies are not written yet.

// ILLUSTRATIVE SKELETON. Port lists are the content. Bodies are omitted.
// This does not implement Ethernet; it states the interfaces.
 
// ── The MAC. Note what its port list does NOT contain: any signal naming a
//    medium, a rate, a wavelength, a lane count or a line code.
module eth_mac #(
  parameter int unsigned CLIENT_W = 8,
  parameter int unsigned IFG_BITS = 96,     // NORMATIVE, every rate
  parameter int unsigned MIN_BITS = 512     // NORMATIVE, every rate
) (
  input  logic clk, rst_n,
  input  logic bit_tick,                    // the ONLY rate dependence
 
  // Client side.
  input  logic                cli_valid,
  input  logic [CLIENT_W-1:0] cli_data,
  input  logic                cli_last,
  output logic                cli_ready,
  output logic                cli_done, cli_ok,
 
  // Toward the reconciliation sublayer. Octets and control, nothing else.
  output logic                mac_tx_en,
  output logic [CLIENT_W-1:0] mac_tx_data,
  output logic                mac_tx_er,
  input  logic                mac_tx_ready,
 
  input  logic                mac_rx_dv,
  input  logic [CLIENT_W-1:0] mac_rx_data,
  input  logic                mac_rx_er
);
endmodule
 
// ── The RS. Knows the MAC's service width and the interface width, and
//    nothing about what the octets mean.
module eth_rs #(
  parameter int unsigned MAC_W = 8,
  parameter int unsigned PHY_W = 4
) (
  input  logic clk, rst_n,
  input  logic             mac_tx_en,
  input  logic [MAC_W-1:0] mac_tx_data,
  input  logic             mac_tx_er,
  output logic             mac_tx_ready,
  output logic             phy_tx_en,
  output logic [PHY_W-1:0] phy_tx_data,
  output logic             phy_tx_er,
  input  logic             phy_rx_dv,
  input  logic [PHY_W-1:0] phy_rx_data,
  input  logic             phy_rx_er,
  output logic             mac_rx_dv,
  output logic [MAC_W-1:0] mac_rx_data,
  output logic             mac_rx_er
);
endmodule
 
// ── The PCS. Deals in code blocks and lanes. NO frame concept appears in
//    this port list, which is the point: it cannot know where a frame
//    starts, and it does not need to.
module eth_pcs #(
  parameter int unsigned PHY_W  = 4,
  parameter int unsigned LANES  = 1,
  parameter int unsigned BLOCK_W = 10       // illustrative coded width
) (
  input  logic clk, rst_n,
  input  logic              phy_tx_en,
  input  logic [PHY_W-1:0]  phy_tx_data,
  input  logic              phy_tx_er,
  output logic [BLOCK_W-1:0] pma_tx_block [LANES],
  input  logic [BLOCK_W-1:0] pma_rx_block [LANES],
  output logic              phy_rx_dv,
  output logic [PHY_W-1:0]  phy_rx_data,
  output logic              phy_rx_er,
  output logic              block_lock,     // boundaries found
  output logic              lane_aligned    // lanes deskewed
);
endmodule
 
// ── The PMA. Serialises and recovers a clock. It does not know what the
//    blocks mean, only how wide they are.
module eth_pma #(
  parameter int unsigned LANES   = 1,
  parameter int unsigned BLOCK_W = 10
) (
  input  logic clk, rst_n,
  input  logic [BLOCK_W-1:0] pcs_tx_block [LANES],
  output logic [BLOCK_W-1:0] pcs_rx_block [LANES],
  output logic               serial_tx [LANES],
  input  logic               serial_rx [LANES],
  output logic               rx_clk_locked,  // clock recovered
  output logic               signal_detect
);
endmodule
 
// ── The top level. The wiring IS the architecture.
module eth_port #(
  parameter int unsigned CLIENT_W = 8,
  parameter int unsigned PHY_W    = 4,
  parameter int unsigned LANES    = 1
) (
  input  logic clk, rst_n, bit_tick,
  input  logic                cli_valid,
  input  logic [CLIENT_W-1:0] cli_data,
  input  logic                cli_last,
  output logic                cli_ready, cli_done, cli_ok,
  output logic                serial_tx [LANES],
  input  logic                serial_rx [LANES],
  output logic                link_up
);
  logic mac_tx_en, mac_tx_er, mac_tx_ready, mac_rx_dv, mac_rx_er;
  logic [CLIENT_W-1:0] mac_tx_data, mac_rx_data;
  logic phy_tx_en, phy_tx_er, phy_rx_dv, phy_rx_er;
  logic [PHY_W-1:0] phy_tx_data, phy_rx_data;
  logic [9:0] tx_block [LANES], rx_block [LANES];
  logic block_lock, lane_aligned, rx_clk_locked, signal_detect;
 
  eth_mac #(.CLIENT_W(CLIENT_W)) u_mac (.*);
  eth_rs  #(.MAC_W(CLIENT_W), .PHY_W(PHY_W)) u_rs (.*);
  eth_pcs #(.PHY_W(PHY_W), .LANES(LANES)) u_pcs (
    .pma_tx_block(tx_block), .pma_rx_block(rx_block), .*);
  eth_pma #(.LANES(LANES)) u_pma (
    .pcs_tx_block(tx_block), .pcs_rx_block(rx_block), .*);
 
  // Link status is built from the BOTTOM UP, and each term belongs to a
  // different layer. A link is not up because one thing is true; it is up
  // because every layer below has satisfied its own condition. This is why
  // Chapter 21.4's link-failure debugging works by descending the stack.
  assign link_up = signal_detect && rx_clk_locked && block_lock && lane_aligned;
endmodule

Classification: illustrative skeleton; elaborates, implements nothing.

What it teaches: read the port lists for what is absent. eth_mac has no signal naming a medium, a rate, a wavelength or a lane count — its whole rate dependence is bit_tick. eth_pcs has no signal naming a frame; it cannot know where one starts, and does not need to. Each absence is a contract clause.

link_up is the most instructive line in the file. It is a conjunction of four terms from three different layers: signal detect from the PMA's view of the medium, clock recovery from the PMA, block lock from the PCS, lane alignment from the PCS. A link is up because every layer below has satisfied its own condition, and each term fails for a different physical reason. That is exactly why debugging a link that will not come up means descending the stack term by term, which Chapter 21.4 builds into a method.

LANES appearing in the PCS and PMA and not in the MAC is the architecture in one parameter. Lane count is a physical-layer property; a MAC that had a lane parameter would need re-verification for every width change, which is the cost Chapter 1.6 argued the interface exists to avoid.

Deliberately simplified: no bodies; a single clock where a real port has several; .* connections that a real design would write explicitly; no management interface, which Chapter 4.5 owns and which is a whole separate path.

Production implication: a real port has a management interface entirely separate from the datapath, several clock domains with explicit crossings, per-layer status and error counters, and a defined reset ordering — because layers must come up bottom-first and a reset that releases them in the wrong order produces a link that never establishes.

9. RTL 4 — Pushing One Frame Down the Stack

A skeleton that is never exercised is a drawing. This is the smallest testbench that makes the interfaces real.

// NON-SYNTHESIZABLE. Pushes one frame from the client to the serial output
// and checks the contract at every boundary on the way down.
module tb_frame_down_the_stack;
 
  localparam int unsigned CLIENT_W = 8;
  localparam int unsigned PHY_W    = 4;
 
  logic clk = 0, rst_n = 0, bit_tick = 0;
  always #5 clk = ~clk;
 
  // A bit tick every fourth cycle, standing in for a rate. The MAC's
  // behaviour must depend on the COUNT of these, never on their period.
  int unsigned tick_div = 0;
  always @(posedge clk) begin
    tick_div <= (tick_div == 3) ? 0 : tick_div + 1;
    bit_tick <= (tick_div == 3);
  end
 
  logic cli_valid, cli_last, cli_ready, cli_done, cli_ok;
  logic [CLIENT_W-1:0] cli_data;
  logic serial_tx [1]; logic serial_rx [1]; logic link_up;
 
  eth_port #(.CLIENT_W(CLIENT_W), .PHY_W(PHY_W), .LANES(1)) dut (.*);
 
  // The frame the client offers. Deliberately below the minimum size, so
  // the padding contract is exercised rather than assumed.
  localparam int unsigned N = 8;
  logic [CLIENT_W-1:0] frame [N];
  initial for (int i = 0; i < N; i++) frame[i] = CLIENT_W'(8'hA0 + i);
 
  // ── Boundary observers. Each one checks a contract clause, and each
  //    clause is a thing a later integration mistake would break.
  int unsigned octets_from_client = 0;
  int unsigned slices_to_phy      = 0;
 
  always @(posedge clk) if (rst_n) begin
    // CLAUSE 1: the client handshake transfers exactly one octet per
    // accepted beat.
    if (cli_valid && cli_ready) octets_from_client++;
 
    // CLAUSE 2: the RS emits exactly MAC_W/PHY_W slices per octet. A ratio
    // mismatch here is the width-configuration error Section 5 guards, and
    // it produces a link that garbles every frame while reporting health.
    if (dut.phy_tx_en) slices_to_phy++;
 
    // CLAUSE 3: the MAC never asserts transmit enable while the RS is
    // holding it off. Violating this loses an octet silently.
    if (dut.mac_tx_en && !dut.mac_tx_ready)
      $error("MAC drove an octet while the RS was not ready");
 
    // CLAUSE 4: no layer below the MAC ever sees a frame boundary, because
    // no such signal crosses the xMII. Asserted structurally: if a future
    // edit adds one, this reference stops compiling, which is the intent.
    // (There is deliberately no `dut.u_pcs.frame_start` to check.)
  end
 
  task automatic send_frame();
    for (int i = 0; i < N; i++) begin
      cli_valid <= 1'b1;
      cli_data  <= frame[i];
      cli_last  <= (i == N - 1);
      @(posedge clk iff cli_ready);
    end
    cli_valid <= 1'b0; cli_last <= 1'b0;
  endtask
 
  initial begin
    cli_valid = 0; cli_last = 0; cli_data = '0;
    repeat (4) @(posedge clk); rst_n = 1;
 
    // The stack must come up BOTTOM-FIRST before anything is offered. A
    // testbench that transmits before link_up is testing an undefined state.
    wait (link_up);
 
    send_frame();
 
    // Completion arrives LATER and out of band — Section 3's ownership rule.
    @(posedge clk iff cli_done);
    assert (cli_ok) else $error("frame reported as failed");
 
    // CLAUSE 2, checked as a ratio rather than a count, so it holds for any
    // width configuration.
    assert (slices_to_phy == octets_from_client * (CLIENT_W / PHY_W))
      else $error("RS emitted %0d slices for %0d octets; expected %0d",
                  slices_to_phy, octets_from_client,
                  octets_from_client * (CLIENT_W / PHY_W));
 
    // CLAUSE 5: the MAC padded a short frame up to the minimum. The client
    // offered fewer octets than the minimum frame requires, so more must
    // have left the MAC than entered it.
    assert (slices_to_phy * PHY_W >= dut.u_mac.MIN_BITS)
      else $error("short frame was not padded to the minimum");
 
    $display("one frame crossed four interfaces: %0d client octets, %0d phy slices",
             octets_from_client, slices_to_phy);
    $finish;
  end
endmodule

Classification: non-synthesizable testbench.

What it teaches: that an architecture is verifiable before any block is implemented. Every clause above checks a contract rather than a behaviour: a conservation relationship between two interfaces, a handshake rule, an ordering requirement. All of them would still be meaningful with the real blocks in place, and all of them catch integration mistakes rather than block-internal ones.

Clause 2 is the one that earns its place. Counting slices against octets and comparing to the ratio rather than to a constant means the check holds for any width configuration — 8-to-4, 8-to-8, 32-to-32. A width mismatch between the RS's parameters and the attached interface produces exactly the failure Section 5 described: a link that reports health and garbles every frame. This assertion catches it in simulation, where the elaboration check cannot, because both parameters were individually legal.

Waiting for link_up before transmitting is not testbench hygiene. It encodes the bottom-up bring-up order Section 8 described, and a bench that transmits before the stack is up is exercising a state the specification does not define.

Deliberately simplified: transmit only; no receive path and therefore no loopback check that the frame survived; no error injection; a fixed short frame.

Production implication: a real integration bench runs frames in both directions simultaneously, loops back at each layer in turn to bisect a failure, injects errors at each boundary to check the error indication propagates, and randomises the width and lane configuration — because the parameters are the most likely thing to be wrong and the least likely to be varied.

10. Waveform — One Octet Across Two Boundaries

Three things in that trace are the chapter.

One octet becomes two slices. cli_data carries A0 once; phy_tx_data carries 0 then A — the low nibble first, then the high. The MAC produced one octet and has no representation of the fact that it left as two pieces.

mac_tx_ready falls while a slice is outstanding. That is the only backpressure in this direction, and it exists purely because the two sides count in different units. Neither side knows the other's width; the RS holds one off while it serves the other.

block_lock and link_up are high throughout, and they are the reason anything is happening at all. They are produced two layers below and consumed at the top. Nothing in this trace transmits unless the PCS has found its block boundaries — a condition the MAC cannot observe, cannot influence, and entirely depends on.

11. What Each Layer Must Not Know

The prohibitions are the architecture, so they are worth stating as a single list.

Each row is a change that has actually happened, and the prohibition is why it did not propagate.

The MAC row is the load-bearing one and Chapter 1.6 §6 demonstrated it: one MAC source across three physical layers thirty years apart.

The PCS row is the one most often violated in practice. It is tempting to give the PCS frame awareness — it would make some optimisations easy, and it is physically adjacent to the data. The moment it has one, a frame-format change reaches the physical layer, and the property that has held Ethernet together since the 1980s is gone.

12. Assertions

Invariants of this skeleton. Only the interframe gap and minimum frame size are normative.

// SVA over the architecture skeleton and its interfaces.
 
// SAFETY — P1: octet conservation across the RS. Exactly RATIO slices leave
// for each octet accepted. Catches the width-configuration error that
// garbles every frame while every status bit reports health.
property p_octet_conservation;
  @(posedge clk) disable iff (!rst_n)
  (mac_tx_en && mac_tx_ready) |-> ##1 (phy_tx_en [*RATIO]);
endproperty
a_octet_conservation : assert property (p_octet_conservation);
 
// SAFETY — P2: the MAC never drives an octet the RS cannot take. Violating
// this loses an octet with no error anywhere.
property p_no_drive_when_not_ready;
  @(posedge clk) disable iff (!rst_n)
  mac_tx_en |-> mac_tx_ready;
endproperty
a_no_drive_unready : assert property (p_no_drive_when_not_ready);
 
// SAFETY — P3: nothing transmits before the stack is up. Bottom-up bring-up
// as a property; transmitting earlier exercises an undefined state.
property p_no_tx_before_link;
  @(posedge clk) disable iff (!rst_n)
  phy_tx_en |-> link_up;
endproperty
a_no_tx_before_link : assert property (p_no_tx_before_link);
 
// CAUSATION — P4: link_up requires every layer's own condition. Catches a
// status aggregation that drops a term — which produces a link declared up
// while one layer is not ready, and frames lost with no explanation.
property p_link_needs_all_layers;
  @(posedge clk) disable iff (!rst_n)
  link_up |-> (signal_detect && rx_clk_locked && block_lock && lane_aligned);
endproperty
a_link_all_layers : assert property (p_link_needs_all_layers);
 
// SAFETY — P5: link_up falls if any layer's condition falls. The converse
// of P4, and the one that catches a latched status that never clears.
property p_link_drops_with_any_layer;
  @(posedge clk) disable iff (!rst_n)
  !(signal_detect && rx_clk_locked && block_lock && lane_aligned) |-> !link_up;
endproperty
a_link_drops : assert property (p_link_drops_with_any_layer);
 
// SAFETY — P6: completion arrives exactly once per accepted frame, and only
// after its last beat. Catches a completion pulsed early, which lets a
// client reuse a buffer that is still being transmitted.
property p_one_completion_per_frame;
  @(posedge clk) disable iff (!rst_n)
  (cli_valid && cli_ready && cli_last) |=> (!cmp_valid throughout (cmp_valid [->1]));
endproperty
a_one_completion : assert property (p_one_completion_per_frame);
 
// SAFETY — P7: the client cannot offer a new frame while one is outstanding.
// The ownership rule from Section 3; without it two frames share one
// completion channel and the client cannot tell them apart.
property p_ownership_respected;
  @(posedge clk) disable iff (!rst_n)
  (state_q == C_WAIT) |-> !core_valid;
endproperty
a_ownership : assert property (p_ownership_respected);
 
// SAFETY — P8: a transmitted frame is at least the minimum size. Normative,
// and it must hold at every width configuration — which is what makes it an
// architecture property rather than a MAC property.
property p_minimum_frame_enforced;
  @(posedge clk) disable iff (!rst_n)
  $fell(phy_tx_en) |-> ($past(slices_sent) * PHY_W >= MIN_BITS);
endproperty
a_minimum_frame : assert property (p_minimum_frame_enforced);
 
// LIVENESS — P9: an accepted frame is eventually completed. ASSUMPTION,
// stated: the stack stays up and the medium eventually permits transmission.
assume property (@(posedge clk) s_eventually (link_up));
property p_frame_completes;
  @(posedge clk) disable iff (!rst_n)
  (cli_valid && cli_ready && cli_last) |-> s_eventually (cmp_valid);
endproperty
a_frame_completes : assert property (p_frame_completes);

The property that must not be written

// FALSE for a correct design. Included as a warning, not as a check.
// property p_client_octets_equal_phy_octets;
//   @(posedge clk) disable iff (!rst_n)
//   ##1 (octets_from_client == slices_to_phy * PHY_W / 8);
// endproperty

It reads like the conservation law the stack ought to obey — what goes in comes out — and it is false at two separate boundaries.

The MAC adds octets. It prepends a preamble and start delimiter, appends an FCS, and pads a short frame up to the minimum. More leaves the MAC than entered it, by an amount that depends on the frame's length. Section 9's clause 5 exists precisely because a short frame must come out longer.

The PCS adds more. Line coding expands the stream: 8B/10B by a quarter, 64B/66B by about three percent. What reaches the medium is not the count of anything above it.

The correct conservation law is about identity, not quantity: the client's octets appear, in order and unaltered, inside what is transmitted — and the far end's client receives exactly those octets after every layer has removed what its peer added. P1 states the local version of that at one boundary, which is the form a checker can actually evaluate.

Writing the wrong version has a specific cost. It fires on the very first short frame, which is the padding case working correctly, and the natural response is to disable it — taking P1 and P8 out of attention along with it.

13. Verification

Monitors observe: the client handshake and completion channel; the MAC-to-RS handshake with its ready; the RS-to-interface slices; the four link-status terms individually; and the bit tick, so that every count can be expressed in bit times.

The scoreboard independently predicts the slice count from the octet count and the width ratio, and the transmitted length from the client length and the normative minimum. It must compute the ratio from its own configuration rather than reading the design's parameter — a checker parameterised from the design agrees with it about a misconfigured width.

Scenarios

1. One frame, well above the minimum. The baseline: verify slice conservation (P1) and one completion (P6).
2. One frame below the minimum. Verify padding, and that more leaves than entered — the case the rejected property gets wrong.
3. Frame exactly at the minimum. Verify no padding is added. The boundary either side of scenario 2.
4. Back-to-back frames. Verify a full interframe gap between them, and that completion for the first arrives before the second is accepted (P7).
5. Client stalls mid-frame. Deassert cli_valid between beats. Verify the MAC reports an underrun rather than emitting a truncated frame silently.
6. RS backpressure. Hold mac_tx_ready low. Verify the MAC waits (P2) and no octet is lost.
7. Width configurations. Elaborate at 8-to-4, 8-to-8 and 32-to-32 and run scenario 1 in each. Verify the conservation ratio holds in all three — the architecture claim, tested.
8. Illegal width ratio. Elaborate with a non-integer ratio and verify the elaboration error fires.
9. Transmit before link up. Offer a frame with link_up low. Verify nothing is transmitted (P3).
10. Each link-status term dropping individually. Four scenarios: drop signal detect, clock lock, block lock, lane alignment. Verify link_up falls in every case (P5) — a latched term that never clears is a real and common defect.
11. Link drops mid-frame. Verify the frame is aborted with an error indication rather than truncated silently.
12. Completion timing. Verify no completion arrives before the last beat is accepted, and exactly one arrives afterwards (P6).
13. Reset at each stage. Verify no stale slice, no stale completion, and a clean first frame after release.

Coverage

Cross the width configuration against every scenario, so scenario 7's claim is exercised throughout rather than once. Cover client frame lengths at 1 octet, MIN-1, MIN, MIN+1 and the maximum. Cover each link-status term dropping alone and in combination. Cover reset asserted in each state of the client port.

A directed stimulus for the width mismatch

// NON-SYNTHESIZABLE — directed stimulus. Elaborates the stack with an RS
// configured for one width and an interface expecting another, and shows
// that every status bit reports health while every frame is wrong.
task automatic width_mismatch_reports_healthy();
  // Both parameters are individually legal, so the elaboration check in
  // Section 5 cannot fire. Only the RELATIONSHIP is wrong.
  wait (link_up);
  assert (link_up)
    else $error("precondition: the stack must report up");
 
  send_frame();
  @(posedge clk iff cli_done);
 
  // The MAC is satisfied. Every layer reports its own condition met.
  assert (cli_ok)
    else $error("MAC reported failure; expected it to report success");
  assert (dut.u_pcs.block_lock && dut.u_pcs.lane_aligned)
    else $error("PCS reported a problem; expected it to report health");
 
  // AND THE FRAME IS WRONG. This is the assertion the task exists for: the
  // conservation relationship is the ONLY local evidence that anything is
  // amiss, which is why P1 is worth writing.
  assert (slices_to_phy == octets_from_client * (CLIENT_W / PHY_W))
    else $error("width mismatch: %0d slices for %0d octets — every frame on this link is garbled, and nothing else reports it",
                slices_to_phy, octets_from_client);
endtask

The two assertions that pass are as important as the one that fails. They establish that the failure is invisible to every status mechanism in the design, which is what makes this bug expensive in the field: the link is up, the counters are clean, the MAC is happy, and the far end receives nothing it can use.

14. Debugging — Descending the Stack

A link that will not carry traffic is diagnosed by descending, because each layer's condition depends on the ones below it and on nothing above.

Read the table top to bottom and stop at the first failing row. Everything below a failure is meaningless; everything above is untested.

The last row is the one with no status bit, and it is why it needs its own detection. Every layer reports its own condition met, because every layer is meeting its own condition — the mismatch is in a relationship between two of them, and no single layer can see a relationship. The conservation check in P1 is the only local evidence, which is the strongest possible argument for writing it.

And the fifth row is where the stack stops helping. Once the physical layer is healthy, the problem is a frame problem, and the descent gives way to the taxonomy Chapter 21.2 builds.

15. Common Misconceptions

"The MAC and the PHY are two chips."

The wrong model: the layering describes a partitioning of silicon.

What it costs: a modern integrated device looks like a violation of the architecture, so the layering is dismissed as legacy — and with it the reason interfaces exist. Then somebody adds a signal that couples two layers, because "they are in the same chip anyway".

The corrected model: the layering is a specification of interfaces, not of packages. A device may implement MAC, RS, PCS and PMA on one die with only the PMD outside, and the contracts still hold — indeed they hold more strictly, because an internal boundary is easier to violate and the violation is harder to see.

"The reconciliation sublayer is just glue."

The wrong model: an adapter, and adapters are accidental complexity.

What it costs: it gets absorbed into the MAC "for efficiency", and the MAC acquires a width and a clocking scheme. The next interface generation then requires MAC changes and MAC re-verification — the exact cost the architecture was built to avoid, paid once per generation forever.

The corrected model: the RS is a deliberate adapter placed where change was predicted, and the prediction was correct: interface widths and clocking schemes have changed repeatedly and every one of those changes landed on the RS. That is architecture working, not glue.

"If the link is up, the physical layer is fine."

The wrong model: link_up is a health indicator for everything below the MAC.

What it costs: the width and ordering mismatches in Section 12 are undiagnosable, because every status bit says healthy while every frame is garbled. Time is spent above the MAC on a fault that lives in a configuration relationship.

The corrected model: link_up is a conjunction of four per-layer conditions, and every one of them is about a layer meeting its own requirement. None is about two layers agreeing with each other. A width mismatch, a bit-ordering mismatch or a coding mismatch satisfies every term and breaks every frame — which is exactly why a conservation check across the boundary is worth its assertion.

"Each layer adds a header, like a protocol stack."

The wrong model: Ethernet's layers encapsulate one another the way network protocol layers do.

What it costs: the PCS is expected to be frame-aware, and the coding expansion is mistaken for framing. It also makes the stack seem more like the OSI model than it is, which Chapter 2.4 has to correct.

The corrected model: only the MAC adds anything frame-shaped — preamble, start delimiter, padding, FCS. Below it, the PCS codes rather than encapsulates: it expands the bit stream and adds control characters and alignment markers that are not headers and are not addressed to anything. The PMA and PMD add no data at all; they change representation. Encapsulation and coding are different operations, and conflating them is what makes people expect the PCS to know where a frame starts.

16. Interview Reasoning

17. Understanding Check

18. What's Next

The stack is now named: client, MAC, reconciliation sublayer, PCS, PMA, PMD, medium — six responsibilities, five interfaces, and a set of prohibitions that has held for three decades.

What this chapter has not done is follow anything through it. The blocks and their contracts are static; a frame is a sequence of events across all of them, and the receive direction is not the transmit direction reversed — transmit decides, receive must discover.

Chapter 2.2 — One Frame, End to End traces application data down through the frame, the MAC, the xMII boundary, the PCS and the PHY onto the wire, then back up the receive path at the far end. It is the chapter the rest of the track refers back to, because every later module is an expansion of one stage of that journey.

Chapter 2.3 then develops layering as an engineering contract in its own right, and Chapter 2.4 pins down where Ethernet stops and what a MAC deliberately does not do.

The full path is on the Ethernet curriculum index.