Skip to content
VLSI Mentor

Ethernet · Module 24

Ethernet against Proprietary Fabrics

A framing tax worth 0.111% on the traffic these fabrics carry, a lookup worth 137.5 ns across five hops, 8.9% of a die — and twenty-nine chapters of ecosystem to rebuild.

Every chapter of Module 23 answered this chapter's question implicitly, by staying inside a standard. This chapter asks what a design gets by leaving one, and the honest answer is smaller and stranger than the marketing.

The framing tax is the argument everyone reaches for first, so derive it first and get it out of the way.

At a 64-octet frame — 84 octets on the wireOctetsShare of the wire
preamble and start delimiter — Chapter 5.289.5238%
interframe gap — Chapter 5.91214.2857%
destination and source addresses — Chapter 5.31214.2857%
frame check sequence — Chapter 5.844.7619%
EtherType — Chapter 5.522.3810%
total3845.2381%

And at the other end of Chapter 8.3 §2's curve.

At a 9 000-octet payload — 9 038 octets on the wireOctetsShare of the wire
preamble and start delimiter80.0885%
interframe gap120.1328%
addresses120.1328%
frame check sequence40.0443%
EtherType20.0221%
total380.4204%

A fabric that deleted every octet of Ethernet's framing would gain 45.24% at minimum frame size and 0.4204% at jumbo — a ratio of 107.6 — and the fabrics that actually leave the standard are the ones carrying large transfers.

So they leave for four tenths of a per cent. Which means the win, if there is one, is somewhere else — and the rest of this chapter goes to find it.


1. Scope — What Leaving Actually Buys and What It Obliges

What a proprietary fabric buys and what it pays. On the buying side: a framing saving of eleven point nine per cent of wire time at sixty four octets, falling to zero point one one one per cent at nine thousand; a forwarding latency of two point five nanoseconds per hop against thirty, which is twelve times and one hundred and thirty seven point five nanoseconds across five hops; and a silicon saving of five point zero four times ten to the seventh bitcell equivalents, which is eight point nine per cent of a merchant switch. On the paying side: the packet buffer at ninety one point one per cent of the die is unchanged, the serialiser at two hundred and fifty six lanes is the same standard intellectual property, twenty nine chapters of ecosystem must be rebuilt, and the number of independent implementations falls to one, which removes the conformance oracle, the ambiguity discovery mechanism, the capture tooling and the second source.Leave thestandardwhat changesFraming0.111% at jumboLookup12× — 137.5 nsArea8.9% of the dieBuffer91.1% — unchangedSerDes256 lanes — sharedEcosystem29 chapters torebuild1 implementationno oracle, no toolingNo second sourcethe roadmap iscoupled12
Figure 1 — four quantities, four different units, and a comparison that forces them into one gets at least two wrong.

This chapter owns four derivations and one conclusion.

What is derived
Sections 2 to 3the framing tax, at both ends of the curve, and what deleting it is worth
Sections 4 to 7the lookup, which is where the real win is: 30 ns per hop against 2.5
Sections 8 to 11what a proprietary fabric must BUILD to replace the ecosystem it left
Sections 12 to 13what it cannot delete — the buffer and the SerDes, which are 91% of the die

What this chapter does not own. It names no vendor and models no specific fabric. Every proprietary interconnect is different and all of them make the same four decisions, which is what makes the comparison possible at all: source routing instead of a lookup, a shorter header, a credit or token link layer, and a controlled endpoint population.

It also does not argue that leaving is wrong. Several of the most successful interconnects in the industry left, deliberately, with clear eyes. The argument is that the win is real and it is not in the unit people quote it in — which is Chapter 23.6 §19's closing thesis arriving in a commercial setting rather than a physical one.


2. The Framing Tax, and What Deleting It Is Worth

Take the table apart field by field and ask what a fabric that deleted each one would actually save, because three of the five cannot be deleted at all.

FieldOctetsCan a proprietary fabric delete it?
preamble and SFD8NO — a receiver's clock recovery needs a known pattern
interframe gap12partly — it can be shortened, not removed
addresses12YES — replaced by a route or a short endpoint id
frame check sequence4NO — a corrupted frame must be detectable
EtherType2YES — if the fabric carries one payload type

Rows one and four are the ones people assume are free and are not.

The preamble is not decoration. Chapter 5.2 §3 established what it is for: a receiver arriving at an idle line must lock its clock recovery and find the frame's first bit, and both need a known pattern. A fabric that deletes it must either run a continuously-scrambled line with block synchronisation — which is what Chapter 3.5's 64B/66B already does above 10 Gb/s — or reintroduce a preamble under another name.

And in fact modern Ethernet already did this. Above 10 Gb/s the eight preamble octets are a legacy the PCS no longer needs for synchronisation; they persist because the MAC interface was defined around them. So a proprietary fabric's preamble saving is real, small, and the same saving 802.3 could have taken and did not, for compatibility.

The frame check sequence cannot go. Chapter 5.8 §2: a 32-bit CRC detects all burst errors up to 32 bits and all single, double and odd-weight errors. A fabric without one delivers corrupted data silently, and every fabric anybody trusts has an equivalent — PCIe has an LCRC, InfiniBand has both a variant and an invariant CRC. The field does not disappear; it is renamed.

So the deletable tax is smaller than the total.

Total taxDeletableActually saved
64-octet frame45.24%addresses 12 + EtherType 2 = 14 octets16.67%
1 518-octet frame2.47%the same 14 octets0.91%
9 018-octet frame0.42%the same 14 octets0.155%

The honest framing saving on jumbo traffic is 0.155% of the wire, and that is the number a proprietary fabric's framing argument is actually worth.

One qualification in the fabric's favour, and it is worth stating because it is the strongest version of the argument. A fabric that knows its endpoint population can use a short route field instead of twelve address octets — Section 4 derives 30 bits for five hops of 64-port switches — so the replacement is smaller rather than absent. Four octets against twelve is an eight-octet saving, and at 64 octets that is 9.52% of the wire, which is the largest honest framing number in this chapter.


3. RTL 1 — The Fabric Package and the Framing-Tax Model

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// ---------------------------------------------------------------------
// propfab_pkg -- constants for comparing a standard frame format
// against a fabric that has left the standard.
//
// Unit: Chapter 23.3 Section 2's bitcell equivalent.
//   1 BCE = one bit of usable on-die SRAM = 0.35 GE
//   1 flip-flop = 20 BCE
//   Chapter 19.7 Section 19's MAC receive datapath = 283 320 BCE
//   Chapter 23.3's 64-port 6.4 Tb/s switch = 5.62e8 BCE
// ---------------------------------------------------------------------
package propfab_pkg;

  localparam int unsigned DATAPATH_BCE  = 283_320;
  localparam int unsigned BCE_PER_FLOP  = 20;
  localparam int unsigned SWITCH_PORTS  = 64;

  // ---- the Ethernet frame's fixed octets -------------------------------
  localparam int unsigned OCT_PREAMBLE  = 8;    // Chapter 5.2
  localparam int unsigned OCT_IFG       = 12;   // Chapter 5.9
  localparam int unsigned OCT_ADDRESSES = 12;   // Chapter 5.3
  localparam int unsigned OCT_FCS       = 4;    // Chapter 5.8
  localparam int unsigned OCT_ETHERTYPE = 2;    // Chapter 5.5
  localparam int unsigned OCT_FIXED     = OCT_PREAMBLE + OCT_IFG
                                        + OCT_ADDRESSES + OCT_FCS
                                        + OCT_ETHERTYPE;   // 38

  // Which of them a proprietary fabric can actually delete. The
  // preamble and the FCS cannot go; the gap can only be shortened.
  localparam int unsigned OCT_DELETABLE = OCT_ADDRESSES + OCT_ETHERTYPE;

  // ---- Chapter 23.3's forwarding state, in BCE -------------------------
  // The three arrays a source-routed fabric does not need.
  localparam int unsigned ACL_TCAM_BCE_E4 = 2_100;   // 2.10e7 / 1e4
  localparam int unsigned L3_LPM_BCE_E4   = 1_680;   // 1.68e7
  localparam int unsigned MAC_TBL_BCE_E4  = 1_260;   // 1.26e7
  localparam int unsigned SWITCH_BCE_E4   = 56_200;  // 5.62e8

  // ---- the pipeline ----------------------------------------------------
  localparam int unsigned LOOKUP_STAGES   = 36;      // Chapter 23.3 Sec 2
  localparam int unsigned SR_STAGES       = 3;       // shift and demux
  localparam int unsigned CORE_MHZ        = 1_200;

  typedef enum logic [1:0] {
    ADDRESSING_NAME  = 2'd0,   // a 48-bit name, looked up
    ADDRESSING_ROUTE = 2'd1,   // a port index per hop
    ADDRESSING_ID    = 2'd2    // a short endpoint identifier
  } addressing_e;

  // ---- derived: framing ------------------------------------------------
  function automatic int unsigned wire_octets(int unsigned payload);
    // payload + 14 header + 4 FCS + 8 preamble + 12 gap
    return payload + OCT_FIXED;
  endfunction

  function automatic int unsigned tax_ppm(int unsigned payload);
    return (OCT_FIXED * 1_000_000) / wire_octets(payload);
  endfunction

  function automatic int unsigned deletable_ppm(int unsigned payload);
    return (OCT_DELETABLE * 1_000_000) / wire_octets(payload);
  endfunction

  // ---- derived: a source route ------------------------------------------
  function automatic int unsigned bits_per_hop(int unsigned radix);
    int unsigned b; int unsigned n;
    b = 0; n = radix - 1;
    while (n > 0) begin b = b + 1; n = n >> 1; end
    return b;
  endfunction

  function automatic int unsigned route_bits(int unsigned radix,
                                             int unsigned hops);
    return bits_per_hop(radix) * hops;
  endfunction

  // ---- derived: latency -------------------------------------------------
  function automatic int unsigned stage_ns_x100(int unsigned stages);
    return (stages * 100_000) / CORE_MHZ;
  endfunction

  function automatic int unsigned datapaths_milli(int unsigned bce);
    return (bce / DATAPATH_BCE) * 1000
         + (((bce % DATAPATH_BCE) * 1000) / DATAPATH_BCE);
  endfunction

endpackage
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// ---------------------------------------------------------------------
// framing_tax_model -- what a frame format costs at a given payload,
// and how much of that a fabric can actually escape.
//
// The output that matters is undeletable_ppm: the preamble and the
// check sequence survive any protocol change, so the saving is always
// smaller than the tax and usually by a factor of nearly three.
// ---------------------------------------------------------------------
module framing_tax_model
  import propfab_pkg::*;
(
  input  logic        clk,
  input  logic        rst_n,

  input  logic [15:0] payload_octets,
  input  logic [15:0] rate_gbps,
  input  logic  [1:0] addressing,         // addressing_e
  input  logic [15:0] route_octets,       // if a route replaces addresses

  output logic [31:0] wire_octets_o,
  output logic [31:0] tax_ppm_o,
  output logic [31:0] deletable_ppm_o,
  output logic [31:0] undeletable_ppm,
  output logic [31:0] actual_saving_ppm,
  output logic [31:0] packet_ns_x100,
  output logic [31:0] packets_per_s_k,
  output logic        preamble_is_deletable,
  output logic        fcs_is_deletable,
  output logic [31:0] c_frames_modelled
);
  always_comb begin
    wire_octets_o   = 32'(wire_octets(int'(payload_octets)));
    tax_ppm_o       = 32'(tax_ppm(int'(payload_octets)));
    deletable_ppm_o = 32'(deletable_ppm(int'(payload_octets)));
    undeletable_ppm = tax_ppm_o - deletable_ppm_o;

    // A route REPLACES the addresses rather than removing them, so the
    // saving is the difference, not the whole field.
    actual_saving_ppm =
      (addressing == 2'(ADDRESSING_NAME)) ? 32'd0
      : ((32'(OCT_DELETABLE) > 32'(route_octets))
         ? (((32'(OCT_DELETABLE) - 32'(route_octets)) * 1_000_000)
            / wire_octets_o)
         : 32'd0);

    // Chapter 8.3's other quantity: the frame RATE, which is what
    // decides the pipeline width rather than the efficiency.
    packet_ns_x100 = (rate_gbps == 0) ? 32'd0
                   : (wire_octets_o * 8 * 100) / 32'(rate_gbps);
    packets_per_s_k = (packet_ns_x100 == 0) ? 32'd0
                    : (100_000_000 / packet_ns_x100);

    // The two that survive any protocol change.
    preamble_is_deletable = 1'b0;
    fcs_is_deletable      = 1'b0;
  end

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) c_frames_modelled <= '0;
    else        c_frames_modelled <= c_frames_modelled + 32'd1;
  end
endmodule

Classification: a division, reported in parts per million so the jumbo case does not round to zero.

What it teaches: that actual_saving_ppm is the number the framing argument is worth and it is not tax_ppm_o. At a 64-octet frame with a four-octet route replacing fourteen octets of address and type, the module reports 119 048 ppm — 11.90%. At 9 018 octets it reports 1 106 ppm — 0.111%. The tax is 45.24% and 0.42% respectively, so the saving is 26.3% of the tax at both ends, because the deletable fraction is a constant share of a constant.

And it teaches that packets_per_s_k is the quantity a fabric designer should be arguing about. At 400 Gb/s the module reports 595 238 thousand packets per second at 64 octets and 5 532 at 9 018 — a factor of 107.6and it is the packet rate that decided Chapter 23.3 §2's eight-wide pipeline, which is real silicon. The efficiency is a percentage; the packet rate is a pipeline.

Deliberately simplified: the model treats the interframe gap as fully present, where a fabric with tighter clock tolerance can shorten it — Chapter 4.4 §4's deletion rate is what bounds how far. route_octets is a constant where a real source route's length varies with the path's hop count, so a fabric with mixed path lengths has a variable-length header, which is a parser cost the model does not charge. And the preamble is charged at eight octets even above 10 Gb/s, where the PCS no longer needs it for synchronisation and it survives as a MAC-interface convention.

Production implication: a variable-length route header is the detail that turns a clean idea into a parser problem. Chapter 19.2 §6 established that a parser's cost is set by how far into the frame a field's position depends on the frame's own content, and a route whose length is the hop count is exactly that: the payload's offset is not knowable until the route has been read. The usual fix is a fixed-width route field sized for the worst path — 30 bits for five hops of 64-port switches, rounded to four octets — which wastes bits on short paths and is what nearly every such fabric does, for the same reason Ethernet's addresses are a fixed six octets.


4. The Lookup, Which Is Where the Win Actually Is

Two forwarding decisions compared. The named path parses the frame, locating fields whose offsets a virtual local area network tag can move; hashes and probes a one hundred and twenty eight thousand entry media access control table; performs a longest prefix match against a one hundred and twenty eight thousand entry routing table; matches sixteen thousand ternary content addressable memory rules; and resolves the disagreeing results by policy before selecting an egress queue. That is thirty six pipeline stages at one point two gigahertz, which is thirty nanoseconds, and it is exactly the lookup term chapter twenty three point one section four assumed. The source routed path takes the low six bits of a thirty bit route field as the egress port and shifts the field right by six. That is three stages, two point five nanoseconds, twelve times faster, and across five hops it saves one hundred and thirty seven point five nanoseconds.Frame arrivesa header to act onParseoffsets the tag canmoveHash, probe, LPM,TCAMfour arraysResolve by policythe results disagree36 stages30.00 nsTake low 6 bitsshift right by 63 stages2.50 ns12× per hop137.5 ns over 512
Figure 2 — thirty-six stages against three, because a name must be resolved against a table and a route is an index.

Section 2 found the framing saving to be a tenth of a per cent on the traffic that matters. The lookup is a different order of magnitude, and it is the thing worth leaving a standard for.

Chapter 23.3 §2 derived the forwarding pipeline's depth from the packet rate: 36 stages at 1.2 GHz, which is 30 ns and is exactly Chapter 23.1 §4's lookup term. Those thirty-six stages do a specific job.

Stage groupWhat it doesWhy it is deep
parselocate the fields, which Chapter 13.2's tag can movethe offsets depend on the content
MAC lookuphash, probe, resolve collisions — Chapter 12.5a 128k-entry table is an SRAM access plus comparison
L3 lookuplongest-prefix matchseveral dependent memory accesses
ACLa TCAM match against 16k ruleswide, and the result must be resolved by priority
resolve and queuecombine the results, pick an egress queuethe outputs disagree and a policy orders them

Now the source-routed alternative. A fabric whose endpoints are enumerated by one authority can put the path in the header: one port index per hop.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
a 64-port switch needs 6 bits to name an egress port
5 hops x 6 bits = 30 bits of route

And the switch's work becomes a shift and a demultiplex.

Named forwardingSource routed
what the switch doesparse, hash, probe, match, resolvetake the low 6 bits, shift the field right
pipeline stages363
at 1.2 GHz30.00 ns2.50 ns
over 5 hops150.0 ns12.5 ns
saving over 5 hops137.5 ns

Twelve times the per-hop latency, and 137.5 nanoseconds saved across a five-stage fabric — which is a real number on a real workload, unlike the framing.

Put that beside the traffic the fabric carries and it becomes a decision rather than a fact.

WorkloadTransferFabric latency at 5 hopsDoes 137.5 ns matter?
Chapter 23.2's all-reduce15.984 GB per participant150 ns of 1.279 s idealno — 0.00001%
a 4 kB storage read4 096 octets150 ns of ~100 µsno — 0.15%
a synchronisation primitive8 octets, and everybody waits150 ns of ~300 nsYES — nearly half
a remote atomic8 octets, round trip300 ns of ~500 nsYES

Rows three and four are the whole case for a proprietary fabric and they are a small, specific, real case. A fabric that exists to make short synchronising operations fast has a genuine 12× lever that Ethernet cannot offer, because Ethernet's addressing model requires a lookup by construction — Chapter 24.1 §2 established that a name must be resolved and a position need not.


5. RTL 2 — The Source-Route Decoder

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// ---------------------------------------------------------------------
// source_route_decoder -- the whole forwarding decision of a
// proprietary fabric, in three pipeline stages.
//
// Set this against Chapter 23.3's thirty-six. The module is short
// because that IS the comparison: the decision is a shift.
// ---------------------------------------------------------------------
module source_route_decoder
  import propfab_pkg::*;
#(
  parameter int unsigned RADIX      = 64,
  parameter int unsigned MAX_HOPS   = 5,
  parameter int unsigned ROUTE_BITS = 30
)(
  input  logic        clk,
  input  logic        rst_n,

  input  logic                  hdr_valid,
  input  logic [ROUTE_BITS-1:0] route_in,
  input  logic            [3:0] hops_remaining_in,

  output logic                  egress_valid,
  output logic            [5:0] egress_port,
  output logic [ROUTE_BITS-1:0] route_out,
  output logic            [3:0] hops_remaining_out,
  output logic                  route_exhausted,
  output logic                  lookup_needed,
  output logic [31:0] stages_used,
  output logic [31:0] latency_ns_x100,
  output logic [31:0] named_latency_ns_x100,
  output logic [31:0] latency_ratio_x10,
  output logic [31:0] c_hops_decoded,
  output logic [31:0] c_exhausted
);
  localparam int unsigned HOP_BITS = 6;   // bits_per_hop(64)

  // Stage 1: register the header. Stage 2: select. Stage 3: shift.
  logic [ROUTE_BITS-1:0] route_s1, route_s2;
  logic            [3:0] hops_s1,  hops_s2;
  logic                  valid_s1, valid_s2;

  always_comb begin
    stages_used           = 32'(SR_STAGES);
    latency_ns_x100       = 32'(stage_ns_x100(int'(SR_STAGES)));
    named_latency_ns_x100 = 32'(stage_ns_x100(int'(LOOKUP_STAGES)));
    latency_ratio_x10     = (latency_ns_x100 == 0) ? 32'd0
                          : (named_latency_ns_x100 * 32'd10)
                            / latency_ns_x100;

    // THE comparison, as a constant. A source-routed fabric performs
    // no lookup at all, so no table can miss and none needs to exist.
    lookup_needed = 1'b0;

    route_exhausted = valid_s2 && (hops_s2 == 4'd0);
  end

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      route_s1 <= '0; route_s2 <= '0; hops_s1 <= '0; hops_s2 <= '0;
      valid_s1 <= 1'b0; valid_s2 <= 1'b0;
      egress_valid <= 1'b0; egress_port <= '0; route_out <= '0;
      hops_remaining_out <= '0;
      c_hops_decoded <= '0; c_exhausted <= '0;
    end else begin
      // Stage 1 -- capture.
      valid_s1 <= hdr_valid;
      route_s1 <= route_in;
      hops_s1  <= hops_remaining_in;

      // Stage 2 -- select this hop's port from the low bits.
      valid_s2 <= valid_s1;
      route_s2 <= route_s1;
      hops_s2  <= hops_s1;

      // Stage 3 -- emit and shift the route down by one hop.
      egress_valid       <= valid_s2 && (hops_s2 != 4'd0);
      egress_port        <= route_s2[HOP_BITS-1:0];
      route_out          <= route_s2 >> HOP_BITS;
      hops_remaining_out <= (hops_s2 == 4'd0) ? 4'd0 : (hops_s2 - 4'd1);

      if (valid_s2 && (hops_s2 != 4'd0))
        c_hops_decoded <= c_hops_decoded + 32'd1;
      if (valid_s2 && (hops_s2 == 4'd0))
        c_exhausted <= c_exhausted + 32'd1;
    end
  end
endmodule

Classification: three registers and a shift, standing in for thirty-six stages of lookup.

What it teaches: that latency_ratio_x10 reports 120 — twelve times — and that the whole difference is the absence of a table. The module contains no memory, no hash, no comparison against stored state and no miss path, because a route is not resolved against anything. Chapter 24.1 §2 made the same observation about PCIe's decode and this is its extreme form: a source route does not even decode, it indexes.

And it teaches that lookup_needed being zero removes an entire failure class rather than a structure. No table means no miss, no flood, no aging, no capacity limit and no thrashChapter 12.5 §7's entire chapter has no subject here. What replaces it is a route computed by somebody else, before the packet existed, and that somebody is the fabric manager.

Deliberately simplified: the route field is fixed at 30 bits for five hops of 64-port switches, so a fabric with deeper paths or wider switches needs a different constant — and a variable-length route is the parser problem Section 3's production note described. There is no error path: a route naming a port that is down, or a route whose hop count disagrees with the actual topology, produces a packet sent somewhere wrong and the module cannot tell. And the three stages assume the route field is already aligned at a fixed offset, which requires the fabric to have no variable-length headers at all.

Production implication: the missing error path is where a source-routed fabric's worst failure lives and it is the exact inverse of Ethernet's. On Ethernet a stale forwarding entry sends a frame to the wrong port and the destination does not answer, so the entry ages out and the system healsChapter 12.5 §9's aging is a self-repair mechanism. On a source-routed fabric a stale route is in the header of every packet the sender emits, and it keeps being wrong until the fabric manager recomputes and redistributes. The blast radius is every packet from that sender to that destination, for the whole reconvergence interval, and the only defence is a fast manager — which is a piece of software, on a control plane, that Section 8 prices.


6. What the Lookup's Deletion Is Worth, in BCE

Twelve times the latency is the visible win. The area saving is the invisible one, and it is smaller than anybody expects.

Chapter 23.3 §2's table, with the three arrays a source-routed fabric does not need marked.

A 64-port 100 Gb/s merchant ASICBCE× the datapathDeleted by source routing?
packet buffer, 64 MB5.12 × 10⁸1 807NO
ACL TCAM, 16k × 256 b2.10 × 10⁷74.0yes
L3 host and LPM, 128k × 128 b1.68 × 10⁷59.2yes
MAC table, 128k × 96 b1.26 × 10⁷44.4yes
RMON counters1.11 × 10⁵0.4NO
queue state, 512 queues3.3 × 10⁴0.1NO
total5.62 × 10⁸1 985

The three deletable rows sum to 5.04 × 10⁷ BCE — 178 datapaths — which is 9.0% of the switch.

Leaving the standard removes 9.0% of the silicon and 100% of the ecosystem, and Section 8 prices the second half.

Three observations, and the third is the one that decides the argument.

One — the buffer is untouched and it is 91.1%. Chapter 23.3 §2 established that the packet buffer dominates a switch by three orders of magnitude over its control logic, and the buffer's size is set by the bandwidth-delay product of the congestion it must absorb, which is a property of the traffic and the topology rather than of the protocol. A proprietary fabric's buffer is the same size.

Two — the SerDes is untouched too, and it is not in the table at all. Chapter 23.3's table counts on-die state; the physical layer is analogue area, and at 64 ports of 100 Gb/s it is a large fraction of the die that BCE does not price. A proprietary fabric at the same rate uses the same SerDes, because the SerDes is where the standards bodies' work is genuinely irreplaceable and because nobody wants to redo it.

Three — the 9.0% is an upper bound and the real figure is smaller. A proprietary fabric still needs some classification: a way to pick a queue, a way to identify control traffic, a way to apply a policy. It deletes the general mechanisms and reintroduces specific ones, and a fabric with eight traffic classes and a small rule set still carries a classifier.

Ethernet's general mechanismA fabric's specific replacementSaving
destination128k-entry MAC tablea route field1.26 × 10⁷ BCE
routing128k-entry LPMnothing — the route is complete1.68 × 10⁷
policy16k × 256 b TCAMperhaps 256 rules2.10 × 10⁷ less ~3 × 10⁵
queueing512 queues of statethe same0

So the achievable saving is about 5.01 × 10⁷ BCE — 8.9% of the switch — and the fabric has given up every mechanism in Modules 12 and 13 to get it.


7. RTL 3 — The Fabric Area Model

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// ---------------------------------------------------------------------
// fabric_area_model -- what a switch costs with and without the
// general forwarding mechanisms, in BCE.
//
// The output that matters is undeletable_share_pct, which stays above
// ninety whatever the protocol, because the packet buffer is sized by
// the traffic rather than by the frame format.
// ---------------------------------------------------------------------
module fabric_area_model
  import propfab_pkg::*;
(
  input  logic        clk,
  input  logic        rst_n,

  input  logic  [1:0] addressing,          // addressing_e
  input  logic [15:0] replacement_rules,   // a fabric still classifies
  input  logic [15:0] rule_width_bits,

  output logic [31:0] deleted_bce_e4,
  output logic [31:0] reintroduced_bce_e4,
  output logic [31:0] net_saving_bce_e4,
  output logic [15:0] net_saving_pct_x10,
  output logic [31:0] undeletable_bce_e4,
  output logic [15:0] undeletable_share_pct,
  output logic [31:0] saving_datapaths_milli,
  output logic        buffer_is_protocol_independent,
  output logic        serdes_is_shared,
  output logic [31:0] c_configs_modelled
);
  always_comb begin
    // The three arrays a route makes unnecessary.
    deleted_bce_e4 = (addressing == 2'(ADDRESSING_NAME))
                   ? 32'd0
                   : (32'(ACL_TCAM_BCE_E4) + 32'(L3_LPM_BCE_E4)
                      + 32'(MAC_TBL_BCE_E4));

    // And what a fabric still needs: a small classifier.
    reintroduced_bce_e4 = (32'(replacement_rules)
                           * 32'(rule_width_bits)) / 32'd10_000;

    net_saving_bce_e4 = (deleted_bce_e4 > reintroduced_bce_e4)
                      ? (deleted_bce_e4 - reintroduced_bce_e4) : 32'd0;

    net_saving_pct_x10 = 16'((net_saving_bce_e4 * 32'd1000)
                             / 32'(SWITCH_BCE_E4));

    undeletable_bce_e4    = 32'(SWITCH_BCE_E4) - net_saving_bce_e4;
    undeletable_share_pct = 16'((undeletable_bce_e4 * 32'd100)
                                / 32'(SWITCH_BCE_E4));

    saving_datapaths_milli = 32'(datapaths_milli(net_saving_bce_e4
                                                 * 32'd10_000));

    // The two facts that cap the saving, stated as constants because
    // they are true of every protocol rather than of this one.
    buffer_is_protocol_independent = 1'b1;
    serdes_is_shared               = 1'b1;
  end

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) c_configs_modelled <= '0;
    else        c_configs_modelled <= c_configs_modelled + 32'd1;
  end
endmodule

Classification: a subtraction from Chapter 23.3's table, with the reintroduced cost charged back.

What it teaches: that undeletable_share_pct reports 91 or higher in every configuration, and that the two constants explaining it — buffer_is_protocol_independent and serdes_is_shared — are properties of physics and economics rather than of either protocol. The buffer's size is a bandwidth-delay product of the congestion, which does not care what the header looks like. The SerDes is where an industry's cumulative analogue effort lives and no fabric vendor redoes it, which is why a proprietary fabric's physical layer is usually an Ethernet or a PCIe one.

And it teaches that reintroduced_bce_e4 is the term that gets left out. A fabric with 256 rules of 256 bits reintroduces 65 536 bits — 6.55 × 10⁴ BCE, which is small — but a fabric that needs a thousand rules and a stateful classifier has given most of the saving back. The model charges it so the comparison is honest.

Deliberately simplified: the reintroduced classifier is priced as an SRAM array at 1 BCE per bit, where a TCAM-based one is five times thatChapter 23.3 §2's factor of 2.5 for the cell and 2 for storing value and mask. The model does not price the fabric manager, which is a control-plane cost rather than a die cost and which Section 9 handles separately. And it takes Chapter 23.3's specific table sizes as the baseline, so a part with a smaller table has a proportionally smaller saving.

Production implication: the table sizes are the lever nobody pulls, and it is worth being explicit because it changes the whole argument. A 128k-entry MAC table is sized for a switch that might be deployed anywhere. A switch deployed as a leaf in Chapter 23.1's fabric sees a few thousand addresses, and a 32k-entry table would be ample — saving 9.45 × 10⁶ BCE, 33.4 datapaths, 1.7% of the die, while remaining fully standard. So a large part of the proprietary fabric's area advantage is available to an Ethernet part that knows its deployment, and the reason merchant silicon does not take it is that a merchant part does not know its deployment. The comparison is therefore not standard against proprietary; it is general-purpose against special-purpose, and that is a much older and better-understood trade.


8. What a Proprietary Fabric Must Build

What a standard supplies beyond a frame format, counted in chapters of this track. Link bring up and capability exchange is four chapters, module eleven. Segmentation and isolation is four chapters, module thirteen. Congestion control is four chapters, module fourteen. Link aggregation and discovery is three chapters, module fifteen. Time synchronisation is five chapters, module sixteen. Statistics and management is one chapter, nineteen point seven. Capture and analysis is one chapter, twenty one point nine. Verification and conformance is six chapters, module twenty. The total is twenty nine chapters, which is twenty two point five per cent of the Ethernet track as it stands, and a fabric that leaves the standard must supply an equivalent for every one of them or decide to go without.The standardsupplies all of itBring-up4 chaptersSegmentation4 chaptersCongestion4 chaptersAggregation3 chaptersTime5 chaptersStats and capture2 chaptersVerification6 chapters29 chapters22.5% of the track12
Figure 3 — twenty-nine chapters of this track are standardised behaviour that has nothing to do with framing or forwarding.

Sections 2 to 7 found the technical savings: 0.155% of the wire and 8.9% of the die. This section is the bill on the other side, and it is not measured in bitcell equivalents.

A standard is not a frame format. It is a frame format plus everything that exists because the frame format is public, and a fabric that leaves must supply all of it.

What Ethernet suppliesWhere this track covers itWhat a proprietary fabric must build
link bring-up and capability exchangeChapter 11.1 to Chapter 11.4 — four chaptersits own negotiation, with its own failure modes
neighbour discovery and topologyChapter 15.3a discovery protocol and a manager
segmentation and isolationChapter 13.1 to Chapter 13.4 — four chaptersa tenancy model, if it needs one
congestion controlChapter 14.1 to Chapter 14.4 — four chaptersits own, and Chapter 24.2 says what that costs
time synchronisationChapter 16.1 to Chapter 16.5 — five chaptersa time protocol, or none
statistics and managementChapter 19.7its own counter definitions and its own tooling
capture and analysisChapter 21.9a tap, a capture device, a decoder — all of them
verification and conformanceChapter 20.1 to Chapter 20.6 — six chaptersa reference model with one author

Twenty-nine chapters of this track are about standardised behaviour that has nothing to do with framing or forwarding22.5% of the Ethernet track as it stands — and a proprietary fabric must supply an equivalent for every row or decide to go without.

Leaving the standard deletes 8.9% of the die and obliges the vendor to rebuild twenty-nine chapters' worth of ecosystem, most of which is software and none of which is a differentiator.

Two rows deserve more than a table entry.

Capture and analysis is the one that hurts during bring-up. Chapter 21.9's entire apparatus presumes a tap, a capture device that can keep up, a decoder that knows the format, and a corpus of known-good traffic to compare against. All four are commercially available for Ethernet at every rate and none exists for a new fabric on day one. So the fabric's first silicon is debugged with the fabric's own counters, which is Chapter 21.9 §20's class 102 territory: the observation apparatus is inside the system it observes.

Verification is the one that hurts permanently, and Section 20's rejected property is about it. Chapter 20.3's scoreboard compares a design against a reference model. For Ethernet the reference is a public specification with a thousand independent implementations to check against. For a proprietary fabric the reference is a document the same organisation wrote, and when the design disagrees with it, amending the document is cheaper than fixing the design.


9. RTL 4 — The Ecosystem Cost Model

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// ---------------------------------------------------------------------
// ecosystem_cost_model -- what a fabric must supply for itself, counted
// in mechanisms rather than in area, because none of it is on the die.
//
// The output that matters is independent_implementations, which is
// one, and which is the input to every other conclusion in Section 8.
// ---------------------------------------------------------------------
module ecosystem_cost_model
  import propfab_pkg::*;
(
  input  logic        clk,
  input  logic        rst_n,

  input  logic        is_standard,
  input  logic [15:0] mechanisms_needed,     // discovery, time, etc.
  input  logic [15:0] mechanisms_supplied,   // by the standard
  input  logic [15:0] vendors,

  output logic [15:0] mechanisms_to_build,
  output logic [15:0] independent_implementations,
  output logic        spec_ambiguity_is_discoverable,
  output logic        conformance_has_an_oracle,
  output logic        capture_tooling_exists,
  output logic [15:0] second_source_count,
  output logic        single_vendor_risk,
  output logic [31:0] c_mechanisms_unsupplied
);
  always_comb begin
    mechanisms_to_build = is_standard
                        ? ((mechanisms_needed > mechanisms_supplied)
                           ? (mechanisms_needed - mechanisms_supplied)
                           : 16'd0)
                        : mechanisms_needed;

    independent_implementations = is_standard ? vendors : 16'd1;

    // THE consequence. A specification needs two independent readings
    // before anybody can discover that it has two readings.
    spec_ambiguity_is_discoverable = (independent_implementations >= 16'd2);

    // And a conformance check needs an oracle its own author did not
    // write. Section 20's rejected property is exactly this.
    conformance_has_an_oracle = (independent_implementations >= 16'd2);

    // Chapter 21.9's apparatus presumes a decoder somebody sells.
    capture_tooling_exists = is_standard;

    second_source_count = is_standard ? vendors : 16'd1;
    single_vendor_risk  = (second_source_count < 16'd2);
  end

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) c_mechanisms_unsupplied <= '0;
    else        c_mechanisms_unsupplied <= 32'(mechanisms_to_build);
  end
endmodule

Classification: a model with no datapath, whose outputs are all consequences of one integer being 1.

What it teaches: that independent_implementations is the input everything else depends on, and it is 1 for a proprietary fabric by definition. From that one value follow spec_ambiguity_is_discoverable false, conformance_has_an_oracle false, capture_tooling_exists false and single_vendor_risk true — four consequences from one fact, none of which is about the frame format.

And it teaches why mechanisms_to_build is 29 rather than 3. A team leaving a standard budgets for the frame format, the forwarding and the link layer. The twenty-six other rows — discovery, time, congestion, segmentation, statistics, capture, conformance — are not in the plan and are not optional, and each is a small project that the standard supplied for free.

Deliberately simplified: the model counts mechanisms rather than effort, where the rows differ by orders of magnitude — a discovery protocol is weeks and a time protocol accurate to Chapter 16.5's limits is years. vendors is a static input where a standard's implementation count grows over decades and a proprietary fabric's stays at one by policy. And the model treats "supplied by the standard" as binary, where several rows are supplied partially: Ethernet supplies PTP, and a deployment still has to configure it.

Production implication: capture_tooling_exists is the row that determines a programme's schedule and it is almost never in the plan. A new fabric's first silicon arrives with no external way to see what is on the wire, so every bring-up bug is diagnosed from the design's own counters — which are exactly the structures most likely to be wrong in first silicon. The teams that get this right build a capture and replay path into the first chip — a mirror port, a timestamped ring buffer, a way to inject a known frame — and treat it as a tape-out requirement rather than a debug convenience. Chapter 21.9 §2 established that an apparatus whose capacity is below the thing it observes records its own drops as the subject's; an apparatus that is the subject has the same problem and no external witness at all.


10. What It Cannot Delete — the Buffer and the SerDes

Two structures survive every protocol change, they are most of the die and most of the schedule, and a fabric comparison that omits them is comparing the small halves.

The buffer first. Chapter 23.3 §2's 64 MB packet buffer is 5.12 × 10⁸ BCE — 91.1% of the switch — and its size is set by a chain of reasoning in which the frame format appears nowhere.

StepWhat decides itChapter
congestion existsseveral ingress ports may target one egressChapter 14.1 §4
the overlap must go somewhereor it is discardedChapter 14.1
how much is a bandwidth-delay productthe burst's duration times the excess rateChapter 23.3 §9
and it is shared, over-committed 2 793% at idlebecause ports are rarely all congested at onceChapter 23.3 §9

None of those four steps mentions a header. A proprietary fabric carrying the same traffic across the same topology at the same rate needs the same buffer, and Chapter 24.2 established that a fabric which tries to avoid it by going lossless pays more rather than less.

Then the SerDes, which is not in Chapter 23.3's table at all because that table counts on-die state.

What it isWho supplies it
the serialiser and clock recoveryanalogue, at 100 Gb/s per lanean IP vendor, or the same one for both
the equalisationChapter 22.1's echo and feed-forward machinerythe same
the line code and scramblerChapter 3.5's 64B/66Bstandardised
the forward error correctionChapter 9.6's RS-FECstandardised

A proprietary fabric at 400 Gb/s uses an Ethernet or a PCIe SerDes, because that is where an industry's accumulated analogue effort lives and nobody redoes it. So the physical layer — the hardest, longest-lead, most schedule-critical part of any high-rate interconnect — is shared.

The two most expensive things in a switch are protocol-independent. A proprietary fabric differentiates in 8.9% of the die and the header, and shares everything else with the standard it left.


11. RTL 5 — The Undeletable Core

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// ---------------------------------------------------------------------
// undeletable_core -- what both fabrics must contain, priced so the
// differentiated fraction can be stated honestly.
//
// The outputs that matter are shared_share_pct and the two booleans
// that explain it: the buffer is sized by traffic and the SerDes by
// physics, and neither cares about the header.
// ---------------------------------------------------------------------
module undeletable_core
  import propfab_pkg::*;
(
  input  logic        clk,
  input  logic        rst_n,

  input  logic [31:0] buffer_bce_e4,
  input  logic [31:0] forwarding_bce_e4,
  input  logic [31:0] other_bce_e4,
  input  logic [15:0] serdes_lanes,
  input  logic        fabric_is_proprietary,

  output logic [31:0] total_bce_e4,
  output logic [31:0] shared_bce_e4,
  output logic [31:0] differentiated_bce_e4,
  output logic [15:0] shared_share_pct,
  output logic [15:0] differentiated_share_pct_x10,
  output logic        buffer_sized_by_traffic,
  output logic        serdes_sized_by_physics,
  output logic        serdes_is_standard_ip,
  output logic [15:0] shared_serdes_lanes,
  output logic [31:0] c_evaluations
);
  always_comb begin
    total_bce_e4 = buffer_bce_e4 + forwarding_bce_e4 + other_bce_e4;

    // A proprietary fabric deletes the forwarding state and keeps the
    // rest. A standard one keeps everything.
    shared_bce_e4 = fabric_is_proprietary
                  ? (buffer_bce_e4 + other_bce_e4)
                  : total_bce_e4;

    differentiated_bce_e4 = total_bce_e4 - shared_bce_e4;

    shared_share_pct = (total_bce_e4 == 0) ? 16'd100
                     : 16'((shared_bce_e4 * 32'd100) / total_bce_e4);

    differentiated_share_pct_x10 =
      (total_bce_e4 == 0) ? 16'd0
                          : 16'((differentiated_bce_e4 * 32'd1000)
                                / total_bce_e4);

    // The three facts that cap every proprietary fabric's advantage.
    buffer_sized_by_traffic = 1'b1;
    serdes_sized_by_physics = 1'b1;
    serdes_is_standard_ip   = 1'b1;

    // And the lane count is the same on both sides at the same rate.
    shared_serdes_lanes = serdes_lanes;
  end

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) c_evaluations <= '0;
    else        c_evaluations <= c_evaluations + 32'd1;
  end
endmodule

Classification: a partition of a die into the part a protocol decides and the part it does not.

What it teaches: that shared_share_pct reports 91 for a proprietary fabric built on Chapter 23.3's numbers, and that the figure is a floor rather than a coincidence: the buffer is 91.1% before anything else is counted. A fabric arguing for a transformative silicon advantage is arguing about 8.9%, and that number is the module's entire output.

And it teaches that the SerDes is absent from the arithmetic and dominant in the schedule. shared_serdes_lanes is reported so a comparison cannot quietly omit it: 64 ports of 400 Gb/s is 256 lanes of 100 Gb/s, each with Chapter 22.1's equalisation machinery, and both fabrics buy the same IP from the same vendors.

Deliberately simplified: the model prices only on-die state and the SerDes is counted in lanes rather than in area, because analogue area does not convert to BCE — it is transistors laid out for matching and noise rather than for density, and Chapter 23.3 §2's gate-equivalent derivation does not apply to it. other_bce_e4 lumps counters and queue state together, which at 0.03% of the die is defensible. And the model assumes the proprietary fabric deletes the whole forwarding block, where Section 7 established it reintroduces a classifier.

Production implication: the SerDes being shared is the fact that most changes a programme's risk profile, and it cuts in the proprietary fabric's favour. A new fabric that reuses a proven 100 Gb/s Ethernet SerDes has eliminated the single largest schedule risk in a high-rate interconnect, because the analogue block is the one that comes back from the fab wrong and takes a respin to fix. A fabric that decides to build its own physical layer as well has taken on a two-year programme to save nothing, since the line code and FEC are public and the equalisation is the same problem. The right shape for a proprietary fabric is therefore a standard physical layer with a proprietary layer above it — which is what Chapter 24.2's RoCE does from the other direction, and it is why the two designs converge in the middle.


12. Where a Proprietary Fabric Actually Fails

Not on bandwidth, not on latency, and not on silicon. It fails on the two things Section 9's model derived from independent_implementations being one.

Failure one — the specification has never been read by anyone who might disagree with it.

A specification's ambiguities are found by two teams implementing it independently and producing incompatible parts. That is what an interoperability event is for, and it is why Chapter 20.6's reusable agent is checkable at all: there is a population of implementations to check against.

A standardA proprietary fabric
readings of the specificationmany, independentone, by the authors
how ambiguity surfacestwo parts fail to interoperateit does not
what happens when design and spec disagreeone of them is wrong and the market says whichthe spec is amended
the conformance oracleother implementationsthe same document

Row three is the mechanism and it is not dishonesty. When first silicon disagrees with the specification, amending the specification is a one-line change and fixing the design is a respin. The amendment is made by engineers acting reasonably, and the effect is that the specification converges on describing whatever the design does — at which point conformance to it establishes nothing.

Failure two — there is no second source, and the consequence is not a price.

What a second source provides
pricea negotiation
supplya fallback when a fab allocation is short
roadmapan alternative if the vendor's priorities change
longevitya part in ten years, when the original vendor has moved on

The last row is the one that decides infrastructure purchases and it is not about cost at all. A data centre built on a fabric with one supplier has coupled its refresh cycle to that supplier's product plan, and there is no engineering mitigation.

13. RTL 6 — The Second-Source Model

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// ---------------------------------------------------------------------
// second_source_model -- the consequences of independent_implementations
// being one, made into signals.
//
// There is no datapath here and that is deliberate: every risk this
// module reports is real, none of it is on the die, and a design
// review that only looks at silicon sees none of it.
// ---------------------------------------------------------------------
module second_source_model
  import propfab_pkg::*;
(
  input  logic        clk,
  input  logic        rst_n,

  input  logic [15:0] implementations,
  input  logic [15:0] spec_amendments_since_freeze,
  input  logic [15:0] amendment_budget,
  input  logic        independent_model_exists,
  input  logic        model_authors_saw_design,

  output logic        oracle_is_independent,
  output logic        conformance_is_meaningful,
  output logic        ambiguity_discoverable,
  output logic        spec_churn_exceeded,
  output logic        price_negotiable,
  output logic        supply_has_fallback,
  output logic        roadmap_is_coupled,
  output logic  [3:0] risks,
  output logic [31:0] c_amendments
);
  always_comb begin
    // An oracle is independent only if somebody wrote it from the
    // document WITHOUT seeing the design. Seeing the design converts
    // the model into a second description of the same artefact.
    oracle_is_independent = independent_model_exists
                         && !model_authors_saw_design;

    conformance_is_meaningful = oracle_is_independent
                             || (implementations >= 16'd2);

    ambiguity_discoverable = (implementations >= 16'd2);

    spec_churn_exceeded = (spec_amendments_since_freeze > amendment_budget);

    price_negotiable    = (implementations >= 16'd2);
    supply_has_fallback = (implementations >= 16'd2);
    roadmap_is_coupled  = (implementations < 16'd2);

    risks = { roadmap_is_coupled, spec_churn_exceeded,
              !conformance_is_meaningful, !ambiguity_discoverable };
  end

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) c_amendments <= '0;
    else        c_amendments <= 32'(spec_amendments_since_freeze);
  end
endmodule

Classification: a risk register expressed as combinational logic, so that a fact usually argued about is evaluated instead.

What it teaches: that oracle_is_independent requires two conditions and teams routinely satisfy only the first. A separate reference model exists in most programmes; a reference model whose authors never saw the design is rare, because the fastest way to write one is to read the RTL. model_authors_saw_design being true collapses the oracle into a second description of the same artefact, which is Section 12's class 114 with extra steps.

And it teaches that spec_churn_exceeded is a measurable, ignored signal. A specification amended repeatedly after a verification freeze is evidence about the verification rather than about the specification, and counting the amendments costs nothing. A programme that freezes a document and then amends it eleven times has, in effect, verified eleven different designs.

Deliberately simplified: implementations is an integer where the useful quantity is independent readings — two vendors who licensed the same reference design are one reading, not two, and this is why some multi-vendor standards behave like single-vendor ones. The risk vector treats all four risks as equal, where roadmap coupling is a decade-scale commercial exposure and spec churn is a quarter-scale engineering one. And there is no term for partial independence, which is what a standard with one dominant implementation actually has.

Production implication: the fix for oracle_is_independent is organisational and it is cheap relative to what it buys. A second team, reading the specification alone, writing a behavioural model before seeing any RTL — and the productive output is not the model but the list of questions they had to ask, because every question is an ambiguity a standard would have found at a plugfest. A programme that runs this exercise and records the questions has reconstructed the one thing leaving a standard gave up, at a cost of a few engineer-months rather than a few engineer-decades. A programme that skips it ships a fabric whose specification says whatever the design does.


14. What a Proprietary Fabric Must Never Do

Five prohibitions, and three of them are about how the fabric describes itself rather than about how it behaves.

#NeverBecause
1quote a framing advantage at minimum frame sizeSection 2: it is 45.24% there and 0.155% on the traffic the fabric carries
2amend the specification to match the design after freezeSection 20: the oracle and the design share a custodian and the amendment is always cheaper
3claim a silicon advantage without naming the fractionSection 6: it is 8.9% of the die; the buffer and the SerDes are shared
4ship first silicon with no capture pathSection 9: no external decoder exists, so the design's own counters are the only witness
5use a variable-length route field without a parser argumentSection 5: the payload's offset then depends on the frame's own content — Chapter 19.2 §6

Row four is the one that costs schedule and it is the cheapest to fix.

A fabric's first silicon arrives into a world with no tap, no capture device and no decoder, so every bring-up question — did the packet leave? did it arrive? what did it look like? — is answered by the chip's own counters. Chapter 21.9 §20's class 102 is precisely this hazard: an observation apparatus whose own failures are recorded as the subject's. Here the apparatus is the subject, so there is not even an independent instrument to be wrong.

The fix is three structures and they are small.

StructureWhat it givesCost
a mirror porttraffic on a wire an external instrument can seeone egress port's worth of replication
a timestamped ring bufferthe last N frames, after a triggerChapter 21.9's capture sink — a few thousand BCE per entry
a frame injectora known frame at a known instanta register bank and a mux

All three are tape-out requirements rather than debug conveniences, because a respin to add them costs more than the whole programme saves by leaving the standard.


15. RTL 7 — Fabric Telemetry

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// ---------------------------------------------------------------------
// propfab_telemetry -- what a fabric with no external instrumentation
// must report about itself, and the one field that says whether the
// report can be trusted.
//
// Design rule: on a fabric with no second implementation, every
// counter is also a claim, so each one carries its provenance.
// ---------------------------------------------------------------------
module propfab_telemetry
  import propfab_pkg::*;
(
  input  logic        clk,
  input  logic        rst_n,
  input  logic        clear,

  input  logic        hop_decoded,
  input  logic        route_exhausted_i,
  input  logic        frame_rx,
  input  logic [15:0] frame_octets,
  input  logic        frame_dropped,
  input  logic        mirror_active,
  input  logic        capture_armed,
  input  logic        injector_used,
  input  logic        external_decoder_present,

  output logic [47:0] c_hops,
  output logic [47:0] c_route_errors,
  output logic [47:0] c_frames,
  output logic [47:0] c_octets,
  output logic [47:0] c_drops,
  output logic [47:0] c_mirror_frames,
  output logic [31:0] mean_frame_octets,
  output logic [31:0] framing_tax_ppm_measured,
  output logic        self_witnessed_only,
  output logic        capture_path_present
);
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n || clear) begin
      c_hops <= '0; c_route_errors <= '0; c_frames <= '0;
      c_octets <= '0; c_drops <= '0; c_mirror_frames <= '0;
    end else begin
      if (hop_decoded)       c_hops         <= c_hops + 48'd1;
      if (route_exhausted_i) c_route_errors <= c_route_errors + 48'd1;
      if (frame_rx) begin
        c_frames <= c_frames + 48'd1;
        c_octets <= c_octets + 48'(frame_octets);
      end
      if (frame_dropped)     c_drops         <= c_drops + 48'd1;
      if (mirror_active && frame_rx)
                             c_mirror_frames <= c_mirror_frames + 48'd1;
    end
  end

  always_comb begin
    mean_frame_octets = (c_frames == 0) ? 32'd0
                      : 32'(c_octets / c_frames);

    // The tax, measured on the traffic that actually ran, rather than
    // quoted at minimum frame size.
    framing_tax_ppm_measured =
      (mean_frame_octets == 0) ? 32'd0
                               : 32'(tax_ppm(int'(mean_frame_octets)));

    // THE field. Without an external decoder, every number above is a
    // claim the design makes about itself.
    self_witnessed_only = !external_decoder_present;

    capture_path_present = mirror_active || capture_armed || injector_used;
  end
endmodule

Classification: a counter bank whose distinguishing output is a statement about the counters' own trustworthiness.

What it teaches: that framing_tax_ppm_measured settles Section 2's argument empirically rather than rhetorically. A fabric reporting a mean frame of 1 500 octets measures a tax of 24 707 ppm — 2.47%; one reporting 9 000 measures 4 204 ppm — 0.42%. The quoted 45.24% corresponds to a mean frame of 46 octets, which the counter will never report on the workloads these fabrics exist for.

And it teaches that self_witnessed_only is the honest label on everything else in the module. Chapter 21.9 §20's class 102 established that an instrument inside the system it measures records its own failures as the subject's; a fabric with no external decoder has only such instruments, and a design that says so in a status bit is doing more than most.

Deliberately simplified: mean_frame_octets is a running mean where the distribution matters more — a bimodal workload of 64-octet control and 9 000-octet bulk has a mean that describes neither. c_mirror_frames counts only while mirroring is active, so it cannot say what fraction of traffic was ever visible externally. And the counters share a clock domain with the datapath, which for a multi-port fabric means the per-port counters must be synchronised before comparison — Chapter 18.1 §19's class 74 once more.

Production implication: the bimodal case is the one that produces a wrong decision, and it is common. A fabric whose frames are 90% 64-octet synchronisation messages and 10% 9 000-octet bulk — which is 94% of its octets — has a mean frame of 958 octets, corresponding to a tax of 3.8% against the true figures of 45.24% and 0.42% for its two populations. The mean describes neither. The useful report is a histogram, which Chapter 21.6 §4 argued for in a different context and for the same reason: a mean over a bimodal distribution is a number that no member of the distribution is near. Sixteen bins of a 48-bit counter is 768 bits, 15 360 BCE, 0.054 datapaths, and it turns a misleading mean into a description.


16. RTL 8 — The Fabric Conformance Monitor

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// ---------------------------------------------------------------------
// propfab_conformance -- the checks that survive Section 20's problem,
// because they follow from arithmetic rather than from a document.
//
// Every check here is chosen for one property: the specification
// cannot be amended to make it pass.
// ---------------------------------------------------------------------
module propfab_conformance
  import propfab_pkg::*;
(
  input  logic        clk,
  input  logic        rst_n,

  input  logic [47:0] octets_in,
  input  logic [47:0] octets_out,
  input  logic [47:0] octets_dropped,
  input  logic [47:0] octets_resident,
  input  logic        window_end,

  input  logic [15:0] route_bits_used,
  input  logic [15:0] radix,
  input  logic [15:0] max_hops,
  input  logic        hop_decoded_i,
  input  logic  [3:0] hops_remaining_i,
  input  logic        egress_valid_i,

  input  logic        claims_conformance,
  input  logic        oracle_independent_i,

  output logic        v_conservation,
  output logic        v_route_too_short,
  output logic        v_hop_after_exhaustion,
  output logic        v_conformance_unfounded,
  output logic  [3:0] violations,
  output logic        conformant
);
  always_comb begin
    // 1. Nothing is created or destroyed. Arithmetic, not a document.
    v_conservation = window_end
                  && (octets_in != (octets_out + octets_dropped
                                    + octets_resident));

    // 2. The route field must be able to name every hop. Arithmetic
    //    again: ceil(log2(radix)) x max_hops.
    v_route_too_short = (32'(route_bits_used)
                         < 32'(route_bits(int'(radix), int'(max_hops))));

    // 3. A packet whose route is exhausted must not be forwarded.
    v_hop_after_exhaustion = egress_valid_i && (hops_remaining_i == 4'd0);

    // 4. Section 20's class 114, as a check: a conformance claim needs
    //    an oracle its own authors did not write.
    v_conformance_unfounded = claims_conformance && !oracle_independent_i;

    violations = { v_conformance_unfounded, v_hop_after_exhaustion,
                   v_route_too_short, v_conservation };
    conformant = (violations == 4'b0000);
  end
endmodule

Classification: four checks selected by a single criterion — that no amendment to a document can make them pass.

What it teaches: that v_conservation is the model check for a fabric whose specification is mutable, because octets in equalling octets out plus dropped plus resident is arithmetic. Nobody can revise it. Chapter 19.7 §19's class 87 is the warning that comes with it — a conservation law over counters committed at different pipeline depths is false for a reason that has nothing to do with the design — so the four terms must be sampled consistently or the check is worse than useless.

And it teaches that v_conformance_unfounded puts Section 20's argument into silicon. A design asserting conformance against a self-authored oracle is making a claim its verification cannot support, and one bit records the fact. It is the only check in this chapter whose subject is a process rather than a signal, and it is included deliberately, because on a fabric with one implementation the process is the risk.

Deliberately simplified: octets_resident must be supplied from the buffer's occupancy and is the term most likely to be sampled at the wrong instant. v_route_too_short is an elaboration check — the route width is fixed at synthesis and the radix at deployment, so it should be a compile-time assertion rather than a runtime one. And oracle_independent_i is an input, so the monitor trusts an organisational fact it cannot verify, which is exactly the recursion class 114 describes and which no amount of RTL escapes.

Production implication: the conservation check is the one to implement and it is routinely implemented wrongly. The four counters must be sampled in the same cycle from the same clock domain, which on a 64-port fabric means a snapshot mechanism rather than four independent reads — a freeze signal, a shadow register per counter, and a single read of the shadows. Without it the check fails intermittently on a correct design, the team disables it, and the one property in the suite that a specification amendment could not have falsified is the one that gets turned off.


17. The Two Fabrics, Priced Side by Side

Two chapters, one shape. Chapter twenty three point six found a complete one hundred gigabit per second field programmable gate array media access controller at eleven point four per cent lookup table utilisation, one point four per cent flip-flops and three point two per cent block random access memory, and it still did not work, because the binding constraints were timing closure and routing congestion and neither appears in a utilisation report. This chapter finds a fabric comparison quoting bandwidth and framing efficiency, where the binding quantity is tail latency on small operations under load, and that quantity cannot be compared across vendors because each would choose a different workload, so it is never published. In both cases the reported unit and the scarce resource are different things, and the remedy in both cases is to name the quantity that actually binds.Chapter 23.611.4% utilised, stillfailsWhat bindsfMAX and routingWhat is reportedLUT, FF, BRAMThis chaptera fabric comparisonWhat bindstail latency, small opsWhat is quotedbandwidth, framingWhy the gap holdstools print, marketscompareName the quantityworkload and percentile12
Figure 4 — the same failure twice: what binds is not what is reported, on an FPGA and in a market.

Everything this chapter derived, in one table.

QuantityEthernetProprietaryThe gap
framing tax at 64 octets45.24%37.84%7.40 points — 11.90% less wire time
framing tax at 9 018 octets0.4204%0.3101%0.110 points — 0.111% less wire time
per-hop forwarding latency30.00 ns — 36 stages2.50 ns — 3 stages12×
across 5 hops150.0 ns12.5 ns137.5 ns
forwarding state5.04 × 10⁷ BCE≈ 3 × 10⁵8.9% of the die
packet buffer5.12 × 10⁸ BCEthe samenone
SerDes, 64 × 400 Gb/s256 lanes of standard IPthe samenone
ecosystem mechanisms to build029 chapters' worthall of it
independent implementationsmany1the conformance oracle

Rows six and seven are blank in the gap column and they are 91% of the die.

A proprietary fabric buys 137.5 nanoseconds and 8.9% of a switch, and pays for it with the conformance oracle, the capture tooling, the second source and twenty-nine chapters of ecosystem.

And the decision is a workload question rather than an engineering one.

WorkloadWhat 137.5 ns is worthVerdict
Chapter 23.2's all-reduce, 15.984 GB/participant0.00001% of a 1.279 s jobnot worth leaving for
a 4 kB storage read, ~100 µs0.15%not worth leaving for
an 8-octet remote atomic, ~500 ns round trip55%worth leaving for
a barrier across 1 024 participantsthe slowest path sets the pace — Chapter 23.2 §4worth leaving for

Rows three and four are real, specific and narrow, and they are the honest case for every proprietary interconnect that has succeeded.


18. What the Comparison Assumes

Eight assumptions, each with the direction the chapter moves if it fails.

#AssumptionIf it is false
1the fabric carries large transfersthe framing argument improves; at 64 octets the honest saving is 11.90 points rather than 0.111
2a route is four octetsa shorter route improves the saving proportionally; a variable-length one costs a parser
3Chapter 23.3's pipeline is 36 stagesthat chapter derived it from the packet rate; a shallower one narrows the 12×
4source-route decode is 3 stagesa design that also classifies needs more, and the 12× falls toward 6×
5the table sizes are 128k MAC, 128k LPM, 16k ACLa part sized for its deployment already has most of the 8.9%
6the SerDes is shareda fabric building its own takes a two-year programme to save nothing
7the packet buffer is protocol-independenttrue by Section 10's four-step derivation; a lossless fabric pays more — Chapter 24.2 §4
8BCE appliesSection 19 examines it and it holds, with the SerDes as an explicit exclusion

Assumption 5 is the one that reframes the argument and it is worth a sentence. A 32k-entry MAC table instead of 128k saves 9.44 × 10⁶ BCE — 33.3 datapaths, 1.7% of the die — while remaining fully standard. So a large part of the proprietary fabric's area advantage is available to any Ethernet part that knows where it will be deployed, and the comparison is really general-purpose against special-purpose rather than standard against proprietary.


19. The Cost, Accounted — in BCE

This chapter's blocks.

BlockFlopsBCE× the datapath
framing_tax_model326400.002
source_route_decoder, 3 stages3 × 35 + 643 3800.012
fabric_area_model326400.002
ecosystem_cost_model326400.002
undeletable_core326400.002
second_source_model326400.002
propfab_telemetry3366 7200.024
propfab_conformance0 — combinational00
this chapter's additions66513 3000.047

Six of the eight blocks are models rather than datapaths, which is characteristic of a comparison chapter: most of it is arithmetic about designs rather than logic inside one. The two that carry real state are the decoder and the telemetry, and together they are 10 100 BCE — 0.036 datapaths.

And the designs the blocks describe.

BCE× the datapath% of Chapter 23.3's switch
forwarding state a source route deletes5.04 × 10⁷1789.0%
net of the classifier it reintroduces5.01 × 10⁷1778.9%
the packet buffer, unchanged5.12 × 10⁸1 80791.1%
a 32k MAC table's saving, inside the standard9.44 × 10⁶33.31.7%
Chapter 23.3's whole switch5.62 × 10⁸1 985100%

Row four against row two is the chapter's quietest finding: 18.8% of the proprietary fabric's entire area advantage is available by choosing a smaller table and staying inside the standard.


20. Properties Worth Asserting, and One Worth Refusing

Fifty-one properties in six groups, and the refused one is the top-level property of every verification plan in this chapter's subject.

Group A — the framing tax (9).

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// A1. The wire slot is the payload plus thirty-eight fixed octets.
p_ft_wire_slot: assert property (@(posedge clk) disable iff (!rst_n)
  (wire_octets_o == 32'(payload_octets) + 32'(OCT_FIXED)));

// A2. The tax falls as the payload grows -- Chapter 8.3's hyperbola.
p_ft_monotone: assert property (@(posedge clk) disable iff (!rst_n)
  (payload_octets > $past(payload_octets)) |->
    (tax_ppm_o <= $past(tax_ppm_o)));

// A3. Only the addresses and the type are deletable.
p_ft_deletable_set: assert property (@(posedge clk) disable iff (!rst_n)
  (deletable_ppm_o == ((32'(OCT_DELETABLE) * 1_000_000) / wire_octets_o)));

// A4. The preamble survives any protocol change.
p_ft_preamble_stays: assert property (@(posedge clk) disable iff (!rst_n)
  (preamble_is_deletable == 1'b0));

// A5. And so does the check sequence.
p_ft_fcs_stays: assert property (@(posedge clk) disable iff (!rst_n)
  (fcs_is_deletable == 1'b0));

// A6. So the undeletable part is always the larger share.
p_ft_undeletable_larger: assert property (@(posedge clk) disable iff (!rst_n)
  (undeletable_ppm > deletable_ppm_o));

// A7. A route REPLACES rather than removes, so the saving is a
//     difference and never the whole field.
p_ft_route_replaces: assert property (@(posedge clk) disable iff (!rst_n)
  ((addressing != 2'(ADDRESSING_NAME)) && (route_octets >= 16'(OCT_DELETABLE)))
    |-> (actual_saving_ppm == 32'd0));

// A8. A named-addressing fabric saves nothing by definition.
p_ft_named_saves_nothing: assert property (@(posedge clk) disable iff (!rst_n)
  (addressing == 2'(ADDRESSING_NAME)) |-> (actual_saving_ppm == 32'd0));

// A9. The packet rate is the wire slot divided into the line rate.
p_ft_packet_rate: assert property (@(posedge clk) disable iff (!rst_n)
  (rate_gbps != 0) |->
    (packet_ns_x100 == (wire_octets_o * 8 * 100) / 32'(rate_gbps)));

Group B — the source route (9).

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// B1. No lookup is performed, ever.
p_sr_no_lookup: assert property (@(posedge clk) disable iff (!rst_n)
  (lookup_needed == 1'b0));

// B2. The decode is three stages.
p_sr_three_stages: assert property (@(posedge clk) disable iff (!rst_n)
  (stages_used == 32'(SR_STAGES)));

// B3. And twelve times faster than a lookup pipeline.
p_sr_ratio: assert property (@(posedge clk) disable iff (!rst_n)
  (latency_ratio_x10 == 32'd120));

// B4. The egress port is the route's low bits.
p_sr_port_is_low_bits: assert property (@(posedge clk) disable iff (!rst_n)
  egress_valid |-> (egress_port == $past(route_s2[5:0])));

// B5. The route shifts down by exactly one hop.
p_sr_shift: assert property (@(posedge clk) disable iff (!rst_n)
  egress_valid |-> (route_out == ($past(route_s2) >> 6)));

// B6. Hops remaining decrements by one and never wraps.
p_sr_hops_decrement: assert property (@(posedge clk) disable iff (!rst_n)
  (egress_valid && ($past(hops_s2) != 4'd0)) |->
    (hops_remaining_out == $past(hops_s2) - 4'd1));

// B7. An exhausted route is not forwarded.
p_sr_no_forward_exhausted: assert property (@(posedge clk) disable iff (!rst_n)
  route_exhausted |-> !egress_valid);

// B8. And it is counted.
p_sr_exhausted_counted: assert property (@(posedge clk) disable iff (!rst_n)
  route_exhausted |=> (c_exhausted == $past(c_exhausted) + 32'd1));

// B9. The route field is wide enough for the topology. An elaboration
//     check, written as a property so it appears in the report.
p_sr_route_wide_enough: assert property (@(posedge clk) disable iff (!rst_n)
  (ROUTE_BITS >= route_bits(RADIX, MAX_HOPS)));

Group C — area (8).

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// C1. A named-addressing fabric deletes nothing.
p_ar_named_deletes_nothing: assert property (@(posedge clk) disable iff (!rst_n)
  (addressing == 2'(ADDRESSING_NAME)) |-> (deleted_bce_e4 == 32'd0));

// C2. A routed one deletes exactly the three arrays.
p_ar_routed_deletes_three: assert property (@(posedge clk) disable iff (!rst_n)
  (addressing != 2'(ADDRESSING_NAME)) |->
    (deleted_bce_e4 == (32'(ACL_TCAM_BCE_E4) + 32'(L3_LPM_BCE_E4)
                        + 32'(MAC_TBL_BCE_E4))));

// C3. The reintroduced classifier is charged back.
p_ar_reintroduction_charged: assert property (@(posedge clk) disable iff (!rst_n)
  (replacement_rules != 16'd0) |-> (reintroduced_bce_e4 != 32'd0));

// C4. The net saving never exceeds the gross.
p_ar_net_bounded: assert property (@(posedge clk) disable iff (!rst_n)
  (net_saving_bce_e4 <= deleted_bce_e4));

// C5. And it is a single-digit percentage of the switch.
p_ar_single_digit: assert property (@(posedge clk) disable iff (!rst_n)
  (net_saving_pct_x10 <= 16'd100));

// C6. The undeletable share is above ninety, always.
p_ar_undeletable_dominates: assert property (@(posedge clk) disable iff (!rst_n)
  (undeletable_share_pct >= 16'd90));

// C7. The buffer is sized by traffic rather than by the header.
p_ar_buffer_independent: assert property (@(posedge clk) disable iff (!rst_n)
  (buffer_sized_by_traffic == 1'b1));

// C8. And the SerDes is the same on both sides at the same rate.
p_ar_serdes_shared: assert property (@(posedge clk) disable iff (!rst_n)
  (serdes_is_shared && serdes_is_standard_ip &&
   (shared_serdes_lanes == serdes_lanes)));

Group D — the ecosystem (8).

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// D1. A proprietary fabric has one implementation by definition.
p_ec_one_impl: assert property (@(posedge clk) disable iff (!rst_n)
  !is_standard |-> (independent_implementations == 16'd1));

// D2. Ambiguity needs two readings to be discoverable.
p_ec_ambiguity: assert property (@(posedge clk) disable iff (!rst_n)
  (independent_implementations < 16'd2) |-> !ambiguity_discoverable);

// D3. And so does a conformance oracle.
p_ec_oracle: assert property (@(posedge clk) disable iff (!rst_n)
  (independent_implementations < 16'd2) |-> !conformance_has_an_oracle);

// D4. Capture tooling exists only for a standard.
p_ec_capture: assert property (@(posedge clk) disable iff (!rst_n)
  (capture_tooling_exists == is_standard));

// D5. A single implementation couples the roadmap.
p_ec_roadmap: assert property (@(posedge clk) disable iff (!rst_n)
  (second_source_count < 16'd2) |-> single_vendor_risk);

// D6. Every mechanism the standard does not supply must be built.
p_ec_build_the_rest: assert property (@(posedge clk) disable iff (!rst_n)
  is_standard |-> (mechanisms_to_build ==
    ((mechanisms_needed > mechanisms_supplied)
     ? (mechanisms_needed - mechanisms_supplied) : 16'd0)));

// D7. A model whose authors saw the design is not an oracle.
p_ec_model_blind: assert property (@(posedge clk) disable iff (!rst_n)
  model_authors_saw_design |-> !oracle_is_independent);

// D8. Spec churn past its budget is a reportable condition.
p_ec_churn: assert property (@(posedge clk) disable iff (!rst_n)
  (spec_amendments_since_freeze > amendment_budget) |-> spec_churn_exceeded);

Group E — telemetry (8).

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// E1. The measured tax follows the measured mean frame.
p_tl_measured_tax: assert property (@(posedge clk) disable iff (!rst_n)
  (mean_frame_octets != 32'd0) |->
    (framing_tax_ppm_measured == 32'(tax_ppm(int'(mean_frame_octets)))));

// E2. Without an external decoder, the fabric witnesses only itself.
p_tl_self_witness: assert property (@(posedge clk) disable iff (!rst_n)
  (self_witnessed_only == !external_decoder_present));

// E3. A capture path is present if any of the three mechanisms is.
p_tl_capture_path: assert property (@(posedge clk) disable iff (!rst_n)
  (capture_path_present == (mirror_active || capture_armed || injector_used)));

// E4. Octets accumulate with frames.
p_tl_octets_track: assert property (@(posedge clk) disable iff (!rst_n)
  frame_rx |=> (c_octets == $past(c_octets) + 48'($past(frame_octets))));

// E5. Mirrored frames never exceed received frames.
p_tl_mirror_bounded: assert property (@(posedge clk) disable iff (!rst_n)
  (c_mirror_frames <= c_frames));

// E6. A route error is counted exactly once.
p_tl_route_error_once: assert property (@(posedge clk) disable iff (!rst_n)
  route_exhausted_i |=> (c_route_errors == $past(c_route_errors) + 48'd1));

// E7. Clearing zeroes every counter.
p_tl_clear: assert property (@(posedge clk) disable iff (!rst_n)
  clear |=> ((c_frames == 48'd0) && (c_octets == 48'd0) &&
             (c_drops == 48'd0)));

// E8. The mean is bounded by the maximum frame.
p_tl_mean_bounded: assert property (@(posedge clk) disable iff (!rst_n)
  (c_frames != 0) |-> (mean_frame_octets <= 32'd9018));

Group F — conformance, chosen so no amendment can make them pass (9).

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// F1. Conservation. Arithmetic, not a document.
p_cf_conservation: assert property (@(posedge clk) disable iff (!rst_n)
  window_end |-> (octets_in == octets_out + octets_dropped
                              + octets_resident));

// F2. The route field must name every hop.
p_cf_route_width: assert property (@(posedge clk) disable iff (!rst_n)
  !v_route_too_short);

// F3. An exhausted route must not be forwarded.
p_cf_no_forward: assert property (@(posedge clk) disable iff (!rst_n)
  !v_hop_after_exhaustion);

// F4. A conformance claim needs an oracle its authors did not write.
p_cf_oracle_required: assert property (@(posedge clk) disable iff (!rst_n)
  claims_conformance |-> oracle_independent_i);

// F5. The violation vector is the monitor's own disjunction.
p_cf_vector: assert property (@(posedge clk) disable iff (!rst_n)
  conformant |-> (violations == 4'b0000));

// F6. Conservation is checked only at a consistent sample point --
//     Chapter 19.7 Section 19's class 87 as a guard rather than a trap.
p_cf_sample_consistent: assert property (@(posedge clk) disable iff (!rst_n)
  window_end |-> $stable(snapshot_taken));

// F7. The reference model's version is pinned at sign-off.
p_cf_version_pinned: assert property (@(posedge clk) disable iff (!rst_n)
  signoff_requested |-> (reference_version == SIGNOFF_SPEC_VERSION));

// F8. And an independently written model agrees.
p_cf_independent_agrees: assert property (@(posedge clk) disable iff (!rst_n)
  transaction_observed |-> (observed == independent_model_predicts));

// F9. Amendments after freeze are budgeted.
p_cf_amendment_budget: assert property (@(posedge clk) disable iff (!rst_n)
  signoff_requested |-> (spec_amendments_since_freeze <= AMENDMENT_BUDGET));

Coverage — the states a fabric comparison only reaches with a real workload.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
c_ft_min_frame:    cover property (@(posedge clk) payload_octets <= 16'd46);
c_ft_jumbo:        cover property (@(posedge clk) payload_octets >= 16'd9000);
c_ft_route_bigger: cover property (@(posedge clk) route_octets >= 16'd14);
c_sr_exhausted:    cover property (@(posedge clk) route_exhausted);
c_sr_max_hops:     cover property (@(posedge clk) hops_remaining_in == 4'd5);
c_ar_reintroduced: cover property (@(posedge clk) reintroduced_bce_e4 != 32'd0);
c_ar_small_table:  cover property (@(posedge clk) net_saving_pct_x10 <= 16'd20);
c_ec_no_oracle:    cover property (@(posedge clk) !oracle_is_independent);
c_ec_churn:        cover property (@(posedge clk) spec_churn_exceeded);
c_ec_blind_model:  cover property (@(posedge clk) !model_authors_saw_design);
c_tl_self_only:    cover property (@(posedge clk) self_witnessed_only);
c_tl_no_capture:   cover property (@(posedge clk) !capture_path_present);
c_cf_conservation: cover property (@(posedge clk) v_conservation);
c_cf_unfounded:    cover property (@(posedge clk) v_conformance_unfounded);


21. Verification Scenarios

Fifty-eight scenarios in six groups, plus one directed test random stimulus will not produce.

Group 1 — the framing tax (10).

#ScenarioExpect
146-octet payload84-octet slot; 452 381 ppm — 45.24%
2256-octet payload294-octet slot; 129 252 ppm — 12.93%
31 500-octet payload1 538-octet slot; 24 707 ppm — 2.47%
49 000-octet payload9 038-octet slot; 4 204 ppm — 0.42%
5a 4-octet route replacing 14119 048 ppm saved at 46; 1 106 at 9 000
6a 14-octet routesaving is zero — p_ft_route_replaces
7ADDRESSING_NAMEsaving is zero by definition
8preamble_is_deletable and fcs_is_deletableboth 0, at every setting
9400 Gb/s at 46 octets595 238 k packets/s; 1.68 ns each
10400 Gb/s at 9 000 octets5 532 k packets/s; 180.76 ns each — a factor of 107.6

Group 2 — the source route (10).

#ScenarioExpect
11radix 64, 5 hops6 bits per hop; 30 bits of route
12radix 32, 5 hops5 bits per hop; 25 bits
13radix 128, 7 hops7 bits per hop; 49 bits
14a 30-bit field for radix 64, 5 hopsp_sr_route_wide_enough holds
15a 24-bit field for the samev_route_too_short
16one hop decodedegress is the low 6 bits; route shifts by 6
17five hops in sequencefive decodes, hops reaching zero
18a sixth hop after exhaustionnot forwarded; c_exhausted increments
19lookup_needed at any setting0
20latency, 3 stages against 36 at 1.2 GHz2.50 ns against 30.00; ratio 120

Group 3 — area (9).

#ScenarioExpect
21ADDRESSING_ROUTE, no classifier5.04 × 10⁷ BCE deleted; 178 datapaths; 9.0%
22the same with 256 rules of 256 bits6.55 × 10⁴ reintroduced; net 8.9%
231 024 rules of 512 bits5.24 × 10⁵ reintroduced; net still 8.9%
24a TCAM-based classifier, 256 × 256 b at ×53.28 × 10⁵ reintroduced; net still 8.9%
25ADDRESSING_NAMEnothing deleted; undeletable_share_pct = 100
26undeletable_share_pct at any setting≥ 90 — p_ar_undeletable_dominates
2764 ports at 400 Gb/s256 SerDes lanes, both fabrics
28a 32k MAC table inside the standard9.44 × 10⁶ BCE saved; 33.3 datapaths; 1.7%
29row 28 against row 2218.8% of the advantage, without leaving

Group 4 — the ecosystem (10).

#ScenarioExpect
30is_standard high, 12 vendorsambiguity_discoverable; conformance_has_an_oracle
31is_standard lowindependent_implementations = 1; all four risks
3229 mechanisms needed, 29 supplied0 to build
3329 needed, 0 supplied29 to build
34an independent model whose authors saw the RTLoracle_is_independent low
35an independent model written from the document aloneoracle_is_independent high; conformance is meaningful
3611 amendments against a budget of 2spec_churn_exceeded
37capture_tooling_exists on a proprietary fabriclow
38one vendorno price negotiation, no supply fallback, roadmap coupled
39two vendors who licensed one reference designthe model reports 2; the readings are 1 — see Section 13

Group 5 — telemetry (9).

#ScenarioExpect
40all frames at 1 500 octetsmean 1 500; measured tax 24 707 ppm
41all frames at 9 000mean 9 000; 4 204 ppm
4290% at 64, 10% at 9 000mean 958; 38 153 ppm — 3.82%, describing neither
43no external decoderself_witnessed_only high
44a mirror port activecapture_path_present; mirrored ≤ received
45a route exhaustionc_route_errors increments once
46clear assertedevery counter zero next cycle
47zero framesmean is 0 and the tax is 0, not a division by zero
4816-bin histogram instead of a mean768 bits; 15 360 BCE; 0.054 datapaths

Group 6 — conformance (10).

#ScenarioExpect
49octets balance at a window endno v_conservation
50octets short by one framev_conservation
51the four terms sampled in different cyclesv_conservation fires on a correct design — class 87
52a snapshot mechanism addedit stops firing
53a 24-bit route for radix 64, 5 hopsv_route_too_short
54forwarding after exhaustionv_hop_after_exhaustion
55claims_conformance with no independent oraclev_conformance_unfounded
56the same with oneclean
57spec amended after the version pinp_cf_version_pinned fires
58all four checks clearconformant high

22. Debugging a Fabric With No Second Implementation

Six symptoms, and the first question is different from every other debugging section in this track, because there is nothing to compare against.

SymptomFirst questionWhere to look
packets arriving at the wrong endpointis the route stale, or is the decode wrong?Section 5 — a stale route repeats on every packet until the manager recomputes
a bring-up bug with no visible trafficis there a mirror port or a capture ring?Section 14 — if not, the design's own counters are the only witness
counters that do not balancewere the four terms sampled in one cycle?Section 16 — Chapter 19.7 §19's class 87
a regression that improved without an RTL changewhat is spec_amendments_since_freeze?Section 21's directed test
two endpoints that disagree about a fieldwhich reading of the specification is each using?Section 12 — there is no third party to settle it
latency worse than the 12× predictedis a classifier in the path?Section 7 — a reintroduced classifier adds stages back

Row five is the one with no procedure, and saying so is more useful than pretending otherwise.

On a standard, two implementations disagreeing about a field is settled by a third: a plugfest, a published test suite, a reference implementation, or simply the weight of a thousand parts that read it the other way. On a proprietary fabric there is no third, so the disagreement is settled by whoever wrote the document, which means it is settled by seniority rather than by evidence.


23. Misconceptions

Misconception 1 — "Ethernet's framing overhead is 45%."

The wrong model: the tax is a property of the frame format, so it is the same everywhere.

What it costs: a fabric built to recover an overhead it does not have. Section 2: 45.24% is the tax at a 46-octet payload and 0.4204% at 9 000, and the fabrics that leave the standard carry large transfers. The honest deletable saving on jumbo traffic is 0.155% of the wire.

The corrected model: the tax is 38 ÷ (payload + 38), a hyperbola whose interesting region is entirely at the small endChapter 8.3 §2 — and quoting the small end for traffic at the large end is choosing the wrong point on a curve everybody knows.

Misconception 2 — "a proprietary fabric can delete the preamble and the check sequence."

The wrong model: those are Ethernet conventions and a new protocol need not have them.

What it costs: a receiver that cannot find the first bit, or a fabric that delivers corrupted data silently. Section 2: the preamble exists for clock recovery and frame delimitation — Chapter 5.2 §3 — and the FCS because wires corrupt. PCIe has an LCRC and InfiniBand has two CRCs; the field is renamed, not removed.

The corrected model: only the addresses and the EtherType are deletable, and a route replaces rather than removes them. Four octets against fourteen is the real saving.

Misconception 3 — "leaving the standard saves most of the switch."

The wrong model: forwarding tables are what a switch is, so deleting them is transformative.

What it costs: a programme justified by an area saving of 8.9%. Section 6: the three deletable arrays are 5.04 × 10⁷ BCE against a 5.62 × 10⁸ BCE switch, and the packet buffer alone is 91.1%. The SerDes is not in the table and is shared.

The corrected model: the two most expensive things in a switch are protocol-independent — the buffer is sized by the traffic and the SerDes by physics — so a proprietary fabric differentiates in 8.9% of the die and the header.

Misconception 4 — "the win is bandwidth."

The wrong model: a purpose-built fabric moves more data than a general-purpose one.

What it costs: competing where the gap is tenths of a per cent. Both fabrics use the same SerDes at the same rate, and Chapter 9.6 took the standard to 800 Gb/s. Section 4: the real win is the lookup — 30 ns against 2.5, twelve times, 137.5 ns across five hops — and it is a latency, quoted as a bandwidth.

The corrected model: name the workload. For Chapter 23.2's all-reduce, 137.5 ns is 0.00001% of the job. For an 8-octet remote atomic on a 500 ns round trip it is 55%. The fabric is worth building for the second.

Misconception 5 — "the ecosystem is a few tools we will write later."

The wrong model: a standard supplies a frame format; everything else is optional tooling.

What it costs: a first silicon with no external decoder, debugged by its own counters. Section 8: twenty-nine chapters of this track — 22.5% of it — are standardised behaviour with nothing to do with framing or forwarding, and Chapter 21.9's entire apparatus presumes a tap, a capture device and a decoder that nobody sells for a new fabric.

The corrected model: the ecosystem is a network effect, so the vendor pays the full cost of every tool rather than a share of it. Build the capture path into first silicon or accept that bring-up has no external witness.

Misconception 6 — "our conformance suite passes, so the design conforms."

The wrong model: a green regression against the reference model means the design matches the specification.

What it costs: sign-off on self-consistency. Section 20's class 114: the design and the oracle share a custodian, so the cheapest repair for a failure is to amend the oracle, and Section 21's directed test shows a property going from 412 failures to zero with unchanged RTL.

The corrected model: an oracle is independent only if its authors never saw the design. The repair is a second team reading the document alone — a few engineer-months, and it reconstructs the one thing leaving the standard gave up. The properties that survive are the ones derived from arithmetic rather than from a document.


24. Interview Questions

Six, with what a strong answer contains.

1. What does Ethernet's framing actually cost?

Thirty-eight octets, and the percentage depends entirely on the payload. 38 ÷ (payload + 38): 45.24% at a 46-octet payload, 2.47% at 1 500, 0.4204% at 9 000. A strong answer goes further: only fourteen of the thirty-eight are deletable — the addresses and the type — because the preamble is needed for clock recovery and the FCS because wires corrupt. So the honest saving on jumbo traffic is 0.155%, and a route field replacing the addresses makes it 0.111%.

2. Where is a proprietary fabric's real advantage?

The lookup. Chapter 23.3 §2's forwarding pipeline is 36 stages at 1.2 GHz — 30 ns; a source-routed decode is a shift and a demultiplex, three stages, 2.50 ns. That is 12× per hop and 137.5 ns across five hops. A strong answer names the structural reason: a name must be resolved against a table and a route is an index, so the saving is not an optimisation, it is the absence of a mechanism.

3. And then what does 137.5 nanoseconds buy?

It depends entirely on the operation. For Chapter 23.2's all-reduce moving 15.984 GB per participant over 1.279 s, it is 0.00001%. For an 8-octet remote atomic with a 500 ns round trip it is 55%. The strong answer says that the fabrics which succeed are the ones built for the second case, and that the comparison cannot be made without naming the workload and the percentile.

4. How much of a switch does leaving the standard actually save?

8.9%. The three deletable arrays are ACL TCAM 2.10 × 10⁷, L3/LPM 1.68 × 10⁷, MAC table 1.26 × 10⁷ — 5.04 × 10⁷ BCE against a 5.62 × 10⁸ BCE switch — less a small reintroduced classifier. The packet buffer is 91.1% and is protocol-independent, and the SerDes is not in the table and is the same IP on both sides. A strong answer adds the quiet finding: a 32k-entry MAC table instead of 128k saves 1.7% while remaining fully standard, which is 18.8% of the whole advantage.

5. What does a fabric with one implementation give up in verification?

The oracle. A conformance property compares a design against a reference model built from a specification; on a standard that specification has a thousand independent readings and cannot be revised to accommodate one part. On a proprietary fabric the design and the document share a custodian, so the cheapest repair for a failure is an amendment, and nothing in the environment distinguishes an amendment from a fix. A strong answer names the repair — a second team writing a model from the document alone, never having seen the RTL — and the cheap partial repair: properties derived from arithmetic, like conservation, which no amendment can falsify.

6. Somebody shows you a fabric comparison. What do you ask?

Four questions. What payload size does the framing argument assume? Is the latency win in the lookup or the framing? What fraction of the die is actually different? How many independent implementations exist? A strong answer notes that the last one determines four separate consequences — ambiguity discovery, the conformance oracle, capture tooling and the second source — and that none of them appears in a technical comparison. The best answers close with the general point: a market's units are a model of what can be compared, not of what is scarce, and the binding quantity here — tail latency on small operations under load — cannot be compared across vendors and so is never published.


25. Questions and Answers


26. What's Next

Module 24 is complete. Three chapters set Ethernet against a load/store fabric, a lossless HPC fabric and a fabric that left the standard, and each derived a number rather than ticking a box.

ChapterThe number
Chapter 24.1425 088 BCE of return-path state at 400 Gb/s, against an Ethernet transmitter's zero
Chapter 24.291.47% of a merchant switch reserved, at 2 km, per lane
this chapter137.5 ns and 8.9% of a die, against twenty-nine chapters of ecosystem

Module 25 turns from what Ethernet is not to what engineers believe it is. Six myths, each of which causes real bugs and bad interviews, and each corrected by deriving the number that kills it rather than by asserting the correction.

Chapter 25.1 takes the largest of them — that Ethernet and TCP/IP are the same thing. It builds the layer boundary in RTL and derives what a MAC can and cannot see: a receive MAC parses eighteen octets of a 1 518-octet frame — 1.19% — and reads exactly zero octets of the IP and TCP headers. Then it prices what believing the myth costs in silicon, because Chapter 23.4's connection context cache is 66.2% of that NIC and every bit of it is above the Ethernet boundary.

Chapter 25.2 takes the second — that MAC addresses are globally unique forever — and derives the collision arithmetic at data-centre scale, which is not the arithmetic people quote.

Continue learning

Standards & specifications

Governing standard
IEEE Std 802.3 (Ethernet)(opens IEEE in a new tab)

Defines the Ethernet MAC, the media-independent interfaces and the physical-layer sublayers, including framing, access control, auto-negotiation and per-rate PHY specifications. VLAN tagging, priority and time-sensitive shaping are defined by IEEE 802.1, not by 802.3.

This page also covers RTL structure, verification approach and debugging technique. Those are engineering practice built on the standard, not requirements the standard itself imposes.

Where this fits

Part of the Ethernet curriculum.