Ethernet · Module 23
An FPGA Ethernet Design
A 1 024-bit datapath, a 42 000-LUT byte aligner, memory that comes in 36 864-bit blocks, and a clock frequency that is an output of the build rather than an input to it.
Every chapter in Modules 19 to 23 priced a design in a unit that assumes a fixed process and the freedom to place anything anywhere. An FPGA offers neither, and three things change.
| An ASIC | An FPGA | |
|---|---|---|
| logic | as many gates as you draw | a fixed inventory of lookup tables |
| memory | any size, anywhere | fixed blocks — 36 kbit at a time |
| clock frequency | an input to the design | an output of place-and-route |
Row three is the chapter's sharpest result and Section 20's refused property. A rate is width × frequency; in an ASIC the designer picks the frequency and the tools meet it, and in an FPGA the tools report what they achieved. A property proved in simulation at 195.3 MHz is asserted about a build that closed at whatever it closed at.
Which forces the first design decision.
| Line rate | Datapath width | Clock required |
|---|---|---|
| 10 Gb/s | 64 bits | 156.2 MHz |
| 25 Gb/s | 128 bits | 195.3 MHz |
| 100 Gb/s | 512 bits | 195.3 MHz |
| 100 Gb/s | 1 024 bits | 97.7 MHz |
| 400 Gb/s | 2 048 bits | 195.3 MHz |
An FPGA reaches 100 Gb/s by going wide, not fast — and going wide has a cost that is not proportional.
| 64-bit | 512-bit | 1 024-bit | |
|---|---|---|---|
| byte positions a frame can start at | 8 | 64 | 128 |
| the aligner, in LUT6 | ≈ 192 | ≈ 10 752 | ≈ 41 984 |
| growth | — | 56× | 219× |
Doubling the datapath width quadruples the byte aligner, because it is a barrel shifter whose positions and whose width both double.
And memory stops being a number.
| Structure | Bits | BRAM36 used | Efficiency |
|---|---|---|---|
| a 64-entry × 96-bit table | 6 144 | 1 | 16.7% |
| Chapter 19.6 §4's 12 KiB reorder buffer | 98 304 | 3 | 88.9% |
| Chapter 23.4's 450-descriptor store | 57 600 | 2 | 78.1% |
| Chapter 23.3's 16k MAC table | 1 572 864 | 43 | 99.2% |
Row one is the quantisation tax. A small table that Chapter 23.3 §2's unit prices at 6 144 BCE occupies 36 864 bits of a block RAM — six times its own size — because a block is the smallest thing an FPGA has. Large structures are efficient and small ones are not, which reverses the ASIC intuition that small is cheap.
1. Scope — A Fixed Inventory, and a Frequency You Are Told
Scope: what a complete Ethernet MAC and PHY interface cost in an FPGA's resources, how the datapath width is chosen, and what the implementation flow does to every rate property in the track.
Not in scope: the SerDes. A modern FPGA's transceivers are hard blocks with their own specifications; this chapter starts at the PCS/PMA boundary Chapter 3.4 §3 drew, and treats the transceiver as a source and sink of aligned symbols.
Three things are different from every earlier chapter, and all three are about the implementation rather than the design.
| Difference | Section | |
|---|---|---|
| 1 | logic is a fixed inventory, not an area | 2 and 19 |
| 2 | memory comes in blocks, so small structures waste | 6 and 7 |
| 3 | the clock frequency is a result, not a choice | 12, 13 and 20 |
Difference 3 is the one that changes verification. Every rate in Modules 19 to 23 — 100 Gb/s, 148.81 Mpps, 6.72 ns per packet — is width × frequency, and in an ASIC the frequency is a target that the flow is obliged to meet. In an FPGA it is a number the flow reports, it varies with the placement seed, the device speed grade, the temperature and the tool version, and a design that closed at 195 MHz last week may close at 188 this week with no source change.
In an ASIC, timing closure is a schedule risk. In an FPGA it is a functional one, because the design's rate is a function of the number that comes out.
And one thing that is not different. Chapter 4.1 §3's MAC/PHY split, Chapter 5.1's frame, Chapter 6.3's residue and Chapter 19.7's counters are all exactly what they were. The protocol does not know what it is implemented in; what changes is the cost of each piece and the confidence with which a rate can be claimed.
2. The Resource Inventory, and What a MAC Costs In It
An FPGA's resources are four kinds and they are not interchangeable.
| Resource | What it is | Granularity |
|---|---|---|
| LUT6 | a 6-input lookup table — any function of 6 inputs | one |
| flip-flop | a register bit, paired with the LUTs | one, and there are ≈2 per LUT |
| BRAM36 | a 36 kbit dual-port block memory | 36 864 bits |
| URAM | a 288 kbit single-clock block | 294 912 bits |
| DSP | a multiplier-accumulator | one |
Row five is worth dismissing immediately: an Ethernet MAC uses almost no DSPs. Chapter 6.4's CRC is XOR trees, Chapter 19.3's assembler is comparisons and counters, and the only arithmetic anywhere in Module 19 is incrementing counters. A design that consumes DSPs in an Ethernet datapath has usually had a multiply inferred by accident.
Now the MAC, mapped.
| From Chapter 19.7 §19 | ASIC | FPGA |
|---|---|---|
| the receive datapath's registers | 14 166 flops | 14 166 flip-flops |
| its combinational logic | 39 445 XOR terms | ≈ 7 900 LUT6 |
| its memory | 55 KiB | 13 BRAM36 |
Row one maps directly and is the only row that does. A flip-flop in an FPGA is a flip-flop; 14 166 of them is about 886 CLB slices' worth, which on a mid-size device is a few per cent of the flip-flops available and is never the binding resource.
Row two needs a conversion and it is the one that surprises. Chapter 19.4 §19's XOR terms map to LUT6s at roughly five terms per LUT, because a LUT6 computes any function of six inputs and an XOR of six inputs is one of them. 39 445 terms is about 7 900 LUT6 — which is small. An FPGA is extremely good at XOR trees, and the CRC that dominates an ASIC MAC's combinational area is not the binding resource here either.
Row three is where the FPGA's shape shows. 55 KiB is 450 560 bits, which needs 13 BRAM36 at 94.0% efficiency — good, because it is one large structure. The trouble is never the large structures.
Which leaves the question of what is binding, and it is none of the above.
| Candidate | Binding? | Why |
|---|---|---|
| flip-flops | no | 14 166 is a few per cent of a mid device |
| LUT6 for CRC and control | no | ≈ 7 900, and FPGAs like XOR |
| BRAM for the FIFOs | no | 13 blocks, 94.0% efficient |
| LUT6 for the byte aligner | YES | ≈ 41 984 at a 1 024-bit datapath — Section 4 |
| timing closure | YES | Sections 12 and 13 |
Rows four and five are the chapter, and neither of them is a quantity that appears anywhere in Modules 19 to 23.
3. RTL 1 — The FPGA Package and the Resource Model
// ---------------------------------------------------------------------
// fpga_pkg -- an Ethernet design's resources in an inventory rather
// than an area.
//
// Chapter 23.3's package held an area model in bitcell equivalents, which
// assumes memory can be any size. An FPGA's cannot: it comes in
// 36-kilobit blocks, and Section 6 is what that does to every small
// structure in the track.
// ---------------------------------------------------------------------
package fpga_pkg;
// ---- the inventory ------------------------------------------------------
localparam int LUT_INPUTS = 6;
localparam int FF_PER_LUT = 2;
localparam int BRAM_BITS = 36864; // 36 kbit
localparam int URAM_BITS = 294912; // 288 kbit
// A mid-size device, for the fit checks in Section 19.
localparam int DEVICE_LUTS = 500000;
localparam int DEVICE_FFS = 1000000;
localparam int DEVICE_BRAMS = 1000;
localparam int DEVICE_URAMS = 320;
// ---- the conversion from Modules 19 to 23's units ------------------------
// An XOR of up to six terms is ONE LUT6, so an n-term XOR tree is
// about n/5 LUTs once the tree's internal nodes are counted.
localparam int TERMS_PER_LUT = 5;
function automatic int luts_of_xor_terms(int terms);
return (terms + TERMS_PER_LUT - 1) / TERMS_PER_LUT;
endfunction
// THE function this chapter exists for: a memory's real cost is the
// number of blocks it occupies, not the bits it holds.
function automatic int brams_of_bits(int bits);
return (bits + BRAM_BITS - 1) / BRAM_BITS;
endfunction
function automatic int bram_efficiency_pct(int bits);
automatic int n = brams_of_bits(bits);
if (n == 0) return 0;
return (bits * 100) / (n * BRAM_BITS);
endfunction
// ---- the rate identity ---------------------------------------------------
// Everything in Modules 19 to 23 is width x frequency. In an ASIC the
// frequency is chosen; here it is reported.
function automatic int mhz_for(int gbps, int width_bits);
return (gbps * 1000) / width_bits;
endfunction
function automatic int gbps_at(int mhz, int width_bits);
return (mhz * width_bits) / 1000;
endfunction
// Section 4: a frame can begin at any octet of a datapath word, so
// the aligner is a barrel shifter over width/8 positions.
function automatic int aligner_luts(int width_bits);
automatic int positions = width_bits / 8;
// ~ (positions/4 + positions/16 + 1) LUT6 per output bit
return width_bits * ((positions / 4) + (positions / 16) + 1);
endfunction
// ---- what the flow reports rather than what the design asks ---------------
typedef struct packed {
logic [15:0] requested_mhz;
logic [15:0] achieved_mhz; // an OUTPUT of place and route
logic [15:0] seed;
logic [7:0] speed_grade;
logic closed;
} timing_result_t;
endpackageClassification: an inventory model whose two most useful functions are a ceiling division and a barrel-shifter estimate.
What it teaches: that brams_of_bits is a ceiling division and that single fact reverses an ASIC intuition. In Chapter 23.3 §2's unit a structure's cost is its size; here a structure's cost is the number of 36 864-bit blocks it lands in. A 6 144-bit table costs a whole block — 16.7% efficient — and a 1 572 864-bit table costs 43 blocks at 99.2%. Small is expensive and large is cheap, which is the opposite of everything the track has assumed.
And it teaches that timing_result_t has achieved_mhz as a field rather than a parameter. Every earlier package in the track declared a clock frequency as a constant the design was built to. This one declares it as a result with a seed and a speed grade attached, because that is what it is — and Section 20's refused property is what happens when a rate property is written as though it were the other kind.
Deliberately simplified: TERMS_PER_LUT = 5 is an approximation that is right for balanced XOR trees and optimistic for logic with many shared sub-expressions. aligner_luts is a rough mux-tree estimate and a real synthesiser does better with dedicated multiplexers in the CLB. DEVICE_* are one device's numbers and FPGA families differ by more than an order of magnitude. And brams_of_bits ignores the width-versus-depth aspect ratio: a 36 kbit block configured 512×72 cannot serve a 1024-bit wide access without several blocks in parallel, so a wide datapath's FIFOs use more blocks than their bit count implies.
Production implication: the aspect-ratio effect is the one that actually blows FPGA memory budgets and the model above hides it. A block RAM is 36 kbit but its widest port is typically 72 bits, so a FIFO for a 1 024-bit datapath needs fifteen blocks in parallel just to be wide enough — regardless of how shallow it is. A 1 024-bit × 512-deep FIFO is 524 288 bits, which is 15 blocks by capacity and 15 by width, so the two agree and it fits exactly — but a 1 024-bit × 64-deep one is 65 536 bits, 2 blocks by capacity and still 15 by width, and it costs seven and a half times its own size. Wide datapaths make shallow memories expensive, and Section 4's width decision therefore reaches into every FIFO in the design.
// ---------------------------------------------------------------------
// resource_model -- add up a design's four resource kinds and report
// which one is binding.
//
// The ASIC question was "how much area". The FPGA question is "which
// of four fixed pools runs out first", and the answer is usually not
// the one the ASIC intuition predicts.
// ---------------------------------------------------------------------
module resource_model
import fpga_pkg::*;
(
input logic clk,
input logic rst_n,
input logic [31:0] flops_required,
input logic [31:0] xor_terms_required,
input logic [31:0] control_luts,
input logic [31:0] aligner_luts_in,
input logic [31:0] memory_bits,
input logic [15:0] memory_structures,
output logic [31:0] luts_used,
output logic [31:0] ffs_used,
output logic [31:0] brams_used,
output logic [15:0] lut_pct,
output logic [15:0] ff_pct,
output logic [15:0] bram_pct,
output logic [1:0] binding_resource,
output logic fits,
output logic [15:0] aligner_share_pct
);
localparam int BIND_LUT = 2'd0;
localparam int BIND_FF = 2'd1;
localparam int BIND_BRAM = 2'd2;
localparam int BIND_NONE = 2'd3;
always_comb begin
luts_used = 32'(luts_of_xor_terms(int'(xor_terms_required)))
+ control_luts + aligner_luts_in;
ffs_used = flops_required;
// Every structure pays its own ceiling, which is Section 6's point:
// the same total bits split across more structures costs more.
brams_used = 32'(memory_structures)
+ 32'(brams_of_bits(int'(memory_bits)));
lut_pct = 16'((luts_used * 100) / DEVICE_LUTS);
ff_pct = 16'((ffs_used * 100) / DEVICE_FFS);
bram_pct = 16'((brams_used * 100) / DEVICE_BRAMS);
if ((lut_pct >= ff_pct) && (lut_pct >= bram_pct)) binding_resource = 2'(BIND_LUT);
else if (ff_pct >= bram_pct) binding_resource = 2'(BIND_FF);
else binding_resource = 2'(BIND_BRAM);
fits = (lut_pct <= 16'd100) && (ff_pct <= 16'd100)
&& (bram_pct <= 16'd100);
// The figure that decides the datapath width.
aligner_share_pct = (luts_used == 0) ? 16'd0
: 16'((aligner_luts_in * 100) / luts_used);
end
endmoduleClassification: four utilisations and a verdict about which pool runs out first.
What it teaches: that aligner_share_pct is typically the largest single item in the LUT budget. With a 1 024-bit datapath the aligner is ≈41 984 LUT6 and the CRC — which dominates the ASIC's combinational area — is ≈7 900. The aligner is five times the CRC, and it exists only because a frame may begin at any octet of a very wide word.
And it teaches that brams_used adds a per-structure term. Splitting the same total bits across more structures costs more blocks, because each pays its own ceiling: four 6 144-bit tables cost four blocks (16.7% each) and one 24 576-bit table costs one (66.7%). In an ASIC those are the same memory; here the second is four times cheaper.
Deliberately simplified: luts_of_xor_terms and aligner_luts_in are estimates the synthesiser will improve on or fail to meet by tens of per cent. There is no routing term at all — an FPGA design can fail to place at 60% LUT utilisation because the interconnect is exhausted, and nothing here expresses that. binding_resource reports one winner where a design is usually constrained by two at once. And the model says nothing about timing, which Sections 12 and 13 show is the constraint that actually binds.
Production implication: the missing routing term is why FPGA utilisation targets are so much lower than the numbers here suggest. A design at 90% LUT utilisation will usually fail to close timing or fail to route, because place-and-route needs slack to find short paths; the practical ceiling is 70 to 80% and the last 20% of a device is effectively unusable for anything with a timing constraint. So a fit check against DEVICE_LUTS is optimistic by about a quarter, and a design that "fits" at 85% by this model is a design that will not close. Budget against 75% and treat the remainder as routing headroom.
4. The Datapath Width, and the Aligner It Forces
One decision determines the whole design, and it is forced by a frequency the designer does not control.
rate = width × frequency
| Line rate | Width | Frequency needed | Achievable in an FPGA? |
|---|---|---|---|
| 10 Gb/s | 64 bits | 156.2 MHz | comfortably |
| 25 Gb/s | 64 bits | 390.6 MHz | no |
| 25 Gb/s | 128 bits | 195.3 MHz | yes, with care |
| 100 Gb/s | 256 bits | 390.6 MHz | no |
| 100 Gb/s | 512 bits | 195.3 MHz | yes, with care |
| 100 Gb/s | 1 024 bits | 97.7 MHz | comfortably |
| 400 Gb/s | 1 024 bits | 390.6 MHz | no |
| 400 Gb/s | 2 048 bits | 195.3 MHz | yes, with care |
An FPGA reaches a rate by going wide, because the frequency ceiling is a property of the fabric — and roughly 200 MHz is where a non-trivial datapath starts to become difficult and 400 MHz is where it stops being realistic.
The cost of going wide is not proportional, and this is the section's result.
| Width | Byte positions | Aligner, LUT6 | Against 64-bit |
|---|---|---|---|
| 64 | 8 | ≈ 192 | 1× |
| 128 | 16 | ≈ 768 | 4× |
| 256 | 32 | ≈ 2 816 | 14.7× |
| 512 | 64 | ≈ 10 752 | 56× |
| 1 024 | 128 | ≈ 41 984 | 219× |
The growth is quadratic, and the reason is structural. A frame may begin at any octet of the datapath word, so the aligner is a barrel shifter over width ÷ 8 positions, and each of its width output bits needs a mux of that many inputs. Doubling the width doubles the positions and doubles the bits, so the cost quadruples.
A 1 024-bit datapath's byte aligner is about 42 000 LUT6 — five times the entire CRC engine, and the largest single structure in an FPGA Ethernet MAC.
Which makes the width decision a three-way trade rather than a two-way one.
| Width | Frequency needed | Aligner cost | Timing risk |
|---|---|---|---|
| 256 | 390.6 MHz | 2 816 LUT | very high |
| 512 | 195.3 MHz | 10 752 | moderate |
| 1 024 | 97.7 MHz | 41 984 | low |
Narrow is cheap in logic and expensive in timing; wide is the reverse — and the correct choice depends on a frequency the designer will not know until the build runs. Section 12 is what that does to the design flow and Section 20 is what it does to the properties.
One mitigation worth naming, because it changes the table. If the design can guarantee that a frame always starts at octet zero of a word — by inserting inter-frame idle to pad every frame to a whole number of words — the aligner disappears entirely. The cost is bandwidth: padding every frame to a 128-octet boundary wastes up to 127 octets per frame, which at minimum frame size is 66% of the wire. So the mitigation is available and is almost never affordable, except on a link whose traffic is known to be large frames.
5. RTL 2 — The Datapath Width Model
// ---------------------------------------------------------------------
// datapath_width_model -- choose a width from a rate and a frequency
// the design does not control.
//
// This is the only block in Modules 19 to 23 whose central input is a
// number that arrives from the implementation flow. Section 20's
// refused property is about writing it as though it did not.
// ---------------------------------------------------------------------
module datapath_width_model
import fpga_pkg::*;
(
input logic clk,
input logic rst_n,
input logic [15:0] line_rate_gbps,
input logic [15:0] achieved_mhz, // from the implementation flow
input logic [15:0] width_bits,
input logic pad_to_word,
output logic [15:0] required_mhz,
output logic [31:0] aligner_luts_o,
output logic [15:0] achieved_gbps,
output logic meets_rate,
output logic [15:0] margin_pct,
output logic [15:0] padding_waste_pct,
output logic [15:0] min_width_for_mhz,
output logic width_is_a_timing_decision
);
always_comb begin
required_mhz = 16'(mhz_for(int'(line_rate_gbps), int'(width_bits)));
achieved_gbps = 16'(gbps_at(int'(achieved_mhz), int'(width_bits)));
// The aligner disappears if every frame starts at octet zero, and
// Section 4 shows what that costs on the wire.
aligner_luts_o = pad_to_word ? 32'd0
: 32'(aligner_luts(int'(width_bits)));
meets_rate = (achieved_mhz >= required_mhz);
margin_pct = (required_mhz == 0) ? 16'd0
: ((achieved_mhz * 16'd100) / required_mhz);
// Padding a 64-octet frame to a 128-octet word wastes 64 of 128.
padding_waste_pct = pad_to_word ? 16'd50 : 16'd0;
// The narrowest datapath that meets the rate at the achieved
// frequency -- which can only be computed after the build.
min_width_for_mhz = (achieved_mhz == 0) ? 16'hFFFF
: 16'((32'(line_rate_gbps) * 1000)
/ 32'(achieved_mhz));
// Always true, and it is the chapter's structural claim.
width_is_a_timing_decision = 1'b1;
end
endmoduleClassification: a width chooser whose most important input arrives after the design is finished.
What it teaches: that min_width_for_mhz can only be computed after a build. The narrowest datapath that meets the rate depends on the frequency the flow achieved, so the design's most structural parameter — the width, which sets the aligner, every FIFO's aspect ratio and the whole pipeline — is chosen from a number that is a result of choosing it. That circularity is real and is why FPGA datapath designs are conservative: pick a width wide enough that any plausible frequency works, and pay the aligner.
And it teaches that padding_waste_pct is 50% at minimum frame size on a 128-octet word and is the reason the mitigation is rarely taken. Padding removes the aligner entirely — ≈41 984 LUT6, the largest structure in the design — at the cost of half the wire on small frames. On a link carrying large frames the trade is excellent; on one carrying minimum frames it is unthinkable, and the same design must usually serve both.
Deliberately simplified: padding_waste_pct is a constant 50 rather than a function of the frame-size distribution — the true figure is between 1.0% for 9 000-octet frames and 50% for 64-octet ones on a 128-octet word. meets_rate compares frequencies and ignores that a design failing timing by 3% may still be usable with a wider datapath and a re-run. achieved_mhz is an input with no notion of a seed or a speed grade, which Section 12 shows is the whole problem. And nothing models the possibility of two clock domains — a wide slow datapath feeding a narrow fast one — which is a real and common arrangement.
Production implication: the two-clock-domain arrangement is the practical escape from this section's trade and it is worth stating because it moves the cost rather than removing it. A design can run a 1 024-bit datapath at 97.7 MHz where the packet's bytes live, and a narrower, faster control path where the per-frame decisions live — so the aligner stays wide and the state machines stay small. What it adds is Chapter 4.4's problem inside the design: a clock-domain crossing on every signal between them, with the synchroniser latency and the metastability analysis that implies. It is the right answer for most FPGA MACs and it is why their block diagrams have more clock domains than an ASIC's.
6. Memory in Blocks, and the Quantisation Tax
Chapter 23.3 §2's unit prices a memory by its bits. An FPGA prices it by the blocks it lands in, and the difference is large for everything small.
Every memory in Modules 19 to 23, mapped to 36 864-bit blocks.
| Structure | Source | Bits | BRAM36 | Efficiency |
|---|---|---|---|---|
| a 64-entry × 96-bit table | a small lookup | 6 144 | 1 | 16.7% |
| Chapter 23.4's 450-descriptor store | §7 | 57 600 | 2 | 78.1% |
| Chapter 19.6's reorder buffer | §4 | 98 304 | 3 | 88.9% |
| Chapter 22.1's 512-entry delay line | §13 | 1 024 | 1 | 2.8% |
| Chapter 19.7's counter bank | §19 | 1 728 | 1 | 4.7% |
| Chapter 23.3's 16k MAC table | §4 | 1 572 864 | 43 | 99.2% |
| Chapter 23.3's 64 MB packet buffer | §4 | 5.12 × 10⁸ | 13 889 | 99.997% |
Rows four and five are the quantisation tax at its worst. Chapter 22.1 §13's delay line is 1 024 bits and occupies a 36 864-bit block — 2.8% used, thirty-six times its own size. Chapter 19.7's counters are 1 728 bits and occupy another.
And row seven is the opposite problem: it does not fit at all. A 64 MB packet buffer needs 13 889 BRAM36 and a large FPGA has around a thousand. Chapter 23.3's switch cannot be built in an FPGA at that buffer depth — not because of area but because the resource does not exist in that quantity, and the design must use external memory, which brings Chapter 19.6's entire latency-hiding problem back.
In an ASIC a memory's cost is its size. In an FPGA it is
ceil(bits ÷ 36 864)blocks, plus enough blocks in parallel to be as wide as the access — so small memories cost a block each and wide ones cost by width regardless of depth.
The rule, stated so it can be applied.
| Structure's shape | Cost driver | Remedy |
|---|---|---|
| small and narrow | the ceiling — one block minimum | merge it into a larger memory |
| small and wide | the width — blocks in parallel | narrow the access, or use registers |
| large and narrow | capacity, efficiently | nothing needed |
| large and wide | capacity, efficiently | nothing needed |
Row one's remedy is the one that changes a design. Four small tables of 6 144 bits cost four blocks at 16.7% each; merged into one 24 576-bit memory with a two-bit table selector prepended to the address, they cost one block at 66.7% — a saving of three blocks for the price of two address bits. In an ASIC the merge is pointless; here it is a factor of four.
Row two's remedy matters at the widths Section 4 forces. A 1 024-bit-wide structure needs 15 blocks in parallel whatever its depth, so a shallow wide FIFO should be built from flip-flops instead: a 1 024 × 8 FIFO is 8 192 flip-flops, which is under 1% of a mid device's supply and costs no blocks at all.
7. RTL 3 — The BRAM Quantiser
// ---------------------------------------------------------------------
// bram_quantiser -- a memory's real cost in an FPGA, which is neither
// its bits nor its bytes.
//
// Chapter 23.3 Section 2's BCE unit assumes memory is continuous. This
// block is what happens when it comes in 36-kilobit blocks with a
// maximum port width.
// ---------------------------------------------------------------------
module bram_quantiser
import fpga_pkg::*;
#(
parameter int MAX_PORT_BITS = 72
)(
input logic clk,
input logic rst_n,
input logic [31:0] depth_entries,
input logic [15:0] entry_bits,
input logic prefer_registers,
output logic [31:0] total_bits,
output logic [15:0] blocks_by_capacity,
output logic [15:0] blocks_by_width,
output logic [15:0] blocks_used,
output logic [15:0] efficiency_pct,
output logic [31:0] ff_alternative,
output logic registers_are_cheaper,
output logic exceeds_device,
output logic [15:0] bce_would_say
);
always_comb begin
total_bits = depth_entries * 32'(entry_bits);
blocks_by_capacity = 16'(brams_of_bits(int'(total_bits)));
// A block's widest port is limited, so a wide access needs blocks
// in parallel however shallow it is.
blocks_by_width = 16'((32'(entry_bits) + MAX_PORT_BITS - 1)
/ MAX_PORT_BITS);
blocks_used = (blocks_by_capacity > blocks_by_width)
? blocks_by_capacity : blocks_by_width;
efficiency_pct = (blocks_used == 0) ? 16'd0
: 16'((total_bits * 100) / (32'(blocks_used) * BRAM_BITS));
// Section 6's row-two remedy: a shallow wide structure belongs in
// flip-flops.
ff_alternative = total_bits;
registers_are_cheaper = prefer_registers ||
(efficiency_pct < 16'd25);
exceeds_device = (blocks_used > 16'(DEVICE_BRAMS));
// What Chapter 23.3's unit would have reported -- correct for an
// ASIC and wrong here by the efficiency factor.
bce_would_say = 16'(total_bits / 1000);
end
endmoduleClassification: a ceiling division, a width division, and a maximum of the two.
What it teaches: that blocks_used is the maximum of two independent constraints and a design can be bound by either. A deep narrow memory is bound by capacity; a shallow wide one by width; and the two give wildly different answers for the same bit count. A 1 024-bit × 64-entry FIFO is 65 536 bits — two blocks by capacity and fifteen by width — so it costs seven and a half times what its size suggests.
And it teaches that registers_are_cheaper has a threshold that is worth taking seriously. Below 25% block efficiency a structure is usually better built from flip-flops: the 1 024 × 8 FIFO above is 8 192 flip-flops against 15 blocks, and flip-flops are the resource an Ethernet design is least short of — Section 2 showed the whole MAC's 14 166 are a few per cent of a device.
Deliberately simplified: MAX_PORT_BITS = 72 is one family's figure and block RAMs can often be cascaded or configured in several aspect ratios, which a real mapper exploits. blocks_by_capacity ignores that two independent memories cannot share a block even when both are small — the model would need a packing pass. ff_alternative reports bits rather than flip-flops plus the multiplexing logic a register file needs, which for a deep structure is substantial. And bce_would_say is included purely to show the gap against Chapter 23.3's unit.
Production implication: the packing pass the model lacks is exactly what a designer must do by hand, and it is the largest available saving in a memory-heavy FPGA design. Every small structure in Modules 19 to 23 — Chapter 19.7's counters at 1 728 bits, Chapter 22.1's delay line at 1 024, a 64-entry lookup at 6 144 — costs one whole block each, so five such structures cost five blocks and use 5% of them. Merged behind a shared address decoder they cost one block at 24%. The merge costs a few LUTs of address arithmetic and saves four blocks, and in a design that is BRAM-bound — which an Ethernet design with deep FIFOs usually is — that is the difference between fitting and not.
8. The Frequency Is an Output, and What That Costs
Every rate in Modules 19 to 23 is width × frequency. This section is what happens when the second factor is reported rather than chosen.
At 100 Gb/s on a 512-bit datapath the requirement is 195.3 MHz, and each megahertz is 0.512 Gb/s.
| Achieved frequency | Rate delivered | Against 100 Gb/s |
|---|---|---|
| 180.0 MHz | 92.16 Gb/s | short by 7.8% |
| 185.0 MHz | 94.72 Gb/s | short by 5.3% |
| 190.0 MHz | 97.28 Gb/s | short by 2.7% |
| 195.3 MHz | 99.99 Gb/s | exactly line rate |
| 200.0 MHz | 102.40 Gb/s | 2.4% of margin |
Row three is the one to sit with. A build that misses its timing constraint by 2.7% does not deliver 97.3% of the performance — it fails to carry line rate, which on an Ethernet port means the receive path backs up, Chapter 19.5's FIFO overflows and the design drops frames.
A 3% timing shortfall in an ASIC is a schedule problem. In an FPGA Ethernet MAC it is a functional failure, because the rate the design must sustain is fixed by the wire and the rate it can sustain is fixed by a number the tools report.
And the number varies between builds of identical source.
| Source of variation | Typical spread |
|---|---|
| the placement seed | ±3 to ±8% |
| the device speed grade | ±15% |
| the tool version | a few per cent, either way |
| temperature and voltage derating | already in the tool's report |
Row one is the one that has no engineering answer. The same RTL, the same constraints, the same device, a different random seed — and the achieved frequency moves by several per cent. So a design that closed at 195.3 MHz has not established that it will close at 195.3 MHz, only that one placement did.
Which forces a margin, and the margin is the design's real target.
| Seed spread assumed | Target to guarantee 195.3 MHz |
|---|---|
| ±3% | 201.4 MHz |
| ±5% | 205.6 MHz |
| ±8% | 212.3 MHz |
Designing for 212 MHz to guarantee 195 is an 8.7% tax on every path in the design, and it is paid in pipeline stages, in wider logic and in the aligner Section 4 priced.
Or the width is increased instead, which is the FPGA answer.
| Width | Frequency needed | At ±8%, the target | Aligner |
|---|---|---|---|
| 512 | 195.3 MHz | 212.3 MHz | ≈ 10 752 LUT |
| 1 024 | 97.7 MHz | 106.1 MHz | ≈ 41 984 LUT |
Thirty-one thousand extra LUTs buys a hundred megahertz of relief, and on a device with 500 000 LUTs that is 6.2% of the logic for a constraint that is otherwise a coin toss. That is the trade an FPGA Ethernet design actually makes, and it is not a trade an ASIC designer ever faces.
9. RTL 4 — The fMAX Budget
// ---------------------------------------------------------------------
// fmax_budget -- the design's rate as a function of a number the
// implementation flow reports.
//
// This block's inputs are unusual: achieved_mhz, seed and speed_grade
// are all properties of a BUILD rather than of a design. Every other
// block in Modules 19 to 23 was a function of its inputs alone.
// ---------------------------------------------------------------------
module fmax_budget
import fpga_pkg::*;
(
input logic clk,
input logic rst_n,
input logic [15:0] line_rate_gbps,
input logic [15:0] width_bits,
input logic [15:0] achieved_mhz,
input logic [15:0] seed_spread_pct,
input logic [7:0] speed_grade,
output logic [15:0] required_mhz,
output logic [15:0] target_mhz,
output logic [15:0] delivered_gbps,
output logic [15:0] shortfall_pct,
output logic meets_line_rate,
output logic would_drop_frames,
output logic [15:0] gbps_per_mhz_x1000,
output logic rate_depends_on_a_build
);
always_comb begin
required_mhz = 16'(mhz_for(int'(line_rate_gbps), int'(width_bits)));
// The number a design must actually aim at, so that every seed
// closes rather than one of them.
target_mhz = (seed_spread_pct >= 16'd100) ? 16'hFFFF
: 16'((32'(required_mhz) * 100)
/ (100 - 32'(seed_spread_pct)));
delivered_gbps = 16'(gbps_at(int'(achieved_mhz), int'(width_bits)));
meets_line_rate = (delivered_gbps >= line_rate_gbps);
shortfall_pct = meets_line_rate ? 16'd0
: (((line_rate_gbps - delivered_gbps) * 16'd100)
/ line_rate_gbps);
// Section 8: a shortfall is not a slowdown. The wire keeps
// delivering, the FIFO fills and Chapter 19.5's overflow follows.
would_drop_frames = !meets_line_rate;
gbps_per_mhz_x1000 = width_bits; // width/1000 Gb/s per MHz, scaled
// Always true, and it is the chapter's structural claim.
rate_depends_on_a_build = 1'b1;
end
endmoduleClassification: a rate calculator whose inputs include a placement seed.
What it teaches: that would_drop_frames is the honest name for a timing shortfall on an Ethernet datapath. In most digital design a design that misses timing runs slower; here the wire does not slow down with it. Chapter 19.5 §4 established that a receive FIFO absorbs a burst and empties because its drain exceeds the line rate — and a datapath 2.7% short of line rate has a drain below it, so the FIFO is Chapter 21.9 §9's countdown rather than a cushion, and the design drops at a steady 2.7%.
And it teaches that target_mhz is the number a constraint file should carry. Constraining to 195.3 MHz means one seed in several will fail to close; constraining to 212.3 assumes an ±8% spread and makes closure reproducible. The difference is an 8.7% tax on every path, and it is the cost of turning a probabilistic result into a deterministic one.
Deliberately simplified: seed_spread_pct is a single number where the real distribution is asymmetric and has a tail. speed_grade is accepted and never used — a faster grade shifts the whole distribution and is the one lever that costs money rather than logic. There is no notion of a partial shortfall being acceptable, which for some workloads it is: a design at 97.3 Gb/s carries 97.3 Gb/s of offered load without dropping anything. And gbps_per_mhz_x1000 returns the width, which is correct and obscure.
Production implication: the unused speed_grade is the lever a programme reaches for last and should consider first when the shortfall is small. A one-grade-faster device typically buys 10 to 15% of frequency — enough to turn a 2.7% shortfall into 8% of margin — and it costs money per unit rather than engineering time. Against the alternatives in Section 8 — an 8.7% pipelining tax on every path, or 31 232 extra LUTs to double the width — a faster speed grade is often the cheapest fix in total cost, and it is the one that does not require the design to change at all. The decision belongs to a programme rather than to an engineer, which is why it tends not to be made.
10. What Varies Between Builds of Identical Source
Section 8 named the sources of variation. This section is what they do to a design's claims, and one of them has no engineering answer at all.
Four sources, and their character.
| Source | Spread | Deterministic? | Under the designer's control? |
|---|---|---|---|
| the placement seed | ±3 to ±8% | no | no |
| the device speed grade | ±15% | yes | yes, for money |
| the tool version | a few per cent | yes, once fixed | yes, by pinning |
| temperature and voltage | in the tool's derating | yes | partly |
Row one is the only one with no answer. The seed is a random number that determines an initial placement; the tools are a heuristic search and a different starting point finds a different local optimum. Rows two to four can be pinned, bought or budgeted; row one can only be sampled.
Which means the honest characterisation of an FPGA design's frequency is a distribution.
| What a single build tells you | What twenty builds tell you | |
|---|---|---|
| the achieved frequency | one sample | a mean and a spread |
| whether the design closes | that this seed did | what fraction of seeds do |
| what to constrain to | nothing | the target that makes closure reliable |
| what to claim | nothing defensible | a rate with a confidence |
Row four is the section's recommendation and it is not how most FPGA designs are signed off. A design is usually declared finished when one build closes; the defensible statement requires a seed sweep, and a sweep of twenty builds is a few hours of compute and is the only way to know whether the margin is 8% or 0.5%.
And the consequence for the design's rate is direct.
| Seeds closing at or above 195.3 MHz | What the design is |
|---|---|
| 20 of 20 | a 100 Gb/s design |
| 17 of 20 | a design that fails to build 15% of the time |
| 10 of 20 | not a design; a coin toss with a compile step |
Row two is the common and dangerous case, because it looks like a working design: the build that shipped closed, the design is in the field, and the next time somebody changes a comment and rebuilds it does not. The failure arrives attached to an unrelated change, which is the worst possible attribution.
An FPGA design's frequency is a random variable and a single build is a single sample. Signing off on one sample is signing off on one draw.
One thing that reduces the variance rather than tolerating it. The spread comes from the tools searching a large space; constraining the search reduces it. Floorplanning the datapath into a region, pinning the wide structures near their memories and registering every module boundary all narrow the distribution — at the cost of engineering effort, and they narrow it rather than removing it. The seed remains a seed.
11. RTL 5 — The Timing Closure Monitor
// ---------------------------------------------------------------------
// timing_closure_monitor -- treat the achieved frequency as the random
// variable it is, and report what a set of builds says.
//
// This block does not run in the FPGA. It is a model of a BUILD FLOW,
// included because the flow is part of this design in a way it is not
// part of any ASIC design in Modules 19 to 23.
// ---------------------------------------------------------------------
module timing_closure_monitor
import fpga_pkg::*;
#(
parameter int SEEDS = 20
)(
input logic clk,
input logic rst_n,
input logic build_done,
input logic [15:0] build_mhz,
input logic [15:0] required_mhz,
output logic [15:0] builds_seen,
output logic [15:0] builds_closing,
output logic [15:0] closure_rate_pct,
output logic [15:0] min_mhz,
output logic [15:0] max_mhz,
output logic [31:0] sum_mhz,
output logic [15:0] mean_mhz,
output logic [15:0] spread_pct,
output logic one_sample_only,
output logic signable
);
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
builds_seen <= '0; builds_closing <= '0;
min_mhz <= 16'hFFFF; max_mhz <= '0; sum_mhz <= '0;
end else if (build_done) begin
builds_seen <= builds_seen + 16'd1;
if (build_mhz >= required_mhz)
builds_closing <= builds_closing + 16'd1;
if (build_mhz < min_mhz) min_mhz <= build_mhz;
if (build_mhz > max_mhz) max_mhz <= build_mhz;
sum_mhz <= sum_mhz + 32'(build_mhz);
end
end
always_comb begin
mean_mhz = (builds_seen == 0) ? 16'd0
: 16'(sum_mhz / 32'(builds_seen));
closure_rate_pct = (builds_seen == 0) ? 16'd0
: (builds_closing * 16'd100) / builds_seen;
spread_pct = (mean_mhz == 0) ? 16'd0
: (((max_mhz - min_mhz) * 16'd100) / mean_mhz);
// Section 10: one build is one sample and says nothing about the
// distribution.
one_sample_only = (builds_seen <= 16'd1);
// The verdict a sign-off should require and usually does not.
signable = (builds_seen >= 16'(SEEDS)) &&
(closure_rate_pct == 16'd100);
end
endmoduleClassification: a model of a build flow rather than of hardware, and the only such block in the track.
What it teaches: that signable requires two things and most sign-offs check neither. It requires enough samples — twenty seeds — and a 100% closure rate across them. A design with 17 of 20 closing has a closure_rate_pct of 85 and is not signable, even though it has built successfully seventeen times and is in the field.
And it teaches that one_sample_only is the default state of most FPGA projects. A design is built, it closes, work moves on. The distribution was never sampled, so nobody knows whether the margin is comfortable or whether the build that closed was lucky — and the two are indistinguishable from inside a single build's report.
Deliberately simplified: the block is written as synthesisable RTL and describes a process that runs on a workstation, which is a modelling convenience rather than a design. spread_pct is a range rather than a standard deviation, so one outlier dominates it. SEEDS = 20 is a round number and the right sample size depends on the spread and the confidence wanted. And nothing here captures why a seed failed, which is where the engineering effort actually goes.
Production implication: running the sweep is cheap and acting on it is where the value is, and the action is usually not more pipelining. Twenty builds of a large FPGA design is a few hours of compute on a machine that is otherwise idle overnight, and the result is a histogram rather than a pass. If the spread is 3% the design needs 3% of margin; if it is 8% it needs 8% — and the design is currently carrying whatever margin the one build happened to have. The most common outcome of a first sweep is the discovery that a design believed to have 5% of margin has 0.4%, which changes the plan immediately and would not otherwise have been found until a rebuild failed.
12. Where an FPGA Ethernet Design Actually Fails
Sections 2 to 11 priced the resources. This section says which of them a real design runs out of, and the answer is neither of the two an ASIC intuition predicts.
A complete 100 Gb/s MAC at a 1 024-bit datapath, mapped.
| Structure | LUT6 | Flip-flops | BRAM36 |
|---|---|---|---|
| byte aligner | 41 984 | 2 048 | 0 |
| CRC-32 engine, 1 024-bit | 3 379 | 1 056 | 0 |
| frame parser and classifier | 4 500 | 3 000 | 0 |
| transmit assembler | 3 800 | 2 500 | 0 |
| receive and transmit FIFOs, 1 024 × 512 | 200 | 200 | 30 |
| statistics counters | 900 | 1 728 | 1 |
| control, registers and telemetry | 2 200 | 3 600 | 1 |
| total | 56 963 | 14 132 | 32 |
Against a mid-size device.
| Resource | Used | Available | Utilisation |
|---|---|---|---|
| LUT6 | 56 963 | 500 000 | 11.4% |
| flip-flops | 14 132 | 1 000 000 | 1.4% |
| BRAM36 | 32 | 1 000 | 3.2% |
Every number is small, and the design still fails — which is the section's point. The binding constraints are not in the table.
| What binds | Why it is not in the table |
|---|---|
| timing closure | it is a property of a placement, not of a resource count |
| routing congestion | a wide datapath's interconnect demand is not a LUT count |
| the transceiver count | a hard resource, and a 100 Gb/s port needs four lanes |
| the external memory interface | if the design needs more buffer than BRAM provides |
Row one is Sections 8 to 11's whole subject. Row two is the one that surprises: a 1 024-bit datapath moves 1 024 wires between every pair of stages, and an FPGA's interconnect is a fixed fabric. A design at 11% LUT utilisation can fail to route because those 1 024-bit buses must all cross the same region — and the remedy is floorplanning rather than logic reduction.
An FPGA Ethernet design is rarely short of logic. It is short of timing and of interconnect, and neither appears in a utilisation report.
Row four is the one that ends Chapter 23.3's ambitions. That chapter's switch needs a 64 MB packet buffer — 13 889 BRAM36 against a device's thousand — so an FPGA switch must use external DRAM, which reintroduces Chapter 19.6 §4's latency-hiding problem inside the switch and adds an interface that is itself a large design.
| In BRAM | In external DRAM | |
|---|---|---|
| capacity | ≈ 4.6 MB on a large device | gigabytes |
| latency | 1 to 2 cycles | hundreds of nanoseconds |
| bandwidth | effectively unlimited | a few hundred Gb/s, shared |
| what it needs | nothing | Chapter 19.6's whole reorder and outstanding machinery |
So an FPGA can build a MAC comfortably, a small switch with difficulty, and Chapter 23.3's 6.4 Tb/s switch not at all — and the reason is a resource inventory rather than an area.
13. RTL 6 — The Rate Derating Model
// ---------------------------------------------------------------------
// rate_derating -- what a design delivers when the flow's number comes
// in below the target, and what the design should do about it.
//
// The important output is not the derated rate; it is that an Ethernet
// datapath cannot BE derated. The wire runs at its rate regardless.
// ---------------------------------------------------------------------
module rate_derating
import fpga_pkg::*;
(
input logic clk,
input logic rst_n,
input logic [15:0] line_rate_gbps,
input logic [15:0] width_bits,
input logic [15:0] achieved_mhz,
input logic [15:0] fifo_depth_words,
output logic [15:0] delivered_gbps,
output logic [15:0] deficit_gbps,
output logic sustained_deficit,
output logic [31:0] fifo_fill_ns,
output logic [31:0] drop_rate_ppm,
output logic can_be_derated,
output logic [15:0] usable_line_rate_gbps
);
always_comb begin
delivered_gbps = 16'(gbps_at(int'(achieved_mhz), int'(width_bits)));
deficit_gbps = (delivered_gbps < line_rate_gbps)
? (line_rate_gbps - delivered_gbps) : 16'd0;
sustained_deficit = (deficit_gbps != 16'd0);
// Chapter 19.5's FIFO with the inequality reversed, exactly as
// Chapter 21.9 Section 9's capture sink: arrival exceeds drain, so
// the depth is a countdown.
fifo_fill_ns = (deficit_gbps == 0) ? 32'hFFFF_FFFF
: ((32'(fifo_depth_words) * 32'(width_bits))
/ 32'(deficit_gbps));
// At a sustained deficit the drop rate is the deficit's share.
drop_rate_ppm = (line_rate_gbps == 0) ? 32'd0
: ((32'(deficit_gbps) * 1_000_000)
/ 32'(line_rate_gbps));
// THE point: a processor can run slower. An Ethernet port cannot.
can_be_derated = 1'b0;
// What the design can honestly carry, which is a different port
// speed rather than a fraction of this one.
usable_line_rate_gbps = delivered_gbps;
end
endmoduleClassification: a deficit model whose central output is a hard-wired 1'b0.
What it teaches: that can_be_derated is false and that is the whole difference from every other digital design. A processor that misses timing runs at a lower clock and does less work per second, which is a performance result. An Ethernet MAC that misses timing still faces a wire running at 100 Gb/s — the arrival rate does not derate with the design — so the deficit is not slower processing, it is loss.
And it teaches that fifo_fill_ns is Chapter 19.5's arithmetic with the inequality reversed, for the third time in the track. Chapter 21.9 §9 found it in a capture apparatus and Chapter 23.3 §9 in a shared buffer; here it is a receive FIFO whose drain is below the line rate because the datapath is 2.7% short. At a 512-word FIFO and a 2.72 Gb/s deficit the fill time is 193 microseconds, after which the design drops at 27 200 parts per million, steadily, forever.
Deliberately simplified: the model assumes the deficit is constant, where a real datapath's throughput varies with the frame-size mix — a design short on the minimum-frame path may be comfortable on large frames. drop_rate_ppm assumes the deficit maps one-for-one onto drops, ignoring flow control. And usable_line_rate_gbps suggests running the port at a lower speed, which is sometimes possible — a 100 Gb/s port configured as 4 × 25 with one lane unused is a real, ugly, occasionally-used answer.
Production implication: flow control is the mitigation the model omits and it converts a drop into a different problem rather than solving it. A design 2.7% short can assert Chapter 14.2's PAUSE — or a priority flow-control frame — and push the deficit back onto the link partner, which stops the drops and moves the congestion to the switch upstream. That is legitimate, it is what the mechanism is for, and it has two consequences worth stating: the port no longer carries line rate and now says so, which is much better than silent loss; and the upstream switch's buffer absorbs the difference, which is Chapter 23.3 §9's shared pool paying for an FPGA's timing closure. The honest position is to declare the port's real rate rather than to claim 100 Gb/s and PAUSE 2.7% of the time.
14. What an FPGA Ethernet Design Must Never Do
Five prohibitions, each derived earlier.
One — never sign off on one build.
| Seeds closing | What the design is |
|---|---|
| 20 of 20 | a design |
| 17 of 20 | a design that fails to build 15% of the time |
| 1 of 1 | one sample, and nothing more |
Section 11: the achieved frequency is a random variable. A sweep of twenty seeds is a few hours of overnight compute and is the only thing that distinguishes 8% of margin from 0.4%.
Two — never constrain to the required frequency.
| Seed spread | Required | Constrain to |
|---|---|---|
| ±3% | 195.3 MHz | 201.4 |
| ±5% | 195.3 | 205.6 |
| ±8% | 195.3 | 212.3 |
Constraining to the requirement guarantees that some seeds fail, which is a build-time lottery attached to whatever change happened to trigger the rebuild.
Three — never treat a timing shortfall as a performance result.
Section 13: an Ethernet port cannot be derated. A datapath 2.7% short faces a wire that is still delivering 100 Gb/s, so Chapter 19.5's FIFO becomes a countdown — 193 microseconds at a 512-word depth — and the design then drops at 27 200 parts per million, steadily. The honest responses are flow control, which declares the deficit, or a lower declared port speed. Silence is not one of them.
Four — never price an FPGA memory by its bits.
| Structure | Bits | Blocks | Efficiency |
|---|---|---|---|
| Chapter 22.1 §13's delay line | 1 024 | 1 | 2.8% |
| a 1 024-bit × 64 FIFO | 65 536 | 15, by width | 11.9% |
| Chapter 23.3's 16k MAC table | 1 572 864 | 43 | 99.2% |
Small costs a whole block; wide costs by width whatever the depth. Merge the small structures behind a shared decoder and build the shallow wide ones from flip-flops, and the savings are factors rather than percentages.
Five — never plan a design whose buffer exceeds the device's blocks.
Section 12: Chapter 23.3's 64 MB packet buffer needs 13 889 BRAM36 and a large device has about a thousand — 4.6 MB. The gap is not closable with a bigger FPGA; it is closable only with external DRAM, which brings Chapter 19.6's entire reorder-and-outstanding machinery inside the design.
15. RTL 7 — FPGA Telemetry
// ---------------------------------------------------------------------
// fpga_telemetry -- what the design knows about itself, plus the three
// build-time facts that no other chapter's telemetry has needed.
//
// Those three are the achieved frequency, the placement seed and the
// device speed grade, and they belong in a readable register because
// they are what the design's rate actually depends on.
// ---------------------------------------------------------------------
module fpga_telemetry
import fpga_pkg::*;
(
input logic clk,
input logic rst_n,
input logic snapshot,
// build-time constants, written by the synthesis flow
input logic [15:0] i_build_mhz,
input logic [15:0] i_build_seed,
input logic [7:0] i_speed_grade,
input logic [15:0] i_required_mhz,
input logic [15:0] i_width_bits,
input logic [31:0] i_build_id,
// run-time
input logic [31:0] i_frames,
input logic [31:0] i_fifo_overflows,
input logic [31:0] i_pause_frames_sent,
input logic [15:0] i_fifo_peak_words,
input logic [15:0] i_fifo_depth_words,
output logic [15:0] o_build_mhz,
output logic [15:0] o_build_seed,
output logic [7:0] o_speed_grade,
output logic [31:0] o_build_id,
output logic [15:0] o_delivered_gbps,
output logic [15:0] o_margin_pct,
output logic [31:0] o_frames,
output logic [15:0] o_overflow_ppm,
output logic [15:0] o_fifo_peak_pct,
output logic [31:0] o_pause_frames_sent,
output logic [2:0] o_caveats,
output logic o_rate_claim_valid
);
localparam int C_NO_MARGIN = 0;
localparam int C_OVERFLOWED = 1;
localparam int C_PAUSING = 2;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
o_build_mhz <= '0; o_build_seed <= '0; o_speed_grade <= '0;
o_build_id <= '0; o_delivered_gbps <= '0; o_margin_pct <= '0;
o_frames <= '0; o_overflow_ppm <= '0; o_fifo_peak_pct <= '0;
o_pause_frames_sent <= '0; o_caveats <= '0;
end else if (snapshot) begin
// The build's identity, which for an FPGA is part of the
// design's specification rather than metadata.
o_build_mhz <= i_build_mhz;
o_build_seed <= i_build_seed;
o_speed_grade <= i_speed_grade;
o_build_id <= i_build_id;
o_delivered_gbps <= 16'(gbps_at(int'(i_build_mhz),
int'(i_width_bits)));
if (i_required_mhz != 0)
o_margin_pct <= ((i_build_mhz * 16'd100) / i_required_mhz) - 16'd100;
o_frames <= i_frames;
o_pause_frames_sent <= i_pause_frames_sent;
if (i_frames != 0)
o_overflow_ppm <= 16'((i_fifo_overflows * 1_000_000) / i_frames);
if (i_fifo_depth_words != 0)
o_fifo_peak_pct <= (i_fifo_peak_words * 16'd100) / i_fifo_depth_words;
o_caveats[C_NO_MARGIN] <= (i_build_mhz < i_required_mhz);
o_caveats[C_OVERFLOWED] <= (i_fifo_overflows != 32'd0);
o_caveats[C_PAUSING] <= (i_pause_frames_sent != 32'd0);
end
end
assign o_rate_claim_valid = !o_caveats[C_NO_MARGIN] &&
!o_caveats[C_OVERFLOWED];
endmoduleClassification: a snapshot bank whose first four fields are properties of a build rather than of a run.
What it teaches: that o_build_seed belongs in a readable register, which sounds absurd until Section 10's argument is accepted. The design's rate is a function of the achieved frequency, the achieved frequency is a function of the seed, and a part in the field that is dropping frames needs its seed recorded so that the build can be reproduced. No ASIC chapter in this track has needed a field like it, and no FPGA design should ship without one.
And it teaches that o_margin_pct is computed from build-time constants and is therefore constant. It is not a measurement — it is the design telling you how much slack the build that made it had. A part reporting 0% is one whose next rebuild is a coin toss, and that is a fact worth knowing from the field rather than from a build log nobody kept.
Deliberately simplified: o_margin_pct underflows if the build is short of the requirement — the subtraction should be signed. The build-time fields must be driven by synthesis-time attributes or a generated package, which the model assumes without showing. o_overflow_ppm normalises by frames where the useful denominator is time, since a sustained deficit drops at a rate rather than a fraction. And there is no record of the tool version, which Section 10 listed as a source of variation.
Production implication: the build-identity fields are the ones to insist on and they cost almost nothing. A 32-bit build identifier, a 16-bit seed, a 16-bit achieved frequency and an 8-bit speed grade are 72 flip-flops — 0.007% of a mid device's supply — and they turn a field failure into a reproducible one. Without them, a part that drops frames in the field is debugged against whatever build the engineer can rebuild today, which by Section 10 is a different placement with a different frequency, and the failure may not reproduce at all. Write the build's identity into the bitstream and read it back from a register.
16. RTL 8 — The FPGA Conformance Monitor
// ---------------------------------------------------------------------
// fpga_conformance_monitor -- properties for an FPGA Ethernet design.
//
// Properties 1 to 9 are about the RTL and hold in simulation at any
// frequency. Properties 10 to 12 are the ones that connect the RTL to
// the build, and they are the only ones that can fail because of a
// placement.
// ---------------------------------------------------------------------
module fpga_conformance_monitor
import fpga_pkg::*;
(
input logic clk,
input logic rst_n,
input logic [15:0] width_bits,
input logic [15:0] required_mhz,
input logic [15:0] achieved_mhz,
input logic [15:0] line_rate_gbps,
input logic [15:0] delivered_gbps,
input logic meets_line_rate,
input logic would_drop_frames,
input logic can_be_derated,
input logic [15:0] blocks_by_capacity,
input logic [15:0] blocks_by_width,
input logic [15:0] blocks_used,
input logic [31:0] aligner_luts_in,
input logic pad_to_word,
input logic [15:0] builds_seen,
input logic [15:0] closure_rate_pct,
input logic signable,
input logic rate_claim_valid,
input logic claim_line_rate
);
// 1. Blocks used is the larger of the two constraints.
p_fp_blocks: assert property (@(posedge clk) disable iff (!rst_n)
blocks_used == ((blocks_by_capacity > blocks_by_width)
? blocks_by_capacity : blocks_by_width));
// 2. A padded datapath needs no aligner.
p_fp_pad: assert property (@(posedge clk) disable iff (!rst_n)
pad_to_word |-> (aligner_luts_in == 32'd0));
// 3. An unpadded one does.
p_fp_aligner: assert property (@(posedge clk) disable iff (!rst_n)
!pad_to_word |-> (aligner_luts_in ==
32'(aligner_luts(int'(width_bits)))));
// 4. The required frequency is the rate over the width.
p_fp_required: assert property (@(posedge clk) disable iff (!rst_n)
required_mhz == 16'(mhz_for(int'(line_rate_gbps), int'(width_bits))));
// 5. Delivered is the achieved frequency times the width.
p_fp_delivered: assert property (@(posedge clk) disable iff (!rst_n)
delivered_gbps == 16'(gbps_at(int'(achieved_mhz), int'(width_bits))));
// 6. Meeting the rate is delivering at least it.
p_fp_meets: assert property (@(posedge clk) disable iff (!rst_n)
meets_line_rate == (delivered_gbps >= line_rate_gbps));
// 7. An Ethernet datapath cannot be derated.
p_fp_no_derate: assert property (@(posedge clk) disable iff (!rst_n)
!can_be_derated);
// 8. Not meeting the rate means dropping.
p_fp_drops: assert property (@(posedge clk) disable iff (!rst_n)
would_drop_frames == !meets_line_rate);
// 9. Efficiency never exceeds 100%.
p_fp_efficiency: assert property (@(posedge clk) disable iff (!rst_n)
bram_efficiency_pct(int'(blocks_used) * BRAM_BITS) <= 100);
// 10. A LINE-RATE CLAIM requires the build to have met the frequency.
p_fp_claim_freq: assert property (@(posedge clk) disable iff (!rst_n)
claim_line_rate |-> (achieved_mhz >= required_mhz));
// 11. And requires the closure to be reproducible, not lucky.
p_fp_claim_signable: assert property (@(posedge clk) disable iff (!rst_n)
claim_line_rate |-> signable);
// 12. And the telemetry's verdict agrees.
p_fp_claim_telemetry: assert property (@(posedge clk) disable iff (!rst_n)
claim_line_rate |-> rate_claim_valid);
endmoduleClassification: nine RTL properties and three that reach into the build flow.
What it teaches: that properties 1 to 9 all hold in simulation at any frequency whatever, because simulation supplies the clock. A testbench running at 1 GHz proves them; a build closing at 180 MHz proves them too. They are statements about the RTL's logic and none of them is a statement about the design's rate.
And it teaches that property 11 is the one that is never written. claim_line_rate |-> signable requires twenty seeds and a 100% closure rate — Section 11's verdict — so a design that closed once may not claim line rate. That is a strong requirement, it is the correct one, and it turns a build result into a statistical statement rather than an event.
Deliberately simplified: property 9 is a tautology as written, since the efficiency of a block-aligned size is 100% by construction. Properties 10 to 12 reference build-time constants that must be driven into the design by the flow, which is a real mechanism and is not shown. Property 3 recomputes the aligner estimate rather than checking the synthesised result, which only a report can supply. And nothing here checks routing, which Section 12 named as a binding constraint that no property can express.
Production implication: the build-time constants in properties 10 to 12 need a mechanism, and the cheap one is a generated package. The synthesis flow writes a small SystemVerilog package containing the achieved frequency, the seed, the speed grade and a build identifier, the design imports it, and the properties above become checkable in a post-synthesis simulation. It costs a script and it closes the loop between what was proved and what was built — which for an ASIC is automatic, because the frequency was an input, and for an FPGA has to be constructed deliberately.
17. An FPGA Against an ASIC, in Both Units
Chapter 23.3 §2 defined a unit that compares a flip-flop against an SRAM bit. This section asks what it says about a design that is neither.
The MAC, in both.
| ASIC (BCE) | FPGA (inventory) | |
|---|---|---|
| Chapter 19.7 §19's receive datapath | 283 320 | 14 166 FF + ≈7 900 LUT6 |
| its memory, 55 KiB | 450 560 | 13 BRAM36 at 94.0% |
| the byte aligner at 1 024 bits | not present — an ASIC is 64 bits at 1.5 GHz | ≈41 984 LUT6 |
| total | 733 880 BCE | 56 963 LUT, 14 132 FF, 32 BRAM |
Row three is the honest observation and it is why the two columns cannot be reconciled. An ASIC MAC at 100 Gb/s runs a 64-bit datapath at 1.5 GHz; an FPGA MAC runs 1 024 bits at 97.7 MHz. They are not the same design expressed in two units — they are different designs, and the FPGA's largest structure does not exist in the ASIC at all.
Which means BCE does not translate, and it is worth saying exactly why.
| BCE assumes | An FPGA provides |
|---|---|
| memory of any size | 36 864-bit blocks |
| memory of any width | a maximum port width per block |
| logic priced by gate equivalents | LUT6s, which are neither gates nor flops |
| a chosen clock frequency | a reported one |
| area as the scarce resource | timing and routing as the scarce resources |
Row five is the deepest difference. Chapter 23.3 §2's unit exists to answer how much area; an FPGA design that fits comfortably in area can still fail, and Section 12's design used 11.4% of the LUTs and failed on timing and routing. The unit answers a question that is not the binding one.
So the FPGA's own unit is the inventory, and the comparison that does work is a ratio of utilisations.
| Design | LUT | FF | BRAM | Binding |
|---|---|---|---|---|
| a 100 Gb/s MAC, 1 024-bit | 11.4% | 1.4% | 3.2% | timing |
| four such MACs | 45.6% | 5.7% | 12.8% | routing |
| Chapter 23.3's switch, buffer excluded | ≈ 25% | ≈ 5% | ≈ 6% | timing |
| Chapter 23.3's switch, buffer included | — | — | 1 389% | impossible |
Row four is the chapter's limit statement. A 6.4 Tb/s switch's 64 MB buffer is 13 889 BRAM36 and a large device has about a thousand — 4.6 MB. The switch is not a large FPGA design; it is not an FPGA design.
An FPGA builds a MAC comfortably, several MACs with care, a small switch with external memory, and a merchant switch not at all — and the boundary is a block count rather than an area.
18. What the FPGA Design Assumes
Seven premises, and four of them are about a build rather than a design.
| Assumption | If it is false |
|---|---|
| the achieved frequency meets the requirement | Section 8 — 2.7% short is a 27 200 ppm drop rate |
| it will meet it on the next build too | Section 10 — the seed is a random variable |
| the tool version is pinned | a few per cent of frequency, either way |
| the speed grade is the one that was budgeted | ±15% |
| memory structures are large | Section 6 — a 1 024-bit structure costs a block each |
| the datapath's buses can be routed | Section 12 — a design at 11% LUT can fail to route |
| BRAM is enough for the buffers | Section 17 — Chapter 23.3's switch needs 14× a device |
Row two is the assumption that is almost never tested, and testing it is Section 11's seed sweep: a few hours of overnight compute against a design that is otherwise signed off.
Row six has no model and no property. Routing congestion is a consequence of where the tools put things, which is downstream of everything a designer writes; the only levers are floorplanning, module boundaries and registering every interface, and none of them is expressible in RTL as a constraint that can be checked.
And one premise that every earlier chapter in the track could take for granted.
That the design's rate is a property of the design. In Modules 19 to 23 a MAC at 100 Gb/s was a 100 Gb/s MAC. Here the rate is
width × achieved frequency, the width is a design choice and the frequency is a build result — so the same source, built twice, is two designs with two rates, and only one of them may meet the requirement.
19. The Cost, Accounted — in the Inventory
This chapter's blocks and the design they describe, in the unit an FPGA actually rations.
| Block | LUT6 | Flip-flops | BRAM36 |
|---|---|---|---|
resource_model | 0 — combinational | 0 | 0 |
datapath_width_model | 0 — combinational | 0 | 0 |
bram_quantiser | 0 — combinational | 0 | 0 |
fmax_budget | 0 — combinational | 0 | 0 |
timing_closure_monitor | 180 | 120 | 0 |
rate_derating | 0 — combinational | 0 | 0 |
fpga_telemetry | 260 | 232 | 0 |
fpga_conformance_monitor | 0 — assertions | 0 | 0 |
| this chapter's additions | 440 | 352 | 0 |
| the MAC they instrument — Section 12 | 56 963 | 14 132 | 32 |
Six of the eight blocks are purely combinational, which is characteristic: most of this chapter is arithmetic about a design rather than logic inside one. The two that carry state are a build-flow model and a telemetry bank, and together they are 440 LUT6 — 0.09% of a mid device and 0.8% of the MAC they instrument.
And the build-identity fields Section 15 argued for.
| Field | Bits | Flip-flops |
|---|---|---|
| build identifier | 32 | 32 |
| placement seed | 16 | 16 |
| achieved frequency | 16 | 16 |
| speed grade | 8 | 8 |
| total | 72 | 72 |
Seventy-two flip-flops — 0.007% of a mid device's supply — and they turn a field failure into a reproducible one, which without them it is not, because the rebuild is a different placement.
The whole design, set against the rest of Module 23.
| Unit | Value | |
|---|---|---|
| Chapter 19.7 §19's ASIC MAC datapath | BCE | 283 320 |
| Chapter 23.3's switch pipeline logic | BCE | 237 200 |
| Chapter 23.4's NIC | BCE | 5.07 × 10⁷ |
| Chapter 23.5's programmable array | BCE | 2.23 × 10⁷ |
| this chapter's FPGA MAC | inventory | 56 963 LUT, 14 132 FF, 32 BRAM |
Row five is in a different unit and that is the chapter's conclusion rather than an inconvenience. Chapter 23.3 §2 defined BCE because the flop had failed five times; BCE fails here for a sixth reason — not because a structure's size is set by data, but because the resources are a fixed inventory and the binding constraint is not a resource at all.
Six chapters have now needed three units: flip-flops for a MAC, bitcell equivalents for a memory-dominated part, and a resource inventory plus a timing distribution for an FPGA. The lesson is not that the earlier units were wrong — it is that a unit is a model of what is scarce, and what is scarce changes.
20. Properties Worth Asserting, and One Worth Refusing
Six groups. Thirty-three of the thirty-four hold in simulation at any clock frequency, which is exactly the problem.
Group A — the resource inventory.
// A1. Blocks used is the larger of capacity and width.
p_rs_blocks: assert property (@(posedge clk) disable iff (!rst_n)
blocks_used == ((blocks_by_capacity > blocks_by_width)
? blocks_by_capacity : blocks_by_width));
// A2. Capacity blocks are the ceiling of bits over the block size.
p_rs_ceiling: assert property (@(posedge clk) disable iff (!rst_n)
blocks_by_capacity == 16'(brams_of_bits(int'(total_bits))));
// A3. Width blocks follow the port width.
p_rs_width: assert property (@(posedge clk) disable iff (!rst_n)
blocks_by_width == 16'((entry_bits + MAX_PORT_BITS - 1) / MAX_PORT_BITS));
// A4. Efficiency never exceeds 100%.
p_rs_eff: assert property (@(posedge clk) disable iff (!rst_n)
efficiency_pct <= 16'd100);
// A5. A structure below 25% efficiency prefers registers.
p_rs_registers: assert property (@(posedge clk) disable iff (!rst_n)
(efficiency_pct < 16'd25) |-> registers_are_cheaper);
// A6. Exceeding the device is reported, not silently clamped.
p_rs_exceeds: assert property (@(posedge clk) disable iff (!rst_n)
(blocks_used > 16'(DEVICE_BRAMS)) |-> exceeds_device);Group B — the datapath width.
// B1. The required frequency is the rate over the width.
p_dw_required: assert property (@(posedge clk) disable iff (!rst_n)
required_mhz == 16'(mhz_for(int'(line_rate_gbps), int'(width_bits))));
// B2. A padded datapath needs no aligner.
p_dw_pad: assert property (@(posedge clk) disable iff (!rst_n)
pad_to_word |-> (aligner_luts_o == 32'd0));
// B3. An unpadded one costs the barrel shifter.
p_dw_aligner: assert property (@(posedge clk) disable iff (!rst_n)
!pad_to_word |-> (aligner_luts_o ==
32'(aligner_luts(int'(width_bits)))));
// B4. The aligner grows quadratically with the width.
p_dw_quadratic: assert property (@(posedge clk) disable iff (!rst_n)
(width_bits == 16'd1024) |-> (aligner_luts_o > 32'd40000));
// B5. Width is a timing decision and the block says so.
p_dw_declares: assert property (@(posedge clk) disable iff (!rst_n)
width_is_a_timing_decision);
// B6. Padding wastes bandwidth at small frame sizes.
p_dw_padding: assert property (@(posedge clk) disable iff (!rst_n)
pad_to_word |-> (padding_waste_pct > 16'd0));Group C — the frequency budget.
// C1. Delivered rate is the achieved frequency times the width.
p_fb_delivered: assert property (@(posedge clk) disable iff (!rst_n)
delivered_gbps == 16'(gbps_at(int'(achieved_mhz), int'(width_bits))));
// C2. Meeting the rate is delivering at least it.
p_fb_meets: assert property (@(posedge clk) disable iff (!rst_n)
meets_line_rate == (delivered_gbps >= line_rate_gbps));
// C3. The target includes the seed margin.
p_fb_target: assert property (@(posedge clk) disable iff (!rst_n)
(seed_spread_pct < 16'd100) |->
(target_mhz == ((required_mhz * 16'd100) / (16'd100 - seed_spread_pct))));
// C4. A shortfall is reported as a drop risk, not as slowness.
p_fb_drops: assert property (@(posedge clk) disable iff (!rst_n)
would_drop_frames == !meets_line_rate);
// C5. The rate depends on a build, and the block says so.
p_fb_declares: assert property (@(posedge clk) disable iff (!rst_n)
rate_depends_on_a_build);
// C6. Shortfall is zero when the rate is met.
p_fb_zero: assert property (@(posedge clk) disable iff (!rst_n)
meets_line_rate |-> (shortfall_pct == 16'd0));Group D — derating.
// D1. An Ethernet datapath cannot be derated.
p_dr_no_derate: assert property (@(posedge clk) disable iff (!rst_n)
!can_be_derated);
// D2. A deficit produces a finite FIFO fill time.
p_dr_fill: assert property (@(posedge clk) disable iff (!rst_n)
sustained_deficit |-> (fifo_fill_ns != 32'hFFFF_FFFF));
// D3. No deficit means no fill.
p_dr_nofill: assert property (@(posedge clk) disable iff (!rst_n)
!sustained_deficit |-> (fifo_fill_ns == 32'hFFFF_FFFF));
// D4. The drop rate is the deficit's share of the line rate.
p_dr_rate: assert property (@(posedge clk) disable iff (!rst_n)
(line_rate_gbps != 0) |->
(drop_rate_ppm == ((32'(deficit_gbps) * 1_000_000)
/ 32'(line_rate_gbps))));
// D5. The usable rate is what the build delivers.
p_dr_usable: assert property (@(posedge clk) disable iff (!rst_n)
usable_line_rate_gbps == delivered_gbps);Group E — the build flow.
// E1. Builds closing never exceed builds seen.
p_bf_bounded: assert property (@(posedge clk) disable iff (!rst_n)
builds_closing <= builds_seen);
// E2. The minimum never exceeds the maximum.
p_bf_order: assert property (@(posedge clk) disable iff (!rst_n)
(builds_seen != 0) |-> (min_mhz <= max_mhz));
// E3. One build is one sample.
p_bf_one: assert property (@(posedge clk) disable iff (!rst_n)
(builds_seen <= 16'd1) |-> one_sample_only);
// E4. Signable requires enough seeds and full closure.
p_bf_signable: assert property (@(posedge clk) disable iff (!rst_n)
signable == ((builds_seen >= 16'(SEEDS)) &&
(closure_rate_pct == 16'd100)));
// E5. The closure rate is a percentage.
p_bf_pct: assert property (@(posedge clk) disable iff (!rst_n)
closure_rate_pct <= 16'd100);Group F — the claim.
// F1. A line-rate claim requires the build to have met the frequency.
p_cl_freq: assert property (@(posedge clk) disable iff (!rst_n)
claim_line_rate |-> (achieved_mhz >= required_mhz));
// F2. And requires the closure to be reproducible.
p_cl_signable: assert property (@(posedge clk) disable iff (!rst_n)
claim_line_rate |-> signable);
// F3. And the telemetry's verdict to agree.
p_cl_telemetry: assert property (@(posedge clk) disable iff (!rst_n)
claim_line_rate |-> rate_claim_valid);
// F4. The build's identity is readable.
p_cl_identity: assert property (@(posedge clk) disable iff (!rst_n)
snapshot |=> ((o_build_seed == $past(i_build_seed)) &&
(o_build_mhz == $past(i_build_mhz))));
// F5. Overflows clear the rate claim.
p_cl_overflow: assert property (@(posedge clk) disable iff (!rst_n)
(o_overflow_ppm != 16'd0) |-> !rate_claim_valid);
// F6. A design short of its requirement is caveated, not silent.
p_cl_caveat: assert property (@(posedge clk) disable iff (!rst_n)
(achieved_mhz < required_mhz) |-> caveat_no_margin);Thirty-four properties, and every one of groups A to E holds in a simulation whose clock is whatever the testbench supplied.
21. Verification Scenarios
Fifty-eight scenarios. Group 5's are builds rather than simulations, and they are the only ones that can find the chapter's failure.
Group 1 — the resource inventory (10).
| # | Scenario | Expect |
|---|---|---|
| 1 | Chapter 19.7's 14 166 flops | 14 166 FFs — 1.4% of a mid device |
| 2 | its 39 445 XOR terms | ≈7 900 LUT6 — FPGAs like XOR |
| 3 | its 55 KiB of memory | 13 BRAM36 at 94.0% |
| 4 | Chapter 22.1 §13's 1 024-bit delay line | 1 BRAM36 at 2.8% — 36× its size |
| 5 | Chapter 19.7's 1 728-bit counter bank | 1 BRAM36 at 4.7% |
| 6 | rows 4 and 5 merged behind a decoder | 1 block at 24% — a saving of one |
| 7 | Chapter 23.3's 16k MAC table | 43 BRAM36 at 99.2% |
| 8 | Chapter 23.3's 64 MB buffer | 13 889 blocks against a device's 1 000 |
| 9 | a 1 024 × 64 FIFO | 2 blocks by capacity, 15 by width — 7.5× |
| 10 | the same built from flip-flops | 8 192 FFs, 0 blocks — cheaper |
Group 2 — the datapath width (10).
| # | Scenario | Expect |
|---|---|---|
| 11 | 10 Gb/s, 64-bit | 156.2 MHz — comfortable |
| 12 | 100 Gb/s, 256-bit | 390.6 MHz — not achievable |
| 13 | 100 Gb/s, 512-bit | 195.3 MHz — with care |
| 14 | 100 Gb/s, 1 024-bit | 97.7 MHz — comfortable |
| 15 | 400 Gb/s, 2 048-bit | 195.3 MHz — with care |
| 16 | the aligner at 64 bits | ≈192 LUT6 |
| 17 | at 512 bits | ≈10 752 — 56× |
| 18 | at 1 024 bits | ≈41 984 — 219× |
| 19 | padding every frame to a 128-octet word | aligner is 0; 50% of the wire wasted at 64 octets |
| 20 | the same on 9 000-octet frames | 1.0% wasted — the trade is excellent |
Group 3 — the frequency budget (10).
| # | Scenario | Expect |
|---|---|---|
| 21 | 512-bit datapath closing at 195.3 MHz | 99.99 Gb/s — exactly line rate |
| 22 | closing at 190.0 MHz | 97.28 Gb/s — short by 2.7% |
| 23 | closing at 180.0 MHz | 92.16 Gb/s — short by 7.8% |
| 24 | closing at 200.0 MHz | 102.40 Gb/s — 2.4% of margin |
| 25 | a ±3% seed spread | constrain to 201.4 MHz |
| 26 | ±5% | 205.6 MHz |
| 27 | ±8% | 212.3 MHz — an 8.7% tax on every path |
| 28 | 1 024-bit instead, at ±8% | 106.1 MHz target; +31 232 LUT |
| 29 | a one-grade-faster device | 10–15% of frequency, for money |
| 30 | meets_line_rate with a shortfall | p_fb_meets fires |
Group 4 — derating, which does not exist (8).
| # | Scenario | Expect |
|---|---|---|
| 31 | a 2.7% deficit, 512-word FIFO | fills in 193 µs, then drops |
| 32 | the steady-state drop rate | 27 200 ppm |
| 33 | the same on a processor | it would just run slower — the contrast |
| 34 | flow control asserted instead | no drops; the congestion moves upstream |
| 35 | the port declared at 97.28 Gb/s | honest; no drops; not 100 Gb/s |
| 36 | the port claimed at 100 Gb/s while pausing 2.7% | the dishonest option |
| 37 | can_be_derated forced true | p_dr_no_derate fires |
| 38 | no deficit | fifo_fill_ns saturates — D3 |
Group 5 — builds, which are not simulations (12).
| # | Scenario | Expect |
|---|---|---|
| 39 | one build closing at 196 MHz | one sample; one_sample_only |
| 40 | twenty builds, all closing | signable |
| 41 | twenty builds, seventeen closing | 85% — not signable, and it looks fine |
| 42 | the spread across twenty seeds | the number the constraint should be set from |
| 43 | a design believed to have 5% of margin | the sweep frequently shows 0.4% |
| 44 | a rebuild after a comment change | a different seed, a different frequency |
| 45 | the same source on a faster speed grade | +10 to 15% |
| 46 | the same source on a newer tool version | a few per cent, either direction |
| 47 | every property in Section 20's groups A–E | holds in simulation at any clock |
| 48 | a simulation at 1 GHz | all properties pass; the build closed at 180 |
| 49 | claim_line_rate after one build | p_cl_signable fires |
| 50 | claim_line_rate after twenty | permitted |
Group 6 — telemetry and the field (8).
| # | Scenario | Expect |
|---|---|---|
| 51 | a part dropping frames in the field | read the seed and the achieved frequency |
| 52 | the same part, rebuilt today | a different placement — may not reproduce |
| 53 | the build identity in a register | 72 flip-flops, 0.007% of a device |
| 54 | o_margin_pct reading 0 | the next rebuild is a coin toss |
| 55 | o_overflow_ppm non-zero | rate_claim_valid low |
| 56 | o_pause_frames_sent non-zero | the deficit is being pushed upstream |
| 57 | a design at 11.4% LUT failing to route | Section 12 — no property expresses it |
| 58 | the whole MAC's utilisation | 11.4% LUT, 1.4% FF, 3.2% BRAM — and it binds on timing |
Scenario 48 is the one that makes the class visible and scenario 41 is the one most designs are in.
The directed test random stimulus will not produce
This test requires the same source, built twice, to produce two designs with two rates — one that meets the requirement and one that does not.
The arrangement:
| Step | Action |
|---|---|
| 1 | a 100 Gb/s MAC on a 512-bit datapath, constrained to 195.3 MHz |
| 2 | build with seed A; it closes at 196.4 MHz |
| 3 | every property in Section 20's groups A to E passes in simulation |
| 4 | build the identical source with seed B; it closes at 190.1 MHz |
| 5 | every property in Section 20's groups A to E passes in simulation |
| 6 | build A carries line rate; build B drops at 27 200 ppm |
Random stimulus will not produce this because the stimulus a generator varies is packets, and nothing about the packets differs between the two builds. The variable is the placement seed, which is an input to a tool rather than to a design, and no amount of simulation explores it — a simulator has no placement at all. The test is a build sweep, not a simulation, and it belongs to a flow that most verification plans do not consider part of verification.
The oracle is in four parts and the first three are expected passes.
| Part | Signal | Required value | Why alone it is not enough |
|---|---|---|---|
| 1 — simulation passes for both builds | Section 20's groups A–E | all 34, both times | the testbench supplies the clock |
| 2 — both builds complete | the tool's exit status | success | neither reports an error |
| 3 — build A meets the rate | 196.4 × 512 ÷ 1000 | 100.6 Gb/s | it is the build that shipped |
| 4 — build B does not | 190.1 × 512 ÷ 1000 | 97.3 Gb/s — short by 2.7% | this is the finding |
Part 1 is the assertion that matters and it asserts a pass. The properties are about the RTL and the RTL did not change; the clock in simulation is whatever the testbench drove, and it is not the clock either build produced. A verification plan that treats simulation coverage as evidence about the rate has made exactly the inference Section 20 refuses.
Part 4 needs the build sweep to have happened, which is why Section 20's third replacement property is a requirement on a flow rather than on a design: builds_seen >= SEEDS && closure_rate_pct == 100. Run the test with one build and the conclusion is the design meets line rate; run it with twenty and the conclusion is a percentage, which is the only defensible form.
22. Debugging an FPGA Ethernet Design
A procedure, and the first three steps are about a build rather than about a design.
| Step | Do | Because |
|---|---|---|
| 1 | read the build's achieved frequency from a register | Section 15 — the rate is width × frequency |
| 2 | compare it against the requirement | a 2.7% shortfall is a 27 200 ppm drop rate |
| 3 | read the seed and the build identifier | Section 10 — a rebuild is a different placement |
| 4 | read the receive FIFO's peak occupancy | a sustained deficit fills it in 193 µs |
| 5 | read the PAUSE-frame count | the deficit may already be pushed upstream |
| 6 | check the memory map for small structures | Section 6 — each costs a whole block |
| 7 | check routing congestion in the implementation report | Section 12 — a design at 11% LUT can fail to route |
| 8 | only now look at the RTL | it is the part that did not change |
Step 8 is last for a reason that is specific to this chapter. In every other chapter of the track a failure is in the design; here the design is a constant across builds and the failure moved with the placement. Starting at the RTL is starting at the one thing that is known not to have changed.
Four signatures.
| Signature | Cause |
|---|---|
| a steady drop rate that is a small percentage, with no error counters | a timing shortfall — Section 13 |
| a design that built last month and does not today | a different seed — Section 10 |
| PAUSE frames on a link with an idle partner | the local datapath is short of line rate |
| a failure to place at low utilisation | routing congestion from a wide datapath |
Row one is the one that is never diagnosed correctly without the frequency register. A 2.7% drop rate with every error counter at zero, every link healthy and every FIFO functioning looks like nothing in Chapter 21.2's taxonomy — because it is not an error, it is a datapath that cannot keep up with the wire, and the only evidence is a number from the build.
Row two is the support case that arrives attached to an innocent change. A comment edit, a rebuild, a different seed, and a design that has been in the field for a year no longer closes. Nothing in the source is wrong; the previous build was a draw and this one is a different draw, and the correct response is Section 11's sweep rather than a search for a regression.
23. Misconceptions
Six, in wrong-model / what-it-costs / corrected-model form.
Misconception 1 — "an FPGA design is an ASIC design with worse numbers."
Wrong model: the same architecture, a few times larger and a few times slower.
What it costs: the architecture. An ASIC MAC at 100 Gb/s runs a 64-bit datapath at 1.5 GHz; an FPGA MAC runs 1 024 bits at 97.7 MHz — and the FPGA's largest structure, a 41 984-LUT byte aligner, does not exist in the ASIC at all.
Corrected model: an FPGA reaches a rate by going wide, because roughly 200 MHz is where a non-trivial datapath becomes difficult and 400 is where it stops being realistic. Going wide quadruples the aligner — it is a barrel shifter whose positions and whose width both double — so the two designs are different designs rather than one design in two technologies.
Misconception 2 — "price the memory by its size."
Wrong model: Chapter 23.3 §2's BCE unit, applied to an FPGA.
What it costs: a factor of thirty-six on the smallest structures. Section 6: Chapter 22.1 §13's 1 024-bit delay line occupies a 36 864-bit block at 2.8% efficiency, and Chapter 19.7's 1 728-bit counter bank occupies another at 4.7%.
Corrected model: an FPGA memory costs ceil(bits ÷ 36 864) blocks, or enough blocks in parallel to be as wide as the access — whichever is larger. A 1 024-bit × 64-entry FIFO is two blocks by capacity and fifteen by width, so it costs seven and a half times its own size. Small costs a block; wide costs by width; large is efficient — the reverse of the ASIC intuition.
Misconception 3 — "we met timing, so we meet the rate."
Wrong model: the build closed, so the design works.
What it costs: the distinction between a sample and a distribution. Section 10: the achieved frequency varies by ±3 to ±8% with the placement seed alone, on identical source, identical constraints and an identical device. A build that closed at 195.3 MHz has established that one placement did.
Corrected model: the frequency is a random variable and one build is one draw. Twenty seeds is a few hours of overnight compute and produces a closure rate — and the common outcome of a first sweep is finding that a design believed to have 5% of margin has 0.4%. Constrain to the requirement divided by one minus the spread: 212.3 MHz to guarantee 195.3 at ±8%.
Misconception 4 — "a timing shortfall is a performance result."
Wrong model: the design missed by 3%, so it runs 3% slower.
What it costs: the design's correctness. Section 13: an Ethernet port cannot be derated. A 512-bit datapath at 190 MHz delivers 97.28 Gb/s against a wire still offering 100 — so Chapter 19.5's receive FIFO becomes a countdown, 193 microseconds at a 512-word depth, and the design drops at 27 200 parts per million, steadily, forever.
Corrected model: the arrival rate does not derate with the design, so a shortfall is loss rather than slowness. The honest responses are flow control, which declares the deficit and pushes it to the upstream switch's buffer, or a lower declared port speed. Claiming 100 Gb/s while pausing 2.7% of the time is neither.
Misconception 5 — "we are only at 11% utilisation, so there is plenty of room."
Wrong model: the utilisation report tells you how full the device is.
What it costs: the two constraints that actually bind. Section 12's complete 100 Gb/s MAC uses 11.4% of the LUTs, 1.4% of the flip-flops and 3.2% of the block RAMs — and it binds on timing and on routing, neither of which appears in a utilisation report.
Corrected model: a 1 024-bit datapath moves 1 024 wires between every pair of stages, and an FPGA's interconnect is a fixed fabric. A design at 11% LUT utilisation can fail to route, and the remedy is floorplanning rather than logic reduction. The practical ceiling for a timing-constrained design is 70 to 80%, so a fit check against the device's totals is optimistic by about a quarter.
Misconception 6 — "the design's rate is a property of the design."
Wrong model: what Modules 19 to 23 have assumed throughout, correctly, until now.
What it costs: the meaning of every rate property in the track. rate = width × frequency; the width is a design choice and the frequency is a build result — so the same source, built twice, is two designs with two rates, and the properties that passed in simulation passed against a clock the testbench invented.
Corrected model: an FPGA design's specification includes its build. The achieved frequency, the seed, the speed grade and a build identifier belong in a readable register — 72 flip-flops, 0.007% of a mid device — because a part dropping frames in the field must be debugged against the build that is running, and a rebuild is a different placement that may not reproduce the failure at all.
24. Interview Questions
Six, with what a strong answer contains.
1. Why does an FPGA Ethernet MAC use a 1 024-bit datapath when an ASIC uses 64?
Because an FPGA reaches a rate by going wide rather than fast. rate = width × frequency, and roughly 200 MHz is where a non-trivial FPGA datapath becomes difficult: 100 Gb/s needs 390.6 MHz at 256 bits, 195.3 at 512 and 97.7 at 1 024. A strong answer names the cost: the byte aligner is a barrel shifter over width ÷ 8 positions and each of width output bits needs a mux of that many inputs, so doubling the width quadruples it — 192 LUT6 at 64 bits and 41 984 at 1 024.
2. How does an FPGA memory's cost differ from an ASIC's?
It is quantised twice. A structure costs ceil(bits ÷ 36 864) blocks by capacity and ceil(width ÷ 72) blocks by width, whichever is larger. So a 1 024-bit delay line costs a whole 36 864-bit block — 2.8% efficient — and a 1 024-bit × 64 FIFO costs fifteen blocks for two blocks' worth of bits. A strong answer gives both remedies: merge small structures behind a shared decoder, and build shallow wide ones from flip-flops, which an Ethernet design has in abundance.
3. What does a 3% timing shortfall cost an Ethernet MAC?
Not 3% of performance — a 27 200 ppm drop rate. A processor short of its frequency does less work per second; an Ethernet MAC faces a wire still delivering 100 Gb/s. A 512-bit datapath at 190 MHz delivers 97.28 Gb/s, so Chapter 19.5's receive FIFO has a drain below its arrival rate and becomes a countdown — 193 microseconds at 512 words — after which the design drops steadily. A strong answer names the honest responses: flow control, which declares the deficit, or a lower declared port speed.
4. Why is signing off on one build not enough?
Because the achieved frequency is a random variable. The same source, the same constraints, the same device and a different placement seed give a frequency that varies by ±3 to ±8%. A build that closed has established that one placement did. A strong answer names the sweep — twenty seeds, a few hours of overnight compute, and a closure rate rather than a pass — and the common finding: a design believed to have 5% of margin frequently has 0.4%.
5. A design uses 11% of the LUTs and fails. Why?
Timing or routing, and neither is in the utilisation report. A 1 024-bit datapath moves 1 024 wires between every pair of stages, and an FPGA's interconnect is a fixed fabric that place-and-route needs slack to navigate. A strong answer gives the practical ceiling — 70 to 80% for a timing-constrained design — and the remedies: floorplanning, registering every module boundary, and pinning wide structures near their memories, none of which reduces logic.
6. What does it mean that a design's rate is a build result?
That the same source, built twice, is two designs. rate = width × frequency; the width is chosen and the frequency is reported — so a property proved in simulation was evaluated against a clock the testbench supplied, and nothing relates it to the clock place-and-route produced. A strong answer names both fixes: a generated package carrying the build's achieved frequency, so a post-synthesis check has the real number, and a build-identity register — 72 flip-flops — so a field failure can be rebuilt against the placement that produced it.
25. Questions and Answers
26. What's Next
Module 23 is complete. Six chapters, one fabric, one workload, one switch and three network interfaces — and three different units, because what is scarce changed three times.
| Chapter | Its result |
|---|---|
| Chapter 23.1 | 2 048 servers from one part number; 28.4% of the bisection reachable by 32 flows |
| Chapter 23.2 | a collective runs at its minimum; one uplink stalls 1 024 accelerators |
| Chapter 23.3 | 1 BCE = one SRAM bit; a switch is 1 985 datapaths and 91% buffer |
| Chapter 23.4 | descriptors are 38% of the payload; the context cache has a cliff at 39.4% |
| Chapter 23.5 | 645 instructions per packet for 197× the area, and a behaviour decided after shipping |
| this chapter | a fixed inventory, a quadratic aligner, and a rate that is a build result |
And the module's through-line is the unit. Chapter 19.7 §19's flop worked for a MAC because a MAC is almost all logic; it failed five times over Modules 21 to 23 and Chapter 23.3 §2 replaced it with the bitcell equivalent, which worked for a switch, a NIC and a programmable array. This chapter needed a third unit — a resource inventory and a timing distribution — because in an FPGA the scarce things are neither gates nor bits.
A unit is a model of what is scarce. Three chapters and three units, and the discipline is to notice when the thing being counted has stopped being the thing that runs out.
Module 24 turns from what Ethernet is to what it is not. Three interconnects sit beside it in real systems and each answers a question Ethernet answers differently: Chapter 24.1 sets a packet network against a load/store fabric, where an address is a destination rather than a name and a read has a return path; Chapter 24.2 sets it against a lossless HPC fabric, where credit-based flow control removes the drop that Chapter 21.6 built a chapter around; and Chapter 24.3 asks what a design gets by leaving a standard at all — which is the question every chapter in Module 23 has implicitly answered by staying inside one.
Continue learning
Related tutorials
- Related topic
PCIe vs Ethernet — Where the Cost of Overload Lands
The same overload into two fabrics: one stalled the sender 59,405 times and lost nothing, the other discarded 59,405 frames. That single choice explains why one needs TCP and the other does not.
- Related topic
The Shared-Medium Problem
Why several independent transmitters on one medium is a distributed timing problem, not a formatting problem. Propagation delay makes every station's view of the medium stale, so two locally correct decisions can still collide — and that is the constraint the Ethernet MAC was built around.
- Related topic
CSMA/CD, Collision Domains and Slot Time
Slot time is the parameter the whole half-duplex MAC hangs on: it bounds medium acquisition, bounds a collision fragment, and is the retransmission quantum. Deriving it from round-trip propagation plus jam is what fixes Ethernet's minimum frame size — a timing constant wearing a frame-format costume.
- Related topic
Packet Switching
A circuit allocates capacity in advance and guarantees it; a packet network allocates on demand and guarantees nothing. The exchange is measurable in RTL — idle reserved slots against buffered, delayed and occasionally dropped packets — and it is why a packet must describe its own extent and destination.
Standards & specifications
- Governing standard
- IEEE Std 802.3 (Ethernet)(opens IEEE in a new tab)
Defines the Ethernet MAC, the media-independent interfaces and the physical-layer sublayers, including framing, access control, auto-negotiation and per-rate PHY specifications. VLAN tagging, priority and time-sensitive shaping are defined by IEEE 802.1, not by 802.3.
This page also covers RTL structure, verification approach and debugging technique. Those are engineering practice built on the standard, not requirements the standard itself imposes.
Where this fits
Part of the Ethernet curriculum.
