Ethernet · Module 12
The MAC Table — CAM, Hashing, Capacity and Aging
An 8192-entry four-way table refuses inserts while 4096 slots are free, and holds 6592 addresses when asked for 8192. Capacity is a property of the hash, not of the memory.
Chapter 12.3 §11 established that the lookup is the only expensive gate in the forwarding decision — 4 to 14 cycles against a 28 ns budget, with every other gate free. Chapter 12.4 established that every table miss costs N−1 copies of egress bandwidth.
This chapter builds the structure both of those depend on, and it opens with the finding that surprises people most.
An 8192-entry forwarding table does not hold 8192 addresses. A four-way set-associative table offered 8192 distinct addresses stores 6592 of them — 80.5% — and refuses the rest. And it begins refusing long before that: at 50% occupancy, with 4096 slots standing empty, 108 of its 2048 sets are already full.
Nothing is broken. The memory is the size it says it is. What has happened is that capacity is not a property of the memory — it is a property of the hash, and the hash is computed from addresses the switch does not choose.
1. Scope — What This Chapter Owns
This chapter owns the structure behind lookup_hit and lookup_port: how a 48-bit key is searched in single-digit cycles, what a CAM costs, how a hash turns a search into an indexed read, what a collision does, why capacity is smaller than the memory, how associativity trades against set count, what gets evicted, and what the ageing sweep actually costs.
It does not own the table's contents — Chapter 12.2 owns learning, the eligibility rules, station moves and why ageing exists at all. This chapter owns the sweep, not the policy.
It does not own the decision — Chapter 12.3 owns the six gates, and consumes this chapter's output as gate 3.
It does not own the cost of a miss — Chapter 12.4 priced that, and this chapter uses the price as the currency in which capacity trades are settled.
And it does not own per-VLAN partitioning. Widening the key from 48 bits to {VID, MAC} changes every number here, and Chapter 13.4 owns it. Section 10's associativity trade is the machinery that chapter will need.
2. What the Table Must Actually Do
Write down the requirement before choosing a structure, because the requirement is unusual and it is what rules most structures out.
| Operation | Rate | Latency bound | Notes |
|---|---|---|---|
| lookup by 48-bit key | 35.71 M/s | 28 ns — 14 cycles at 500 MHz | Chapter 12.3 §11 |
| insert / refresh / move | 35.71 M/s | must complete before the lookup | Chapter 12.2 §7's ordering |
| age every entry | once per second | none | Section 13 |
| flush by port | on link-down | microseconds | Chapter 12.2 §15 |
The first two rows together are the constraint that shapes everything. Chapter 12.2 §7 established that learning must complete before the lookup on the same frame, so every frame is a write followed by a read — 71.42 M accesses per second, which at 500 MHz is 14.3% of a single-ported memory's cycles.
Bandwidth, therefore, is not the problem. 14.3% leaves ample headroom. Latency is the problem, and it is a hard bound: the answer must exist 28 ns after the question, because Chapter 12.3 §4's deadline expires and the frame floods.
Which immediately rules out the obvious structure. A linear search of 8192 entries at one entry per cycle takes 8192 cycles — 585 times the budget. Chapter 12.2 §6's behavioural store used exactly that search and said so; this chapter is why.
3. RTL 1 — The CAM, and What Parallel Search Costs
A content-addressable memory answers the question directly: present the key, and every entry compares itself against it simultaneously.
// -----------------------------------------------------------------------
// mactable_pkg -- shared types for the forwarding table.
// -----------------------------------------------------------------------
package mactable_pkg;
localparam int ADDR_W = 48;
// What an insert did. Chapter 12.2 Section 6 named the first four; this
// chapter adds the two that only a hashed structure can produce.
typedef enum logic [2:0] {
TI_INSERT = 3'd0, // new key, free way in its set
TI_REFRESH = 3'd1, // same key, same port -- age reset
TI_MOVE = 3'd2, // same key, different port
TI_EVICT = 3'd3, // new key, set full -- something was displaced
TI_SET_FULL = 3'd4, // new key, set full, eviction NOT permitted
TI_NONE = 3'd5
} table_insert_e;
// An entry. Chapter 12.2 Section 6 established the five fields; the
// hashed structure adds nothing, because the hash is recomputed rather
// than stored.
typedef struct packed {
logic valid;
logic [ADDR_W-1:0] key; // the FULL address -- a hash is not
// reversible, so the tag is 48 bits
logic [4:0] port;
logic [8:0] age; // 300 s at a 1 s tick
logic stat; // operator-asserted, never ages
} tentry_t;
endpackage// -----------------------------------------------------------------------
// cam_lookup_model -- a binary content-addressable memory.
//
// BEHAVIOURAL. A real CAM is a custom cell array, not a synthesized
// comparator tree, and this module exists to make the COST structure
// explicit rather than to be built. Every entry compares in parallel; the
// answer is available in one compare plus one priority encode.
// -----------------------------------------------------------------------
module cam_lookup_model
import mactable_pkg::*;
#(
parameter int N_ENTRIES = 8192,
parameter int IDX_W = 13,
parameter int CNT_W = 32
)(
input logic clk,
input logic rst_n,
input logic search_valid,
input logic [ADDR_W-1:0] search_key,
input logic write_valid,
input logic [IDX_W-1:0] write_index,
input tentry_t write_entry,
output logic hit,
output logic [IDX_W-1:0] hit_index,
output tentry_t hit_entry,
output logic multi_hit, // the same key stored twice
// Cost accounting -- the reason this module is in the chapter.
output logic [CNT_W-1:0] c_searches,
output logic [CNT_W-1:0] c_match_line_toggles,
output logic [31:0] compare_bits_per_search
);
tentry_t mem [N_ENTRIES];
logic [N_ENTRIES-1:0] match;
// THE DEFINING PROPERTY. Every entry compares against the key, every
// search, in one cycle. That is what buys the latency and what costs
// the power: N_ENTRIES x ADDR_W comparison bits toggle per search,
// whether or not any of them match.
always_comb begin
for (int i = 0; i < N_ENTRIES; i++)
match[i] = mem[i].valid && (mem[i].key == search_key);
end
// Priority encode. A well-formed table has at most one match; multi_hit
// means the same key was inserted twice, which a CAM does not prevent.
always_comb begin
hit = 1'b0;
hit_index = '0;
hit_entry = '0;
multi_hit = 1'b0;
if (search_valid) begin
hit = |match;
multi_hit = ($countones(match) > 1);
for (int i = N_ENTRIES-1; i >= 0; i--)
if (match[i]) begin
hit_index = IDX_W'(i);
hit_entry = mem[i];
end
end
end
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
for (int i = 0; i < N_ENTRIES; i++) mem[i] <= '0;
c_searches <= '0;
c_match_line_toggles <= '0;
end else begin
if (write_valid) mem[write_index] <= write_entry;
if (search_valid) begin
c_searches <= c_searches + 1'b1;
// Every match line is evaluated on every search. This counter is
// a proxy for the dynamic power a hashed structure avoids.
c_match_line_toggles <= c_match_line_toggles + CNT_W'(N_ENTRIES);
end
end
end
assign compare_bits_per_search = 32'(N_ENTRIES) * 32'(ADDR_W);
endmoduleClassification: behavioural model — a production CAM is a custom cell array with match lines, sense amplifiers and a priority encoder, and no synthesis tool produces one from this description.
What it teaches: that the CAM's latency is a constant and its cost is the whole array. One compare cycle plus one encode cycle answers any key in 2 cycles, comfortably inside Chapter 12.3's 14-cycle budget and independent of how full the table is. And it teaches what buys that: 8192 × 48 = 393 216 comparison bits are evaluated on every single search, at 35.71 M searches per second, whether the key is present or not.
And it teaches that a CAM does not enforce key uniqueness. multi_hit exists because nothing in the structure prevents the same address occupying two entries — a hashed table cannot do this, because a key has exactly one set — and a duplicated key means two answers, of which the priority encoder silently picks one.
Deliberately simplified: a binary CAM with no masking. Real forwarding CAMs are often ternary, supporting don't-care bits so that one entry can match a range — which is essential for the longest-prefix matching a router does and entirely unnecessary for the exact 48-bit match a switch does. Paying for ternary here buys nothing.
Production implication: c_match_line_toggles is the number that decides the structure. At 35.71 M searches per second, an 8192-entry CAM evaluates 35.71 M × 8192 = 292 billion match-line evaluations per second. A four-way hashed table evaluates four comparators per search — 2048× fewer — and that ratio, not the area, is why the hashed structure wins in anything power-constrained.
4. Why Nobody Builds an 8192-Entry CAM for This
The CAM is correct, fast and simple. It loses on two numbers, and both scale the wrong way.
| 8192 × 48-bit CAM | 8192 × 4-way hashed SRAM | |
|---|---|---|
| storage cells | 393 216 CAM cells | 393 216 SRAM cells |
| transistors per cell | 10–16 | 6 |
| relative area | 1.7× to 2.7× | 1× |
| comparators active per search | 8192 | 4 |
| match-line evaluations at 35.71 M/s | 292 × 10⁹ per second | 143 × 10⁶ per second |
| lookup latency | 2 cycles | 4 cycles |
| usable capacity | 100% | 80.5% — Section 9 |
The area ratio is bad and survivable. A 1.7× to 2.7× area penalty on a 393 kbit structure is real but not decisive on a switch die that also contains 24 SerDes, a packet buffer and a fabric.
The activity ratio is the one that decides it. 8192 ÷ 4 = 2048× more comparator activity per search, at 35.71 M searches per second, permanently, whether the network is busy or idle. A CAM burns that power to answer questions whose answers are usually "no".
And the third column of the trade is what this chapter is really about. The hashed table gives back area and power and takes payment in capacity that cannot be predicted from the memory size — which is the entire remainder of this chapter, and which turns out to be the cheaper currency because Chapter 12.4 already established what a miss costs and it is bounded.
5. RTL 2 — The Hash, and What Makes One Good
A hash turns a 48-bit search into an 11-bit index. Everything that follows in this chapter is a consequence of how well it spreads.
// -----------------------------------------------------------------------
// address_hash -- 48 bits to an index, plus the quality instrumentation
// that says whether it is spreading.
//
// The requirement is unusual: the input is NOT uniformly distributed. A
// MAC address is a 24-bit OUI followed by a 24-bit serial (Chapter 5.3),
// and a rack of identical machines shares the OUI and often has serials
// that differ in a handful of low bits. A hash that ignores the low bits,
// or that is dominated by the OUI, clusters catastrophically.
// -----------------------------------------------------------------------
module address_hash
import mactable_pkg::*;
#(
parameter int IDX_W = 11, // 2048 sets
parameter int CNT_W = 32
)(
input logic clk,
input logic rst_n,
input logic key_valid,
input logic [ADDR_W-1:0] key,
output logic [IDX_W-1:0] index,
// Quality instrumentation. A hash cannot be judged by inspection; it
// has to be measured against the addresses actually present.
input logic sample_enable,
output logic [CNT_W-1:0] c_hashed,
output logic [IDX_W-1:0] observed_min_index,
output logic [IDX_W-1:0] observed_max_index,
output logic low_bits_only // index ignores the OUI
);
// A CRC-derived fold. The low 24 bits (the vendor's serial) carry the
// entropy; the top 24 (the OUI) are near-constant across a rack, so the
// hash must not be dominated by them -- but it must not IGNORE them
// either, or two vendors' identical serials collide.
logic [IDX_W-1:0] h;
always_comb begin
automatic logic [23:0] lo = key[23:0];
automatic logic [23:0] hi = key[47:24];
automatic logic [23:0] mix;
// Fold the OUI into the serial with a rotate, so that neither half
// can dominate and identical serials from different vendors separate.
mix = lo ^ {hi[11:0], hi[23:12]} ^ {lo[7:0], lo[23:8]};
h = mix[IDX_W-1:0] ^ mix[23:24-IDX_W];
end
assign index = h;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
c_hashed <= '0;
observed_min_index <= '1;
observed_max_index <= '0;
low_bits_only <= 1'b0;
end else if (key_valid && sample_enable) begin
c_hashed <= c_hashed + 1'b1;
if (h < observed_min_index) observed_min_index <= h;
if (h > observed_max_index) observed_max_index <= h;
// A cheap standing check: if the index never depends on the OUI,
// an entire rack of one vendor's machines lands in a narrow band.
if (h == key[IDX_W-1:0]) low_bits_only <= 1'b1;
end
end
endmoduleClassification: synthesizable.
What it teaches: that the input to this hash is not uniformly distributed and that is the entire difficulty. Chapter 5.3 established the 48-bit layout: a 24-bit OUI identifying the manufacturer, and 24 bits the manufacturer assigns. A rack of forty identical servers shares the OUI exactly and frequently has serial numbers that differ only in the low six or seven bits — they came off one production line, in sequence.
So the two obvious hashes both fail, in opposite directions. Taking the low 11 bits directly maps that rack into 40 consecutive sets and ignores the OUI entirely, so two vendors' machines with the same serial collide every time. Taking the high bits maps the whole rack into one set, which is the pathological case Section 7 quantifies.
Deliberately simplified: an XOR-and-rotate fold. Production designs use a proper CRC — usually the same CRC-32 polynomial Chapter 5.8 already requires in the datapath, which makes it nearly free — and often make the polynomial or a seed programmable, so that a deployment suffering pathological clustering can be re-hashed without new silicon.
Production implication: low_bits_only is a standing check for the mistake that is easiest to make and hardest to notice. A hash that reduces to the low index bits works perfectly in simulation with random addresses — random addresses spread under any function of them — and clusters severely in a real rack. The failure appears only after deployment, on one customer's network, as a table that refuses inserts at 30% occupancy.
6. RTL 3 — The Set-Associative Store
One SRAM read returns a whole set. Four comparators check it. That is the entire lookup, and it is four cycles.
// -----------------------------------------------------------------------
// hashed_set_table -- W-way set-associative forwarding table.
//
// THE CENTRAL STRUCTURE OF THIS CHAPTER. The key hashes to exactly one
// set; the set is read in one access; the ways are compared in parallel.
// A key that is not in its set is not in the table -- the hash is
// deterministic, so a miss here is a real miss and not a partial search.
// -----------------------------------------------------------------------
module hashed_set_table
import mactable_pkg::*;
#(
parameter int N_SETS = 2048,
parameter int WAYS = 4,
parameter int IDX_W = 11,
parameter int WAY_W = 2,
parameter int PORT_W = 5,
parameter int AGE_W = 9,
parameter int CNT_W = 32
)(
input logic clk,
input logic rst_n,
// ---- Lookup -------------------------------------------------------
input logic lk_valid,
input logic [IDX_W-1:0] lk_index, // from address_hash
input logic [ADDR_W-1:0] lk_key,
output logic lk_hit,
output logic [PORT_W-1:0] lk_port,
output logic [WAY_W-1:0] lk_way,
// ---- Insert / refresh / move --------------------------------------
input logic ins_valid,
input logic [IDX_W-1:0] ins_index,
input logic [ADDR_W-1:0] ins_key,
input logic [PORT_W-1:0] ins_port,
input logic evict_permitted,
input logic [WAY_W-1:0] evict_way, // from set_eviction_selector
output table_insert_e ins_result,
output logic [PORT_W-1:0] displaced_port,
output logic [ADDR_W-1:0] displaced_key,
// ---- Age ----------------------------------------------------------
input logic age_write,
input logic [IDX_W-1:0] age_index,
input logic [WAY_W-1:0] age_way,
input logic age_expire,
input logic [AGE_W-1:0] age_next,
output tentry_t age_read_entry,
// ---- Observability -------------------------------------------------
output logic [3:0] set_occupancy, // ways used in the last set read
output logic set_was_full,
output logic [CNT_W-1:0] occupancy,
output logic [CNT_W-1:0] c_set_full_refusals
);
// The array. One read port serving lookup, insert and the ageing sweep;
// Section 2 showed the combined demand is 14.3% of a 500 MHz memory, so
// one port suffices and the arbitration is trivial.
tentry_t mem [N_SETS][WAYS];
// ---- Lookup: read the set, compare the ways ------------------------
logic [WAYS-1:0] way_match;
logic [3:0] used_ways;
always_comb begin
way_match = '0;
used_ways = '0;
for (int w = 0; w < WAYS; w++) begin
if (mem[lk_index][w].valid) begin
used_ways = used_ways + 4'd1;
// THE COMPARE IS AGAINST THE FULL 48-BIT KEY. A hash is not
// reversible, so the index does not narrow the stored key at all
// -- unlike a cache tag, nothing may be omitted.
if (mem[lk_index][w].key == lk_key) way_match[w] = 1'b1;
end
end
end
always_comb begin
lk_hit = 1'b0;
lk_port = '0;
lk_way = '0;
if (lk_valid) begin
lk_hit = |way_match;
for (int w = WAYS-1; w >= 0; w--)
if (way_match[w]) begin
lk_port = mem[lk_index][w].port;
lk_way = WAY_W'(w);
end
end
end
assign set_occupancy = used_ways;
assign set_was_full = (used_ways == 4'(WAYS));
// ---- Insert --------------------------------------------------------
logic [WAYS-1:0] ins_match, ins_free;
always_comb begin
ins_match = '0;
ins_free = '0;
for (int w = 0; w < WAYS; w++) begin
if (mem[ins_index][w].valid) begin
if (mem[ins_index][w].key == ins_key) ins_match[w] = 1'b1;
end else begin
ins_free[w] = 1'b1;
end
end
end
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
for (int s = 0; s < N_SETS; s++)
for (int w = 0; w < WAYS; w++) mem[s][w] <= '0;
ins_result <= TI_NONE;
displaced_port <= '0;
displaced_key <= '0;
occupancy <= '0;
c_set_full_refusals <= '0;
end else begin
ins_result <= TI_NONE;
if (ins_valid) begin
automatic int hw = -1;
automatic int fw = -1;
for (int w = 0; w < WAYS; w++) begin
if (ins_match[w]) hw = w;
if (ins_free[w] && (fw < 0)) fw = w;
end
if (hw >= 0) begin
// The key is present in its set. Refresh or move -- Chapter
// 12.2 Section 6's distinction, unchanged by the structure.
if (mem[ins_index][hw].port == ins_port) begin
mem[ins_index][hw].age <= '0;
ins_result <= TI_REFRESH;
end else begin
displaced_port <= mem[ins_index][hw].port;
mem[ins_index][hw].port <= ins_port;
mem[ins_index][hw].age <= '0;
ins_result <= TI_MOVE;
end
end else if (fw >= 0) begin
mem[ins_index][fw] <= '{valid: 1'b1, key: ins_key,
port: ins_port, age: '0, stat: 1'b0};
occupancy <= occupancy + 1'b1;
ins_result <= TI_INSERT;
end else if (evict_permitted) begin
// THE SET IS FULL AND THE TABLE MAY NOT BE. Something must be
// displaced, and Section 11 chooses which -- from four
// candidates the arriving key's hash selected.
displaced_key <= mem[ins_index][evict_way].key;
displaced_port <= mem[ins_index][evict_way].port;
mem[ins_index][evict_way] <= '{valid: 1'b1, key: ins_key,
port: ins_port, age: '0,
stat: 1'b0};
ins_result <= TI_EVICT;
end else begin
// REFUSE. Chapter 12.2 Section 12 established why refusing beats
// evicting under attack: invented addresses cannot displace
// stations that are actually communicating.
ins_result <= TI_SET_FULL;
if (!(&c_set_full_refusals))
c_set_full_refusals <= c_set_full_refusals + 1'b1;
end
end
// ---- Ageing write-back ------------------------------------------
if (age_write) begin
if (age_expire) begin
mem[age_index][age_way].valid <= 1'b0;
occupancy <= occupancy - 1'b1;
end else begin
mem[age_index][age_way].age <= age_next;
end
end
end
end
assign age_read_entry = mem[age_index][age_way];
endmoduleClassification: synthesizable, with mem inferring a two-dimensional SRAM whose row is one full set.
What it teaches: that the whole set is one memory word. A four-way set of {valid, 48-bit key, 5-bit port, 9-bit age, static} is 4 × 64 = 256 bits, read in a single access. That is why the lookup is four cycles and not sixteen — the ways are not searched one at a time, they arrive together and four comparators resolve them in parallel.
And it teaches the one place a forwarding table differs from a cache at the bit level. A cache stores a tag — the address bits the index did not cover — because the index is a slice of the address and can be reconstructed. A hash is not reversible, so the index reconstructs nothing and the full 48-bit key must be stored. There is no tag saving here at all, which is why the entry width in Chapter 12.2 §12 was 64 bits regardless of structure.
Deliberately simplified: a single memory port shared by lookup, insert and the sweep, with no arbitration shown. Section 2's arithmetic justifies the sharing — 14.3% duty — but a real design still needs an arbiter, and the sweep must yield to both other clients since neither can wait.
Production implication: ins_result distinguishing TI_SET_FULL from TI_EVICT is the field that makes the next four sections diagnosable. TI_SET_FULL means an address could not be learned while the table had free slots elsewhere — a hash-distribution event, not a capacity event — and occupancy sitting far below N_SETS × WAYS while refusals climb is the signature Section 9 quantifies. A design that reports only "table full" merges the two and sends an operator to buy capacity that will not help.
7. The Collision, and What It Actually Costs
Two addresses whose hashes are equal share a set. With four ways that is harmless. With five it is not, and five arrive sooner than intuition suggests.
This is the classic balls-in-bins problem with a capacity limit, and the arithmetic is unforgiving. With n addresses spread over S = 2048 sets, the load on any one set is approximately Poisson with mean λ = n ÷ S, and a set overflows when more than W = 4 addresses land in it.
| Table occupancy | Entries present | Sets overflowing | Sets, of 2048 |
|---|---|---|---|
| 12.5% | 1024 | 0.02% | 0.4 |
| 25% | 2048 | 0.37% | 7.5 |
| 37.5% | 3072 | 1.86% | 38.0 |
| 50% | 4096 | 5.27% | 107.8 |
| 62.5% | 5120 | 10.88% | 222.9 |
| 75% | 6144 | 18.47% | 378.3 |
| 90% | 7372 | 29.35% | 601.1 |
Read the 50% row. A table with 4096 entries free is already refusing inserts for any address that hashes to one of 108 sets. From the outside it is half empty. From the point of view of an address unlucky in its hash, it is full.
And the cost of that refusal is exactly Chapter 12.4's price. The station is never learned, every frame to it floods to N−1 ports, and — since Chapter 12.2 established that a station only appears in the table when it transmits — the condition persists for as long as that station's set stays full.
8. RTL 4 — Watching the Distribution, Not the Total
Occupancy is the number everyone reports and it is the wrong one. What matters is how unevenly the sets are loaded.
// -----------------------------------------------------------------------
// set_occupancy_monitor -- a histogram over set load, and the derived
// quantities that say whether the hash is doing its job.
//
// Total occupancy answers "how full is the table". It cannot answer "can
// the next address be learned", which depends entirely on ONE set chosen
// by that address's hash. Section 9 shows the two diverge sharply.
// -----------------------------------------------------------------------
module set_occupancy_monitor
import mactable_pkg::*;
#(
parameter int N_SETS = 2048,
parameter int WAYS = 4,
parameter int IDX_W = 11,
parameter int CNT_W = 32
)(
input logic clk,
input logic rst_n,
// Sampled from the sweep, which already visits every set once a second.
input logic sample_valid,
input logic [IDX_W-1:0] sample_index,
input logic [3:0] sample_used_ways,
input logic sweep_wrapped,
input logic [CNT_W-1:0] occupancy,
input logic [CNT_W-1:0] set_full_refusals,
output logic [CNT_W-1:0] sets_with [WAYS+1], // histogram: 0..WAYS used
output logic [15:0] occupancy_pct,
output logic [15:0] full_sets_pct,
output logic [IDX_W-1:0] hottest_set,
output logic hash_clustering, // distribution is skewed
output logic refusing_while_empty, // the Section 9 condition
output logic histogram_valid
);
logic [CNT_W-1:0] acc [WAYS+1];
logic [CNT_W-1:0] acc_full;
logic [CNT_W-1:0] prev_refusals;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
for (int i = 0; i <= WAYS; i++) begin
acc[i] <= '0; sets_with[i] <= '0;
end
acc_full <= '0;
hottest_set <= '0;
occupancy_pct <= '0;
full_sets_pct <= '0;
hash_clustering <= 1'b0;
refusing_while_empty <= 1'b0;
histogram_valid <= 1'b0;
prev_refusals <= '0;
end else begin
histogram_valid <= 1'b0;
if (sample_valid) begin
acc[sample_used_ways] <= acc[sample_used_ways] + 1'b1;
if (sample_used_ways == 4'(WAYS)) begin
acc_full <= acc_full + 1'b1;
hottest_set <= sample_index;
end
end
// One full pass of the sweep is one complete census of the table.
if (sweep_wrapped) begin
for (int i = 0; i <= WAYS; i++) begin
sets_with[i] <= acc[i];
acc[i] <= '0;
end
occupancy_pct <= 16'((occupancy * CNT_W'(100)) /
CNT_W'(N_SETS * WAYS));
full_sets_pct <= 16'((acc_full * CNT_W'(100)) / CNT_W'(N_SETS));
// THE CONDITION THIS MODULE EXISTS FOR. Inserts are being refused
// while the table as a whole is far from full -- which is a hash
// distribution problem and NOT a capacity problem, and the two
// have entirely different remedies.
refusing_while_empty <=
((set_full_refusals != prev_refusals) &&
(occupancy < CNT_W'((N_SETS * WAYS * 3) / 4)));
// Skew: more full sets than a well-spread hash would produce at
// this occupancy. At 50% occupancy a good hash gives about 5% of
// sets full; three times that is clustering.
hash_clustering <= (acc_full > ((CNT_W'(N_SETS) * CNT_W'(15)) / 100)) &&
(occupancy < CNT_W'((N_SETS * WAYS) / 2));
prev_refusals <= set_full_refusals;
acc_full <= '0;
histogram_valid <= 1'b1;
end
end
end
endmoduleClassification: synthesizable.
What it teaches: that the census is free, because something already walks every set once a second. Section 13's ageing sweep visits every entry to decrement its timer; sampling the used-way count on the way past costs a four-bit adder and produces a complete distribution histogram every sweep period. A design that reports only total occupancy has walked the whole table and thrown the interesting number away.
And refusing_while_empty is the finding. Chapter 12.2 §12's attack fills a table and is visible as high occupancy. This is the opposite condition — refusals with occupancy at 30% — and it means the hash is clustering on the addresses this particular network happens to contain. Nothing about it is visible from any total.
Deliberately simplified: a histogram over the whole table with one hot-set register. Production monitors keep the top few hot sets and often a per-set refusal count, because a single pathological set caused by one rack of sequentially-addressed machines is a different remedy from broad clustering.
Production implication: hash_clustering distinguishes a design problem from a deployment problem, and the distinction determines who fixes it. Broad clustering at low occupancy means the hash function is wrong — Section 5's low_bits_only failure, shipped. A single hot set means one customer's address block is unlucky, which a programmable hash seed fixes in the field without new silicon, and which is why production hashes are seeded.
9. Full While Empty — Effective Capacity
Section 7 measured how many sets are full at a given occupancy. Turn the question round: offer the table 8192 distinct addresses and count how many it actually stores.
The answer is the sum over sets of min(load, W), and with n = 8192 addresses over S = 2048 sets of W = 4 ways it is 6592.
| Structure | Offered 8192 addresses | Stored | Effective capacity |
|---|---|---|---|
| direct-mapped, 8192 sets × 1 | 8192 | 5178 | 63.2% |
| 2-way, 4096 sets | 8192 | 5975 | 72.9% |
| 4-way, 2048 sets | 8192 | 6592 | 80.5% |
| 8-way, 1024 sets | 8192 | 7049 | 86.0% |
| 16-way, 512 sets | 8192 | 7379 | 90.1% |
| CAM, 8192 entries | 8192 | 8192 | 100% |
A datasheet saying "8192 MAC addresses" describes the memory, and a four-way implementation of it holds 6592. The missing 1600 addresses are not lost to a defect — they are addresses whose sets were already full while other sets had room.
And it gets worse before it gets better, because offering more addresses fills the sparse sets:
| Addresses offered to the 4-way table | Stored | Effective capacity |
|---|---|---|
| 8 192 | 6592 | 80.5% |
| 16 384 | 8070 | 98.5% |
| 32 768 | 8192 | 100% |
The table only reaches its nominal capacity when it is offered four times that many addresses — by which point 24 576 of them have been refused. "Full" is a state the table reaches by rejecting three addresses for every one it keeps.
10. Associativity Against Depth
At constant capacity, more ways means fewer sets. The trade is not free in either direction and both ends are bad.
| Structure | Index bits | Set width | Sets full at 50% | Effective capacity |
|---|---|---|---|---|
| 1-way × 8192 | 13 | 64 bits | 9.02% — 739 sets | 63.2% |
| 2-way × 4096 | 12 | 128 bits | 8.03% — 329 sets | 72.9% |
| 4-way × 2048 | 11 | 256 bits | 5.27% — 108 sets | 80.5% |
| 8-way × 1024 | 10 | 512 bits | 2.14% — 22 sets | 86.0% |
| 16-way × 512 | 9 | 1024 bits | 0.37% — 2 sets | 90.1% |
More associativity is strictly better for capacity and strictly worse for everything else.
The set width is the constraint. A 16-way set is 16 × 64 = 1024 bits read in one access, and 16 comparators of 48 bits each must resolve within the cycle. That is the CAM's problem reappearing at one sixteenth of its scale — and at some associativity the structure has simply become a small CAM with an index in front of it.
The comparator count is the power. Section 4's decisive number was 8192 ÷ 4 = 2048× activity reduction. At 16 ways it is 8192 ÷ 16 = 512× — still enormous, and four times worse than the four-way design.
And the marginal return is falling. Going from 4-way to 8-way buys 86.0 − 80.5 = 5.5 percentage points of effective capacity for double the set width and double the comparators. Going from 8 to 16 buys 4.1 more points for another doubling.
Which is why four and eight ways are what actually gets built. Four-way sits at the knee: it recovers most of the capacity a direct-mapped structure loses — 80.5% against 63.2% — while keeping the set to one comfortable memory word and the comparator count to four.
11. RTL 5 — Choosing What to Evict
When a set is full and the design has decided to evict rather than refuse, something must be chosen. The choice is made from four candidates, and which four was decided by the arriving address's hash.
// -----------------------------------------------------------------------
// set_eviction_selector -- picks a way to displace within one set.
//
// READ THE SCOPE CAREFULLY. This selects the best candidate IN THIS SET.
// It cannot select the best candidate in the table, because the other
// 2047 sets are not readable in the cycle budget and are not candidates
// at all. Section 12 and Section 18's rejected property are about the
// gap between those two sentences.
// -----------------------------------------------------------------------
module set_eviction_selector
import mactable_pkg::*;
#(
parameter int WAYS = 4,
parameter int WAY_W = 2,
parameter int AGE_W = 9,
parameter int CNT_W = 32
)(
input logic clk,
input logic rst_n,
input logic sel_valid,
input tentry_t set_ways [WAYS],
input logic attack_suspected, // Chapter 12.2's rate limiter
output logic evict_permitted,
output logic [WAY_W-1:0] evict_way,
output logic [AGE_W-1:0] evict_age,
output logic all_ways_fresh, // nothing is a good victim
output logic [CNT_W-1:0] c_evictions,
output logic [CNT_W-1:0] c_refusals,
output logic [CNT_W-1:0] c_fresh_evictions, // displaced something ACTIVE
output logic [AGE_W-1:0] youngest_evicted
);
// Oldest of the four. Static entries are never candidates -- Chapter
// 12.2 Section 6 established that an operator asserted them, so no
// automatic mechanism may retract them.
logic [WAY_W-1:0] oldest_way;
logic [AGE_W-1:0] oldest_age;
logic any_candidate;
always_comb begin
oldest_way = '0;
oldest_age = '0;
any_candidate = 1'b0;
for (int w = 0; w < WAYS; w++) begin
if (set_ways[w].valid && !set_ways[w].stat) begin
if (!any_candidate || (set_ways[w].age > oldest_age)) begin
oldest_age = set_ways[w].age;
oldest_way = WAY_W'(w);
any_candidate = 1'b1;
end
end
end
end
// "FRESH" means every candidate in this set was refreshed recently, so
// every one of them belongs to a station that is actively
// communicating. Evicting here displaces a working conversation.
localparam int FRESH_AGE = 10; // seconds
assign all_ways_fresh = any_candidate && (oldest_age < AGE_W'(FRESH_AGE));
always_comb begin
evict_permitted = 1'b0;
if (sel_valid && any_candidate) begin
// REFUSE UNDER ATTACK. Chapter 12.2 Section 12 established the
// asymmetry: refusing lets an attacker prevent new learning;
// evicting lets them REMOVE stations that are working. The second
// is strictly worse, so a suspected attack switches the policy.
if (attack_suspected) evict_permitted = 1'b0;
// REFUSE WHEN EVERYTHING IS FRESH. Displacing an active station to
// admit an unknown one trades a working conversation for a guess.
else if (all_ways_fresh) evict_permitted = 1'b0;
else evict_permitted = 1'b1;
end
end
assign evict_way = oldest_way;
assign evict_age = oldest_age;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
c_evictions <= '0;
c_refusals <= '0;
c_fresh_evictions <= '0;
youngest_evicted <= '1;
end else if (sel_valid) begin
if (evict_permitted) begin
if (!(&c_evictions)) c_evictions <= c_evictions + 1'b1;
// The age of what we displaced is the honest measure of the
// eviction's damage. A 250-second-old entry was probably dead
// anyway; a 3-second-old entry was a live conversation.
if (oldest_age < youngest_evicted) youngest_evicted <= oldest_age;
if (oldest_age < AGE_W'(FRESH_AGE))
if (!(&c_fresh_evictions))
c_fresh_evictions <= c_fresh_evictions + 1'b1;
end else begin
if (!(&c_refusals)) c_refusals <= c_refusals + 1'b1;
end
end
end
endmoduleClassification: synthesizable.
What it teaches: that the eviction decision has two guards and both are about the adversary Chapter 12.2 §12 introduced. attack_suspected switches the policy from evict to refuse, because refusing lets an attacker prevent new learning while evicting lets them remove stations that are already working — and the second is strictly worse. all_ways_fresh refuses for the same reason without needing to detect an attack at all: if every candidate was refreshed in the last ten seconds, all four are live conversations, and displacing one to admit an unknown address trades a certainty for a guess.
And youngest_evicted is the honest measure of damage. An entry aged 250 seconds was probably a station that has left. An entry aged 3 seconds was mid-conversation, and displacing it means every subsequent frame to that station floods until it transmits again — Chapter 12.4's price, charged to a station chosen by a hash.
Deliberately simplified: oldest-first within the set, using the ageing counter Chapter 12.2 §6 already maintains. Production designs often add a small per-way reference bit for a clock-style policy, or bias against ways whose entries have moved recently — but every one of them still chooses from the same four candidates, which is Section 12's point.
Production implication: c_fresh_evictions is a counter that should be zero in a healthy deployment and is the first thing to look at when users report intermittent unreachability with a table well below capacity. It means the structure is displacing live conversations to make room, and the remedies are more associativity, a better hash seed, or — if the arrivals are invented — Chapter 12.2 §11's rate limiter, which is upstream of this module entirely.
12. The Superlative Problem
Every replacement policy is described with a superlative — least recently used, oldest, least valuable. In a set-associative structure every one of those words is false, and the falsehood is not a rounding error.
Section 11 selects the oldest of four. The oldest entry in the table is almost certainly not among those four, and it is not reachable: the other 2047 sets are not read in this cycle, are not in the budget, and are not candidates.
So the honest statement of what the module does is:
it evicts the oldest of the four entries whose set the arriving address's hash selected.
And the arriving address is chosen by the network.
| The claim | What is actually true |
|---|---|
| "we evict the oldest entry" | the oldest of four, in a set we did not choose |
| "the least useful entry is displaced" | the least useful of four, which may all be highly useful |
| "the policy is LRU" | LRU within a set of four, which is not LRU |
| "capacity is used efficiently" | efficient within each set, and 19.5% of slots are unreachable |
The gap has a concrete consequence that Section 11's all_ways_fresh guard exists to blunt. An arriving address whose hash lands in a set of four actively communicating stations displaces one of them — and the displaced station is not the least useful station on the network, or even a below-average one. It is simply the one whose address hashed to the same 11 bits.
Which means an adversary who can choose addresses can choose the victim. Chapter 12.2 §12's attacker invented addresses to fill a table; an attacker who knows the hash can invent five addresses that land in one specific set and evict one specific station, at a cost of five frames, while the table sits at 20% occupancy. attack_suspected and all_ways_fresh are both defences against exactly this, and neither of them is a replacement policy — they are refusals to apply one.
13. RTL 6 — The Ageing Sweep
Chapter 12.2 §7 established why entries expire. This is what walking 8192 of them costs, and the answer is less than anybody budgets for.
// -----------------------------------------------------------------------
// age_sweep_engine -- walks the table incrementally, decrementing timers
// and expiring entries, while yielding to lookups and inserts.
//
// The naive implementation ages every entry on every tick, which needs
// 8192 read-modify-writes inside one clock. This one spreads the work
// across the whole interval and is therefore nearly free -- Section 14
// quantifies "nearly".
// -----------------------------------------------------------------------
module age_sweep_engine
import mactable_pkg::*;
#(
parameter int N_SETS = 2048,
parameter int WAYS = 4,
parameter int IDX_W = 11,
parameter int WAY_W = 2,
parameter int AGE_W = 9,
parameter int CNT_W = 32
)(
input logic clk,
input logic rst_n,
input logic [AGE_W-1:0] age_limit, // 300 s, Chapter 12.2
input logic second_tick,
input logic mem_grant, // the memory is ours
output logic mem_request,
output logic [IDX_W-1:0] sweep_index,
output logic [WAY_W-1:0] sweep_way,
input tentry_t sweep_entry,
output logic age_write,
output logic age_expire,
output logic [AGE_W-1:0] age_next,
output logic sweep_wrapped,
output logic [3:0] set_used_ways,
output logic sample_valid,
output logic [CNT_W-1:0] c_expired,
output logic [CNT_W-1:0] c_visited,
output logic [15:0] sweep_duty_ppm // parts per million
);
logic [IDX_W-1:0] idx_q;
logic [WAY_W-1:0] way_q;
logic pass_active_q;
logic [3:0] used_acc;
logic [CNT_W-1:0] cycles_this_second;
logic [CNT_W-1:0] busy_this_second;
// A pass begins on the second tick and runs at whatever rate the memory
// arbiter allows. It MUST complete within the second, and Section 14
// shows it completes in 33 microseconds of a 1000 millisecond budget.
assign mem_request = pass_active_q;
assign sweep_index = idx_q;
assign sweep_way = way_q;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
idx_q <= '0;
way_q <= '0;
pass_active_q <= 1'b0;
used_acc <= '0;
age_write <= 1'b0;
age_expire <= 1'b0;
age_next <= '0;
sweep_wrapped <= 1'b0;
sample_valid <= 1'b0;
set_used_ways <= '0;
c_expired <= '0;
c_visited <= '0;
cycles_this_second <= '0;
busy_this_second <= '0;
sweep_duty_ppm <= '0;
end else begin
age_write <= 1'b0;
age_expire <= 1'b0;
sweep_wrapped <= 1'b0;
sample_valid <= 1'b0;
cycles_this_second <= cycles_this_second + 1'b1;
if (pass_active_q) busy_this_second <= busy_this_second + 1'b1;
if (second_tick) begin
pass_active_q <= 1'b1;
idx_q <= '0;
way_q <= '0;
used_acc <= '0;
// Duty in parts per million, so the "ageing is expensive" claim
// can be checked rather than assumed.
sweep_duty_ppm <= (cycles_this_second == '0) ? 16'd0
: 16'((busy_this_second * CNT_W'(1_000_000)) /
cycles_this_second);
cycles_this_second <= '0;
busy_this_second <= '0;
end
// ONE ENTRY PER GRANTED CYCLE. The sweep never blocks a lookup or
// an insert -- both are on the critical path and neither can wait,
// while the sweep has a whole second.
if (pass_active_q && mem_grant) begin
if (!(&c_visited)) c_visited <= c_visited + 1'b1;
if (sweep_entry.valid) begin
used_acc <= used_acc + 4'd1;
if (!sweep_entry.stat) begin
if (sweep_entry.age >= age_limit) begin
age_write <= 1'b1;
age_expire <= 1'b1;
if (!(&c_expired)) c_expired <= c_expired + 1'b1;
end else begin
age_write <= 1'b1;
age_next <= sweep_entry.age + 1'b1;
end
end
end
// Advance. On the last way of a set, publish the occupancy sample
// Section 8 consumes -- the census is a by-product of a walk that
// was happening anyway.
if (way_q == WAY_W'(WAYS-1)) begin
set_used_ways <= used_acc + (sweep_entry.valid ? 4'd1 : 4'd0);
sample_valid <= 1'b1;
used_acc <= '0;
way_q <= '0;
if (idx_q == IDX_W'(N_SETS-1)) begin
idx_q <= '0;
pass_active_q <= 1'b0;
sweep_wrapped <= 1'b1;
end else begin
idx_q <= idx_q + 1'b1;
end
end else begin
way_q <= way_q + 1'b1;
end
end
end
end
endmoduleClassification: synthesizable.
What it teaches: that the sweep is a background walk with a one-second deadline, and it must never win against a lookup. A lookup has 28 ns and a frame waiting; the sweep has a second and 8192 entries to visit in it. The arbiter therefore gives the sweep the lowest priority unconditionally, and the sweep still finishes with five orders of magnitude to spare.
And it teaches that the census is a by-product. The sweep visits every way of every set anyway, so counting valid ways per set costs a four-bit accumulator and hands Section 8 a complete distribution histogram once per interval. A design that walks the whole table and reports only expiries has done all the work and kept a tenth of the result.
Deliberately simplified: one entry per granted cycle with a single pass per second. Production designs often walk faster and mark rather than expire, so that expiry happens in a second pass — which matters when a topology change requires the whole table's age to be recomputed at once (Chapter 12.2 §15's accelerated interval).
Production implication: sweep_duty_ppm exists to falsify a claim that gets made in design reviews. "Ageing will cost us memory bandwidth" is stated confidently and is wrong by four orders of magnitude, and Section 14 does the arithmetic. Publishing the measured duty ends the argument with a number instead of an opinion.
14. Ageing Is Nearly Free, and Everyone Budgets For It Anyway
Do the arithmetic, because the intuition that a full-table walk is expensive is strong and wrong.
The sweep visits 2048 × 4 = 8192 entries. Each visit is a read and, for a valid non-static entry, a write.
| accesses | at 500 MHz | duty over one second | |
|---|---|---|---|
| read every entry | 8192 | 16.38 µs | 0.00164% |
| read and write every entry | 16 384 | 32.77 µs | 0.00328% |
Against the forwarding traffic, which is the load that actually matters:
| Client | Accesses per second | Share of a 500 MHz port |
|---|---|---|
| lookups — Chapter 12.3 §11 | 35.71 M | 7.14% |
| inserts — Chapter 12.2 §7's ordering | 35.71 M | 7.14% |
| the ageing sweep | 16 384 | 0.0033% |
| total | 71.43 M | 14.29% |
The sweep is 0.0033 ÷ 14.29 = 0.023% of the memory's committed bandwidth — one part in four thousand of what the table is already doing.
And the total is 14.29%, which is the more important finding. Section 2 asserted that bandwidth is not the constraint; this is the arithmetic behind it. A single-ported memory at 500 MHz has 485 M spare accesses per second after serving every lookup, every insert and the sweep.
Which relocates the entire design problem onto latency, exactly where Chapter 12.3 §11 put it. The table is not short of bandwidth. It is short of time on any individual lookup, and that is a structural question — how many comparisons resolve in one access — rather than a throughput one.
15. RTL 7 — Capacity Accounting
Nominal capacity is a constant printed on a datasheet. Effective capacity is a measurement, and it is the one an operator needs.
// -----------------------------------------------------------------------
// table_capacity_accountant -- what the table is actually holding against
// what it was asked to hold.
//
// The distinction this module exists to preserve: an address that was
// never learned because its SET was full is a different event from one
// never learned because the TABLE was full, and both are different from
// one that aged out. All three end with a station being flooded to.
// -----------------------------------------------------------------------
module table_capacity_accountant
import mactable_pkg::*;
#(
parameter int N_SETS = 2048,
parameter int WAYS = 4,
parameter int CNT_W = 32,
parameter int WINDOW = 1_000_000
)(
input logic clk,
input logic rst_n,
input logic ins_valid,
input table_insert_e ins_result,
input logic lookup_valid,
input logic lookup_hit,
input logic [CNT_W-1:0] occupancy,
output logic [CNT_W-1:0] c_insert,
output logic [CNT_W-1:0] c_refresh,
output logic [CNT_W-1:0] c_move,
output logic [CNT_W-1:0] c_evict,
output logic [CNT_W-1:0] c_set_full,
output logic window_valid,
output logic [15:0] hit_rate_pct,
output logic [15:0] occupancy_pct,
output logic [15:0] effective_cap_pct, // stored / distinct offered
output logic capacity_bound, // table genuinely full
output logic hash_bound // sets full, table is not
);
logic [CNT_W-1:0] win_lk, win_hit, win_distinct;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
c_insert <= '0; c_refresh <= '0; c_move <= '0;
c_evict <= '0; c_set_full <= '0;
win_lk <= '0; win_hit <= '0; win_distinct <= '0;
window_valid <= 1'b0;
hit_rate_pct <= '0;
occupancy_pct <= '0;
effective_cap_pct <= 16'd100;
capacity_bound <= 1'b0;
hash_bound <= 1'b0;
end else begin
window_valid <= 1'b0;
if (ins_valid) begin
unique case (ins_result)
TI_INSERT: begin c_insert <= c_insert + 1'b1;
win_distinct <= win_distinct + 1'b1; end
TI_REFRESH: c_refresh <= c_refresh + 1'b1;
TI_MOVE: c_move <= c_move + 1'b1;
TI_EVICT: begin c_evict <= c_evict + 1'b1;
win_distinct <= win_distinct + 1'b1; end
// A REFUSED address is still a distinct address the table was
// offered. Counting it is what makes effective capacity
// measurable rather than theoretical.
TI_SET_FULL: begin c_set_full <= c_set_full + 1'b1;
win_distinct <= win_distinct + 1'b1; end
default: ;
endcase
end
if (lookup_valid) begin
win_lk <= win_lk + 1'b1;
if (lookup_hit) win_hit <= win_hit + 1'b1;
end
if (win_lk >= CNT_W'(WINDOW)) begin
hit_rate_pct <= 16'((win_hit * CNT_W'(100)) / win_lk);
occupancy_pct <= 16'((occupancy * CNT_W'(100)) /
CNT_W'(N_SETS * WAYS));
// Stored against distinct offered. On a four-way table offered
// 8192 distinct addresses this converges near 80.
effective_cap_pct <= (win_distinct == '0) ? 16'd100
: 16'((occupancy * CNT_W'(100)) / win_distinct);
// THE TWO CONDITIONS THAT LOOK IDENTICAL FROM OUTSIDE. Both
// produce refusals and flooding; one is fixed by more memory and
// the other is not.
capacity_bound <= (occupancy >= CNT_W'((N_SETS * WAYS * 95) / 100));
hash_bound <= (c_set_full != '0) &&
(occupancy < CNT_W'((N_SETS * WAYS * 3) / 4));
win_lk <= '0; win_hit <= '0; win_distinct <= '0;
window_valid <= 1'b1;
end
end
end
endmoduleClassification: synthesizable.
What it teaches: that a refused address must still be counted as offered. TI_SET_FULL increments win_distinct — the table was asked to hold this address and did not — and without that term effective_cap_pct would be trivially 100% by construction, since it would only ever count successes.
And capacity_bound against hash_bound is the whole point of the module. Both conditions produce refusals, both produce flooding, and from a port statistic they are indistinguishable. One means the table is genuinely full and more memory helps. The other means sets are full while the table is at 60% and more memory helps not at all — the remedies are a better hash, a different seed, or more associativity, and each is a different conversation.
Deliberately simplified: effective capacity is computed against inserts observed in one window rather than against a true count of distinct addresses ever offered. A production accountant keeps a longer-horizon estimate, because a window of a million lookups may contain very few distinct new addresses on a stable network.
Production implication: hit_rate_pct is the number an operator already watches, and it is the last one to move. A table that has started refusing shows an unchanged hit rate for as long as the refused stations are quiet — the hit rate only falls when somebody tries to reach them. hash_bound fires immediately, on the first refusal, at 60% occupancy, which is weeks before anybody notices intermittent unreachability.
16. Meeting the 28 ns Deadline
Assemble the structure and count the cycles, because Chapter 12.3 §4's deadline is what all of this was for.
| Stage | Cycles | What happens |
|---|---|---|
| hash | 1 | 48 bits to an 11-bit index — Section 5 |
| memory read | 1 | one access returns the whole 256-bit set |
| compare | 1 | four 48-bit comparators in parallel |
| select | 1 | one-hot to port and way |
| total | 4 | 8 ns at 500 MHz |
Four cycles against Chapter 12.3's 14, leaving 10 cycles of margin — which is what pays for the insert that Chapter 12.2 §7 requires to complete first, plus arbitration, plus the pipeline registers a real floorplan needs.
Compare the alternatives against the same budget:
| Structure | Cycles | Against 14 | Verdict |
|---|---|---|---|
| CAM | 2 | 7× margin | fits, and costs Section 4's power |
| 4-way hashed | 4 | 3.5× margin | fits comfortably |
| 16-way hashed | 4–5 | fits, but the set is 1024 bits and 16 comparators | fits, expensively |
| linear search, 8192 | 8192 | 585× over | does not fit by any margin |
The linear row is why Chapter 12.2 §6's store was labelled behavioural. It is a correct description of what a table does and it misses the deadline by more than two orders of magnitude — which is the gap between a specification and an implementation, made numeric.
17. RTL 8 — Conformance for a Structure With Unreachable State
The monitor's difficulty here is new: parts of the table are correct and unreachable, and a check that cannot tell those apart from corruption is worse than none.
// -----------------------------------------------------------------------
// table_conformance_monitor -- checks the structure's invariants.
//
// What it CAN check: a key appears at most once; a key is only ever in
// the set its hash names; occupancy matches the valid bits; every
// refusal had a genuinely full set; no static entry ever expired.
//
// What it CANNOT check: that the table contains the addresses it "should"
// -- Chapter 12.2 Section 16 established that the correct contents of a
// forwarding table are not a property of the design.
// -----------------------------------------------------------------------
module table_conformance_monitor
import mactable_pkg::*;
#(
parameter int N_SETS = 2048,
parameter int WAYS = 4,
parameter int IDX_W = 11,
parameter int WAY_W = 2,
parameter int CNT_W = 32
)(
input logic clk,
input logic rst_n,
// Observed from the sweep, which reads every entry once a second.
input logic sweep_valid,
input logic [IDX_W-1:0] sweep_index,
input logic [WAY_W-1:0] sweep_way,
input tentry_t sweep_entry,
input logic [IDX_W-1:0] recomputed_index, // hash of sweep_entry.key
input logic sweep_wrapped,
input logic ins_valid,
input table_insert_e ins_result,
input logic [3:0] ins_set_used_ways,
input logic evict_was_static,
input logic expire_was_static,
input logic [CNT_W-1:0] occupancy,
output logic [CNT_W-1:0] v_wrong_set, // key not in its hash set
output logic [CNT_W-1:0] v_static_evicted,
output logic [CNT_W-1:0] v_static_expired,
output logic [CNT_W-1:0] v_false_refusal, // refused a non-full set
output logic [CNT_W-1:0] v_occupancy_drift,
output logic conformant
);
logic [CNT_W-1:0] counted_valid;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
counted_valid <= '0;
v_wrong_set <= '0;
v_static_evicted <= '0;
v_static_expired <= '0;
v_false_refusal <= '0;
v_occupancy_drift <= '0;
end else begin
if (sweep_valid) begin
if (sweep_entry.valid) begin
counted_valid <= counted_valid + 1'b1;
// THE STRUCTURAL INVARIANT. A key lives in exactly the set its
// hash names. If it does not, either the hash changed under a
// live table -- a reseed without a flush -- or an insert wrote
// the wrong index, and in both cases the key is UNFINDABLE:
// every lookup for it will search a different set and miss,
// while the entry sits there consuming a way forever.
if (recomputed_index != sweep_index)
if (!(&v_wrong_set)) v_wrong_set <= v_wrong_set + 1'b1;
end
// Chapter 12.2 Section 6: an operator asserted a static entry, so
// no automatic mechanism may retract it.
if (expire_was_static)
if (!(&v_static_expired)) v_static_expired <= v_static_expired + 1'b1;
end
if (ins_valid) begin
if ((ins_result == TI_EVICT) && evict_was_static)
if (!(&v_static_evicted)) v_static_evicted <= v_static_evicted + 1'b1;
// A refusal must have a full set behind it. Refusing with a free
// way is a lost address that nothing would ever explain.
if ((ins_result == TI_SET_FULL) && (ins_set_used_ways < 4'(WAYS)))
if (!(&v_false_refusal)) v_false_refusal <= v_false_refusal + 1'b1;
end
// One complete pass is a census. The valid bits counted must equal
// the running occupancy; drift means an insert or an expiry failed
// to adjust it, and every capacity decision downstream is then made
// on a wrong number.
if (sweep_wrapped) begin
if (counted_valid != occupancy)
if (!(&v_occupancy_drift))
v_occupancy_drift <= v_occupancy_drift + 1'b1;
counted_valid <= '0;
end
end
end
assign conformant = (v_wrong_set == '0) &&
(v_static_evicted == '0) &&
(v_static_expired == '0) &&
(v_false_refusal == '0) &&
(v_occupancy_drift == '0);
endmoduleClassification: synthesizable, and intended to stay in silicon.
What it teaches: that v_wrong_set catches a failure with no other symptom whatsoever. An entry stored in the wrong set is unfindable: every lookup for that key hashes to a different set and misses, so the station is flooded to forever — while the entry sits in the table consuming a way, counted in occupancy, visible in a table dump, and looking entirely correct. The only way to detect it is to recompute the hash of the stored key and compare it against where the key actually is, which the sweep can do for free because it is reading the key anyway.
And it teaches the one operational cause of that failure, which is not a logic bug. Section 5 noted that production hashes are seeded so a deployment suffering clustering can be re-hashed in the field. Changing the seed on a live table invalidates every entry's placement at once — every key is now in the wrong set. The reseed must be accompanied by a full flush, and v_wrong_set is what catches a firmware update that forgot.
Deliberately simplified: the census compares a count against a register once per sweep. A production monitor also checks for duplicate keys across sets, which the structure should make impossible but a corrupted index bit does not.
Production implication: conformant here means the structure's invariants hold — keys are where the hash says, occupancy is honest, static entries survive, refusals were genuine. It does not mean the table has the right contents, which Chapter 12.2 §16 established is not a property of any design, and it does not mean every address that should have been learnable was learned — Section 9's 19.5% is fully conformant and entirely absent.
18. Properties Worth Asserting, and One Worth Refusing
The structural invariants are absolute. The capacity properties are measurements, not guarantees. And the replacement policy's property is where the superlative gets away from people.
The hash and the structure
// P1. A key lives in exactly the set its hash names. An entry elsewhere
// is UNFINDABLE and consumes a way forever.
property p_key_in_its_hash_set;
@(posedge clk) disable iff (!rst_n)
(sweep_valid && sweep_entry.valid) |-> (recomputed_index == sweep_index);
endproperty
a_key_placement: assert property (p_key_in_its_hash_set);
// P2. The hash is a function -- the same key always yields the same
// index. A seeded hash must therefore not change under a live table.
property p_hash_is_deterministic;
@(posedge clk) disable iff (!rst_n)
(key_valid && (key == $past(key)) && $stable(hash_seed))
|-> (index == $past(index));
endproperty
a_hash_stable: assert property (p_hash_is_deterministic);
// P3. The index is in range.
property p_index_in_range;
@(posedge clk) disable iff (!rst_n)
key_valid |-> (index < IDX_W'(N_SETS));
endproperty
a_index_range: assert property (p_index_in_range);
// P4. A lookup compares the FULL 48-bit key. A hash is not reversible, so
// unlike a cache tag nothing may be omitted.
property p_full_key_compared;
@(posedge clk) disable iff (!rst_n)
(lk_valid && lk_hit) |-> (mem[lk_index][lk_way].key == lk_key);
endproperty
a_full_key: assert property (p_full_key_compared);
// P5. A miss means the key is in no way of its set -- which, because the
// hash is deterministic, means it is not in the table at all.
property p_miss_means_absent;
@(posedge clk) disable iff (!rst_n)
(lk_valid && !lk_hit) |-> (way_match == '0);
endproperty
a_miss_complete: assert property (p_miss_means_absent);
// P6. A key appears at most once. The structure makes this automatic --
// a key has one set -- and asserting it catches an index-bit fault.
property p_key_unique;
@(posedge clk) disable iff (!rst_n)
lk_valid |-> $onehot0(way_match);
endproperty
a_key_unique: assert property (p_key_unique);Insert, refuse and evict
// P7. A free way is used before any eviction is considered.
property p_free_way_preferred;
@(posedge clk) disable iff (!rst_n)
(ins_valid && !(|ins_match) && (|ins_free)) |=> (ins_result == TI_INSERT);
endproperty
a_free_first: assert property (p_free_way_preferred);
// P8. A refusal implies the set really was full. Refusing with a way free
// is an address lost for no reason and with no explanation.
property p_refusal_implies_full_set;
@(posedge clk) disable iff (!rst_n)
(ins_valid && (ins_result == TI_SET_FULL)) |-> (ins_set_used_ways == 4'(WAYS));
endproperty
a_refusal_genuine: assert property (p_refusal_implies_full_set);
// P9. TI_SET_FULL is distinct from a table-full condition. The two have
// different remedies and only one is fixed by more memory.
property p_set_full_distinct_from_table_full;
@(posedge clk) disable iff (!rst_n)
(ins_valid && (ins_result == TI_SET_FULL) && (occupancy < CNT_W'(N_SETS*WAYS)))
|-> hash_bound;
endproperty
a_set_vs_table: assert property (p_set_full_distinct_from_table_full);
// P10. A static entry is NEVER evicted. Chapter 12.2 Section 6: an
// operator asserted it, so no automatic mechanism may retract it.
property p_static_never_evicted;
@(posedge clk) disable iff (!rst_n)
(ins_valid && (ins_result == TI_EVICT)) |-> !set_ways[evict_way].stat;
endproperty
a_static_survives_evict: assert property (p_static_never_evicted);
// P11. A static entry is NEVER expired.
property p_static_never_expired;
@(posedge clk) disable iff (!rst_n)
age_expire |-> !sweep_entry.stat;
endproperty
a_static_survives_age: assert property (p_static_never_expired);
// P12. Eviction is REFUSED while an attack is suspected. Chapter 12.2
// Section 12: refusing lets an attacker block new learning; evicting
// lets them remove stations that are working.
property p_no_evict_under_attack;
@(posedge clk) disable iff (!rst_n)
(sel_valid && attack_suspected) |-> !evict_permitted;
endproperty
a_refuse_under_attack: assert property (p_no_evict_under_attack);
// P13. Eviction is REFUSED when every candidate is fresh -- displacing a
// live conversation to admit an unknown address trades a certainty for a
// guess.
property p_no_evict_when_all_fresh;
@(posedge clk) disable iff (!rst_n)
(sel_valid && all_ways_fresh) |-> !evict_permitted;
endproperty
a_refuse_when_fresh: assert property (p_no_evict_when_all_fresh);
// P14. The chosen victim is the oldest NON-STATIC way in this set. Note
// the scope: in THIS SET. See the rejected property below.
property p_victim_is_oldest_in_set;
@(posedge clk) disable iff (!rst_n)
(sel_valid && evict_permitted) |->
(set_ways[evict_way].age == max_nonstatic_age_in_set);
endproperty
a_oldest_in_set: assert property (p_victim_is_oldest_in_set);
// P15. Every eviction records what it displaced, so the damage is
// measurable rather than inferred.
property p_eviction_records_victim;
@(posedge clk) disable iff (!rst_n)
(ins_valid && (ins_result == TI_EVICT))
|=> ((displaced_key != '0) && $changed(c_evictions));
endproperty
a_eviction_recorded: assert property (p_eviction_records_victim);Ageing
// P16. The sweep completes a full pass within its interval. It has a
// second and needs 33 microseconds -- but only if it is granted.
property p_sweep_completes_in_interval;
@(posedge clk) disable iff (!rst_n)
second_tick |-> ##[1:$] sweep_wrapped within (!second_tick[*1:$] ##1 second_tick);
endproperty
a_sweep_completes: assert property (p_sweep_completes_in_interval);
// P17. The sweep NEVER wins against a lookup or an insert. It has a
// second; they have 28 nanoseconds.
property p_sweep_lowest_priority;
@(posedge clk) disable iff (!rst_n)
(mem_request && (lk_valid || ins_valid)) |-> !mem_grant;
endproperty
a_sweep_yields: assert property (p_sweep_lowest_priority);
// P18. An entry expires only at or after the configured interval --
// Chapter 12.2's P14, restated at the structure.
property p_no_early_expiry;
@(posedge clk) disable iff (!rst_n)
age_expire |-> (sweep_entry.age >= age_limit);
endproperty
a_no_early_expiry: assert property (p_no_early_expiry);
// P19. Every visited valid entry either ages or expires -- an entry the
// sweep skips never expires at all.
property p_every_entry_aged;
@(posedge clk) disable iff (!rst_n)
(mem_grant && pass_active_q && sweep_entry.valid && !sweep_entry.stat)
|-> age_write;
endproperty
a_all_aged: assert property (p_every_entry_aged);
// P20. Expiry decrements occupancy exactly once.
property p_expiry_adjusts_occupancy;
@(posedge clk) disable iff (!rst_n)
(age_write && age_expire) |=> (occupancy == $past(occupancy) - 1);
endproperty
a_expiry_occupancy: assert property (p_expiry_adjusts_occupancy);
// P21. The census matches the running occupancy at the end of a pass.
property p_occupancy_census_agrees;
@(posedge clk) disable iff (!rst_n)
sweep_wrapped |-> (counted_valid == occupancy);
endproperty
a_occupancy_honest: assert property (p_occupancy_census_agrees);Capacity and latency
// P22. Occupancy never exceeds nominal capacity.
property p_occupancy_bounded;
@(posedge clk) disable iff (!rst_n)
(occupancy <= CNT_W'(N_SETS * WAYS));
endproperty
a_occupancy_bounded: assert property (p_occupancy_bounded);
// P23. A refused address is still counted as OFFERED, or effective
// capacity is trivially 100% by construction.
property p_refusal_counted_as_offered;
@(posedge clk) disable iff (!rst_n)
(ins_valid && (ins_result == TI_SET_FULL)) |=> $changed(win_distinct);
endproperty
a_refusal_offered: assert property (p_refusal_counted_as_offered);
// P24. The two bound conditions are distinguished, because one is fixed
// by more memory and the other is not.
property p_bounds_are_distinct;
@(posedge clk) disable iff (!rst_n)
window_valid |-> !(capacity_bound && hash_bound);
endproperty
a_bounds_distinct: assert property (p_bounds_are_distinct);
// P25. The lookup completes within Chapter 12.3's budget, every time.
property p_lookup_meets_deadline;
@(posedge clk) disable iff (!rst_n)
lk_valid |-> ##[1:4] lk_done;
endproperty
a_lookup_deadline: assert property (p_lookup_meets_deadline);
// P26. The lookup latency is CONSTANT -- it does not depend on how full
// the set is, which is what makes the deadline assertable at all.
property p_lookup_latency_constant;
@(posedge clk) disable iff (!rst_n)
lk_done |-> (lk_latency_cy == 3'd4);
endproperty
a_constant_latency: assert property (p_lookup_latency_constant);
// P27. Conformance means the structure's invariants hold -- never that
// the table has the right contents.
property p_conformant_definition;
@(posedge clk) disable iff (!rst_n)
conformant |-> ((v_wrong_set == '0) && (v_static_evicted == '0) &&
(v_static_expired == '0) && (v_false_refusal == '0) &&
(v_occupancy_drift == '0));
endproperty
a_conformant_def: assert property (p_conformant_definition);
// P28. A hash reseed is accompanied by a flush, or every key is now in
// the wrong set.
property p_reseed_implies_flush;
@(posedge clk) disable iff (!rst_n)
$changed(hash_seed) |-> ##[0:2] table_flush;
endproperty
a_reseed_flushes: assert property (p_reseed_implies_flush);19. Verification Scenarios
Fifty-eight scenarios. The structural ones have no acceptable failure; the capacity ones have expected outcomes that include refusing addresses while the table is half empty.
The hash
| # | Scenario | Expected |
|---|---|---|
| 1 | The same key twice | the same index, always |
| 2 | 8192 random 48-bit keys | index distribution within a few percent of uniform |
| 3 | 40 keys sharing an OUI, serials differing in the low 6 bits | spread across many sets, not clustered |
| 4 | Two vendors, identical 24-bit serials | different indices — the OUI must contribute |
| 5 | A hash reduced to key[10:0] | low_bits_only asserts |
| 6 | Any key | index < N_SETS |
| 7 | Hash seed changed | table_flush follows — P28 |
| 8 | Hash seed changed without a flush | v_wrong_set rises on the next sweep |
Lookup
| # | Scenario | Expected |
|---|---|---|
| 9 | Key present in way 2 of its set | hit, lk_way = 2, correct port |
| 10 | Key absent | miss — and the miss is complete, not partial |
| 11 | Key present in a different set (corrupted index) | miss, and v_wrong_set on the sweep |
| 12 | Set with 4 valid ways, key in none | miss in 4 cycles, same as a hit |
| 13 | Empty set | miss in 4 cycles |
| 14 | Any lookup | exactly 4 cycles — latency does not depend on occupancy |
| 15 | 1000 lookups at varying set occupancy | lk_latency_cy constant at 4 throughout |
| 16 | Lookup against Chapter 12.3's budget | 4 of 14 cycles, 10 cycles of margin |
Insert, refuse, evict
| # | Scenario | Expected |
|---|---|---|
| 17 | New key, set has a free way | TI_INSERT, occupancy +1 |
| 18 | Known key, same port | TI_REFRESH, occupancy unchanged |
| 19 | Known key, different port | TI_MOVE, displaced_port = old port |
| 20 | New key, set full, eviction permitted | TI_EVICT, victim recorded |
| 21 | New key, set full, eviction not permitted | TI_SET_FULL |
| 22 | TI_SET_FULL with a free way in the set | v_false_refusal — a bug |
| 23 | Set full of static entries | TI_SET_FULL — no static entry is a candidate |
| 24 | Set with 3 fresh entries and 1 aged 250 s | evicts the 250 s entry |
| 25 | Set with 4 entries all aged under 10 s | refused — all_ways_fresh |
| 26 | attack_suspected high, set full | refused, never evicted |
| 27 | Any eviction | displaced_key recorded, c_evictions increments |
| 28 | Eviction of an entry aged 3 s | c_fresh_evictions increments |
Capacity
| # | Scenario | Expected |
|---|---|---|
| 29 | 4-way table at 12.5% occupancy | 0.4 sets full — effectively none |
| 30 | 4-way at 25% | 7.5 sets full |
| 31 | 4-way at 50% | 107.8 sets full — with 4096 slots free |
| 32 | 4-way at 75% | 378.3 sets full |
| 33 | 8192 distinct addresses offered to 4-way | 6592 stored — 80.5% |
| 34 | 8192 offered to direct-mapped | 5178 stored — 63.2% |
| 35 | 8192 offered to 8-way | 7049 stored — 86.0% |
| 36 | 8192 offered to a CAM | 8192 stored — 100% |
| 37 | 16 384 offered to 4-way | 8070 stored — 98.5% |
| 38 | 32 768 offered to 4-way | 8192 — nominal, after refusing 24 576 |
| 39 | Refusals rising, occupancy 60% | hash_bound, not capacity_bound |
| 40 | Refusals rising, occupancy 97% | capacity_bound, not hash_bound |
| 41 | One rack of sequential addresses, weak hash | hash_clustering asserts |
| 42 | effective_cap_pct on a 4-way table | converges near 80 |
Ageing
| # | Scenario | Expected |
|---|---|---|
| 43 | Full sweep of 8192 entries | completes in 32.77 µs of a 1 s budget |
| 44 | Sweep duty over one second | ≈ 33 ppm — sweep_duty_ppm |
| 45 | Sweep contending with a lookup | the lookup wins, every time |
| 46 | Sweep contending with an insert | the insert wins |
| 47 | Entry at age_limit − 1 | survives |
| 48 | Entry at age_limit | expires, occupancy −1 |
| 49 | Static entry idle for an hour | never expires |
| 50 | Expiry below age_limit | v_static_expired or an early-expiry violation |
| 51 | One complete pass | sweep_wrapped, census published, histogram valid |
| 52 | Census against occupancy | equal — v_occupancy_drift = 0 |
Bandwidth and conformance
| # | Scenario | Expected |
|---|---|---|
| 53 | Lookups + inserts + sweep at line rate | 14.29% of a 500 MHz memory port |
| 54 | Sweep share of that total | 0.023% — one part in four thousand |
| 55 | Sustained line rate, 1 hour | no lookup ever misses its deadline |
| 56 | Entry written to the wrong set | v_wrong_set on the next sweep |
| 57 | Static entry evicted | v_static_evicted |
| 58 | Healthy run, one million lookups | conformant high throughout |
20. Debugging a Forwarding Table
Every row produces a switch forwarding correctly with a table reporting healthy occupancy. The third column is the observable that separates them.
| Symptom | Likely cause | The observable that decides it |
|---|---|---|
| Flooding for some stations, table 60% full | sets full while the table is not | hash_bound and c_set_full — more memory will not help |
| Flooding for some stations, table 97% full | genuinely out of capacity | capacity_bound — more memory will help |
| Refusals appeared after a firmware update | the hash seed changed without a flush | v_wrong_set on the next sweep |
| One rack of servers never learned | the hash is dominated by the OUI, or ignores it | hash_clustering, hottest_set, and Section 5's low_bits_only |
| Two specific stations alternately flooded | a direct-mapped collision oscillating | one entry for the pair; Chapter 12.2's move detector is silent |
| One host intermittently unreachable, table 20% full | a fresh eviction, possibly targeted | c_fresh_evictions and youngest_evicted |
| A station in the table but never found | the entry is in the wrong set | v_wrong_set — the entry is visible in a dump and unfindable |
| Occupancy disagrees with a table dump | an insert or expiry failed to adjust the counter | v_occupancy_drift at the next sweep wrap |
| Static entries disappearing | the sweep is not honouring stat | v_static_expired |
| Lookups occasionally over budget | the sweep is winning arbitration | sweep_duty_ppm far above 33, and P17 failing |
| Hit rate fell suddenly with no config change | entries expired en masse | Chapter 12.2 §15's port-down flush, or an accelerated interval |
| Hit rate normal, users reporting flooding | the refused stations are quiet | hit rate is the last metric to move — read c_set_full |
21. Common Misconceptions
1 — "An 8192-entry table holds 8192 addresses."
The wrong model: capacity is the memory size.
What it costs: every sizing decision is wrong by about 20%, and the error appears only in deployment. A four-way table offered 8192 distinct addresses stores 6592 — 80.5% — and the missing 1600 are not lost to a defect: their sets were full while other sets had room. A design targeting 8192 stations has provisioned for 6592.
The corrected model: capacity is a property of the hash, not the memory, because which set a key lands in is decided by the address and the address comes from the network. Effective capacity is Σ min(load, W) over sets, it depends on the associativity, and it is never on a datasheet because stating it would require stating an address distribution.
2 — "A table that is half empty will accept any address."
The wrong model: free slots mean room.
What it costs: the most confusing failure in this chapter. At 50% occupancy, 108 of 2048 sets are already full, so an address unlucky in its hash is refused while 4096 slots stand free. The station is never learned, every frame to it floods forever, and every capacity metric reads healthy.
The corrected model: the question is never "is the table full" but "is this address's set full" — and the answer depends on 11 bits of a hash of an address nobody chose. c_set_full and hash_bound exist precisely because the aggregate cannot express this.
3 — "The ageing sweep costs significant memory bandwidth."
The wrong model: walking 8192 entries is expensive, so ageing needs a budget and perhaps a dedicated port.
What it costs: design effort spent on a non-problem — a second memory port, a coarser ageing interval, a partial sweep — each of which trades away correctness for bandwidth that was never scarce.
The corrected model: the sweep is 16 384 accesses per second, which at 500 MHz is 32.77 µs — 0.0033% duty, against lookups and inserts at 14.29%. The sweep is one part in four thousand of what the table already does. Bandwidth is not the constraint; latency on an individual lookup is, which is why the structure matters and the sweep does not.
4 — "Least-recently-used replacement evicts the least recently used entry."
The wrong model: the policy's name describes what it does.
What it costs: the entire threat model. LRU is a performance policy, evaluated against hit-rate curves, and nobody threat-models a hit-rate curve. So a design named lru_policy invites reasoning that misses the five-frame targeted eviction in Section 19.
The corrected model: it evicts the oldest of four, and the four were chosen by the hash of the arriving address — an input from the network. The superlative's domain is selected by the adversary. Name the module for its scope (set_eviction_selector), assert the scope (P14), and add the guards (P12, P13) that are the actual defence.
5 — "A CAM is obviously better; it is just more expensive."
The wrong model: the CAM wins on merit and loses on price, so the hashed table is a compromise.
What it costs: the decision is made on area, which is the smaller of the two penalties. A CAM's area is 1.7×–2.7× the SRAM; its comparator activity is 2048× the four-way table's — 292 × 10⁹ match-line evaluations per second at line rate, permanently, whether or not anything matches.
The corrected model: it is a genuine trade with a currency on each side. The CAM pays power for exact capacity; the hashed table pays capacity for power — and the hashed table wins because Chapter 12.4 already priced its loss, and a bounded amount of flooding is cheaper than a permanently hot comparator array.
6 — "A miss in a hashed table might mean the key is elsewhere."
The wrong model: the hash only narrows the search, so a miss is inconclusive.
What it costs: a design that searches further on a miss — a second probe, a victim buffer, an overflow list — and blows Chapter 12.3's deadline exactly when the table is under pressure, converting a bounded flood into a variable-latency lookup that misses its budget.
The corrected model: the hash is deterministic, so a key has exactly one set. A miss in that set means the key is not in the table, conclusively, in four cycles. That determinism is what makes the constant lookup latency assertable (P26), and it is the property the whole structure is built on.
22. Interview Reasoning
Q1 — "How many MAC addresses does an 8192-entry four-way set-associative table hold?"
Reason through it. Not 8192. Offered 8192 distinct addresses it stores 6592 — 80.5% — because the addresses distribute unevenly over 2048 sets and a set that receives more than four entries refuses the rest. The stored count is Σ min(load, W) over sets, and with λ = 4 per set the shortfall is about a fifth. The strong answer goes further: it only reaches nominal capacity when offered about four times as many addresses — 32 768 offered gives 8192 stored, having refused 24 576 — so "full" is a state the table reaches by rejecting three addresses for every one it keeps. And it names the consequence: each refused address is a station that floods forever, at Chapter 12.4's price.
Q2 — "A customer reports flooding. The table is 60% full. What do you check, and what do you tell them?"
Reason through it. 60% occupancy is well inside the range where sets are already full — Section 7's table gives 5.27% of sets full at 50%, so at 60% roughly 200 of 2048 sets are refusing. The check is c_set_full and hash_bound: refusals with occupancy below three quarters means the sets are full while the table is not. The strong answer states plainly what this rules out: a larger table will not help, because the failure is distribution, not capacity. The remedies are more associativity, a different hash seed, or a better hash — and if the addresses are one rack of sequentially-numbered machines from one vendor, hash_clustering and hottest_set will say so directly.
Q3 — "Why is a CAM not used for MAC forwarding tables, given that it has exact capacity and lower latency?"
Reason through it. Area is the smaller penalty — 1.7× to 2.7×, real but survivable on a die with 24 SerDes and a packet buffer. The decisive number is comparator activity: a CAM evaluates all 8192 match lines on every search, and at 35.71 M searches per second that is 292 billion match-line evaluations per second, permanently, whether the network is busy or idle. A four-way hashed table evaluates four comparators — 2048× fewer. The strong answer frames it as a currency trade rather than a compromise: the CAM pays power for capacity, the hashed table pays capacity for power, and the hashed table wins because Chapter 12.4 already priced the flooding its missing 19.5% causes, and that price is bounded.
Q4 — "Your replacement policy is documented as LRU. What is wrong with that description, and why does it matter?"
Reason through it. It evicts the oldest of four, not the oldest in the table — the other 2047 sets are not read, are not in the four-cycle budget, and are not candidates. And the four candidates were chosen by the hash of the arriving address, which comes from the network. So the superlative's domain is selected by the input. The strong answer draws the security consequence: an attacker who knows the hash needs five frames to evict one specific victim — compute five addresses that hash to the victim's set, send them, and the fifth eviction takes the victim — while the table sits at 20% occupancy and no counter moves. LRU is a performance policy evaluated against hit-rate curves, and nobody threat-models a hit-rate curve, which is precisely why the name is dangerous. The fix is naming the scope, asserting the scope, and adding the two guards — refuse under suspected attack, refuse when every candidate is fresh.
Q5 — "Does the ageing sweep need its own memory port?"
Reason through it. No, by four orders of magnitude. The sweep is 8192 reads and up to 8192 writes per second — 32.77 µs at 500 MHz, a duty of 0.0033%. Lookups and inserts together are 71.43 M accesses per second, 14.29% of the same port. The sweep is one part in four thousand of the traffic the port already carries. The strong answer adds the design consequence: the sweep must be the lowest-priority client unconditionally — it has a full second and 8192 entries, while a lookup has 28 ns and a frame waiting — and it still finishes with five orders of magnitude of margin. It then relocates the real problem: bandwidth is not the constraint anywhere in this structure; latency on an individual lookup is, which is why the associativity and the set width matter and the sweep does not.
Q6 — "A station appears in a table dump but is never found by a lookup. Explain."
Reason through it. The entry is in the wrong set — a key lives in exactly the set its hash names, and this one does not, so every lookup hashes to a different set and misses while the entry sits there consuming a way. The entry is valid, counted in occupancy, and visible to any dump, which is why nothing else reports it. The strong answer names the operational cause: production hashes are seeded so clustering can be fixed in the field, and changing the seed on a live table invalidates every entry's placement at once. A reseed must be accompanied by a full flush (P28), and v_wrong_set — recomputing the hash of each stored key during the sweep, which is reading the key anyway — is what catches a firmware update that forgot.
23. Understanding Check
24. What's Next
This chapter and the three before it have built a complete switch: Chapter 12.1's per-frame decision, Chapter 12.2's learning, Chapter 12.3's six gates, Chapter 12.4's flooding, and this chapter's table. Every one of them assumed the whole frame had arrived.
Chapter 12.6 — Store-and-Forward against Cut-Through removes that assumption, and the consequence is sharper than Chapter 12.1 §10 could state: a cut-through switch must run all six of Chapter 12.3's gates before the frame has finished arriving, and one of the six needs the last four octets. That gate cannot be run at all, which is not a defect to fix but a property of the discipline.
Then Chapter 13.1 — Why VLANs Exist takes up Chapter 12.4 §12's unfinished argument. The only variable in (N − 1) × B a designer controls is N, and reducing it is what segmentation does — at a cost that lands squarely on this chapter's structure, because Chapter 13.4 will widen the key from 48 bits to {VID, MAC} and every number in Sections 9 and 10 will have to be recomputed.
Continue learning
Related tutorials
- Related topic
Hash-Based Distribution and Frame Ordering
The distributor's field selection, polynomial and reduction, why a modulo moves three quarters of the flows when one member fails, and the conversation that pins itself to one link.
- Related topic
The Shared-Medium Problem
Why several independent transmitters on one medium is a distributed timing problem, not a formatting problem. Propagation delay makes every station's view of the medium stale, so two locally correct decisions can still collide — and that is the constraint the Ethernet MAC was built around.
- Related topic
CSMA/CD, Collision Domains and Slot Time
Slot time is the parameter the whole half-duplex MAC hangs on: it bounds medium acquisition, bounds a collision fragment, and is the retransmission quantum. Deriving it from round-trip propagation plus jam is what fixes Ethernet's minimum frame size — a timing constant wearing a frame-format costume.
- Related topic
Packet Switching
A circuit allocates capacity in advance and guarantees it; a packet network allocates on demand and guarantees nothing. The exchange is measurable in RTL — idle reserved slots against buffered, delayed and occasionally dropped packets — and it is why a packet must describe its own extent and destination.
Standards & specifications
- Governing standard
- IEEE Std 802.3 (Ethernet)(opens IEEE in a new tab)
Defines the Ethernet MAC, the media-independent interfaces and the physical-layer sublayers, including framing, access control, auto-negotiation and per-rate PHY specifications. VLAN tagging, priority and time-sensitive shaping are defined by IEEE 802.1, not by 802.3.
This page also covers RTL structure, verification approach and debugging technique. Those are engineering practice built on the standard, not requirements the standard itself imposes.
Where this fits
Part of the Ethernet curriculum.
