CXL · Module 30
Performance Review Checklist
A working pre-tapeout performance audit. Eleven review dimensions — offered against completed, the denominator, latency origin, the tail, Little's Law as a consistency check, bandwidth-delay, head-of-line blocking, counter wrap, warm-up, instrument cost and the saturation knee — each with the defect, the arithmetic, and the telemetry that makes the number falsifiable.
30.4 closed the correctness half of this module. This chapter reviews the numbers, and it is the first review where every claim under examination is arithmetically true and can still be wrong.
The review question this chapter turns on, asked once per dimension:
What exactly was measured, at which boundary, over which interval — and what makes that number describe the system rather than the instrument?
A correctness review asks whether something can happen. A performance review asks how much, how often, and how you know, and the answers are numbers somebody chose how to produce.
1. Safety, Liveness And Performance Are Three Claims
Module 30 has been about the first two. This chapter is about the third, and collapsing them is the commonest category error in a design review.
| Claim | Says | Fails as |
|---|---|---|
| Safety | this never happens | data loss, two owners |
| Liveness | this eventually happens | a hang, a livelock |
| Performance | this happens within a cost | slow, or short of promise |
They differ in what they need before they can be checked. Safety needs nothing. Liveness needs an assumption, stated. Performance needs an assumption and a measurement, which is what makes it the one claim in this module that can be wrong about itself.
A design that misses a performance target is slow. A design that violates a safety invariant is broken. They are not the same finding, they do not get the same priority, and a review that reports them in one list has lost the distinction that decides what ships.
Performance is the only one of the three that requires an instrument, which is why this chapter spends most of its length on instruments rather than on designs. Every model here is built twice: once with an honest measurement, and once with a flattering one that measures something slightly different and reports a better number for identical behaviour.
2. How To Use This Chapter
Each of the eleven review dimensions below is a working review item, and every one answers the same eight questions:
| Facet | What it settles |
|---|---|
| Under review | the specific number being examined |
| Claim at risk | what becomes untrue if the number is wrong |
| Where it lives | the counter, the divisor, the timestamp |
| Evidence to demand | what the reviewer should ask to see |
| What escapes | the decision that gets made on a bad number |
| How DV proves it | the stimulus that separates the two readings |
| Telemetry | what makes the number checkable in the field |
| Misleading evidence | what makes the flattering number look rigorous |
3. The One-Sentence Model
A performance review is sound when a target was met, when the measurement boundary is named, when the denominator is the observation window, when the tail is published beside the mean, when the binding resource is named, and when the measured interval is stated — and "it met the target" is bit 0.
4. What This Chapter Owns
| Ground | Owner |
|---|---|
| Reviewing the architecture before RTL exists | 30.1 |
| Reviewing the RTL against the architecture | 30.2 |
| Reviewing the environment that judges the RTL | 30.3 |
| Reviewing coherency invariants across agents | 30.4 |
| Reviewing package-level integration | 30.6 |
| Reviewing a failing link | 30.7 |
| Reviewing the numbers a design publishes | this chapter |
The boundary with 30.3 is close enough to state. That chapter asks whether a checker could fail. This one asks whether a measurement could be wrong while every checker passes — and the answer is yes, routinely, because a measurement is not a check and nothing in a regression compares it against anything.
5. Teaching-Model Boundary And Source Discipline
Every model in this chapter is a teaching model. Each isolates one measurement property so it can be examined, mutated and broken on purpose. None is a production CXL controller, a performance monitor, or an implementation of any specification flow.
Nothing here states a normative CXL detail. No opcode, packet layout, bit position, field width, response encoding, snoop encoding, retry rule, timeout constant, latency figure, bandwidth number or register definition from the specification appears anywhere in this chapter. The review dimensions — offered against completed, the denominator, the tail, the binding resource — are general measurement properties that any interconnect's performance claims must satisfy, and they are examined in their general form deliberately, so the technique transfers.
| Claim class | How it is marked |
|---|---|
| General measurement reasoning | stated plainly |
| Teaching abstraction | declared in the model header |
| Illustrative parameter | every concrete figure in a model or table |
| Simulator-derived result | quoted from a run and asserted |
| Derived arithmetic | shown with its inputs |
Every rate in this chapter is in arbitrary units per hundred cycles. That is a deliberate choice: a number with no unit anybody recognises cannot be mistaken for a specification figure.
6. Review Item 1 — Offered, Accepted Or Completed?
Under review. Every counter whose value is called throughput.
Claim at risk. The rate the design delivers.
Where it lives. The counter's enable condition.
The distinction. A request is offered when the producer presents it, accepted when the consumer takes it, and completed when its response retires. Under backpressure the three diverge, and a design that publishes the first is publishing what it attempted.
// RTL 1 - offered, accepted and completed are three different numbers.
//
// "Throughput" names none of them on its own. A request is OFFERED when the
// producer presents it, ACCEPTED when the consumer takes it, and COMPLETED when
// its response retires. Under backpressure the three diverge, and a design that
// reports the first one is reporting what it TRIED to do.
//
// BAD : rate = offered / window // what was attempted
// GOOD : rate = completed / window // what was delivered
//
// TEACHING MODEL. Isolates one performance-measurement property; it is not a
// production CXL controller, and contains no opcode, packet layout, field
// width, encoding, timing guarantee or register definition from any
// specification.
module offered_vs_completed #(parameter int COUNT_OFFERED = 0) (
input logic clk, rst_n,
input logic valid, ready, resp_valid, sample,
output logic accepted,
output logic [7:0] n_offered, n_accepted, n_completed, n_window,
output logic [15:0] rate_pct,
output logic thr_err
);
logic [31:0] r_q;
logic [7:0] reported;
assign accepted = valid && ready;
// The build under review picks which number it calls throughput.
assign reported = (COUNT_OFFERED != 0) ? n_offered : n_completed;
assign r_q = (n_window == 8'd0) ? 32'd0
: (({24'd0, reported} * 32'd100) / {24'd0, n_window});
assign rate_pct = r_q[15:0];
// SAFETY-OF-EVIDENCE VIOLATION: the published rate exceeds what was actually
// delivered. A design cannot complete more than it completed.
assign thr_err = sample && (reported > n_completed);
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
n_offered <= 8'd0; n_accepted <= 8'd0; n_completed <= 8'd0; n_window <= 8'd0;
end else begin
n_window <= n_window + 8'd1;
if (valid) n_offered <= n_offered + 8'd1;
if (accepted) n_accepted <= n_accepted + 8'd1;
if (resp_valid) n_completed <= n_completed + 8'd1;
end
end
endmoduleThe measurement. One transaction presented for four cycles, taken on the fourth, retired one cycle later:
4 offers, 1 acceptance : offered=4 accepted=1 completed=1One delivered transaction and four offers. Over the five-cycle window the completion-based rate is 100 / 5 = 20 percent and the offer-based rate is 400 / 5 = 80 percent — four times the delivered work, from the same interface, in the same window.
The error is not a fixed offset. It is the stall depth, so it grows with congestion and vanishes on an unloaded interface — which the run drives explicitly: with ready tied high the two builds agree exactly, and the flattering build is correct.
Evidence to demand. Which of the three the number counts, in one word. If the answer takes a paragraph, the number counts something else.
What escapes. A capacity plan built on a rate the design has never delivered.
How DV proves it. One transaction against N stall cycles; the count must be exactly one for every N.
Telemetry. Publish all three. Their differences are the interface's efficiency, and any one of them alone is unfalsifiable.
Misleading evidence. A counter that increments exactly when valid rises — which is what a reviewer expects to see, and exactly what the defect produces on a transaction accepted immediately.
Figure 1 — one interface, one window, two rates that differ by four times. Neither counter is broken. They count different events, and only one of them counts work the system actually finished.
7. Review Item 2 — What Is The Denominator?
Under review. Every percentage, every rate, every ratio.
Claim at risk. All of them. A numerator with the wrong divisor is not approximately right.
Where it lives. The divisor, which the reader never sees.
The failure. Utilisation is busy time over some interval. Divide by the observation window and you learn how hard the block worked. Divide by the time the block was busy and the answer is a hundred percent by construction — it is not a measurement at all.
// RTL 2 - the denominator decides the answer.
//
// Utilisation is busy time over SOME interval, and which interval is a choice
// the reader never sees. Divide by the observation window and you learn how
// hard the block worked. Divide by the time the block was busy and you learn
// nothing at all - the answer is a hundred percent by construction.
//
// BAD : util = busy / busy // always 100
// GOOD : util = busy / window // busy + idle
//
// TEACHING MODEL.
module denominator_choice #(parameter int DIVIDE_BY_BUSY = 0) (
input logic clk, rst_n,
input logic active, measuring, report_now,
output logic [7:0] busy_cycles, idle_cycles, window_cycles,
output logic [15:0] util_pct, honest_pct,
output logic den_err
);
logic [31:0] u_q, h_q;
logic [7:0] denom;
// The honest denominator, computed in BOTH builds so the model can detect
// its own flattering build.
assign h_q = (window_cycles == 8'd0) ? 32'd0
: (({24'd0, busy_cycles} * 32'd100) / {24'd0, window_cycles});
assign honest_pct = h_q[15:0];
assign denom = (DIVIDE_BY_BUSY != 0) ? busy_cycles : window_cycles;
assign u_q = (denom == 8'd0) ? 32'd0
: (({24'd0, busy_cycles} * 32'd100) / {24'd0, denom});
assign util_pct = u_q[15:0];
// SAFETY-OF-EVIDENCE VIOLATION: the published figure claims more utilisation
// than the observation window can support, and idle time existed.
assign den_err = report_now && (util_pct > honest_pct) && (idle_cycles != 8'd0);
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
busy_cycles <= 8'd0; idle_cycles <= 8'd0; window_cycles <= 8'd0;
end else if (measuring) begin
window_cycles <= window_cycles + 8'd1;
if (active) busy_cycles <= busy_cycles + 8'd1;
else idle_cycles <= idle_cycles + 8'd1;
end
end
endmoduleThe measurement. Ten measured cycles, active on four:
4 busy of 10 : honest=40% window_denom=40% busy_denom=100%Both builds counted the same four busy cycles and the same six idle ones. The entire difference is the divisor, and it moves the published figure from 40 to 100.
The run also drives the case where they agree: on a fully busy window there is no idle time to divide away, and the flattering denominator is honest by accident. That is the case a reviewer is most likely to be shown.
Evidence to demand. The divisor, as a number, and what interval it covers. "Utilisation" without an interval is not a figure.
What escapes. A block believed saturated that is idle two thirds of the time, and a capacity decision made on that belief.
How DV proves it. Drive a window with known idle time and require the published figure to be below a hundred.
Telemetry. Publish busy, idle and window separately. A consumer can then compute the ratio it wants, and any inconsistency between the three is visible.
8. Review Item 3 — Latency From Where?
Under review. Every latency figure.
Claim at risk. How long the system takes, as a user experiences it.
Where it lives. The timestamp the measurement starts from.
The distinction. Total latency is arrival to completion and has two parts: the time queued waiting to be accepted, and the time in service. A design that timestamps at acceptance measures only the second — and under load the first is the whole story.
// RTL 3 - latency from WHERE? Queueing delay is not service time.
//
// A transaction's total latency is the time from ARRIVAL to COMPLETION. It has
// two parts: the time it spent queued waiting to be accepted, and the time it
// spent being served. A design that timestamps at ACCEPTANCE measures only the
// second part - and under load the first part is the whole story.
//
// BAD : latency = done - accepted // service time only
// GOOD : latency = done - arrived // queueing + service
//
// TEACHING MODEL. Sequential.
// State remembered : the arrival and acceptance timestamps of one in-flight
// transaction, and an elapsed-cycle counter.
// Safety : the reported latency is never less than the service time.
module latency_origin #(parameter int START_AT_ACCEPT = 0) (
input logic clk, rst_n,
input logic arrive, accept, done,
output logic [7:0] now_t, arrival_t, accept_t,
output logic [7:0] queued, service, total_lat, reported_lat,
output logic in_flight, lat_err
);
logic [7:0] t_q, arr_q, acc_q, rep_q, tot_q, que_q, srv_q;
logic fl_q;
assign now_t = t_q;
assign arrival_t = arr_q;
assign accept_t = acc_q;
assign queued = que_q;
assign service = srv_q;
assign total_lat = tot_q;
assign reported_lat = rep_q;
assign in_flight = fl_q;
// SAFETY-OF-EVIDENCE VIOLATION: the published latency is smaller than the
// total the transaction actually experienced, while queueing delay existed.
assign lat_err = (rep_q < tot_q) && (que_q != 8'd0);
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
t_q <= 8'd0; arr_q <= 8'd0; acc_q <= 8'd0;
rep_q <= 8'd0; tot_q <= 8'd0; que_q <= 8'd0; srv_q <= 8'd0; fl_q <= 1'b0;
end else begin
if (t_q != 8'hFF) t_q <= t_q + 8'd1;
if (arrive) begin arr_q <= t_q; fl_q <= 1'b1; end
if (accept) acc_q <= t_q;
if (done) begin
fl_q <= 1'b0;
// The truth, computed the same way in BOTH builds.
que_q <= acc_q - arr_q;
srv_q <= t_q - acc_q;
tot_q <= t_q - arr_q;
// The whole review point: which timestamp the instrument started from.
rep_q <= (START_AT_ACCEPT != 0) ? (t_q - acc_q) : (t_q - arr_q);
end
end
end
endmoduleThe measurement. Arrival, five cycles queued, acceptance, three cycles in service:
queued=5 service=3 total=8 : from_arrival=8 from_accept=3Both builds compute the same total. The arrival-based build publishes 8; the acceptance-based build publishes 3, and is not wrong about anything — it is answering a different question from the one the reader will assume.
The boundary case is where they agree: a transaction accepted in the cycle it arrives has no queueing delay, and both builds publish the service time. A directed test with an idle consumer produces exactly that case.
Evidence to demand. The two timestamps, named. If only one exists, the queueing delay is not being measured by anybody.
What escapes. A latency budget met in the lab and missed in the field, by exactly the amount of queueing the lab did not have.
How DV proves it. Hold the consumer off for N cycles and require the published latency to grow by N.
Telemetry. Publish queueing and service separately. Their ratio is the fastest diagnosis available: a rising total with flat service is a queue problem, and a rising service time is not.
9. Review Item 4 — What Does The Mean Hide?
Under review. Every average.
Claim at risk. The experience of the worst-served request.
Where it lives. The choice to publish one number for a distribution.
The mechanism. A mean is the statistic a single large outlier moves least. A run whose worst transaction took ten times its budget can have an entirely respectable average, and every user who hit that transaction saw the ten.
// RTL 4 - the mean hides the tail, and the tail is the customer's experience.
//
// A mean is one number standing in for a distribution, and it is the number a
// single enormous outlier moves least. A run whose worst transaction took ten
// times the budget can have a perfectly respectable average, and every user who
// hit that transaction saw the ten.
//
// BAD : report the mean alone
// GOOD : report the mean AND the peak, and alarm on the peak
//
// TEACHING MODEL.
// Safety : the published figure never understates the worst case observed.
module mean_hides_tail #(parameter int MEAN_ONLY = 0) (
input logic clk, rst_n,
input logic sample_valid, check_budget,
input logic [7:0] sample_lat, budget,
output logic [15:0] sum_lat, mean_lat,
output logic [7:0] peak_lat, n_samples, n_over_budget,
output logic [15:0] reported,
output logic over_budget, tail_err
);
logic [15:0] s_q;
logic [7:0] p_q, n_q, ob_q;
logic [31:0] m_q;
assign sum_lat = s_q;
assign peak_lat = p_q;
assign n_samples = n_q;
assign n_over_budget = ob_q;
assign m_q = (n_q == 8'd0) ? 32'd0 : (({16'd0, s_q} * 32'd1) / {24'd0, n_q});
assign mean_lat = m_q[15:0];
// The build under review decides which single number it publishes.
assign reported = (MEAN_ONLY != 0) ? mean_lat : {8'd0, p_q};
// The truth, computed the same way in both builds.
assign over_budget = (p_q > budget) && (budget != 8'd0);
// SAFETY-OF-EVIDENCE VIOLATION: a sample exceeded the budget and the
// published figure is inside it.
assign tail_err = check_budget && over_budget && (reported <= {8'd0, budget});
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
s_q <= 16'd0; p_q <= 8'd0; n_q <= 8'd0; ob_q <= 8'd0;
end else if (sample_valid) begin
s_q <= s_q + {8'd0, sample_lat};
n_q <= n_q + 8'd1;
if (sample_lat > p_q) p_q <= sample_lat;
if ((budget != 8'd0) && (sample_lat > budget)) ob_q <= ob_q + 8'd1;
end
end
endmoduleThe measurement. Samples of 10, 10, 10 and 90 against a budget of 50:
samples 10,10,10,90 vs budget 50 : mean=30 peak=90 over=1The mean is 30, comfortably inside the budget. The peak is 90, nearly twice it. The mean-only build publishes 30 and reports no violation; the peak-publishing build publishes 90 and reports one. Both counted the same over-budget sample.
The run drives the exact boundary too — a sample landing precisely on the budget. The bound is stated as "over budget", so equal is inside it, and that case is the only one that separates a strict comparison from an inclusive one. It was a mutation survivor before it was a test; section 21 records it.
Evidence to demand. The maximum, beside the mean, from the same run. A percentile is better and a maximum is the minimum acceptable.
What escapes. A tail nobody has looked at, which is where every complaint comes from.
Telemetry. Peak, and a count of samples over budget. The count is what distinguishes one bad transaction from a systematic problem, and the mean distinguishes neither.
10. Review Item 5 — Do The Three Numbers Agree?
Under review. Occupancy, arrival rate and latency, when all three are published.
Claim at risk. That they describe the same interval.
Where it lives. Nowhere — which is the point. This is a consistency check, not a measurement.
The relationship. For a system in steady state, the average number of items in it equals the arrival rate times the average time each spends in it:
L = lambda x WIt is not a performance target. It is a constraint relating three quantities a design already publishes, and three numbers that do not satisfy it cannot all be measurements of the same interval. Used this way it is the performance equivalent of a conservation equation: it catches an instrument, not a design.
// RTL 5 - Little's Law as a consistency check on three published numbers.
//
// For a system in steady state, the average number of items in it equals the
// arrival rate times the average time each spends in it:
//
// L = lambda * W
//
// It is not a performance target. It is a CONSTRAINT relating three quantities
// a design already publishes, and three numbers that do not satisfy it cannot
// all be measurements of the same interval. Used this way it is the performance
// equivalent of a conservation equation: it catches an INSTRUMENT, not a design.
//
// Two assumptions make it applicable, and both are published here rather than
// assumed, because assuming them is how the check gets misused:
//
// 1. STEADY STATE - arrivals balance departures across the window.
// 2. L is a TIME AVERAGE - the mean occupancy over the window, not the
// occupancy sampled at the end of it. Sampling the instantaneous value is
// the most common way this check is applied wrongly, so the model
// integrates occupancy over the window and divides.
//
// TEACHING MODEL. Rates and averages are scaled by 100 to stay in integer
// arithmetic. Nothing here is a CXL specification number.
module little_consistency #(parameter int SKIP_THE_CHECK = 0) (
input logic clk, rst_n,
input logic arrive, depart, measuring, report_now,
input logic [7:0] lat_avg,
output logic [7:0] occupancy, n_arrivals, n_departs, window_cycles,
output logic [15:0] occ_integral, little_lhs, little_rhs, lambda_x100,
output logic steady, consistent, truly_consistent, lit_err
);
logic [7:0] occ_q, arr_q, dep_q, win_q;
logic [15:0] integ_q;
logic [31:0] lam, l_x100;
assign occupancy = occ_q;
assign n_arrivals = arr_q;
assign n_departs = dep_q;
assign window_cycles = win_q;
assign occ_integral = integ_q;
// Steady state is arrivals matching departures over the window. Without it
// the law does not apply and the comparison would be meaningless.
assign steady = (arr_q == dep_q);
// L, scaled by 100: the TIME AVERAGE occupancy over the window.
assign l_x100 = (win_q == 8'd0) ? 32'd0
: (({16'd0, integ_q} * 32'd100) / {24'd0, win_q});
assign little_lhs = l_x100[15:0];
// lambda, scaled by 100: arrivals per cycle.
assign lam = (win_q == 8'd0) ? 32'd0
: (({24'd0, arr_q} * 32'd100) / {24'd0, win_q});
assign lambda_x100 = lam[15:0];
// lambda * W at the same scale.
assign little_rhs = lambda_x100 * {8'd0, lat_avg};
// Integer arithmetic makes an exact equality the wrong test; the honest test
// is agreement within one occupancy unit, which is one hundred at this scale.
// The truth is computed in BOTH builds so the model can detect its own
// unchecked build.
assign truly_consistent = (little_lhs >= little_rhs)
? ((little_lhs - little_rhs) <= 16'd100)
: ((little_rhs - little_lhs) <= 16'd100);
// The whole review point: whether this build performs the check at all.
assign consistent = (SKIP_THE_CHECK != 0) ? 1'b1 : truly_consistent;
// SAFETY-OF-EVIDENCE VIOLATION: three numbers that cannot all describe the
// same steady-state interval, published as consistent.
assign lit_err = report_now && steady && !truly_consistent && consistent;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
occ_q <= 8'd0; arr_q <= 8'd0; dep_q <= 8'd0; win_q <= 8'd0;
integ_q <= 16'd0;
end else begin
// One assignment to the occupancy counter, computed from both events.
case ({arrive, depart})
2'b10: occ_q <= occ_q + 8'd1;
2'b01: occ_q <= (occ_q == 8'd0) ? 8'd0 : occ_q - 8'd1;
default: occ_q <= occ_q;
endcase
if (measuring) begin
if (win_q != 8'hFF) win_q <= win_q + 8'd1;
integ_q <= integ_q + {8'd0, occ_q};
if (arrive) arr_q <= arr_q + 8'd1;
if (depart) dep_q <= dep_q + 8'd1;
end
end
end
endmoduleTwo assumptions, published rather than assumed
Steady state. Arrivals must balance departures across the window. The model publishes that balance, and withholds the check when it does not hold — the run drives that case explicitly, because applying the law outside its assumptions is how it gets misused.
L is a time average. It is the mean occupancy over the window, not the occupancy sampled at the end of it. Sampling the instantaneous value is the most common way this check is applied wrongly, so the model integrates occupancy across the window and divides.
The measurement. A queue primed to an occupancy of 2, then sixteen measured cycles with an arrival and a departure landing together on eight of them:
L=200 lambda=50 W=9 : lhs=200 rhs=450 checked=0 skipped=1L is exactly 2 and lambda is exactly 0.5 per cycle, so the law requires W = 4. The run asserts that at W = 4 the three agree. At a published W of 9, the right-hand side is 4.5 against a left-hand side of 2 — the three numbers cannot all describe the same interval. The checking build says so; the unchecked build reports consistent regardless.
Evidence to demand. All three numbers, and the arithmetic. If a design publishes only two of them, the third is unfalsifiable.
What escapes. Three plausible figures on one slide that contradict each other, and a plan built on whichever one was convenient.
Telemetry. Occupancy integrated over the window, arrivals, and mean latency — the three inputs, not the conclusion.
11. Review Item 6 — Is That Bandwidth Reachable?
Under review. Every bandwidth figure.
Claim at risk. The sustained rate the design can achieve.
Where it lives. The outstanding-transaction depth, which is usually in a different document from the bandwidth claim.
The relationship. Sustained bandwidth is bounded by how much work can be in flight divided by how long each piece takes to come back:
achievable = outstanding / round_tripA design with four outstanding slots on a six-hundred-cycle round trip cannot use a link of any width. Quoting the link rate as bandwidth quotes a number the block's own structure forbids it from reaching.
// RTL 6 - the link rate is not the bandwidth you get.
//
// Sustained bandwidth is bounded by how much work can be in flight at once
// divided by how long each piece takes to come back:
//
// achievable = outstanding / round_trip
//
// A design with four outstanding slots on a six-hundred-cycle round trip cannot
// use a link of any width. Quoting the link rate as "bandwidth" is quoting a
// number the block's own structure forbids it from reaching.
//
// BAD : bandwidth = link_rate
// GOOD : bandwidth = min(link_rate, outstanding / round_trip)
//
// TEACHING MODEL. Rates are in illustrative units per hundred cycles so the
// arithmetic stays integer and the relationship stays visible.
module bandwidth_delay #(parameter int QUOTE_LINK_RATE = 0) (
input logic clk, rst_n,
input logic evaluate,
input logic [7:0] link_rate, rtt_cycles, outstanding,
output logic [15:0] structural_cap, achievable, reported, needed_outstanding,
output logic cap_binds, bw_err,
output logic [7:0] n_evals, n_overclaims
);
logic [31:0] cap, need;
// What the structure permits, per hundred cycles.
assign cap = (rtt_cycles == 8'd0) ? 32'd65535
: (({24'd0, outstanding} * 32'd100) / {24'd0, rtt_cycles});
assign structural_cap = (cap > 32'd65535) ? 16'd65535 : cap[15:0];
// The truth, computed the same way in BOTH builds.
assign achievable = (structural_cap < {8'd0, link_rate}) ? structural_cap
: {8'd0, link_rate};
assign cap_binds = (structural_cap < {8'd0, link_rate});
// How many slots the link rate would actually need.
assign need = ({24'd0, link_rate} * {24'd0, rtt_cycles}) / 32'd100;
assign needed_outstanding = (need > 32'd65535) ? 16'd65535 : need[15:0];
// The whole review point.
assign reported = (QUOTE_LINK_RATE != 0) ? {8'd0, link_rate} : achievable;
// SAFETY-OF-EVIDENCE VIOLATION: a bandwidth is published that the design's
// own outstanding depth forbids.
assign bw_err = evaluate && (reported > achievable);
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
n_evals <= 8'd0; n_overclaims <= 8'd0;
end else if (evaluate) begin
n_evals <= n_evals + 8'd1;
if (reported > achievable) n_overclaims <= n_overclaims + 8'd1;
end
end
endmoduleThe measurement. Twenty outstanding slots, a fifty-cycle round trip, a link offering 80 per hundred cycles:
link=80 rtt=50 outstanding=20 : cap=40 achievable=40 quoted=80 needed=40The structure binds at 40 — half the link rate — and reaching 80 would need 40 slots, twice what the design has. The run also drives the boundary at exactly 40 slots, where the structure stops binding and the two builds agree, and the degenerate zero round trip, where the cap saturates rather than dividing by zero.
Evidence to demand. Outstanding depth and round-trip time, as two numbers, beside any bandwidth claim. The third number is then arithmetic and not an opinion.
What escapes. A link sized for a bandwidth the requester cannot request, and the cost of the extra width.
Telemetry. Outstanding-depth high-water mark. A design that never approaches its own limit is not limited by it, and one that sits at it permanently has found its bottleneck.
12. Review Item 7 — Whose Throughput Is Healthy?
Under review. Aggregate throughput and port utilisation.
Claim at risk. That the fabric is delivering what it could.
Where it lives. The choice of which quantity to publish.
The failure. One shared queue, several destinations. The entry at the head is bound for a congested destination, so it cannot move — and everything behind it waits, including entries whose own destinations are free and idle.
What makes this hard to see is that the obvious instruments all look fine. The congested port is a hundred percent utilised, because it is genuinely busy. The queue is full, which reads as healthy demand.
// RTL 7 - head-of-line blocking, and the metric that reports it as health.
//
// One shared queue, several destinations. The entry at the head is bound for a
// congested destination, so it cannot move - and everything behind it waits,
// including entries whose own destinations are free and idle. The fabric is
// delivering far less than it could.
//
// What makes this hard to see is that the obvious instruments all look FINE.
// The congested port is a hundred percent utilised, because it is genuinely
// busy. The queue is full, which reads as healthy demand. Only a metric that
// compares what COULD have been delivered against what WAS reveals the gap.
//
// BAD : report port utilisation, or queue occupancy
// GOOD : report deliverable-minus-delivered - the work that was ready and
// had a free destination and still did not move
//
// TEACHING MODEL. Sequential.
// Safety : none. Head-of-line blocking is a performance failure, not a
// correctness one - nothing is lost, everything is late.
// Liveness : every entry eventually moves - ASSUMING the head clears.
module head_of_line #(parameter int REPORT_UTILISATION = 0) (
input logic clk, rst_n,
input logic head_dest_busy, report_now,
input logic [7:0] ready_behind, stall_limit,
output logic [7:0] delivered, deliverable, blocked_work,
output logic [7:0] n_delivered, n_blocked_cycles, port_busy_cycles,
output logic [15:0] port_util_pct,
output logic reported_healthy, truly_blocked, hol_err
);
logic [7:0] nd_q, nb_q, pb_q, win_q;
logic [31:0] u_q;
// The head moves only when its destination is free.
assign delivered = head_dest_busy ? 8'd0 : 8'd1;
// What the fabric could have moved this cycle: the head, plus everything
// behind it whose own destination is idle.
assign deliverable = head_dest_busy ? ready_behind : (ready_behind + 8'd1);
// Work that is ready, has a free destination, and cannot move BECAUSE THE
// HEAD IS STUCK. Entries waiting their ordinary turn behind a moving head
// are queueing, not head-of-line blocking, and are not counted here.
assign blocked_work = head_dest_busy ? ready_behind : 8'd0;
assign n_delivered = nd_q;
assign n_blocked_cycles = nb_q;
assign port_busy_cycles = pb_q;
// The flattering instrument: the congested port's utilisation. It is high
// precisely BECAUSE the port is the bottleneck.
assign u_q = (win_q == 8'd0) ? 32'd0
: (({24'd0, pb_q} * 32'd100) / {24'd0, win_q});
assign port_util_pct = u_q[15:0];
// The truth, computed the same way in BOTH builds.
assign truly_blocked = (stall_limit != 8'd0) && (nb_q >= stall_limit);
// The whole review point: which figure the review is shown.
assign reported_healthy = (REPORT_UTILISATION != 0) ? (port_util_pct >= 16'd50)
: !truly_blocked;
// SAFETY-OF-EVIDENCE VIOLATION: ready work with free destinations has been
// stalled past the bound, and the instrument reports the block as healthy.
assign hol_err = report_now && truly_blocked && reported_healthy;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
nd_q <= 8'd0; nb_q <= 8'd0; pb_q <= 8'd0; win_q <= 8'd0;
end else begin
if (win_q != 8'hFF) win_q <= win_q + 8'd1;
if (head_dest_busy && (pb_q != 8'hFF)) pb_q <= pb_q + 8'd1;
if (!head_dest_busy && (nd_q != 8'hFF)) nd_q <= nd_q + 8'd1;
// A blocked cycle is one where work was ready behind a stalled head.
if (head_dest_busy && (ready_behind != 8'd0) && (nb_q != 8'hFF))
nb_q <= nb_q + 8'd1;
end
end
endmoduleThe measurement. The head stuck for five cycles with three ready entries behind it:
head stuck 5 cycles, 3 ready behind : delivered=0 blocked=3 util=100%Nothing moved and the port reports a hundred percent utilisation. Both builds compute the same figure; the utilisation-reporting build calls it healthy and the blocked-work build calls it a stall.
The model distinguishes blocking from queueing, which is the distinction that makes the metric useful: entries waiting their ordinary turn behind a moving head are queueing and are not counted. The run asserts that when the head clears, blocked work goes to zero while entries are still waiting.
Evidence to demand. Deliverable minus delivered — work that was ready, had a free destination, and did not move. Utilisation cannot express it.
What escapes. A fabric delivering a fraction of its capacity, with every dashboard green.
Telemetry. Blocked cycles, and blocked work per flow. The aggregate is the one number that cannot show this.
13. Review Item 8 — Did The Counter Wrap?
Under review. Every counter read at an interval.
Claim at risk. Every rate derived from a difference of two readings.
Where it lives. The counter's width against the measurement interval.
The failure. A rate is a difference between two readings divided by an interval. If the counter wraps between them, the difference is small, the rate looks low, and nothing says a wrap happened. The failure is silent and it always understates — which is the direction nobody investigates.
// RTL 8 - a counter that wraps inside the measurement window.
//
// A rate is a difference between two counter readings divided by an interval.
// If the counter wraps between the readings, the difference is small, the rate
// looks low, and nothing anywhere says a wrap happened. The failure is silent
// and it always understates - which is the direction nobody investigates.
//
// BAD : an 8-bit byte counter, free-running, read once per long window
// GOOD : saturate instead of wrapping, and publish a sticky wrap flag
//
// TEACHING MODEL.
// Safety : a published rate never understates the traffic it measured.
module counter_wrap #(parameter int WRAP_SILENTLY = 0) (
input logic clk, rst_n,
input logic tick, sample,
input logic [7:0] increment,
output logic [7:0] raw_count, reported_count,
output logic [15:0] true_total,
output logic wrapped, truly_wrapped, saturated, wrap_err,
output logic [7:0] n_wraps
);
logic [7:0] raw_q;
logic [15:0] true_q;
logic wrap_q;
logic [8:0] sum;
assign sum = {1'b0, raw_q} + {1'b0, increment};
assign raw_count = raw_q;
assign truly_wrapped = wrap_q;
// The whole review point: a silently-wrapping counter publishes no flag,
// so a reader cannot tell a low rate from a lost one.
assign wrapped = (WRAP_SILENTLY != 0) ? 1'b0 : wrap_q;
assign saturated = (raw_q == 8'hFF);
assign true_total = true_q;
// The whole review point: what the instrument publishes after a wrap.
assign reported_count = (WRAP_SILENTLY != 0) ? raw_q
: (wrap_q ? 8'hFF : raw_q);
// SAFETY-OF-EVIDENCE VIOLATION: the published count is below the true total
// and nothing reports that a wrap occurred.
assign wrap_err = sample && ({8'd0, reported_count} < true_q) && !wrapped;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
raw_q <= 8'd0; true_q <= 16'd0; wrap_q <= 1'b0; n_wraps <= 8'd0;
end else if (tick) begin
// The truth, maintained at full width in BOTH builds.
true_q <= true_q + {8'd0, increment};
if (sum[8]) begin
wrap_q <= 1'b1;
n_wraps <= n_wraps + 8'd1;
// The flattering build wraps; the honest build saturates.
raw_q <= (WRAP_SILENTLY != 0) ? sum[7:0] : 8'hFF;
end else begin
// No guard is needed here. In the saturating build `wrap_q` is set only
// by the branch above, which assigns 8'hFF in the same cycle - so
// `wrap_q` implies `raw_q == 8'hFF`, and reaching this branch from that
// state requires an increment of zero, which rewrites 255 with 255.
// A guard that can never change an outcome is dead code, and the
// mutation campaign is what proved it.
raw_q <= sum[7:0];
end
end
end
endmoduleThe measurement. Three ticks of a hundred through an eight-bit counter:
true=300 : saturating=255 flag=1 | wrapping=44 flag=0Three hundred bytes reported as forty-four. The saturating build stops at 255 and raises a flag; the wrapping build lands on 300 − 256 = 44 and publishes nothing. Both know they wrapped — the wrapping build's internal flag is set — and only one of them tells anybody.
The boundary is the case that agrees. At exactly 255 nothing has wrapped and both builds report the same number; one byte later the wrapping build reads zero — a full counter reported as empty.
A dead guard was found here and deleted, not patched: section 21 records it.
Evidence to demand. Counter width, expected rate, and measurement interval — three numbers whose product decides whether a wrap is possible. If a wrap is possible, the sticky flag is not optional.
What escapes. A traffic figure a fraction of the truth, and a link believed under-used.
Telemetry. A sticky wrap bit per counter. It costs one flop and it is the difference between a low number and an unknown one.
14. Review Item 9 — Which Interval Was Measured?
Under review. Every figure described as steady state.
Claim at risk. That the number describes the system's ongoing behaviour.
Where it lives. The cycle the measurement started on.
The failure. A measurement that starts at reset includes the interval in which the design was filling pipelines, populating caches and building outstanding depth. Averaged in, that interval drags the number away from the steady-state behaviour the reader will assume — and the shorter the run, the more it dominates.
// RTL 9 - measuring the warm-up as if it were the steady state.
//
// A measurement that starts at reset includes the interval in which the design
// was filling its pipelines, populating its caches and building up outstanding
// depth. Averaged into the result, that interval drags the number away from the
// steady-state behaviour the reader will assume it describes - and the shorter
// the run, the more it dominates.
//
// BAD : measure from reset release
// GOOD : discard a stated warm-up interval, then measure
//
// TEACHING MODEL.
module warmup_window #(parameter int MEASURE_FROM_RESET = 0) (
input logic clk, rst_n,
input logic running, report_now,
input logic [7:0] warm_limit, work_this_cycle,
output logic [7:0] total_cycles, steady_cycles, warm_cycles,
output logic [15:0] total_work, steady_work,
output logic [15:0] reported_rate, steady_rate,
output logic warm_done, warm_err
);
logic [7:0] tc_q, sc_q, wc_q;
logic [15:0] tw_q, sw_q;
logic [31:0] r_all, r_steady;
assign total_cycles = tc_q;
assign steady_cycles = sc_q;
assign warm_cycles = wc_q;
assign total_work = tw_q;
assign steady_work = sw_q;
assign warm_done = (wc_q >= warm_limit);
assign r_all = (tc_q == 8'd0) ? 32'd0
: (({16'd0, tw_q} * 32'd100) / {24'd0, tc_q});
assign r_steady = (sc_q == 8'd0) ? 32'd0
: (({16'd0, sw_q} * 32'd100) / {24'd0, sc_q});
assign steady_rate = r_steady[15:0];
// The whole review point: which interval the published rate covers.
assign reported_rate = (MEASURE_FROM_RESET != 0) ? r_all[15:0] : r_steady[15:0];
// SAFETY-OF-EVIDENCE VIOLATION: a rate published as steady state that the
// warm-up interval has measurably moved.
assign warm_err = report_now && warm_done && (reported_rate != steady_rate);
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
tc_q <= 8'd0; sc_q <= 8'd0; wc_q <= 8'd0; tw_q <= 16'd0; sw_q <= 16'd0;
end else if (running) begin
tc_q <= tc_q + 8'd1;
tw_q <= tw_q + {8'd0, work_this_cycle};
if (wc_q < warm_limit) begin
wc_q <= wc_q + 8'd1;
end else begin
sc_q <= sc_q + 8'd1;
sw_q <= sw_q + {8'd0, work_this_cycle};
end
end
end
endmoduleThe measurement. Four warm-up cycles delivering 2 each, then six steady cycles delivering 10 each:
warm 4 idle, steady 6 at 10 : steady_rate=1000 from_reset=680The steady rate is 60 over 6 cycles — 1000 per hundred. Measured from reset it is 68 over 10 — 680, thirty-two percent low, from a run that is entirely representative once it has warmed up.
A filling pipeline delivers less work, not no work, and the stimulus reflects that deliberately: with zero warm-up work the total and steady numerators are the same number and a mutation on the numerator cannot be killed. Section 21 records it.
Evidence to demand. The warm-up interval, stated, and the rate computed with and without it. A single figure with no interval is not a steady-state measurement.
What escapes. A design believed slower than it is, and time spent optimising a pipeline fill.
Telemetry. Publish the warm-up cycle count. A consumer can then discard it, which is impossible if the boundary was never recorded.
15. Review Item 10 — Did Measuring It Change It?
Under review. The instrument.
Claim at risk. That the number describes the system when nobody is watching.
Where it lives. Whatever the counter read shares with the datapath.
The failure. Reading a performance counter is not free if the read shares a port, a bus or a pipeline stage. Sampled rarely the cost disappears into the noise; sampled often enough to see a transient, the instrument becomes part of what it is measuring.
// RTL 10 - the instrument that costs what it measures.
//
// Reading a performance counter is not free if the read shares a port, a bus or
// a pipeline stage with the datapath. Sampled rarely the cost disappears into
// the noise; sampled often enough to see a transient, the instrument becomes
// part of what it is measuring, and the number it returns describes a system
// that only exists while it is being watched.
//
// BAD : sample the counter from the datapath port, as often as you like
// GOOD : shadow the counters, or publish the cycles the instrument consumed
//
// TEACHING MODEL. Sequential.
// Safety : the published throughput accounts for cycles the instrument took.
module instrument_cost #(parameter int SAMPLE_IS_FREE = 0) (
input logic clk, rst_n,
input logic work_ready, sample_req, report_now,
output logic [7:0] n_work_done, n_samples_taken, n_stolen, n_cycles,
output logic [15:0] reported_rate, true_rate,
output logic datapath_stall, perturb_err
);
logic [7:0] wd_q, st_q, sm_q, cy_q;
logic [31:0] r_rep, r_true;
assign n_work_done = wd_q;
assign n_samples_taken = sm_q;
assign n_stolen = st_q;
assign n_cycles = cy_q;
// A sample and a work item contend for the same port in the real build.
// The stall is physical and happens in both builds. Only the accounting differs.
assign datapath_stall = sample_req && work_ready;
// The truth: work delivered over every cycle the window contained.
assign r_true = (cy_q == 8'd0) ? 32'd0
: (({24'd0, wd_q} * 32'd100) / {24'd0, cy_q});
assign true_rate = r_true[15:0];
// The flattering build charges the stolen cycles to nobody, so its
// denominator is the cycles the datapath was ALLOWED to use.
assign r_rep = ((cy_q - st_q) == 8'd0) ? 32'd0
: (({24'd0, wd_q} * 32'd100) / {24'd0, (cy_q - st_q)});
assign reported_rate = (SAMPLE_IS_FREE != 0) ? r_rep[15:0] : r_true[15:0];
// SAFETY-OF-EVIDENCE VIOLATION: the instrument consumed datapath cycles and
// the published rate excludes them from its denominator.
assign perturb_err = report_now && (st_q != 8'd0) && (reported_rate > true_rate);
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
wd_q <= 8'd0; st_q <= 8'd0; sm_q <= 8'd0; cy_q <= 8'd0;
end else begin
if (cy_q != 8'hFF) cy_q <= cy_q + 8'd1;
if (sample_req) sm_q <= sm_q + 8'd1;
// A stolen cycle is one the instrument took from ready work.
if (sample_req && work_ready) st_q <= st_q + 8'd1;
// Work advances unless the instrument took the cycle - in BOTH builds.
if (work_ready && !sample_req) wd_q <= wd_q + 8'd1;
end
end
endmoduleThe measurement. Twelve cycles, work always ready, a sample every third cycle:
12 cycles, 4 samples : true=66 reported_excluding_stolen=100Four samples each steal a cycle from ready work, so eight items complete. The honest rate is 800 / 12 = 66 per hundred. Charging the stolen cycles to nobody — dividing by the eight cycles the datapath was allowed — gives 800 / 8 = 100.
The cost is physical in both builds. Both complete eight items and both count four stolen cycles. Only the denominator differs, which is section 7 arriving through a door nobody watches.
Evidence to demand. What the counter read costs, in cycles, and whether it is inside the denominator. "Negligible" is a claim; the stolen-cycle count is a measurement.
What escapes. A throughput figure that only exists while the profiler is attached, and a regression nobody can reproduce without it.
Telemetry. Publish the stolen-cycle count. An instrument that reports its own cost is one a reviewer can subtract.
16. Review Item 11 — What Saturates First?
Under review. Every peak number, and every plan to improve it.
Claim at risk. That raising the thing you are about to raise will help.
Where it lives. The comparison that decides which capacity binds.
The structure. A system has one binding constraint at a time. Below the knee, offered and delivered load track each other and latency is flat. At the knee the first resource saturates; past it, delivered load stops rising and latency grows without bound. Raising that resource does not remove the knee — it moves it.
// RTL 11 - the knee, and where the bottleneck goes when you fix it.
//
// A system has one binding constraint at a time. Below the knee, offered load
// and delivered load track each other and latency is flat. At the knee the
// first resource saturates; past it, delivered load stops rising and latency
// grows without bound. Raising that resource does not remove the knee - it
// MOVES it, to whichever resource saturates next.
//
// BAD : quote a peak number with no statement of what was binding
// GOOD : publish which resource saturated, and at what offered load
//
// TEACHING MODEL. Three illustrative resources with independent capacities.
module saturation_knee #(parameter int HIDE_THE_LIMITER = 0) (
input logic clk, rst_n,
input logic evaluate,
input logic [7:0] offered, cap_queue, cap_link, cap_tags,
output logic [7:0] delivered, knee_at, limiter, n_evals, n_saturated,
output logic saturated, limiter_known, knee_err
);
logic [7:0] min_cap, m1;
// The binding constraint is the smallest capacity.
assign m1 = (cap_queue < cap_link) ? cap_queue : cap_link;
assign min_cap = (m1 < cap_tags) ? m1 : cap_tags;
assign knee_at = min_cap;
assign delivered = (offered < min_cap) ? offered : min_cap;
assign saturated = (offered >= min_cap);
// Which resource it is: 1 queue, 2 link, 3 tags, 0 none.
assign limiter = (HIDE_THE_LIMITER != 0) ? 8'd0
: ((min_cap == cap_queue) ? 8'd1
: ((min_cap == cap_link) ? 8'd2 : 8'd3));
assign limiter_known = (limiter != 8'd0);
// SAFETY-OF-EVIDENCE VIOLATION: the design is saturated and the instrument
// cannot say which resource is binding, so no fix can be aimed.
assign knee_err = evaluate && saturated && !limiter_known;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
n_evals <= 8'd0; n_saturated <= 8'd0;
end else if (evaluate) begin
n_evals <= n_evals + 8'd1;
if (saturated) n_saturated <= n_saturated + 8'd1;
end
end
endmoduleThe measurement. Capacities of 30 queue, 80 link and 50 tags, against an offered load of 60:
offered=60 caps 30/80/50 : delivered=30 knee=30 limiter=1 hidden=0The queue binds at 30. Raise it to 100 and the run asserts what happens next: the knee moves to the tag pool at 50, and delivered load rises to 50 rather than to 60. The bottleneck migrated; it did not disappear.
Evidence to demand. The binding resource, named, at the offered load in question. A peak number with no limiter named cannot be acted on — any change is a guess with a one-in-three chance.
What escapes. An expensive widening of a resource that was never binding.
Telemetry. Publish which resource is saturated, as an enumerated value. The three capacities alone are not enough, because which one binds depends on the load.
Figure 3 — the knee moved; it did not disappear. Raising the queue from 30 to 100 bought 20 units of delivered load, not the 30 the offered load would have taken, because the tag pool was 20 away. A plan that budgets for the full 30 has budgeted for a resource that was never binding, and the only thing that would have shown it in advance is the limiter the design declines to publish.
17. The Review Assembled
Eleven dimensions, one summary — and the same trap every chapter in this module has found at its own level.
// RTL 12 - a performance review assembled. Eleven review dimensions, one
// summary. "It met the target" is bit 0: a number was produced, and one sixth
// of a review.
module perf_review_signoff #(parameter int TARGET_MET_IS_PROOF = 0) (
input logic clk, rst_n,
input logic review,
input logic target_met, boundary_named, denominator_honest,
input logic tail_published, limiter_named, window_stated,
output logic [5:0] fail_mask,
output logic [15:0] conditions_met, sound_pct,
output logic sound,
output logic [7:0] n_reviews, n_sound, n_claimed,
output logic perf_err
);
logic [31:0] s_q;
logic truly_sound, claimed;
assign fail_mask[0] = ~target_met;
assign fail_mask[1] = ~boundary_named;
assign fail_mask[2] = ~denominator_honest;
assign fail_mask[3] = ~tail_published;
assign fail_mask[4] = ~limiter_named;
assign fail_mask[5] = ~window_stated;
assign conditions_met = {15'd0, target_met} + {15'd0, boundary_named}
+ {15'd0, denominator_honest} + {15'd0, tail_published}
+ {15'd0, limiter_named} + {15'd0, window_stated};
assign s_q = ({16'd0, conditions_met} * 32'd100) / 32'd6;
// No clamp: six one-bit values over six cannot exceed a hundred.
assign sound_pct = s_q[15:0];
assign truly_sound = (fail_mask == 6'd0);
assign claimed = (TARGET_MET_IS_PROOF != 0) ? target_met : truly_sound;
assign sound = claimed;
assign perf_err = review && !truly_sound && claimed;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
n_reviews <= 8'd0; n_sound <= 8'd0; n_claimed <= 8'd0;
end else if (review) begin
n_reviews <= n_reviews + 8'd1;
if (truly_sound) n_sound <= n_sound + 8'd1;
if (claimed) n_claimed <= n_claimed + 8'd1;
end
end
endmoduleThe measurement. Two views of the same performance report:
window not stated : mask=100000 met=5 sound=83%
a number was produced, and nothing else : mask=111110 met=1 sound=16%The first line is a real review with one finding open — the measured interval was never stated. Five of six conditions met, and the report is one sentence away from being usable.
The second line is what this chapter exists to prevent. A target was met. A number was produced, it was compared against a goal, and not one of the other five conditions was established. Sixteen percent of a review, reported as a review.
18. Quantitative Reasoning
Every figure here is a teaching parameter or a value derived from one and asserted by the testbench. None is a measurement of a real system, and none is a CXL specification number. Rates are in arbitrary units per hundred cycles.
The offered-versus-completed error is the stall depth plus one. Four offers for one completion is a factor of four; the general form is offers = 1 + stalls, so the error is zero on an idle interface and unbounded on a congested one. The defect's magnitude is a function of the traffic, which is why a nominal regression never finds it.
The denominator moves 40 to 100. Four busy cycles of ten is 400 / 10 = 40 percent. The same four over a busy-time divisor is 400 / 4 = 100 percent. The numerator never changed.
Latency by origin. Five cycles queued and three in service is a total of 8 and a service time of 3 — the acceptance-based figure is 37.5 percent of the number a user experiences.
The mean against the tail. Samples of 10, 10, 10 and 90 sum to 120 over four, a mean of 30 against a peak of 90 — the peak is three times the mean and 180 percent of a budget the mean clears comfortably.
Little's Law, derived. Occupancy integrated at 2 over 16 cycles gives L = 32 / 16 = 2. Eight arrivals over 16 cycles gives lambda = 0.5 per cycle. The law then requires W = L / lambda = 4 cycles. A published W of 9 gives lambda x W = 4.5, which differs from L by 2.25 items — more than the entire occupancy, so the three cannot describe one interval.
Bandwidth-delay, derived. Twenty slots over a fifty-cycle round trip is 20 / 50 = 0.4 per cycle, or 40 per hundred cycles. Against a link offering 80, the structure binds at half the link rate. Reaching 80 needs 80 x 50 / 100 = 40 slots — double what the design has. The slots are the cheap half of that sentence, and the link width is the expensive half.
Head-of-line arithmetic. Three ready entries behind a stuck head, over five cycles, is fifteen entry-cycles of deliverable work that did not move, while the congested port reports 100 percent utilisation.
Counter wrap, derived. An eight-bit counter holds 255. Three increments of 100 total 300; the wrapping build publishes 300 − 256 = 44, which is 14.7 percent of the truth. The general rule is a decision: for a counter of width W read every T cycles at a rate R, a wrap is possible when R x T > 2^W − 1. Those three numbers are always available, so the question is always answerable before tapeout.
Warm-up, derived. Four warm cycles at 2 and six steady cycles at 10 gives 68 over 10 = 680 against a steady 60 over 6 = 1000 — the measured figure is 68 percent of the steady one. Halve the run and the distortion roughly doubles, which is why short benchmarks flatter long pipelines least.
Instrument cost, derived. Four samples in twelve cycles steal a third of the datapath's opportunities: eight items in twelve cycles is 66 per hundred, and excluding the stolen cycles gives 100 — a 51 percent overstatement produced entirely by a divisor.
The knee. Capacities of 30, 80 and 50 bind at 30. Raising the queue to 100 moves the knee to 50 — delivered load rises by 20, not by the 30 the offered load would have taken, because the next resource was 20 away.
The sign-off arithmetic. Six conditions; five met is 5 x 100 / 6 = 83 percent under integer division, and one met is 100 / 6 = 16 percent.
19. Verification Method
Order of work
compile → inspect warnings → legal baseline → reset → boundaries → simultaneous events → abuse and error cases → configuration contrasts → structural gates → PASS → mutation campaign
A mutation campaign on a failing baseline is invalid, and every campaign in this chapter ran against a green one. The baseline was re-run after every testbench modification before the campaign was re-run.
Independent oracles
| Model | Oracle |
|---|---|
| offered vs completed | 4 valid cycles, 1 accept, 1 response → 4 / 1 / 1 over a 5-cycle window |
| denominator | 4 busy of 10 → 40 percent; over busy time → 100 |
| latency origin | arrive, 5 cycles, accept, 3 cycles, done → 5 / 3 / 8 |
| mean vs tail | 10+10+10+90 = 120 over 4 → mean 30, peak 90 |
| Little's Law | occupancy 2 over 16 cycles, 8 arrivals → L = 2, lambda = 0.5, W must be 4 |
| bandwidth-delay | 20 slots / 50 cycles → 40; link 80 → cap binds; need 40 slots |
| head-of-line | head stuck, 3 behind → 0 delivered, 3 blocked, port 100 percent |
| counter wrap | 3 x 100 = 300 by hand; 300 − 256 = 44 |
| warm-up | 2x4 + 10x6 = 68 over 10 = 680; 60 over 6 = 1000 |
| instrument | 12 cycles, 4 samples, 8 done → 66; over 8 cycles → 100 |
| knee | min(30, 80, 50) = 30; raise the queue and min becomes 50 |
| sign-off | five of six → 83 percent; one of six → 16 percent |
chkv prints got against expected, which is what lets an oracle be wrong out loud. In this chapter it caught three, all mine, recorded in section 20.
X and Z rejected explicitly
chk(c, …) tests c !== 1'b1, so an X-valued condition fails rather than passing. chkv(got, exp, …) reduces the result and reports an explicit X/Z failure before comparing.
Pulses are latched, never sampled
Every evidence output — thr_err, den_err, lat_err, tail_err, lit_err, bw_err, hol_err, wrap_err, warm_err, perturb_err, knee_err, perf_err — is caught by a continuous always @(posedge clk) monitor into a sticky bit, asserted outside any conditional, in a window containing a clock edge.
Stimulus never lands on the active edge, and reset is released after it
step_clk is @(posedge clk); #1;. Reset release was moved one delta past the edge after a free-running counter exposed a race that every earlier chapter's idiom had contained without anybody noticing. Section 20 records it.
Both builds are always instantiated
Every model has both its honest and flattering instrument wired to the same stimulus. In ten of the twelve, the flattering build computes the honest figure internally and declines to publish it — the difference under review is never the information available, always which number is put on the slide.
Safety, liveness and performance kept apart
Safety — a published rate never exceeds what was delivered; a published latency never understates the total; a published count never falls below the truth without a flag. None requires an assumption.
Liveness — nothing in this chapter is a liveness claim. Head-of-line blocking is explicitly labelled a performance failure: nothing is lost, everything is late.
Performance — a 50000 limit, a budget of 50, a stall bound of 4. These are targets.
20. Baseline Defects Found Before Mutation
RTL findings — one, and the campaign found it
A dead guard in the wrapping counter. else if (!(wrap_q && (WRAP_SILENTLY == 0))) guarded the non-carry update path and can never change an outcome: in the saturating build wrap_q is set only by the carry branch, which assigns 8'hFF in the same cycle, so wrap_q implies raw_q == 8'hFF; reaching the guarded branch from that state requires an increment of zero, which rewrites 255 with 255.
Found as an unkillable mutation survivor. The correct response was neither an assertion nor a withdrawal but deleting the dead code and writing the invariant down. domcheck reported zero on this chapter: it models domination between a guard and an enclosing condition, and this domination runs between a guard and a state invariant established by a sibling branch.
Testbench defects — three
| Where | Fault |
|---|---|
| reset sequence | reset was RELEASED on the active clock edge, racing the design's own sampling of rst_n |
| m3 violation window | the violation was asserted in a window containing no clock edge, so the sticky monitor had nothing to sample |
| m10 abuse case | the checker ran after its own stimulus had been withdrawn, so it evaluated where every boolean operator agrees |
The first is the one worth keeping. Every chapter in Module 30 had used the same reset idiom and it had never mattered, because no earlier model contained a counter that free-runs from reset release. A latent testbench race is not found by passing tests; it is found by a design that happens to be sensitive to it.
Wrong oracles — three, all mine
| Expectation | Truth | Why |
|---|---|---|
| a six-cycle window | five | I counted stimulus lines, not clock edges |
| arrival stamped 1, acceptance stamped 6 | 0 and 5 | the timestamp counter is read before it steps |
| five blocked cycles after the report | six | the report window's own clock edge is a blocked cycle |
The third became prose. It is the chapter's subject met on the way past: the instrument is inside the interval it observes, and here it cost exactly one cycle of the quantity being measured.
Coverage gaps found by the structural gates
| Gate | Finding | Closed by |
|---|---|---|
outscan | 47 unasserted output nets | value assertions on every one |
displaycheck | 1 displayed value with no assertion | assertion added |
Compiler-warning findings
Under -Wall the twelve models produce no truncation, select-width, signedness, cast, multiplication-width, shift-width or loop-width warnings. The only warning is a missing timescale on models with no delay constructs — inspected and recorded as benign.
One width result was reasoned rather than trusted. little_rhs is a 16-bit by 8-bit product assigned to a 16-bit net. The tool does not warn. At the illustrative parameters the product stays under 65,536, and the model documents that as a stated bound rather than an assumption — which is 30.2 section 9's rule applied to this chapter's own arithmetic.
Simulator constraints
Icarus Verilog 13.0 rejects ref task arguments, carried forward from every chapter in this module.
21. Mutation Testing
105 mutations attempted, 105 non-equivalent, 105 killed. Zero unexplained survivors, zero equivalent mutants withdrawn.
| Reported separately | Count |
|---|---|
| Mutants attempted | 105 |
| Withdrawn as equivalent | 0 |
| Non-equivalent mutants | 105 |
| Killed | 105 |
| Unexplained survivors | 0 |
| Model | Dimension | Muts |
|---|---|---|
| m1 | offered vs completed | 9 |
| m2 | the denominator | 8 |
| m3 | latency origin | 8 |
| m4 | mean against tail | 8 |
| m5 | Little's Law | 10 |
| m6 | bandwidth-delay | 8 |
| m7 | head-of-line | 9 |
| m8 | counter wrap | 8 |
| m9 | warm-up window | 8 |
| m10 | instrument cost | 8 |
| m11 | the knee | 9 |
| m12 | review sign-off | 12 |
Eleven survivors across the two first runs, every one classified before anything was changed.
Two missing checkers on one output
rate_pct was a published output that nothing asserted anywhere, and two mutations lived in it — an inverted ratio and a degenerate-window default. outscan reported zero unasserted output nets on this chapter while that was true: it matches a port to a testbench net through the instantiation and the connection itself satisfied the match.
This is the fifth distinct way a structural tool in this track has reported a confident zero on something it could not read — after a changed assertion idiom in 29.5, a changed net name in 30.1, a changed counter suffix in 30.2, and a domination shape outside the model in 30.4. The standing conclusion is unchanged: a hit is real; a zero is unread until independently confirmed.
One vacuous checker, one statement too late
The instrument-cost abuse case drove exactly the state that separates && from || — a sample with no work ready — and then withdrew the stimulus on the line above the assertion. The check evaluated in a quiet cycle where every boolean operator agrees.
A check that runs after its own stimulus has been withdrawn is vacuous, and it looks exactly like a check that passed.
Eight boundaries the stimulus approached and never landed on
| Boundary | What was driven | What was not |
|---|---|---|
| peak against budget | 90 and 20 against 50 | exactly 50 |
| blocked cycles against the limit | 5 against 4 | exactly 4 |
| lambda's source | steady windows, arrivals equal to exits | a window where they differ |
| the tolerance comparison | gaps in one direction | the other direction |
| the wrap flag's value | the saturating build | the wrapping build's own figure |
| ticks after a wrap | none | the cycle after |
| warm-up numerators | zero work during warm-up | non-zero work |
| in-flight state | after completion | during the flight |
Every one is a value the stimulus went past rather than stopped on. The habit that prevents the whole family is cheap and specific:
For every comparison in a model, drive the value below it, above it, and exactly on it.
That single rule would have killed six of the eleven survivors before the campaign ran.
The classification rule
Never add an assertion for a survivor before classifying it.
| Class | Means, and what to do |
|---|---|
| Equivalent | no input tells the two apart — withdraw it, never count a kill |
| Stimulus gap | the case is never driven — extend the stimulus |
| Missing checker | the case is driven and nothing looks — add the checker |
| Vacuous checker | the check cannot fail — fix the check, not the design |
| Unreachable | its guard never holds — fix the guard |
| Masked | another mechanism hides it — expose it, or say why you cannot |
| Coincidental | the arithmetic happens to agree — change the stimulus |
| Missing config | the build that differs is never built — instantiate it |
| Other | anything else — state it precisely |
22. Synthesis And Implementation Reality
An honest counter costs the same as a flattering one. valid && ready instead of valid is a two-input gate on an enable. There is no area argument for the wrong one, which removes the only argument ever made for it.
Publishing three counters instead of one costs two counters. For 16-bit counts that is 32 flops on a block that already has thousands. The three numbers are what make any of them falsifiable, and that is the cheapest verification the design will ever buy.
A timestamp at arrival costs a register per in-flight entry. For an 8-bit timestamp and 64 entries that is 512 flops — genuinely not free, and the reason acceptance-based latency exists. The honest answer is to state which you measured, not to pay for what you do not need.
A peak register is one comparator and one register. A full histogram is a small RAM and an indexed increment. The peak is the cheap ninety percent of the value a distribution provides, and there is no excuse for publishing only a mean.
Little's Law needs an occupancy integrator, which is an adder and a wide accumulator running every cycle in the measurement window. At 16 bits over a 256-cycle window that is comfortably sized, and it is the only structure in this chapter that costs more than a handful of flops. It is also the only one that can catch an instrument lying about itself.
Counter width is the decision, and it is decidable in advance. A counter of width W read every T cycles at rate R wraps when R x T exceeds 2^W − 1. Widening by 8 bits costs 8 flops and multiplies the safe interval by 256. A sticky wrap bit costs one flop and converts a silent understatement into a known-unknown, which is the better trade almost every time.
A shadow counter removes the instrument's cost from the datapath at the price of duplicating the counter and a synchroniser. That is the fix for section 15, and its cost is exactly the cost of not perturbing the thing being measured.
Publishing the limiter costs an encoder over the capacity comparators that already exist to compute the minimum. The comparison is being done anyway; only the reporting is optional, and it is the part that makes the number actionable.
No area, frequency or power figures appear in this chapter, because none was measured.
23. Silicon Observability
| Telemetry | What it exposes |
|---|---|
| offered, accepted and completed, published separately | which number a rate was computed from; any inconsistency between the three |
| busy, idle and window cycles | the denominator, so a consumer can compute its own ratio |
| queueing and service latency, separately | whether a rising total is a queue or a slower service |
| peak latency and a count over budget | one bad transaction against a systematic problem |
| occupancy integrated over the window | the only input that makes Little's Law checkable in the field |
| outstanding high-water mark | whether the structure ever approached its own bandwidth cap |
| blocked cycles, and blocked work per flow | head-of-line blocking, which utilisation reports as health |
| a sticky wrap bit per counter | the difference between a low number and an unknown one |
| the warm-up cycle count | lets a consumer discard an interval the producer included |
| stolen cycles taken by the instrument | an instrument that reports its own cost |
| the binding resource, as an enumerated value | which capacity to raise, rather than which to guess |
| saturation counts on every age and rate | a design telling you it hit a limit |
Three counters here must read permanently zero — unowned overclaims, blocked-work reported as healthy, and a wrap with no flag. Each costs almost nothing and each captures a number that would otherwise be believed.
The pattern to read on the offered-accepted-completed triple: offers far above completions on a stalled interface is a healthy design in a busy fabric. The same gap with a rate computed from the first number is a different report.
A saturation counter reading zero means one of two things — the limit was never reached, or the design wraps and cannot count. Distinguishing them requires the wrap flag, which is why it is not optional.
Blocked work is the one to keep if only one survives area review. It is a subtraction between two quantities the fabric already knows, it is valid continuously, and it catches the failure every other instrument in this chapter reports as success.
24. DebugLabs
Lab 1 — Throughput is a quarter of the model and the counter agrees with the model
Symptom. A block specified at one transaction per two cycles measures one per eight in the lab. Its own transaction counter reports the specified rate.
Evidence. The counter's rate matches the specification; the observed data rate does not. The interface is visibly backpressured.
Hypothesis. The counter is not counting what the datapath is doing.
Investigation. Compare the counter against bytes moved divided by transaction size. They disagree by a factor tracking the measured backpressure.
Root cause. The counter is gated on valid alone, so every stall cycle is counted as a transaction.
Fix. Gate on valid && ready, and publish offered, accepted and completed separately.
Prevention. A review rule that reads every rate counter's enable, and a DV check driving one transaction against N stall cycles.
Silicon observability. The three counters. Their inconsistency is the signature and it is readable without any access to the design.
Lab 2 — A block is believed saturated and is idle two thirds of the time
Symptom. A capacity plan calls for a second instance of a block reporting 100 percent utilisation. A trace shows it idle most cycles.
Evidence. The utilisation figure is a hundred percent in every window, at every load.
Hypothesis. A figure that is a hundred percent at every load is not a measurement.
Investigation. Read the divisor. It is the busy-cycle count.
Root cause. Busy over busy, which is one by construction.
Fix. Divide by the observation window, and publish busy, idle and window separately.
Prevention. A ratio with no stated interval is not a figure. Ask for the divisor as a number.
Silicon observability. All three counters, so a consumer computes its own ratio.
Lab 3 — Latency meets budget in the lab and misses it in the field
Symptom. A path measured at 3 cycles in bring-up is reported at 8 by customers.
Evidence. The design's own counter reports 3 under both conditions.
Hypothesis. The counter starts somewhere the customer's experience does not.
Investigation. Instrument arrival separately from acceptance. Under load the gap is five cycles.
Root cause. The timestamp is taken at acceptance, so the figure is service time and excludes all queueing.
Fix. Timestamp at arrival, and publish queueing and service separately.
Prevention. Hold the consumer off for N cycles and require the published latency to grow by N. A bring-up test with an idle consumer produces the one case where the two agree.
Silicon observability. Queueing and service as two numbers. Their ratio diagnoses the next step immediately.
Lab 4 — Every metric is inside budget and customers report timeouts
Symptom. Average latency is 30 against a budget of 50. A fraction of a percent of requests time out.
Evidence. The design publishes a mean. There is no maximum anywhere.
Hypothesis. The distribution has a tail the mean cannot show.
Investigation. Add a peak register. It reads 90 — nearly twice the budget.
Root cause. One number standing in for a distribution, and it is the number an outlier moves least.
Fix. Publish the peak and a count of samples over budget.
Prevention. A mean alone is never sufficient for a latency claim. The maximum is the minimum acceptable addition.
Silicon observability. Peak and over-budget count. The count separates one bad transaction from a systematic problem, and the mean separates neither.
Lab 5 — Three published figures that cannot all be true
Symptom. A report gives occupancy, arrival rate and mean latency. A reviewer cannot make them agree.
Evidence. L is 2, lambda is 0.5 per cycle, and W is published as 9.
Hypothesis. The three were measured over different intervals, or one of them is an instantaneous sample.
Investigation. Apply Little's Law: lambda x W is 4.5 against an L of 2. Then check the assumptions — arrivals and departures balance, so steady state holds, and the occupancy figure turns out to have been sampled at the end of the window rather than averaged over it.
Root cause. L is a time average and the instrument reported an instantaneous value.
Fix. Integrate occupancy across the window and divide. Publish the integral.
Prevention. Run the law as a consistency check whenever all three are published. It catches the instrument, not the design.
Silicon observability. The occupancy integral, arrivals, and mean latency — the three inputs rather than the conclusion.
Lab 6 — A wider link delivers no more bandwidth
Symptom. A link is widened and sustained throughput does not move.
Evidence. The link rate doubled. Delivered bandwidth is unchanged.
Hypothesis. Something other than the link is binding.
Investigation. Compute outstanding over round trip. It equals the delivered figure exactly, before and after.
Root cause. The outstanding-transaction depth caps the achievable rate below the old link rate, so the width was never the constraint.
Fix. Size the outstanding depth to the bandwidth-delay product, and publish the cap beside any bandwidth claim.
Prevention. Never quote a link rate as a bandwidth without the two numbers that bound it.
Silicon observability. Outstanding high-water mark. A design that never approaches its own limit is not limited by it.
Lab 7 — A traffic counter reads a seventh of the truth
Symptom. A byte counter reports a rate far below what the interface is visibly carrying.
Evidence. The figure is low, stable, and plausible. No error is reported.
Hypothesis. The counter wrapped between readings.
Investigation. Compute width against interval against rate: the product exceeds the counter's range within the sampling period.
Root cause. An eight-bit counter read over an interval carrying three hundred bytes, reporting 44.
Fix. Saturate instead of wrapping, publish a sticky wrap bit, or widen the counter so a wrap is impossible in the interval.
Prevention. The three numbers that decide this are always available before tapeout. A wrap is a design decision, not an accident.
Silicon observability. The sticky wrap bit. It converts a wrong number into a known-unknown, which is the only useful thing to do with it.
Lab 8 — A benchmark improves when it is run for longer
Symptom. The same workload reports a better rate on a long run than on a short one, with no change to the design.
Evidence. The improvement scales with run length.
Hypothesis. A fixed cost at the start is being averaged over a variable interval.
Investigation. Split the measurement at a warm-up boundary. The steady rate is identical in both runs; only the included fill differs.
Root cause. The measurement starts at reset and includes the pipeline fill.
Fix. Discard a stated warm-up interval, and publish its length.
Prevention. A steady-state figure with no interval stated is not a steady-state figure. Ask which cycle the measurement started on.
Silicon observability. The warm-up cycle count, so a consumer can discard what the producer included.
25. Coverage Reasoning
Functional coverage measures what the stimulus reached. It does not measure whether a number was checked, and this chapter contains twelve defects that a fully covered environment would still publish.
Three coverage models are worth adding to any environment reviewed with this chapter:
Denominator coverage. A bin per published ratio, crossed with whether the window contained idle time. The bin a busy-denominator design can never hit is "ratio below 100 with idle time present" — and that unreachable bin is the proof.
Boundary coverage per comparison. For every comparison in the design, three bins: below, above, and exactly on. Section 21 records eight survivors that this one model would have caught before the campaign ran.
Instrument-cost coverage. A cross of sample rate against datapath occupancy. The interesting bin is "sample requested while work was ready" — the cell where the instrument and the thing measured contend, which a low sample rate never populates.
The bin the flattering design cannot hit is the most valuable bin in any model. In section 7 it is "utilisation below a hundred with idle time". In section 13 it is "wrap flag set". In section 16 it is "limiter named while saturated". Each is unreachable in the flattering build and trivial in the honest one, which makes the coverage report a direct test of the review item.
26. How This Appears In Real Engineering
The offered-versus-completed defect is the commonest performance-counter finding there is, and it survives because it is exact on an unloaded interface — which is the interface most unit benchmarks present.
The busy-denominator utilisation figure is usually an honest mistake about what "utilisation" means, made once, in a script, and then quoted for years.
Acceptance-based latency exists because arrival timestamps cost registers. It is a defensible engineering decision that becomes a defect the moment the number is published without saying which it is.
Means are published because they are one number, and one number fits on a slide. The tail is where every escalation comes from.
Little's Law is taught and almost never used as a check, because it is presented as a queueing-theory result rather than as three numbers a design already has.
Bandwidth is quoted as link rate because the link rate is on the datasheet and the outstanding depth is in an RTL parameter file that the person writing the slide has never opened.
Head-of-line blocking survives because every instrument reports it as health. The congested port really is busy and the queue really is full.
Counter wraps are silent by construction, and always understate — the direction that produces no investigation.
Warm-up is included because the measurement harness starts at reset, which is the easiest place to start it.
The instrument's cost is assumed negligible because at the sample rates used during bring-up it is, and nobody revisits the assumption when the rate goes up to chase a transient.
The limiter is not published because the comparison that computes it is internal, and exposing it feels like exposing an implementation detail rather than the one fact that makes the number actionable.
27. Common Misconceptions
"Throughput is throughput." It is one of three numbers, and they diverge exactly when the interface is interesting.
"Utilisation is a hundred percent, so we need another one." Ask for the divisor. A figure that is a hundred percent at every load is not a measurement.
"The latency counter says 3." From where? Queueing delay is the part that grows under load, and an acceptance-based counter cannot see it.
"The average is inside budget." The average is the statistic an outlier moves least. Ask for the maximum.
"Little's Law is queueing theory, not something we can use." It is three numbers the design already publishes and one multiplication. Used as a check it catches an instrument, not a design.
"The link is 80, so the bandwidth is 80." Not if the requester can only hold 20 transactions in flight over a 50-cycle round trip. The structure caps it at 40 and no link width changes that.
"The port is fully utilised, so the fabric is working hard." Or the port is the bottleneck and everything behind it is stalled. Those are the same reading.
"The counter is fine, it never reports an error." A wrapping counter cannot report one. Silence is what wrapping sounds like.
"We measured from the start of the run." Then the figure includes the pipeline fill, and it is not the steady-state number the reader will assume.
"Reading a counter is free." At the sample rate you used during bring-up. At the rate needed to see a transient, the instrument is part of the system.
"We improved the bottleneck." You moved it. Name the new one before deciding whether the change was worth it.
"The number met the target." That is bit 0, and it is worth one sixth of a review.
28. Interview And Design-Review Questions
Measurement and boundaries
1. Name the three numbers that get called throughput. Offered, accepted and completed. They diverge under backpressure, and only the third counts work the system finished.
2. A counter reports four transactions and one was delivered. What happened? It is gated on valid alone, so every stall cycle counted. The error is the stall depth plus one.
3. Why does that defect survive unit test? With ready tied high the three numbers are identical. Its magnitude is a function of backpressure, and a unit test usually has none.
4. What is wrong with a utilisation figure of a hundred percent? Possibly nothing — or the divisor is the busy time, which makes it a hundred by construction. Ask for the divisor as a number.
5. Give the two parts of total latency. Queueing delay and service time. A design that timestamps at acceptance publishes only the second.
6. Which of the two grows under load? Queueing. Which is why the lab figure and the field figure diverge exactly when it matters.
7. What does a bring-up test with an idle consumer measure? The one case where arrival-based and acceptance-based latency agree.
Distributions and relationships
8. Why is the mean the wrong statistic for a latency claim? It is the statistic a single large outlier moves least. A run with a 90 against a budget of 50 can average 30.
9. What is the minimum acceptable addition to a mean? The maximum. A percentile distribution is better; a maximum is the floor.
10. State Little's Law and say what it is for here. L equals lambda times W. It is not a target — it is a constraint relating three numbers a design already publishes, and it catches an instrument.
11. Name its two assumptions. Steady state, and that L is a time average rather than an instantaneous sample.
12. Which of the two is more often violated in practice? The time average. Sampling occupancy at the end of the window is the most common misuse.
13. L is 2, lambda is 0.5, W is published as 9. What do you conclude? That the three were not measured over the same interval. The law requires W to be 4.
14. What bounds sustained bandwidth? Outstanding transactions divided by round-trip time, and the link rate — whichever is smaller.
15. Twenty slots, a fifty-cycle round trip, a link offering 80. What is achievable? Forty. Reaching 80 would need forty slots, twice what the design has.
16. Which half of that sentence is cheap? The slots. Widening the link to a rate the requester cannot request is the expensive half.
Instruments and their failure modes
17. A port reports a hundred percent utilisation and nothing is moving. Explain. Head-of-line blocking. The congested port is genuinely busy, which is what makes it the bottleneck.
18. What metric shows it? Deliverable minus delivered — ready work with a free destination that did not move. Utilisation cannot express it.
19. Distinguish head-of-line blocking from ordinary queueing. Queueing is waiting your turn behind a moving head. Blocking is waiting behind a head that cannot move, with your own destination idle.
20. When is a counter wrap possible? When rate times interval exceeds two to the width, minus one. All three numbers are available before tapeout.
21. Which direction does a wrap error go? It always understates, which is the direction nobody investigates.
22. What does a sticky wrap bit buy? It converts a wrong number into a known-unknown, for one flop.
23. Why does a benchmark improve when run for longer? A fixed warm-up cost averaged over a longer interval. The steady rate did not change.
24. What must a steady-state figure state? The interval it covers, and the warm-up it discarded.
25. When does an instrument become part of the system? When its reads contend with the datapath at a rate high enough to matter — which is exactly the rate needed to observe a transient.
26. How do you account for that honestly? Put the stolen cycles inside the denominator, and publish the count.
27. What is the fix that removes the cost rather than accounting for it? Shadow counters, at the price of duplication and a synchroniser.
The knee and the review
28. What happens at the knee? The first resource saturates. Below it offered and delivered track and latency is flat; above it delivered stops rising and latency grows without bound.
29. What happens when you raise the binding resource? The knee moves to whichever resource saturates next. It does not disappear.
30. Capacities of 30, 80 and 50 at an offered load of 60 — what is delivered, and what binds? Thirty, and the queue. Raise the queue to 100 and 50 is delivered, bound by the tag pool.
31. Why must a peak figure name its limiter? Because otherwise any improvement is a guess with a one-in-three chance, and the two wrong choices cost money.
32. Distinguish safety, liveness and performance. Safety: this never happens, no assumptions. Liveness: this eventually happens, assumptions required. Performance: this happens within a cost, assumptions and a measurement required.
33. Which of the three needs an instrument? Only performance, which is why most of the defects in this chapter are in instruments rather than designs.
34. A design misses a performance target. Is that a bug? It is slow. A safety violation is broken. Reporting both in one list loses the distinction that decides what ships.
Campaign and method
35. What single condition makes a mutation campaign invalid? A failing baseline. Every mutation then fails for the reason the baseline does.
36. Eight survivors in this chapter were one family. Which? A boundary the stimulus approached and never landed on.
37. State the rule that kills that family. For every comparison, drive below it, above it, and exactly on it.
38. A survivor cannot be killed. What are the three possibilities? It is equivalent, the model cannot express the distinguishing case, or the code is dead. This chapter produced the third.
39. What is the right response to dead code found by a campaign? Delete it and write down the invariant that made it dead — not an assertion, and not a withdrawal.
40. A structural tool reports zero. What must you confirm? That it read its input. Five distinct times in this track a tool has reported a confident zero on something it could not parse.
41. Your checker ran and the mutation survived. Name two reasons that are not "the checker is wrong". The stimulus was withdrawn before it ran, or the state it needed was never driven.
42. Why was the reset-release race not found in four earlier chapters? No earlier model contained a counter that free-runs from reset release. A latent race is found by a design sensitive to it, not by passing tests.
43. Which telemetry in this chapter is evaluable in the field? All of it — that is the selection criterion. The occupancy integral, the three throughput counters and the wrap bits are all readable from outside the design.
44. If you could keep three counters from this chapter, which? Blocked work, the offered-accepted-completed triple, and a sticky wrap bit on every counter. The first catches the failure every other instrument calls success, the second makes a rate falsifiable, and the third turns a wrong number into a known-unknown.
45. What question would you ask first about any performance number you are shown? What was measured, at which boundary, over which interval. Three answers, and most reports do not have them.
46. What is the performance equivalent of a conservation equation? Little's Law, applied as a consistency check on three published figures rather than as a prediction.
29. Exercises
1 — Design review · Advanced. Builds: reading a published figure as an engineering claim. You are handed a block reporting "94 percent utilisation, 2.1 average latency, 12 GB/s". Bounded scope: write the three questions you would ask before accepting any of the three numbers, and say what a satisfactory answer to each looks like. Hint: each number has a hidden choice in it — a divisor, an origin, and a bound.
2 — Trade-off · Advanced. Builds: sizing a structure from a relationship rather than a figure. A link offers 200 units per hundred cycles over a 400-cycle round trip. Bounded scope: compute the outstanding depth needed to saturate it, then compute what is achievable with 64 slots, and argue whether the extra slots or a narrower link is the better spend. Hint: state the storage cost per slot before you argue.
3 — Code review · Advanced. Builds: finding a measurement defect by reading. Review a counter declared logic [15:0] bytes_moved; read once per millisecond on a link carrying up to 8 bytes per cycle at 1 GHz. Bounded scope: decide whether a wrap is possible, show the arithmetic, and state the two fixes and their costs. Hint: rate times interval against two to the width.
4 — Debug · Advanced. Builds: separating an instrument defect from a design defect. A fabric reports every port at 97 percent utilisation and delivers a third of its specified aggregate. Bounded scope: give the hypothesis, the one measurement that would confirm it, and the metric you would add. Hint: utilisation is high because the bottleneck is busy.
5 — Design · Advanced. Builds: instrumenting a claim so it can be falsified. Specify the performance telemetry for a block that must sustain a rate and meet a latency budget. Bounded scope: list every counter, say what each makes falsifiable, and identify which two you would keep if area review cut the rest. Hint: a number nothing can contradict is not evidence.
6 — Spec read · Intermediate. Builds: reading a performance claim for its assumptions. Take any published interconnect performance claim you have access to. Bounded scope: identify the boundary, the interval, and the denominator it used, and list what it does not state. Hint: the omissions are usually the interval and the tail.
7 — Debug · Advanced. Builds: applying a consistency check to published figures. A report gives occupancy 8, arrival rate 0.25 per cycle, mean latency 12 cycles. Bounded scope: apply Little's Law, state whether the three can describe one interval, and give the two most likely explanations if they cannot. Hint: check the assumptions before concluding the numbers are wrong.
8 — Design · Expert. Builds: coverage that targets an unreachable bin. Define a functional coverage model that would find the busy-denominator defect without anybody suspecting it. Bounded scope: specify the bins, the cross, and identify precisely which bin the flattering design can never hit. Hint: the proof is the unreachable cell, not the covered ones.
30. Summary
Safety, liveness and performance are three different claims, and only the third requires an instrument — which is why most performance defects are in instruments rather than designs.
Offered, accepted and completed are three numbers, and the gap between them is the stall depth.
The denominator decides the answer. Busy over busy is a hundred percent by construction and is not a measurement.
Latency measured from acceptance is service time, and queueing delay is the part that grows under load.
The mean is the statistic an outlier moves least. Publish the peak beside it, or publish nothing.
Little's Law is a consistency check on three numbers a design already has — and it catches the instrument, not the design.
Sustained bandwidth is outstanding depth over round-trip time, and the link rate is an upper bound the structure may forbid.
Head-of-line blocking is reported as health by every obvious instrument, because the congested port is genuinely busy.
A counter wrap is silent and always understates. Rate times interval against two to the width decides it before tapeout.
A measurement that starts at reset includes the pipeline fill, and the shorter the run the more it dominates.
An instrument sampled fast enough to see a transient is part of the system it measures, and its cost belongs inside the denominator.
Raising the binding resource moves the knee, it does not remove it — so name the new limiter before deciding the change was worth it.
Six conditions, and "it met the target" is one of them. A real review with one finding open is 83 percent. A number and nothing else is 16.
Continue learning
Related tutorials
- Related topic
Architecture Review Checklist
A working pre-RTL review document. Nine review dimensions — mechanism, authority, state placement, failure domain, ordering, conservation, timeout authority, backpressure and liveness — each with the invariant at risk, the evidence to demand, what escapes if it is wrong, and the telemetry that exposes it after tapeout.
- Related topic
RTL Review Checklist
A working pre-tapeout RTL review document. Nine review dimensions — handshake acceptance, transition completeness, single-driver discipline, identity lifetime, arithmetic width, recovery completeness, retry state, combinational completeness and behavioural telemetry — each with the defect, the code that produces it, what escapes, and the telemetry that exposes it in silicon.
- Related topic
Verification Review Checklist
A working review document for the verification environment itself. Nine review dimensions — oracle independence, unknown-value vacuity, checker reachability, pulse observation, transaction identity, duplicate responses, exact versus bound checking, timeout authority and fairness — each with the escape, the executable contrast, and the campaign discipline that makes a passing regression mean something.
- Related topic
Coherency Review Checklist
A working pre-tapeout coherency review. Nine review dimensions — newest-data authority, writer exclusion, dirty ownership, acknowledgement conservation, stale and duplicate acknowledgements, transient states, same-line concurrency, deadlock against livelock against starvation, and recovery reclamation — each with the invariant, the executable contrast, and the telemetry that exposes it in silicon.
Standards & specifications
- Governing standard
- CXL Specification (CXL Consortium)(opens CXL Consortium in a new tab)
Defines CXL.io, CXL.cache and CXL.mem, and the coherence and memory-pooling behaviour built on them. System design and deployment topology are not mandated.
This page also covers RTL structure, verification approach and debugging technique. Those are engineering practice built on the standard, not requirements the standard itself imposes.
Where this fits
Part of the CXL curriculum.
