Performance-Analysis Discipline

18.1 decomposed a latency. 18.2 found the binding ceiling. 18.3 added everything the operating system does. 18.4 scaled it across a fabric.

Four chapters of models, and none of them says which model to reach for, how far to trust it, or what to do with the answer. That is this chapter.

1. The Engineering Problem — A Number Is Not An Analysis

Six things separate an analysis somebody can act on from a measurement somebody took.

A model needs its workload characterised, and the inputs it is not given it will assume. The assumption does not appear in the answer. Section 5.

What a fix is worth is bounded by the share it addresses. A 2× speedup on half the work is a 1.33× overall gain, and no speedup on that half can ever exceed 2×. Section 6.

Only one input usually moves the answer, and an analysis that has not identified which one is spending its time on the others. Section 7.

A regression gate without a noise band fails on noise, and a gate that fails on noise is a gate people learn to ignore. Section 9.

A model is valid over the range it was fitted on, and answering outside that range without saying so is the most common way a good model produces a wrong decision. Section 10.

And a decision nobody recorded the basis of cannot be revisited when the inputs change — which they will. Section 14.

This chapter against the previous four, stated precisely. They own what to compute. This one owns whether the computation is worth acting on, and every model here takes another chapter's output as its input.

2. The One-Sentence Model

An analysis is actionable when it states what it assumed, what share it addresses, how far it can be trusted, and how confident it is — and every defect below is a number missing one of those four.

3. What This Chapter Owns

Deferred:

4. Teaching-Model Boundary

Four characterisation inputs, three bottleneck shares, a single noise band and integer percentages are far coarser than a real analysis. They are sized so every boundary is reachable and every result recomputable on paper.

What is not simplified is the structure: a model gated on having its inputs, a gain bounded by the share it addresses, a sensitivity ranked across candidate inputs, a regression compared against noise, a query compared against a fitted range, and a report gated on sample count and spread.

Three things are absent by design. Statistical method is out of scope beyond section 12's sample count and spread — a real analysis needs distributions, confidence intervals and a stated test, and this chapter models only the two properties whose absence is most common. There is no cost model — section 6 says what a fix is worth in speedup and nothing about what it costs to build. And multi-variable interaction is absent: section 7 ranks inputs independently, where real systems have inputs that only matter together.

5. RTL 1 — A Model Assumes What It Is Not Told

// A model needs its workload characterised; the inputs it does not have it will
// assume, and the assumption is invisible in the answer.
module characterisation #(parameter int ASSUME_DEFAULTS = 0) (
  input  logic clk, rst_n,
  input  logic       model,
  input  logic       rw_known, size_known, locality_known, concurrency_known,
  output logic       characterised,
  output logic [3:0] missing_mask,
  output logic [7:0] n_models, n_assumed,
  output logic       silent_default_err
);
  assign missing_mask[0] = ~rw_known;
  assign missing_mask[1] = ~size_known;
  assign missing_mask[2] = ~locality_known;
  assign missing_mask[3] = ~concurrency_known;
  // The assuming build fills every gap with a default and reports a model.
  assign characterised = (ASSUME_DEFAULTS != 0) ? 1'b1 : (missing_mask == 4'd0);
  // Producing a model from inputs that were never supplied.
  assign silent_default_err = model && characterised && (missing_mask != 4'd0);
  // ... model counters omitted for length
endmodule

Five modelling attempts, one with everything and four each missing one input:

  characterise: models=5 assumed=4 | assuming silent defaults=4

The four inputs are exactly the four that the previous chapters need, and each one is a chapter:

Both builds count four attempts with a missing input. n_assumed is 4 in each — the assuming build knows the input was missing and produces a model anyway. The difference is entirely in characterised, which is the flag anybody downstream would read.

6. RTL 2 — What A Fix Is Worth

// What a fix is worth is bounded by the share it addresses. Amdahl, applied to
// a bottleneck ranking.
module fix_value #(parameter int ASSUME_FULL = 0) (
  input  logic clk, rst_n,
  input  logic        evaluate,
  input  logic [7:0]  share_pct, speedup_x,
  output logic [7:0]  new_share_pct, overall_pct,
  output logic [15:0] gain_x100, ceiling_x100,
  output logic        worthwhile, overstated_err
);
  logic [31:0] ns_q, ov_q, g_q, c_q;
  logic [31:0] unfixed;
  // The part the fix does not touch.
  assign unfixed = 32'd100 - {24'd0, share_pct};
  assign ns_q = (speedup_x == 8'd0) ? {24'd0, share_pct}
                                    : ({24'd0, share_pct} / {24'd0, speedup_x});
  assign new_share_pct = ns_q[7:0];
  assign ov_q = unfixed + ns_q;
  assign overall_pct = (ov_q > 32'd255) ? 8'hFF : ov_q[7:0];
  // Overall speedup, times a hundred: the original 100 units of work over what
  // remains after the fix.
  assign g_q = (ASSUME_FULL != 0) ? ({24'd0, speedup_x} * 32'd100)
             : ((ov_q == 32'd0) ? 32'd0 : (32'd10000 / ov_q));
  assign gain_x100 = (g_q > 32'd65535) ? 16'hFFFF : g_q[15:0];
  // The ceiling: even an infinite speedup leaves the unfixed part, so no fix
  // can ever beat 100 over what it does not address.
  assign c_q = (unfixed == 32'd0) ? 32'd65535 : (32'd10000 / unfixed);
  assign ceiling_x100 = (c_q > 32'd65535) ? 16'hFFFF : c_q[15:0];
  // The bar is 1.11x rather than 1.10x because 10000/ov is never exactly 110 in
  // integer arithmetic -- ov=90 gives 111 and ov=91 gives 109 -- so a 110 bar has
  // a boundary no input can land on and no test can distinguish.
  assign worthwhile = (gain_x100 > 16'd111);
  // Claiming a gain the addressed share cannot deliver even at infinite speedup.
  assign overstated_err = evaluate && (gain_x100 > ceiling_x100);
endmodule

  fix: share=50% speedup=2x gain=1.33x ceiling=2.00x | assuming overstated=3

A 2× speedup on half the work is a 1.33× overall gain, and the assuming build reports 2.00×. The gap is not an approximation — it is the half of the work the fix does not touch, which does not get faster no matter what.

The ceiling_x100 output is the part most analyses lack. A fix addressing 10% of the cost cannot deliver more than 1.11× however good it is — and knowing that before the work starts is the difference between a project and a wasted quarter.

7. RTL 3 — Which Input Moves The Answer

// Sensitivity: which input actually moves the answer, and which one an analysis
// has been spending its time on.
module sensitivity (
  input  logic clk, rst_n,
  input  logic        analyse,
  input  logic [15:0] base_ns,
  input  logic [15:0] d_latency, d_bandwidth, d_concurrency, d_placement,
  output logic [15:0] worst_delta,
  output logic [1:0]  most_sensitive,
  output logic [7:0]  worst_pct,
  output logic        flat, no_lever_err
);
  // ... selection omitted for length
  // A model where nothing moves the answer has no lever, which means the model
  // is missing the input that matters rather than the system being insensitive.
  assign flat = (worst_delta == 16'd0);
  assign no_lever_err = analyse && flat && (base_ns != 16'd0);
endmodule

  sensitivity: worst=400ns input=0 pct=40%

Four candidate inputs against a 1000 ns base, and the sensitive one is driven to three different positions so the answer cannot be a fixed index:

The fourth row is the model's most useful output. A system where no modelled input moves the answer does not have an insensitive system — it has an incomplete model. Something is driving the cost and it is not among the four things being varied, which is a stronger and more actionable conclusion than "we found no lever".

no_lever_err requires a non-zero base, because a model of nothing is flat for the trivial reason.

8. Waveform — A Fix's Value Falling With Its Share

Transcribed from the printed trace. One stimulus stream, both builds.

9. RTL 4 — A Gate Needs A Noise Band

// A performance gate: what a CI check must do to catch a regression without
// failing on noise.
module regression_gate #(parameter int NO_NOISE_BAND = 0) (
  input  logic clk, rst_n,
  input  logic        run,
  input  logic [15:0] baseline_ns, measured_ns, noise_ns,
  output logic [15:0] delta_ns, threshold_ns,
  output logic        regressed, flaky,
  output logic [7:0]  n_runs, n_failed, n_flaky,
  output logic        noise_fail_err
);
  // A regression must exceed the measurement noise before it is a regression.
  assign delta_ns     = (measured_ns > baseline_ns) ? (measured_ns - baseline_ns) : 16'd0;
  assign threshold_ns = (NO_NOISE_BAND != 0) ? 16'd0 : noise_ns;
  assign regressed    = (delta_ns > threshold_ns);
  // A delta inside the noise band that the gate called a regression.
  assign flaky        = regressed && (delta_ns <= noise_ns);
  assign noise_fail_err = run && flaky;
  // ... run counters omitted for length
endmodule

A 1000 ns baseline with 50 ns of measurement noise:

  gate: runs=4 failed=1 | no-band failed=3 flaky=2

The no-band gate failed three of four runs and two of those were noise. A gate with that failure rate is a gate whose failures get re-run until they pass, which is the same as not having one.

The improvement case matters: delta_ns is guarded so a faster run produces a delta of zero rather than an unsigned wrap. Without the guard, every improvement is the largest regression the gate has ever seen.

The repeat from SUSPECT is the mechanism a noise band makes affordable. A gate that fails 60% of runs cannot afford to re-run every failure; a gate that fails rarely can, and the repeat is what separates a real regression from a slow machine. Removing the band removes the SUSPECT state, and with it the only cheap way to tell the two apart.

10. RTL 5 — A Model Is Valid Where It Was Fitted

// A model is valid over the range it was fitted on and no further.
module extrapolation #(parameter int UNBOUNDED = 0) (
  input  logic clk, rst_n,
  input  logic        query,
  input  logic [15:0] fit_lo, fit_hi, at_point,
  output logic        in_range, trustworthy,
  output logic [15:0] distance,
  output logic [7:0]  overreach_pct, n_query, n_extrap,
  output logic        extrapolation_err
);
  logic [31:0] op_q;
  logic [15:0] span;
  assign span     = fit_hi - fit_lo;
  assign in_range = (at_point >= fit_lo) && (at_point <= fit_hi);
  assign distance = in_range ? 16'd0
                  : ((at_point > fit_hi) ? (at_point - fit_hi) : (fit_lo - at_point));
  // How far outside the fitted span the query sits, as a share of that span.
  assign op_q = (span == 16'd0) ? 32'd0
              : (({16'd0, distance} * 32'd100) / {16'd0, span});
  assign overreach_pct = (op_q > 32'd255) ? 8'hFF : op_q[7:0];
  // The unbounded build answers any query as though it were fitted there.
  assign trustworthy = (UNBOUNDED != 0) ? 1'b1 : in_range;
  // Answering outside the fitted range without saying so.
  assign extrapolation_err = query && trustworthy && !in_range;
  // ... query counters omitted for length
endmodule

A model fitted between 10 and 50:

  extrapolate: queries=6 outside=3 overreach=0% | unbounded extrapolated=3

Both edges are inside. A model fitted from 10 to 50 is valid at 10 and at 50; excluding the endpoints throws away the two points the fit is most confident about.

overreach_pct is the share of the fitted span, which makes it comparable across models. A query 40 beyond the top of a 40-wide fit is 100% overreach — the model is being asked about a region as large as the one it knows.

11. RTL 6 — A Number Needs A Sample Count

// A measurement without a sample count and a spread is a number, not a result.
module confidence #(parameter int SINGLE_SAMPLE = 0) (
  input  logic clk, rst_n,
  input  logic        report,
  input  logic [15:0] samples, spread_ns, mean_ns,
  output logic [7:0]  spread_pct,
  output logic        enough_samples, tight, reportable,
  output logic [7:0]  n_reports, n_weak,
  output logic        single_sample_err
);
  // ... spread share omitted for length
  assign enough_samples = (samples >= 16'd10);
  assign tight          = (spread_pct <= 8'd10);
  // The single-sample build reports anything it was handed.
  assign reportable = (SINGLE_SAMPLE != 0) ? 1'b1 : (enough_samples && tight);
  // Reporting a result from one measurement.
  assign single_sample_err = report && reportable && !enough_samples;
  // ... report counters omitted for length
endmodule

  confidence: reports=7 weak=4 spread=6% | single-sample claims=2

Seven reports, four of them weak — and they are weak for two different reasons:

Both conditions are necessary and neither is sufficient. Fifty samples with a 30% spread is not a result, and one sample with a 6% spread is not a spread at all. The two mutations that drop either term individually are killed by different rows of that table.

single_sample_err fires only on the sample-count failure, not on the spread failure. A loose result is a weak result honestly reported; a one-sample result presented as a measurement is a different claim.

12. RTL 7 — Which Regime Is The Workload In

// Roofline: whether a workload is limited by bandwidth or by latency, and which
// of the previous three chapters therefore applies.
module roofline #(parameter int ASSUME_BW = 0) (
  input  logic clk, rst_n,
  input  logic        classify,
  input  logic [15:0] bytes_per_op, ops_per_sec_k,
  input  logic [15:0] bw_ceiling_gbps, latency_ns, outstanding,
  output logic [31:0] demand_gbps, latency_bound_gbps,
  output logic [1:0]  regime,
  output logic        bw_bound, lat_bound, misclassified_err
);
  logic [31:0] lb_q;
  assign demand_gbps = ({16'd0, bytes_per_op} * {16'd0, ops_per_sec_k} * 32'd8) / 32'd1000;
  // What the outstanding count can sustain at this latency, per 18.2 section 7.
  assign lb_q = (latency_ns == 16'd0) ? 32'd0
              : (({16'd0, outstanding} * {16'd0, bytes_per_op} * 32'd8000)
                 / {16'd0, latency_ns});
  assign latency_bound_gbps = lb_q;
  // Whichever ceiling is lower is the binding regime.
  assign bw_bound  = ({16'd0, bw_ceiling_gbps} <= latency_bound_gbps);
  assign lat_bound = (latency_bound_gbps < {16'd0, bw_ceiling_gbps});
  assign regime = (ASSUME_BW != 0) ? 2'd0 : (bw_bound ? 2'd0 : 2'd1);
  // Calling a workload bandwidth bound when concurrency is the lower ceiling.
  assign misclassified_err = classify && (regime == 2'd0) && lat_bound;
endmodule

  roofline: demand=512 lat_bound=65536 regime=0 | assuming misclassified=1

The regime is the answer to "which chapter applies". A bandwidth-bound workload is 18.2's problem; a latency-bound one is 18.1's, and optimising the wrong one is work on a ceiling that is not binding.

The tie is resolved toward bandwidth, and driven exactly: one outstanding request at 1280 ns sustains precisely the 400 Gbps ceiling. Both bw_bound and lat_bound are asserted at that point, because a model where a tie satisfies both or neither has an ambiguous regime.

13. RTL 8 — Rank The Bottlenecks Completely

// Ranking bottlenecks: the order to work in, and what the second one is worth
// once the first is fixed.
module bottleneck_rank (
  input  logic clk, rst_n,
  input  logic        rank,
  input  logic [7:0]  s0, s1, s2,       // shares, percent
  output logic [7:0]  biggest_pct, second_pct,
  output logic [1:0]  first_id, second_id,
  output logic [7:0]  total_pct, unexplained_pct,
  output logic        complete, unexplained_err
);
  logic [7:0] hi, lo;
  assign hi = (s0 > s1) ? s0 : s1;
  assign lo = (s0 > s1) ? s1 : s0;
  assign biggest_pct = (hi > s2) ? hi : s2;
  // The second largest of three: the loser of the top pair, or the third value,
  // whichever is larger.
  assign second_pct  = (hi > s2) ? ((lo > s2) ? lo : s2) : hi;
  // ... identity selection omitted for length
  assign total_pct = s0 + s1 + s2;
  assign unexplained_pct = (total_pct >= 8'd100) ? 8'd0 : (8'd100 - total_pct);
  assign complete = (unexplained_pct == 8'd0);
  // A ranking that does not account for the whole cost has a term nobody named.
  assign unexplained_err = rank && (unexplained_pct != 8'd0);
endmodule

  rank: biggest=50% (term 0) second=30% (term 1) unexplained=0%

The third row is the one that matters most. A ranking accounting for 75% of the cost has a 25% term that nobody has identified — and that unnamed term could be larger than the second-ranked one being worked on. unexplained_err exists to make that visible before anybody starts optimising.

The fourth row is the opposite failure and the model reports it honestly: shares totalling 130% leave nothing unexplained and are double counting. The guard on the subtraction is what prevents an unsigned wrap there, and the row exists to drive it.

second_pct is the second-largest of three, not the loser of the first comparison — and section 12 of 18.2 made the same mistake in a different model. The second-largest of three is the loser of the top pair or the third value, whichever is larger, and getting it wrong reports the wrong thing to work on next.

14. RTL 9 — A Decision Must Be Revisitable

// A performance decision must record what it was based on, or it cannot be
// revisited when the inputs change.
module decision_record #(parameter int RECORD_RESULT_ONLY = 0) (
  input  logic clk, rst_n,
  input  logic       decide,
  input  logic       inputs_recorded, assumptions_recorded,
  input  logic       method_recorded, range_recorded,
  output logic       revisitable,
  output logic [3:0] gap_mask,
  output logic [7:0] n_decisions, n_orphaned,
  output logic       orphan_err
);
  assign gap_mask[0] = ~inputs_recorded;
  assign gap_mask[1] = ~assumptions_recorded;
  assign gap_mask[2] = ~method_recorded;
  assign gap_mask[3] = ~range_recorded;
  // The result-only build records the answer and nothing that produced it.
  assign revisitable = (RECORD_RESULT_ONLY != 0) ? 1'b0 : (gap_mask == 4'd0);
  // A decision nobody can revisit when an input changes.
  assign orphan_err = decide && !revisitable;
  // ... decision counters omitted for length
endmodule

  record: decisions=5 orphaned=4 | result-only orphaned=5

Four things a performance decision must record, each falsified alone:

The result-only build orphans all five decisions, including the one where everything was recorded — because it records nothing regardless of what was available. That is the honest model of a decision captured as a number in a slide: the analysis existed, and none of it survived.

This is where the chapter's models compose. A decision made on section 5's characterisation, bounded by section 6's ceiling, valid over section 10's range and confident by section 11's test is exactly the four things this model asks to be recorded.

15. RTL 10 — The Discipline Assembled

// The analysis discipline assembled: every property an actionable performance
// claim needs.
module analysis_discipline #(parameter int NUMBER_ONLY = 0) (
  input  logic clk, rst_n,
  input  logic       assess,
  input  logic       characterised,   // the workload's inputs are known
  input  logic       ranked,          // the cost is fully attributed
  input  logic       bounded,         // the model states its valid range
  input  logic       confident,       // enough samples, tight enough spread
  input  logic       recorded,        // the decision can be revisited
  output logic       actionable,
  output logic [4:0] fail_mask,
  output logic [7:0] n_assess, n_actionable,
  output logic       bare_number_err
);
  assign fail_mask[0] = ~characterised;
  assign fail_mask[1] = ~ranked;
  assign fail_mask[2] = ~bounded;
  assign fail_mask[3] = ~confident;
  assign fail_mask[4] = ~recorded;
  // The number-only build reports a measurement and calls it an analysis.
  assign actionable = (NUMBER_ONLY != 0) ? confident : (fail_mask == 5'd0);
  assign bare_number_err = assess && actionable && (fail_mask != 5'd0);
  // ... assessment counters omitted for length
endmodule

  discipline: assessed=6 actionable=1 | number-only actionable=5

One actionable analysis out of six, and the number-only build found five. Its four extra are characterisation, ranking, bounds and the record — every property that is about the analysis rather than about the measurement:

This is the fifth assembled model in two batches with the same structure, and the most general statement of it: a simplified model passes the gates that are visible from where it stands, and this one stands at the measurement. 17.4 stood at the datasheet, 18.1 at the bench, 18.3 at the hardware specification, and 18.4 at the switch.

16. Quantitative Reasoning

Every number is from a printed line above.

Characterisation. Five modelling attempts, four with a missing input — and the assuming build produced a model for all five.

Fix value. 2× on a 50% share is 1.33×. 100× on the same share is 2.00×, exactly the ceiling. 100× on a 5% share is 1.05×.

Sensitivity. A 400 ns delta on a 1000 ns base is 40% — and with all deltas zero the model reports no lever, not an insensitive system.

Regression gate. Four runs: the banded gate failed one, the no-band gate failed three, of which two were noise.

Extrapolation. A fit from 10 to 50: both edges inside, 70 is 50% overreach, 90 is 100% — as far outside as the fit is wide.

Confidence. Seven reports, four weak — two for too few samples and two for too loose a spread.

Roofline. 64 outstanding at 500 ns sustains 65,536 Gbps against a 400 Gbps ceiling: bandwidth bound. One outstanding at 2000 ns sustains 256 — latency bound.

Ranking. 40/20/15 leaves 25% unexplained — potentially larger than the second-ranked term.

Discipline. Six assessments, one actionable, against a number-only build's five.

17. Assertions

Every property is an immediate check written as cond !== 1'b1, sampled after a settle. 178 assertion sites across two testbenches.

18. Mutation Testing

79 mutations, one at a time, each required to make the baseline print RESULT: FAIL.

79 of 79 were killed.

The first run killed 77 and left 2 survivors — the lowest first-run survivor count of the batch, because the lessons of the previous four chapters were applied at authoring time rather than discovered at mutation time.

Both are worth stating, because neither is a testbench gap.

"Worthwhile boundary off by one" could not be killed by any stimulus. The gain is 10000 / ov in integer arithmetic, which yields 111 at ov = 90 and 109 at ov = 91 — it never produces exactly 110. A threshold of > 110 and one of >= 110 are therefore identical for every possible input, and no test can distinguish them. The fix was to move the bar to 1.11×, a value the arithmetic does produce and the 10%-share case lands on exactly.

"Overreach inverted" survived because the only outside query had distance equal to span. At a fit of 10–50 queried at 90, the distance is 40 and the span is 40, so distance × 100 / span and span × 100 / distance both give 100. Adding a query at 70 — distance 20 against a span of 40 — distinguishes 50% from 200%.

A representative sample:

19. Verification Strategy

Two builds, one stimulus. The parameter is the only difference.

Check that a threshold is reachable before trusting a boundary test. 10000 / ov never equals 110. A threshold placed on an unreachable value makes its own boundary untestable.

Check that a boundary case actually distinguishes the expressions. Distance equal to span makes a ratio and its inverse identical, so the "far outside" query proved nothing about the direction of the division.

Move the answer across every index. The sensitive input to three positions; the largest bottleneck to two.

Drive the case where the shortcut is correct. A fully characterised workload, a fit queried inside its range, a measurement with enough samples and a tight spread. In each, a checker that fired would fire on the good case.

Drive both failure modes of a conjunction separately. Section 11's weak reports are weak for two different reasons, and the two mutations dropping either term are killed by different rows.

Guard every unsigned subtraction and drive past the guard. An improvement in section 9; shares totalling over 100 in section 13.

20. Synthesis and Implementation Reality

None of this is hardware. These are executable specifications for a method, and their value is that the method has boundaries a document does not force you to state.

Section 11's confidence test is deliberately crude. Ten samples and a 10% spread are not a statistical test — they are the two properties whose absence is most common, and a real analysis needs a distribution, a stated hypothesis and an interval. The model is a floor, not a standard.

Section 7's sensitivity ranks inputs independently. Real systems have inputs that only matter together — a large working set matters only at high concurrency — and a one-at-a-time sweep misses interactions entirely. That is a known limitation of the method, not of the model.

Section 10's fitted range assumes the model was fitted at all. Many performance models are constructed from first principles rather than fitted, and their valid range is a judgement rather than a measured interval. The discipline is the same: state it, and refuse outside it.

Section 14's record is process, not code. Nothing enforces it. The model exists to make the four things explicit enough to put on a checklist, because a decision recorded as a number in a slide is section 14's result-only build.

21. Silicon Observability

This chapter's observables are not counters in a device — they are properties of an analysis.

The first row is the one that decays fastest. A number that was correct when measured becomes wrong when an input changes, and without a record of which inputs were assumed, nobody can tell whether a given change invalidates it.

22. Debug Lab

Symptom: a performance analysis produced a decision that turned out wrong.

Check which inputs were measured and which were assumed. Section 5: an assumed concurrency figure produces a bandwidth number that is right only if the assumption was.

Check whether the ranking summed to 100%. Section 13: a 25% unexplained remainder may have been larger than the term that was optimised.

Check the share the fix addressed against the gain it claimed. Section 6: a fix on 10% of the cost cannot deliver more than 1.11× whatever its speedup.

Check whether the model was used inside its fitted range. Section 10: nothing refuses to answer outside it, and the answer looks the same.

Check the sample count. Section 11: one sample is not a measurement, and a 30% spread across fifty samples is not a result either.

Check the regime. Section 12: work on a bandwidth ceiling when the workload is latency bound is work on a ceiling that is not binding.

23. Design Review

Which of the workload's inputs were measured, and which were assumed?

Do your ranked shares sum to a hundred? If not, what is the remainder and could it be bigger than the second term?

What share of the cost does the proposed fix address, and what is its ceiling?

Over what range was the model validated, and is the question inside it?

How many samples, and what spread?

Is the workload bandwidth bound or latency bound at the operating point in question?

Where is the basis of this decision written down? If the answer is a number in a slide, it is section 14's result-only build.

24. How This Appears In Real Engineering

Analysis failures do not present as wrong numbers. They present as correct numbers that led somewhere wrong, and the number is usually defensible in isolation.

The characteristic case is an optimisation that shipped and did not help. Section 6: the fix was real, the speedup was real, and the share it addressed was 8% — so the ceiling was 1.09× and the measured end-to-end improvement was inside the noise.

The second is a benchmark result extrapolated. Section 10: a model validated between 10 and 50 answering a question at 90, confidently, with no flag.

The third is a CI gate everybody ignores. Section 9: no noise band, a failure rate dominated by noise, and a culture of re-running until green — at which point the gate catches nothing.

The fourth is a decision nobody can revisit. Section 14: the inputs changed, the number is still on the slide, and no record exists of what it assumed.

25. Common Misconceptions

"The model gave us a number." From which inputs? Section 5's assuming build produces a number from any subset and marks none of them.

"We got a 3x speedup on the memory path." On what share? At a 20% share that is a 1.19× overall gain, and the ceiling is 1.25×.

"We swept the parameters and nothing mattered." Then the input that matters was not in the sweep. Section 7 calls that no lever rather than an insensitive system.

"The gate is too noisy so we re-run it." A gate without a noise band fails on noise by construction. Section 9's no-band build failed three of four runs and two were noise.

"The model extrapolates well." Nothing about a model refuses to answer outside its range. It returns a number with the same confidence it returns any other.

"We measured it." How many times? Section 11's single-sample build reports anything it is handed, and a 30% spread is not a measurement either.

"We know where the time goes — it is 40% here and 20% there." That is 60%. The other 40% is a term nobody named and it is larger than the second one.

"The result is in the deck." Then the inputs, the assumptions, the method and the range are not, and the decision cannot be revisited when any of them changes.

26. Interview Reasoning

Q1. Somebody hands you a performance model's output. What do you ask first? Which inputs were measured and which were assumed. A model fills its gaps and the answer carries no mark saying which.

Q2. A fix gives a 2x speedup on half the workload. What is the overall gain? 1.33×. And no fix on that half can ever beat 2.00×, which is worth knowing before the work starts.

Q3. A fix addresses 5 percent of the cost. Is it worth building? Its ceiling is 1.05×. Whether that is worth building is a business question, but it is bounded and the bound is one division.

Q4. You sweep every parameter and the answer does not move. What have you learned? That the input which matters is not in your sweep. That is a stronger conclusion than "the system is insensitive" and it points somewhere.

Q5. Your CI performance gate fails 60 percent of runs. What is wrong? Most likely no noise band. A gate that fails inside the measurement noise fails on noise, and a gate people re-run is a gate that catches nothing.

Q6. How wide should the noise band be? Wide enough that the false-failure rate is acceptable and narrow enough to catch the smallest regression you care about. Both need measuring; neither is a default.

Q7. A model fitted from 10 to 50 is asked about 90. What happens? It returns a number, confidently. Nothing about a fitted model refuses, which is why the range has to be carried alongside it and checked.

Q8. What is a 100 percent overreach? A query as far outside the fitted range as the range is wide. At a fit of 10 to 50, that is 90.

Q9. How many samples do you need? More than one, and enough that the spread is meaningful. Section 11 uses ten and 10% as a floor, not a standard — the real answer needs a distribution and a stated test.

Q10. Fifty samples with a 30 percent spread — is that a result? No. Enough samples and a tight enough spread are both necessary and neither is sufficient.

Q11. How do you decide whether to work on latency or bandwidth? Compare the bandwidth ceiling against what your concurrency sustains at your latency. Whichever is lower is binding, and that selects which chapter applies.

Q12. Your bottleneck shares are 40, 20 and 15. What do you do? Find the missing 25 before optimising anything. It is larger than the second-ranked term and nobody has named it.

Q13. The shares total 130 percent. What does that mean? Double counting — two terms are measuring overlapping work. Nothing is unexplained and the ranking is still wrong.

Q14. What must a performance decision record? The inputs, the assumptions, the method and the valid range. Without all four it cannot be revisited when any input changes, and inputs change.

Q15. Which of the five properties in section 15 is about the measurement? Only confidence. The other four are about the analysis surrounding it, which is why a correct measurement can still fail the chart.

Q16. Your threshold sits at a value the arithmetic never produces. Why does that matter? Because the boundary is then untestable — the comparisons on either side of it are identical for every possible input, and a bug that moves it cannot be detected.

27. Exercises

1. Extend RTL 1 with a confidence level per input — measured, estimated or assumed. How does characterised change, and what should a model do with an estimate?

2. RTL 2 assumes the fix has no cost. Add a build cost in engineer-weeks and derive the share below which no speedup justifies the work.

3. In RTL 3, make two inputs interact — a delta that appears only when both are varied. Does the one-at-a-time ranking still find it?

4. RTL 4 uses a fixed noise band. Derive it from a rolling spread of recent runs and find the window length at which the gate stops responding to real regressions.

5. In RTL 5, model a fit whose confidence decays with distance rather than stopping at the edge. What replaces in_range?

6. RTL 6 uses a fixed sample floor. Make it depend on the spread — more samples needed when the spread is wide — and find the spread at which the required count exceeds a practical budget.

7. Compose RTL 7 with 18.1 section 9's utilisation-dependent latency. At what utilisation does a bandwidth-bound workload become latency bound?

8. Add a sixth property to RTL 10 for reproducibility — can somebody else obtain the same number? Where does it belong in the mask, and is it implied by the other five?

28. Summary

Four chapters of models are useless without a method, and this chapter builds the method.

A model assumes what it is not told. Five attempts, four with a missing input, and the assuming build produced an answer for every one.

A fix is bounded by its share. 2× on half is 1.33×; 100× on a twentieth is 1.05×, and the ceiling is one division away.

Only one input usually moves the answer — and when none of them does, the model is incomplete rather than the system insensitive.

A gate needs a noise band. Three failures in four without one, two of them noise.

A model is valid where it was fitted. Both edges inside, and nothing refuses to answer at 100% overreach.

A number needs a sample count and a spread, and both are necessary while neither is sufficient.

The regime selects the chapter. Bandwidth bound at 64 outstanding; latency bound at one.

A ranking must sum to a hundred. 40/20/15 leaves 25% nobody named, potentially larger than the term being optimised.

And a decision must record its basis, or it cannot be revisited when the inputs change — which they will.

Module 19 — CXL Security takes a fabric that performs and asks the question every chapter so far has assumed away: whether the device on the other end is the device it claims to be.