DDR · Module 30
Senior Performance Tuning
Bandwidth names five quantities, and the gaps between them are the diagnosis. Three of the four gaps lie outside the memory controller, and the ceiling of your own lever is a finding rather than an excuse.
“The memory subsystem is too slow. What do you do?” The senior tier begins here, and the questions stop having correct answers.
The failure mode is proposing an optimisation. Every candidate has a list — improve the row-hit rate, batch the writes, interleave the banks — and a proposal offered before a measurement is a guess with engineering vocabulary.
Two laws, and the second is the one that separates senior from strong:
“Bandwidth” names five different quantities. Until you say which one is low, the question has not been asked.
Before committing to an optimisation, compute its ceiling: the bandwidth you would achieve if your lever were perfect. If the ceiling is below the requirement, the work cannot succeed however well it is done — and knowing that before starting is the deliverable.
1. What the Question Is Grading
Four things, and the first three happen before any DDR knowledge is used.
| What is graded | How it shows |
|---|---|
| Whether you disambiguate before answering | the first question you ask back |
| Whether you measure before proposing | whether a number precedes a verb |
| Whether you can bound your own lever | §6 |
| Whether you know whose problem it is | §4's owner column |
Row four is the one that changes careers. The largest gap in a real memory-performance problem is usually owned by a different team, and the engineer who can say so with the measurement that shows it is doing the job. The engineer who accepts the framing spends a quarter optimising the third-largest term.
2. Five Bandwidths
Each rung is an upper bound on the next. The ladder descends, and every step down has exactly one named cause.
| Rung | Quantity | What sets it |
|---|---|---|
| 1 | Peak | the interface: data rate × bus width. A property of the part and the clock — 23.2 §2 |
| 2 | Usable | peak minus unconditional device overhead — refresh, which no controller can avoid |
| 3 | Workload-achievable | the best any controller could extract from this request stream: its locality is a property of the address pattern, not of the scheduler |
| 4 | Controller-achieved | what this controller extracted: scheduling quality, direction handling, queue depth |
| 5 | Application-observed | what the software saw: interconnect transit, outstanding-transaction limits, and whether the code is latency-bound at all |
Rung 3 is the one candidates do not have, and it is the rung that assigns blame correctly.
CURRICULUM-DERIVED from 23.3, which owns the fact that access structure, stride and the address mapping produce the hit, miss and conflict counts: those counts are a property of the request stream. So a controller facing a conflict-heavy stream is not a bad controller — the ceiling it is working under was set upstream, and separating rung 3 from rung 4 is what makes that measurable rather than arguable.
And rung 5 is where the complaint came from. Nobody reports that rung 4 is low. The report is always at rung 5, and four of the five rungs are above the reporter's visibility.
3. The Ladder, Measured
DERIVED from a stated model. Normalised so peak = 100.00, so every figure is a percentage of peak.
| Rung | Factor applied | Value | Gap from the rung above |
|---|---|---|---|
| 1 Peak | — | 100.00 | — |
| 2 Usable | × 0.9551 — CURRICULUM-DERIVED | 95.51 | 4.49 |
| 3 Workload-achievable | × 0.70 — ILLUSTRATIVE locality | 66.86 | 28.65 |
| 4 Controller-achieved | × 0.85 — ILLUSTRATIVE scheduling and turnaround | 56.83 | 10.03 |
| 5 Application-observed | × 0.67 — ILLUSTRATIVE concurrency and interconnect | 38.08 | 18.75 |
The refresh factor is the one verified number here. CURRICULUM-DERIVED from 23.2 §6, which derives 350 ns / 7.8 µs = 4.49% from two verified inputs and reproduces 15.5's published table by a second route. Everything else is ILLUSTRATIVE and chosen to make the arithmetic legible.
The factors are applied multiplicatively, and that is not a stylistic choice. CURRICULUM-DERIVED from 23.2 §5, which owns the result: independent fractional costs multiply, because they apply to different denominators, and adding them double-counts the overlap — with several costs the additive total can exceed 100% and produce a negative achieved bandwidth, which is how you know the method is wrong rather than merely imprecise.
DERIVED check: the four gaps sum to 4.49 + 28.65 + 10.03 + 18.75 = 61.92, and 100.00 − 38.08 = 61.92. The decomposition is exhaustive, which is 23.2 §9's standard applied to a five-rung ladder rather than to cycles.
4. Four Gaps, Four Owners
The bottom row is the diagnosis and the top row is the report.
| Gap | ILLUSTRATIVE size | Cause | Whose lever | Owner chapter |
|---|---|---|---|---|
| 1→2 | 4.49 | refresh | nobody's — unconditional | 15.5, 23.2 §6 |
| 2→3 | 28.65 | the request stream's locality | software and address mapping | 23.3, 18.2 |
| 3→4 | 10.03 | scheduling and turnaround | the controller team | 23.4, 23.5 |
| 4→5 | 18.75 | concurrency and interconnect | the SoC and the requester | 29.1, 29.3 |
The largest gap is not the controller's, and the controller team is the one being asked. That is the result this chapter exists for, and it generalises past the illustrative numbers: three of the four gaps are outside the memory controller, so the prior probability that a memory-performance problem is a controller problem is low.
Saying it well matters. The measurement says the largest single loss is upstream of us; here is the number, here is how I measured it, and here is the most our own work could deliver even if it were perfect is collaborative. Not our problem is not, and it is also unfalsifiable.
5. Working the Question
Q1 (the first move). What do you ask before answering?
Four questions, and the order is graded:
1 WHICH BANDWIDTH is low? (§2's five rungs)
-- almost always rung 5, and the reporter cannot see the rest
2 Compared with WHAT?
-- a requirement, a previous release, or a competitor?
"too slow" with no reference is not a measurement
3 Is the workload BANDWIDTH-bound or LATENCY-bound?
-- 30.4 §4 showed CL at ~10% of a loaded read; a
latency-bound workload does not respond to bandwidth
work AT ALL, and this is the cheapest thing to settle
4 What is the DEMAND?
-- the bus can only be busy if somebody asked. §9's RTL is
the instrument that answers this, and §10 is what happens
when it cannotQuestion three is the one that saves a quarter of work. CURRICULUM-DERIVED from 23.2 §1, which owns the result that latency and bandwidth can move in opposite directions: a reordering that raises throughput raises tail latency, so “make it faster” is ambiguous in a way that determines the entire approach.
Q2 (applying the ladder). You have the five numbers in §3. What do you do first?
Nothing to the controller. The largest gap is 2→3 at 28.65, and it is the request stream's locality — a property of access structure, stride and mapping, which 23.3 owns.
The three actions, in cost order:
| Action | Cost | Expected effect |
|---|---|---|
| Measure the stride and access structure | hours | tells you whether the mapping or the software is the lever |
| Try an address-mapping change | days, no RTL | moves gap 2→3 directly — 18.2 |
| Change the access pattern in software | weeks, another team | the largest lever and the longest lead time |
And the sentence that makes this an answer rather than a deflection: I will also compute what our own scheduling work could deliver, so we know whether to do both. That is §6.
Q3 (the corner case). The row-hit rate is 85%. Is that good?
Unanswerable as posed, and two reasons make it so.
First, a hit rate is not a cost. CURRICULUM-DERIVED from 23.3, which owns the quantified cost difference between a miss and a conflict: the remaining 15% is composed of misses and conflicts, and a conflict costs strictly more than a miss because it pays a precharge before the activate. So 85% hits with a conflict-heavy remainder is materially worse than 85% hits with a miss-heavy one, and the single number cannot tell them apart.
Second, identical counts can hide opposite structure. Chapter 23.3 owns that result directly: the same hit, miss and conflict counts can arise from opposite temporal structures, and the temporal structure is what determines whether a scheduler can do anything about them. A hit rate is a summary of a summary.
The complete answer: 85% of what, with what miss-to-conflict split, and clustered or dispersed in time — and then the number means something.
6. The Ceiling of Your Own Lever
Q4 (the senior turn). Management wants 1.5×. Your team owns scheduling. Can you deliver it?
No, and proving it before starting is the answer.
DERIVED from §3's model, holding every factor fixed except the one being improved:
current application-observed 38.08
scenario A: PERFECT scheduling and turnaround (0.85 -> 1.00)
rung 4 becomes 66.86
rung 5 becomes 66.86 x 0.67 = 44.80
gain = 44.80 / 38.08 = 1.176x <-- the CEILING of our lever
scenario B: PERFECT locality (0.70 -> 1.00), software + mapping
rung 3 becomes 95.51
rung 4 becomes 95.51 x 0.85 = 81.18
rung 5 becomes 81.18 x 0.67 = 54.39
gain = 1.428x
scenario C: PERFECT above the controller (0.67 -> 1.00)
rung 5 becomes 56.83
gain = 1.492x
scenario D: all three perfect
rung 5 becomes 95.51
gain = 2.508xSo the answer is arithmetic, not opinion: our lever's ceiling is 1.176×, and the target is 1.5×. No scheduling work can reach it, however good. Two of the three levers must move, and D shows the combination is sufficient.
Three things this computation does that a proposal cannot.
It makes “no” a technical result. A refusal backed by a ceiling is a finding; a refusal backed by judgement is an opinion, and opinions lose to schedules.
It ranks the levers by what they can deliver rather than by what they cost. Scenario C is the second-largest lever and needs no DDR work at all — increasing the requester's outstanding transactions is an interconnect and requester change, and 29.3 owns why a workload that cannot keep requests in flight cannot use bandwidth.
And it survives being wrong about the factors. The ILLUSTRATIVE numbers are chosen; the method is not. Substituting measured factors changes every figure and changes nothing about the procedure — which is the right thing to say if an interviewer challenges the numbers.
The follow-up an interviewer should ask: what if the ceiling is above the target? Then the work is possible, not justified — the ceiling is achieved only by perfection, and a realistic fraction of it is what should be promised. A ceiling of 1.6× against a target of 1.5× is a project with no margin, and saying so is the same skill as saying no.
7. Every Optimisation Has a Cost — Name It
Q5 (removing an assumption). You propose reordering to raise throughput. What does it cost?
An optimisation offered without its cost is half an answer, and an interviewer will supply the missing half as a follow-up if you do not.
CURRICULUM-DERIVED from 23.4, which owns what reordering gains in the throughput of 23.2 and costs in the latency of 23.1, and from 23.2 §1: latency and bandwidth can move in opposite directions.
| Optimisation | What it raises | What it costs | The bound to state |
|---|---|---|---|
| Reorder for row hits | throughput, rung 4 | tail latency, and starvation risk | the starvation bound — 23.4, 30.5 §8 |
| Open-page policy | hit rate on a local stream | precharge cost on a conflict-heavy stream | the crossover — 23.5 |
| Write batching | slot utilisation — 12.4 | read latency for reads behind a batch | the maximum batch length — 30.5 §7 |
| Deeper queues | locality visible to the scheduler | latency variance, and area | the queue depth against the age threshold |
| Larger bursts | command efficiency | payload efficiency when the requester wanted less | the per-request waste — 12.4 |
Two rules make this a senior answer.
Every row costs a tail, not an average, and CURRICULUM-DERIVED from 23.1 — which owns why the average is the wrong statistic and what to use instead — that is the statistic that must be quoted. An optimisation that improves the mean and doubles the 99th percentile has failed a real-time requirement while passing a benchmark.
And every row needs a stated bound rather than a stated intention. We will not let it starve is not a bound; no entry waits more than N cycles, and here is the mechanism that guarantees it is — 30.5 §8. The bound is what makes the trade reviewable.
The follow-up an interviewer should ask: which of these is safe to deploy without a workload study? None of them, and saying so is correct. CURRICULUM-DERIVED from 23.5, which owns the result that neither fixed page policy wins across workloads and that hysteresis in an adaptive policy is mandatory rather than an optimisation. A policy chosen without a workload is a guess with a crossover formula attached.
8. A Regression With Nothing Changed
Q6 (diagnosing). Bandwidth dropped 15% between two releases. Nothing changed in the memory controller. Where do you look?
Rung by rung, and the ladder makes the question answerable — because the only ratio you own is rung 4 divided by rung 3, and reporting it as unchanged is both a defence and a diagnosis.
| What the rungs show | Conclusion | Owner |
|---|---|---|
| Rung 4 / rung 3 unchanged, rung 3 fell | the request stream changed — stride, footprint or mapping | 23.3, 18.2 |
| Rung 4 / rung 3 unchanged, rung 5 / rung 4 fell | above the controller — fewer outstanding transactions, or more interconnect contention | 29.1, 29.3 |
| Rung 2 fell | refresh rate rose — a thermal change, not a code change | 15.4, 15.5 |
| Rung 4 / rung 3 fell | it is the controller — configuration, not RTL | 23.5 |
| Rung 1 fell | the part is running at a different rate than before | 30.4 §2 |
Row three is the hypothesis nobody offers and it needs no software change at all. CURRICULUM-DERIVED from 15.4, which owns per-temperature retention variation, and 15.5: a hotter part refreshes more often, so a denser rack, a failed fan or a heavier neighbouring workload reduces usable bandwidth with every line of code identical. The discriminator is whether the regression correlates with temperature rather than with the release.
Row four is the one to check before defending. Nothing changed in the controller usually means no RTL changed, and a configuration change — a page policy, a queue depth, a timing register, an address map — is a controller change that leaves the RTL untouched. CURRICULUM-DERIVED from 23.5: a page-policy change alone can move the result across its crossover. So the first thing to diff is the configuration, not the source.
And the general shape of the answer: the controller's own efficiency is unchanged at X%; the loss is in gap 2→3 and correlates with the new allocator, not with our release. A ratio that is stable across releases is the single most useful artifact a controller team can keep, and it exists only because rung 3 and rung 4 were separated — which is §2's whole point arriving as a regression defence.
9. RTL Review — An Efficiency Monitor
The intended contract:
- Over a configurable window of
WINcycles, countdata_cycles— cycles in which the data bus carried payload. - Report utilisation =
data_cycles / WIN: the bus's duty cycle. - Report efficiency =
data_cycles / demand_cycles, where a demand cycle is one in which at least one request is outstanding — accepted and not yet complete. Not a cycle in which a request arrived. - Both must be reported. Neither alone identifies a bottleneck, because utilisation cannot distinguish a slow controller from an idle one and efficiency cannot see absent demand.
- Counters must be sized for
WINand must not wrap; on window end, latch both results and clear. Ifdemand_cyclesis zero, efficiency is invalid rather than zero or full.
// ---------------------------------------------------------------------
// bw_efficiency_mon -- INTENTIONALLY DEFECTIVE, for review (§9).
//
// CLASSIFICATION: verification/telemetry, ILLUSTRATIVE, CONTAINS A BUG.
//
// WHAT IT IS MEANT TO DO: the five-clause contract above -- report
// bus utilisation AND controller efficiency separately, so that §2's
// rung-3-versus-rung-4 distinction is measurable.
//
// WHY IT EXISTS HERE: §5's fourth question is "what is the demand?"
// and this block is the instrument that answers it. 12.4 owns why
// the four efficiency measures must stay distinct; this is what
// happens when the DENOMINATOR of one of them is wrong.
//
// HOW TO RUN IT: a latency-bound stream -- one outstanding request at
// a time, long gaps, low bus occupancy.
// EXPECTED RESULT under clause 3: efficiency is LOW, because the bus
// was idle during most cycles in which a request was outstanding.
// EXPECTED TRACE: demand_cycles must count every cycle with an
// outstanding request, so it must exceed data_cycles.
//
// SYNTHESIS: three counters plus two dividers used only at window
// end. Telemetry only -- it drives nothing.
// LIMITATIONS: measures rung 4 against rung 3. Rungs 1, 2 and 5 are
// not observable here: refresh is 15.5's, and application-observed
// bandwidth requires a requester-side counter (29.3). That is a
// STATED limitation and NOT the bug.
// ---------------------------------------------------------------------
module bw_efficiency_mon #(
parameter int WIN = 65536,
// COUNT, not INDEX: a counter that must REPRESENT WIN needs
// $clog2(WIN+1) bits. Sized $clog2(WIN) it saturates one short of
// the window it measures, understating a fully busy bus -- the
// instrument failing in exactly the case it exists to certify.
parameter int CNT_W = $clog2(WIN + 1)
)(
input logic clk,
input logic rst_n,
input logic data_cycle, // bus carried payload
input logic req_accepted, // a request was accepted
input logic any_outstanding, // >=1 request in flight
input logic win_tick, // end of the window
output logic [CNT_W-1:0] r_data,
output logic [CNT_W-1:0] r_demand,
output logic r_valid_eff,
output logic r_ready
);
initial begin
if (WIN < 2) $fatal(1, "bw_efficiency_mon: WIN must be >= 2");
end
logic [CNT_W-1:0] c_data, c_demand;
always_ff @(posedge clk) begin
if (!rst_n) begin
c_data <= '0; c_demand <= '0;
r_data <= '0; r_demand <= '0;
r_valid_eff <= 1'b0;
r_ready <= 1'b0;
end else if (win_tick) begin
// Latch and clear -- clause 5.
r_data <= c_data;
r_demand <= c_demand;
r_valid_eff <= (c_demand != '0);
r_ready <= 1'b1;
c_data <= '0;
c_demand <= '0;
end else begin
r_ready <= 1'b0;
if (data_cycle && c_data != {CNT_W{1'b1}}) c_data <= c_data + 1'b1;
if (req_accepted && c_demand != {CNT_W{1'b1}}) c_demand <= c_demand + 1'b1;
// ^^^^^^^^^^^^ THE DEFECT
end
end
endmoduleBefore reading on: which clause, and which direction does the reported number move?
10. The Defect — An Instrument That Reports the Opposite
The violated clause is 3, and the defect is one signal: demand_cycles increments on req_accepted — a request arriving — where the contract requires any_outstanding, a request being in flight.
any_outstanding is declared on the port list and never read, which is the same static tell as 30.7 §9's unread trained input — and it is again findable by lint before any simulation.
The trace, and the direction is the point:
ILLUSTRATIVE: a latency-bound stream. One request at a time.
Each request is accepted on one cycle, is outstanding for 120
cycles, and its burst occupies 4 data cycles. 100 requests in a
12,000-cycle window.
the CONTRACT's numbers (clause 3):
data_cycles = 100 x 4 = 400
demand_cycles = 100 x 120 = 12,000
utilisation = 400 / 12,000 = 3.3%
efficiency = 400 / 12,000 = 3.3%
reading: the bus is nearly idle AND the controller returned
almost nothing while work was outstanding. LATENCY-BOUND.
THIS BLOCK's numbers:
data_cycles = 400
demand_cycles = 100 <-- one per acceptance
utilisation = 400 / 12,000 = 3.3%
efficiency = 400 / 100 = 400%
reading: 3.3% busy and 400% efficient.Efficiency above 100% is the giveaway and nobody sees it, because telemetry dashboards clamp, and a clamped 400% renders as a reassuring 100%. So the instrument reports a perfect controller on the workload where the controller is least relevant — and the team, told the controller is already at 100%, looks upstream. In this instance that happens to be the right place to look, which is worse: the instrument was wrong and the conclusion was right, so nobody discovers the defect until a case where it matters.
And the inverse case is the damaging one. A bandwidth-bound stream with many outstanding requests makes req_accepted frequent, so efficiency reads plausibly and wrongly by a factor nobody can bound — a number that is neither obviously broken nor correct, which is the worst kind of measurement.
The correction:
// CORRECTED. Clause 3: a demand cycle is a cycle in which work
// was OUTSTANDING, not a cycle in which work ARRIVED. The
// distinction is the whole instrument: utilisation answers "was
// the bus busy?", and efficiency answers "when somebody wanted
// the bus, was it busy?". Only the second separates rung 3 from
// rung 4 (§2), and only `any_outstanding` can express it.
if (any_outstanding && c_demand != {CNT_W{1'b1}}) c_demand <= c_demand + 1'b1;And the review finding that outlives the fix: req_accepted should be counted too, as a third output. It is a genuinely useful number — the request rate — and having it alongside the other two makes §5's fourth question answerable without a new build. CURRICULUM-DERIVED from 12.4, which owns why the four efficiency measures must stay distinct rather than be blended: a monitor that reports three separate numbers is worth more than one reporting a blended score, because only separate numerators and denominators support attribution.
11. SVA Review — A Range Check on a Derived Statistic
Q. What single assertion would have caught §9's defect in the first simulation?
A range check on the derived percentage — and it is the check nobody writes, because a counter does not look like something that can be wrong.
// The one-liner. Efficiency cannot exceed 100%, so data_cycles can
// never exceed demand_cycles -- every payload cycle is a cycle in
// which something was outstanding, BY DEFINITION of the two terms.
// On the defective block this fails within one window.
property p_efficiency_is_a_fraction;
@(posedge clk) disable iff (!rst_n)
r_ready |-> (r_data <= r_demand);
endproperty
assert property (p_efficiency_is_a_fraction)
else $error("data cycles exceed demand cycles: denominator is wrong");
// Clause 2's denominator, checked the same way.
property p_utilisation_is_a_fraction;
@(posedge clk) disable iff (!rst_n)
r_ready |-> (r_data <= WIN);
endproperty
assert property (p_utilisation_is_a_fraction)
else $error("data cycles exceed the window: counter or window wrong");
// Clause 5's invalid case, as an implication in both directions --
// because reporting a valid efficiency with a zero denominator and
// reporting invalid with a non-zero one are different bugs.
property p_eff_valid_iff_demand;
@(posedge clk) disable iff (!rst_n)
r_ready |-> (r_valid_eff == (r_demand != '0));
endproperty
assert property (p_eff_valid_iff_demand)
else $error("efficiency validity disagrees with its denominator");
// Saturation must never be reached, or every result is a floor
// rather than a count -- the COUNT-versus-INDEX hazard as a
// runtime check rather than an elaboration guard.
property p_no_saturation;
@(posedge clk) disable iff (!rst_n)
r_ready |-> (r_data != {CNT_W{1'b1}} && r_demand != {CNT_W{1'b1}});
endproperty
assert property (p_no_saturation)
else $error("a counter saturated: results are floors, not counts");
// And the covers that make the assertions meaningful: the two
// REGIMES must both occur, or the monitor was only ever validated
// in one of them -- §10's point that the defect is obvious in one
// regime and unbounded in the other.
cover property (@(posedge clk) disable iff (!rst_n)
r_ready && (r_data * 10 < r_demand)); // latency-bound
cover property (@(posedge clk) disable iff (!rst_n)
r_ready && (r_data * 10 > r_demand * 8)); // bandwidth-bound
cover property (@(posedge clk) disable iff (!rst_n)
r_ready && (r_demand == '0)); // no demandThe general rule, and it is the fifth distinct variety in this module:
Every derived statistic has a range that follows from its definition. Assert the range. A measurement that can report an impossible value has never been checked at all — and the defect that produces it is in the definition, not the arithmetic.
Held against the four before it: 30.4 §8's property was too narrow in time; 30.5 §11's never named the contract's key signal; 30.6 §11's obligation was inexpressible without a model; 30.7 §10's shared a wrong constant with the environment. This one is different again: the property was trivially writable, trivially cheap, and simply never occurred to anybody — which is why 30.9 treats the question as a checklist rather than as insight.
Follow-up: why do the two regime covers matter more than the assertions? Because §10 showed the defect is glaring in one regime and unbounded in the other. A monitor validated only on bandwidth-bound traffic reports plausible nonsense on latency-bound traffic, and a green regression on one regime is not evidence about the other. Covering both regimes is the stimulus requirement, and it is the same shape as 30.7 §10's demand that the trained input be swept.
12. What Would You Measure?
Q. Instrument the ladder. What, in what order, and what does each rung cost to obtain?
| Rung or gap | Instrument | Cost | What it settles |
|---|---|---|---|
| 5 Observed | requester-side bytes and elapsed time | minutes | whether the complaint is real, and against what |
| Latency- or bandwidth-bound | outstanding-transaction count over time | minutes | §5 Q1's third question, the cheapest fork in the chapter |
| 4 Controller | §9's monitor — both numbers | hours | rung 4, and demand |
| 2→3 locality | hit / miss / conflict split, not a hit rate | hours | §5 Q3 — the split, because a conflict costs more |
| 2→3 structure | stride and temporal clustering | days | whether mapping or software is the lever — 23.3 |
| 3 Workload-achievable | replay the trace against a model controller | days | the rung that separates blame — 23.5's counterfactual |
| 1→2 refresh | derive it; do not measure it | minutes | it is tRFC / tREFI — 23.2 §6 |
Row two is the highest-value minute in the chapter. If the workload is latency-bound, every bandwidth instrument below it will report low numbers that are symptoms rather than causes, and the entire investigation changes direction.
Row six is the one that needs stating even though it is expensive. Rung 3 is not directly measurable — it is what a different controller would have achieved. CURRICULUM-DERIVED from 23.5, which owns the counterfactual measurement that lets a design evaluate a policy it has not deployed: the technique is to replay the observed request trace against an alternative policy. Without something of that shape, gap 2→3 and gap 3→4 cannot be separated, and the blame question stays arguable.
And row seven is a discipline point. Refresh overhead is derivable from two published parameters and measuring it wastes a day. Deriving what can be derived and measuring only what cannot is the difference between a plan and a wish list.
13. Common Wrong Answers
“Increase the row-hit rate.” §1, §5. An optimisation proposed before a measurement. And gap 2→3 may not be yours to move.
“Bandwidth is low.” §2. Which of five? The complaint arrives at rung 5 and four rungs are invisible to the reporter.
“Peak is 25.6 GB/s so we should be getting close to that.” §3. Refresh alone caps it — 23.2 §6 — and three more factors compose beneath that.
“Add the losses: 20% turnaround plus 4.5% refresh is 24.5%.” §3. They multiply, because they apply to different denominators — 23.2 §5. With several costs the additive total can exceed 100%.
“The controller is only achieving 56% of peak, so it is a bad controller.” §2, §4. Rung 3 was 66.86 — the stream's own locality set that ceiling. The controller's own gap was 10.03.
“85% row hits is good.” §5 Q3. Not without the miss-to-conflict split, since a conflict costs more, and not without the temporal structure, since identical counts hide opposite structures.
“We can get 1.5× with better scheduling.” §6. The ceiling of that lever is 1.176× in this model. Compute it before promising.
“The ceiling is 1.6× and the target is 1.5×, so we are fine.” §6. The ceiling assumes perfection. A project with no margin between ceiling and target is a project that will miss.
“It is not our problem.” §4. Unfalsifiable and uncollaborative. The measurement plus your own ceiling is the same claim, stated as a finding.
“Utilisation is 3%, so the controller is wasting the bus.” §9, §10. Utilisation cannot distinguish a slow controller from an absent workload. That needs the second number.
“Efficiency is 100%, so there is nothing to do here.” §10. Check whether it can exceed 100%. A clamped dashboard renders a broken denominator as a perfect score.
“It is only a counter; it cannot be wrong.” §11. Its denominator can be, and one range assertion finds it in the first window.
“The monitor passed regression.” §11. On which regime? A monitor validated only on bandwidth-bound traffic reports unbounded nonsense on latency-bound traffic.
“Reordering raises throughput.” §7. True, and it costs tail latency and risks starvation. An optimisation without its cost and a stated bound is half an answer.
“We improved the average latency by 20%.” §7. The average is the wrong statistic — 23.1 owns why. An optimisation that halves the mean and doubles the 99th percentile fails a real-time requirement and passes a benchmark.
“Open-page is the better policy.” §7. Neither fixed policy wins across workloads — 23.5 — and an adaptive one needs hysteresis as a requirement, not a refinement.
“Nothing changed in the controller.” §8. Usually means no RTL changed. Diff the configuration first — a page policy or a timing register is a controller change with an untouched source tree.
“A regression means somebody broke something.” §8. A hotter part refreshes more often, so rung 2 falls with every line of code identical. The discriminator is whether it correlates with temperature or with the release.
“Make it faster.” §5. Latency and bandwidth can move in opposite directions — 23.2 §1 — so the instruction is ambiguous in a way that determines the whole approach.
“Our part gets N GB/s in the benchmark.” A benchmark's number belongs to a workload, a controller configuration and a board. Quoting one as a property of the memory is the rung confusion of §2 with a marketing figure attached — and the interviewer's next question is which rung it measured.
14. Self-Check
-
Name the five bandwidths and state, for each adjacent pair, the single cause of the gap and the team that owns it.
-
Using §3's factors, recompute the ladder with locality at 0.90 instead of 0.70. State the new application-observed figure and which gap becomes the largest.
-
Explain in two sentences why the factors multiply, and give the symptom that shows someone has added them instead.
-
Your lever is the 4→5 gap and the concurrency factor is 0.67. Compute your ceiling, and say what you would report if the target were 1.3×.
-
Someone reports 85% row hits. Write the two follow-up questions that make the number meaningful, and say what each would change.
-
Pick two optimisations from §7. For each, name what it raises, what it costs, the statistic the cost must be quoted in, and the bound you would commit to.
-
Bandwidth fell 15% and your rung-4-over-rung-3 ratio is unchanged. Give the two remaining rung-level explanations and one observation that separates them — then give the explanation that requires no code change at all.
-
Find the defect in §9 without reading §10. Say which regime makes it obvious and which makes it dangerous.
-
Write the one-line assertion that catches it, and state the general rule about derived statistics that it instances.
-
Rung 3 cannot be measured directly. Describe the counterfactual technique that estimates it, name the chapter that owns it, and say what cannot be separated without it.
15. Where This Goes
A performance question is a measurement question with an ownership answer. Disambiguate the five bandwidths before answering; measure all five, because the gaps and not the values are the diagnosis; settle latency-bound versus bandwidth-bound first, because it is the cheapest fork and it changes everything; compose costs multiplicatively; and compute the ceiling of your own lever before committing to it, because a ceiling below the target is a finding rather than an excuse.
Three results carry forward. Three of the four gaps are outside the memory controller, so the prior probability that a memory-performance problem is a controller problem is low. An instrument whose denominator is wrong reports the opposite of the truth and renders as a reassuring number. And every derived statistic has a range that follows from its definition, so assert the range — the fifth distinct way a property can be absent or useless, and the last one needed before the taxonomy becomes a method.
Chapter 30.9 is that method. Five chapters have now produced five different reasons a green assertion proves nothing, and the senior verification question — “how would you verify a DDR controller?” — is failed by answering use UVM, add assertions, add coverage. The question is what each of those would prove, and the only defensible answer starts from the obligations rather than from the tools.
Continue learning
Related tutorials
- Related topic
The Memory Wall Problem
Why increasing compute capability eventually stops translating into proportional system performance. Latency against bandwidth, an RTL latency model that makes waiting visible, memory-level parallelism, queueing under saturation, and why DRAM is slow is the wrong way to say it.
- Related topic
SDR SDRAM
Making DRAM synchronous replaced an analog timing negotiation with a clocked contract, which is what made pipelining and counted bursts possible. It also fixes the vocabulary the rest of the curriculum depends on: clock frequency, transfer rate, data rate and bandwidth are four different quantities.
- Related topic
HBM Overview
HBM reaches hundreds of GB/s with a per-pin rate lower than DDR5's. It wins on width, not speed — and getting that width required changing the packaging, which adds a fourth design layer to array physics, device architecture and the interface.
- Related topic
DDR Bandwidth
Chapter 12.4 named four efficiency measures and built three. This builds the fourth: every bus cycle charged to exactly one named cause, with the categories provably summing to the window.
Standards & specifications
- Governing standard
- JEDEC JESD79 (DDR SDRAM)(opens JEDEC Solid State Technology Association in a new tab)
Defines the DDR SDRAM device itself — signals, command encoding, mode registers, timing parameters and the initialisation sequence — one document per generation. Memory-controller microarchitecture, address-mapping policy, PHY training algorithms and board-level design are not specified by it.
This page also covers RTL structure, verification approach and debugging technique. Those are engineering practice built on the standard, not requirements the standard itself imposes.
Where this fits
Part of the DDR curriculum.
