Skip to content
VLSI Mentor

DDR · Module 30

Senior Performance Tuning

Bandwidth names five quantities, and the gaps between them are the diagnosis. Three of the four gaps lie outside the memory controller, and the ceiling of your own lever is a finding rather than an excuse.

“The memory subsystem is too slow. What do you do?” The senior tier begins here, and the questions stop having correct answers.

The failure mode is proposing an optimisation. Every candidate has a list — improve the row-hit rate, batch the writes, interleave the banks — and a proposal offered before a measurement is a guess with engineering vocabulary.

Two laws, and the second is the one that separates senior from strong:

“Bandwidth” names five different quantities. Until you say which one is low, the question has not been asked.

Before committing to an optimisation, compute its ceiling: the bandwidth you would achieve if your lever were perfect. If the ceiling is below the requirement, the work cannot succeed however well it is done — and knowing that before starting is the deliverable.

1. What the Question Is Grading

Four things, and the first three happen before any DDR knowledge is used.

What is gradedHow it shows
Whether you disambiguate before answeringthe first question you ask back
Whether you measure before proposingwhether a number precedes a verb
Whether you can bound your own lever§6
Whether you know whose problem it is§4's owner column

Row four is the one that changes careers. The largest gap in a real memory-performance problem is usually owned by a different team, and the engineer who can say so with the measurement that shows it is doing the job. The engineer who accepts the framing spends a quarter optimising the third-largest term.

2. Five Bandwidths

Each rung is an upper bound on the next. The ladder descends, and every step down has exactly one named cause.

RungQuantityWhat sets it
1Peakthe interface: data rate × bus width. A property of the part and the clock — 23.2 §2
2Usablepeak minus unconditional device overhead — refresh, which no controller can avoid
3Workload-achievablethe best any controller could extract from this request stream: its locality is a property of the address pattern, not of the scheduler
4Controller-achievedwhat this controller extracted: scheduling quality, direction handling, queue depth
5Application-observedwhat the software saw: interconnect transit, outstanding-transaction limits, and whether the code is latency-bound at all

Rung 3 is the one candidates do not have, and it is the rung that assigns blame correctly.

CURRICULUM-DERIVED from 23.3, which owns the fact that access structure, stride and the address mapping produce the hit, miss and conflict counts: those counts are a property of the request stream. So a controller facing a conflict-heavy stream is not a bad controller — the ceiling it is working under was set upstream, and separating rung 3 from rung 4 is what makes that measurable rather than arguable.

And rung 5 is where the complaint came from. Nobody reports that rung 4 is low. The report is always at rung 5, and four of the five rungs are above the reporter's visibility.

3. The Ladder, Measured

DERIVED from a stated model. Normalised so peak = 100.00, so every figure is a percentage of peak.

RungFactor appliedValueGap from the rung above
1 Peak100.00
2 Usable× 0.9551 — CURRICULUM-DERIVED95.514.49
3 Workload-achievable× 0.70 — ILLUSTRATIVE locality66.8628.65
4 Controller-achieved× 0.85 — ILLUSTRATIVE scheduling and turnaround56.8310.03
5 Application-observed× 0.67 — ILLUSTRATIVE concurrency and interconnect38.0818.75

The refresh factor is the one verified number here. CURRICULUM-DERIVED from 23.2 §6, which derives 350 ns / 7.8 µs = 4.49% from two verified inputs and reproduces 15.5's published table by a second route. Everything else is ILLUSTRATIVE and chosen to make the arithmetic legible.

The factors are applied multiplicatively, and that is not a stylistic choice. CURRICULUM-DERIVED from 23.2 §5, which owns the result: independent fractional costs multiply, because they apply to different denominators, and adding them double-counts the overlap — with several costs the additive total can exceed 100% and produce a negative achieved bandwidth, which is how you know the method is wrong rather than merely imprecise.

DERIVED check: the four gaps sum to 4.49 + 28.65 + 10.03 + 18.75 = 61.92, and 100.00 − 38.08 = 61.92. The decomposition is exhaustive, which is 23.2 §9's standard applied to a five-rung ladder rather than to cycles.

4. Four Gaps, Four Owners

A block diagram of the five-rung bandwidth ladder with the cause of each gap shown beside it. On the left, peak bandwidth, set by the interface data rate and bus width. Peak feeds usable bandwidth, and the loss between them is refresh, an unconditional device overhead that no controller can avoid, worth four point four nine percent of peak. Usable feeds workload-achievable bandwidth, and the loss between them is the request stream's own locality, which is determined by access structure, stride and address mapping rather than by the scheduler; in the illustrative model this is the largest gap at twenty-eight point six five percent of peak, and it is owned by software and by the address mapping rather than by the memory controller team. Workload-achievable feeds controller-achieved bandwidth, and the loss between them is scheduling quality and direction turnaround, which is the memory controller team's own lever and the third largest gap at ten point zero three percent. Controller-achieved feeds application-observed bandwidth, and the loss between them is everything above the controller, meaning interconnect transit and the number of transactions the requester can keep outstanding, worth eighteen point seven five percent. The structural claim is that the complaint always arrives at the last rung while four of the five rungs lie above the reporter's visibility, and that the largest gap in this model belongs to a different team from the one usually asked to fix it.1 Peak 100.00Refresh 4.492 Usable 95.51Locality 28.653 Workload 66.86Scheduling 10.034 Controller56.83Concurrency18.755 Observed 38.08The complaintminusdeviceminussoftwareminusyoursminusSoCmeasured12

The bottom row is the diagnosis and the top row is the report.

GapILLUSTRATIVE sizeCauseWhose leverOwner chapter
1→24.49refreshnobody's — unconditional15.5, 23.2 §6
2→328.65the request stream's localitysoftware and address mapping23.3, 18.2
3→410.03scheduling and turnaroundthe controller team23.4, 23.5
4→518.75concurrency and interconnectthe SoC and the requester29.1, 29.3

The largest gap is not the controller's, and the controller team is the one being asked. That is the result this chapter exists for, and it generalises past the illustrative numbers: three of the four gaps are outside the memory controller, so the prior probability that a memory-performance problem is a controller problem is low.

Saying it well matters. The measurement says the largest single loss is upstream of us; here is the number, here is how I measured it, and here is the most our own work could deliver even if it were perfect is collaborative. Not our problem is not, and it is also unfalsifiable.

5. Working the Question

Q1 (the first move). What do you ask before answering?

Four questions, and the order is graded:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
   1  WHICH BANDWIDTH is low?  (§2's five rungs)
        -- almost always rung 5, and the reporter cannot see the rest

   2  Compared with WHAT?
        -- a requirement, a previous release, or a competitor?
           "too slow" with no reference is not a measurement

   3  Is the workload BANDWIDTH-bound or LATENCY-bound?
        -- 30.4 §4 showed CL at ~10% of a loaded read; a
           latency-bound workload does not respond to bandwidth
           work AT ALL, and this is the cheapest thing to settle

   4  What is the DEMAND?
        -- the bus can only be busy if somebody asked. §9's RTL is
           the instrument that answers this, and §10 is what happens
           when it cannot

Question three is the one that saves a quarter of work. CURRICULUM-DERIVED from 23.2 §1, which owns the result that latency and bandwidth can move in opposite directions: a reordering that raises throughput raises tail latency, so “make it faster” is ambiguous in a way that determines the entire approach.


Q2 (applying the ladder). You have the five numbers in §3. What do you do first?

Nothing to the controller. The largest gap is 2→3 at 28.65, and it is the request stream's locality — a property of access structure, stride and mapping, which 23.3 owns.

The three actions, in cost order:

ActionCostExpected effect
Measure the stride and access structurehourstells you whether the mapping or the software is the lever
Try an address-mapping changedays, no RTLmoves gap 2→3 directly — 18.2
Change the access pattern in softwareweeks, another teamthe largest lever and the longest lead time

And the sentence that makes this an answer rather than a deflection: I will also compute what our own scheduling work could deliver, so we know whether to do both. That is §6.


Q3 (the corner case). The row-hit rate is 85%. Is that good?

Unanswerable as posed, and two reasons make it so.

First, a hit rate is not a cost. CURRICULUM-DERIVED from 23.3, which owns the quantified cost difference between a miss and a conflict: the remaining 15% is composed of misses and conflicts, and a conflict costs strictly more than a miss because it pays a precharge before the activate. So 85% hits with a conflict-heavy remainder is materially worse than 85% hits with a miss-heavy one, and the single number cannot tell them apart.

Second, identical counts can hide opposite structure. Chapter 23.3 owns that result directly: the same hit, miss and conflict counts can arise from opposite temporal structures, and the temporal structure is what determines whether a scheduler can do anything about them. A hit rate is a summary of a summary.

The complete answer: 85% of what, with what miss-to-conflict split, and clustered or dispersed in time — and then the number means something.

6. The Ceiling of Your Own Lever

Q4 (the senior turn). Management wants 1.5×. Your team owns scheduling. Can you deliver it?

No, and proving it before starting is the answer.

DERIVED from §3's model, holding every factor fixed except the one being improved:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
   current application-observed  38.08

   scenario A: PERFECT scheduling and turnaround (0.85 -> 1.00)
     rung 4 becomes  66.86
     rung 5 becomes  66.86 x 0.67 = 44.80
     gain = 44.80 / 38.08 = 1.176x        <-- the CEILING of our lever

   scenario B: PERFECT locality (0.70 -> 1.00), software + mapping
     rung 3 becomes  95.51
     rung 4 becomes  95.51 x 0.85 = 81.18
     rung 5 becomes  81.18 x 0.67 = 54.39
     gain = 1.428x

   scenario C: PERFECT above the controller (0.67 -> 1.00)
     rung 5 becomes  56.83
     gain = 1.492x

   scenario D: all three perfect
     rung 5 becomes  95.51
     gain = 2.508x

So the answer is arithmetic, not opinion: our lever's ceiling is 1.176×, and the target is 1.5×. No scheduling work can reach it, however good. Two of the three levers must move, and D shows the combination is sufficient.

Three things this computation does that a proposal cannot.

It makes “no” a technical result. A refusal backed by a ceiling is a finding; a refusal backed by judgement is an opinion, and opinions lose to schedules.

It ranks the levers by what they can deliver rather than by what they cost. Scenario C is the second-largest lever and needs no DDR work at all — increasing the requester's outstanding transactions is an interconnect and requester change, and 29.3 owns why a workload that cannot keep requests in flight cannot use bandwidth.

And it survives being wrong about the factors. The ILLUSTRATIVE numbers are chosen; the method is not. Substituting measured factors changes every figure and changes nothing about the procedure — which is the right thing to say if an interviewer challenges the numbers.

The follow-up an interviewer should ask: what if the ceiling is above the target? Then the work is possible, not justified — the ceiling is achieved only by perfection, and a realistic fraction of it is what should be promised. A ceiling of 1.6× against a target of 1.5× is a project with no margin, and saying so is the same skill as saying no.

7. Every Optimisation Has a Cost — Name It

Q5 (removing an assumption). You propose reordering to raise throughput. What does it cost?

An optimisation offered without its cost is half an answer, and an interviewer will supply the missing half as a follow-up if you do not.

CURRICULUM-DERIVED from 23.4, which owns what reordering gains in the throughput of 23.2 and costs in the latency of 23.1, and from 23.2 §1: latency and bandwidth can move in opposite directions.

OptimisationWhat it raisesWhat it costsThe bound to state
Reorder for row hitsthroughput, rung 4tail latency, and starvation riskthe starvation bound — 23.4, 30.5 §8
Open-page policyhit rate on a local streamprecharge cost on a conflict-heavy streamthe crossover — 23.5
Write batchingslot utilisation — 12.4read latency for reads behind a batchthe maximum batch length — 30.5 §7
Deeper queueslocality visible to the schedulerlatency variance, and areathe queue depth against the age threshold
Larger burstscommand efficiencypayload efficiency when the requester wanted lessthe per-request waste — 12.4

Two rules make this a senior answer.

Every row costs a tail, not an average, and CURRICULUM-DERIVED from 23.1 — which owns why the average is the wrong statistic and what to use instead — that is the statistic that must be quoted. An optimisation that improves the mean and doubles the 99th percentile has failed a real-time requirement while passing a benchmark.

And every row needs a stated bound rather than a stated intention. We will not let it starve is not a bound; no entry waits more than N cycles, and here is the mechanism that guarantees it is — 30.5 §8. The bound is what makes the trade reviewable.

The follow-up an interviewer should ask: which of these is safe to deploy without a workload study? None of them, and saying so is correct. CURRICULUM-DERIVED from 23.5, which owns the result that neither fixed page policy wins across workloads and that hysteresis in an adaptive policy is mandatory rather than an optimisation. A policy chosen without a workload is a guess with a crossover formula attached.

8. A Regression With Nothing Changed

Q6 (diagnosing). Bandwidth dropped 15% between two releases. Nothing changed in the memory controller. Where do you look?

Rung by rung, and the ladder makes the question answerable — because the only ratio you own is rung 4 divided by rung 3, and reporting it as unchanged is both a defence and a diagnosis.

What the rungs showConclusionOwner
Rung 4 / rung 3 unchanged, rung 3 fellthe request stream changed — stride, footprint or mapping23.3, 18.2
Rung 4 / rung 3 unchanged, rung 5 / rung 4 fellabove the controller — fewer outstanding transactions, or more interconnect contention29.1, 29.3
Rung 2 fellrefresh rate rose — a thermal change, not a code change15.4, 15.5
Rung 4 / rung 3 fellit is the controller — configuration, not RTL23.5
Rung 1 fellthe part is running at a different rate than before30.4 §2

Row three is the hypothesis nobody offers and it needs no software change at all. CURRICULUM-DERIVED from 15.4, which owns per-temperature retention variation, and 15.5: a hotter part refreshes more often, so a denser rack, a failed fan or a heavier neighbouring workload reduces usable bandwidth with every line of code identical. The discriminator is whether the regression correlates with temperature rather than with the release.

Row four is the one to check before defending. Nothing changed in the controller usually means no RTL changed, and a configuration change — a page policy, a queue depth, a timing register, an address map — is a controller change that leaves the RTL untouched. CURRICULUM-DERIVED from 23.5: a page-policy change alone can move the result across its crossover. So the first thing to diff is the configuration, not the source.

And the general shape of the answer: the controller's own efficiency is unchanged at X%; the loss is in gap 2→3 and correlates with the new allocator, not with our release. A ratio that is stable across releases is the single most useful artifact a controller team can keep, and it exists only because rung 3 and rung 4 were separated — which is §2's whole point arriving as a regression defence.

9. RTL Review — An Efficiency Monitor

The intended contract:

  1. Over a configurable window of WIN cycles, count data_cycles — cycles in which the data bus carried payload.
  2. Report utilisation = data_cycles / WIN: the bus's duty cycle.
  3. Report efficiency = data_cycles / demand_cycles, where a demand cycle is one in which at least one request is outstanding — accepted and not yet complete. Not a cycle in which a request arrived.
  4. Both must be reported. Neither alone identifies a bottleneck, because utilisation cannot distinguish a slow controller from an idle one and efficiency cannot see absent demand.
  5. Counters must be sized for WIN and must not wrap; on window end, latch both results and clear. If demand_cycles is zero, efficiency is invalid rather than zero or full.
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// ---------------------------------------------------------------------
// bw_efficiency_mon -- INTENTIONALLY DEFECTIVE, for review (§9).
//
// CLASSIFICATION: verification/telemetry, ILLUSTRATIVE, CONTAINS A BUG.
//
// WHAT IT IS MEANT TO DO: the five-clause contract above -- report
// bus utilisation AND controller efficiency separately, so that §2's
// rung-3-versus-rung-4 distinction is measurable.
//
// WHY IT EXISTS HERE: §5's fourth question is "what is the demand?"
// and this block is the instrument that answers it. 12.4 owns why
// the four efficiency measures must stay distinct; this is what
// happens when the DENOMINATOR of one of them is wrong.
//
// HOW TO RUN IT: a latency-bound stream -- one outstanding request at
// a time, long gaps, low bus occupancy.
// EXPECTED RESULT under clause 3: efficiency is LOW, because the bus
// was idle during most cycles in which a request was outstanding.
// EXPECTED TRACE: demand_cycles must count every cycle with an
// outstanding request, so it must exceed data_cycles.
//
// SYNTHESIS: three counters plus two dividers used only at window
// end. Telemetry only -- it drives nothing.
// LIMITATIONS: measures rung 4 against rung 3. Rungs 1, 2 and 5 are
// not observable here: refresh is 15.5's, and application-observed
// bandwidth requires a requester-side counter (29.3). That is a
// STATED limitation and NOT the bug.
// ---------------------------------------------------------------------
module bw_efficiency_mon #(
  parameter int WIN   = 65536,
  // COUNT, not INDEX: a counter that must REPRESENT WIN needs
  // $clog2(WIN+1) bits. Sized $clog2(WIN) it saturates one short of
  // the window it measures, understating a fully busy bus -- the
  // instrument failing in exactly the case it exists to certify.
  parameter int CNT_W = $clog2(WIN + 1)
)(
  input  logic             clk,
  input  logic             rst_n,

  input  logic             data_cycle,    // bus carried payload
  input  logic             req_accepted,  // a request was accepted
  input  logic             any_outstanding, // >=1 request in flight
  input  logic             win_tick,      // end of the window

  output logic [CNT_W-1:0] r_data,
  output logic [CNT_W-1:0] r_demand,
  output logic             r_valid_eff,
  output logic             r_ready
);
  initial begin
    if (WIN < 2) $fatal(1, "bw_efficiency_mon: WIN must be >= 2");
  end

  logic [CNT_W-1:0] c_data, c_demand;

  always_ff @(posedge clk) begin
    if (!rst_n) begin
      c_data  <= '0;  c_demand <= '0;
      r_data  <= '0;  r_demand <= '0;
      r_valid_eff <= 1'b0;
      r_ready <= 1'b0;
    end else if (win_tick) begin
      // Latch and clear -- clause 5.
      r_data      <= c_data;
      r_demand    <= c_demand;
      r_valid_eff <= (c_demand != '0);
      r_ready     <= 1'b1;
      c_data      <= '0;
      c_demand    <= '0;
    end else begin
      r_ready <= 1'b0;
      if (data_cycle   && c_data   != {CNT_W{1'b1}}) c_data   <= c_data   + 1'b1;
      if (req_accepted && c_demand != {CNT_W{1'b1}}) c_demand <= c_demand + 1'b1;
      //  ^^^^^^^^^^^^ THE DEFECT
    end
  end
endmodule

Before reading on: which clause, and which direction does the reported number move?

10. The Defect — An Instrument That Reports the Opposite

The violated clause is 3, and the defect is one signal: demand_cycles increments on req_accepted — a request arriving — where the contract requires any_outstanding, a request being in flight.

any_outstanding is declared on the port list and never read, which is the same static tell as 30.7 §9's unread trained input — and it is again findable by lint before any simulation.

The trace, and the direction is the point:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
   ILLUSTRATIVE: a latency-bound stream. One request at a time.
   Each request is accepted on one cycle, is outstanding for 120
   cycles, and its burst occupies 4 data cycles. 100 requests in a
   12,000-cycle window.

   the CONTRACT's numbers (clause 3):
     data_cycles   = 100 x 4   = 400
     demand_cycles = 100 x 120 = 12,000
     utilisation   = 400 / 12,000 =  3.3%
     efficiency    = 400 / 12,000 =  3.3%
     reading: the bus is nearly idle AND the controller returned
             almost nothing while work was outstanding. LATENCY-BOUND.

   THIS BLOCK's numbers:
     data_cycles   = 400
     demand_cycles = 100            <-- one per acceptance
     utilisation   = 400 / 12,000 =  3.3%
     efficiency    = 400 / 100    = 400%
     reading: 3.3% busy and 400% efficient.

Efficiency above 100% is the giveaway and nobody sees it, because telemetry dashboards clamp, and a clamped 400% renders as a reassuring 100%. So the instrument reports a perfect controller on the workload where the controller is least relevant — and the team, told the controller is already at 100%, looks upstream. In this instance that happens to be the right place to look, which is worse: the instrument was wrong and the conclusion was right, so nobody discovers the defect until a case where it matters.

And the inverse case is the damaging one. A bandwidth-bound stream with many outstanding requests makes req_accepted frequent, so efficiency reads plausibly and wrongly by a factor nobody can bound — a number that is neither obviously broken nor correct, which is the worst kind of measurement.

The correction:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
      // CORRECTED. Clause 3: a demand cycle is a cycle in which work
      // was OUTSTANDING, not a cycle in which work ARRIVED. The
      // distinction is the whole instrument: utilisation answers "was
      // the bus busy?", and efficiency answers "when somebody wanted
      // the bus, was it busy?". Only the second separates rung 3 from
      // rung 4 (§2), and only `any_outstanding` can express it.
      if (any_outstanding && c_demand != {CNT_W{1'b1}}) c_demand <= c_demand + 1'b1;

And the review finding that outlives the fix: req_accepted should be counted too, as a third output. It is a genuinely useful number — the request rate — and having it alongside the other two makes §5's fourth question answerable without a new build. CURRICULUM-DERIVED from 12.4, which owns why the four efficiency measures must stay distinct rather than be blended: a monitor that reports three separate numbers is worth more than one reporting a blended score, because only separate numerators and denominators support attribution.

11. SVA Review — A Range Check on a Derived Statistic

Q. What single assertion would have caught §9's defect in the first simulation?

A range check on the derived percentage — and it is the check nobody writes, because a counter does not look like something that can be wrong.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
  // The one-liner. Efficiency cannot exceed 100%, so data_cycles can
  // never exceed demand_cycles -- every payload cycle is a cycle in
  // which something was outstanding, BY DEFINITION of the two terms.
  // On the defective block this fails within one window.
  property p_efficiency_is_a_fraction;
    @(posedge clk) disable iff (!rst_n)
      r_ready |-> (r_data <= r_demand);
  endproperty
  assert property (p_efficiency_is_a_fraction)
    else $error("data cycles exceed demand cycles: denominator is wrong");

  // Clause 2's denominator, checked the same way.
  property p_utilisation_is_a_fraction;
    @(posedge clk) disable iff (!rst_n)
      r_ready |-> (r_data <= WIN);
  endproperty
  assert property (p_utilisation_is_a_fraction)
    else $error("data cycles exceed the window: counter or window wrong");

  // Clause 5's invalid case, as an implication in both directions --
  // because reporting a valid efficiency with a zero denominator and
  // reporting invalid with a non-zero one are different bugs.
  property p_eff_valid_iff_demand;
    @(posedge clk) disable iff (!rst_n)
      r_ready |-> (r_valid_eff == (r_demand != '0));
  endproperty
  assert property (p_eff_valid_iff_demand)
    else $error("efficiency validity disagrees with its denominator");

  // Saturation must never be reached, or every result is a floor
  // rather than a count -- the COUNT-versus-INDEX hazard as a
  // runtime check rather than an elaboration guard.
  property p_no_saturation;
    @(posedge clk) disable iff (!rst_n)
      r_ready |-> (r_data != {CNT_W{1'b1}} && r_demand != {CNT_W{1'b1}});
  endproperty
  assert property (p_no_saturation)
    else $error("a counter saturated: results are floors, not counts");

  // And the covers that make the assertions meaningful: the two
  // REGIMES must both occur, or the monitor was only ever validated
  // in one of them -- §10's point that the defect is obvious in one
  // regime and unbounded in the other.
  cover property (@(posedge clk) disable iff (!rst_n)
                  r_ready && (r_data * 10 < r_demand));        // latency-bound
  cover property (@(posedge clk) disable iff (!rst_n)
                  r_ready && (r_data * 10 > r_demand * 8));    // bandwidth-bound
  cover property (@(posedge clk) disable iff (!rst_n)
                  r_ready && (r_demand == '0));                // no demand

The general rule, and it is the fifth distinct variety in this module:

Every derived statistic has a range that follows from its definition. Assert the range. A measurement that can report an impossible value has never been checked at all — and the defect that produces it is in the definition, not the arithmetic.

Held against the four before it: 30.4 §8's property was too narrow in time; 30.5 §11's never named the contract's key signal; 30.6 §11's obligation was inexpressible without a model; 30.7 §10's shared a wrong constant with the environment. This one is different again: the property was trivially writable, trivially cheap, and simply never occurred to anybody — which is why 30.9 treats the question as a checklist rather than as insight.

Follow-up: why do the two regime covers matter more than the assertions? Because §10 showed the defect is glaring in one regime and unbounded in the other. A monitor validated only on bandwidth-bound traffic reports plausible nonsense on latency-bound traffic, and a green regression on one regime is not evidence about the other. Covering both regimes is the stimulus requirement, and it is the same shape as 30.7 §10's demand that the trained input be swept.

12. What Would You Measure?

Q. Instrument the ladder. What, in what order, and what does each rung cost to obtain?

Rung or gapInstrumentCostWhat it settles
5 Observedrequester-side bytes and elapsed timeminuteswhether the complaint is real, and against what
Latency- or bandwidth-boundoutstanding-transaction count over timeminutes§5 Q1's third question, the cheapest fork in the chapter
4 Controller§9's monitor — both numbershoursrung 4, and demand
2→3 localityhit / miss / conflict split, not a hit ratehours§5 Q3 — the split, because a conflict costs more
2→3 structurestride and temporal clusteringdayswhether mapping or software is the lever — 23.3
3 Workload-achievablereplay the trace against a model controllerdaysthe rung that separates blame — 23.5's counterfactual
1→2 refreshderive it; do not measure itminutesit is tRFC / tREFI23.2 §6

Row two is the highest-value minute in the chapter. If the workload is latency-bound, every bandwidth instrument below it will report low numbers that are symptoms rather than causes, and the entire investigation changes direction.

Row six is the one that needs stating even though it is expensive. Rung 3 is not directly measurable — it is what a different controller would have achieved. CURRICULUM-DERIVED from 23.5, which owns the counterfactual measurement that lets a design evaluate a policy it has not deployed: the technique is to replay the observed request trace against an alternative policy. Without something of that shape, gap 2→3 and gap 3→4 cannot be separated, and the blame question stays arguable.

And row seven is a discipline point. Refresh overhead is derivable from two published parameters and measuring it wastes a day. Deriving what can be derived and measuring only what cannot is the difference between a plan and a wish list.

13. Common Wrong Answers

“Increase the row-hit rate.” §1, §5. An optimisation proposed before a measurement. And gap 2→3 may not be yours to move.

“Bandwidth is low.” §2. Which of five? The complaint arrives at rung 5 and four rungs are invisible to the reporter.

“Peak is 25.6 GB/s so we should be getting close to that.” §3. Refresh alone caps it — 23.2 §6 — and three more factors compose beneath that.

“Add the losses: 20% turnaround plus 4.5% refresh is 24.5%.” §3. They multiply, because they apply to different denominators — 23.2 §5. With several costs the additive total can exceed 100%.

“The controller is only achieving 56% of peak, so it is a bad controller.” §2, §4. Rung 3 was 66.86 — the stream's own locality set that ceiling. The controller's own gap was 10.03.

“85% row hits is good.” §5 Q3. Not without the miss-to-conflict split, since a conflict costs more, and not without the temporal structure, since identical counts hide opposite structures.

“We can get 1.5× with better scheduling.” §6. The ceiling of that lever is 1.176× in this model. Compute it before promising.

“The ceiling is 1.6× and the target is 1.5×, so we are fine.” §6. The ceiling assumes perfection. A project with no margin between ceiling and target is a project that will miss.

“It is not our problem.” §4. Unfalsifiable and uncollaborative. The measurement plus your own ceiling is the same claim, stated as a finding.

“Utilisation is 3%, so the controller is wasting the bus.” §9, §10. Utilisation cannot distinguish a slow controller from an absent workload. That needs the second number.

“Efficiency is 100%, so there is nothing to do here.” §10. Check whether it can exceed 100%. A clamped dashboard renders a broken denominator as a perfect score.

“It is only a counter; it cannot be wrong.” §11. Its denominator can be, and one range assertion finds it in the first window.

“The monitor passed regression.” §11. On which regime? A monitor validated only on bandwidth-bound traffic reports unbounded nonsense on latency-bound traffic.

“Reordering raises throughput.” §7. True, and it costs tail latency and risks starvation. An optimisation without its cost and a stated bound is half an answer.

“We improved the average latency by 20%.” §7. The average is the wrong statistic — 23.1 owns why. An optimisation that halves the mean and doubles the 99th percentile fails a real-time requirement and passes a benchmark.

“Open-page is the better policy.” §7. Neither fixed policy wins across workloads — 23.5 — and an adaptive one needs hysteresis as a requirement, not a refinement.

“Nothing changed in the controller.” §8. Usually means no RTL changed. Diff the configuration first — a page policy or a timing register is a controller change with an untouched source tree.

“A regression means somebody broke something.” §8. A hotter part refreshes more often, so rung 2 falls with every line of code identical. The discriminator is whether it correlates with temperature or with the release.

“Make it faster.” §5. Latency and bandwidth can move in opposite directions — 23.2 §1 — so the instruction is ambiguous in a way that determines the whole approach.

“Our part gets N GB/s in the benchmark.” A benchmark's number belongs to a workload, a controller configuration and a board. Quoting one as a property of the memory is the rung confusion of §2 with a marketing figure attached — and the interviewer's next question is which rung it measured.

14. Self-Check

  1. Name the five bandwidths and state, for each adjacent pair, the single cause of the gap and the team that owns it.

  2. Using §3's factors, recompute the ladder with locality at 0.90 instead of 0.70. State the new application-observed figure and which gap becomes the largest.

  3. Explain in two sentences why the factors multiply, and give the symptom that shows someone has added them instead.

  4. Your lever is the 4→5 gap and the concurrency factor is 0.67. Compute your ceiling, and say what you would report if the target were 1.3×.

  5. Someone reports 85% row hits. Write the two follow-up questions that make the number meaningful, and say what each would change.

  6. Pick two optimisations from §7. For each, name what it raises, what it costs, the statistic the cost must be quoted in, and the bound you would commit to.

  7. Bandwidth fell 15% and your rung-4-over-rung-3 ratio is unchanged. Give the two remaining rung-level explanations and one observation that separates them — then give the explanation that requires no code change at all.

  8. Find the defect in §9 without reading §10. Say which regime makes it obvious and which makes it dangerous.

  9. Write the one-line assertion that catches it, and state the general rule about derived statistics that it instances.

  10. Rung 3 cannot be measured directly. Describe the counterfactual technique that estimates it, name the chapter that owns it, and say what cannot be separated without it.

15. Where This Goes

A performance question is a measurement question with an ownership answer. Disambiguate the five bandwidths before answering; measure all five, because the gaps and not the values are the diagnosis; settle latency-bound versus bandwidth-bound first, because it is the cheapest fork and it changes everything; compose costs multiplicatively; and compute the ceiling of your own lever before committing to it, because a ceiling below the target is a finding rather than an excuse.

Three results carry forward. Three of the four gaps are outside the memory controller, so the prior probability that a memory-performance problem is a controller problem is low. An instrument whose denominator is wrong reports the opposite of the truth and renders as a reassuring number. And every derived statistic has a range that follows from its definition, so assert the range — the fifth distinct way a property can be absent or useless, and the last one needed before the taxonomy becomes a method.

Chapter 30.9 is that method. Five chapters have now produced five different reasons a green assertion proves nothing, and the senior verification question — “how would you verify a DDR controller?” — is failed by answering use UVM, add assertions, add coverage. The question is what each of those would prove, and the only defensible answer starts from the obligations rather than from the tools.

Continue learning

Standards & specifications

Governing standard
JEDEC JESD79 (DDR SDRAM)(opens JEDEC Solid State Technology Association in a new tab)

Defines the DDR SDRAM device itself — signals, command encoding, mode registers, timing parameters and the initialisation sequence — one document per generation. Memory-controller microarchitecture, address-mapping policy, PHY training algorithms and board-level design are not specified by it.

This page also covers RTL structure, verification approach and debugging technique. Those are engineering practice built on the standard, not requirements the standard itself imposes.

Where this fits

Part of the DDR curriculum.