Row-Buffer Question

This question is used constantly because it has a shallow entrance and a deep floor. The opening is a definition almost everyone gives correctly. The third follow-up has no correct short answer, and the fifth requires holding a scheduler, a second requester and an evidence argument in mind at once.

The ladder interviewers actually walk:

   Q1  what is a row hit?                          -> definition
   Q2  why is it cheaper than a conflict?          -> command count
   Q3  does maximising row hits maximise           -> NO SHORT ANSWER
       system performance?
   Q4  construct a case where preferring a row     -> requires a second
       hit starves another request                    requester
   Q5  what evidence distinguishes a row-locality  -> requires choosing
       problem from a bank-level bottleneck?           observability

Q1 and Q2 are recall. Q3 onwards is the interview. This chapter is built on that boundary.

1. Three Outcomes, Not Two

The first place a weak answer reveals itself is the count.

CURRICULUM-DERIVED from Module 9: an access to a bank has three possible relationships to that bank's current state, not two.

A candidate who says "hit or miss" has collapsed the two expensive cases, and they are not equally expensive: a miss pays an activate, a conflict pays a precharge and an activate. That is the difference between two commands and three, and it is the difference between waiting tRCD and waiting tRP + tRCD.

The distinction has a second consequence worth volunteering: an idle bank is a good state to find, and a controller under a close-page policy deliberately produces it. Chapter 23.5 owns that policy and its crossover. So "miss" is not simply "hit that failed" — under one policy it is the expected steady state.

2. Foundation — Q1 and Q2

Q1. What is a row hit?

What a strong answer establishes: that the row buffer holds one row per bank, and that a hit is an access whose row is the one already held.

Reasoning path: Chapter 9.2 owns the row buffer. Activating a row transfers it into the sense amplifiers; the column command then selects within it. A hit is therefore not a cache hit — nothing was retained speculatively, and there is exactly one row per bank, not a set.

Common wrong answer: the row buffer is a cache of recently used rows. Why it fails: it implies capacity and replacement. There is one row per bank and no choice about which to keep — opening a new row necessarily closes the old one, which is precisely why the conflict case exists.

Q2. Why is a hit cheaper than a conflict?

What a strong answer establishes: a command count and a timing consequence, in that order.

DERIVED under a stated ILLUSTRATIVE model — tRP = 14, tRCD = 14 command clocks, and a column command issuable immediately when the row is open:

   cost to the FIRST data of an access, by outcome

   row hit       RD                         earliest column cmd: now
                                            commands: 1

   row miss      ACT, then RD               RD no earlier than
                                            ACT + tRCD  = +14
                                            commands: 2

   row conflict  PRE, then ACT, then RD      ACT no earlier than
                                            PRE + tRP   = +14
                                            RD  no earlier than
                                            ACT + tRCD  = +28
                                            commands: 3

   so under this ILLUSTRATIVE model the conflict costs twice the
   miss's delay and three times the hit's command count.

   and note what is NOT in this arithmetic: tRAS. A row must stay
   open a minimum time (14.3) before it may be precharged, so a
   conflict arriving soon after the previous ACT can be delayed
   FURTHER by a constraint this model omits.

The last paragraph is the part that distinguishes the answer. Chapter 14.3 owns tRAS and 14.4 owns tRC; a conflict is not simply tRP + tRCD when the open row is young. Volunteering that the simple sum is a lower bound shows you know which constraints were assumed away.

3. Q3 — The Question With No Short Answer

Does maximising row hits maximise system performance?

No, and there is no correct one-sentence version of why. The honest answer names the gap between a local objective and a global one, then gives at least two independent mechanisms.

Mechanism one — it competes with bank parallelism. Chapter 16.1 owns bank-level parallelism. Serving every queued request to one open row maximises hits on that bank and leaves every other bank idle. Meanwhile a scattered stream across sixteen banks has a terrible hit rate and can overlap sixteen activates.

DERIVED, ILLUSTRATIVE, to make the trade concrete:

   8 requests, ILLUSTRATIVE: tRCD = tRP = 14, tRC = 42, a column
   command occupying 4 cycles of data bus.

   (A) all 8 to one open row -- perfect locality
       8 x RD, back-to-back, limited by column spacing
       ~ 8 x 4 = 32 cycles of bus, one bank busy, 15 banks idle

   (B) 8 to eight DIFFERENT idle banks -- zero locality
       8 x (ACT then RD). The ACTs are spaced by cross-bank
       activate constraints, but the tRCD waits OVERLAP across
       banks (16.1), so the 8 accesses pipeline rather than
       serialise.
       total time is bounded by the activate spacing and the bus,
       NOT by 8 x 14.

   the point is not which number wins -- that depends on the
   parameters and the stream. The point is that (B)'s 0% hit rate
   is NOT 8 x worse than (A)'s 100%, because the penalty is paid
   in PARALLEL. A hit-rate metric alone cannot express that.

Mechanism two — it competes with fairness. Chapter 17.4 owns starvation demonstrated concretely and the bounded bypass that fixes it. A scheduler that always prefers the row hit will serve a requester that keeps hitting indefinitely, while a requester needing a different row in the same bank waits. §4 constructs it.

Mechanism three, if you want the complete answer — it competes with the policy. Chapter 23.5 owns open-page versus close-page and the crossover between them, and establishes that neither fixed policy wins across workloads. So "maximise hits" is itself a policy choice that is wrong for some streams.

Common wrong answer: yes, because a hit is the cheapest access. Why it fails: it is a true statement about one access used to justify a claim about a stream. Cheapest-per-access and best-in-aggregate are different objectives, and the whole of 23.4 exists because they diverge.

4. Q4 — Construct the Starvation Case

Interviewers ask candidates to construct this rather than describe it, because construction proves the model.

A minimal construction, and it needs only two requesters and one bank:

   ILLUSTRATIVE. Bank 5 currently has row R_a open.

   requester P : a stream of accesses all to row R_a in bank 5
   requester Q : one access to row R_b in bank 5

   a strict row-hit-first scheduler evaluates:
     P's next access  -> row hit on R_a    -> cheapest
     Q's access       -> row conflict       -> most expensive

   it serves P. P issues another access to R_a. Repeat.

   as long as P keeps offering, Q is never selected -- not because
   Q is low priority, but because Q is always the most EXPENSIVE
   candidate, and the policy selects on cost.

   Q's wait is UNBOUNDED. That is starvation, and no priority field
   was involved.

The insight worth stating: starvation here is produced by the optimisation itself, not by a priority setting. A policy that selects the cheapest candidate starves whoever is structurally expensive — and in a bank, the expensive requester is the one that needs a different row, which is exactly the requester whose wait a locality-optimising scheduler will never shorten.

Follow-up you will get: what fixes it? Chapter 17.4 owns the answer and it is not "add priority": a bounded bypass — an age bound after which the expensive candidate is served regardless of cost. The bound converts an unbounded wait into a provable limit, and it costs exactly one row conflict each time it fires.

Follow-up after that: what state does the scheduler need? Per-request age, and a comparison against a bound. Not per-requester priority — priority does not solve this, because Q may be the highest-priority requester and still be the most expensive candidate. §6's RTL is the state this needs.

5. Q5 — Which Problem Am I Looking At?

You measure a 20% row-hit rate and low bandwidth. Is the problem locality?

Not established — and this is the question where choosing observability matters more than knowing definitions.

The last row is the one candidates miss, and it is the cheapest to check. A 20% hit rate with an almost-empty queue is not a locality problem — the controller had nothing to work with, which is a requester concurrency problem and no map or policy change will touch it.

The discriminating measurement, in order of cost:

1. Queue occupancy and data-bus utilisation together. Low/low means the memory system is idle — go upstream.
2. Bank distribution. Skewed means a bank bottleneck; even means locality.
3. Hit rate with one requester versus several. A fall implicates interleaving.
4. Hit rate under a different mapping policy. A change implicates the map.

A strong answer gives that order and says why: each step is cheaper than the next, and each one's result changes which of the remaining hypotheses are still live.

6. Q6 — Have I Eliminated Conflicts?

“If I close the page after every access, I never get a conflict. Is that an improvement?”

You have eliminated conflicts and you have not removed their cost — you have converted it into a different, unconditional cost, and naming the conversion is the answer.

   DERIVED from a stated model. ILLUSTRATIVE tRP = 14, tRCD = 14,
   and command counts from §1.

   OPEN page, a stream with hit rate h, and the remaining
   (1 - h) split between miss m and conflict c:

     cost per access  =  h*(1 cmd)  +  m*(2 cmds)  +  c*(3 cmds)

   CLOSE page -- precharge immediately after every access:

     every access finds its bank idle, so c = 0 and m = 1 - 0 = 1
     ... except the hits are gone too: h = 0.
     cost per access  =  1*(2 cmds)   for EVERY access
     plus the precharge is paid even when the next access
     would have HIT the row you just closed.

So the trade is exact rather than rhetorical. Close-page converts a variable cost into a fixed one. It wins when conflicts dominate and loses when hits do — and CURRICULUM-DERIVED from 23.5, which owns open-page, close-page and adaptive policies, the crossover between the two fixed policies and its closed form, and why neither fixed policy wins across workloads.

Two things a strong answer adds.

The precharge can be hidden, which changes the arithmetic rather than the structure. An auto-precharge or a precharge issued while the bank is otherwise idle overlaps the cost with other banks' work — so the command count is unchanged and the latency impact is not. Command count and observed latency are different currencies, and a candidate who conflates them will claim close-page is free on a multi-bank stream.

And an adaptive policy needs hysteresis as a requirement, not a refinement. CURRICULUM-DERIVED from 23.5, which owns that result: a predictor that flips policy on every change of evidence pays the switching cost continuously and can be worse than either fixed policy. Saying adaptive without saying with hysteresis is an incomplete answer to a question about policy.

The follow-up: so which would you ship? Neither, without the workload — and that is not a dodge, it is 23.5's published result. The answerable question is which measurement would decide it, and the measurement is the hit rate with its miss-to-conflict split, because the crossover is stated in exactly those terms. A hit rate alone cannot locate the crossover.

7. Two Levers, Two Teams

Q7. The hit rate is 20% and you have been asked to improve it. What can you actually change?

The hit, miss and conflict counts are produced by the access structure and the address map, and those are owned by different people.

CURRICULUM-DERIVED from 23.3, which owns what produces those counts — access structure, stride and the address mapping — and from 18.2, which owns how mapping drives the row-buffer-hit rate.

The distinction that makes this an answer: the first three do not change the stream's inherent locality; they change how much of it survives. A stream with genuine reuse and a map that scatters it has locality the controller cannot see — so the map change is the cheapest real lever and it is usually available.

And the honest boundary, which 30.8 §4 develops into a method: if the stream has no reuse at all, no map, policy or scheduler recovers any — and the correct report is that the ceiling is set upstream, with the measurement that shows it. Row four's limit is worth stating too: reordering changes which hits are realised, not how many exist.

8. The Three Outcomes as a Sequence

Answer D is the one worth noticing. Chapter 7.4 establishes that reporting a confident wrong row is worse than reporting unknown — so a row-state tracker has three answers plus an absence, not three answers. A controller that treats unknown as idle will issue ACT to a bank that already has a row open, which is an illegal command rather than a slow one.

And the diagram makes §1's closing point visible: the same access takes one, two or three commands depending on state left by earlier traffic. The cost of an access is a property of the stream.

9. RTL Review — A Row-State Tracker

The intended contract:

1. For each bank, track whether a row is open and which row it is.
2. ACT to a bank records that bank's open row.
3. PRE to a bank records that bank as idle.
4. A precharge-all records every bank as idle.
5. The classification output for a presented access must reflect the state before any command issued in the same cycle.
6. After reset, and before any command to a bank, that bank's state is unknown — not idle.

// ---------------------------------------------------------------------
// row_state_tracker -- INTENTIONALLY DEFECTIVE, for review (§9).
//
// CLASSIFICATION: synthesisable, ILLUSTRATIVE, and CONTAINS A BUG.
//
// WHAT IT IS MEANT TO DO: the six-clause contract above.
//
// WHY IT EXISTS HERE: row-state tracking is the state a scheduler
// needs to classify an access (§1), and the classification decides
// whether an access costs one, two or three commands. A tracker that
// is subtly wrong produces a scheduler that issues ILLEGAL commands,
// not merely slow ones.
//
// HOW TO RUN IT: drive ACT/PRE/PREA sequences and present accesses;
// compare `outcome` against an independent model.
// EXPECTED RESULT under the contract: an access presented in the SAME
// cycle as an ACT to its bank must classify against the PRE-ACT
// state.
// EXPECTED TRACE: `outcome` for a given (bank,row) must not change in
// the cycle a command to that bank is accepted.
//
// SYNTHESIS: a small register file plus comparators. No memory.
// LIMITATIONS: models no timing whatsoever -- Module 14 owns timing,
// and this block answers only "which row is open".
// ---------------------------------------------------------------------
typedef enum logic [1:0] {
  OUT_UNKNOWN  = 2'd0,   // clause 6: no command seen for this bank yet
  OUT_HIT      = 2'd1,
  OUT_MISS     = 2'd2,   // bank idle
  OUT_CONFLICT = 2'd3
} row_outcome_e;

module row_state_tracker #(
  parameter int BANKS = 16,
  parameter int ROW_W = 17
)(
  input  logic                       clk,
  input  logic                       rst_n,

  // ---- observed commands
  input  logic                       act_valid,
  input  logic [$clog2(BANKS)-1:0]   act_bank,
  input  logic [ROW_W-1:0]           act_row,
  input  logic                       pre_valid,
  input  logic [$clog2(BANKS)-1:0]   pre_bank,
  input  logic                       pre_all,

  // ---- the access being classified
  input  logic                       q_valid,
  input  logic [$clog2(BANKS)-1:0]   q_bank,
  input  logic [ROW_W-1:0]           q_row,
  output row_outcome_e               outcome
);
  initial begin
    if (BANKS < 2) $fatal(1, "row_state_tracker: BANKS must be >= 2 (got %0d)", BANKS);
    if (ROW_W < 1) $fatal(1, "row_state_tracker: ROW_W must be >= 1");
  end

  logic             open_q [BANKS];   // is a row open?
  logic             seen_q [BANKS];   // clause 6: any command seen?
  logic [ROW_W-1:0] row_q  [BANKS];

  always_ff @(posedge clk) begin
    if (!rst_n) begin
      for (int b = 0; b < BANKS; b++) begin
        open_q[b] <= 1'b0;
        seen_q[b] <= 1'b0;
        row_q[b]  <= '0;
      end
    end else begin
      if (pre_all) begin
        for (int b = 0; b < BANKS; b++) begin
          open_q[b] <= 1'b0;
          seen_q[b] <= 1'b1;
        end
      end else if (pre_valid) begin
        open_q[pre_bank] <= 1'b0;
        seen_q[pre_bank] <= 1'b1;
      end

      if (act_valid) begin
        open_q[act_bank] <= 1'b1;
        row_q[act_bank]  <= act_row;
        seen_q[act_bank] <= 1'b1;
      end
    end
  end

  // Classification.
  always_comb begin
    outcome = OUT_UNKNOWN;
    if (q_valid) begin
      if (!seen_q[q_bank])                           outcome = OUT_UNKNOWN;
      else if (act_valid && act_bank == q_bank) begin
        // a command to this bank is being accepted right now
        outcome = (act_row == q_row) ? OUT_HIT : OUT_CONFLICT;
      end
      else if (!open_q[q_bank])                      outcome = OUT_MISS;
      else if (row_q[q_bank] == q_row)               outcome = OUT_HIT;
      else                                           outcome = OUT_CONFLICT;
    end
  end
endmodule

Before reading on: which clause does this violate, what stimulus exposes it, and what would the wrong answer cause a scheduler to do?

10. The Defect, and Why It Is Dangerous

The violated clause is 5. The classification must reflect the state before any command issued in the same cycle, and the else if (act_valid && act_bank == q_bank) branch deliberately classifies against the in-flight ACT instead.

The stimulus that exposes it: present an access to bank 5 row R_b in the same cycle as an ACT to bank 5 row R_b, while bank 5 currently has row R_a open.

What a scheduler does with the wrong answer. It believes the access needs one command. So it schedules a column command against a bank whose row is still R_a — reading the wrong row's data, or issuing a column command during the activate's own tRCD window. That is a correctness failure and an illegal-command failure, not a performance one.

Why this bug survives testing. It requires a same-cycle coincidence of an access presentation and an ACT to the same bank — common only at high throughput, which is exactly where a directed test is least likely to look and a random test is least likely to be checked closely.

This is the stale-state family, inverted. The usual bug reads state that is too old; this one reads state that is too new. Both violate the same contract clause, which is why clause 5 is written about the state before any command in this cycle rather than about staleness.

The corrected classification:

  // CORRECTED. Clause 5: classify against the registered state only.
  // A command accepted this cycle changes the state for the NEXT
  // cycle's classification -- non-blocking assignment already
  // guarantees that, so the combinational path must not peek at the
  // command inputs at all.
  //
  // The removed branch was the entire defect. Nothing replaces it:
  // the correct behaviour is to ignore in-flight commands here, and
  // the consumer is responsible for not presenting an access it is
  // simultaneously activating for.
  always_comb begin
    outcome = OUT_UNKNOWN;
    if (q_valid) begin
      if (!seen_q[q_bank])              outcome = OUT_UNKNOWN;
      else if (!open_q[q_bank])         outcome = OUT_MISS;
      else if (row_q[q_bank] == q_row)  outcome = OUT_HIT;
      else                              outcome = OUT_CONFLICT;
    end
  end

A second defect worth noticing, because an interviewer may push for it. The sequential block gives act_valid priority over pre_valid for the same bank in the same cycle — the ACT assignment comes last and wins. Whether that is right depends on a contract this module does not state, and the honest review comment is that a same-bank PRE and ACT in one cycle is either illegal upstream or needs a stated resolution order. Leaving it implicit is a specification gap, not a code bug — and saying so is a stronger answer than inventing a rule.

11. SVA Review — What Does This Property Prove?

A property offered as proof that the tracker is correct:

  // Offered as "proves the tracker tracks the open row correctly".
  property p_act_sets_row;
    @(posedge clk) disable iff (!rst_n)
      act_valid |=> (row_q[$past(act_bank)] == $past(act_row));
  endproperty
  assert property (p_act_sets_row)
    else $error("tracker: ACT did not record its row");

Q. This property passes on the defective code in §9. Why is the design still wrong?

Because the property constrains the storage and the defect is in the classification. row_q is updated correctly in both versions; the bug is that the combinational output ignores row_q in favour of the in-flight input. The property never reads outcome at all.

This is the most valuable SVA-review pattern there is: a property that is true, non-vacuous, correctly written, and proves something adjacent to the claim made for it.

The property that would have caught it:

  // Clause 5, asserted. The classification must match what the
  // REGISTERED state implies -- so it must not change in the cycle a
  // command to that bank is accepted.
  property p_classify_against_registered_state;
    @(posedge clk) disable iff (!rst_n)
      (q_valid && seen_q[q_bank] && open_q[q_bank])
        |-> (outcome == ((row_q[q_bank] == q_row) ? OUT_HIT : OUT_CONFLICT));
  endproperty
  assert property (p_classify_against_registered_state)
    else $error("tracker: classified against something other than registered state");

  // And the antecedent must be published, or a run that never
  // presented an access to an open bank passes silently.
  cover property (@(posedge clk) disable iff (!rst_n)
                  q_valid && open_q[q_bank] && act_valid && act_bank == q_bank);

The cover is the important half. The defect only appears when an access and an ACT to the same bank coincide; a property without that cover passes on a run that never created the coincidence, which is how the bug reaches silicon with a green regression.

Follow-up an interviewer may ask: is the corrected property enough? No. It constrains the hit/conflict decision for an open bank and says nothing about OUT_MISS or OUT_UNKNOWN. A complete check needs the unknown case too — and that one matters most, because 7.4's principle is that a confident wrong answer beats no answer only in the wrong direction.

12. What Would You Measure?

Q. You suspect a row-locality problem. What do you instrument?

A strong answer names fields and what each distinguishes, not "add a hit counter".

Row one is the cheapest improvement most telemetry is missing. A single miss rate cannot tell you whether you are paying tRCD or tRP + tRCD, and the two have different fixes — a miss-dominated profile suggests a close-page policy is closing rows you would have reused, while a conflict-dominated profile suggests the stream or the map is mixing rows within a bank.

Row six is the senior answer. If the tracker is reporting unknown, every hit-rate number derived from it is suspect — and that is a measurement about the instrument rather than the system, which §11's cover argument is the verification form of.

13. Common Wrong Answers

“The row buffer is a cache of recent rows.” §2 Q1. One row per bank, no replacement choice. Opening a row necessarily closes the previous one.

“There are two outcomes: hit and miss.” §1. Three. A miss costs ACT; a conflict costs PRE and ACT, and under 14.3 it can cost more when the open row is young.

“A conflict costs tRP + tRCD.” §2 Q2. That is a lower bound. tRAS and tRC can delay the precharge itself.

“Always prefer the row hit.” §3, §4. It competes with bank parallelism, it starves whoever is structurally expensive, and 23.5 establishes no fixed policy wins across workloads.

“Starvation is a priority problem.” §4. Here it is produced by the optimisation itself — the starved requester may be the highest priority and still be the most expensive candidate.

“A low hit rate means poor locality.” §5. It is equally consistent with a bank bottleneck, interleaving, a map mismatch, or nothing offered at all.

“An idle bank is a failed hit.” §1. Under a close-page policy an idle bank is the intended steady state.

“Close-page eliminates conflicts, so it is better.” §6. It eliminates conflicts by converting a variable cost into an unconditional one, and it removes the hits too. 23.5 owns the crossover.

“Auto-precharge makes closing the page free.” §6. It hides the precharge from the command count, not from the observed latency. Command count and latency are different currencies.

“Use an adaptive page policy.” §6. Without hysteresis it pays the switching cost continuously and can be worse than either fixed policy — 23.5 owns that as a requirement, not a refinement.

“A hit rate tells you which policy to use.” §6. The crossover is stated in terms of the miss-to-conflict split, so a hit rate alone cannot locate it.

“Improving the hit rate is a scheduler problem.” §7. The counts are produced by the access structure and the address map — 23.3. The scheduler changes which hits are realised, not how many exist.

“A better address map always helps.” §7. It changes how much of the stream's existing reuse survives. If the stream has none, no map recovers any, and the correct report is that the ceiling is upstream.

“This property proves the tracker is correct.” §11. It proves the storage is correct and never reads the classification output.

“Unknown row state can be treated as idle.” §8. That produces an ACT to a bank with a row open — an illegal command, not a slow one.

14. Self-Check

1. Give the command count and the earliest-data delay for all three outcomes under a stated timing model, then name the constraint your model omitted.

2. Construct the starvation case with two requesters and one bank, and state the smallest scheduler state addition that bounds the wait.

3. A 20% hit rate is reported. List the five hypotheses from §5 and order the measurements by cost, saying what each one's result eliminates.

4. A colleague proposes close-page to eliminate conflicts. Write the cost-per-access expression for each policy from §6, and name the single measurement that would decide between them.

5. Give the four levers on the hit rate from §7 with their owners, and say which one changes how much existing locality survives rather than how much exists.

6. Find the defect in §9 without reading §10, then write the stimulus that exposes it as a three-cycle trace.

7. Explain why §11's first property passes on defective code, and write one additional property plus the cover that would catch it.

8. Argue both sides of should a scheduler prefer a row hit over a request that has waited 1,000 cycles? and state the state each side needs.

15. Where This Goes

The row-buffer question is shallow at the entrance and deep at the floor. Three outcomes rather than two; a command-count derivation whose simple sum is a lower bound; a local objective that competes with bank parallelism, with fairness, and with the page policy; starvation produced by the optimisation rather than by priority; and a low hit rate that is consistent with five different problems including "nothing was offered".

Three results carry forward. A cheapest-per-access rule starves whoever is structurally expensive — which reappears as the composition problem in 30.5. A property can be true, non-vacuous and prove something adjacent to its claim, which is 30.9's central skill. And an instrument reporting unknown invalidates every number derived from it, which 30.10 treats as the first thing to check.

Chapter 30.3 narrows to one parameter, and the narrowing is deceptive. What is tRCD is acronym expansion; the real question gives you a cycle number and asks when the next command may legally issue — and the correct answer usually begins by saying what information is still missing, because a single parameter never determines a legal issue time on its own.