Performance Optimisation

Chapter 16.2 ended with two legal candidates and one command bus, and declined to choose: a row hit in one bank against an ACTIVATE that would open a second.

This chapter takes up that choice, and the result is sharper than a preference:

Both choices are legal. They produce identical total completion time and identical row-hit rate. And one request waits seventeen cycles instead of five.

The two quantities people reach for when optimising bank utilisation — throughput and hit rate — are both blind to the difference. That is the mechanism this chapter owns, and it is why neither policy can be declared the winner.

1. The Situation, and Two Defensible Answers

Take Chapter 16.2 §6's ending and give it more traffic.

  bank 0   OPEN, row 12     four pending requests, all to row 12   → four row HITS
  bank 1   CLOSED           one pending request, row 3             → needs ACTIVATE

Both a column command to bank 0 and an ACTIVATE to bank 1 are legal at cycle 20 — Chapter 16.2 §6 verified that. The command bus carries one.

Policy A — take the row hits. Serve bank 0's four hits back to back, then open bank 1. Maximises use of an already-open row; every command until the last is the cheapest kind.

Policy B — open the other bank first. Issue bank 1's ACTIVATE now, then interleave. Gets a second bank into service early so its row is ready when its request's turn comes.

Both are reasonable. Both are what a real controller might do. §2 works out what each actually costs.

2. The Trace, Worked

Policy A — row hits first:

  cycle  command      why this cycle
  ──────────────────────────────────────────────────────────
    20   col b0  #1   tCCD from cycle 16 → 20
    24   col b0  #2   tCCD from 20
    28   col b0  #3   tCCD from 24
    32   col b0  #4   tCCD from 28
    33   ACT b1       first free cycle after the last column
    37   col b1       tRCD from 33 → 37; tCCD from 32 → 36; max = 37
  ──────────────────────────────────────────────────────────
  bank 0 finished at 32     bank 1 request served at 37

Policy B — activate first:

  cycle  command      why this cycle
  ──────────────────────────────────────────────────────────
    20   ACT b1       bus free, activate legal
    21   col b0  #1   tCCD from 16 → 20, so legal
    25   col b1       tRCD from 20 → 24; tCCD from 21 → 25; max = 25
    29   col b0  #2   tCCD from 25
    33   col b0  #3   tCCD from 29
    37   col b0  #4   tCCD from 33
  ──────────────────────────────────────────────────────────
  bank 0 finished at 37     bank 1 request served at 25

3. What the Proxies Missed, and Why

The two numbers most often used to judge a memory scheduler both report these policies as identical, and it is worth being precise about why.

Makespan is identical because the bottleneck is shared. Every column command in both schedules is spaced by tCCD on the shared data path. Five column commands at four-cycle spacing occupy 16 cycles whichever bank they target, so the last one lands at the same cycle. Reordering work across banks does not change how long the shared resource is busy — it only changes whose work goes first.

Row-hit rate is identical because it counts classifications, not schedules. Four of the five requests are hits in both policies, because whether a request is a hit depends on the bank's state when it arrives — not on the order in which the controller serves it. Chapter 9.3 owns the classification and it is policy-independent here.

4. Row-Hit Rate Is Not Bank-Level Parallelism

§2's workload was constructed so the two policies did not damage each other's hit rate. In general they do, and the mechanism is worth stating because it is the core of the trade.

The two objectives pull in opposite directions when the workload has locality in more than one bank.

  maximise row hits     →  keep serving the bank whose row is open
                        →  work concentrates in that bank
                        →  other banks stay closed, unused

  maximise bank spread  →  move to a different bank
                        →  that bank may need ACT (and PRE first)
                        →  a row hit was available and was not taken

They are different quantities measuring different things. Row-hit rate is about how much work each row opening serves — the amortisation of an expensive operation. Bank-level parallelism is about how many independent row contexts are in play — the overlap Chapter 16.1 bounded.

5. The Concentration Pathology

§4's first limit deserves its own treatment, because it is the failure that a hit-rate-maximising policy produces and because hit rate reports it as a success.

A policy that always prefers a row hit, given a workload with sustained traffic to one bank, never leaves that bank. The consequences compound:

The other banks stay closed. No activation is issued to them, so they contribute no concurrency. Chapter 16.1's budget permits four rows open and one is in use.

Requests to other banks wait indefinitely. Not for a bounded time — for as long as the favoured bank has traffic. That is starvation, and it is a fairness failure rather than a timing one, so nothing in the timing machinery detects it.

And the hit rate looks excellent. Every served request is a hit, so the metric a team is watching reports the best possible number while some requesters are not being served at all.

6. RTL — Observing Both Quantities

Collision check. Chapter 9.6's row_class_counters counts row hits, misses and conflicts over a window — the hit-rate side, and it is reused for that. Chapter 16.2's bank_candidate_mask produces the legality vector. Chapter 16.1's activate_concurrency_budget reports the activation budget. Chapter 12.4's burst_efficiency_monitor counts data-bus slot use.

None of them measures bank spread, and none reports the two quantities together. row_class_counters counts classifications without reference to which bank produced them, so a window of four hits all in one bank and a window of four hits in four banks are indistinguishable to it — which §5 just showed is precisely the distinction that matters.

The engineering problem. Over an observation window, report the row-hit rate and how many distinct banks were served and the worst per-bank service gap — so that §5's pathology, which the hit rate reports as success, is visible in the same view.

Classification: verification-oriented observation. Counts events and cycles; measures and never gates.

What it does not model. No array internals, no analog behaviour. No legality — Chapter 16.2 owns it. No policy and no fairness mechanism — it observes the consequence of a policy and corrects nothing; Chapter 17.4 owns the correction. And it is not a performance model: it counts what happened, and whether that was good is Module 23's question.

// ─────────────────────────────────────────────────────────────────────
//  bank_concurrency_observer
//
//  CLASSIFICATION
//    Verification-oriented observation. Counts events and cycles.
//    Measures; never gates, never corrects.
//
//  WHAT IT MODELS
//    §5's two quantities, side by side over one window:
//      row-hit rate      -- how well each row opening was amortised
//      bank spread       -- how many DISTINCT banks were served
//      worst service gap -- longest any requesting bank went unserved
//    §4 argues these are different quantities that can move in
//    opposite directions, and §5 shows the hit rate REPORTS the
//    concentration pathology as a success.
//
//  WHAT IT DOES NOT MODEL
//    No array internals, no analog behaviour.
//    No legality -- Chapter 16.2's bank_candidate_mask owns it.
//    No policy and NO FAIRNESS MECHANISM. This block observes the
//      consequence of a policy; Chapter 17.4 owns preventing the
//      pathology. Adding correction here would build Module 17.
//    NOT a performance model. It counts what happened. Whether that
//      was good is Module 23's question, and §1's callout explains
//      why this chapter declines it.
//
//  RELATIONSHIP TO EXISTING RTL
//    Chapter 9.6's row_class_counters counts hits/misses/conflicts and
//    is the right block for the hit-rate side. What it cannot do is
//    distinguish four hits in ONE bank from four hits in FOUR banks --
//    it counts classifications without a bank dimension, and §5 shows
//    that is exactly the distinction that hides the pathology. This
//    block adds the bank dimension and the service-gap measure.
//
//  THE SPREAD MEASURE
//    A bit per bank, set when that bank is served, counted at the end
//    of the window. Deliberately a POPCOUNT and not a rate: §5's
//    failure is "one bank was served" and a rate would average it
//    away. Storage is one bit per bank, which is negligible.
//
//  CONVENTION
//    A bank is "served" on the cycle a command targeting it is
//    accepted -- Chapter 13.2 §6's sampling convention.
//    A bank is "requesting" while it has a pending request, which is
//    an INPUT: this block has no queue visibility (Chapter 17.2).
//
//  SIMULTANEITY
//    service and window roll-over together -> credited to the CLOSING
//        window, matching Chapter 15.1 §6's rule so observers agree.
//    two banks served in one cycle -> impossible on one command bus;
//        REPORTED on multi_service rather than assumed away.
//    reset -> counts cleared; no verdict for a partial window.
// ─────────────────────────────────────────────────────────────────────
module bank_concurrency_observer #(
  parameter int NUM_BANKS     = 16,
  parameter int WINDOW_CYCLES = 256,
  parameter int BK_W  = (NUM_BANKS     <= 1) ? 1 : $clog2(NUM_BANKS),
  parameter int CNT_W = (WINDOW_CYCLES <= 1) ? 1 : $clog2(WINDOW_CYCLES + 1),
  // Spread is a COUNT of banks and can equal NUM_BANKS.
  parameter int SPR_W = (NUM_BANKS     <= 1) ? 1 : $clog2(NUM_BANKS + 1),
  // Service gap saturates at the window length.
  parameter int GAP_W = CNT_W
) (
  input  logic                     clk,
  input  logic                     rst_n,

  // ── A command was accepted this cycle, and which bank it targeted.
  input  logic                     cmd_accepted,
  input  logic [BK_W-1:0]          cmd_bank,
  // Was the command a column command serving a row hit? From
  // Chapter 9.3's row_request_classifier; this block does not
  // classify.
  input  logic                     cmd_is_row_hit,
  // Did the command serve a request, as opposed to being maintenance
  // such as ACTIVATE or PRECHARGE? Separated because §2 showed both
  // policies issue identical command COUNTS and differ in ordering.
  input  logic                     cmd_serves_request,

  // ── Which banks currently have pending work. AN INPUT: queue
  //    visibility belongs to Chapter 17.2.
  input  logic [NUM_BANKS-1:0]     bank_requesting,

  // ── Hit-rate side.
  output logic [CNT_W-1:0]         served_count,
  output logic [CNT_W-1:0]         hit_count,

  // ── Spread side. §5's measure: how many DISTINCT banks were served.
  output logic [NUM_BANKS-1:0]     banks_served_mask,
  output logic [SPR_W-1:0]         banks_served_count,

  // ── §5's pathology, made directly visible: the longest any bank
  //    went while requesting and unserved. Saturating.
  output logic [GAP_W-1:0]         worst_service_gap,
  output logic [BK_W-1:0]          worst_gap_bank,

  // ── THE observation of §5: high hit rate with low spread. Not a
  //    verdict -- a flag saying the two quantities have diverged and
  //    the hit rate alone would be misleading.
  output logic                     concentration_observed,

  output logic                     window_complete,
  // Two banks claimed in one cycle. Impossible on one command bus.
  output logic                     multi_service
);

  // ── Elaboration guards.
  if (NUM_BANKS < 1) begin : g_banks
    initial $fatal(1, "bank_concurrency_observer: NUM_BANKS must be >= 1");
  end
  if (WINDOW_CYCLES < 1) begin : g_win
    initial $fatal(1, "bank_concurrency_observer: WINDOW_CYCLES must be >= 1");
  end

  localparam int GAP_MAX = WINDOW_CYCLES;

  logic [CNT_W-1:0]     wcnt, srv_q, hit_q;
  logic [NUM_BANKS-1:0] mask_q;
  logic [GAP_W-1:0]     gap_q [NUM_BANKS];
  logic                 rolling;

  logic [SPR_W-1:0] spread;
  logic [GAP_W-1:0] wgap;
  int unsigned      wbank;

  always_comb begin
    rolling         = (wcnt == CNT_W'(WINDOW_CYCLES - 1));
    window_complete = rolling;

    served_count      = srv_q;
    hit_count         = hit_q;
    banks_served_mask = mask_q;

    // Spread: popcount of the served mask. A COUNT, not a rate --
    // see the header. $countones is legal in an always_comb.
    spread             = SPR_W'($countones(mask_q));
    banks_served_count = spread;

    // Worst gap among banks that are currently requesting. A bank
    // with no pending work is not being starved.
    wgap  = '0;
    wbank = 0;
    for (int unsigned b = 0; b < NUM_BANKS; b++) begin
      if (bank_requesting[b] && (gap_q[b] > wgap)) begin
        wgap  = gap_q[b];
        wbank = b;
      end
    end
    worst_service_gap = wgap;
    worst_gap_bank    = BK_W'(wbank);

    // §5's divergence flag. Deliberately conservative: it fires only
    // when several banks WANTED service and few got it, so a genuinely
    // single-bank workload does not trip it.
    concentration_observed =
        ( ($countones(bank_requesting) > 1)
       && (spread <= SPR_W'(1))
       && (srv_q != '0) );

    multi_service = 1'b0;   // one command bus; see the header
  end

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      wcnt   <= '0;
      srv_q  <= '0;
      hit_q  <= '0;
      mask_q <= '0;
      for (int unsigned b = 0; b < NUM_BANKS; b++) begin
        gap_q[b] <= '0;
      end
    end else begin
      // Per-bank service gap: reset on service, otherwise advance
      // while requesting. Saturating, so severe starvation reports
      // truthfully rather than wrapping to a small number --
      // Chapter 15.2 §7's argument, unchanged.
      for (int unsigned b = 0; b < NUM_BANKS; b++) begin
        if (cmd_accepted && cmd_serves_request && (cmd_bank == BK_W'(b))) begin
          gap_q[b] <= '0;
        end else if (bank_requesting[b] && (gap_q[b] != GAP_W'(GAP_MAX))) begin
          gap_q[b] <= gap_q[b] + 1'b1;
        end
      end

      if (rolling) begin
        // Credit this cycle to the closing window, then restart.
        wcnt   <= '0;
        srv_q  <= '0;
        hit_q  <= '0;
        mask_q <= '0;
      end else begin
        wcnt <= wcnt + 1'b1;
        if (cmd_accepted && cmd_serves_request) begin
          srv_q          <= srv_q + 1'b1;
          mask_q[cmd_bank] <= 1'b1;
          if (cmd_is_row_hit) begin
            hit_q <= hit_q + 1'b1;
          end
        end
      end
    end
  end

endmodule

Interface contract. bank_requesting is an input because queue visibility is Chapter 17.2's, and cmd_is_row_hit comes from Chapter 9.3's classifier — this block classifies nothing. cmd_serves_request separates requests served from maintenance commands, which matters because §2 showed both policies issue identical command counts and differ only in ordering.

Parameter contract. SPR_W is $clog2(NUM_BANKS + 1) because the spread can equal the bank count, while BK_W is $clog2(NUM_BANKS) because an index cannot — the same ten-character distinction Chapter 14.8 §5 flagged. NUM_BANKS == 1 is legal and makes the spread measure degenerate, which is the configuration in which §5's pathology cannot exist.

Why the spread is a popcount and not a rate. §5's failure is “one bank was served”, and any rate averages that away — a window with 100 services all in one bank and a window with 100 services across 16 banks have identical service rates. The count is the measure that distinguishes them, and it costs one bit per bank.

Why concentration_observed is conservative. It requires more than one bank requesting and a spread of at most one. A workload that genuinely only touches one bank is not being starved and must not trip the flag — otherwise the signal becomes noise and gets ignored, which is Chapter 14.9 §7's over-blocking argument in a different setting.

Corner cases. No requests at all: gaps do not advance, since a bank with no pending work is not starved. A bank served while not requesting: gap resets, harmlessly. All banks requesting and one served repeatedly: gaps on the others climb to saturation and worst_service_gap reports truthfully. Window roll-over with a service: credited to the closing window. NUM_BANKS == 1: concentration_observed can never assert, correctly.

Synthesis implications. Two counters, a bank mask, NUM_BANKS gap counters, a popcount and a max-reduction. For 16 banks with 9-bit gaps that is roughly 170 flops — this is verification instrumentation and would not normally ship.

Failure modes. Reporting only the hit rate — §5's pathology reads as success. Making spread a rate — averages away the one-bank case. Advancing gaps for non-requesting banks — reports starvation where there is no demand, and the flag stops being believed. Wrapping the gap counter — severe starvation looks mild. Counting maintenance commands as service — a policy that issues activations to a bank without ever serving its request would appear to be serving it.

7. Both Quantities, in Cycles

  4 banks, all requesting throughout. Window = 16 cycles.
  Policy: always take the row hit in bank 0.

  cyc  served  hit  bank  srv  hits  mask  spread  worst gap (bank)
  ─────────────────────────────────────────────────────────────────
    0    yes    Y    b0    1     1   0001     1        1  (b1)
    4    yes    Y    b0    2     2   0001     1        5  (b1)
    8    yes    Y    b0    3     3   0001     1        9  (b1)
   12    yes    Y    b0    4     4   0001     1       13  (b1)
   15     ·     ·     ·    4     4   0001     1       16  (b1)
  ─────────────────────────────────────────────────────────────────
  hit rate  = 4/4 = 100 %        ← the metric being watched
  spread    = 1 of 4 banks       ← the problem
  worst gap = 16 cycles, saturated, bank 1
  concentration_observed = TRUE

Three observations.

A perfect hit rate and a starving system. Every served request was a hit, so the number a team watches reports 100 % while three of four banks were never served. §5's signature, in a table.

worst_service_gap saturates at the window length, which is the honest report: bank 1 went the entire window unserved, and the block says so rather than wrapping to something reassuring.

concentration_observed is the flag that makes the divergence actionable. Hit rate alone says success; spread alone says one bank; the two together say the hit rate is misleading, which is the statement a reader needs.

8. Four Assertions Worth Writing

// ── P1. The hit count never exceeds the served count. A structural
//    sanity check: a hit is a kind of service, so counting more hits
//    than services means the classification input is being counted
//    independently of the service condition.
property p_hits_are_a_subset_of_services;
  @(posedge clk) disable iff (!rst_n)
    (hit_count <= served_count);
endproperty
a_hits_are_a_subset_of_services: assert property (p_hits_are_a_subset_of_services);

// ── P2. The spread is the popcount of the served mask, and a bank
//    appears in the mask only if it was served. Catches a spread
//    computed as a rate or derived from requests rather than
//    services -- §6's argument that a rate averages away the
//    one-bank case.
property p_spread_is_the_served_popcount;
  @(posedge clk) disable iff (!rst_n)
    (banks_served_count == SPR_W'($countones(banks_served_mask)));
endproperty
a_spread_is_the_served_popcount: assert property (p_spread_is_the_served_popcount);

// ── P3. A bank's service gap advances only while it is REQUESTING,
//    and resets on service. Catches a gap that advances for idle
//    banks, which reports starvation where there is no demand and
//    makes the flag noise.
generate
  for (genvar gb = 0; gb < NUM_BANKS; gb++) begin : g_gap
    property p_gap_tracks_demand;
      @(posedge clk) disable iff (!rst_n)
        ( (!$past(bank_requesting[gb], 1)
           && !($past(cmd_accepted, 1) && $past(cmd_serves_request, 1)
                && ($past(cmd_bank, 1) == BK_W'(gb))))
            |-> (gap_q[gb] == $past(gap_q[gb], 1)) )
        and ( ($past(cmd_accepted, 1) && $past(cmd_serves_request, 1)
               && ($past(cmd_bank, 1) == BK_W'(gb)))
            |-> (gap_q[gb] == '0) );
    endproperty
    a_gap_tracks_demand: assert property (p_gap_tracks_demand);
  end
endgenerate

// ── P4. The concentration flag requires genuine contention. Catches
//    a flag that fires on a single-bank workload, which is not
//    starvation and which would make the signal noise -- and a
//    signal that is noise gets switched off, taking the real cases
//    with it.
property p_concentration_requires_contention;
  @(posedge clk) disable iff (!rst_n)
    concentration_observed |-> (($countones(bank_requesting) > 1)
                             && (banks_served_count <= SPR_W'(1)));
endproperty
a_concentration_requires_contention:
  assert property (p_concentration_requires_contention);

// ── C1. BOTH regimes are reached. The concentration cover is the one
//    people omit, and a monitor never observed to flag concentration
//    is not known able to.
c_saw_concentration: cover property (@(posedge clk) disable iff (!rst_n)
                                       concentration_observed);
c_saw_wide_spread:   cover property (@(posedge clk) disable iff (!rst_n)
                                       (banks_served_count >= SPR_W'(3)));
c_window_done:       cover property (@(posedge clk) disable iff (!rst_n)
                                       window_complete);

What these prove. That hits are a subset of services; that the spread is a genuine popcount over served banks; that gaps track demand rather than idleness; and that the concentration flag requires real contention.

What these do not prove, and the first is the chapter's honest limit.

Nothing here proves the policy is good. The block measures; it has no notion of what a system wants. A high concentration flag on a workload that genuinely should favour one bank is a correct measurement and a wrong conclusion, and no property can tell the difference — that judgement is Module 23's and the fairness mechanism is Chapter 17.4's.

Nothing proves bank_requesting is honest. A design that never asserts it makes every gap zero and the flag permanently low — the most reassuring possible wrong answer. C1's first cover is the only thing that says whether the divergent regime was exercised.

And nothing here is a bandwidth or latency measurement. §2's difference was a latency distribution, and this block reports a gap, which is a proxy for it and not the thing itself.

Vacuity. P3's first conjunct needs a non-requesting, non-served bank; P4 needs the flag. C1's three covers exist because a short run with a realistic window completes none.

9. DV — Measuring the Divergence Independently

Invert the representation. The block accumulates counters and a mask. A checker should hold a list of served (cycle, bank) pairs and compute both quantities by scanning at window close — a recount rather than an accumulation, so it cannot share a counting or roll-over bug.

Derive service from the command stream, not from the design's flags. cmd_serves_request and cmd_is_row_hit are inputs, and §8's second limitation is that a design under-reporting either produces flattering numbers. A checker should classify from the observed commands and reconstructed bank state, which is Chapter 9.3 §8's rule.

Report both quantities together, always. This is the chapter-specific obligation: a report giving hit rate without spread reproduces exactly the blindness §5 is about.

  BANK UTILISATION OBSERVATION            (not a verdict — see below)
    window            : cycles 8192 .. 8447   (256 cycles)
    banks present     : 16
    ─────────────────────────────────────────────────────────────────
    requests served   : 41
    row hits          : 39          → hit rate 95.1 %
    distinct banks    : 2 of 16     → spread 12.5 %
    worst service gap : 256 cycles, SATURATED, bank 9
    banks requesting  : 11
    ─────────────────────────────────────────────────────────────────
    concentration     : OBSERVED
    note              : hit rate alone reports this window as
                        near-optimal; 9 of 11 requesting banks were
                        never served
    NOT A VERDICT     : whether this is correct depends on what the
                        system requires of those 9 banks. Module 23
                        owns that judgement; Chapter 17.4 owns the
                        fairness mechanism if one is wanted.

Two things to steal. Reporting hit rate and spread on adjacent lines is the whole point — separating them into different reports recreates the blindness. And the explicit NOT A VERDICT line, because a measurement that looks like a judgement invites someone to optimise the number rather than the system, which is the failure this chapter warns about twice.

10. Debugging

Symptom. Throughput or latency is worse than expected, and the row-hit rate looks excellent.

The discriminator that defines this chapter is measuring spread and hit rate together. Either alone is misleading in a specific and opposite direction: hit rate hides concentration, and spread hides locality destruction. One line of instrumentation showing both settles which regime you are in, and they lead to opposite fixes.

The second discriminator is whether any policy changes the outcome. If concentration persists no matter how the controller chooses, the scheduler is not the problem — the mapping is, and that is Module 18.

Responsible layer. If spread is healthy, hit rate is reasonable, and the makespan matches the shared-bottleneck bound, then nothing here is wrong — the memory system is delivering what the access pattern and the device allow, and any further improvement comes from changing the access pattern.

11. Common Misconceptions

“A row hit is always the best request to schedule.” Tempting because a hit genuinely is the cheapest command sequence — one command against three for a conflict. Why it is wrong: cheapest for that request is not best for the system. §2 showed a policy taking every available hit leaving one request waiting 17 cycles where an alternative served it in 5, with identical makespan and hit rate. Consequence: a policy that concentrates work and starves other banks while its metric reports success. Replacement model: cheapness is one input to a choice, not the choice. Debugging clue: excellent hit rate with poor worst-case latency.

“Maximum row-hit rate means maximum memory performance.” Tempting because hit rate is the easiest quantity to measure and it does correlate with efficiency in many workloads. Why it is wrong: §3 — the hit rate counts classifications, not schedules, and §2's two policies had identical rates with very different latency distributions. §5 shows the pathological case where the rate is 100 % and three of four banks starve. Consequence: optimisation effort aimed at a number the decision does not move. Replacement model: hit rate measures amortisation of row openings; it says nothing about distribution. Debugging clue: hit rate improved and nothing else did.

“Row-hit rate and bank-level parallelism are the same thing.” Tempting because both are “good” bank-related properties that a well-behaved controller should have. Why it is wrong: §4 — one measures how much work each row opening serves, the other how many independent row contexts are in play, and maximising either alone degrades the other. Consequence: a design that reports one and believes it has characterised both. Replacement model: two quantities, measured separately, traded deliberately. Debugging clue: a report containing only one of them.

“If two requests are legal, there is always an objectively better choice.” Tempting because engineering problems usually have a better answer, and it feels like the numbers should decide. Why it is wrong: §2's two policies produce identical makespan and identical hit rate and differ in which request waits. Preferring one means preferring one requester, which is a system decision rather than a memory-controller fact. Consequence: a policy chosen by intuition and defended as optimal, and fairness properties that cannot be stated. Replacement model: legality has one answer; policy has a system-dependent one. Debugging clue: a scheduler whose policy is not written down anywhere.

“Bank conflicts are just cache misses.” Tempting because both are “the fast path failed” and both cost extra time. Why it is wrong: a cache miss fetches data that was not present; a bank conflict is a resource state problem — the data is present and the bank is holding the wrong row, so the fix is three commands rather than a fetch. And Chapter 9.3 §3 established row numbers repeat per bank, so conflict is about this bank's current row, not about global residency. Consequence: reasoning about bank behaviour with cache intuitions, which get the remedy wrong — associativity and replacement have no analogue here. Replacement model: a row buffer is a state, not a cache. Debugging clue: a proposed fix involving replacement policy.

“Spreading work across banks is always better.” Tempting because it is the natural correction once you have seen §5's concentration pathology. Why it is wrong: §4's second limit — a policy that moves banks at every opportunity opens rows it barely uses, paying tRCD and later tRP for each, and runs into Chapter 16.1's activation budget, which caps openings at four per window regardless of willingness. Consequence: an over-correction that destroys locality the workload had. Replacement model: both extremes are bad and the useful behaviour is in between. Debugging clue: activations per served request rising while hit rate falls.

12. Interview Reasoning

“Why might maximising row hits reduce useful concurrency?” Because a policy that always prefers an available hit never leaves the bank whose row is open, so no other bank gets activated and the concurrency the device permits goes unused. The uncomfortable part is that the hit rate reports this as a success — every served request was a hit — while requests to other banks wait unboundedly. It is a fairness failure rather than a timing one, so nothing in the timing machinery detects it.

“Two legal choices — a row hit in one bank or an activate in another. Which is better?” Neither, in general, and I can make that concrete. Working a small trace with four hits pending in one bank and one request to a closed bank, both policies produce the same total completion time and the same hit rate, because the bottleneck is the shared column spacing and the hit rate counts classifications rather than order. What differs is the latency distribution — one request waits 17 cycles in one policy and 5 in the other. So the choice is about whose request waits, which is a system question.

“What is the difference between a row hit and bank-level parallelism?” A row hit is about amortisation — how much work a single row opening serves. Bank-level parallelism is about concurrency — how many independent row contexts are in play at once. They are different quantities and they conflict: staying in a bank raises the first and lowers the second, and moving between banks does the reverse. Measuring only one characterises neither.

“How would you detect a scheduler that has concentrated all its work in one bank?” Count distinct banks served over a window alongside the hit rate, and measure the longest any requesting bank went unserved. Hit rate alone is exactly the wrong instrument — it reports concentration as 100 % success. The two together are informative in a way neither is alone, and the gap measure should saturate rather than wrap so that severe starvation does not report as mild.

“Your hit rate is 95 % and throughput is disappointing. Where do you look?” First at the spread — how many distinct banks were actually served — because a high hit rate with a low spread is the concentration signature and the two numbers point at opposite fixes. If the spread is healthy, compute whether the makespan matches the shared column-spacing bound; if it does, reordering cannot help and the answer is fewer or wider accesses rather than better scheduling. And if concentration persists under every policy, the scheduler is not the problem — the address mapping is not distributing requests across banks in the first place.

“Is there a best bank scheduling policy?” No, and the reason is structural rather than a lack of study. The decision trades latency between requesters, and which requester should wait is a property of the system rather than of the memory. What a controller can do is compute legality correctly, measure both quantities rather than one, and make the policy explicit so its fairness behaviour can be reasoned about — which is why legality and policy are kept in separate stages.

13. Engineering Exercises

1. Reproduce the divergence. Using §2's educational values, work both policies for six hits pending in bank 0 and one request to closed bank 1. Give makespan, hit rate and bank 1's latency under each.

Worked: Policy A serves six hits at 20, 24, 28, 32, 36, 40, then ACT at 41 and bank 1's column at 45 — makespan 45, bank 1 latency 25. Policy B activates at 20, serves bank 0 at 21, bank 1 at 25, then bank 0's remaining five at 29, 33, 37, 41, 45 — makespan 45, bank 1 latency 5. Identical makespan and hit rate again, with bank 1's latency now differing by 20 cycles rather than 12. The divergence grows with the queue depth, which is the generalisation worth extracting.

2. Find the bottleneck. In exercise 1, why is the makespan identical? Compute the bound that sets it.

Worked: seven column commands at tCCD of 4 occupy 6 × 4 = 24 cycles from the first, so the last lands at 21 + 24 = 45 regardless of order — the shared data path is saturated. Reordering cannot beat a saturated shared resource; it can only change whose work goes first. That is §3's argument as arithmetic.

3. Detect the pathology. A 64-cycle window reports 15 services, 15 row hits, and 2 distinct banks served, with 9 banks requesting. State what each number says and what you would check next.

Worked: hit rate 100 % — every row opening was fully amortised. Spread 2 of 9 requesting — seven banks got nothing. Together: concentration. What to check next is whether it persists under a different policy: if it does, the requests are not landing in different banks and the problem is Module 18's mapping; if it does not, the policy is starving banks and Chapter 17.4 owns the fix.

4. Cost the over-correction. A policy moves to a different bank on every opportunity. With tRCD of 4 and one column command per activation, compute the commands per served request and compare against a policy serving four hits per activation.

Worked: spreading gives ACTIVATE + column = 2 commands per request, plus an eventual tRP. Four hits per activation gives (1 + 4) / 4 = 1.25 commands per request. So the spreading policy issues 60 % more commands for the same work — on a command bus that carries one per cycle, which §3 identified as the bottleneck. Spreading is not free and the cost lands on the shared resource.

5. Explain the blind spot. Why can row-hit rate not detect §5's pathology even in principle? Give the property of the metric that causes it.

Worked: the hit rate is a ratio over served requests, so requests that are never served do not appear in the denominator at all. Starving a bank removes its requests from the measurement rather than penalising it — the metric improves as starvation worsens. That is a structural property of any ratio over completed work, and it is why the spread must be counted separately rather than derived.

6. Choose a measure. §6 reports spread as a count of distinct banks. Argue for reporting it instead as the fraction of requesting banks served, then say which you would ship and why.

7. Bound the trade. Construct a workload for which policy A and policy B give genuinely identical outcomes on every measure, and say what property of the workload makes the choice irrelevant.

14. Summary

Chapter 16.2 produced two legal candidates and declined to choose. This chapter worked the choice and found that neither is faster.

Identical makespan, identical row-hit rate, identical command counts — and one request waits 17 cycles instead of 5. Policy A finishes bank 0's stream five cycles earlier; policy B serves bank 1 twelve cycles earlier. The policies do not differ in speed; they differ in who waits.

Both common proxies are structurally blind to it. Makespan is set by the shared column spacing, so reordering across banks cannot change it once the data path is saturated. Row-hit rate counts classifications, which depend on bank state when a request arrives rather than on the order of service. The quantity that differs is the latency distribution, which neither contains.

Row-hit rate and bank-level parallelism are different quantities that conflict. One measures how much work each row opening serves; the other how many independent row contexts are in play. Maximising the first concentrates work and leaves the activation budget unused; maximising the second opens rows it barely uses and pays tRCD and tRP for each. Both limits are bad and neither is the objective.

The concentration pathology is the sharpest case, because the metric most likely to be watched reports it as success: every served request is a hit, other banks starve, nothing is illegal, and no timing check fires. §13's fifth exercise explains why the blindness is structural — a ratio over served requests removes starved requests from its own denominator, so the metric improves as starvation worsens.

bank_concurrency_observer reports hit rate and distinct banks served together, with a saturating per-bank service gap and a conservative divergence flag that requires genuine contention. The spread is a popcount and not a rate, because any rate averages away the one-bank case that is the whole problem.

And this chapter measures without judging. Whether a given concentration is wrong depends on what the system requires — Module 23 owns that, and Chapter 17.4 owns the fairness mechanism. The contribution here is that the two numbers must be read together, because each hides what the other reveals.

15. What Comes Next

Three chapters have treated banks as interchangeable: 16.1's budget counted activations without asking which bank, 16.2's vector treated every bank identically, and this chapter's spread counted distinct banks without distinguishing them.

They are not interchangeable. Chapter 16.4 introduces the level of hierarchy that Chapter 5.3 established exists and this module has been quietly relying on: bank groups, the scope at which part of the column data path is shared.

Two accesses to different banks in the same group are spaced differently from two accesses to different banks in different groups — and from verified figures that difference reaches a factor of two at DDR4-3200. So “a different bank” is not one relationship but two, the choice between them is available to a scheduler, and the organisation that determines which is which differs between DDR4 and DDR5 in ways that need stating carefully.