CXL Memory Controllers

24.2 worked hard to preserve an order: a read must not pass a write to its address, completions must retire against their own tag, and hazards must be held or forwarded. This chapter is the next block along, and its whole job is to reorder.

A DRAM scheduler that serves requests in the order they arrive delivers a third of the bandwidth the pins are capable of. Reordering for row locality and bank spread is not an optimisation; it is the difference between a memory controller and a queue. And it must never move a request past one the pipeline ordered it behind — which is section 7, and it is the only place in this chapter where a performance mechanism can produce wrong data.

Everything else here is bandwidth, and there is much less of it than the pin count suggests.

1. The Engineering Problem — Peak Is A Pin Count

A DRAM access costs one, two or three timings. An open row reads in 15 cycles; the wrong row open costs 45 — a three-times spread on the same device. Section 5.

Banks work in parallel only if the addresses reach them. Sixteen requests over eight banks is 8 cycles; over one bank it is 64. Section 6.

Refresh steals bandwidth on a schedule. Sixteen refreshes of 350 cycles in 64,000 is 8% gone before a single request is served. Section 8.

Turning the bus around costs idle cycles. Four direction changes at 8 cycles each against 64 cycles of data is 66% efficiency. Section 9.

And the three compound. Eight, twelve and ten percent lost is 560 Gbps delivered from an 800 Gbps part. Section 13.

This chapter against 24.2, stated precisely. That one owns preserving an order. This one owns breaking it safely for throughput — which is why section 7 exists at all, and why it is the only model here whose broken build corrupts rather than slows.

2. The One-Sentence Model

A memory controller is correct when the DRAM answers, addresses reach more than one bank, reordering never breaks a dependency the pipeline established, refresh is in the bandwidth plan, reads and writes are batched to limit turnaround, and the check bits cost capacity in the plan — and every defect below is a controller that answers correctly at a fraction of the rate it was bought for.

3. What This Chapter Owns

Deferred:

4. Teaching-Model Boundary

Every model is a small synchronous block isolating one cost. A real memory controller is a bank state machine per bank, a command scheduler, a refresh engine, a write buffer, an ECC encoder and decoder and a PHY training block, and none of that is reproduced. What is reproduced is the arithmetic each performs, and the shape of the mistake when it does not.

Three simplifications are worth stating. Section 5 takes the row state as an input rather than tracking it. Section 6 takes the number of distinct banks touched rather than deriving it from addresses. Section 12 models a queue's hit rate as linear in its depth up to saturation, which is a first-order approximation of a strongly workload-dependent curve. In each case the conclusion is the same and the model is abbreviated.

Each model is built twice — a correct build and a broken build selected by a parameter. Every broken build here is a datasheet. Every access hits. Every bank is available. Refresh is free. The bus never turns around. Check bits cost nothing. Each is what the peak-bandwidth number on a part's front page assumes, and none of them is a lie — they are the conditions under which that number is achievable, stated as though they always hold.

Figure 1 — Three outcomes on one device, a factor of three apart, and the access itself is identical in all three. Which one it gets is decided by what the scheduler did with the previous access to that bank — which is why sections 7 and 12 are about the scheduler rather than about the memory.

5. RTL 1 — An Access Costs One, Two Or Three Timings

// RTL 1 - the row buffer. A DRAM access costs one, two or three timings
// depending on what is already open in the bank it lands in.
module row_buffer #(parameter int ASSUME_ALL_HITS = 0) (
  input  logic clk, rst_n,
  input  logic       access,
  input  logic       row_open, row_match,
  input  logic [7:0] t_cl, t_rcd, t_rp,
  output logic [7:0] latency_cycles, best_latency,
  output logic       hit, conflict, miss, at_best,
  output logic [7:0] n_accesses, n_degraded,
  output logic       hit_assumed_err
);
  logic [15:0] l_q;
  assign hit      = row_open && row_match;
  assign conflict = row_open && !row_match;
  assign miss     = !row_open;
  assign best_latency = t_cl;
  // A hit reads the open row; a miss opens one first; a conflict closes the
  // open row before opening the right one.
  assign l_q = (ASSUME_ALL_HITS != 0) ? {8'd0, t_cl}
             : (hit ? {8'd0, t_cl}
               : (miss ? ({8'd0, t_rcd} + {8'd0, t_cl})
                 : ({8'd0, t_rp} + {8'd0, t_rcd} + {8'd0, t_cl})));
  assign latency_cycles = (l_q > 16'd255) ? 8'hFF : l_q[7:0];
  assign at_best = (latency_cycles <= best_latency);
  // An access that did not hit the open row, costed as if it had.
  assign hit_assumed_err = access && !hit && (t_rcd != 8'd0)
                           && (latency_cycles == best_latency);

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_accesses <= 8'd0; n_degraded <= 8'd0;
    end else if (access) begin
      n_accesses <= n_accesses + 8'd1;
      if (!at_best) n_degraded <= n_degraded + 8'd1;
    end
  end
endmodule

Six accesses. Column, row-to-column and precharge times of fifteen cycles each.

Three degraded when the row state is modelled; none when every access is assumed a hit.

Row four is the case that catches people writing this model. A row number matching the request means nothing if the bank has no row open — the match is only meaningful against something that is actually open, and a hit test that checks the match alone reports hits on a closed bank. Section 17's mutation set injects exactly that.

Row five is where the all-hits model is right, and it is not hypothetical: a part whose activate and precharge times are negligible against its column time really does make every access cost the same. The error is assuming that shape rather than measuring it.

Three times is the whole reason a scheduler exists. A stream of accesses that alternates between two rows in one bank costs 45 cycles each; the same accesses reordered so that same-row requests are grouped cost 15, and nothing about the memory changed.

6. RTL 2 — Banks Work In Parallel Only If The Addresses Reach Them

// RTL 2 - bank parallelism. Banks serve requests concurrently only to the
// extent the addresses land in different ones.
module bank_parallelism #(parameter int ONE_BANK = 0) (
  input  logic clk, rst_n,
  input  logic       schedule,
  input  logic [7:0] banks, distinct_banks, requests, bank_busy_cycles,
  output logic [7:0] concurrency, batches, service_cycles,
  output logic       efficient,
  output logic [7:0] n_schedules, n_serial,
  output logic       parallelism_ignored_err
);
  logic [15:0] c_q, b_q, s_q;
  // Concurrency is the smaller of the banks present and the banks touched.
  assign c_q = (ONE_BANK != 0) ? 16'd1
             : ((distinct_banks > banks) ? {8'd0, banks} : {8'd0, distinct_banks});
  assign concurrency = (c_q > 16'd255) ? 8'hFF : c_q[7:0];
  assign b_q = (concurrency == 8'd0) ? 16'd0
             : (({8'd0, requests} + {8'd0, concurrency} - 16'd1)
                / {8'd0, concurrency});
  assign batches = (b_q > 16'd255) ? 8'hFF : b_q[7:0];
  assign s_q = {8'd0, batches} * {8'd0, bank_busy_cycles};
  assign service_cycles = (s_q > 16'd255) ? 8'hFF : s_q[7:0];
  assign efficient = (service_cycles <= 8'd16);
  // Several banks touched, served one at a time.
  assign parallelism_ignored_err = schedule && (distinct_banks > 8'd1)
                                   && (banks > 8'd1) && (concurrency == 8'd1);

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_schedules <= 8'd0; n_serial <= 8'd0;
    end else if (schedule) begin
      n_schedules <= n_schedules + 8'd1;
      if (!efficient) n_serial <= n_serial + 8'd1;
    end
  end
endmodule

Eight schedules. Sixteen requests, four cycles of bank occupancy each.

Two served serially when the banks are used; seven when they are not.

Row three is the workload the scheduler cannot help. Sixteen requests into one bank is sixteen serialised accesses however clever the controller is — the parallelism is in the address stream, not in the scheduler — which is why section 11's interleaving is the model that actually fixes it.

Row eight is a device rather than a workload, and the distinction matters for the check: a single-bank part serving requests to eight logically distinct banks is not ignoring parallelism it has, it has none. parallelism_ignored_err requires more than one bank to exist, and section 17 records that this case was demanded by a surviving mutation.

Row four is the address decode disagreeing with the device. More distinct banks named than the part contains means the bank-select bits are wrong; the concurrency caps at what exists, which is correct behaviour on top of a configuration error.

Both traces show a bus that is busy every cycle, which is the trap in reading a utilisation counter on a memory controller. Busy is not the same as productive, and the difference between eight requests and one in the same four cycles is invisible to anything counting occupancy.

7. RTL 3 — Reorder For Locality, Never Past A Dependency

// RTL 3 - scheduling against ordering. Reordering for row locality raises
// throughput and must never move a request past one it is ordered against.
module scheduler_reorder #(parameter int REORDER_FREELY = 0) (
  input  logic clk, rst_n,
  input  logic       pick,
  input  logic       candidate_is_row_hit, ordered_against_head,
  input  logic [7:0] head_latency, hit_latency,
  output logic [7:0] chosen_latency, saved_cycles,
  output logic       picked_candidate, order_broken,
  output logic [7:0] n_picks, n_reordered,
  output logic       ordering_violated_err
);
  // A candidate may be promoted only if nothing orders it behind the head.
  assign picked_candidate = (REORDER_FREELY != 0)
                            ? candidate_is_row_hit
                            : (candidate_is_row_hit && !ordered_against_head);
  assign chosen_latency = picked_candidate ? hit_latency : head_latency;
  assign saved_cycles = (head_latency > chosen_latency)
                        ? (head_latency - chosen_latency) : 8'd0;
  assign order_broken = picked_candidate && ordered_against_head;
  // A request promoted past one it was ordered against.
  assign ordering_violated_err = pick && order_broken;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_picks <= 8'd0; n_reordered <= 8'd0;
    end else if (pick) begin
      n_picks <= n_picks + 8'd1;
      if (picked_candidate) n_reordered <= n_reordered + 8'd1;
    end
  end
endmodule

Six scheduling picks. A head at 45 cycles and a row-hit candidate at 15.

Three promotions with ordering respected; four without it.

Row two is the only defect in this chapter that produces wrong data. Everything else here costs bandwidth; promoting a read past a write it depends on returns the previous value — 24.2 §10's hazard, arriving from the other direction. The pipeline held the read and the scheduler let it past.

Row five is worth stating because it bounds the mechanism's value. Promoting a candidate when the head is also a row hit saves nothing — the gain comes from the difference between a hit and a conflict, not from reordering as such — so a workload with high row locality already gets very little from the scheduler.

Row six is a candidate that is slower than the head, which the model allows and floors at zero saving. It is perverse and reachable: a scheduler choosing on row-hit status alone can promote a request that happens to have a longer latency, and a saving computed without the floor wraps to a large positive.

8. RTL 4 — Refresh Steals Bandwidth On A Schedule

// RTL 4 - refresh. DRAM must be refreshed on a schedule, and the controller
// cannot serve a rank while it is refreshing.
module refresh_overhead #(parameter int IGNORE_REFRESH = 0) (
  input  logic clk, rst_n,
  input  logic        account,
  input  logic [15:0] refresh_interval, refresh_cycles, window_cycles,
  output logic [15:0] refreshes, lost_cycles, usable_cycles, overhead_pct,
  output logic        acceptable,
  output logic [7:0]  n_accounts, n_costly,
  output logic        refresh_ignored_err
);
  logic [31:0] r_q, l_q, o_q;
  assign r_q = (refresh_interval == 16'd0) ? 32'd0
             : ({16'd0, window_cycles} / {16'd0, refresh_interval});
  assign refreshes = (r_q > 32'd65535) ? 16'hFFFF : r_q[15:0];
  // Every refresh takes the rank away from the request stream.
  assign l_q = (IGNORE_REFRESH != 0) ? 32'd0
             : ({16'd0, refreshes} * {16'd0, refresh_cycles});
  assign lost_cycles = (l_q > 32'd65535) ? 16'hFFFF : l_q[15:0];
  assign usable_cycles = (window_cycles > lost_cycles)
                         ? (window_cycles - lost_cycles) : 16'd0;
  assign o_q = (window_cycles == 16'd0) ? 32'd0
             : (({16'd0, lost_cycles} * 32'd100) / {16'd0, window_cycles});
  assign overhead_pct = (o_q > 32'd65535) ? 16'hFFFF : o_q[15:0];
  assign acceptable = (overhead_pct <= 16'd5);
  // Refreshes that happened and cost nothing.
  assign refresh_ignored_err = account && (refreshes != 16'd0)
                               && (refresh_cycles != 16'd0) && (lost_cycles == 16'd0);

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_accounts <= 8'd0; n_costly <= 8'd0;
    end else if (account) begin
      n_accounts <= n_accounts + 8'd1;
      if (!acceptable) n_costly <= n_costly + 8'd1;
    end
  end
endmodule

Seven accountings. A 64,000-cycle window.

Three costly when refresh is counted; none when it is not.

Eight percent is gone before the first request is served, and it is not recoverable by any scheduling decision — the rank is unavailable while it refreshes. The only levers are the part's refresh duration and whether the controller can refresh banks individually, both of which are chosen at part selection rather than at design time.

Row six is why denser parts are not free. A larger array takes longer to refresh, so the same interval costs seventeen percent instead of eight — the capacity gain and the bandwidth loss come from the same physics, and a capacity plan that does not carry the refresh number is optimistic by the difference.

Row seven is a part refreshing so often it never serves a request, driven because the usable-cycle floor exists for it. A mis-programmed interval register produces exactly this, and the floor reports nothing usable rather than wrapping to a plausible-looking figure.

9. RTL 5 — Turning The Bus Around Costs Idle Cycles

// RTL 5 - bus turnaround. Switching the data bus between reads and writes costs
// idle cycles, so the number of switches matters more than the number of writes.
module bus_turnaround #(parameter int IGNORE_TURNAROUND = 0) (
  input  logic clk, rst_n,
  input  logic       account,
  input  logic [7:0] switches, turnaround_cycles, transfers, transfer_cycles,
  output logic [7:0] data_cycles, lost_cycles, total_cycles, efficiency_pct,
  output logic       acceptable,
  output logic [7:0] n_accounts, n_costly,
  output logic       turnaround_ignored_err
);
  logic [15:0] d_q, l_q, t_q, e_q;
  assign d_q = {8'd0, transfers} * {8'd0, transfer_cycles};
  assign data_cycles = (d_q > 16'd255) ? 8'hFF : d_q[7:0];
  // Each direction change idles the bus for a turnaround.
  assign l_q = (IGNORE_TURNAROUND != 0) ? 16'd0
             : ({8'd0, switches} * {8'd0, turnaround_cycles});
  assign lost_cycles = (l_q > 16'd255) ? 8'hFF : l_q[7:0];
  assign t_q = {8'd0, data_cycles} + {8'd0, lost_cycles};
  assign total_cycles = (t_q > 16'd255) ? 8'hFF : t_q[7:0];
  assign e_q = (total_cycles == 8'd0) ? 16'd0
             : (({8'd0, data_cycles} * 16'd100) / {8'd0, total_cycles});
  assign efficiency_pct = (e_q > 16'd255) ? 8'hFF : e_q[7:0];
  assign acceptable = (efficiency_pct >= 8'd80);
  // Direction changes that happened and cost nothing.
  assign turnaround_ignored_err = account && (switches != 8'd0)
                                  && (turnaround_cycles != 8'd0)
                                  && (lost_cycles == 8'd0);

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_accounts <= 8'd0; n_costly <= 8'd0;
    end else if (account) begin
      n_accounts <= n_accounts + 8'd1;
      if (!acceptable) n_costly <= n_costly + 8'd1;
    end
  end
endmodule

Six accountings. Sixteen four-cycle transfers — 64 cycles of data — with an 8-cycle turnaround.

Two costly when turnaround is counted; one when it is not.

The count of switches matters and the count of writes does not. Sixteen reads followed by sixteen writes is one switch; alternating them is thirty-two, with identical traffic and identical data. That is entirely a scheduling decision, and it is the cheapest bandwidth in the chapter to recover.

Row six is the degenerate case that shows what a write buffer prevents. Turnarounds with no data between them is a bus that spends every cycle changing direction — a controller that services one read and one write alternately, with no batching at all — and it is what a naive first-come-first-served scheduler produces on a mixed workload.

Row five is the batching target. Two switches over sixteen transfers is exactly 80% efficiency, which sets the batch size: eight transfers per direction. That is a write-buffer depth derived from a bandwidth requirement, which is the direction the calculation should run.

10. RTL 6 — Error Correction Costs Capacity Always And Latency Sometimes

// RTL 6 - error correction. ECC costs capacity always and latency only when it
// has something to correct, and both are real.
module ecc_overhead #(parameter int IGNORE_ECC = 0) (
  input  logic clk, rst_n,
  input  logic       account,
  input  logic [7:0] data_bits, check_bits, base_latency, correct_latency,
  input  logic       correcting,
  output logic [7:0] total_bits, capacity_cost_pct, latency_cycles,
  output logic       acceptable,
  output logic [7:0] n_accounts, n_costly,
  output logic       ecc_ignored_err
);
  logic [15:0] t_q, c_q;
  logic [7:0]  eff_check;
  // A device without an error-correction budget carries no check bits and
  // corrects nothing.
  assign eff_check = (IGNORE_ECC != 0) ? 8'd0 : check_bits;
  assign t_q = {8'd0, data_bits} + {8'd0, eff_check};
  assign total_bits = (t_q > 16'd255) ? 8'hFF : t_q[7:0];
  assign c_q = (total_bits == 8'd0) ? 16'd0
             : (({8'd0, eff_check} * 16'd100) / {8'd0, total_bits});
  assign capacity_cost_pct = (c_q > 16'd255) ? 8'hFF : c_q[7:0];
  assign latency_cycles = ((IGNORE_ECC != 0) || !correcting)
                          ? base_latency : correct_latency;
  assign acceptable = (capacity_cost_pct <= 8'd15);
  // Check bits configured and neither stored nor counted.
  assign ecc_ignored_err = account && (check_bits != 8'd0) && (eff_check == 8'd0);

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_accounts <= 8'd0; n_costly <= 8'd0;
    end else if (account) begin
      n_accounts <= n_accounts + 8'd1;
      if (!acceptable) n_costly <= n_costly + 8'd1;
    end
  end
endmodule

Six accountings. Sixty-four data bits, a base latency of fifteen and a correction latency of twenty.

One too costly when the check bits are stored; none when they are not.

Capacity is paid on every bit and latency only on a correction, and the two are budgeted in different places. Eleven percent of the array is check bits before the device stores a byte of user data — a figure that belongs in the capacity plan alongside the part's density, and that a datasheet's raw capacity does not carry.

Row two is the latency that only shows up in the tail. Corrections are rare, so a mean latency measurement will not see them; a ninety-ninth-percentile measurement will, which is why section 14 asks for a histogram rather than an average.

Row three is a stronger code, and the trade is explicit. Sixteen check bits correct more and cost twenty percent of the array — the reliability decision and the capacity decision are the same decision, and section 22's review question asks for the code before it asks for the capacity.

11. RTL 7 — Which Address Bits Select The Bank

// RTL 7 - address interleaving. Which address bits select the bank decides how
// a stride spreads across banks, and a bad choice puts every access in one.
module interleave_spread #(parameter int NO_INTERLEAVE = 0) (
  input  logic clk, rst_n,
  input  logic       map,
  input  logic [7:0] banks, stride_lines, accesses,
  output logic [7:0] banks_touched, spread_pct,
  output logic       spread_well,
  output logic [7:0] n_maps, n_clustered,
  output logic       interleave_ignored_err
);
  logic [15:0] b_q, s_q;
  logic [7:0]  reach;
  // A stride that shares a factor with the bank count revisits the same banks.
  assign reach = (stride_lines == 8'd0) ? 8'd1
               : ((stride_lines >= banks) ? 8'd1 : banks);
  assign b_q = (NO_INTERLEAVE != 0) ? 16'd1
             : ((accesses < {8'd0, reach}) ? {8'd0, accesses} : {8'd0, reach});
  assign banks_touched = (b_q > 16'd255) ? 8'hFF : b_q[7:0];
  assign s_q = (banks == 8'd0) ? 16'd0
             : (({8'd0, banks_touched} * 16'd100) / {8'd0, banks});
  assign spread_pct = (s_q > 16'd255) ? 8'hFF : s_q[7:0];
  assign spread_well = (spread_pct >= 8'd50);
  // Several banks available and every access landing in one.
  assign interleave_ignored_err = map && (banks > 8'd1) && (accesses > 8'd1)
                                  && (stride_lines != 8'd0)
                                  && (stride_lines < banks) && (banks_touched == 8'd1);

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_maps <= 8'd0; n_clustered <= 8'd0;
    end else if (map) begin
      n_maps <= n_maps + 8'd1;
      if (!spread_well) n_clustered <= n_clustered + 8'd1;
    end
  end
endmodule

Seven mappings. Sixteen banks.

Four clustered when the map interleaves; six when it does not.

Row two is the classic and it is entirely a bit-selection choice. A stride equal to the bank count means every access has the same bank-select bits — sixteen banks and one of them working — and the fix is choosing different address bits, not a better scheduler.

Row four is the access pattern rather than the map. Every access to the same line touches one bank because there is one line; that is the workload, and interleave_ignored_err exempts it, which is why the check requires a non-zero stride. Section 17 records that this exemption was needed before the mutation set could be built.

And row three is the acceptance boundary. Eight accesses across sixteen banks reach half of them — which is the most an eight-access burst can do, so "spread well" at 50% is a statement about the burst as much as about the map.

12. RTL 8 — A Deeper Queue Finds More Row Hits

// RTL 8 - the scheduler queue. A deeper queue sees more candidates, so it finds
// more row hits, and the hit rate is what the queue depth actually buys.
module queue_locality #(parameter int SHALLOW_IS_ENOUGH = 0) (
  input  logic clk, rst_n,
  input  logic       schedule,
  input  logic [7:0] queue_depth, rows_open, hit_rate_per_entry_pct,
  output logic [7:0] candidates, hit_rate_pct, miss_rate_pct,
  output logic       good_locality,
  output logic [7:0] n_schedules, n_poor,
  output logic       depth_ignored_err
);
  logic [15:0] c_q, h_q;
  // A scheduler can only choose among the entries it can see.
  assign c_q = (SHALLOW_IS_ENOUGH != 0) ? 16'd1 : {8'd0, queue_depth};
  assign candidates = (c_q > 16'd255) ? 8'hFF : c_q[7:0];
  // Each visible entry is another chance to find an open row, up to saturation.
  assign h_q = (({8'd0, candidates} * {8'd0, hit_rate_per_entry_pct}) > 16'd100)
               ? 16'd100
               : ({8'd0, candidates} * {8'd0, hit_rate_per_entry_pct});
  assign hit_rate_pct = (h_q > 16'd255) ? 8'hFF : h_q[7:0];
  assign miss_rate_pct = 8'd100 - hit_rate_pct;
  assign good_locality = (hit_rate_pct >= 8'd60);
  // A deep queue scheduled as if it could see one entry.
  assign depth_ignored_err = schedule && (queue_depth > 8'd1)
                             && (candidates == 8'd1);

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_schedules <= 8'd0; n_poor <= 8'd0;
    end else if (schedule) begin
      n_schedules <= n_schedules + 8'd1;
      if (!good_locality) n_poor <= n_poor + 8'd1;
    end
  end
endmodule

Six schedules. A fifteen percent chance per visible entry of finding an open row.

Three with poor locality when the queue is used; all six when it is not.

A scheduler can only choose among what it can see, so queue depth buys row hits directly. Going from one visible entry to four takes the hit rate from 15% to 60% — and section 5 says a hit costs 15 cycles against a conflict's 45, so that depth change is worth roughly a factor of two in effective latency.

Row one is the saturation, and it is the sizing answer. Beyond about seven entries the hit rate is capped by the workload's locality rather than by the queue — more depth buys nothing and costs area, which is the calculation 24.5 will make in general.

The per-entry chance is a workload property and it is the input nobody has. Fifteen percent against twenty-five percent is the difference between needing seven entries and needing four; it comes from a trace, not from a specification, which is why section 14 asks for the hit rate to be measured rather than assumed.

13. RTL 9 — What The Controller Delivers

// RTL 9 - what the controller delivers. Peak bandwidth is a pin count; delivered
// bandwidth is what is left after every cycle the controller could not use.
module controller_efficiency #(parameter int PEAK_BANDWIDTH = 0) (
  input  logic clk, rst_n,
  input  logic        assess,
  input  logic [15:0] peak_gbps,
  input  logic [7:0]  refresh_loss_pct, turnaround_loss_pct, row_miss_loss_pct,
  output logic [7:0]  total_loss_pct, delivered_pct,
  output logic [15:0] delivered_gbps,
  output logic        acceptable,
  output logic [7:0]  n_assessments, n_poor,
  output logic        losses_ignored_err
);
  logic [15:0] t_q, d_q;
  // The three losses compound onto the same bus; a peak figure carries none.
  assign t_q = (PEAK_BANDWIDTH != 0) ? 16'd0
             : ({8'd0, refresh_loss_pct} + {8'd0, turnaround_loss_pct}
                + {8'd0, row_miss_loss_pct});
  assign total_loss_pct = (t_q > 16'd100) ? 8'd100 : t_q[7:0];
  assign delivered_pct = 8'd100 - total_loss_pct;
  assign d_q = ({16'd0, delivered_pct} * {16'd0, peak_gbps}) / 32'd100;
  assign delivered_gbps = (d_q > 16'hFFFF) ? 16'hFFFF : d_q[15:0];
  assign acceptable = (delivered_pct >= 8'd70);
  // Losses that were measured and not subtracted.
  assign losses_ignored_err = assess
                              && ((refresh_loss_pct + turnaround_loss_pct
                                   + row_miss_loss_pct) != 8'd0)
                              && (total_loss_pct == 8'd0);

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_assessments <= 8'd0; n_poor <= 8'd0;
    end else if (assess) begin
      n_assessments <= n_assessments + 8'd1;
      if (!acceptable) n_poor <= n_poor + 8'd1;
    end
  end
endmodule

Six assessments. An 800 Gbps part.

Two poor when the losses are counted; none when they are not.

Five hundred and sixty of eight hundred is what the part actually does, and none of the three losses is a defect: refresh is physics, turnaround is a bus property, and row misses are the workload meeting the scheduler. A peak figure is achievable only under conditions that never all hold at once.

Row three is the difference between a tuned controller and an untuned one. Twenty percent turnaround and twenty percent row-miss loss instead of twelve and ten is 416 Gbps against 560 — a 26% difference from scheduling policy alone, on identical silicon and identical memory.

Row four is a clamp rather than an arithmetic nicety. Losses summing past a hundred percent means the measurements overlap or are wrong; reporting nothing delivered is the honest answer, and a model without the clamp would produce a negative that wraps into a large positive.

Figure 3 — Thirty percent gone, and twenty-two of it is the scheduler's rather than the memory's. That is the argument for every model in sections 7, 9, 11 and 12: refresh is fixed, and the rest is design.

14. RTL 10 — A CXL Memory Controller Assembled

// RTL 10 - a CXL memory controller assembled. Everything that must hold before
// DRAM that answers is DRAM that answers at the rate it was bought for.
module memory_controller_model #(parameter int DRAM_RESPONDS = 0) (
  input  logic clk, rst_n,
  input  logic       evaluate,
  input  logic       dram_responds,       // reads return data
  input  logic       banks_spread,        // addresses reach more than one bank
  input  logic       ordering_preserved,  // reordering never breaks a dependency
  input  logic       refresh_budgeted,    // refresh is in the bandwidth plan
  input  logic       turnaround_grouped,  // reads and writes are batched
  input  logic       ecc_accounted,       // check bits cost capacity in the plan
  output logic       correct,
  output logic [5:0] fail_mask,
  output logic [7:0] n_eval, n_correct,
  output logic       false_correct_err
);
  assign fail_mask[0] = ~dram_responds;
  assign fail_mask[1] = ~banks_spread;
  assign fail_mask[2] = ~ordering_preserved;
  assign fail_mask[3] = ~refresh_budgeted;
  assign fail_mask[4] = ~turnaround_grouped;
  assign fail_mask[5] = ~ecc_accounted;
  // The DRAM-responds build is what a memory bring-up reports.
  assign correct = (DRAM_RESPONDS != 0) ? dram_responds : (fail_mask == 6'd0);
  assign false_correct_err = evaluate && correct && (fail_mask != 6'd0);

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_eval <= 8'd0; n_correct <= 8'd0;
    end else if (evaluate) begin
      n_eval <= n_eval + 8'd1;
      if (correct) n_correct <= n_correct + 8'd1;
    end
  end
endmodule

One correct configuration of six, and four false claims.

"The DRAM answers" is what a memory bring-up reports and it is right about one of the six. Row two is the most expensive to discover late: every read returns correct data at a sixth of the expected bandwidth, because the bank-select bits were chosen from the wrong part of the address and no functional test can see it.

Figure 4 — Four causes and one exit that is not a defect. The order is deliberate: interleaving first because it is the largest single factor and the cheapest to check, then queue depth, then the part, then batching — which is roughly the order of the bandwidth each can recover.

15. Quantitative Reasoning

Row buffer. A hit is 15 cycles, a miss 30 and a conflict 45 — a three-times spread on identical hardware, decided by what the previous access did.

Bank parallelism. Sixteen requests over eight banks is 8 cycles; over one bank it is 64 — and both keep the bus busy every cycle.

Scheduling. Promoting a row hit past a conflicting head saves 30 cycles; promoting it past an ordering dependency returns the wrong data.

Refresh. Sixteen refreshes of 350 cycles in 64,000 is 8% gone; a denser part at 700 cycles is 17%.

Turnaround. Four direction changes at 8 cycles against 64 of data is 66% efficiency; two changes is exactly 80%.

Error correction. Eight check bits on sixty-four is 11% of the array; sixteen is 20%, and a correction costs 20 cycles against a base of 15.

Interleaving. A unit stride across sixteen banks touches all sixteen; a stride equal to the bank count touches one.

Queue depth. One visible entry gives a 15% hit rate; four gives 60%, and eight saturates.

Delivered bandwidth. Eight, twelve and ten percent lost leaves 560 Gbps of 800 — and 22 of the 30 points are the scheduler's rather than the memory's.

The assembled model. Six properties, six configurations, one correct. The DRAM-responds milestone reported five.

16. Assertions

Every check is an explicit comparison against an exact value. Icarus Verilog 13.0 has no concurrent assertion support, so each is a procedural comparison against 1'b1, and every one is an equality.

Every inclusive threshold is driven at exactly equal, every ceiling both on and off its boundary, and every floor past it — the three rules this batch inherits.

Row buffer. A matching row in a closed bank is asserted a miss, and a part whose activate and precharge times are instant is asserted as not an assumed hit.

chk(wGh == 1'b0,  "a match means nothing with no row open");
chk(wGl == 8'd30, "at the open-and-read cost");

Bank parallelism. A service time of exactly sixteen cycles is constructed from five banks, more distinct banks than the device holds is asserted capped, and a single-bank part is asserted as not ignored parallelism.

Scheduling. A promotion that saves nothing and one whose candidate is slower than the head are both driven, and the second is asserted to floor at zero saving.

Refresh. An overhead of exactly five percent is constructed, and a part refreshing so often it never serves a request is asserted to leave nothing usable rather than wrapping.

chk(fGl == 16'hFFFF, "saturating the reported loss");
chk(fGu == 16'd0,    "which leaves nothing usable, not a wrapped count");

Turnaround. Efficiency of exactly 80% is constructed from two switches, and turnarounds with no data between them are driven.

Error correction. A capacity cost of exactly fifteen percent is constructed from twelve check bits, and both the correcting and non-correcting latencies are driven.

Interleaving. A spread of exactly fifty percent is driven, a stride of zero is asserted as not an ignored interleave, and a device with no bank count is driven.

Queue depth. A hit rate of exactly sixty percent is constructed from four entries, and a one-deep queue is asserted as one the shallow model gets right.

Delivered bandwidth. Seventy percent delivered is driven at the acceptance threshold, and losses summing past a hundred are asserted clamped.

The assembled model. Every fail mask is asserted as an exact six-bit value, and each of the six bits is driven false alone.

Totals: 297 checks across two testbenches, 156 on the front five models and 141 on the back five, all passing on the unmutated sources.

17. Mutation Testing

Sixty-four mutations were injected one at a time. 64 injected, 64 killed, after three survivors.

All three survivors were the class this batch has now met in every chapter: a guard for a configuration outside the model's own premise.

Survivor 1 — a single-bank part. Section 6's banks > 1 term survived because every case had several banks. A part with one bank serving requests to several logically distinct banks is not ignoring parallelism it has — and driving it kills the mutation and makes the check's meaning precise.

Survivor 2 — a part that refreshes more than it serves. Section 8's usable-cycle floor needed the lost cycles to exceed the window, which needs a refresh interval short enough that the refreshes never stop. A mis-programmed interval register produces exactly that, and without the floor the subtraction wraps.

Survivor 3 — a device with no bank count. Section 11's spread guard needed a bank count of zero, which is the pre-enumeration state of every controller. Without it the division is undefined.

Two guards were proved dominated and deleted before mutating, following the same discipline: section 9's saving floor in 24.2, and the three range-size guards in that chapter's decoder. This chapter kept every guard it wrote, which is the first time in the batch that has happened — and the difference is that all three of these have inputs the arithmetic genuinely cannot exclude.

The complete set was re-run after every stimulus change, per standing discipline, and all sixty-four held.

18. Verification Strategy

What a testbench for a memory controller must cover.

Drive the device that is not the model's premise. A single-bank part in a bank-parallelism model. A zero-bank device in an interleaving model. A part that refreshes constantly in a refresh model. All three of this chapter's survivors were this, and the pattern is now reliable enough to check for before mutating.

Distinguish the workload from the design. Section 6's row three and section 11's row four are both cases where the outcome is bad and nothing is wrong — one bank touched because the workload touches one bank, and one bank touched because every access is to the same line. A check that fires on those would report a defect on correct hardware, and each needed an explicit exemption.

Drive every conditional's branches separately. Section 5's hit test has two terms and each was mutated alone; a hit test that checks the open flag alone and one that checks the match alone are both wrong in different ways, and only a case with a match against a closed bank separates them.

The cases where the datasheet is right. An access to an open matching row. Every request in one bank on a one-bank part. A head that is already a hit. A part with no refresh requirement. A stream with no direction changes. A device with no error correction. Six cases across nine models, each exempted explicitly.

Counters as a second signature. Ten models, ten pairs of totals, differing in all ten — three degraded against none, two serial against seven, three costly against none, four clustered against six.

What a real controller needs that these models do not have. Bank state over time. Section 5 takes the row state as an input; a real controller derives it from what it scheduled, which makes the row-hit rate a consequence of the scheduler rather than an input to it — and that feedback is the whole difficulty of memory scheduling. Section 26 exercises 1 and 8 are where it comes back.

19. Synthesis and Implementation Reality

Section 5's row state is a small state machine per bank, and the number of them is the bank count — sixteen or thirty-two small machines, which is cheap. What is not cheap is the scheduler that reads all of them every cycle.

Section 7's ordering constraint is a dependency matrix between queue entries, and it grows as the square of the queue depth. That is the real cost of section 12's deeper queue, and it is why "just make the queue bigger" stops being free quickly.

Section 8's refresh engine competes with the request scheduler for command slots, and modern parts allow per-bank refresh, which turns section 8's whole-rank stall into a bank-level one. That changes the model's arithmetic and not its conclusion.

Section 9's turnaround is a property of the bus and the memory, and the controller's only lever is the write buffer: deeper batching means fewer switches and more write latency, which is a direct trade against read latency for anything that must be ordered behind the writes.

Section 10's ECC sits in the datapath on every access, so its encode and decode latencies are on the critical path — and 24.2 §12's poison bit is what carries an uncorrectable result out of it.

Section 11's interleaving is a wire permutation and costs nothing in gates. It is the highest-value, lowest-cost decision in the chapter, and it is made once, early, usually without a trace to justify it.

20. Silicon Observability

"Cycles the bus was busy against requests retired" is the counter figure 2 argues for. A memory controller running one bank at full occupancy and one running eight look identical on a utilisation gauge — both are busy every cycle — and only the retirement rate separates them. A controller reporting 100% utilisation and a third of its bandwidth is the normal state of a badly interleaved device.

21. Debug Lab

Symptom. A CXL memory device is measured at just over a third of its expected bandwidth. The DRAM passes every functional test, ECC reports clean, latency under a single-threaded load is exactly as specified, and the memory controller's utilisation counter reads 98%.

Step 1 — is the controller idle or busy? Bus busy: 98%. The controller is not waiting for requests, which eliminates the host side and the transaction pipeline and points at what the controller is doing with the cycles it has.

Step 2 — is it retiring anything? Requests retired against cycles busy: one request per 45 cycles. Section 5's conflict latency exactly. Busy and unproductive, which is the distinction figure 2 exists for.

Step 3 — why is every access a conflict? Row hits, misses and conflicts: 93% conflicts. Not misses — conflicts, which means a row is open and it is the wrong one, every time.

Step 4 — where are the accesses landing? Accesses per bank: one bank has 94% of them. Section 11. The bank-select bits were taken from address bits that this workload's stride does not vary — a 4 KB stride against a bank-select field below bit 12 — so every access in the stream has identical bank bits.

Step 5 — confirm the mechanism. One bank, sequential accesses to different rows within it: every access closes the open row and opens another. That is section 5's 45-cycle conflict on section 6's single active bank, and the two together are the entire deficit.

Step 6 — check the rest before fixing. Refresh at 8%, turnaround at 11%, queue occupancy at its maximum. All three are within budget — the scheduler is working correctly and has nothing to work with, because the address stream gives it one bank.

The finding. One decision, made once, in a wire permutation: the bank-select bits were chosen above the stride's varying bits rather than below them. Everything downstream — the row state machines, the scheduler, the queue depth — is correct and powerless.

The fix. Move the bank-select field below bit 12 so the stride varies it. The change is a permutation of wires and costs no gates, and it takes the device from one active bank to sixteen. Expected recovery: section 6's row one against row three, which is the factor of eight the measurement is missing.

What made this hard. Every counter that existed read healthy: utilisation 98%, ECC clean, latency to specification, refresh and turnaround within budget. The two counters that would have found it in an hour — conflicts separately from misses, and accesses per bank as a histogram — are both counters that only matter once, and are therefore the first two to be cut.

22. Design Review

1. Which address bits select the bank, and against which workload's stride? The highest-value, lowest-cost decision here. Section 11.

2. Are row hits, misses and conflicts counted separately? They have different causes and different fixes. Section 5.

3. How deep is the scheduler queue, and what row-hit rate does that buy on a real trace? The per-entry hit rate comes from a trace, not a specification. Section 12.

4. Can the scheduler promote a request past one it is ordered against? The only wrong-data defect in this chapter. Section 7.

5. What is the refresh overhead as a fraction, and does the plan carry it? Eight percent before anything is served, seventeen on a denser part. Section 8.

6. How many bus direction changes per window, and how deep is the write buffer? The count of switches, not of writes. Section 9.

7. What fraction of the array is check bits, and is it in the capacity figure? Eleven percent before a byte of user data. Section 10.

8. Is there a counter for requests retired as well as for cycles busy? A badly interleaved controller reads 98% utilised. Figure 2 and section 20.

9. What are the three losses separately, and which are the scheduler's? Twenty-two of thirty points are recoverable by design. Section 13.

10. Which of the six properties does "the DRAM answers" imply? Section 14 exists because the answer is the first one only.

23. How This Appears In Real Engineering

A memory-controller designer owns sections 5, 7 and 12 as one problem: the row-hit rate is a consequence of the scheduler and an input to it, which makes memory scheduling a feedback problem rather than a selection one. That is why the queue depth, the dependency matrix and the row-state machines are all sized together.

A system architect owns section 11 and usually makes it in an afternoon. Which address bits select the bank is a wire permutation that decides a factor of sixteen, and it is made before there is a trace to justify it — which is section 21.

A component engineer owns sections 8 and 10, and both are part-selection questions rather than design ones. Refresh duration and error-correction strength are properties of the part, and both cost bandwidth and capacity that the datasheet's headline numbers do not carry.

A performance-verification engineer meets the whole chapter as one number: delivered against peak. The counters that decompose it — conflicts against misses, accesses per bank, switches per window — are individually unglamorous and collectively the only way to attribute the gap, and section 21 is a device where every counter that existed read healthy.

24. Common Misconceptions

"The memory is 800 Gbps." At the pins. Delivered, 560. Section 13.

"The controller is 98% utilised, so it is working." Busy is not productive — that can be one request per 45 cycles. Figure 2.

"A row miss and a row conflict are both misses." One costs 30 cycles and the other 45, and they have different fixes. Section 5.

"Sixteen banks means sixteen-way parallelism." Only if the addresses reach them. Section 6.

"The scheduler can reorder freely — it is just memory." Not past a dependency the pipeline established. Section 7.

"Refresh is a small overhead." Eight percent, and seventeen on a denser part. Section 8.

"We do more reads than writes, so turnaround is fine." It is the count of switches, not the ratio. Section 9.

"The device has 64 GB." Before eleven percent of check bits. Section 10.

"A deeper queue is always better." Until the hit rate saturates, after which it is area. Section 12.

"The DRAM answers." One property of six. Section 14.

25. Interview Reasoning

Q. A memory controller reads 98% utilised and delivers a third of its bandwidth. Explain.

Busy is not productive. A controller serving one bank with a 45-cycle row conflict on every access is busy every cycle and retires one request per 45 — the utilisation counter cannot tell that from eight banks retiring eight requests per four cycles. The counter that separates them is requests retired against cycles busy.

Q. What is the difference between a row miss and a row conflict?

A miss finds the bank closed and costs an activate plus a read — 30 cycles. A conflict finds the wrong row open and must precharge first — 45. They have different causes: a miss means the bank was idle, a conflict means the previous access to that bank went somewhere else, which is a scheduling and interleaving question rather than a memory one.

Q. Sixteen banks and you are getting one bank's bandwidth. Where do you look?

At which address bits select the bank, against the workload's stride. If the stride does not vary the bank-select field, every access has the same bank bits — sixteen banks and one of them working. The fix is a wire permutation costing no gates, and it is the highest-value decision in a memory controller.

Q. When must a memory scheduler not reorder?

When a request is ordered behind another by something upstream — a read behind a write to the same address, or a completion ordering the transaction pipeline established. Reordering for row locality is worth 30 cycles; reordering past a dependency returns the previous value, and it is the only place in a memory controller where a performance mechanism produces wrong data.

Q. How much bandwidth does refresh cost, and can you get it back?

Around eight percent on a typical part and seventeen on a denser one, and no — the rank is unavailable while it refreshes. The levers are part selection and per-bank refresh, both of which are chosen before the controller is designed. It is the one loss in the chapter that is physics rather than design.

Q. Your bus efficiency is 66% and the workload is 80% reads. Why?

Because efficiency depends on the number of direction changes, not the read-write ratio. Sixteen reads then sixteen writes is one switch; alternating them is thirty-two — identical traffic, identical data, and a 34-point efficiency difference. The fix is a write buffer deep enough to batch, which trades write latency for bus efficiency.

26. Exercises

1. Give RTL 1 a bank state machine and derive the row state from what the scheduler issued, closing the feedback loop.

2. Extend RTL 2 to derive the distinct-bank count from an address stream and a bank-select field.

3. Combine RTL 3 and RTL 8: find the queue depth at which the dependency matrix costs more area than the row hits are worth.

4. Model per-bank refresh in RTL 4 and find how much of the eight percent it recovers.

5. Give RTL 5 a write buffer and find the depth at which the bus reaches 90% efficiency for a given read-write mix.

6. Extend RTL 6 to a code that corrects two errors and price the capacity against the failure rate it buys.

7. Drive RTL 7 with several strides at once and find the bank-select field that maximises the worst case.

8. Measure RTL 8's per-entry hit rate from a real trace instead of assuming it, and show how much the sizing changes.

9. Model section 21 end to end: a bank-select field above the stride, and the conflict rate it produces.

10. Add a seventh property to RTL 10. If it is implied by one of the six, say which; if not, give the controller it catches that the current mask calls correct.

27. Summary

24.2 preserved an order. This chapter breaks it on purpose, and the arithmetic says why: a scheduler that does not reorder delivers a third of what the pins can carry.

An access costs one, two or three timings. A hit reads in 15 cycles, a miss opens a row for 30, and a conflict closes one first for 45 — a three-times spread decided entirely by what the previous access to that bank did.

Banks work in parallel only if the addresses reach them. Sixteen requests over eight banks is 8 cycles and over one is 64 — and both keep the bus busy every single cycle, which is why a utilisation counter cannot tell them apart.

Reorder for locality, never past a dependency. Promoting a row hit saves 30 cycles; promoting it past an ordering constraint returns the previous value — the only defect in this chapter that produces wrong data rather than fewer bytes.

Refresh steals bandwidth on a schedule. Sixteen refreshes of 350 cycles in 64,000 is 8% gone before a request is served, and a denser part is 17% — physics rather than design, and the only loss here that no scheduler recovers.

Turning the bus around costs idle cycles, and it is the count of switches rather than of writes: four changes against 64 cycles of data is 66% efficiency, and two is exactly 80%.

Error correction costs capacity always and latency sometimes. Eight check bits on sixty-four is 11% of the array before a byte of user data, and a correction costs 20 cycles against 15.

Which address bits select the bank decides a factor of sixteen. A unit stride reaches all sixteen banks; a stride equal to the bank count reaches one — and the fix is a wire permutation costing no gates.

A deeper queue finds more row hits. One visible entry gives a 15% hit rate and four gives 60%, which at section 5's timings is worth roughly a factor of two in effective latency — and it saturates, so the depth has an answer.

And the three losses compound. Eight, twelve and ten percent leaves 560 Gbps of 800 — with 22 of the 30 points recoverable by the scheduler and only 8 belonging to the memory.

Three mutations survived on guards for devices outside the models' premises — a single-bank part, a part that refreshes more than it serves, and a device with no bank count. This is the first chapter in the batch that kept every guard it wrote, because all three have inputs the arithmetic genuinely cannot exclude.

The DRAM answering is one property of six. The milestone a memory bring-up reports called five of six controllers correct when one was — and section 21 is a device where every counter that existed read healthy while it delivered a third of its bandwidth.

24.4 — Device Architecture puts 24.1, 24.2 and this chapter into one device, and asks the question none of them can answer alone: where the boundaries between them should fall.