DDR5

Chapter 4.5 ended on one line of arithmetic:

   16 beats × 64 bits  =  128 bytes   -- twice a cache line.  Fatal.
   16 beats × 32 bits  =   64 bytes   -- a cache line.  UNCHANGED.

DDR5 is what you build when you take the second line seriously. It transfers 16 beats per burst, and it keeps granularity identical to DDR4's by halving the width the burst crosses — splitting each 64-bit channel into two independent 32-bit sub-channels, each with its own command path and its own bank hierarchy.

The arithmetic is exact rather than approximate, and that is what makes the design compelling rather than merely clever. Doubling the beats and halving the width cancel precisely, so the access size a requester sees does not move at all while the beat count — and therefore the sustainable transfer rate per pin — doubles.

And the split buys something the arithmetic does not show. Two independent sub-channels are two independent request streams. That is the memory-level parallelism Chapter 1.8 §7 argued matters more than peak bandwidth, arriving here as a structural property of the interface rather than as something the requester has to manufacture.

1. Why Halving the Width Is Not a Downgrade

The instinct is that a narrower channel is a worse channel. Work through what actually changes and the instinct turns out to be measuring the wrong thing.

   DDR4, one channel:
     width 64 bits · burst 8 beats
     granularity  =  8 × 64 / 8   =  64 bytes
     independent request streams  =  1

   DDR5, one channel split into two sub-channels:
     width 32 bits each · burst 16 beats
     granularity  =  16 × 32 / 8  =  64 bytes    <-- IDENTICAL
     independent request streams  =  2            <-- DOUBLED
     total pins carrying data     =  64           <-- IDENTICAL

Three quantities matter and none of them got worse. The access size is the same. The data pin count is the same. And the number of accesses that can be outstanding independently doubled.

What improved is the beat count per burst, which is the quantity that converts into transfer rate. A longer burst means the per-pin signalling rate can rise while the command rate per access stays manageable — because one command still produces one access, and that access now occupies twice as many beats. A deeper burst is how a faster pin is fed without asking the command path to keep pace.

The middle column is the point. It does not change. Everything DDR5 does to the interface is arranged so that the middle column does not change, because the moment it does, the extra bandwidth stops being useful — which is the lesson 4.5 §1 extracted from DDR4's refusal to deepen its prefetch.

2. What "Independent" Has to Mean

A sub-channel is not a relabelling of half the data pins. For two sub-channels to be independent request streams, everything an access needs must be duplicated, and the list is instructive.

Its own command and address path. This is the load-bearing one. If the two halves shared a command bus, a command issued to one would occupy the bus and delay the other, and the streams would not be independent at all — they would be one stream serving two data paths. Each DDR5 sub-channel has its own CA bus, which is why the split is an architectural change rather than a configuration.

Its own bank and bank-group hierarchy. Chapter 4.5 §2 established that the achievable rate depends on the internal column-path resource a set of banks shares. A sub-channel with its own bank groups has its own copies of that resource, so its access timing is unaffected by what the other sub-channel is doing.

Its own timing and trained state. Following DDR3's pattern, each sub-channel's path is measured and calibrated separately, because each has its own routing and its own receivers.

What the two sub-channels do share is the device and the module: the supply, the package, the thermal environment, and the physical array's process characteristics. So they are independent in scheduling and shared in physics — a distinction that matters when reasoning about power and thermal behaviour, where the two halves are not independent at all.

And the consequence for the controller is another doubling of responsibility. Two sub-channels need two schedulers, two command streams, two sets of timing state, and an address mapping that decides which sub-channel an address belongs to. A controller that treats them as one wide channel gets DDR4's parallelism out of DDR5 hardware — the same failure mode as 4.5's bank-group mapping, one level up.

3. The Other Changes, and the Pressure Behind Each

DDR5's remaining features are usually listed. Each one answers a specific pressure, and naming the pressure is what keeps this from being trivia.

Error correction moved inside the device — on-die ECC. The device itself encodes data on write and checks and corrects it on read, internally.

The pressure is manufacturability. Chapter 1.6 established that DRAM economics are dominated by bits per unit area, and Chapter 2.2 §3 established that a smaller cell stores less charge and therefore has less margin against leakage and defects. Pushing to a denser process produces more cells that are marginal, and without internal correction that puts a floor under how aggressive the process can be. On-die ECC raises the floor: the device can tolerate a rate of internal single-bit failures that would otherwise make the part unusable. It exists to make a denser array economically viable, which is a manufacturing argument rather than a reliability feature for the host's benefit.

And it has a genuinely awkward consequence. Errors the device corrects internally are, by construction, less visible to the host. A system that used to see a rising correctable-error rate as an early warning of a failing part may now see a clean interface until the internal correction is exhausted. On-die ECC improves the part and degrades the host's visibility into it — and system-level ECC remains a separate, host-visible mechanism for exactly that reason. Treating on-die ECC as a substitute for system-level correction is a category error: one protects the array's manufacturability, the other protects the system's data.

Refresh became more targeted — same-bank refresh. Chapter 2.3 established that refresh is mandatory and Chapter 2.6 that it consumes the array. The pressure is that refresh time is time the device cannot serve requests, and as capacity grows the number of rows grows, so the refresh burden grows with it. A refresh that occupies all banks blocks everything. A refresh targeted at one bank per bank group leaves the other banks in each group available. Refresh stops being a device-wide stall and becomes a partial one — which matters more the larger the device gets, and is another instance of the module's recurring move: convert a global cost into a local one so the rest can proceed.

Protection against a disturbance mechanism — refresh management. Chapter 2.2 described that activity on one row can disturb the charge stored in physically nearby rows. As cells shrink and get closer together, that coupling strengthens, and repeated, targeted activation of one row can degrade its neighbours faster than the normal refresh interval repairs them.

DDR5 adds refresh management: the controller tracks activation activity and issues a management command that directs the device to perform additional, targeted refresh where the activity warrants it. Forms exist for all-bank and for same-bank scope. This is a mitigation whose necessity is a direct consequence of the density trend Chapter 1.6 described — the physics got worse because the economics demanded it, so a mechanism was added to hold the line. Modules 15 and 25 own the command and the tracking obligation.

The receiver learned to compensate for the channel — equalisation. At DDR5 rates the channel distorts the signal in a way that depends on what was previously transmitted: energy from an earlier bit is still present when a later one is sampled. DDR5's receiver can apply decision feedback equalisation, using its own recent decisions to subtract that residual influence before judging the current bit.

This is DDR3's calibration pattern taken one step further. ZQ calibration adjusted impedance against a reference. DDR4's reference-voltage training adjusted a threshold against the measured signal. Equalisation adjusts for the channel's dynamic behaviour — not a static offset but a response that depends on the data history. The progression is from "designed", to "measured once", to "measured and adapted per bit", and it is the clearest single indicator of how much of a modern interface's correctness lives outside the digital domain.

Power regulation moved onto the module. A DDR5 module carries its own power management, taking a higher input voltage and generating the supplies the devices need locally, rather than having the board deliver each final supply.

The pressure is current delivery. A module drawing its supplies from the board must have them delivered across connectors and traces, and the resistive loss and noise on that path scale with the current and worsen as the supply voltage falls. Regulating locally means the long path carries a higher voltage at a lower current — a straightforwardly better arrangement for loss — and puts the regulation close to the load where it can respond to fast changes in demand. It also makes each module responsible for its own supply quality, which is another instance of a cost being localised rather than shared.

4. The Generation, With Layers Labelled

The two "unchanged" rows in the middle are the design. Granularity and pin count held constant while the burst doubled — that is the whole architectural claim, and everything above and below it is either a consequence or an answer to a pressure the higher rate created.

And the last row still reads "unchanged", six generations in. The array is denser, its cells hold less charge, and it needs internal correction and disturbance mitigation to survive that density — but it is still a 1T1C cell with a destructive read and a mandatory refresh. Six generations of interface evolution have been wrapped around the same storage principle.

5. RTL — Granularity as an Elaboration Invariant, Parallelism as State

Problem

Two things need expressing, and they are the chapter's two claims.

The granularity invariant is a design rule, not a runtime condition. burst × width must equal the target access size, and a parameterisation that violates it is a broken architecture — which means it should fail at elaboration, not produce a device that runs and wastes.

The parallelism is runtime state. Two sub-channels each tracking their own outstanding accesses give twice the concurrency of one, and that is worth being able to observe.

Classification

SYNTHESIZABLE RTL, with a COMPILE-TIME architectural check. The in-flight tracking and the routing are real controller logic; the granularity check is elaboration-time arithmetic that makes an architectural rule unrepresentable when violated.

What it does not model. No data, no beats, no commands, no bank state, no timing. No on-die ECC, no refresh, no refresh management, no equalisation, no power regulation — §3's entire subject matter is physical, analog or device-internal, and none of it is representable here. The block answers one question: how many accesses can be outstanding, and per which sub-channel.

Interface

req_valid with req_sc presents an access already mapped to a sub-channel. req_accept grants it. cpl_valid with cpl_sc retires one. inflight_total and max_inflight_seen are the concurrency telemetry. sc_blocked and cpl_unexpected report the two failure conditions.

How to simulate it

vlog subchannel_router.sv tb_subchannel_router.sv then vsim -c tb_subchannel_router -do "run -all"; VCS vcs -sverilog subchannel_router.sv tb_subchannel_router.sv && ./simv; Xcelium xrun -sv subchannel_router.sv tb_subchannel_router.sv.

// ─────────────────────────────────────────────────────────────────────────
// SUBCHANNEL ROUTER.
// Classification: SYNTHESIZABLE RTL + COMPILE-TIME ARCHITECTURAL CHECK.
//
// Expresses DDR5's two claims:
//
//   1. GRANULARITY IS AN INVARIANT. burst_beats x sub_channel_width must
//      equal the target access size. This is checked at ELABORATION, so a
//      parameterisation that breaks the architecture cannot elaborate --
//      which is where an architectural rule belongs. A runtime check would
//      let a broken device build and then waste bandwidth silently.
//
//   2. PARALLELISM IS STATE. Each sub-channel tracks its own outstanding
//      accesses, so N sub-channels give N times the concurrency of one.
//
// WHAT THIS DOES NOT MODEL: data, beats, commands, bank state, timing,
// on-die ECC, refresh, refresh management, equalisation, power regulation.
// Section 3's subject matter is physical, analog or device-internal and no
// register represents any of it.
// ─────────────────────────────────────────────────────────────────────────
module subchannel_router #(
  parameter int SUBCHANNELS  = 2,
  // Width of ONE sub-channel's data path, in bits.
  parameter int SUB_BITS     = 32,
  parameter int BURST_BEATS  = 16,
  // The access size the architecture must preserve, in bytes. DDR4's 64.
  parameter int TARGET_GRAN  = 64,
  // Outstanding accesses trackable per sub-channel.
  parameter int MAX_INFLIGHT = 2,

  // DERIVED. The granularity this parameterisation actually produces.
  parameter int GRAN_BYTES   = (SUB_BITS * BURST_BEATS) / 8,
  parameter int SC_W         = (SUBCHANNELS <= 1) ? 1 : $clog2(SUBCHANNELS),
  parameter int IF_W         = $clog2(MAX_INFLIGHT + 1),
  parameter int TOT_W        = $clog2((MAX_INFLIGHT * SUBCHANNELS) + 1)
) (
  input  logic              clk,
  input  logic              rst_n,

  input  logic              req_valid,
  // Which sub-channel this access belongs to. Decided UPSTREAM by the
  // address mapping -- and that mapping is what determines whether a given
  // access stride uses both sub-channels or piles into one, exactly as
  // Chapter 4.5 said of the bank-group field.
  input  logic [SC_W-1:0]   req_sc,
  output logic              req_accept,

  input  logic              cpl_valid,
  input  logic [SC_W-1:0]   cpl_sc,

  output logic [TOT_W-1:0]  inflight_total,
  output logic [TOT_W-1:0]  max_inflight_seen,

  // The requested sub-channel has no room. Distinguished from "no request"
  // so a scheduler can tell a full sub-channel from an idle one.
  output logic              sc_blocked,
  // A completion arrived for a sub-channel with nothing outstanding:
  // a protocol violation upstream. Reported, never absorbed.
  output logic              cpl_unexpected
);

  // ── COMPILE-TIME ARCHITECTURAL CHECK. This is the block's most
  //    important lines, and they generate no hardware at all.
  //
  //    The granularity rule is a property of the PARAMETERISATION, so it is
  //    enforced where parameterisation is resolved. A 16-beat burst on a
  //    64-bit sub-channel is 128 bytes and will not elaborate here -- which
  //    is precisely the design DDR5 rejected.
  if (SUBCHANNELS < 1) begin : g_sc_min
    initial $fatal(1, "subchannel_router: SUBCHANNELS must be >= 1");
  end
  if (SUB_BITS < 8) begin : g_sb_min
    initial $fatal(1, "subchannel_router: SUB_BITS must be >= 8");
  end
  if (((SUB_BITS * BURST_BEATS) % 8) != 0) begin : g_gran_bytes
    initial $fatal(1, "subchannel_router: burst x width must be a whole number of bytes");
  end
  if (GRAN_BYTES != TARGET_GRAN) begin : g_gran_invariant
    initial $fatal(1,
      "subchannel_router: granularity %0d B != target %0d B -- burst x width must be held constant",
      GRAN_BYTES, TARGET_GRAN);
  end
  if (MAX_INFLIGHT < 1) begin : g_if_min
    initial $fatal(1, "subchannel_router: MAX_INFLIGHT must be >= 1");
  end

  logic [IF_W-1:0] inflight [SUBCHANNELS];

  // ── Concurrency total. A small adder tree over the per-sub-channel
  //    counters; at the sub-channel counts in question this is trivial.
  always_comb begin
    inflight_total = '0;
    for (int g = 0; g < SUBCHANNELS; g++) begin
      inflight_total = inflight_total + TOT_W'(inflight[g]);
    end
  end

  logic sc_ok, cpl_ok, has_room, cpl_same_sc;

  // ── Range checks on the sub-channel indices.
  //
  //    NOT `req_sc < SC_W'(SUBCHANNELS)`: SC_W is $clog2(SUBCHANNELS), so
  //    that cast TRUNCATES whenever SUBCHANNELS is a power of two. At the
  //    default SUBCHANNELS = 2, SC_W is 1 and 1'(2) is 0, so the comparison
  //    would never hold and nothing would ever be accepted. When the count
  //    fills its index width every value is in range; otherwise compare one
  //    bit wider so the bound is representable.
  if (SUBCHANNELS >= (1 << SC_W)) begin : g_all_sc_valid
    assign sc_ok  = 1'b1;
    assign cpl_ok = 1'b1;
  end else begin : g_sc_range_check
    assign sc_ok  = ({1'b0, req_sc} < (SC_W+1)'(SUBCHANNELS));
    assign cpl_ok = ({1'b0, cpl_sc} < (SC_W+1)'(SUBCHANNELS));
  end

  assign cpl_same_sc = cpl_valid && cpl_ok && sc_ok && (cpl_sc == req_sc);

  // A retirement in the same cycle frees a slot, so a full sub-channel can
  // still accept. Without this, a saturated sub-channel would stall for a
  // cycle after every completion and lose a measurable fraction of its
  // achievable concurrency -- a real and commonly missed throughput bug.
  assign has_room = sc_ok
                 && ((inflight[req_sc] < IF_W'(MAX_INFLIGHT)) || cpl_same_sc);

  assign req_accept = req_valid && has_room;
  assign sc_blocked = req_valid && sc_ok && !has_room;

  assign cpl_unexpected = cpl_valid && (!cpl_ok || (cpl_ok && (inflight[cpl_sc] == '0)));

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      for (int g = 0; g < SUBCHANNELS; g++) inflight[g] <= '0;
      max_inflight_seen <= '0;
    end else begin
      for (int g = 0; g < SUBCHANNELS; g++) begin
        logic acc_g, ret_g;
        acc_g = req_accept && sc_ok && (req_sc == SC_W'(g));
        ret_g = cpl_valid && cpl_ok && (cpl_sc == SC_W'(g)) && (inflight[g] != '0);

        // Both in one cycle leaves the count unchanged, which is the whole
        // reason has_room accounts for a same-cycle completion.
        if (acc_g && !ret_g)      inflight[g] <= inflight[g] + 1'b1;
        else if (!acc_g && ret_g) inflight[g] <= inflight[g] - 1'b1;
      end

      // High-water mark of achieved concurrency. Sampled from the
      // combinational total, so it lags a real peak by one cycle and
      // converges -- adequate for telemetry, and stated rather than hidden.
      if (inflight_total > max_inflight_seen) begin
        max_inflight_seen <= inflight_total;
      end
    end
  end

endmodule

Combinational logic

has_room, and specifically its cpl_same_sc term. Without that term a sub-channel at its limit would refuse a request in the same cycle a slot is freed, costing a cycle after every completion — invisible functionally, measurable in throughput. sc_blocked distinguishes a full sub-channel from an idle one so a scheduler can tell them apart.

Sequential logic

One counter per sub-channel, each updated by the accept and retire signals for that sub-channel only — which is the independence, expressed structurally. The accept-and-retire-together case leaves the count unchanged.

Cycle trace

SUBCHANNELS = 2, MAX_INFLIGHT = 2, SUB_BITS = 32, BURST_BEATS = 16, TARGET_GRAN = 64 — elaborates, because 16 × 32 / 8 = 64.

Four accesses outstanding. Set SUBCHANNELS = 1 with MAX_INFLIGHT = 2 and the ceiling is two. Twice the concurrency, from two independent counters rather than from anything getting faster — which is exactly what the split buys beyond the granularity arithmetic.

And the check that matters most is the one that does not run. Set SUB_BITS = 64 with BURST_BEATS = 16 and elaboration fails with the granularity message. That non-event is the architecture being enforced.

Simulation

A directed test should fill both sub-channels and confirm the total reaches MAX_INFLIGHT × SUBCHANNELS; confirm SUBCHANNELS = 1 reaches only MAX_INFLIGHT; drive requests to one sub-channel only and confirm the total is capped at MAX_INFLIGHT while the other sub-channel stays empty — which is the concentration failure mode; confirm accept-in-the-same-cycle-as-completion at a full sub-channel; drive an unmatched completion and confirm cpl_unexpected with no underflow; and attempt to elaborate at least one deliberately invalid granularity and confirm it fails.

That last item is a test of the build, not of the simulation, and it is the one most often omitted — an elaboration check nobody has ever seen fire may not work.

Synthesis

SUBCHANNELS small counters, an adder tree, comparators, and a high-water register. The adder tree is the only thing that grows, and at realistic sub-channel counts it is negligible. The granularity check synthesises to nothing, which is the ideal cost for an architectural constraint.

Corner cases

SUBCHANNELS == 1 reproduces the pre-split behaviour and is the comparison that demonstrates the benefit. MAX_INFLIGHT == 1 gives IF_W == 1. A completion arriving in the same cycle as an accept to the same sub-channel leaves the count unchanged. A completion for an empty sub-channel is reported without underflowing. An out-of-range req_sc or cpl_sc is reported rather than masked, since masking would silently retarget an access.

Verification

What DV must prove: per-sub-channel counts never exceed MAX_INFLIGHT and never underflow; the total reaching MAX_INFLIGHT × SUBCHANNELS under a balanced pattern, which is the architectural claim and the property a purely forbidding suite would miss, exactly as 4.5's P4 warned; one sub-channel's activity not affecting the other's count; accept-with-same-cycle-completion; unmatched completions reported; and the elaboration check firing on an invalid granularity.

Debugging

If the total never exceeds MAX_INFLIGHT, the requests are all landing on one sub-channel — check the upstream address mapping, not this block. If a sub-channel stalls for a cycle after each completion, the cpl_same_sc term in has_room is missing. If counts drift downward over a long run, check the inflight[g] != '0 guard in ret_g — an unguarded decrement underflows to the maximum and makes a sub-channel appear permanently full. If max_inflight_seen reads one below the peak the trace shows, that is the documented one-cycle sampling lag rather than a bug.

Limitations

No data, no beats, no timing, no bank state — so it cannot say whether an outstanding access is making progress, only that it is outstanding. No latency modelling. No address mapping: req_sc arrives already decided, and the mapping is where the real design risk is. And nothing from §3 at all, for the reasons the header gives.

6. Four Assertions Worth Writing

// VERIFICATION-ONLY, inside subchannel_router.

// P1 -- per-sub-channel bounds. Safety: exceeding MAX_INFLIGHT means
// tracking state that does not exist.
property p_inflight_bounded;
  @(posedge clk) disable iff (!rst_n)
    (inflight_total <= TOT_W'(MAX_INFLIGHT * SUBCHANNELS));
endproperty
assert property (p_inflight_bounded);

// P2 -- INDEPENDENCE. Activity on one sub-channel does not change another's
// count. This is the architectural claim of the whole chapter: without it,
// "two sub-channels" is a relabelling of one. Written per pair with a
// genvar, because the property is about a RELATIONSHIP between counters and
// a single scalar property cannot express it.
generate
  for (genvar a = 0; a < SUBCHANNELS; a++) begin : g_indep
    property p_other_sc_unchanged;
      @(posedge clk) disable iff (!rst_n)
        (!(req_accept && (req_sc == SC_W'(a)))
         && !(cpl_valid && (cpl_sc == SC_W'(a))))
          |=> (inflight[a] == $past(inflight[a]));
    endproperty
    assert property (p_other_sc_unchanged);
  end
endgenerate

// P3 -- THE LIVENESS-FLAVOURED PROPERTY. A request to a sub-channel with
// room MUST be accepted. Chapter 4.5's P4 lesson applied: P1 and P2 forbid
// things, and a design that accepts nothing satisfies both perfectly.
property p_accepts_when_room;
  @(posedge clk) disable iff (!rst_n)
    (req_valid && sc_ok && (inflight[req_sc] < IF_W'(MAX_INFLIGHT)))
      |-> req_accept;
endproperty
assert property (p_accepts_when_room);

// P4 -- an unmatched completion is reported rather than acted on. Note the
// |=> rather than |->: cpl_unexpected is combinational and describes THIS
// cycle, while the state it must not disturb is not visible until the NEXT
// one. A |-> here would compare the completion against a total it cannot
// yet have affected, and would pass on a design that underflows.
property p_no_underflow;
  @(posedge clk) disable iff (!rst_n)
    (cpl_unexpected && !req_accept)
      |=> (inflight_total == $past(inflight_total));
endproperty
assert property (p_no_underflow);

P2 is the assertion that earns the chapter. Every other property here would hold on a design that merged both sub-channels into one shared counter — the counts would be bounded, requests would be accepted when there was room, nothing would underflow. Only P2 distinguishes two independent sub-channels from one wide one, which is the entire architectural difference between DDR5 and DDR4 at this level.

The pattern generalises: when a design's claim is that two things are independent, the assertion that matters is one stating that activity on one leaves the other unchanged. Independence is a property of a relationship, and relationships need properties written per pair — which is why P2 needs a genvar and cannot be a single scalar assertion.

What none of them prove. Nothing about granularity, because granularity is not a runtime quantity — §5's elaboration check is the only place it can be enforced, and no simulation can substitute for it. And nothing about on-die ECC, refresh management, equalisation or power regulation, all of which are device-internal, analog or physical. The verification gap between §5's RTL and §3's content is the largest in this module, and being able to state precisely where it lies is the point of classifying every block.

7. Two Streams in Cycles

inflight[0] and inflight[1] move independently. That is the figure's content — not the total, but the fact that the two counters respond only to their own sub-channel's traffic. The completion on cycle 6 changes inflight[0] and leaves inflight[1] untouched, which is P2 visible as a waveform.

Run the identical request stream at SUBCHANNELS = 1 and the total caps at 2. Same request rate, same per-sub-channel depth, half the concurrency — and by Chapter 1.8 §7's argument, concurrency is what converts a fixed access latency into throughput. The split doubled the available memory-level parallelism without making any single access one cycle faster.

What the figure does not claim. No timing: the completions here arrive when the stimulus says, not after any modelled latency. No data, no beats. And nothing about granularity, which §5's elaboration check owns.

8. The New Costs

Cost 1 — the controller doubled again. Two schedulers, two command streams, two sets of trained and timing state, and an address mapping deciding which sub-channel an access belongs to. A controller that treats a DDR5 channel as one wide channel extracts DDR4's parallelism from DDR5 hardware — 4.5 §10's bank-group failure mode, one level up, and with the same diagnosis: look at whether both halves are busy before looking anywhere else.

Cost 2 — a single access is not faster. Halving the width doubles the beats needed per access. At a given pin rate the burst takes the same time; at DDR5's higher pin rate it takes less, but the improvement comes from the rate and not from the split. The split buys throughput and concurrency, never single-access latency — and a latency-bound workload sees correspondingly little from it.

Cost 3 — on-die ECC reduced the host's visibility. Internally corrected errors do not reach the host, so a rising internal failure rate that used to surface as a growing correctable-error count may now surface as nothing until correction is exhausted. The part got better and the observability got worse, and system-level ECC remains a separate mechanism for a separate purpose.

Cost 4 — more mechanism that degrades silently when misconfigured. Refresh management requires the controller to track activation activity and act on it; equalisation must be trained; on-module regulation must be configured and monitored. Each is invisible when working, which is the fault class 4.4 §9 named and which has grown with every generation since.

9. Where the Chain Has Arrived

Six chapters, and the chain is worth reading end to end, because the shape of it is the module's real content.

Three things are true of every row. The array barely changed. The response was at the interface or the device architecture, never at the cell. And the controller absorbed more responsibility — for electrical state, then measured timing state, then access-pattern structure, then sub-channel scheduling.

And one more observation that sets up the last two chapters. Every row optimises the same objective under the same constraints: bandwidth per channel, in a system where the memory sits in a socket on a board, powered from a wall or a large supply, cooled by air, and expected to be replaceable and multi-vendor.

Change the constraints and the chain diverges. A device in a phone has a fixed battery, no socket, no airflow worth the name, and a package shared with the processor. A device feeding an accelerator has a bandwidth requirement that no number of 32-bit sub-channels can reach and a power budget that forbids getting there by raising the rate.

Those are not later generations of this chain. They are different chains, run against different constraints, and the next two chapters are about what the same engineering reasoning produces when the objective changes.

10. Common Misconceptions

"DDR5 sub-channels are just the channel split in half, so each is worse." Wrong mental model: a narrower channel is a downgrade. Engineering action: comparing a DDR5 sub-channel against a DDR4 channel on width; assuming access size halved; assuming bandwidth per pin fell. Observable failure / bad conclusion: concluding DDR5 offers no benefit at equal pin count, and — the practical version — building a controller that reassembles the two sub-channels into one wide logical channel, which throws away the entire parallelism benefit while keeping all the complexity. Correct model: the burst doubled as the width halved, so granularity is exactly unchanged at 64 bytes and the data pin count is exactly unchanged. What doubled is the beat count per burst — which is what lets the pin rate rise — and the number of independent request streams. Prevention: compute burst × width for both. If the product is the same, nothing about access size changed, and the comparison has to be about something else.

"Each sub-channel has its own data pins but shares the command bus." Wrong mental model: the split is a data-path partitioning. Engineering action: designing one scheduler and one command stream for both halves; assuming a command to one sub-channel is free with respect to the other. Observable failure / bad conclusion: the two halves contending on the shared path, so they behave as one stream serving two data paths — DDR4's concurrency with DDR5's complexity, and a measured parallelism that never reaches the expected ceiling. Correct model: each sub-channel has its own command and address path, plus its own bank and bank-group hierarchy and its own trained timing state. Without a separate CA bus the streams would not be independent, and independence is the entire point. Prevention: ask what would have to be shared for the streams to interfere. If the command path is shared, they are not independent.

"On-die ECC means the system does not need ECC." Wrong mental model: error correction is error correction. Engineering action: omitting system-level ECC on the basis that the device corrects internally; treating a clean interface as evidence the memory is healthy. Observable failure / bad conclusion: no protection against errors introduced anywhere outside the array — on the interface, in the controller, on the module — and loss of the early warning that a rising correctable-error rate used to provide. Failures arrive without the gradual degradation that used to precede them. Correct model: on-die ECC exists to make a denser array manufacturable by tolerating marginal cells. It is internal, it protects the array only, and it deliberately hides what it corrects. System-level ECC protects the system's data across the whole path and reports what it sees. They serve different purposes and neither substitutes for the other. Prevention: ask what each mechanism protects and who can see its reports. One protects the array and reports to nobody; the other protects the path and reports to the host.

"Refresh management is just more refresh." Wrong mental model: it is a refresh rate increase. Engineering action: treating it as a device-internal matter; not implementing activation tracking in the controller; assuming a conservative refresh rate covers it. Observable failure / bad conclusion: no mitigation against a targeted, activity-driven disturbance, because the mechanism it defends against is not uniform in time or in address — a workload that repeatedly activates one row degrades its neighbours faster than any fixed interval repairs them, and a uniformly higher refresh rate pays cost everywhere while still not covering the worst case. Correct model: the disturbance Chapter 2.2 described is driven by activation activity on nearby rows. Refresh management requires the controller to track that activity and direct additional targeted refresh where it is warranted. It is a controller obligation, not a device setting. Prevention: ask whether the threat is uniform or targeted. A targeted threat needs a targeted response, and targeting requires someone to be counting.

11. Debugging — Measured Parallelism Is Half the Expected Value

Symptom. A DDR5 system is functional and its sustained bandwidth is roughly what a DDR4 system of the same pin count and rate would deliver. The expected improvement from two sub-channels is absent. No errors.

"DDR5 performing like DDR4" is a strong hint that something is collapsing the two sub-channels into one, and there are several ways for that to happen at different layers.

Mechanism 1 — the address mapping sends everything to one sub-channel. Inspect: which address bit selects the sub-channel, and how the workload's stride interacts with it. Expected evidence: one sub-channel's occupancy high and the other's near zero. Discriminator: per-sub-channel occupancy, which is §5's inflight[] directly. If one is idle, nothing else matters. This is first because it is a configuration fix and because the measurement is a counter read.

Mechanism 2 — the controller reassembles the sub-channels into one logical channel. Inspect: whether the controller issues each access across both sub-channels together rather than to one. Expected evidence: both sub-channels busy, always in lockstep, with concurrency never exceeding one access at a time. Discriminator: both busy but correlated. This is cleanly different from mechanism 1 — there, one half is idle; here, both are busy and synchronised. It is a design decision rather than a mapping error, and it is a surprisingly common one because it makes a DDR5 controller look like a DDR4 controller. And note the honest exception: reassembling is correct when the workload's access size genuinely is 128 bytes, since one 128-byte access across both sub-channels wastes nothing. The defect is doing it unconditionally.

Mechanism 3 — the request source has no parallelism to exploit. Inspect: how many requests the requester has outstanding at once. Expected evidence: both sub-channels correctly configured and lightly occupied, with the requester issuing one access and waiting. Discriminator: occupancy low rather than blocked. Two independent streams cannot help a requester that only ever has one access in flight — this is Chapter 1.8 §7's point, and the fix is upstream of the memory system entirely.

Mechanism 4 — bank-group concentration inside a sub-channel. Inspect: 4.5's same-group versus cross-group stall counts, per sub-channel. Expected evidence: both sub-channels busy and both stalling on same-group conflicts. Discriminator: the stall cause within each half. The sub-channel split is working and the DDR4-level problem is present underneath it — the two mechanisms are independent and can both be true.

Mechanism 5 — the ceiling is granularity, not parallelism. Inspect: useful bytes over transferred bytes. Expected evidence: both sub-channels fully busy and delivered useful bandwidth well below the transferred figure. Discriminator: busy and wasteful rather than idle or blocked. No amount of parallelism fixes transferring data nobody wanted, and a 16-beat burst on a 32-bit sub-channel is still a 64-byte minimum.

Discrimination, cheapest first. Read per-sub-channel occupancy — that single counter pair separates mechanisms 1, 2 and 3 immediately, because the signatures are one idle, both busy in lockstep, and both lightly loaded. Then read useful-over-transferred bytes. Then read per-sub-channel same-group stall counts.

The reasoning lesson. When a structural feature delivers nothing, the question is not "is it broken" but "is it being used". Sub-channels, bank groups and prefetch all have the property that the hardware works perfectly while delivering nothing, because their benefit is conditional on the access pattern reaching them in the right shape. From DDR4 onward, every performance feature in this module is conditional in that way — which is why the diagnosis always begins by measuring whether the condition holds, and why per-cause, per-partition telemetry is the difference between a day and a month.

12. Interview Reasoning

"Why does DDR5 split its channel into two sub-channels?" To make a 16-beat burst affordable. Granularity is burst times width, and a 16-beat burst on a 64-bit channel would be 128 bytes — twice a typical cache line, so every access would transfer a wanted line and an unwanted one, raising peak bandwidth without raising delivered bandwidth. Halving the width to 32 bits while doubling the burst to 16 beats gives exactly 64 bytes, identical to DDR4. So the split is what lets the beat count — and therefore the per-pin rate — rise with the access size held precisely constant. The data pin count is unchanged too; what changes is the beat count and the number of independent request streams.

"What makes the two sub-channels independent rather than just two halves of a bus?" Each has its own command and address path. That is the load-bearing part: with a shared CA bus, a command to one would occupy the bus and delay the other, so the two would be one request stream serving two data paths. Each also has its own bank and bank-group hierarchy — so its column-path resources are its own — and its own trained timing state, because each has its own routing and receivers. What they do share is the device and module: supply, package and thermal environment. So they are independent in scheduling and shared in physics.

"What does the split buy beyond the granularity arithmetic?" Memory-level parallelism. Two independent streams double the number of accesses that can be outstanding, which is the quantity that converts a fixed access latency into throughput — a single access is not one cycle faster, and in fact needs twice as many beats because the channel is half as wide. So the benefit is throughput and concurrency, never single-access latency, and a latency-bound workload with one access in flight sees very little from it.

"Does on-die ECC replace system-level ECC?" No, and the reason is that they exist for different purposes. On-die ECC lets the device tolerate marginal cells, which is what makes a denser array manufacturable — DRAM economics are dominated by bits per area, and a smaller cell holds less charge and has less margin. It protects the array only, and it deliberately hides what it corrects. System-level ECC protects the system's data across the whole path, including the interface and the controller, and reports what it sees. There is also a real observability cost: internally corrected errors do not reach the host, so the rising correctable-error rate that used to provide early warning of a failing part may now be invisible until internal correction is exhausted.

"A DDR5 system performs like a DDR4 system of the same rate. How do you find out why?" Read per-sub-channel occupancy first, because three distinct causes have three distinct signatures there. One sub-channel idle means the address mapping is sending everything to the other, which is a configuration fix. Both busy but always in lockstep means the controller is reassembling them into one wide logical channel and issuing each access across both — which is correct if the workload's access size really is 128 bytes and wrong if it is done unconditionally. Both lightly loaded means the requester never has more than one access outstanding, so there is no parallelism to exploit and the fix is upstream of memory entirely. After that, read useful bytes over transferred bytes to rule out a granularity ceiling, and read same-group stall counts per sub-channel to see whether the DDR4-level bank-group problem is present underneath a correctly working split.

13. Engineering Check

An educational DDR5-style channel: two 32-bit sub-channels, 16-beat bursts, 4 accesses trackable per sub-channel. Educational figures, not any real part.

1. Granularity per sub-channel access? 16 beats × 32 bits ÷ 8 = 64 bytes. Identical to DDR4's 8 × 64 ÷ 8 — which is the design.

2. Data pins per channel, compared with DDR4? 2 × 32 = 64. Identical. No pins were spent to get the extra parallelism, which is what makes the trade compelling rather than merely possible.

3. Maximum outstanding accesses, versus a single 64-bit channel tracking 4? 2 × 4 = 8 versus 4. Twice the concurrency at the same pin count and the same per-stream depth.

4. A workload issues 64-byte requests, perfectly alternating sub-channels. Useful fraction? 64 ÷ 64 = 100%, with both sub-channels busy. The best case, and the one the architecture was designed around.

5. The same workload, but the address mapping sends every request to sub-channel 0. Useful fraction and concurrency? Useful fraction is still 100% — no bytes are wasted — but concurrency caps at 4 instead of 8 and sub-channel 1 is idle. Half the hardware unused, with nothing malfunctioning and no efficiency loss visible in a bytes-wasted metric. This is why §11's first measurement is per-sub-channel occupancy: the metric that would catch a granularity problem is completely blind to this one.

6. A workload issues 32-byte requests, alternating sub-channels. Useful fraction and concurrency? Concurrency is the full 8, and the useful fraction is 32 ÷ 64 = 50%. The parallelism is perfect and half the bandwidth is wasted. Together with question 5 this is the chapter's most useful pair: the two failure modes are orthogonal, they need different measurements, and a system can be maximally broken in one while perfect in the other. No single number describes memory performance, which is Chapter 1.8's thesis reaching its most concrete form.

14. Summary

DDR5 transfers 16 beats per burst and keeps granularity identical to DDR4 by halving the width the burst crosses — splitting each 64-bit channel into two independent 32-bit sub-channels. The cancellation is exact: 16 × 32 ÷ 8 = 64 bytes, the same as 8 × 64 ÷ 8, at the same data pin count. The general principle is that a design pinned at a granularity ceiling has no freedom in the product width × burst but complete freedom in the factors.

Independence requires duplication, and the command path is the load-bearing part. Each sub-channel has its own CA bus — without which the two would be one request stream serving two data paths — plus its own bank and bank-group hierarchy and its own trained timing state. They share the device: supply, package, thermals. Independent in scheduling, shared in physics.

The split buys concurrency, not latency. Two independent streams double the accesses that can be outstanding, which is what converts a fixed access latency into throughput. A single access is not faster, and needs twice the beats.

The supporting features each answer a pressure the higher density or higher rate created. On-die ECC makes a denser array manufacturable by tolerating marginal cells — and reduces the host's visibility into the array's health, which is why system-level ECC remains separate. Same-bank refresh converts a device-wide stall into a partial one as row counts grow. Refresh management answers the activity-driven disturbance of Chapter 2.2, and requires the controller to track activation activity. Decision feedback equalisation extends DDR3's calibration pattern from "measured once" to "adapted per bit". On-module power regulation moves the long supply path to a higher voltage and lower current, and localises responsibility for supply quality.

And the chain's shape is the module's real content. Six generations, and in every one the array barely changed, the response was at the interface or the device architecture, and the controller absorbed more responsibility — electrical state, then measured timing, then access-pattern structure, then sub-channel scheduling. Every generation also optimised the same objective under the same constraints: bandwidth per channel, in a socket, on a board, with a large supply and air cooling.

15. What Comes Next

Chapter 4.7 changes the constraints.

A memory device in a battery-powered product has no socket, no meaningful airflow, a fixed energy budget, and a package shared with the processor. Under those constraints the objective is not bandwidth per channel — it is bandwidth per joule, and the same engineering reasoning produces visibly different answers: a different signalling scheme chosen for energy rather than rate, a separation of the command clock from the data clock, power states deep enough to be genuinely expensive to leave, and more channels that are each narrower rather than fewer that are wide.

It is not a later generation of the DDR chain. It is a parallel chain, and reading it as a low-power variant of DDR is the single most common error about it.

Return to DDR4 for the granularity wall this chapter answers, DDR3 for the calibration pattern equalisation extends, or Charge Storage and Leakage for the disturbance mechanism refresh management mitigates. Module 25 is the DDR5 architecture in full. The full path is on the DDR tutorials index.