DIMMs

Every level so far has been logical. A bank, a bank group, a rank, a channel are structures a controller reasons about, and none of them has a physical boundary of its own.

A memory module does. It is a physical object: devices mounted on a board, joined to the system by a connector, replaceable as a unit.

And it is where this module's logical hierarchy meets an electrical constraint it has been able to ignore until now. The chapter's question is not what a module contains — that is mostly assembly. It is:

What scaling problem makes a buffering layer necessary, and what does that buffer cost?

1. A Module Is Not a Rank

Because module descriptions mix the two, it is worth separating them before anything else.

A rank is a logical grouping — the set of devices that together supply the data width, selected as a unit, sharing the data bus with other ranks.

A module is a physical object — a board carrying devices and a connector.

A module carries one or more ranks. Configurations are commonly written in a form that states both: a 1R module carries one rank, a 2R module carries two, and so on. Chapter 5.4 §13 worked a concrete case — a single-rank x8 DDR5 ECC module carries ten devices, five per 40-bit sub-channel — and the useful habit it demonstrates is that rank count and device count are different numbers arrived at by different reasoning: the rank count is a configuration choice, and the device count follows from the width divided by the device width.

And a module is not a channel either. A channel may have several module slots; a module sits in one. Chapter 5.5 §4 covered the one case that genuinely blurs this — a DDR5 module carrying two sub-channels — and the resolution there applies: independent in scheduling, shared in physics.

2. The Scaling Problem

Now the constraint that drives everything in this chapter.

Every device attached to a shared bus is an electrical load. Chapter 4.3 §3 established that every attachment is also a discontinuity that reflects, and Chapter 4.4 §2 that the signals reaching every device — command, address and clock — are the most heavily loaded in the system and the first to fail as rates rise.

Loading has two consequences and they pull against each other.

More devices means a lower achievable frequency. Each additional load makes the bus harder to drive cleanly, so a bus with many attached devices cannot run as fast as one with few.

But more devices is exactly how capacity grows. Chapter 5.4 §4 established that ranks add capacity without adding wires, which is their whole purpose — and each added rank adds a full rank's worth of devices to the bus.

So capacity and frequency are in direct conflict, mediated by load, and that conflict is the reason module forms differ at all. A design wanting both must find a way to add devices without adding proportional load.

A buffer is a re-driver. The controller drives one input; the buffer regenerates the signal and drives the devices behind it. From the controller's point of view the load is one device regardless of how many are actually present — which decouples capacity from frequency, and is the entire idea.

3. Three Module Forms, One Progression

The module forms differ in which signals get a buffer, and reading them as a progression makes the logic obvious.

Unbuffered. Command, address and data all run directly from the connector to the devices. No register, no buffer. Simplest, lowest latency, and most heavily loaded — so it supports the fewest ranks and, at a given frequency, the smallest configurations.

Registered. A registering clock driver sits between the connector and the devices, receiving the command, address and clock signals, buffering them, and re-driving them to the devices. The data lines still connect directly to the devices.

Why the command path first? Because Chapter 4.4 §2's argument says it is the worst case: command, address and clock must reach every device on the module, while data lines can be organised so each device owns a subset. Buffering the most heavily loaded signals first is where the benefit per buffer is largest.

Load-reduced. Adds data buffers on the data path as well, so the module presents a single load on both the command/address bus and the data bus. That is what permits the largest rank counts — with the data path buffered too, adding ranks no longer adds data-bus load, so configurations become possible that a registered module cannot support.

Read the progression as answering one question repeatedly: which bus is now the limit? Buffer the command path, and the data path becomes the limit. Buffer that, and the load ceases to scale with rank count at all. Each buffering layer appears when the previous one stopped being the binding constraint — which is the same pattern Module 4 followed generation by generation.

4. What a Buffer Costs

A buffer receives a signal and re-drives it, and that takes time. The signal emerges later than it would have.

The digital consequence is a pipeline stage, and three properties of it make it manageable rather than problematic:

It is deterministic. The delay is a fixed, known quantity, not a variable one. It is uniform. Every command through the buffer is delayed identically — there is no reordering and no per-command variation. It is therefore accountable. A controller absorbs it by shifting its timing model by a constant.

5. RTL — The Buffer's Digital Consequence

Engineering problem

A buffering layer inserts a fixed, uniform pipeline delay into the command/address path, and possibly into the data path. The controller must know the delay, and the delay must preserve ordering exactly. Model that — and model nothing else, because nothing else about a buffer is digital.

Classification

SYNTHESIZABLE RTL. Two parameterised pipelines and an in-flight counter.

It models the added pipeline stage. It does not model why the buffer exists. No loading, no impedance, no drive strength, no reflections, no signal integrity, no frequency benefit. §2's entire argument is analog and is unrepresentable here — the RTL cannot show that a buffer permits a higher frequency, only that it delays a command. Writing RTL that claimed otherwise would be exactly the fake-physics error this curriculum has refused throughout.

The parameterisation covers all three module forms, which is the block's most useful property:

   CA_STAGES = 0, DQ_STAGES = 0   ->  unbuffered
   CA_STAGES = N, DQ_STAGES = 0   ->  registered
   CA_STAGES = N, DQ_STAGES = M   ->  load-reduced

Interface

ca_in_* and ca_out_* are the command path; dq_in_* and dq_out_* the data path. ca_latency and dq_latency expose the constants a controller must account for. ca_in_flight is observability.

State

The pipeline registers, and a counter of commands currently inside the command pipeline.

Combinational logic

Almost none — the zero-stage case is a wire, and that is the point of the generate.

Sequential logic

Shift registers. Nothing conditional, which is what guarantees uniformity: there is no path by which one command could be delayed differently from another.

Simulation

vlog module_ca_pipeline.sv tb_module_ca_pipeline.sv then vsim -c tb_module_ca_pipeline -do "run -all"; VCS vcs -sverilog module_ca_pipeline.sv tb_module_ca_pipeline.sv && ./simv; Xcelium xrun -sv module_ca_pipeline.sv tb_module_ca_pipeline.sv.

Expected output: every command emerges exactly CA_STAGES cycles after entering, in the order it entered, with CA_STAGES = 0 producing a pure pass-through in the same cycle.

// ─────────────────────────────────────────────────────────────────────────
// MODULE CA PIPELINE.  Classification: SYNTHESIZABLE RTL.
//
// Models the ONE digital consequence of a module-level buffering layer: the
// fixed, uniform pipeline delay it adds. Three module forms are three
// parameterisations:
//
//   CA_STAGES = 0, DQ_STAGES = 0   unbuffered   (CA and DQ direct)
//   CA_STAGES = N, DQ_STAGES = 0   registered   (CA buffered, DQ direct)
//   CA_STAGES = N, DQ_STAGES = M   load-reduced (both buffered)
//
// WHAT THIS DOES NOT MODEL, AND CANNOT: electrical loading, impedance,
// drive strength, reflections, signal integrity, or the frequency headroom
// a buffer actually buys. SECTION 2'S ENTIRE ARGUMENT IS ANALOG. This RTL
// can show that a buffer DELAYS a command; it cannot show WHY anyone would
// accept that delay. Diagrams and prose carry the reason.
//
// The delay's value is a PARAMETER, not a claim: real buffer latency is
// device- and generation-specific and belongs to the module's own
// specification and to Modules 13/14/19.
//
// UNIFORMITY IS THE POINT. Nothing here is conditional, so no command can
// be delayed differently from another -- which is what makes the cost a
// constant the controller can absorb rather than a negotiation it cannot.
// ─────────────────────────────────────────────────────────────────────────
module module_ca_pipeline #(
  // Width of the command/address bundle. Abstract: Module 7 owns the real
  // command encodings, and this block never interprets the bits.
  parameter int CA_W      = 24,
  parameter int DQ_W      = 64,
  // Buffer depth on each path. 0 means no buffer on that path.
  parameter int CA_STAGES = 1,
  parameter int DQ_STAGES = 0,
  parameter int CNT_W     = (CA_STAGES <= 1) ? 1 : $clog2(CA_STAGES + 1)
) (
  input  logic              clk,
  input  logic              rst_n,

  input  logic              ca_in_valid,
  input  logic [CA_W-1:0]   ca_in,
  output logic              ca_out_valid,
  output logic [CA_W-1:0]   ca_out,

  input  logic              dq_in_valid,
  input  logic [DQ_W-1:0]   dq_in,
  output logic              dq_out_valid,
  output logic [DQ_W-1:0]   dq_out,

  // Exposed so the controller's timing model can be shifted by a constant
  // rather than having the value hard-coded in two places.
  output logic [7:0]        ca_latency,
  output logic [7:0]        dq_latency,

  output logic [CNT_W-1:0]  ca_in_flight
);

  // ── COMPILE-TIME legality.
  if (CA_STAGES < 0) begin : g_ca_min
    initial $fatal(1, "module_ca_pipeline: CA_STAGES must be >= 0");
  end
  if (DQ_STAGES < 0) begin : g_dq_min
    initial $fatal(1, "module_ca_pipeline: DQ_STAGES must be >= 0");
  end
  // A data buffer with no command buffer is not a module form that exists:
  // the command path is the more heavily loaded one (Chapter 4.4 Section 2)
  // and is always buffered first. Rejected so a nonsensical configuration
  // cannot be built by accident.
  if ((DQ_STAGES > 0) && (CA_STAGES == 0)) begin : g_form
    initial $fatal(1, "module_ca_pipeline: DQ buffering without CA buffering is not a module form");
  end
  if (CA_W < 1 || DQ_W < 1) begin : g_w_min
    initial $fatal(1, "module_ca_pipeline: CA_W and DQ_W must be >= 1");
  end

  assign ca_latency = 8'(CA_STAGES);
  assign dq_latency = 8'(DQ_STAGES);

  // ── Command/address path.
  //    The zero-stage case must be a WIRE, not a register. A generate is
  //    the only way to express that: a shift register of depth zero is not
  //    a declarable object, and writing `if (CA_STAGES == 0)` inside an
  //    always_ff would still infer a flop.
  if (CA_STAGES == 0) begin : g_ca_direct
    assign ca_out_valid = ca_in_valid;
    assign ca_out       = ca_in;
    assign ca_in_flight = '0;
  end else begin : g_ca_pipe
    logic            ca_v_q [CA_STAGES];
    logic [CA_W-1:0] ca_d_q [CA_STAGES];

    always_ff @(posedge clk or negedge rst_n) begin
      if (!rst_n) begin
        for (int s = 0; s < CA_STAGES; s++) begin
          ca_v_q[s] <= 1'b0;
          ca_d_q[s] <= '0;
        end
      end else begin
        // UNCONDITIONAL shift. There is deliberately no enable, no stall
        // and no bypass: any of them would make the delay data-dependent,
        // and a variable delay breaks the counted-cycle contract the whole
        // interface is built on (Section 4).
        ca_v_q[0] <= ca_in_valid;
        ca_d_q[0] <= ca_in;
        for (int s = 1; s < CA_STAGES; s++) begin
          ca_v_q[s] <= ca_v_q[s-1];
          ca_d_q[s] <= ca_d_q[s-1];
        end
      end
    end

    assign ca_out_valid = ca_v_q[CA_STAGES-1];
    assign ca_out       = ca_d_q[CA_STAGES-1];

    // Observability: how many commands are currently inside the buffer.
    always_comb begin
      ca_in_flight = '0;
      for (int s = 0; s < CA_STAGES; s++) begin
        if (ca_v_q[s]) ca_in_flight = ca_in_flight + CNT_W'(1);
      end
    end
  end

  // ── Data path. Same structure; present only on a load-reduced module.
  if (DQ_STAGES == 0) begin : g_dq_direct
    assign dq_out_valid = dq_in_valid;
    assign dq_out       = dq_in;
  end else begin : g_dq_pipe
    logic            dq_v_q [DQ_STAGES];
    logic [DQ_W-1:0] dq_d_q [DQ_STAGES];

    always_ff @(posedge clk or negedge rst_n) begin
      if (!rst_n) begin
        for (int s = 0; s < DQ_STAGES; s++) begin
          dq_v_q[s] <= 1'b0;
          dq_d_q[s] <= '0;
        end
      end else begin
        dq_v_q[0] <= dq_in_valid;
        dq_d_q[0] <= dq_in;
        for (int s = 1; s < DQ_STAGES; s++) begin
          dq_v_q[s] <= dq_v_q[s-1];
          dq_d_q[s] <= dq_d_q[s-1];
        end
      end
    end

    assign dq_out_valid = dq_v_q[DQ_STAGES-1];
    assign dq_out       = dq_d_q[DQ_STAGES-1];
  end

endmodule

Cycle trace

CA_STAGES = 2, DQ_STAGES = 1:

Every command takes exactly two cycles and they emerge in order. There is no configuration of inputs that changes either fact, which is §4's uniformity guarantee expressed structurally rather than promised.

Waveform expectation

§7. Watch that the gap between a command entering and emerging is identical for every command, and that ca_in_flight rises and falls purely with occupancy.

Synthesis implication

CA_STAGES × (CA_W + 1) flip-flops plus the same for the data path — and at zero stages, nothing at all: the generate produces wires. A real registering clock driver is a separate device with its own I/O, its own clocking and its own configuration; this is the timing shape it imposes, not the component.

Corner cases

CA_STAGES == 0 produces pure combinational pass-through, which is the unbuffered module and is the case the generate exists to make representable. CA_STAGES == 1 gives CNT_W == 1 through the guard. A data buffer without a command buffer does not elaborate, because it is not a module form that exists — the command path is the more heavily loaded one and is always buffered first. Reset clears the pipelines, so commands in flight across a reset are lost rather than delivered late, which is correct: a reset invalidates them.

Verification

What DV must prove: exact latency — every command emerges exactly CA_STAGES cycles after entry; ordering — commands emerge in entry order with no reordering, duplication or loss; uniformity — the delay is identical regardless of command content, spacing or occupancy, which is the property that matters most and is easiest to take for granted; the zero-stage pass-through; ca_in_flight matching an independent count; and that the illegal form does not elaborate.

SVA

§6.

Debugging

If commands emerge in the wrong order, the shift loop is running in the wrong direction — ca_v_q[s] <= ca_v_q[s-1] ascending is a delay line; descending would overwrite. If the delay varies, something conditional has been added to the shift — an enable, a stall or a bypass — and that is a correctness bug rather than an optimisation, because the controller's timing model assumes a constant. If ca_out_valid never asserts, check the output tap is CA_STAGES-1 and not CA_STAGES. If a controller's timing is off by exactly the buffer depth, check that it is reading ca_latency rather than assuming an unbuffered module — that is by far the most common integration error with a buffered module, and it produces a system that is wrong by a constant everywhere.

Limitations

No electrical modelling whatsoever — the header says why at length, and it is the whole reason the buffer exists. No modelling of the frequency headroom gained, no loading, no rank-count benefit. No buffer configuration or its own initialisation, which real registering clock drivers have. No error detection on the buffered path. And no notion of the asymmetry between the paths: a real data buffer is bidirectional and must turn around, which this unidirectional model does not represent.

6. Four Assertions Worth Writing

// VERIFICATION-ONLY. The first three live inside the g_ca_pipe branch,
// where a pipeline actually exists; the last is in g_ca_direct.

// P1 -- EXACT LATENCY. A command entering emerges exactly CA_STAGES cycles
// later. This is the property the controller's whole timing model rests on:
// Section 4's argument is that the delay is a CONSTANT it can add, and this
// is that claim made checkable.
property p_exact_latency;
  @(posedge clk) disable iff (!rst_n)
    ca_in_valid |-> ##CA_STAGES ca_out_valid;
endproperty
assert property (p_exact_latency);

// P2 -- and the DATA emerges with it, unchanged. Latency without integrity
// would be a delay line that forgets what it delayed, and P1 alone permits
// exactly that.
property p_data_preserved;
  logic [CA_W-1:0] d;
  @(posedge clk) disable iff (!rst_n)
    (ca_in_valid, d = ca_in)
      |-> ##CA_STAGES (ca_out_valid && (ca_out == d));
endproperty
assert property (p_data_preserved);

// P3 -- NO OUTPUT WITHOUT AN INPUT. P1 and P2 both start from an input, so
// neither forbids the pipeline producing a command nobody issued. Given a
// fixed latency, every output must have had an input exactly CA_STAGES ago
// -- and together with P1 that makes the mapping a bijection, which is why
// ordering needs no separate property here.
property p_no_spontaneous_output;
  @(posedge clk) disable iff (!rst_n)
    ca_out_valid |-> $past(ca_in_valid, CA_STAGES);
endproperty
assert property (p_no_spontaneous_output);

// P4 -- the unbuffered form is a WIRE, not a fast pipeline. Combinational,
// so an immediate assertion: inventing a clock to host a property on
// combinational logic would give the block a timing dependency it does not
// have.
always_comb begin
  a_direct_passthrough: assert (ca_out_valid == ca_in_valid);
  if (ca_in_valid) a_direct_data: assert (ca_out == ca_in);
end

P1 and P3 together are the interesting pair, because of what they make unnecessary. P1 says every input produces an output CA_STAGES later; P3 says every output had an input CA_STAGES earlier. Together they establish a bijection between inputs and outputs at a fixed offset — and a fixed-offset bijection cannot reorder.

So ordering needs no property here, and that is a fact about the structure, not an oversight. It is worth being explicit about why: §5's shift is unconditional. Add an enable, a stall or a bypass — any of which looks like a reasonable optimisation — and the mapping stops being a fixed offset, at which point ordering becomes a separate property that must be written. A reviewer who cannot say why a property is absent cannot tell an omission from a consequence.

P4 guards the degenerate form, and it earns its place because CA_STAGES == 0 is the unbuffered module — a real configuration, not a corner case. A generate arm that is rarely exercised is exactly where a pass-through quietly becomes a register.

What none of them prove, and it is nearly the whole subject: nothing about loading, impedance, drive strength, reflections, signal integrity, or the frequency headroom the buffer actually buys. §2's entire argument is analog. These properties establish that the buffer delays commands correctly; they cannot establish that anyone should want it to, and no assertion in any simulation can. That gap — between what the RTL can be verified to do and what the component exists for — is the widest in this module, and naming it is the point of classifying every block.

7. Latency, Applied Uniformly

The point of this figure is that it is boring, and deliberately so. ACT enters at cycle 0 and emerges at 2. RD enters at 1 and emerges at 3. PRE enters at 7, after a gap, with an empty pipeline — and emerges at 9, exactly two cycles later, just like the others.

Nothing changes the delay: not the command's content, not whether the pipeline was busy, not the spacing between commands. That invariance is the entire reason a buffering layer is acceptable, and §4 explained why: a fixed delay is a constant the controller adds to its model, while a variable one would break the counted-cycle contract the interface is built on.

And note what the figure does not show, which is most of the subject. It does not show the electrical loading that motivated the buffer, the frequency headroom it buys, or the additional ranks it permits. Those are the reasons the buffer exists and none of them is visible in any waveform — which is why §2 and §3 are prose and diagrams, and why §5's classification says so explicitly.

8. The Module, Answered Systematically

The "how is it selected" row is the one that distinguishes a module from every other level. Banks, groups, ranks and channels are all selected by a field. A module is not addressed at all — it is a packaging boundary, and the controller reaches its contents through the ranks on it. That is the clearest statement of why a module is a physical object rather than a logical level, and why it appears last in this module's progression.

9. Common Misconceptions

"A DIMM is a rank." Wrong mental model: module and rank are the same unit. Engineering action: reading a module count as a rank count; assuming one module is one rank when computing loading or capacity. Observable failure / bad conclusion: configuration and loading calculations wrong by the rank count, and an inability to interpret any real configuration — a 2R module is one physical object carrying two ranks, which share the channel's bus exactly as two ranks on separate modules would. Correct model: a rank is a logical grouping of devices supplying the data width and selected as a unit. A module is a physical board carrying one or more ranks, plus any buffering. They are counted separately and neither determines the other. Prevention: ask what is selected. A rank is selected by a chip select; a module is not selected at all.

"Registered and load-reduced modules are slower." Wrong mental model: buffering adds latency, therefore performance is worse. Engineering action: avoiding buffered modules for latency-sensitive work without examining the system-level consequence; comparing module forms on the buffer delay alone. Observable failure / bad conclusion: missing that buffering is what permits the higher frequency and larger capacity in the first place — so the comparison is not "same system, plus a delay" but "a system that can run faster and hold more, at the cost of a small fixed delay". A buffered configuration is frequently better on both latency and bandwidth despite the extra stage. Correct model: a buffer adds a fixed pipeline delay to latency and does not reduce throughput — a pipeline stage delays commands without limiting their rate. It buys frequency headroom and rank count by decoupling load from device count. Prevention: compare achievable configurations, not just the delay. The unbuffered alternative at the same capacity may not exist at the same frequency.

"Buffering is about making the signal stronger." Wrong mental model: the buffer is an amplifier for a weak signal. Engineering action: reasoning about buffering as drive strength; expecting it to help in situations where load is not the issue. Observable failure / bad conclusion: misunderstanding why data buffers appear only after command buffers, and why buffering helps rank count specifically — an amplifier model predicts neither. Correct model: the buffer isolates load. The controller drives one input instead of many devices, so the load it sees is constant regardless of how many devices are behind it. That is what decouples capacity from frequency — and it is why the most heavily loaded signals, command and address, are buffered first. Prevention: ask what the controller is driving. If the answer changes from "N devices" to "one buffer", the mechanism is load isolation.

"The buffer delay is variable and must be handshaked." Wrong mental model: a buffer is a queue with variable occupancy. Engineering action: designing a controller that waits for or negotiates with the buffer; treating buffer latency as a range rather than a constant. Observable failure / bad conclusion: unnecessary complexity, and a misunderstanding of why the interface has no such handshake — Chapter 4.1 §1 established the whole interface is a counted-cycle contract, and a variable buffer delay would make the count unknowable. Correct model: the delay is fixed, uniform and deterministic — the same for every command regardless of content, spacing or occupancy. The controller absorbs it by shifting its timing model by a constant. Predictability, not speed, is the engineering requirement on the buffer. Prevention: ask whether anything in the design could make one command take longer than another. In a correct buffer, nothing can — which is why §5's shift is unconditional.

10. Debugging — A Buffered Module Fails Where an Unbuffered One Works

Symptom. A system works with unbuffered modules and fails when buffered modules are fitted, or fails when a buffered configuration is populated more fully. Failures look like timing violations rather than data corruption — commands landing in unexpected states.

Swapping module form changes exactly two things: the electrical loading and the added latency. The first is the reason to swap and the second is the cost, and the failure is almost always in how the second was accounted for.

Mechanism 1 — the controller's timing model does not include the buffer latency. Inspect: whether the controller is configured for the module form actually fitted, and whether it is reading the buffer latency or assuming zero. Expected evidence: every timing relationship off by exactly the buffer depth — a constant error, not a variable one. Discriminator: is the error a constant? This is first because it is overwhelmingly the most common integration error with buffered modules and because the signature is unmistakable: a system wrong by the same amount everywhere is a configuration constant, not a marginality. §5's debugging note flags it for the same reason.

Mechanism 2 — the module form was mis-detected. Inspect: what the system believes is fitted, against what is fitted. Expected evidence: a configuration describing a different form than the hardware. Discriminator: read back the detected configuration. Distinguished from mechanism 1 by where the wrong constant came from — here the controller correctly applies a latency for the form it thinks is present, and the form is wrong. Same symptom, different fix, and both are free to check.

Mechanism 3 — per-rank training did not cover the added ranks. Inspect: whether training ran for every rank in the more fully populated configuration. Expected evidence: the additional ranks behaving marginally while the original ones are fine. Discriminator: does it correlate with which rank, rather than with all traffic? Chapter 5.4 §11's first mechanism, appearing again because a buffered module's purpose is to permit more ranks — so the configurations it enables are precisely the ones most likely to expose incomplete per-rank training.

Mechanism 4 — the data path is buffered and the model accounted only for the command path. Inspect: whether the configuration is load-reduced, and whether the controller's read and write timing includes the data buffer's contribution separately. Expected evidence: command-path timing correct and data-path timing off by a constant. Discriminator: are commands landing correctly while data timing is wrong? This separates a registered module's single latency from a load-reduced module's two, and it is easy to miss because the two paths are configured separately.

Mechanism 5 — not latency: the loading assumption was wrong in the other direction. Inspect: whether the configuration exceeds what the module form supports, or whether an unbuffered module is being used in a configuration that requires buffering. Expected evidence: marginality that worsens with population and does not correspond to any constant offset. Discriminator: does the failure scale with how many ranks are populated, rather than being a fixed offset? This is the genuinely electrical case, and it is the one where the answer is "this configuration is not supported" rather than "a constant is wrong".

Discrimination, cheapest first. Ask whether the error is a constant or scales with population — that single question separates mechanisms 1, 2 and 4 from mechanism 5, and it is answerable from the symptom alone. Then read back the detected module form and the configured latencies, which resolves 1, 2 and 4 among themselves. Then check per-rank training coverage.

The reasoning lesson. A constant error and a marginality look different and are diagnosed differently, and the distinction is available before any measurement. Something wrong by exactly the same amount everywhere is a number — a configuration constant, a model parameter, a detected value — and it is found by reading configuration, not by instrumenting hardware. Something that worsens as you add load is physics, and no amount of configuration review will find it. Classify the symptom before choosing the tool, because the two investigations share no steps and picking the wrong one costs days. This is the same first move Chapter 4.1 §11 recommended for a clean factor of two: arithmetic before instruments.

11. Interview Reasoning

"Why do registered and load-reduced modules exist?" Because electrical load couples capacity to frequency. Every device attached to a shared bus is a load, more load means a lower achievable frequency, and adding capacity means adding ranks and therefore devices — so capacity and frequency pull directly against each other. A buffer breaks the coupling: the controller drives one input, the buffer re-drives the signal to the devices behind it, and the load the controller sees is constant regardless of how many devices are present. Registered modules buffer the command, address and clock path, which is the most heavily loaded because those signals reach every device. Load-reduced modules additionally buffer the data path, so load stops scaling with rank count on both buses, which is what permits the largest configurations.

"What does a buffer cost, and why is that acceptable?" A fixed pipeline delay — the signal is received and re-driven, which takes time. It is acceptable because it is deterministic and uniform: every command is delayed identically regardless of content, spacing or occupancy, so the controller absorbs it by shifting its timing model by a constant. A variable delay would be far worse than a larger fixed one, because the whole DDR interface is a counted-cycle contract in which obligations are expressed in cycles, and a delay that sometimes differed would make the count unknowable and require a handshake the interface does not have. The cost also lands on latency rather than throughput — a pipeline stage delays commands without limiting their rate.

"Is a buffered module slower than an unbuffered one?" Higher latency by the buffer depth, same peak bandwidth, and frequently better overall — because the comparison is not "the same system plus a delay". Buffering is what permits the higher frequency and the larger rank count in the first place, so the unbuffered alternative at the same capacity may not exist at the same frequency. The right comparison is between achievable configurations, not between the delays.

"What is the difference between a DIMM and a rank?" A rank is logical — the set of devices that together supply the data width, selected as a unit by a chip select. A module is physical — a board carrying one or more ranks plus any buffering, joined by a connector. A 2R module is one physical object carrying two ranks, and those two ranks share the channel's bus exactly as two ranks on separate modules would. The clearest distinction is selection: a rank is selected by a field, and a module is not addressed at all. The controller reaches a module's contents through the ranks on it, which is why the module is a packaging boundary rather than a level of the addressing hierarchy.

"A system works with unbuffered modules and fails with buffered ones. Where do you start?" By asking whether the error is a constant or scales with population, because that distinguishes the two things swapping module form actually changes. If every timing relationship is off by exactly the buffer depth, the controller's model does not include the buffer latency — either it was not configured for the form fitted, or the form was mis-detected — and both are found by reading back configuration rather than by instrumenting anything. If the data path is buffered as well, the command and data latencies are configured separately and it is easy to account for one and not the other, which shows up as commands landing correctly while data timing is wrong. If instead the failure worsens as more ranks are populated with no fixed offset, that is genuinely electrical and the configuration may simply exceed what the form supports. Constant errors are found by reading numbers; marginality is found by measurement, and the two investigations share no steps.

12. Engineering Check

A channel supports modules of any form. Reason structurally; specific rank limits and latencies are module-specification and Module 13/14 material.

1. A 2R module is fitted. How many ranks share the channel's data bus? Two — and they share it exactly as two ranks on two separate modules would. Chapter 5.4 §3's ownership applies unchanged: one drives at a time, with a handoff on every change. Being on one module changes nothing about the sharing.

2. Two 2R modules are fitted on one channel. Ranks, and bandwidth? Four ranks, one data bus, bandwidth unchanged. Capacity doubled again; the wires did not. Also four ranks' worth of devices now load that bus, which is §2's constraint arriving.

3. Why does buffering the command path help more than buffering the data path, if you can only do one? Because command, address and clock reach every device on the module, while data lines can be organised so each device owns a subset. The command path is the most heavily loaded signal set, so isolating it removes the largest share of the load per buffer. That is why registered modules exist as a form and "data-buffered only" does not — and why §5's RTL refuses to elaborate that configuration.

4. A module form adds a fixed 2-cycle command latency. What must the controller change? One constant. Every command-to-command and command-to-data timing relationship shifts by 2 cycles. Nothing structural changes — no queues resize, no state machines change, no handshake appears — because the delay is uniform and deterministic. That is the entire content of §4.

5. What would change if the delay were sometimes 2 and sometimes 3? Everything. The interface's timing is a counted-cycle contract, so an unknown count means the controller cannot know when data will appear. It would need a handshake — a per-transfer acknowledgement that DDR does not have and Chapter 4.1 §4 explained it deliberately does not have, because the counted contract is what makes the interface efficient. A variable delay would not be a worse buffer; it would be an incompatible interface.

6. A system is off by exactly 2 cycles on every timing relationship after fitting buffered modules. Electrical problem or configuration problem? Configuration, conclusively, and the reasoning is available before any measurement. A constant error everywhere is a number, and numbers come from configuration — here, a controller applying zero buffer latency to a module that has two cycles of it. An electrical problem produces marginality that scales with load and varies with conditions; it does not produce a clean uniform offset. This is §10's first discrimination and it costs nothing to make.

13. Summary

A memory module is the one level in this hierarchy that is physical rather than logical. It carries one or more ranks, plus any buffering, and — uniquely — it is not addressed at all. The controller reaches its contents through the ranks on it, which is why it is a packaging boundary rather than a level of the selection hierarchy.

The scaling problem is load. Every attached device is an electrical load; more load means a lower achievable frequency; and capacity grows by adding ranks, which adds devices. So capacity and frequency pull directly against each other, and a design wanting both must add devices without adding proportional load.

A buffer breaks that coupling by isolating load — the controller drives one input, and the buffer re-drives to the devices behind it, so the load it sees is constant regardless of device count. It is not an amplifier; it is an isolator.

The forms are a progression of which bus is now the limit. Unbuffered: command, address and data all direct — simplest, most loaded, fewest ranks. Registered: a registering clock driver buffers command, address and clock, with data still direct — the command path first because those signals reach every device and are the most loaded. Load-reduced: adds data buffers, so load stops scaling with rank count on both buses, permitting the largest configurations.

The cost is a fixed pipeline delay, and three properties make it acceptable: it is deterministic, uniform across every command, and therefore accountable by shifting the controller's timing model by a constant. A variable delay would be far worse than a larger fixed one, because the interface is a counted-cycle contract and an unknown count would require a handshake DDR does not have. Predictability, not speed, is the requirement on a buffer. And the cost is to latency, never throughput.

Which makes buffered modules frequently better on both counts, because the unbuffered alternative at the same capacity may not exist at the same frequency.

And the debugging discriminator generalises: a constant error everywhere is a configuration number, found by reading configuration; a failure that scales with population is electrical, found by measurement. The two investigations share no steps, and the classification is available from the symptom alone.

14. What Comes Next

Chapter 5.7 assembles everything.

Six chapters have each introduced one level — device structure, banks, bank groups, ranks, channels and modules — with its own shared resource, its own selection mechanism and its own conflict. The final chapter puts them in one picture and traces a single request all the way through it, from a requester through the controller and PHY to a set of cells and back.

Its RTL is the module's integrative block: a structural front end that takes a request's decomposed fields and answers, in one place, which resources it needs, which are available, what class of access it is, and which level of the hierarchy is blocking it — composing the bank state of 5.2, the classification of 5.3 and the ownership of 5.4.

That composed picture is what every later DDR module references.

Return to Ranks for the logical grouping modules package, Channels for the bus a module attaches to, or DDR3 for the topology work that made command-path loading the binding constraint. Module 22 covers signal integrity properly. The full path is on the DDR tutorials index.