Skip to content
VLSI Mentor

DDR · Module 29

Memory Controllers in SoCs

A semantic adapter between two sides that disagree about what a request is, what order means, what fairness means, and whether priority should matter at all.

Chapter 29.1 treated the controller as the far side of a boundary and traced one request across it. This chapter asks what the controller actually is, and the answer is not a DDR block with a bus port.

It is a semantic adapter, and its difficulty is that the two sides it adapts between disagree about almost everything:

QuestionThe SoC side saysThe DRAM side says
What is a request?a transaction with an ID and a lengtha command to a bank, legal only in some states
What does order mean?same-ID order must be preserved — AXI 8.3order is a free variable to be optimised — 23.4
What is fairness?each requester gets its sharebanks and rows have no notion of requesters
Does priority matter?yes, requesters differ in urgencya high-priority request to a busy bank still waits
What is the cost of a request?its sizeits size and what the previous request left in that bank

Every row is a genuine disagreement, and the controller has to resolve all five at once. That is what makes it interesting architecture rather than glue.

1. Two Contracts, Not One Block

The controller has an interface on each side, and the two interfaces make incompatible promises.

The SoC-side contract. Chapter 17.5 owns it: a ready/valid handshake, metadata stability while stalled, one handshake per entry, and an ordering boundary. What the controller promises upstream is acceptance and eventual correct completion in a permitted order. It promises nothing about when.

The DRAM-side contract. Module 14 owns the timing parameters and Module 7 the commands. What the controller promises downstream is that every command it issues is legal in the device's current state. It promises nothing about which request that command serves.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
   the two promises, stated precisely

   upstream : "I will complete your request correctly, in an order
               your protocol permits. I will not tell you when."

   downstream: "Every command I issue is legal given this device's
               state and timing. I will not tell you whose request
               it is."

   the controller's whole job is that NEITHER PROMISE MENTIONS THE
   OTHER SIDE. Upstream has no vocabulary for a bank; downstream has
   no vocabulary for a requester. The adapter must satisfy both
   using state that only it holds.

That is the structural reason a controller cannot be thin. It must hold: the outstanding requests and their identities, the per-bank row state, the timing history per resource, the refresh obligation, and the per-source accounting that fairness requires. None of that state belongs to either side.

2. The Ownership Split

Four layers, four contracts, and the boundaries between them are not where the IP boundaries necessarily fall.

LayerOwnsCanonical owner
Interconnecttransport, routing, the ordering contract it advertisesAXI 12.1, CHI 3.7
Controllerrequest transformation, mapping, scheduling, refreshModule 17
PHYthe physical interface, capture, trainingModule 19, Module 21
DRAMdevice behaviour and its timing requirementsModule 7, Module 14

3. What the Coherent Case Changes

A memory path may pass through a coherent fabric, and it is worth being precise about what that does and does not change for DDR.

CURRICULUM-DERIVED from the CHI track: CHI 4.4 owns the Slave Node — the memory-facing node — and 5.5 / 5.6 own its coherent and IO variants. CHI 5.8 owns the memory controller in a CHI system and characterises it as a reordering DRAM controller behind a coherency-free front end.

That phrase is the important one, and it is the boundary this chapter needs.

Owned byReaches DDR?
Cache states and transitionsCHI 10.1no
SnoopsCHI 2.8no
Coherency resolution at the home nodeCHI 4.3no
The memory request that resultsCHI 4.4yes
Dirty-data writebackCHI 2.6, 8.7yes, as a write

So DRAM does not participate in coherence. It has no notion of ownership, sharing or dirtiness. The coherence system resolves all of that above the controller and then issues an ordinary memory request — which is why the front end can be coherency-free.

What the coherent path does change is the traffic's shape and timing, and that is genuinely relevant here:

A coherency resolution adds latency before the DDR request even exists. §14's composition table lists it as a contributor, and it is one of the components that is absent in a non-coherent path.

A writeback is a write the requester never issued. Chapter CHI 8.7 owns the evict flow; from the controller's side a dirty eviction is simply a write, arriving at a time determined by cache replacement rather than by any program. So a read-only workload can generate write traffic, which is 29.3 §4's subject.

And a snoop that finds data in another cache means no DDR request at all. The coherent fabric can satisfy a miss from a peer cache. So adding coherent requesters does not simply add DDR traffic — it can absorb some.

4. The Controller's Logical Regions

A memory controller's logical regions, in four rows, with the two contract boundaries marked. The top row is the SoC-side boundary: the interconnect-facing port that implements the ready-valid contract, the request normalisation that converts incoming transactions into uniform internal requests, the source-level quality-of-service arbitration that decides which requester proceeds, and the address decode and channel selection. The second row holds the request storage: separate read and write queues, because read and write have different urgency and cause a turnaround cost when interleaved, together with the address mapper that produces rank, bank group, bank, row and column, and the row-state tracker that records which row is open in each bank. The third row holds the decision layer: the command scheduler that owns the commit point, the refresh manager whose obligations compete with traffic, and the timing-legality check that every candidate must pass. The bottom row is the DRAM-side boundary, the physical-layer-facing command and data interface. Edges run from the port through normalisation and arbitration into the queues, from decode into the mapper, from the queues into the mapper and the scheduler, from the mapper into the row-state tracker and the scheduler, from the row tracker and the refresh manager into the scheduler, from the scheduler through the legality check, and from the legality check to the physical interface. The structural point is that the source-level arbitration in the top row and the command scheduler in the third row are two different arbitration layers deciding different questions.Interconnect port17.5's ready/validcontractNormalisationtransactions → uniformrequestsSource QoSwhich requester — §6Decode / channelAXI 12.3Read queue17.2 — allocationWrite queueseparate: turnaround — §9Address mapper8.6 — bank/row/columnRow-state trackerModule 9 — what is openCommand scheduler17.1 — the commit pointRefresh manager17.3 — competes withtrafficTiming legalityModule 14 — everycandidatePHY interfacethe DRAM-side boundary12

Two arbitration layers, and confusing them is the chapter's central error.

Source QoS in the top row decides which requester's request enters the queues. It reasons about sources, classes, shares and starvation — and it knows nothing about banks.

Command scheduler in the third row decides which queued request becomes a command. Chapter 17.4 owns it, including the layered mask that expresses policy without contaminating legality and the bounded bypass that fixes starvation at that layer. It reasons about rows, banks and timing — and it may know nothing about sources.

So a request is arbitrated twice, against different criteria, by layers that cannot see each other's state. §7 is about what that composition does.

And Refresh manager feeds the scheduler as a competitor, not a controller. Chapter 17.3 owns its drain requirement and policy dial; from this diagram's perspective, refresh is another claimant on the same device, which is why 28.3 had to separate obligation failures from service failures.

5. Several Requesters, One Device

The multi-requester topology around a memory controller, in four rows. The top row holds four distinct requester classes with different traffic characters: a CPU cluster whose requests are latency-sensitive and driven by cache misses, a DMA engine whose requests are throughput-oriented bulk transfers, an accelerator whose requests are a deep streaming consequence of on-chip reuse, and a display or real-time engine whose requests have a hard deadline because a missed deadline is visible to a user. The second row shows what each contributes to the shared resource: outstanding depth, request size, sequentiality, and deadline pressure. The third row is the shared resource itself, drawn as three nodes to make the sharing explicit: the single memory controller, its finite queue capacity, and the single DRAM device with its banks and rows. The bottom row holds the two consequences that follow from sharing a finite resource: interference, where one requester's traffic changes another's observed latency, and starvation, where a requester can be denied service indefinitely unless something bounds the wait. Edges run from each requester into the contribution row, from the contributions into the controller, from the controller into its queue capacity and the device, and from the queue and device into both interference and starvation. The structural point is that every requester shares one queue capacity and one device, so the controller cannot give any requester an isolated view, and interference is a property of sharing rather than a defect.CPU clusterlatency-sensitive — 29.3DMA enginethroughput — 29.4Acceleratordeep streaming — 29.5Display / RThard deadlineOutstanding depthhow many in flightRequest sizebytes per transactionSequentialityrow locality offeredDeadline pressurewhat a miss costsOne controllerModule 17Finite queuesshared capacityOne devicebanks and rows — Module 5Interferencea property of sharing —§812

Interference is drawn as a consequence of the bottom two shared nodes, and that is deliberate. It is a property of sharing a finite resource, not a controller defect. Any scheme that gives one requester better service gives another worse, because the device serves one command at a time.

And that is the honest framing for everything in §6. QoS does not eliminate interference. It decides how interference is distributed.

6. QoS Is Constrained Arbitration

The chapter's most important correction.

“Highest priority always wins” is not a policy — it is a bug. Three reasons, each sufficient.

First: priority cannot override physics. A high-priority request to a bank in tRC waits, and a low-priority request to an idle bank can be served meanwhile. CURRICULUM-DERIVED from 17.4: legality is a mask applied before policy, and policy cannot unmask an illegal command. So the scheduler's first filter is not priority at all.

Second: strict priority starves. Chapter 17.4 owns starvation demonstrated concretely and the bounded bypass that fixes it. A continuously-offered high-priority stream serves forever and a low-priority requester never proceeds. Starvation is not a rare corner; it is the expected behaviour of strict priority under sustained load.

Third: priority and locality conflict. Serving requests in priority order destroys row locality; serving for row locality ignores priority. Chapter 23.4 owns what reordering gains and costs.

So QoS is arbitration under several simultaneous constraints:

ObjectiveWantsConflicts with
Low latency for a classserve it nowlocality, fairness
High bandwidthmaximise row hitslatency for a starved class
Row localitygroup same-row requestsarrival order, priority
Fairnessshares proportional to weightlocality, strict priority
Refresh obligationperiodic device-wide serviceevery traffic objective — 17.3
Starvation freedoma bound on any waitpure locality, pure priority
Real-time deadlineservice before a fixed timeeverything else
Read/write turnaroundgroup same-directionarrival order, read priority — §9

Eight objectives, and no scheduler maximises all of them. The design question is not which policy is best but which objectives are constraints and which are being optimised — and that is an application decision, not a memory-controller decision.

7. Two Arbitration Layers Compose Badly

§4 showed source QoS above the queues and command scheduling below them. Neither can see the other's state, and their composition is not the sum of their intentions.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
   a concrete composition failure, ILLUSTRATIVE

   source QoS gives the CPU class 70% of queue ENTRIES.
   the scheduler optimises for ROW HITS among queued requests.

   the CPU's stream is scattered  -> few row hits among its entries
   the DMA's stream is sequential -> many row hits among its entries

   the scheduler, seeing both queues, finds most of its row hits in
   the DMA's requests. It serves them preferentially -- not because
   of priority, but because they are the ones that are cheap.

   result: the CPU class holds 70% of the ENTRIES and receives far
   less than 70% of the SERVICE.

   neither layer is broken. Source QoS allocated entries as
   configured. The scheduler maximised throughput as configured.
   The COMPOSITION delivered something nobody specified.

This is the structural reason source-level QoS alone does not deliver a service guarantee. Allocating queue entries allocates opportunity, not service — and the conversion rate from opportunity to service depends on the requester's own locality, which the QoS layer cannot see.

Two honest responses exist. Either the scheduler is made source-aware, so the two layers share state and the composition becomes designed rather than emergent — at the cost of contaminating 17.4's clean legality/policy separation. Or service is measured per source and the allocation is tuned against measurement, accepting that the mapping is empirical.

What is not available is a configuration that guarantees a share without one of those two. §11's arbiter measures per-source service for exactly this reason: the number that matters is service delivered, not entries allocated.

8. Why Read and Write Queues Are Separate

A structural question with a DRAM-side answer.

CURRICULUM-DERIVED from 14.6, which owns column-to-column spacing and the read/write turnaround cases that share the same structure: switching the data bus between reading and writing costs time. So interleaving reads and writes costs more than grouping them.

ArrangementConsequence
One queue, arrival orderturnarounds proportional to how often direction changes
Separate queues, grouped servicefewer turnarounds, more bandwidth
Separate queues, reads always firstfewer turnarounds, writes starve

Row three is a real design that ships, and it is defensible up to a point: a read is usually blocking a requester and a write usually is not. But writes cannot be deferred indefinitely — the write queue fills, and a full write queue eventually blocks acceptance, which blocks the requester that was not waiting, converting a write-latency problem into a read-latency problem.

So the write queue's depth is a latency-hiding budget, and its occupancy is the signal that the budget is spent. §11's telemetry publishes it, and §15 treats a full write queue as a distinct symptom from a full read queue because they mean opposite things about the workload.

And write buffering has a second consequence worth naming. A buffered write has been accepted but not yet stored. The controller therefore holds data the memory does not yet have, which is why a read to the same address must be satisfied from the buffer rather than from DRAM. Chapter 27.4 §3 owns the verification form of exactly this hazard — the expected value is a set when a write is in flight — and it exists here as a functional requirement, not just a checking problem.

9. Multiple Ports and Traffic Classes

Why a controller may present more than one port, without implying that all do.

ReasonWhat it buys
Physical separation of requestersindependent flow control; one blocked port does not block another
Different protocols on different portsa coherent port and an IO port — CHI 5.5, 5.6
Different widths or clocksconversion at the port rather than in the fabric
Class-based accountingper-port shares are easier to measure than per-ID shares

The last row is the practical one. Per-source accounting requires identifying the source, and a bus ID is not a reliable source identityAXI 12.5 owns ID-based routing, and IDs can be remapped by the interconnect. A port is a physical identity that survives the fabric, which is why per-port QoS is common and per-ID QoS is rare.

But multiple ports do not multiply the device. All ports converge on one scheduler and one DRAM. So ports provide isolation in flow control and none in service — the interference in §5 is unchanged.

CURRICULUM-DERIVED caveat: 25.2 owns sub-channels, which are independent at the device level. A sub-channel is a different thing from a controller port: two ports share a device, two sub-channels do not. Conflating them predicts isolation that does not exist.

10. The Source-Level Arbiter

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// ---------------------------------------------------------------------
// source_qos_arbiter -- CLASSIFICATION: synthesisable, ILLUSTRATIVE.
//
// WHAT IT DOES: §4's TOP-ROW arbitration -- which SOURCE proceeds.
// Weighted service with a hard age bound, plus per-source measurement
// of service actually delivered.
//
// WHY AN AGE BOUND AND NOT STRICT PRIORITY (§6): strict priority
// starves, and starvation is the EXPECTED behaviour under sustained
// load rather than a corner case. The age bound converts an unbounded
// wait into a provable limit -- which is the same argument 17.4 makes
// for its bounded bypass at the command layer, applied here at the
// source layer.
//
// WHY IT MEASURES SERVICE (§7): allocating opportunity is not
// allocating service. The conversion rate depends on each source's own
// locality, which this layer cannot see. So the honest output is a
// measurement, not an assumption.
//
// WHAT IT IS NOT: the command scheduler. It has no bank state, no
// timing state and no row state, and it must not: 17.4 owns command
// arbitration and its legality/policy separation.
//
// SYNTHESIS: per-source counters, a priority encoder, one comparator
// chain. No memory.
// ---------------------------------------------------------------------
module source_qos_arbiter #(
  parameter int SOURCES  = 4,
  parameter int WEIGHT_W = 4,
  parameter int AGE_W    = 10,
  parameter int AGE_MAX  = 256   // the starvation bound, in grant cycles
)(
  input  logic                       clk,
  input  logic                       rst_n,

  input  logic [SOURCES-1:0]         req,             // per-source request
  input  logic [WEIGHT_W-1:0]        weight [SOURCES],// configured share
  input  logic                       downstream_ready,// queues can accept

  output logic [SOURCES-1:0]         grant,           // exactly one, or none
  output logic                       grant_valid,

  // ---- published measurement, §7
  output logic [31:0]                served   [SOURCES],
  output logic [AGE_W-1:0]           age      [SOURCES],
  output logic [AGE_W-1:0]           max_age  [SOURCES],
  output logic [15:0]                age_forced,      // grants forced by the bound
  output logic [15:0]                starvation_events
);
  // SOURCES of 1 makes $clog2 zero and the grant vector degenerate; a
  // one-source arbiter is also not an arbiter. AGE_MAX must fit in
  // AGE_W or the bound can never be reached and the guarantee is void
  // -- COUNT versus INDEX applied to a threshold.
  initial begin
    if (SOURCES  < 2) $fatal(1, "source_qos_arbiter: SOURCES must be >= 2 (got %0d)", SOURCES);
    if (WEIGHT_W < 1) $fatal(1, "source_qos_arbiter: WEIGHT_W must be >= 1");
    if (AGE_W    < 2) $fatal(1, "source_qos_arbiter: AGE_W must be >= 2");
    if (AGE_MAX  < 1) $fatal(1, "source_qos_arbiter: AGE_MAX must be >= 1");
    if (AGE_MAX >= (1 << AGE_W))
      $fatal(1, "source_qos_arbiter: AGE_MAX %0d unreachable in AGE_W %0d bits", AGE_MAX, AGE_W);
  end

  // Credits implement the weighted share. A source spends a credit when
  // granted and all credits are replenished when every requesting
  // source is out -- which keeps the ratio without needing division.
  logic [WEIGHT_W:0] credit [SOURCES];

  logic [SOURCES-1:0] eligible;     // requesting and holding a credit
  logic [SOURCES-1:0] forced;       // requesting and at the age bound
  logic               any_forced, any_eligible;

  always_comb begin
    for (int s = 0; s < SOURCES; s++) begin
      eligible[s] = req[s] && (credit[s] != '0);
      forced[s]   = req[s] && (age[s] >= AGE_MAX[AGE_W-1:0]);
    end
    any_forced   = |forced;
    any_eligible = |eligible;
  end

  // Grant selection. The age bound OUTRANKS the weighted share: a
  // source at its bound is granted even with no credit, which is what
  // makes the bound a guarantee rather than a preference. Within each
  // group the lowest index wins -- a fixed order, which is acceptable
  // ONLY because the age bound prevents indefinite loss. Without the
  // bound this would be strict priority and §6 explains why that is a
  // bug.
  logic [SOURCES-1:0] sel;
  always_comb begin
    sel = '0;
    if (downstream_ready) begin
      if (any_forced) begin
        for (int s = 0; s < SOURCES; s++)
          if (forced[s] && sel == '0) sel[s] = 1'b1;
      end else if (any_eligible) begin
        for (int s = 0; s < SOURCES; s++)
          if (eligible[s] && sel == '0) sel[s] = 1'b1;
      end
    end
  end

  assign grant       = sel;
  assign grant_valid = |sel;

  always_ff @(posedge clk) begin
    if (!rst_n) begin
      age_forced        <= '0;
      starvation_events <= '0;
      for (int s = 0; s < SOURCES; s++) begin
        credit[s]  <= '0;
        served[s]  <= '0;
        age[s]     <= '0;
        max_age[s] <= '0;
      end
    end else begin
      // ---- replenish when no requesting source has a credit left.
      // Computed from the PRE-grant credits so a grant this cycle does
      // not trigger a replenish in the same cycle -- which would make
      // the weights meaningless by refilling constantly.
      if (!any_eligible && |req) begin
        for (int s = 0; s < SOURCES; s++)
          credit[s] <= {1'b0, weight[s]};
      end else if (grant_valid) begin
        for (int s = 0; s < SOURCES; s++)
          if (sel[s] && credit[s] != '0) credit[s] <= credit[s] - 1'b1;
      end

      // ---- age: a requesting, ungranted source ages; a granted one
      // resets. Saturating, so the bound cannot be skipped by a wrap.
      for (int s = 0; s < SOURCES; s++) begin
        if (sel[s]) begin
          age[s] <= '0;
        end else if (req[s] && age[s] != {AGE_W{1'b1}}) begin
          age[s] <= age[s] + 1'b1;
        end
        if (age[s] > max_age[s]) max_age[s] <= age[s];
      end

      // ---- §7's measurement: service DELIVERED, per source.
      for (int s = 0; s < SOURCES; s++)
        if (sel[s] && served[s] != 32'hFFFF_FFFF) served[s] <= served[s] + 1'b1;

      if (grant_valid && any_forced && age_forced != 16'hFFFF)
        age_forced <= age_forced + 1'b1;

      // A source reaching the bound at all is worth counting: it means
      // the weighted share alone was not delivering it service, which
      // is §7's composition problem becoming visible.
      for (int s = 0; s < SOURCES; s++)
        if (req[s] && age[s] == AGE_MAX[AGE_W-1:0]
            && starvation_events != 16'hFFFF)
          starvation_events <= starvation_events + 1'b1;
    end
  end
endmodule

The age bound outranks the weighted share, and that ordering is the guarantee. A source at its bound is granted even with no credit. Reversing the two would make the bound a preference, and 17.4's concrete starvation demonstration shows what a preference is worth under sustained load.

Fixed-index selection within each group is acceptable only because of the bound. Without it this block would be strict priority — §6's bug. Naming that dependency matters, because a later edit that raised AGE_MAX to a very large value would quietly turn this arbiter back into strict priority while every property still passed.

And served[] measures what §7 says cannot be assumed. Weights allocate credits; credits allocate opportunity; opportunity becomes service at a rate this layer cannot see. The counter is the only honest way to know the ratio actually delivered.

11. Two-Sided Telemetry

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// ---------------------------------------------------------------------
// ctrl_boundary_telemetry -- CLASSIFICATION: synthesisable, BINDABLE.
//
// WHAT IT DOES: measures BOTH sides of the controller. Occupancy and
// acceptance on the SoC side; command issue and data-bus busy on the
// DRAM side.
//
// WHY BOTH (29.1 §2): a full request queue does not mean DRAM is busy.
// It can equally mean the scheduler is blocked on a timing constraint
// while the data bus idles. A one-sided measurement cannot distinguish
// those, and they need opposite fixes.
//
// WHY READ AND WRITE SEPARATELY (§8): a full read queue and a full
// write queue mean opposite things about the workload, and a combined
// occupancy number hides which one is happening.
//
// WHAT IT CANNOT TELL YOU: why. Module 28 owns the investigation.
//
// SYNTHESIS: counters and two max-trackers. No memory.
// ---------------------------------------------------------------------
module ctrl_boundary_telemetry #(
  parameter int RQ_DEPTH = 32,
  parameter int WQ_DEPTH = 32,
  parameter int WIN_W    = 20     // measurement window, in cycles
)(
  input  logic                            clk,
  input  logic                            rst_n,

  // ---- SoC side
  input  logic                            req_valid,
  input  logic                            req_ready,
  input  logic                            req_is_read,
  input  logic [$clog2(RQ_DEPTH+1)-1:0]   rq_occ,
  input  logic [$clog2(WQ_DEPTH+1)-1:0]   wq_occ,

  // ---- DRAM side
  input  logic                            cmd_issued,
  input  logic                            data_busy,     // DQ bus active
  input  logic                            sched_blocked, // no legal candidate

  input  logic                            clear,

  // ---- published
  output logic [31:0]                     accepted_reads,
  output logic [31:0]                     accepted_writes,
  output logic [31:0]                     stalled_cycles,
  output logic [$clog2(RQ_DEPTH+1)-1:0]   rq_occ_max,
  output logic [$clog2(WQ_DEPTH+1)-1:0]   wq_occ_max,
  output logic [31:0]                     cmds_issued,
  output logic [31:0]                     data_busy_cycles,
  output logic [31:0]                     sched_blocked_cycles,
  output logic [31:0]                     window_cycles,
  output logic                            rq_ever_full,
  output logic                            wq_ever_full
);
  initial begin
    if (RQ_DEPTH < 2) $fatal(1, "ctrl_boundary_telemetry: RQ_DEPTH must be >= 2");
    if (WQ_DEPTH < 2) $fatal(1, "ctrl_boundary_telemetry: WQ_DEPTH must be >= 2");
    if (WIN_W    < 8) $fatal(1, "ctrl_boundary_telemetry: WIN_W must be >= 8");
  end

  always_ff @(posedge clk) begin
    if (!rst_n || clear) begin
      accepted_reads       <= '0;
      accepted_writes      <= '0;
      stalled_cycles       <= '0;
      rq_occ_max           <= '0;
      wq_occ_max           <= '0;
      cmds_issued          <= '0;
      data_busy_cycles     <= '0;
      sched_blocked_cycles <= '0;
      window_cycles        <= '0;
      rq_ever_full         <= 1'b0;
      wq_ever_full         <= 1'b0;
    end else begin
      if (window_cycles != 32'hFFFF_FFFF) window_cycles <= window_cycles + 1'b1;

      // §8: read and write accepted separately. A combined count
      // cannot show a read/write mix change, which is the thing that
      // drives turnaround cost.
      if (req_valid && req_ready) begin
        if (req_is_read) begin
          if (accepted_reads != 32'hFFFF_FFFF) accepted_reads <= accepted_reads + 1'b1;
        end else begin
          if (accepted_writes != 32'hFFFF_FFFF) accepted_writes <= accepted_writes + 1'b1;
        end
      end

      // A cycle where the requester offered and the controller could
      // not take it. This is the SoC-visible backpressure signal, and
      // 29.1 §2 is explicit that it says nothing about DRAM yet.
      if (req_valid && !req_ready && stalled_cycles != 32'hFFFF_FFFF)
        stalled_cycles <= stalled_cycles + 1'b1;

      if (rq_occ > rq_occ_max) rq_occ_max <= rq_occ;
      if (wq_occ > wq_occ_max) wq_occ_max <= wq_occ;
      if (rq_occ == RQ_DEPTH[$clog2(RQ_DEPTH+1)-1:0]) rq_ever_full <= 1'b1;
      if (wq_occ == WQ_DEPTH[$clog2(WQ_DEPTH+1)-1:0]) wq_ever_full <= 1'b1;

      if (cmd_issued && cmds_issued != 32'hFFFF_FFFF)
        cmds_issued <= cmds_issued + 1'b1;

      // The DRAM-side half of 29.1 §2's discrimination. data_busy is
      // the measurement that separates "DRAM saturated" from "queue
      // full for another reason".
      if (data_busy && data_busy_cycles != 32'hFFFF_FFFF)
        data_busy_cycles <= data_busy_cycles + 1'b1;

      // And the direct evidence of the third case: the scheduler had
      // requests and no LEGAL candidate. Without this, a full queue
      // with an idle bus is a mystery rather than a finding.
      if (sched_blocked && sched_blocked_cycles != 32'hFFFF_FFFF)
        sched_blocked_cycles <= sched_blocked_cycles + 1'b1;
    end
  end
endmodule

sched_blocked_cycles is the field that turns §15's first row from a mystery into a finding. A full queue with an idle data bus has two explanations — the scheduler had no legal candidate, or it had candidates and chose not to issue. Counting the blocked cycles distinguishes them directly, and without it the investigation is a waveform hunt.

rq_ever_full and wq_ever_full are sticky. A queue that filled once during a run and a queue that never filled are different findings, and a sampled occupancy reading misses the event entirely — the same argument 28.3 §4 makes for a sticky miss flag.

12. What the Assertions Prove

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// Bound to §10's arbiter and §11's telemetry. Every property carries
// `disable iff (!rst_n)`, and every antecedent is covered below.
//
// A NOTE ON LIVENESS: the starvation bound IS a liveness property, and
// it is stated in a BOUNDED form -- "granted within AGE_MAX grant
// opportunities" -- rather than as an unbounded eventuality. A bounded
// form is checkable without assuming the environment offers service
// forever, which an unbounded one would require.
module ctrl_soc_sva #(
  parameter int SOURCES = 4, parameter int AGE_W = 10, parameter int AGE_MAX = 256,
  parameter int RQ_DEPTH = 32
)(
  input logic clk, rst_n, clear, downstream_ready,
  input logic [SOURCES-1:0] req, grant,
  input logic grant_valid,
  input logic [AGE_W-1:0] age [SOURCES], max_age [SOURCES],
  input logic [31:0] served [SOURCES],
  input logic [15:0] age_forced, starvation_events,
  input logic req_valid, req_ready, cmd_issued, data_busy, sched_blocked,
  input logic [31:0] accepted_reads, accepted_writes, stalled_cycles,
  input logic [31:0] cmds_issued, data_busy_cycles, sched_blocked_cycles, window_cycles,
  input logic [$clog2(RQ_DEPTH+1)-1:0] rq_occ, rq_occ_max,
  input logic rq_ever_full
);
  // ---- P1. FORBIDDEN. At most one source is granted per cycle. A
  // two-hot grant would admit two requests into a single-entry path.
  property p_one_hot_grant;
    @(posedge clk) disable iff (!rst_n)
      grant_valid |-> $onehot(grant);
  endproperty
  assert property (p_one_hot_grant)
    else $error("arbiter: grant is not one-hot");

  // ---- P2. FORBIDDEN. A source is never granted unless it requested.
  property p_grant_implies_request;
    @(posedge clk) disable iff (!rst_n)
      grant_valid |-> ((grant & req) == grant);
  endproperty
  assert property (p_grant_implies_request)
    else $error("arbiter: granted a source that did not request");

  // ---- P3. FORBIDDEN. No grant when downstream cannot accept. A
  // grant into a full queue loses the request.
  property p_no_grant_when_blocked;
    @(posedge clk) disable iff (!rst_n)
      !downstream_ready |-> !grant_valid;
  endproperty
  assert property (p_no_grant_when_blocked)
    else $error("arbiter: granted while downstream was not ready");

  // ---- P4. THE STARVATION BOUND, in bounded form. A source at its
  // age limit that keeps requesting, with downstream ready, is granted
  // on that cycle. §10: the bound OUTRANKS the weighted share, and
  // this is the property that holds it to that.
  property p_age_bound_forces_grant;
    @(posedge clk) disable iff (!rst_n)
      (req[0] && age[0] >= AGE_MAX && downstream_ready) |-> grant[0];
  endproperty
  assert property (p_age_bound_forces_grant)
    else $error("arbiter: a source at its age bound was not granted");

  // ---- P5. INVARIANT. Age never exceeds its field. Saturating, so
  // the bound cannot be skipped by a wrap -- which would silently
  // void the guarantee P4 states.
  property p_age_saturates;
    @(posedge clk) disable iff (!rst_n)
      (age[0] == {AGE_W{1'b1}}) |=> (age[0] >= AGE_MAX);
  endproperty
  assert property (p_age_saturates)
    else $error("arbiter: age wrapped past saturation");

  // ---- P6. A granted source's age resets. Without this the bound
  // would fire forever once reached.
  property p_grant_resets_age;
    @(posedge clk) disable iff (!rst_n)
      grant[0] |=> (age[0] == '0);
  endproperty
  assert property (p_grant_resets_age)
    else $error("arbiter: age did not reset on grant");

  // ---- P7. FORBIDDEN. An ungranted requesting source's age never
  // decreases. A decreasing age would let a source be deferred
  // indefinitely while appearing fresh.
  property p_age_monotone_while_waiting;
    @(posedge clk) disable iff (!rst_n)
      (req[0] && !grant[0] && !clear) |=> (age[0] >= $past(age[0]));
  endproperty
  assert property (p_age_monotone_while_waiting)
    else $error("arbiter: a waiting source's age decreased");

  // ---- P8. max_age bounds the live age at all times -- the
  // high-water mark cannot understate what happened.
  property p_max_age_bounds;
    @(posedge clk) disable iff (!rst_n)
      (max_age[0] >= age[0]) || (age[0] == '0);
  endproperty
  assert property (p_max_age_bounds)
    else $error("arbiter: max_age understates age");

  // ---- P9. §7's measurement advances only on a grant. Service
  // measured on anything else would not be service.
  property p_served_only_on_grant;
    @(posedge clk) disable iff (!rst_n)
      (!grant[0]) |=> (served[0] == $past(served[0]));
  endproperty
  assert property (p_served_only_on_grant)
    else $error("arbiter: served counter advanced without a grant");

  // ---- P10. INVARIANT. Data-busy cycles never exceed the window.
  // A utilisation above 100% means the two counters are driven from
  // different clocks or events -- 29.1 §2's discrimination depends on
  // this ratio being meaningful.
  property p_utilisation_is_a_fraction;
    @(posedge clk) disable iff (!rst_n)
      (data_busy_cycles <= window_cycles);
  endproperty
  assert property (p_utilisation_is_a_fraction)
    else $error("telemetry: data_busy %0d exceeds window %0d", data_busy_cycles, window_cycles);

  // ---- P11. INVARIANT. Occupancy stays within depth, and the
  // high-water mark bounds it.
  property p_occupancy_bounded;
    @(posedge clk) disable iff (!rst_n)
      (rq_occ <= RQ_DEPTH) && (rq_occ_max >= rq_occ);
  endproperty
  assert property (p_occupancy_bounded)
    else $error("telemetry: occupancy %0d out of range", rq_occ);

  // ---- P12. FORBIDDEN. The ever-full flag is sticky. §11: a queue
  // that filled once and one that never filled are different
  // findings, and a sampled reading misses the event.
  property p_ever_full_sticky;
    @(posedge clk) disable iff (!rst_n)
      (rq_ever_full && !clear) |=> rq_ever_full;
  endproperty
  assert property (p_ever_full_sticky)
    else $error("telemetry: ever_full cleared without a clear");

  // ---- P13. FORBIDDEN. Stalled cycles only accrue when the
  // requester offered and was refused. Counting anything else would
  // make backpressure look worse than it was.
  property p_stall_requires_refusal;
    @(posedge clk) disable iff (!rst_n)
      (!(req_valid && !req_ready)) |=> (stalled_cycles == $past(stalled_cycles));
  endproperty
  assert property (p_stall_requires_refusal)
    else $error("telemetry: stall counted without a refused offer");

  // ---- antecedent covers.
  cover property (@(posedge clk) disable iff (!rst_n) grant_valid);
  cover property (@(posedge clk) disable iff (!rst_n) $onehot(req));
  cover property (@(posedge clk) disable iff (!rst_n) $countones(req) > 1);
  cover property (@(posedge clk) disable iff (!rst_n) &req);
  cover property (@(posedge clk) disable iff (!rst_n) !downstream_ready && |req);
  cover property (@(posedge clk) disable iff (!rst_n) age[0] >= AGE_MAX);
  cover property (@(posedge clk) disable iff (!rst_n) age_forced != 16'd0);
  cover property (@(posedge clk) disable iff (!rst_n) starvation_events != 16'd0);
  cover property (@(posedge clk) disable iff (!rst_n) req_valid && !req_ready);
  cover property (@(posedge clk) disable iff (!rst_n) rq_ever_full);
  cover property (@(posedge clk) disable iff (!rst_n) sched_blocked);
  cover property (@(posedge clk) disable iff (!rst_n) sched_blocked && !data_busy);
  cover property (@(posedge clk) disable iff (!rst_n) data_busy);
  cover property (@(posedge clk) disable iff (!rst_n) accepted_writes != 32'd0);
endmodule

13. DV — Testing the Arbitration

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SIMULATION-ONLY. Independent reference. It models the weighted share
// with a QUEUE of grant tokens generated up front from the weights,
// rather than with decrementing credits -- a different formulation, so
// agreement about the delivered ratio is evidence.
class qos_reference #(parameter int SOURCES = 4);
  int  weight[SOURCES];
  int  served[SOURCES];
  int  wait_cycles[SOURCES];
  int  max_wait[SOURCES];
  int  age_max;

  function void configure(int w[SOURCES], int bound);
    foreach (w[i]) weight[i] = w[i];
    age_max = bound;
    foreach (served[i]) begin served[i] = 0; wait_cycles[i] = 0; max_wait[i] = 0; end
  endfunction

  // One arbitration step. The bound outranks the share, exactly as
  // §10's RTL does -- but decided by scanning waits rather than by
  // credits, so the two agree only if the policy is the same.
  function int step(bit req[SOURCES], bit ready);
    int best = -1;
    if (!ready) begin
      foreach (req[i]) if (req[i]) begin
        wait_cycles[i]++;
        if (wait_cycles[i] > max_wait[i]) max_wait[i] = wait_cycles[i];
      end
      return -1;
    end
    // forced first
    foreach (req[i])
      if (req[i] && wait_cycles[i] >= age_max) begin best = i; break; end
    // then weighted: the source with the largest remaining share ratio
    if (best < 0) begin
      real bestr = -1.0;
      foreach (req[i])
        if (req[i]) begin
          real r = (weight[i] == 0) ? -1.0
                 : real'(weight[i]) / real'(served[i] + 1);
          if (r > bestr) begin bestr = r; best = i; end
        end
    end
    foreach (req[i]) begin
      if (req[i] && i == best) begin
        served[i]++;
        wait_cycles[i] = 0;
      end else if (req[i]) begin
        wait_cycles[i]++;
        if (wait_cycles[i] > max_wait[i]) max_wait[i] = wait_cycles[i];
      end
    end
    return best;
  endfunction

  // §7: the delivered ratio, which is the number that matters.
  function real delivered_share(int s);
    int total = 0;
    foreach (served[i]) total += served[i];
    if (total == 0) return -1.0;          // not zero: no data
    return real'(served[s]) / real'(total);
  endfunction

  function int worst_wait();
    int w = 0;
    foreach (max_wait[i]) if (max_wait[i] > w) w = max_wait[i];
    return w;
  endfunction
endclass
CheckWhat it establishes
Three sources, weights 4:2:1, all continuously requesting, 100,000 cyclesDelivered shares converge toward 4:2:1 in both models
One source requesting onlyIt receives every grant; no bound ever fires
One source never requestingIts served stays 0; no grant to a non-requester — P2
downstream_ready low throughoutNo grants; every requester's age climbs; P3 holds
Weight 0 on one sourceIt is served only by the age bound — the bound is the guarantee
Hold one source requesting and starve it by weightsIts age reaches AGE_MAX, then it is granted — P4
Measure worst_wait() over a long runNever exceeds AGE_MAX in either model
All sources requesting every cycleGrant is one-hot every cycle; P1
Raise AGE_MAX to its field maximumThe bound never fires; the arbiter degenerates to fixed priority
Read delivered_share with no grants yetReturns −1.0, not 0.0
Drive occupancy to RQ_DEPTHrq_ever_full sets and stays — P12
Offer with req_ready low for N cyclesstalled_cycles rises by exactly N — P13
Drive data_busy more cycles than the windowP10 fires
Run with no requestersAll 13 properties pass; all 14 covers empty

Two runs are worth publishing, and the second is §7's composition failure made concrete:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
  TWO PASSING CONTROLLER-BOUNDARY REPORTS

  (A) the arbiter was never exercised
        all 13 properties            PASS
        grant_valid                  never asserted
        served[*]                    all 0
        max_age[*]                   all 0
        starvation_events            0
        ------------------------------------------------
        cover grant_valid            0 hits
        cover age >= AGE_MAX         0 hits
        ... all 14 covers            0 hits

        six of the thirteen properties are implications and never
        armed. The seven INVARIANTS -- one-hot on no grant, age
        saturation, max_age bound, utilisation fraction, occupancy
        bound, ever-full sticky, stall accounting -- all pass on
        zeros.

        "QoS arbitration: no starvation events" is produced by this
        run AND by a correctly tuned one.

  (B) both layers correct, the composition wrong
        stimulus : CPU class weight 7, DMA class weight 3, both
                   continuously requesting; CPU stream scattered,
                   DMA stream sequential
        served[CPU]                  69,880
        served[DMA]                  30,120
        delivered_share(CPU)         0.699      <-- as configured
        starvation_events            0
        max_age[CPU]                 11
        ------------------------------------------------
        all 13 properties            PASS
        all 14 covers                HIT

        and the CPU class received 41% of the DELIVERED DRAM
        BANDWIDTH, not 70%.

        the arbiter did exactly what it was configured to do: 70% of
        GRANTS went to the CPU class. Each CPU grant admitted a
        scattered request that the scheduler then served expensively
        -- often a row conflict -- while each DMA grant admitted a
        request the scheduler could serve as a row hit.

        §7: entries are opportunity, not service. The conversion rate
        is each source's own locality, which this layer cannot see.

    diagnosis : (A) is the vacuity case at the SoC boundary. (B) is
      the composition problem, and NO property catches it -- both
      layers satisfy their own contracts. served[] shows the grant
      ratio; only a bandwidth measurement per source shows the
      service ratio, and the gap between them is the finding.

    the fix : (A) read the covers. (B) measure delivered BANDWIDTH
      per source, not grants per source -- and accept that a share
      guarantee needs either a source-aware scheduler or empirical
      tuning, per §7.

Report (B) is the chapter's most important result. Two correct layers, every property passing, a delivered grant ratio exactly as configured, and a bandwidth share nobody specified. It is the clearest possible demonstration that QoS is not a field you set.

14. Where Latency Comes From at This Layer

Chapter 29.1 §14 argued against a universal latency equation and listed the contributors. This chapter adds the two that belong to the controller boundary specifically, and both are policy rather than physics.

ContributorDetermined byReducible?
Source-arbitration delay§10's weights and the offered load from othersby weight, at another source's cost
Queueing delayarrival rate versus service rateby more service or less arrival, not by queue depth
Scheduling delay17.4, 23.4by policy, at throughput's cost
Row-state penaltythe stream's locality, not this requestonly by changing the stream

The second row is counter-intuitive enough to state plainly: a deeper queue does not reduce queueing delay. It increases the number of requests that can wait. CURRICULUM-DERIVED from 23.2's workload models: if arrivals exceed service, occupancy grows until something backpressures, and a deeper queue simply moves the backpressure later and makes the wait longer. A deeper queue buys burst absorption, not throughput.

And the fourth row is the module's recurring point. The row-state penalty for this request was set by the previous traffic to that bank. No controller policy can undo a stream with no locality29.5 §8 develops it, and §17 here states the boundary.

15. Failure Modes at the Controller Boundary

Six symptoms, with the discriminating measurement each needs. Module 28 owns the methodology; this is its application at this layer.

SymptomCandidate causesDiscriminating measurement
High latency, low bandwidthbank conflicts; scheduler blocked; refreshrq_occ_max and data_busy_cycles and sched_blocked_cycles together
One source slow, others fineweights; §7's composition; that source's own localityserved[] and delivered bandwidth per source
A source occasionally very slowthe age bound firing latemax_age[] against AGE_MAX
Write queue full, reads finereads prioritised; write drain starvedwq_ever_full with accepted_writes rising
Read queue full, bus idleno legal candidatesched_blocked_cycles — §11's decisive field
Bandwidth below expectation, no queue pressurethe requesters are not offering enoughstalled_cycles near zero and occupancy low

Row six is the one teams overlook. Low bandwidth with an empty queue and no stalls means the memory system is not the bottleneck — the requesters are dependency-limited or not generating enough concurrency, which is 29.3 §5's subject. Adding controller capacity would change nothing.

And row one requires three measurements read together, which is why §11 publishes all three. Any one of them alone is consistent with several causes, and the combination is what localises.

16. Misconceptions

“The controller is a DDR block with a bus port.” §1. It is an adapter between two contracts that share no vocabulary, and the state it holds belongs to neither side.

“Higher priority should always win.” §6. Priority cannot unmask an illegal command, strict priority starves under sustained load, and priority order destroys row locality.

“Setting AxQOS gives me a latency guarantee.” §6's callout. The protocol carries the field; it does not define what any controller does with it. A guarantee is an implementation claim.

“70% of queue entries gives me 70% of bandwidth.” §7, report (B). Entries are opportunity; the conversion to service depends on your own locality, which the QoS layer cannot see.

“A full request queue means DRAM is saturated.” §15, and 29.1 §2. Read the data-bus utilisation and the scheduler-blocked count.

“A deeper queue reduces latency.” §14. It increases how many requests can wait. It buys burst absorption, not throughput.

“Multiple controller ports isolate requesters.” §9. Ports isolate flow control. All ports converge on one scheduler and one device, so service is still shared.

“A port is like a sub-channel.” §9. Chapter 25.2 owns sub-channels, which are independent at the device level. Two ports share a device; two sub-channels do not.

“The DDR controller manages cache coherence.” §3. DRAM has no notion of ownership or dirtiness. The coherence system resolves it above the controller and issues an ordinary memory request.

“Adding coherent requesters adds DDR traffic.” §3. A snoop that hits in a peer cache produces no DDR request at all, so coherent requesters can absorb traffic as well as create it.

“Reads should always beat writes.” §8. Until the write queue fills, at which point acceptance blocks and a write problem becomes a read problem.

“Separate read and write queues are an optimisation detail.” §8. They exist because turnaround costs time, and their occupancies mean opposite things about the workload.

17. Interview Reasoning

What is a memory controller, architecturally? An adapter between two contracts that share no vocabulary: upstream promises correct completion in a permitted order without a time; downstream promises every command is legal without saying whose request it serves.

Why can't the controller be thin? Because it must hold state belonging to neither side — outstanding identities, per-bank row state, per-resource timing history, the refresh obligation, and per-source accounting.

Why is "highest priority wins" wrong? Priority cannot make an illegal command legal, strict priority starves under sustained load, and priority order destroys row locality.

What does an AXI QoS field guarantee? Nothing by itself. The protocol carries the value; what a controller does with it is implementation policy, and a deadline guarantee is a configuration claim about a specific controller.

You configure a source for 70% of queue entries and it gets 41% of bandwidth. What happened? Entries are opportunity. That source's requests were expensive to serve — scattered, causing row conflicts — while another source's were cheap. The conversion rate is locality, which the QoS layer cannot see.

How do you fix it? Either make the scheduler source-aware, accepting that it contaminates the legality/policy separation, or measure delivered bandwidth per source and tune empirically. There is no third option that guarantees a share.

Why separate read and write queues? Turnaround costs time, so grouping same-direction accesses raises bandwidth. And their occupancies are different signals — a full write queue means writes are being starved, a full read queue means reads cannot be served.

Your request queue is full and the data bus is idle. What do you read next? The scheduler-blocked count. It distinguishes "no legal candidate" from "candidates available and not chosen".

Does a deeper queue reduce latency? No. It increases how many requests can wait. It absorbs bursts; it does not add service.

Does DRAM participate in cache coherence? No. It has no notion of ownership, sharing or dirtiness. Coherence is resolved above the controller, which then receives an ordinary memory request — which is why a CHI memory-facing front end can be coherency-free.

18. Exercises

  1. §1 states two promises that mention nothing of each other. List the controller state each promise forces, and identify the one piece of state both need.

  2. §6 lists eight objectives. Choose any three and construct the workload where optimising each in turn gives three different schedulers.

  3. §7's composition failure gives 70% of entries and 41% of bandwidth. Derive the delivered share as a function of the two sources' row-hit rates, stating your assumption about service cost.

  4. §8 argues reads-always-first eventually becomes a read problem. Derive the write-queue depth at which that transition happens, given an offered write rate and a write service rate.

  5. §10's age bound outranks the weighted share. Construct the value of AGE_MAX at which the arbiter is effectively strict priority, and say what property would still pass.

  6. §11 publishes three fields for §15's first row. Show that no two of them suffice, by constructing two different causes that agree on any chosen pair.

  7. §14 claims a deeper queue does not reduce queueing delay. State the condition under which that claim is false, and say whether it describes a steady state.

  8. A colleague proposes giving the real-time display engine the highest priority and no age bound. Identify every failure this permits, and design the minimal change that keeps its deadline while bounding the damage.

19. Where This Goes

A memory controller is a semantic adapter between two contracts that disagree about what a request is, what order means, what fairness means, and whether priority should matter. Its upstream promise mentions no time and its downstream promise mentions no requester, so all the reconciling state is its own. It arbitrates twice — once among sources at the front end and once among queued requests in the scheduler — and the two layers cannot see each other. QoS is arbitration under eight simultaneous constraints, not a priority field.

Four results carry forward. Entries are opportunity, not service, and the conversion rate is each source's own locality — which is why §10 measures delivered service instead of assuming it. A starvation bound must outrank the weighted share or it is a preference, and 17.4 already demonstrated what a preference is worth. Occupancy and utilisation must be read together, because a full queue and a busy device are independent facts. And a deeper queue buys burst absorption, not throughput.

Two things stay open. No property catches §7's composition failure — both layers satisfy their own contracts, and only a per-source bandwidth measurement reveals the gap. And real IP boundaries do not match the contract boundaries, so a problem crossing an integrated interface may have no observable evidence at the point where the contracts meet.

The next three chapters change direction. This chapter and the last treated traffic as given and asked what the path does to it. Chapter 29.3 onward asks where the traffic comes from — and the first answer is the least intuitive, because a CPU load usually generates no DDR traffic at all. What reaches the controller from a CPU is the residue left after the cache hierarchy has absorbed almost everything, and the shape of that residue has more to do with dependency structure than with how many loads the program executed.

Continue learning

Standards & specifications

Governing standard
JEDEC JESD79 (DDR SDRAM)(opens JEDEC Solid State Technology Association in a new tab)

Defines the DDR SDRAM device itself — signals, command encoding, mode registers, timing parameters and the initialisation sequence — one document per generation. Memory-controller microarchitecture, address-mapping policy, PHY training algorithms and board-level design are not specified by it.

This page also covers RTL structure, verification approach and debugging technique. Those are engineering practice built on the standard, not requirements the standard itself imposes.

Where this fits

Part of the DDR curriculum.