DDR · Module 29
Memory Controllers in SoCs
A semantic adapter between two sides that disagree about what a request is, what order means, what fairness means, and whether priority should matter at all.
Chapter 29.1 treated the controller as the far side of a boundary and traced one request across it. This chapter asks what the controller actually is, and the answer is not a DDR block with a bus port.
It is a semantic adapter, and its difficulty is that the two sides it adapts between disagree about almost everything:
| Question | The SoC side says | The DRAM side says |
|---|---|---|
| What is a request? | a transaction with an ID and a length | a command to a bank, legal only in some states |
| What does order mean? | same-ID order must be preserved — AXI 8.3 | order is a free variable to be optimised — 23.4 |
| What is fairness? | each requester gets its share | banks and rows have no notion of requesters |
| Does priority matter? | yes, requesters differ in urgency | a high-priority request to a busy bank still waits |
| What is the cost of a request? | its size | its size and what the previous request left in that bank |
Every row is a genuine disagreement, and the controller has to resolve all five at once. That is what makes it interesting architecture rather than glue.
1. Two Contracts, Not One Block
The controller has an interface on each side, and the two interfaces make incompatible promises.
The SoC-side contract. Chapter 17.5 owns it: a ready/valid handshake, metadata stability while stalled, one handshake per entry, and an ordering boundary. What the controller promises upstream is acceptance and eventual correct completion in a permitted order. It promises nothing about when.
The DRAM-side contract. Module 14 owns the timing parameters and Module 7 the commands. What the controller promises downstream is that every command it issues is legal in the device's current state. It promises nothing about which request that command serves.
the two promises, stated precisely
upstream : "I will complete your request correctly, in an order
your protocol permits. I will not tell you when."
downstream: "Every command I issue is legal given this device's
state and timing. I will not tell you whose request
it is."
the controller's whole job is that NEITHER PROMISE MENTIONS THE
OTHER SIDE. Upstream has no vocabulary for a bank; downstream has
no vocabulary for a requester. The adapter must satisfy both
using state that only it holds.That is the structural reason a controller cannot be thin. It must hold: the outstanding requests and their identities, the per-bank row state, the timing history per resource, the refresh obligation, and the per-source accounting that fairness requires. None of that state belongs to either side.
2. The Ownership Split
Four layers, four contracts, and the boundaries between them are not where the IP boundaries necessarily fall.
| Layer | Owns | Canonical owner |
|---|---|---|
| Interconnect | transport, routing, the ordering contract it advertises | AXI 12.1, CHI 3.7 |
| Controller | request transformation, mapping, scheduling, refresh | Module 17 |
| PHY | the physical interface, capture, training | Module 19, Module 21 |
| DRAM | device behaviour and its timing requirements | Module 7, Module 14 |
3. What the Coherent Case Changes
A memory path may pass through a coherent fabric, and it is worth being precise about what that does and does not change for DDR.
CURRICULUM-DERIVED from the CHI track: CHI 4.4 owns the Slave Node — the memory-facing node — and 5.5 / 5.6 own its coherent and IO variants. CHI 5.8 owns the memory controller in a CHI system and characterises it as a reordering DRAM controller behind a coherency-free front end.
That phrase is the important one, and it is the boundary this chapter needs.
| Owned by | Reaches DDR? | |
|---|---|---|
| Cache states and transitions | CHI 10.1 | no |
| Snoops | CHI 2.8 | no |
| Coherency resolution at the home node | CHI 4.3 | no |
| The memory request that results | CHI 4.4 | yes |
| Dirty-data writeback | CHI 2.6, 8.7 | yes, as a write |
So DRAM does not participate in coherence. It has no notion of ownership, sharing or dirtiness. The coherence system resolves all of that above the controller and then issues an ordinary memory request — which is why the front end can be coherency-free.
What the coherent path does change is the traffic's shape and timing, and that is genuinely relevant here:
A coherency resolution adds latency before the DDR request even exists. §14's composition table lists it as a contributor, and it is one of the components that is absent in a non-coherent path.
A writeback is a write the requester never issued. Chapter CHI 8.7 owns the evict flow; from the controller's side a dirty eviction is simply a write, arriving at a time determined by cache replacement rather than by any program. So a read-only workload can generate write traffic, which is 29.3 §4's subject.
And a snoop that finds data in another cache means no DDR request at all. The coherent fabric can satisfy a miss from a peer cache. So adding coherent requesters does not simply add DDR traffic — it can absorb some.
4. The Controller's Logical Regions
Two arbitration layers, and confusing them is the chapter's central error.
Source QoS in the top row decides which requester's request enters the queues. It reasons about sources, classes, shares and starvation — and it knows nothing about banks.
Command scheduler in the third row decides which queued request becomes a command. Chapter 17.4 owns it, including the layered mask that expresses policy without contaminating legality and the bounded bypass that fixes starvation at that layer. It reasons about rows, banks and timing — and it may know nothing about sources.
So a request is arbitrated twice, against different criteria, by layers that cannot see each other's state. §7 is about what that composition does.
And Refresh manager feeds the scheduler as a competitor, not a controller. Chapter 17.3 owns its drain requirement and policy dial; from this diagram's perspective, refresh is another claimant on the same device, which is why 28.3 had to separate obligation failures from service failures.
5. Several Requesters, One Device
Interference is drawn as a consequence of the bottom two shared nodes, and that is deliberate. It is a property of sharing a finite resource, not a controller defect. Any scheme that gives one requester better service gives another worse, because the device serves one command at a time.
And that is the honest framing for everything in §6. QoS does not eliminate interference. It decides how interference is distributed.
6. QoS Is Constrained Arbitration
The chapter's most important correction.
“Highest priority always wins” is not a policy — it is a bug. Three reasons, each sufficient.
First: priority cannot override physics. A high-priority request to a bank in tRC waits, and a low-priority request to an idle bank can be served meanwhile. CURRICULUM-DERIVED from 17.4: legality is a mask applied before policy, and policy cannot unmask an illegal command. So the scheduler's first filter is not priority at all.
Second: strict priority starves. Chapter 17.4 owns starvation demonstrated concretely and the bounded bypass that fixes it. A continuously-offered high-priority stream serves forever and a low-priority requester never proceeds. Starvation is not a rare corner; it is the expected behaviour of strict priority under sustained load.
Third: priority and locality conflict. Serving requests in priority order destroys row locality; serving for row locality ignores priority. Chapter 23.4 owns what reordering gains and costs.
So QoS is arbitration under several simultaneous constraints:
| Objective | Wants | Conflicts with |
|---|---|---|
| Low latency for a class | serve it now | locality, fairness |
| High bandwidth | maximise row hits | latency for a starved class |
| Row locality | group same-row requests | arrival order, priority |
| Fairness | shares proportional to weight | locality, strict priority |
| Refresh obligation | periodic device-wide service | every traffic objective — 17.3 |
| Starvation freedom | a bound on any wait | pure locality, pure priority |
| Real-time deadline | service before a fixed time | everything else |
| Read/write turnaround | group same-direction | arrival order, read priority — §9 |
Eight objectives, and no scheduler maximises all of them. The design question is not which policy is best but which objectives are constraints and which are being optimised — and that is an application decision, not a memory-controller decision.
7. Two Arbitration Layers Compose Badly
§4 showed source QoS above the queues and command scheduling below them. Neither can see the other's state, and their composition is not the sum of their intentions.
a concrete composition failure, ILLUSTRATIVE
source QoS gives the CPU class 70% of queue ENTRIES.
the scheduler optimises for ROW HITS among queued requests.
the CPU's stream is scattered -> few row hits among its entries
the DMA's stream is sequential -> many row hits among its entries
the scheduler, seeing both queues, finds most of its row hits in
the DMA's requests. It serves them preferentially -- not because
of priority, but because they are the ones that are cheap.
result: the CPU class holds 70% of the ENTRIES and receives far
less than 70% of the SERVICE.
neither layer is broken. Source QoS allocated entries as
configured. The scheduler maximised throughput as configured.
The COMPOSITION delivered something nobody specified.This is the structural reason source-level QoS alone does not deliver a service guarantee. Allocating queue entries allocates opportunity, not service — and the conversion rate from opportunity to service depends on the requester's own locality, which the QoS layer cannot see.
Two honest responses exist. Either the scheduler is made source-aware, so the two layers share state and the composition becomes designed rather than emergent — at the cost of contaminating 17.4's clean legality/policy separation. Or service is measured per source and the allocation is tuned against measurement, accepting that the mapping is empirical.
What is not available is a configuration that guarantees a share without one of those two. §11's arbiter measures per-source service for exactly this reason: the number that matters is service delivered, not entries allocated.
8. Why Read and Write Queues Are Separate
A structural question with a DRAM-side answer.
CURRICULUM-DERIVED from 14.6, which owns column-to-column spacing and the read/write turnaround cases that share the same structure: switching the data bus between reading and writing costs time. So interleaving reads and writes costs more than grouping them.
| Arrangement | Consequence |
|---|---|
| One queue, arrival order | turnarounds proportional to how often direction changes |
| Separate queues, grouped service | fewer turnarounds, more bandwidth |
| Separate queues, reads always first | fewer turnarounds, writes starve |
Row three is a real design that ships, and it is defensible up to a point: a read is usually blocking a requester and a write usually is not. But writes cannot be deferred indefinitely — the write queue fills, and a full write queue eventually blocks acceptance, which blocks the requester that was not waiting, converting a write-latency problem into a read-latency problem.
So the write queue's depth is a latency-hiding budget, and its occupancy is the signal that the budget is spent. §11's telemetry publishes it, and §15 treats a full write queue as a distinct symptom from a full read queue because they mean opposite things about the workload.
And write buffering has a second consequence worth naming. A buffered write has been accepted but not yet stored. The controller therefore holds data the memory does not yet have, which is why a read to the same address must be satisfied from the buffer rather than from DRAM. Chapter 27.4 §3 owns the verification form of exactly this hazard — the expected value is a set when a write is in flight — and it exists here as a functional requirement, not just a checking problem.
9. Multiple Ports and Traffic Classes
Why a controller may present more than one port, without implying that all do.
| Reason | What it buys |
|---|---|
| Physical separation of requesters | independent flow control; one blocked port does not block another |
| Different protocols on different ports | a coherent port and an IO port — CHI 5.5, 5.6 |
| Different widths or clocks | conversion at the port rather than in the fabric |
| Class-based accounting | per-port shares are easier to measure than per-ID shares |
The last row is the practical one. Per-source accounting requires identifying the source, and a bus ID is not a reliable source identity — AXI 12.5 owns ID-based routing, and IDs can be remapped by the interconnect. A port is a physical identity that survives the fabric, which is why per-port QoS is common and per-ID QoS is rare.
But multiple ports do not multiply the device. All ports converge on one scheduler and one DRAM. So ports provide isolation in flow control and none in service — the interference in §5 is unchanged.
CURRICULUM-DERIVED caveat: 25.2 owns sub-channels, which are independent at the device level. A sub-channel is a different thing from a controller port: two ports share a device, two sub-channels do not. Conflating them predicts isolation that does not exist.
10. The Source-Level Arbiter
// ---------------------------------------------------------------------
// source_qos_arbiter -- CLASSIFICATION: synthesisable, ILLUSTRATIVE.
//
// WHAT IT DOES: §4's TOP-ROW arbitration -- which SOURCE proceeds.
// Weighted service with a hard age bound, plus per-source measurement
// of service actually delivered.
//
// WHY AN AGE BOUND AND NOT STRICT PRIORITY (§6): strict priority
// starves, and starvation is the EXPECTED behaviour under sustained
// load rather than a corner case. The age bound converts an unbounded
// wait into a provable limit -- which is the same argument 17.4 makes
// for its bounded bypass at the command layer, applied here at the
// source layer.
//
// WHY IT MEASURES SERVICE (§7): allocating opportunity is not
// allocating service. The conversion rate depends on each source's own
// locality, which this layer cannot see. So the honest output is a
// measurement, not an assumption.
//
// WHAT IT IS NOT: the command scheduler. It has no bank state, no
// timing state and no row state, and it must not: 17.4 owns command
// arbitration and its legality/policy separation.
//
// SYNTHESIS: per-source counters, a priority encoder, one comparator
// chain. No memory.
// ---------------------------------------------------------------------
module source_qos_arbiter #(
parameter int SOURCES = 4,
parameter int WEIGHT_W = 4,
parameter int AGE_W = 10,
parameter int AGE_MAX = 256 // the starvation bound, in grant cycles
)(
input logic clk,
input logic rst_n,
input logic [SOURCES-1:0] req, // per-source request
input logic [WEIGHT_W-1:0] weight [SOURCES],// configured share
input logic downstream_ready,// queues can accept
output logic [SOURCES-1:0] grant, // exactly one, or none
output logic grant_valid,
// ---- published measurement, §7
output logic [31:0] served [SOURCES],
output logic [AGE_W-1:0] age [SOURCES],
output logic [AGE_W-1:0] max_age [SOURCES],
output logic [15:0] age_forced, // grants forced by the bound
output logic [15:0] starvation_events
);
// SOURCES of 1 makes $clog2 zero and the grant vector degenerate; a
// one-source arbiter is also not an arbiter. AGE_MAX must fit in
// AGE_W or the bound can never be reached and the guarantee is void
// -- COUNT versus INDEX applied to a threshold.
initial begin
if (SOURCES < 2) $fatal(1, "source_qos_arbiter: SOURCES must be >= 2 (got %0d)", SOURCES);
if (WEIGHT_W < 1) $fatal(1, "source_qos_arbiter: WEIGHT_W must be >= 1");
if (AGE_W < 2) $fatal(1, "source_qos_arbiter: AGE_W must be >= 2");
if (AGE_MAX < 1) $fatal(1, "source_qos_arbiter: AGE_MAX must be >= 1");
if (AGE_MAX >= (1 << AGE_W))
$fatal(1, "source_qos_arbiter: AGE_MAX %0d unreachable in AGE_W %0d bits", AGE_MAX, AGE_W);
end
// Credits implement the weighted share. A source spends a credit when
// granted and all credits are replenished when every requesting
// source is out -- which keeps the ratio without needing division.
logic [WEIGHT_W:0] credit [SOURCES];
logic [SOURCES-1:0] eligible; // requesting and holding a credit
logic [SOURCES-1:0] forced; // requesting and at the age bound
logic any_forced, any_eligible;
always_comb begin
for (int s = 0; s < SOURCES; s++) begin
eligible[s] = req[s] && (credit[s] != '0);
forced[s] = req[s] && (age[s] >= AGE_MAX[AGE_W-1:0]);
end
any_forced = |forced;
any_eligible = |eligible;
end
// Grant selection. The age bound OUTRANKS the weighted share: a
// source at its bound is granted even with no credit, which is what
// makes the bound a guarantee rather than a preference. Within each
// group the lowest index wins -- a fixed order, which is acceptable
// ONLY because the age bound prevents indefinite loss. Without the
// bound this would be strict priority and §6 explains why that is a
// bug.
logic [SOURCES-1:0] sel;
always_comb begin
sel = '0;
if (downstream_ready) begin
if (any_forced) begin
for (int s = 0; s < SOURCES; s++)
if (forced[s] && sel == '0) sel[s] = 1'b1;
end else if (any_eligible) begin
for (int s = 0; s < SOURCES; s++)
if (eligible[s] && sel == '0) sel[s] = 1'b1;
end
end
end
assign grant = sel;
assign grant_valid = |sel;
always_ff @(posedge clk) begin
if (!rst_n) begin
age_forced <= '0;
starvation_events <= '0;
for (int s = 0; s < SOURCES; s++) begin
credit[s] <= '0;
served[s] <= '0;
age[s] <= '0;
max_age[s] <= '0;
end
end else begin
// ---- replenish when no requesting source has a credit left.
// Computed from the PRE-grant credits so a grant this cycle does
// not trigger a replenish in the same cycle -- which would make
// the weights meaningless by refilling constantly.
if (!any_eligible && |req) begin
for (int s = 0; s < SOURCES; s++)
credit[s] <= {1'b0, weight[s]};
end else if (grant_valid) begin
for (int s = 0; s < SOURCES; s++)
if (sel[s] && credit[s] != '0) credit[s] <= credit[s] - 1'b1;
end
// ---- age: a requesting, ungranted source ages; a granted one
// resets. Saturating, so the bound cannot be skipped by a wrap.
for (int s = 0; s < SOURCES; s++) begin
if (sel[s]) begin
age[s] <= '0;
end else if (req[s] && age[s] != {AGE_W{1'b1}}) begin
age[s] <= age[s] + 1'b1;
end
if (age[s] > max_age[s]) max_age[s] <= age[s];
end
// ---- §7's measurement: service DELIVERED, per source.
for (int s = 0; s < SOURCES; s++)
if (sel[s] && served[s] != 32'hFFFF_FFFF) served[s] <= served[s] + 1'b1;
if (grant_valid && any_forced && age_forced != 16'hFFFF)
age_forced <= age_forced + 1'b1;
// A source reaching the bound at all is worth counting: it means
// the weighted share alone was not delivering it service, which
// is §7's composition problem becoming visible.
for (int s = 0; s < SOURCES; s++)
if (req[s] && age[s] == AGE_MAX[AGE_W-1:0]
&& starvation_events != 16'hFFFF)
starvation_events <= starvation_events + 1'b1;
end
end
endmoduleThe age bound outranks the weighted share, and that ordering is the guarantee. A source at its bound is granted even with no credit. Reversing the two would make the bound a preference, and 17.4's concrete starvation demonstration shows what a preference is worth under sustained load.
Fixed-index selection within each group is acceptable only because of the bound. Without it this block would be strict priority — §6's bug. Naming that dependency matters, because a later edit that raised AGE_MAX to a very large value would quietly turn this arbiter back into strict priority while every property still passed.
And served[] measures what §7 says cannot be assumed. Weights allocate credits; credits allocate opportunity; opportunity becomes service at a rate this layer cannot see. The counter is the only honest way to know the ratio actually delivered.
11. Two-Sided Telemetry
// ---------------------------------------------------------------------
// ctrl_boundary_telemetry -- CLASSIFICATION: synthesisable, BINDABLE.
//
// WHAT IT DOES: measures BOTH sides of the controller. Occupancy and
// acceptance on the SoC side; command issue and data-bus busy on the
// DRAM side.
//
// WHY BOTH (29.1 §2): a full request queue does not mean DRAM is busy.
// It can equally mean the scheduler is blocked on a timing constraint
// while the data bus idles. A one-sided measurement cannot distinguish
// those, and they need opposite fixes.
//
// WHY READ AND WRITE SEPARATELY (§8): a full read queue and a full
// write queue mean opposite things about the workload, and a combined
// occupancy number hides which one is happening.
//
// WHAT IT CANNOT TELL YOU: why. Module 28 owns the investigation.
//
// SYNTHESIS: counters and two max-trackers. No memory.
// ---------------------------------------------------------------------
module ctrl_boundary_telemetry #(
parameter int RQ_DEPTH = 32,
parameter int WQ_DEPTH = 32,
parameter int WIN_W = 20 // measurement window, in cycles
)(
input logic clk,
input logic rst_n,
// ---- SoC side
input logic req_valid,
input logic req_ready,
input logic req_is_read,
input logic [$clog2(RQ_DEPTH+1)-1:0] rq_occ,
input logic [$clog2(WQ_DEPTH+1)-1:0] wq_occ,
// ---- DRAM side
input logic cmd_issued,
input logic data_busy, // DQ bus active
input logic sched_blocked, // no legal candidate
input logic clear,
// ---- published
output logic [31:0] accepted_reads,
output logic [31:0] accepted_writes,
output logic [31:0] stalled_cycles,
output logic [$clog2(RQ_DEPTH+1)-1:0] rq_occ_max,
output logic [$clog2(WQ_DEPTH+1)-1:0] wq_occ_max,
output logic [31:0] cmds_issued,
output logic [31:0] data_busy_cycles,
output logic [31:0] sched_blocked_cycles,
output logic [31:0] window_cycles,
output logic rq_ever_full,
output logic wq_ever_full
);
initial begin
if (RQ_DEPTH < 2) $fatal(1, "ctrl_boundary_telemetry: RQ_DEPTH must be >= 2");
if (WQ_DEPTH < 2) $fatal(1, "ctrl_boundary_telemetry: WQ_DEPTH must be >= 2");
if (WIN_W < 8) $fatal(1, "ctrl_boundary_telemetry: WIN_W must be >= 8");
end
always_ff @(posedge clk) begin
if (!rst_n || clear) begin
accepted_reads <= '0;
accepted_writes <= '0;
stalled_cycles <= '0;
rq_occ_max <= '0;
wq_occ_max <= '0;
cmds_issued <= '0;
data_busy_cycles <= '0;
sched_blocked_cycles <= '0;
window_cycles <= '0;
rq_ever_full <= 1'b0;
wq_ever_full <= 1'b0;
end else begin
if (window_cycles != 32'hFFFF_FFFF) window_cycles <= window_cycles + 1'b1;
// §8: read and write accepted separately. A combined count
// cannot show a read/write mix change, which is the thing that
// drives turnaround cost.
if (req_valid && req_ready) begin
if (req_is_read) begin
if (accepted_reads != 32'hFFFF_FFFF) accepted_reads <= accepted_reads + 1'b1;
end else begin
if (accepted_writes != 32'hFFFF_FFFF) accepted_writes <= accepted_writes + 1'b1;
end
end
// A cycle where the requester offered and the controller could
// not take it. This is the SoC-visible backpressure signal, and
// 29.1 §2 is explicit that it says nothing about DRAM yet.
if (req_valid && !req_ready && stalled_cycles != 32'hFFFF_FFFF)
stalled_cycles <= stalled_cycles + 1'b1;
if (rq_occ > rq_occ_max) rq_occ_max <= rq_occ;
if (wq_occ > wq_occ_max) wq_occ_max <= wq_occ;
if (rq_occ == RQ_DEPTH[$clog2(RQ_DEPTH+1)-1:0]) rq_ever_full <= 1'b1;
if (wq_occ == WQ_DEPTH[$clog2(WQ_DEPTH+1)-1:0]) wq_ever_full <= 1'b1;
if (cmd_issued && cmds_issued != 32'hFFFF_FFFF)
cmds_issued <= cmds_issued + 1'b1;
// The DRAM-side half of 29.1 §2's discrimination. data_busy is
// the measurement that separates "DRAM saturated" from "queue
// full for another reason".
if (data_busy && data_busy_cycles != 32'hFFFF_FFFF)
data_busy_cycles <= data_busy_cycles + 1'b1;
// And the direct evidence of the third case: the scheduler had
// requests and no LEGAL candidate. Without this, a full queue
// with an idle bus is a mystery rather than a finding.
if (sched_blocked && sched_blocked_cycles != 32'hFFFF_FFFF)
sched_blocked_cycles <= sched_blocked_cycles + 1'b1;
end
end
endmodulesched_blocked_cycles is the field that turns §15's first row from a mystery into a finding. A full queue with an idle data bus has two explanations — the scheduler had no legal candidate, or it had candidates and chose not to issue. Counting the blocked cycles distinguishes them directly, and without it the investigation is a waveform hunt.
rq_ever_full and wq_ever_full are sticky. A queue that filled once during a run and a queue that never filled are different findings, and a sampled occupancy reading misses the event entirely — the same argument 28.3 §4 makes for a sticky miss flag.
12. What the Assertions Prove
// Bound to §10's arbiter and §11's telemetry. Every property carries
// `disable iff (!rst_n)`, and every antecedent is covered below.
//
// A NOTE ON LIVENESS: the starvation bound IS a liveness property, and
// it is stated in a BOUNDED form -- "granted within AGE_MAX grant
// opportunities" -- rather than as an unbounded eventuality. A bounded
// form is checkable without assuming the environment offers service
// forever, which an unbounded one would require.
module ctrl_soc_sva #(
parameter int SOURCES = 4, parameter int AGE_W = 10, parameter int AGE_MAX = 256,
parameter int RQ_DEPTH = 32
)(
input logic clk, rst_n, clear, downstream_ready,
input logic [SOURCES-1:0] req, grant,
input logic grant_valid,
input logic [AGE_W-1:0] age [SOURCES], max_age [SOURCES],
input logic [31:0] served [SOURCES],
input logic [15:0] age_forced, starvation_events,
input logic req_valid, req_ready, cmd_issued, data_busy, sched_blocked,
input logic [31:0] accepted_reads, accepted_writes, stalled_cycles,
input logic [31:0] cmds_issued, data_busy_cycles, sched_blocked_cycles, window_cycles,
input logic [$clog2(RQ_DEPTH+1)-1:0] rq_occ, rq_occ_max,
input logic rq_ever_full
);
// ---- P1. FORBIDDEN. At most one source is granted per cycle. A
// two-hot grant would admit two requests into a single-entry path.
property p_one_hot_grant;
@(posedge clk) disable iff (!rst_n)
grant_valid |-> $onehot(grant);
endproperty
assert property (p_one_hot_grant)
else $error("arbiter: grant is not one-hot");
// ---- P2. FORBIDDEN. A source is never granted unless it requested.
property p_grant_implies_request;
@(posedge clk) disable iff (!rst_n)
grant_valid |-> ((grant & req) == grant);
endproperty
assert property (p_grant_implies_request)
else $error("arbiter: granted a source that did not request");
// ---- P3. FORBIDDEN. No grant when downstream cannot accept. A
// grant into a full queue loses the request.
property p_no_grant_when_blocked;
@(posedge clk) disable iff (!rst_n)
!downstream_ready |-> !grant_valid;
endproperty
assert property (p_no_grant_when_blocked)
else $error("arbiter: granted while downstream was not ready");
// ---- P4. THE STARVATION BOUND, in bounded form. A source at its
// age limit that keeps requesting, with downstream ready, is granted
// on that cycle. §10: the bound OUTRANKS the weighted share, and
// this is the property that holds it to that.
property p_age_bound_forces_grant;
@(posedge clk) disable iff (!rst_n)
(req[0] && age[0] >= AGE_MAX && downstream_ready) |-> grant[0];
endproperty
assert property (p_age_bound_forces_grant)
else $error("arbiter: a source at its age bound was not granted");
// ---- P5. INVARIANT. Age never exceeds its field. Saturating, so
// the bound cannot be skipped by a wrap -- which would silently
// void the guarantee P4 states.
property p_age_saturates;
@(posedge clk) disable iff (!rst_n)
(age[0] == {AGE_W{1'b1}}) |=> (age[0] >= AGE_MAX);
endproperty
assert property (p_age_saturates)
else $error("arbiter: age wrapped past saturation");
// ---- P6. A granted source's age resets. Without this the bound
// would fire forever once reached.
property p_grant_resets_age;
@(posedge clk) disable iff (!rst_n)
grant[0] |=> (age[0] == '0);
endproperty
assert property (p_grant_resets_age)
else $error("arbiter: age did not reset on grant");
// ---- P7. FORBIDDEN. An ungranted requesting source's age never
// decreases. A decreasing age would let a source be deferred
// indefinitely while appearing fresh.
property p_age_monotone_while_waiting;
@(posedge clk) disable iff (!rst_n)
(req[0] && !grant[0] && !clear) |=> (age[0] >= $past(age[0]));
endproperty
assert property (p_age_monotone_while_waiting)
else $error("arbiter: a waiting source's age decreased");
// ---- P8. max_age bounds the live age at all times -- the
// high-water mark cannot understate what happened.
property p_max_age_bounds;
@(posedge clk) disable iff (!rst_n)
(max_age[0] >= age[0]) || (age[0] == '0);
endproperty
assert property (p_max_age_bounds)
else $error("arbiter: max_age understates age");
// ---- P9. §7's measurement advances only on a grant. Service
// measured on anything else would not be service.
property p_served_only_on_grant;
@(posedge clk) disable iff (!rst_n)
(!grant[0]) |=> (served[0] == $past(served[0]));
endproperty
assert property (p_served_only_on_grant)
else $error("arbiter: served counter advanced without a grant");
// ---- P10. INVARIANT. Data-busy cycles never exceed the window.
// A utilisation above 100% means the two counters are driven from
// different clocks or events -- 29.1 §2's discrimination depends on
// this ratio being meaningful.
property p_utilisation_is_a_fraction;
@(posedge clk) disable iff (!rst_n)
(data_busy_cycles <= window_cycles);
endproperty
assert property (p_utilisation_is_a_fraction)
else $error("telemetry: data_busy %0d exceeds window %0d", data_busy_cycles, window_cycles);
// ---- P11. INVARIANT. Occupancy stays within depth, and the
// high-water mark bounds it.
property p_occupancy_bounded;
@(posedge clk) disable iff (!rst_n)
(rq_occ <= RQ_DEPTH) && (rq_occ_max >= rq_occ);
endproperty
assert property (p_occupancy_bounded)
else $error("telemetry: occupancy %0d out of range", rq_occ);
// ---- P12. FORBIDDEN. The ever-full flag is sticky. §11: a queue
// that filled once and one that never filled are different
// findings, and a sampled reading misses the event.
property p_ever_full_sticky;
@(posedge clk) disable iff (!rst_n)
(rq_ever_full && !clear) |=> rq_ever_full;
endproperty
assert property (p_ever_full_sticky)
else $error("telemetry: ever_full cleared without a clear");
// ---- P13. FORBIDDEN. Stalled cycles only accrue when the
// requester offered and was refused. Counting anything else would
// make backpressure look worse than it was.
property p_stall_requires_refusal;
@(posedge clk) disable iff (!rst_n)
(!(req_valid && !req_ready)) |=> (stalled_cycles == $past(stalled_cycles));
endproperty
assert property (p_stall_requires_refusal)
else $error("telemetry: stall counted without a refused offer");
// ---- antecedent covers.
cover property (@(posedge clk) disable iff (!rst_n) grant_valid);
cover property (@(posedge clk) disable iff (!rst_n) $onehot(req));
cover property (@(posedge clk) disable iff (!rst_n) $countones(req) > 1);
cover property (@(posedge clk) disable iff (!rst_n) &req);
cover property (@(posedge clk) disable iff (!rst_n) !downstream_ready && |req);
cover property (@(posedge clk) disable iff (!rst_n) age[0] >= AGE_MAX);
cover property (@(posedge clk) disable iff (!rst_n) age_forced != 16'd0);
cover property (@(posedge clk) disable iff (!rst_n) starvation_events != 16'd0);
cover property (@(posedge clk) disable iff (!rst_n) req_valid && !req_ready);
cover property (@(posedge clk) disable iff (!rst_n) rq_ever_full);
cover property (@(posedge clk) disable iff (!rst_n) sched_blocked);
cover property (@(posedge clk) disable iff (!rst_n) sched_blocked && !data_busy);
cover property (@(posedge clk) disable iff (!rst_n) data_busy);
cover property (@(posedge clk) disable iff (!rst_n) accepted_writes != 32'd0);
endmodule13. DV — Testing the Arbitration
// SIMULATION-ONLY. Independent reference. It models the weighted share
// with a QUEUE of grant tokens generated up front from the weights,
// rather than with decrementing credits -- a different formulation, so
// agreement about the delivered ratio is evidence.
class qos_reference #(parameter int SOURCES = 4);
int weight[SOURCES];
int served[SOURCES];
int wait_cycles[SOURCES];
int max_wait[SOURCES];
int age_max;
function void configure(int w[SOURCES], int bound);
foreach (w[i]) weight[i] = w[i];
age_max = bound;
foreach (served[i]) begin served[i] = 0; wait_cycles[i] = 0; max_wait[i] = 0; end
endfunction
// One arbitration step. The bound outranks the share, exactly as
// §10's RTL does -- but decided by scanning waits rather than by
// credits, so the two agree only if the policy is the same.
function int step(bit req[SOURCES], bit ready);
int best = -1;
if (!ready) begin
foreach (req[i]) if (req[i]) begin
wait_cycles[i]++;
if (wait_cycles[i] > max_wait[i]) max_wait[i] = wait_cycles[i];
end
return -1;
end
// forced first
foreach (req[i])
if (req[i] && wait_cycles[i] >= age_max) begin best = i; break; end
// then weighted: the source with the largest remaining share ratio
if (best < 0) begin
real bestr = -1.0;
foreach (req[i])
if (req[i]) begin
real r = (weight[i] == 0) ? -1.0
: real'(weight[i]) / real'(served[i] + 1);
if (r > bestr) begin bestr = r; best = i; end
end
end
foreach (req[i]) begin
if (req[i] && i == best) begin
served[i]++;
wait_cycles[i] = 0;
end else if (req[i]) begin
wait_cycles[i]++;
if (wait_cycles[i] > max_wait[i]) max_wait[i] = wait_cycles[i];
end
end
return best;
endfunction
// §7: the delivered ratio, which is the number that matters.
function real delivered_share(int s);
int total = 0;
foreach (served[i]) total += served[i];
if (total == 0) return -1.0; // not zero: no data
return real'(served[s]) / real'(total);
endfunction
function int worst_wait();
int w = 0;
foreach (max_wait[i]) if (max_wait[i] > w) w = max_wait[i];
return w;
endfunction
endclass| Check | What it establishes |
|---|---|
| Three sources, weights 4:2:1, all continuously requesting, 100,000 cycles | Delivered shares converge toward 4:2:1 in both models |
| One source requesting only | It receives every grant; no bound ever fires |
| One source never requesting | Its served stays 0; no grant to a non-requester — P2 |
downstream_ready low throughout | No grants; every requester's age climbs; P3 holds |
| Weight 0 on one source | It is served only by the age bound — the bound is the guarantee |
| Hold one source requesting and starve it by weights | Its age reaches AGE_MAX, then it is granted — P4 |
Measure worst_wait() over a long run | Never exceeds AGE_MAX in either model |
| All sources requesting every cycle | Grant is one-hot every cycle; P1 |
Raise AGE_MAX to its field maximum | The bound never fires; the arbiter degenerates to fixed priority |
Read delivered_share with no grants yet | Returns −1.0, not 0.0 |
Drive occupancy to RQ_DEPTH | rq_ever_full sets and stays — P12 |
Offer with req_ready low for N cycles | stalled_cycles rises by exactly N — P13 |
Drive data_busy more cycles than the window | P10 fires |
| Run with no requesters | All 13 properties pass; all 14 covers empty |
Two runs are worth publishing, and the second is §7's composition failure made concrete:
TWO PASSING CONTROLLER-BOUNDARY REPORTS
(A) the arbiter was never exercised
all 13 properties PASS
grant_valid never asserted
served[*] all 0
max_age[*] all 0
starvation_events 0
------------------------------------------------
cover grant_valid 0 hits
cover age >= AGE_MAX 0 hits
... all 14 covers 0 hits
six of the thirteen properties are implications and never
armed. The seven INVARIANTS -- one-hot on no grant, age
saturation, max_age bound, utilisation fraction, occupancy
bound, ever-full sticky, stall accounting -- all pass on
zeros.
"QoS arbitration: no starvation events" is produced by this
run AND by a correctly tuned one.
(B) both layers correct, the composition wrong
stimulus : CPU class weight 7, DMA class weight 3, both
continuously requesting; CPU stream scattered,
DMA stream sequential
served[CPU] 69,880
served[DMA] 30,120
delivered_share(CPU) 0.699 <-- as configured
starvation_events 0
max_age[CPU] 11
------------------------------------------------
all 13 properties PASS
all 14 covers HIT
and the CPU class received 41% of the DELIVERED DRAM
BANDWIDTH, not 70%.
the arbiter did exactly what it was configured to do: 70% of
GRANTS went to the CPU class. Each CPU grant admitted a
scattered request that the scheduler then served expensively
-- often a row conflict -- while each DMA grant admitted a
request the scheduler could serve as a row hit.
§7: entries are opportunity, not service. The conversion rate
is each source's own locality, which this layer cannot see.
diagnosis : (A) is the vacuity case at the SoC boundary. (B) is
the composition problem, and NO property catches it -- both
layers satisfy their own contracts. served[] shows the grant
ratio; only a bandwidth measurement per source shows the
service ratio, and the gap between them is the finding.
the fix : (A) read the covers. (B) measure delivered BANDWIDTH
per source, not grants per source -- and accept that a share
guarantee needs either a source-aware scheduler or empirical
tuning, per §7.Report (B) is the chapter's most important result. Two correct layers, every property passing, a delivered grant ratio exactly as configured, and a bandwidth share nobody specified. It is the clearest possible demonstration that QoS is not a field you set.
14. Where Latency Comes From at This Layer
Chapter 29.1 §14 argued against a universal latency equation and listed the contributors. This chapter adds the two that belong to the controller boundary specifically, and both are policy rather than physics.
| Contributor | Determined by | Reducible? |
|---|---|---|
| Source-arbitration delay | §10's weights and the offered load from others | by weight, at another source's cost |
| Queueing delay | arrival rate versus service rate | by more service or less arrival, not by queue depth |
| Scheduling delay | 17.4, 23.4 | by policy, at throughput's cost |
| Row-state penalty | the stream's locality, not this request | only by changing the stream |
The second row is counter-intuitive enough to state plainly: a deeper queue does not reduce queueing delay. It increases the number of requests that can wait. CURRICULUM-DERIVED from 23.2's workload models: if arrivals exceed service, occupancy grows until something backpressures, and a deeper queue simply moves the backpressure later and makes the wait longer. A deeper queue buys burst absorption, not throughput.
And the fourth row is the module's recurring point. The row-state penalty for this request was set by the previous traffic to that bank. No controller policy can undo a stream with no locality — 29.5 §8 develops it, and §17 here states the boundary.
15. Failure Modes at the Controller Boundary
Six symptoms, with the discriminating measurement each needs. Module 28 owns the methodology; this is its application at this layer.
| Symptom | Candidate causes | Discriminating measurement |
|---|---|---|
| High latency, low bandwidth | bank conflicts; scheduler blocked; refresh | rq_occ_max and data_busy_cycles and sched_blocked_cycles together |
| One source slow, others fine | weights; §7's composition; that source's own locality | served[] and delivered bandwidth per source |
| A source occasionally very slow | the age bound firing late | max_age[] against AGE_MAX |
| Write queue full, reads fine | reads prioritised; write drain starved | wq_ever_full with accepted_writes rising |
| Read queue full, bus idle | no legal candidate | sched_blocked_cycles — §11's decisive field |
| Bandwidth below expectation, no queue pressure | the requesters are not offering enough | stalled_cycles near zero and occupancy low |
Row six is the one teams overlook. Low bandwidth with an empty queue and no stalls means the memory system is not the bottleneck — the requesters are dependency-limited or not generating enough concurrency, which is 29.3 §5's subject. Adding controller capacity would change nothing.
And row one requires three measurements read together, which is why §11 publishes all three. Any one of them alone is consistent with several causes, and the combination is what localises.
16. Misconceptions
“The controller is a DDR block with a bus port.” §1. It is an adapter between two contracts that share no vocabulary, and the state it holds belongs to neither side.
“Higher priority should always win.” §6. Priority cannot unmask an illegal command, strict priority starves under sustained load, and priority order destroys row locality.
“Setting AxQOS gives me a latency guarantee.” §6's callout. The protocol carries the field; it does not define what any controller does with it. A guarantee is an implementation claim.
“70% of queue entries gives me 70% of bandwidth.” §7, report (B). Entries are opportunity; the conversion to service depends on your own locality, which the QoS layer cannot see.
“A full request queue means DRAM is saturated.” §15, and 29.1 §2. Read the data-bus utilisation and the scheduler-blocked count.
“A deeper queue reduces latency.” §14. It increases how many requests can wait. It buys burst absorption, not throughput.
“Multiple controller ports isolate requesters.” §9. Ports isolate flow control. All ports converge on one scheduler and one device, so service is still shared.
“A port is like a sub-channel.” §9. Chapter 25.2 owns sub-channels, which are independent at the device level. Two ports share a device; two sub-channels do not.
“The DDR controller manages cache coherence.” §3. DRAM has no notion of ownership or dirtiness. The coherence system resolves it above the controller and issues an ordinary memory request.
“Adding coherent requesters adds DDR traffic.” §3. A snoop that hits in a peer cache produces no DDR request at all, so coherent requesters can absorb traffic as well as create it.
“Reads should always beat writes.” §8. Until the write queue fills, at which point acceptance blocks and a write problem becomes a read problem.
“Separate read and write queues are an optimisation detail.” §8. They exist because turnaround costs time, and their occupancies mean opposite things about the workload.
17. Interview Reasoning
What is a memory controller, architecturally? An adapter between two contracts that share no vocabulary: upstream promises correct completion in a permitted order without a time; downstream promises every command is legal without saying whose request it serves.
Why can't the controller be thin? Because it must hold state belonging to neither side — outstanding identities, per-bank row state, per-resource timing history, the refresh obligation, and per-source accounting.
Why is "highest priority wins" wrong? Priority cannot make an illegal command legal, strict priority starves under sustained load, and priority order destroys row locality.
What does an AXI QoS field guarantee? Nothing by itself. The protocol carries the value; what a controller does with it is implementation policy, and a deadline guarantee is a configuration claim about a specific controller.
You configure a source for 70% of queue entries and it gets 41% of bandwidth. What happened? Entries are opportunity. That source's requests were expensive to serve — scattered, causing row conflicts — while another source's were cheap. The conversion rate is locality, which the QoS layer cannot see.
How do you fix it? Either make the scheduler source-aware, accepting that it contaminates the legality/policy separation, or measure delivered bandwidth per source and tune empirically. There is no third option that guarantees a share.
Why separate read and write queues? Turnaround costs time, so grouping same-direction accesses raises bandwidth. And their occupancies are different signals — a full write queue means writes are being starved, a full read queue means reads cannot be served.
Your request queue is full and the data bus is idle. What do you read next? The scheduler-blocked count. It distinguishes "no legal candidate" from "candidates available and not chosen".
Does a deeper queue reduce latency? No. It increases how many requests can wait. It absorbs bursts; it does not add service.
Does DRAM participate in cache coherence? No. It has no notion of ownership, sharing or dirtiness. Coherence is resolved above the controller, which then receives an ordinary memory request — which is why a CHI memory-facing front end can be coherency-free.
18. Exercises
-
§1 states two promises that mention nothing of each other. List the controller state each promise forces, and identify the one piece of state both need.
-
§6 lists eight objectives. Choose any three and construct the workload where optimising each in turn gives three different schedulers.
-
§7's composition failure gives 70% of entries and 41% of bandwidth. Derive the delivered share as a function of the two sources' row-hit rates, stating your assumption about service cost.
-
§8 argues reads-always-first eventually becomes a read problem. Derive the write-queue depth at which that transition happens, given an offered write rate and a write service rate.
-
§10's age bound outranks the weighted share. Construct the value of
AGE_MAXat which the arbiter is effectively strict priority, and say what property would still pass. -
§11 publishes three fields for §15's first row. Show that no two of them suffice, by constructing two different causes that agree on any chosen pair.
-
§14 claims a deeper queue does not reduce queueing delay. State the condition under which that claim is false, and say whether it describes a steady state.
-
A colleague proposes giving the real-time display engine the highest priority and no age bound. Identify every failure this permits, and design the minimal change that keeps its deadline while bounding the damage.
19. Where This Goes
A memory controller is a semantic adapter between two contracts that disagree about what a request is, what order means, what fairness means, and whether priority should matter. Its upstream promise mentions no time and its downstream promise mentions no requester, so all the reconciling state is its own. It arbitrates twice — once among sources at the front end and once among queued requests in the scheduler — and the two layers cannot see each other. QoS is arbitration under eight simultaneous constraints, not a priority field.
Four results carry forward. Entries are opportunity, not service, and the conversion rate is each source's own locality — which is why §10 measures delivered service instead of assuming it. A starvation bound must outrank the weighted share or it is a preference, and 17.4 already demonstrated what a preference is worth. Occupancy and utilisation must be read together, because a full queue and a busy device are independent facts. And a deeper queue buys burst absorption, not throughput.
Two things stay open. No property catches §7's composition failure — both layers satisfy their own contracts, and only a per-source bandwidth measurement reveals the gap. And real IP boundaries do not match the contract boundaries, so a problem crossing an integrated interface may have no observable evidence at the point where the contracts meet.
The next three chapters change direction. This chapter and the last treated traffic as given and asked what the path does to it. Chapter 29.3 onward asks where the traffic comes from — and the first answer is the least intuitive, because a CPU load usually generates no DDR traffic at all. What reaches the controller from a CPU is the residue left after the cache hierarchy has absorbed almost everything, and the shape of that residue has more to do with dependency structure than with how many loads the program executed.
Continue learning
Related tutorials
- Related topic
Arbitration
Arbitration runs after legality has already decided. It narrows subsets rather than comparing priority numbers, advances its pointer only on commit, and needs a bounded bypass because row-hit-first genuinely starves.
- Related topic
AXI-to-DDR Flow
One AXI transaction is not one DDR command, acceptance is not service, and the ordering the bus guarantees is not the ordering the scheduler uses. This is the transformation between them.
- Related topic
CPU Access Patterns
Most loads never reach DRAM. What does is shaped less by how many loads a program executes than by its dependency structure — and that decides whether latency or bandwidth matters.
- Related topic
DMA Access Patterns
A DMA transfer is not one stream. It is a payload stream and a descriptor stream with opposite properties, and the second is the one nobody counts.
Standards & specifications
- Governing standard
- JEDEC JESD79 (DDR SDRAM)(opens JEDEC Solid State Technology Association in a new tab)
Defines the DDR SDRAM device itself — signals, command encoding, mode registers, timing parameters and the initialisation sequence — one document per generation. Memory-controller microarchitecture, address-mapping policy, PHY training algorithms and board-level design are not specified by it.
This page also covers RTL structure, verification approach and debugging technique. Those are engineering practice built on the standard, not requirements the standard itself imposes.
Where this fits
Part of the DDR curriculum.
