AMBA AXI · Module 19

DMA-to-DDR Transfers

Trace a DMA engine streaming data to a DDR controller over AXI — a throughput-oriented path contrasting the latency-sensitive CPU path: long INCR bursts, deep outstanding for sustained bandwidth, how the DDR controller's row/bank scheduling shapes efficiency, and why DMA traffic is engineered to maximize bus and DRAM utilization.

The CPU-memory path (19.1) is latency-sensitive — a core waits on each miss, so the design hides latency. A DMA-to-DDR path is the opposite: throughput-oriented. A DMA engine moves bulk data (a network packet to memory, a frame buffer, a disk block) and doesn't care about per-transaction latency — it cares about sustained bandwidth. This flips the AXI design emphasis: where the CPU path optimizes for fast critical-word return, the DMA path optimizes for long bursts, deep outstanding, and full bus/DRAM utilization. Tracing a DMA write stream to a DDR controller shows the throughput-oriented use of AXI, how it contrasts with the CPU path, and how the DDR controller's internal scheduling interacts with the AXI traffic to determine real efficiency. This is the second canonical real path, and the contrast with the CPU path is the lesson.

1. The Path and the Throughput Mindset

A DMA engine is an AXI manager that streams data between a source and a destination — here, writing bulk data to DDR through the interconnect and a DDR memory controller. Unlike the CPU, the DMA isn't blocked waiting; it's been told "move N bytes from A to B" and its goal is to do so as fast as possible. So every AXI choice is throughput-oriented: maximize the data carried per address phase (long bursts), keep many transactions in flight (deep outstanding), and use the full bus width — the design question is "how do we saturate the bus and DRAM?", not "how do we return one word fast?".

Figure 1 — the DMA-to-DDR path and its throughput mindset. A DMA engine (AXI manager) streams bulk data through the interconnect to a DDR memory controller (AXI subordinate) driving DRAM. Unlike the latency-sensitive CPU path, the DMA's goal is sustained bandwidth — it isn't blocked waiting, so every AXI choice optimizes throughput: long bursts (max data per address), deep outstanding (many in flight), full bus width. The design question is 'how do we saturate the bus and DRAM?', not 'how do we return one word fast?'.

2. Long INCR Bursts: Amortizing Overhead

The DMA's first throughput lever is long INCR bursts. Each burst pays an address-phase overhead once and then streams many data beats, so longer bursts amortize that overhead over more data — driving efficiency toward 100%. A DMA moving a contiguous region issues INCR bursts as long as allowed (up to 256 beats in AXI4), bounded by the 4 KB rule (a burst can't cross a 4 KB page), so it typically issues max-length bursts within each 4 KB region. This is the inverse of the CPU's short, line-sized WRAP bursts: the DMA wants the longest bursts possible because it's optimizing throughput, not latency.

Long INCR write burst from DMA

11 cycles

Figure 2 — long INCR bursts amortize address overhead. Each burst pays one address phase then streams many data beats, so the longer the burst, the smaller the per-beat overhead and the higher the efficiency. A DMA moving a contiguous region issues the longest INCR bursts allowed (up to 256 beats), bounded by the 4 KB rule (no burst crosses a page), so it streams max-length bursts within each 4 KB region. This is the inverse of the CPU's short line-sized WRAP bursts — the DMA optimizes throughput, so it wants the longest bursts possible.

3. Deep Outstanding and DDR Row/Bank Scheduling

The DMA's second lever is deep outstanding — issuing many bursts before earlier ones complete — to keep the bus and DRAM continuously busy across the DDR controller's latency (Little's Law again: depth ≈ bandwidth × latency). But the DDR controller adds a twist the SRAM-like CPU model didn't: DRAM has rows and banks, and accessing an already-open row (a "row hit") is far faster than opening a new row ("row miss" → precharge + activate). So the DDR controller reorders the outstanding AXI requests to maximize row hits and bank parallelism — which is why it needs multiple outstanding requests to reorder among, and why AXI's out-of-order completion (different IDs) matters here.

Figure 3 — deep outstanding feeds the DDR controller's row/bank scheduling. The DMA issues many outstanding bursts; the DDR controller reorders them to maximize row hits (accessing an already-open DRAM row is far faster than opening a new one) and to exploit bank parallelism. This is why the DDR controller needs multiple outstanding requests — to have a pool to reorder among — and why AXI's out-of-order completion (different IDs) matters: the controller returns responses in the order that's most DRAM-efficient, not issue order.

4. The Contrast: Throughput vs. Latency Optimization

The DMA-to-DDR and CPU-to-memory paths optimize opposite things, and the contrast is the lesson. The CPU path is latency-sensitive: short line-sized WRAP bursts, critical-word-first (unblock the core ASAP), and outstanding to hide latency. The DMA path is throughput-oriented: long INCR bursts (amortize overhead), no critical-word-first (order doesn't matter, just get it all moved), deep outstanding to sustain bandwidth, and tolerance for the DDR controller's reordering. Same protocol, opposite tuning — which is exactly why AXI's flexibility (configurable burst length/type, outstanding depth, out-of-order completion) matters: one protocol serves both the latency-critical core and the throughput-critical DMA.

Figure 4 — DMA (throughput) vs. CPU (latency) optimization, the lesson of the contrast. CPU path: short line-sized WRAP bursts, critical-word-first, outstanding to hide latency — optimized for fast single-access response. DMA path: long INCR bursts, no critical-word-first, deep outstanding to sustain bandwidth, tolerates DDR reordering — optimized for total throughput. Same AXI protocol, opposite tuning — which is exactly why AXI's flexibility (burst length/type, outstanding depth, out-of-order completion) matters: one protocol serves both.

5. Common Misconceptions

Myths to unlearn

"DMA and CPU traffic use AXI the same way." They optimize opposite things — CPU for latency (short WRAP bursts, critical-word-first), DMA for throughput (long INCR bursts, deep outstanding). Same protocol, opposite tuning; treating them identically misses the design point.
"DMA cares about per-transaction latency." No — DMA cares about sustained bandwidth. It's not blocked waiting on each transaction (unlike a core on a miss), so it tolerates latency and reordering as long as throughput stays high. Latency-per-transaction is the wrong metric for DMA.
"Longer bursts are unconditionally better for DMA." Longer amortizes overhead (good), but the 4 KB rule caps burst span, and very long bursts can hurt fairness (starving other masters) and QoS — so DMA uses max-length bursts within the 4 KB limit and the interconnect arbitrates for fairness.
"The DDR controller serves requests in order." No — it reorders outstanding requests to maximize row hits and bank parallelism (a row hit is far faster than a row miss), so responses can come back out of order (different IDs). This reordering is a major efficiency lever, and AXI's out-of-order completion enables it.
"DRAM is like SRAM — uniform access time." DRAM has rows and banks with very non-uniform access time: an open-row hit is fast, a row miss (precharge + activate) is slow. The DDR controller's whole job is scheduling AXI requests to favor row hits — which the SRAM-like model of the CPU path ignored.
"Bandwidth depends only on the AXI bus." Real DMA-to-DDR bandwidth depends on DRAM efficiency too — row-hit rate, bank parallelism, refresh overhead — so even a saturated AXI bus won't achieve peak if the access pattern causes constant row misses. The DRAM scheduling is part of the throughput story.

6. Debugging Insight

🔍 Debugging insight — diagnose DMA-to-DDR throughput shortfalls by lever

When a DMA-to-DDR path underperforms, check the throughput levers in order:

Bursts too short. Per-burst address overhead dominates — check the DMA's burst length; it should issue max-length INCR bursts within the 4 KB limit. Short bursts (or fragmentation forcing short bursts) waste address cycles.
Insufficient outstanding. The DMA can't keep enough bursts in flight to cover the DDR controller's latency (Little's Law gap) — the bus idles between bursts. Check the DMA's outstanding depth and the path's outstanding support.
Poor DRAM row-hit rate. The access pattern causes constant row misses (precharge+activate each time) — even a saturated bus won't hit peak. Check the address pattern and whether the DDR controller has enough outstanding requests to reorder for row hits; sequential access (good locality) maximizes row hits.
Address fragmentation / 4 KB crossings. Non-contiguous or page-crossing transfers force short bursts. Check the descriptor/address layout; contiguous, page-aligned regions enable long bursts.
Interconnect contention / unfair arbitration. The DMA competing with other masters (CPU, GPU) may be starved or may starve them — check QoS/arbitration and whether the DMA's long bursts are blocking latency-sensitive CPU traffic.
Width underutilization. Narrow transfers not using the full bus width waste bandwidth per beat — check AWSIZE matches the bus width for bulk moves.

The method: compute peak (bus and DRAM), measure actual, and check the levers — burst length, outstanding depth, DRAM row-hit rate, fragmentation, arbitration, width — to find the dominant limiter. The DMA-specific addition over the generic performance method is the DRAM row/bank efficiency lever, which the SRAM-like CPU model didn't have.

7. Verification Insight

✅ System insight — the DMA path teaches throughput tuning and the DRAM dimension

The DMA-to-DDR path is the throughput-oriented counterpart to the latency-sensitive CPU path, and the contrast is the central lesson: the same AXI protocol is tuned oppositely for the two paths, which is exactly why AXI's flexibility (configurable burst length and type, outstanding depth, out-of-order completion) exists — one protocol must serve both the latency-critical core and the throughput-critical DMA. The DMA's throughput levers, all studied earlier, now appear in their natural role: long INCR bursts amortize address overhead (the 4 KB rule caps them, so max-length-within-a-page); deep outstanding sustains bandwidth across the DDR controller's latency (Little's Law); full bus width maximizes data per beat; and the DMA tolerates reordering because it cares about aggregate throughput, not per-transaction order. The new dimension this path adds — invisible in the SRAM-like CPU model — is the DDR controller's row/bank scheduling: DRAM access time is highly non-uniform (open-row hit fast, row miss = precharge+activate slow), so the controller reorders the outstanding AXI requests to maximize row hits and bank parallelism, which means (1) the DMA must supply enough outstanding requests for the controller to have a reorder pool, and (2) AXI's out-of-order completion (different IDs) is what lets the controller return DRAM-optimal order — so real bandwidth depends on DRAM efficiency, not just the AXI bus. The system-level lessons: throughput tuning is a different discipline from latency tuning (maximize bursts and outstanding and DRAM locality, vs. minimize critical-word return); a saturated AXI bus is necessary but not sufficient for peak bandwidth (DRAM row-hit rate matters); and the interconnect must arbitrate fairly between the DMA's long throughput bursts and the CPU's latency-sensitive short bursts — a classic QoS tension where the DMA's bandwidth-greedy traffic can starve the latency-critical core if not managed. For verification, the DMA path stresses sustained-bandwidth scenarios (long bursts, deep outstanding, contiguous and fragmented patterns) and the DRAM-efficiency interaction (row-hit vs. row-miss patterns), complementing the CPU path's latency-hiding scenarios. The deeper takeaway, building on 19.1: real systems have multiple traffic classes with opposite requirements sharing one AXI fabric, and understanding AXI means understanding how its flexibility serves them all and how the interconnect mediates their contention — which is the system-architecture perspective the case studies build. Next, a third pattern entirely: a CPU configuring a peripheral over AXI4-Lite, where the access is small, infrequent, and control-oriented rather than bulk data.

8. Interview Questions

9. Summary

The DMA-to-DDR path is the throughput-oriented counterpart to the latency-sensitive CPU path, and the contrast is the lesson. A DMA engine (AXI manager) streams bulk data to a DDR controller (subordinate); it isn't blocked waiting, so every choice maximizes sustained bandwidth: long INCR bursts (amortize the per-burst address overhead — max-length within the 4 KB-rule limit), deep outstanding (sustain bandwidth across the controller's latency, Little's Law), and full bus width. The new dimension over the SRAM-like CPU model is the DDR controller's row/bank scheduling: DRAM access is non-uniform (open-row hit fast, row miss = precharge+activate slow), so the controller reorders outstanding requests to maximize row hits and bank parallelism — which needs a deep outstanding pool to reorder among and uses AXI's out-of-order completion (different IDs) to return DRAM-optimal order. So real DMA bandwidth depends on DRAM efficiency, not just the AXI bus.

The CPU and DMA paths optimize opposite things with the same protocol — short WRAP/critical-word-first/hide-latency vs. long INCR/deep-outstanding/sustain-bandwidth — which is precisely why AXI is flexible (configurable burst length/type, outstanding depth, out-of-order completion): one protocol serves both the latency-critical core and the throughput-critical DMA, and the interconnect arbitrates their contention (QoS — the DMA's long bursts must not starve the CPU's latency-critical misses). Achieved bandwidth is a min across levels (DMA config, interconnect, DRAM scheduling), and a well-tuned bulk DMA makes every level above DRAM non-binding so it's DRAM-bandwidth-limited. The deep lesson: the contrast reveals AXI as a parameterized space — what's fundamental (handshake, transaction model) vs. tunable (burst/outstanding/ordering) — which is the protocol's design rationale and the generative understanding the case studies build. Next, a third pattern: a CPU configuring a peripheral over AXI4-Lite — small, infrequent, control-oriented access.

10. What Comes Next

You've seen the throughput path; next, the small control-access pattern:

19.3 — AXI4-Lite CSR Access (coming next) — a CPU configuring a peripheral over AXI4-Lite, a small, infrequent, control-oriented access pattern that contrasts with both the CPU-memory and DMA-DDR bulk paths.

Previous: 19.1 — CPU-to-Memory Path. Related: 7.3 — INCR Bursts for the DMA's long bursts, 13.3 — Outstanding Depth Tuning for the bandwidth sizing, and 8.4 — Different-ID Ordering for the out-of-order completion the DDR controller exploits.