AMBA AXI · Module 13

Latency Analysis

Decomposing AXI transaction latency into its contributors — address/arbitration, interconnect decode/routing, conversion, slave access, and response return — plus first-word vs last-word latency and why latency is hidden, not eliminated.

Bandwidth (Chapter 13.1) is the aggregate rate; latency is the per-transaction time — how long from issuing a transaction to getting its result. They're different axes: a path can be high-bandwidth yet high-latency (lots of data, but each transaction takes a while). Latency matters for responsiveness, for real-time deadlines, and — crucially — because it sets the outstanding depth needed to hide it (Little's law, feeding Chapter 13.3). This chapter decomposes a transaction's latency into its contributors (address, decode, conversion, slave access, response return), distinguishes first-word vs last-word latency, and explains why latency is hidden, not eliminated.

1. Latency Is Additive Along the Path

A transaction's latency is the sum of the time at each stage along its path from manager to subordinate and back:

Address/arbitration — time for the address to be accepted, including any arbitration wait if the subordinate is contended (Chapter 12.4).
Interconnect decode/routing — the decoder's latency (Chapter 12.3) plus any pipeline registers through the fabric.
Conversion — cycles added by width converters or clock-domain crossings on the path (Chapter 12.7).
Slave/memory access — the subordinate's own time to produce the data or accept the write (DRAM row activation, a peripheral's response time) — usually the largest contributor.
Response/data return — time for the data/response to travel back through the interconnect (decode/CDC/conversion again, in reverse).

These add up: latency = address/arb + decode + conversion + slave access + return. The slave access often dominates (DRAM is tens of cycles), but on a deep interconnect the fabric stages and conversions add meaningfully. Knowing the breakdown tells you where the time goes — and which stage to attack if latency is too high.

Figure 1 — latency contributors along a transaction's path. The manager's request traverses address/arbitration → interconnect decode/routing → conversion (if any) → slave access (often the largest) → and the response returns back through the fabric. Latency is the sum of all these stages; the slave access typically dominates but fabric/conversion stages add up on a deep interconnect.

2. On the Wire — First-Word Latency

For a read, latency is most visible as the gap between the address being accepted and the first data beat returning:

latency-breakdown — read first-word latency then burst streaming

8 cycles

Figure 2 — latency-breakdown: a read's first-word latency. ARVALID is accepted, then several cycles pass (address/arb + decode + slave access + return) before the first R beat (D0) arrives — the first-word latency. After that, the burst's beats stream back at full rate. The initial gap is the latency; the streaming is bandwidth.

3. Read vs Write, First-Word vs Last-Word

Latency is measured differently per direction and per metric:

Read latency: from the AR address being accepted to the first R beat (first-word latency) — and to the last R beat (last-word latency = first-word + (beats−1) × beat-time) for the whole burst.
Write latency: from AW/W being accepted to the B response — the write isn't complete (globally visible) until B returns, even if W was accepted earlier (the RAW-dependency point, Chapter 9.2).

The first-word vs last-word distinction matters: a CPU waiting on a single critical word cares about first-word latency (get me the word I stalled on — hence WRAP critical-word-first, Chapter 7.4); a bulk transfer cares about last-word (when is it all here). For a burst, first-word latency is the fixed pipeline+access cost; the rest is bandwidth (beats streaming). So latency dominates short transfers (mostly the fixed first-word cost) while bandwidth dominates long ones (mostly streaming).

Figure 3 — read vs write, first-word vs last-word. Read latency: AR accepted → first R (first-word) → last R (last-word = first-word + remaining beats). Write latency: AW/W accepted → B response (write not complete until B). First-word latency dominates short transfers; bandwidth (streaming) dominates long ones. Critical-word-first cares about first-word; bulk cares about last-word.

4. Latency Is Hidden, Not Eliminated

The crucial performance insight: latency cannot be eliminated — only hidden. The slave's access time, the fabric's pipeline depth, the conversion cycles are physical; a single transaction always pays them. What you can do is overlap the latency of many transactions so the aggregate throughput doesn't suffer — that's exactly outstanding transactions (Chapter 8.1) and Little's law: with enough in flight, the data channel stays busy through each transaction's latency, so the latency is hidden behind concurrency even though every individual transaction still takes the full time.

So latency and bandwidth decouple: you hide latency (via outstanding depth) to protect bandwidth, but a single transaction's latency is irreducible. This matters for two cases the hiding doesn't help: a dependency chain (each transaction waits on the previous one's result — can't overlap, so you pay full latency each time, e.g., pointer chasing) and real-time deadlines (a transaction must complete within a bounded time — worst-case tail latency under contention matters, not average). For those, you must actually reduce latency (faster slave, shallower fabric, higher QoS to cut arbitration wait), not just hide it. The outstanding-depth needed to hide latency for throughput is the subject of Chapter 13.3.

Figure 4 — latency is hidden, not eliminated. A single transaction always pays the full latency (slave access + fabric + conversion — physical, irreducible). Outstanding transactions overlap many latencies so aggregate throughput is protected (latency hidden behind concurrency). But hiding fails for dependency chains (can't overlap) and real-time deadlines (tail latency) — those require actually reducing latency.

5. Common Misconceptions

Myths to unlearn

"Latency and bandwidth are the same thing." No — latency is per-transaction time, bandwidth is aggregate rate. A path can be high-bandwidth and high-latency at once.
"More outstanding transactions reduce latency." They hide latency (protect throughput), they don't reduce a single transaction's latency — that stays the full physical time.
"The interconnect adds negligible latency." A deep fabric (decode pipeline, CDC, width conversion) adds real cycles each way; on a multi-hop path they add up, though the slave access usually still dominates.
"A write is done when W is accepted." It's complete (globally visible) only at the B response — write latency runs to B, which matters for read-after-write ordering (Chapter 9.2).
"Hiding latency always works." It fails for dependency chains (can't overlap serial-dependent transactions) and real-time deadlines (tail latency) — those need latency reduced, not hidden.
"First-word and last-word latency are the same." For a burst they differ by the streaming time of the remaining beats; short transfers are first-word-dominated, long ones bandwidth-dominated.

6. Debugging Insight

🔍 Debugging insight — decompose latency to find the dominant contributor

When latency is too high, decompose it to find where the time goes:

Measure the breakdown — address/arb, decode/route, conversion, slave access, return. The largest contributor is the target. Often it's the slave access (DRAM page miss, slow peripheral), but on a deep fabric it can be the interconnect.
First-word latency high? The fixed pipeline+access cost is large — check slave access time (page policy, banking) and fabric depth (decode pipelining, CDC, conversion stages).
Latency spikes under load (tail latency)? Arbitration wait at a contended subordinate (Chapter 12.4) — the transaction is queuing behind others. Raise its QoS or reduce contention.
Throughput fine but a dependency chain is slow? Latency can't be hidden for serial-dependent transactions (each waits on the prior) — you pay full latency each hop. Reduce latency (faster path) or restructure to remove the dependency (prefetch, parallelize).
Write "completes" but a later read sees old data? Confusing W-accept with B — the write isn't done until B; a dependent read must wait for B (RAW, Chapter 9.2).
Conversion-heavy path slow? Width/clock converters add latency each way — minimize conversions on latency-critical paths.

The triage: decompose → biggest contributor; high fixed cost → slave/fabric; load-dependent spikes → arbitration/contention; dependency-chain slow → irreducible latency (reduce, don't hide). Latency debugging is always "where do the cycles go?"

7. Verification Insight

✅ Verification insight — measure latency distribution, not just averages

Latency is a performance property, so verify by measuring — and measure the distribution, not just the average, because tail latency drives real-time correctness. Build monitors that timestamp each transaction's issue and completion and compute first-word and last-word latency per transaction, then report the distribution (min, mean, and especially worst-case/tail) per path. Assert latency targets: average within budget, and — critically for real-time paths — the worst-case within the deadline under worst-case contention (all masters loading the shared subordinate). Decompose where possible (instrument fabric stages) so a latency miss points at the contributor.

Coverage should include: uncontended latency (the floor — fabric + slave with no competition); contended latency (the tail — under maximum multi-master load); first-word vs last-word for various burst lengths; write latency to B; and dependency-chain scenarios (serial-dependent transactions, where latency can't be hidden — confirm the chain's total time). The highest-value measurement is the worst-case (tail) latency under contention — it's what real-time deadlines depend on and what average-case testing completely misses. Like bandwidth, latency bugs are correct-but-slow (a transaction that completes correctly but too late causes a real-time failure), so dedicated latency measurement under worst-case load is essential — and the uncontended-vs-contended gap quantifies the contention cost (feeding QoS/arbitration tuning).

8. Interview Questions

9. Summary

Transaction latency is the per-transaction time (distinct from bandwidth's aggregate rate), and it's the sum of contributors along the path: address/arbitration, interconnect decode/routing, conversion (width/CDC), slave access (usually the largest), and response return. For reads, first-word latency (to the first beat) is the fixed pipeline+access cost and last-word latency adds the streaming time of the remaining beats; for writes, latency runs to the B response (a write isn't complete until B). Short transfers are first-word/latency-dominated; long ones bandwidth-dominated.

The defining principle: latency is hidden, not eliminated — the contributors are physical, so a single transaction always pays the full time, but outstanding transactions overlap many latencies to protect aggregate throughput (Little's law, Chapter 13.3). Hiding fails for dependency chains (serial-dependent transactions can't overlap) and real-time deadlines (tail latency), which require actually reducing latency (faster/closer slave, shallower fabric, QoS to cut arbitration wait). Latency bugs are correct-but-slow, so verify by measuring the distribution and tail under worst-case contention — the tail is what real-time deadlines depend on and what average-case testing misses. Next: outstanding depth and buffering — sizing the concurrency that hides this latency.

10. What Comes Next

You've decomposed latency; next, sizing the depth that hides it:

13.3 — Outstanding Depth & Buffering (coming next) — relating outstanding depth and buffer sizing to latency hiding (the bandwidth-latency product in practice).

Previous: 13.1 — Bandwidth & Throughput. Related: 8.1 — Why Outstanding Transactions Exist for latency hiding, and 12.4 — Arbiter & Arbitration for the contention that drives tail latency. For the broader protocol catalog, see the AMBA family overview doc.