Mastering Memory over AXI

Chapter 18.3 and Chapter 18.4 both used a 16-beat burst limit and neither justified it. This chapter does, and the justification produces a result that inverts halfway through.

Take a 1518-octet frame on a 512-bit bus. Twenty-four data beats, and one address cycle per burst.

Two things in that table are worth reading carefully.

Longer bursts are monotonically better and the returns stop abruptly. 4 to 32 beats buys sixteen percentage points; 32 to 64 buys nothing, because a 1518-octet frame already fits in one 2048-octet burst and a larger limit has nothing to do.

And the misalignment penalty looks small: 1.40 percentage points at 32 beats.

That second reading is wrong, and the reason it is wrong is the chapter's central point.

Chapter 18.1 §12 established that the MAC is transaction-bound rather than bandwidth-bound, so the second row is the one that binds and the first is nearly irrelevant. An event that costs 1.4% of the bus's cycles costs 37% of its scarcest resource, and a design measuring efficiency will conclude alignment does not matter.

This chapter is about allocating three resources — burst length, outstanding transactions, and AXI IDs — and about the fact that the third one is not fungible.

1. Scope, and Why This Chapter Is About Resources

Chapter 18.1 to Chapter 18.4 settled the MAC's correctness. What is left is how much of a shared bus it may take.

Row four is the one this chapter exists for. Bandwidth, transactions and outstanding slots are all quantities: more is better, they are interchangeable, and allocating them is arithmetic. An AXI ID is not a quantity. Each ID carries an ordering contract — transactions sharing an ID complete in order — and two transactions that share an ID are constrained whether or not the design wanted them to be.

Which means an ID allocator that treats IDs as a pool of interchangeable tokens is correct by every obvious measure and wrong. Section 19's rejected class is exactly that.

What this chapter establishes:

What it does not do: it does not revisit the ordering requirements — those are settled — only the mechanism's cost. And it does not address multiple queues, which is Chapter 18.7.

2. The Three Limits on a Burst

Every burst this module issues is bounded by three things and the smallest wins. Chapter 18.3 §7 and Chapter 18.4 §5 both computed the minimum; neither said which limit usually binds.

Which binds depends on the frame size and it changes across a single frame.

A 1518-octet frame into a 2 KiB page-aligned buffer, 32-beat limit on a 512-bit bus: the burst limit is 2048 octets, the buffer has 2048, the page has 4096 — so the frame binds and one burst carries all 1518 octets. No limit was reached at all.

The same frame 3 KiB into a page: the page boundary is 1024 octets away. First burst 1024, second burst 494. The page bound, which the MAC cannot see and the driver chose by accident.

A 9000-octet jumbo frame into 2 KiB buffers: the buffer binds on every burst but the last, and the page bound never fires if the buffers are 2 KiB-aligned — which they are, if they came from a slab allocator.

Row four is the only one where the design's own choice is the constraint, which is worth noticing: at any sensible burst limit, the MAC's parameter is rarely what binds. The buffer size and the alignment — both the driver's — decide almost every burst.

And that has a direct consequence for where tuning effort goes. Raising MAX_BEATS from 16 to 32 moves one row of that table. Aligning the buffer pool moves another, and enlarging the buffers moves a third — and two of the three are software changes that cost nothing.

3. RTL 1 — The Burst-Length Selector

The block that takes Section 2's three limits and produces one length, and records which limit bound it.

// -----------------------------------------------------------------------
// axishape_pkg -- burst shaping, IDs and outstanding-transaction policy.
// -----------------------------------------------------------------------
package axishape_pkg;

  localparam int ADDR_W    = 64;
  localparam int BUS_BYTES = 64;         // 512-bit
  localparam int PAGE_B    = 4096;
  localparam int MAX_ID    = 16;
  localparam int MAX_OUT   = 32;

  // Which limit shortened a burst. Section 2: the identity of the
  // winner is more useful than the value, and it is free -- the
  // comparison has already been made.
  typedef enum logic [1:0] {
    BOUND_LENGTH = 2'd0,                 // the whole transfer fitted
    BOUND_BURST  = 2'd1,                 // MAX_BEATS
    BOUND_PAGE   = 2'd2,                 // the 4 KiB rule
    BOUND_BUFFER = 2'd3                  // the descriptor's buffer ended
  } bound_e;

  // What a transaction is FOR. This is the ID allocator's input and
  // section 8's subject: two transactions with the same purpose may
  // share an ID; two with different purposes must not, because an ID
  // imposes ordering neither of them asked for.
  typedef enum logic [2:0] {
    PURP_DESC_FETCH   = 3'd0,
    PURP_DESC_WRITE   = 3'd1,            // 18.2's ordering domain
    PURP_RX_DATA      = 3'd2,
    PURP_TX_DATA      = 3'd3,
    PURP_STATUS_WRITE = 3'd4
  } purpose_e;

  typedef struct packed {
    logic [ADDR_W-1:0] addr;
    logic [15:0]       bytes;
    purpose_e          purpose;
    logic              is_write;
    logic [15:0]       stream_id;        // which frame / which chain
  } req_t;

endpackage

// -----------------------------------------------------------------------
// burst_length_selector -- one length, and the reason for it.
//
// The three-way minimum is 18.3 section 7's and 18.4 section 5's.
// What is new is BOUND, which turns "something limited this" into
// "this limited this" for one comparator's worth of logic.
// -----------------------------------------------------------------------
module burst_length_selector
  import axishape_pkg::*;
#(
  parameter int MAX_BEATS = 32
)(
  input  logic               clk,
  input  logic               rst_n,

  input  logic               req_valid,
  input  logic [ADDR_W-1:0]  req_addr,
  input  logic [15:0]        req_remaining,      // left in this buffer
  input  logic [15:0]        req_total_left,     // left in this transfer

  output logic [15:0]        burst_bytes,
  output logic [7:0]         burst_len,          // beats minus one
  output bound_e             bound,
  output logic               burst_is_aligned,

  output logic [31:0]        c_bound_length,
  output logic [31:0]        c_bound_burst,
  output logic [31:0]        c_bound_page,
  output logic [31:0]        c_bound_buffer,
  output logic [31:0]        c_bursts,
  output logic [31:0]        c_beats,
  output logic [15:0]        mean_beats_x10
);

  localparam int MAX_BYTES = MAX_BEATS * BUS_BYTES;

  // Distance to the next page boundary.
  wire [12:0] to_page = 13'(PAGE_B) - {1'b0, req_addr[11:0]};

  wire [15:0] by_total  = req_total_left;
  wire [15:0] by_buffer = req_remaining;
  wire [15:0] by_burst  = 16'(MAX_BYTES);
  wire [15:0] by_page   = {3'b0, to_page};

  // The minimum, and the identity of the winner. Priority when two
  // tie: report the SOFTER constraint, because that is the one a
  // driver can act on. A tie between page and buffer reports buffer.
  always_comb begin
    burst_bytes = by_total;
    bound       = BOUND_LENGTH;

    if (by_buffer < burst_bytes) begin
      burst_bytes = by_buffer;
      bound       = BOUND_BUFFER;
    end
    if (by_burst < burst_bytes) begin
      burst_bytes = by_burst;
      bound       = BOUND_BURST;
    end
    if (by_page < burst_bytes) begin
      burst_bytes = by_page;
      bound       = BOUND_PAGE;
    end
  end

  assign burst_len = 8'(((burst_bytes + BUS_BYTES - 1) / BUS_BYTES) - 1);

  // An aligned burst is one whose start is on a bus-width boundary.
  // A misaligned start wastes the first beat's leading strobes and,
  // worse, makes every subsequent beat unaligned too.
  assign burst_is_aligned = (req_addr[$clog2(BUS_BYTES)-1:0] == '0);

  always_comb begin
    if (c_bursts == '0) mean_beats_x10 = 16'd0;
    else mean_beats_x10 = 16'((c_beats * 32'd10) / c_bursts);
  end

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      c_bound_length <= '0; c_bound_burst <= '0;
      c_bound_page <= '0; c_bound_buffer <= '0;
      c_bursts <= '0; c_beats <= '0;
    end else if (req_valid) begin
      c_bursts <= c_bursts + 1;
      c_beats  <= c_beats + {24'b0, burst_len} + 32'd1;
      unique case (bound)
        BOUND_LENGTH: c_bound_length <= c_bound_length + 1;
        BOUND_BURST:  c_bound_burst  <= c_bound_burst + 1;
        BOUND_PAGE:   c_bound_page   <= c_bound_page + 1;
        BOUND_BUFFER: c_bound_buffer <= c_bound_buffer + 1;
      endcase
    end
  end

endmodule

Classification: a four-way minimum with winner identification and per-cause accounting.

What it teaches: that mean_beats_x10 is the single number that says whether the burst policy is working, and it is almost never what the parameter says. A design configured for 32-beat bursts whose mean is 9 beats is issuing three and a half times as many transactions as its configuration implies — and the four c_bound_* counters say why: buffer-bound means the buffers are small, page-bound means they are unaligned, burst-bound means the parameter is the limit and could be raised.

And it teaches that the tie-breaking priority is a deliberate choice with a diagnostic purpose. When the page bound and the buffer bound coincide — a 2 KiB buffer starting 2 KiB into a page — both are true and only one is actionable. Reporting BOUND_BUFFER sends an integrator to the buffer size, which is a real lever; reporting BOUND_PAGE sends them to alignment, which in that case is already perfect. The listing reports whichever check fires last in the chain, and the ordering is chosen so that the page check wins only when it strictly binds.

Deliberately simplified: the divide computing burst_len is a real divide; BUS_BYTES is a power of two so a production design shifts. mean_beats_x10 is a combinational divide that software should do instead. And burst_is_aligned checks only the start address, where a full treatment also considers whether the length leaves the next burst aligned — which it does not, whenever a page bound truncates a burst to a non-multiple of the bus width.

Production implication: the ratio c_bound_page / c_bursts is Chapter 18.3 §7's c_page_splits with the denominator attached, and it is the form an integrator can act on. A raw split count means nothing without the burst count; a ratio above a few per cent on a receive path is a misaligned pool and a free fix. On a transmit path it is Chapter 18.4 §5's application-supplied buffer and usually is not.

4. Burst Efficiency, and the Two Verdicts on Misalignment

Section 3 produced lengths. This section asks what they are worth, and the answer depends on which resource is being counted.

The efficiency model is one line. A burst costs one address cycle plus its data beats. A 1518-octet frame is 24 data beats on a 512-bit bus regardless of how it is split, so efficiency is 24 / (24 + bursts).

Three readings, and the third is the one that matters.

Reading 1 — longer is better, monotonically. There is no optimum in the interior; the efficiency rises all the way to 32 beats. A design choosing 16 beats "because longer bursts hog the bus" is giving up 3.69 percentage points for a fairness concern the interconnect's arbiter already handles.

Reading 2 — the returns stop abruptly and completely. 32 to 64 beats buys exactly nothing on a 1518-octet frame, because the frame already fits in one 2048-octet burst. A 64-beat burst on a 512-bit bus is 4096 octets — precisely one page — so it can only ever be issued page-aligned, and any other alignment guarantees a split. The parameter is worse than useless above 32 for this traffic.

Reading 3 — the misalignment penalty is 1.40 percentage points, and that is the wrong way to count it.

Chapter 18.1 §12 established the MAC is transaction-bound, so the second row is the binding one — and a team measuring bus efficiency will see 1.4% and conclude alignment is a rounding error.

But there is a second inversion, and it stops the conclusion being simple.

At 100 Gb/s the transaction budget binds on minimum-size frames, and a 64-octet frame crosses a page boundary only 1.54% of the time.

Read the last column. Misalignment costs 0.0092 transactions per cycle where the budget is tight, and 0.0121 where the budget has 96% slack.

So the honest statement is the one nobody expects: alignment matters most for the frames whose transaction budget matters least. The two never bind together — minimum-size frames saturate the address channel and rarely split; maximum-size frames split constantly and use 4% of the channel.

Which does not make alignment unimportant, and the reason is worth being precise about. The address channel is not the only thing a transaction consumes: it consumes an outstanding slot for a full round trip — Section 6 — and an entry in Section 16's reorder buffer. A 37% increase in transactions on large frames is a 37% increase in both, and those are sized for the worst case rather than the average.

5. RTL 2 — The Alignment Splitter

The block that turns one logical request into a sequence of legal bursts, and the one place where the 4 KiB rule is enforced rather than assumed.

// -----------------------------------------------------------------------
// alignment_splitter -- walks a request, emitting legal bursts.
//
// The 4 KiB rule is not negotiable and is not visible to anything
// above this block. A frame lands wherever the descriptor's buffer
// pointer says; this module is the only place that knows the bus has
// an opinion about it.
// -----------------------------------------------------------------------
module alignment_splitter
  import axishape_pkg::*;
#(
  parameter int MAX_BEATS = 32
)(
  input  logic               clk,
  input  logic               rst_n,

  input  logic               req_valid,
  output logic               req_ready,
  input  req_t               req,

  // Legal bursts out.
  output logic               burst_valid,
  input  logic               burst_ready,
  output logic [ADDR_W-1:0]  burst_addr,
  output logic [7:0]         burst_len,
  output logic [15:0]        burst_bytes,
  output purpose_e           burst_purpose,
  output logic [15:0]        burst_stream,
  output logic               burst_is_last,          // of this request
  output bound_e             burst_bound,

  output logic [31:0]        c_requests,
  output logic [31:0]        c_bursts,
  output logic [31:0]        c_split_requests,       // needed >1 burst
  output logic [15:0]        worst_bursts_per_req,
  output logic               illegal_burst           // must never assert
);

  logic [ADDR_W-1:0] addr;
  logic [15:0]       left;
  logic              active;
  logic [15:0]       bursts_this_req;

  localparam int MAX_BYTES = MAX_BEATS * BUS_BYTES;

  wire [12:0] to_page = 13'(PAGE_B) - {1'b0, addr[11:0]};

  logic [15:0] this_bytes;
  bound_e      this_bound;
  always_comb begin
    this_bytes = left;
    this_bound = BOUND_LENGTH;
    if (16'(MAX_BYTES) < this_bytes) begin
      this_bytes = 16'(MAX_BYTES);
      this_bound = BOUND_BURST;
    end
    if ({3'b0, to_page} < this_bytes) begin
      this_bytes = {3'b0, to_page};
      this_bound = BOUND_PAGE;
    end
  end

  assign req_ready     = !active;
  assign burst_valid   = active && (left != 16'd0);
  assign burst_addr    = addr;
  assign burst_bytes   = this_bytes;
  assign burst_len     = 8'(((this_bytes + BUS_BYTES - 1) / BUS_BYTES) - 1);
  assign burst_purpose = req.purpose;
  assign burst_stream  = req.stream_id;
  assign burst_is_last = (this_bytes == left);
  assign burst_bound   = this_bound;

  // The invariant this block exists for. A burst that crosses a page
  // boundary is a protocol violation the interconnect may answer with
  // a SLVERR, a wrapped access, or silent corruption -- the standard
  // does not say, because the master is simply forbidden to do it.
  assign illegal_burst = burst_valid &&
                         ((addr[11:0] + this_bytes) > 16'(PAGE_B));

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      addr <= '0; left <= '0; active <= 1'b0; bursts_this_req <= '0;
      c_requests <= '0; c_bursts <= '0; c_split_requests <= '0;
      worst_bursts_per_req <= '0;
    end else begin
      if (req_valid && req_ready) begin
        addr            <= req.addr;
        left            <= req.bytes;
        active          <= 1'b1;
        bursts_this_req <= '0;
        c_requests      <= c_requests + 1;
      end

      if (burst_valid && burst_ready) begin
        addr            <= addr + {{(ADDR_W-16){1'b0}}, this_bytes};
        left            <= left - this_bytes;
        bursts_this_req <= bursts_this_req + 16'd1;
        c_bursts        <= c_bursts + 1;

        if (burst_is_last) begin
          active <= 1'b0;
          if (bursts_this_req != 16'd0)
            c_split_requests <= c_split_requests + 1;
          if ((bursts_this_req + 16'd1) > worst_bursts_per_req)
            worst_bursts_per_req <= bursts_this_req + 16'd1;
        end
      end
    end
  end

endmodule

Classification: a page-aware request walker with a protocol-violation guard.

What it teaches: that the 4 KiB rule has no defined failure mode, which is why illegal_burst must be an assertion rather than a counter. The AXI specification forbids a burst from crossing a 4 KiB boundary and does not say what happens if one does — because the master is simply not allowed to. An interconnect may return an error, may wrap the address within the boundary, or may forward it and let a downstream slave decide. All three have been seen, and the second is the worst: the write lands 4 KiB earlier than intended, silently.

And it teaches why the rule exists, which explains why it cannot be relaxed. A 4 KiB boundary is the smallest page an interconnect must assume, and a burst crossing one may need to be routed to two different slaves — two different memory controllers, or memory and a peripheral. A burst is a single routing decision, so a burst spanning two routes has no meaning.

Deliberately simplified: the splitter handles one request at a time, which serialises the address path and at Chapter 18.4 §16's rates is exactly the loop-versus-pipeline error that chapter warned about. A production splitter pipelines the address computation and holds several requests. worst_bursts_per_req is sticky with no reset, so a single pathological request marks the counter for ever. And the BOUND_BUFFER case from Section 3 is absent, because this block receives a request already limited to one buffer.

Production implication: c_split_requests / c_requests is the fraction of transfers that needed more than one burst, and comparing it against the theoretical fraction is how an integrator separates alignment from size. At a 32-beat limit, a 1518-octet frame should split 37.04% of the time from alignment alone. A measured 40% is alignment; a measured 95% means the buffers are smaller than the frames and Section 2's buffer bound is firing, which the c_bound_* counters confirm directly.

6. The Outstanding-Transaction Budget

Burst length decides how much each transaction carries. This section decides how many may be in flight, and the answer is Little's law with a bus in it.

A transaction occupies an outstanding slot from issue to response. If the round trip is RT and each transaction carries P bits, O outstanding transactions sustain O × P / RT bits per second.

Those figures are for 16-beat bursts — 1024 octets each. Chapter 18.1 §4 established that a 100 Gb/s port needs 114.29 Gb/s in each direction:

Which is a much smaller number than Chapter 18.2 §9's table of 32 and Chapter 18.4 §16's 45 frames, and the discrepancy is worth resolving because it looks like a contradiction.

It is not. Those chapters counted frames in flight, not transactions. A 64-octet frame is one transaction; the 45-frame figure at 100 Gb/s is 45 tiny transactions, carrying 64 octets each rather than 1024. Little's law with P = 512 bits instead of 8192 gives sixteen times the count — and 45 is what it gives.

So the outstanding budget is set by minimum-size frames, exactly as the transaction budget was — and for the same reason: small transfers occupy a slot for a full round trip while carrying almost nothing.

And there is a hard ceiling this arithmetic runs into. An interconnect offers a fixed number of outstanding transactions per master, frequently 8 or 16, and a design needing 67 cannot simply ask for more. The lever is the burst length again: at 64-octet frames there is nothing to lengthen, which is why Chapter 18.3 §17's answer was to reduce the transaction count by batching descriptors rather than to increase concurrency.

7. RTL 3 — The Outstanding Governor

The block that enforces Section 6's budget, and that has to divide it between purposes that need very different amounts.

// -----------------------------------------------------------------------
// outstanding_governor -- per-purpose outstanding limits with a
// shared pool and a reservation.
//
// A single global limit starves whichever purpose asks last. A purely
// per-purpose partition wastes slots when one purpose is idle. The
// structure here is a reserved minimum per purpose plus a shared
// remainder, which is the standard answer and is worth stating
// because designs routinely do neither.
// -----------------------------------------------------------------------
module outstanding_governor
  import axishape_pkg::*;
#(
  parameter int NUM_PURP = 5,
  parameter int TOTAL    = MAX_OUT
)(
  input  logic               clk,
  input  logic               rst_n,

  // Reserved slots per purpose. Must sum to <= TOTAL.
  input  logic [5:0]         cfg_reserved [NUM_PURP],

  input  logic               issue_valid,
  input  purpose_e           issue_purpose,
  output logic               issue_grant,

  input  logic               retire_valid,
  input  purpose_e           retire_purpose,

  output logic [5:0]         in_use [NUM_PURP],
  output logic [5:0]         shared_free,
  output logic [31:0]        c_grants,
  output logic [31:0]        c_stalls [NUM_PURP],
  output logic [5:0]         peak_total,
  output logic               reservation_overcommitted
);

  logic [5:0] used [NUM_PURP];
  logic [5:0] total_used;

  always_comb begin
    int i;
    total_used = '0;
    for (i = 0; i < NUM_PURP; i++) total_used = total_used + used[i];
  end

  // A reservation that oversubscribes the pool is a configuration
  // error whose symptom is a purpose that is starved despite having
  // a reservation -- which looks exactly like a hardware bug.
  logic [7:0] reserved_sum;
  always_comb begin
    int i;
    reserved_sum = '0;
    for (i = 0; i < NUM_PURP; i++) reserved_sum = reserved_sum + 8'(cfg_reserved[i]);
  end
  assign reservation_overcommitted = (reserved_sum > 8'(TOTAL));

  // How many slots are not spoken for by anybody's reservation.
  logic [7:0] reserved_unused;
  always_comb begin
    int i;
    reserved_unused = '0;
    for (i = 0; i < NUM_PURP; i++)
      if (used[i] < cfg_reserved[i])
        reserved_unused = reserved_unused + 8'(cfg_reserved[i] - used[i]);
  end
  assign shared_free = 6'(8'(TOTAL) - 8'(total_used) - reserved_unused);

  // A purpose may issue if it is inside its own reservation, or if
  // the shared remainder has room. The first clause is what prevents
  // a high-rate purpose from starving a low-rate one.
  wire within_reservation = (used[issue_purpose] < cfg_reserved[issue_purpose]);
  wire shared_available   = (shared_free != 6'd0);

  assign issue_grant = issue_valid &&
                       (within_reservation || shared_available);

  always_ff @(posedge clk or negedge rst_n) begin
    int i;
    if (!rst_n) begin
      for (i = 0; i < NUM_PURP; i++) begin
        used[i] <= '0; c_stalls[i] <= '0;
      end
      c_grants <= '0; peak_total <= '0;
    end else begin
      if (issue_valid && issue_grant) begin
        used[issue_purpose] <= used[issue_purpose] + 6'd1;
        c_grants <= c_grants + 1;
      end
      if (issue_valid && !issue_grant)
        c_stalls[issue_purpose] <= c_stalls[issue_purpose] + 1;

      if (retire_valid && (used[retire_purpose] != 6'd0))
        used[retire_purpose] <= used[retire_purpose] - 6'd1;

      if (total_used > peak_total) peak_total <= total_used;
    end
  end

  assign in_use = used;

endmodule

Classification: a reserved-plus-shared credit pool with per-purpose starvation accounting.

What it teaches: that a single global outstanding limit starves the purpose that asks least often, and a pure partition wastes the pool. The receive data path issues constantly; a descriptor fetch issues once per eight frames — so under a global limit the fetch competes for a pool the data path has already emptied, and a descriptor fetch delayed is Chapter 18.2 §3's starved, which becomes a dropped frame. The reservation guarantees the fetch a slot without permanently removing it from the data path, because the shared calculation returns unused reservations to the pool.

And it teaches that c_stalls must be per purpose. A single stall counter says the pool is too small; five say which purpose is being squeezed, and the remedies differ: data-path stalls mean the pool is too small, descriptor-fetch stalls mean the reservation is too small, and those are opposite changes to the same configuration.

Deliberately simplified: reserved_unused and total_used are both computed combinationally by summing five counters every cycle, which at 400 MHz needs a maintained running total instead. The reservation is static where a real design may adapt it — though a static reservation that is occasionally wrong is much safer than an adaptive one that is occasionally wrong, because its failure is a known throughput loss rather than an unbounded one. And retirement assumes the purpose is carried back with the response, which requires the ID or the tag to encode it — Section 9's job.

Production implication: peak_total against TOTAL says whether the master is actually using the concurrency it was given. A peak well below the limit means the MAC never had that many requests to make — so raising the interconnect's limit would change nothing, and the bottleneck is upstream. A peak pinned at the limit with c_stalls rising means the limit binds, and Section 6's table says what it should be for this traffic. Without both numbers, an integrator negotiating for more outstanding slots has no case.

8. What an AXI ID Actually Buys, and What It Costs

This is the section the chapter exists for. An ID is not a slot, a credit or a tag — it is a contract, and a design that allocates IDs as though they were interchangeable breaks something no counter will show.

AXI's ordering model in one sentence: transactions issued with the same ID complete in order; transactions with different IDs have no ordering relationship at all.

Which means an ID does two things at once, and they point in opposite directions.

Row two is the failure mode. Two transactions that share an ID are ordered whether or not anybody wanted it — so an allocator handing out IDs round-robin from a pool imposes ordering between unrelated transactions and gives up their concurrency for no benefit.

And the reverse is the failure mode that corrupts rather than slows. Two transactions that needed ordering and were given different IDs have no ordering, and Chapter 18.2 §10's mode 0 stops working — silently, on some interconnects, some of the time.

So the allocation rule is not "spread the load" but "group by ordering requirement":

Two transactions share an ID if and only if the design requires them to complete in order.

Which turns the ID allocator into a classifier rather than a distributor, and the classification is by purpose:

Row three is the one that costs, and it is Chapter 18.4 §5's c_reorder_stalls explained. A transmit frame's data must reach the FIFO in frame order. Giving the whole frame one ID gets that from the protocol and serialises the frame's own bursts — a 1518-octet frame at 32 beats is one burst, so nothing is lost, but a 9000-octet frame at 32 beats is five bursts and they are now sequential.

Row three is why jumbo transmit is slower than it looks on a same-ID design, and Section 12 is how to get it back without spending five IDs on one frame.

9. RTL 4 — The ID Allocator

Not a pool. A classifier that maps an ordering requirement onto an identifier, and refuses to reuse one whose contract is still live.

// -----------------------------------------------------------------------
// axi_id_allocator -- maps (purpose, stream) onto an AXI ID.
//
// The rule from section 8: two transactions share an ID if and only
// if they must complete in order. So the ID is a function of the
// ORDERING DOMAIN, not of load. Reuse is permitted only when the
// previous holder's domain is closed -- section 19's rejected class
// is an allocator that reuses on the basis of the slot being free.
// -----------------------------------------------------------------------
module axi_id_allocator
  import axishape_pkg::*;
#(
  parameter int NUM_ID = MAX_ID
)(
  input  logic               clk,
  input  logic               rst_n,

  input  logic               req_valid,
  input  purpose_e           req_purpose,
  input  logic [15:0]        req_stream,          // frame or ring number
  output logic               req_grant,
  output logic [$clog2(NUM_ID)-1:0] req_id,

  // A response retires one transaction under an ID.
  input  logic               rsp_valid,
  input  logic [$clog2(NUM_ID)-1:0] rsp_id,

  // A stream declares itself finished -- its ordering domain closes
  // and its ID may be rebound to a different domain.
  input  logic               stream_closed,
  input  logic [15:0]        closed_stream,
  input  purpose_e           closed_purpose,

  output logic [31:0]        c_grants,
  output logic [31:0]        c_no_id,
  output logic [31:0]        c_rebinds,
  output logic [5:0]         ids_bound,
  output logic               id_reused_while_live    // must never assert
);

  // Each ID is BOUND to an ordering domain while it has outstanding
  // transactions. The binding is the contract; the outstanding count
  // is what keeps it alive.
  logic                      bound   [NUM_ID];
  purpose_e                  b_purp  [NUM_ID];
  logic [15:0]               b_strm  [NUM_ID];
  logic [5:0]                b_out   [NUM_ID];

  // Does an ID already carry this exact domain?
  logic                      hit;
  logic [$clog2(NUM_ID)-1:0] hit_i;
  always_comb begin
    int i;
    hit = 1'b0; hit_i = '0;
    for (i = NUM_ID-1; i >= 0; i--)
      if (bound[i] && (b_purp[i] == req_purpose) &&
          (b_strm[i] == req_stream)) begin
        hit   = 1'b1;
        hit_i = i[$clog2(NUM_ID)-1:0];
      end
  end

  // A free ID is one with nothing outstanding AND no live binding.
  // Those are different conditions: an ID may have retired every
  // transaction and still be bound, because its stream has not
  // declared itself finished and a later burst of the same frame
  // must still be ordered against the earlier ones.
  logic                      free;
  logic [$clog2(NUM_ID)-1:0] free_i;
  always_comb begin
    int i;
    free = 1'b0; free_i = '0;
    for (i = NUM_ID-1; i >= 0; i--)
      if (!bound[i]) begin
        free   = 1'b1;
        free_i = i[$clog2(NUM_ID)-1:0];
      end
  end

  assign req_grant = req_valid && (hit || free);
  assign req_id    = hit ? hit_i : free_i;

  // The invariant. An ID whose binding is live must not be handed to
  // a different domain, however many of its transactions have
  // retired. This is the whole difference between an ID and a slot.
  assign id_reused_while_live = req_valid && !hit && free &&
                                bound[free_i];

  always_comb begin
    int i;
    ids_bound = '0;
    for (i = 0; i < NUM_ID; i++) if (bound[i]) ids_bound = ids_bound + 6'd1;
  end

  always_ff @(posedge clk or negedge rst_n) begin
    int i;
    if (!rst_n) begin
      for (i = 0; i < NUM_ID; i++) begin
        bound[i] <= 1'b0; b_purp[i] <= PURP_DESC_FETCH;
        b_strm[i] <= '0;  b_out[i]  <= '0;
      end
      c_grants <= '0; c_no_id <= '0; c_rebinds <= '0;
    end else begin
      if (req_valid && req_grant) begin
        c_grants <= c_grants + 1;
        if (hit) begin
          b_out[hit_i] <= b_out[hit_i] + 6'd1;
        end else begin
          bound[free_i]  <= 1'b1;
          b_purp[free_i] <= req_purpose;
          b_strm[free_i] <= req_stream;
          b_out[free_i]  <= 6'd1;
          c_rebinds      <= c_rebinds + 1;
        end
      end

      if (req_valid && !req_grant) c_no_id <= c_no_id + 1;

      if (rsp_valid && (b_out[rsp_id] != 6'd0))
        b_out[rsp_id] <= b_out[rsp_id] - 6'd1;

      // Unbinding is explicit and is NOT triggered by the outstanding
      // count reaching zero.
      for (i = 0; i < NUM_ID; i++)
        if (stream_closed && bound[i] &&
            (b_strm[i] == closed_stream) && (b_purp[i] == closed_purpose) &&
            (b_out[i] == 6'd0))
          bound[i] <= 1'b0;
    end
  end

endmodule

Classification: a content-addressed binding table keyed on an ordering domain, with explicit release.

What it teaches: that an ID becomes free in two stages and a design that collapses them is broken. An ID whose transactions have all responded still carries its binding, because a later burst of the same frame must be ordered against the earlier ones — and if the ID has been rebound to another frame in the meantime, that ordering is gone. The outstanding count reaching zero means the ID is idle; only stream_closed means it is free.

And it teaches that c_no_id is a throughput limit with a completely different cause from c_stalls in Section 7. Both stall a request. Running out of outstanding slots means the memory system is the limit; running out of IDs means too many independent ordering domains are open at once — which at Chapter 18.4 §16's 45 transmit chains in flight is 45 domains against 16 IDs, and the resolution is Section 12's.

Deliberately simplified: the hit and free searches are combinational over sixteen entries and evaluated every cycle, which is a large amount of logic in the address path; a production allocator keeps a small CAM and a free list. The unbind loop writes bound[i] for all sixteen entries in one cycle, which is a sixteen-way comparison plus a write. And there is no ageing: a stream that is never closed holds its ID for ever, which a production design bounds with a timeout.

Production implication: ids_bound against NUM_ID is the number that decides whether an interconnect's ID width is adequate, and it is a negotiation an integrator usually loses for lack of evidence. An interconnect offering 4-bit IDs gives sixteen; a design needing more must either reduce its ordering domains or accept c_no_id stalls. The counter converts "we would like more IDs" into "we exhaust sixteen for 14% of requests", which is an argument.

10. The ID Budget, Already Partly Spent

Before this chapter allocates IDs, two earlier chapters have already committed some. This section adds them up, and the total does not fit.

Chapter 18.2 §10 offered three barrier strategies and mode 0 was "give both writes the same AXI ID." That is a commitment: the descriptor writebacks for a ring must all share one ID, or the ordering that mode depends on is gone.

Chapter 18.4 §5 requires a transmit frame's data to reach the FIFO in frame order, and the cheapest way to get that is one ID per frame — which at Section 6's 45 frames in flight is 45 IDs.

Add it up for a single-queue port:

The transmit data row is the whole problem and it is 90% of the requirement.

Three ways out, and they are not equally good.

Row one is a real option and it is the one most designs take without realising. A design with 16 IDs and one per transmit frame has 16 frames in flight, which at 300 ns and 6.72 ns per minimum-size frame is 16/45 = 36% of line rate — and nothing reports it as a limit, because c_no_id is a counter most designs do not have.

Row two is worse than it looks for jumbo frames and free for ordinary ones, because a 1518-octet frame at a 32-beat limit is one burst and cannot be reordered with itself.

Row three is the right answer and Section 12 develops it. The key realisation is that frame order is needed at the FIFO, not on the bus — so a small buffer that restores order locally buys back every ID the transmit path was spending, and the buffer's size is Section 16's subject.

11. RTL 5 — The Ordering-Domain Tracker

The block that knows which transactions must be ordered against which, independently of how the bus is told about it.

// -----------------------------------------------------------------------
// ordering_domain_tracker -- the design's own model of what must be
// ordered, kept separately from the ID assignment.
//
// The separation is the point. Section 9 maps domains onto IDs when
// there are enough IDs; section 12 restores order locally when there
// are not. Both need the same underlying fact -- which transactions
// belong to which domain -- and that fact is this block.
// -----------------------------------------------------------------------
module ordering_domain_tracker
  import axishape_pkg::*;
#(
  parameter int NUM_DOMAIN = 64
)(
  input  logic               clk,
  input  logic               rst_n,

  input  logic               open_valid,
  input  purpose_e           open_purpose,
  input  logic [15:0]        open_stream,
  output logic               open_grant,
  output logic [$clog2(NUM_DOMAIN)-1:0] open_domain,

  // A transaction joins a domain and takes a sequence number within
  // it. The sequence number is what section 12's reorder buffer
  // uses; the ID is what the bus uses. They are not the same thing.
  input  logic               join_valid,
  input  logic [$clog2(NUM_DOMAIN)-1:0] join_domain,
  output logic [7:0]         join_seq,
  output logic               join_grant,

  input  logic               retire_valid,
  input  logic [$clog2(NUM_DOMAIN)-1:0] retire_domain,

  input  logic               close_valid,
  input  logic [$clog2(NUM_DOMAIN)-1:0] close_domain,

  output logic [7:0]         domain_next_expect [NUM_DOMAIN],
  output logic [15:0]        domains_open,
  output logic [31:0]        c_opens,
  output logic [31:0]        c_no_domain,
  output logic [31:0]        c_joins,
  output logic               closed_with_outstanding    // must never assert
);

  logic                  d_open [NUM_DOMAIN];
  purpose_e              d_purp [NUM_DOMAIN];
  logic [15:0]           d_strm [NUM_DOMAIN];
  logic [7:0]            d_next [NUM_DOMAIN];       // next seq to allocate
  logic [7:0]            d_exp  [NUM_DOMAIN];       // next seq to deliver
  logic [5:0]            d_out  [NUM_DOMAIN];

  logic                  free;
  logic [$clog2(NUM_DOMAIN)-1:0] free_i;
  always_comb begin
    int i;
    free = 1'b0; free_i = '0;
    for (i = NUM_DOMAIN-1; i >= 0; i--)
      if (!d_open[i]) begin
        free   = 1'b1;
        free_i = i[$clog2(NUM_DOMAIN)-1:0];
      end
  end

  assign open_grant  = open_valid && free;
  assign open_domain = free_i;

  assign join_grant = join_valid && d_open[join_domain];
  assign join_seq   = d_next[join_domain];

  // Closing a domain with transactions still outstanding loses the
  // ordering the domain existed to express. It is the domain-level
  // form of section 9's id_reused_while_live.
  assign closed_with_outstanding = close_valid && d_open[close_domain] &&
                                   (d_out[close_domain] != 6'd0);

  always_comb begin
    int i;
    domains_open = '0;
    for (i = 0; i < NUM_DOMAIN; i++)
      if (d_open[i]) domains_open = domains_open + 16'd1;
  end

  always_ff @(posedge clk or negedge rst_n) begin
    int i;
    if (!rst_n) begin
      for (i = 0; i < NUM_DOMAIN; i++) begin
        d_open[i] <= 1'b0; d_purp[i] <= PURP_DESC_FETCH;
        d_strm[i] <= '0; d_next[i] <= '0; d_exp[i] <= '0; d_out[i] <= '0;
      end
      c_opens <= '0; c_no_domain <= '0; c_joins <= '0;
    end else begin
      if (open_valid && open_grant) begin
        d_open[free_i] <= 1'b1;
        d_purp[free_i] <= open_purpose;
        d_strm[free_i] <= open_stream;
        d_next[free_i] <= '0;
        d_exp[free_i]  <= '0;
        d_out[free_i]  <= '0;
        c_opens        <= c_opens + 1;
      end
      if (open_valid && !open_grant) c_no_domain <= c_no_domain + 1;

      if (join_valid && join_grant) begin
        d_next[join_domain] <= d_next[join_domain] + 8'd1;
        d_out[join_domain]  <= d_out[join_domain] + 6'd1;
        c_joins             <= c_joins + 1;
      end

      // Retiring advances the EXPECTED sequence, which is what the
      // reorder buffer compares against. Note it advances by one per
      // retirement regardless of which sequence retired -- section 12
      // holds anything early.
      if (retire_valid) begin
        d_exp[retire_domain] <= d_exp[retire_domain] + 8'd1;
        if (d_out[retire_domain] != 6'd0)
          d_out[retire_domain] <= d_out[retire_domain] - 6'd1;
      end

      if (close_valid && (d_out[close_domain] == 6'd0))
        d_open[close_domain] <= 1'b0;
    end
  end

  assign domain_next_expect = d_exp;

endmodule

Classification: a domain table carrying two sequence numbers — one for allocation, one for delivery.

What it teaches: that the design's ordering model and the bus's ordering mechanism are separate things, and separating them is what makes Section 12 possible. A design that expresses ordering only by ID assignment has no ordering model left when it runs out of IDs. This block keeps the model — domain, sequence number, next-expected — independently of whether the bus is enforcing it, so the same information serves both the ID policy and the local reorder buffer.

And it teaches that two sequence numbers are needed rather than one. d_next is what the next transaction takes; d_exp is what the reorder buffer will deliver next. They differ by exactly the number of transactions in flight, and collapsing them into one counter makes it impossible to tell a transaction that arrived early from one that arrived on time.

Deliberately simplified: 64 domains with combinational search and a 64-entry parallel update is far more logic than a real design spends — a production tracker keeps a free list and indexes directly, since the domain number is known to the requester. The sequence numbers are 8-bit and wrap at 256, which bounds a domain to 256 transactions with no check. And domain_next_expect is exported as a 64-entry array, which is a wide bus to Section 12 rather than the single lookup a real design would use.

Production implication: c_no_domain and Section 9's c_no_id together are the diagnostic pair for Section 10's arithmetic. Domains exhausted means the design has more independent streams than it can track — Chapter 18.4 §16's 45 chains against 64 domains is comfortable. IDs exhausted with domains available means the mapping is the limit, which is exactly the condition Section 12 removes, and the pair is how an integrator knows which of the two to spend area on.

12. Getting the Ordering Back Without Spending an ID

Section 10 ended with 50 IDs wanted and 16 available. This section is the way out, and it rests on one observation.

Frame order is needed at the FIFO. It is not needed on the bus.

Chapter 18.4 §5's gather engine must deliver octets into the transmit FIFO in frame order, because that is the order they go onto the wire. It does not follow that the bus must deliver them in that order — only that something between the bus and the FIFO must.

And the buffer is small, because the reordering window is bounded by the number of outstanding transactions rather than by the number of frames.

The rule is (O − 1) × burst payload: with O transactions in flight, at most O − 1 can have arrived out of order while the design waits for the one it wants. At eight outstanding and 16-beat bursts that is 7 KiB — against Chapter 18.1 §18's 32 KiB receive FIFO at 100 Gb/s, 21.9%.

So the trade is explicit: 7 KiB of SRAM buys back 44 AXI IDs, and with them Chapter 18.4 §16's full pipeline depth.

Two details make this work and both are easy to get wrong.

First, the buffer is per ordering domain, not global. A transmit frame's bursts must be ordered against each other, not against another frame's — so the reorder buffer is indexed by Section 11's domain and sequence number, and a design with one flat buffer reintroduces exactly the cross-frame ordering it was trying to avoid.

Second, the buffer must be sized for the worst case, not the average. Out-of-order responses are rare on a well-behaved interconnect — c_reorder_stalls on a typical port is near zero — and a buffer sized for the average is a buffer that overflows on the day the memory controller reorders. An overflow means a burst's data has nowhere to go and the bus must be stalled, which is safe but slow; a design that instead drops the data has silently corrupted a frame.

And the result closes Chapter 18.4's open question. That chapter's c_reorder_stalls counted responses arriving early with no buffer to hold them. With the buffer, the counter changes meaning: it becomes a measure of how much reordering the interconnect actually does, which is the input to deciding whether the buffer could be smaller.

13. RTL 6 — The Efficiency Monitor

The block that measures Section 4's two verdicts separately, because a design that measures only one of them draws the wrong conclusion.

// -----------------------------------------------------------------------
// efficiency_monitor -- bus efficiency AND transaction count, kept
// apart.
//
// Section 4: the same misalignment is 1.40 points of cycle efficiency
// and 37.0% of transactions. A monitor reporting one number reports
// whichever the designer happened to care about, and the two give
// opposite verdicts on whether alignment matters.
// -----------------------------------------------------------------------
module efficiency_monitor
  import axishape_pkg::*;
(
  input  logic        clk,
  input  logic        rst_n,

  input  logic        addr_issued,
  input  logic        beat_transferred,
  input  logic        bus_idle,
  input  logic        stalled_no_slot,
  input  logic        stalled_no_id,
  input  logic        stalled_reorder,

  input  logic        req_started,
  input  logic [15:0] req_bytes,

  output logic [31:0] c_addr_cycles,
  output logic [31:0] c_data_cycles,
  output logic [31:0] c_idle_cycles,
  output logic [31:0] c_stall_slot,
  output logic [31:0] c_stall_id,
  output logic [31:0] c_stall_reorder,
  output logic [31:0] c_requests,
  output logic [31:0] c_request_octets,

  // Two independent verdicts.
  output logic [15:0] cycle_efficiency_pct,      // data / (data + addr)
  output logic [15:0] octets_per_txn,            // the transaction verdict
  output logic [15:0] bus_occupancy_pct
);

  wire [31:0] busy = c_addr_cycles + c_data_cycles;
  wire [31:0] total = busy + c_idle_cycles;

  always_comb begin
    // Verdict 1: of the cycles the bus spent on us, what fraction
    // moved data? This is what "efficiency" usually means and it is
    // insensitive to the thing that matters.
    if (busy == '0) cycle_efficiency_pct = 16'd0;
    else cycle_efficiency_pct = 16'((c_data_cycles * 32'd100) / busy);

    // Verdict 2: how much payload did each transaction carry? This
    // is the one that binds, because 18.1 section 12's limit is one
    // transaction per cycle regardless of its size.
    if (c_addr_cycles == '0) octets_per_txn = 16'd0;
    else octets_per_txn = 16'(c_request_octets / c_addr_cycles);

    if (total == '0) bus_occupancy_pct = 16'd0;
    else bus_occupancy_pct = 16'((busy * 32'd100) / total);
  end

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      c_addr_cycles <= '0; c_data_cycles <= '0; c_idle_cycles <= '0;
      c_stall_slot <= '0; c_stall_id <= '0; c_stall_reorder <= '0;
      c_requests <= '0; c_request_octets <= '0;
    end else begin
      if (addr_issued)      c_addr_cycles    <= c_addr_cycles + 1;
      if (beat_transferred) c_data_cycles    <= c_data_cycles + 1;
      if (bus_idle)         c_idle_cycles    <= c_idle_cycles + 1;
      if (stalled_no_slot)  c_stall_slot     <= c_stall_slot + 1;
      if (stalled_no_id)    c_stall_id       <= c_stall_id + 1;
      if (stalled_reorder)  c_stall_reorder  <= c_stall_reorder + 1;
      if (req_started) begin
        c_requests       <= c_requests + 1;
        c_request_octets <= c_request_octets + {16'b0, req_bytes};
      end
    end
  end

endmodule

Classification: a dual-verdict bus monitor with three independent stall causes.

What it teaches: that octets_per_txn is the number Chapter 18.1 §12's wall is actually about, and almost no bus monitor reports it. A conventional monitor reports utilisation and efficiency — both cycle-based — and both are insensitive to the transaction count, because a transaction's address cycle is one cycle whether it carries 64 octets or 2048. A port at 96% cycle efficiency and 64 octets per transaction is in serious trouble; one at 80% and 1024 octets per transaction is fine.

And it teaches that the three stall counters name three different owners. c_stall_slot is the memory system — Section 7's pool. c_stall_id is this chapter's allocation policy — Section 10's arithmetic. c_stall_reorder is the interconnect's behaviour — how much it reorders, which nobody controls. A single "stalled" counter cannot separate a memory problem from a design parameter from a third party's behaviour.

Deliberately simplified: three combinational divides, which software should do from the raw counters. bus_idle is taken as an input where a real monitor derives it from the handshakes. And the counters are global rather than per purpose, so a port whose descriptor traffic is efficient and whose data traffic is not reports an average that describes neither — which Section 14's telemetry splits.

Production implication: the pair cycle_efficiency_pct and octets_per_txn should be reported together and never separately, because Section 4's whole finding is that they disagree. An integration report quoting 94.6% bus efficiency on a port issuing 1.370 transactions where 1 would do is quoting the number that does not bind, and the design will be signed off as adequate and will not reach line rate on minimum-size frames.

14. RTL 7 — Shaping Telemetry

Per-purpose accounting, because Section 13's global numbers hide the thing that would fix them.

// -----------------------------------------------------------------------
// axishape_telemetry -- the same measurements, split by purpose.
//
// A port's descriptor traffic and its frame data have completely
// different shapes: 16-octet single-beat transactions against
// 1024-octet bursts. Averaging them describes neither.
// -----------------------------------------------------------------------
module axishape_telemetry
  import axishape_pkg::*;
#(
  parameter int NUM_PURP = 5
)(
  input  logic        clk,
  input  logic        rst_n,

  input  logic        txn_valid,
  input  purpose_e    txn_purpose,
  input  logic [15:0] txn_bytes,
  input  logic [7:0]  txn_len,
  input  bound_e      txn_bound,
  input  logic        txn_split,

  input  logic        rsp_valid,
  input  purpose_e    rsp_purpose,
  input  logic [15:0] rsp_latency_cycles,

  output logic [31:0] c_txn      [NUM_PURP],
  output logic [31:0] c_octets   [NUM_PURP],
  output logic [31:0] c_beats    [NUM_PURP],
  output logic [31:0] c_splits   [NUM_PURP],
  output logic [31:0] c_lat_sum  [NUM_PURP],
  output logic [31:0] c_lat_n    [NUM_PURP],
  output logic [15:0] worst_lat  [NUM_PURP],
  output logic [31:0] c_bound    [4],
  output logic [15:0] octets_per_txn [NUM_PURP]
);

  always_comb begin
    int p;
    for (p = 0; p < NUM_PURP; p++)
      octets_per_txn[p] = (c_txn[p] == '0) ? 16'd0
                                           : 16'(c_octets[p] / c_txn[p]);
  end

  always_ff @(posedge clk or negedge rst_n) begin
    int p;
    if (!rst_n) begin
      for (p = 0; p < NUM_PURP; p++) begin
        c_txn[p] <= '0; c_octets[p] <= '0; c_beats[p] <= '0;
        c_splits[p] <= '0; c_lat_sum[p] <= '0; c_lat_n[p] <= '0;
        worst_lat[p] <= '0;
      end
      for (p = 0; p < 4; p++) c_bound[p] <= '0;
    end else begin
      if (txn_valid) begin
        c_txn[txn_purpose]    <= c_txn[txn_purpose] + 1;
        c_octets[txn_purpose] <= c_octets[txn_purpose] + {16'b0, txn_bytes};
        c_beats[txn_purpose]  <= c_beats[txn_purpose] + {24'b0, txn_len} + 32'd1;
        if (txn_split) c_splits[txn_purpose] <= c_splits[txn_purpose] + 1;
        c_bound[txn_bound] <= c_bound[txn_bound] + 1;
      end

      // Per-purpose latency. The descriptor fetch's latency is what
      // 18.2 section 3's prefetch window must cover; the data write's
      // is what 18.3 section 10's barrier waits for. They are
      // different numbers and both are needed.
      if (rsp_valid) begin
        c_lat_sum[rsp_purpose] <= c_lat_sum[rsp_purpose] +
                                  {16'b0, rsp_latency_cycles};
        c_lat_n[rsp_purpose]   <= c_lat_n[rsp_purpose] + 1;
        if (rsp_latency_cycles > worst_lat[rsp_purpose])
          worst_lat[rsp_purpose] <= rsp_latency_cycles;
      end
    end
  end

endmodule

Classification: a per-purpose transaction and latency accumulator.

What it teaches: that per-purpose latency is the measurement Module 18 has needed since Chapter 18.1 §17 and has been estimating instead. Three chapters have used "a 300 ns round trip" as a placeholder: Chapter 18.2 §9 sized an in-flight table with it, Chapter 18.3 §10 sized a barrier with it, Chapter 18.4 §16 derived a pipeline depth from it. worst_lat measures it, per purpose, on the real system — and the read latency and the write-response latency are frequently very different numbers.

And it teaches that splitting by purpose changes what the split counter means. A global c_splits mixes descriptor fetches — 16 octets, never split — with frame data. Per purpose, c_splits[PURP_RX_DATA] / c_txn[PURP_RX_DATA] is directly comparable against Section 4's theoretical 37.04%, which is what makes it actionable rather than merely nonzero.

Deliberately simplified: NUM_PURP divides evaluated combinationally; the latency measurement requires a timestamp carried with each transaction, which is a field this listing assumes exists and does not create. The accumulators wrap, and a windowed version reset on read is what a real design provides. And c_bound is indexed by a two-bit enum cast to an array index, which works and is not how a production design would write it.

Production implication: worst_lat[PURP_DESC_FETCH] is the number Chapter 18.2 §5's low-water mark should be set from, and it is measurable only here. A prefetch window refilled at a threshold chosen from a datasheet figure is a guess; one refilled at a threshold derived from the measured worst-case fetch latency is a design. The same argument applies to Chapter 18.4 §7's cut-through lead, which is computed from a worst-case stall figure an integrator supplies and this counter can check.

15. RTL 8 — The Shaping Conformance Monitor

The last block, and the first verdict in Module 18 that is about a policy rather than a fault.

// -----------------------------------------------------------------------
// axishape_conformance_monitor -- is the shaping policy working?
//
// Unlike the monitors in 18.1 to 18.4, most of these verdicts are
// not faults. They are a policy that could be better, and the
// distinction matters: a fault stops the port, a policy costs
// throughput nobody has attributed to it.
// -----------------------------------------------------------------------
module axishape_conformance_monitor
  import axishape_pkg::*;
#(
  parameter int NUM_PURP = 5,
  parameter int MAX_BEATS = 32
)(
  input  logic        clk,
  input  logic        rst_n,

  input  logic [31:0] c_bound [4],
  input  logic [31:0] c_txn   [NUM_PURP],
  input  logic [15:0] octets_per_txn [NUM_PURP],
  input  logic [15:0] mean_beats_x10,
  input  logic [31:0] c_stall_slot,
  input  logic [31:0] c_stall_id,
  input  logic [31:0] c_stall_reorder,
  input  logic [31:0] c_no_id,
  input  logic [31:0] c_no_domain,
  input  logic [5:0]  peak_total,
  input  logic [5:0]  ids_bound,
  input  logic [15:0] worst_lat [NUM_PURP],

  input  logic        illegal_burst,
  input  logic        id_reused_while_live,
  input  logic        closed_with_outstanding,
  input  logic        reservation_overcommitted,

  output logic        shaping_ok,
  output logic        protocol_violation,
  output logic        cfg_fault,
  output logic        bursts_too_short,
  output logic        alignment_dominating,
  output logic        id_starved,
  output logic        outstanding_starved,
  output logic        reorder_pressure,
  output logic        none_of_the_above
);

  // Faults. These stop the port or corrupt it.
  assign protocol_violation = illegal_burst | id_reused_while_live |
                              closed_with_outstanding;
  assign cfg_fault          = reservation_overcommitted;

  // Policy findings. These cost throughput nobody has attributed.
  assign bursts_too_short = (mean_beats_x10 < 16'(MAX_BEATS * 10 / 2));

  // More than a fifth of bursts shortened by the page rule means
  // alignment is the dominant limit -- section 4's 37.04% is what
  // pure misalignment looks like on maximum-size frames.
  assign alignment_dominating =
    ((c_bound[BOUND_PAGE] + c_bound[BOUND_BURST] +
      c_bound[BOUND_BUFFER] + c_bound[BOUND_LENGTH]) > 32'd1000) &&
    (c_bound[BOUND_PAGE] >
     ((c_bound[BOUND_PAGE] + c_bound[BOUND_BURST] +
       c_bound[BOUND_BUFFER] + c_bound[BOUND_LENGTH]) >> 3));

  assign id_starved          = (c_no_id > 32'd1000) ||
                               (c_stall_id > 32'd1000);
  assign outstanding_starved = (c_stall_slot > 32'd1000);
  assign reorder_pressure    = (c_stall_reorder > 32'd1000);

  assign shaping_ok = !protocol_violation && !cfg_fault;

  assign none_of_the_above = shaping_ok && !bursts_too_short &&
                             !alignment_dominating && !id_starved &&
                             !outstanding_starved && !reorder_pressure;

  // ---- properties -------------------------------------------------

  p_no_illegal_burst:
    assert property (@(posedge clk) disable iff (!rst_n)
      !illegal_burst)
    else $error("a burst crossed a 4 KiB boundary");

  p_no_live_id_reuse:
    assert property (@(posedge clk) disable iff (!rst_n)
      !id_reused_while_live)
    else $error("an ID was rebound while its ordering domain was live");

  p_violation_excludes_ok:
    assert property (@(posedge clk) disable iff (!rst_n)
      protocol_violation |-> !shaping_ok)
    else $error("shaping_ok asserted with a protocol violation");

  p_peak_within_pool:
    assert property (@(posedge clk) disable iff (!rst_n)
      peak_total <= MAX_OUT)
    else $error("outstanding transactions exceeded the pool");

  p_ids_bound_within_pool:
    assert property (@(posedge clk) disable iff (!rst_n)
      ids_bound <= MAX_ID)
    else $error("more IDs bound than exist");

endmodule

Classification: a verdict generator that separates protocol faults from policy findings.

What it teaches: that most of what this chapter measures is not a fault, and saying so explicitly changes how it is acted on. illegal_burst stops the port. bursts_too_short costs a few per cent of the bus and nobody notices for a year — and the two arriving through the same reporting channel with the same urgency is how a real fault gets lost among tuning suggestions.

And it teaches that alignment_dominating needs a denominator that sums all four bound causes. A page-split count alone is meaningless; a page-split count against the total burst count is Section 4's 37.04% made comparable. The threshold of one eighth is deliberately below that figure: a port at 12.5% page-bound is already worth looking at, and one at 37% is textbook misalignment.

Deliberately simplified: bursts_too_short fires at half the configured maximum, which is arbitrary and would be a register. The four-way sum of c_bound is recomputed twice combinationally in one expression. And there is no verdict for the interaction Section 4 identified — a port whose large frames are misaligned and whose small frames saturate the channel is fine on both counts separately and should be reported as such, which needs the per-purpose split from Section 14 rather than these globals.

Production implication: none_of_the_above for the fifth and last time in Module 18. Five monitors now cover the MAC's interfaces, its ring, both DMA directions and its bus shaping. A port on which all five assert has no known hardware or driver-contract problem, and the remaining explanations are the application, the network, or the offered load — which is where Chapter 18.1 §21's complaint 2 wanted to send an investigation and had no evidence to.

16. The Reorder Buffer, Sized

Section 12 asserted a rule. This section derives it and then argues about the worst case, because the worst case is what a buffer must be sized for and it is not the common one.

The rule: with O transactions outstanding in one ordering domain, at most O − 1 responses can have arrived out of order while the design waits for the one it needs.

The reasoning is short. The design is waiting for sequence k. Every other outstanding transaction has a sequence above k — sequences below k have already been delivered — so at most O − 1 of them can be sitting in the buffer. Each occupies one burst's payload.

And the table shows why burst length and reorder buffer are the same decision. Doubling the burst length halves the transaction count — Section 4 — and doubles the reorder buffer. A design choosing 32-beat bursts and 8 outstanding is choosing 14 KiB of SRAM, which against Chapter 18.1 §18's 32 KiB receive FIFO is 44% and is emphatically not free.

Now the worst case, and this is where designs get it wrong.

On a well-behaved interconnect, out-of-order responses are rare. Chapter 18.4 §5's c_reorder_stalls on a typical port is near zero, because most interconnects return responses in issue order most of the time. So a buffer sized for the measured behaviour is nearly empty.

And a buffer sized for the measured behaviour overflows on the day the memory controller reorders, which happens when:

Every row is ordinary and every row is intermittent, which is Chapter 18.2 §19's class-75 environment all over again: a failure rate that is a property of the system rather than of the input.

So the buffer must be sized for O − 1 and the overflow behaviour must be safe.

Row one is the only acceptable answer, and it is available because AXI's read data channel has a ready signal. A design that cannot stall — because it has already accepted the beat — must have the buffer space, which is the argument for sizing at the worst case rather than handling the overflow.

17. What the Bus Assumes About the MAC

Chapter 18.1 §17 listed what the MAC assumes about the system. This chapter is the only one in Module 18 where the obligations run the other way.

Row five is the one with system-wide consequences and the least local visibility. A master that asserts AWVALID and waits for something that depends on the write data channel making progress — which depends on the write address being accepted — has deadlocked the interconnect for every master on it. The MAC's own counters show nothing; the symptom is the whole SoC stopping.

And the rule that prevents it is a discipline rather than a check: never make the acceptance of one channel conditional on progress in another. Chapter 18.4 §5's gather engine is the block that could get this wrong — it issues a read address and consumes read data, and if the address issue waited for the FIFO to drain it would be conditioning one channel on another.

Which is why that listing gates ar_valid on !fifo_full rather than on the FIFO having room for the whole burst. The distinction is small and the consequence is not: a master that will always eventually accept its read data cannot deadlock the read channel; one that may refuse indefinitely can.

Now the reciprocal list, because this chapter has also established what the MAC needs from the bus:

Rows two and three are the ones an integration negotiation is actually about, and this chapter's contribution is that both now have numbers attached. "We need more outstanding transactions" is a request; "we stall on 14% of requests at eight outstanding and Little's law says we need five for 1518-octet frames and 67 for 64-octet ones" is an argument.

And row one is worth a closing note because it is the only requirement the MAC cannot verify. An interconnect that claims same-ID ordering and does not honour it produces Chapter 18.2 §20's run C — corrupted descriptors, intermittently, with every component behaving as documented. The only defence is Chapter 18.2 §11's mode 1, which depends on nothing, and the only reason not to use it is the round trip Section 6 has now shown how to pipeline away.

18. The Cost, Accounted

Eight blocks, and this chapter's cost is unusual: the logic is significant, the memory is significant, and the parameters are what really cost.

About 5 500 flops, the largest of Module 18's five chapters — and two thirds of it is the domain tracker and the telemetry, both of which are bookkeeping rather than datapath.

The memory:

The second row is the one to carry forward, because §4 chose 32-beat bursts. Chapter 19.6 §19 shows that pairing the 16-beat figure with a 32-beat design is not a budget slip: 7 KiB holds 3.5 of this datapath's 2 KiB bursts, so the fourth simultaneous out-of-order response overflows it — and Section 16's "the overflow behaviour must be safe" then has no safe answer.

And the parameters, which is where the real cost is, because each is a negotiation with somebody outside the MAC team:

Row three is the one this chapter resolves and it resolves it by spending memory. Sixteen IDs and no reorder buffer is 36% of line rate; sixteen IDs and 7 KiB of reorder buffer is full rate. The 7 KiB buys 44 IDs.

Module 18's whole memory bill, now complete, for a 100 Gb/s port:

Just over one per cent of a chip's SRAM for one 100 Gb/s port's system interface, of which 58% is the receive FIFO — and at this rate three quarters of that is the memory-stall buffer rather than Chapter 14.2's headroom. At 1 Gb/s the proportions invert entirely and the headroom is 87% of a 1.9 KiB FIFO, which is Chapter 18.1 §18's crossover at about 9 Gb/s.

19. Properties Worth Asserting, and One Worth Refusing

This chapter's properties are mostly about resource limits, which makes the rejected one particularly easy to write: it is a resource property that is true of the resource and false of what the resource carries.

Burst shaping.

// No burst crosses a 4 KiB boundary. Non-negotiable.
p_burst_within_page:
  assert property (@(posedge clk) disable iff (!rst_n)
    (burst_valid && burst_ready) |->
      ((burst_addr[11:0] + burst_bytes) <= PAGE_B))
  else $error("a burst crossed a 4 KiB boundary");

// No burst exceeds the negotiated length.
p_burst_within_max:
  assert property (@(posedge clk) disable iff (!rst_n)
    (burst_valid && burst_ready) |-> (burst_len < MAX_BEATS))
  else $error("a burst exceeded the negotiated maximum length");

// The bound reported matches the length produced.
p_bound_matches_length:
  assert property (@(posedge clk) disable iff (!rst_n)
    (req_valid && (bound == BOUND_PAGE)) |->
      (burst_bytes == {3'b0, to_page}))
  else $error("a page-bound burst was not the page distance");

// A request is fully consumed before the next is accepted.
p_one_request_at_a_time:
  assert property (@(posedge clk) disable iff (!rst_n)
    active |-> !req_ready)
  else $error("a request was accepted while one was in progress");

// The remaining count only decreases.
p_left_monotonic:
  assert property (@(posedge clk) disable iff (!rst_n)
    active |=> (left <= $past(left)))
  else $error("the remaining byte count increased");

// The last burst of a request consumes exactly what is left.
p_last_burst_exact:
  assert property (@(posedge clk) disable iff (!rst_n)
    (burst_valid && burst_ready && burst_is_last) |->
      (burst_bytes == left))
  else $error("the final burst did not consume the remainder");

// Every request eventually completes.
p_request_completes:
  assert property (@(posedge clk) disable iff (!rst_n)
    (req_valid && req_ready) |-> ##[1:$] (burst_valid && burst_ready &&
                                          burst_is_last))
  else $error("a request never produced a final burst");

Outstanding transactions.

// The pool is never exceeded.
p_pool_not_exceeded:
  assert property (@(posedge clk) disable iff (!rst_n)
    peak_total <= TOTAL)
  else $error("outstanding transactions exceeded the pool");

// A grant inside a reservation is always given.
p_reservation_honoured:
  assert property (@(posedge clk) disable iff (!rst_n)
    (issue_valid && (used[issue_purpose] < cfg_reserved[issue_purpose]))
      |-> issue_grant)
  else $error("a purpose was refused inside its own reservation");

// A retirement never underflows.
p_retire_no_underflow:
  assert property (@(posedge clk) disable iff (!rst_n)
    retire_valid |-> (used[retire_purpose] != 6'd0))
  else $error("a retirement arrived for a purpose with nothing outstanding");

// An overcommitted reservation is flagged, not silently clipped.
p_overcommit_flagged:
  assert property (@(posedge clk) disable iff (!rst_n)
    (reserved_sum > TOTAL) |-> reservation_overcommitted)
  else $error("an overcommitted reservation was not flagged");

// Every issued transaction eventually retires.
p_issue_retires:
  assert property (@(posedge clk) disable iff (!rst_n)
    (issue_valid && issue_grant) |-> ##[1:$] retire_valid)
  else $error("an issued transaction never retired");