HBM Integration

Chapter 17.1 established that a memory die is a stateful endpoint. Chapter 17.3 found that a single memory controller becomes the contended resource. This chapter confronts a subsystem whose entire value is that it has many independent paths — and asks what it takes to keep all of them busy from across a chiplet boundary.

1. The One-Sentence Model

HBM bandwidth is a concurrency problem before it is a pin-width number. Peak capability matters only if the system creates enough independent useful work across channels, banks, controllers and queues — and can return the results without serialising them somewhere else.

2. What This Chapter Owns

What is new here, and none of it exists in 17.1:

17.1 taught one memory endpoint. This chapter has N of them behind one ingress, and almost every failure below needs N > 1 to exist: admission against the wrong channel (§17), hotspot imbalance (§20), head-of-line blocking across independent channels (§24), one channel's maintenance stalling the rest (§29), and a channel mask changed under live traffic (§39).

A mandatory architectural clarification in §4, because "HBM attached over UCIe" is routinely read as something it is not.

Protection domains that do not overlap (§33) — a clean UCIe CRC proves a transport object survived a link and nothing else.

And bottleneck arithmetic (§41–§44) that composes the link, the ingress path, the controller, the media and the source into one honest bound.

3. Sourcing, and the Version Gate

4. What "HBM Attached Over UCIe" Does and Does Not Mean

This section is mandatory reading before any of the rest, because the phrase is routinely read as a claim about physical connection that no source supports.

The structure that makes the phrase meaningful:

compute / accelerator die
        │  UCIe link(s) — a die-to-die interconnect between LOGIC dies
        ▼
memory-side LOGIC chiplet
        ├── UCIe-facing ingress, address mapping, per-channel queues
        └── local HBM controller(s) and PHY
                    │  native HBM interface — NOT UCIe
                    ▼
              HBM stack / media

Three consequences, and each shapes a later section.

UCIe's boundary is between two logic dies. Everything from the memory-side ingress downward is ordinary memory-subsystem engineering that happens to sit on a separate die. That is why §22's scheduler and §28's maintenance behave exactly as 17.1 described them — the link changed who is upstream, not what DRAM is.

The exact physical integration varies by architecture and package, and no single arrangement is claimed here. A design may place the controller with the stack, or on a separate logic die, or split it — and §8's mapping, §16's admission and §38's channel mask are needed in all of those.

And it means the UCIe link and the HBM interface have completely independent bandwidths. §41's arithmetic exists because those two numbers have no reason to match, and assuming they do is the most common integration mistake in this chapter.

5. The Responsibility Stack

Read the third layer. UCIe is one of six and is bounded above and below. Improving it addresses one layer's contribution, which §42 quantifies and which is frequently not the binding one.

6. The Ingress Path

Three structures in that picture are where the chapter's failures live. The mapper (§9, §20), the per-channel queues with their admission (§17), and the return path (§26). The controllers and the stack behave exactly as 17.1 described; nothing new is claimed about them.

7. The Parallelism Hierarchy

stack
  └── channel          — independently addressable; its own controller interface
        └── bank       — independently busy; its own recovery window (17.1 §6)

No count is stated at any level (§3). What is architectural is the independence:

Bandwidth is the sum over independent resources that are simultaneously busy. Not over resources that exist. §20 is the gap between those two sentences, and it is where most disappointing measurements come from.

8. Address to Channel

// ILLUSTRATIVE address-to-channel mapping. Valid ONLY when NUM_CHANNELS is a
// power of two — see Section 9 for why that qualifier matters, and Section 10
// for the general case.
localparam int CH_W = $clog2(NUM_CHANNELS);
 
logic [CH_W-1:0] channel_sel;
assign channel_sel = req_addr[INTERLEAVE_LSB +: CH_W];

Architecture. A field extraction from the address. Cheap, single-cycle, and entirely determined by INTERLEAVE_LSB, which is the granularity decision in §12.

State. None — combinational, and carried with the request rather than recomputed downstream, for the same reason 17.2 §16 captures a target: a recomputation after a configuration change routes to a different place.

Cycle behaviour. Evaluated once at ingress. channel_sel then indexes the queue array (§13), the occupancy array (§15) and the admission check (§16). Three consumers, one value, computed once.

Contract. Every downstream structure assumes channel_sel < NUM_CHANNELS and that it is stable for the request's whole life. The first is guaranteed by construction only when NUM_CHANNELS is a power of two — §9.

Failure. §9 when the parameter is not a power of two. Also placing INTERLEAVE_LSB below the access size, which makes consecutive bytes of one access land on different channels and splits every request.

DV. Cover every channel as selected (§53), and cover addresses that differ only in the interleave field.

9. Wrong RTL — the Mask Assumes a Power of Two

// WRONG when NUM_CHANNELS is not a power of two.
assign channel_sel = (req_addr >> INTERLEAVE_LSB) & (NUM_CHANNELS - 1);

Suppose a configuration exposes 6 usable channels — which is entirely ordinary after a repair or a degraded-channel decision (§37).

NUM_CHANNELS = 6
NUM_CHANNELS - 1 = 5 = 3'b101      <- NOT a contiguous mask
 
masked value can be:  0, 1, 4, 5   only
channels 2 and 3:      never selected
channels 0,1,4,5:      receive all traffic

Three consequences.

Two channels are silently unused. The hardware is present, powered, and idle. Measured bandwidth is two-thirds of what the configuration should deliver, with no error anywhere.

And the distribution across the remaining four is not uniform either, because the reachable values are not contiguous — so the imbalance is both a capacity loss and a hotspot.

The bug appears only at non-power-of-two configurations, which are exactly the ones produced by degradation. A design verified at 8 channels and deployed with 6 after a repair fails in the field, and the symptom is a performance regression that looks like a failing device.

The fix is either a general mapping (§10) or a parameter check that refuses the configuration (§11). Choosing the second is legitimate — many designs mandate power-of-two channel counts — but the choice must be enforced, not assumed.

10. A Mapping That Does Not Assume a Power of Two

// ILLUSTRATIVE general mapping. A modulo is expensive in one cycle; a small
// lookup keyed by a hashed slice is the usual synthesis-friendly compromise.
localparam int HASH_W = 8;
 
logic [HASH_W-1:0]  addr_slice;
logic [CH_W-1:0]    channel_lut [2**HASH_W];   // built at configuration time
logic [CH_W-1:0]    channel_sel;
 
assign addr_slice  = req_addr[INTERLEAVE_LSB +: HASH_W];
assign channel_sel = channel_lut[addr_slice];

Architecture. A configuration-time table replacing a runtime modulo. The table is populated so that the 2**HASH_W slice values are distributed as evenly as possible over the NUM_CHANNELS active channels — which for a non-power-of-two count means some channels receive one more slice than others.

State. 2**HASH_W entries of CH_W bits. HASH_W trades table size against distribution quality: too small and the residual imbalance is significant; too large and the table dominates the ingress area.

Cycle behaviour. One table read, combinational or registered. The table is written only at a channel-mask commit (§38), never during live traffic — the same requested-versus-active discipline as everywhere else in this curriculum.

Contract. Every entry must name an active channel. A table entry pointing at a disabled channel routes requests into a queue that will never be scheduled, which is §39 with the mask and the table disagreeing.

Failure. Rebuilding the table incrementally under live traffic, which produces a window where some addresses map to the old channel and some to the new — and a request in flight to the old one has no owner at the new.

DV. After every commit, check that every table entry names an active channel and that the distribution is within a stated bound of uniform.

11. Parameter Checks at Elaboration

// ILLUSTRATIVE elaboration-time checks. A bad parameter should fail the build,
// not produce a silently degraded design (Section 9).
initial begin
  assert (NUM_CHANNELS > 0)
    else $fatal(1, "NUM_CHANNELS must be positive");
 
  assert (DEPTH_PER_CHANNEL > 0)
    else $fatal(1, "DEPTH_PER_CHANNEL must be positive");
 
  // If the design MANDATES a power-of-two channel count, say so here rather
  // than relying on Section 8's mask being correct by accident.
  assert ((NUM_CHANNELS & (NUM_CHANNELS - 1)) == 0)
    else $fatal(1, "NUM_CHANNELS must be a power of two for the mask mapping");
 
  assert (INTERLEAVE_LSB >= $clog2(ACCESS_BYTES))
    else $fatal(1, "INTERLEAVE_LSB below access size splits every request");
end

Architecture. Four checks that turn silent misconfiguration into a build failure.

Why the third is worth more than it looks. It converts §9 from a field performance regression into a compile error. A design that mandates power-of-two channels is perfectly reasonable; a design that assumes it without saying so is one repair away from the §9 failure.

Why the fourth matters. An interleave granularity smaller than the access size means a single request's bytes belong to two channels. The design must then split every request or produce wrong data, and neither is what the parameter's author intended.

DV. These are elaboration checks, not runtime assertions. They cost nothing at simulation time and nothing in silicon, and they are the cheapest defence in the chapter.

12. Interleave Granularity

Two consequences, and no universally best value is claimed.

Fine granularity converts one sequential stream into perfect channel utilisation and simultaneously destroys row locality within each channel, which raises bank-recovery stalls (17.1 §6).

Coarse granularity preserves locality and creates hotspots when the active address range is smaller than the granularity times the channel count. A workload touching one coarse block uses one channel — §20, produced entirely by a configuration parameter.

The right value depends on the access pattern, and the honest engineering answer is to measure it — 15.5 §21's sweep methodology applies directly, with granularity as the swept variable and per-channel utilisation as the observable.

13. Per-Channel Queues

// ILLUSTRATIVE per-channel request storage. NOT an HBM-defined structure.
typedef struct packed {
  logic [REQ_ID_W-1:0] id;         // the REQUESTER's identity — carried (17.1 §9)
  logic [ADDR_W-1:0]   addr;
  logic                is_write;
  logic [BE_W-1:0]     byte_enables;
} hbm_req_t;
 
// Payload storage — inferred RAM, deliberately NOT reset (Section 14).
hbm_req_t                        chan_mem  [NUM_CHANNELS][DEPTH_PER_CHANNEL];
 
// Control state — small, and reset (Section 14).
logic [PTR_W-1:0]                wr_ptr_q  [NUM_CHANNELS];
logic [PTR_W-1:0]                rd_ptr_q  [NUM_CHANNELS];
logic [OCC_W-1:0]                occ_q     [NUM_CHANNELS];

Architecture. One queue per channel rather than one shared queue. The separation is the design: a shared queue means a request for a full channel blocks a request for an idle one, which is head-of-line blocking across resources that are independent by construction (§24).

State. NUM_CHANNELS × DEPTH_PER_CHANNEL payload entries plus three small control registers per channel. The payload is inferred RAM; the control is flops.

Cycle behaviour. Push on admission (§16), pop on a scheduler grant that fired. occ_q is the authority for admission, and it must be updated in one place (§15).

Contract. The scheduler assumes occ_q[c] accurately reflects what is in channel c's queue. Admission assumes the same value. Two consumers, one counter, and §15 is why it needs a single owner.

Failure. Sizing DEPTH_PER_CHANNEL uniformly when the workload is skewed — which wastes storage on cold channels and backs up hot ones. A shared pool with per-channel reservations is the alternative, and it trades §17's simplicity for utilisation.

DV. Cover each channel empty, partially full and completely full; cover simultaneous push and pop on the same channel (§15).

14. Reset Discipline for Large Arrays

// ILLUSTRATIVE. Reset the CONTROL state; do not reset the payload RAM.
always_ff @(posedge clk or negedge rst_n)
  if (!rst_n) begin
    for (int c = 0; c < NUM_CHANNELS; c++) begin
      wr_ptr_q[c] <= '0;
      rd_ptr_q[c] <= '0;
      occ_q[c]    <= '0;
    end
  end else begin
    // ... normal pointer and occupancy updates
  end
 
// The payload is written on push and read on pop. It is never reset, and it is
// never READ before it is written, because occ_q gates every pop.
always_ff @(posedge clk)
  if (push_fire[c]) chan_mem[c][wr_ptr_q[c]] <= push_data;

Architecture. Reset only what determines behaviour. occ_q reading zero after reset means no pop can occur, so the payload's contents are unobservable — which is the argument that makes not resetting it safe.

State. The distinction is the section: NUM_CHANNELS × DEPTH_PER_CHANNEL × $bits(hbm_req_t) bits of payload against a few dozen bits of control.

Cycle behaviour. A reset loop over the payload would be enormous. In synthesis it prevents RAM inference entirely — the array becomes flops, and an ingress buffer that should have been a few block RAMs becomes hundreds of thousands of registers.

Contract. The safety argument must be explicit: no read occurs before a write, because occ_q gates every pop and starts at zero. A design that pops speculatively, or that exposes queue contents to a debug path, breaks that argument and then does need the reset.

Failure. Resetting the payload "for determinism in simulation" and discovering the area cost at synthesis. Or the reverse — omitting the reset in a design that does read speculatively, which produces X propagation from an uninitialised RAM that simulation shows and silicon does not.

DV. Assert that no pop occurs while occ_q[c] == 0 — which is the property the whole argument rests on, and it is one line.

15. Occupancy Accounting

// ILLUSTRATIVE. One owner per channel counter; simultaneous push and pop
// handled explicitly rather than by two independent statements.
always_ff @(posedge clk or negedge rst_n)
  if (!rst_n) begin
    for (int c = 0; c < NUM_CHANNELS; c++) occ_q[c] <= '0;
  end else begin
    for (int c = 0; c < NUM_CHANNELS; c++) begin
      unique case ({push_fire[c], pop_fire[c]})
        2'b10: occ_q[c] <= occ_q[c] + 1'b1;
        2'b01: occ_q[c] <= occ_q[c] - 1'b1;
        default: ;                       // 2'b00 and 2'b11 both hold
      endcase
    end
  end

Architecture. One unique case per channel with all four combinations enumerated. The 2'b11 arm holding is the point — a simultaneous push and pop leaves occupancy unchanged, and two independent if statements produce a net change of one in whichever order the tool resolves them.

State. NUM_CHANNELS counters, each $clog2(DEPTH_PER_CHANNEL+1) bits wide. The +1 matters: a counter that can represent DEPTH but not DEPTH distinct values plus empty will alias full and empty.

Cycle behaviour. Both push_fire and pop_fire are handshake-qualified — a grant the controller refused is not a pop, and an admission the ingress did not take is not a push. Crediting either is 13.4 §18's pointer bug.

Contract. Admission (§16) and the scheduler both read this. An overcount refuses admission to a channel with room; an undercount admits into a full one and overwrites a live request.

Failure. The two-if form. The drift is slow and monotonic, so a design works for a long time and then either refuses all traffic on one channel or corrupts it — and by then the correlation with load is invisible.

DV. §18's bounds and conservation properties. Cover the simultaneous push-and-pop cycle explicitly per channel, because it is exactly where the bug lives and it is rare at low load.

16. Admission Uses the Selected Channel

// ILLUSTRATIVE. The request's OWN channel must have room. An aggregate count
// is not an answer to this question — Section 17.
assign ingress_ready =
      channel_active_q[channel_sel]                       // Section 38
   && (occ_q[channel_sel] < DEPTH_PER_CHANNEL[OCC_W-1:0]) // THIS channel
   && outstanding_space                                    // return-path identity
   && return_path_ready;                                   // Section 41's fourth term

Architecture. Four terms, and the second is the one this section exists for. The channel is already known at ingress (§8), so the check costs one indexed comparison — there is no efficiency argument for using an aggregate instead.

State. None of its own; a conjunction over per-channel and global state.

Cycle behaviour. Combinational into the ingress handshake. channel_sel must be valid in the same cycle, which is why §8 computes it at ingress rather than downstream.

Contract. Accepting a request promises a queue slot in a specific channel, a return-path identity, and the ability to return the result. All three, or the request must not be accepted.

Failure. §17. Also omitting return_path_ready — which admits requests whose responses cannot be returned, and the resulting backpressure appears as a memory slowdown rather than a return-path limit (§41).

DV. Force each term false alone. The second must be forced per channel, since a design can check the right channel for channel 0 and the wrong one for the rest.

17. Wrong RTL — Admission Against an Aggregate Free Count

// WRONG — total free space says nothing about the channel this request needs.
logic [TOT_W-1:0] total_free;
always_comb begin
  total_free = '0;
  for (int c = 0; c < NUM_CHANNELS; c++)
    total_free = total_free + (DEPTH_PER_CHANNEL - occ_q[c]);
end
 
assign ingress_ready = (total_free != '0);       // ← the wrong question

Illustrative, with 8 channels of depth 16.

A request mapping to channel 0 is accepted, because total_free is 112. There is nowhere to put it.

Three outcomes, all bad, depending on what the design does next.

It overwrites. The write pointer advances past a live entry and a request in the queue is destroyed. A read whose request was overwritten never returns; a write whose request was overwritten never lands. No error is reported.

Or it drops. Slightly better, and still an accepted request that vanishes — which is 10.1 §10's rule violated: an accepted object cannot disappear.

Or it stalls after acceptance. The request sits in the ingress stage blocking everything behind it, including requests for the seven idle channels. A single hot channel now stalls the entire subsystem, which is §19's failure arrived at by a different route.

And the bug is invisible under balanced traffic. With traffic spread evenly, no single channel fills before the aggregate does. It appears exactly under the hotspot conditions of §20, which is when the system is already performing badly for a different reason — so the two failures mask each other.

18. SVA — Admission, Occupancy and Conservation

// MANDATORY. Admission implies room in the SELECTED channel (Section 16).
property p_admit_implies_selected_channel_room;
  @(posedge clk) disable iff (!rst_n)
    ingress_fire |-> (occ_q[channel_sel] < DEPTH_PER_CHANNEL);
endproperty
a_admit_implies_selected_channel_room:
  assert property (p_admit_implies_selected_channel_room);
 
// Occupancy is bounded on both sides.
property p_occupancy_bounded(int c);
  @(posedge clk) disable iff (!rst_n)
    (occ_q[c] <= DEPTH_PER_CHANNEL);
endproperty
a_occupancy_bounded: assert property (p_occupancy_bounded(CH_UT));
 
// A pop never occurs from an empty queue (Section 14's safety argument).
property p_no_pop_when_empty(int c);
  @(posedge clk) disable iff (!rst_n)
    (occ_q[c] == '0) |-> !pop_fire[c];
endproperty
a_no_pop_when_empty: assert property (p_no_pop_when_empty(CH_UT));
 
// Simultaneous push and pop leaves occupancy unchanged (Section 15).
property p_simultaneous_push_pop_holds(int c);
  @(posedge clk) disable iff (!rst_n)
    (push_fire[c] && pop_fire[c]) |=> (occ_q[c] == $past(occ_q[c]));
endproperty
a_simultaneous_push_pop_holds:
  assert property (p_simultaneous_push_pop_holds(CH_UT));
 
// The selected channel is always in range and always active.
property p_channel_sel_valid;
  @(posedge clk) disable iff (!rst_n)
    ingress_fire |-> ((channel_sel < NUM_CHANNELS) && channel_active_q[channel_sel]);
endproperty
a_channel_sel_valid: assert property (p_channel_sel_valid);

Architecture. Five properties: correct admission, bounds, empty-pop safety, the simultaneous case, and channel-selection validity.

Why the fourth is not implied by the second. Bounds catch a counter that has already gone wrong. The fourth catches the mechanism — the specific cycle where a two-if design loses an update — and it fires the first time that cycle occurs rather than after enough drift to breach a bound.

Why the fifth exists. §9's mask bug produces a channel_sel that is in range but names an inactive channel after a degraded configuration. The channel_active_q term is what catches it, and it is the only assertion that does.

DV. All five are always-on. The fourth needs a simultaneous push and pop on the same channel, which is a coverage bin (§53), not a lucky occurrence.

19. Per-Channel Backpressure, Not a Global Stop

The trade, stated honestly. Per-channel backpressure requires the ingress to pass over a request for a full channel and admit a later one for an idle channel. That reorders requests relative to arrival, which is fine when they are independent and is not fine when the system requires an order between them.

So the design must know which requests may be reordered. Same-address ordering, and any ordering the memory model requires, must be preserved — and 16.3 §17's principle applies exactly: the transport may reorder freely where no semantic dependency exists, and must respect order where one does.

The usual resolution is that the ingress reorders freely across channels — different channels mean different addresses under any sane mapping — and never within one. That is cheap and correct, and it delivers most of the benefit.

20. Load Imbalance — What Hardware Cannot Fix

A perfectly fair scheduler cannot help a workload whose addresses all map to one channel.

ILLUSTRATIVE. 8 channels, one workload.
 
BALANCED    : demand per channel = 12.5% each
              achievable = sum over 8 busy channels
 
HOTSPOT     : demand = 100% on channel 3, 0% elsewhere
              achievable = ONE channel's bandwidth
              measured    ≈ 1/8 of the balanced case
              hardware peak: UNCHANGED

Three consequences.

The subsystem is not broken and the number is real. Every component works; the workload asked one channel for everything. A benchmark reporting one-eighth of peak has correctly measured its own address pattern.

And there are three distinct causes, which need distinguishing: the workload's natural pattern, the interleave granularity being too coarse for the active range (§12), and the mapping being wrong (§9). Only the last is a hardware bug, and only per-channel counters (§21) tell them apart.

The remedy is a mapping change, not a scheduler change. Finer granularity, a different address slice, or a hashed mapping (§10) redistributes the same traffic. Adding queue depth to the hot channel treats the symptom and adds latency.

21. Per-Channel Instrumentation

// Diagnostic only. PER CHANNEL, because the aggregate hides exactly what
// matters — 15.3 Section 14's argument, one layer down.
logic [63:0] channel_accepted_q  [NUM_CHANNELS];   // requests admitted
logic [63:0] channel_issued_q    [NUM_CHANNELS];   // scheduler grants that FIRED
logic [63:0] channel_bytes_q     [NUM_CHANNELS];   // on an actual transfer beat
logic [63:0] channel_busy_q      [NUM_CHANNELS];   // controller occupied
logic [63:0] channel_full_q      [NUM_CHANNELS];   // cycles at DEPTH_PER_CHANNEL
logic [63:0] channel_hol_stall_q [NUM_CHANNELS];   // eligible work blocked (Section 24)
logic [63:0] channel_maint_q     [NUM_CHANNELS];   // unavailable for maintenance

Two properties.

Every event is defined at the point it is counted. channel_bytes_q increments on an actual transfer beat, not on a grant — the difference is 15.3 §14's qualification rule, and a counter incremented on grant reports bandwidth the system never delivered.

And channel_accepted_q compared across channels is the single most useful number in the chapter. A ratio of 8:0:0:0:0:0:0:0 is §20; a ratio of 2:2:0:0:2:2:0:0 is §9. Two different bugs, distinguished by one array.

22. The Scheduler Needs State

17.1 §6 established that service depends on history. A queued request is not necessarily an issuable one.

23. The Eligibility Bitmap

// ILLUSTRATIVE. Queued and eligible are different things. Symbolic timing
// throughout — no JEDEC constraint is named or valued (Section 3).
logic [DEPTH_PER_CHANNEL-1:0] queued;      // this entry holds a valid request
logic [DEPTH_PER_CHANNEL-1:0] eligible;    // ...and may legally issue NOW
logic [DEPTH_PER_CHANNEL-1:0] grant;
 
always_comb
  for (int e = 0; e < DEPTH_PER_CHANNEL; e++)
    eligible[e] = queued[e]
               && bank_ready(entry_bank[e])       // 17.1 Section 6
               && timing_ok(entry_bank[e])        // symbolic constraint set
               && !maintenance_blocks(entry_bank[e]);
 
// Age-bounded selection among eligible entries.
always_comb begin
  grant = '0;
  if (|eligible)
    grant = (|(eligible & aged_out)) ? lowest_set(eligible & aged_out)
                                     : select_best(eligible);
end

Architecture. Two bitmaps and a selection. The separation of queued from eligible is the whole idea — a scheduler that selects from queued issues illegal accesses, and one that treats !eligible as !queued forgets the work exists.

State. Two DEPTH_PER_CHANNEL-wide vectors, recomputed each cycle from bank and timing state.

Cycle behaviour. eligible is combinational from state that changes as accesses issue and retire. The aged_out override is what bounds reordering's unfairness — select_best may optimise freely among eligible entries, and an entry past its age bound wins regardless (17.1 §18's anti-starvation property).

Contract. The controller relies on never receiving an ineligible request. The requester relies on the age bound. Neither is visible at this block's interface.

Failure. select_best chosen purely for efficiency with no age override — which starves a request to a hot bank indefinitely while the channel serves a stream to idle banks, and the channel's throughput numbers look excellent throughout.

DV. Assert grant implies eligible (§27). Cover the case where entries are queued and none is eligible, which is the state §24's bug mishandles.

24. Wrong Scheduler — the Queue Head Blocks Everything

// WRONG — strict FIFO. If the head is not eligible, nothing issues.
assign grant = (queued[rd_ptr] && eligible[rd_ptr]) ? (1 << rd_ptr) : '0;

Illustrative. Channel 3's queue holds five requests. The head targets a bank in its recovery window; the other four target different, idle banks.

Three readings.

The channel idles while three legal accesses wait. Throughput collapses to whatever fraction of accesses happen to be eligible at the head. Bank parallelism — the entire reason the hierarchy in §7 exists — is unused.

It is head-of-line blocking within a channel, and it is the same shape as §17's blocking across channels. A design can fix one and not the other, and the two produce similar symptoms with different fixes.

And it needs no bug to appear. The code is correct FIFO. It is a design choice whose cost only shows under mixed bank targeting, which a sequential benchmark never produces — so it passes the obvious test and fails the real workload.

25. Reordering and Return Identity

Two reasons responses do not return in issue order, and both are present here:

The scheduler reorders within a channel (§23), deliberately, because that is how bank parallelism is exploited.

And channels complete independently — a request to channel 1 issued later can complete before one to channel 6 issued earlier, because they share nothing.

// ILLUSTRATIVE outstanding table. The requester's identity is carried, never
// replaced by a local index (17.1 Section 19).
typedef struct packed {
  logic                 valid;
  logic [REQ_ID_W-1:0]  req_id;      // the REQUESTER's
  logic [CH_W-1:0]      channel;
  logic                 is_read;
  logic [AGE_W-1:0]     age;         // saturating
} hbm_outstanding_t;
 
hbm_outstanding_t out_q [MAX_OUTSTANDING];

Architecture. One entry per outstanding request. MAX_OUTSTANDING bounds system concurrency and is therefore a first-order bandwidth parameter — §44 derives it.

State. MAX_OUTSTANDING entries. The channel field is what lets a completion be attributed to a channel for §21's counters and what lets §39's channel-removal check find affected requests.

Cycle behaviour. Allocated at admission, freed when the response is sent — not when the controller returns data, which is 15.2 §15's retirement-point rule.

Contract. The return path matches by req_id. Everything downstream of the memory assumes that identity is the requester's own.

Failure. §26. Also freeing the entry when the data arrives from the controller rather than when the response is sent, which loses the binding while the response is still in the return path.

DV. §27's properties, plus coverage of deep reordering (§53).

26. Wrong RTL — Return Order Implies the Requester

// WRONG — responses are matched to requests by position.
always_ff @(posedge clk)
  if (resp_data_valid) begin
    deliver_to(pending_fifo[deliver_ptr]);      // ← oldest request gets this data
    deliver_ptr <= deliver_ptr + 1'b1;
  end

Illustrative. Read A is issued to channel 6; read B is issued to channel 1 one cycle later. Channel 1 is idle; channel 6's bank is recovering.

The two reads receive each other's data.

Four properties.

The transport is perfect. Both objects crossed with clean CRC, exactly once, in the correct order for their respective channels. The memory returned exactly the right bytes for both addresses.

The corruption is a pure association error — no bit was altered anywhere. 17.1 §23's point exactly: a clean CRC proves a transport object survived a link and nothing about which request it belongs to.

And it is silent and data-dependent. If A and B happen to read the same value, nothing is observably wrong. The failure rate tracks how often reordering occurs, which tracks bank conflict, which tracks the address pattern — so it appears under load and vanishes in isolation.

The pending_fifo looks like a scoreboard and is not one. It records order; order is exactly the property that does not survive. The fix is 17.1 §19's: carry the requester's identity and match on it.

27. SVA — Identity, Eligibility and Exactly-Once

// MANDATORY. A returned identity must be live (Section 26).
property p_returned_id_live;
  @(posedge clk) disable iff (!rst_n)
    resp_valid |-> (out_q[resp_slot].valid && (out_q[resp_slot].req_id == resp_id));
endproperty
a_returned_id_live: assert property (p_returned_id_live);
 
// No identity is outstanding twice.
property p_id_unique_while_outstanding;
  @(posedge clk) disable iff (!rst_n)
    ingress_fire |-> !id_currently_outstanding(req_id_in);
endproperty
a_id_unique_while_outstanding: assert property (p_id_unique_while_outstanding);
 
// Exactly one completion per request.
property p_one_completion_per_request;
  @(posedge clk) disable iff (!rst_n)
    (resp_valid && resp_fire) |=> !out_q[$past(resp_slot)].valid;
endproperty
a_one_completion_per_request: assert property (p_one_completion_per_request);
 
// A grant implies the entry was ELIGIBLE, not merely queued (Section 23).
property p_grant_implies_eligible;
  @(posedge clk) disable iff (!rst_n)
    (|grant) |-> ((grant & eligible) == grant);
endproperty
a_grant_implies_eligible: assert property (p_grant_implies_eligible);
 
// Counters increment only on real events (Section 21).
property p_bytes_counted_on_transfer_only;
  @(posedge clk) disable iff (!rst_n)
    $changed(channel_bytes_q[CH_UT]) |-> $past(transfer_beat[CH_UT]);
endproperty
a_bytes_counted_on_transfer_only:
  assert property (p_bytes_counted_on_transfer_only);

Architecture. Five properties covering association, uniqueness, exactly-once, scheduler legality, and counter honesty.

Why the last one is worth an assertion. A performance counter that increments on a grant rather than a transfer reports bandwidth the system never delivered — and every conclusion drawn from it is then wrong in the optimistic direction. The assertion costs nothing and makes the measurement trustworthy, which for a chapter about bandwidth is not optional.

Why uniqueness is separate from liveness. A design can return the correct identity for every response and still have admitted two live requests with the same identity, in which case the first response resolves the wrong one (17.1 §18).

DV. All five always-on. The first fails on the very first reordered completion in §26's design, which is why deep-reorder coverage matters.

28. Maintenance Across Parallel Channels

17.1 §11 established maintenance as an obligation with a deadline. With many channels, the question becomes scope.

And a second-order benefit worth naming. With independent channels, maintenance on one overlaps with service on the others. Parallelism improves availability as well as bandwidth (17.1 §15), and a design that serialises maintenance across channels gives that up for nothing.

29. Wrong Design — One Channel's Maintenance Stalls the Stack

// WRONG — any channel in maintenance stops the whole ingress.
assign ingress_ready = ready_terms && !(|channel_maint_active);

Illustrative, with 8 channels. If each channel is unavailable for maintenance an illustrative 2% of the time and the windows are independent:

probability all 8 channels are simultaneously available
    = (1 - 0.02)^8 = 0.98^8 ≈ 0.851
 
fraction of time this design accepts NO requests ≈ 14.9%
 
fraction of capacity lost to maintenance if handled per channel ≈ 2%

The design gives up roughly 15% of its throughput to recover 2% of unavailability — a factor of about seven, purely from the scope of a single signal.

Three readings.

It is functionally correct. No request is ever issued to an unavailable channel; nothing is corrupted; every access completes. A functional regression passes.

And it gets worse as channels are added. With 16 channels at the same rate, 0.98^16 ≈ 0.724 — over 27% lost. The failure scales in the wrong direction with the very parameter that is supposed to deliver the bandwidth.

The diagnostic signature is misleading. Utilisation is low across all channels simultaneously, which looks like a source-side or link-side shortage. Only channel_maint_q correlated against ingress stall cycles reveals it — which is why §21's per-channel maintenance counter exists.

30. Read/Write Direction and Batching

Switching the data bus between directions has a cost at the controller level, and no value for it is stated here (§3). What is architectural is the trade:

Two consequences.

A read-heavy workload with occasional writes suffers most from aggressive batching, because each rare write waits for a read batch to complete. The tail latency of the minority direction is the cost, and it is invisible in an average.

And the batch size must be bounded by a starvation guarantee (§31, §32), not only by an efficiency target. A design that batches until the queue is empty starves the other direction indefinitely under sustained one-directional load.

31. Read and Write Queues, and Starvation

// ILLUSTRATIVE direction scheduling with bounded batching. Per channel.
logic [BATCH_W-1:0] dir_run_q;        // how many consecutive same-direction issues
logic               cur_dir_q;        // 0 = read batch, 1 = write batch
logic [AGE_W-1:0]   opp_wait_q;       // saturating age of the opposite direction
 
wire opp_pending = cur_dir_q ? read_pending : write_pending;
wire must_switch = (dir_run_q >= BATCH_MAX) || (opp_wait_q >= DIR_SERVICE_BOUND);
 
always_ff @(posedge clk or negedge rst_n)
  if (!rst_n) begin
    dir_run_q  <= '0;
    cur_dir_q  <= 1'b0;
    opp_wait_q <= '0;
  end else begin
    if (issue_fire) begin
      if (must_switch && opp_pending) begin
        cur_dir_q  <= ~cur_dir_q;
        dir_run_q  <= '0;
        opp_wait_q <= '0;
      end else begin
        dir_run_q <= (dir_run_q == BATCH_MAX) ? BATCH_MAX : dir_run_q + 1'b1;
      end
    end
 
    if (opp_pending && !(issue_fire && must_switch))
      opp_wait_q <= (opp_wait_q == AGE_MAX) ? AGE_MAX : opp_wait_q + 1'b1;
  end

Architecture. Two independent switch triggers: a batch-size cap for efficiency, and an age bound for liveness. Either alone is insufficient — a cap does not bound the opposite direction's wait when batches are back to back, and an age bound alone gives up the batching benefit.

State. Three small registers per channel. Both counters saturate, for the reason they always do in this curriculum: a wrapping age reports a fresh wait at the moment the wait is longest.

Cycle behaviour. The direction changes only on an issue that fired, and only when the opposite direction actually has work. Switching to an empty direction wastes a turnaround for nothing.

Contract. The requester relies on both directions being served within a bound. Neither bound is visible at the interface, which is §32.

Failure. Setting BATCH_MAX to the queue depth, which removes the cap while leaving the code. Or omitting the opp_pending qualification, which switches direction on an empty opposite queue and pays the turnaround repeatedly.

DV. Drive sustained one-directional load with a trickle of the other and confirm opp_wait_q never reaches AGE_MAX.

32. SVA — Both Directions Are Served, Under Assumptions

// LIVENESS, bounded, both directions, assumptions stated (15.2 Section 36).
//
//   A1: an issued access completes within a bound
//   A2: a pending direction stays pending until served
//   A3: maintenance terminates (17.1 Section 14)
assume property (@(posedge clk) disable iff (!rst_n)
  issue_fire |-> ##[1:ACCESS_BOUND] access_done);
 
property p_reads_served_within_bound;
  @(posedge clk) disable iff (!rst_n)
    read_pending |-> ##[1:DIR_SERVICE_BOUND] read_issued;
endproperty
a_reads_served_within_bound: assert property (p_reads_served_within_bound);
 
property p_writes_served_within_bound;
  @(posedge clk) disable iff (!rst_n)
    write_pending |-> ##[1:DIR_SERVICE_BOUND] write_issued;
endproperty
a_writes_served_within_bound: assert property (p_writes_served_within_bound);
 
// Every channel with eligible work is eventually served — no channel starves.
property p_channel_not_starved(int c);
  @(posedge clk) disable iff (!rst_n)
    (|eligible_in_channel(c)) |-> ##[1:CHANNEL_SERVICE_BOUND] channel_issued[c];
endproperty
a_channel_not_starved: assert property (p_channel_not_starved(CH_UT));

Architecture. Three bounded liveness properties: read direction, write direction, and per channel.

Why the third is separate. The direction bounds are per channel; the third says no channel is starved by whatever arbitrates across channels for shared ingress or return-path resources. A design can serve both directions correctly within each channel and still never schedule channel 7.

Why the assumptions must be explicit. Without A3, the read property fails on a design that is correct and merely waiting out a maintenance window. Stating it converts "this hangs" into "this hangs because maintenance did not terminate" — a diagnosis rather than a symptom.

DV. Prove all three. Then set BATCH_MAX to the queue depth and confirm the opposite-direction property fails, verifying the cap is load-bearing.

33. Protection Domains Do Not Overlap

17.1 §23 established that correction lives in more than one place. With a parallel subsystem behind a link, the domains must be named precisely.

A clean UCIe CRC proves that a transport object survived a link. It does not prove the controller's SRAM held the address correctly, that the media stored the right value, that the metadata is intact, or that the response is associated with the right request. §26 is a corruption in which every CRC passes and every bit is correct.

Two consequences.

Counting must attribute each event to one domain (17.1 §23). One physical error visible at two check points is one error, and double-counting misjudges a device's health.

And the gaps between domains are where end-to-end protection earns its cost. Every boundary between two mechanisms is a region covered by neither, and §34 is the specific gap that produces the worst failure.

34. Data Is Protected and Metadata Is Not

The gap that matters most, stated as a question: if the ingress queue protects the data field but not the address, identity or byte enables — what happens when one of those bits flips?

Three of those five are silent corruptions, and the data-protection mechanism reports success in every one.

Protect the whole entry, not the payload. An address corrupted in a queue is indistinguishable from a request that was always for that address, and nothing downstream can tell.

35. A Protected Queue Entry

// ILLUSTRATIVE internal protection. NOT an HBM-defined structure and NOT a
// claim about any device's ECC (Section 3). Parity shown for brevity; a real
// design chooses strength from its own error budget.
typedef struct packed {
  logic [ADDR_W-1:0]   addr;
  logic [REQ_ID_W-1:0] id;
  logic                is_write;
  logic [BE_W-1:0]     byte_enables;
  logic [DATA_W-1:0]   data;
  logic [PAR_W-1:0]    parity;      // over EVERY field above (Section 34)
} protected_req_t;
 
// Checked on read-out, before the request is used for anything.
wire entry_ok = (calc_parity(entry_read) == entry_read.parity);
 
always_comb begin
  issue_valid = queued_valid && entry_ok;
  entry_error = queued_valid && !entry_ok;      // reported, never ignored
end

Architecture. One check word over the whole entry, verified before the entry influences anything. That ordering is 14.1 §19's delivery gate: check, then act, never the reverse.

State. PAR_W extra bits per entry. The width is an error-budget decision — parity detects, it does not correct, and a design that needs correction needs a stronger code and more bits.

Cycle behaviour. Combinational on read-out, gating issue_valid. A registered check issues one request before deciding it was corrupt (14.1 §20), and by then the access has been sent.

Contract. Everything downstream assumes the entry is what was written. entry_error must be reported and counted, never silently dropped — a design that discards a corrupt entry without reporting turns a detected error into a lost request.

Failure. Covering only data (§34). Or checking after the address has already been used to select a bank, which is the same one-cycle-too-late error in a different place.

DV. Inject a flip in each field separately and confirm detection and reporting. Five directed tests, one per field, and they are the only proof that the check covers what §34 requires.

And the wrong form, which is the common one:

// WRONG — parity over the data field alone.
assign entry_parity = ^entry.data;               // ← addr, id, be, is_write unprotected

A single flip in addr writes correct, parity-clean data to the wrong location. Two addresses are now wrong — the intended one still holds its old value, and an unrelated one has been overwritten — and every check in the system passes.

The signature is indistinguishable from §17's overflow and from a mapping error, which is why the per-field injection tests matter: they are what let a debugging engineer exclude this mechanism rather than suspect it forever.

36. Repair and Degradation at System Scope

17.1 §23 introduced repair as device state. At subsystem scope the consequences are architectural:

37. Requested and Active Channel Configuration

// ILLUSTRATIVE. The same discipline as 14.4's lane repair and 17.2's memory map,
// at memory-resource scope.
logic [NUM_CHANNELS-1:0] requested_channel_mask_q;   // software writes here
logic [NUM_CHANNELS-1:0] active_channel_mask_q;      // ingress reads here
logic [CH_W-1:0]         requested_channel_lut [2**HASH_W];
logic [CH_W-1:0]         active_channel_lut    [2**HASH_W];
 
assign channel_commit_allowed =
      requested_mask_validated                 // every LUT entry names an active channel
   && (outstanding_on_removed_channels == '0)  // Section 39's guard
   && (occ_on_removed_channels == '0);         // their queues have drained
 
always_ff @(posedge clk)
  if (channel_commit_fire) begin
    active_channel_mask_q <= requested_channel_mask_q;   // atomic, together
    active_channel_lut    <= requested_channel_lut;      // with the mapping
  end

Architecture. Two copies of both the mask and the mapping table, committed together and atomically. Committing the mask without the table, or the reverse, produces a window where a request maps to a channel that is no longer active — §39.

State. Two masks and two tables. The duplication is the cost of atomicity, and for a mask it is trivial; for the LUT it is the price of not corrupting traffic during a repair.

Cycle behaviour. One-cycle transfer of both structures. The three-term guard must hold: the new mapping is internally valid, no request is outstanding to a channel being removed, and those channels' queues have drained.

Contract. The ingress sees a complete, consistent configuration or the previous one. Never a mixture.

Failure. §39. Also validating the mask and not the LUT, which permits a table entry pointing at a removed channel — and requests then enter a queue that is never scheduled.

DV. After every commit, assert every LUT entry names an active channel; assert the commit was guarded (§40).

38. Wrong RTL — Removing a Channel Under Live Traffic

// WRONG — the mask is applied immediately, with requests still outstanding.
always_ff @(posedge clk)
  if (channel_disable_req)
    active_channel_mask_q[disable_ch] <= 1'b0;   // ← no guard, no drain

1. Channel 5 has 12 requests queued and 4 outstanding at the controller.
2. Software disables channel 5. The mask clears immediately.
3. The scheduler no longer schedules channel 5 — the 12 queued requests
   will never issue.
4. The controller completes the 4 outstanding accesses.
5. Their responses arrive at an ingress that no longer considers channel 5
   active. Depending on the design they are dropped, or matched against
   an outstanding table that has been cleared.
6. -> 12 requests silently lost, and 4 responses orphaned.

Three properties.

No component failed. Software issued a legal request; the mask logic did what it was told; the controller completed its accesses. The error is doing it at the wrong moment.

And it is the same configuration-lifetime principle as everywhere else in this curriculum — 14.4's lane repair, 17.2 §15's remap, 16.5 §14's destination recomputation. Here the consequence is lost requests rather than misrouted ones, because the resource disappears rather than changing identity.

The correct sequence is quiesce, drain, validate, commit (§37): stop admitting to the affected channels, wait for their queues and outstanding counts to reach zero, validate the new mapping, then commit the mask and the table together.

39. SVA — the Channel Configuration Is Stable While Work Depends On It

// MANDATORY. The active mask changes only at a guarded commit.
property p_mask_changes_only_at_commit;
  @(posedge clk) disable iff (!rst_n)
    !channel_commit_fire |=> $stable(active_channel_mask_q);
endproperty
a_mask_changes_only_at_commit: assert property (p_mask_changes_only_at_commit);
 
// A commit requires the removed channels to be fully drained.
property p_commit_requires_drain;
  @(posedge clk) disable iff (!rst_n)
    channel_commit_fire |-> ((outstanding_on_removed_channels == '0)
                          && (occ_on_removed_channels == '0));
endproperty
a_commit_requires_drain: assert property (p_commit_requires_drain);
 
// The mask and the mapping table change together — never one without the other.
property p_mask_and_lut_commit_together;
  @(posedge clk) disable iff (!rst_n)
    $changed(active_channel_mask_q) |-> channel_commit_fire;
endproperty
a_mask_and_lut_commit_together:
  assert property (p_mask_and_lut_commit_together);
 
// Every mapping-table entry names an active channel.
property p_lut_entries_active;
  @(posedge clk) disable iff (!rst_n)
    channel_commit_fire |=> all_lut_entries_active(active_channel_lut, active_channel_mask_q);
endproperty
a_lut_entries_active: assert property (p_lut_entries_active);
 
// A live request's channel is never changed under it.
property p_request_channel_immutable;
  @(posedge clk) disable iff (!rst_n)
    out_q[IDX].valid |-> $stable(out_q[IDX].channel);
endproperty
a_request_channel_immutable: assert property (p_request_channel_immutable);

Architecture. Five properties: commit-only changes, drain guard, joint commit, table validity, and per-request immutability.

Why the fourth fires at the commit rather than on use. A table entry naming a removed channel produces a request that enters a never-scheduled queue — a hang whose cause is thousands of cycles upstream. Checking at the commit names the configuration instead of its first victim, which is 17.2 §10's argument.

Why the fifth is separate from the first. A stable mask does not prove a request's recorded channel is stable; a design can freeze the configuration correctly and still let a downstream stage re-derive the channel from the live mapper. Same shape as 17.2 §14's captured-route property.

DV. The second needs a commit attempted with outstanding work — a directed test, and §53's cp_commit_context bin.

40. UCIe Bandwidth Against Memory Bandwidth

The arithmetic that keeps peak numbers honest.

effective_system_bandwidth  <=  min(
    UCIe useful bandwidth to this subsystem,     // 15.1's useful, not raw
    ingress/return path sustainable rate,
    memory controller sustainable rate,
    media sustainable rate (channels × per-channel, minus maintenance),
    source demand                                 // what the workload asks for
)

Three regimes, and each has a different fix:

The third regime is the most common and the least suspected, because both the link and the memory report healthy utilisation figures — which are low. A system with 10% link utilisation and 10% channel utilisation is not two problems; it is one source problem.

41. A Worked Bandwidth Bound

Illustrative numbers throughout. No source is claimed for any figure.

effective <= min(400, 500, 640, 620, 700) = 400 GB/s      <- UCIe binds

Three readings.

The link binds at 400 GB/s, and the media is idle 35% of the time (1 − 400/620). Adding channels raises 620 and changes nothing.

Doubling the link to 800 GB/s moves the bound to the media at 620 GB/s — a 55% improvement, not 100%. The next constraint is always closer than the ratio of the one you fixed.

And if the source only demanded 300 GB/s, the answer would be 300 with everything else idle. The same subsystem, the same hardware, and a completely different diagnosis — which is why §46's classifier includes a source-empty reason.

42. Never Sum Peaks

WRONG:  4 stacks × peak_per_stack  +  N links × peak_per_link  =  system bandwidth

Four reasons the sum is not reachable, and each is a shared resource the peaks ignore:

Two consequences.

This is 15.4's min-cut argument at memory scope. The reachable bandwidth is bounded by the narrowest cut between the source and the media, and the narrowest cut is frequently not the media.

And "peak" for the media already excludes maintenance (§28) — so even the media term in a sum is optimistic unless the maintenance overhead has been deducted. Two optimisms compounding is how a system is specified at twice its achievable rate.

43. Outstanding Depth for a Parallel Subsystem

17.2 §22's formula, with a parallelism term:

required_outstanding  ≈  target_throughput × round_trip_latency

But there is a floor that has nothing to do with latency:

required_outstanding  >=  NUM_CHANNELS      (to have work in every channel at all)

Worked, illustratively. 8 channels, a round trip of 120 cycles, targeting one request per cycle:

latency-driven   : 1 req/cycle × 120 cycles = 120 outstanding
parallelism floor: 8
required         : max(120, 8) = 120

And with only 4 outstanding requests:

at most 4 channels can have work at any instant
-> at least 4 of 8 channels are idle by construction
-> achievable <= 4/8 = 50% of the balanced case, before any other limit

Three readings.

The parallelism floor is usually the smaller of the two and it is the one people check. "We have more outstanding requests than channels" is a true statement that guarantees almost nothing. The latency term is 15× larger here.

Concurrency below the channel count is a hard structural cap, visible in per-channel utilisation as several channels reading zero (§21).

And this composes with §40. If the source cannot generate 120 outstanding requests, the source is the binding term and no amount of link or memory improvement changes the measurement.

44. Thermal and Power Coupling

A memory subsystem and its compute die share a package.

One consequence for measurement. A short benchmark measures the unthrottled rate; a long one measures the sustainable rate. 15.5 §17's warmup discipline is therefore not optional here — and a design that reports only the burst figure has reported a number the system cannot hold.

45. Performance Counters

// Diagnostic only. Every counter's incrementing EVENT is defined (Section 27's
// last property enforces it).
logic [63:0] channel_busy_q      [NUM_CHANNELS];  // controller occupied by a transfer
logic [63:0] channel_bytes_q     [NUM_CHANNELS];  // on an actual transfer beat
logic [63:0] channel_full_q      [NUM_CHANNELS];  // occ == DEPTH_PER_CHANNEL
logic [63:0] channel_hol_stall_q [NUM_CHANNELS];  // queued>0, eligible>0, no grant
logic [63:0] channel_maint_q     [NUM_CHANNELS];  // unavailable for maintenance
logic [63:0] channel_ineligible_q[NUM_CHANNELS];  // queued>0, eligible==0
logic [63:0] ucie_stall_q;                        // ingress blocked by transport
logic [63:0] return_stall_q;                      // completion blocked on return path
logic [63:0] outstanding_at_max_q;                // Section 43's constraint
logic [63:0] source_empty_q;                      // nothing offered

Two counters here answer questions no other pair can.

channel_hol_stall_q against channel_ineligible_q. The first means work was legal and the scheduler did not issue it — a scheduler problem (§24). The second means work was queued and none was legal — a timing or maintenance situation, which is correct behaviour. Identical from outside, opposite conclusions.

And return_stall_q against ucie_stall_q. Both look like "the link is the problem"; the first is the completion path and the second is the request path. They are fixed by different resources, and a single "link busy" counter names neither.

46. One Primary Reason Per Cycle

// ILLUSTRATIVE. 15.5 Section 10's causal-priority classifier, at the memory
// ingress. Exactly one bin per cycle; the ten sum to elapsed cycles.
typedef enum logic [3:0] {
  HBM_TRANSFER      = 4'd0,   // useful work happened
  HBM_SOURCE_EMPTY  = 4'd1,   // nothing was offered              <- Section 40 regime 3
  HBM_UCIE_BLOCKED  = 4'd2,   // request path blocked
  HBM_RETURN_FULL   = 4'd3,   // completion path blocked
  HBM_CHANNEL_FULL  = 4'd4,   // the SELECTED channel had no room  <- Section 17
  HBM_NO_OUTSTANDING= 4'd5,   // identity table full               <- Section 43
  HBM_INELIGIBLE    = 4'd6,   // queued, none legal to issue
  HBM_MAINT         = 4'd7,   // maintenance                       <- correct behaviour
  HBM_COMMIT_DRAIN  = 4'd8,   // deliberate quiesce                <- Section 37
  HBM_SCHED_IDLE    = 4'd9    // eligible work not granted         <- Section 24, a BUG
} hbm_reason_e;
 
hbm_reason_e reason_d;
 
always_comb begin
  unique case (1'b1)
    transfer_beat                        : reason_d = HBM_TRANSFER;
    !req_offered                         : reason_d = HBM_SOURCE_EMPTY;
    commit_drain_active                  : reason_d = HBM_COMMIT_DRAIN;
    maintenance_blocking                 : reason_d = HBM_MAINT;
    !ucie_ingress_ready                  : reason_d = HBM_UCIE_BLOCKED;
    !return_path_ready                   : reason_d = HBM_RETURN_FULL;
    !outstanding_space                   : reason_d = HBM_NO_OUTSTANDING;
    (occ_q[channel_sel] == DEPTH_PER_CHANNEL) : reason_d = HBM_CHANNEL_FULL;
    (|queued && !(|eligible))            : reason_d = HBM_INELIGIBLE;
    (|eligible && !(|grant))             : reason_d = HBM_SCHED_IDLE;
    default                              : reason_d = HBM_TRANSFER;
  endcase
end

Architecture. Ten mutually exclusive reasons in causal priority. HBM_SOURCE_EMPTY is second so idle cycles never inflate a resource bin — the most common way a classifier lies (17.2 §32).

Contract. The ten counters sum to elapsed cycles. That conservation is asserted, and it is what makes a residual meaningful (15.5 §12).

The bin that matters most. HBM_SCHED_IDLE is the only reason in the list that is unambiguously a bug — eligible work existed and the scheduler issued nothing, which is §24. Every other bin describes a resource limit or correct behaviour. A non-zero value there is a defect, and no other instrument distinguishes it from HBM_INELIGIBLE.

Failure. Independent counters per condition, which double-count every cycle where two hold, and the percentages exceed 100% in a way nobody notices until they are summed.

47. The HBM Scoreboard

// Verification-only. FOUR models — the fourth is what makes a performance
// claim checkable rather than anecdotal.
class hbm_scoreboard;
 
  // ---- Layer 1: ADDRESS/CHANNEL model. Predicted independently, per epoch.
  typedef struct {
    int channel;
    int config_epoch;
  } map_model_t;
  map_model_t predicted_map [bit [ADDR_W-1:0]];
 
  function int predict_channel(bit [ADDR_W-1:0] a, int epoch);
    return lut_model[epoch][hash_slice(a)];      // every epoch retained
  endfunction
 
  // ---- Layer 2: REQUEST LIFETIME model.
  typedef struct {
    bit              live;
    bit [ADDR_W-1:0] addr;
    int              accepted_channel;   // what the DESIGN chose
    int              predicted_channel;  // what the MODEL says
    bit              is_read;
    int              completions;        // MUST be <= 1
    int              issue_order;
    int              complete_order;     // may differ — Section 25
  } req_model_t;
  req_model_t reqs [int];                // keyed by requester identity
 
  // ---- Layer 3: MEMORY CONTENT model.
  bit [DATA_W-1:0] mem [bit [ADDR_W-1:0]];
 
  // ---- Layer 4: PERFORMANCE model, per channel.
  typedef struct {
    int accepted;
    int issued;
    int completed;
    longint bytes;
    int max_occupancy;
    int maint_cycles;
  } perf_model_t;
  perf_model_t perf [int];               // keyed by channel
 
  // ---- Catches Section 9 and Section 39.
  function void check_channel_selection(int id);
    if (reqs[id].accepted_channel != reqs[id].predicted_channel)
      $error("CHANNEL MISMAP id %0d addr %0h: design chose %0d, model says %0d",
             id, reqs[id].addr, reqs[id].accepted_channel, reqs[id].predicted_channel);
  endfunction
 
  // ---- Catches Section 26 — the association error no value check finds.
  function void check_association(int id, bit [DATA_W-1:0] got);
    if (reqs[id].is_read && (got !== mem[reqs[id].addr]))
      $error("ASSOCIATION/DATA ERROR id %0d addr %0h: got %0h expected %0h "
           , "(issue order %0d, completion order %0d)",
             id, reqs[id].addr, got, mem[reqs[id].addr],
             reqs[id].issue_order, reqs[id].complete_order);
  endfunction
 
  // ---- Catches Section 20 and Section 29 — a PERFORMANCE bug, not a functional one.
  function void check_balance(int total_accepted);
    foreach (perf[c])
      if (perf[c].accepted * NUM_CHANNELS < total_accepted / IMBALANCE_TOL)
        $display("NOTE: channel %0d received %0d of %0d — imbalance (Section 20)",
                 c, perf[c].accepted, total_accepted);
  endfunction
 
endclass

Architecture. Four models keyed by address-and-epoch, by requester identity, by address, and by channel.

Why the fourth layer exists at all. Layers 1 to 3 prove correctness. §20's hotspot, §24's head-of-line blocking and §29's over-global stall are all functionally correct and cost most of the bandwidth — and no correctness model detects any of them. A subsystem whose entire justification is bandwidth needs a model that checks bandwidth.

Why issue_order and complete_order are both recorded. They differ, legitimately (§25). Recording both is what turns "the data was wrong" into "the data was wrong on a completion that arrived 6 positions out of order" — which points directly at §26.

And why every epoch's LUT is retained. A request accepted under one channel configuration must be checked against that configuration, not the current one — the same reason 17.2 §35 keeps every map epoch.

48. Flagship Trace 1 — Balanced Striping

Illustrative. Four requests arriving on consecutive cycles, striping across four channels. Cycle numbers illustrative.

Four readings.

Each admission checked occ[channel_sel], not an aggregate. §17's design would have checked a total of 4D every time and been right by luck here — and wrong the moment one channel filled.

All four channels are busy from cycle 8, which is what §7's independence buys and what §20's hotspot destroys.

Completions arrive in the order 11, 13, 10, 12 — none of them issue order. Every one is matched by req_id. §26's design delivers channel 1's data to id 10.

And four outstanding requests is exactly NUM_CHANNELS in this illustration — the parallelism floor of §43, and nowhere near the latency-driven requirement. Sustaining this rate would need far more in flight.

49. Flagship Trace 2 — a Hotspot

Same subsystem, an address pattern whose interleave field is constant.

Four readings.

Measured bandwidth is one channel's worth. Roughly a quarter of the balanced case in this illustration. The hardware peak is unchanged.

Three of four channels are idle and there is no bug. Every component is correct; the workload asked one channel for everything (§20).

The stall bin is HBM_CHANNEL_FULL, not HBM_UCIE_BLOCKED — and that distinction is the whole diagnosis. A single "stalled" counter would have pointed at the link, which is idle.

And §17's design behaves differently and worse here. With an aggregate free count of 3D, it accepts the request for the full channel 2 — overwriting, dropping, or blocking the ingress. The hotspot and the admission bug compound, which is why §17's failure is invisible until §20's conditions arise.

50. Flagship Trace 3 — One Channel in Maintenance

And the wrong design (§29), same event:

Three readings.

In the correct design, three channels never stop. Only requests mapping to channel 2 wait, and only its queue fills. The subsystem loses roughly one channel's share for the duration.

In the wrong design all four stop, and §29's arithmetic shows the compounding: with more channels the loss grows even though each channel's maintenance fraction is unchanged.

And the reason bin distinguishes them immediately. The correct design reports HBM_CHANNEL_FULL on a minority of cycles; the wrong one reports HBM_MAINT on all of them. Same event, same hardware, two completely different histograms.

51. Flagship Trace 4 — the Link Is Faster Than the Memory

Three readings.

The queues fill and then the system reaches equilibrium at the memory's rate. That is correct behaviour, not a fault. The link's extra capacity is simply unused.

The binding term is the media, and the histogram says so. HBM_CHANNEL_FULL dominates while HBM_UCIE_BLOCKED stays at zero. §40's second regime, identified in one measurement.

And a faster link changes nothing. The equilibrium is set by the memory. A design team that reads "the ingress queues are full" as "we need more link bandwidth" has read the symptom backwards — the queues are full because the link is already faster than the drain.

52. Coverage

covergroup cg_hbm_integration @(posedge clk);
  option.per_instance = 1;
 
  // --- Mapping and balance (Sections 8-12, 20).
  cp_channel : coverpoint channel_sel { bins each[] = {[0:NUM_CHANNELS-1]}; }
  cp_balance : coverpoint traffic_balance_class {
    bins balanced = {0};
    bins skewed   = {1};
    bins hotspot  = {2};                 // Section 49
  }
  cp_nonpow2_config : coverpoint num_active_channels_is_power_of_two {
    bins pow2     = {1};
    bins non_pow2 = {0};                 // Section 9's regime
  }
 
  // --- Queues and admission (Sections 13-18).
  cp_channel_occ : coverpoint occ_q_ut {
    bins empty = {0};
    bins mid   = {[1:DEPTH_PER_CHANNEL-1]};
    bins full  = {DEPTH_PER_CHANNEL};    // Section 17
  }
  cp_one_full_others_empty : coverpoint one_channel_full_others_empty;  // Section 17
  cp_simul_push_pop : coverpoint push_and_pop_same_channel_same_cycle;  // Section 15
 
  // --- Scheduler (Sections 22-24).
  cp_eligibility : coverpoint eligibility_class {
    bins none_queued      = {0};
    bins queued_none_elig = {1};         // Section 24's precondition
    bins some_eligible    = {2};
    bins all_eligible     = {3};
  }
  cp_aged_out : coverpoint age_override_fired;
 
  // --- Return path (Sections 25-27).
  cp_reorder : coverpoint response_reorder_distance {
    bins in_order = {0}; bins near = {[1:3]}; bins far = {[4:$]};
  }
  cp_cross_channel_reorder : coverpoint completions_interleaved_across_channels;
  cp_outstanding : coverpoint outstanding_occupancy {
    bins below_channels = {[1:NUM_CHANNELS-1]};   // Section 43's floor violated
    bins at_channels    = {NUM_CHANNELS};
    bins deep           = {[NUM_CHANNELS+1:MAX_OUTSTANDING-1]};
    bins full           = {MAX_OUTSTANDING};
  }
 
  // --- Direction (Sections 30-32).
  cp_mix : coverpoint read_write_mix_class {
    bins read_heavy = {0}; bins write_heavy = {1}; bins mixed = {2};
  }
  cp_dir_switch : coverpoint direction_switch_cause {
    bins batch_cap = {0}; bins age_bound = {1};   // both must fire
  }
 
  // --- Maintenance and configuration (Sections 28-29, 36-39).
  cp_maint : coverpoint channels_in_maintenance {
    bins none = {0}; bins one = {1}; bins several = {[2:$]};
  }
  cp_commit_context : coverpoint channel_commit_attempt {
    bins idle              = {0};
    bins blocked_by_outst  = {1};        // Section 39's guard working
    bins forced_with_outst = {2};        // guard disabled — Section 38
  }
  cp_degraded_mask : coverpoint active_channel_count_reduced;
 
  // --- Protection (Sections 33-35).
  cp_protection_inject : coverpoint protection_injection_field {
    bins none = {0}; bins data = {1}; bins addr = {2};
    bins id   = {3}; bins be   = {4}; bins dir  = {5};   // ALL five, Section 34
  }
 
  // --- Attribution (Section 46) — every reason must be reachable.
  cp_reason : coverpoint reason_d { bins each[] = {[0:9]}; }
 
  // --- Crosses that carry the information.
  x_balance_occ    : cross cp_balance, cp_channel_occ;      // Section 49
  x_maint_reason   : cross cp_maint, cp_reason;             // Section 50
  x_commit_outst   : cross cp_commit_context, cp_outstanding;
  x_reorder_chan   : cross cp_reorder, cp_channel;          // Section 26
  x_nonpow2_chan   : cross cp_nonpow2_config, cp_channel;   // Section 9
endcovergroup

Seven bins worth calling out:

cp_nonpow2_config.non_pow2 crossed with cp_channel. §9's failure. A design verified only at 8 channels never reaches it, and it appears after the first repair.

cp_one_full_others_empty. §17's exact precondition, and the state a balanced regression never produces.

cp_eligibility.queued_none_elig. §24's precondition — work present, none legal. A sequential benchmark never reaches it.

cp_outstanding.below_channels. §43's floor violated, so several channels are idle by construction. Covering it proves the environment can produce the low-concurrency regime that §40's third case describes.

cp_protection_inject — all five fields. §34's requirement. Injecting only into data verifies the one field that was already protected.

cp_commit_context.forced_with_outst. Run with the guard disabled, to prove the per-request channel immutability is a real second defence rather than dead code.

And cp_reason — every one of the ten. In particular HBM_SCHED_IDLE should be unreachable in a correct design; reaching it in a regression is §24.

53. Debug Taxonomy

Row 8 is the one worth memorising. A channel idling with a full queue has two opposite explanations — the scheduler failing to issue legal work, or no work being legal — and channel_hol_stall_q against channel_ineligible_q is the only pair that separates them.

54. Debug Checklist

1. Which channel did the address map to, and under which configuration epoch? (§8, §37)
2. Is NUM_CHANNELS a power of two, and does the mapping assume it is? (§9, §11)
3. What is channel_accepted_q across all channels? (§21)
4. Which channels have zero accepted requests? (§9 vs §20)
5. What is the interleave granularity, and how large is the active address range? (§12)
6. Did admission check the selected channel or an aggregate? (§16, §17)
7. What is each channel's occupancy, and which are at depth? (§15)
8. How many requests are outstanding, and is it above NUM_CHANNELS? (§43)
9. Is the outstanding table ever at maximum? (§43, §45)
10. For a stalled channel: is work queued but ineligible, or eligible and ungranted? (§45, §53)
11. Did the age override ever fire? (§23)
12. Is any channel in maintenance, and is the stall scoped to it? (§28, §29)
13. Was the returned identity the one issued, and how far out of order? (§25, §26)
14. Does the internal protection cover address, identity, byte enables and direction — or only data? (§34)
15. Was a channel-mask commit attempted, and was it guarded by a drain? (§37, §39)
16. Does every mapping-table entry name an active channel? (§39)
17. What does the reason histogram say, and is HBM_SCHED_IDLE non-zero? (§46)
18. Do the ten reason counters sum to elapsed cycles? (§46)
19. Which of the five bandwidth terms is the minimum? (§40, §41)
20. Was the measurement window long enough to include throttling? (§44)
21. Which of the four scoreboard layers diverged first? (§47)

55. Common Misconceptions

"HBM bandwidth comes from one giant bus." It comes from many independently addressable channels, each with independently busy banks. Bandwidth is the sum over resources that are simultaneously busy, not over resources that exist (§7, §20).

"HBM attached over UCIe means UCIe speaks to DRAM arrays." It does not. UCIe attaches a logic chiplet; the memory-side die contains or drives the controller and the native memory interface, which UCIe never carries. The two interfaces have completely independent bandwidths, which is why §40's arithmetic is necessary (§4).

"Peak HBM bandwidth is application bandwidth." Achievable bandwidth is the minimum of the link's useful rate, the ingress and return paths, the controllers, the media after maintenance, and the source's demand. The media term is the one people quote and frequently not the binding one (§40, §41).

"More channels automatically mean balanced traffic." Balance is a property of the address pattern and the mapping, not of the channel count. A workload whose interleave field is constant uses one channel regardless of how many exist, and adding channels makes the ratio worse (§20).

"One full channel should stall the whole subsystem." It should not. Per-channel backpressure lets the other channels continue, and a global stall gives up throughput proportional to the channel count — an illustrative 15% at eight channels and over 27% at sixteen, to avoid 2% of unavailability (§19, §29).

"FIFO order can be used for read-return matching." The scheduler reorders within a channel and channels complete independently, so return order carries no information. Matching by position delivers each read the other's data with clean CRC and correct bits throughout (§25, §26).

"UCIe CRC replaces memory ECC." A clean CRC proves a transport object survived a link. It proves nothing about the controller's storage, the media, the metadata, or which request a response belongs to. The domains do not overlap, and the gaps between them are where the worst failures live (§33, §34).

"Protecting the data field is enough." A flip in the address writes correct, check-clean data to the wrong location; a flip in the identity delivers it to the wrong requester; a flip in the byte enables writes the wrong extent. Three of five fields produce silent corruption that data protection reports as success (§34, §35).

"A wider or faster UCIe link always improves HBM performance." Only when the link is the binding term. In the worked example, doubling a binding 400 GB/s link moves the bound to the media at 620 GB/s — a 55% gain, not 100% — and if the memory were already binding it would be a 0% gain (§41).

"More outstanding requests always help." They help until the latency-driven requirement is met and then they do nothing. Below the channel count they are a hard structural cap; above the latency requirement they add queueing delay without adding throughput (§43).

"Refresh or maintenance means the memory is failing." It is correct, specified, unavoidable behaviour (17.1 §11). Classifying it as a fault leads to starving it, which loses data (§28).

"Removing a degraded channel is just a routing-bit change." It changes capacity, bandwidth, and the mapping — potentially to a non-power-of-two count that breaks a mask mapping — and it must be quiesced, drained, validated and committed atomically with the mapping table, or queued requests are lost and outstanding responses are orphaned (§36, §38).

56. Understanding Check

57. Summary and What Comes Next

HBM bandwidth is a concurrency problem before it is a pin-width number. Peak capability is reachable only with enough independent work across channels and banks, and with a return path that does not re-serialise it.

UCIe attaches a logic chiplet, never a DRAM array. The link and the native memory interface have independent bandwidths, which is why the effective rate is a minimum over five terms and why fixing the wrong one changes nothing.

Admission must consult the selected channel. An aggregate free count accepts a request for a full channel while seven others are empty, and the result is an overwrite, a drop, or a subsystem-wide stall.

A scheduler must distinguish queued from eligible. FIFO head blocking idles a channel while legal work waits behind it, and only a pair of counters separates that defect from the correct case of nothing being eligible.

Return order carries no information. Matching by position delivers each read the other's data with clean CRC and correct bits throughout.

One channel's maintenance must not stall the stack — a global scope costs an illustrative 15% at eight channels and over 27% at sixteen, to avoid 2% of unavailability, and it worsens with the parameter meant to deliver the bandwidth.

Protect the whole entry, not the payload. A flipped address writes check-clean data to the wrong place, and every mechanism in the system reports success.

And a channel set is configuration with a lifetime. Quiesce, drain, validate, commit the mask and the mapping together — or queued requests are lost and outstanding responses are orphaned.

Module 17 has now moved from a single memory chiplet as a stateful endpoint, through remote capacity joining one map, through computation moving toward the data, to a massively parallel subsystem behind a chiplet boundary. The thread running through all four is the same: memory is not a passive target, and every one of its properties — availability, ordering, identity, protection, configuration — has a lifetime the rest of the system must respect.

• 17.5 — Future Memory Architectures — where memory chiplets are going next.

Browse the full path on the UCIe tutorials index.