Memory Chiplets

Chapter 2.4 established why memory separates from logic and why capacity and bandwidth are different products. Module 16 has spent five chapters on coherence operating over memory. This chapter turns to the memory chiplet itself, as a manufactured die with obligations of its own.

1. The One-Sentence Model

A memory chiplet is not a passive target at the end of a link. It is a stateful endpoint with its own timing constraints, its own maintenance obligations, and periods of scheduled unavailability that the requester cannot see and must not misread as failure.

2. What This Chapter Owns

What this chapter owns, and it is a genuinely different layer from 2.4:

A memory die is stateful in a way a logic endpoint is not. Bank state, timing history, refresh position and repair configuration are all live state that determines whether the next request can be served now. §5 is that argument, and everything after it follows.

Availability is scheduled, not binary. A DRAM array is periodically unavailable for maintenance, and that unavailability is correct behaviour. §11 through §14 are about not mistaking it for a fault.

Controller placement moves specific state, not a vague responsibility. 2.4 asked where the controller sits; this chapter enumerates exactly which structures move with it and what each side must then know (§8).

HBM's bandwidth is a parallelism structure. It comes from many independent access paths, not from one very wide one — and that difference changes what a request stream must look like to extract it (§15).

And identity survives reordering. A memory scheduler reorders for efficiency, so responses return out of order and must carry identity. §17 is that contract, and §19 is the bug.

3. Sourcing

4. What a Memory Chiplet Actually Is

Three properties that distinguish it from every other chiplet in this curriculum.

It is the only endpoint whose content is the product. A compute chiplet is valuable for what it does; a memory chiplet is valuable for what it holds. Losing its state is not a performance event — it is the loss of the thing itself.

It is the only endpoint that must do work when nobody asked it to. Maintenance happens on the memory's schedule, not the requester's (§11).

And it is the only endpoint whose ability to serve depends on what it did recently. A read to a bank that was just accessed cannot be served immediately, regardless of link state, credits or queue space. The memory's own history is a resource constraint (§6).

5. The Memory Chiplet as an Endpoint

Read the two arrows into the array. The scheduler and the maintenance engine both need it, and maintenance is not optional — which means the scheduler's throughput is bounded by something the requester cannot see, did not cause, and cannot influence.

6. Service Depends on History

The property that makes memory unlike every other target.

// ILLUSTRATIVE bank state. Symbolic timing parameters throughout — no JEDEC
// value is used or implied (Section 3).
typedef struct packed {
  logic                     row_open;
  logic [ROW_W-1:0]         open_row;
  logic [TIMER_W-1:0]       busy_until;      // cycles remaining before reusable
} bank_state_t;
 
bank_state_t bank_q [NUM_BANKS];
 
// A request is ISSUABLE only if its bank is out of its recovery window.
function automatic logic bank_issuable(int b);
  return (bank_q[b].busy_until == '0);
endfunction

Architecture. Per-bank state recording whether a row is open, which one, and how long the bank must wait before it can accept another access. The busy_until field is the whole idea: an access consumes the bank for a period afterwards, so recent history determines current availability.

State. NUM_BANKS entries, each a small counter and a row register. The counters are the design's model of the array's physics, and they must be conservative — a controller that believes a bank is free before it is has produced a timing violation, which corrupts data rather than delaying it.

Cycle behaviour. busy_until is loaded when an access issues and decrements each cycle. A bank at zero is issuable; a bank above zero is not, no matter how urgent the request.

Contract. The requester upstream sees only latency. It has no way to know whether a slow response means a busy bank, a maintenance window, a congested link or a scheduler decision — which is why §21's timeout must not assume any of them.

Failure. Decrementing on a stalled cycle when the array is not actually progressing. Or sizing the counter too small so it saturates and reads as "free" early — which is the same class of error as a wrapping age counter (13.4 §13), with a far worse consequence.

DV. Assert that no access issues to a bank whose busy_until is non-zero. Then check the converse is exercised: that the test actually creates back-to-back same-bank accesses, which a random address stream with many banks rarely does.

7. Bank Parallelism Is the Whole Performance Story

Consecutive accesses to different banks can overlap; consecutive accesses to the same bank cannot.

Two consequences that matter at the chiplet boundary.

Address mapping decides how well a workload performs, because it decides which banks a request stream touches. A mapping that puts sequential addresses in one bank turns a streaming workload into a serialised one — and the link, the controller and the array are all functioning perfectly while it happens.

And a benchmark can be accidentally designed to succeed or fail. A sweep whose addresses happen to rotate through banks measures near-peak; the same sweep with a different stride can measure a fraction of it. This is 15.5 §26's A/B discipline applied to memory: change one variable, and know which one.

8. Controller Placement Moves State, Not a Responsibility

2.4 §5 asked where the memory controller sits. This chapter answers what moves with it, because that is what an integration engineer must actually plan for.

The right-hand column is why a standalone memory chiplet usually carries its own controller. It converts a timing-sensitive interface into a request/response interface — and a request/response interface is what a die-to-die link can carry robustly across latency, retry and recovery.

Two consequences of that conversion.

The link stops being on the timing-critical path. A retry or a recovery delays a request; it does not violate an array timing constraint, because the constraint is enforced locally by state that never crossed the link (§6).

But identity becomes essential. Once the memory side reorders (§17), the link must carry enough identity to put responses back together — which is the trade the conversion makes, and §19 is what happens when it is not honoured.

9. The Endpoint Request

// ILLUSTRATIVE endpoint request. NOT a CXL, UCIe or JEDEC structure.
typedef enum logic [1:0] { OP_READ, OP_WRITE, OP_MASKED_WRITE } mem_op_e;
 
typedef struct packed {
  logic                  valid;
  logic [REQ_ID_W-1:0]   req_id;        // the REQUESTER's identity — carried, not reassigned
  mem_op_e               op;
  logic [ADDR_W-1:0]     addr;
  logic [BE_W-1:0]       byte_enables;
  logic [DATA_W-1:0]     wdata;
} mem_request_t;

Architecture. One packed object. req_id is the requester's, carried unchanged through the memory chiplet and returned with the response — because the memory side reorders and the requester must be able to reassemble.

State. One register per pipeline stage, plus an entry in the outstanding table (§17) for every read.

Cycle behaviour. Formed at the endpoint, held stable while it waits (§20). Never mutated after acceptance, and in particular the address is never re-derived — which is 16.5 §14's rule at the memory endpoint.

Contract. The requester relies on req_id coming back untouched. A memory chiplet that reassigns identity locally has broken the only mechanism the requester has to match a response to a request.

Failure. Reassigning req_id to a local index and returning the local index. Or dropping byte_enables for full-width writes and reconstructing them — the same class of optimisation 11.4 §2 documents in CXL's link layer, and it is only safe if the reconstruction rule is exact.

DV. Assert that every returned identity was an identity that was issued, exactly once (§18).

10. The Write Obligation

A write is not complete when it is accepted, and it is not complete when it is scheduled.

Two consequences.

A write acknowledgement means whatever the interface defines it to mean, and a design must know which of the four rows it corresponds to. An acknowledgement at row 1 is a flow-control signal; an acknowledgement at row 4 is a durability statement, and treating the first as the second is the memory-endpoint form of 16.5 §11's bug.

And a read that follows a write to the same address must see it. However that is achieved — scheduler ordering, a bypass, or a rule that same-address requests do not reorder — it must be achieved deliberately, and §17's reordering is exactly what threatens it.

11. Maintenance Is an Obligation, Not an Error

A DRAM array requires periodic maintenance to retain its contents. During that maintenance the affected part of the array cannot serve requests.

This is correct, specified, unavoidable behaviour. It is not a fault, not congestion, and not a link problem — and the single most common integration mistake in this chapter is a mechanism upstream that treats it as one.

Three properties of maintenance that shape everything around it:

It is not optional and not deferrable indefinitely. Postponing it too long risks the stored state. So a scheduler may delay maintenance for a bounded time and must then perform it, which makes maintenance a liveness obligation with a deadline — the mirror image of every service bound in Module 13.

It is invisible from outside. The requester sees a longer latency and nothing else. No signal distinguishes "your read is waiting for maintenance" from "your read is waiting behind other traffic", unless the endpoint deliberately exposes one (§22).

And it consumes bandwidth that no workload asked for. Peak achievable throughput is therefore below the array's raw capability by a maintenance overhead — which belongs in 15.1's units discipline as structural overhead, not as an efficiency loss.

12. Refresh Scheduling With a Deficit Counter

// ILLUSTRATIVE maintenance scheduler. Symbolic interval and deadline — no
// JEDEC value is used (Section 3). The structure is what is being taught.
logic [CNT_W-1:0] interval_q;      // counts toward the next required maintenance
logic [DEF_W-1:0] deficit_q;       // how many are owed but not yet performed
logic             maint_request;
logic             maint_urgent;
 
always_ff @(posedge clk or negedge rst_n)
  if (!rst_n) begin
    interval_q <= '0;
    deficit_q  <= '0;
  end else begin
    // Time always advances the obligation.
    if (interval_q == REFRESH_INTERVAL - 1) begin
      interval_q <= '0;
      if (deficit_q != DEF_MAX) deficit_q <= deficit_q + 1'b1;   // saturating
    end else begin
      interval_q <= interval_q + 1'b1;
    end
 
    // Performing maintenance discharges one unit of the obligation.
    if (maint_done && (deficit_q != '0))
      deficit_q <= deficit_q - 1'b1;
  end
 
assign maint_request = (deficit_q != '0);
assign maint_urgent  = (deficit_q >= DEFICIT_LIMIT);   // must win from here on

Architecture. A deficit rather than a periodic trigger. The obligation accumulates with time and is discharged by performing maintenance, so the scheduler is free to defer within a budget and is forced to comply beyond it. That single structure is what lets maintenance be flexible without ever being skipped.

State. One interval counter and one saturating deficit counter. Saturating deliberately — a deficit that wraps reports no obligation at the exact moment the obligation is most severe, which is the same failure shape as 13.4 §13's age counter and here risks the stored data.

Cycle behaviour. interval_q advances unconditionally; time does not stop because the array is busy. deficit_q increments on interval expiry and decrements on completion, and the two events are independent — which is exactly why a single "refresh now" pulse is not sufficient: a pulse that arrives while the array is busy is simply lost.

Contract. The request scheduler must yield when maint_urgent is asserted. That is a hard priority inversion in favour of maintenance, and it is correct — the alternative is losing state.

Failure. Using a periodic pulse instead of a deficit, so a maintenance opportunity missed under load is never made up. Or allowing maint_urgent to be overridden by a high-priority request, which trades correctness for latency and is never the right trade here.

DV. Saturate the endpoint with requests and confirm deficit_q never reaches DEF_MAX and that maintenance still completes. Then confirm the urgent path actually preempts — with a directed test, because a random stream will not reliably reach DEFICIT_LIMIT.

13. Wrong RTL — Maintenance Deferred Whenever Traffic Exists

// WRONG — maintenance is treated as background work.
assign do_maintenance = maint_request && !any_request_pending;

Under sustained load any_request_pending is permanently true and maintenance never happens.

Three properties, and the third is what makes it dangerous.

It is the same shape as 13.4 §16's strict-priority starvation — a class that always yields is a class that never runs. But the starved party here is not a traffic class; it is the array's data retention.

The symptom is not a hang. Unlike every other starvation failure in this curriculum, this one does not stop the system. It returns wrong data, and only for the regions whose maintenance was skipped longest.

And it is load-correlated in the worst possible way. The failure appears only under sustained load, which is exactly the condition a performance campaign creates and a functional regression does not. A design can pass every functional test and fail in production.

The fix is §12's deficit plus an urgency threshold that cannot be overridden — and the assertion in §14 is the thing that proves the threshold is real rather than nominal.

14. SVA — Maintenance Completes Within Its Deadline

// MANDATORY. Maintenance is a bounded liveness obligation with a hard deadline.
//
//   A1: the array eventually accepts an issued maintenance operation
assume property (@(posedge clk) disable iff (!rst_n)
  maint_issue |-> ##[1:MAINT_ARRAY_BOUND] maint_done);
 
property p_maintenance_within_deadline;
  @(posedge clk) disable iff (!rst_n)
    maint_request |-> ##[1:MAINT_DEADLINE] maint_done;
endproperty
a_maintenance_within_deadline: assert property (p_maintenance_within_deadline);
 
// The deficit never saturates — saturation means the deadline logic failed.
property p_deficit_never_saturates;
  @(posedge clk) disable iff (!rst_n)
    (deficit_q != DEF_MAX);
endproperty
a_deficit_never_saturates: assert property (p_deficit_never_saturates);
 
// Urgent maintenance is never preempted by a normal request.
property p_urgent_maintenance_not_preempted;
  @(posedge clk) disable iff (!rst_n)
    (maint_urgent && array_issue) |-> array_issue_is_maintenance;
endproperty
a_urgent_maintenance_not_preempted:
  assert property (p_urgent_maintenance_not_preempted);

Architecture. A bounded deadline, a saturation guard, and a preemption guard.

Why the second property is the most valuable. MAINT_DEADLINE is a parameter someone chose, and it may be wrong. deficit_q saturating is a design-independent statement that the obligation is outrunning the discharge, and it fires whether or not the deadline parameter was set correctly.

Why the third is not implied by the first. A design can meet its deadline on average and still let a burst of high-priority requests preempt an urgent maintenance operation. The third property forbids that specific override, which is §13's fix expressed as a rule rather than a hope.

DV. Sustained-load soak with all three enabled. This is one of the few properties in the curriculum whose failure means data loss rather than a hang, and it deserves a dedicated long-duration test rather than a slot in a random regression.

15. HBM — Bandwidth From Parallelism

A stacked memory component gets its bandwidth from having many independent access paths, not from having one very fast one.

Three consequences, and the second is the one that surprises people.

Concurrency is a requirement, not an optimisation. A workload that issues one request at a time cannot use a parallel structure at all, regardless of how large each request is — which is 15.5 §22's outstanding-depth sweep with a physical cause.

Peak bandwidth is only reachable with a spread request stream. Requests concentrated on one path measure a fraction of peak while every component functions perfectly. A benchmark that reports low bandwidth may be measuring its own address pattern (§16).

And maintenance becomes less disruptive. With independent paths, maintenance on one does not stop the others — so parallelism improves availability as well as bandwidth, which is a genuine second-order benefit and rarely stated.

16. Wrong Assumption — Bandwidth Is a Property of the Memory

CLAIM:   "The memory chiplet delivers X GB/s, so our workload will get X GB/s."
REALITY: X is reachable only with enough concurrent requests, spread across
         enough independent access paths, with maintenance overhead deducted.

The fourth row is the one to instrument for, because it is the one where the memory is blameless and the memory is blamed. 15.5 §33's residual analysis is the method: attribute every non-useful cycle to a cause, and the answer will name the link, the concurrency, the spread or the maintenance — but it will name one of them.

17. The Scheduler Reorders — Deliberately

A memory scheduler reorders requests to exploit §6 and §7. Serving a request to an idle bank before one to a busy bank is not opportunism; it is the whole reason the scheduler exists.

// ILLUSTRATIVE outstanding-request table. The scheduler reorders, so responses
// return out of order and MUST carry identity back.
typedef struct packed {
  logic                  valid;
  logic [REQ_ID_W-1:0]   req_id;        // the REQUESTER's, unchanged (Section 9)
  logic [BANK_W-1:0]     bank;
  logic                  is_read;
  logic [AGE_W-1:0]      age;           // saturating — for anti-starvation
} outstanding_t;
 
outstanding_t out_q [MAX_OUTSTANDING];

Architecture. One entry per accepted request, holding the identity that must be returned and the age that stops reordering from starving anyone.

State. MAX_OUTSTANDING entries. This bounds concurrency, and §15 established that concurrency is what reaches peak bandwidth — so this parameter is a first-order performance decision, not a convenience.

Cycle behaviour. Allocated at acceptance, freed when the response is sent for a read, or when durability is reached for a write (§10). The two are different events and the table must distinguish them.

Contract. The requester relies on identity to reassemble. The scheduler relies on age never wrapping, because a wrapped age makes the oldest request look youngest and reordering will then starve it indefinitely.

Failure. §19. Also freeing a read's entry when the array returns data rather than when the response is sent — which loses the entry while the data is still in the response path, and is 15.2 §15's retirement-point error at the memory endpoint.

DV. Assert identity uniqueness and return-completeness (§18). Cover deep reordering — a response returning many positions away from its issue order, which a shallow queue never produces.

18. SVA — Every Request Returns, Exactly Once, With Its Own Identity

// MANDATORY. The contract a reordering endpoint owes its requester.
property p_returned_id_was_issued;
  @(posedge clk) disable iff (!rst_n)
    resp_valid |-> out_q[resp_slot].valid && (out_q[resp_slot].req_id == resp_id);
endproperty
a_returned_id_was_issued: assert property (p_returned_id_was_issued);
 
// No identity is outstanding twice at the same time.
property p_id_unique_while_outstanding;
  @(posedge clk) disable iff (!rst_n)
    req_accept_fire |-> !id_currently_outstanding(req_id_in);
endproperty
a_id_unique_while_outstanding: assert property (p_id_unique_while_outstanding);
 
// Every accepted request eventually returns — bounded, with assumptions.
//   A1: the array completes issued accesses within a bound
//   A2: maintenance terminates (Section 14)
property p_every_request_returns;
  @(posedge clk) disable iff (!rst_n)
    req_accept_fire |-> ##[1:REQ_SERVICE_BOUND] resp_for(req_id_in);
endproperty
a_every_request_returns: assert property (p_every_request_returns);
 
// Reordering does not starve: the oldest entry is served within a bound.
property p_oldest_not_starved;
  @(posedge clk) disable iff (!rst_n)
    (out_q[IDX].valid && out_q[IDX].age >= AGE_LIMIT)
      |-> ##[1:STARVATION_BOUND] resp_for(out_q[IDX].req_id);
endproperty
a_oldest_not_starved: assert property (p_oldest_not_starved);

Architecture. Identity correctness, identity uniqueness, completion liveness, and anti-starvation.

Why identity uniqueness is separate from identity correctness. A design can return the right identity for every response and still have accepted two live requests with the same identity — at which point the first response resolves the wrong one. The uniqueness check is at acceptance, and it is the requester's contract as much as the endpoint's.

Why anti-starvation needs its own property. Reordering is designed to be unfair in the short term. Without an age bound, a stream of requests to idle banks can indefinitely postpone one request to a hot bank, and no other property notices — the endpoint is serving beautifully and one requester is stuck.

DV. The fourth property requires a sustained skewed access pattern, which is a directed test rather than a random one.

19. Wrong RTL — Identity Reassigned Locally

// WRONG — the endpoint replaces the requester's identity with its own index.
always_ff @(posedge clk)
  if (req_accept_fire)
    out_q[alloc_slot].req_id <= alloc_slot;    // ← the requester's id is discarded
 
assign resp_id = out_q[resp_slot].req_id;      // returns a LOCAL index

The requester receives an identity it never issued.

Three consequences.

The requester cannot match the response to anything. Its own outstanding table is keyed by the identity it issued, which no longer exists anywhere in the system.

And if the requester is tolerant, it is worse. A requester that matches by address instead will work — until two outstanding requests target the same address, at which point it matches the wrong one and the failure is a silent data mismatch rather than a hang.

The bug survives casual testing perfectly. With one request outstanding at a time, the local index and the requester's identity happen to correspond, and everything works. It breaks the moment concurrency rises — which is the moment §15 says performance requires.

The correct form carries the requester's identity through untouched, and uses the local slot only as a local index:

// ILLUSTRATIVE. Local slot for local bookkeeping; the requester's identity is
// carried, not replaced.
always_ff @(posedge clk)
  if (req_accept_fire)
    out_q[alloc_slot].req_id <= req_id_in;     // carried through
 
assign resp_id = out_q[resp_slot].req_id;      // the requester's own identity

20. The Request Is Stable While It Waits

// MANDATORY. A memory request offered and not accepted does not change.
property p_request_stable_under_stall;
  @(posedge clk) disable iff (!rst_n)
    (req_valid && !req_ready) |=> (req_valid && $stable(req_payload));
endproperty
a_request_stable_under_stall: assert property (p_request_stable_under_stall);
 
// And the address is never re-derived after acceptance (16.5 Section 14).
property p_address_immutable_after_accept;
  @(posedge clk) disable iff (!rst_n)
    out_q[IDX].valid |-> $stable(out_q[IDX].bank);
endproperty
a_address_immutable_after_accept: assert property (p_address_immutable_after_accept);

Architecture. Two properties: stability while waiting, and immutability after acceptance.

Why the second belongs here as well as in 16.5. The memory endpoint decodes an address into bank, row and column at acceptance. If any part of that decode is re-evaluated later against changed configuration, the access lands somewhere else — and it is the same silent-corruption shape as a recomputed destination, one layer further down.

DV. Both are cheap, both are always-on, and the first catches a class of integration bug that produces corrupted addresses with no other symptom.

21. Wrong RTL — a Timeout That Does Not Know About Maintenance

// WRONG — the requester's timeout assumes memory always responds promptly.
localparam int MEM_TIMEOUT = 512;   // measured on an idle, just-refreshed array
 
always_ff @(posedge clk)
  if (outstanding[id] && (age_q[id] >= MEM_TIMEOUT))
    declare_memory_failed(id);      // ← fires on a healthy, busy memory

What the bound must actually accommodate:

Three consequences.

A correct, healthy memory is declared failed. All six contributions are legitimate; a bound measured with none of them present will be exceeded when several coincide.

And the response to the false fault is worse than the fault. Declaring a memory failed may trigger recovery, re-issue, or escalation (14.5) — against a device that was about to answer, and a re-issue may duplicate a non-idempotent operation.

The compounding case is the dangerous one. Any single contribution alone fits comfortably. A same-bank access, during a maintenance window, on a link that recently degraded, is all three at once — and that combination is rare enough never to appear in a directed test and common enough to appear in the field.

The fix is that the bound is derived, not measured once:

// ILLUSTRATIVE. The bound accounts for the mechanisms that legitimately extend
// service time, and it scales with the ACTIVE link configuration.
assign mem_timeout_cycles =
      base_service_bound(active_width_q, active_rate_q)   // 16.5 Section 27
    + max_maintenance_stall                                // Section 11
    + max_bank_recovery                                    // Section 6
    + max_retry_allowance;                                 // 14.3

22. Observability the Endpoint Should Provide

§11 established that maintenance is invisible from outside. It does not have to be.

Three properties of this list.

Every row answers a question that is otherwise attributed to the wrong component, which is exactly 14.5's argument for observability as a verification category alongside safety and liveness.

The counters are cheap and the alternatives are not. Each is a saturating counter. The alternative to the third row is data loss discovered in the field.

And the last row is a trend, not a threshold. A rising corrected-error rate is the earliest available signal that a device is degrading, and it is only useful if the count is retained across the events that reset everything else (14.5's diagnostics-survive rule).

23. Correction and Repair Live in More Than One Place

Errors can be detected and corrected at several points in the path, and each placement catches a different set.

Three consequences.

They are not alternatives and they are not redundant. Link protection cannot catch a bit that was already wrong when it left the array; array protection cannot catch a bit corrupted in transit. A design that has one and assumes coverage of the other has a gap it cannot see.

Counting must not conflate them. One physical error corrected in two places is one error, and a system that counts it twice will misjudge a device's health. This is 14.1's alignment argument as an accounting rule: attribute each event to one detection point.

And repair is separate from correction. Correction fixes a value in flight; repair changes which physical resource is used, permanently, and is configuration state that must survive power cycles and must be consistent with whatever mapping the controller believes.

// ILLUSTRATIVE repair state. Configuration, not a runtime decision.
typedef struct packed {
  logic                    valid;
  logic [RESOURCE_W-1:0]   failed_resource;
  logic [RESOURCE_W-1:0]   spare_resource;
} repair_entry_t;
 
repair_entry_t repair_q [NUM_SPARES];

Architecture. A small substitution table applied to resource selection. Small deliberately — spares are a finite, manufactured quantity, and exhausting them is a device-level end of life.

Contract. The repair map and the controller's model of the array must agree. A controller addressing a resource the repair map has retired, or vice versa, accesses the wrong physical storage — with no error reported by either, because each is behaving correctly according to its own state.

Failure. Applying repair on one access path and not another. Or allowing the map to change while accesses are in flight, which is 14.4's requested-versus-active configuration problem in a place where the consequence is wrong data rather than a retrain.

DV. Assert that no access targets a resource marked failed, and that spare exhaustion is reported rather than silently ignored.

24. State Lifetimes at the Memory Chiplet

Two readings.

The third row is the one people miss. A link recovery takes time, and the maintenance obligation accumulated during it. A design that resumes and immediately serves a backlog of requests without first discharging the accumulated deficit has quietly extended the maintenance interval across the whole recovery — which is §13's failure, triggered by a link event.

And the fourth row is why controller placement matters (§8). Bank and timing state on the memory die never crossed the link, so a link event cannot invalidate it. The same state on the far side of the link is exposed to every transport event there is.

25. Flagship Trace — One Read, With Maintenance in the Way

Illustrative. A read arrives while its bank is recovering and a maintenance window is pending.

Five readings.

Cycle 6 is the section's point. The bank became free, and the request still could not issue — because the array was performing maintenance. Two independent conditions, both required, and a requester upstream can distinguish neither.

Fourteen cycles from acceptance to response, of which four were maintenance. A timeout calibrated on an idle array would not have included them (§21).

Cycle 3: the obligation appeared while the request was already queued. Maintenance is not scheduled around traffic; time advances it regardless (§12).

Cycle 15 returns identity 0x2A — the requester's, not the local slot. §19's bug returns the slot index here.

And cycle 16 frees the entry when the response is sent. Freeing it at cycle 14, when the data was ready, would have lost the binding while the response was still in the path.

26. Failure Trace — Maintenance Starved Under Load

Four readings, and this is the most dangerous trace in the chapter.

The system runs correctly for a long time. Nothing is wrong for the first hundred thousand cycles under load, and a test that stops there passes.

There is no hang and no error signal. Every request is served, on time, with clean CRC. The link is healthy, the scheduler is efficient, and the throughput numbers are excellent.

The failure is wrong data, appearing gradually, in the regions whose maintenance was skipped longest — so it looks like a random, region-correlated corruption with no obvious cause.

And the only pre-failure signal is deficit_q (§22). A design that does not expose it has no warning at all, and a design that exposes it has a metric that goes red hundreds of thousands of cycles before the first wrong bit.

27. Coverage

covergroup cg_memory_chiplet @(posedge clk);
  option.per_instance = 1;
 
  // --- Access-pattern dimension (Sections 6, 7).
  cp_bank_locality : coverpoint consecutive_same_bank {
    bins spread    = {0};
    bins same_once = {1};
    bins same_run  = {[2:$]};        // serialised by busy_until
  }
  cp_bank_conflict_stall : coverpoint stalled_on_bank_recovery;
 
  // --- Concurrency (Sections 15, 17).
  cp_outstanding : coverpoint outstanding_count {
    bins one  = {1};                 // Section 19's bug hides here
    bins few  = {[2:MAX_OUTSTANDING/2]};
    bins deep = {[MAX_OUTSTANDING/2+1:MAX_OUTSTANDING]};
  }
  cp_reorder_distance : coverpoint response_reorder_distance {
    bins in_order = {0};
    bins near     = {[1:3]};
    bins far      = {[4:$]};         // Section 18's identity contract
  }
 
  // --- Maintenance (Sections 11-14).
  cp_deficit : coverpoint deficit_q {
    bins none    = {0};
    bins some    = {[1:DEFICIT_LIMIT-1]};
    bins urgent  = {[DEFICIT_LIMIT:DEF_MAX-1]};
    bins saturated = {DEF_MAX};      // MUST stay at zero — Section 14
  }
  cp_maint_vs_request : coverpoint maintenance_preempted_request;
  cp_request_during_maint : coverpoint request_arrived_during_maintenance;
 
  // --- Correction and repair (Section 23).
  cp_error_site : coverpoint error_detected_at {
    bins none   = {0};
    bins array  = {1};
    bins link   = {2};
    bins both   = {3};               // must be counted ONCE
  }
  cp_repair : coverpoint repair_state {
    bins none = {0}; bins some = {[1:NUM_SPARES-1]}; bins exhausted = {NUM_SPARES};
  }
 
  // --- Composition with the link (Sections 21, 24).
  cp_link_event : coverpoint link_event_during_outstanding {
    bins none = {0}; bins retry = {1}; bins recovery = {2}; bins degraded = {3};
  }
 
  // --- Crosses that carry the information.
  x_maint_load    : cross cp_deficit, cp_outstanding;          // Section 26
  x_bank_conc     : cross cp_bank_locality, cp_outstanding;    // Section 16
  x_link_maint    : cross cp_link_event, cp_deficit;           // Section 24
  x_reorder_conc  : cross cp_reorder_distance, cp_outstanding; // Section 19
endcovergroup

Six bins worth calling out:

cp_deficit.saturated must stay at zero. Like 16.5 §35's retire-event bins, this one exists as a detector. Hitting it is §13.

cp_outstanding.one versus .deep. §19's identity bug is invisible at one outstanding request and obvious at many. A regression that only ever has one request in flight has verified nothing about identity.

cp_reorder_distance.far. A response returning many positions from its issue order — the case §18's contract exists for, and one that a shallow queue never produces.

cp_request_during_maint. §25's cycle 6. Without it, the interaction between the two independent issue conditions is never exercised.

cp_error_site.both. One physical error visible at two detection points, to prove it is counted once (§23).

And x_link_maint. A link recovery while the maintenance deficit is non-zero — §24's third row, and the case where a link event silently extends a maintenance interval.

28. Debug Taxonomy

Row 7 is the one to memorise. Gradual read errors after long sustained load, with a healthy link and excellent throughput is the signature of starved maintenance, and it is the only failure in this chapter that destroys data rather than delaying or misrouting it.

29. Debug Checklist

1. How many requests are outstanding? (§15, §22)
2. How are addresses distributed across banks or access paths? (§7)
3. How many accesses stalled on bank recovery? (§6)
4. What is the current and maximum maintenance deficit? (§12, §22)
5. How many cycles was the array unavailable for maintenance? (§11)
6. Was maintenance ever preempted by a request while urgent? (§14)
7. Is the returned identity the one the requester issued? (§19)
8. Was any identity outstanding twice at once? (§18)
9. What is the maximum observed request age? (§18)
10. When is a request's entry freed — on data-ready or on response-sent? (§17)
11. What does a write acknowledgement mean here — accepted, or durable? (§10)
12. What is the read timeout composed of, and does it include maintenance and bank recovery? (§21)
13. Does it scale with the active link width and rate? (16.5 §27)
14. Did a link retry, recovery or degradation occur while requests were outstanding? (§24)
15. Was the maintenance deficit discharged after the recovery, before serving the backlog? (§24)
16. Where was each error detected, and is it counted once? (§23)
17. Does the repair map agree with the addressing the controller is using? (§23)
18. Are any spares exhausted? (§23)

30. Common Misconceptions

"A memory chiplet is a passive target." It is a stateful endpoint whose ability to serve depends on what it did recently, which performs mandatory work nobody requested, and which deliberately reorders what it is given (§4).

"Refresh is overhead you can schedule away." It is a maintenance obligation with a deadline. It can be deferred within a budget and must then be performed. A design that defers it whenever traffic exists will never perform it under sustained load, and will lose data (§11, §13).

"A slow response means something is broken." It may mean a bank is recovering, maintenance is in progress, the link is retrying, or the link recovered narrower. All four are healthy, and a timeout that excludes them manufactures faults (§21).

"Peak bandwidth is a property of the memory." It is reachable only with enough concurrent requests, spread across enough independent access paths, with maintenance overhead deducted. A single-outstanding-request workload cannot approach it regardless of request size (§15, §16).

"HBM is fast because it is wide." Its bandwidth comes from many independent access paths, which is a different thing with different consequences: concurrency becomes a requirement rather than an optimisation, and the access pattern matters far more (§15).

"The memory returns responses in order." A scheduler reorders deliberately, because that is how it exploits bank parallelism. Responses must therefore carry identity, and the identity must be the requester's (§17, §19).

"The endpoint can use its own local index as the response identity." It cannot. The requester's table is keyed by what it issued. The bug is invisible at one outstanding request and appears exactly when concurrency rises — which is when performance requires it (§19).

"ECC somewhere means ECC everywhere." Array protection cannot catch a bit corrupted in transit; link protection cannot catch a bit that was already wrong when it left the array. They are complementary, not redundant, and one physical error visible at both must be counted once (§23).

"A UCIe recovery does not affect the memory." It does not affect the array contents, the repair map, the outstanding obligations or the local timing state. It does let the maintenance deficit accumulate throughout, and resuming into a backlog without discharging it extends the maintenance interval across the whole recovery (§24).

"Controller placement is a floorplanning decision." It decides which state crosses the link and therefore what the link must guarantee — timing fidelity, or delivery and identity. Those are entirely different links (§8).

31. Understanding Check

32. Summary and What Comes Next

A memory chiplet is a stateful endpoint, not a passive target. Its content is the product, it performs mandatory work nobody requested, and its ability to serve depends on what it did recently.

Service depends on history. A bank consumed by a recent access cannot serve another regardless of link state or queue space — which makes address mapping a first-order performance decision and makes bank parallelism the whole performance story.

Maintenance is an obligation with a deadline, not an error. A deficit counter lets it be deferred within a budget and forces compliance beyond it; a periodic pulse loses every opportunity missed under load; and a gate that defers maintenance whenever traffic exists loses data under sustained load, silently, after passing every functional test.

Bandwidth comes from parallelism, so concurrency is a requirement. A single-outstanding-request workload cannot approach peak regardless of request size, and a concentrated request stream measures a fraction of it while every component works perfectly.

The scheduler reorders deliberately, so identity must survive. The requester's identity is carried through untouched — a locally-reassigned index works at one outstanding request and fails exactly when concurrency rises.

Timeouts must be composed, not measured. Link traversal, queueing, bank recovery, maintenance, retry and a degraded link are all legitimate contributions, and a bound that excludes them declares a healthy memory failed — then triggers a response worse than the fault.

Correction lives in more than one place and each placement catches a different set — and one physical error visible at two of them is one error.

And controller placement moves state, which decides what the link must guarantee: timing fidelity, or delivery and identity. Those are entirely different links, and the second is the one a die-to-die boundary carries well.

This chapter treated the memory chiplet as a device. The next asks what changes when the memory a host wants is not merely on the package but reached through the CXL-over-UCIe expansion path — capacity that the host discovers, maps and grows into.

• 17.2 — Memory Expansion — the CXL-over-UCIe memory-expansion path.

Browse the full path on the UCIe tutorials index.