UCIe · Module 12
Response Flow
How a response finds the one request that created its obligation, survives independent backpressure and transport retry, and retires the transaction exactly once — the outstanding table, one-hot identity matching, out-of-order returns, why retiring on arrival loses data, the stale-response hazard after a timeout, generation quarantine, and request/response conservation.
Chapter 12.1 ended on a row worth repeating. Transport was entirely finished — the replay entry retired, the bits long gone, the remote consumer in possession of the request — and the transaction was still open, because a response had not returned.
That row is this chapter. And the return path is not the forward path reversed: it has its own queues, its own backpressure, its own ordering freedom, and one problem the forward path does not have at all. A request creates its own destination by existing. A response has to go and find one.
Everything hard here follows from that asymmetry. A response arriving with no matching request is not an error the transport can detect. A response matched against the wrong request is not an error anything can detect. And a response arriving after its request gave up is a hazard that produces intermittent corruption weeks into a bring-up.
1. The One-Sentence Model
A response closes an obligation created earlier, so response processing cannot be understood without the retained request state that defines the obligation.
That is why this chapter spends its first third on a request-side structure. The outstanding table is not response-path plumbing — it is the response path's entire reason for being able to work.
2. Sourcing, and What Is Symbolic
3. Why the Return Path Is Not the Forward Path Reversed
Five differences, and each generates sections.
A response must find its destination; a request creates one. A request arrives at a consumer that is prepared to accept any request. A response arrives at a matcher that must locate one specific entry, and if it cannot, there is nothing sensible to do with it. §5 through §10.
Ordering freedom differs. A request path that preserves acceptance order is being conservative and correct. A response path that assumes return order is making a claim about the protocol above it, and if that claim is wrong it delivers one requester's data to another. §15.
Backpressure is independent in the two directions. The forward path's readiness says nothing about the return path's, and coupling them can deadlock a full-duplex link. §11.
Retirement is a choice with a wrong answer. A request has no retirement point on the initiator side — it is handed on. A response retires an obligation, and choosing the earliest plausible moment loses data. §17.
And the return path has a hazard with no forward-path analogue. A response can arrive after its request has given up, into a system that may have reused its identity. §25 through §27, and it is the chapter's signature failure.
4. The Return Path and Its Buffers
The dashed conceptual point that the figure makes structurally: the outstanding table is connected to the matcher by a lookup, not by the datapath. The response does not pass through the table. So a design can have a perfectly healthy response datapath and a table that has forgotten the obligation, and the two facts are independent — which is §8.
5. The Outstanding-Request Table
// ILLUSTRATIVE outstanding-request state. Builds on Chapter 5.1 §5, which
// introduced tag allocation and the two basic properties; this adds the fields
// the response path actually needs. NOT a UCIe or CXL structure.
typedef struct packed {
logic valid;
logic [ID_W-1:0] id; // the identity the far side will reflect
logic [GEN_W-1:0] gen; // §26 — which attempt this entry is
logic [META_W-1:0] meta; // what was requested: address, direction, extent
logic [AGE_W-1:0] age; // §32 — ticks since the obligation was created
logic rsp_seen; // §17 — a response arrived and is not yet consumed
logic [MON_ID_W-1:0] mon_id; // VERIFICATION ONLY
} outstanding_t;
outstanding_t outstanding_q [MAX_OUTSTANDING];Architecture. This is the structure that makes a response mean something. Without it, returning data is a value with no destination — and note that the far side is under no obligation to tell you what it is responding to beyond echoing an identity, so meta exists so that the local side can check the response against what was asked rather than merely routing it.
State. MAX_OUTSTANDING entries with per-request lifetime: allocated before transmission, freed exactly once at a retirement point chosen deliberately (§17).
Cycle behaviour. Allocation gates request acceptance (Chapter 12.1 §10). Lookup on every arriving response. Retirement on one defined event.
Contract. The requester relies on receiving data for its own request and no other. The far side relies on the identity being meaningful for as long as the transaction is open.
Failure. Every failure in this chapter is a failure of this table: cleared too early (§8), holding two live entries with one identity (§9), matched by position instead of identity (§16), retired before the consumer took the data (§18), or reused too soon (§27).
DV. Every occupancy from empty to full; allocation and retirement in the same cycle (§20); and — the one that matters most — a response arriving for an entry that does not exist, which must be reported rather than absorbed.
6. Matching, and What "Match" Must Mean
// ILLUSTRATIVE response matching. The identity is the protocol's reflected
// value (§5); the lookup structure is this chapter's own.
typedef struct packed {
logic [ID_W-1:0] id; // reflected identity
logic [GEN_W-1:0] gen; // §26 — reflected generation, if the design carries one
logic [DATA_W-1:0] data;
logic [STS_W-1:0] status; // symbolic: no encoding is asserted
logic [MON_ID_W-1:0] mon_id; // VERIFICATION ONLY
} response_t;
logic [MAX_OUTSTANDING-1:0] match_vec;
always_comb begin
for (int e = 0; e < MAX_OUTSTANDING; e++)
match_vec[e] = outstanding_q[e].valid
&& (outstanding_q[e].id == rsp_in.id)
&& (outstanding_q[e].gen == rsp_in.gen); // §26
end
wire rsp_hit = (match_vec != '0);
wire rsp_orphan = rsp_valid && !rsp_hit;
wire [OUT_IDX_W-1:0] rsp_idx = onehot_index(match_vec);Architecture. A match is four conjuncts, not one. The entry must be valid — matching a stale entry is worse than not matching. The identity must be equal. The generation must be equal (§26). And in a design carrying more than one class of transaction, the response's class must be the one the entry expects — a data response arriving for an entry that expects a completion without data is a malformed pairing that will otherwise be accepted and mis-consumed.
State. None of its own; a combinational lookup over §5's table.
Cycle behaviour. One lookup per arriving response. In a large table this is a wide comparison and real designs use a directly-indexed table where the identity is the index — which is cheaper and shifts the problem to §9, since a direct index makes duplicate identities structurally impossible but makes generation checking essential.
Contract. Exactly one entry, or a reported orphan. There is no third legal outcome, and §7 is that made checkable.
Failure. Zero matches absorbed silently loses the response and leaves the requester waiting forever. Multiple matches means the table holds two live entries with one identity — §9 — and whichever is picked, one of them is wrong.
DV. Match at the first entry, the last, and the middle; a response with an identity never issued; a response whose identity is valid but whose generation is stale; and a response arriving in the same cycle a new entry is allocated with the same identity, which is the race §9's allocator must prevent.
7. SVA — Exactly One Match, or a Reported Orphan
// Illustrative. The cheapest high-value property on the response path.
property p_response_matches_exactly_one;
@(posedge clk) disable iff (!rst_n)
(rsp_valid && rsp_hit) |-> $onehot(match_vec);
endproperty
a_response_matches_exactly_one: assert property (p_response_matches_exactly_one);
// A response matching nothing is an EVENT, not noise. Whether it is an
// assertion or a coverage-plus-reporting obligation depends on whether the
// architecture permits late responses at all (§25) — choose deliberately.
property p_orphan_response_is_reported;
@(posedge clk) disable iff (!rst_n)
rsp_orphan |-> orphan_reported;
endproperty
a_orphan_response_is_reported: assert property (p_orphan_response_is_reported);
// The matched entry must actually describe the request this response answers.
// Catches an index/identity mismatch that routes the right data to the wrong
// entry — which the two properties above cannot see.
property p_match_agrees_with_request;
@(posedge clk) disable iff (!rst_n)
rsp_retire_fire |-> (outstanding_q[rsp_idx].meta == rsp_expected_meta);
endpropertyWhy $onehot here and $onehot0 in Chapter 11.2 §9. The address-window case had a legal zero-match outcome — an address in no window is unmapped, which is a real situation. Here zero matches is not a legal outcome of a matched response, so the property is written over rsp_hit and uses $onehot, while the zero case gets its own property demanding it be reported. Two properties instead of one weaker one, because the two failures are different: $onehot catches a corrupted table, and the orphan property catches a lost obligation.
On making the orphan property an assertion versus a cover. If the architecture states that late responses cannot occur, it is an assertion. If the architecture permits them after an abandonment (§25), it must be a cover point plus a reporting requirement, because otherwise it fires legitimately and gets waived — and a waived property protects nothing. Encode the contract you actually have.
8. Wrong RTL — Clear the Outstanding Entry on Transmission
// WRONG — the entry freed when the request leaves.
always_ff @(posedge clk) begin
if (request_transmitted) outstanding_q[tx_idx].valid <= 1'b0;
endArchitecture. The plane that knows the bytes went is being allowed to close an obligation that only the far side can discharge. Chapter 11.5 §14 argued this at the integration level; here it is the specific, most common instance.
Cycle behaviour. One pulse and the entry is free — and, crucially, available for reuse by the next request.
Failure, in three escalating stages.
The response matches nothing. For a read, the data arrives with no entry to route it to. Dropped, the requester waits forever.
Or it matches the wrong entry. If the identity has been reused by a later request, the response matches that entry and one requester receives another's data. This is the silent-misdelivery family, and it is worse than the first outcome because there is no stall to notice — everything completes, with wrong values.
And the transport retry case makes it non-deterministic. Chapter 12.1 §17 established that a request may be transmitted more than once under replay. So request_transmitted may pulse twice for one request: the first pulse frees the entry, and the second frees whatever now occupies that slot. A transport retry has corrupted an unrelated transaction.
Why it survives review. The condition reads as a natural completion, and in a testbench where responses return in a few cycles and identities are never recycled, it works perfectly.
// Illustrative — the obligation survives every transport milestone.
property p_outstanding_survives_transmission;
@(posedge clk) disable iff (!rst_n)
(request_transmitted && outstanding_q[tx_idx].valid
&& !rsp_retire_fire) |=> outstanding_q[tx_idx].valid;
endproperty
a_outstanding_survives_transmission:
assert property (p_outstanding_survives_transmission);
// And the retry variant specifically.
property p_retry_does_not_free_outstanding;
@(posedge clk) disable iff (!rst_n)
transport_retry_event |=> $stable(outstanding_valid_vec);
endproperty9. Identity Allocation, and Not Reusing a Live One
// ILLUSTRATIVE identity allocator. Chapter 5.1 §5 showed the in-use bitmap and
// the no-reuse property; this adds what the response path needs — the
// generation, and the quarantine of §26.
logic [MAX_ID-1:0] id_in_use_q;
logic [MAX_ID-1:0] id_quarantined_q; // §26 — retired but not yet safe to reuse
logic [GEN_W-1:0] id_gen_q [MAX_ID]; // increments each time an identity is reissued
wire [MAX_ID-1:0] id_allocatable = ~id_in_use_q & ~id_quarantined_q;
wire id_available = (id_allocatable != '0);
wire [ID_W-1:0] alloc_id = lowest_set_index(id_allocatable);
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
id_in_use_q <= '0;
id_quarantined_q <= '0;
for (int i = 0; i < MAX_ID; i++) id_gen_q[i] <= '0;
end else begin
if (alloc_fire) begin
id_in_use_q[alloc_id] <= 1'b1;
// The generation increments at ALLOCATION, so the value carried by a
// request is the value a legitimate response will reflect (§26).
id_gen_q[alloc_id] <= id_gen_q[alloc_id] + 1'b1;
end
if (retire_fire) id_in_use_q[retire_id] <= 1'b0;
// A NORMAL retirement makes the identity immediately reusable. An
// ABANDONED transaction does not — §26.
if (abandon_fire) id_quarantined_q[abandon_id] <= 1'b1;
if (quarantine_release_fire) id_quarantined_q[release_id] <= 1'b0;
end
endClassification: synthesizable, illustrative.
Architecture. Two bitmaps rather than one, because "not in use" and "safe to use" are different questions and conflating them is §27. A normally-retired identity is immediately safe. An abandoned one is not, because a response for it may still be in flight.
State. Two bits and a generation counter per identity, per-identity lifetime. The generation is the only field here whose value must survive retirement — it is what makes a stale response recognisable.
Cycle behaviour. Allocation consumes an identity and bumps its generation. Retirement releases it. Abandonment quarantines it. Note the generation increments on allocation, not retirement, so the request carries the value a legitimate response will reflect.
Contract. The response matcher relies on at most one live entry per identity, and on the generation distinguishing attempts.
Failure. Allocating a live identity makes returning responses ambiguous — Chapter 5.1 §5's failure, where a completion for A is matched against the entry now describing B. Allocating a quarantined identity is §27.
DV. Exhaust all identities; retire and immediately reallocate; abandon and attempt to reallocate (must be refused); and wrap the generation counter, verifying the wrap is handled rather than aliasing.
// Illustrative — no two live entries share an identity. The pairwise form for
// a small table; the affordable form for a large one is the onehot check of §7
// at every allocation.
property p_no_duplicate_live_id;
@(posedge clk) disable iff (!rst_n)
(outstanding_q[a].valid && outstanding_q[b].valid && (a != b))
|-> (outstanding_q[a].id != outstanding_q[b].id);
endproperty
a_no_duplicate_live_id: assert property (p_no_duplicate_live_id);
// Illustrative — never allocate an identity that is in use or quarantined.
property p_alloc_only_free_id;
@(posedge clk) disable iff (!rst_n)
alloc_fire |-> (!id_in_use_q[alloc_id] && !id_quarantined_q[alloc_id]);
endproperty
a_alloc_only_free_id: assert property (p_alloc_only_free_id);10. Match Before Buffer, or Buffer Before Match
A real architectural choice with a real trade, and designs pick it by accident more often than deliberately.
| Match before buffer | Buffer before match | |
|---|---|---|
| Structure | look up the identity as the response arrives, then queue the matched result | queue the raw response, look up later |
| Timing | lookup in the arrival path — a wide comparison at line rate | decoupled — the lookup runs at the queue's drain rate |
| What the queue stores | resolved entry index plus payload | the response's full identity plus payload |
| Orphan detection | immediate, at arrival | delayed until drain — the orphan sat in a queue meanwhile |
| Table pressure | one lookup port at arrival rate | one lookup port at drain rate |
| Risk | timing closure on a wide match | the table may change while the response waits |
The risk row is the one that decides it. With buffer-before-match, a response can sit in the queue while its entry is retired by something else — a timeout, an abandonment, a reset. When it is finally looked up, the entry is gone or has been reused, and this is §25's stale-response hazard created inside the local design rather than arriving from the link.
So buffer-before-match is not wrong, but it requires the generation check of §26 as a matter of correctness rather than as a defence against the far side. Match-before-buffer needs it too, for the link-side case — but buffer-before-match needs it for a purely local reason, and a design that chose buffer-before-match for timing without adding the generation check has introduced a hazard it did not have.
Whichever is chosen, write it down. The failure mode of not choosing is a design where the answer varies by module and the assumptions do not compose.
11. The Response Queue, and Independent Backpressure
// ILLUSTRATIVE response queue. Separate from the request path's storage, with
// its own occupancy and its own ready — see the wrong version in §12.
response_t rsp_q [RSP_DEPTH];
logic [RQ_W-1:0] rsp_occ_q;
wire rsp_push = rsp_in_valid && rsp_in_ready;
wire rsp_pop = rsp_out_valid && rsp_out_ready;
// The local Adapter may hand up a response only if there is room for it. This
// is the return direction's OWN condition and shares nothing with the forward
// direction's admission gate (Ch 12.1 §10).
assign rsp_in_ready = (rsp_occ_q != RSP_DEPTH);
assign rsp_out_valid = (rsp_occ_q != '0);
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
rsp_occ_q <= '0;
end else begin
unique case ({rsp_push, rsp_pop})
2'b10: rsp_occ_q <= rsp_occ_q + 1'b1;
2'b01: rsp_occ_q <= rsp_occ_q - 1'b1;
2'b11: rsp_occ_q <= rsp_occ_q; // one in, one out
2'b00: rsp_occ_q <= rsp_occ_q;
endcase
end
endArchitecture. The return path needs storage for the same reason the forward path does — Chapter 12.1 §7's argument applies unchanged — and it needs its own storage because the two directions stall independently.
State. RSP_DEPTH slots and a counter, per-response lifetime.
Cycle behaviour. The unique case is the same discipline as Chapter 12.1 §18 and it is here for a different consequence: §21.
Contract. The local Adapter relies on rsp_in_ready meaning the response will be retained. The consumer relies on responses being held until it takes them.
Failure. §12 for the coupling error, §21 for the counting error.
DV. Fill and drain; simultaneous push and pop at every occupancy; and hold the consumer stalled with the queue full while the forward direction continues to issue requests — which is the case that proves the two directions are independent.
12. Wrong RTL — Shared Readiness Across Directions
// WRONG — one readiness notion for both directions.
assign rsp_in_ready = fwd_path_ready; // the forward path's conditionArchitecture. It looks like an economy. The link is full-duplex; the design is not.
Failure, and it is a deadlock rather than a data error. Consider the state a busy system reaches naturally:
- The forward path is stalled because credits are exhausted — the far side's receive buffer is full.
- The far side's buffer is full because it is waiting for the local side to consume responses.
- The local side cannot accept responses because
rsp_in_readyis derived fromfwd_path_ready, which is low. - So responses are not consumed, the far side does not free buffer space, credits do not return, and the forward path stays stalled.
A closed cycle, built from two individually reasonable conditions. No message is corrupted, no CRC fails, no counter goes out of range, and every safety assertion passes — Chapter 5.5 §12's observation, arriving here through a one-line coupling.
A second, subtler version of the same mistake shares a valid bit rather than a ready: one xfer_valid used for both a request going down and a response coming up. Then a response arriving in the same cycle a request is presented either overwrites it or is lost, depending on which path samples first. Full-duplex means two independent handshakes, not one shared one.
// Illustrative — the return direction makes progress independently of the
// forward direction's readiness. Written as a progress property with the
// assumptions stated, because it is liveness (§29).
property p_response_progress_independent;
@(posedge clk) disable iff (!rst_n || error_injection_active)
(rsp_out_valid && rsp_consumer_ready)[*MAX_RSP_WAIT]
|-> rsp_pop[->1];
endproperty13. Response Metadata Must Stay Bundled
The forward path's misalignment failure (Chapter 12.1 §15) has a return-path twin, and the twin is worse.
// WRONG — the identity and the payload pipelined at different depths.
always_ff @(posedge clk) rsp_id_q <= rsp_in.id; // 1 stage
always_ff @(posedge clk) rsp_dat_q <= rsp_in.data;
always_ff @(posedge clk) rsp_dat_q2 <= rsp_dat_q; // 2 stages
// The matcher sees response N's identity with response N-1's data.Failure. Request A receives Request B's data. Both requests complete. Both report success. Both requesters believe they have their data, and one of them is holding a value from an address it never asked about.
Why this is worse than the forward-path version. The forward version produced a wrong write at a legitimate address — bad, and confined to that address. This one corrupts the requester's own view of memory, and the requester may be a core that proceeds to compute on the value. There is also no CRC-level protection to appeal to: §24.
And the symptom is maximally confusing, because it is symmetrical. If responses A and B are swapped, both look plausible, and a scoreboard checking only "did every request get a response" passes. The only check that sees it is one comparing each response's data against what that specific request asked for — which is why §5's table carries meta.
// Illustrative — one object, one enable.
response_t rsp_obj_q, rsp_obj_q2;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
rsp_obj_q <= '0;
rsp_obj_q2 <= '0;
end else if (rsp_pipe_en) begin
rsp_obj_q <= rsp_in;
rsp_obj_q2 <= rsp_obj_q;
end
end// Illustrative — the bundle holds still under stall, all fields together.
property p_response_stable_under_stall;
@(posedge clk) disable iff (!rst_n)
(rsp_out_valid && !rsp_out_ready)
|=> ($stable(rsp_obj_q) && rsp_out_valid);
endproperty
a_response_stable_under_stall: assert property (p_response_stable_under_stall);
// Illustrative — the identity and the data delivered together came from the
// same source response. Checkable only with the verification-only tag.
property p_response_id_matches_data_source;
@(posedge clk) disable iff (!rst_n)
rsp_deliver |-> (delivered_id_mon == delivered_data_mon);
endproperty
a_response_id_matches_data_source:
assert property (p_response_id_matches_data_source);14. Responses Need Not Return In Order
The assumption that produces §16, stated carefully because the correct statement is narrower than either extreme.
What is not assertable here. Whether this protocol permits out-of-order responses is the carried protocol's business, and it varies: some protocols require in-order completion, some permit reordering freely, and some permit it between specified classes only. This chapter does not state which, for any protocol, because that requires specification text it has not inspected for every case.
What is verified and directly relevant. The CXL 1.0 specification, in the same restrictions Chapter 11.3 §15 quotes, states that multiple reads to the same cache line are allowed and that the Host can freely reorder requests, so the device is responsible for ordering requests when required. Chapter 11.3 §24 quotes the more general form: the host will not preserve the order of CXL.cache requests as delivered by the device.
So for at least one protocol carried over UCIe, reordering is explicit and the burden is on the agent. Which settles the design question:
The matching table exists precisely because issue order is not sufficient to identify a response. If issue order were always sufficient, an identity would be unnecessary — and the fact that real protocols define and reflect one is the strongest available evidence that it is not.
Two design consequences.
Never derive the destination from position. §16.
And out-of-order completion interacts with the consumer's readiness. If responses may complete out of order but the consumer accepts them in order, the response queue becomes a reordering buffer — which is a design decision with a depth requirement, not a free property. §31.
15. Out-of-Order in Practice
Requests issued A, B, C. Responses return B, A, C.
With identity matching, each response finds its own entry and each requester gets its own data. The return order is irrelevant to correctness, and the only question is whether the consumer requires ordering — which is a separate concern with a separate mechanism.
With a FIFO assumption, the first response to arrive is paired with the oldest outstanding request:
| Response arrives | FIFO assumption pairs it with | Correct pairing | Result |
|---|---|---|---|
| B | A (oldest) | B | A receives B's data |
| A | B | A | B receives A's data |
| C | C | C | correct — by coincidence |
Two transactions silently exchange data, and the third is right by luck. Every request completed. Every response was consumed. Counts balance. The link is clean. And two requesters are computing on values from addresses they never asked about.
The detail that makes this survive testing: with a single outstanding request at a time, the FIFO assumption is always correct. So a design carrying this bug works perfectly at low load and fails only when concurrency rises — which is the same profile as a performance problem, and it is where people look first.
16. Wrong RTL — Use the Oldest Outstanding Request
// WRONG unless the protocol guarantees in-order completion — and even then,
// it encodes an assumption that nothing in this module records.
assign rsp_idx = oldest_outstanding_idx;Architecture. It replaces a lookup with a pointer, which is genuinely cheaper: no wide comparison, no identity storage, no allocator.
Cycle behaviour. Correct whenever exactly one request is outstanding, and correct whenever responses happen to return in order.
Failure. §15's table. And note the second-order damage: because the response is consumed and the entry is retired, the outstanding table stays perfectly consistent. The bug produces no bookkeeping anomaly at all, so conservation checks pass, occupancy is right, and the only evidence is the data itself.
When it is legitimate, because it sometimes is: a protocol that guarantees in-order completion permits this, and in that case it is a reasonable optimisation. But it must be justified in a comment against the protocol's guarantee, and it must be asserted, because the guarantee is not visible in this module:
// Illustrative — if the design assumes in-order completion, ASSERT it rather
// than relying on it. This fires the first time the far side reorders, which
// is exactly the event the optimisation is betting will never happen.
property p_responses_return_in_order;
@(posedge clk) disable iff (!rst_n)
rsp_valid |-> (rsp_in.id == outstanding_q[oldest_outstanding_idx].id);
endproperty
a_responses_return_in_order: assert property (p_responses_return_in_order);That property is the whole lesson of the section. An optimisation that depends on an external guarantee is fine; an optimisation that depends on an unstated external guarantee is a latent bug with a performance-shaped symptom.
17. The Retirement Point Is a Choice
Four candidate moments to free the outstanding entry, and only one is right.
| Candidate | What it means | Verdict |
|---|---|---|
| request transmitted | the bytes left | wrong — §8 |
| response arrives at the local Adapter | the bytes came back | wrong — §18 |
| response matched | the obligation is identified | wrong — identified is not delivered |
| response accepted by the consumer | the requester has the data | correct |
Why the fourth and not the third. Matching identifies the entry. It does not put the data anywhere the requester can see it. Between matching and consumption the response lives in a queue, and the entry is the only record of what that queued response is for. Free it at matching and a consumer stall leaves a response in a queue with no context — §18.
Why "accepted by the consumer" and not "consumed and acted upon". Because the layers above have their own retirement rules, and this layer's obligation ends when it has handed the data to the party that asked. Chapter 5.1 §3's lifetime argument: the semantic transaction above may remain open longer, and that is not this table's business.
Whatever is chosen, say so explicitly in the design and in the scoreboard. Chapter 11.2 §30 made this point for the memory model and it applies with more force here: a scoreboard whose retirement point differs from the design's produces a stream of failures whose only cause is the disagreement, those get waived, and the real ones get waived alongside them.
18. Wrong RTL — Retire on Response Arrival
// WRONG — the obligation closed when the response reached the chip.
always_ff @(posedge clk) begin
if (rsp_valid && rsp_hit) outstanding_q[rsp_idx].valid <= 1'b0;
endArchitecture. Arrival treated as delivery. They differ by however long the consumer stalls.
Cycle behaviour. The entry frees on arrival. The response goes into the queue. The consumer is not ready.
Failure. The response is now in a queue with no context. Concretely:
The identity has been released, so it may be reallocated to a new request while the old response is still queued. When the queue drains, the response is delivered against an entry describing a different transaction — §27's hazard, generated locally.
Or the queue overflows and the response is dropped. The entry is already gone, so nothing records that anything was lost. The requester waits forever, and the conservation equation of §29 shows a request that was neither completed nor failed — which is the only evidence, and it appears at end of test rather than at the moment of loss.
Or a reset clears the queue. The response is gone, the entry is gone, and there is no record that an obligation existed.
Why it survives review. rsp_valid && rsp_hit reads exactly like completion, and with a consumer that is always ready — which is what a simple testbench provides — arrival and acceptance are the same cycle and the bug is invisible.
// Illustrative — the entry is retained while a matched response waits.
// The `rsp_seen` field of §5 is what makes this expressible.
property p_outstanding_retained_while_response_stalled;
@(posedge clk) disable iff (!rst_n)
(outstanding_q[e].rsp_seen && !rsp_consumer_accept[e])
|=> outstanding_q[e].valid;
endproperty
a_outstanding_retained_while_response_stalled:
assert property (p_outstanding_retained_while_response_stalled);
// Illustrative — retirement happens at the CHOSEN point and no other.
property p_retire_only_on_consumer_accept;
@(posedge clk) disable iff (!rst_n)
$fell(outstanding_q[e].valid) |-> $past(rsp_consumer_accept[e]);
endproperty
a_retire_only_on_consumer_accept:
assert property (p_retire_only_on_consumer_accept);p_retire_only_on_consumer_accept is the strongest property in the chapter, because it is written from the effect rather than the cause: any path that clears a valid bit for any reason other than consumer acceptance fires it. That catches §8, §18, a stray reset, and a debug backdoor, without needing to enumerate them.
19. Simultaneous Allocate and Retire
The outstanding table's counterpart to Chapter 12.1 §18, and the context makes the consequence different.
// ILLUSTRATIVE outstanding-count maintenance. The explicit case is the point.
logic [OUT_W-1:0] out_count_q;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
out_count_q <= '0;
end else begin
unique case ({alloc_fire, retire_fire})
2'b10: out_count_q <= out_count_q + 1'b1;
2'b01: out_count_q <= out_count_q - 1'b1;
2'b11: out_count_q <= out_count_q; // A retires as D allocates
2'b00: out_count_q <= out_count_q;
endcase
end
end
// Asserted, never saturated (Ch 9.5 §8).
assign out_space = (out_count_q != MAX_OUTSTANDING);Architecture. One writer, four enumerated outcomes.
Cycle behaviour. The 2'b11 row is common rather than rare here: a system running at its outstanding limit retires and allocates in the same cycle continuously, because retirement is what creates the space allocation consumes. So this row is the steady state, not a corner case — which is the difference from the forward path, where simultaneous push/pop happens only when the queue is transiently non-empty.
Contract. out_space gates request acceptance (Chapter 12.1 §10). A drifting count therefore corrupts the forward path.
Failure. §20.
DV. Run at the outstanding limit for an extended period, which exercises 2'b11 on nearly every cycle and makes any drift accumulate fast.
20. Wrong RTL — Two Independent Count Updates, Here
// WRONG — and the consequence is specific to this context.
always_ff @(posedge clk) begin
if (alloc_fire) out_count_q <= out_count_q + 1'b1;
if (retire_fire) out_count_q <= out_count_q - 1'b1;
endArchitecture. Same shape as Chapter 12.1 §19. Different consequence, and worth working through rather than assuming it transfers.
Cycle behaviour. On a simultaneous allocate and retire, the decrement wins and the increment is lost, so the count drifts downward.
Failure — and here downward drift is the dangerous direction, which is the opposite of the forward path. In Chapter 12.1 §19 downward drift caused a queue to accept a request it could not hold. Here the count does not protect storage; it protects the identity space. A count that reads lower than reality means out_space is asserted when the table is actually full — so a new request is allocated into a table with no free entry.
What happens next depends on the allocator. If the allocator independently checks id_in_use_q (§9), the allocation fails at the identity level and the design stalls confusingly. If it trusts the count, it allocates a live entry — and now two transactions share one identity, which is §9's failure and produces a returning response matched against the wrong entry.
And because the steady state at the outstanding limit hits 2'b11 on almost every cycle, the drift here accumulates far faster than on the forward path. A design running near its limit can drift by hundreds within a short burst.
The cheapest detector, and it is worth writing even with no reference model:
// Illustrative — the count must equal the number of valid entries. This is a
// redundant computation on purpose: the two must never disagree.
property p_count_matches_valid_entries;
@(posedge clk) disable iff (!rst_n)
out_count_q == $countones(outstanding_valid_vec);
endproperty
a_count_matches_valid_entries: assert property (p_count_matches_valid_entries);$countones over the valid bits is the whole check. It catches every drift in the cycle it happens, it needs no testbench model, and it costs one property.
21. Response Transport Replay
The return direction has its own replay, and it needs its own duplicate suppression.
The transport that carried the request also carries the response — with the same CRC, the same Ack/Nak sequence numbering, and the same replay-on-failure behaviour (Chapter 12.1 §9). So the response can be retransmitted, and, as Chapter 9.4 §12 established, retransmission also happens when a confirmation is lost — in which case the original arrived intact and the local side sees the same response object twice, both with passing CRC.
// WRONG — semantic completion keyed on response arrival.
assign transaction_complete = rsp_valid && rsp_crc_ok;Failure. The transaction completes twice:
- The consumer receives the response twice. For a read, the requester may accept the data twice, which is harmless for a pure register read and not harmless for a FIFO-style consumer.
- The entry retires twice. The second retirement frees whatever now occupies the slot — and if the identity was reallocated, it terminates a live, unrelated transaction, whose requester then waits forever.
- The count decrements twice, so §20's drift appears without any counting bug at all.
And the fix belongs below the semantic boundary, in the mapping layer, exactly as Chapter 11.4 §17 argued for the forward direction. A response matcher that has to defend itself against duplicates has been handed a problem it cannot see the inputs for, because transport identity is invisible above the mapping layer.
// Illustrative — completion gated on transport resolution plus duplicate
// suppression, not on arrival.
assign transaction_complete = rsp_reconstructed
&& !rsp_is_duplicate
&& rsp_consumer_accept;// Illustrative — a replayed response produces at most one semantic completion.
// One of the two or three most important properties in Module 12.
property p_replay_does_not_duplicate_completion;
@(posedge clk) disable iff (!rst_n)
transaction_complete |-> !completed_mon[rsp_obj_q.mon_id];
endproperty
a_replay_does_not_duplicate_completion:
assert property (p_replay_does_not_duplicate_completion);
// Illustrative — a duplicate arriving must be SUPPRESSED, not merely ignored:
// it must be counted, so a regression can prove the mechanism was exercised.
property p_duplicate_is_counted;
@(posedge clk) disable iff (!rst_n)
(rsp_valid && rsp_is_duplicate) |=> (dup_count_q == $past(dup_count_q) + 1'b1);
endpropertyOn p_duplicate_is_counted. A duplicate-suppression mechanism that has never fired is indistinguishable from one that does not work. The counter is what turns "no duplicates were delivered" into evidence, and §34's coverage requires the count to be non-zero rather than the failures to be zero.
22. A Clean CRC Does Not Prove the Response Is Right
Worth its own short section, because the CRC's strength invites over-trust.
The UCIe material states that the Adapter's CRC covers a 128-byte payload with a triple-bit-flip detection guarantee in 16 bits, and that a failure causes replay. That is a strong, quantified guarantee, and it is a guarantee about exactly one thing: that the bytes that arrived are the bytes that were sent.
It says nothing about any of the following, all of which are mapping or integration failures rather than transmission failures:
| Failure | Why CRC cannot see it |
|---|---|
| Wrong identity reflected | the far side put a legal value in the field; the bytes crossed intact |
| Wrong data associated with a correct identity | §13 — the misalignment happened before the CRC was computed |
| Wrong status | a legal encoding, correctly transmitted |
| A duplicate delivery | both copies pass CRC; that is the whole problem (§21) |
| A stale response from a previous attempt | it was a valid response once (§25) |
| A response matched to the wrong entry | the corruption is local, after reception |
CRC proves transmission integrity. Every failure in this chapter is a failure of association, and association is not a property of the bytes.
That is also the reason §35's taxonomy separates "response never arrives" from "response arrives but is wrong" — the first is a transport question and the second never is.
23. Timeouts, and the Response That Arrives Afterwards
Chapter 12.1 §25 established that a timeout is an absence of information. Here is the consequence on the return path, and it is the chapter's signature hazard.
A request times out. The local side gives up — reports a failure upward, or escalates. And then the response arrives.
It was always going to. The request had been delivered and executed; the response was queued behind something, or the link was recovering, or the far side was simply slow. Nothing about the timeout stopped the far side from responding, because the timeout was a local decision that the far side never learned about.
Now the design faces a response for a transaction it has already failed. Three things must be true and none of them is automatic:
It must not be delivered as though the transaction succeeded. The requester has already been told the transaction failed. Delivering data afterwards means the requester may act on data for an operation it believes did not happen.
It must not be silently dropped either. A late response is the single most valuable diagnostic in the system — it proves the request was delivered, which converts an unexplained timeout into a latency problem. Dropping it discards that.
And it must not match a new transaction. Which is the part that requires mechanism, because if the identity was recycled, it will match — and match cleanly.
A timeout does not make late responses harmless. It makes them dangerous, because the local state they refer to no longer exists.
24. Generation Quarantine
// ILLUSTRATIVE stale-response defence. The GENERATION and the QUARANTINE are
// implementation techniques, NOT anything UCIe or CXL mandates. What is being
// solved is real; this is one solution among several.
//
// TECHNIQUE 1 — GENERATION. Each identity carries an attempt number, allocated
// with the request (§9) and reflected in the response. A response whose
// generation does not match the current entry is from a previous attempt.
wire rsp_generation_stale = rsp_hit_by_id && (rsp_in.gen != outstanding_q[rsp_idx].gen);
// TECHNIQUE 2 — QUARANTINE. An identity belonging to an ABANDONED transaction
// is not reallocated until it is safe. "Safe" is an architectural judgement:
// after a bounded interval, after a link epoch change, or after an explicit
// acknowledgement from the far side that it holds nothing for that identity.
logic [QUAR_W-1:0] quarantine_timer_q [MAX_ID];
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
for (int i = 0; i < MAX_ID; i++) quarantine_timer_q[i] <= '0;
end else begin
for (int i = 0; i < MAX_ID; i++) begin
if (abandon_fire && (abandon_id == i[ID_W-1:0]))
quarantine_timer_q[i] <= QUARANTINE_TICKS;
else if (quarantine_timer_q[i] != '0)
quarantine_timer_q[i] <= quarantine_timer_q[i] - 1'b1;
end
end
end
assign quarantine_release_fire = (quarantine_timer_q[release_id] == '0)
&& id_quarantined_q[release_id];Classification: synthesizable, illustrative — and a design may implement one technique, both, or a different one entirely.
Architecture. Two independent defences, and they cover different cases. The generation catches a stale response whose identity has already been reissued — it arrives, matches by identity, and is rejected by generation. The quarantine reduces how often that happens at all, by not reissuing the identity while a response might still be in flight. Belt and braces, and the belt is the generation: quarantine is a probabilistic improvement, generation is a check.
State. A generation per identity — which must survive retirement, since its whole purpose is to distinguish attempts — plus a quarantine timer per identity. Per-identity lifetime, and the generation is one of very few fields in this curriculum whose value is meaningless except in comparison with a previous value of itself.
Cycle behaviour. The generation increments at allocation. The quarantine timer is loaded on abandonment and counts down.
Contract. The far side must reflect the generation for this to work — which is a protocol-level requirement and therefore not something a design can add unilaterally. If the carried protocol has no field available, the design cannot use the generation technique and must rely on quarantine plus explicit reporting, which is weaker. Say which situation you are in.
Failure. With neither technique, §27. With quarantine alone, the window is narrowed and not closed — a response delayed longer than QUARANTINE_TICKS still aliases. With the generation alone, correctness holds but every abandoned transaction's identity may produce a rejected response that must be reported.
DV. Abandon a transaction, reallocate its identity as soon as the design permits, and inject the late response — verifying it is rejected and reported rather than delivered. This is a directed test; nothing random produces it.
// Illustrative — a stale response is never delivered.
property p_stale_response_not_delivered;
@(posedge clk) disable iff (!rst_n)
rsp_generation_stale |-> !transaction_complete;
endproperty
a_stale_response_not_delivered: assert property (p_stale_response_not_delivered);
// Illustrative — and it is reported, because it is diagnostic gold (§23).
property p_stale_response_reported;
@(posedge clk) disable iff (!rst_n)
rsp_generation_stale |-> stale_reported;
endproperty25. Wrong RTL — Immediate Identity Reuse
The flagship bug of this chapter. Trace it fully, because every step is individually reasonable.
// WRONG — a timed-out identity returned to the free pool immediately.
always_ff @(posedge clk) begin
if (request_timeout[t]) begin
outstanding_q[t].valid <= 1'b0;
id_in_use_q[outstanding_q[t].id] <= 1'b0; // "the transaction is over"
end
endThe sequence:
- Request A is issued with identity 5. It is delivered and executed remotely.
- A's response is delayed — queued behind a burst, or the link is recovering.
- A times out. The design reports the failure upward and frees identity 5.
- Request B is issued and allocated identity 5. It is a different request, to a different address, from a different requester.
- A's response arrives, carrying identity 5.
- It matches B's entry. Cleanly.
$onehotpasses. The valid bit is set. The identity is equal. - B's requester receives A's data, and B's entry retires as though B had completed.
- B's real response arrives later, matches nothing, and is dropped as an orphan — or, if identity 5 has been recycled again, matches a third transaction.
Every check in this chapter passes except the ones that exist for this. The match is one-hot. The counts balance. The CRC was clean both times. The conservation equation of §29 balances too, because the number of completions equals the number of requests — they are just paired wrongly.
Why the symptom is so hard to recognise. It requires a timeout to have occurred earlier, so it appears as intermittent data corruption that begins after a period of link stress and then continues. The corruption is not correlated with the corrupted transaction — it is correlated with a timeout that happened before it. Anyone looking at the failing transaction finds nothing wrong with it.
And it gets worse under load, because load causes the delays that cause the timeouts. So the failure rate rises with exactly the conditions that make debugging hardest.
The fix is §26 — and note that the fix has two halves and both are needed. Quarantine narrows the window. The generation check is what actually closes it, because there is no interval after which a response is guaranteed not to arrive.
26. Flagship Trace 1 — Response Held While the Consumer Stalls
Fifteen illustrative cycles. Latencies illustrative; the point is which state persists.
Request A was issued with identity 7 at some earlier cycle and is outstanding.
| Cyc | Remote | Transport | Local Adapter | Matcher | Rsp queue | Consumer | Entry 7 |
|---|---|---|---|---|---|---|---|
| 1 | A's response produced | — | — | — | — | — | valid |
| 2 | handed to Adapter | — | — | — | — | — | valid |
| 3 | — | in flight | — | — | — | — | valid |
| 5 | — | arrived | reconstructing | — | — | — | valid |
| 6 | — | — | complete, CRC ok | — | — | — | valid |
| 7 | — | — | — | matched → entry 7 | — | — | valid, rsp_seen |
| 8 | — | — | — | — | held | not ready | valid, rsp_seen |
| 9 | — | — | — | — | held | not ready | valid, rsp_seen |
| 12 | — | — | — | — | held | not ready | valid, rsp_seen |
| 13 | — | — | — | — | held | ready | valid, rsp_seen |
| 14 | — | — | — | — | — | accepted | valid, rsp_seen |
| 15 | — | — | — | — | — | — | retired |
Five things to read off it.
Cycle 6 is not completion. CRC passed and the object is whole. That is transport's job finished and nothing more — §22.
Cycle 7 is not completion either. The response has been identified. It has not been delivered. rsp_seen is set precisely so that the difference is representable.
Cycles 8 through 12 are the whole point. The consumer is stalled for five cycles and entry 7 stays valid throughout. A design retiring at cycle 7 has, for those five cycles, a response in a queue with no context and an identity that may be reallocated — §18.
Cycle 14 is the retirement trigger and cycle 15 is the retirement. One cycle apart, and the ordering matters: the entry is freed after the consumer has the data, never before.
And the row that never appears: no cycle in which the identity is free while the response is still in the queue. That absence is the correctness property, and p_retire_only_on_consumer_accept is what enforces it.
27. Flagship Trace 2 — The Late Stale Response
The signature example. Two designs, same stimulus, different outcomes.
Request A with identity 5, delivered and executed remotely. Its response is delayed.
| Cyc | Event | Naïve design | With generation + quarantine |
|---|---|---|---|
| 1 | A issued, identity 5, gen 1 | entry 5 valid | entry 5 valid, gen 1 |
| 40 | A's response still not returned | — | — |
| 50 | A times out | entry 5 cleared, id 5 free | entry 5 cleared, id 5 quarantined |
| 51 | failure reported upward for A | reported | reported |
| 55 | request B needs an identity | allocates id 5 | id 5 unavailable → allocates id 9 |
| 56 | B issued | id 5, no generation | id 9, gen 1 |
| 70 | A's response arrives, id 5 | matches B's entry | matches nothing — id 5 not live |
| 71 | resolution | B's requester gets A's data | reported as a late response |
| 72 | B's real response arrives | matches nothing → dropped | matches B's entry correctly |
| 73 | outcome for B | wrong data, completed | correct data, completed |
| — | outcome for the engineer | intermittent corruption, uncorrelated | a log line naming the request and its age |
And the harder variant, where quarantine alone is not enough. Suppose the quarantine expires at cycle 65 and identity 5 is reallocated to request C at cycle 66. A's response arrives at cycle 70 carrying generation 1; C's entry holds generation 2. The identity matches and the generation does not, so it is rejected and reported. Quarantine narrowed the window; the generation closed it — which is why §24 presents them as two techniques rather than one with a fallback.
The row worth dwelling on is the last. The difference between the two designs is not only correctness. It is that one produces intermittent corruption whose cause is a timeout that happened twenty cycles earlier, and the other produces a log line that says a response arrived for an abandoned transaction and how old it was. The second design is debuggable and the first is not, and that gap is larger than the correctness gap.
28. Recovery With Responses Outstanding
A link recovery lands with several requests outstanding, some of which the far side has already executed.
What the local side knows: which entries are open, and (with §9's epoch record from Chapter 11.5 §9) which link epoch each was accepted under.
What the local side cannot know: whether the far side executed each request, and whether a response was generated and lost. Both are on the other die.
So the same taxonomy as Chapter 12.1 §25 applies, and the disposition is architectural:
Retain and wait is right when recovery is expected to restore the transport and the far side retained its own state — the good case, and worth designing for, because it makes the whole event invisible.
Escalate is right when recovery failed or exceeded its budget, and it must report the outcome as unknown rather than as "not delivered".
Reissue only where the carried protocol says a reissue is safe for that operation — idempotent reads often, non-idempotent writes usually not (Chapter 12.1 §26).
And in every case, quarantine the identities of abandoned transactions. This is the recovery-specific reason §24 exists: a recovery can abandon many transactions at once, so the quarantine pressure is highest exactly when the identity space is most stressed — and a design that quarantines without enough identities to keep working will stall. Size the identity space for the abandonment case, not the steady state.
What this chapter does not do is state the recovery flow, the thresholds, or the reporting encodings. Those require specification text this chapter has not inspected, and Chapter 11.5 §17 made the same demarcation for the coherence case.
29. Head-of-Line Blocking on the Return Path
Response B is ready. Response A is ahead of it in a single global response queue, and A's consumer is stalled. Can B progress?
With one global FIFO: no. B is blocked behind A. That is head-of-line blocking (Chapter 9.3 §7), and it is a correct design — simple, small, and with a predictable worst case.
With per-class or per-consumer queues: yes, at the cost of more storage and an arbiter, plus a new question about whether reordering between the queues is permitted.
The design trade, stated honestly:
| One global queue | Per-class queues | |
|---|---|---|
| Storage | one queue | N queues, each sized for its worst case |
| Blocking | head-of-line across all consumers | isolated per class |
| Ordering | trivially preserved | needs an explicit policy |
| Deadlock risk | a stalled consumer stalls everything | lower, but arbitration fairness now matters |
And the deadlock note is the important one. With one global queue, a consumer that stalls indefinitely stalls the whole return path — which stops responses reaching other consumers, which may be what would have unblocked the first one. That is Chapter 11.4 §22's argument in the return direction: if the classes can depend on each other, isolating them is a correctness requirement rather than a performance choice.
This chapter does not prescribe which. It prescribes knowing which, and asserting the consequence:
// Illustrative — if the design claims class isolation, assert it: a stalled
// consumer of one class must not prevent another class from making progress.
property p_class_isolation_on_return;
@(posedge clk) disable iff (!rst_n || error_injection_active)
(rsp_ready_class[b] && rsp_valid_class[b] && !rsp_ready_class[a])[*MAX_WAIT]
|-> rsp_pop_class[b][->1];
endproperty30. Latency Decomposition
Where response latency actually goes, and why measuring the total tells you almost nothing.
| Segment | Owned by | Typical variability |
|---|---|---|
| remote execution | the far side's engine or media | high — media, contention, coherence resolution |
| remote response queue | far-side buffering | load-dependent |
| remote Adapter framing + transmit | transport | low, but quantised by framing (Ch 12.1 §12) |
| the link itself | PHY | low and nearly fixed |
| local Adapter reconstruct + CRC | transport | low |
| local matching | this chapter | low |
| response queue wait | local consumer readiness | high — and entirely local |
| consumer acceptance | above this layer | protocol-dependent |
The published UCIe KPI is the useful anchor here. The Consortium's target is latency (Tx + Rx) under 2 ns, stated as including the D2D Adapter and PHY — "FDI to bump and back." Read that carefully: it covers three of the eight rows — Adapter framing, the link, and Adapter reconstruction, in both directions.
So if a response's round-trip latency is dominated by anything, it is not those rows. The conclusion for a debugging engineer is direct: a latency problem on this path is overwhelmingly likely to be remote execution or local consumer backpressure, and instrumenting the transport is looking in the two places the architecture has already made small.
That is why §31's age field is per-request and why §35's taxonomy lists "response stuck despite healthy UCIe" as its own row.
31. Per-Request Age
// ILLUSTRATIVE diagnostic. Not a UCIe mechanism, and NO timeout value is
// asserted. In a verification environment a timestamp is usually better; in
// silicon an age field is what you get to keep.
always_ff @(posedge clk) begin
for (int e = 0; e < MAX_OUTSTANDING; e++) begin
if (alloc_fire && (alloc_idx == e[OUT_IDX_W-1:0]))
outstanding_q[e].age <= '0;
else if (outstanding_q[e].valid && (outstanding_q[e].age != {AGE_W{1'b1}}))
outstanding_q[e].age <= outstanding_q[e].age + 1'b1; // SATURATES
end
end
// Sticky diagnostics: the oldest age ever observed, and the age of the last
// transaction to be abandoned. Both survive recovery; both clear only on a
// broad deliberate reset.
logic [AGE_W-1:0] max_age_seen_q;
logic [AGE_W-1:0] last_abandon_age_q;Architecture. An age per entry plus two high-water marks. The high-water marks are what make this useful after the fact: the interesting number is the worst case and it will not be present when you look.
State. Per-request for age; diagnostic lifetime for the two sticky registers, which must survive link recovery and clear only on a deliberate reset.
Cycle behaviour. Cleared at allocation — not at retirement, because a stale age from the previous occupant of the slot is worse than no age. Saturates rather than wrapping, for the reason Chapter 12.1 §27 gives: a wrapped age reads as a young request, which is the one reading that hides the oldest transaction in the system.
Contract. A timeout policy, if the architecture has one, reads age. Debug reads all three.
Failure. Wrapping instead of saturating hides the failure. Clearing the sticky registers on recovery destroys the evidence of the event you are investigating.
DV. Verify saturation; verify max_age_seen_q is monotonic across a recovery; and verify age is zeroed at allocation rather than at retirement.
32. Request/Response Conservation
The invariant that makes double completion and silent loss detectable.
JOIN KEY — the verification-only monitor ID. Required because the protocol
identity is reused and, with §24, may be reused with a different generation.
per request (by mon_id):
issued_at, id_used, gen_used
responses_received : count — may be > 1 under transport replay
completions : count — must be exactly 1
disposition : completed | abandoned_reported | in_flight
stale_rejections : count — responses rejected by generation (§24)
aggregate:
requests_accepted, completed, in_flight, abandonedThe equation:
requests_accepted = completed + in_flight + abandonedChecked every cycle, and at end of test with in_flight required to be zero under progress assumptions.
Five checks, and each catches a specific bug in this chapter.
The equation, continuously. Catches silent loss — a dropped response with its entry already retired (§18) makes the left side exceed the right.
completions == 1 per request. Catches §21's duplicate completion, and note the diagnostic value is in rows where responses_received > 1 and completions == 1: that is duplicate suppression visibly working, and a regression in which that combination never appears has not tested the mechanism.
Every completion's data matches what that request asked for. This is the check that catches §13 and §15 and §27, and it is the only one that does. All three produce perfectly balanced counts — the pairings are simply wrong. The model must therefore hold the expected result per request, not just the fact of a response, which is why §5's table carries meta and why the model must too.
stale_rejections is non-zero in any run that injects a late response. A zero here after such injection means the generation check did not fire, which means it is not working.
No completion for a request whose disposition is already abandoned_reported. Catches §27 directly: a late response delivered against a reused identity shows up as a completion for a request that was already reported failed.
On what the model must not do. It must not read the design's out_count_q to derive in_flight — §20's counter is a suspect, not a source. Count from interface observations so a drifting design counter appears as a disagreement rather than being adopted as truth.
33. Coverage
// Illustrative response-flow coverage. Not UCIe-defined. Every bin exists to
// reach a specific failure in this chapter.
covergroup cg_response_flow @(posedge clk iff rsp_event);
cp_order : coverpoint response_order_vs_issue {
bins in_order = {0}; bins out_of_order = {1}; // §15
}
cp_out_occ : coverpoint out_count_q {
bins e = {0}; bins mid = {[1:$-1]}; bins full = {MAX_OUTSTANDING};
}
cp_rq_occ : coverpoint rsp_occ_q {
bins e = {0}; bins mid = {[1:$-1]}; bins full = {RSP_DEPTH};
}
cp_simul : coverpoint alloc_retire_same_cycle; // §19's 2'b11
cp_stall : coverpoint consumer_stalled_cycles {
bins none = {0}; bins few = {[1:4]}; bins many = {[5:$]};
}
cp_dup : coverpoint response_was_duplicate; // §21
cp_orphan : coverpoint response_was_orphan; // §7
cp_stale : coverpoint response_was_stale; // §24 — MUST be non-zero
cp_timeout : coverpoint request_timed_out;
cp_reuse : coverpoint id_reallocated_after_abandon; // §27
cp_gen_wrap : coverpoint generation_wrapped; // §9's DV note
cp_recovery : coverpoint recovery_with_responses_outstanding;
cp_status : coverpoint response_status_class; // where the architecture defines error responses
// Out-of-order returns at real concurrency — §15/§16. At one outstanding
// request the FIFO assumption is always right, so this cross is the bug's
// only route.
x_order_occ : cross cp_order, cp_out_occ;
// A stalled consumer with the response queue full — §18's window, maximised.
x_stall_full : cross cp_stall, cp_rq_occ;
// Simultaneous allocate/retire at the outstanding limit — §19's steady state.
x_simul_full : cross cp_simul, cp_out_occ;
// THE cross for §27: an abandoned identity reallocated, with a late response.
x_reuse_stale : cross cp_reuse, cp_stale;
// A duplicate arriving while the queue is full and the consumer stalled.
x_dup_pressure : cross cp_dup, cp_rq_occ, cp_stall;
// Recovery with responses in flight at every outstanding occupancy.
x_recovery_occ : cross cp_recovery, cp_out_occ;
endgroupThree notes, because the bins encode arguments.
cp_stale must be non-zero, not zero. It is the only bin in the group whose value is proof that a defence works. A regression reporting zero stale responses has either never injected one or has a broken generation check, and those are indistinguishable from the outside.
x_order_occ is the only route to §16. At one outstanding request, the oldest-outstanding assumption is always correct. The bug is unreachable without concurrency, which means it is unreachable in exactly the tests written first.
x_reuse_stale is the batch's most valuable single cross. It requires a timeout, an identity reallocation, and a late response for the original — three events in a specific order that no random stimulus produces. It is a directed test, and it is the one that finds the chapter's flagship bug.
34. Diagnostic Taxonomy
| Symptom | Where it is | First move |
|---|---|---|
| Response never arrives | remote execution, transport, or routing | is the request even outstanding remotely? Then Ch 12.1 §31 |
| Response arrives, matches nothing | the outstanding table | §8 — was the entry cleared early? Then §27 — was the identity reused? |
| Wrong data delivered to a request | identity/payload association | §13 for misalignment, §16 for a FIFO assumption, §27 for a stale response |
| A transaction completes twice | transport-to-semantic boundary | §21 — completion keyed on arrival; look for a lost confirmation |
| Intermittent corruption starting after link stress | stale response after identity reuse | §27 — correlate with earlier timeouts, not with the failing transaction |
| Response stuck despite a healthy UCIe link | local response queue or consumer | §11, §29 — and §30 says this is the likely place |
| Everything works at low load, fails at high | out-of-order responses, or count drift | §15 and §20 — both have exactly this profile |
| Outstanding count wrong but no loss | counting, not data | §20's $countones check |
| A late response after a recovery | expected — check it is handled | §28, and it should produce a log line rather than a symptom |
The fifth row is the one worth memorising, because the instinct it defeats is universal. The corruption is not correlated with the corrupted transaction. It is correlated with a timeout that happened earlier, on a different transaction, possibly to a different requester. An engineer who does not know to look backwards will examine the failing transaction, find it flawless, and conclude the problem is elsewhere.
35. Debug Checklist
- Which request created the obligation? Identity, generation, and monitor ID.
- Is its outstanding entry still valid? If not, when did it go, and why (§8, §18)?
- Did the remote side generate a response? If not, this is a forward-path or remote-execution question.
- Was the response transported? CRC, retry count, arrival cycle.
- Did the transport retry? And was a duplicate suppressed and counted (§21)?
- Did the local Adapter reconstruct one complete response?
- Did the identity match exactly one live entry? §7 — and if zero, is this an orphan or a stale response?
- Does the generation match? §24 — this is the question that separates "unexpected" from "late".
- Did the identity and the data arrive from the same source response? §13, checkable only with the monitor tag.
- Does the data match what that specific request asked for? §32's third check. This is where §15's and §27's bugs surface.
- Was the consumer ready, and for how long was it not? §31's age and the queue occupancy.
- Was the entry retired at the chosen point and no other? §18's second property.
- Did a timeout occur for this identity, or for a previous occupant of it? §27 — and note the second half of that question.
- Was the identity reused, and after a quarantine? §24.
- Does the conservation equation balance, and does every completion carry the right data? §32. Balance alone is not enough — §27 balances perfectly.
Step 15 has two halves and the second is the one people omit. Conservation catches loss and duplication. Only the data check catches mispairing, and mispairing is the failure class this chapter exists for.
36. Common Misconceptions
"Responses always return in request order." For at least one protocol carried over UCIe the specification says the opposite — the host does not preserve the order of requests as delivered by the device, and the device owns its ordering. The matching table exists precisely because issue order is not sufficient to identify a response (§14).
"The request can be forgotten once it leaves." The obligation is discharged by the far side, not by the transport. Freeing the entry on transmission means a returning response matches nothing — or matches a later request that reused the identity, delivering one requester's data to another (§8).
"A response ID match is optional if a FIFO is used." Only if the protocol guarantees in-order completion — and if it does, assert that guarantee, because it is not visible in the module making the assumption. Without it, two transactions silently exchange data and every count balances (§16).
"Outstanding state can retire when the response reaches the chip." Arrival is not delivery. Between matching and consumption the response sits in a queue, and the entry is the only record of what that queued response is for (§17, §18).
"Replay may create another semantic completion." A lost confirmation makes the transport deliver the same response twice, both with passing CRC. Completion keyed on arrival retires the entry twice — and the second retirement terminates whatever now occupies the slot (§21).
"Timeout makes late responses harmless." It makes them dangerous, because the local state they refer to no longer exists. The far side never learned about the timeout and was always going to respond (§23).
"IDs can be reused immediately." A timed-out identity reused for a new request will be matched cleanly by the old request's late response, and the new requester receives the old request's data — with a one-hot match, balanced counts, and a clean CRC (§25, §27).
"Request and response backpressure are the same." Deriving one from the other builds a deadlock out of two reasonable conditions: the forward path stalls on credits, credits do not return because responses are not consumed, and responses are not consumed because the forward path is stalled. Every safety assertion passes (§12).
"A clean CRC proves the response matched the right request." CRC proves the bytes that arrived are the bytes that were sent. Wrong identity, wrong data association, duplicate delivery, and stale responses all pass it — because they are failures of association, and association is not a property of the bytes (§22).
"Legal FIFO counts prove there are no stale-response bugs." The flagship bug of this chapter produces a one-hot match, a balanced conservation equation, correct occupancy, and wrong data. Only a check comparing each response against what that specific request asked for sees it (§32).
37. Understanding Check
38. Summary and What Comes Next
A response closes an obligation created earlier, so response processing cannot be understood without the retained request state that defines the obligation.
The foundation is a quoted contract: a request identity pre-allocated for the duration of the transaction and reflected in the response so it can be routed appropriately. Three rules follow — the entry exists before the request is sent, its lifetime belongs to the protocol rather than to any transport milestone, and a response is not self-describing.
The mechanisms: a four-conjunct match — valid entry, equal identity, equal generation, expected class — asserted $onehot with orphans reported separately, because a corrupted table and a lost obligation are different failures. A deliberately chosen retirement point at consumer acceptance, enforced by a property written from the effect so that every early-retirement path fires it. Independent backpressure in both directions, because deriving one from the other builds a deadlock from two reasonable conditions. One bundled response object, because a swapped identity and payload corrupts the requester's own view of memory and looks symmetrical from every count. An explicit case over allocate and retire, because at the outstanding limit that collision is the steady state rather than a corner case. And duplicate suppression below the semantic boundary, counted rather than merely working.
The hazard this chapter exists for: a timeout does not make late responses harmless — it makes them dangerous, because the state they refer to no longer exists. Reuse the identity and the late response matches cleanly, delivering one requester's data to another with a one-hot match, balanced counts, and a clean CRC. Quarantine narrows the window; the generation closes it.
And the two checks to write first: requests_accepted = completed + in_flight + abandoned, and — because balance alone is satisfied by every mispairing bug here — every completion's data must match what that specific request asked for.
Request and response paths have now been studied independently, each with its own ownership boundaries, buffers, and lifetimes. What has been deliberately left out of both is the payload itself: how data beats move, how a multi-beat transfer relates to the request that named it, and what happens when both directions are carrying data at once:
- 12.3 — Data Flow — payload-data movement direction and beat sequencing.
Browse the full path on the UCIe tutorials index.