CXL · Module 14
State Transitions
13.4 owns which transitions exist. This owns when they fire: the gap between requesting a transition and committing it, what happens when two flows interleave on one line, and the state an abandoned flow leaves behind.
13.4 built the state space: which states exist, which moves are legal, and the machinery that applies one.
14.1 through 14.4 built the flows that walk it.
This chapter is the intersection: which transitions actually fire, when they fire relative to the messages that caused them, and what a flow that stops half-way leaves behind.
1. The Engineering Problem — A Transition Is Not Instantaneous
Every diagram in Module 13 draws a transition as an arrow. In a real controller it is two events separated by time.
A transition is requested when the flow decides and committed when the state register changes. Between those two the flow believes one thing and the line says another, and that interval is not zero in any design where the authorising message crosses a link. Section 5 measures it.
Confirmations arrive out of order. Two transitions of one line, requested in one sequence and authorised in another. The commit order is what the rest of the system observes, and it must match the order the serialisation point decided rather than the order the messages arrived.
Two flows interleave on one line. The guard is not "one flow at a time" — flows legitimately overlap. It is that a flow's transition must be evaluated against the state the other flow left, not the one it started with.
And a flow can stop half-way. It has committed some of its transitions and not others. The line is in a state no complete flow produces, and the only question that matters is whether anything will ever move it again.
14.5 against 13.4, stated precisely. That chapter owns which transitions exist and which moves are legal. This one owns when they commit, in what order, and what happens when a flow is interrupted. If a section here could be moved into 13.4 without loss, it is in the wrong chapter.
2. The One-Sentence Model
A transition is requested, then authorised, then committed — and everything interesting happens in the gaps: between request and commit, between one flow's commit and another's evaluation, and between a flow stopping and the line being put back.
Call it request, authorise, commit. Every defect in this chapter is a commit that happened without its authorisation, in the wrong order, against a stale state, or not at all.
3. What This Chapter Owns
| Ground | Owner |
|---|---|
| The state space, the legal-edge graph, the transition machinery | 13.4 |
| The read flow | 14.1 |
| The write flow and the upgrade transaction | 14.2 |
| The transfer as a message sequence | 14.3 |
| Host and device cache interaction | 14.4 |
| When transitions commit, in what order, and what an interrupted flow leaves | this chapter |
Deferred:
| Deferred ground | Owner |
|---|---|
| Directory scaling and snoop filters | Modules 15 and 16 |
| Latency anatomy and bandwidth modelling | Module 18 |
| Device architecture | Modules 20 and 21 |
4. Teaching-Model Boundary
Every model below is a teaching model, compiled and simulated with Icarus Verilog 13.0, checked by a testbench whose oracle is structurally different from the design.
What these models are not: a coherency controller. There is no link layer, no transaction table, no MSHR allocation and no arbitration policy. A production controller has one transition context per outstanding transaction and a commit path shared between them.
The conventions from Module 13 and 14.1 carry over. Every checker tests cond !== 1'b1. Unreachable monitors get an extra build of the same source. Comparisons are one source under a parameter, instantiated twice, driven from one stimulus stream. Every displayed value is a captured signal, and every combinational sample is preceded by a settle.
This chapter uses seven parameterised twin builds — COMMIT_ON_REQUEST, COMMIT_ON_ARRIVAL, STALE_EVAL, NO_UNWIND, FAULT_INJECT on the transient owner, RECOVER_FORWARD and RECOVER_TO_SHARED, and SKIP_AUDIT.
5. RTL 1 — The Gap Between Deciding And Changing
// COMMIT_ON_REQUEST changes the state when the flow decides, not when the
// authorising message arrives -- so the state is right before it is allowed.
assign committed = pend_q && ((COMMIT_ON_REQUEST != 0) || confirmed);
assign premature_err = committed && !confirmed;
assign lost_request_err = request && pend_q;with the gap measured rather than assumed:
gap_q <= gap_q + 8'd1;
// Latch the widest request-to-commit gap: it is the interval in
// which the flow and the line disagree.
if (gap_q + 8'd1 > max_gap) max_gap <= gap_q + 8'd1;Measured:
commit : gap=3 latched=4 | premature correct=0 commit-on-request=1 cancelled=1
second request while pending reported=1 state held at 0The state held at INVALID for the whole gap while the flow had already decided to move it. The COMMIT_ON_REQUEST build changed it immediately, and premature_err fired — the state was right before it was allowed to be.
That distinction is the whole chapter in one signal. A state that is correct and unauthorised is not a benign optimisation: every other agent reads the state, and reading a state the protocol has not yet permitted is reading a claim nobody has agreed to.
A cancellation leaves the state untouched, and a second request while one is pending is reported rather than silently replacing the target. Both are cases a flow-level test never produces, because a well-behaved flow issues one request at a time.
6. RTL 2 — Commit Order, Not Arrival Order
Two transitions of one line, authorised out of order:
// A confirmation for a later transition must WAIT for the earlier one. The
// arrival-order build commits whatever arrives, which reorders the line's
// observable history.
assign held = conf && (conf_seq != expect_q) && (COMMIT_ON_ARRIVAL == 0);
assign commit_en = conf && ((COMMIT_ON_ARRIVAL != 0) || (conf_seq == expect_q));
assign out_of_order_err = commit_en && (conf_seq != expect_q);The later transition was confirmed first. Measured:
order : held=1 commits=2 | out-of-order correct=0 arrival-order build=1The correct build held it until the earlier one committed, then committed both in order. The arrival-order build committed the later one immediately, which reorders the line's observable history — and the check that catches it is not on the messages but on the commits.
This is 13.2's out-of-order response result applied to state changes rather than to data. Responses may arrive in any order; commits may not happen in any order, because the sequence of states a line passes through is what every other agent's view is built from.
The arrival-order build's expectation counter also does not advance on an out-of-order commit — checking that is what separates "committed the wrong one" from "committed the wrong one and lost track".
7. RTL 3 — Two Flows On One Line
The guard is not exclusivity:
// A step is legal only if the state the flow assumed is the state the line
// is actually in. STALE_EVAL skips that check, so a flow applies a
// transition computed against a state that no longer exists.
assign a_ok = flow_a_step && ((STALE_EVAL != 0) || (a_assumed_st == cur_st));Measured:
interleave: A and B both blocked on stale assumptions | reevals=2 steps=3
stale build stepped anyway=1 both at once=1 | idle line held at 3Both flows were blocked on stale assumptions, at different points in the run. Flow A was blocked when it assumed a state the line was not in; flow B was blocked after A moved the line out from under it, and then succeeded once it re-evaluated.
That is the model's whole content: a flow is not blocked because another flow is running, it is blocked because the state it computed its transition against no longer exists. The re-evaluation is cheap; applying the stale transition is not.
The STALE_EVAL build drops both the freshness check and the arbitration, and the reasoning is worth stating: a design that does not check what state a flow assumed has no basis on which to order two flows against each other either. That build applied both flows' transitions to one state.
8. Waveform — Four Cycles Of Disagreement
Transcribed from the printed cycle trace of the commit-point model in section 5.
A transition pending across an authorisation round trip
9 cyclesThe two state rows diverge for four cycles and then agree again. Everything before cycle 3 and after cycle 7 is identical, which is why a test that samples only at the ends of a flow cannot tell the two designs apart.
9. RTL 4 — What A Broken Flow Leaves
// A flow that stops part-way must unwind the transitions it committed.
// NO_UNWIND just stops, leaving the line where the last committed step put
// it -- which is a state no complete flow ever produces.
assign left_partial_err = !live_q && !unw_q && (done_q != 3'd0);A four-step flow was abandoned after two steps. Measured:
partial : complete=1 abandoned=1 unwound=1 | left partial correct=0 no-unwind=1The correct build unwound both committed steps; the NO_UNWIND build stopped where it was. The resulting state is not illegal — it is a state the graph has — but it is one that no complete flow produces, and nothing will ever move it again because the flow that owned it is gone.
A start attempted during the unwind was refused. That matters because the unwind is itself a sequence of transitions, and a new flow entering half-way through would interleave with a flow that no longer exists.
One redundancy was removed here. The monitor originally also tested done != total, but a flow whose steps are all done completes rather than being abandoned, so that term could never be the deciding one. It was a provably equivalent mutation pointing at a redundant condition; the condition was removed and the mutation replaced.
10. RTL 5 — Every Transient Belongs To One Flow
// A transient state with no flow owning it can never be left: nothing will
// ever supply the event that completes it.
assign orphan_transient_err = (st > 4'd4) && !own_q;
assign two_owner_err = enter && own_q && (flow_id != id_q) && (FAULT_INJECT == 0);
assign wrong_leaver_err = leave && own_q && (flow_id != id_q);Measured:
owner : two_owner=1 orphan=1 wrong_leaver=1 | faulty build owner=2
stable state flagged as orphaned=0This is the fact that makes unwinding possible at all. A transient can only be left by the flow that entered it, because only that flow knows what event completes it. Three failures, three separate signals:
orphan_transient_err— a transient with no owner. Nothing will ever supply the completing event, so the line is stuck permanently.two_owner_err— two flows claiming one transient. Each will supply a completing event, and the second arrives for a transition that already happened.wrong_leaver_err— a flow leaving a transient it does not own. The owning flow is still waiting, and its event will arrive at a line that has moved on.
The orphan monitor is scoped to transients specifically. A stable state with no owner is entirely normal, and a monitor that flagged it would fire constantly — the bench checks that explicitly.
11. RTL 6 — The Transitions That Only Exist For Recovery
// Recovery goes BACKWARD, to the state the flow started from -- never
// forward to the state it was heading for, because the events that would
// have justified the forward move never happened.Three policies were driven on identical stimulus. Measured:
recovery: SM_A back to 1 (S=1); forward build to 3 (M=3) | forward errors=3
IS_D: correct to 0 (I); recover-to-shared to 1 (S=1), flagged=1The upgrade transient recovers to SHARED — the state its flow started from. The forward build takes it to MODIFIED, which grants write permission that nothing authorised: the invalidations that would have justified it never completed.
The third policy is the interesting one. Recovering the fill transient to SHARED rather than INVALID looks defensible — the flow was heading for a shared copy — and it is still a grant, by one level rather than two. The flow never obtained read permission either. That build exists because it is the only way to make each transient's true origin observable: with only the correct and forward builds, the fill transient's origin could be recorded as either INVALID or SHARED and nothing would notice.
These transitions are not in the flow graph, because no flow uses them. They are the reason a real controller's state machine is larger than its specification's, and they are the transitions least likely to be exercised by any directed test.
12. RTL 7 — What Each Flow Actually Costs In Transitions
The payoff of 14.1 through 14.4: those chapters described messages; this counts the state changes they cause.
case (k)
2'd0: len_of = 3'd2; // I -> IS_D -> S/E
2'd1: len_of = 3'd2; // S -> SM_A -> M
2'd2: len_of = 3'd3; // I -> IM_D -> SM_A -> M
default: len_of = 3'd2; // transfer: O -> (handover) -> I
endcaseMeasured:
traces : read miss=2 upgrade=2 write-from-invalid=3 | total transitions=9 over 4 flowsA write from invalid makes three state changes and a read miss makes two. The write passes through two transients — the fill and the upgrade — because 14.2 established that a fetch is not permission. That is the single most useful number in this chapter for anyone sizing a transition path: the worst flow is 50% longer than the common one.
Each flow's starting and landing state was checked against a hand-written oracle, and the intermediate states too: the three cache flows enter a transient on their first step, while a transfer does not — the line stays OWNED while the duty moves, which is 14.3's overlap seen from the state's side.
13. RTL 8 — Where A Line Actually Spends Its Time
logic [31:0] weighted; // 16 x 100 needs 23 bits; 16 would silently wrap
assign transient_pct = (t_total == 16'd0) ? 8'd0
: (weighted / {16'd0, t_total});Thirty samples across a line's life. Measured:
residency: stable=21 transient=9 of 30 (30%) longest run=4Thirty percent, and a longest unbroken transient run of four cycles. 13.4 measured 16% on a synthetic stimulus; this is the same measurement taken across real flow shapes, and it is higher because the flows here include the write-from-invalid case that passes through two transients.
The longest-run counter is separate from the percentage for a reason. A line spending 30% of its time in transients as many short visits is a busy line; the same 30% as a few very long visits is a line that is getting stuck, and the two need different investigations. The run is latched and reset by any stable sample.
14. RTL 9 — Auditing Every Commit Against The Graph
14.1 built the path check as a testbench device. Here it is promoted to a monitor the design carries:
if (!legal) begin
n_illegal <= n_illegal + 8'd1;
run_q <= run_q + 8'd1;
// A burst of illegal commits is a different fault from an isolated
// one; the longest run separates a glitch from a broken table.
if (run_q + 8'd1 > worst_run) worst_run <= run_q + 8'd1;All 384 source-cause-destination combinations were swept against an independently transcribed edge list, then real commits were driven. Measured:
audit : 28 legal edges; illegal commits=3 of 4 audited, worst run=3Twenty-eight edges, matching 13.4 and 14.1 exactly. Three illegal commits were driven consecutively, and the worst run of 3 is the number that matters: an isolated illegal commit is a glitch, while an unbroken run of them is a broken transition table. A later shorter run did not reduce the latched peak, and a legal commit broke the run.
15. RTL 10 — The Machinery Assembled
// The audit runs on the committed transition, not on the request -- which is
// what makes it a monitor rather than a filter.
assign applied = pend_q && auth && (legal_w || (SKIP_AUDIT != 0));
assign refused = pend_q && auth && !legal_w && (SKIP_AUDIT == 0);
assign bad_state_err = (st_q > T_SM);Measured:
assembled: applied=3 refused=2 | skip-audit build applied the illegal one=1
state without authorisation held at 5 | shorter illegal run kept peak at 3
skip-audit build reached an undefined encoding=1Three legal transitions applied, two illegal ones refused. The state did not move without authorisation — held at the fill transient across multiple cycles with the request pending.
The SKIP_AUDIT build applied an illegal transition and, driven with a target outside the eight defined encodings, landed there. bad_state_err caught it. That monitor is unreachable in the audited build, which is exactly why the skip-audit build has to exist: the four-bit state field from 13.4 makes an undefined encoding representable, and only a build that will write one makes the monitor testable.
16. Quantitative Reasoning
The request-to-commit gap is an authorisation round trip. Measured at 4 cycles here; on a CXL link it is the full round trip from 13.2. Every transition in every flow carries that gap, so a flow with 3 transitions has three of them — which is why the write-from-invalid flow is the expensive one in both messages and time.
Transitions per flow, measured. A read miss and an upgrade take 2 state changes each; a write from invalid takes 3. Nine transitions across four flows. At a 4-cycle commit gap, the write-from-invalid flow spends 12 cycles in request-to-commit gaps alone, against 8 for the others.
Transient residency across real flows: 30%. Higher than 13.4's 16% on synthetic stimulus, because the flow mix here includes the two-transient write. The longest unbroken run of 4 is the number that distinguishes a busy line from a stuck one, and it must be latched separately from the percentage.
The audit is a lookup per commit. At 28 legal edges out of 384 combinations, 93% of the space is illegal, and checking each commit costs one table read. That is cheap enough to leave in silicon, and the run-length counter costs one comparator on top — which is what separates a glitch from a broken table.
Unwinding costs one transition per committed step. A flow abandoned after 2 of 4 steps unwinds 2. That is bounded by the flow's length rather than by anything external, so the worst-case unwind is the worst-case flow: 3 transitions.
17. Assertions
Presented as SystemVerilog and executed as procedural checkers — see section 19.
A transition never commits without its authorisation.
property p_no_premature_commit;
@(posedge clk) disable iff (!rst_n) committed |-> confirmed;
endpropertyThe state does not move while a transition is merely pending.
property p_state_held;
@(posedge clk) disable iff (!rst_n)
(pending && !confirmed) |-> $stable(state);
endpropertyA cancelled transition leaves the state untouched.
property p_cancel_is_inert;
@(posedge clk) disable iff (!rst_n) cancelled |-> $stable(state);
endpropertyCommits happen in the order the serialisation point decided.
property p_commit_order;
@(posedge clk) disable iff (!rst_n) commit_en |-> (commit_seq == next_expected);
endpropertyA flow never steps against a state it did not assume.
property p_fresh_evaluation;
@(posedge clk) disable iff (!rst_n) a_ok |-> (a_assumed_st == cur_st);
endpropertyNever two flows stepping one line at once.
property p_one_step;
@(posedge clk) disable iff (!rst_n) !(a_ok && b_ok);
endpropertyAn abandoned flow leaves nothing partial.
property p_no_partial;
@(posedge clk) disable iff (!rst_n)
(!in_flow && !unwinding) |-> (steps_done == 0);
endpropertyEvery transient has exactly one owning flow.
property p_transient_owned;
@(posedge clk) disable iff (!rst_n) (state > O) |-> transient_owned;
endpropertyOnly the owning flow may leave a transient.
property p_owner_leaves;
@(posedge clk) disable iff (!rst_n)
(leave && transient_owned) |-> (flow_id == owner);
endpropertyA recovery never grants a permission the flow did not obtain.
property p_recovery_backward;
@(posedge clk) disable iff (!rst_n)
is_recovery |-> (to_st <= flow_origin(from_st));
endpropertyEvery committed transition is an edge the graph has.
property p_audited;
@(posedge clk) disable iff (!rst_n) commit |-> legal;
endpropertyThe state never leaves the defined encodings.
property p_state_defined;
@(posedge clk) disable iff (!rst_n) (state <= T_SM);
endproperty18. Mutation Testing
85 mutations were injected into the ten models, one at a time, each a single-line change a competent engineer could plausibly write. Every one must make the testbench print RESULT: FAIL.
| Model | Mutations killed |
|---|---|
commit_point | 10 / 10 |
transition_order | 8 / 8 |
interleave_guard | 8 / 8 |
partial_flow | 10 / 10 |
transient_owner | 8 / 8 |
recovery_transitions | 7 / 7 |
flow_trace | 10 / 10 |
transient_residency | 8 / 8 |
transition_audit | 8 / 8 |
transition_top | 8 / 8 |
| Total | 85 / 85 |
Representative mutations, all killed:
| Mutation | What it models |
|---|---|
| The correct build also commits on request | a state right before it is allowed |
| A cancellation applies the transition | a rejected move applied anyway |
| A later confirmation commits immediately | the line's observable history reordered |
| The expected sequence advances on any commit | losing track of what was ordered |
| The correct build also skips the freshness check | a transition computed against a state that is gone |
| Both flows stepping is not detected | two transitions from one state |
| The correct build also leaves the flow unwound | a state no complete flow produces |
| A flow starts while an unwind is running | interleaving with a flow that no longer exists |
| An orphaned transient is not reported | a line nothing will ever move again |
| The correct build also lets a second flow claim it | two completing events for one transition |
| Any flow may leave the transient | the owning flow's event arriving at a moved line |
| The correct build also recovers forward | a permission nothing authorised |
| A write from invalid takes two transitions | the upgrade transient skipped |
| A transient is counted as stable | the residency measurement hiding its subject |
| The worst illegal run is sampled rather than latched | a glitch and a broken table indistinguishable |
| The correct build also skips the audit | an illegal commit applied |
| An undefined encoding is not detected | a state nothing downstream can decode |
Sixteen mutations survived the first run. None was patched away.
Nine stimulus gaps. The bench never checked each flow's starting or landing state (only its length), never checked the intermediate state of the write-from-invalid flow, never attempted a second run while a flow was live, never attempted a start during an unwind, never blocked flow A on a stale assumption (only flow B), never drove a cycle in which no flow stepped, never followed an illegal-commit run with a shorter one, never held a transition pending without authorisation, and never checked that a stable state is not flagged as an orphaned transient.
Three unobserved outputs. The step count after a completed flow, the arrival-order build's expectation counter, and the audit's total against its illegal count.
One unreachable checker. bad_state_err cannot fire in the audited build — the audit refuses any target outside the eight defined encodings. The SKIP_AUDIT build is what makes it reachable, and this is the payoff of 13.4's decision to widen the state field: an undefined encoding has to be representable before a monitor for it can be testable.
One dead-ish clause, removed from the design. left_partial_err also tested done != total, but a flow whose steps are all done completes rather than being abandoned, so that term could never be the deciding one. It was removed and the mutation replaced with one on a condition that can decide.
One model gained a third policy so a monitor became observable. from_st_stable records which stable state each transient's flow began from, and it is used only to decide whether a recovery moved forward. With only the correct and forward builds, the fill transient's origin could have been recorded as either INVALID or SHARED and nothing would have noticed. A third policy that recovers the fill transient to SHARED — a plausible wrong answer — makes the distinction observable. This is the same shape as 13.5's third mapping policy, and it recurs for the same reason: a monitor comparing against a recorded value needs stimulus that makes the recorded value matter.
A methodology finding worth recording separately. During this chapter's verification a mutation run was executed against a failing baseline, and reported every mutation as killed — because the harness declares a kill when the bench prints RESULT: FAIL, and the bench was already failing for an unrelated reason. A mutation run is only meaningful against a passing baseline. Two mutations that appeared killed in that run were genuine survivors, found only when the baseline was fixed and the suite re-run. Every mutation figure in this batch was produced against a passing baseline; this one was caught because the count that follows a failing run looks too good.
19. Verification Strategy
The oracle must not be the design. Each testbench models the same behaviour in a structurally different representation.
For commit_point the design holds a pending flag and a target register. The oracle holds two independent booleans — has it been asked for, has it been authorised — with the commit as their AND, so a design that collapses the two cannot be validated by a reference that has already collapsed them.
For flow_trace the design computes each flow's length, start and end from a case statement over a flow kind. The oracle holds three separate hand-written tables, one for each, so a bug in the flow encoding cannot appear identically in the reference.
For transition_audit the design is a nested case statement; the oracle is an explicit edge list written as integer comparisons, transcribed independently from 13.4, and swept across all 384 combinations. As in 14.1, the sweep is run with the commit enable low — otherwise 356 illegal combinations pour into the counter the real audit uses.
For recovery_transitions the oracle is an explicit map from each transient to the stable state its flow started from, which is the fact the recovery direction depends on.
Every displayed value is a captured signal, latched before any later event changes it. Two counters in this chapter continue to rise after their assertions and are captured at the assertion point.
Delta-cycle discipline. Every sample of a combinational output is preceded by a settle — including one added during development, where an input was driven and the dependent monitor read in the same delta.
Coverage recorded: 158 assertion sites across three testbenches; all four flows driven with their start, intermediate and landing states checked; a second run attempted during each; transitions confirmed in order and out of order on both builds; both flows blocked on stale assumptions and a cycle with neither stepping; a flow completed, abandoned, unwound, and a start attempted during the unwind; transients driven orphaned, claimed twice, and left by the wrong flow; all three recovery policies driven across all three transients; all 384 audit combinations swept plus consecutive illegal commits; and the assembled machinery driven legal, illegal, unauthorised, and with an undefined target.
20. Synthesis and Implementation Reality
The pending register is per outstanding transition, not per line. A controller with 16 transactions in flight has 16 of these, each holding a target state, a cause, and a gap counter. The gap counter feeds only the diagnostic.
The commit path is shared and the request path is not. Many flows request; one commit path applies. That is where the ordering constraint from section 6 lives, and it is why the expectation counter is a real structure rather than a verification device.
The audit is a table read on the commit path. At 28 edges it is small enough to be a lookup rather than logic, and it is on the same path as the state write — which is what makes it a monitor rather than a filter: it observes what was committed rather than gating what may be.
Unwinding needs the flow's committed-step count, which means the flow context outlives the flow. A design that frees the context on abandonment cannot unwind, which is exactly the NO_UNWIND failure — and it is an attractive optimisation because the context is the scarce resource.
The transient-owner field is one identifier per transient line. It is what makes unwinding possible and what makes orphan_transient_err checkable, and it is the smallest structure in this chapter that a design is most likely to omit.
Reset must leave nothing pending and no transient owned. A pending register surviving reset commits a transition nobody requested the moment an authorisation arrives for something else.
21. Silicon Observability
| Counter | Question it answers |
|---|---|
max_gap | the longest a transition was pending, latched |
n_cancelled | how often decided transitions are abandoned |
n_held | how much reordering the commit path is absorbing |
n_reevals | how often flows collide on one line |
n_abandoned against n_unwound | whether every abandoned flow was cleaned up |
transient_pct | how much of a line's life is mid-transition |
longest_transient | whether that time is many short visits or a few stuck ones |
n_illegal and worst_run | isolated glitches against a broken transition table |
n_refused | how often the audit caught something |
Five error signals belong in silicon. premature_err, left_partial_err, orphan_transient_err, illegal_commit_err and bad_state_err all detect states from which no correct behaviour is possible, and each is a handful of gates over registers that already exist.
n_abandoned minus n_unwound should be zero. A persistent difference means a flow stopped without cleaning up, and the line it owned is stuck in a transient. That is silent until something touches the line, at which point every access is blocked forever.
worst_run is the counter that classifies a fault. One illegal commit is a glitch worth logging. Three in a row is a transition table that disagrees with the graph, and the two need entirely different investigations — which is why the run length is latched separately from the count.
22. Debug Lab
A line's state is correct and nobody authorised it
PREMATURE-COMMITAnother agent reads a line's state and acts on it, then the authorising transaction is refused. The state was correct at the moment it was read and became wrong afterwards.
commit : gap=3 latched=4 | premature correct=0 commit-on-request=1premature_err fires when a transition commits without its authorisation. Read max_gap too — the longer the request-to-commit interval, the wider the window in which the two designs differ.
A state update at the request rather than at the authorisation; a speculative update with no squash path; a pipeline where the state write and the authorisation check are in different stages.
Request a transition and hold it pending across several cycles without authorising. The correct build's state does not move; the early build's does. Then cancel a pending transition and confirm the state is still untouched — a cancellation that applies the move is the same bug seen from the other end.
A state that is correct and unauthorised is not a benign optimisation. Every other agent reads the state, and reading one the protocol has not yet permitted is reading a claim nobody agreed to.
assign committed = pend_q && confirmed; // never on the request aloneLatch the gap. It is the width of the window in which the flow and the line disagree, and it is invisible in any test where the authorisation is prompt.
A line passes through states in an order nobody ordered
ARRIVAL-ORDER-COMMITTwo transitions of one line complete, and an observer sees the line in a state sequence that matches neither the request order nor any legal path.
order : held=1 commits=2 | out-of-order correct=0 arrival-order build=1out_of_order_err fires when a commit does not match the expected sequence. Check the commits, not the messages — responses may arrive in any order, but commits may not happen in any order.
A commit path driven directly by arriving confirmations; no sequence tracking on the transition path; an assumption that the link preserves order between transactions.
Request two transitions and confirm the later one first. The correct build holds it; the arrival-order build commits it. Then check the expectation counter on the faulty build — an out-of-order commit must not advance it, or the design has lost track as well as reordered.
The sequence of states a line passes through is what every other agent's view is built from. Reordering commits reorders that history.
assign commit_en = conf && (conf_seq == next_expected);
assign held = conf && (conf_seq != next_expected);This is 13.2's out-of-order response result applied to state changes rather than to data. The same reasoning, a different consequence.
A transition is applied against a state that no longer exists
STALE-EVALUATIONUnder concurrent access, a line ends up in a state that is legal but wrong for what the flows requested. Every individual transition was legal.
interleave: A and B both blocked on stale assumptions | reevals=2 steps=3stale_eval_err fires when a flow steps with an assumed state that does not match the line. Check whether the design records what state each flow computed its transition against.
A flow that computes its transition once and applies it later; no freshness check on the step; two flows arbitrated by priority without re-evaluation.
Have flow A move the line, then have flow B step with its original assumption. B must be blocked and re-evaluate. Then block A the same way — a bench that only ever blocks the second flow misses a symmetric bug in the first.
A flow is not blocked because another is running; it is blocked because the state it computed against no longer exists. Re-evaluation is cheap; applying the stale transition is not.
assign a_ok = flow_a_step && (a_assumed_st == cur_st);A design with no freshness check has no basis on which to order two flows either. Both properties are lost together, which is why the faulty build in this chapter drops both.
A line is stuck in a state no flow produces
PARTIAL-FLOWOne line stops responding. Its state is legal but is not the start or end of any flow. No flow is running against it.
partial : complete=1 abandoned=1 unwound=1 | left partial correct=0 no-unwind=1left_partial_err fires when a flow has ended with committed steps and no unwind running. Compare n_abandoned against n_unwound — a persistent difference means flows are stopping without cleaning up.
An error path that frees the flow context without unwinding; a timeout that abandons a flow; a reset of one flow's context that did not touch the line.
Abandon a four-step flow after two steps. The correct build unwinds both; the no-unwind build stops. Then attempt to start a new flow during the unwind and confirm it is refused — the unwind is itself a sequence of transitions, and a new flow entering half-way would interleave with a flow that no longer exists.
Unwinding needs the flow's committed-step count, so the flow context must outlive the flow. Freeing it on abandonment is attractive because the context is scarce, and it is exactly what makes unwinding impossible.
if (abandon) begin
live_q <= 1'b0;
unw_q <= 1'b1; // unwind rather than stop
endExport n_abandoned minus n_unwound. It should be zero, and a drift is silent until something touches the affected line.
A transient state that nothing will ever leave
ORPHANED-TRANSIENTEvery access to one line is blocked, indefinitely. The line is in a transient. No transaction is outstanding against it.
owner : two_owner=1 orphan=1 wrong_leaver=1 | faulty build owner=2orphan_transient_err is true whenever a transient has no owning flow. Note it is scoped to transients — a stable state with no owner is entirely normal, and a monitor that flagged it would fire constantly.
A flow that entered a transient and was destroyed; an owner field not written on entry; a second flow that claimed the transient and left, releasing it on the first flow's behalf.
Drive a transient with no owner and confirm it is reported. Then have a second flow claim an owned transient — refused and reported — and have a non-owning flow attempt to leave one, which must also be refused. Three separate signals, three different failures.
A transient can only be left by the flow that entered it, because only that flow knows what event completes it. With no owner, nothing will ever supply that event.
assign orphan_transient_err = (st > O) && !owned;
assign two_owner_err = enter && owned && (flow_id != owner);
assign wrong_leaver_err = leave && owned && (flow_id != owner);The owner field is the smallest structure in this chapter and the one a design is most likely to omit. Without it, unwinding is impossible and the orphan condition is undetectable.
A recovery grants a permission nobody authorised
FORWARD-RECOVERYAfter an error recovery, an agent holds write permission on a line whose invalidations never completed. The recovery reported success.
recovery: SM_A back to 1 (S=1); forward build to 3 (M=3) | forward errors=3forward_recovery_err compares the recovered state against the stable state the transient's flow began from. Recovery goes backward; anything forward is a grant.
A recovery table that maps each transient to its intended destination; a recovery written from the flow's goal rather than its origin; a partial completion treated as a completion.
Recover from each of the three transients. Each must return to the state its flow started from. Note that recovering the fill transient to SHARED rather than INVALID looks defensible and is still a grant — by one level rather than two — which is what makes each transient's true origin worth checking individually.
The events that would have justified the forward move never happened. Recovering forward grants a permission on the strength of a transaction that failed.
case (from_st)
T_IS: to_st = I; // the fill flow started from INVALID
T_IM: to_st = I;
T_SM: to_st = S; // the upgrade flow started from SHARED
endcaseThese transitions are not in the flow graph, because no flow uses them. They are the reason a real controller's state machine is larger than its specification's, and the least likely to be exercised by any directed test.
An illegal transition is committed and the state is legal
UNAUDITED-COMMITA line's state history contains a move that no legal path produces. Each individual state is valid. Nothing was reported.
audit : 28 legal edges; illegal commits=3 of 4 audited, worst run=3illegal_commit_err checks each commit against the graph. Then read worst_run — an isolated illegal commit is a glitch, while an unbroken run is a transition table that disagrees with the graph.
A transition table transcribed separately from the graph; an optimisation that skips the fill transient; a commit path that trusts its caller.
Sweep all 384 source-cause-destination combinations against an independently written edge list and confirm 28 are legal. Then drive three illegal commits consecutively and check both the count and the run length. A legal commit must break the run, and a later shorter run must not reduce the latched peak.
Every state in the history was valid; the moves between them were not. Only a per-commit check against the graph can see that.
assign illegal_commit_err = commit && !legal;Run the exhaustive sweep with the commit enable low. Otherwise 356 illegal combinations pour into the counter the real audit uses, and the counter means nothing.
A state field holds a value nothing can decode
UNDEFINED-ENCODINGDownstream logic derives inconsistent permissions from one line. Different consumers disagree about what the line's state means.
assembled: applied=3 refused=2 | skip-audit build reached an undefined encoding=1bad_state_err compares the state against the highest defined encoding. It is unreachable in a design that audits its commits, which is why a zero reading proves nothing without a build that will write an undefined value.
A commit path with no audit; a reset value in a reserved encoding; a state field written by more than one source.
Request a transition to a target outside the eight defined encodings. The audited build refuses it; the skip-audit build applies it and lands there. This is the payoff of the four-bit state field from 13.4 — an undefined encoding has to be representable before a monitor for it can be testable.
An undefined encoding is not a wrong answer but an uninterpretable one, so every consumer derives something different from it.
assign applied = pend_q && auth && legal_w; // audit gates the commit
assign bad_state_err = (st_q > T_SM);Treat the reset value as part of the check. A controller that comes out of reset in a reserved encoding has no legal move at all.
23. Design Review
What was built. Ten models: a commit point with a commit-on-request twin, an ordering constraint with an arrival-order twin, an interleave guard with a stale-evaluation twin, a partial-flow tracker with a no-unwind twin, a transient owner with a fault-injection build, a recovery table in three policies, a flow tracer for all four flows, a residency meter, a live transition audit, and the machinery assembled with a skip-audit twin.
What was measured. A 4-cycle latched request-to-commit gap with the state held throughout, against an early build that moved immediately. A later confirmation held until its predecessor committed, against an arrival-order build that reordered the history. Both flows blocked on stale assumptions, with the stale build applying both at once. A four-step flow abandoned after two and unwound completely, against a build that stopped. Three transient-ownership failures, each on its own signal. Three recovery policies with the correct one going backward and both others granting. 2, 2 and 3 transitions for the read miss, upgrade and write-from-invalid flows — nine over four flows. 30% transient residency with a longest run of 4. 28 legal edges of 384, with three consecutive illegal commits and a latched worst run of 3. And a skip-audit build reaching an undefined encoding.
What would be different in production. Every structure here is per outstanding transition, and the commit path is shared between them — which is where the ordering constraint actually lives. The audit becomes a table read rather than logic. The unwind is driven by a transaction table rather than a step counter. None of that changes the three-part shape; all of it multiplies where it can be got wrong.
The strongest argument against this design. Holding the flow context alive through the unwind means a context cannot be freed on abandonment, and the context is the scarce resource that bounds concurrency. That argument is correct, and the honest response is that the unwind needs the step count, not the whole context — a two-bit counter can outlive the transaction table entry, exactly as 14.3 separated the liability flag from the transfer context. The model keeps them together because that is the clearer teaching shape, and a production design should not.
What would be built differently next time. from_st_stable records where each transient's flow began, and with only two recovery policies its fill-transient entry could have been wrong without anything noticing. The third policy was added to make it observable. The general lesson is that a monitor comparing against a recorded value needs stimulus that makes the recorded value matter — and that is now the second time in this module a third build has been required for exactly that reason.
24. How This Appears In Real Engineering
In a microarchitecture review, the question that exposes a premature-commit design is when the state register is written relative to the authorisation. "When we know" and "when we are told" are the same sentence in a specification and a round trip apart in silicon.
In bring-up, n_abandoned minus n_unwound is the counter that finds stuck lines. A drift means flows are stopping without cleaning up, and the affected line is blocked forever with nothing reporting it.
In a performance investigation, transient_pct alongside longest_transient distinguishes a busy line from a stuck one. The same 30% as many short visits and as a few long ones need entirely different work.
In a verification plan review, ask whether the flow machine has been held with its authorisation absent, in every phase. That is the only stimulus that separates a commit driven by an event from one driven by elapsed time.
In silicon debug, worst_run classifies the fault before anyone looks at a waveform. One illegal commit is a glitch; three consecutively is a transition table that disagrees with the graph.
In a design review of somebody else's controller, ask what happens to a line when a flow is killed. If the answer is "the context is freed", the line is stuck in whatever transient the flow left it in.
25. Common Misconceptions
"A transition is an event." It is two events separated by an authorisation round trip. Measured: a 4-cycle latched gap with the state deliberately unchanged throughout.
"Committing early is a harmless optimisation." A state that is correct and unauthorised is a claim nobody agreed to, and every other agent reads it. Measured: the early build's state was right from cycle 3 and permitted from cycle 7.
"Responses arrive out of order, so commits can too." Responses may; commits may not. The sequence of states a line passes through is what every other agent's view is built from.
"Two flows on one line must be serialised." They interleave legitimately. The guard is that each flow's transition is evaluated against the state the other left — not that only one runs at a time.
"An abandoned flow just stops." It must unwind. Measured: the no-unwind build left the line in a state no complete flow produces, and nothing will ever move it again.
"A transient state is just a state." It belongs to exactly one flow, because only that flow knows what event completes it. Measured: three separate ownership failures, each with its own signal.
"Recovery restores the line to where it was going." It restores it to where the flow started. The events that would have justified the forward move never happened, and recovering forward grants a permission on the strength of a failed transaction.
"Auditing every commit is too expensive." It is a table read against 28 edges, plus one comparator for the run length. That is cheap enough for silicon, and it is the only thing that distinguishes a legal state reached illegally.
26. Interview Reasoning
27. Exercises
-
Calculation. A system runs 1000 flows in the ratio 5:3:2 of read miss, upgrade and write-from-invalid, at a 4-cycle commit gap. Compute the total number of transitions and the total time spent in request-to-commit gaps.
-
Analysis. A line reports 30% transient residency with a longest run of 40 cycles, on a design whose flows are at most 3 transitions. State what that combination implies, and which of this chapter's counters you would read next.
-
RTL task. Extend
partial_flowto unwind two flows concurrently on different lines. State what state that requires per flow, and the failure that becomes possible if the two share one step counter. -
Assertion task. Write the property proving the state does not move while a transition is merely pending. Then explain why it passes trivially on a design where
pendingis derived from the state register, and what independent source of the pending flag is required. -
Design task. Add a speculative commit with a squash path, keeping the state authoritative. State what must be squashed, what may not be, and which of this chapter's monitors must change.
-
Testbench design. Design the stimulus that distinguishes a commit driven by authorisation from one driven by elapsed time. Explain why every test with a prompt authorisation passes on both, and state the minimum stimulus that separates them.
-
Debug task. A mutation suite reports 100% kills on a chapter that has been producing survivors all day. Give your first check, explain what it would reveal, and state why the result is not trustworthy until it is done.
-
Design review. A colleague proposes dropping the transient-owner field, arguing that a line in a transient always has exactly one flow against it by construction. Give the strongest version of that argument, then the two failures that become undetectable, and the cost of the field.
28. Summary
A transition is requested, then authorised, then committed.
- The gap is an authorisation round trip. Latched at 4 cycles, with the state held throughout — and a build that committed at the request was right before it was allowed.
- Commit order, not arrival order. A later confirmation was held until its predecessor committed; the arrival-order build reordered the line's observable history.
- Flows interleave; stale evaluations do not. Both flows were blocked at different points and both succeeded after re-evaluating against the state the other left.
- An abandoned flow unwinds. Two committed steps of a four-step flow undone; the no-unwind build left the line in a state no complete flow produces.
- Every transient belongs to exactly one flow. Orphaned, doubly-claimed and wrongly-released transients each on their own signal — and a stable state with no owner is normal.
- Recovery goes backward. The upgrade transient to SHARED, the fill transients to INVALID. Both forward policies granted a permission nothing authorised, one by two levels and one by one.
- Transitions per flow: 2, 2 and 3. The write from invalid is the expensive one, passing through two transients because a fetch is not permission.
- 30% transient residency, longest run 4. Higher than 13.4's synthetic 16% because of the two-transient write, and the run length is what separates a busy line from a stuck one.
- 28 legal edges of 384, audited on every commit, with three consecutive illegal commits producing a latched worst run of 3 — a broken table rather than a glitch.
- The state never leaves the defined encodings. The skip-audit build reached one, which is the payoff of the four-bit field: representable first, then testable.
- Verification: 158 assertion sites, 85 of 85 mutations killed, zero surviving. Sixteen first-run escapes were nine stimulus gaps, three unobserved outputs, one unreachable checker, one dead-ish clause removed from the design, and one model that gained a third policy so a recorded value became observable. Plus one methodology finding: a mutation run against a failing baseline reports every mutation as killed, and two genuine survivors were hidden by exactly that before the baseline was fixed.
Module 14 is complete. 14.1 built the read, 14.2 the write and its acquisition, 14.3 the transfer, 14.4 the boundary between two caches, and this chapter the transitions all four of them cause. Module 15 takes these flows to a fabric — more agents, switches between them, and a directory that can no longer name every sharer individually.
Continue learning
Related tutorials
- Related topic
CXL-over-UCIe Integration
Composing a CXL-coherent chiplet from three state planes that must agree — memory mapping, coherence ownership, and transport. Why one plane being valid proves nothing about another, why one transaction occupies four tracking entries that are not duplicates, why semantic state must not retire at a transport event, and the three-model scoreboard that attributes a failure to a plane.
- Related topic
UCIe and CXL
Which responsibilities belong to CXL and which to UCIe — why native mapping and streaming are different relationships, the three subprotocols' three different boundary contracts, why a UCIe send is not a CXL completion, the destination decided once and never recomputed, CXL.mem traffic that starves CXL.cache into deadlock, what a UCIe recovery may and may not touch, why a degraded link is slower and not different, and a layer-attribution method for deciding which protocol to blame.
- Related topic
CXL State Management
A coherency state is not a name, it is a tuple of facts, and most of the state space is the transient states nobody draws. The encoding, the legal-edge graph, the machinery that applies a transition atomically, and what happens to a snoop that arrives mid-flight.
- Related topic
Why CXL Matters
Why PCIe transaction semantics are insufficient for coherent memory and accelerator attach — CXL.io, CXL.cache and CXL.mem as the CXL Consortium defines them, device types, why coherence needs distributed per-line state, memory-window routing, what coherence costs, and why a clean transport proves nothing about coherence.
Standards & specifications
- Governing standard
- CXL Specification (CXL Consortium)(opens CXL Consortium in a new tab)
Defines CXL.io, CXL.cache and CXL.mem, and the coherence and memory-pooling behaviour built on them. System design and deployment topology are not mandated.
This page also covers RTL structure, verification approach and debugging technique. Those are engineering practice built on the standard, not requirements the standard itself imposes.
Where this fits
Part of the CXL curriculum.
