Ownership-Transfer Flows

13.3 established what ownership is — a debt, not a right — and stated the rule for moving it: the responsibilities must overlap, never abut.

This is the message sequence that produces that overlap, and the three things it has to survive: a directory that wants to publish the new owner early, a third agent that asks for the line while it is in mid-air, and a transfer that fails half-way.

1. The Engineering Problem — The Line Is Always Somewhere

A transfer moves data from one cache to another. That part is easy. What makes it hard is that the line has to remain answerable for every cycle of the move.

The old owner must stay liable longer than feels natural. The instinct is to hand responsibility over when the data is sent. That is one message too early: the acknowledgement has not arrived, so the new owner may not have the line, and for the duration of that latency nobody can answer. Section 5 measures the gap this produces.

The directory must not publish the new owner until it can answer. A directory that names the destination when the data leaves points every subsequent lookup at a cache that is still empty. The lookup does not fail — it is answered by an agent with nothing, which is worse.

A third agent is entitled to ask at the worst possible moment. An agent that is neither the source nor the destination reads the line mid-transfer. It can always be served, because the line is always somewhere, and stalling it is a choice rather than a necessity.

And the transfer can fail. A failure must leave exactly one owner, and it must be the old one — the new owner may hold a partial copy and cannot be trusted to answer from it.

2. The One-Sentence Model

Ownership moves by overlapping, not by handing off. Both agents are liable for the interval between the data landing and the old owner releasing, the directory names whoever can actually answer, and a failed transfer leaves the line exactly where it started.

Call it overlap, then release. Every defect in this chapter is a release that happened before the take, or a name published before the thing it names could answer.

3. What This Chapter Owns

Deferred:

13.3 owns the rule; this chapter owns the sequence. Where that chapter proved a three-phase handover keeps the line answerable, this one builds the five-phase message flow, the directory's publication timing, and the third-party and failure cases it did not reach.

4. Teaching-Model Boundary

Every model below is a teaching model, compiled and simulated with Icarus Verilog 13.0, checked by a testbench whose oracle is structurally different from the design.

What these models are not: a coherency controller. There is no link layer, no credit scheme, no address decode, and no migration policy — the decision to move ownership is somebody else's, and this chapter only executes it.

The conventions from Module 13 and 14.1 carry over. Every checker tests cond !== 1'b1. Unreachable monitors get an extra build of the same source. Comparisons are one source under a parameter, instantiated twice, driven from one stimulus stream. Every displayed value is a captured signal, and every combinational sample is preceded by a settle.

This chapter uses nine parameterised twin builds — RELEASE_ON_SEND, PUBLISH_EARLY, DEFER_ALWAYS, ALWAYS_NEW, ACCEPT_ANY, DROP_THEN_TAKE, ABORT_TO_NEW, QUEUE_AS_ISSUED, TWO_OF_THREE and NO_OVERLAP.

5. RTL 1 — Five Phases, And Who Answers In Each

  // The old owner answers until the ACK, not until the send. RELEASE_ON_SEND
  // hands responsibility over one message too early, which opens a gap the
  // length of the acknowledgement latency.
  assign old_still_liable = (ph_q == ASKED) || (ph_q == PREP)
                            || (ph_q == FLIGHT)
                            || ((ph_q == SETTLE) && (RELEASE_ON_SEND == 0));
  assign responder_valid  = old_still_liable || resp_v_q;
  assign gap_err          = read_arrives && !responder_valid;

Reads were driven in every phase, on both builds, from one stimulus stream. Measured:

  handover : gaps correct=1 early-release=2 | old liable at settle=1 early=0
             held in prepare=2 in flight=3
             held settling=4

One gap in the correct build, and it was the deliberate one — a read issued before any agent had ever owned the line. The early-release build had more, and the extra ones are the point: the very read that arrives during the settling phase is answered by the correct build and unanswerable in the other.

The three hold checks are worth naming. The flow stayed in preparation until the old owner had stopped writing, in flight until the data was actually sent, and in settling until the acknowledgement arrived. Each of those is an event, and a phase machine that advances on elapsed time passes every test in which the event is prompt.

The oracle is a single "who can answer right now" integer, advanced by events, with no notion of a phase at all — so a bug in the phase encoding cannot appear in the reference.

The data goes from the old owner to the new one directly. The directory is told about the transfer and never carries the line — which is why section 9 has to check that the new owner accepts data only from the agent it was told to expect.

6. RTL 2 — Publishing The New Owner

The directory must name whoever can actually answer:

  // The invariant: whoever the directory names must be able to answer.
  assign points_at_empty_err = lookup && pv_q && (pub_q == new_owner)
                               && !newhas_q;

with the publication gated on the acknowledgement rather than the send:

      if (data_sent && moving_q) begin
        // PUBLISH_EARLY names the new owner when the data leaves, not when it
        // arrives -- a window in which the directory points at an empty cache.

Lookups were driven before, during and after the transfer. Measured:

  publish  : at send correct names 1 early build names 2 | misdirected correct=0 early=1
             lookups=3

At the moment the data leaves, the correct directory still names agent 1 and the early-publish directory already names agent 2 — which has nothing. A lookup in that window is not a miss; it is answered by an empty cache, which is a failure that presents as data corruption rather than as a lookup failure.

Both publication paths are gated on moving_q, a transfer this directory actually started. Without that guard a data message belonging to some other transfer moves the published owner — a defect the testbench found before any mutation ran, because two models shared one data_acked stimulus and the directory reacted to a transfer it had never been told about.

7. RTL 3 — The Third Agent

An agent that is neither source nor destination reads the line mid-transfer:

  // The line is always somewhere. During a transfer it is with the old owner
  // until the data lands, and with the new one afterwards -- so a third-party
  // read can always be served, and deferring it is a choice rather than a
  // necessity.
  assign server    = (ALWAYS_NEW != 0) ? new_owner
                   : (data_with_new ? new_owner : old_owner);

Measured across three builds:

  third pty: served by agent 1 mid-transfer | defer build deferred=1 starved=1 released=1
             always-new build served from an empty agent=1

The third-party read was served immediately, by the old owner, which still had the data. The DEFER_ALWAYS build stalled it instead — legal, and it produced a request that aged past the starvation threshold before the transfer completed. Deferring is a real design choice with a real cost, and the model measures the cost rather than asserting it.

The ALWAYS_NEW build is the third one, and it exists solely to reach served_by_empty_err. Both other builds name whoever holds the data, so the monitor is unreachable in each — the same pattern 14.1 and 14.2 each needed once.

8. Waveform — Three Cycles With No Answerer

Transcribed from the printed cycle trace of the assembled flow in section 15.

Three cycles with nobody able to answer. That is the acknowledgement latency, and on a CXL link it is a round trip rather than three cycles. Over the full trace the correct build recorded 1 unanswerable probe — the deliberate pre-ownership one — against 5 for the no-overlap build.

9. RTL 4 — Where The Data Comes From

The new owner receives a line from an agent it never contacted, so it has to check the sender:

  assign accept = data_valid && armed_q
                  && ((ACCEPT_ANY != 0) || (data_sender == src_q));
  // Routing the line through the directory doubles the transfer latency for
  // no correctness benefit. It is not an error, it is a cost -- counted.
  assign via_home_err = data_valid && home_carries;

Measured:

  data path: wrong_sender=1 accept-any took it=1 via-home=1 accepted=2

The sender check is the same reasoning as 14.1's forwarded read: a cache that held the line a moment ago and has a data message already in flight will deliver a well-formed stale copy, and the only evidence is that it came from an agent nobody named.

via_home_err is deliberately not an error. Routing the line through the directory is correct and slow — it doubles the transfer latency for no correctness benefit — so the model counts it rather than flagging it. Building it as a counter rather than an alarm is the honest choice, and a mutation that turns it into an unconditional flag is killed by a transfer that legitimately took the direct path.

10. RTL 5 — The Duty Moves By Overlapping

13.3 said the responsibilities must overlap. This measures what happens when they do not:

  // A dirty line with nobody owing it is 13.3's data-loss condition. During a
  // transfer the safe order is TAKE then DROP; DROP_THEN_TAKE inverts it.
  assign nobody_owes_err = live_q && !old_q && !new_q;

with the faulty build releasing at the send:

        // DROP_THEN_TAKE releases the old owner when the data is SENT. The new
        // owner does not owe anything until it LANDS, so the interval between
        // the two is a window in which nobody owes the data at all.
        if (data_sent && (DROP_THEN_TAKE != 0)) old_q <= 1'b0;

Measured:

  duty     : overlap=1 latched=3 | nobody-owes correct=0 drop-then-take=1
             drop-then-take gap cycles=2

Both agents owed the data during the overlap, and that is safe. Two agents owing one line is not a two-owner failure — it is a period in which either can answer, and 13.3's registry invariant is about who may claim the duty, not about how many agents are discharging it during a handover.

The faulty build produced two cycles in which nobody owed the data, which is 13.3's orphaned-dirty-line condition arriving as a transient rather than as a permanent state. The peak overlap is latched at 3 cycles: a shrinking overlap is what precedes a gap, so the widest one is the safety margin worth watching.

11. RTL 6 — A Transfer That Fails

  // ABORT_TO_NEW leaves the new owner named after a failure, so the line's
  // only recorded holder is the agent that received an incomplete copy.
  assign partial_owner_err = !live_q && ov_q && (own_q == new_owner)
                             && aborted_q;

Measured:

  abort    : owner after abort correct=1 abort-to-new=2 partial reported=1 orphan=0
             orphaned before any transfer=1

A failed transfer leaves the OLD owner holding the line. The new owner may have received part of it, and a directory that names it is naming an agent that will answer reads from an incomplete copy. Rolling back to the source is the only safe outcome, and it is safe precisely because the source never gave anything up.

no_owner_err is reachable at exactly one moment — before any transfer has ever run, when the line genuinely has no recorded owner. That is a real condition rather than an artefact, and the bench checks it explicitly.

One redundancy was found and removed here. The completion path cleared an aborted_q flag that the start path already cleared, and no stimulus could ever make the second clear load-bearing. It was a provably equivalent mutation pointing at a redundant assignment; the assignment was removed and the mutation replaced with one on the clear that actually matters.

12. RTL 7 — Two Transfers Of One Line

The second transfer's source is whatever the first leaves behind:

          to_q   <= qto_q;
          // The retarget: the queued transfer starts from wherever the line
          // actually is now, not from where the requester thought it was.
          from_q <= (QUEUE_AS_ISSUED != 0) ? qfrom_q : to_q;

Measured:

  serialise: retarget source=1 (queue-as-issued keeps 0) overlap detected=1 stale source=1

The queued transfer was retargeted to source from agent 1 — where the line actually is — while the queue-as-issued build still sources it from agent 0, which let go. That is 14.2's reclassification result in a different guise: a queued request encodes an assumption about a state that the transfer ahead of it has already changed.

stale_source_err had to be rewritten during verification. As first written it tested q_q && !busy_q, a combination the design can never produce — the queue clears in the same cycle the next transfer starts. That is a dead monitor rather than an unreachable one, and the fix was in the condition: checking while the transfer is still queued, which is both reachable and the moment the information is actually useful.

13. RTL 8 — A Transfer Finishes On Three Things

  assign complete = live_q && ((TWO_OF_THREE != 0) ? (cnt >= 2'd2)
                                                   : (d_q && a_q && p_q));
  assign partial_complete_err = complete && !(d_q && a_q && p_q);

The data landed, the new owner acknowledged, and the directory published. Measured:

  complete : two-of-three declared early=1 partial=1 tag_mismatch=1

The two-of-three build declared the transfer complete with the directory still naming the old owner. Every agent involved believes the transfer happened; the directory does not, so every subsequent lookup goes to an agent that no longer has the line. That is the worst of the three partial states, because the two participants agree with each other and disagree with the system.

All three parts are tag-matched. The publish half is the one usually left unchecked, and a mutation removing that check survives any test that only mismatches the data or the acknowledgement.

14. RTL 9 — What A Transfer Costs, And Whether It Paid

  // The payoff: how many local hits the new owner got out of the transfer. A
  // transfer that yields fewer hits than it cost cycles was not worth making.
  assign hits_per_xfer = (n_xfers == 16'd0) ? 8'd0
                       : ({16'd0, n_hits_after} / {16'd0, n_xfers});

Measured:

  cost     : xfers=2 mean=50 cycles | stall mean=12 | hits after=20 = 10 per transfer

Ten local hits per transfer, at 50 cycles per transfer. That is the number that decides whether moving the duty was worth it: a transfer that yields two hits has cost more than it saved, and one that yields fifty has paid for itself many times over. Without it, n_xfers measures activity rather than value, and a line ping-ponging between two agents looks identical to one that migrated once and stayed.

The third-party stall counter is separate because it is a cost borne by an agent that had nothing to do with the decision. At a 12-cycle mean it is small; on a design that defers third-party reads for the whole transfer it is the transfer latency.

15. RTL 10 — The Flow Assembled

  // The old owner answers until the acknowledgement. NO_OVERLAP hands over at
  // the send, leaving the acknowledgement latency with no answerer at all.
  assign answerable = (st_q == IDLE) ? cur_v_q
                    : ((st_q == SETTLE) ? (NO_OVERLAP == 0) : 1'b1);
  assign unanswerable_err = probe && !answerable;

Probes were driven in every phase. Measured:

  assembled: unanswerable correct=1 no-overlap=2 | answerable at settle correct=1 no-overlap=0
  holds    : prep=1 flight=2 settle=3 (each waits for its own event)

One unanswerable probe in the correct build across the whole run, and it was the deliberate one before any agent owned the line. Each of the three phases waited for its own event rather than for elapsed time.

16. Quantitative Reasoning

The gap is the acknowledgement latency. Measured at 3 cycles in the trace; on a CXL link it is a full round trip. At the 13.2 figure of a 4-to-8-cycle round trip, releasing at the send opens a gap of that length on every transfer, during which any read of the line has no correct answer.

The overlap is the safety margin, and it is free. Both agents owing the line costs nothing — no extra message, no extra state beyond one bit. The measured peak was 3 cycles. A design that shortens it is trading a margin for nothing, because the transfer's total latency is set by the data movement and the acknowledgement, not by how long the old owner keeps a flag set.

Publishing early buys nothing and costs a window. The directory's publication is not on any critical path — the new owner does not need it to receive the data, and the old owner does not need it to release. Publishing at the send moves it earlier by exactly the acknowledgement latency and creates a window in which every lookup is misdirected.

The payoff test. At a measured 50-cycle transfer and 10 local hits afterwards, the transfer pays for itself if a remote access costs more than 5 cycles more than a local one. Below that threshold the migration is a net loss, and hits_per_xfer is the counter that says which side of it a workload sits on.

A deferred third-party read costs the remainder of the transfer. Measured at a 12-cycle mean stall in this model; on a design that defers for the whole transfer it is the full 50. Serving from whoever currently holds the line costs nothing and is always possible, because the line is always somewhere.

17. Assertions

Presented as SystemVerilog and executed as procedural checkers — see section 19.

The line is answerable in every phase of a transfer.

property p_always_answerable;
  @(posedge clk) disable iff (!rst_n)  (phase != IDLE) |-> responder_valid;
endproperty

The old owner answers until the acknowledgement.

property p_old_liable_until_ack;
  @(posedge clk) disable iff (!rst_n)  (phase == SETTLE) |-> old_still_liable;
endproperty

The directory never names an agent that cannot answer.

property p_published_can_answer;
  @(posedge clk) disable iff (!rst_n)
    (lookup && published == new_owner) |-> new_owner_has_data;
endproperty

A third-party read is always servable.

property p_third_party_servable;
  @(posedge clk) disable iff (!rst_n)  req |-> (serve_now || defer_req);
endproperty

Nobody is ever served by an agent with no data.

property p_no_empty_server;
  @(posedge clk) disable iff (!rst_n)
    (serve_now && server == new_owner) |-> data_with_new;
endproperty

Data is accepted only from the named sender.

property p_named_sender;
  @(posedge clk) disable iff (!rst_n)  accept |-> (data_sender == expected_sender);
endproperty

Somebody always owes the data during a transfer.

property p_someone_owes;
  @(posedge clk) disable iff (!rst_n)  xfer_live |-> (old_owes || new_owes);
endproperty

A failed transfer leaves the source holding the line.

property p_abort_to_source;
  @(posedge clk) disable iff (!rst_n)  aborted |-> (owner_after == old_owner);
endproperty

A queued transfer sources from where the line actually is.

property p_retargeted;
  @(posedge clk) disable iff (!rst_n)
    (queue_starts) |-> (new_source == previous_destination);
endproperty

A transfer completes on all three parts.

property p_three_parts;
  @(posedge clk) disable iff (!rst_n)
    complete |-> (got_data && got_ack && got_pub);
endproperty

Never two transfers of one line at once.

property p_serialised;
  @(posedge clk) disable iff (!rst_n)  !(busy && queue_running);
endproperty

A probe is never unanswerable during a transfer.

property p_no_unanswerable;
  @(posedge clk) disable iff (!rst_n)  (probe && stage != IDLE) |-> answerable;
endproperty

18. Mutation Testing

88 mutations were injected into the ten models, one at a time, each a single-line change a competent engineer could plausibly write. Every one must make the testbench print RESULT: FAIL.

Representative mutations, all killed:

Fifteen mutations survived the first run. None was patched away.

Eight stimulus gaps. The bench never held the flow in its settling phase, never held the assembled flow in preparation, flight or settling, never checked the lookup count, never checked the deferral count, never issued a second start while a transfer was running, and never drove a queued transfer whose recorded source differed from where the line would be.

Three unobserved outputs. The lookup total, the deferral total, and the peak overlap after the handoff completed — the last of which needed care, because the peak was first sampled during the overlap rather than after it, so a later shorter handoff appeared to reduce it when it had not.

Two unreachable checkers. served_by_empty_err needed a third build of third_party, ALWAYS_NEW, because both existing builds name whoever holds the data. This is the third consecutive chapter in which a monitor needed an extra parameter for exactly this reason, and the pattern is now clear: a monitor that watches for "answering from nothing" is unreachable in any design that computes the answerer correctly.

One dead monitor, fixed in the design. stale_source_err tested q_q && !busy_q — a combination the design can never produce, because the queue clears in the same cycle the next transfer starts. No stimulus and no fault-injection hook could reach it. The fix was in the condition: checking while the transfer is still queued, which is both reachable and the moment the information is actually useful. This is the same class as 13.4's three-bit range check — a monitor watching for something that cannot exist, as distinct from one watching for something a correct design does not do.

One provably equivalent mutation, fixed by removing a redundancy. The completion path cleared an abort flag that the start path already cleared, and no stimulus could make the second clear load-bearing — a transfer can only complete after a begin. The redundant assignment was removed and the mutation replaced with one on the clear that actually matters.

One defect found before any mutation ran. Two models shared a data_acked stimulus, and the directory reacted to a transfer it had never been told about — moving the published owner on somebody else's flow. Both publication paths are now gated on a transfer this directory started. That is a real design fix that the shared-stimulus discipline surfaced.

19. Verification Strategy

The oracle must not be the design. Each testbench models the same behaviour in a structurally different representation.

For xfer_flow the design holds a five-value phase register. The oracle holds a single "who can answer right now" integer, advanced by events, with no notion of a phase at all:

  integer o_answerer;   // -1 nobody, else the agent id
  task o_event(input integer ev);   // 0 start 1 sent 2 acked

Note which events move it: start and acked, and not sent. That is the overlap rule expressed as a reference rather than as an assertion, and a design that hands over at the send cannot agree with it.

For duty_handoff the design has one nobody_owes expression; the oracle holds two independent booleans and an "at least one" test.

For xfer_completion the design has one complete expression over three flags; the oracle holds three independent booleans and an explicit AND.

Every displayed value is a captured signal, latched into a named integer before any later event changes it. This chapter needed that twice: the peak overlap, which grows after it is first sampled, and the served-by count, which continues to rise after its assertion.

Delta-cycle discipline. Every sample of a combinational output is preceded by a settle. One check additionally needed an extra clock, because a released deferral is combinationally true in the cycle the transfer ends and the pending flag clears at the following edge.

Coverage recorded: 171 assertion sites across three testbenches; reads driven in all five handover phases on both builds; lookups driven before, during and after publication; third-party reads driven before and after the data landed, across three builds; data driven from the right sender, the wrong sender and via the directory; the duty driven through send, land and release with the faulty ordering compared on the same stimulus; a transfer aborted and one completed, with a second start attempted during each; two transfers queued with and without a stale source; completion driven with each part alone, all three, and mismatched tags in all three directions; and the assembled flow probed in every phase with each phase held.

20. Synthesis and Implementation Reality

The transfer state machine is per line in flight, not per line. A directory tracking millions of lines has a handful of transfer contexts, and the number of them bounds how many migrations can be in progress. Every structure in this chapter is replicated that many times and no more.

The overlap costs one bit. Both agents owing the line is one extra flag on the old owner, held for the acknowledgement latency. That is the entire cost of the property this whole chapter exists to preserve, and it is the cheapest correctness margin in the module.

The publication is a directory write, and its timing is a design decision rather than a constraint. Nothing on the critical path needs it early. Placing it after the acknowledgement costs the acknowledgement latency in staleness and removes the misdirected-lookup window entirely.

The queue is one entry deep here and deeper in production, and every entry needs retargeting when the one ahead of it completes. That retarget is a write to the source field of every queued entry, which at depth becomes a small broadcast — and a design that skips it queues requests that encode assumptions already invalidated.

unanswerable_err and nobody_owes_err are both a handful of gates and both belong in silicon. Each is unreachable in a correct design, so each needs its fault-injection build in the regression before a zero reading means anything.

Reset must leave no transfer in flight. A transfer context that comes out of reset in the settling phase will wait forever for an acknowledgement to a message that was never sent, and its line has an old owner that no longer believes it is liable.

21. Silicon Observability

Five error signals belong in silicon. gap_err, points_at_empty_err, served_by_empty_err, nobody_owes_err and partial_owner_err all detect states from which no correct behaviour is possible, and each is a handful of gates over signals that already exist.

hits_per_xfer is the counter that turns migration from a policy into a measurement. A system moving ownership aggressively and a system moving it well look identical in n_xfers; only the payoff distinguishes them. A falling value while transfers rise is a line ping-ponging, and the fix is in the migration policy rather than anywhere in this chapter.

max_overlap is a shrinking-margin detector. The overlap is what stands between a correct handover and a gap. A design whose peak overlap is falling over time — because acknowledgements are arriving faster relative to the release — is approaching the boundary, and the latched peak is the only thing that shows it before it is crossed.

22. Debug Lab

  handover : gaps correct=1 early-release=2 | old liable at settle=1 early=0

assign old_still_liable = (phase != IDLE) && (phase != DONE);   // through settling

  publish  : at send correct names 1 early build names 2 | misdirected correct=0 early=1

if (data_acked && moving_q) begin
  pub_q <= new_owner;          // publish only when the new owner can answer
end

  third pty: served by agent 1 mid-transfer | defer build deferred=1 starved=1 released=1

assign server    = data_with_new ? new_owner : old_owner;
assign serve_now = req && !defer_req;

  data path: wrong_sender=1 accept-any took it=1 via-home=1 accepted=2

assign accept = data_valid && armed_q && (data_sender == expected_sender);

  duty     : overlap=1 latched=3 | nobody-owes correct=0 drop-then-take=1
             drop-then-take gap cycles=2

if (data_landed)  new_q <= 1'b1;     // take first
if (old_released) old_q <= 1'b0;     // then drop

  abort    : owner after abort correct=1 abort-to-new=2 partial reported=1 orphan=0

if (abort) begin
  own_q  <= old_owner;      // roll back to the source
  live_q <= 1'b0;
end

  serialise: retarget source=1 (queue-as-issued keeps 0) overlap detected=1 stale source=1

from_q <= to_q;      // source from where the line actually is now

  complete : two-of-three declared early=1 partial=1 tag_mismatch=1

assign complete = live_q && got_data && got_ack && got_pub;

23. Design Review

What was built. Ten models: a five-phase handover with a release-on-send twin, a directory publisher with a publish-early twin, a third-party server in three builds, a data-path acceptor with an accept-any twin, a duty tracker with a drop-then-take twin, an abort path with a roll-forward twin, a serialiser with a queue-as-issued twin, a three-part completion with a two-of-three twin, a cost model with a payoff metric, and an assembled flow with a no-overlap twin.

What was measured. One gap in the correct handover — the deliberate pre-ownership one — against more in the early-release build, with all three phases held. A directory naming agent 1 at the send where the early build named agent 2, and one misdirected lookup against zero. A third-party read served immediately by the old owner, while the deferring build's request aged past starvation. Data rejected from an unnamed sender and accepted from the named one, with the routed-through-directory case counted rather than flagged. A 3-cycle latched overlap against 2 cycles of nobody owing the data in the faulty build. An abort leaving the source holding the line while the roll-forward build named a partial copy. A queued transfer retargeted to source from agent 1 where the queue-as-issued build kept agent 0. A completion requiring all three parts. 50-cycle transfers yielding 10 local hits each. Three cycles with no answerer in the no-overlap build.

What would be different in production. The transfer context count bounds concurrent migrations and every structure is replicated per context. The queue is deeper and the retarget becomes a small broadcast across its entries. The migration policy — the decision to move ownership at all — sits above everything here and is where hits_per_xfer actually gets used.

The strongest argument against this design. Keeping the old owner liable through the settling phase means it cannot free its transfer context until the acknowledgement returns, which on a long link is the dominant term in migration throughput. That argument is correct, and the honest response is that the alternative is not "release early" but "overlap on a cheaper structure": the liability is one flag, and it can outlive the full context. Releasing the context while retaining the flag keeps the margin and recovers most of the throughput.

What would be built differently next time. stale_source_err was written before the design it monitors was finished, and it ended up testing a state combination the design cannot produce. Writing a monitor against an intended behaviour rather than against the implemented one is how dead monitors get created — and this is the second chapter in this module to produce one. The discipline that catches it is asking, for every monitor, which stimulus makes this true, before the mutation suite asks the same question less politely.

24. How This Appears In Real Engineering

In a microarchitecture review, the question that separates a specified transfer from an aspirational one is when the old owner stops answering. "When it sends" and "when the acknowledgement returns" are one message apart in a diagram and a full round trip apart in silicon.

In bring-up, n_gap and n_misdirected are the two counters to watch. Both detect windows in which the line is unreachable or misdirected, both are silent everywhere else, and both are cheap if they were designed in.

In a performance investigation, hits_per_xfer is what distinguishes aggressive migration from good migration. A falling value while transfer counts rise is a line ping-ponging between two agents, and the fix is in the policy rather than in the flow.

In a verification plan review, ask whether the transfer has been probed in every phase. The settling phase is the one that separates a correct handover from an early release, and it is the phase a directed test spends the least time in.

In silicon debug, a lookup that reaches an empty cache presents as data corruption rather than as a lookup failure — the directory answered, the agent answered, and the answer was wrong. points_at_empty_err is the only thing that names it.

In a design review of somebody else's transfer path, ask what a failure leaves behind. If the answer is "the destination", the design is naming an agent that may hold half a line.

25. Common Misconceptions

"A handover is a moment." It is five phases, and the line must be answerable in all of them. Measured: the correct build answered reads in every phase; the early-release build could not answer during settling.

"Two agents owing one line is a two-owner failure." It is the overlap, and it is safe. 13.3's registry invariant is about who may claim the duty, not about how many agents are discharging it during a handover. Zero agents owing it is the failure.

"The directory should be updated as early as possible." Nothing on any critical path needs it early, and publishing at the send names a cache that is still empty. Measured: one misdirected lookup in the window.

"A third-party read has to wait for the migration." The line is always somewhere. Measured: served immediately by the old owner before the data landed and by the new one afterwards. Deferring is a policy choice whose cost falls on an uninvolved agent.

"The data can go via the directory." It can, and it doubles the transfer latency for no correctness benefit. That is why the model counts it rather than flagging it — it is a cost, not an error.

"A failed transfer should complete anyway." It must roll back to the source, which never relinquished anything. Naming the destination names an agent that may hold a partial copy.

"A queued transfer can be replayed as issued." Its source is whatever the transfer ahead of it leaves behind. Measured: retargeted to agent 1 where the naive build kept agent 0, which had let go.

"Data landing plus an acknowledgement means the transfer is done." The directory has to agree. Measured: the two-of-three build retired with the directory still naming the old owner — the two participants agreeing with each other and disagreeing with the system.

26. Interview Reasoning

27. Exercises

1. Calculation. A link has an 8-cycle acknowledgement latency and a system performs 500 migrations. Compute the total exposure a release-on-send design creates, and express it as a fraction of the total migration time at a 50-cycle mean transfer.

2. Analysis. A system reports 200 transfers with a hits_per_xfer of 3 and a mean transfer latency of 50 cycles. State whether the migrations are paying for themselves, what remote-access penalty would be required to break even, and what the counter would look like for a line that is ping-ponging.

3. RTL task. Extend xfer_flow to support cancelling a transfer during the flight phase, after the data has been sent but before it is acknowledged. State what the old owner must do, what the new owner must do with data that may already have arrived, and which monitor must change.

4. Assertion task. Write the property proving the line is answerable in every phase of a transfer. Then explain why it passes trivially on a design that derives answerable from the phase register alone, and what independent source of the answerer is required to make it meaningful.

5. Design task. Add a second transfer context so two lines can migrate concurrently. State what must be true about the mapping from line to context, and the failure that becomes possible if one line can reach both.

6. Testbench design. Design the stimulus that distinguishes a handover that overlaps from one that hands off at the send. Explain why any test with a zero-latency acknowledgement passes on both, and state the minimum stimulus that separates them.

7. Debug task. A directory lookup returns an agent that reports a miss, and the directory's publication logic looks correct in isolation. Give your investigation order, and name the guard whose absence lets an unrelated flow move the ownership record.

8. Design review. A colleague proposes routing transfer data through the directory so the destination need not validate the sender. Give the strongest version of that argument, then the latency cost, and the reason the sender check is cheaper than the routing.

28. Summary

Ownership moves by overlapping, not by handing off.

• The old owner answers until the acknowledgement, through four of the five phases. Measured: one gap in the correct build — the deliberate pre-ownership one — against more in the release-on-send build, with three cycles of no answerer on the trace.
• Two agents owing one line is safe; zero is the failure. The faulty ordering produced 2 cycles in which nobody owed the data, and the peak overlap was latched at 3.
• The directory publishes on the acknowledgement. At the send the correct directory still named agent 1 while the early build named agent 2 — which had nothing, producing one misdirected lookup.
• A third-party read is always servable, because the line is always somewhere. Served by the old owner before the data landed and the new one afterwards; the deferring build's request aged past starvation.
• The data goes cache to cache, so the destination must check the sender. Rejected from an unnamed agent, accepted from the named one, and the routed-through-directory case counted rather than flagged — it is a cost, not an error.
• A failed transfer rolls back to the source, which never relinquished anything. The roll-forward build named a destination holding a partial copy.
• A queued transfer is retargeted, not replayed. Sourced from agent 1 where the naive build kept agent 0, which had let go.
• A transfer completes on all three parts. The two-of-three build retired with the directory still naming the old owner — the participants agreeing with each other and disagreeing with the system.
• The payoff decides whether it was worth it. 50-cycle transfers yielding 10 local hits each; n_xfers alone cannot distinguish good migration from a line ping-ponging.
• Verification: 171 assertion sites, 88 of 88 mutations killed, zero surviving. Fifteen first-run escapes were eight stimulus gaps, three unobserved outputs, two unreachable checkers — one needing a third build — one dead monitor fixed in the design, and one equivalent mutation that exposed a redundant assignment. Plus one design defect found before any mutation ran, by the shared-stimulus discipline.

Next: 14.4 Cache-Interaction Flows, which puts a host cache and a device cache on opposite sides of a link and asks what each may do without asking the other.