Memory-Access Issues

26.3 was about a line whose value is wrong because coherency failed. This chapter is about a read that went somewhere else entirely — and the difference matters because the two produce the same complaint and need different instruments.

The completion status says the access finished. Eighty reads of a thousand were answered by a device that does not own the address, and every one of them completed normally.

1. The Engineering Problem — Completion Is Not Correctness

A read is answered by whatever claims its address. Eighty of a thousand answered by another device and twenty by nothing is eighty-two percent correct on a completion rate of ninety-eight. Section 5.

Host and device must compute the same device from the same address. A 256-byte host granule against a 1024-byte device one is seventy-five percent of blocks landing on the wrong device — and every one of those accesses completes. Section 6.

A device that cannot answer correctly can say so. Thirty-two of forty bad reads poisoned leaves eight returned as ordinary data, indistinguishable from correct ones. Section 7.

A partial write changes the bytes it names, or the line. Eight bytes enabled on a sixty-four-byte line, written whole, is fifty-six bytes of somebody else's data overwritten. Section 9.

This chapter against 26.3, stated precisely. That one owns a value made wrong by coherency. This one owns a value made wrong by routing, masking or signalling — which is why every model here is about where an access went and what it carried, and why section 14's weak definition is a completion status.

2. The One-Sentence Model

A memory access is sound when it completed, it went to the device that owns the address, host and device compute the same device, a device that cannot answer says so, a partial write changed only its own bytes, and no two windows claim one address — and "the read completed" is one of those six.

3. What This Chapter Owns

Deferred:

4. Teaching-Model Boundary

Every model takes one property of a memory access and computes what it costs when it is wrong. A real investigation has a host's decoder configuration, a device's window registers, an analyser on the link and a workload, and none of that is reproduced. What is reproduced is the arithmetic of routing and masking — how many accesses land in the wrong place, how many bytes a write touches, how far a poisoned line spreads.

Three simplifications are worth stating. Section 6 treats interleave disagreement as a uniform fraction where the real pattern is structured by address bits. Section 8 prices poison spread with a flat fanout. Section 12 treats corrected-error accumulation as linear in time. In each case the conclusion is the same and the model is abbreviated.

Each model is built twice — a correct build and a broken build selected by a parameter. Every broken build here is what a completion status supports: an answer is the right answer, granularity is a detail, silence is acceptable, poison stays put, a mask is advisory, first match wins, slow is a failure, corrected is clean, one address is like another. Each is a true statement about something, and none of them is about whether the data was right.

Figure 1 — Ninety-eight percent completed and eighty-two percent were answered by the right device. The sixteen-point gap is invisible to any counter that measures completions, which is every counter a host keeps by default.

5. RTL 1 — A Read Is Answered By Whatever Claims Its Address

// RTL 1 - address decode. A read is answered by whatever claims its address,
// and a window programmed wrongly is answered by the wrong thing.
module decode_window #(parameter int COMPLETION_IS_CORRECTNESS = 0) (
  input  logic clk, rst_n,
  input  logic        access,
  input  logic [15:0] reads, in_window, claimed_by_other, unclaimed,
  output logic [15:0] correct, misrouted, unanswered, correct_pct,
  output logic        decode_sound,
  output logic [7:0]  n_accesses, n_misrouted,
  output logic        completion_taken_as_correct_err
);
  logic [31:0] p_q;
  assign misrouted = (COMPLETION_IS_CORRECTNESS != 0) ? 16'd0 : claimed_by_other;
  assign unanswered = unclaimed;
  assign correct = (in_window > misrouted) ? (in_window - misrouted) : 16'd0;
  assign p_q = (reads == 16'd0) ? 32'd100
             : (({16'd0, correct} * 32'd100) / {16'd0, reads});
  assign correct_pct = (p_q > 32'd100) ? 16'd100 : p_q[15:0];
  assign decode_sound = (misrouted == 16'd0) && (unanswered == 16'd0);
  // Reads answered by a device that does not own the address.
  assign completion_taken_as_correct_err = access && (claimed_by_other != 16'd0)
                                           && (misrouted == 16'd0);

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_accesses <= 8'd0; n_misrouted <= 8'd0;
    end else if (access) begin
      n_accesses <= n_accesses + 8'd1;
      if (!decode_sound) n_misrouted <= n_misrouted + 8'd1;
    end
  end
endmodule

Five access sets. A thousand reads unless stated.

Three access sets with an unsound decode; two when a completion is taken as correctness.

A CXL memory read goes to whichever device's window claims the address, and a window programmed with the wrong base or size claims addresses it does not own. The read completes. It returns data — the wrong device's data, from the right offset within the wrong window — and it looks exactly like a correct read.

Row four is the failure that is easy to see and it is worth contrasting. Forty reads that nothing claims produce an error response or a timeout — visible, logged, and investigated. Eighty reads answered by the wrong device produce nothing at all. The louder failure is the less dangerous one.

Row three is the resolution. One misrouted read of a thousand is enough to call the decode unsound, and it is precisely the case that a completion-rate metric rounds away.

The two failure modes need different evidence, which is the practical consequence. An unanswered read is in the host's error log with its address; a misrouted read is in nobody's log at all, and finding it means comparing the address against the decoder configuration that should have claimed it. The first is a search through records that exist and the second is a search through records that have to be created — which is why section 20's first entry is a per-window comparison of reads issued against reads answered.

6. RTL 2 — Host And Device Must Agree On The Interleave

// RTL 2 - interleave agreement. Host and device must compute the same device
// from the same address, and a granularity mismatch sends every other block
// to the wrong place.
module interleave_match #(parameter int GRANULARITY_IS_A_DETAIL = 0) (
  input  logic clk, rst_n,
  input  logic        check_it,
  input  logic [15:0] blocks, host_gran, device_gran, ways,
  output logic [15:0] agreement, disagreement, wrong_pct, ratio,
  output logic        interleave_agrees,
  output logic [7:0]  n_checks, n_mismatched,
  output logic        granularity_ignored_err
);
  logic [31:0] p_q;
  // Where the granularities differ, the fraction of blocks that land on the
  // same device is one in the ratio between them.
  assign ratio = (host_gran == device_gran) ? 16'd1
               : ((host_gran > device_gran) ? (host_gran / ((device_gran == 16'd0)
                  ? 16'd1 : device_gran))
                  : (device_gran / ((host_gran == 16'd0) ? 16'd1 : host_gran)));
  assign agreement = (GRANULARITY_IS_A_DETAIL != 0) ? blocks
                   : ((ratio == 16'd0) ? blocks : (blocks / ratio));
  assign disagreement = (blocks > agreement) ? (blocks - agreement) : 16'd0;
  assign p_q = (blocks == 16'd0) ? 32'd0
             : (({16'd0, disagreement} * 32'd100) / {16'd0, blocks});
  assign wrong_pct = (p_q > 32'd100) ? 16'd100 : p_q[15:0];
  assign interleave_agrees = (disagreement == 16'd0);
  // A granularity mismatch reported as agreement.
  assign granularity_ignored_err = check_it && (host_gran != device_gran)
                                   && interleave_agrees;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_checks <= 8'd0; n_mismatched <= 8'd0;
    end else if (check_it) begin
      n_checks <= n_checks + 8'd1;
      if (!interleave_agrees) n_mismatched <= n_mismatched + 8'd1;
    end
  end
endmodule

Five checks. Eight hundred blocks unless stated.

Three checks mismatched; none when granularity is a detail.

Interleaving is a computation both ends perform independently on the same address, and they must produce the same device. A host striping at 256 bytes against a device expecting 1024 puts three of every four blocks on a device that is not the one the host thinks — and all four complete.

Row four is worth stating because it is symmetric. It does not matter which end is coarser: the ratio is the same and the failure is the same. The investigation that matters is comparing the two configured values, not deciding which is right.

Row three is the gentlest version and the hardest to see. A factor of two means half the blocks are correct, so a workload that touches a small region may be entirely fine and the same workload on a different region entirely wrong. The address dependence is what makes section 13's pattern analysis the right next step.

7. RTL 3 — A Device That Cannot Answer Can Say So

// RTL 3 - poison against silence. A device that cannot answer correctly can
// say so, and a device that returns data anyway cannot be distinguished from
// one that is right.
module poison_signal #(parameter int SILENT_IS_FINE = 0) (
  input  logic clk, rst_n,
  input  logic        respond,
  input  logic [15:0] bad_reads, poisoned, consumers_checking,
  output logic [15:0] flagged, silent, detectable, undetectable,
  output logic        failures_visible,
  output logic [7:0]  n_responses, n_silent,
  output logic        silent_corruption_err
);
  assign flagged = (SILENT_IS_FINE != 0) ? 16'd0
                 : ((poisoned > bad_reads) ? bad_reads : poisoned);
  assign silent = (bad_reads > flagged) ? (bad_reads - flagged) : 16'd0;
  // A flagged failure is detectable only by a consumer that checks.
  assign detectable = (consumers_checking > flagged) ? flagged : consumers_checking;
  assign undetectable = (bad_reads > detectable) ? (bad_reads - detectable) : 16'd0;
  assign failures_visible = (silent == 16'd0);
  // Bad reads returned as ordinary data.
  assign silent_corruption_err = respond && (bad_reads != 16'd0)
                                 && (flagged == 16'd0);

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_responses <= 8'd0; n_silent <= 8'd0;
    end else if (respond) begin
      n_responses <= n_responses + 8'd1;
      if (!failures_visible) n_silent <= n_silent + 8'd1;
    end
  end
endmodule

Five response sets. Forty bad reads unless stated.

One response set with a silent failure; four when silence is accepted.

Poison is the mechanism by which a device says "this data is not right" without failing the transaction, and it is the difference between a detectable corruption and an undetectable one. Eight silent reads of forty is eight values that enter a computation looking exactly like data.

Row four separates the two halves and it matters. Forty reads flagged and nobody checking the flag gives forty undetectable failures — the device did everything right. Signalling and consumption are different problems with different owners, and a model that reported this as a device fault would be sending the wrong engineer.

Row one is the arithmetic of partial signalling. Thirty-two flagged and twelve consumers checking gives twelve detectable of forty bad reads — the two mechanisms multiply rather than add, which is why improving either one alone has limited return.

8. RTL 4 — Poison That Is Consumed Rather Than Contained

// RTL 4 - a poisoned line spreads. Poison that is consumed rather than
// contained propagates into everything computed from it.
module poison_spread #(parameter int POISON_STAYS_PUT = 0) (
  input  logic clk, rst_n,
  input  logic        assess,
  input  logic [15:0] poisoned_lines, consumers, fanout, containment_pct,
  output logic [15:0] contained, escaped, downstream, spread_ratio,
  output logic        poison_contained,
  output logic [7:0]  n_assessments, n_spreading,
  output logic        spread_ignored_err
);
  logic [31:0] c_q, d_q;
  assign c_q = ({16'd0, poisoned_lines} * {16'd0, containment_pct}) / 32'd100;
  assign contained = (POISON_STAYS_PUT != 0) ? poisoned_lines
                   : ((c_q > 32'd65535) ? 16'hFFFF : c_q[15:0]);
  assign escaped = (poisoned_lines > contained)
                   ? (poisoned_lines - contained) : 16'd0;
  assign d_q = {16'd0, escaped} * {16'd0, fanout};
  assign downstream = (d_q > 32'd65535) ? 16'hFFFF : d_q[15:0];
  assign spread_ratio = (poisoned_lines == 16'd0) ? 16'd0
                      : (downstream / poisoned_lines);
  assign poison_contained = (escaped == 16'd0);
  // Poison that reached a consumer, reported as contained.
  assign spread_ignored_err = assess && (containment_pct < 16'd100)
                              && (poisoned_lines != 16'd0) && poison_contained;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_assessments <= 8'd0; n_spreading <= 8'd0;
    end else if (assess) begin
      n_assessments <= n_assessments + 8'd1;
      if (!poison_contained) n_spreading <= n_spreading + 8'd1;
    end
  end
endmodule

Five assessments. Sixty poisoned lines, a fanout of twelve.

Three assessments spreading; none when poison is assumed to stay put.

A poisoned line that reaches a computation contaminates everything computed from it, and the fanout decides how far. Fifteen escaped lines at a fanout of twelve is a hundred and eighty downstream values — three for every poisoned line in the original set.

Row four is the sensitivity and it is the argument for containment. Ninety-nine percent containment still leaves twelve downstream values from one escaped line. Poison spread is multiplicative, so the last percent of containment is worth as much as the first ninety.

Row three is the case where containment was never attempted. Sixty escaped at a fanout of twelve is seven hundred and twenty values, and the ratio of twelve-to-one is what makes a containment mechanism worth its cost.

9. RTL 5 — A Partial Write Changes Its Bytes Or The Line

// RTL 5 - partial writes. A write with byte enables changes the bytes it
// names, and a device that ignores them changes the whole line.
module write_mask #(parameter int MASK_IS_ADVISORY = 0) (
  input  logic clk, rst_n,
  input  logic        write_it,
  input  logic [15:0] writes, line_bytes, enabled_bytes, neighbours_live,
  output logic [15:0] changed, should_change, clobbered, correct_pct,
  output logic        mask_honoured,
  output logic [7:0]  n_writes, n_clobbering,
  output logic        mask_ignored_err
);
  logic [31:0] p_q;
  assign should_change = enabled_bytes;
  assign changed = (MASK_IS_ADVISORY != 0) ? line_bytes : enabled_bytes;
  assign clobbered = (changed > should_change) ? (changed - should_change) : 16'd0;
  assign p_q = (line_bytes == 16'd0) ? 32'd100
             : ((({16'd0, line_bytes} - {16'd0, clobbered}) * 32'd100)
                / {16'd0, line_bytes});
  assign correct_pct = (p_q > 32'd100) ? 16'd100 : p_q[15:0];
  assign mask_honoured = (clobbered == 16'd0);
  // Bytes outside the mask overwritten, on a write reported as correct.
  assign mask_ignored_err = write_it && (clobbered != 16'd0)
                            && (neighbours_live != 16'd0);

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_writes <= 8'd0; n_clobbering <= 8'd0;
    end else if (write_it) begin
      n_writes <= n_writes + 8'd1;
      if (!mask_honoured) n_clobbering <= n_clobbering + 8'd1;
    end
  end
endmodule

Five write sets. A 64-byte line unless stated.

The mask model never clobbers; the advisory view clobbers on three of five.

Byte enables are not a hint. A write with eight of sixty-four bytes enabled changes eight bytes, and a device that writes the whole line overwrites fifty-six bytes belonging to whatever else shares that line — a different structure, a different thread's data, a lock.

Row four is the exemption and it is why the error signal needs the neighbour input. Fifty-six bytes clobbered with nothing live in them is a correctness violation with no consequence, and the model reports the clobber without raising the error. A defect and a loss are different facts.

Row three is the smallest instance, and it is the one that survives testing: sixty-three of sixty-four bytes enabled looks like a full-line write in every test that writes full lines.

Figure 3 — Both writes complete and both report success. The fifty-six bytes on the bottom right belonged to something else — another field, another thread's data, a lock — and nothing in the write path knows or reports that they changed.

This is the failure that most resembles a performance optimisation. Coalescing a partial write into a full-line write is a real and legitimate technique when the device holds the rest of the line, and the defect is doing it when it does not. A device that read-modify-writes correctly and one that simply writes the line are indistinguishable except by a test that puts a known pattern in the untouched bytes and checks it afterwards — which is section 19's point and is a test rather than an inspection.

10. RTL 6 — Two Windows Claiming One Address

// RTL 6 - overlapping windows. Two decoders that both claim an address answer
// the same read differently depending on which one wins.
module window_overlap #(parameter int FIRST_MATCH_WINS = 0) (
  input  logic clk, rst_n,
  input  logic        program_it,
  input  logic [15:0] windows, overlapping_pairs, addresses_k, overlap_k,
  output logic [15:0] ambiguous_k, unique_k, ambiguous_pct, conflicts,
  output logic        map_unambiguous,
  output logic [7:0]  n_programmings, n_conflicting,
  output logic        overlap_ignored_err
);
  logic [31:0] p_q;
  // A first-match rule produces an answer; it does not remove the ambiguity.
  assign ambiguous_k = (FIRST_MATCH_WINS != 0) ? 16'd0 : overlap_k;
  assign unique_k = (addresses_k > ambiguous_k)
                    ? (addresses_k - ambiguous_k) : 16'd0;
  assign p_q = (addresses_k == 16'd0) ? 32'd0
             : (({16'd0, ambiguous_k} * 32'd100) / {16'd0, addresses_k});
  assign ambiguous_pct = (p_q > 32'd100) ? 16'd100 : p_q[15:0];
  assign conflicts = overlapping_pairs;
  assign map_unambiguous = (ambiguous_k == 16'd0);
  // Overlapping windows, reported as an unambiguous map.
  assign overlap_ignored_err = program_it && (overlap_k != 16'd0)
                               && map_unambiguous;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_programmings <= 8'd0; n_conflicting <= 8'd0;
    end else if (program_it) begin
      n_programmings <= n_programmings + 8'd1;
      if (!map_unambiguous) n_conflicting <= n_conflicting + 8'd1;
    end
  end
endmodule

Five programmings. 4,096k addresses unless stated.

Three programmings with a conflict; none when first match wins.

A decoder with a first-match rule always produces an answer, and the answer is not the question. Two windows claiming 512k of addresses means those addresses have two owners — the read goes to whichever the decoder checks first, and the other device's copy of that memory is simply unreachable.

Row one is why this survives configuration. The system works. Twelve percent of the address space is silently owned by one device instead of two, and the symptom is a device that appears smaller than it is, or data written to one address readable at another.

Row three is the resolution. One kilobyte of overlap is enough to break the map, and a first-match rule makes it produce a consistent, wrong answer — which is harder to find than an inconsistent one.

Row four is the configuration that is obviously broken and rarely happens. Every address claimed twice produces a system where one device is entirely invisible, which fails at boot and gets fixed in an hour. The dangerous version is row one: twelve percent, in a region nothing touches during bring-up, discovered when a workload grows into it months later.

11. RTL 7 — Slow Is Not Wrong

// RTL 7 - slow is not wrong. An access that takes far longer than the others
// is a different investigation from one that returns the wrong value.
module slow_or_wrong #(parameter int SLOW_IS_A_FAILURE = 0) (
  input  logic clk, rst_n,
  input  logic        classify,
  input  logic [15:0] complaints, wrong_data, over_budget, budget_ns,
  output logic [15:0] correctness_cases, performance_cases, misfiled, split_pct,
  output logic        cases_separated,
  output logic [7:0]  n_classifications, n_mixed,
  output logic        misfiled_err
);
  logic [31:0] p_q;
  assign correctness_cases = wrong_data;
  assign performance_cases = (SLOW_IS_A_FAILURE != 0) ? 16'd0 : over_budget;
  // Slow cases filed as correctness bugs go to the wrong engineer.
  assign misfiled = (SLOW_IS_A_FAILURE != 0) ? over_budget : 16'd0;
  assign p_q = (complaints == 16'd0) ? 32'd0
             : (({16'd0, correctness_cases} * 32'd100) / {16'd0, complaints});
  assign split_pct = (p_q > 32'd100) ? 16'd100 : p_q[15:0];
  assign cases_separated = (misfiled == 16'd0);
  assign misfiled_err = classify && (misfiled != 16'd0);

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_classifications <= 8'd0; n_mixed <= 8'd0;
    end else if (classify) begin
      n_classifications <= n_classifications + 8'd1;
      if (!cases_separated) n_mixed <= n_mixed + 8'd1;
    end
  end
endmodule

Five classifications. Forty complaints unless stated.

The separated model never mixes them; the slow-is-a-failure view mixes three of five.

A memory access that is slow and one that is wrong go to different engineers. Twenty-eight slow accesses filed as correctness bugs are twenty-eight investigations looking for a data error that is not there — and the actual latency outlier is nobody's ticket.

Row three is the pure case and it is common. Every complaint slow and none wrong: the correctness share is zero, and an investigation that started from "memory is returning bad data" has not begun looking at the right thing.

Row one is the split that makes the triage possible, and it needs one question asked of each complaint: was the value wrong, or was it late? That question costs nothing and is skipped whenever the complaint arrives as "memory problems."

12. RTL 8 — Corrected Errors Accumulate Between Scrubs

// RTL 8 - corrected memory errors and the scrub. A single-bit error corrected
// on read is free; two in one word are not, and the scrub rate decides how
// often the second happens.
module scrub_rate #(parameter int CORRECTED_IS_CLEAN = 0) (
  input  logic clk, rst_n,
  input  logic        judge,
  input  logic [15:0] ce_per_day, scrub_hours, words_k, accumulation_hours,
  output logic [15:0] accumulated, ue_risk_ppm, scrub_period_h, headroom_h,
  output logic        scrub_adequate,
  output logic [7:0]  n_judgements, n_marginal,
  output logic        corrected_ignored_err
);
  logic [31:0] a_q, r_q;
  // Errors accumulate between scrubs; a second in the same word is fatal.
  assign a_q = (CORRECTED_IS_CLEAN != 0) ? 32'd0
             : (({16'd0, ce_per_day} * {16'd0, accumulation_hours}) / 32'd24);
  assign accumulated = (a_q > 32'd65535) ? 16'hFFFF : a_q[15:0];
  assign r_q = (words_k == 16'd0) ? 32'd0
             : (({16'd0, accumulated} * 32'd1000) / {16'd0, words_k});
  assign ue_risk_ppm = (r_q > 32'd65535) ? 16'hFFFF : r_q[15:0];
  assign scrub_period_h = scrub_hours;
  assign headroom_h = (accumulation_hours > scrub_hours)
                      ? (accumulation_hours - scrub_hours) : 16'd0;
  assign scrub_adequate = (scrub_hours <= accumulation_hours)
                          && (accumulated == 16'd0 || scrub_hours != 16'd0);
  // Corrected errors accumulating between scrubs, reported as clean.
  assign corrected_ignored_err = judge && (ce_per_day != 16'd0)
                                 && (accumulated == 16'd0);

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_judgements <= 8'd0; n_marginal <= 8'd0;
    end else if (judge) begin
      n_judgements <= n_judgements + 8'd1;
      if (!scrub_adequate) n_marginal <= n_marginal + 8'd1;
    end
  end
endmodule

Five judgements. Twelve corrected errors a day, 100k words.

One judgement with an inadequate scrub — the same one either way, because the scrub period is a fact.

A single-bit error corrected on read is free and a second in the same word is not, so the question is how long errors are allowed to accumulate. Twelve a day over forty-eight hours is twenty-four sitting in memory, and the scrub period decides whether a second one lands in the same word before the first is cleaned.

Rows two and three are the policy boundary. A scrub exactly as long as the accumulation window is adequate; one longer is not, and the two differ by whether the errors are ever removed faster than they arrive.

Row five is the missing denominator. Twenty-four accumulated errors with no word count to express them against is a number with no meaning — the rate is per-word, and the model reports the count without inventing the rate.

13. RTL 9 — Which Addresses Fail Is The Cheapest Signal

// RTL 9 - narrowing by address. Which addresses fail is the cheapest signal
// a memory investigation has, and a pattern names the decoder that produced it.
module address_pattern #(parameter int ANY_ADDRESS_IS_THE_SAME = 0) (
  input  logic clk, rst_n,
  input  logic        analyse,
  input  logic [15:0] failing_addrs, sharing_bit, candidates_all, ranges,
  output logic [15:0] pattern_strength, candidates, ruled_out, strength_pct,
  output logic        pattern_found,
  output logic [7:0]  n_analyses, n_unpatterned,
  output logic        pattern_unused_err
);
  logic [31:0] p_q;
  // A bit shared by every failing address names a decoder stage.
  assign pattern_strength = (ANY_ADDRESS_IS_THE_SAME != 0) ? 16'd0
                          : ((sharing_bit > failing_addrs) ? failing_addrs
                             : sharing_bit);
  assign p_q = (failing_addrs == 16'd0) ? 32'd0
             : (({16'd0, pattern_strength} * 32'd100) / {16'd0, failing_addrs});
  assign strength_pct = (p_q > 32'd100) ? 16'd100 : p_q[15:0];
  // A pattern that holds for every failing address rules out the ranges that
  // do not have the bit.
  assign ruled_out = (strength_pct >= 16'd100) ? ranges : 16'd0;
  assign candidates = (candidates_all > ruled_out)
                      ? (candidates_all - ruled_out) : 16'd0;
  // No guard on failing_addrs is needed: strength_pct is zero whenever there
  // are no failing addresses, so it cannot reach a hundred.
  assign pattern_found = (strength_pct >= 16'd100);
  // Every failing address sharing a bit, and nothing concluded from it.
  assign pattern_unused_err = analyse && (failing_addrs != 16'd0)
                              && (sharing_bit >= failing_addrs) && !pattern_found;

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_analyses <= 8'd0; n_unpatterned <= 8'd0;
    end else if (analyse) begin
      n_analyses <= n_analyses + 8'd1;
      if (!pattern_found) n_unpatterned <= n_unpatterned + 8'd1;
    end
  end
endmodule

Five analyses. Twenty failing addresses, twelve candidate causes, eight ranges.

Three analyses with no pattern; all five when the addresses are ignored.

The list of failing addresses is free and it names a decoder stage. If every failing address has a particular bit set, the decoder stage that uses that bit is where to look — eight ranges ruled out and eight candidates of twelve gone, for the price of a bitwise comparison.

Row two is why the pattern must be total. Nineteen of twenty sharing a bit rules out nothing — one counterexample means the bit is not the discriminator, and a nineteen-twentieths pattern is a coincidence with a good story. The model requires a hundred percent and reports ninety-five as no pattern.

Row four is the honest negative. No shared bit means the addresses have nothing in common, and the model says so rather than finding a weaker pattern. That is a real outcome — a fault that is not address-dependent — and it is worth knowing in one comparison.

14. RTL 10 — A Memory-Access Diagnosis Assembled

// RTL 10 - a memory-access diagnosis assembled. Everything that must hold
// before "the read completed" is a claim about where it went and what it
// carried.
module access_signoff #(parameter int IT_COMPLETED = 0) (
  input  logic clk, rst_n,
  input  logic       evaluate,
  input  logic       completed,        // the access returned without a timeout
  input  logic       decode_sound,     // it went to the device that owns the address
  input  logic       interleave_agrees,// host and device compute the same device
  input  logic       failures_visible, // a device that cannot answer says so
  input  logic       mask_honoured,    // partial writes changed only their bytes
  input  logic       map_unambiguous,  // no two windows claim one address
  output logic       access_sound,
  output logic [5:0] fail_mask,
  output logic [7:0] n_eval, n_sound,
  output logic       false_completion_err
);
  assign fail_mask[0] = ~completed;
  assign fail_mask[1] = ~decode_sound;
  assign fail_mask[2] = ~interleave_agrees;
  assign fail_mask[3] = ~failures_visible;
  assign fail_mask[4] = ~mask_honoured;
  assign fail_mask[5] = ~map_unambiguous;
  // The it-completed build is what a completion status reports.
  assign access_sound = (IT_COMPLETED != 0) ? completed : (fail_mask == 6'd0);
  assign false_completion_err = evaluate && access_sound && (fail_mask != 6'd0);

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_eval <= 8'd0; n_sound <= 8'd0;
    end else if (evaluate) begin
      n_eval <= n_eval + 8'd1;
      if (access_sound) n_sound <= n_sound + 8'd1;
    end
  end
endmodule

Seven configurations.

One configuration sound under the full diagnosis; six under the completion status.

Row four is the one that makes the point without argument. Bad reads returned as ordinary data complete normally because completing normally is what returning data means — the completion status is measuring the transaction, and the transaction was fine.

Rows two, three and six are three routes to the same symptom. A mis-programmed window, a granularity mismatch and an overlap all produce a read answered by the wrong device, and they are distinguished by which configuration you compare against which. Section 13's address pattern is what separates them cheaply.

These six are not 26.3's six. That chapter's failures all end with a value that was correct somewhere and stale here; these end with a value that was never the right one to return — and the two produce the same complaint from a user.

Figure 4 — Routing is asked first because three of the six failures are routing and the address pattern separates them in one comparison. Every exit above is an access that completed.

15. Quantitative Reasoning

Decode. Eighty of a thousand reads answered by another device and twenty by nothing is 82% correct on a 98% completion rate — a sixteen-point gap no completion counter can see.

Interleave. A 256-byte host granule against a 1024-byte device one is a ratio of four and 75% of blocks on the wrong device; a factor of two is 50%.

Poison. Thirty-two of forty bad reads flagged with twelve consumers checking is twelve detectable and twenty-eight not — the two mechanisms multiply.

Spread. Fifteen escaped lines at a fanout of twelve is a hundred and eighty downstream values; even 99% containment leaves twelve.

Write masks. Eight bytes enabled on a 64-byte line written whole is fifty-six bytes clobbered — 12% correct.

Overlap. 512k of 4,096k claimed by two windows is twelve percent of the map ambiguous, answered consistently and wrongly by a first-match rule.

Filing. Twelve wrong-data and twenty-eight over-budget complaints is a 30% correctness share — and calling slow a failure misfiles twenty-eight.

Scrub. Twelve corrected errors a day over a 48-hour window is twenty-four accumulated and 240 parts per million of words at risk.

Address pattern. Twenty failing addresses all sharing a bit rules out eight ranges and eight of twelve candidates; nineteen of twenty rules out nothing.

The assembled diagnosis. Six properties, seven configurations, one sound. The completion status called six sound.

16. Assertions

Every check is an explicit comparison against an exact value. Icarus Verilog 13.0 has no concurrent assertion support, so each is a procedural comparison against 1'b1, and every one is an equality.

Three scripted checks ran before the campaign. The mutual-exclusivity check reported clean on all ten models. The output-listing step reported twenty unasserted nets and three were real — a correct-percentage, an undetectable count and a unique-address count, all three the broken build's own claim. And every code block is generated from its verified source file, so the chapter and the simulation cannot disagree.

Alongside those: every inclusive threshold at exactly equal, every clamp driven past its cap, and both builds asserted on every degenerate case.

Decode. Unanswered reads are driven separately from misrouted ones, because they are different failures reaching the same boolean.

chk(dGu == 16'd40, "forty unanswered");
chk(dGk == 1'b0,   "which is also not sound");

Interleave. The granularity mismatch is driven in both directions, host-coarser and device-coarser, to show the ratio is symmetric.

Poison. Nobody checking the flag is driven — the configuration where the device did everything right and the failures are still undetectable.

chk(pGd == 16'd0,  "nothing detectable");
chk(pGv == 1'b1,   "though the failures were flagged");

Spread. Ninety-nine percent containment is driven, where one escaped line is twelve downstream values.

Write masks. A clobber with no live neighbour is driven, where the mask is broken and nothing was lost — the case the error signal must not fire on.

Overlap. One kilobyte of overlap and a fully-overlapping map are both driven.

Filing. Every complaint slow, and every complaint wrong, are both driven.

Scrub. A scrub period exactly equal to the accumulation window is driven, and a judgement with no word count where the accumulation is real and the rate is not expressible.

Address pattern. Nineteen of twenty sharing a bit is driven — the near-pattern that rules out nothing — and more sharing the bit than there are failures, exercising the clamp.

The assembled diagnosis. Every fail mask is asserted as an exact six-bit value, and each of the six bits is driven false alone.

Totals: 290 checks across two testbenches, 149 on the front five models and 141 on the back five, all passing on the unmutated sources.

17. Mutation Testing

Seventy-three mutations were injected one at a time. 73 injected, 73 killed, after one survivor and three anchors that matched nothing.

The one survivor was a dominated guard: section 13's pattern_found required a non-zero failing-address count alongside a hundred-percent strength, and the strength is zero whenever there are no failing addresses. It was deleted with the dominator named and the mutation replaced with one on the threshold.

Three anchors matched nothing, and all three were self-inflicted. Four mutations in this chapter needed whole-statement replacements — a parameter-selected ternary cannot be mutated by changing one branch without leaving the other behind — and rewriting those four invalidated three other anchors that referenced the same text. The harness reported ANCHOR x0 on each rather than counting them, which is the behaviour batch 024 established; without it the score would have read 73 of 73 while injecting 70.

18. Verification Strategy

What a testbench for a routing model must cover.

Re-check every anchor after editing a mutation's target. Three failures here, all from rewriting statements that other anchors referenced. A mutation list is coupled to the source text, and editing one entry can silently invalidate another.

Drive the defect with and without its consequence. §9's clobber with no live neighbour is a correctness violation that lost nothing — the error signal must distinguish the fault from the loss, and only a case with both halves separated proves it does.

Drive the near-miss, not just the hit and the absence. §13's nineteen-of-twenty is neither a pattern nor no pattern; it is the configuration that tests whether the threshold is really total, and a testbench with only 100% and 0% would pass a 90% threshold.

Drive a rate with no denominator. §12's word count at zero leaves a real accumulation and no rate — the model must report the count without inventing the rate, and the case is easy to omit.

The cases where the completion is enough. Every read to its owner. Matching granularities. Every bad read flagged. Full containment. A full-line write. No overlap. Every complaint wrong. No corrected errors. Eight exemptions across nine models.

Counters as a second signature. Ten models, ten pairs of totals, differing in nine. The tenth is §12's marginal count, deliberately equal because the scrub period is a configured fact rather than a reading.

What a real investigation needs that these models do not have. An address-structured interleave model for §6, a per-consumer fanout for §8, and a non-linear accumulation for §12. All three are abbreviations that preserve the conclusion, and section 26 exercises 2, 4 and 8 are where they come back.

19. Synthesis and Implementation Reality

The host's decoder configuration and the device's window registers are two places, and section 5's failure is a disagreement between them. Reading both and comparing is a ten-minute job that is rarely the first thing done, because the symptom points at data rather than at configuration.

Interleave parameters are programmed at enumeration and never re-read. Section 6's mismatch is two numbers in two registers, and the check is a comparison nobody automates.

Poison is an encoding on the completion, so a device that supports it and a device that does not are distinguishable only by testing with a known-bad line. Section 7's silent fraction is a device capability, and it belongs in a bring-up checklist.

Byte enables travel with the write and a device may legally coalesce, which makes section 9's failure look like an optimisation. The test is a partial write to a line whose other bytes hold a known pattern — and it is a test, not an inspection.

Window overlap is detectable at configuration time by sorting the windows and checking adjacency. Section 10's failure is prevented by a loop over the decoder table, run once at boot, and almost nothing runs it.

Corrected-error counters and the scrub period are both in the device, and section 12 compares them. The comparison is arithmetic on two exported numbers and produces a yes-or-no about whether errors are removed faster than they arrive.

20. Silicon Observability

"Decoder windows sorted and checked for overlap at boot" is the one that prevents rather than detects. Every other entry helps after a wrong value appears. This one runs once, costs a sort, and makes section 10 impossible — and it is absent from most platforms because nothing in the specification requires it.

21. Debug Lab

Symptom. A CXL type-3 memory expander in a database server returns wrong data on roughly one read in a hundred thousand, always in a particular table's address range. The device reports no errors. The host reports no errors. Every access completes.

Step 1 — wrong, or slow? Section 11. The complaints are checked: twelve are wrong values and twenty-eight are latency outliers that had been filed together. The twenty-eight go to a different investigation, and the correctness question is now about twelve.

Step 2 — which addresses? Section 13. The twelve failing addresses are dumped and their common bits computed. All twelve share bit 18 set — a total pattern, which rules out eight of twelve candidate causes and points at a decoder stage that uses that bit.

Step 3 — which decoder. Section 5. The host's decoder configuration and the device's window registers are read and compared for the first time. They agree on base and size. That removes the mis-programmed-window branch and leaves the interleave.

Step 4 — the interleave. Section 6. The host is configured for a 256-byte interleave granule; the device reports 512. A factor of two, which puts half the blocks on the wrong device — and bit 18 is exactly the bit the two configurations disagree about.

Step 5 — why only one table. Half the blocks land wrongly, but most of the address space is backed by a single device in this configuration, so a wrong device selection resolves to the same physical memory for most ranges. The table in question straddles the one boundary where it does not.

Step 6 — why no error. Section 7. The wrong device answers, correctly, from its own memory at that offset. There is nothing to poison — the access is well-formed, the device owns the address it was given, and the data it returns is genuinely what it holds. Every layer behaved correctly on the information it had.

The finding. A one-bit interleave granularity mismatch between two configuration registers, found by comparing two numbers after an address pattern named the bit — and reached in three observations, two of which were free.

The fix. Correct the host's interleave granule. For the platform: log both ends' interleave parameters at boot and compare them (section 20), because the two registers are the whole bug and nothing reads them together. And separate latency complaints from correctness complaints at intake, because twenty-eight of forty had been in the wrong queue from the start.

What made this hard. Nothing was broken and nothing was reported. Every access completed, every device answered from memory it owns, and the only evidence was twelve addresses that shared a bit.

22. Design Review

1. Were the complaints separated into wrong values and slow accesses? Twelve and twenty-eight, and only one is a correctness problem. Section 11.

2. What do the failing addresses have in common? The cheapest narrowing available, and it names a decoder stage. Sections 13 and 20.

3. Do the host's decoder configuration and the device's window registers agree? Two places, ten minutes. Sections 5 and 19.

4. Do both ends report the same interleave granularity and ways? Two numbers nobody compares. Sections 6 and 20.

5. Does the device issue poison responses, and does anything consume them? Two mechanisms that multiply. Section 7.

6. Are the decoder windows checked for overlap at boot? A sort and a loop, run once, and it prevents rather than detects. Sections 10 and 20.

7. Does a partial write to a line with a known pattern leave the other bytes intact? A test, not an inspection. Sections 9 and 19.

8. What is the corrected-error rate against the scrub period? Two exported numbers whose ratio is the answer. Section 12.

9. What is the completion rate, and what is the correct-answer rate? They differ by sixteen points in §5 and no default counter shows it. Section 5.

10. What does "the access completed" establish? Section 14 exists because the answer is the first property only.

23. How This Appears In Real Engineering

A platform engineer owns both halves of every configuration comparison in this chapter — decoder windows, interleave parameters, overlap. Three of the six failures are two registers disagreeing, and nothing reads them together.

A device engineer owns poison signalling and write masking, and both are correct-by-construction or not at all. Section 7's silent fraction is a capability, and section 9's mask is a test that has to be written.

A support engineer receives "memory problems" and files them as one queue. Section 11 is the intake question — was it wrong or was it late — and it costs nothing and is skipped.

A database or application engineer sees a wrong value and has no way to tell a coherency bug from a routing bug. 26.3 and this chapter produce identical complaints, and the separating observation is the address pattern.

24. Common Misconceptions

"The access completed, so the address was valid." It was valid for whatever answered it (section 5). Eighty reads of a thousand answered by the wrong device complete perfectly and return that device's data.

"Both ends are configured for the same memory, so the interleave is fine." Base and size can agree while granularity does not (section 6) — a factor of two puts half the blocks on the wrong device, and both configurations look correct in isolation.

"There were no poison responses, so the data was good." Or the device does not issue them (section 7). Eight silent bad reads of forty are indistinguishable from correct ones, and the difference is a device capability.

"The write only touched its own bytes." If the device honoured the mask. Eight bytes enabled and sixty-four written is fifty-six bytes of a neighbour's data (section 9), and it looks like a coalescing optimisation.

"First match wins, so the decode is deterministic." Deterministic and wrong (section 10). 512k of addresses have two owners, and one of them is unreachable.

"Memory is returning bad data." Twelve of forty complaints were wrong values; twenty-eight were slow (section 11), and they are a different investigation with a different owner.

25. Interview Reasoning

"A CXL memory read returns wrong data and completes normally. Where do you start?" The failing addresses. A common bit names a decoder stage and rules out most of the cause set for the price of a bitwise AND — and it is free, unlike anything involving an analyser.

"The host and device agree on the memory window's base and size. Is the decode correct?" Not yet — the interleave granularity and ways are separate parameters (section 6), and a mismatch there puts blocks on the wrong device while base and size look right.

"Why is a misrouted read more dangerous than an unclaimed one?" The unclaimed read produces an error response and gets investigated. The misrouted one completes and returns data (section 5) — the louder failure is the safer one.

"What does a poison response buy you?" The difference between a detectable corruption and an undetectable one — and only if something consumes it (section 7). The follow-up worth asking back: does anything in this stack check?

"Forty people report memory problems. What is your first question?" Was the value wrong, or was it late? Twelve and twenty-eight (section 11) — and the two go to different engineers, so asking it at intake is worth more than any later analysis.

26. Exercises

1. Compare the two configurations. List every decoder parameter on the host side and its counterpart on the device, and write the boot-time comparison that would have caught §21.

2. Model the interleave by address bits. §6 uses a uniform fraction. Work out which address bits select the device under each granularity and show exactly which blocks land wrongly under a factor-of-two mismatch.

3. Test the write mask. Design the partial-write test that distinguishes a device honouring byte enables from one coalescing, and say what pattern the untouched bytes must hold.

4. Price poison containment. §8 uses a flat fanout. Model a per-consumer fanout and find the containment percentage at which the downstream count falls below the poisoned-line count.

5. Check for overlap. Write the boot-time window-overlap check (§10) — sort, compare adjacent, report — and state what it does when it finds one.

6. Separate the queues. Design the intake question and the two queues for §11, and estimate the misfiling rate without it from your own bug tracker.

7. Find the pattern. Given twenty failing addresses, compute every bit shared by all of them and say how many decoder stages each shared bit implicates.

8. Model accumulation properly. §12 is linear. Add a birthday-problem calculation for two errors landing in the same word, and re-derive the required scrub period for 12 and 120 corrected errors a day.

9. Distinguish the two chapters. Design the minimal observation that separates a coherency bug (26.3) from a routing bug (this chapter), given only a wrong value and its address.

10. Add the seventh property. Propose one none of §14's six implies, name its section, and construct the configuration where the six hold and it fails. A property that cannot fail alone is not a seventh property.

27. Summary

A completion is not a correct answer. Eighty reads of a thousand answered by a device that does not own the address is eighty-two percent correct on a ninety-eight percent completion rate — and no default counter shows the gap.

And the louder failure is the safer one. Forty unclaimed reads produce error responses and get investigated; eighty misrouted ones produce data.

Host and device compute the device independently and must agree. A 256-byte granule against a 1024-byte one is seventy-five percent of blocks on the wrong device, with base and size matching perfectly in both configurations.

A device that cannot answer correctly can say so, and something has to listen. Thirty-two of forty flagged with twelve consumers checking is twelve detectable failures of forty — the two mechanisms multiply, so improving either alone has limited return.

Poison that escapes is multiplied by the fanout. Fifteen escaped lines at twelve is a hundred and eighty downstream values, and even ninety-nine percent containment leaves twelve.

A partial write changes its bytes or the line. Eight enabled on sixty-four written whole is fifty-six bytes of a neighbour's data — and it looks like a coalescing optimisation.

A first-match decoder always produces an answer. 512k of addresses with two owners is twelve percent of the map, answered consistently and wrongly, with one device's memory simply unreachable.

Slow and wrong go to different engineers. Twelve wrong values and twenty-eight latency outliers is a thirty percent correctness share, and filing them together sends twenty-eight investigations looking for a data error that is not there.

Corrected errors accumulate between scrubs. Twelve a day over forty-eight hours is twenty-four sitting in memory, and whether that matters is the ratio of two numbers the device already exports.

And the failing addresses are free. Twenty sharing a bit rules out eight ranges and eight of twelve candidates — while nineteen of twenty rules out nothing, because a pattern with a counterexample is a coincidence with a good story.

The one mutation survivor was a dominated guard — a pattern-found condition requiring failing addresses alongside a strength that is zero without them. Three anchors matched nothing, and all three were self-inflicted: four mutations needed whole-statement rewrites, and rewriting them invalidated anchors that referenced the same text. The harness reported each as not-injected, which is the only reason 73 of 73 is a score rather than a count of substitutions that happened to apply.

"The read completed" is one property of six. The completion status called six of seven configurations sound when one was — and §21 is a database server losing one read in a hundred thousand, explained by two configuration registers disagreeing about a single interleave parameter, found by twelve addresses that shared a bit.

Module 26 continues. 26.1 was a device the host never saw, 26.2 a link that came back as something less, 26.3 a value made stale, and this chapter a value that was never the right one to return. Every one of the four ended the same way: the indicator was accurate and the question it answered was narrower than the one being asked — which is Module 25's finding, arriving from the other side of the tape-out.