Skip to content

PCIe · Module 24

Error Injection — One Contract at a Time, and Prove It Recovers

A useful negative test corrupts exactly one rule and leaves everything else legal. The harder half is what comes after: detection is not recovery, and a design that reports the fault but leaks a Tag has failed the test it appeared to pass.

Module 24 built observers. 24.2 bound properties, 24.3 tracked transactions, 24.4 measured reach, 24.6 composed them.

Every one of them has branches that no correct traffic can reach. The terminal-status path in the scoreboard, the credit-exhaustion cover property, the error bins — all of it is dead code until something deliberately misbehaves.

1. Sources, Scope, and What This Chapter Refuses to Fabricate

2. One Contract at a Time

The difference between a negative test and noise:

TestWhat you learn
legal packet + wrong integrity valuewhether the integrity checker fires, and what recovery follows
random bits on the wirethat the design dislikes garbage

The discipline, as a procedure:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
1. establish a baseline legal transaction that PASSES
2. identify the single field or event that carries the contract
3. corrupt exactly that, after everything else is computed
4. observe WHICH checker fires FIRST
5. observe the required recovery or containment
6. prove no unrelated resource leaked
7. prove the NEXT clean transaction succeeds

Step 3's "after everything else is computed" is not a detail. §4 shows the specific way it goes wrong: corrupt the payload before the integrity value is computed and the integrity value is computed over the corrupted payload — so it is valid, the checker correctly says nothing, and the test silently proves nothing.

And step 4's word "first" matters. With several checkers armed, the useful evidence is which one fires firstChapter 21.4 §11 measured last-error reporting naming a non-root-cause 96.8% of the time, and an injected fault produces exactly the cascade that exploits it.

3. The Error Matrix by Layer

Faults belong to layers, and the layer determines where the test must inject and where it must watch.

LayerFaultInjected whereFirst checker
Transactionrequest no Function claimsat the requesterUnsupported Request path (21.1 §6)
Completion with an unknown identityat the completer/VIPthe context matcher (23.5 §3)
Completion with an error statusat the completer/VIPthe terminal path (13.2)
byte count inconsistent with the requestat the completerscoreboard coverage (24.3 §5)
Data Linkbad integrity valueafter computation (§4)the integrity checker (14.4)
sequence discontinuityin the transmitted sequence fieldthe sequence checker
acknowledgement withheldat the link partnerthe replay mechanism
Flow Controlissue without credit — internalinside the DUT's schedulerSVA, before the wire (24.2 §10)
incorrect credit advertisement — peerat the VIP/partnerthe DUT's credit accounting
Linkexpected training event withheldat the partner/PHY modelLTSSM progress (25.3)

The Flow Control rows are deliberately split, and the distinction is the most important one in the table.

An internal credit fault is a DUT mutation: the scheduler issues although the credit manager says zero. The correct outcome is that an assertion fires before anything reaches the wire — nothing illegal is ever transmitted, and the test proves the internal contract.

A peer credit fault is an environment behaviour: the link partner advertises credit incorrectly. That tests the DUT's own accounting, and it is only injectable if the environment can model it.

Conflating them produces a test that cannot be interpreted — you do not know whether you proved the DUT's scheduler or its accounting, and §10's mutation 16 is exactly that confusion.

4. Injection Is Not Corruption

5. Detection Is Not Recovery

6. The Waveform

Arm, inject once, detect, recover — and the clean packet that follows

10 cycles
Ten cycles of an error-injection harness. A clean packet transfers at cycles 0 and 1. The fault arm is asserted from cycle 1. At cycle 2 inject now is asserted and the packet is corrupted. At cycle 3 the checker error fires and the first error record latches and stays latched. Recovery is pending from cycle 3 to cycle 5. At cycle 6 recovery completes and the fault is consumed. Clean packets transfer again at cycles 7 and 8.fault lands on ONE packet onlyfault lands on ONE packetonlychecker fires; first error latchedchecker fires; first errorlatchedrecovery done, fault consumedrecovery done, faultconsumednext clean packet succeedsnext clean packet succeedsclkfault_arminject_nowpkt_xferchecker_errfirst_errrecoveringclean_okt0t1t2t3t4t5t6t7t8t9
Figure 1 — one injected fault, from arm to recovered. A clean packet transfers, the injector is armed, the fault lands on exactly one packet, the checker fires and the first-error record latches, a retry follows, recovery completes and the fault is consumed. The two packets after recovery transfer normally, which is the part of the test that proves the design recovered rather than merely detected.

Four things to read out of the figure.

inject_now is one cycle wide. The fault lands on exactly one packet (§2); a corruptor still asserted at cycle 3 would damage the next one too, and the recovery test would then run against a second fault.

first_err is sticky from cycle 3 onward. Later cascading errors must not overwrite it (23.6 §7) — the whole value of the record is that it names the first cause.

recovering has a definite end. Cycle 6 is where the design says it is finished; without that event a watchdog cannot distinguish "recovering" from "hung" (§7).

And cycles 7–8 are the test. A trace that stops at cycle 6 has demonstrated detection. The clean packets after it are what demonstrate recovery (§5).

7. RTL — The Injection and Recovery Harness

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// VERIFICATION-ONLY. Fault taxonomy and control.
// LOCAL VERIFICATION POLICY -- PCIe defines none of these values (§1).
package fault_pkg;
 
  parameter int TAGS   = 8;
  parameter int TAG_W  = (TAGS <= 1) ? 1 : $clog2(TAGS);
  parameter int SEQ_W  = 12;
  parameter int CNT_W  = 32;
 
  typedef enum logic [3:0] {
    FI_NONE            = 4'd0,
    FI_BAD_INTEGRITY   = 4'd1,   // Data Link: corrupt the COMPUTED value (§4)
    FI_SEQ_CORRUPT     = 4'd2,   // Data Link: sequence discontinuity
    FI_DROP_ACK        = 4'd3,   // Data Link: withhold acknowledgement
    FI_BAD_CPL_ID      = 4'd4,   // Transaction: wrong (Requester ID, Tag)
    FI_CPL_ERR_STATUS  = 4'd5,   // Transaction: terminal status (13.2)
    FI_CREDIT_OVERSPEND= 4'd6,   // INTERNAL mutation -- SVA must fire (§3)
    FI_DROP_TRAIN_EVENT= 4'd7    // Link: withhold a training event
  } fault_kind_e;
 
  typedef enum logic [1:0] { FM_ONESHOT, FM_REPEAT, FM_DISABLED } fault_mode_e;
 
  typedef struct packed {
    logic            armed;
    fault_kind_e     kind;
    fault_mode_e     mode;
    logic [CNT_W-1:0] target_event;    // fire on the Nth qualifying EVENT
    logic [2:0]      target_port;
    logic [2:0]      target_function;
  } fault_cfg_t;
 
  // Which checker SHOULD fire first for each fault. The test asserts this
  // mapping -- a fault that trips the wrong checker is §10's mutation 4.
  function automatic string expected_checker(input fault_kind_e k);
    case (k)
      FI_BAD_INTEGRITY:    return "dl_integrity";
      FI_SEQ_CORRUPT:      return "dl_sequence";
      FI_DROP_ACK:         return "dl_replay";
      FI_BAD_CPL_ID:       return "tl_unknown_identity";
      FI_CPL_ERR_STATUS:   return "tl_terminal_status";
      FI_CREDIT_OVERSPEND: return "sva_credit_reservation";
      FI_DROP_TRAIN_EVENT: return "ltssm_progress";
      default:             return "none";
    endcase
  endfunction
 
endpackage
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import fault_pkg::*;
 
// VERIFICATION-ONLY. One-shot arm and trigger (§2).
// The event counter advances on the ACCEPTED event only -- counting offers
// would fire the fault on a packet that was never sent (§10, mutation 18).
// §9: one-shot fires exactly 1; a repeating variant fired 2,987.
module fault_trigger (
  input  logic clk,
  input  logic rst_n,
  input  fault_cfg_t cfg,
  input  logic       event_fire,       // a qualifying event TRANSFERRED
  input  logic       fault_consumed,   // the DUT finished recovering
 
  output logic       inject_now,
  output fault_kind_e active_kind,
  output logic       pending,
  output logic [CNT_W-1:0] event_count,
  output logic       err_fire_unarmed  // sticky: fired without an arm
);
  logic [CNT_W-1:0] cnt_q;
  logic             armed_q, pend_q, e_q;
 
  assign event_count      = cnt_q;
  assign active_kind      = cfg.kind;
  assign pending          = pend_q;
  assign err_fire_unarmed = e_q;
 
  // SAME-CYCLE RULE (§8): arm and the target event in one cycle FIRE. The
  // arm is a level, and the count is compared against the next value.
  logic [CNT_W-1:0] cnt_next;
  assign cnt_next   = event_fire ? (cnt_q + CNT_W'(1)) : cnt_q;
  assign inject_now = armed_q && event_fire
                   && (cnt_next == cfg.target_event)
                   && (cfg.mode != FM_DISABLED);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      cnt_q <= '0; armed_q <= 1'b0; pend_q <= 1'b0; e_q <= 1'b0;
    end else begin
      cnt_q <= cnt_next;
      // Arming is edge-insensitive: a level that is high enables firing.
      if (cfg.armed && !armed_q) armed_q <= 1'b1;
      if (inject_now) begin
        pend_q <= 1'b1;
        if (cfg.mode == FM_ONESHOT) armed_q <= 1'b0;   // FIRES EXACTLY ONCE
      end
      if (fault_consumed) pend_q <= 1'b0;
      // A fire with no arm cannot happen; prove it rather than trust it.
      if (event_fire && (cnt_next == cfg.target_event) && !armed_q && cfg.armed)
        e_q <= 1'b1;
    end
  end
endmodule
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import fault_pkg::*;
 
// VERIFICATION-ONLY. Data Link integrity corruptor (§4).
// It mutates the COMPUTED value, never the payload -- corrupting the
// payload first makes the computed value CORRECT for the corrupted data,
// and the checker rightly says nothing (§10, mutation 14).
module dl_integrity_injector #(parameter int W = 32) (
  input  logic         inject_now,
  input  logic [W-1:0] integrity_in,     // already computed over the packet
  input  logic [$clog2(W)-1:0] bit_sel,
 
  output logic [W-1:0] integrity_out,
  output logic         corrupted
);
  // Exactly one bit, exactly one packet. Everything else is untouched, so
  // the only broken contract is the integrity check (P8, P9).
  assign integrity_out = inject_now ? (integrity_in ^ (W'(1) << bit_sel))
                                    : integrity_in;
  assign corrupted     = inject_now;
endmodule
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import fault_pkg::*;
 
// VERIFICATION-ONLY. Sequence-number injector.
// The WIDTH and the legal progression are the Data Link chapter's (14.4);
// this block only perturbs the transmitted value by a parameterized delta,
// so nothing here asserts a sequence-number rule.
module dl_sequence_injector (
  input  logic             inject_now,
  input  logic [SEQ_W-1:0] seq_in,
  input  logic signed [3:0] delta,       // +1 skip, -1 duplicate, etc.
 
  output logic [SEQ_W-1:0] seq_out,
  output logic             corrupted
);
  assign seq_out   = inject_now ? (seq_in + SEQ_W'(delta)) : seq_in;
  assign corrupted = inject_now;
endmodule
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import fault_pkg::*;
 
// VERIFICATION-ONLY. Wrong-identity Completion injector (§3).
// The packet stays perfectly legal -- correct format, correct length, a
// valid identity. Only the WRONG one. A conformance checker sees nothing;
// the project scoreboard's matcher must catch it (24.5 §6's distinction).
module tl_cpl_identity_injector (
  input  logic            inject_now,
  input  logic [15:0]     rid_in,
  input  logic [TAG_W-1:0] tag_in,
  input  logic [TAG_W-1:0] wrong_tag,
 
  output logic [15:0]     rid_out,
  output logic [TAG_W-1:0] tag_out,
  output logic            corrupted
);
  assign rid_out   = rid_in;                       // still a legal Requester ID
  assign tag_out   = inject_now ? wrong_tag : tag_in;
  assign corrupted = inject_now;
endmodule
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import fault_pkg::*;
 
// SYNTHESIZABLE debug hook. First-error capture, injection-aware.
// It records WHICH checker fired first AND what fault was pending, so a
// cascade cannot bury the mapping the test is asserting (§2, §5).
module injection_first_error (
  input  logic clk,
  input  logic rst_n,
  input  logic [7:0]  checker_id,        // which checker fired
  input  logic        checker_fire,
  input  fault_kind_e pending_kind,
  input  logic        pending_valid,
  input  logic [15:0] pkt_rid,
  input  logic [TAG_W-1:0] pkt_tag,
  input  logic [3:0]  link_state,
  input  logic [CNT_W-1:0] cycle,
  input  logic        clear,
 
  output logic        captured,
  output logic [7:0]  first_checker,
  output fault_kind_e first_fault,
  output logic [15:0] first_rid,
  output logic [TAG_W-1:0] first_tag,
  output logic [3:0]  first_link_state,
  output logic [CNT_W-1:0] first_cycle,
  output logic [CNT_W-1:0] later_errors
);
  logic cap_q; logic [7:0] chk_q; fault_kind_e fk_q;
  logic [15:0] rid_q; logic [TAG_W-1:0] tag_q; logic [3:0] ls_q;
  logic [CNT_W-1:0] cyc_q, later_q;
 
  assign captured=cap_q; assign first_checker=chk_q; assign first_fault=fk_q;
  assign first_rid=rid_q; assign first_tag=tag_q; assign first_link_state=ls_q;
  assign first_cycle=cyc_q; assign later_errors=later_q;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n || clear) begin
      cap_q<=1'b0; chk_q<='0; fk_q<=FI_NONE; rid_q<='0; tag_q<='0;
      ls_q<='0; cyc_q<='0; later_q<='0;
    end else if (checker_fire) begin
      if (!cap_q) begin
        // LATCH ONCE. 21.4 §11 measured last-write-wins naming a
        // non-root-cause 96.8% of the time.
        cap_q<=1'b1; chk_q<=checker_id;
        fk_q <= pending_valid ? pending_kind : FI_NONE;
        rid_q<=pkt_rid; tag_q<=pkt_tag; ls_q<=link_state; cyc_q<=cycle;
      end else if (!(&later_q)) later_q <= later_q + CNT_W'(1);
    end
  end
endmodule
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import fault_pkg::*;
 
// VERIFICATION-ONLY. Recovery watchdog (§5).
// A TEST BUDGET, not a PCIe timer. It measures how long the DUT stayed in
// its declared recovery state and flags a design that detected but never
// finished. Labelled deliberately -- 24.3 §8's distinction.
module recovery_watchdog #(parameter int unsigned BUDGET = 4096) (
  input  logic clk,
  input  logic rst_n,
  input  logic fault_injected,       // one-cycle pulse
  input  logic recovery_done,        // the DUT's declared completion event
 
  output logic recovery_pending,
  output logic [CNT_W-1:0] recovery_cycles,
  output logic watchdog_expired,     // sticky
  output logic [CNT_W-1:0] worst_recovery
);
  logic pend_q, exp_q;
  logic [CNT_W-1:0] cyc_q, worst_q;
 
  assign recovery_pending = pend_q;
  assign recovery_cycles  = cyc_q;
  assign watchdog_expired = exp_q;
  assign worst_recovery   = worst_q;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin pend_q<=1'b0; cyc_q<='0; exp_q<=1'b0; worst_q<='0; end
    else begin
      // SAME-CYCLE: injection and completion in one cycle means the design
      // recovered immediately -- a legal zero-cycle recovery, not an error.
      if (recovery_done && pend_q) begin
        pend_q <= 1'b0;
        if (cyc_q > worst_q) worst_q <= cyc_q;
      end else if (fault_injected && !pend_q) begin
        pend_q <= 1'b1; cyc_q <= '0;
      end else if (pend_q) begin
        cyc_q <= cyc_q + CNT_W'(1);
        if (cyc_q >= CNT_W'(BUDGET)) exp_q <= 1'b1;   // test budget elapsed
      end
    end
  end
endmodule
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
import fault_pkg::*;
 
// SYNTHESIZABLE debug hook. THE RECOVERY INVARIANT MONITOR (§5).
// This is the block that turns "the checker fired" into "the design
// recovered". §9: every fault kind returned to 8/8 free Tags.
module recovery_invariant_monitor (
  input  logic clk,
  input  logic rst_n,
  input  logic [TAGS-1:0] tag_free_map,
  input  logic [TAGS-1:0] tag_owned_map,
  input  logic [CNT_W-1:0] buffer_reserved,
  input  logic            replay_pending,
  input  logic            fault_injected,
  input  logic            recovery_done,
  input  logic            clean_txn_completed,   // a LEGAL transaction after recovery
 
  output logic err_tag_leak,
  output logic err_buffer_leak,
  output logic err_replay_stuck,
  output logic err_no_clean_after_recovery,   // sticky
  output logic recovered_cleanly
);
  logic [TAGS-1:0] baseline_free_q;
  logic [CNT_W-1:0] baseline_buf_q;
  logic seen_fault_q, seen_recovery_q, seen_clean_q, e_q;
 
  assign err_tag_leak    = seen_recovery_q && (tag_free_map != baseline_free_q);
  assign err_buffer_leak = seen_recovery_q && (buffer_reserved != baseline_buf_q);
  assign err_replay_stuck = seen_recovery_q && replay_pending;
  assign err_no_clean_after_recovery = e_q;
  assign recovered_cleanly = seen_recovery_q && seen_clean_q
                          && !err_tag_leak && !err_buffer_leak && !err_replay_stuck;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      baseline_free_q <= {TAGS{1'b1}}; baseline_buf_q <= '0;
      seen_fault_q<=1'b0; seen_recovery_q<=1'b0; seen_clean_q<=1'b0; e_q<=1'b0;
    end else begin
      // Snapshot the resting state BEFORE the fault -- the comparison the
      // whole recovery test depends on.
      if (!seen_fault_q && !fault_injected) begin
        baseline_free_q <= tag_free_map;
        baseline_buf_q  <= buffer_reserved;
      end
      if (fault_injected)      seen_fault_q    <= 1'b1;
      if (recovery_done)       seen_recovery_q <= 1'b1;
      if (clean_txn_completed) seen_clean_q    <= 1'b1;
      // Detection without a subsequent clean transaction is a FAILED test,
      // even though every checker behaved correctly (§5).
      if (seen_recovery_q && !seen_clean_q && (tag_owned_map == '0)) e_q <= 1'b1;
    end
  end
endmodule

Classification: six verification-only, two synthesizable debug hooks.

Failure — eight. Corrupting before the integrity value is computed (§4). A fault that outlives its packet. Counting offers rather than accepted events. A one-shot that repeats (2,987 fires, §9). First-error overwritten by the cascade. A recovery watchdog labelled as a PCIe timer. No baseline snapshot, so leaks are invisible. And no clean transaction after recovery — the test that looks passed and is not.

8. Same-Cycle Audit and Assertions

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// ---- INJECTION DISCIPLINE (§2, §4) ------------------------------
// P1: no injection unless armed.
property p_no_inject_unarmed;
  @(posedge clk) disable iff (!rst_n) inject_now |-> cfg.armed;
endproperty
 
// P2: injection requires an ACCEPTED event, never an offer.
property p_inject_on_transfer;
  @(posedge clk) disable iff (!rst_n) inject_now |-> event_fire;
endproperty
 
// P3: the fault lands on the EXACT target event.
property p_exact_target_event;
  @(posedge clk) disable iff (!rst_n)
    inject_now |-> (event_count + CNT_W'(1) == cfg.target_event);
endproperty
 
// P4: a one-shot fault fires exactly once. §9: a repeating variant fired
// 2,987 times against the correct design's 1.
property p_oneshot_fires_once;
  @(posedge clk) disable iff (!rst_n)
    (inject_now && (cfg.mode == FM_ONESHOT)) |=> !inject_now;
endproperty
 
// P5: a second arm while a fault is pending is refused.
property p_no_rearm_while_pending;
  @(posedge clk) disable iff (!rst_n)
    (pending && cfg.armed) |-> !inject_now;
endproperty
 
// P6: firing without an arm is REPORTED, never silent.
property p_unarmed_fire_flagged;
  @(posedge clk) disable iff (!rst_n)
    (event_fire && (event_count + CNT_W'(1) == cfg.target_event)
      && !armed_q && cfg.armed) |=> err_fire_unarmed;
endproperty
 
// P7: the event counter advances only on an accepted event.
property p_event_count_on_transfer;
  @(posedge clk) disable iff (!rst_n)
    (event_count != $past(event_count)) |-> $past(event_fire);
endproperty
 
// P8: the corruptor mutates the COMPUTED value only -- the payload is
// untouched, so exactly one contract is broken (§4).
property p_payload_untouched;
  @(posedge clk) disable iff (!rst_n)
    inject_now |-> $stable(packet_payload);
endproperty
 
// P9: a non-target packet is bit-identical to the baseline.
property p_non_target_unmodified;
  @(posedge clk) disable iff (!rst_n)
    (!inject_now && pkt_valid) |-> (integrity_out == integrity_in);
endproperty
 
// P10: exactly one fault kind is active at a time.
property p_single_active_fault;
  @(posedge clk) disable iff (!rst_n)
    pending |-> (active_kind != FI_NONE);
endproperty
 
// ---- DETECTION (§2, §3) -----------------------------------------
// P11: the injected fault trips the EXPECTED checker, not another.
property p_expected_checker_fires;
  @(posedge clk) disable iff (!rst_n)
    (captured && (first_fault != FI_NONE))
      |-> (checker_name(first_checker) == expected_checker(first_fault));
endproperty
 
// P12: a clean packet never trips a checker. §9: the baseline fired NOTHING.
property p_no_false_detection;
  @(posedge clk) disable iff (!rst_n)
    (pkt_valid && !inject_now && !pending) |-> !checker_fire;
endproperty
 
// P13: an injected packet is counted as ONE transaction, not two.
property p_injected_counted_once;
  @(posedge clk) disable iff (!rst_n)
    inject_now |=> (txn_count == $past(txn_count) + 1);
endproperty
 
// P14: the first error is sticky.
property p_first_error_sticky;
  @(posedge clk) disable iff (!rst_n)
    (captured && !clear) |=> (captured && $stable(first_checker)
                                       && $stable(first_fault));
endproperty
 
// P15: later errors are counted, never stored over the first.
property p_later_errors_counted;
  @(posedge clk) disable iff (!rst_n)
    (captured && checker_fire && !clear) |=> $stable(first_cycle);
endproperty
 
// ---- RECOVERY (§5) ----------------------------------------------
// P16: recovery is pending for a bounded interval under the TEST BUDGET.
// This bounds a verification watchdog, not a PCIe timer (§1).
property p_recovery_bounded;
  @(posedge clk) disable iff (!rst_n)
    recovery_pending |-> (recovery_cycles <= CNT_W'(BUDGET)) || watchdog_expired;
endproperty
 
// P17: every Tag returns to the pre-fault baseline. §9: 8/8 for every kind.
property p_no_tag_leak_after_recovery;
  @(posedge clk) disable iff (!rst_n)
    (recovery_done && (tag_owned_map == '0)) |=> !err_tag_leak;
endproperty
 
// P18: buffer reservations return to baseline.
property p_no_buffer_leak;
  @(posedge clk) disable iff (!rst_n)
    (recovery_done && (tag_owned_map == '0)) |=> !err_buffer_leak;
endproperty
 
// P19: replay state clears -- a replayed packet is not a new transaction.
property p_replay_state_clears;
  @(posedge clk) disable iff (!rst_n)
    recovery_done |=> !err_replay_stuck;
endproperty
 
// P20: THE PROPERTY MOST OFTEN MISSING. A clean transaction after recovery
// completes normally. Detection without this is a failed test (§5).
property p_clean_after_recovery;
  @(posedge clk) disable iff (!rst_n)
    recovery_done |-> s_eventually clean_txn_completed;
endproperty
 
// P21: the error does not repeat indefinitely from one fault.
property p_no_error_storm;
  @(posedge clk) disable iff (!rst_n)
    (later_errors > CNT_W'(STORM_LIMIT)) |-> error_storm_reported;
endproperty
 
// ---- CROSS-LAYER AND HYGIENE ------------------------------------
// P22: reset clears the arm and any pending fault.
property p_reset_clears_injection;
  @(posedge clk)
    (!rst_n) |=> (!inject_now && !pending && !captured);
endproperty
 
// P23: a wrong-identity Completion never retires an INNOCENT context
// (23.5 §3's matcher must report unknown rather than default).
property p_wrong_identity_no_innocent_retire;
  @(posedge clk) disable iff (!rst_n)
    (inject_now && (active_kind == FI_BAD_CPL_ID)) |=> !unrelated_ctx_retired;
endproperty
 
// P24: an internal credit overspend is caught BEFORE the wire (§3).
property p_credit_fault_caught_internally;
  @(posedge clk) disable iff (!rst_n)
    (inject_now && (active_kind == FI_CREDIT_OVERSPEND)) |-> !tx_wire_valid;
endproperty
 
// P25: the credit ledger never underflows even under injection.
property p_ledger_no_underflow_under_injection;
  @(posedge clk) disable iff (!rst_n)
    (avail_credit <= advertised_credit);
endproperty
 
// P26: the packet owner is held through EOP even during injection --
// corruption does not release arbitration (21.2 §11's contract).
property p_packet_lock_holds_under_injection;
  @(posedge clk) disable iff (!rst_n)
    (owner_valid && inject_now) |=> (owner_valid && $stable(owner));
endproperty
 
// P27: injection logic never drives functional traffic.
property p_injector_non_functional;
  @(posedge clk) disable iff (!rst_n)
    $stable({pkt_valid, pkt_ready}) or !$stable({event_count, later_errors});
endproperty
 
// P28: EVIDENCE -- a fault was actually injected in this run.
property c_fault_injected;
  @(posedge clk) disable iff (!rst_n) inject_now;
endproperty
 
// P29: EVIDENCE -- a clean transaction followed a recovery.
property c_clean_after_recovery_reached;
  @(posedge clk) disable iff (!rst_n) (recovery_done ##[1:$] clean_txn_completed);
endproperty

Twenty-nine properties. P17–P20 are the recovery contract, and P20 is the one this chapter argues is most often absent. P28 and P29 are the cover properties — without them an injection suite can report a clean run because it never actually injected anything (24.2 §4's vacuity measured at 60,205 versus 0).

9. Measured Behaviour

10. Verification — Mutations

#MutationSymptomCaught by
1Fault fires before the armcorruption in the baseline phase (§9)P1, P6
2One-shot fires repeatedly2,987 fires; recovery never tested cleanlyP4
3Corruption persists into the next packetrecovery tested against a second faultP9
4The fault trips a different checker than expectedthe mapping the test asserts is wrongP11
5A clean packet trips a checkerfalse failures; the baseline is unusableP12
6The injected packet counted as two transactionsconservation breaks in the scoreboardP13
7First error overwritten by the cascaderoot cause lost 96.8% of the timeP14, P15
8Fault state survives resetthe next test starts pre-corruptedP22
9Tag leaked after a bad Completionwedges after ~14 jobs (23.3 §14)P17
10Buffer reservation leaked after an errorthroughput decays run over runP18
11Replay state never retiresthe retry path stays armed foreverP19
12The recovery watchdog called a PCIe timeran unobserved mechanism claimed (§1)review
13Multi-field random corruptionthe failure names nothing (§2)review
14Payload corrupted before the integrity value is computedthe value is valid; the test silently passes (§4)P8
15The "sequence" fault mutates a transaction-layer fieldthe test's name and its layer disagree (§3)review
16Internal and peer credit faults conflatedyou cannot tell scheduler from accounting (§3)P24
17The error path never exercised with a TX stallstall-only ownership bugs unreachedP26
18Injection triggered on valid rather than the transferthe fault lands on a packet never sentP2, P7
19Target event counter off by onethe fault lands on the wrong packetP3
20No clean transaction attempted after recoverydetection mistaken for recovery (§5)P20, P29
21VIP reports the first error; the project checker ignores ittwo observers disagree and neither is read (24.5 §10)review
22Expected errors suppressed too broadly for the whole runreal failures after the injected one are hiddenreview
23Reset used to clear residue before the leak checkthe leak is masked by the cleanupP17, P18
24The same fault injected in every sequenceone fault class tested, the rest neverreview
25An impossible behaviour presented as a legal PCIe scenariothe DUT fails a test the protocol forbidsreview
26A wrong-identity Completion retires an innocent contexta valid transaction corrupted by the testP23
27A replayed packet counted as new application payloadefficiency and throughput inflate (22.5 §6)P19
28The error result pulse lost under backpressurethe failure is never reportedP21
29The checker and the injector share a helper functionboth wrong together (24.1 §5)review
30Credit ledger underflows under injectionthe engine then believes it owns huge creditP25
31The packet owner released when corruption is detectedheader and payload split across ownersP26
32No cover property on the injection itselfa suite that never injected reports clean (§8)P28
33Injection logic gates a functional signalthe measured design is not the shipping oneP27

Two counterexamples worth stating explicitly.

Mutation 14 is the one that makes a whole test suite worthless while every run is green. The injector corrupts a payload byte, and then the harness computes the integrity value over the packet it is about to send. The value is correct for the corrupted payload. The receiver checks it, finds it valid, and accepts the packet. No checker fires, the test reports "error injected, no failure detected" — and if the expectation is written loosely, that reads as a pass. §7's corruptor operates on the computed value for exactly this reason, and P8 asserts the payload is untouched.

Mutation 20 is the methodological one, and §9 is built to expose it. A suite that injects, observes the checker, and ends has verified detection. Chapter 23.3 §14's engine passed that test and wedged after 14 jobs because the error path leaked a Tag. The fix costs one more transaction at the end of each test — and P20 plus the baseline comparison in §7's monitor is what turns it into an assertion rather than a habit.

11. Debugging the Testbench

Symptom — the fault is injected and no checker fires. Suspect the injection before the DUT (§4, mutation 14). Confirm the corruption reached the wire: read the injector's corrupted output and compare the transmitted value against the baseline. The most common cause is corrupting before the protected value is computed, which makes the packet internally consistent.

Symptom — the checker fires one packet later than expected. Target-event off-by-one (mutation 19), or the counter is advancing on offers rather than transfers (mutation 18). event_count at the moment of injection is the evidence — P3 pins it.

Symptom — the expected error appears and the design wedges afterwards. Detection without recovery (§5). Read the recovery invariant monitor: err_tag_leak, err_buffer_leak, err_replay_stuck. §9's right-hand columns are the shape of a healthy result; a wedge means one of those three is set.

Symptom — a different checker fires first than the one the test expects. Either the fault is in the wrong layer (mutation 15) or the DUT genuinely detects it earlier. Both are findings. The first-error record names the checker and the pending fault together, so the mapping is readable directly — and if the DUT's earlier detection is legitimate, the expectation in expected_checker() is what needs updating.

Symptom — the test passes without backpressure and fails with it. The error path has never been exercised under stall (mutation 17). Every ownership defect in Modules 22–23 needed a stall to appear (24.1 §7). Add a stalling consumer to the recovery phase specifically, not just to the traffic phase.

Symptom — reset leaves a fault armed. Mutation 8. The next test begins pre-corrupted, and its baseline is not clean — which invalidates every comparison in §9. P22 asserts the clear, and the symptom is a suite where test order changes results.

Symptom — one fault produces thousands of errors. An error storm (P21). Distinguish a genuine cascade from a repeating injection: read the one-shot's pending and event_count. §9 measured a repeating trigger firing 2,987 times — which looks exactly like a design that cannot recover.

Symptom — the project scoreboard fails while the protocol checker reports successful recovery. They are answering different questions (24.5 §6). The link recovered; the transaction did not — most often a Tag or context that the recovery path did not release. This is the wrong-identity injection's signature, and P23 is the property.

12. Misconceptions

"Error injection means flipping random bits." It means breaking exactly one contract and leaving everything else legal (§2).

"Corrupt the payload — that's the data." Then the protecting value is computed over the corruption and the packet is consistent (§4).

"If the checker fired, the test passed." Detection is half the test (§5).

"The design reported the error, so it recovered." Chapter 23.3 §14's engine reported and wedged after 14 jobs.

"A recovery timer is the Completion Timeout." It is a test budget unless it is the actual mechanism (§1, §7).

"One-shot versus repeat is a detail." 1 fire versus 2,987 (§9).

"Inject on valid — the packet is right there." The packet may never be sent (P2).

"An internal credit bug and a bad peer are the same test." One tests the scheduler, the other the accounting (§3).

"Suppress expected errors for the whole test." Then the failures after the injected one are hidden too (mutation 22).

"Clear the residue with a reset before checking." That masks the leak the test exists to find (mutation 23).

"A replayed packet is new traffic." It is the same payload again (22.5 §6).

"If the suite is green, faults were injected." Not without a cover property on the injection itself (P28).

13. Understanding Check

Q1. Your bad-integrity test injects a fault, the checker never fires, and the suite reports a pass. What happened? Almost certainly the corruption preceded the computation (§4, mutation 14). The protecting value was computed over the already-corrupted packet, making it internally consistent — so the receiver correctly accepted it. The injector must mutate the computed value, after everything else is final, and P8 asserts the payload is untouched by injection.

Q2. All five fault kinds fire their expected checkers. What have you still not tested? Recovery (§5). The middle of §9's table would look identical for a design that leaks a Tag on every errored transaction. The right-hand columns — return to 8/8 free Tags and clean transactions afterwards — are the other half, and 23.3 §14 measured what their absence costs: a wedge after 14 jobs.

Q3. Why does the trigger count events rather than cycles, and why on the transfer? Because a fault must land on a specific packet to be interpretable (§2). Counting cycles lands it wherever the pipeline happens to be; counting offers lands it on a packet that may never be sent, so the corruption is discarded and nothing is tested. P2 and P7 pin both, and §9 shows the injector waiting for a packet of the targeted class — which is why 1,965 injections produced 787 integrity errors.

Q4. An internal credit overspend and a peer advertising credit incorrectly — why must these be separate tests? Because they prove different contracts (§3). The internal mutation must be caught by an assertion before anything reaches the wire (P24) — nothing illegal is transmitted, and the test proves the scheduler honours the reservation. The peer fault tests the DUT's own credit accounting. Conflating them yields a failure you cannot attribute to either.

Q5. Your recovery watchdog expires at 4,096 cycles. Is that a Completion Timeout violation? No. It is a test budget (§7). The watchdog measures how long the DUT stayed in its declared recovery state and flags a design that never finished — a true statement about the simulation. A Completion Timeout is a device mechanism with its own configuration, which this chapter does not model and 25.7 owns diagnosing.

Q6. Which single property would you add to an error-injection suite you inherited? P20 — a clean transaction completes after recovery. It requires no protocol knowledge, it is one extra transaction per test, and it is the property that separates "the design noticed" from "the design survived." P28's cover property is the close second, because a suite that silently stopped injecting reports the same green as one that never had a bug.

14. Module 24 Complete

Seven chapters built a verification stack. 24.1 framed the three contracts, 24.2 proved the local ones, 24.3 proved end-to-end conservation, 24.4 measured reach, 24.5 divided the labour with a vendor participant, 24.6 composed them — and this chapter gave them something to catch.

Every observer in Module 24 has branches that only a fault reaches. The terminal-status path, the unknown-identity error, the credit-exhaustion cover property, the coverage error bins. They are now reachable, and §9 shows each one firing exactly when it should and never otherwise.

Module 25 changes the question entirely. Verification asks did the design meet its contract, under stimulus I chose. Debugging asks what is wrong with this system, which I did not build and cannot re-run at will — fewer observation points, no golden model, and a symptom instead of a testbench.

Chapter 25.1 establishes the method: symptom → layer → last known-good event → first missing event → distinguishing experiment. And the first-error capture in §7 is the bridge — the same block, in silicon instead of simulation, is what makes that method possible at all.