Verification Checklist — Can This Environment Fail?

30.2 asked whether the RTL is faithful. This gate asks something uncomfortable about the thing that judged it: is this environment capable of failing?

1. What This Gate Owns

Verification is complete only when every architectural contract has independent stimulus, an independent checker, closure evidence and a known failure signature.

Four requirements, and dropping any one of them produces a familiar kind of false confidence.

Three readings.

Test count is not closure. Ten thousand tests over one path is one test run ten thousand times. The measure is contracts covered, not seeds consumed.

Coverage percentage is not closure either. Code coverage says which lines executed; it says nothing about whether the interesting simultaneous conditions (30.2 §1) ever co-occurred. §9 is about what to measure instead.

And the fourth requirement is the one this chapter is really about. A checker that has never fired is not a checker; it is a hypothesis. §7 and §8 exist to convert hypotheses into evidence.

This gate inherits every contract recorded by 30.1 §4 and every hook required by 30.2 §15 Q48. It does not re-teach UVM — 24.1, 24.2, 24.3, 24.4, 24.5, 24.6 and 24.7 own the mechanisms.

2. Independence, Drawn

The property to read off the picture: there is no path from the DUT's output to the expected stream. Trace backwards from the scoreboard's two inputs and they meet only at the stimulus. If they meet anywhere else, §4 applies.

3. Requirement Traceability

One row per contract from 30.1 §4. A requirement is not verified until every column is filled.

Three readings.

A requirement with stimulus and no checker is not verified — the design was exercised and nobody looked. A requirement with a checker and no negative test is weakly verified: the checker exists and may be inert.

The last row is what the matrix is for. "Descriptor ownership handoff" has a contract from 30.1 §13 Q17 and nothing behind it. In a test-count-based review that gap is invisible; in this matrix it is one blank row.

And "Status: CLOSED" should mean the negative test ran and failed for the right reason — not that the checker exists. §7 makes that a gate.

4. The Circular Predictor

The most damaging verification failure available, because its symptom is universal success.

// WRONG. ILLUSTRATIVE. The expected stream is built from the DUT's own output.
// This happens by increments: someone needs a field the input side does not
// carry, takes it from the response "just for now", and the environment is
// hollow from then on.
class pcie_env extends uvm_env;
  function void connect_phase(uvm_phase phase);
    // The output monitor feeds BOTH the predictor and the scoreboard's actual
    // port. The two are now the same observation with extra steps.
    out_mon.ap.connect(predictor.analysis_export);   // <-- expected
    out_mon.ap.connect(sb.actual_export);            // <-- actual
    predictor.ap.connect(sb.expected_export);
  endfunction
endclass

Why it is so hard to spot. Nothing looks wrong at the call site. Each line is a legitimate TLM connection; the class names are right; the predictor genuinely computes something. The defect is in the graph, not in any statement — and code review reads statements.

What it produces.

The detection method is structural, and it takes minutes. Trace both scoreboard inputs backwards and find where they meet. They must meet at the stimulus and nowhere else. Any shared ancestor downstream of the DUT is a fail — and this should be an explicit review artefact, not a hope.

The corrected connection.

// CORRECT. Expected descends from the INPUT side only. If the input side does
// not carry enough information to predict the output, that is a finding about
// the reference model — not a licence to peek at the answer.
in_mon.ap.connect(predictor.analysis_export);      // expected: input-derived
predictor.ap.connect(sb.expected_export);
out_mon.ap.connect(sb.actual_export);              // actual: output-derived

And the confirming test is §8's mutation. Break one RTL condition. If the regression stays green, the environment is circular somewhere — this is the one check that detects the failure without reading any code.

5. Monitor Sampling

A monitor publishes events, and 30.2 §2's question applies unchanged: which event?

// ILLUSTRATIVE. A passive monitor. Every decision here is a review item.
class pcie_req_monitor extends uvm_monitor;
  `uvm_component_utils(pcie_req_monitor)
  virtual pcie_if      vif;
  uvm_analysis_port #(pcie_req_txn) ap;
 
  task run_phase(uvm_phase phase);
    pcie_req_txn tx;
    forever begin
      @(vif.mon_cb);
 
      // WRONG: if (vif.mon_cb.req_valid)
      //   -- a held offer under back-pressure publishes the SAME request once
      //      per cycle, and the scoreboard sees N requests where there was one.
      //      This is 30.2 §3's bug, reproduced in the testbench, where it is
      //      harder to see because nothing in the DUT is wrong.
      if (vif.mon_cb.req_valid && vif.mon_cb.req_ready) begin
        tx = pcie_req_txn::type_id::create("tx");
        tx.tag        = vif.mon_cb.req_tag;
        tx.bytes      = vif.mon_cb.req_bytes;
        tx.timestamp  = $time;
 
        // Publish a COPY. Analysis subscribers may queue the handle for many
        // cycles; reusing one object overwrites data the scoreboard has not
        // read yet, producing "wrong" comparisons with no DUT bug behind them.
        ap.write(tx);
      end
    end
  endtask
endclass

Six lenses. Architecture: a passive observer of the same events the RTL acts on, so DUT and model agree on what happened. State: none intentionally — a monitor that accumulates state is a second model, and it will drift. Event: req_valid && req_ready, sampled through a clocking block so there is no race with the driver. Contract: subscribers may hold the handle indefinitely, so each publish must be a fresh object. Failure: sampling valid alone duplicates transactions under back-pressure; reusing one object corrupts queued data and produces phantom mismatches that consume days. DV/debug: the mutation in §8 — change the sample condition and confirm the environment notices.

And the clocking block is not a formality. Sampling combinationally races the driver's non-blocking updates, and the resulting failures are seed-dependent and simulator-dependent — the worst debugging category, because reproduction is unreliable.

6. Scoreboard Identity

Comparing by order works until the design is allowed to reorder — and PCIe permits Completions to be split and delivered out of order (13.3 · 13.4).

// ILLUSTRATIVE. Identity-based scoreboard. Keyed by tag AND generation, so a
// reused tag is a different entry rather than the same one twice.
class pcie_scoreboard extends uvm_scoreboard;
  `uvm_component_utils(pcie_scoreboard)
 
  typedef struct {
    int unsigned bytes_expected;
    int unsigned bytes_seen;
    bit [1:0]    status_acc;
    time         issued;
  } entry_t;
 
  // Associative array keyed by {generation, tag} — NOT a queue. A queue encodes
  // an ordering assumption that the protocol does not owe us.
  entry_t m_live [bit [11:0]];
 
  int unsigned m_orphan, m_dup, m_stale;
 
  function bit [11:0] key(bit [3:0] gen, bit [7:0] tag);
    return {gen, tag};
  endfunction
 
  // Expected: a request was accepted.
  function void write_expected(pcie_req_txn tx);
    bit [11:0] k = key(tx.gen, tx.tag);
    if (m_live.exists(k))
      `uvm_error("SB", $sformatf("tag %0d gen %0d allocated while live", tx.tag, tx.gen))
    m_live[k] = '{bytes_expected: tx.bytes, bytes_seen: 0, status_acc: 0, issued: $time};
  endfunction
 
  // Actual: a Completion was observed.
  function void write_actual(pcie_cpl_txn tx);
    bit [11:0] k = key(tx.gen, tx.tag);
 
    if (!m_live.exists(k)) begin
      // Distinguish the two ways this happens — they are different bugs and
      // reporting them as one number loses the diagnosis (29.6 §7's lesson).
      if (m_seen_gen.exists(tx.tag) && (tx.gen != m_seen_gen[tx.tag])) begin
        m_stale++;   // a stale generation: 30.2 §7's tag-reuse hazard
        `uvm_error("SB", $sformatf("stale completion tag %0d gen %0d", tx.tag, tx.gen))
      end else begin
        m_orphan++;  // no request ever matched this
        `uvm_error("SB", $sformatf("orphan completion tag %0d", tx.tag))
      end
      return;
    end
 
    m_live[k].bytes_seen  += tx.bytes;
    m_live[k].status_acc  |= tx.status;      // accumulate — never overwrite
 
    if (m_live[k].bytes_seen > m_live[k].bytes_expected)
      `uvm_error("SB", $sformatf("over-delivery tag %0d", tx.tag))
 
    if (m_live[k].bytes_seen == m_live[k].bytes_expected) begin
      m_seen_gen[tx.tag] = tx.gen;
      m_live.delete(k);                       // retire exactly once
    end
  endfunction
 
  // The check most environments omit: nothing is still outstanding at the end.
  function void check_phase(uvm_phase phase);
    foreach (m_live[k])
      `uvm_error("SB", $sformatf("request never completed: key %0h, issued %0t",
                                 k, m_live[k].issued))
  endfunction
endclass

Six lenses. Architecture: an associative array keyed by identity, because the protocol does not owe us ordering. State: one entry per live request, with bytes and accumulated status. Event: write_expected on an accepted request; write_actual on an observed Completion; retirement only when the byte count is exactly met. Contract: the monitors must supply gen, which requires 30.2 §15 Q48's visibility — a scoreboard cannot check what the interface does not expose. Failure: using a queue silently asserts FIFO ordering (30.1 §9); deleting on the first Completion loses split data; |= replaced by = turns a failed transfer into a successful one. DV/debug: the three counters m_orphan, m_dup, m_stale distinguish three different bugs that a single "mismatch" count merges.

And check_phase is the quietly important part. A test that ends with entries still in m_live had requests that never completed, and without this loop the test passes. The most common real bug it catches is the environment's own — see §11.

7. Every Checker Must Have Been Seen to Fail

Rule: for each checker, a negative test exists, is run, and fails for the stated reason.

Three readings.

A checker that has never fired is a hypothesis. It may be mis-wired, mis-keyed, subscribed to the wrong port, or disabled by a filter — and none of those are visible while everything passes. The negative test converts it to evidence.

Row 4 is deliberately inverted and belongs on the list. Out-of-order delivery must pass. A "negative" suite where every case fails is testing the checkers' sensitivity and not their specificity — and an over-sensitive scoreboard that flags legal behaviour gets its errors demoted to warnings, which disables it.

And "fails for the stated reason" is stricter than "fails." A test that fails with a null-pointer error in the environment has not demonstrated the checker. The error message matters, which is why §6's scoreboard distinguishes stale from orphan.

8. Mutation Testing — the Gate That Cannot Be Faked

Break the RTL deliberately. If the environment stays green, verification is not closed.

Three readings.

This is the only check that detects a circular predictor without reading code (§4). A hollow environment cannot detect any mutation, so a single surviving mutation is a strong signal and a row of surviving mutations is a diagnosis.

A surviving mutation has exactly three explanations, and all three are findings: the stimulus never reaches that condition; the checker cannot see it; or the checker is not independent. None is acceptable at sign-off.

And the mutations should be drawn from 30.2's wrong-RTL patterns, not invented. The list above is the previous chapter's failure catalogue used as a test suite — which is the cheapest way to build a mutation set that targets realistic bugs rather than arbitrary ones.

9. Semantic Coverage

Coverage percentage is not the goal. The question is: which important combinations were never observed?

// ILLUSTRATIVE. Cross-coverage aimed at the conditions 30.2 says actually
// break endpoint RTL — simultaneity, occupancy extremes, and reset under load.
covergroup cg_pcie_semantic @(posedge clk);
  cp_class    : coverpoint req_class      { bins b[] = {BULK, STREAM, PROGRESS}; }
  cp_depth    : coverpoint outstanding_q  { bins low = {[0:7]};
                                            bins mid = {[8:47]};
                                            bins near_max = {[48:63]};
                                            bins at_max   = {64}; }
  cp_bp       : coverpoint req_ready      { bins stalled = {0}; bins open = {1}; }
  cp_reset    : coverpoint reset_kind     { bins none, recovery, func, local; }
  cp_err      : coverpoint err_kind       { bins none, cpl_status, timeout, ecrc; }
  cp_cfg      : coverpoint cfg_commit_fire{ bins idle = {0}; bins commit = {1}; }
 
  // The simultaneity bin. This is 30.2 §4's coincidence, and it will not arise
  // reliably from random stimulus at low occupancy.
  cp_simul    : coverpoint {alloc_fire, retire_fire} { bins both = {2'b11}; }
 
  // The crosses that matter. Each names a real failure from Module 29 or 30.2.
  x_depth_bp   : cross cp_depth, cp_bp;        // window binds under back-pressure
  x_reset_load : cross cp_reset, cp_depth;     // reset with work outstanding
  x_cfg_load   : cross cp_cfg,  cp_depth;      // commit while busy
  x_err_class  : cross cp_err,  cp_class;      // error on each class
endgroup

Six lenses. Architecture: coverage aimed at conditions, not lines. State: the sampled variables must be the same ones the checkers use, or coverage certifies a state the scoreboard never judged. Event: sampled every cycle here; a per-transaction group would miss cp_simul. Contract: every bin must be reachable — an unreachable bin is a permanently open item that eventually gets excluded, taking real gaps with it. Failure: at_max never hit means the outstanding limit was never reached, so 30.2 §15's bound is untested. DV/debug: x_reset_load with only the low depth bin filled means reset was only ever tested near-idle — §10's gap exactly.

And the review question is the inverse of the usual one. Not "what percentage?" but "list the crosses at zero, and justify each." Most will be genuinely unreachable or genuinely unimportant; the two or three that are neither are the finding.

10. Reset and Recovery Under Load

Testing reset from idle verifies that registers clear. It verifies nothing about lifetimes.

Two readings.

The scoreboard needs a reset contract of its own, and this is where environments quietly break: on reset, does the scoreboard discard live entries, retain them, or mark them abandoned? All three are implementable and only one matches the architecture (30.1 §6's matrix). An environment that discards will not notice the DUT retiring an orphan.

And these scenarios need a reset agent independent of the sequence, because the interesting timings are between transactions, not between tests. A reset that only ever happens at test boundaries tests nothing on this list.

11. End-of-Test Drain

// WRONG. ILLUSTRATIVE. The objection is dropped when the driver has taken the
// last item. Sequence completion is treated as semantic completion.
task body();
  repeat (N) begin
    `uvm_do(req)                  // driver accepts...
  end
  // BUG: the last request's Completion may be hundreds of cycles away. The
  //      test ends, the scoreboard's check_phase runs against a half-finished
  //      state, or shutdown begins before the response arrives at all.
endtask

Failure — the timeline.

Two outcomes, and both are bad. Either the test fails spuriously on a request that would have completed — and the usual fix is to delete the check_phase loop, removing a genuine checker; or check_phase is not reached in time and the test passes without ever checking the last transactions.

// CORRECT. Drain on SEMANTIC completion — outstanding work is zero — with an
// explicit bound so a hang fails loudly instead of hanging the regression.
task body();
  repeat (N) `uvm_do(req)
  fork
    begin : drain
      wait (sb.outstanding() == 0);
    end
    begin : watchdog
      #(DRAIN_TIMEOUT_NS * 1ns);
      `uvm_error("DRAIN", $sformatf("%0d requests outstanding at drain timeout",
                                    sb.outstanding()))
    end
  join_any
  disable fork;
endtask

Six lenses. Architecture: the test ends when the design is finished, not when the sequence is. State: the scoreboard's live count is the authority — it already exists (§6). Event: outstanding() == 0, or the watchdog. Contract: the drain bound must come from the architecture's completion-timeout policy (30.1 §13 Q11), not a round number. Failure: an unbounded wait converts a design hang into a regression hang, which is worse — it consumes a machine and produces no diagnosis. DV/debug: the error message reports the count, so a drain failure names how much work was stranded.

12. Assertions Versus Scoreboard Versus Formal

They catch different things, and using one for another's job leaves a hole.

The interface assertions the environment itself must carry. These are distinct from 30.2 §13's — those watch internal state, these watch the boundary and belong in the interface so every environment that binds it inherits them.

// ILLUSTRATIVE. Bound to the interface, not the DUT. These are the properties a
// testbench is entitled to assume, and the reason to put them here rather than
// in the scoreboard is that they fail at the CYCLE of the violation instead of
// at the transaction that eventually goes wrong.
interface pcie_req_if (input logic clk, input logic rst_n);
  logic        req_valid, req_ready;
  logic [7:0]  req_tag;
  logic [15:0] req_bytes;
 
  // English: an offer, once made, is not withdrawn before acceptance. Catches a
  // driver that pulls valid low on a stall — which corrupts the DUT's view and
  // is a TESTBENCH bug that presents as a DUT bug.
  a_if_no_retract: assert property (
    @(posedge clk) disable iff (!rst_n)
      (req_valid && !req_ready) |=> req_valid
  );
 
  // English: the payload is stable while the offer is pending. The mirror of
  // 30.2 §3's property, asserted from the other side of the boundary.
  a_if_stable: assert property (
    @(posedge clk) disable iff (!rst_n)
      (req_valid && !req_ready) |=> ($stable(req_tag) && $stable(req_bytes))
  );
 
  // English: no request is offered with a zero byte count. A degenerate request
  // is a stimulus bug, and without this it reaches the scoreboard as a
  // never-completing entry reported at check_phase — hundreds of cycles from
  // its cause.
  a_if_nonzero: assert property (
    @(posedge clk) disable iff (!rst_n)
      req_valid |-> (req_bytes != '0)
  );
 
  // English: a tag is not offered again while the scoreboard still has it live.
  // A stimulus-side guard: the sequence must not generate a colliding tag, and
  // this fires at generation rather than at the resulting mismatch.
  a_if_tag_not_reoffered: assert property (
    @(posedge clk) disable iff (!rst_n)
      (req_valid && req_ready) |-> !sb_tag_live[req_tag]
  );
endinterface

Six lenses. Architecture: interface-bound, so every environment inherits them and no testbench can omit them by accident. State: none of their own except the scoreboard's live vector, referenced read-only. Event: the same valid && ready boundary §5's monitor samples — deliberately the same event, so a disagreement between assertion and monitor is impossible. Contract: these state what the testbench owes the DUT, which is the half usually left unwritten; §7's negative tests must not violate them accidentally. Failure: without a_if_no_retract, a driver that withdraws an offer produces DUT behaviour that looks like an RTL bug and consumes days. DV/debug: each fires at the cycle of the violation rather than at the transaction that eventually mismatches, which is typically hundreds of cycles earlier and is the entire value.

And a_if_nonzero earns its place for a reason worth generalising. A zero-byte request is a stimulus defect, and without the assertion it surfaces as a scoreboard entry that never completes — reported by check_phase at end of test, maximally distant from its cause. Cheap boundary assertions convert late, confusing failures into immediate, obvious ones.

Two readings.

Formal's sweet spot here is unusually large — the first four are all small-state safety properties on structures (30.2 §6's tag table) that fit comfortably. These are the properties random stimulus is worst at, because they require specific simultaneous conditions.

And the last row is a boundary worth defending. Formal does not prove system throughput, and an assertion cannot check reassembled data content. Claiming either is how a gap gets closed on paper.

13. The Verification Checklist — 47 Questions

Plan and traceability

Environment independence

Monitors

Scoreboard

Stimulus and sequences

Coverage

Assertions and formal

Errors, reset, drain, debug

14. Misconceptions

"Ten thousand tests pass, so we're done." §1, §4: if the predictor is circular they would pass with no DUT at all.

"Coverage is at 98 %." §9: of lines. The question is which crosses are at zero, and whether each is justified.

"The checker is there, so the requirement is covered." §7: a checker that has never fired is a hypothesis — it may be mis-keyed, mis-connected or filtered out.

"Formal proves it." §12: formal proves small-state safety properties beautifully and does not prove throughput (22.1) or data content.

"Reset is tested." §10: from idle, which verifies that registers clear. The lifetime questions all live under load.

"item_done means the transaction finished." §11: it means the driver finished. The Completion may be hundreds of cycles away.

"Out-of-order delivery should fail the test." §7 row 4: it is legal. A suite where every negative case fails is testing sensitivity, not specificity — and an over-sensitive scoreboard gets demoted to warnings.

"The monitor is passive, so it can't be wrong." §5: sampling valid alone duplicates transactions; reusing one object corrupts queued data. Both produce failures with no DUT bug behind them.

15. Understanding Check

Q1. A regression has been green for eight months. What is the first thing to check?

Where the expected data comes from (§4). If the output monitor feeds both the predictor and the scoreboard's actual port, the scoreboard is comparing the DUT to itself and would have been green on day one for the same reason — including with a real bug injected, because the expected value moves with the actual. It is hard to spot because the defect is in the connection graph, not in any statement: every line of connect_phase is a legitimate TLM call, and code review reads statements.

Two checks, and the second needs no code reading. Structurally, trace both scoreboard inputs backwards and find their common ancestor — it must be the stimulus and nothing downstream of the DUT; this should be a review artefact, not a hope. Behaviourally, run mutations (§8): break valid && ready to valid, remove the generation term from the Completion match, change status_acc |= to =. A hollow environment cannot detect any of them, so a row of surviving mutations is a diagnosis rather than a hint.

Q2. Why is "the checker exists" not enough, and what is the standard instead?

Because a checker that has never fired is a hypothesis (§7). It may be subscribed to the wrong port, keyed on the wrong field, or disabled by a filter — and none of those are visible while everything passes. The standard is a negative test that ran and failed for the stated reason: inject a duplicate Completion, a stale generation, a missing Completion, a good-status-after-bad split.

Two refinements matter. "Fails for the stated reason" is stricter than "fails" — a null-pointer crash in the environment demonstrates nothing, which is why §6's scoreboard distinguishes m_stale from m_orphan from m_dup. And the negative suite must include a case that must PASS: out-of-order Completion delivery is legal (13.4), and a suite where everything fails is testing the checkers' sensitivity while ignoring their specificity. An over-sensitive scoreboard that flags legal behaviour gets its errors demoted to warnings, which disables it entirely.

Q3. A test drops its objection after the last uvm_do. What are the two outcomes?

Both bad, and one of them is silent (§11). The last request's Completion may be hundreds of cycles away. Either check_phase runs against a half-finished state and the test fails spuriously on a request that would have completed — and the usual "fix" is to delete the check_phase loop, removing a genuine checker — or shutdown outruns the check and the test passes without ever examining the final transactions.

The correct drain is on semantic completion: wait (sb.outstanding() == 0), using the scoreboard's live count, which §6 already maintains. It must be bounded, forked against a watchdog that reports the stranded count — because an unbounded wait converts a design hang into a regression hang, which consumes a machine and produces no diagnosis. And the bound should come from the architecture's completion-timeout policy (30.1 §13 Q11), not a round number.

Q4. Design the coverage that would catch the failure class 30.2 says actually breaks endpoint RTL.

Target simultaneity, occupancy extremes and reset under load — none of which are line coverage (§9). The specific bins: cp_simul = {alloc_fire, retire_fire} == 2'b11, the coincidence that 30.2 §4's two-NBA counter loses, which will not arise reliably from random stimulus at low occupancy; cp_depth.at_max, because an outstanding bound that is never reached is never tested; and cp_bp.stalled, because with ready always high the valid-versus-valid && ready bug cannot appear.

Then the crosses, each named after a real failure. x_reset_load — reset crossed with occupancy, because reset from idle only verifies that registers clear and every lifetime question lives under load (§10). x_cfg_load — commit crossed with outstanding work, which is a_cfg_stable_when_busy's stimulus. x_err_class — each error type on each traffic class.

And the review question is inverted from the usual one. Not "what percentage?" but "list every cross sitting at zero and justify each." Most will be genuinely unreachable or unimportant; the two or three that are neither are the finding. Exclusions must be reviewed rather than accumulated, because a long unreviewed exclusion list is where real gaps go to hide.

16. What Comes Next

This gate asked whether the environment can fail. The next asks whether any of it survives contact with the SoC.

30.4 covers the failures that appear only when the subsystem meets real clocks, resets, an address map, an IOMMU, firmware and software — disagreements between individually reasonable assumptions, where the PCIe traffic is perfectly legal and lands in the wrong memory.