USB · Module 27
Senior Silicon Debug
Walk a bring-up debug session from an analyser trace — narrowing by what the evidence excludes, never by what it suggests, because every USB symptom is consistent with five different faults.
The third senior question, and the only one in this module that cannot be prepared by reading a specification.
1. The Question
"A board has just come back from fab. The device enumerates intermittently. Here is an analyser trace. Walk me through it."
There is no correct sequence of steps to recite, which is exactly why it is asked. What the interviewer is watching is how you narrow — and specifically whether you narrow in the right direction.
2. Why the Suggestive Direction Fails
Because the symptom is shared. USB has a small number of ways to report that something went wrong, and a large number of things that go wrong — so every symptom is many-to-one, and reasoning from symptom to cause is reasoning backwards along a collapsing map.
| Symptom in the trace | Causes it is consistent with |
|---|---|
| CRC errors | signal integrity · unsolicited device traffic · hub repeater · brown-out · host turnaround |
| Timeouts | device stalled its own control endpoint · address applied early · power · firmware hang |
| Unexpected NAKs | buffer never filled · DMA not started · endpoint not configured · toggle desync |
| Babble | device clock wrong · length field wrong · hub translator |
| Works when plugged in directly | hub translator · cable · power budget · or a device bug the hub masks |
The last row is the one that ends careers-worth of debugging time. "It works without the hub" feels like a hub problem and is very often a device problem that a forgiving hub happens to paper over — the exact shape of chapter 26.3's AXI boundary bug, one layer up.
Reasoning FROM the symptom:
CRC errors -> "probably signal integrity"
-> reflow the board, swap the cable, scope D+/D-
-> two days, no change, and the shortlist never shrank
Reasoning FROM exclusion:
one CLEAN packet from the device
-> a marginal PHY does not selectively corrupt
-> signal integrity is OUT
a TIMEOUT
-> a device that talks too much does not go silent
-> unsolicited traffic is OUT
a STALL
-> a considered response means packets arrive intact both ways
-> the hub is OUT
Three events. Three classes gone. Nothing was guessed.3. What We Are Building
usb_trace_triage is that discipline as hardware. It does not classify failures. For each candidate fault class it holds the evidence that would rule it out, and reports which classes remain possible.
The output is not a diagnosis. It is a shortlist, and its most valuable field is how many classes are still alive.
Six hypotheses, eliminated by contradiction
A session narrowing, one piece of evidence at a time
Cycle 8 is the case nobody builds and everybody needs: the shortlist empties, and conclusive goes low. A trace that contradicts every hypothesis is informative — the fault is outside the model — but it is not a diagnosis, and reporting it as one would be the worst possible outcome.
4. The Six Elimination Rules
Every one has the form "if X is in the trace then class Y is impossible." None has the form "X means Y", and that asymmetry is the design.
| Evidence | Rules out | Because |
|---|---|---|
| a clean packet from the device | signal integrity | a marginal PHY does not selectively corrupt |
| an ACK at a non-zero address | address applied early | the device is reachable where the host thinks it is |
| two DATA packets with different toggles | toggle desynchronised | the sequence is advancing correctly |
| a timeout | unsolicited transmission | a device that talks too much does not go silent |
| a STALL | hub fault | a considered response means packets arrive intact both ways |
| activity outside the post-reset window | brown-out | a device that resets under load does not keep running |
5. Seven Properties
| # | Property |
|---|---|
| 1 | Every class starts possible. |
| 2 | A class is eliminated only by evidence that contradicts it. |
| 3 | A class is eliminated the moment something contradicts it. |
| 4 | Elimination is monotonic — nothing comes back. |
| 5 | n_alive is the number of surviving classes. |
| 6 | conclusive means exactly one survives. |
| 7 | An empty shortlist is not conclusive. |
Properties 2 and 3 are the pair that matters, and 2 is the dangerous one. A tool that eliminates the real fault sends the whole team to look somewhere else — with the authority of a shortlist.
6. Verilog-2005 RTL
// =====================================================================
// usb_trace_triage -- "Walk a bring-up debug session" in hardware.
//
// An analyser trace of a failing USB bring-up contains millions of
// packets and one interesting moment. The skill the question tests is
// not reading packets; it is NARROWING -- and the discipline that makes
// narrowing work is counter-intuitive:
//
// Narrow by what the evidence EXCLUDES, not by what it suggests.
//
// A trace that shows CRC errors SUGGESTS signal integrity. What it
// EXCLUDES is nothing at all, because a CRC error is also what a device
// driving the bus at the wrong time looks like, and what a hub with a
// failing repeater looks like, and what a host with a marginal
// turnaround timeout looks like.
//
// So this module does not classify failures. It maintains, for each
// candidate fault class, the evidence that would RULE IT OUT -- and
// reports which classes remain possible. A class is eliminated when
// something in the trace is inconsistent with it, never when something
// merely looks like something else.
//
// The output is therefore not a diagnosis. It is a SHORTLIST, and the
// most valuable field on it is how many classes are still alive.
// =====================================================================
module usb_trace_triage (
input wire clk,
input wire rst_n,
// ---- one trace event per cycle ----
input wire ev_valid,
input wire [2:0] ev_kind,
// ---- attributes of the event, as an analyser would report them ----
input wire ev_from_device, // the device transmitted it
input wire ev_crc_bad,
input wire [6:0] ev_addr,
input wire ev_addr_is_zero,
input wire ev_after_reset, // within the post-reset window
input wire ev_toggle,
// ---- which fault classes are STILL POSSIBLE ----
//
// All start possible. Each is cleared by evidence that CONTRADICTS it.
output wire c_signal, // signal integrity / marginal PHY
output wire c_addr_early, // SET_ADDRESS applied too early
output wire c_toggle, // data toggle desynchronised
output wire c_unsolicited, // device transmits without a token
output wire c_hub, // hub repeater or translator fault
output wire c_power, // brown-out / insufficient current
// ---- the shortlist, and the number that matters ----
output wire [2:0] n_alive,
output wire conclusive, // exactly one class remains
// ---- observability ----
output wire [31:0] n_events,
output wire [31:0] n_crc_bad,
output wire [31:0] n_dev_tx,
output wire [31:0] n_eliminations
);
localparam [2:0] E_SOF = 3'd0,
E_TOKEN = 3'd1,
E_DATA = 3'd2,
E_ACK = 3'd3,
E_NAK = 3'd4,
E_STALL = 3'd5,
E_TIMEOUT = 3'd6,
E_RESET = 3'd7;
// Each bit is "this class is still possible". They start SET.
reg c_sig_r, c_addr_r, c_tog_r, c_unsol_r, c_hub_r, c_pwr_r;
reg [31:0] ev_c, crc_c, devtx_c, elim_c;
// toggle history, for the one class that needs a sequence rather than
// a single event
reg last_tog;
reg have_tog;
assign c_signal = c_sig_r;
assign c_addr_early = c_addr_r;
assign c_toggle = c_tog_r;
assign c_unsolicited = c_unsol_r;
assign c_hub = c_hub_r;
assign c_power = c_pwr_r;
assign n_events = ev_c;
assign n_crc_bad = crc_c;
assign n_dev_tx = devtx_c;
assign n_eliminations = elim_c;
// ---- the shortlist size ----
wire [2:0] alive = {2'd0, c_sig_r} + {2'd0, c_addr_r} + {2'd0, c_tog_r}
+ {2'd0, c_unsol_r} + {2'd0, c_hub_r} + {2'd0, c_pwr_r};
assign n_alive = alive;
// ---- CONCLUSIVE means exactly one class survives ----
//
// Not "one class looks likely". A trace that eliminates five of six is
// a diagnosis; a trace that makes one of six look probable is a guess,
// and the whole point of this module is that it refuses to report the
// second as though it were the first.
assign conclusive = (alive == 3'd1);
// =================================================================
// THE ELIMINATION RULES.
//
// Each is a statement of the form "if X is in the trace then class Y
// is impossible". Note that NONE of them says "X means Y" -- that is
// the direction that produces wrong diagnoses, and the direction
// every one of these deliberately avoids.
// =================================================================
// A CLEAN packet from the device rules out signal integrity as the
// WHOLE story: a marginal PHY does not selectively corrupt.
wire e_sig = ev_valid && ev_from_device && !ev_crc_bad;
// Any successful transaction at a NON-ZERO address rules out the
// address applying early: the device is reachable where the host
// thinks it is.
wire e_addr = ev_valid && (ev_kind == E_ACK) && !ev_addr_is_zero;
// Two consecutive DATA packets with DIFFERENT toggles rule out a
// desynchronised toggle.
wire e_tog = ev_valid && (ev_kind == E_DATA) && have_tog
&& (ev_toggle != last_tog);
// A TIMEOUT rules out unsolicited transmission as the cause: a device
// that talks too much does not produce silence.
wire e_unsol = ev_valid && (ev_kind == E_TIMEOUT);
// A STALL rules out a hub fault: a stall is a considered response from
// a device whose packets are therefore arriving intact in both
// directions.
wire e_hub = ev_valid && (ev_kind == E_STALL);
// Activity outside the post-reset window rules out brown-out: a device
// that resets under load does not keep running.
wire e_pwr = ev_valid && !ev_after_reset && (ev_kind != E_RESET);
wire [2:0] elim_now = {2'd0, (e_sig && c_sig_r)}
+ {2'd0, (e_addr && c_addr_r)}
+ {2'd0, (e_tog && c_tog_r)}
+ {2'd0, (e_unsol && c_unsol_r)}
+ {2'd0, (e_hub && c_hub_r)}
+ {2'd0, (e_pwr && c_pwr_r)};
always @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
// Everything starts possible. A triage tool that begins with a
// favourite has already made the mistake it exists to prevent.
c_sig_r <= 1'b1;
c_addr_r <= 1'b1;
c_tog_r <= 1'b1;
c_unsol_r <= 1'b1;
c_hub_r <= 1'b1;
c_pwr_r <= 1'b1;
ev_c <= 32'd0;
crc_c <= 32'd0;
devtx_c <= 32'd0;
elim_c <= 32'd0;
last_tog <= 1'b0;
have_tog <= 1'b0;
end else begin
if (ev_valid) begin
ev_c <= ev_c + 32'd1;
if (ev_crc_bad) crc_c <= crc_c + 32'd1;
if (ev_from_device) devtx_c <= devtx_c + 32'd1;
// ---- elimination is MONOTONIC ----
//
// A class, once ruled out, stays ruled out. Evidence does not
// expire, and a later event that merely looks consistent with a
// class cannot revive it -- which is what stops this from
// oscillating and what makes the shortlist mean something.
if (e_sig) c_sig_r <= 1'b0;
if (e_addr) c_addr_r <= 1'b0;
if (e_tog) c_tog_r <= 1'b0;
if (e_unsol) c_unsol_r <= 1'b0;
if (e_hub) c_hub_r <= 1'b0;
if (e_pwr) c_pwr_r <= 1'b0;
// One add of the combinational count, not six increments of one
// register: six non-blocking assignments to elim_c would be six
// writes and only the last would take effect.
if (elim_now != 3'd0) elim_c <= elim_c + {29'd0, elim_now};
// toggle history advances on every DATA packet
if (ev_kind == E_DATA) begin
last_tog <= ev_toggle;
have_tog <= 1'b1;
end
end
end
end
endmodule7. SystemVerilog RTL
// =====================================================================
// usb_trace_triage -- SystemVerilog.
//
// The event kinds and the fault classes become named enumerations, which
// matters here more than usual: the output of this module is a SHORTLIST
// of hypotheses, and a shortlist printed as `3'b101` is not a shortlist
// anybody can act on.
//
// Originally: "Walk a bring-up debug session" in hardware.
//
// An analyser trace of a failing USB bring-up contains millions of
// packets and one interesting moment. The skill the question tests is
// not reading packets; it is NARROWING -- and the discipline that makes
// narrowing work is counter-intuitive:
//
// Narrow by what the evidence EXCLUDES, not by what it suggests.
//
// A trace that shows CRC errors SUGGESTS signal integrity. What it
// EXCLUDES is nothing at all, because a CRC error is also what a device
// driving the bus at the wrong time looks like, and what a hub with a
// failing repeater looks like, and what a host with a marginal
// turnaround timeout looks like.
//
// So this module does not classify failures. It maintains, for each
// candidate fault class, the evidence that would RULE IT OUT -- and
// reports which classes remain possible. A class is eliminated when
// something in the trace is inconsistent with it, never when something
// merely looks like something else.
//
// The output is therefore not a diagnosis. It is a SHORTLIST, and the
// most valuable field on it is how many classes are still alive.
// =====================================================================
module usb_trace_triage (
input logic clk,
input logic rst_n,
// ---- one trace event per cycle ----
input logic ev_valid,
input logic [2:0] ev_kind,
// ---- attributes of the event, as an analyser would report them ----
input logic ev_from_device, // the device transmitted it
input logic ev_crc_bad,
input logic [6:0] ev_addr,
input logic ev_addr_is_zero,
input logic ev_after_reset, // within the post-reset window
input logic ev_toggle,
// ---- which fault classes are STILL POSSIBLE ----
//
// All start possible. Each is cleared by evidence that CONTRADICTS it.
output logic c_signal, // signal integrity / marginal PHY
output logic c_addr_early, // SET_ADDRESS applied too early
output logic c_toggle, // data toggle desynchronised
output logic c_unsolicited, // device transmits without a token
output logic c_hub, // hub repeater or translator fault
output logic c_power, // brown-out / insufficient current
// ---- the shortlist, and the number that matters ----
output logic [2:0] n_alive,
output logic conclusive, // exactly one class remains
// ---- observability ----
output logic [31:0]n_events,
output logic [31:0]n_crc_bad,
output logic [31:0]n_dev_tx,
output logic [31:0]n_eliminations
);
// A shortlist printed as 3'b101 is not a shortlist anybody can act on.
typedef enum logic [2:0] {
E_SOF = 3'd0,
E_TOKEN = 3'd1,
E_DATA = 3'd2,
E_ACK = 3'd3,
E_NAK = 3'd4,
E_STALL = 3'd5,
E_TIMEOUT = 3'd6,
E_RESET = 3'd7
} ev_e;
// Each bit is "this class is still possible". They start SET.
logic c_sig_r, c_addr_r, c_tog_r, c_unsol_r, c_hub_r, c_pwr_r;
logic [31:0] ev_c, crc_c, devtx_c, elim_c;
// toggle history, for the one class that needs a sequence rather than
// a single event
logic last_tog;
logic have_tog;
assign c_signal = c_sig_r;
assign c_addr_early = c_addr_r;
assign c_toggle = c_tog_r;
assign c_unsolicited = c_unsol_r;
assign c_hub = c_hub_r;
assign c_power = c_pwr_r;
assign n_events = ev_c;
assign n_crc_bad = crc_c;
assign n_dev_tx = devtx_c;
assign n_eliminations = elim_c;
// ---- the shortlist size ----
wire [2:0] alive = {2'd0, c_sig_r} + {2'd0, c_addr_r} + {2'd0, c_tog_r}
+ {2'd0, c_unsol_r} + {2'd0, c_hub_r} + {2'd0, c_pwr_r};
assign n_alive = alive;
// ---- CONCLUSIVE means exactly one class survives ----
//
// Not "one class looks likely". A trace that eliminates five of six is
// a diagnosis; a trace that makes one of six look probable is a guess,
// and the whole point of this module is that it refuses to report the
// second as though it were the first.
assign conclusive = (alive == 3'd1);
// =================================================================
// THE ELIMINATION RULES.
//
// Each is a statement of the form "if X is in the trace then class Y
// is impossible". Note that NONE of them says "X means Y" -- that is
// the direction that produces wrong diagnoses, and the direction
// every one of these deliberately avoids.
// =================================================================
// A CLEAN packet from the device rules out signal integrity as the
// WHOLE story: a marginal PHY does not selectively corrupt.
wire e_sig = ev_valid && ev_from_device && !ev_crc_bad;
// Any successful transaction at a NON-ZERO address rules out the
// address applying early: the device is reachable where the host
// thinks it is.
wire e_addr = ev_valid && (ev_kind == E_ACK) && !ev_addr_is_zero;
// Two consecutive DATA packets with DIFFERENT toggles rule out a
// desynchronised toggle.
wire e_tog = ev_valid && (ev_kind == E_DATA) && have_tog
&& (ev_toggle != last_tog);
// A TIMEOUT rules out unsolicited transmission as the cause: a device
// that talks too much does not produce silence.
wire e_unsol = ev_valid && (ev_kind == E_TIMEOUT);
// A STALL rules out a hub fault: a stall is a considered response from
// a device whose packets are therefore arriving intact in both
// directions.
wire e_hub = ev_valid && (ev_kind == E_STALL);
// Activity outside the post-reset window rules out brown-out: a device
// that resets under load does not keep running.
wire e_pwr = ev_valid && !ev_after_reset && (ev_kind != E_RESET);
wire [2:0] elim_now = {2'd0, (e_sig && c_sig_r)}
+ {2'd0, (e_addr && c_addr_r)}
+ {2'd0, (e_tog && c_tog_r)}
+ {2'd0, (e_unsol && c_unsol_r)}
+ {2'd0, (e_hub && c_hub_r)}
+ {2'd0, (e_pwr && c_pwr_r)};
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
// Everything starts possible. A triage tool that begins with a
// favourite has already made the mistake it exists to prevent.
c_sig_r <= 1'b1;
c_addr_r <= 1'b1;
c_tog_r <= 1'b1;
c_unsol_r <= 1'b1;
c_hub_r <= 1'b1;
c_pwr_r <= 1'b1;
ev_c <= 32'd0;
crc_c <= 32'd0;
devtx_c <= 32'd0;
elim_c <= 32'd0;
last_tog <= 1'b0;
have_tog <= 1'b0;
end else begin
if (ev_valid) begin
ev_c <= ev_c + 32'd1;
if (ev_crc_bad) crc_c <= crc_c + 32'd1;
if (ev_from_device) devtx_c <= devtx_c + 32'd1;
// ---- elimination is MONOTONIC ----
//
// A class, once ruled out, stays ruled out. Evidence does not
// expire, and a later event that merely looks consistent with a
// class cannot revive it -- which is what stops this from
// oscillating and what makes the shortlist mean something.
if (e_sig) c_sig_r <= 1'b0;
if (e_addr) c_addr_r <= 1'b0;
if (e_tog) c_tog_r <= 1'b0;
if (e_unsol) c_unsol_r <= 1'b0;
if (e_hub) c_hub_r <= 1'b0;
if (e_pwr) c_pwr_r <= 1'b0;
// One add of the combinational count, not six increments of one
// register: six non-blocking assignments to elim_c would be six
// writes and only the last would take effect.
if (elim_now != 3'd0) elim_c <= elim_c + {29'd0, elim_now};
// toggle history advances on every DATA packet
if (ev_kind == E_DATA) begin
last_tog <= ev_toggle;
have_tog <= 1'b1;
end
end
end
end
endmodule8. VHDL-2008 RTL
-- =====================================================================
-- usb_trace_triage -- VHDL-2008.
--
-- An analyser trace of a failing USB bring-up contains millions of
-- packets and one interesting moment. The skill the interview tests is
-- not reading packets; it is NARROWING -- and the discipline that makes
-- narrowing work is counter-intuitive:
--
-- Narrow by what the evidence EXCLUDES, not by what it suggests.
--
-- A trace full of CRC errors SUGGESTS signal integrity. It EXCLUDES
-- nothing, because a CRC error is also what a device driving the bus at
-- the wrong time looks like, and what a failing hub repeater looks like,
-- and what a marginal host turnaround looks like.
--
-- So this entity does not classify. For each candidate fault class it
-- maintains the evidence that would RULE IT OUT, and reports which
-- classes remain possible. The output is a SHORTLIST, and its most
-- valuable field is how many classes are still alive.
-- =====================================================================
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
package td_pkg is
type ev_t is (EV_SOF, EV_TOKEN, EV_DATA, EV_ACK,
EV_NAK, EV_STALL, EV_TIMEOUT, EV_RESET);
function ev_of(v : std_logic_vector(2 downto 0)) return ev_t;
end package;
package body td_pkg is
function ev_of(v : std_logic_vector(2 downto 0)) return ev_t is
begin
case v is
when "000" => return EV_SOF;
when "001" => return EV_TOKEN;
when "010" => return EV_DATA;
when "011" => return EV_ACK;
when "100" => return EV_NAK;
when "101" => return EV_STALL;
when "110" => return EV_TIMEOUT;
when others => return EV_RESET;
end case;
end function;
end package body;
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
use work.td_pkg.all;
entity usb_trace_triage is
port (
clk : in std_logic;
rst_n : in std_logic;
ev_valid : in std_logic;
ev_kind : in std_logic_vector(2 downto 0);
ev_from_device : in std_logic;
ev_crc_bad : in std_logic;
ev_addr : in std_logic_vector(6 downto 0);
ev_addr_is_zero : in std_logic;
ev_after_reset : in std_logic;
ev_toggle : in std_logic;
-- which fault classes are STILL POSSIBLE. All start possible; each is
-- cleared only by evidence that CONTRADICTS it.
c_signal : out std_logic;
c_addr_early : out std_logic;
c_toggle : out std_logic;
c_unsolicited : out std_logic;
c_hub : out std_logic;
c_power : out std_logic;
n_alive : out std_logic_vector(2 downto 0);
conclusive : out std_logic;
n_events : out std_logic_vector(31 downto 0);
n_crc_bad : out std_logic_vector(31 downto 0);
n_dev_tx : out std_logic_vector(31 downto 0);
n_eliminations : out std_logic_vector(31 downto 0)
);
end entity;
architecture rtl of usb_trace_triage is
signal c_sig_r, c_addr_r, c_tog_r : std_logic := '1';
signal c_unsol_r, c_hub_r, c_pwr_r : std_logic := '1';
signal ev_c, crc_c, devtx_c, elim_c : unsigned(31 downto 0)
:= (others => '0');
signal last_tog, have_tog : std_logic := '0';
signal e_sig, e_addr, e_tog, e_unsol, e_hub, e_pwr : std_logic;
signal alive, elim_now : natural range 0 to 6;
signal kind : ev_t;
function b2n(b : std_logic) return natural is
begin
if b = '1' then return 1; else return 0; end if;
end function;
begin
kind <= ev_of(ev_kind);
c_signal <= c_sig_r;
c_addr_early <= c_addr_r;
c_toggle <= c_tog_r;
c_unsolicited <= c_unsol_r;
c_hub <= c_hub_r;
c_power <= c_pwr_r;
n_events <= std_logic_vector(ev_c);
n_crc_bad <= std_logic_vector(crc_c);
n_dev_tx <= std_logic_vector(devtx_c);
n_eliminations <= std_logic_vector(elim_c);
alive <= b2n(c_sig_r) + b2n(c_addr_r) + b2n(c_tog_r)
+ b2n(c_unsol_r) + b2n(c_hub_r) + b2n(c_pwr_r);
n_alive <= std_logic_vector(to_unsigned(alive, 3));
-- ---- CONCLUSIVE means exactly ONE class survives ----
--
-- Not "one class looks likely". A trace that eliminates five of six is a
-- diagnosis; one that makes a class look probable is a guess, and this
-- entity refuses to report the second as though it were the first.
--
-- Nor is an EMPTY shortlist conclusive: a trace that contradicts every
-- hypothesis is informative -- the fault is outside the model -- but it
-- is not a diagnosis.
conclusive <= '1' when alive = 1 else '0';
-- =================================================================
-- THE ELIMINATION RULES.
--
-- Each has the form "if X is in the trace then class Y is impossible".
-- None says "X means Y" -- that is the direction that produces wrong
-- diagnoses, and every one of these deliberately avoids it.
-- =================================================================
-- A CLEAN packet from the device rules out signal integrity as the whole
-- story: a marginal PHY does not selectively corrupt.
e_sig <= '1' when (ev_valid = '1' and ev_from_device = '1'
and ev_crc_bad = '0') else '0';
-- Any successful transaction at a NON-ZERO address rules out the address
-- applying early: the device is reachable where the host thinks it is.
e_addr <= '1' when (ev_valid = '1' and kind = EV_ACK
and ev_addr_is_zero = '0') else '0';
-- Two consecutive DATA packets with DIFFERENT toggles rule out a
-- desynchronised toggle. The have_tog term matters: a single DATA packet
-- is not evidence about a sequence.
e_tog <= '1' when (ev_valid = '1' and kind = EV_DATA and have_tog = '1'
and ev_toggle /= last_tog) else '0';
-- A TIMEOUT rules out unsolicited transmission: a device that talks too
-- much does not produce silence.
e_unsol <= '1' when (ev_valid = '1' and kind = EV_TIMEOUT) else '0';
-- A STALL rules out a hub fault: a stall is a considered response from a
-- device whose packets are therefore arriving intact in both directions.
e_hub <= '1' when (ev_valid = '1' and kind = EV_STALL) else '0';
-- Activity outside the post-reset window rules out brown-out: a device
-- that resets under load does not keep running.
e_pwr <= '1' when (ev_valid = '1' and ev_after_reset = '0'
and kind /= EV_RESET) else '0';
elim_now <= b2n(e_sig and c_sig_r) + b2n(e_addr and c_addr_r)
+ b2n(e_tog and c_tog_r) + b2n(e_unsol and c_unsol_r)
+ b2n(e_hub and c_hub_r) + b2n(e_pwr and c_pwr_r);
main : process(clk, rst_n)
begin
if rst_n = '0' then
-- Everything starts possible. A triage tool that begins with a
-- favourite has already made the mistake it exists to prevent.
c_sig_r <= '1';
c_addr_r <= '1';
c_tog_r <= '1';
c_unsol_r <= '1';
c_hub_r <= '1';
c_pwr_r <= '1';
ev_c <= (others => '0');
crc_c <= (others => '0');
devtx_c <= (others => '0');
elim_c <= (others => '0');
last_tog <= '0';
have_tog <= '0';
elsif rising_edge(clk) then
if ev_valid = '1' then
ev_c <= ev_c + 1;
if ev_crc_bad = '1' then crc_c <= crc_c + 1; end if;
if ev_from_device = '1' then devtx_c <= devtx_c + 1; end if;
-- ---- elimination is MONOTONIC ----
--
-- A class, once ruled out, stays ruled out. Evidence does not
-- expire, and a later event that merely looks consistent with a
-- class cannot revive it -- which is what stops the shortlist
-- oscillating and what makes its size mean something.
if e_sig = '1' then c_sig_r <= '0'; end if;
if e_addr = '1' then c_addr_r <= '0'; end if;
if e_tog = '1' then c_tog_r <= '0'; end if;
if e_unsol = '1' then c_unsol_r <= '0'; end if;
if e_hub = '1' then c_hub_r <= '0'; end if;
if e_pwr = '1' then c_pwr_r <= '0'; end if;
-- One add of the combinational count. Six increments of one signal
-- would be six assignments and only the last would take effect.
if elim_now /= 0 then
elim_c <= elim_c + to_unsigned(elim_now, 32);
end if;
if kind = EV_DATA then
last_tog <= ev_toggle;
have_tog <= '1';
end if;
end if;
end if;
end process;
end architecture;9. The Testbench: Two Sides of One Property
A triage tool is verified the way chapter 27.8's checker is, and for the same reason — the two failure directions are not symmetric.
NO FALSE ELIMINATION -- a class must never be ruled out by
evidence that does not contradict it.
NO FALSE SURVIVAL -- a class must be ruled out the moment
something does.
The first is the dangerous one. A tool that eliminates the
REAL fault does not merely fail to help -- it actively
misdirects, and it does so with the authority of a
shortlist.So the shadow recomputes all six elimination conditions independently from the event attributes, and compares all six classes every cycle — not just the one that changed. A class cleared as a side effect of another's evidence would otherwise go unnoticed, and that is precisely mutation I5.
Verilog-2005 testbench
// =====================================================================
// Testbench for usb_trace_triage.
//
// The property that matters is NOT "does it find the right fault". It is
// the two-sided one that a triage tool lives or dies by:
//
// NO FALSE ELIMINATION -- a class is never ruled out by evidence that
// does not actually contradict it.
// NO FALSE SURVIVAL -- a class IS ruled out the moment something
// contradicts it.
//
// The first is the dangerous direction. A tool that eliminates the real
// fault sends the whole team to look somewhere else, and it does so with
// the authority of a shortlist.
//
// So the shadow recomputes all six elimination conditions from the event
// attributes, independently, and compares the full set every cycle --
// not just the class that changed.
// =====================================================================
`timescale 1ns/1ps
module tb_td_v;
localparam [2:0] E_SOF = 3'd0, E_TOKEN = 3'd1, E_DATA = 3'd2, E_ACK = 3'd3,
E_NAK = 3'd4, E_STALL = 3'd5, E_TIMEOUT = 3'd6,
E_RESET = 3'd7;
reg clk = 1'b0, rst_n = 1'b0;
reg ev_valid = 1'b0;
reg [2:0] ev_kind = E_SOF;
reg ev_from_device = 1'b0;
reg ev_crc_bad = 1'b0;
reg [6:0] ev_addr = 7'd0;
reg ev_addr_is_zero = 1'b1;
reg ev_after_reset = 1'b1;
reg ev_toggle = 1'b0;
wire c_signal, c_addr_early, c_toggle, c_unsolicited, c_hub, c_power;
wire [2:0] n_alive;
wire conclusive;
wire [31:0] n_events, n_crc_bad, n_dev_tx, n_eliminations;
usb_trace_triage dut (
.clk(clk), .rst_n(rst_n),
.ev_valid(ev_valid), .ev_kind(ev_kind),
.ev_from_device(ev_from_device), .ev_crc_bad(ev_crc_bad),
.ev_addr(ev_addr), .ev_addr_is_zero(ev_addr_is_zero),
.ev_after_reset(ev_after_reset), .ev_toggle(ev_toggle),
.c_signal(c_signal), .c_addr_early(c_addr_early), .c_toggle(c_toggle),
.c_unsolicited(c_unsolicited), .c_hub(c_hub), .c_power(c_power),
.n_alive(n_alive), .conclusive(conclusive),
.n_events(n_events), .n_crc_bad(n_crc_bad),
.n_dev_tx(n_dev_tx), .n_eliminations(n_eliminations)
);
always #5 clk = ~clk;
integer errors = 0, checks = 0, steps = 0;
integer seed;
function [31:0] urand;
input dummy;
begin urand = $random(seed) & 32'h3FFF_FFFF; end
endfunction
// ---- the shadow: six independent booleans ----
reg s_sig, s_addr, s_tog, s_unsol, s_hub, s_pwr;
reg s_last_tog, s_have_tog;
reg [31:0] x_ev, x_crc, x_dev, x_elim;
// ---- the two headline counters ----
// Run-wide totals: the DUT's counters are cleared by every reset, and
// the random phase resets every 16 events.
integer g_ev = 0, g_crc = 0, g_dev = 0, g_elim = 0;
integer n_false_elim = 0; // a class cleared without contradiction
integer n_false_surv = 0; // a class survived a contradiction
// ---- exhaustive reach over (kind, from_dev, crc, addr0, after_reset) --
// The event TOGGLE is a sixth dimension, not a constant. Without it the
// first DATA packet of every scenario carries toggle 0, which equals the
// reset value of the history -- so a rule that wrongly ignores "is there
// any history" never fires, and mutation I6 scored 0 directed.
reg reach [0:255];
integer ri, n_reach;
task ck(input cond, input [255:0] what);
begin
checks = checks + 1;
if (!cond) begin
errors = errors + 1;
if (errors <= 20)
$display(" ERROR @%0t step=%0d: %0s", $time, steps, what);
end
end
endtask
// ---------------------------------------------------------------
// One trace event.
// ---------------------------------------------------------------
task ev(input v, input [2:0] k, input fd, input cb,
input az, input ar, input tg);
reg p_sig, p_addr, p_tog, p_unsol, p_hub, p_pwr;
reg q_sig, q_addr, q_tog, q_unsol, q_hub, q_pwr;
reg [2:0] e_alive;
begin
ev_valid = v; ev_kind = k; ev_from_device = fd;
ev_crc_bad = cb; ev_addr_is_zero = az;
ev_after_reset = ar; ev_toggle = tg;
ev_addr = az ? 7'd0 : 7'd42;
// remember the state BEFORE this event, for the monotonicity check
p_sig = s_sig; p_addr = s_addr; p_tog = s_tog;
p_unsol = s_unsol; p_hub = s_hub; p_pwr = s_pwr;
// ---- the six elimination conditions, recomputed independently ----
q_sig = v && fd && !cb;
q_addr = v && (k == E_ACK) && !az;
q_tog = v && (k == E_DATA) && s_have_tog && (tg != s_last_tog);
q_unsol = v && (k == E_TIMEOUT);
q_hub = v && (k == E_STALL);
q_pwr = v && !ar && (k != E_RESET);
if (v) begin
x_ev = x_ev + 1; g_ev = g_ev + 1;
if (cb) begin x_crc = x_crc + 1; g_crc = g_crc + 1; end
if (fd) begin x_dev = x_dev + 1; g_dev = g_dev + 1; end
x_elim = x_elim
+ ((q_sig && s_sig) ? 1 : 0)
+ ((q_addr && s_addr) ? 1 : 0)
+ ((q_tog && s_tog) ? 1 : 0)
+ ((q_unsol && s_unsol) ? 1 : 0)
+ ((q_hub && s_hub) ? 1 : 0)
+ ((q_pwr && s_pwr) ? 1 : 0);
g_elim = g_elim
+ ((q_sig && s_sig) ? 1 : 0)
+ ((q_addr && s_addr) ? 1 : 0)
+ ((q_tog && s_tog) ? 1 : 0)
+ ((q_unsol && s_unsol) ? 1 : 0)
+ ((q_hub && s_hub) ? 1 : 0)
+ ((q_pwr && s_pwr) ? 1 : 0);
if (q_sig) s_sig = 1'b0;
if (q_addr) s_addr = 1'b0;
if (q_tog) s_tog = 1'b0;
if (q_unsol) s_unsol = 1'b0;
if (q_hub) s_hub = 1'b0;
if (q_pwr) s_pwr = 1'b0;
if (k == E_DATA) begin
s_last_tog = tg;
s_have_tog = 1'b1;
end
end
@(posedge clk);
#1;
steps = steps + 1;
ev_valid = 1'b0;
// ---- PROPERTY 1: every class matches the shadow ----
//
// All six compared every cycle, not just the one that changed. A
// class cleared as a side effect of another's evidence would
// otherwise go unnoticed.
ck(c_signal === s_sig, "c_signal disagrees");
ck(c_addr_early === s_addr, "c_addr_early disagrees");
ck(c_toggle === s_tog, "c_toggle disagrees");
ck(c_unsolicited === s_unsol, "c_unsolicited disagrees");
ck(c_hub === s_hub, "c_hub disagrees");
ck(c_power === s_pwr, "c_power disagrees");
// ---- PROPERTY 2: NO FALSE ELIMINATION ----
//
// The dangerous direction. A class may only go from possible to
// impossible if its OWN condition held this cycle.
if (p_sig && !c_signal && !q_sig) n_false_elim = n_false_elim + 1;
if (p_addr && !c_addr_early && !q_addr) n_false_elim = n_false_elim + 1;
if (p_tog && !c_toggle && !q_tog) n_false_elim = n_false_elim + 1;
if (p_unsol && !c_unsolicited && !q_unsol) n_false_elim = n_false_elim + 1;
if (p_hub && !c_hub && !q_hub) n_false_elim = n_false_elim + 1;
if (p_pwr && !c_power && !q_pwr) n_false_elim = n_false_elim + 1;
ck(n_false_elim == 0,
"a fault class was eliminated by evidence that does not contradict it");
// ---- PROPERTY 3: NO FALSE SURVIVAL ----
if (q_sig && c_signal) n_false_surv = n_false_surv + 1;
if (q_addr && c_addr_early) n_false_surv = n_false_surv + 1;
if (q_tog && c_toggle) n_false_surv = n_false_surv + 1;
if (q_unsol && c_unsolicited) n_false_surv = n_false_surv + 1;
if (q_hub && c_hub) n_false_surv = n_false_surv + 1;
if (q_pwr && c_power) n_false_surv = n_false_surv + 1;
ck(n_false_surv == 0,
"a fault class survived evidence that contradicts it");
// ---- PROPERTY 4: elimination is MONOTONIC ----
//
// Evidence does not expire. A class that was ruled out stays ruled
// out, and no later event can revive it.
ck(!(!p_sig && c_signal), "c_signal came back from the dead");
ck(!(!p_addr && c_addr_early), "c_addr_early came back from the dead");
ck(!(!p_tog && c_toggle), "c_toggle came back from the dead");
ck(!(!p_unsol && c_unsolicited), "c_unsolicited came back from the dead");
ck(!(!p_hub && c_hub), "c_hub came back from the dead");
ck(!(!p_pwr && c_power), "c_power came back from the dead");
// ---- PROPERTY 5: the shortlist size and the verdict ----
e_alive = {2'd0, s_sig} + {2'd0, s_addr} + {2'd0, s_tog}
+ {2'd0, s_unsol} + {2'd0, s_hub} + {2'd0, s_pwr};
ck(n_alive === e_alive, "the shortlist size disagrees");
ck(conclusive === (e_alive == 3'd1), "conclusive disagrees");
// ---- PROPERTY 6: zero survivors is NOT conclusive ----
//
// A trace that contradicts every hypothesis is informative -- the
// fault is outside the model -- but it is not a diagnosis, and
// reporting it as one would be the worst possible outcome.
ck(!(conclusive && (e_alive == 3'd0)),
"an empty shortlist was reported as conclusive");
// ---- PROPERTY 7: the counters agree ----
ck(n_events === x_ev, "event count disagrees");
ck(n_crc_bad === x_crc, "CRC-error count disagrees");
ck(n_dev_tx === x_dev, "device-transmission count disagrees");
ck(n_eliminations === x_elim, "elimination count disagrees");
end
endtask
task reset_dut;
begin
rst_n = 1'b0;
ev_valid = 0;
@(posedge clk); @(posedge clk);
rst_n = 1'b1;
// everything starts possible
s_sig = 1'b1; s_addr = 1'b1; s_tog = 1'b1;
s_unsol = 1'b1; s_hub = 1'b1; s_pwr = 1'b1;
s_last_tog = 1'b0; s_have_tog = 1'b0;
x_ev = 0; x_crc = 0; x_dev = 0; x_elim = 0;
@(posedge clk); #1;
end
endtask
integer ki, fi, ci, ai, ri2, tgi, k;
initial begin
for (ri = 0; ri < 256; ri = ri + 1) reach[ri] = 1'b0;
seed = 32'd27009;
// =============================================================
// PHASE 1 (DIRECTED, EXHAUSTIVE) -- every event attribute
// combination, against a freshly reset shortlist.
// 8 kinds x from_device x crc x addr0 x after_reset x toggle = 256.
// =============================================================
for (ki = 0; ki < 8; ki = ki + 1)
for (fi = 0; fi < 2; fi = fi + 1)
for (ci = 0; ci < 2; ci = ci + 1)
for (ai = 0; ai < 2; ai = ai + 1)
for (ri2 = 0; ri2 < 2; ri2 = ri2 + 1)
for (tgi = 0; tgi < 2; tgi = tgi + 1) begin
reset_dut;
ck(n_alive === 3'd6, "the shortlist did not start with all six classes");
ck(conclusive === 1'b0, "a fresh shortlist was reported as conclusive");
ev(1'b1, ki[2:0], fi[0], ci[0], ai[0], ri2[0], tgi[0]);
ev(1'b0, E_SOF, 1'b0, 1'b0, 1'b1, 1'b1, 1'b0);
ri = (ki << 5) | (fi << 4) | (ci << 3) | (ai << 2) | (ri2 << 1) | tgi;
reach[ri] = 1'b1;
end
// =============================================================
// PHASE 2 (DIRECTED) -- THE POINT OF THE CHAPTER.
//
// A real debug session, narrowed one piece of evidence at a time,
// with the shortlist size checked after each step. The fault is a
// device that applies SET_ADDRESS too early, and the trace looks
// at first exactly like signal integrity.
// =============================================================
reset_dut;
ck(n_alive === 3'd6, "the session did not start with six classes");
// (1) The trace is full of CRC errors. This SUGGESTS signal integrity
// and EXCLUDES nothing -- the shortlist must not move at all.
//
// The toggle is held CONSTANT here on purpose. Alternating it
// would eliminate the toggle class as a side effect, and the
// first version of this phase did exactly that and then asserted
// that nothing had been eliminated. The assertion was right and
// the stimulus was wrong.
for (k = 0; k < 8; k = k + 1)
ev(1'b1, E_DATA, 1'b1, 1'b1, 1'b1, 1'b1, 1'b0);
ck(n_alive === 3'd6,
"CRC errors narrowed the shortlist, but they contradict nothing");
// (2) One clean packet from the device, same toggle. Signal integrity
// cannot be the whole story: a marginal PHY does not selectively
// corrupt.
ev(1'b1, E_DATA, 1'b1, 1'b0, 1'b1, 1'b1, 1'b0);
ck(c_signal === 1'b0, "a clean device packet did not rule out signal integrity");
ck(n_alive === 3'd5, "the shortlist did not shrink to five");
// (3) Now a DATA packet with the OTHER toggle: the two alternate, so a
// desynchronised toggle is ruled out.
ev(1'b1, E_DATA, 1'b0, 1'b1, 1'b1, 1'b1, 1'b1);
ck(c_toggle === 1'b0, "alternating toggles did not rule out desynchronisation");
ck(n_alive === 3'd4, "the shortlist did not shrink to four");
// (4) A timeout: the device went silent, so it is not talking too
// much. Unsolicited transmission is out.
ev(1'b1, E_TIMEOUT, 1'b0, 1'b0, 1'b1, 1'b1, 1'b0);
ck(c_unsolicited === 1'b0, "a timeout did not rule out unsolicited traffic");
// (5) A STALL: a considered response, so packets arrive intact both
// ways. A hub fault is out.
ev(1'b1, E_STALL, 1'b1, 1'b0, 1'b1, 1'b1, 1'b0);
ck(c_hub === 1'b0, "a STALL did not rule out a hub fault");
// (6) Activity outside the post-reset window: the device is not
// browning out. Power is out.
ev(1'b1, E_TOKEN, 1'b0, 1'b0, 1'b1, 1'b0, 1'b0);
ck(c_power === 1'b0, "activity outside the reset window did not rule out power");
// ---- and now exactly one class remains ----
ck(n_alive === 3'd1, "the session did not narrow to a single class");
ck(c_addr_early === 1'b1, "the surviving class is not the one the evidence leaves");
ck(conclusive === 1'b1, "a single surviving class was not reported as conclusive");
// (7) The confirming evidence: an ACK at a non-zero address would
// rule the last class out too -- and then NOTHING survives,
// which must NOT be reported as a diagnosis.
ev(1'b1, E_ACK, 1'b0, 1'b0, 1'b0, 1'b1, 1'b0);
ck(c_addr_early === 1'b0, "an ACK at a non-zero address did not rule out the last class");
ck(n_alive === 3'd0, "the shortlist did not empty");
ck(conclusive === 1'b0, "an empty shortlist was reported as conclusive");
// =============================================================
// PHASE 3 (DIRECTED, EXHAUSTIVE) -- each class eliminated last.
//
// Six orderings, each ending with a different survivor, so the
// "exactly one remains" state is reached for every class rather
// than for one convenient one.
// =============================================================
for (k = 0; k < 6; k = k + 1) begin
reset_dut;
// eliminate all but class k
if (k != 0) ev(1'b1, E_DATA, 1'b1, 1'b0, 1'b1, 1'b1, 1'b0);
if (k != 1) ev(1'b1, E_ACK, 1'b0, 1'b0, 1'b0, 1'b1, 1'b0);
if (k != 2) begin
ev(1'b1, E_DATA, 1'b0, 1'b1, 1'b1, 1'b1, 1'b0);
ev(1'b1, E_DATA, 1'b0, 1'b1, 1'b1, 1'b1, 1'b1);
end
if (k != 3) ev(1'b1, E_TIMEOUT, 1'b0, 1'b0, 1'b1, 1'b1, 1'b0);
if (k != 4) ev(1'b1, E_STALL, 1'b0, 1'b0, 1'b1, 1'b1, 1'b0);
if (k != 5) ev(1'b1, E_TOKEN, 1'b0, 1'b0, 1'b1, 1'b0, 1'b0);
ck(n_alive === 3'd1, "the ordering did not leave exactly one class");
ck(conclusive === 1'b1, "a single survivor was not conclusive");
end
// =============================================================
// PHASE 3b (DIRECTED, EXHAUSTIVE) -- drive the shortlist to EMPTY
// by six different routes.
//
// An empty shortlist means the trace contradicts every hypothesis in
// the model -- which is informative (the fault is outside the model)
// and is NOT a diagnosis. Reporting it as conclusive would be the
// worst possible outcome, so it is checked on every route rather than
// on one convenient one. Checked once, it killed its mutation 3 times.
// =============================================================
for (k = 0; k < 6; k = k + 1) begin
reset_dut;
// the six eliminations, rotated so a different one lands last
for (ki = 0; ki < 6; ki = ki + 1) begin
ri2 = (ki + k) % 6;
if (ri2 == 0) ev(1'b1, E_DATA, 1'b1, 1'b0, 1'b1, 1'b1, 1'b0);
if (ri2 == 1) ev(1'b1, E_ACK, 1'b0, 1'b0, 1'b0, 1'b1, 1'b0);
if (ri2 == 2) begin
ev(1'b1, E_DATA, 1'b0, 1'b1, 1'b1, 1'b1, 1'b0);
ev(1'b1, E_DATA, 1'b0, 1'b1, 1'b1, 1'b1, 1'b1);
end
if (ri2 == 3) ev(1'b1, E_TIMEOUT, 1'b0, 1'b0, 1'b1, 1'b1, 1'b0);
if (ri2 == 4) ev(1'b1, E_STALL, 1'b0, 1'b0, 1'b1, 1'b1, 1'b0);
if (ri2 == 5) ev(1'b1, E_TOKEN, 1'b0, 1'b0, 1'b1, 1'b0, 1'b0);
// after every step: a shortlist of one is conclusive, none is not
ck(conclusive === (n_alive === 3'd1),
"conclusive does not mean exactly one survivor");
ck(!(conclusive && (n_alive === 3'd0)),
"an empty shortlist was reported as conclusive");
end
ck(n_alive === 3'd0, "the six eliminations did not empty the shortlist");
ck(conclusive === 1'b0, "an empty shortlist was reported as conclusive");
// and further events must not revive anything
ev(1'b1, E_SOF, 1'b0, 1'b0, 1'b1, 1'b1, 1'b0);
ck(n_alive === 3'd0, "a later event revived an eliminated class");
ck(conclusive === 1'b0, "an empty shortlist became conclusive");
end
// =============================================================
// PHASE 4 (RANDOM) -- arbitrary traces.
// =============================================================
`ifndef DIRECTED_ONLY
for (k = 0; k < 24000; k = k + 1) begin
if ((k % 16) == 0) reset_dut;
ev((urand(0) % 4) != 0, urand(0) % 8, (urand(0) % 2) == 0,
(urand(0) % 3) == 0, (urand(0) % 2) == 0, (urand(0) % 4) != 0,
(urand(0) % 2) == 0);
end
`endif
n_reach = 0;
for (ri = 0; ri < 256; ri = ri + 1) if (reach[ri]) n_reach = n_reach + 1;
$display("steps=%0d checks=%0d reach=%0d/256 errors=%0d",
steps, checks, n_reach, errors);
$display("[trace] events=%0d crc_bad=%0d dev_tx=%0d eliminations=%0d",
g_ev, g_crc, g_dev, g_elim);
$display("[the whole point] false eliminations = %0d, false survivals = %0d",
n_false_elim, n_false_surv);
if (n_reach != 256) begin
$display("FAIL: exhaustive sweep incomplete"); errors = errors + 1;
end
if (errors == 0) $display("PASS: 0 errors in %0d checks", checks);
else $display("FAIL: %0d errors in %0d checks", errors, checks);
$finish;
end
endmoduleSystemVerilog testbench
// =====================================================================
// Testbench for usb_trace_triage.
//
// The property that matters is NOT "does it find the right fault". It is
// the two-sided one that a triage tool lives or dies by:
//
// NO FALSE ELIMINATION -- a class is never ruled out by evidence that
// does not actually contradict it.
// NO FALSE SURVIVAL -- a class IS ruled out the moment something
// contradicts it.
//
// The first is the dangerous direction. A tool that eliminates the real
// fault sends the whole team to look somewhere else, and it does so with
// the authority of a shortlist.
//
// So the shadow recomputes all six elimination conditions from the event
// attributes, independently, and compares the full set every cycle --
// not just the class that changed.
// =====================================================================
`timescale 1ns/1ps
module tb_td_sv;
localparam [2:0] E_SOF = 3'd0, E_TOKEN = 3'd1, E_DATA = 3'd2, E_ACK = 3'd3,
E_NAK = 3'd4, E_STALL = 3'd5, E_TIMEOUT = 3'd6,
E_RESET = 3'd7;
logic clk = 1'b0, rst_n = 1'b0;
logic ev_valid = 1'b0;
logic [2:0] ev_kind = E_SOF;
logic ev_from_device = 1'b0;
logic ev_crc_bad = 1'b0;
logic [6:0] ev_addr = 7'd0;
logic ev_addr_is_zero = 1'b1;
logic ev_after_reset = 1'b1;
logic ev_toggle = 1'b0;
logic c_signal, c_addr_early, c_toggle, c_unsolicited, c_hub, c_power;
logic [2:0] n_alive;
logic conclusive;
logic [31:0] n_events, n_crc_bad, n_dev_tx, n_eliminations;
usb_trace_triage dut (
.clk(clk), .rst_n(rst_n),
.ev_valid(ev_valid), .ev_kind(ev_kind),
.ev_from_device(ev_from_device), .ev_crc_bad(ev_crc_bad),
.ev_addr(ev_addr), .ev_addr_is_zero(ev_addr_is_zero),
.ev_after_reset(ev_after_reset), .ev_toggle(ev_toggle),
.c_signal(c_signal), .c_addr_early(c_addr_early), .c_toggle(c_toggle),
.c_unsolicited(c_unsolicited), .c_hub(c_hub), .c_power(c_power),
.n_alive(n_alive), .conclusive(conclusive),
.n_events(n_events), .n_crc_bad(n_crc_bad),
.n_dev_tx(n_dev_tx), .n_eliminations(n_eliminations)
);
always #5 clk = ~clk;
integer errors = 0, checks = 0, steps = 0;
integer seed;
function automatic logic [31:0] urand(bit dummy);
return $random(seed) & 32'h3FFF_FFFF;
endfunction
// ---- the shadow: six independent booleans ----
logic s_sig, s_addr, s_tog, s_unsol, s_hub, s_pwr;
logic s_last_tog, s_have_tog;
logic [31:0] x_ev, x_crc, x_dev, x_elim;
// ---- the two headline counters ----
// Run-wide totals: the DUT's counters are cleared by every reset, and
// the random phase resets every 16 events.
integer g_ev = 0, g_crc = 0, g_dev = 0, g_elim = 0;
integer n_false_elim = 0; // a class cleared without contradiction
integer n_false_surv = 0; // a class survived a contradiction
// ---- exhaustive reach over (kind, from_dev, crc, addr0, after_reset) --
// The event TOGGLE is a sixth dimension, not a constant. Without it the
// first DATA packet of every scenario carries toggle 0, which equals the
// reset value of the history -- so a rule that wrongly ignores "is there
// any history" never fires, and mutation I6 scored 0 directed.
logic reach [0:255];
integer ri, n_reach;
task ck(input logic cond, input logic [255:0] what);
begin
checks = checks + 1;
if (!cond) begin
errors = errors + 1;
if (errors <= 20)
$display(" ERROR @%0t step=%0d: %0s", $time, steps, what);
end
end
endtask
// ---------------------------------------------------------------
// One trace event.
// ---------------------------------------------------------------
task ev(input logic v, input logic [2:0] k, input logic fd, input logic cb,
input logic az, input logic ar, input logic tg);
logic p_sig, p_addr, p_tog, p_unsol, p_hub, p_pwr;
logic q_sig, q_addr, q_tog, q_unsol, q_hub, q_pwr;
logic [2:0] e_alive;
begin
ev_valid = v; ev_kind = k; ev_from_device = fd;
ev_crc_bad = cb; ev_addr_is_zero = az;
ev_after_reset = ar; ev_toggle = tg;
ev_addr = az ? 7'd0 : 7'd42;
// remember the state BEFORE this event, for the monotonicity check
p_sig = s_sig; p_addr = s_addr; p_tog = s_tog;
p_unsol = s_unsol; p_hub = s_hub; p_pwr = s_pwr;
// ---- the six elimination conditions, recomputed independently ----
q_sig = v && fd && !cb;
q_addr = v && (k == E_ACK) && !az;
q_tog = v && (k == E_DATA) && s_have_tog && (tg != s_last_tog);
q_unsol = v && (k == E_TIMEOUT);
q_hub = v && (k == E_STALL);
q_pwr = v && !ar && (k != E_RESET);
if (v) begin
x_ev = x_ev + 1; g_ev = g_ev + 1;
if (cb) begin x_crc = x_crc + 1; g_crc = g_crc + 1; end
if (fd) begin x_dev = x_dev + 1; g_dev = g_dev + 1; end
x_elim = x_elim
+ ((q_sig && s_sig) ? 1 : 0)
+ ((q_addr && s_addr) ? 1 : 0)
+ ((q_tog && s_tog) ? 1 : 0)
+ ((q_unsol && s_unsol) ? 1 : 0)
+ ((q_hub && s_hub) ? 1 : 0)
+ ((q_pwr && s_pwr) ? 1 : 0);
g_elim = g_elim
+ ((q_sig && s_sig) ? 1 : 0)
+ ((q_addr && s_addr) ? 1 : 0)
+ ((q_tog && s_tog) ? 1 : 0)
+ ((q_unsol && s_unsol) ? 1 : 0)
+ ((q_hub && s_hub) ? 1 : 0)
+ ((q_pwr && s_pwr) ? 1 : 0);
if (q_sig) s_sig = 1'b0;
if (q_addr) s_addr = 1'b0;
if (q_tog) s_tog = 1'b0;
if (q_unsol) s_unsol = 1'b0;
if (q_hub) s_hub = 1'b0;
if (q_pwr) s_pwr = 1'b0;
if (k == E_DATA) begin
s_last_tog = tg;
s_have_tog = 1'b1;
end
end
@(posedge clk);
#1;
steps = steps + 1;
ev_valid = 1'b0;
// ---- PROPERTY 1: every class matches the shadow ----
//
// All six compared every cycle, not just the one that changed. A
// class cleared as a side effect of another's evidence would
// otherwise go unnoticed.
ck(c_signal === s_sig, "c_signal disagrees");
ck(c_addr_early === s_addr, "c_addr_early disagrees");
ck(c_toggle === s_tog, "c_toggle disagrees");
ck(c_unsolicited === s_unsol, "c_unsolicited disagrees");
ck(c_hub === s_hub, "c_hub disagrees");
ck(c_power === s_pwr, "c_power disagrees");
// ---- PROPERTY 2: NO FALSE ELIMINATION ----
//
// The dangerous direction. A class may only go from possible to
// impossible if its OWN condition held this cycle.
if (p_sig && !c_signal && !q_sig) n_false_elim = n_false_elim + 1;
if (p_addr && !c_addr_early && !q_addr) n_false_elim = n_false_elim + 1;
if (p_tog && !c_toggle && !q_tog) n_false_elim = n_false_elim + 1;
if (p_unsol && !c_unsolicited && !q_unsol) n_false_elim = n_false_elim + 1;
if (p_hub && !c_hub && !q_hub) n_false_elim = n_false_elim + 1;
if (p_pwr && !c_power && !q_pwr) n_false_elim = n_false_elim + 1;
ck(n_false_elim == 0,
"a fault class was eliminated by evidence that does not contradict it");
// ---- PROPERTY 3: NO FALSE SURVIVAL ----
if (q_sig && c_signal) n_false_surv = n_false_surv + 1;
if (q_addr && c_addr_early) n_false_surv = n_false_surv + 1;
if (q_tog && c_toggle) n_false_surv = n_false_surv + 1;
if (q_unsol && c_unsolicited) n_false_surv = n_false_surv + 1;
if (q_hub && c_hub) n_false_surv = n_false_surv + 1;
if (q_pwr && c_power) n_false_surv = n_false_surv + 1;
ck(n_false_surv == 0,
"a fault class survived evidence that contradicts it");
// ---- PROPERTY 4: elimination is MONOTONIC ----
//
// Evidence does not expire. A class that was ruled out stays ruled
// out, and no later event can revive it.
ck(!(!p_sig && c_signal), "c_signal came back from the dead");
ck(!(!p_addr && c_addr_early), "c_addr_early came back from the dead");
ck(!(!p_tog && c_toggle), "c_toggle came back from the dead");
ck(!(!p_unsol && c_unsolicited), "c_unsolicited came back from the dead");
ck(!(!p_hub && c_hub), "c_hub came back from the dead");
ck(!(!p_pwr && c_power), "c_power came back from the dead");
// ---- PROPERTY 5: the shortlist size and the verdict ----
e_alive = {2'd0, s_sig} + {2'd0, s_addr} + {2'd0, s_tog}
+ {2'd0, s_unsol} + {2'd0, s_hub} + {2'd0, s_pwr};
ck(n_alive === e_alive, "the shortlist size disagrees");
ck(conclusive === (e_alive == 3'd1), "conclusive disagrees");
// ---- PROPERTY 6: zero survivors is NOT conclusive ----
//
// A trace that contradicts every hypothesis is informative -- the
// fault is outside the model -- but it is not a diagnosis, and
// reporting it as one would be the worst possible outcome.
ck(!(conclusive && (e_alive == 3'd0)),
"an empty shortlist was reported as conclusive");
// ---- PROPERTY 7: the counters agree ----
ck(n_events === x_ev, "event count disagrees");
ck(n_crc_bad === x_crc, "CRC-error count disagrees");
ck(n_dev_tx === x_dev, "device-transmission count disagrees");
ck(n_eliminations === x_elim, "elimination count disagrees");
end
endtask
task reset_dut;
begin
rst_n = 1'b0;
ev_valid = 0;
@(posedge clk); @(posedge clk);
rst_n = 1'b1;
// everything starts possible
s_sig = 1'b1; s_addr = 1'b1; s_tog = 1'b1;
s_unsol = 1'b1; s_hub = 1'b1; s_pwr = 1'b1;
s_last_tog = 1'b0; s_have_tog = 1'b0;
x_ev = 0; x_crc = 0; x_dev = 0; x_elim = 0;
@(posedge clk); #1;
end
endtask
integer ki, fi, ci, ai, ri2, tgi, k;
initial begin
for (ri = 0; ri < 256; ri = ri + 1) reach[ri] = 1'b0;
seed = 32'd27009;
// =============================================================
// PHASE 1 (DIRECTED, EXHAUSTIVE) -- every event attribute
// combination, against a freshly reset shortlist.
// 8 kinds x from_device x crc x addr0 x after_reset x toggle = 256.
// =============================================================
for (ki = 0; ki < 8; ki = ki + 1)
for (fi = 0; fi < 2; fi = fi + 1)
for (ci = 0; ci < 2; ci = ci + 1)
for (ai = 0; ai < 2; ai = ai + 1)
for (ri2 = 0; ri2 < 2; ri2 = ri2 + 1)
for (tgi = 0; tgi < 2; tgi = tgi + 1) begin
reset_dut;
ck(n_alive === 3'd6, "the shortlist did not start with all six classes");
ck(conclusive === 1'b0, "a fresh shortlist was reported as conclusive");
ev(1'b1, ki[2:0], fi[0], ci[0], ai[0], ri2[0], tgi[0]);
ev(1'b0, E_SOF, 1'b0, 1'b0, 1'b1, 1'b1, 1'b0);
ri = (ki << 5) | (fi << 4) | (ci << 3) | (ai << 2) | (ri2 << 1) | tgi;
reach[ri] = 1'b1;
end
// =============================================================
// PHASE 2 (DIRECTED) -- THE POINT OF THE CHAPTER.
//
// A real debug session, narrowed one piece of evidence at a time,
// with the shortlist size checked after each step. The fault is a
// device that applies SET_ADDRESS too early, and the trace looks
// at first exactly like signal integrity.
// =============================================================
reset_dut;
ck(n_alive === 3'd6, "the session did not start with six classes");
// (1) The trace is full of CRC errors. This SUGGESTS signal integrity
// and EXCLUDES nothing -- the shortlist must not move at all.
//
// The toggle is held CONSTANT here on purpose. Alternating it
// would eliminate the toggle class as a side effect, and the
// first version of this phase did exactly that and then asserted
// that nothing had been eliminated. The assertion was right and
// the stimulus was wrong.
for (k = 0; k < 8; k = k + 1)
ev(1'b1, E_DATA, 1'b1, 1'b1, 1'b1, 1'b1, 1'b0);
ck(n_alive === 3'd6,
"CRC errors narrowed the shortlist, but they contradict nothing");
// (2) One clean packet from the device, same toggle. Signal integrity
// cannot be the whole story: a marginal PHY does not selectively
// corrupt.
ev(1'b1, E_DATA, 1'b1, 1'b0, 1'b1, 1'b1, 1'b0);
ck(c_signal === 1'b0, "a clean device packet did not rule out signal integrity");
ck(n_alive === 3'd5, "the shortlist did not shrink to five");
// (3) Now a DATA packet with the OTHER toggle: the two alternate, so a
// desynchronised toggle is ruled out.
ev(1'b1, E_DATA, 1'b0, 1'b1, 1'b1, 1'b1, 1'b1);
ck(c_toggle === 1'b0, "alternating toggles did not rule out desynchronisation");
ck(n_alive === 3'd4, "the shortlist did not shrink to four");
// (4) A timeout: the device went silent, so it is not talking too
// much. Unsolicited transmission is out.
ev(1'b1, E_TIMEOUT, 1'b0, 1'b0, 1'b1, 1'b1, 1'b0);
ck(c_unsolicited === 1'b0, "a timeout did not rule out unsolicited traffic");
// (5) A STALL: a considered response, so packets arrive intact both
// ways. A hub fault is out.
ev(1'b1, E_STALL, 1'b1, 1'b0, 1'b1, 1'b1, 1'b0);
ck(c_hub === 1'b0, "a STALL did not rule out a hub fault");
// (6) Activity outside the post-reset window: the device is not
// browning out. Power is out.
ev(1'b1, E_TOKEN, 1'b0, 1'b0, 1'b1, 1'b0, 1'b0);
ck(c_power === 1'b0, "activity outside the reset window did not rule out power");
// ---- and now exactly one class remains ----
ck(n_alive === 3'd1, "the session did not narrow to a single class");
ck(c_addr_early === 1'b1, "the surviving class is not the one the evidence leaves");
ck(conclusive === 1'b1, "a single surviving class was not reported as conclusive");
// (7) The confirming evidence: an ACK at a non-zero address would
// rule the last class out too -- and then NOTHING survives,
// which must NOT be reported as a diagnosis.
ev(1'b1, E_ACK, 1'b0, 1'b0, 1'b0, 1'b1, 1'b0);
ck(c_addr_early === 1'b0, "an ACK at a non-zero address did not rule out the last class");
ck(n_alive === 3'd0, "the shortlist did not empty");
ck(conclusive === 1'b0, "an empty shortlist was reported as conclusive");
// =============================================================
// PHASE 3 (DIRECTED, EXHAUSTIVE) -- each class eliminated last.
//
// Six orderings, each ending with a different survivor, so the
// "exactly one remains" state is reached for every class rather
// than for one convenient one.
// =============================================================
for (k = 0; k < 6; k = k + 1) begin
reset_dut;
// eliminate all but class k
if (k != 0) ev(1'b1, E_DATA, 1'b1, 1'b0, 1'b1, 1'b1, 1'b0);
if (k != 1) ev(1'b1, E_ACK, 1'b0, 1'b0, 1'b0, 1'b1, 1'b0);
if (k != 2) begin
ev(1'b1, E_DATA, 1'b0, 1'b1, 1'b1, 1'b1, 1'b0);
ev(1'b1, E_DATA, 1'b0, 1'b1, 1'b1, 1'b1, 1'b1);
end
if (k != 3) ev(1'b1, E_TIMEOUT, 1'b0, 1'b0, 1'b1, 1'b1, 1'b0);
if (k != 4) ev(1'b1, E_STALL, 1'b0, 1'b0, 1'b1, 1'b1, 1'b0);
if (k != 5) ev(1'b1, E_TOKEN, 1'b0, 1'b0, 1'b1, 1'b0, 1'b0);
ck(n_alive === 3'd1, "the ordering did not leave exactly one class");
ck(conclusive === 1'b1, "a single survivor was not conclusive");
end
// =============================================================
// PHASE 3b (DIRECTED, EXHAUSTIVE) -- drive the shortlist to EMPTY
// by six different routes.
//
// An empty shortlist means the trace contradicts every hypothesis in
// the model -- which is informative (the fault is outside the model)
// and is NOT a diagnosis. Reporting it as conclusive would be the
// worst possible outcome, so it is checked on every route rather than
// on one convenient one. Checked once, it killed its mutation 3 times.
// =============================================================
for (k = 0; k < 6; k = k + 1) begin
reset_dut;
// the six eliminations, rotated so a different one lands last
for (ki = 0; ki < 6; ki = ki + 1) begin
ri2 = (ki + k) % 6;
if (ri2 == 0) ev(1'b1, E_DATA, 1'b1, 1'b0, 1'b1, 1'b1, 1'b0);
if (ri2 == 1) ev(1'b1, E_ACK, 1'b0, 1'b0, 1'b0, 1'b1, 1'b0);
if (ri2 == 2) begin
ev(1'b1, E_DATA, 1'b0, 1'b1, 1'b1, 1'b1, 1'b0);
ev(1'b1, E_DATA, 1'b0, 1'b1, 1'b1, 1'b1, 1'b1);
end
if (ri2 == 3) ev(1'b1, E_TIMEOUT, 1'b0, 1'b0, 1'b1, 1'b1, 1'b0);
if (ri2 == 4) ev(1'b1, E_STALL, 1'b0, 1'b0, 1'b1, 1'b1, 1'b0);
if (ri2 == 5) ev(1'b1, E_TOKEN, 1'b0, 1'b0, 1'b1, 1'b0, 1'b0);
// after every step: a shortlist of one is conclusive, none is not
ck(conclusive === (n_alive === 3'd1),
"conclusive does not mean exactly one survivor");
ck(!(conclusive && (n_alive === 3'd0)),
"an empty shortlist was reported as conclusive");
end
ck(n_alive === 3'd0, "the six eliminations did not empty the shortlist");
ck(conclusive === 1'b0, "an empty shortlist was reported as conclusive");
// and further events must not revive anything
ev(1'b1, E_SOF, 1'b0, 1'b0, 1'b1, 1'b1, 1'b0);
ck(n_alive === 3'd0, "a later event revived an eliminated class");
ck(conclusive === 1'b0, "an empty shortlist became conclusive");
end
// =============================================================
// PHASE 4 (RANDOM) -- arbitrary traces.
// =============================================================
`ifndef DIRECTED_ONLY
for (k = 0; k < 24000; k = k + 1) begin
if ((k % 16) == 0) reset_dut;
ev((urand(0) % 4) != 0, urand(0) % 8, (urand(0) % 2) == 0,
(urand(0) % 3) == 0, (urand(0) % 2) == 0, (urand(0) % 4) != 0,
(urand(0) % 2) == 0);
end
`endif
n_reach = 0;
for (ri = 0; ri < 256; ri = ri + 1) if (reach[ri]) n_reach = n_reach + 1;
$display("steps=%0d checks=%0d reach=%0d/256 errors=%0d",
steps, checks, n_reach, errors);
$display("[trace] events=%0d crc_bad=%0d dev_tx=%0d eliminations=%0d",
g_ev, g_crc, g_dev, g_elim);
$display("[the whole point] false eliminations = %0d, false survivals = %0d",
n_false_elim, n_false_surv);
if (n_reach != 256) begin
$display("FAIL: exhaustive sweep incomplete"); errors = errors + 1;
end
if (errors == 0) $display("PASS: 0 errors in %0d checks", checks);
else $display("FAIL: %0d errors in %0d checks", errors, checks);
$finish;
end
endmoduleVHDL-2008 testbench
-- =====================================================================
-- Testbench for usb_trace_triage (VHDL-2008).
--
-- The property that matters is not "does it find the right fault". It is
-- the two-sided one a triage tool lives or dies by:
--
-- NO FALSE ELIMINATION -- a class is never ruled out by evidence that
-- does not actually contradict it.
-- NO FALSE SURVIVAL -- a class IS ruled out the moment something
-- contradicts it.
--
-- The first is the dangerous direction: a tool that eliminates the real
-- fault sends the whole team elsewhere, with the authority of a shortlist.
-- =====================================================================
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
use std.textio.all;
use work.td_pkg.all;
entity tb_td_vhdl is
generic (DIRECTED_ONLY : boolean := false);
end entity;
architecture sim of tb_td_vhdl is
signal clk : std_logic := '0';
signal rst_n : std_logic := '0';
signal ev_valid : std_logic := '0';
signal ev_kind : std_logic_vector(2 downto 0) := "000";
signal ev_from_device : std_logic := '0';
signal ev_crc_bad : std_logic := '0';
signal ev_addr : std_logic_vector(6 downto 0) := (others => '0');
signal ev_addr_is_zero : std_logic := '1';
signal ev_after_reset : std_logic := '1';
signal ev_toggle : std_logic := '0';
signal c_signal, c_addr_early, c_toggle : std_logic;
signal c_unsolicited, c_hub, c_power : std_logic;
signal n_alive : std_logic_vector(2 downto 0);
signal conclusive : std_logic;
signal n_events, n_crc_bad, n_dev_tx, n_eliminations
: std_logic_vector(31 downto 0);
signal done : boolean := false;
begin
dut : entity work.usb_trace_triage
port map (
clk => clk, rst_n => rst_n,
ev_valid => ev_valid, ev_kind => ev_kind,
ev_from_device => ev_from_device, ev_crc_bad => ev_crc_bad,
ev_addr => ev_addr, ev_addr_is_zero => ev_addr_is_zero,
ev_after_reset => ev_after_reset, ev_toggle => ev_toggle,
c_signal => c_signal, c_addr_early => c_addr_early,
c_toggle => c_toggle, c_unsolicited => c_unsolicited,
c_hub => c_hub, c_power => c_power,
n_alive => n_alive, conclusive => conclusive,
n_events => n_events, n_crc_bad => n_crc_bad,
n_dev_tx => n_dev_tx, n_eliminations => n_eliminations);
clk <= not clk after 5 ns when not done else '0';
stim : process
variable errors : natural := 0;
variable checks : natural := 0;
variable steps : natural := 0;
variable s_sig, s_addr, s_tog : std_logic := '1';
variable s_unsol, s_hub, s_pwr : std_logic := '1';
variable s_last_tog, s_have_tog : std_logic := '0';
variable x_ev, x_crc, x_dev, x_elim : natural := 0;
-- Run-wide totals: the DUT's counters are cleared by every reset, and
-- the random phase resets every 16 events.
variable g_ev, g_crc, g_dev, g_elim : natural := 0;
variable n_false_elim, n_false_surv : natural := 0;
-- The event TOGGLE is a sixth dimension, not a constant. Without it the
-- first DATA packet of every scenario carries toggle 0, which equals the
-- reset value of the history -- so a rule that wrongly ignores "is there
-- any history" never fires, and mutation I6 scored 0 directed.
variable reach : std_logic_vector(0 to 255) := (others => '0');
variable n_reach : natural := 0;
variable rnd : unsigned(31 downto 0) := x"000AE3F1";
variable ln : line;
procedure ck(cond : boolean; what : string) is
begin
checks := checks + 1;
if not cond then
errors := errors + 1;
if errors <= 20 then
write(ln, string'(" ERROR step=") & integer'image(steps)
& string'(": ") & what);
writeline(output, ln);
end if;
end if;
end procedure;
impure function nxt return natural is
begin
rnd := rnd xor (rnd sll 13);
rnd := rnd xor (rnd srl 17);
rnd := rnd xor (rnd sll 5);
return to_integer(rnd(14 downto 0));
end function;
function sl_of(b : boolean) return std_logic is
begin
if b then return '1'; else return '0'; end if;
end function;
function b2n(b : std_logic) return natural is
begin
if b = '1' then return 1; else return 0; end if;
end function;
procedure ev(v : std_logic; k : ev_t;
fd, cb, az, ar, tg : std_logic) is
variable p_sig, p_addr, p_tog, p_unsol, p_hub, p_pwr : std_logic;
variable q_sig, q_addr, q_tog, q_unsol, q_hub, q_pwr : std_logic;
variable e_alive : natural;
variable kv : std_logic_vector(2 downto 0);
begin
case k is
when EV_SOF => kv := "000";
when EV_TOKEN => kv := "001";
when EV_DATA => kv := "010";
when EV_ACK => kv := "011";
when EV_NAK => kv := "100";
when EV_STALL => kv := "101";
when EV_TIMEOUT => kv := "110";
when EV_RESET => kv := "111";
end case;
ev_valid <= v; ev_kind <= kv; ev_from_device <= fd;
ev_crc_bad <= cb; ev_addr_is_zero <= az;
ev_after_reset <= ar; ev_toggle <= tg;
if az = '1' then ev_addr <= (others => '0');
else ev_addr <= std_logic_vector(to_unsigned(42, 7));
end if;
-- the state BEFORE this event, for the monotonicity check
p_sig := s_sig; p_addr := s_addr; p_tog := s_tog;
p_unsol := s_unsol; p_hub := s_hub; p_pwr := s_pwr;
-- the six elimination conditions, recomputed independently
q_sig := sl_of(v = '1' and fd = '1' and cb = '0');
q_addr := sl_of(v = '1' and k = EV_ACK and az = '0');
q_tog := sl_of(v = '1' and k = EV_DATA and s_have_tog = '1'
and tg /= s_last_tog);
q_unsol := sl_of(v = '1' and k = EV_TIMEOUT);
q_hub := sl_of(v = '1' and k = EV_STALL);
q_pwr := sl_of(v = '1' and ar = '0' and k /= EV_RESET);
if v = '1' then
x_ev := x_ev + 1; g_ev := g_ev + 1;
if cb = '1' then x_crc := x_crc + 1; g_crc := g_crc + 1; end if;
if fd = '1' then x_dev := x_dev + 1; g_dev := g_dev + 1; end if;
x_elim := x_elim + b2n(q_sig and s_sig) + b2n(q_addr and s_addr)
+ b2n(q_tog and s_tog) + b2n(q_unsol and s_unsol)
+ b2n(q_hub and s_hub) + b2n(q_pwr and s_pwr);
g_elim := g_elim + b2n(q_sig and s_sig) + b2n(q_addr and s_addr)
+ b2n(q_tog and s_tog) + b2n(q_unsol and s_unsol)
+ b2n(q_hub and s_hub) + b2n(q_pwr and s_pwr);
if q_sig = '1' then s_sig := '0'; end if;
if q_addr = '1' then s_addr := '0'; end if;
if q_tog = '1' then s_tog := '0'; end if;
if q_unsol = '1' then s_unsol := '0'; end if;
if q_hub = '1' then s_hub := '0'; end if;
if q_pwr = '1' then s_pwr := '0'; end if;
if k = EV_DATA then
s_last_tog := tg;
s_have_tog := '1';
end if;
end if;
wait until rising_edge(clk);
wait for 1 ns;
steps := steps + 1;
ev_valid <= '0';
-- PROPERTY 1: every class matches the shadow. All six compared every
-- cycle, not just the one that changed -- a class cleared as a side
-- effect of another's evidence would otherwise go unnoticed.
ck(c_signal = s_sig, "c_signal disagrees");
ck(c_addr_early = s_addr, "c_addr_early disagrees");
ck(c_toggle = s_tog, "c_toggle disagrees");
ck(c_unsolicited = s_unsol, "c_unsolicited disagrees");
ck(c_hub = s_hub, "c_hub disagrees");
ck(c_power = s_pwr, "c_power disagrees");
-- PROPERTY 2: NO FALSE ELIMINATION -- the dangerous direction
if p_sig = '1' and c_signal = '0' and q_sig = '0' then n_false_elim := n_false_elim + 1; end if;
if p_addr = '1' and c_addr_early = '0' and q_addr = '0' then n_false_elim := n_false_elim + 1; end if;
if p_tog = '1' and c_toggle = '0' and q_tog = '0' then n_false_elim := n_false_elim + 1; end if;
if p_unsol = '1' and c_unsolicited = '0' and q_unsol = '0' then n_false_elim := n_false_elim + 1; end if;
if p_hub = '1' and c_hub = '0' and q_hub = '0' then n_false_elim := n_false_elim + 1; end if;
if p_pwr = '1' and c_power = '0' and q_pwr = '0' then n_false_elim := n_false_elim + 1; end if;
ck(n_false_elim = 0,
"a fault class was eliminated by evidence that does not contradict it");
-- PROPERTY 3: NO FALSE SURVIVAL
if q_sig = '1' and c_signal = '1' then n_false_surv := n_false_surv + 1; end if;
if q_addr = '1' and c_addr_early = '1' then n_false_surv := n_false_surv + 1; end if;
if q_tog = '1' and c_toggle = '1' then n_false_surv := n_false_surv + 1; end if;
if q_unsol = '1' and c_unsolicited = '1' then n_false_surv := n_false_surv + 1; end if;
if q_hub = '1' and c_hub = '1' then n_false_surv := n_false_surv + 1; end if;
if q_pwr = '1' and c_power = '1' then n_false_surv := n_false_surv + 1; end if;
ck(n_false_surv = 0,
"a fault class survived evidence that contradicts it");
-- PROPERTY 4: elimination is MONOTONIC. Evidence does not expire.
ck(not (p_sig = '0' and c_signal = '1'), "c_signal came back from the dead");
ck(not (p_addr = '0' and c_addr_early = '1'), "c_addr_early came back from the dead");
ck(not (p_tog = '0' and c_toggle = '1'), "c_toggle came back from the dead");
ck(not (p_unsol = '0' and c_unsolicited = '1'), "c_unsolicited came back from the dead");
ck(not (p_hub = '0' and c_hub = '1'), "c_hub came back from the dead");
ck(not (p_pwr = '0' and c_power = '1'), "c_power came back from the dead");
-- PROPERTY 5: the shortlist size and the verdict
e_alive := b2n(s_sig) + b2n(s_addr) + b2n(s_tog)
+ b2n(s_unsol) + b2n(s_hub) + b2n(s_pwr);
ck(to_integer(unsigned(n_alive)) = e_alive, "the shortlist size disagrees");
ck((conclusive = '1') = (e_alive = 1), "conclusive disagrees");
-- PROPERTY 6: zero survivors is NOT conclusive. A trace that
-- contradicts every hypothesis is informative -- the fault is outside
-- the model -- but it is not a diagnosis.
ck(not (conclusive = '1' and e_alive = 0),
"an empty shortlist was reported as conclusive");
-- PROPERTY 7: the counters agree
ck(to_integer(unsigned(n_events)) = x_ev, "event count disagrees");
ck(to_integer(unsigned(n_crc_bad)) = x_crc, "CRC-error count disagrees");
ck(to_integer(unsigned(n_dev_tx)) = x_dev, "device-transmission count disagrees");
ck(to_integer(unsigned(n_eliminations)) = x_elim, "elimination count disagrees");
end procedure;
procedure reset_dut is
begin
rst_n <= '0';
ev_valid <= '0';
wait until rising_edge(clk);
wait until rising_edge(clk);
rst_n <= '1';
s_sig := '1'; s_addr := '1'; s_tog := '1';
s_unsol := '1'; s_hub := '1'; s_pwr := '1';
s_last_tog := '0'; s_have_tog := '0';
x_ev := 0; x_crc := 0; x_dev := 0; x_elim := 0;
wait until rising_edge(clk);
wait for 1 ns;
end procedure;
type ev_arr is array (0 to 7) of ev_t;
constant kinds : ev_arr := (EV_SOF, EV_TOKEN, EV_DATA, EV_ACK,
EV_NAK, EV_STALL, EV_TIMEOUT, EV_RESET);
variable ri : natural;
begin
-- PHASE 1 (DIRECTED, EXHAUSTIVE) -- every event attribute combination
-- against a freshly reset shortlist. 8 x 2 x 2 x 2 x 2 x 2 = 256.
for ki in 0 to 7 loop
for fi in 0 to 1 loop
for ci in 0 to 1 loop
for ai in 0 to 1 loop
for r2 in 0 to 1 loop
for tgi in 0 to 1 loop
reset_dut;
ck(to_integer(unsigned(n_alive)) = 6,
"the shortlist did not start with all six classes");
ck(conclusive = '0', "a fresh shortlist was reported as conclusive");
ev('1', kinds(ki), sl_of(fi = 1), sl_of(ci = 1),
sl_of(ai = 1), sl_of(r2 = 1), sl_of(tgi = 1));
ev('0', EV_SOF, '0', '0', '1', '1', '0');
ri := ki*32 + fi*16 + ci*8 + ai*4 + r2*2 + tgi;
reach(ri) := '1';
end loop;
end loop;
end loop;
end loop;
end loop;
end loop;
-- PHASE 2 (DIRECTED) -- THE POINT OF THE CHAPTER.
--
-- A real debug session, narrowed one piece of evidence at a time. The
-- fault is a device that applies SET_ADDRESS too early, and the trace
-- looks at first exactly like signal integrity.
reset_dut;
ck(to_integer(unsigned(n_alive)) = 6, "the session did not start with six classes");
-- (1) The trace is full of CRC errors. This SUGGESTS signal integrity
-- and EXCLUDES nothing -- the shortlist must not move. The toggle
-- is held CONSTANT here on purpose: alternating it would eliminate
-- the toggle class as a side effect.
for k in 0 to 7 loop
ev('1', EV_DATA, '1', '1', '1', '1', '0');
end loop;
ck(to_integer(unsigned(n_alive)) = 6,
"CRC errors narrowed the shortlist, but they contradict nothing");
-- (2) One clean packet from the device, same toggle: signal integrity
-- cannot be the whole story.
ev('1', EV_DATA, '1', '0', '1', '1', '0');
ck(c_signal = '0', "a clean device packet did not rule out signal integrity");
ck(to_integer(unsigned(n_alive)) = 5, "the shortlist did not shrink to five");
-- (3) A DATA packet with the OTHER toggle: the two alternate.
ev('1', EV_DATA, '0', '1', '1', '1', '1');
ck(c_toggle = '0', "alternating toggles did not rule out desynchronisation");
ck(to_integer(unsigned(n_alive)) = 4, "the shortlist did not shrink to four");
-- (4) A timeout: the device went silent, so it is not talking too much.
ev('1', EV_TIMEOUT, '0', '0', '1', '1', '0');
ck(c_unsolicited = '0', "a timeout did not rule out unsolicited traffic");
-- (5) A STALL: a considered response, so packets arrive intact both ways.
ev('1', EV_STALL, '1', '0', '1', '1', '0');
ck(c_hub = '0', "a STALL did not rule out a hub fault");
-- (6) Activity outside the post-reset window: not browning out.
ev('1', EV_TOKEN, '0', '0', '1', '0', '0');
ck(c_power = '0', "activity outside the reset window did not rule out power");
ck(to_integer(unsigned(n_alive)) = 1, "the session did not narrow to a single class");
ck(c_addr_early = '1', "the surviving class is not the one the evidence leaves");
ck(conclusive = '1', "a single surviving class was not reported as conclusive");
-- (7) An ACK at a non-zero address rules the last class out too -- and
-- then NOTHING survives, which must NOT be reported as a diagnosis.
ev('1', EV_ACK, '0', '0', '0', '1', '0');
ck(c_addr_early = '0', "an ACK at a non-zero address did not rule out the last class");
ck(to_integer(unsigned(n_alive)) = 0, "the shortlist did not empty");
ck(conclusive = '0', "an empty shortlist was reported as conclusive");
-- PHASE 3 (DIRECTED, EXHAUSTIVE) -- each class eliminated last, so the
-- "exactly one remains" state is reached for every class rather than
-- for one convenient one.
for k in 0 to 5 loop
reset_dut;
if k /= 0 then ev('1', EV_DATA, '1', '0', '1', '1', '0'); end if;
if k /= 1 then ev('1', EV_ACK, '0', '0', '0', '1', '0'); end if;
if k /= 2 then
ev('1', EV_DATA, '0', '1', '1', '1', '0');
ev('1', EV_DATA, '0', '1', '1', '1', '1');
end if;
if k /= 3 then ev('1', EV_TIMEOUT, '0', '0', '1', '1', '0'); end if;
if k /= 4 then ev('1', EV_STALL, '0', '0', '1', '1', '0'); end if;
if k /= 5 then ev('1', EV_TOKEN, '0', '0', '1', '0', '0'); end if;
ck(to_integer(unsigned(n_alive)) = 1, "the ordering did not leave exactly one class");
ck(conclusive = '1', "a single survivor was not conclusive");
end loop;
-- PHASE 3b (DIRECTED, EXHAUSTIVE) -- drive the shortlist to EMPTY by six
-- different routes. An empty shortlist means the trace contradicts every
-- hypothesis in the model -- informative, and NOT a diagnosis. Reporting
-- it as conclusive would be the worst possible outcome, so it is checked
-- on every route rather than on one convenient one.
for k in 0 to 5 loop
reset_dut;
for ki in 0 to 5 loop
case (ki + k) mod 6 is
when 0 => ev('1', EV_DATA, '1', '0', '1', '1', '0');
when 1 => ev('1', EV_ACK, '0', '0', '0', '1', '0');
when 2 =>
ev('1', EV_DATA, '0', '1', '1', '1', '0');
ev('1', EV_DATA, '0', '1', '1', '1', '1');
when 3 => ev('1', EV_TIMEOUT, '0', '0', '1', '1', '0');
when 4 => ev('1', EV_STALL, '0', '0', '1', '1', '0');
when others => ev('1', EV_TOKEN, '0', '0', '1', '0', '0');
end case;
ck((conclusive = '1') = (to_integer(unsigned(n_alive)) = 1),
"conclusive does not mean exactly one survivor");
ck(not (conclusive = '1' and to_integer(unsigned(n_alive)) = 0),
"an empty shortlist was reported as conclusive");
end loop;
ck(to_integer(unsigned(n_alive)) = 0,
"the six eliminations did not empty the shortlist");
ck(conclusive = '0', "an empty shortlist was reported as conclusive");
ev('1', EV_SOF, '0', '0', '1', '1', '0');
ck(to_integer(unsigned(n_alive)) = 0, "a later event revived an eliminated class");
ck(conclusive = '0', "an empty shortlist became conclusive");
end loop;
-- PHASE 4 (RANDOM) -- arbitrary traces
if not DIRECTED_ONLY then
for k in 0 to 23999 loop
if (k mod 16) = 0 then reset_dut; end if;
ev(sl_of((nxt mod 4) /= 0), kinds(nxt mod 8),
sl_of((nxt mod 2) = 0), sl_of((nxt mod 3) = 0),
sl_of((nxt mod 2) = 0), sl_of((nxt mod 4) /= 0),
sl_of((nxt mod 2) = 0));
end loop;
end if;
n_reach := 0;
for i in 0 to 255 loop
if reach(i) = '1' then n_reach := n_reach + 1; end if;
end loop;
write(ln, string'("steps=") & integer'image(steps)
& string'(" checks=") & integer'image(checks)
& string'(" reach=") & integer'image(n_reach) & string'("/256")
& string'(" errors=") & integer'image(errors));
writeline(output, ln);
write(ln, string'("[trace] events=") & integer'image(g_ev)
& string'(" crc_bad=") & integer'image(g_crc)
& string'(" dev_tx=") & integer'image(g_dev)
& string'(" eliminations=") & integer'image(g_elim));
writeline(output, ln);
write(ln, string'("[the whole point] false eliminations = ")
& integer'image(n_false_elim)
& string'(", false survivals = ") & integer'image(n_false_surv));
writeline(output, ln);
if n_reach /= 256 then
write(ln, string'("FAIL: exhaustive sweep incomplete"));
writeline(output, ln);
errors := errors + 1;
end if;
if errors = 0 then
write(ln, string'("PASS: 0 errors in ") & integer'image(checks)
& string'(" checks"));
else
write(ln, string'("FAIL: ") & integer'image(errors)
& string'(" errors in ") & integer'image(checks) & string'(" checks"));
end if;
writeline(output, ln);
done <= true;
wait;
end process;
end architecture;10. Exhaustive Verification
| Measure | Verilog | SystemVerilog | VHDL |
|---|---|---|---|
| (kind × from-device × CRC × addr0 × post-reset × toggle) reached | 256 / 256 | 256 / 256 | 256 / 256 |
| …reached by directed stimulus alone | 256 / 256 | 256 / 256 | 256 / 256 |
| survivor-orderings swept | 6 / 6 | 6 / 6 | 6 / 6 |
| empty-shortlist routes swept | 6 / 6 | 6 / 6 | 6 / 6 |
| Steps | 24609 | 24609 | 24609 |
| Checks executed | 517424 | 517424 | 517424 |
| trace events | 18391 | 18391 | 18408 |
| CRC errors seen | 6137 | 6137 | 6106 |
| device transmissions | 9119 | 9119 | 9180 |
| eliminations performed | 6841 | 6841 | 6812 |
| false eliminations | 0 | 0 | 0 |
| false survivals | 0 | 0 | 0 |
| Result | PASS | PASS | PASS |
The event toggle is a sixth dimension of the sweep rather than a constant, and that was not the first plan. Section 12 explains what it cost.
11. Mutation Testing
These mutations are defects in reasoning rather than in logic, which is what makes them worth studying: each one is a plausible-looking inference that a tired engineer would accept at two in the morning.
| # | Mutation | Verilog | SysVer | VHDL |
|---|---|---|---|---|
| I5 | one rule eliminates the wrong class | 94102 | 94102 | 93371 |
| I1 | a CRC error "rules out" signal integrity — narrowing by suggestion | 83612 | 83612 | 83334 |
| I4 | one class starts already eliminated | 57198 | 57198 | 57337 |
| I6 | the toggle rule fires with no history | 41776 | 41776 | 42087 |
| I2 | elimination is not monotonic — a class comes back | 35867 | 35867 | 34811 |
| I7 | conclusive reports any non-empty shortlist | 21363 | 21363 | 21457 |
| I3 | conclusive reports an EMPTY shortlist | 811 | 811 | 819 |
| — | unmutated baseline | 0 | 0 | 0 |
All seven die in all three languages.
I1 is the mutation this chapter exists for. It reverses one condition so that a CRC error eliminates signal integrity instead of a clean packet doing so — which is narrowing by what the evidence suggests, expressed as a rule. It is the most common debugging error there is, and here it is a single !.
I3 scores lowest at 811 and is the subtlest. It calls an empty shortlist conclusive — so a trace that has contradicted every hypothesis is reported as a diagnosis. Everything else about the module still works: every rule is right, every elimination is correct, the count is accurate. It simply reports "the fault is X" at the moment the evidence has ruled out X.
Directed against random
| # | V all | V directed | V random | VHDL all | VHDL directed | VHDL random |
|---|---|---|---|---|---|---|
| I1 | 83612 | 2293 | 81319 | 83334 | 2293 | 81041 |
| I2 | 35867 | 163 | 35704 | 34811 | 163 | 34648 |
| I3 | 811 | 51 | 760 | 819 | 51 | 768 |
| I4 | 57198 | 1835 | 55363 | 57337 | 1835 | 55502 |
| I5 | 94102 | 938 | 93164 | 93371 | 938 | 92433 |
| I6 | 41776 | 575 | 41201 | 42087 | 575 | 41512 |
| I7 | 21363 | 862 | 20501 | 21457 | 862 | 20595 |
| — | BASE 0 | 0 | 0 | 0 | 0 | 0 |
Every directed column identical, and the directed-only baseline reaches 256/256 with 0 errors.
12. Two Mutations Scored 0 and 3, and Both Were Sweep Gaps
I6 survived directed stimulus entirely
I6 removes the have_tog term, so a single DATA packet "rules out" a desynchronised toggle — eliminating on evidence that does not exist yet, which is the mirror image of I1's mistake.
It scored 0 directed. The reason was embarrassingly simple: every scenario in the exhaustive sweep drove its event with toggle = 0, and the reset value of the toggle history is also 0. So ev_toggle != last_tog was false on every first DATA packet, and the mutation never had a chance to fire.
Sweep: 8 kinds x from_dev x crc x addr0 x post_reset = 128
...with toggle HARDCODED to 0.
A rule that wrongly ignores "is there any history yet" can
only fire when the first packet's toggle DIFFERS from the
history's reset value -- which that sweep never produced.
Adding toggle as a sixth dimension: 256 points, I6 -> 575.I3 scored 3, which is indistinguishable from luck
I3 calls an empty shortlist conclusive. The session in phase 2 empties the shortlist exactly once, at its final step, and checks conclusive there. One scenario, three checks, three kills.
The fix was to reach the empty state by six different routes — rotating which elimination lands last — and to check the shortlist-size-to-verdict relationship after every elimination rather than only at the end. I3 went from 3 to 51.
That is still the lowest score in the table, and it should be: an empty shortlist is a rare state, and the property is narrow. What matters is that 51 is a number produced by 6 deliberate scenarios rather than by one accident.
13. The Session, Step by Step
This is the answer to the interview question, as a transcript. The fault is a device that applies SET_ADDRESS too early — chapter 27.3's mutation C1, in silicon.
The report. The device enumerates about one time in five. The failure rate changes when the host is changed and does not change when the cable is changed.
Step 1 — what does the trace show? Thousands of CRC errors and a scattering of timeouts. The shortlist is six. Nothing has been excluded.
Step 2 — is there a single clean packet from the device? Yes, several. A marginal PHY does not selectively corrupt: it corrupts at a rate set by its margin, and it does not produce long clean runs. Signal integrity is out. Five.
Step 3 — do consecutive DATA packets alternate their toggle? Yes. Toggle desynchronisation is out. Four.
Step 4 — is there a timeout? Yes. A device that transmits when it should not does not produce silence; a timeout is the absence of traffic. Unsolicited transmission is out. Three.
Step 5 — is there a STALL anywhere in the trace? Yes, one. A STALL is a considered response — the device parsed a request and declined it — which means packets are arriving intact in both directions. A hub or repeater fault is out. Two.
Step 6 — is there activity outside the post-reset window? Yes, plenty. A device browning out under load does not keep running between resets. Power is out. One.
Step 7 — what survives? The address being applied too early. And now the confirming test, which is the one worth designing: address the device at address 0 immediately after the SET_ADDRESS status stage. A correct device answers; this one does not, because it has already moved.
14. Follow-Ups the Interviewer Will Ask
"What do you do first with a trace?" Look for what cannot be true. Not for what looks wrong — everything looks wrong in a failing trace.
"The trace is full of CRC errors. Where do you start?" Not with the scope. A CRC error is consistent with at least five different faults, so it narrows nothing. Find a clean device packet first.
"It works when plugged in directly rather than through a hub." That is weak evidence about the hub and strong evidence that something is timing- or translation-sensitive. Very often it is a device bug that a forgiving hub masks — the same shape as an interconnect that silently splits an illegal AXI burst.
"How do you tell a device bug from a host bug?" Change the host. If the failure rate changes and the device does not, the device is sensitive to something the host varies — which usually means timing.
"What if nothing is excluded?" Then you need a different measurement, not more of the same trace. That is what an empty shortlist is telling you when it happens: the fault is outside your model.
"The bug is intermittent. How do you make it reproducible before you debug it?" Find the variable it depends on, then make that variable extreme. If it depends on bus load, saturate the bus; if on buffer alignment, force the worst alignment. An intermittent bug that you can make continuous is an ordinary bug.
"When do you stop and ask for help?" When the shortlist stops shrinking. Two more hours of the same trace will not shrink it; a different instrument might.
15. UVM: Triage as a Scoreboard
// Triage in a verification environment rather than on a bench. The value
// is the same and the mechanism is identical: maintain, per fault class,
// the evidence that would rule it out -- and NEVER the evidence that
// suggests it.
//
// Used at regression scale this answers a question no single test can:
// across ten thousand failing runs, which hypotheses survive ALL of them?
typedef enum {
FC_SIGNAL, // PHY / cable / termination
FC_ADDR_EARLY, // SET_ADDRESS applied before the status stage
FC_TOGGLE, // data toggle desynchronised
FC_UNSOLICITED, // device transmits without a token
FC_HUB, // repeater or transaction translator
FC_POWER // brown-out / insufficient current
} fault_class_e;
class trace_event extends uvm_sequence_item;
`uvm_object_utils(trace_event)
rand bit from_device;
rand bit crc_bad;
rand bit addr_is_zero;
rand bit after_reset;
rand bit toggle;
rand int kind; // SOF / TOKEN / DATA / ACK / NAK / STALL / TIMEOUT / RESET
function new(string name = "trace_event"); super.new(name); endfunction
endclass
class triage_scoreboard extends uvm_scoreboard;
`uvm_component_utils(triage_scoreboard)
uvm_analysis_imp #(trace_event, triage_scoreboard) ap;
// Every class starts POSSIBLE. A triage tool that begins with a
// favourite has already made the mistake it exists to prevent.
bit alive [fault_class_e];
// The evidence that eliminated each class, kept so the report can say
// WHY -- an elimination nobody can justify is an elimination nobody
// should trust.
string why [fault_class_e];
bit have_toggle;
bit last_toggle;
int unsigned n_events, n_crc, n_eliminations;
function new(string name, uvm_component parent);
super.new(name, parent);
ap = new("ap", this);
foreach (alive[c]) alive[c] = 1'b1;
alive[FC_SIGNAL] = 1'b1;
alive[FC_ADDR_EARLY] = 1'b1;
alive[FC_TOGGLE] = 1'b1;
alive[FC_UNSOLICITED] = 1'b1;
alive[FC_HUB] = 1'b1;
alive[FC_POWER] = 1'b1;
endfunction
// Elimination is MONOTONIC: a class, once ruled out, stays ruled out.
// Evidence does not expire, and a later event that merely looks
// consistent with a class cannot revive it -- otherwise the shortlist
// oscillates and its size means nothing.
function void eliminate(fault_class_e c, string evidence);
if (!alive[c]) return;
alive[c] = 1'b0;
why[c] = evidence;
n_eliminations++;
`uvm_info("TRIAGE",
$sformatf("%s ruled out: %s", c.name(), evidence), UVM_LOW)
endfunction
function void write(trace_event t);
n_events++;
if (t.crc_bad) n_crc++;
// ---- NOTE WHAT IS ABSENT ----
//
// There is no rule here of the form "a CRC error suggests signal
// integrity". A CRC error is consistent with every class in the list,
// so it eliminates nothing and therefore appears nowhere.
// A CLEAN packet from the device: a marginal PHY does not selectively
// corrupt, so signal integrity cannot be the whole story.
if (t.from_device && !t.crc_bad)
eliminate(FC_SIGNAL,
"a clean packet arrived from the device; a marginal PHY does not selectively corrupt");
// An ACK at a NON-ZERO address: the device is reachable where the host
// thinks it is, so the address did not move early.
if (t.kind == 3 /* ACK */ && !t.addr_is_zero)
eliminate(FC_ADDR_EARLY,
"a transaction completed at a non-zero address; the device is reachable where the host expects");
// Two consecutive DATA packets with DIFFERENT toggles. The have_toggle
// guard matters: a single DATA packet is not evidence about a sequence.
if (t.kind == 2 /* DATA */) begin
if (have_toggle && t.toggle != last_toggle)
eliminate(FC_TOGGLE,
"consecutive DATA packets alternated their toggle; the sequence is advancing");
last_toggle = t.toggle;
have_toggle = 1'b1;
end
// A TIMEOUT: a device that transmits when it should not does not
// produce silence.
if (t.kind == 6 /* TIMEOUT */)
eliminate(FC_UNSOLICITED,
"a turnaround timeout occurred; a device transmitting out of turn does not go silent");
// A STALL: a considered response, so packets arrive intact both ways.
if (t.kind == 5 /* STALL */)
eliminate(FC_HUB,
"the device STALLed, which is a parsed response; packets arrive intact in both directions");
// Activity outside the post-reset window: not browning out.
if (!t.after_reset && t.kind != 7 /* RESET */)
eliminate(FC_POWER,
"traffic occurred outside the post-reset window; a browning-out device does not keep running");
endfunction
function int unsigned n_alive();
int unsigned n = 0;
foreach (alive[c]) if (alive[c]) n++;
return n;
endfunction
function void report_phase(uvm_phase phase);
super.report_phase(phase);
`uvm_info("TRIAGE",
$sformatf("%0d events, %0d CRC errors, %0d eliminations, %0d classes remain",
n_events, n_crc, n_eliminations, n_alive()), UVM_LOW)
foreach (alive[c])
if (alive[c])
`uvm_info("TRIAGE/ALIVE",
$sformatf("still possible: %s", c.name()), UVM_LOW)
else
`uvm_info("TRIAGE/OUT",
$sformatf("ruled out: %s -- %s", c.name(), why[c]), UVM_LOW)
// ---- the verdict, and the two ways it can be misreported ----
//
// Exactly one survivor is a diagnosis. Anything else is not, and the
// two failure modes are opposite: reporting a guess as a diagnosis,
// and reporting "the fault is X" when the evidence has ruled out X.
if (n_alive() == 1) begin
foreach (alive[c])
if (alive[c])
`uvm_info("TRIAGE/VERDICT",
$sformatf("CONCLUSIVE: %s is the only class consistent with this trace. Design a test that distinguishes it from its absence.",
c.name()), UVM_LOW)
end else if (n_alive() == 0) begin
// NOT a diagnosis. The fault is outside the model, which is a real
// and useful conclusion -- and reporting it as a diagnosis would be
// the worst possible outcome.
`uvm_error("TRIAGE/OUTSIDE",
"every candidate class was ruled out: the fault is outside this model and a different measurement is needed")
end else begin
`uvm_info("TRIAGE/VERDICT",
$sformatf("INCONCLUSIVE: %0d classes remain. This trace cannot distinguish them; find a measurement that excludes one.",
n_alive()), UVM_LOW)
end
endfunction
endclass16. Common Misconceptions
"CRC errors mean signal integrity." They are consistent with at least five faults. They narrow nothing.
"Start with the most likely cause." Start with what the evidence excludes. Likelihood is a prior; exclusion is data.
"It works without the hub, so it is the hub." Very often it is a device bug that a forgiving hub masks.
"An intermittent bug is hard to debug." An intermittent bug whose variable you have identified is an ordinary bug. Find the variable, then make it extreme.
"A shortlist of two is nearly a diagnosis." It is two hypotheses. Pick a measurement that excludes one.
"An empty shortlist means the tool is broken." It means the fault is outside the model — a real conclusion, and not a diagnosis.
"A wrong elimination is a minor error." It is worse than no elimination: nobody re-examines an eliminated class, and the shortlist lends it authority.
"Once narrowed to one class, you are done." You have a hypothesis. The last step is always an experiment designed to distinguish it from its absence.
"Evidence can be re-evaluated later." Not in this scheme, and deliberately so. Non-monotonic elimination makes the shortlist oscillate and its size meaningless.
17. Exercises
1. Take the five symptoms in section 2 and, for each, list every fault class it is consistent with. Then write the exclusion rule that each symptom's absence would give you.
2. I1 reverses one condition and scores 83,612. Write the reversed rule out in English and explain why it is persuasive despite being backwards.
3. I5 applies correct evidence to the wrong class and scores highest of the seven. Explain why that failure mode costs more human time than a tool that eliminates nothing.
4. I6 scored 0 against a 128-point sweep that was genuinely complete. Identify the missing dimension from the design's port list alone, and give a procedure that would find such gaps systematically.
5. Design the confirming experiment for each of the six fault classes — a test that distinguishes the class from its absence, not one that merely looks for it.
6. Add a seventh class: a host that violates the turnaround timeout. Give its elimination rule and say which existing rules it interacts with.
7. The scoreboard in section 15 records why each class was eliminated. Design a regression-scale report that aggregates shortlists across ten thousand failing runs, and say what its most useful output is.
18. Summary
| Idea | Why it matters |
|---|---|
| Narrow by exclusion, not suggestion | every USB symptom is many-to-one |
| A CRC error narrows nothing | 6137 of them, zero eliminations |
| Every class starts possible | a favourite is the mistake to avoid |
| A clean packet excludes signal integrity | a marginal PHY does not selectively corrupt |
| A timeout excludes unsolicited traffic | talking too much does not cause silence |
| A STALL excludes a hub fault | a parsed response means packets arrive |
| Elimination is monotonic | or the shortlist oscillates and means nothing |
| A wrong elimination is worse than none | nobody re-examines a cleared class |
| One survivor is a hypothesis | the last step is always an experiment |
| An empty shortlist is not a diagnosis | the fault is outside the model |
| "Works without the hub" is weak evidence | often a device bug the hub masks |
| Record why each class was eliminated | an unjustifiable elimination is untrustworthy |
| An attribute held constant is not swept | I6 scored 0 against a complete 128-point sweep |
| A property checked once dies on one seed | I3 scored 3 until six routes reached the state |
| 256 states, 7 mutations, 3 languages | 0 false eliminations in 517,424 checks |
Tooling
| Step | Command |
|---|---|
| Verilog-2005 | iverilog -g2005 -o td_v.out td_v.v td_v_tb.v && ./td_v.out |
| SystemVerilog | iverilog -g2012 -o td_sv.out td_sv.sv td_sv_tb.sv && ./td_sv.out |
| VHDL-2008 analyse | nvc --std=2008 -a td_vhdl.vhd td_vhdl_tb.vhd |
| VHDL-2008 elaborate | nvc --std=2008 -e tb_td_vhdl |
| VHDL-2008 run | nvc --std=2008 -r tb_td_vhdl |
| One mutation | iverilog -g2005 -DMUT_I1 -o mm td_v_mut.v td_v_tb.v && ./mm |
| Directed only (Verilog) | iverilog -g2005 -DDIRECTED_ONLY -o mm td_v_mut.v td_v_tb.v && ./mm |
| Directed only (VHDL) | nvc --std=2008 -e -gDIRECTED_ONLY=true tb_td_vhdl |
All three implementations pass with 0 errors: all 256 combinations of event kind, direction, CRC status, address, reset window and toggle — reached by directed stimulus alone; six orderings that each leave a different class as the sole survivor; six routes that empty the shortlist entirely; zero false eliminations and zero false survivals in 517,424 checks; and every one of the seven mutations killed by directed stimulus alone, with all seven directed scores identical across languages.
Chapter 27.10 — Senior Architecture Tradeoffs closes the module and the track's interview tier: given an SoC role, choose between USB, PCIe and Thunderbolt and defend it. Its central discipline is the one this chapter applies to faults, applied instead to requirements — the question is never which bus is better, it is which constraint you are unwilling to relax.
Continue learning
Related tutorials
- Related topic
Transfer Errors
Retries mask the error rate — a link losing a third of its traffic reports a perfect transfer success rate, because the denominator everybody uses is the wrong one.
- Related topic
What Is USB?
The opening interview question answered with one load-bearing idea instead of a list — USB is host-scheduled, and polling, NAK, the frame and the missing interrupt line are all consequences of it.
- Related topic
Host / Device / Hub Identification
Host and device take fifteen seconds; the hub is where the interview is decided — a hub is a repeater, not a switch, and no downstream port can ever reach another one.
- Related topic
The Enumeration Question
Attach to configured, with the one step almost everybody gets backwards — SET_ADDRESS takes effect after the status stage, and a device that switches early is invisible to the host.
Standards & specifications
- Governing standard
- USB-IF (Universal Serial Bus Specification)(opens USB Implementers Forum (USB-IF) in a new tab)
Defines the USB bus — its electrical signalling, connectors, packet and transaction model, device framework and the descriptors a device must expose — together with the device-class specifications layered on it. It does not define host-controller register interfaces (xHCI and EHCI are separate documents) nor any operating system's driver architecture.
This page also covers RTL structure, verification approach and debugging technique. Those are engineering practice built on the standard, not requirements the standard itself imposes.
Where this fits
Part of the USB curriculum.
