USB · Module 27
Senior Verification Strategy
The UVM component list is the easy half — the answer that gets hired is a checker that fails when the stimulus was too weak to prove anything, and counts the opportunities to prove it.
The second senior question, and the one most likely to be asked of anybody applying to a verification team.
1. The Question
"Design a UVM environment for a USB device controller."
Everybody answers this the same way: driver, monitor, sequencer, sequence library, scoreboard, coverage collector, agent, environment, virtual sequencer. It is a correct list. It is also on the first page of every UVM tutorial, and reciting it tells the interviewer only that you have read one.
2. Why This Is the Answer
Consider the most common way a verification effort fails. It is not a missing checker. It is a checker that was present, correct, and never in a position to fire.
- A device that never had data pending cannot transmit unsolicited, so the unsolicited check passes trivially.
- A bus on which every token is addressed to the device under test cannot distinguish one with an address comparator from one without.
- A run with a single
DATApacket in it makes every data-toggle check unreachable. - A
SETUPthat never arrives while the endpoint is halted means the "SETUP is never stalled" rule was never tested.
Every one of those produces a green report. Every one of them is a device that ships broken.
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
The report everybody writes:
0 errors in 40,000,000 cycles. PASS
The report that means something:
0 violations
11229 cycles in which the device was addressed
5013 cycles with data pending and no token
3785 tokens for other devices
423 SETUPs while halted
1651 consecutive DATA pairs
VERDICT: CONCLUSIVE, PASS
The second one can be wrong. The first cannot even be
checked.3. What We Are Building
usb_protocol_checker is that idea as synthesisable hardware — which is the useful form, because it can then sit in emulation and in silicon bring-up as well as in simulation.
It has two kinds of output:
- VIOLATIONS — things the device did wrong.
- OPPORTUNITIES — situations in which it could have done them.
And one output that combines them, which is the point of the whole module:
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
verdict_valid = every opportunity class occurred at least once
verdict_pass = verdict_valid AND no violations
There is NO output on this module that says "pass" without
also asserting that the run was capable of failing.Two kinds of counter, and a verdict that needs both
The verdict staying inconclusive until the last class arrives
Cycles 0 to 5 have zero violations and verdict_pass is low. That is the entire thesis on one waveform.
4. The Four Rules and the Five Opportunities
| Violation | What it catches |
|---|---|
v_unsolicited | the device transmitted without being asked |
v_silent | the device was asked and said nothing |
v_stall_setup | the device stalled a SETUP, locking itself out |
v_bad_toggle | two consecutive DATA packets with the same toggle |
| Opportunity | Why a zero here invalidates the run |
|---|---|
o_addressed | nothing was asked, so "silent" was unreachable |
o_idle_with_data | the device never wanted to speak, so "unsolicited" was unreachable |
o_foreign_token | every token was ours, so the address check was untested |
o_setup_halted | no SETUP arrived while halted — the rarest of the five |
o_toggle_pair | fewer than two DATA packets, so the toggle check had nothing to compare |
5. Verilog-2005 RTL
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// =====================================================================
// usb_protocol_checker -- "Design a UVM environment" answered in
// hardware, because the answer that gets hired is not a list of
// components. It is this:
//
// A checker that FAILS when the stimulus was too weak to prove
// anything.
//
// Anybody can name a driver, a monitor, a sequencer and a scoreboard.
// What separates a verification engineer from somebody who has read
// about verification is knowing that "0 errors" is not a result --
// it is a result ONLY IF the run was capable of producing errors, and
// the only way to know that is to measure the OPPORTUNITIES and fail
// when there were none.
//
// So this module is a synthesisable protocol checker with two kinds of
// output:
//
// VIOLATIONS -- things the device did wrong.
// OPPORTUNITIES -- situations in which it COULD have done them.
//
// A run with zero violations and zero opportunities is a failed run,
// and this design says so on a pin.
// =====================================================================
module usb_protocol_checker #(
parameter integer TURNAROUND = 16 // cycles the host waits for a reply
) (
input wire clk,
input wire rst_n,
// ---- the bus, as a monitor sees it ----
input wire tok_valid,
input wire [1:0] tok_pid, // T_OUT / T_IN / T_SOF / T_SETUP
input wire tok_for_us,
input wire dev_tx, // the device is transmitting
input wire [2:0] dev_pid, // R_NONE/R_DATA/R_NAK/R_STALL/R_ACK
input wire dev_toggle,
input wire halted,
input wire data_avail,
// ---- VIOLATIONS: things that are wrong ----
output wire v_unsolicited, // spoke without being asked
output wire v_silent, // was asked and said nothing
output wire v_stall_setup, // stalled a SETUP
output wire v_bad_toggle, // repeated a toggle it should have flipped
output wire [31:0] n_violations,
// ---- OPPORTUNITIES: situations in which it could have gone wrong ----
//
// These are the outputs that make a zero meaningful. Each counts a
// cycle in which a BROKEN device would have been caught.
output wire [31:0] o_addressed, // was asked for something
output wire [31:0] o_idle_with_data,// had data and no token
output wire [31:0] o_foreign_token, // a token for somebody else
output wire [31:0] o_setup_halted, // a SETUP while halted
output wire [31:0] o_toggle_pair, // two consecutive DATA responses
// ---- THE VERDICT, and it is not "no violations" ----
output wire verdict_valid, // the run proved something
output wire verdict_pass // ...and the device was correct
);
localparam [1:0] T_OUT = 2'd0, T_IN = 2'd1, T_SOF = 2'd2, T_SETUP = 2'd3;
localparam [2:0] R_NONE = 3'd0, R_DATA = 3'd1, R_NAK = 3'd2,
R_STALL = 3'd3, R_ACK = 3'd4;
// ---- what happened in the previous cycle ----
reg p_asked; // a token addressed to us, expecting a reply
reg [1:0] p_pid;
reg p_halted;
// ---- toggle history, for the flip check ----
reg last_tog;
reg have_last_tog;
reg [31:0] viol_c;
reg [31:0] ad_c, idle_c, foreign_c, setuph_c, togp_c;
reg vu_r, vs_r, vss_r, vbt_r;
assign v_unsolicited = vu_r;
assign v_silent = vs_r;
assign v_stall_setup = vss_r;
assign v_bad_toggle = vbt_r;
assign n_violations = viol_c;
assign o_addressed = ad_c;
assign o_idle_with_data = idle_c;
assign o_foreign_token = foreign_c;
assign o_setup_halted = setuph_c;
assign o_toggle_pair = togp_c;
// =================================================================
// THE VERDICT.
//
// A run is only conclusive if EVERY opportunity class was exercised.
// Zero violations against zero opportunities is not a pass -- it is a
// run that proved nothing, and reporting it as a pass is the single
// most common way verification lies.
//
// Note that verdict_pass is deliberately ANDed with verdict_valid:
// there is no output on this module that says "pass" without also
// asserting that the run was capable of failing.
// =================================================================
// ONE source of truth. Deriving each output from the five counters
// independently makes them two parallel derivations, and a mutation of
// the validity rule then breaks only one of the two -- which showed up
// as directed columns that differed where identical stimulus against
// identical logic must give the same number.
wire v_valid = (ad_c != 32'd0) &&
(idle_c != 32'd0) &&
(foreign_c != 32'd0) &&
(setuph_c != 32'd0) &&
(togp_c != 32'd0);
assign verdict_valid = v_valid;
// Derived FROM v_valid, so there is no output on this module that says
// "pass" without also asserting that the run was capable of failing.
assign verdict_pass = v_valid && (viol_c == 32'd0);
wire asked_now = tok_valid && tok_for_us && (tok_pid != T_SOF);
// =================================================================
// THE FOUR VIOLATION CONDITIONS, as combinational wires.
//
// Computed here rather than incremented in four separate statements
// inside the clocked block. Four non-blocking `viol_c <= viol_c + 1`
// assignments in one always block are four writes to one register and
// only the last takes effect -- so two violations in one cycle would be
// counted as one.
//
// That is not hypothetical: it is the same defect this track already
// found in a hub's blocked-port counter, and the bench caught it here
// the same way, as 42 "violation count disagrees" errors.
// =================================================================
wire w_unsol = dev_tx && !p_asked;
// The device was asked and said nothing. Guarded against w_unsol so the
// two stay mutually exclusive, as an if/else-if would make them.
// Silence is not a safe default: the host waits out the turnaround
// timeout, retries, and eventually reports the endpoint as absent.
wire w_silent = !w_unsol && p_asked && !dev_tx;
// A SETUP may never be refused: clearing a halt is itself a control
// transfer, so a device that stalls SETUP has locked itself out.
wire w_stallsetup = p_asked && (p_pid == T_SETUP)
&& dev_tx && (dev_pid == R_STALL);
// Two consecutive DATA responses with the same toggle. The host cannot
// tell the second from a retransmission of the first, and discards it.
wire w_badtoggle = dev_tx && (dev_pid == R_DATA)
&& have_last_tog && (dev_toggle == last_tog);
wire [2:0] n_viol_now = {2'd0, w_unsol} + {2'd0, w_silent}
+ {2'd0, w_stallsetup} + {2'd0, w_badtoggle};
always @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
p_asked <= 1'b0;
p_pid <= T_OUT;
p_halted <= 1'b0;
last_tog <= 1'b0;
have_last_tog <= 1'b0;
viol_c <= 32'd0;
ad_c <= 32'd0;
idle_c <= 32'd0;
foreign_c <= 32'd0;
setuph_c <= 32'd0;
togp_c <= 32'd0;
vu_r <= 1'b0;
vs_r <= 1'b0;
vss_r <= 1'b0;
vbt_r <= 1'b0;
end else begin
// ---- the four violations, and ONE add of their count ----
vu_r <= w_unsol;
vs_r <= w_silent;
vss_r <= w_stallsetup;
vbt_r <= w_badtoggle;
if (n_viol_now != 3'd0) viol_c <= viol_c + {29'd0, n_viol_now};
// The toggle history advances on every DATA response, whether or not
// it was the right one.
if (dev_tx && (dev_pid == R_DATA)) begin
last_tog <= dev_toggle;
have_last_tog <= 1'b1;
end
// ===========================================================
// OPPORTUNITIES. Every one of these counts a cycle in which a
// BROKEN device would have been caught -- which is what makes
// a zero in the violation counters mean anything at all.
// ===========================================================
// The device was asked for something, so "silent" was reachable.
if (asked_now) ad_c <= ad_c + 32'd1;
// The device had data pending and no token on the bus, so
// "unsolicited" was reachable. Without this, a device that never
// had anything to send would pass violation 1 trivially.
if (!tok_valid && data_avail) idle_c <= idle_c + 32'd1;
// A token for somebody else, so answering out of turn was
// reachable. A bus on which every token is ours cannot
// distinguish a device with an address comparator from one
// without.
if (tok_valid && !tok_for_us && (tok_pid != T_SOF))
foreign_c <= foreign_c + 32'd1;
// A SETUP arriving while halted, so "stalled a SETUP" was
// reachable. This is the rarest situation of the five and the
// one a random test is least likely to produce.
if (asked_now && (tok_pid == T_SETUP) && halted)
setuph_c <= setuph_c + 32'd1;
// Two consecutive DATA responses, so the toggle check had
// something to compare. One DATA packet in a whole run makes
// violation 4 unreachable.
if (dev_tx && (dev_pid == R_DATA) && have_last_tog)
togp_c <= togp_c + 32'd1;
// ---- advance the history ----
p_asked <= asked_now;
p_halted <= halted;
if (asked_now) p_pid <= tok_pid;
end
end
endmodule6. SystemVerilog RTL
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// =====================================================================
// usb_protocol_checker -- SystemVerilog.
//
// The violation and opportunity sets become named enumerations, which is
// the point rather than decoration: this module's whole output is a
// DIAGNOSIS, and a verification report that says "3 violations" is
// useless where one that says "v_stall_setup" is actionable.
//
// Originally: "Design a UVM environment" answered in
// hardware, because the answer that gets hired is not a list of
// components. It is this:
//
// A checker that FAILS when the stimulus was too weak to prove
// anything.
//
// Anybody can name a driver, a monitor, a sequencer and a scoreboard.
// What separates a verification engineer from somebody who has read
// about verification is knowing that "0 errors" is not a result --
// it is a result ONLY IF the run was capable of producing errors, and
// the only way to know that is to measure the OPPORTUNITIES and fail
// when there were none.
//
// So this module is a synthesisable protocol checker with two kinds of
// output:
//
// VIOLATIONS -- things the device did wrong.
// OPPORTUNITIES -- situations in which it COULD have done them.
//
// A run with zero violations and zero opportunities is a failed run,
// and this design says so on a pin.
// =====================================================================
module usb_protocol_checker #(
parameter int TURNAROUND = 16 // cycles the host waits for a reply
) (
input logic clk,
input logic rst_n,
// ---- the bus, as a monitor sees it ----
input logic tok_valid,
input logic [1:0] tok_pid, // T_OUT / T_IN / T_SOF / T_SETUP
input logic tok_for_us,
input logic dev_tx, // the device is transmitting
input logic [2:0] dev_pid, // R_NONE/R_DATA/R_NAK/R_STALL/R_ACK
input logic dev_toggle,
input logic halted,
input logic data_avail,
// ---- VIOLATIONS: things that are wrong ----
output logic v_unsolicited, // spoke without being asked
output logic v_silent, // was asked and said nothing
output logic v_stall_setup, // stalled a SETUP
output logic v_bad_toggle, // repeated a toggle it should have flipped
output logic [31:0]n_violations,
// ---- OPPORTUNITIES: situations in which it could have gone wrong ----
//
// These are the outputs that make a zero meaningful. Each counts a
// cycle in which a BROKEN device would have been caught.
output logic [31:0]o_addressed, // was asked for something
output logic [31:0]o_idle_with_data,// had data and no token
output logic [31:0]o_foreign_token, // a token for somebody else
output logic [31:0]o_setup_halted, // a SETUP while halted
output logic [31:0]o_toggle_pair, // two consecutive DATA responses
// ---- THE VERDICT, and it is not "no violations" ----
output logic verdict_valid, // the run proved something
output logic verdict_pass // ...and the device was correct
);
typedef enum logic [1:0] { T_OUT = 2'd0, T_IN = 2'd1,
T_SOF = 2'd2, T_SETUP = 2'd3 } tok_e;
typedef enum logic [2:0] { R_NONE = 3'd0, R_DATA = 3'd1, R_NAK = 3'd2,
R_STALL = 3'd3, R_ACK = 3'd4 } rsp_e;
// ---- what happened in the previous cycle ----
logic p_asked; // a token addressed to us, expecting a reply
logic [1:0] p_pid;
logic p_halted;
// ---- toggle history, for the flip check ----
logic last_tog;
logic have_last_tog;
logic [31:0] viol_c;
logic [31:0] ad_c, idle_c, foreign_c, setuph_c, togp_c;
logic vu_r, vs_r, vss_r, vbt_r;
assign v_unsolicited = vu_r;
assign v_silent = vs_r;
assign v_stall_setup = vss_r;
assign v_bad_toggle = vbt_r;
assign n_violations = viol_c;
assign o_addressed = ad_c;
assign o_idle_with_data = idle_c;
assign o_foreign_token = foreign_c;
assign o_setup_halted = setuph_c;
assign o_toggle_pair = togp_c;
// =================================================================
// THE VERDICT.
//
// A run is only conclusive if EVERY opportunity class was exercised.
// Zero violations against zero opportunities is not a pass -- it is a
// run that proved nothing, and reporting it as a pass is the single
// most common way verification lies.
//
// Note that verdict_pass is deliberately ANDed with verdict_valid:
// there is no output on this module that says "pass" without also
// asserting that the run was capable of failing.
// =================================================================
// ONE source of truth. Deriving each output from the five counters
// independently makes them two parallel derivations, and a mutation of
// the validity rule then breaks only one of the two -- which showed up
// as directed columns that differed where identical stimulus against
// identical logic must give the same number.
wire v_valid = (ad_c != 32'd0) &&
(idle_c != 32'd0) &&
(foreign_c != 32'd0) &&
(setuph_c != 32'd0) &&
(togp_c != 32'd0);
assign verdict_valid = v_valid;
// Derived FROM v_valid, so there is no output on this module that says
// "pass" without also asserting that the run was capable of failing.
assign verdict_pass = v_valid && (viol_c == 32'd0);
wire asked_now = tok_valid && tok_for_us && (tok_pid != T_SOF);
// =================================================================
// THE FOUR VIOLATION CONDITIONS, as combinational wires.
//
// Computed here rather than incremented in four separate statements
// inside the clocked block. Four non-blocking `viol_c <= viol_c + 1`
// assignments in one always block are four writes to one register and
// only the last takes effect -- so two violations in one cycle would be
// counted as one.
//
// That is not hypothetical: it is the same defect this track already
// found in a hub's blocked-port counter, and the bench caught it here
// the same way, as 42 "violation count disagrees" errors.
// =================================================================
wire w_unsol = dev_tx && !p_asked;
// The device was asked and said nothing. Guarded against w_unsol so the
// two stay mutually exclusive, as an if/else-if would make them.
// Silence is not a safe default: the host waits out the turnaround
// timeout, retries, and eventually reports the endpoint as absent.
wire w_silent = !w_unsol && p_asked && !dev_tx;
// A SETUP may never be refused: clearing a halt is itself a control
// transfer, so a device that stalls SETUP has locked itself out.
wire w_stallsetup = p_asked && (p_pid == T_SETUP)
&& dev_tx && (dev_pid == R_STALL);
// Two consecutive DATA responses with the same toggle. The host cannot
// tell the second from a retransmission of the first, and discards it.
wire w_badtoggle = dev_tx && (dev_pid == R_DATA)
&& have_last_tog && (dev_toggle == last_tog);
wire [2:0] n_viol_now = {2'd0, w_unsol} + {2'd0, w_silent}
+ {2'd0, w_stallsetup} + {2'd0, w_badtoggle};
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
p_asked <= 1'b0;
p_pid <= T_OUT;
p_halted <= 1'b0;
last_tog <= 1'b0;
have_last_tog <= 1'b0;
viol_c <= 32'd0;
ad_c <= 32'd0;
idle_c <= 32'd0;
foreign_c <= 32'd0;
setuph_c <= 32'd0;
togp_c <= 32'd0;
vu_r <= 1'b0;
vs_r <= 1'b0;
vss_r <= 1'b0;
vbt_r <= 1'b0;
end else begin
// ---- the four violations, and ONE add of their count ----
vu_r <= w_unsol;
vs_r <= w_silent;
vss_r <= w_stallsetup;
vbt_r <= w_badtoggle;
if (n_viol_now != 3'd0) viol_c <= viol_c + {29'd0, n_viol_now};
// The toggle history advances on every DATA response, whether or not
// it was the right one.
if (dev_tx && (dev_pid == R_DATA)) begin
last_tog <= dev_toggle;
have_last_tog <= 1'b1;
end
// ===========================================================
// OPPORTUNITIES. Every one of these counts a cycle in which a
// BROKEN device would have been caught -- which is what makes
// a zero in the violation counters mean anything at all.
// ===========================================================
// The device was asked for something, so "silent" was reachable.
if (asked_now) ad_c <= ad_c + 32'd1;
// The device had data pending and no token on the bus, so
// "unsolicited" was reachable. Without this, a device that never
// had anything to send would pass violation 1 trivially.
if (!tok_valid && data_avail) idle_c <= idle_c + 32'd1;
// A token for somebody else, so answering out of turn was
// reachable. A bus on which every token is ours cannot
// distinguish a device with an address comparator from one
// without.
if (tok_valid && !tok_for_us && (tok_pid != T_SOF))
foreign_c <= foreign_c + 32'd1;
// A SETUP arriving while halted, so "stalled a SETUP" was
// reachable. This is the rarest situation of the five and the
// one a random test is least likely to produce.
if (asked_now && (tok_pid == T_SETUP) && halted)
setuph_c <= setuph_c + 32'd1;
// Two consecutive DATA responses, so the toggle check had
// something to compare. One DATA packet in a whole run makes
// violation 4 unreachable.
if (dev_tx && (dev_pid == R_DATA) && have_last_tog)
togp_c <= togp_c + 32'd1;
// ---- advance the history ----
p_asked <= asked_now;
p_halted <= halted;
if (asked_now) p_pid <= tok_pid;
end
end
endmodule7. VHDL-2008 RTL
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
-- =====================================================================
-- usb_protocol_checker -- VHDL-2008.
--
-- "Design a UVM environment" answered in hardware, because the answer
-- that gets hired is not a list of components. It is this:
--
-- A checker that FAILS when the stimulus was too weak to prove
-- anything.
--
-- Anybody can name a driver, a monitor, a sequencer and a scoreboard.
-- What separates a verification engineer from somebody who has read
-- about verification is knowing that "0 errors" is not a result -- it is
-- a result ONLY IF the run was capable of producing errors, and the only
-- way to know that is to count the OPPORTUNITIES and fail when there
-- were none.
--
-- So this module has two kinds of output: VIOLATIONS, things the device
-- did wrong; and OPPORTUNITIES, situations in which it could have. A run
-- with zero of each is a failed run, and this design says so on a pin.
-- =====================================================================
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
package vc_pkg is
constant TOK_OUT : std_logic_vector(1 downto 0) := "00";
constant TOK_IN : std_logic_vector(1 downto 0) := "01";
constant TOK_SOF : std_logic_vector(1 downto 0) := "10";
constant TOK_SETUP : std_logic_vector(1 downto 0) := "11";
constant RSP_NONE : std_logic_vector(2 downto 0) := "000";
constant RSP_DATA : std_logic_vector(2 downto 0) := "001";
constant RSP_NAK : std_logic_vector(2 downto 0) := "010";
constant RSP_STALL : std_logic_vector(2 downto 0) := "011";
constant RSP_ACK : std_logic_vector(2 downto 0) := "100";
end package;
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
use work.vc_pkg.all;
entity usb_protocol_checker is
generic (
TURNAROUND : natural := 16
);
port (
clk : in std_logic;
rst_n : in std_logic;
tok_valid : in std_logic;
tok_pid : in std_logic_vector(1 downto 0);
tok_for_us : in std_logic;
dev_tx : in std_logic;
dev_pid : in std_logic_vector(2 downto 0);
dev_toggle : in std_logic;
halted : in std_logic;
data_avail : in std_logic;
-- VIOLATIONS: things that are wrong
v_unsolicited : out std_logic;
v_silent : out std_logic;
v_stall_setup : out std_logic;
v_bad_toggle : out std_logic;
n_violations : out std_logic_vector(31 downto 0);
-- OPPORTUNITIES: situations in which it could have gone wrong. These
-- are the outputs that make a zero mean something: each counts a cycle
-- in which a BROKEN device would have been caught.
o_addressed : out std_logic_vector(31 downto 0);
o_idle_with_data : out std_logic_vector(31 downto 0);
o_foreign_token : out std_logic_vector(31 downto 0);
o_setup_halted : out std_logic_vector(31 downto 0);
o_toggle_pair : out std_logic_vector(31 downto 0);
-- THE VERDICT, and it is not "no violations"
verdict_valid : out std_logic;
verdict_pass : out std_logic
);
end entity;
architecture rtl of usb_protocol_checker is
signal p_asked : std_logic := '0';
signal p_pid : std_logic_vector(1 downto 0) := TOK_OUT;
signal p_halted : std_logic := '0';
signal last_tog : std_logic := '0';
signal have_last_tog : std_logic := '0';
signal viol_c : unsigned(31 downto 0) := (others => '0');
signal ad_c, idle_c, foreign_c, setuph_c, togp_c : unsigned(31 downto 0)
:= (others => '0');
signal vu_r, vs_r, vss_r, vbt_r : std_logic := '0';
signal asked_now : std_logic;
-- ONE source of truth for the verdict. Computing verdict_pass from the
-- five counters independently makes it a second, parallel derivation --
-- and a mutation of the validity rule then breaks only one of the two
-- outputs. That asymmetry showed up as directed columns of 694 against
-- 693 where identical stimulus and identical logic must give the same
-- number.
signal v_valid : std_logic;
-- ---- the four violation conditions, computed combinationally ----
--
-- Four separate increments of one counter inside a clocked process are
-- four signal assignments and only the last takes effect, so two
-- violations in one cycle would be counted as one. The bench caught
-- exactly that, as 42 "violation count disagrees" errors.
signal w_unsol, w_silent, w_stallsetup, w_badtoggle : std_logic;
signal n_viol_now : natural range 0 to 4;
-- std_logic to 0/1. A conditional expression in argument position --
-- `(1 when b = '1' else 0)` -- is VHDL-2019, not 2008, so summing four
-- flags needs a function rather than an inline conditional.
function b2n(b : std_logic) return natural is
begin
if b = '1' then return 1; else return 0; end if;
end function;
begin
asked_now <= '1' when (tok_valid = '1' and tok_for_us = '1'
and tok_pid /= TOK_SOF) else '0';
w_unsol <= '1' when (dev_tx = '1' and p_asked = '0') else '0';
-- The device was asked and said nothing. Guarded against w_unsol so the
-- two stay mutually exclusive. Silence is not a safe default: the host
-- waits out the turnaround timeout, retries, and eventually reports the
-- endpoint as absent.
w_silent <= '1' when (w_unsol = '0' and p_asked = '1' and dev_tx = '0')
else '0';
-- A SETUP may never be refused: clearing a halt is itself a control
-- transfer, so a device that stalls SETUP has locked itself out.
w_stallsetup <= '1' when (p_asked = '1' and p_pid = TOK_SETUP
and dev_tx = '1' and dev_pid = RSP_STALL)
else '0';
-- Two consecutive DATA responses with the same toggle. The host cannot
-- tell the second from a retransmission of the first, and discards it.
w_badtoggle <= '1' when (dev_tx = '1' and dev_pid = RSP_DATA
and have_last_tog = '1' and dev_toggle = last_tog)
else '0';
n_viol_now <= b2n(w_unsol) + b2n(w_silent)
+ b2n(w_stallsetup) + b2n(w_badtoggle);
v_unsolicited <= vu_r;
v_silent <= vs_r;
v_stall_setup <= vss_r;
v_bad_toggle <= vbt_r;
n_violations <= std_logic_vector(viol_c);
o_addressed <= std_logic_vector(ad_c);
o_idle_with_data <= std_logic_vector(idle_c);
o_foreign_token <= std_logic_vector(foreign_c);
o_setup_halted <= std_logic_vector(setuph_c);
o_toggle_pair <= std_logic_vector(togp_c);
-- =================================================================
-- THE VERDICT.
--
-- A run is conclusive only if EVERY opportunity class was exercised.
-- Zero violations against zero opportunities is not a pass -- it is a
-- run that proved nothing, and reporting it as a pass is the single
-- most common way verification lies.
--
-- verdict_pass is deliberately ANDed with verdict_valid: there is no
-- output on this entity that says "pass" without also asserting that
-- the run was capable of failing.
-- =================================================================
v_valid <= '1' when (ad_c /= 0 and idle_c /= 0 and foreign_c /= 0
and setuph_c /= 0 and togp_c /= 0) else '0';
verdict_valid <= v_valid;
-- Derived FROM v_valid, so there is no output on this entity that says
-- "pass" without also asserting that the run was capable of failing.
verdict_pass <= '1' when (v_valid = '1' and viol_c = 0) else '0';
main : process(clk, rst_n)
begin
if rst_n = '0' then
p_asked <= '0';
p_pid <= TOK_OUT;
p_halted <= '0';
last_tog <= '0';
have_last_tog <= '0';
viol_c <= (others => '0');
ad_c <= (others => '0');
idle_c <= (others => '0');
foreign_c <= (others => '0');
setuph_c <= (others => '0');
togp_c <= (others => '0');
vu_r <= '0';
vs_r <= '0';
vss_r <= '0';
vbt_r <= '0';
elsif rising_edge(clk) then
-- ---- the four violations, and ONE add of their count ----
vu_r <= w_unsol;
vs_r <= w_silent;
vss_r <= w_stallsetup;
vbt_r <= w_badtoggle;
if n_viol_now /= 0 then
viol_c <= viol_c + to_unsigned(n_viol_now, 32);
end if;
-- The toggle history advances on every DATA response, whether or not
-- it was the right one.
if dev_tx = '1' and dev_pid = RSP_DATA then
last_tog <= dev_toggle;
have_last_tog <= '1';
end if;
-- ===========================================================
-- OPPORTUNITIES. Every one counts a cycle in which a BROKEN
-- device would have been caught -- which is what makes a zero in
-- the violation counters mean anything at all.
-- ===========================================================
-- The device was asked for something, so "silent" was reachable.
if asked_now = '1' then ad_c <= ad_c + 1; end if;
-- The device had data pending and no token on the bus, so
-- "unsolicited" was reachable. Without this, a device that never had
-- anything to send passes the unsolicited check trivially.
if tok_valid = '0' and data_avail = '1' then idle_c <= idle_c + 1; end if;
-- A token for somebody else, so answering out of turn was reachable.
-- A bus on which every token is ours cannot distinguish a device
-- with an address comparator from one without.
if tok_valid = '1' and tok_for_us = '0' and tok_pid /= TOK_SOF then
foreign_c <= foreign_c + 1;
end if;
-- A SETUP arriving while halted: the rarest of the five situations
-- and the one a random test is least likely ever to produce.
if asked_now = '1' and tok_pid = TOK_SETUP and halted = '1' then
setuph_c <= setuph_c + 1;
end if;
-- Two consecutive DATA responses, so the toggle check had something
-- to compare. One DATA packet in a whole run makes it unreachable.
if dev_tx = '1' and dev_pid = RSP_DATA and have_last_tog = '1' then
togp_c <= togp_c + 1;
end if;
-- ---- advance the history ----
p_asked <= asked_now;
p_halted <= halted;
if asked_now = '1' then p_pid <= tok_pid; end if;
end if;
end process;
end architecture;8. How You Verify a Checker
This is the part of the chapter that generalises furthest, because a checker is verified differently from a design and the difference is where most people stop.
For a design you ask one question: does it do the right thing? For a checker you ask two:
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
TRUE POSITIVES -- does it fire when the defect is present?
FALSE POSITIVES -- does it stay SILENT when it is not?
The second is the one that gets skipped, and the second is
the one that decides whether the checker survives contact
with a real project.A checker with false positives gets commented out. The commit message always says "noisy assert", and after that the check does not exist. So this bench:
- runs a correct device model for the overwhelming majority of its cycles — including a 30,000-cycle random phase that injects no defect at all — and requires the violation count to be exactly zero;
- then injects each of five defect shapes in turn and requires exactly the matching output to fire;
- and checks that a defect never makes the run inconclusive, because a checker that disqualified its own evidence when it found something would be hiding failures.
Verilog-2005 testbench
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// =====================================================================
// Testbench for usb_protocol_checker.
//
// A checker is verified differently from a design, and the difference
// is the whole reason this chapter exists.
//
// For a design you ask "does it do the right thing". For a checker you
// ask TWO things, and the second is the one people skip:
//
// TRUE POSITIVES -- does it fire when the defect is present?
// FALSE POSITIVES -- does it stay silent when it is NOT?
//
// A checker with false positives gets commented out, and the commit
// message always says "noisy assert". So this bench runs a CORRECT
// device model for the majority of its cycles and requires the
// violation count to be exactly zero -- and only then injects each
// defect in turn and requires exactly the matching output to fire.
//
// It also checks the thing the checker exists to provide: that
// verdict_valid is FALSE until every opportunity class has been seen.
// =====================================================================
`timescale 1ns/1ps
module tb_vc_v;
localparam integer TURNAROUND = 16;
localparam [1:0] T_OUT = 2'd0, T_IN = 2'd1, T_SOF = 2'd2, T_SETUP = 2'd3;
localparam [2:0] R_NONE = 3'd0, R_DATA = 3'd1, R_NAK = 3'd2,
R_STALL = 3'd3, R_ACK = 3'd4;
// the injected defects the device model can exhibit
localparam integer D_NONE = 0, D_UNSOL = 1, D_SILENT = 2,
D_STALLSETUP = 3, D_TOGGLE = 4,
// A device that transmits unsolicited with NOTHING to send. It matters
// because it is the only defect that distinguishes "the device spoke
// when it should not have" from the narrower "the device spoke when it
// had data and should not have" -- and the narrow version is a
// plausible-looking optimisation that a reviewer would wave through.
//
// Without this shape, a checker that requires data_avail passes every
// test in this suite. It scored a clean 0.
D_UNSOL_NODATA = 5;
reg clk = 1'b0, rst_n = 1'b0;
reg tok_valid = 1'b0;
reg [1:0] tok_pid = T_OUT;
reg tok_for_us = 1'b0;
reg dev_tx = 1'b0;
reg [2:0] dev_pid = R_NONE;
reg dev_toggle = 1'b0;
reg halted = 1'b0;
reg data_avail = 1'b0;
wire v_unsolicited, v_silent, v_stall_setup, v_bad_toggle;
wire [31:0] n_violations;
wire [31:0] o_addressed, o_idle_with_data, o_foreign_token,
o_setup_halted, o_toggle_pair;
wire verdict_valid, verdict_pass;
usb_protocol_checker #(.TURNAROUND(TURNAROUND)) dut (
.clk(clk), .rst_n(rst_n),
.tok_valid(tok_valid), .tok_pid(tok_pid), .tok_for_us(tok_for_us),
.dev_tx(dev_tx), .dev_pid(dev_pid), .dev_toggle(dev_toggle),
.halted(halted), .data_avail(data_avail),
.v_unsolicited(v_unsolicited), .v_silent(v_silent),
.v_stall_setup(v_stall_setup), .v_bad_toggle(v_bad_toggle),
.n_violations(n_violations),
.o_addressed(o_addressed), .o_idle_with_data(o_idle_with_data),
.o_foreign_token(o_foreign_token), .o_setup_halted(o_setup_halted),
.o_toggle_pair(o_toggle_pair),
.verdict_valid(verdict_valid), .verdict_pass(verdict_pass)
);
always #5 clk = ~clk;
integer errors = 0, checks = 0, steps = 0;
integer seed;
function [31:0] urand;
input dummy;
begin urand = $random(seed) & 32'h3FFF_FFFF; end
endfunction
// ---- the shadow: the same predicates, recomputed independently ----
reg s_asked;
reg [1:0] s_pid;
reg s_last_tog, s_have_tog;
reg [31:0] x_viol, x_ad, x_idle, x_foreign, x_setuph, x_togp;
// ---- the two headline counters for a CHECKER ----
integer n_false_pos = 0; // fired on a correct device
integer n_missed = 0; // failed to fire on a real defect
// ---- exhaustive reach over (pid, ours, halted, data, defect) ----
reg reach [0:191];
integer ri, n_reach;
task ck(input cond, input [255:0] what);
begin
checks = checks + 1;
if (!cond) begin
errors = errors + 1;
if (errors <= 20)
$display(" ERROR @%0t step=%0d: %0s", $time, steps, what);
end
end
endtask
// ---------------------------------------------------------------
// A device model, with a selectable defect.
//
// The CORRECT branch is the one that matters most: every cycle it
// runs is a cycle in which the checker must stay silent.
// ---------------------------------------------------------------
task drive(input tv, input [1:0] tp, input fu,
input hl, input da, input integer defect);
reg d_tx;
reg [2:0] d_pid;
reg d_tog;
reg asked;
begin
tok_valid = tv; tok_pid = tp; tok_for_us = fu;
halted = hl; data_avail = da;
// was the device asked in the PREVIOUS cycle?
asked = s_asked;
// ---- the device's response this cycle ----
d_tx = 1'b0;
d_pid = R_NONE;
d_tog = s_last_tog;
if (asked) begin
d_tx = 1'b1;
if (s_pid == T_IN) begin
if (hl) d_pid = R_STALL;
else if (da) begin
d_pid = R_DATA;
// a correct device FLIPS the toggle on every DATA packet
d_tog = s_have_tog ? ~s_last_tog : 1'b0;
end
else d_pid = R_NAK;
end else begin
// a SETUP is never stalled; other OUT traffic on a halted
// endpoint is
if (hl && (s_pid != T_SETUP)) d_pid = R_STALL;
else d_pid = R_ACK;
end
end
// ---- inject the selected defect ----
case (defect)
D_UNSOL: if (!asked && da) begin d_tx = 1'b1; d_pid = R_DATA; end
D_UNSOL_NODATA:
if (!asked && !da) begin d_tx = 1'b1; d_pid = R_NAK; end
D_SILENT: if (asked) begin d_tx = 1'b0; d_pid = R_NONE; end
D_STALLSETUP: if (asked && (s_pid == T_SETUP)) d_pid = R_STALL;
D_TOGGLE: if (d_tx && (d_pid == R_DATA)) d_tog = s_last_tog;
default: ; // D_NONE: a correct device
endcase
dev_tx = d_tx; dev_pid = d_pid; dev_toggle = d_tog;
// ---- the shadow's expectation, recomputed from scratch ----
if (d_tx && !asked) x_viol = x_viol + 1;
else if (asked && !d_tx) x_viol = x_viol + 1;
if (asked && (s_pid == T_SETUP) && d_tx && (d_pid == R_STALL))
x_viol = x_viol + 1;
if (d_tx && (d_pid == R_DATA) && s_have_tog && (d_tog == s_last_tog))
x_viol = x_viol + 1;
if (tv && fu && (tp != T_SOF)) x_ad = x_ad + 1;
if (!tv && da) x_idle = x_idle + 1;
if (tv && !fu && (tp != T_SOF)) x_foreign = x_foreign + 1;
if (tv && fu && (tp == T_SETUP) && hl) x_setuph = x_setuph + 1;
if (d_tx && (d_pid == R_DATA) && s_have_tog) x_togp = x_togp + 1;
@(posedge clk);
#1;
steps = steps + 1;
// ---- PROPERTY 1: each violation fires exactly when it should ----
ck(v_unsolicited === (d_tx && !asked),
"v_unsolicited disagrees");
ck(v_silent === (asked && !d_tx),
"v_silent disagrees");
ck(v_stall_setup === (asked && (s_pid == T_SETUP) && d_tx
&& (d_pid == R_STALL)),
"v_stall_setup disagrees");
ck(v_bad_toggle === (d_tx && (d_pid == R_DATA) && s_have_tog
&& (d_tog == s_last_tog)),
"v_bad_toggle disagrees");
// ---- PROPERTY 2: NO FALSE POSITIVES on a correct device ----
//
// The most important check in the file. A checker that fires on
// correct behaviour is a checker somebody deletes.
if (defect == D_NONE) begin
if (v_unsolicited || v_silent || v_stall_setup || v_bad_toggle)
n_false_pos = n_false_pos + 1;
ck(!v_unsolicited, "false positive: unsolicited, on a correct device");
ck(!v_silent, "false positive: silent, on a correct device");
ck(!v_stall_setup, "false positive: stalled SETUP, on a correct device");
ck(!v_bad_toggle, "false positive: bad toggle, on a correct device");
end
ck(n_false_pos == 0, "the checker fired on a correct device");
// ---- PROPERTY 3: the counters agree ----
ck(n_violations === x_viol, "violation count disagrees");
ck(o_addressed === x_ad, "addressed-opportunity count disagrees");
ck(o_idle_with_data === x_idle, "idle-with-data count disagrees");
ck(o_foreign_token === x_foreign,"foreign-token count disagrees");
ck(o_setup_halted === x_setuph, "setup-while-halted count disagrees");
ck(o_toggle_pair === x_togp, "toggle-pair count disagrees");
// ---- PROPERTY 4: the verdict requires every opportunity class ----
ck(verdict_valid === ((x_ad != 0) && (x_idle != 0) && (x_foreign != 0)
&& (x_setuph != 0) && (x_togp != 0)),
"verdict_valid disagrees with the opportunity counters");
// ---- PROPERTY 5: there is no pass without a valid verdict ----
//
// The claim this whole module exists to make. `verdict_pass` must
// NEVER be high while the run is inconclusive, no matter how clean
// the violation counters are.
// Compared against the SHADOW's own recomputation, never against the
// DUT's verdict_valid output. Using a DUT output inside an expected
// value makes the check self-referential: mutation H1 forces
// verdict_valid high, the expectation moves with it, and the check
// passes vacuously. It reported 694 kills where the independent
// version reports 1254.
ck(verdict_pass === (((x_ad != 0) && (x_idle != 0) && (x_foreign != 0)
&& (x_setuph != 0) && (x_togp != 0))
&& (x_viol == 0)),
"verdict_pass disagrees");
ck(!(verdict_pass && !verdict_valid),
"the checker reported PASS on an inconclusive run");
// advance the shadow's history
s_asked <= 1'b0;
s_asked = tv && fu && (tp != T_SOF);
if (s_asked) s_pid = tp;
if (d_tx && (d_pid == R_DATA)) begin
s_last_tog = d_tog;
s_have_tog = 1'b1;
end
tok_valid = 1'b0; dev_tx = 1'b0; dev_pid = R_NONE;
end
endtask
task reset_dut;
begin
rst_n = 1'b0;
tok_valid = 0; dev_tx = 0; halted = 0; data_avail = 0;
@(posedge clk); @(posedge clk);
rst_n = 1'b1;
s_asked = 1'b0; s_pid = T_OUT;
s_last_tog = 1'b0; s_have_tog = 1'b0;
x_viol = 0; x_ad = 0; x_idle = 0; x_foreign = 0; x_setuph = 0; x_togp = 0;
@(posedge clk); #1;
end
endtask
integer pi, oi, hi, di, dfi, k;
initial begin
for (ri = 0; ri < 192; ri = ri + 1) reach[ri] = 1'b0;
seed = 32'd27008;
// =============================================================
// PHASE 1 (DIRECTED, EXHAUSTIVE) -- every bus situation against
// every device behaviour. 4 pids x ours x halted x data x 6
// defects = 192.
// =============================================================
for (dfi = 0; dfi < 6; dfi = dfi + 1)
for (pi = 0; pi < 4; pi = pi + 1)
for (oi = 0; oi < 2; oi = oi + 1)
for (hi = 0; hi < 2; hi = hi + 1)
for (di = 0; di < 2; di = di + 1) begin
reset_dut;
// a token, then the response cycle, then two idle cycles
drive(1'b1, pi[1:0], oi[0], hi[0], di[0], dfi);
drive(1'b0, T_OUT, 1'b0, hi[0], di[0], dfi);
drive(1'b0, T_OUT, 1'b0, hi[0], di[0], dfi);
ri = (dfi * 32) + (pi * 8) + (oi * 4) + (hi * 2) + di;
reach[ri] = 1'b1;
end
// =============================================================
// PHASE 2 (DIRECTED) -- THE POINT OF THE CHAPTER.
//
// A long, entirely CORRECT run, and the verdict must be
// INCONCLUSIVE until every opportunity class has occurred --
// then and only then may it report a pass.
//
// The opportunity classes are introduced one at a time, in the
// order of how likely a random test is to produce them, and the
// verdict is checked after each.
// =============================================================
reset_dut;
// (a) only SOF traffic: nothing is asked, nothing can be proved
for (k = 0; k < 8; k = k + 1) drive(1'b1, T_SOF, 1'b1, 1'b0, 1'b0, D_NONE);
ck(verdict_valid === 1'b0, "the verdict was valid after SOFs alone");
ck(verdict_pass === 1'b0, "the checker passed a run of nothing but SOFs");
// (b) the device is addressed: "silent" becomes reachable
for (k = 0; k < 4; k = k + 1) begin
drive(1'b1, T_IN, 1'b1, 1'b0, 1'b1, D_NONE);
drive(1'b0, T_OUT, 1'b0, 1'b0, 1'b1, D_NONE);
end
ck(o_addressed !== 32'd0, "the addressed opportunity was not counted");
ck(verdict_valid === 1'b0, "the verdict was valid with 4 of 5 classes missing");
// (c) idle with data pending: "unsolicited" becomes reachable
for (k = 0; k < 4; k = k + 1) drive(1'b0, T_OUT, 1'b0, 1'b0, 1'b1, D_NONE);
ck(o_idle_with_data !== 32'd0, "the idle-with-data opportunity was not counted");
ck(verdict_valid === 1'b0, "the verdict was valid with 3 of 5 classes missing");
// (d) a token for somebody else
for (k = 0; k < 4; k = k + 1) drive(1'b1, T_IN, 1'b0, 1'b0, 1'b1, D_NONE);
ck(o_foreign_token !== 32'd0, "the foreign-token opportunity was not counted");
ck(verdict_valid === 1'b0, "the verdict was valid with 2 of 5 classes missing");
// (e) two consecutive DATA responses, so the toggle can be compared
for (k = 0; k < 4; k = k + 1) begin
drive(1'b1, T_IN, 1'b1, 1'b0, 1'b1, D_NONE);
drive(1'b0, T_OUT, 1'b0, 1'b0, 1'b1, D_NONE);
end
ck(o_toggle_pair !== 32'd0, "the toggle-pair opportunity was not counted");
ck(verdict_valid === 1'b0, "the verdict was valid with 1 of 5 classes missing");
// (f) and finally a SETUP while halted -- the rarest of the five,
// and the one a random test is least likely ever to produce
drive(1'b1, T_SETUP, 1'b1, 1'b1, 1'b0, D_NONE);
drive(1'b0, T_OUT, 1'b0, 1'b1, 1'b0, D_NONE);
ck(o_setup_halted !== 32'd0, "the setup-while-halted opportunity was not counted");
ck(verdict_valid === 1'b1, "the verdict was still invalid with all 5 classes seen");
ck(verdict_pass === 1'b1, "a correct device did not pass a conclusive run");
// =============================================================
// PHASE 3 (DIRECTED, EXHAUSTIVE) -- each defect, in turn, must
// make the verdict fail while leaving it VALID.
//
// A defect must not be allowed to make the run inconclusive --
// that would be a checker that hides failures by disqualifying
// its own evidence.
// =============================================================
for (dfi = 1; dfi < 6; dfi = dfi + 1) begin
reset_dut;
// build up all five opportunity classes with a correct device
for (k = 0; k < 6; k = k + 1) begin
drive(1'b1, T_IN, 1'b1, 1'b0, 1'b1, D_NONE);
drive(1'b0, T_OUT, 1'b0, 1'b0, 1'b1, D_NONE);
end
for (k = 0; k < 4; k = k + 1) drive(1'b1, T_IN, 1'b0, 1'b0, 1'b1, D_NONE);
drive(1'b1, T_SETUP, 1'b1, 1'b1, 1'b0, D_NONE);
drive(1'b0, T_OUT, 1'b0, 1'b1, 1'b0, D_NONE);
ck(verdict_valid === 1'b1, "the opportunity build-up did not conclude");
ck(verdict_pass === 1'b1, "a correct build-up did not pass");
// now introduce the defect
for (k = 0; k < 6; k = k + 1) begin
drive(1'b1, T_IN, 1'b1, 1'b0, 1'b1, dfi);
drive(1'b0, T_OUT, 1'b0, 1'b0, 1'b1, dfi);
end
if (dfi == D_STALLSETUP) begin
drive(1'b1, T_SETUP, 1'b1, 1'b1, 1'b0, dfi);
drive(1'b0, T_OUT, 1'b0, 1'b1, 1'b0, dfi);
end
if (dfi == D_UNSOL)
for (k = 0; k < 4; k = k + 1) drive(1'b0, T_OUT, 1'b0, 1'b0, 1'b1, dfi);
if (dfi == D_UNSOL_NODATA)
for (k = 0; k < 4; k = k + 1) drive(1'b0, T_OUT, 1'b0, 1'b0, 1'b0, dfi);
ck(n_violations !== 32'd0, "a defective device produced no violations");
ck(verdict_valid === 1'b1, "a defect made the run inconclusive");
ck(verdict_pass === 1'b0, "a defective device passed");
end
// =============================================================
// PHASE 4 (RANDOM) -- a correct device on a busy bus.
//
// Deliberately CORRECT throughout: this phase exists to find
// false positives, and it is the longest phase in the suite for
// exactly that reason.
// =============================================================
`ifndef DIRECTED_ONLY
reset_dut;
for (k = 0; k < 30000; k = k + 1)
drive((urand(0) % 3) != 0, urand(0) % 4, (urand(0) % 4) != 0,
(urand(0) % 8) == 0, (urand(0) % 2) == 0, D_NONE);
ck(n_violations === 32'd0,
"the checker reported violations against a correct device over 30000 cycles");
`endif
n_reach = 0;
for (ri = 0; ri < 192; ri = ri + 1) if (reach[ri]) n_reach = n_reach + 1;
$display("steps=%0d checks=%0d reach=%0d/192 errors=%0d",
steps, checks, n_reach, errors);
$display("[opportunities] addressed=%0d idle_with_data=%0d foreign=%0d setup_halted=%0d toggle_pairs=%0d",
o_addressed, o_idle_with_data, o_foreign_token,
o_setup_halted, o_toggle_pair);
$display("[the whole point] false positives = %0d, missed defects = %0d",
n_false_pos, n_missed);
if (n_reach != 192) begin
$display("FAIL: exhaustive sweep incomplete"); errors = errors + 1;
end
if (errors == 0) $display("PASS: 0 errors in %0d checks", checks);
else $display("FAIL: %0d errors in %0d checks", errors, checks);
$finish;
end
endmoduleSystemVerilog testbench
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// =====================================================================
// Testbench for usb_protocol_checker.
//
// A checker is verified differently from a design, and the difference
// is the whole reason this chapter exists.
//
// For a design you ask "does it do the right thing". For a checker you
// ask TWO things, and the second is the one people skip:
//
// TRUE POSITIVES -- does it fire when the defect is present?
// FALSE POSITIVES -- does it stay silent when it is NOT?
//
// A checker with false positives gets commented out, and the commit
// message always says "noisy assert". So this bench runs a CORRECT
// device model for the majority of its cycles and requires the
// violation count to be exactly zero -- and only then injects each
// defect in turn and requires exactly the matching output to fire.
//
// It also checks the thing the checker exists to provide: that
// verdict_valid is FALSE until every opportunity class has been seen.
// =====================================================================
`timescale 1ns/1ps
module tb_vc_sv;
localparam integer TURNAROUND = 16;
localparam [1:0] T_OUT = 2'd0, T_IN = 2'd1, T_SOF = 2'd2, T_SETUP = 2'd3;
localparam [2:0] R_NONE = 3'd0, R_DATA = 3'd1, R_NAK = 3'd2,
R_STALL = 3'd3, R_ACK = 3'd4;
// the injected defects the device model can exhibit
localparam integer D_NONE = 0, D_UNSOL = 1, D_SILENT = 2,
D_STALLSETUP = 3, D_TOGGLE = 4,
// A device that transmits unsolicited with NOTHING to send. It matters
// because it is the only defect that distinguishes "the device spoke
// when it should not have" from the narrower "the device spoke when it
// had data and should not have" -- and the narrow version is a
// plausible-looking optimisation that a reviewer would wave through.
//
// Without this shape, a checker that requires data_avail passes every
// test in this suite. It scored a clean 0.
D_UNSOL_NODATA = 5;
logic clk = 1'b0, rst_n = 1'b0;
logic tok_valid = 1'b0;
logic [1:0] tok_pid = T_OUT;
logic tok_for_us = 1'b0;
logic dev_tx = 1'b0;
logic [2:0] dev_pid = R_NONE;
logic dev_toggle = 1'b0;
logic halted = 1'b0;
logic data_avail = 1'b0;
logic v_unsolicited, v_silent, v_stall_setup, v_bad_toggle;
logic [31:0] n_violations;
logic [31:0] o_addressed, o_idle_with_data, o_foreign_token,
o_setup_halted, o_toggle_pair;
logic verdict_valid, verdict_pass;
usb_protocol_checker #(.TURNAROUND(TURNAROUND)) dut (
.clk(clk), .rst_n(rst_n),
.tok_valid(tok_valid), .tok_pid(tok_pid), .tok_for_us(tok_for_us),
.dev_tx(dev_tx), .dev_pid(dev_pid), .dev_toggle(dev_toggle),
.halted(halted), .data_avail(data_avail),
.v_unsolicited(v_unsolicited), .v_silent(v_silent),
.v_stall_setup(v_stall_setup), .v_bad_toggle(v_bad_toggle),
.n_violations(n_violations),
.o_addressed(o_addressed), .o_idle_with_data(o_idle_with_data),
.o_foreign_token(o_foreign_token), .o_setup_halted(o_setup_halted),
.o_toggle_pair(o_toggle_pair),
.verdict_valid(verdict_valid), .verdict_pass(verdict_pass)
);
always #5 clk = ~clk;
integer errors = 0, checks = 0, steps = 0;
integer seed;
function automatic logic [31:0] urand(bit dummy);
return $random(seed) & 32'h3FFF_FFFF;
endfunction
// ---- the shadow: the same predicates, recomputed independently ----
logic s_asked;
logic [1:0] s_pid;
logic s_last_tog, s_have_tog;
logic [31:0] x_viol, x_ad, x_idle, x_foreign, x_setuph, x_togp;
// ---- the two headline counters for a CHECKER ----
integer n_false_pos = 0; // fired on a correct device
integer n_missed = 0; // failed to fire on a real defect
// ---- exhaustive reach over (pid, ours, halted, data, defect) ----
logic reach [0:191];
integer ri, n_reach;
task ck(input logic cond, input logic [255:0] what);
begin
checks = checks + 1;
if (!cond) begin
errors = errors + 1;
if (errors <= 20)
$display(" ERROR @%0t step=%0d: %0s", $time, steps, what);
end
end
endtask
// ---------------------------------------------------------------
// A device model, with a selectable defect.
//
// The CORRECT branch is the one that matters most: every cycle it
// runs is a cycle in which the checker must stay silent.
// ---------------------------------------------------------------
task drive(input logic tv, input logic [1:0] tp, input logic fu,
input logic hl, input logic da, input integer defect);
logic d_tx;
logic [2:0] d_pid;
logic d_tog;
logic asked;
begin
tok_valid = tv; tok_pid = tp; tok_for_us = fu;
halted = hl; data_avail = da;
// was the device asked in the PREVIOUS cycle?
asked = s_asked;
// ---- the device's response this cycle ----
d_tx = 1'b0;
d_pid = R_NONE;
d_tog = s_last_tog;
if (asked) begin
d_tx = 1'b1;
if (s_pid == T_IN) begin
if (hl) d_pid = R_STALL;
else if (da) begin
d_pid = R_DATA;
// a correct device FLIPS the toggle on every DATA packet
d_tog = s_have_tog ? ~s_last_tog : 1'b0;
end
else d_pid = R_NAK;
end else begin
// a SETUP is never stalled; other OUT traffic on a halted
// endpoint is
if (hl && (s_pid != T_SETUP)) d_pid = R_STALL;
else d_pid = R_ACK;
end
end
// ---- inject the selected defect ----
case (defect)
D_UNSOL: if (!asked && da) begin d_tx = 1'b1; d_pid = R_DATA; end
D_UNSOL_NODATA:
if (!asked && !da) begin d_tx = 1'b1; d_pid = R_NAK; end
D_SILENT: if (asked) begin d_tx = 1'b0; d_pid = R_NONE; end
D_STALLSETUP: if (asked && (s_pid == T_SETUP)) d_pid = R_STALL;
D_TOGGLE: if (d_tx && (d_pid == R_DATA)) d_tog = s_last_tog;
default: ; // D_NONE: a correct device
endcase
dev_tx = d_tx; dev_pid = d_pid; dev_toggle = d_tog;
// ---- the shadow's expectation, recomputed from scratch ----
if (d_tx && !asked) x_viol = x_viol + 1;
else if (asked && !d_tx) x_viol = x_viol + 1;
if (asked && (s_pid == T_SETUP) && d_tx && (d_pid == R_STALL))
x_viol = x_viol + 1;
if (d_tx && (d_pid == R_DATA) && s_have_tog && (d_tog == s_last_tog))
x_viol = x_viol + 1;
if (tv && fu && (tp != T_SOF)) x_ad = x_ad + 1;
if (!tv && da) x_idle = x_idle + 1;
if (tv && !fu && (tp != T_SOF)) x_foreign = x_foreign + 1;
if (tv && fu && (tp == T_SETUP) && hl) x_setuph = x_setuph + 1;
if (d_tx && (d_pid == R_DATA) && s_have_tog) x_togp = x_togp + 1;
@(posedge clk);
#1;
steps = steps + 1;
// ---- PROPERTY 1: each violation fires exactly when it should ----
ck(v_unsolicited === (d_tx && !asked),
"v_unsolicited disagrees");
ck(v_silent === (asked && !d_tx),
"v_silent disagrees");
ck(v_stall_setup === (asked && (s_pid == T_SETUP) && d_tx
&& (d_pid == R_STALL)),
"v_stall_setup disagrees");
ck(v_bad_toggle === (d_tx && (d_pid == R_DATA) && s_have_tog
&& (d_tog == s_last_tog)),
"v_bad_toggle disagrees");
// ---- PROPERTY 2: NO FALSE POSITIVES on a correct device ----
//
// The most important check in the file. A checker that fires on
// correct behaviour is a checker somebody deletes.
if (defect == D_NONE) begin
if (v_unsolicited || v_silent || v_stall_setup || v_bad_toggle)
n_false_pos = n_false_pos + 1;
ck(!v_unsolicited, "false positive: unsolicited, on a correct device");
ck(!v_silent, "false positive: silent, on a correct device");
ck(!v_stall_setup, "false positive: stalled SETUP, on a correct device");
ck(!v_bad_toggle, "false positive: bad toggle, on a correct device");
end
ck(n_false_pos == 0, "the checker fired on a correct device");
// ---- PROPERTY 3: the counters agree ----
ck(n_violations === x_viol, "violation count disagrees");
ck(o_addressed === x_ad, "addressed-opportunity count disagrees");
ck(o_idle_with_data === x_idle, "idle-with-data count disagrees");
ck(o_foreign_token === x_foreign,"foreign-token count disagrees");
ck(o_setup_halted === x_setuph, "setup-while-halted count disagrees");
ck(o_toggle_pair === x_togp, "toggle-pair count disagrees");
// ---- PROPERTY 4: the verdict requires every opportunity class ----
ck(verdict_valid === ((x_ad != 0) && (x_idle != 0) && (x_foreign != 0)
&& (x_setuph != 0) && (x_togp != 0)),
"verdict_valid disagrees with the opportunity counters");
// ---- PROPERTY 5: there is no pass without a valid verdict ----
//
// The claim this whole module exists to make. `verdict_pass` must
// NEVER be high while the run is inconclusive, no matter how clean
// the violation counters are.
// Compared against the SHADOW's own recomputation, never against the
// DUT's verdict_valid output. Using a DUT output inside an expected
// value makes the check self-referential: mutation H1 forces
// verdict_valid high, the expectation moves with it, and the check
// passes vacuously. It reported 694 kills where the independent
// version reports 1254.
ck(verdict_pass === (((x_ad != 0) && (x_idle != 0) && (x_foreign != 0)
&& (x_setuph != 0) && (x_togp != 0))
&& (x_viol == 0)),
"verdict_pass disagrees");
ck(!(verdict_pass && !verdict_valid),
"the checker reported PASS on an inconclusive run");
// advance the shadow's history
s_asked <= 1'b0;
s_asked = tv && fu && (tp != T_SOF);
if (s_asked) s_pid = tp;
if (d_tx && (d_pid == R_DATA)) begin
s_last_tog = d_tog;
s_have_tog = 1'b1;
end
tok_valid = 1'b0; dev_tx = 1'b0; dev_pid = R_NONE;
end
endtask
task reset_dut;
begin
rst_n = 1'b0;
tok_valid = 0; dev_tx = 0; halted = 0; data_avail = 0;
@(posedge clk); @(posedge clk);
rst_n = 1'b1;
s_asked = 1'b0; s_pid = T_OUT;
s_last_tog = 1'b0; s_have_tog = 1'b0;
x_viol = 0; x_ad = 0; x_idle = 0; x_foreign = 0; x_setuph = 0; x_togp = 0;
@(posedge clk); #1;
end
endtask
integer pi, oi, hi, di, dfi, k;
initial begin
for (ri = 0; ri < 192; ri = ri + 1) reach[ri] = 1'b0;
seed = 32'd27008;
// =============================================================
// PHASE 1 (DIRECTED, EXHAUSTIVE) -- every bus situation against
// every device behaviour. 4 pids x ours x halted x data x 6
// defects = 192.
// =============================================================
for (dfi = 0; dfi < 6; dfi = dfi + 1)
for (pi = 0; pi < 4; pi = pi + 1)
for (oi = 0; oi < 2; oi = oi + 1)
for (hi = 0; hi < 2; hi = hi + 1)
for (di = 0; di < 2; di = di + 1) begin
reset_dut;
// a token, then the response cycle, then two idle cycles
drive(1'b1, pi[1:0], oi[0], hi[0], di[0], dfi);
drive(1'b0, T_OUT, 1'b0, hi[0], di[0], dfi);
drive(1'b0, T_OUT, 1'b0, hi[0], di[0], dfi);
ri = (dfi * 32) + (pi * 8) + (oi * 4) + (hi * 2) + di;
reach[ri] = 1'b1;
end
// =============================================================
// PHASE 2 (DIRECTED) -- THE POINT OF THE CHAPTER.
//
// A long, entirely CORRECT run, and the verdict must be
// INCONCLUSIVE until every opportunity class has occurred --
// then and only then may it report a pass.
//
// The opportunity classes are introduced one at a time, in the
// order of how likely a random test is to produce them, and the
// verdict is checked after each.
// =============================================================
reset_dut;
// (a) only SOF traffic: nothing is asked, nothing can be proved
for (k = 0; k < 8; k = k + 1) drive(1'b1, T_SOF, 1'b1, 1'b0, 1'b0, D_NONE);
ck(verdict_valid === 1'b0, "the verdict was valid after SOFs alone");
ck(verdict_pass === 1'b0, "the checker passed a run of nothing but SOFs");
// (b) the device is addressed: "silent" becomes reachable
for (k = 0; k < 4; k = k + 1) begin
drive(1'b1, T_IN, 1'b1, 1'b0, 1'b1, D_NONE);
drive(1'b0, T_OUT, 1'b0, 1'b0, 1'b1, D_NONE);
end
ck(o_addressed !== 32'd0, "the addressed opportunity was not counted");
ck(verdict_valid === 1'b0, "the verdict was valid with 4 of 5 classes missing");
// (c) idle with data pending: "unsolicited" becomes reachable
for (k = 0; k < 4; k = k + 1) drive(1'b0, T_OUT, 1'b0, 1'b0, 1'b1, D_NONE);
ck(o_idle_with_data !== 32'd0, "the idle-with-data opportunity was not counted");
ck(verdict_valid === 1'b0, "the verdict was valid with 3 of 5 classes missing");
// (d) a token for somebody else
for (k = 0; k < 4; k = k + 1) drive(1'b1, T_IN, 1'b0, 1'b0, 1'b1, D_NONE);
ck(o_foreign_token !== 32'd0, "the foreign-token opportunity was not counted");
ck(verdict_valid === 1'b0, "the verdict was valid with 2 of 5 classes missing");
// (e) two consecutive DATA responses, so the toggle can be compared
for (k = 0; k < 4; k = k + 1) begin
drive(1'b1, T_IN, 1'b1, 1'b0, 1'b1, D_NONE);
drive(1'b0, T_OUT, 1'b0, 1'b0, 1'b1, D_NONE);
end
ck(o_toggle_pair !== 32'd0, "the toggle-pair opportunity was not counted");
ck(verdict_valid === 1'b0, "the verdict was valid with 1 of 5 classes missing");
// (f) and finally a SETUP while halted -- the rarest of the five,
// and the one a random test is least likely ever to produce
drive(1'b1, T_SETUP, 1'b1, 1'b1, 1'b0, D_NONE);
drive(1'b0, T_OUT, 1'b0, 1'b1, 1'b0, D_NONE);
ck(o_setup_halted !== 32'd0, "the setup-while-halted opportunity was not counted");
ck(verdict_valid === 1'b1, "the verdict was still invalid with all 5 classes seen");
ck(verdict_pass === 1'b1, "a correct device did not pass a conclusive run");
// =============================================================
// PHASE 3 (DIRECTED, EXHAUSTIVE) -- each defect, in turn, must
// make the verdict fail while leaving it VALID.
//
// A defect must not be allowed to make the run inconclusive --
// that would be a checker that hides failures by disqualifying
// its own evidence.
// =============================================================
for (dfi = 1; dfi < 6; dfi = dfi + 1) begin
reset_dut;
// build up all five opportunity classes with a correct device
for (k = 0; k < 6; k = k + 1) begin
drive(1'b1, T_IN, 1'b1, 1'b0, 1'b1, D_NONE);
drive(1'b0, T_OUT, 1'b0, 1'b0, 1'b1, D_NONE);
end
for (k = 0; k < 4; k = k + 1) drive(1'b1, T_IN, 1'b0, 1'b0, 1'b1, D_NONE);
drive(1'b1, T_SETUP, 1'b1, 1'b1, 1'b0, D_NONE);
drive(1'b0, T_OUT, 1'b0, 1'b1, 1'b0, D_NONE);
ck(verdict_valid === 1'b1, "the opportunity build-up did not conclude");
ck(verdict_pass === 1'b1, "a correct build-up did not pass");
// now introduce the defect
for (k = 0; k < 6; k = k + 1) begin
drive(1'b1, T_IN, 1'b1, 1'b0, 1'b1, dfi);
drive(1'b0, T_OUT, 1'b0, 1'b0, 1'b1, dfi);
end
if (dfi == D_STALLSETUP) begin
drive(1'b1, T_SETUP, 1'b1, 1'b1, 1'b0, dfi);
drive(1'b0, T_OUT, 1'b0, 1'b1, 1'b0, dfi);
end
if (dfi == D_UNSOL)
for (k = 0; k < 4; k = k + 1) drive(1'b0, T_OUT, 1'b0, 1'b0, 1'b1, dfi);
if (dfi == D_UNSOL_NODATA)
for (k = 0; k < 4; k = k + 1) drive(1'b0, T_OUT, 1'b0, 1'b0, 1'b0, dfi);
ck(n_violations !== 32'd0, "a defective device produced no violations");
ck(verdict_valid === 1'b1, "a defect made the run inconclusive");
ck(verdict_pass === 1'b0, "a defective device passed");
end
// =============================================================
// PHASE 4 (RANDOM) -- a correct device on a busy bus.
//
// Deliberately CORRECT throughout: this phase exists to find
// false positives, and it is the longest phase in the suite for
// exactly that reason.
// =============================================================
`ifndef DIRECTED_ONLY
reset_dut;
for (k = 0; k < 30000; k = k + 1)
drive((urand(0) % 3) != 0, urand(0) % 4, (urand(0) % 4) != 0,
(urand(0) % 8) == 0, (urand(0) % 2) == 0, D_NONE);
ck(n_violations === 32'd0,
"the checker reported violations against a correct device over 30000 cycles");
`endif
n_reach = 0;
for (ri = 0; ri < 192; ri = ri + 1) if (reach[ri]) n_reach = n_reach + 1;
$display("steps=%0d checks=%0d reach=%0d/192 errors=%0d",
steps, checks, n_reach, errors);
$display("[opportunities] addressed=%0d idle_with_data=%0d foreign=%0d setup_halted=%0d toggle_pairs=%0d",
o_addressed, o_idle_with_data, o_foreign_token,
o_setup_halted, o_toggle_pair);
$display("[the whole point] false positives = %0d, missed defects = %0d",
n_false_pos, n_missed);
if (n_reach != 192) begin
$display("FAIL: exhaustive sweep incomplete"); errors = errors + 1;
end
if (errors == 0) $display("PASS: 0 errors in %0d checks", checks);
else $display("FAIL: %0d errors in %0d checks", errors, checks);
$finish;
end
endmoduleVHDL-2008 testbench
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
-- =====================================================================
-- Testbench for usb_protocol_checker (VHDL-2008).
--
-- A checker is verified differently from a design, and the difference is
-- the whole reason this chapter exists. For a design you ask "does it do
-- the right thing". For a checker you ask TWO things, and the second is
-- the one people skip:
--
-- TRUE POSITIVES -- does it fire when the defect is present?
-- FALSE POSITIVES -- does it stay silent when it is NOT?
--
-- A checker with false positives gets commented out, and the commit
-- message always says "noisy assert". So this bench runs a CORRECT device
-- model for most of its cycles and requires the violation count to be
-- exactly zero, and only then injects each defect in turn.
-- =====================================================================
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
use std.textio.all;
use work.vc_pkg.all;
entity tb_vc_vhdl is
generic (DIRECTED_ONLY : boolean := false);
end entity;
architecture sim of tb_vc_vhdl is
constant D_NONE : natural := 0;
constant D_UNSOL : natural := 1;
constant D_SILENT : natural := 2;
constant D_STALLSETUP : natural := 3;
constant D_TOGGLE : natural := 4;
-- A device that transmits unsolicited with NOTHING to send. The only
-- defect that distinguishes "spoke when it should not have" from the
-- narrower "spoke when it had data and should not have" -- and the
-- narrow version is a plausible optimisation a reviewer would wave
-- through. Without this shape it scored a clean 0.
constant D_UNSOL_NODATA : natural := 5;
signal clk : std_logic := '0';
signal rst_n : std_logic := '0';
signal tok_valid : std_logic := '0';
signal tok_pid : std_logic_vector(1 downto 0) := TOK_OUT;
signal tok_for_us : std_logic := '0';
signal dev_tx : std_logic := '0';
signal dev_pid : std_logic_vector(2 downto 0) := RSP_NONE;
signal dev_toggle : std_logic := '0';
signal halted : std_logic := '0';
signal data_avail : std_logic := '0';
signal v_unsolicited, v_silent, v_stall_setup, v_bad_toggle : std_logic;
signal n_violations : std_logic_vector(31 downto 0);
signal o_addressed, o_idle_with_data, o_foreign_token,
o_setup_halted, o_toggle_pair : std_logic_vector(31 downto 0);
signal verdict_valid, verdict_pass : std_logic;
signal done : boolean := false;
begin
dut : entity work.usb_protocol_checker
generic map (TURNAROUND => 16)
port map (
clk => clk, rst_n => rst_n,
tok_valid => tok_valid, tok_pid => tok_pid, tok_for_us => tok_for_us,
dev_tx => dev_tx, dev_pid => dev_pid, dev_toggle => dev_toggle,
halted => halted, data_avail => data_avail,
v_unsolicited => v_unsolicited, v_silent => v_silent,
v_stall_setup => v_stall_setup, v_bad_toggle => v_bad_toggle,
n_violations => n_violations,
o_addressed => o_addressed, o_idle_with_data => o_idle_with_data,
o_foreign_token => o_foreign_token, o_setup_halted => o_setup_halted,
o_toggle_pair => o_toggle_pair,
verdict_valid => verdict_valid, verdict_pass => verdict_pass);
clk <= not clk after 5 ns when not done else '0';
stim : process
variable errors : natural := 0;
variable checks : natural := 0;
variable steps : natural := 0;
variable s_asked : std_logic := '0';
variable s_pid : std_logic_vector(1 downto 0) := TOK_OUT;
variable s_last_tog : std_logic := '0';
variable s_have_tog : std_logic := '0';
variable x_viol, x_ad, x_idle, x_foreign, x_setuph, x_togp : natural := 0;
variable n_false_pos, n_missed : natural := 0;
variable reach : std_logic_vector(0 to 191) := (others => '0');
variable n_reach : natural := 0;
variable rnd : unsigned(31 downto 0) := x"0009D1B3";
variable ln : line;
procedure ck(cond : boolean; what : string) is
begin
checks := checks + 1;
if not cond then
errors := errors + 1;
if errors <= 20 then
write(ln, string'(" ERROR step=") & integer'image(steps)
& string'(": ") & what);
writeline(output, ln);
end if;
end if;
end procedure;
impure function nxt return natural is
begin
rnd := rnd xor (rnd sll 13);
rnd := rnd xor (rnd srl 17);
rnd := rnd xor (rnd sll 5);
return to_integer(rnd(14 downto 0));
end function;
function sl_of(b : boolean) return std_logic is
begin
if b then return '1'; else return '0'; end if;
end function;
-- A device model with a selectable defect. The CORRECT branch is the
-- one that matters most: every cycle it runs is a cycle in which the
-- checker must stay silent.
procedure drive(tv : std_logic; tp : std_logic_vector(1 downto 0);
fu, hl, da : std_logic; defect : natural) is
variable d_tx : std_logic;
variable d_pid : std_logic_vector(2 downto 0);
variable d_tog : std_logic;
variable asked : std_logic;
begin
tok_valid <= tv; tok_pid <= tp; tok_for_us <= fu;
halted <= hl; data_avail <= da;
asked := s_asked;
d_tx := '0';
d_pid := RSP_NONE;
d_tog := s_last_tog;
if asked = '1' then
d_tx := '1';
if s_pid = TOK_IN then
if hl = '1' then
d_pid := RSP_STALL;
elsif da = '1' then
d_pid := RSP_DATA;
-- a correct device FLIPS the toggle on every DATA packet
if s_have_tog = '1' then d_tog := not s_last_tog;
else d_tog := '0';
end if;
else
d_pid := RSP_NAK;
end if;
else
-- a SETUP is never stalled; other traffic on a halted endpoint is
if hl = '1' and s_pid /= TOK_SETUP then d_pid := RSP_STALL;
else d_pid := RSP_ACK;
end if;
end if;
end if;
-- inject the selected defect
if defect = D_UNSOL then
if asked = '0' and da = '1' then d_tx := '1'; d_pid := RSP_DATA; end if;
elsif defect = D_UNSOL_NODATA then
if asked = '0' and da = '0' then d_tx := '1'; d_pid := RSP_NAK; end if;
elsif defect = D_SILENT then
if asked = '1' then d_tx := '0'; d_pid := RSP_NONE; end if;
elsif defect = D_STALLSETUP then
if asked = '1' and s_pid = TOK_SETUP then d_pid := RSP_STALL; end if;
elsif defect = D_TOGGLE then
if d_tx = '1' and d_pid = RSP_DATA then d_tog := s_last_tog; end if;
end if;
dev_tx <= d_tx; dev_pid <= d_pid; dev_toggle <= d_tog;
-- the shadow's expectation, recomputed from scratch
if d_tx = '1' and asked = '0' then
x_viol := x_viol + 1;
elsif asked = '1' and d_tx = '0' then
x_viol := x_viol + 1;
end if;
if asked = '1' and s_pid = TOK_SETUP and d_tx = '1' and d_pid = RSP_STALL then
x_viol := x_viol + 1;
end if;
if d_tx = '1' and d_pid = RSP_DATA and s_have_tog = '1'
and d_tog = s_last_tog then
x_viol := x_viol + 1;
end if;
if tv = '1' and fu = '1' and tp /= TOK_SOF then x_ad := x_ad + 1; end if;
if tv = '0' and da = '1' then x_idle := x_idle + 1; end if;
if tv = '1' and fu = '0' and tp /= TOK_SOF then
x_foreign := x_foreign + 1;
end if;
if tv = '1' and fu = '1' and tp = TOK_SETUP and hl = '1' then
x_setuph := x_setuph + 1;
end if;
if d_tx = '1' and d_pid = RSP_DATA and s_have_tog = '1' then
x_togp := x_togp + 1;
end if;
wait until rising_edge(clk);
wait for 1 ns;
steps := steps + 1;
-- PROPERTY 1: each violation fires exactly when it should
ck((v_unsolicited = '1') = (d_tx = '1' and asked = '0'),
"v_unsolicited disagrees");
ck((v_silent = '1') = (asked = '1' and d_tx = '0'),
"v_silent disagrees");
ck((v_stall_setup = '1') = (asked = '1' and s_pid = TOK_SETUP
and d_tx = '1' and d_pid = RSP_STALL),
"v_stall_setup disagrees");
ck((v_bad_toggle = '1') = (d_tx = '1' and d_pid = RSP_DATA
and s_have_tog = '1' and d_tog = s_last_tog),
"v_bad_toggle disagrees");
-- PROPERTY 2: NO FALSE POSITIVES on a correct device. The most
-- important check in the file: a checker that fires on correct
-- behaviour is a checker somebody deletes.
if defect = D_NONE then
if v_unsolicited = '1' or v_silent = '1'
or v_stall_setup = '1' or v_bad_toggle = '1' then
n_false_pos := n_false_pos + 1;
end if;
ck(v_unsolicited = '0', "false positive: unsolicited, on a correct device");
ck(v_silent = '0', "false positive: silent, on a correct device");
ck(v_stall_setup = '0', "false positive: stalled SETUP, on a correct device");
ck(v_bad_toggle = '0', "false positive: bad toggle, on a correct device");
end if;
ck(n_false_pos = 0, "the checker fired on a correct device");
-- PROPERTY 3: the counters agree
ck(to_integer(unsigned(n_violations)) = x_viol, "violation count disagrees");
ck(to_integer(unsigned(o_addressed)) = x_ad, "addressed-opportunity count disagrees");
ck(to_integer(unsigned(o_idle_with_data)) = x_idle, "idle-with-data count disagrees");
ck(to_integer(unsigned(o_foreign_token)) = x_foreign,"foreign-token count disagrees");
ck(to_integer(unsigned(o_setup_halted)) = x_setuph, "setup-while-halted count disagrees");
ck(to_integer(unsigned(o_toggle_pair)) = x_togp, "toggle-pair count disagrees");
-- PROPERTY 4: the verdict requires every opportunity class
ck((verdict_valid = '1') = (x_ad /= 0 and x_idle /= 0 and x_foreign /= 0
and x_setuph /= 0 and x_togp /= 0),
"verdict_valid disagrees with the opportunity counters");
-- PROPERTY 5: there is no pass without a valid verdict. The claim
-- this whole entity exists to make.
ck((verdict_pass = '1') = ((x_ad /= 0 and x_idle /= 0 and x_foreign /= 0
and x_setuph /= 0 and x_togp /= 0)
and x_viol = 0),
"verdict_pass disagrees");
ck(not (verdict_pass = '1' and verdict_valid = '0'),
"the checker reported PASS on an inconclusive run");
-- advance the shadow's history
s_asked := '0';
if tv = '1' and fu = '1' and tp /= TOK_SOF then s_asked := '1'; end if;
if s_asked = '1' then s_pid := tp; end if;
if d_tx = '1' and d_pid = RSP_DATA then
s_last_tog := d_tog;
s_have_tog := '1';
end if;
tok_valid <= '0'; dev_tx <= '0'; dev_pid <= RSP_NONE;
end procedure;
procedure reset_dut is
begin
rst_n <= '0';
tok_valid <= '0'; dev_tx <= '0'; halted <= '0'; data_avail <= '0';
wait until rising_edge(clk);
wait until rising_edge(clk);
rst_n <= '1';
s_asked := '0'; s_pid := TOK_OUT;
s_last_tog := '0'; s_have_tog := '0';
x_viol := 0; x_ad := 0; x_idle := 0; x_foreign := 0;
x_setuph := 0; x_togp := 0;
wait until rising_edge(clk);
wait for 1 ns;
end procedure;
variable ri : natural;
variable tpv : std_logic_vector(1 downto 0);
begin
-- PHASE 1 (DIRECTED, EXHAUSTIVE) -- every bus situation against every
-- device behaviour. 4 pids x ours x halted x data x 6 defects = 192.
for dfi in 0 to 5 loop
for pi in 0 to 3 loop
for oi in 0 to 1 loop
for hi in 0 to 1 loop
for di in 0 to 1 loop
reset_dut;
tpv := std_logic_vector(to_unsigned(pi, 2));
drive('1', tpv, sl_of(oi = 1), sl_of(hi = 1), sl_of(di = 1), dfi);
drive('0', TOK_OUT, '0', sl_of(hi = 1), sl_of(di = 1), dfi);
drive('0', TOK_OUT, '0', sl_of(hi = 1), sl_of(di = 1), dfi);
ri := dfi*32 + pi*8 + oi*4 + hi*2 + di;
reach(ri) := '1';
end loop;
end loop;
end loop;
end loop;
end loop;
-- PHASE 2 (DIRECTED) -- THE POINT OF THE CHAPTER.
--
-- A long, entirely CORRECT run, and the verdict must be INCONCLUSIVE
-- until every opportunity class has occurred. The classes are
-- introduced one at a time, in order of how likely a random test is to
-- produce them, and the verdict is checked after each.
reset_dut;
for k in 0 to 7 loop
drive('1', TOK_SOF, '1', '0', '0', D_NONE);
end loop;
ck(verdict_valid = '0', "the verdict was valid after SOFs alone");
ck(verdict_pass = '0', "the checker passed a run of nothing but SOFs");
for k in 0 to 3 loop
drive('1', TOK_IN, '1', '0', '1', D_NONE);
drive('0', TOK_OUT, '0', '0', '1', D_NONE);
end loop;
ck(o_addressed /= x"00000000", "the addressed opportunity was not counted");
ck(verdict_valid = '0', "the verdict was valid with 4 of 5 classes missing");
for k in 0 to 3 loop
drive('0', TOK_OUT, '0', '0', '1', D_NONE);
end loop;
ck(o_idle_with_data /= x"00000000", "the idle-with-data opportunity was not counted");
ck(verdict_valid = '0', "the verdict was valid with 3 of 5 classes missing");
for k in 0 to 3 loop
drive('1', TOK_IN, '0', '0', '1', D_NONE);
end loop;
ck(o_foreign_token /= x"00000000", "the foreign-token opportunity was not counted");
ck(verdict_valid = '0', "the verdict was valid with 2 of 5 classes missing");
for k in 0 to 3 loop
drive('1', TOK_IN, '1', '0', '1', D_NONE);
drive('0', TOK_OUT, '0', '0', '1', D_NONE);
end loop;
ck(o_toggle_pair /= x"00000000", "the toggle-pair opportunity was not counted");
ck(verdict_valid = '0', "the verdict was valid with 1 of 5 classes missing");
-- and finally a SETUP while halted: the rarest of the five, and the one
-- a random test is least likely ever to produce
drive('1', TOK_SETUP, '1', '1', '0', D_NONE);
drive('0', TOK_OUT, '0', '1', '0', D_NONE);
ck(o_setup_halted /= x"00000000", "the setup-while-halted opportunity was not counted");
ck(verdict_valid = '1', "the verdict was still invalid with all 5 classes seen");
ck(verdict_pass = '1', "a correct device did not pass a conclusive run");
-- PHASE 3 (DIRECTED, EXHAUSTIVE) -- each defect must make the verdict
-- fail while leaving it VALID. A defect must not be allowed to make the
-- run inconclusive: that would be a checker hiding failures by
-- disqualifying its own evidence.
for dfi in 1 to 5 loop
reset_dut;
for k in 0 to 5 loop
drive('1', TOK_IN, '1', '0', '1', D_NONE);
drive('0', TOK_OUT, '0', '0', '1', D_NONE);
end loop;
for k in 0 to 3 loop
drive('1', TOK_IN, '0', '0', '1', D_NONE);
end loop;
drive('1', TOK_SETUP, '1', '1', '0', D_NONE);
drive('0', TOK_OUT, '0', '1', '0', D_NONE);
ck(verdict_valid = '1', "the opportunity build-up did not conclude");
ck(verdict_pass = '1', "a correct build-up did not pass");
for k in 0 to 5 loop
drive('1', TOK_IN, '1', '0', '1', dfi);
drive('0', TOK_OUT, '0', '0', '1', dfi);
end loop;
if dfi = D_STALLSETUP then
drive('1', TOK_SETUP, '1', '1', '0', dfi);
drive('0', TOK_OUT, '0', '1', '0', dfi);
end if;
if dfi = D_UNSOL then
for k in 0 to 3 loop
drive('0', TOK_OUT, '0', '0', '1', dfi);
end loop;
end if;
if dfi = D_UNSOL_NODATA then
for k in 0 to 3 loop
drive('0', TOK_OUT, '0', '0', '0', dfi);
end loop;
end if;
ck(n_violations /= x"00000000", "a defective device produced no violations");
ck(verdict_valid = '1', "a defect made the run inconclusive");
ck(verdict_pass = '0', "a defective device passed");
end loop;
-- PHASE 4 (RANDOM) -- a correct device on a busy bus.
--
-- Deliberately CORRECT throughout: this phase exists to find false
-- positives, and it is the longest phase in the suite for that reason.
if not DIRECTED_ONLY then
reset_dut;
for k in 0 to 29999 loop
drive(sl_of((nxt mod 3) /= 0),
std_logic_vector(to_unsigned(nxt mod 4, 2)),
sl_of((nxt mod 4) /= 0),
sl_of((nxt mod 8) = 0),
sl_of((nxt mod 2) = 0),
D_NONE);
end loop;
ck(n_violations = x"00000000",
"the checker reported violations against a correct device over 30000 cycles");
end if;
n_reach := 0;
for i in 0 to 191 loop
if reach(i) = '1' then n_reach := n_reach + 1; end if;
end loop;
write(ln, string'("steps=") & integer'image(steps)
& string'(" checks=") & integer'image(checks)
& string'(" reach=") & integer'image(n_reach) & string'("/192")
& string'(" errors=") & integer'image(errors));
writeline(output, ln);
write(ln, string'("[opportunities] addressed=") & integer'image(x_ad)
& string'(" idle_with_data=") & integer'image(x_idle)
& string'(" foreign=") & integer'image(x_foreign)
& string'(" setup_halted=") & integer'image(x_setuph)
& string'(" toggle_pairs=") & integer'image(x_togp));
writeline(output, ln);
write(ln, string'("[the whole point] false positives = ")
& integer'image(n_false_pos) & string'(", missed defects = ")
& integer'image(n_missed));
writeline(output, ln);
if n_reach /= 192 then
write(ln, string'("FAIL: exhaustive sweep incomplete"));
writeline(output, ln);
errors := errors + 1;
end if;
if errors = 0 then
write(ln, string'("PASS: 0 errors in ") & integer'image(checks)
& string'(" checks"));
else
write(ln, string'("FAIL: ") & integer'image(errors)
& string'(" errors in ") & integer'image(checks) & string'(" checks"));
end if;
writeline(output, ln);
done <= true;
wait;
end process;
end architecture;9. Exhaustive Verification
| Measure | Verilog | SystemVerilog | VHDL |
|---|---|---|---|
| (PID × ours × halted × data × defect) reached | 192 / 192 | 192 / 192 | 192 / 192 |
| …reached by directed stimulus alone | 192 / 192 | 192 / 192 | 192 / 192 |
| defect shapes injected | 5 | 5 | 5 |
| Steps | 30770 | 30770 | 30770 |
| Checks executed | 551699 | 551699 | 551699 |
o_addressed | 11229 | 11229 | 11230 |
o_idle_with_data | 5013 | 5013 | 4923 |
o_foreign_token | 3785 | 3785 | 3674 |
o_setup_halted | 423 | 423 | 474 |
o_toggle_pair | 1651 | 1651 | 1612 |
| false positives | 0 | 0 | 0 |
| missed defects | 0 | 0 | 0 |
| Result | PASS | PASS | PASS |
423 is the smallest of the five opportunity counters and it is the one that matters most. A SETUP arriving while an endpoint is halted is the rarest situation a USB device sees, and it is the one whose rule — a SETUP may never be stalled — has the worst consequence if broken: the device locks itself out permanently, because clearing a halt is itself a control transfer.
10. Mutation Testing
These mutations are different in kind from every other chapter's, because the thing being mutated is a checker. Three of them make it miss something; four make it lie about the verdict — and the second kind is more dangerous, because a checker that reports a pass it cannot justify is worse than no checker at all.
| # | Mutation | Verilog | SysVer | VHDL |
|---|---|---|---|---|
| H7 | v_stall_setup fires on any stall — a false positive | 92587 | 92587 | 92577 |
| H5 | the toggle history is never marked valid, so the check never fires | 90271 | 90271 | 90333 |
| H6 | an opportunity is counted when it did not arise | 30570 | 30570 | 30448 |
| H1 | the verdict no longer requires the opportunities | 1406 | 1406 | 1324 |
| H2 | pass no longer requires a valid verdict | 1273 | 1273 | 1191 |
| H4 | the unsolicited check is narrowed to require pending data | 104 | 104 | 104 |
| H3 | the silence check is removed | 66 | 66 | 66 |
| — | unmutated baseline | 0 | 0 | 0 |
All seven die in all three languages.
H6 is the subtlest of the seven. It counts the SETUP-while-halted opportunity on any addressed token, so the verdict becomes valid on a run that never exercised the class. Every violation check still works perfectly. The module still finds every defect it is given. Its verdict is simply no longer trustworthy — and nothing about its output looks wrong.
Directed against random
| # | V all | V directed | V random | VHDL all | VHDL directed | VHDL random |
|---|---|---|---|---|---|---|
| H1 | 1406 | 1254 | 152 | 1324 | 1254 | 70 |
| H2 | 1273 | 1121 | 152 | 1191 | 1121 | 70 |
| H3 | 66 | 66 | 0 | 66 | 66 | 0 |
| H4 | 104 | 104 | 0 | 104 | 104 | 0 |
| H5 | 90271 | 430 | 89841 | 90333 | 430 | 89903 |
| H6 | 30570 | 432 | 30138 | 30448 | 432 | 30016 |
| H7 | 92587 | 815 | 91772 | 92577 | 815 | 91762 |
| — | BASE 0 | 0 | 0 | 0 | 0 | 0 |
Every directed column identical, and the directed-only baseline reaches 192/192 with 0 errors.
H3 and H4 have a random contribution of exactly zero, which is the expected shape: both are defect-injection checks, and the random phase deliberately injects no defect. They exist only because somebody wrote the phases for them.
11. The Bench Had a Self-Referential Check
The most instructive finding in this chapter is a bug in the bench, and it is one that a verification engineer will meet again.
The Verilog bench originally checked verdict_pass like this:
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
ck(verdict_pass === (verdict_valid && (x_viol == 0)));
^^^^^^^^^^^^^
the DUT's OWN OUTPUT
Under mutation H1, verdict_valid is forced high. The
EXPECTATION moves with it. The check passes vacuously.The VHDL bench happened to recompute the expectation from the shadow's own counters, so it caught what the Verilog missed — and the symptom was a directed column reading 694 against 1254 where identical stimulus and identical logic must produce the same number.
Removing the DUT output from the expectation raised three mutation scores at once:
| # | Before | After |
|---|---|---|
| H1 | 770 | 1406 |
| H2 | 1273 | 1273 |
| H5 | 60308 | 90271 |
12. Now the Component List — With the Reason for Each
Having led with the verdict, the standard architecture is worth walking, because each piece now has a justification rather than a name.
| Component | Why it exists | The mistake to avoid |
|---|---|---|
| Interface + clocking block | one place that defines sampling and driving edges | sampling on the driving edge, which works in simulation and not in silicon |
| Driver | turns transactions into pin wiggles | putting protocol rules here; the driver must be able to drive illegal traffic |
| Monitor | reconstructs transactions from pins, passively | reusing driver code, which makes the monitor blind to what the driver cannot produce |
| Sequencer + sequences | composes stimulus; holds the constraints | constraints that only produce legal traffic, so no rejection path is ever tested |
| Reference model | predicts what the device should do | mirroring the RTL's structure, so both make the same mistake |
| Scoreboard | compares prediction with observation | comparing only data, which misses every timing and ordering rule |
| Protocol checker | asserts the rules, transaction-independently | rules that can never fire — which is what this chapter is about |
| Coverage collector | records what was reached | functional coverage that counts stimulus rather than situations |
| Opportunity audit | fails the run if a rule was never at risk | not having one, which is the default |
13. Follow-Ups the Interviewer Will Ask
"How do you know your coverage model is right?" You do not, directly — which is why the opportunity counters are separate from the coverage model and are checked as errors. A coverage hole is a warning; a rule that was never at risk is a failed run.
"What is the difference between code coverage and functional coverage here?" Code coverage tells you the line executed. o_setup_halted tells you the situation occurred. A single line of checker code can execute a million times without the situation it guards ever arising.
"How would you test the checker itself?" Inject defects and require each to fire; then run a long, entirely correct stimulus and require silence. The second half is the one that keeps it enabled — and mutation testing of the checker, as in section 10, is the systematic version.
"Where do you put the protocol rules — driver, monitor or checker?" The checker. A driver that enforces the rules cannot generate the illegal traffic that tests them, and a monitor that enforces them will reject the very transactions you need it to report.
"What do you do about a check that keeps false-positiving?" Fix it or delete it. A disabled check is worse than a missing one, because it is still in the coverage report. Both of chapter 27.3's self-check attempts produced thousands of false alarms before the third got it right.
"How much of this is reusable across protocols?" The opportunity-audit pattern is entirely protocol-independent. The rules are not. That division is the right one to draw when planning a verification IP.
"What would you do first on a new device controller?" The monitor and the protocol checker, before any stimulus at all — because they are what tells you whether the stimulus you write next is doing anything.
14. UVM: The Environment, With the Audit Wired In
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// The standard component list, assembled -- with the one addition that
// makes the other eight into evidence rather than activity.
//
// Note what is NOT in the driver: any protocol rule at all. A driver that
// enforces the rules cannot generate the illegal traffic that tests them.
class usb_dev_env extends uvm_env;
`uvm_component_utils(usb_dev_env)
usb_agent agent; // driver + monitor + sequencer
usb_ref_model model; // predicts; does NOT mirror the RTL
usb_scoreboard scb; // compares prediction with observation
usb_protocol_chk chk; // asserts the rules
usb_coverage cov; // records what was reached
usb_opportunity_audit audit; // FAILS THE RUN if a rule was never at risk
function new(string name, uvm_component parent); super.new(name, parent);
endfunction
function void build_phase(uvm_phase phase);
super.build_phase(phase);
agent = usb_agent::type_id::create("agent", this);
model = usb_ref_model::type_id::create("model", this);
scb = usb_scoreboard::type_id::create("scb", this);
chk = usb_protocol_chk::type_id::create("chk", this);
cov = usb_coverage::type_id::create("cov", this);
audit = usb_opportunity_audit::type_id::create("audit", this);
endfunction
function void connect_phase(uvm_phase phase);
super.connect_phase(phase);
// The monitor feeds everything. It is passive and it is the ONLY
// source of observed behaviour -- the driver's intent is never used as
// evidence of what happened.
agent.mon.ap.connect(model.analysis_export);
agent.mon.ap.connect(scb.obs_export);
agent.mon.ap.connect(chk.analysis_export);
agent.mon.ap.connect(cov.analysis_export);
agent.mon.ap.connect(audit.analysis_export);
model.pred_ap.connect(scb.pred_export);
endfunction
endclass
// =====================================================================
// THE COMPONENT THAT IS USUALLY MISSING.
//
// It contains no rules and predicts nothing. Its entire job is to
// answer one question at the end of the run: was this run CAPABLE of
// failing?
// =====================================================================
class usb_opportunity_audit extends uvm_subscriber #(usb_txn);
`uvm_component_utils(usb_opportunity_audit)
// One counter per rule the environment asserts. The names deliberately
// match the rules, so a zero points straight at the check it invalidates.
int unsigned o_addressed; // -> the "must answer" rule
int unsigned o_idle_with_data; // -> the "must not initiate" rule
int unsigned o_foreign_token; // -> the address-decode rule
int unsigned o_setup_halted; // -> the "SETUP is never stalled" rule
int unsigned o_toggle_pair; // -> the data-toggle rule
int unsigned o_nak; // -> the "NAK is not terminal" rule
int unsigned o_stall_cleared; // -> the halt-recovery rule
function new(string name, uvm_component parent); super.new(name, parent);
endfunction
function void write(usb_txn t);
if (t.addressed_us && t.pid != PID_SOF) o_addressed++;
if (!t.token_present && t.data_pending) o_idle_with_data++;
if (t.token_present && !t.addressed_us
&& t.pid != PID_SOF) o_foreign_token++;
if (t.addressed_us && t.pid == PID_SETUP
&& t.endpoint_halted) o_setup_halted++;
if (t.response == RSP_DATA && t.had_prior_data) o_toggle_pair++;
if (t.response == RSP_NAK) o_nak++;
if (t.clear_halt_completed) o_stall_cleared++;
endfunction
// ---- and this is the part that is an ERROR, not a warning ----
//
// A coverage hole is a warning: you meant to reach something and did
// not. A rule that was never AT RISK is different in kind -- it means an
// assertion in this environment has never been in a position to fire,
// and reporting a pass on that basis is not a weak result but a wrong
// one.
function void report_phase(uvm_phase phase);
super.report_phase(phase);
audit_one(o_addressed, "the device was never addressed",
"the \"a device must answer when addressed\" rule");
audit_one(o_idle_with_data, "the device never had data pending while idle",
"the \"a device may never initiate\" rule");
audit_one(o_foreign_token, "every token was addressed to the DUT",
"the address-decode rule");
audit_one(o_setup_halted, "no SETUP ever arrived while an endpoint was halted",
"the \"a SETUP is never stalled\" rule");
audit_one(o_toggle_pair, "fewer than two DATA packets were ever seen",
"the data-toggle rule");
audit_one(o_nak, "the device never NAKed",
"the \"NAK is not terminal\" rule");
audit_one(o_stall_cleared, "no halt was ever cleared",
"the halt-recovery rule");
`uvm_info("AUDIT",
$sformatf("opportunities: addressed=%0d idle=%0d foreign=%0d setup_halted=%0d toggle=%0d nak=%0d cleared=%0d",
o_addressed, o_idle_with_data, o_foreign_token,
o_setup_halted, o_toggle_pair, o_nak, o_stall_cleared),
UVM_LOW)
endfunction
// The message names the RULE, not the counter. "o_setup_halted is zero"
// means nothing to whoever reads the report at 2am; "the SETUP-is-never-
// stalled rule was never tested" tells them what to go and write.
function void audit_one(int unsigned n, string situation, string rule);
if (n == 0)
`uvm_error("AUDIT/INCONCLUSIVE",
$sformatf("%s, so %s was never exercised: this run cannot be reported as a pass",
situation, rule))
endfunction
endclass15. Common Misconceptions
"0 errors means it works." It means the run found nothing. Whether it could have is a separate question, and the one that decides what the number means.
"Coverage closure means verification is done." Coverage says the stimulus reached a state. It does not say a checker was watching.
"More random cycles is more confidence." Not for rules whose situations well-behaved stimulus avoids. Five of the mutations in this module have a random contribution of exactly zero.
"Put the protocol rules in the driver." Then the driver cannot generate the illegal traffic that tests them.
"The reference model should mirror the RTL." Then it makes the same mistakes. Every shadow in this module is deliberately a different formulation.
"A noisy assert can be disabled for now." A disabled check is worse than a missing one, because it is still in the report.
"Checking is cheaper than stimulus." A defect in the checking is orders of magnitude harder to find: H1 and H2 score 1406 and 1273 where a design defect scores 92,587.
"An expected value can use a DUT output if it is convenient." Then the check is a tautology. It cost three mutation scores here before it was found.
"The verdict is: no violations." The verdict is: no violations and the run was capable of producing them.
16. Exercises
1. Give the answer to the interview question in three sentences, leading with the verdict rather than the component list.
2. H1 and H2 score 1406 and 1273 while H7 scores 92,587. Explain the ratio, and say what it implies about how much effort belongs in verifying the verification.
3. The bench's verdict_pass check originally contained a DUT output. Write a procedure for auditing an existing environment for checks of that shape.
4. Five mutations across this module have a random contribution of exactly zero. List them, find what their situations have in common, and generalise.
5. H6 counts an opportunity that did not arise. Argue why this is more dangerous than H3 (removing a check outright), and design a check that would catch it.
6. Add two more rules to the checker — "NAK is not terminal" and "a cleared halt resets the toggle" — with the opportunity counters each needs.
7. The opportunity-audit pattern is protocol-independent. Write it as reusable verification IP, and say what the user must supply.
17. Summary
| Idea | Why it matters |
|---|---|
| A checker that fails on weak stimulus | the answer that gets hired |
| "0 errors" is not a result | it is one only if errors were possible |
| Count opportunities beside violations | a zero in both is a failed run |
| No output says pass without saying valid | one source of truth for the verdict |
v_silent is the half people omit | silence is a failure, not caution |
| The rarest opportunity guards the worst rule | a SETUP while halted, 423 in 30770 |
| Verify a checker twice: true and false positives | the false-positive half keeps it enabled |
| The longest phase injects no defect | it exists to prove the checker is quiet |
| A defect in the checking hides best | 1406 against 92587 |
| Rules go in the checker, not the driver | a driver with rules cannot test them |
| The model must not mirror the RTL | or it repeats its mistakes |
| An expected value must contain no DUT output | otherwise the check is a tautology |
| An accumulator written from two places | must be combinational and added once |
| A coverage hole is a warning; an untested rule is an error | different in kind |
| Name the rule in the message, not the counter | so the report says what to write next |
| 192 states, 5 defect shapes, 7 mutations | 0 false positives in 551,699 checks |
Tooling
| Step | Command |
|---|---|
| Verilog-2005 | iverilog -g2005 -o vc_v.out vc_v.v vc_v_tb.v && ./vc_v.out |
| SystemVerilog | iverilog -g2012 -o vc_sv.out vc_sv.sv vc_sv_tb.sv && ./vc_sv.out |
| VHDL-2008 analyse | nvc --std=2008 -a vc_vhdl.vhd vc_vhdl_tb.vhd |
| VHDL-2008 elaborate | nvc --std=2008 -e tb_vc_vhdl |
| VHDL-2008 run | nvc --std=2008 -r tb_vc_vhdl |
| One mutation | iverilog -g2005 -DMUT_H1 -o mm vc_v_mut.v vc_v_tb.v && ./mm |
| Directed only (Verilog) | iverilog -g2005 -DDIRECTED_ONLY -o mm vc_v_mut.v vc_v_tb.v && ./mm |
| Directed only (VHDL) | nvc --std=2008 -e -gDIRECTED_ONLY=true tb_vc_vhdl |
All three implementations pass with 0 errors: all 192 combinations of token type, addressing, halt state, data availability and injected defect — reached by directed stimulus alone; five distinct defect shapes each required to fire exactly the matching output; a 30,000-cycle correct-device phase whose only assertion is silence; zero false positives and zero missed defects in 551,699 checks; and every one of the seven mutations killed by directed stimulus alone, with all seven directed scores identical across languages.
Chapter 27.9 — Senior Silicon Debug is the third senior question and the one that cannot be prepared by reading: walk a real bring-up debug session from an analyser trace. Its central discipline is the one this chapter's audit is built on, applied to a live board — narrow by what the evidence excludes, not by what it suggests.
Continue learning
Related tutorials
- Related topic
USB Protocol Checkers
Every ordering rule says what must happen next, and none of them fires when nothing happens at all — the timeout is a checker's only liveness tool, and a checker with false positives gets switched off.
- Related topic
What Is USB?
The opening interview question answered with one load-bearing idea instead of a list — USB is host-scheduled, and polling, NAK, the frame and the missing interrupt line are all consequences of it.
- Related topic
Host / Device / Hub Identification
Host and device take fifteen seconds; the hub is where the interview is decided — a hub is a repeater, not a switch, and no downstream port can ever reach another one.
- Related topic
The Enumeration Question
Attach to configured, with the one step almost everybody gets backwards — SET_ADDRESS takes effect after the status stage, and a device that switches early is invisible to the host.
Standards & specifications
- Governing standard
- USB-IF (Universal Serial Bus Specification)(opens USB Implementers Forum (USB-IF) in a new tab)
Defines the USB bus — its electrical signalling, connectors, packet and transaction model, device framework and the descriptors a device must expose — together with the device-class specifications layered on it. It does not define host-controller register interfaces (xHCI and EHCI are separate documents) nor any operating system's driver architecture.
This page also covers RTL structure, verification approach and debugging technique. Those are engineering practice built on the standard, not requirements the standard itself imposes.
Where this fits
Part of the USB curriculum.