SPI · Module 17
Constrained-Random Sequences
A constraint set is a specification in a solver's language, and it fails in two directions a coverage report cannot tell apart. A weighted set closes at draw 42 where a uniform one needs 2120; an over-constrained set stalls silently; and an inconsistent one must report rather than emit.
Chapter 17.3 built the coverage model. This chapter builds what reaches it — and measures the two ways a constraint set goes wrong that a coverage report is blind to.
Two constraint sets, both legal, both eventually closing the same bins. One needs fifty times as many transactions. Nothing in the coverage report says why.
1. A Constraint Set Is A Specification, And It Can Be Wrong Twice
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
OVER-CONSTRAINED the set forbids something the plan requires. Every
transaction is legal, the generator reports success on
every one of them, and a coverage bin is never reached.
The symptom is indistinguishable from "we need a longer
run", and teams spend weeks on the wrong remedy.
INCONSISTENT the set has no solution at all in some corner. A real
solver reports this. A hand-rolled generator usually does
something far worse: it gives up quietly and emits its
last draw -- an item that violates the specification,
produced by the component whose job is to enforce it.2. Four Sets
| Set | What it is |
|---|---|
CS_LOOSE | legal, and uniform over the datasheet's ranges rather than over its interesting values |
CS_SPEC | the device's actual specification: widths weighted to the ones that matter, timing biased onto the minimum, and the master's mode derived from the slave's |
CS_OVER | CS_SPEC plus one clause that looks harmless and is in no datasheet |
CS_BAD | a set with no solution in a corner: in mode 2 the width must be 32 and must be 1 |
CS_LOOSE is the one to sit with, because it is what a generator usually is. It draws the frame width as a number in 1..32 and the gap as a cycle count in 0..255 and maps each to a class afterwards — because the ranges are what the datasheet lists. The consequence is that the extremes arrive at their share of the range: one width in sixteen, one gap in 256.
3. What Legal Traffic Looks Like Under Each Set
Twelve draws, two distributions
12 cyclesNeither row is illegal. The difference is where the mass is, and a >= MIN requirement is only distinguished from a buggy > MIN by the value MIN itself.
4. The Measurement
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
constraint set curated bins closed at draw solver failures
loose 29 2120 0
spec 29 42 0
over 28 0 0Both legal sets close. One needs fifty times as many draws.
The loose set was given more than six times the budget so the comparison would be about the rate rather than about whether the run was long enough — and it closed at draw 2,120 against the weighted set's 42. Same axes, same bins, same generator, same coverage model from Chapter 17.3. The entire difference is the distribution: a gap minimum drawn uniformly from 0..255 arrives once in 256 draws, and it then has to be crossed with four modes before its bin closes.
Nobody chose that rate. It is what writing a legal range instead of a weighted clause chooses for you.
5. Over-Constraining Is Silent
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
over 28 0 0CS_OVER adds one clause — never a 32-bit frame in mode 0 — that appears in no datasheet. The result: 28 of 29 bins in 1,200 draws, and zero reported failures.
Every transaction was legal. The generator succeeded on every draw. The coverage report says only that a bin is missing, which is indistinguishable from needing a longer run — and no number of draws will change it.
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
the diagnosis has to come from reading the CONSTRAINT SET, not the coverage
reportThe cheap first move remains the one from Chapter 16.2: change the seed. A hole that moves is a distribution problem. A hole that does not move is structural, and then there are exactly two candidates — a field that cannot reach the bin, or a clause that forbids it.
6. An Inconsistent Set Must Fail Loudly
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
the inconsistent set, 400 draws:
items emitted ..................... 305
solver failures reported .......... 95
redraws spent ..................... 3063
emitted items violating the spec .. 0Both halves are the measurement. 95 reported failures, and zero emitted items that violate the specification.
A generator that gave up and emitted its last candidate would also report failures — and it would be the component whose job is to enforce the specification producing the violation.
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
unbounded rejection sampling hangs
bounded and silent lies
bounded and reported tells the truthThe difference between the last two is one output port.
7. Sequences: Order And Reproducibility
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
the sequence: 6 items requested, 6 delivered; replay mismatches 0 of 6The same seed produced the identical six-item stream twice. A random sequence that cannot be replayed is a failure nobody can debug — and reproducibility is also what lets the three language versions of this bench be three measurements of one experiment rather than three unrelated numbers.
8. Building It — Three HDLs
Azvya Education Pvt. Ltd.VLSI Mentor
spi_seq_gen.sv — four constraint sets, with a bounded rejection loop and a reported failure
// spi_seq_gen.sv
//
// Chapter 17.4 -- constraints that encode a device specification, and the two ways a constraint
// set fails that a regression cannot tell apart from bad luck.
//
// A CONSTRAINT SET IS A SPECIFICATION WRITTEN IN A SOLVER'S LANGUAGE, and like any specification
// it can be wrong in two directions:
//
// OVER-CONSTRAINED the set forbids something the plan requires. Every transaction is legal,
// the generator reports success on every one of them, and a coverage bin
// is never reached. The symptom is indistinguishable from "we need a
// longer run", and teams spend weeks on the wrong remedy.
//
// INCONSISTENT the set has no solution at all in some corner. A real solver reports
// this; a hand-rolled generator usually does something far worse, which is
// to give up quietly and emit its last draw -- an item that violates the
// specification, produced by the component whose job is to enforce it.
//
// This generator implements four constraint sets so that both failures can be measured against
// a working one:
//
// CS_LOOSE LEGAL, and uniform over the device's RANGES rather than over its interesting
// values. The frame width is drawn as a number and mapped to a class, the gap as a
// cycle count and mapped to a class -- which is how a real generator is usually
// written, because the ranges are what the datasheet lists. The consequence is that
// the extremes are as rare as their share of the range: one width in sixteen, one
// gap in two hundred and fifty-six.
//
// CS_SPEC the device's actual specification: widths weighted to the ones that matter,
// timing deliberately biased onto the minimum, and the master's mode DERIVED from
// the slave's rather than drawn independently (Chapter 16.2's result: two
// independent bits disagree three times in four, which makes the common case rare).
//
// CS_OVER CS_SPEC with one extra clause that looks harmless -- "never a 32-bit frame in
// mode 0" -- added by somebody who misread a table. Nothing fails. A bin goes
// unreached forever.
//
// CS_BAD a set with no solution in a corner: "width must be 32" and "width must be 1"
// both apply when the mode is 2. The generator must REPORT this, not resolve it.
//
// THE ATTEMPT BOUND IS THE WHOLE OF THE FOURTH SET'S LESSON. Rejection sampling without a bound
// hangs; with a bound and no report, it emits an illegal item; with a bound and a report, it
// tells you the truth. The third of those is the only acceptable behaviour and it is the one
// that costs an extra output port.
`timescale 1ns/1ps
module spi_seq_gen #(
parameter int DW = 32,
parameter int LEN_W = 6,
parameter int CNT_W = 16,
parameter int TRIES = 32 // the rejection-sampling bound
) (
input wire clk,
input wire rst_n,
input wire [1:0] cset, // 0 = LOOSE, 1 = SPEC, 2 = OVER, 3 = BAD
input wire [31:0] seed,
input wire reseed,
input wire req, // draw one item
output reg item_valid,
output reg [1:0] mode, // {cpol, cpha} -- the SLAVE's mode
output reg [1:0] m_mode, // the MASTER's, derived unless a mismatch is asked for
output reg [1:0] width_c, // 0 = 1 bit, 1 = 8 bits, 2 = 32 bits
output reg order_c,
output reg [1:0] gap_c, // 0 = min, 1 = mid, 2 = long
output reg [DW-1:0] data,
// THE PORT THAT MAKES AN INCONSISTENT SET REPORTABLE. Without it the only honest options are
// to hang or to lie.
output reg solve_fail,
output reg [CNT_W-1:0] n_items,
output reg [CNT_W-1:0] n_fail,
// How many draws the solver needed. A set that is merely tight shows up here long before it
// shows up as a missing bin.
output reg [CNT_W-1:0] n_redraws
);
localparam [1:0] CS_LOOSE = 2'd0,
CS_SPEC = 2'd1,
CS_OVER = 2'd2,
CS_BAD = 2'd3;
reg [31:0] rng;
function automatic [31:0] nxt(input [31:0] s);
reg [31:0] x;
begin
x = s;
x = x ^ (x << 13);
x = x ^ (x >> 17);
x = x ^ (x << 5);
nxt = x;
end
endfunction
// The width draw. CS_LOOSE is uniform over the three classes; every other set weights them,
// because the interesting widths are the extremes and the byte and a uniform draw over a
// range spends most of its time in the middle.
function automatic [1:0] draw_width(input [31:0] r, input [1:0] cs);
reg [3:0] q;
begin
q = r[19:16];
if (cs == CS_LOOSE) begin
// UNIFORM OVER THE RANGE 1..32, mapped to the class afterwards. One draw in
// sixteen is the narrowest frame and one in sixteen is the widest, because that
// is their share of the range -- which is a rate nobody chose.
if (q == 4'd0) draw_width = 2'd0;
else if (q == 4'd15) draw_width = 2'd2;
else draw_width = 2'd1;
end
else if (q < 4'd4) draw_width = 2'd0; // 1 bit, deliberately often
else if (q < 4'd12) draw_width = 2'd1; // 8 bits
else draw_width = 2'd2; // 32 bits
end
endfunction
// The gap draw. CS_LOOSE is uniform; the others put deliberate weight on the MINIMUM,
// because `>= MIN` is only distinguished from `> MIN` by the value MIN itself.
function automatic [1:0] draw_gap(input [31:0] r, input [1:0] cs);
reg [3:0] q;
reg [7:0] wide;
begin
q = r[27:24];
wide = r[27:20];
if (cs == CS_LOOSE) begin
// UNIFORM OVER A GAP OF 0..255 CYCLES. The minimum is one value out of the range,
// so it arrives once in 256 draws -- which is the rate at which a `>= MIN`
// requirement is actually tested by a uniform generator.
if (wide == 8'd0) draw_gap = 2'd0;
else if (wide < 8'd128) draw_gap = 2'd1;
else draw_gap = 2'd2;
end
else if (q < 4'd6) draw_gap = 2'd0; // at the minimum, deliberately often
else if (q < 4'd12) draw_gap = 2'd1;
else draw_gap = 2'd2;
end
endfunction
// Does a candidate satisfy the selected set? Everything the sets differ by lives here, in
// one function, so that "the constraint" is a single readable object rather than a
// behaviour spread across a draw.
function automatic integer satisfies(input [1:0] cs, input [1:0] m, input [1:0] w);
begin
satisfies = 1;
// THE DEVICE SPECIFICATION, which EVERY set honours -- including the loose one,
// because the point of the loose set is bad distributions, not illegal traffic: no
// 32-bit frame in mode 3.
if (m == 2'd3 && w == 2'd2) satisfies = 0;
// CS_OVER's extra clause. It looks like the line above and it is not in any
// datasheet.
if (cs == CS_OVER && m == 2'd0 && w == 2'd2) satisfies = 0;
// CS_BAD: in mode 2 the width must be 32 AND must be 1. No value satisfies both.
if (cs == CS_BAD && m == 2'd2 && w != 2'd2) satisfies = 0;
if (cs == CS_BAD && m == 2'd2 && w != 2'd0) satisfies = 0;
end
endfunction
integer tries;
reg [1:0] cand_m, cand_w;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
rng <= 32'h1;
item_valid <= 1'b0;
solve_fail <= 1'b0;
mode <= 2'd0;
m_mode <= 2'd0;
width_c <= 2'd0;
order_c <= 1'b0;
gap_c <= 2'd0;
data <= {DW{1'b0}};
n_items <= {CNT_W{1'b0}};
n_fail <= {CNT_W{1'b0}};
n_redraws <= {CNT_W{1'b0}};
end else begin
item_valid <= 1'b0;
solve_fail <= 1'b0;
if (reseed) begin
// A ZERO SEED IS REFUSED, because zero is a fixed point of xorshift and a
// generator emitting a constant looks exactly like a suite that ran.
rng <= (seed == 32'd0) ? 32'h1234_5678 : seed;
end else if (req) begin
rng = nxt(rng);
cand_m = rng[1:0];
cand_w = draw_width(rng, cset);
tries = 0;
// REJECTION SAMPLING WITH A BOUND. Unbounded, this hangs on CS_BAD. Bounded
// without a report, it emits the last candidate -- an item that violates the
// specification, produced by the component whose job is to enforce it.
// Only the CONSTRAINED variable is redrawn. Redrawing the mode as well would make
// an inconsistent set look solvable -- and it would also stop the generator from
// sampling the distribution it was asked for, because the antecedent of the
// constraint would be chosen to suit the constraint. A solver may reorder its
// variables; a rejection sampler that redraws its antecedent is answering a
// different question.
while (satisfies(cset, cand_m, cand_w) == 0 && tries < TRIES) begin
rng = nxt(rng);
cand_w = draw_width(rng, cset);
tries = tries + 1;
end
n_redraws <= n_redraws + tries[CNT_W-1:0];
if (satisfies(cset, cand_m, cand_w) == 0) begin
// THE HONEST FAILURE. No item is emitted; the caller is told.
solve_fail <= 1'b1;
n_fail <= n_fail + 1'b1;
end else begin
mode <= cand_m;
width_c <= cand_w;
order_c <= rng[30];
gap_c <= draw_gap(rng, cset);
data <= {rng[15:0], rng[31:16]};
// THE MASTER'S MODE IS DERIVED, not drawn. Chapter 16.2 measured why: two
// independent bits disagree three times in four, so an independently drawn
// master mode makes the MISMATCH case 75% of the suite and the agreeing
// case -- the one every real transfer uses -- the corner case.
m_mode <= (rng[29:28] == 2'd0) ? (cand_m ^ 2'd1) : cand_m;
item_valid <= 1'b1;
n_items <= n_items + 1'b1;
end
end
end
end
endmoduleAzvya Education Pvt. Ltd.VLSI Mentor
spi_seq_gen.v — the same design in Verilog-2001
// spi_seq_gen.v
//
// Chapter 17.4 -- constraints that encode a device specification, and the two ways a constraint
// set fails that a regression cannot tell apart from bad luck.
//
// A CONSTRAINT SET IS A SPECIFICATION WRITTEN IN A SOLVER'S LANGUAGE, and like any specification
// it can be wrong in two directions:
//
// OVER-CONSTRAINED the set forbids something the plan requires. Every transaction is legal,
// the generator reports success on every one of them, and a coverage bin
// is never reached. The symptom is indistinguishable from "we need a
// longer run", and teams spend weeks on the wrong remedy.
//
// INCONSISTENT the set has no solution at all in some corner. A real solver reports
// this; a hand-rolled generator usually does something far worse, which is
// to give up quietly and emit its last draw -- an item that violates the
// specification, produced by the component whose job is to enforce it.
//
// This generator implements four constraint sets so that both failures can be measured against
// a working one:
//
// CS_LOOSE LEGAL, and uniform over the device's RANGES rather than over its interesting
// values. The frame width is drawn as a number and mapped to a class, the gap as a
// cycle count and mapped to a class -- which is how a real generator is usually
// written, because the ranges are what the datasheet lists. The consequence is that
// the extremes are as rare as their share of the range: one width in sixteen, one
// gap in two hundred and fifty-six.
//
// CS_SPEC the device's actual specification: widths weighted to the ones that matter,
// timing deliberately biased onto the minimum, and the master's mode DERIVED from
// the slave's rather than drawn independently (Chapter 16.2's result: two
// independent bits disagree three times in four, which makes the common case rare).
//
// CS_OVER CS_SPEC with one extra clause that looks harmless -- "never a 32-bit frame in
// mode 0" -- added by somebody who misread a table. Nothing fails. A bin goes
// unreached forever.
//
// CS_BAD a set with no solution in a corner: "width must be 32" and "width must be 1"
// both apply when the mode is 2. The generator must REPORT this, not resolve it.
//
// THE ATTEMPT BOUND IS THE WHOLE OF THE FOURTH SET'S LESSON. Rejection sampling without a bound
// hangs; with a bound and no report, it emits an illegal item; with a bound and a report, it
// tells you the truth. The third of those is the only acceptable behaviour and it is the one
// that costs an extra output port.
`timescale 1ns/1ps
module spi_seq_gen #(
parameter DW = 32,
parameter LEN_W = 6,
parameter CNT_W = 16,
parameter TRIES = 32 // the rejection-sampling bound
) (
input wire clk,
input wire rst_n,
input wire [1:0] cset, // 0 = LOOSE, 1 = SPEC, 2 = OVER, 3 = BAD
input wire [31:0] seed,
input wire reseed,
input wire req, // draw one item
output reg item_valid,
output reg [1:0] mode, // {cpol, cpha} -- the SLAVE's mode
output reg [1:0] m_mode, // the MASTER's, derived unless a mismatch is asked for
output reg [1:0] width_c, // 0 = 1 bit, 1 = 8 bits, 2 = 32 bits
output reg order_c,
output reg [1:0] gap_c, // 0 = min, 1 = mid, 2 = long
output reg [DW-1:0] data,
// THE PORT THAT MAKES AN INCONSISTENT SET REPORTABLE. Without it the only honest options are
// to hang or to lie.
output reg solve_fail,
output reg [CNT_W-1:0] n_items,
output reg [CNT_W-1:0] n_fail,
// How many draws the solver needed. A set that is merely tight shows up here long before it
// shows up as a missing bin.
output reg [CNT_W-1:0] n_redraws
);
localparam [1:0] CS_LOOSE = 2'd0,
CS_SPEC = 2'd1,
CS_OVER = 2'd2,
CS_BAD = 2'd3;
reg [31:0] rng;
function [31:0] nxt;
input [31:0] s;
reg [31:0] x;
begin
x = s;
x = x ^ (x << 13);
x = x ^ (x >> 17);
x = x ^ (x << 5);
nxt = x;
end
endfunction
// The width draw. CS_LOOSE is uniform over the three classes; every other set weights them,
// because the interesting widths are the extremes and the byte and a uniform draw over a
// range spends most of its time in the middle.
function [1:0] draw_width;
input [31:0] r;
input [1:0] cs;
reg [3:0] q;
begin
q = r[19:16];
if (cs == CS_LOOSE) begin
// UNIFORM OVER THE RANGE 1..32, mapped to the class afterwards. One draw in
// sixteen is the narrowest frame and one in sixteen is the widest, because that
// is their share of the range -- which is a rate nobody chose.
if (q == 4'd0) draw_width = 2'd0;
else if (q == 4'd15) draw_width = 2'd2;
else draw_width = 2'd1;
end
else if (q < 4'd4) draw_width = 2'd0; // 1 bit, deliberately often
else if (q < 4'd12) draw_width = 2'd1; // 8 bits
else draw_width = 2'd2; // 32 bits
end
endfunction
// The gap draw. CS_LOOSE is uniform; the others put deliberate weight on the MINIMUM,
// because `>= MIN` is only distinguished from `> MIN` by the value MIN itself.
function [1:0] draw_gap;
input [31:0] r;
input [1:0] cs;
reg [3:0] q;
reg [7:0] wide;
begin
q = r[27:24];
wide = r[27:20];
if (cs == CS_LOOSE) begin
// UNIFORM OVER A GAP OF 0..255 CYCLES. The minimum is one value out of the range,
// so it arrives once in 256 draws -- which is the rate at which a `>= MIN`
// requirement is actually tested by a uniform generator.
if (wide == 8'd0) draw_gap = 2'd0;
else if (wide < 8'd128) draw_gap = 2'd1;
else draw_gap = 2'd2;
end
else if (q < 4'd6) draw_gap = 2'd0; // at the minimum, deliberately often
else if (q < 4'd12) draw_gap = 2'd1;
else draw_gap = 2'd2;
end
endfunction
// Does a candidate satisfy the selected set? Everything the sets differ by lives here, in
// one function, so that "the constraint" is a single readable object rather than a
// behaviour spread across a draw.
function integer satisfies;
input [1:0] cs;
input [1:0] m;
input [1:0] w;
begin
satisfies = 1;
// THE DEVICE SPECIFICATION, which EVERY set honours -- including the loose one,
// because the point of the loose set is bad distributions, not illegal traffic: no
// 32-bit frame in mode 3.
if (m == 2'd3 && w == 2'd2) satisfies = 0;
// CS_OVER's extra clause. It looks like the line above and it is not in any
// datasheet.
if (cs == CS_OVER && m == 2'd0 && w == 2'd2) satisfies = 0;
// CS_BAD: in mode 2 the width must be 32 AND must be 1. No value satisfies both.
if (cs == CS_BAD && m == 2'd2 && w != 2'd2) satisfies = 0;
if (cs == CS_BAD && m == 2'd2 && w != 2'd0) satisfies = 0;
end
endfunction
integer tries;
reg [1:0] cand_m, cand_w;
always @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
rng <= 32'h1;
item_valid <= 1'b0;
solve_fail <= 1'b0;
mode <= 2'd0;
m_mode <= 2'd0;
width_c <= 2'd0;
order_c <= 1'b0;
gap_c <= 2'd0;
data <= {DW{1'b0}};
n_items <= {CNT_W{1'b0}};
n_fail <= {CNT_W{1'b0}};
n_redraws <= {CNT_W{1'b0}};
end else begin
item_valid <= 1'b0;
solve_fail <= 1'b0;
if (reseed) begin
// A ZERO SEED IS REFUSED, because zero is a fixed point of xorshift and a
// generator emitting a constant looks exactly like a suite that ran.
rng <= (seed == 32'd0) ? 32'h1234_5678 : seed;
end else if (req) begin
rng = nxt(rng);
cand_m = rng[1:0];
cand_w = draw_width(rng, cset);
tries = 0;
// REJECTION SAMPLING WITH A BOUND. Unbounded, this hangs on CS_BAD. Bounded
// without a report, it emits the last candidate -- an item that violates the
// specification, produced by the component whose job is to enforce it.
// Only the CONSTRAINED variable is redrawn. Redrawing the mode as well would make
// an inconsistent set look solvable -- and it would also stop the generator from
// sampling the distribution it was asked for, because the antecedent of the
// constraint would be chosen to suit the constraint. A solver may reorder its
// variables; a rejection sampler that redraws its antecedent is answering a
// different question.
while (satisfies(cset, cand_m, cand_w) == 0 && tries < TRIES) begin
rng = nxt(rng);
cand_w = draw_width(rng, cset);
tries = tries + 1;
end
n_redraws <= n_redraws + tries[CNT_W-1:0];
if (satisfies(cset, cand_m, cand_w) == 0) begin
// THE HONEST FAILURE. No item is emitted; the caller is told.
solve_fail <= 1'b1;
n_fail <= n_fail + 1'b1;
end else begin
mode <= cand_m;
width_c <= cand_w;
order_c <= rng[30];
gap_c <= draw_gap(rng, cset);
data <= {rng[15:0], rng[31:16]};
// THE MASTER'S MODE IS DERIVED, not drawn. Chapter 16.2 measured why: two
// independent bits disagree three times in four, so an independently drawn
// master mode makes the MISMATCH case 75% of the suite and the agreeing
// case -- the one every real transfer uses -- the corner case.
m_mode <= (rng[29:28] == 2'd0) ? (cand_m ^ 2'd1) : cand_m;
item_valid <= 1'b1;
n_items <= n_items + 1'b1;
end
end
end
end
endmoduleAzvya Education Pvt. Ltd.VLSI Mentor
spi_seq_gen.vhd — the same design in VHDL
-- spi_seq_gen.vhd
--
-- Chapter 17.4 -- constraints that encode a device specification, and the two ways a constraint
-- set fails that a regression cannot tell apart from bad luck.
--
-- A CONSTRAINT SET IS A SPECIFICATION WRITTEN IN A SOLVER'S LANGUAGE, and like any specification
-- it can be wrong in two directions:
--
-- OVER-CONSTRAINED the set forbids something the plan requires. Every transaction is legal,
-- the generator reports success on every one of them, and a coverage bin
-- is never reached. The symptom is indistinguishable from "we need a
-- longer run", and teams spend weeks on the wrong remedy.
--
-- INCONSISTENT the set has no solution at all in some corner. A real solver reports
-- this; a hand-rolled generator usually does something far worse, which is
-- to give up quietly and emit its last draw -- an item that violates the
-- specification, produced by the component whose job is to enforce it.
--
-- This generator implements four constraint sets so that both failures can be measured against
-- a working one:
--
-- CS_LOOSE LEGAL, and uniform over the device's RANGES rather than over its interesting
-- values. The frame width is drawn as a number and mapped to a class, the gap as a
-- cycle count and mapped to a class -- which is how a real generator is usually
-- written, because the ranges are what the datasheet lists. The consequence is that
-- the extremes are as rare as their share of the range: one width in sixteen, one
-- gap in two hundred and fifty-six.
--
-- CS_SPEC the device's actual specification: widths weighted to the ones that matter,
-- timing deliberately biased onto the minimum, and the master's mode DERIVED from
-- the slave's rather than drawn independently (Chapter 16.2's result: two
-- independent bits disagree three times in four, which makes the common case rare).
--
-- CS_OVER CS_SPEC with one extra clause that looks harmless -- "never a 32-bit frame in
-- mode 0" -- added by somebody who misread a table. Nothing fails. A bin goes
-- unreached forever.
--
-- CS_BAD a set with no solution in a corner: "width must be 32" and "width must be 1"
-- both apply when the mode is 2. The generator must REPORT this, not resolve it.
--
-- THE ATTEMPT BOUND IS THE WHOLE OF THE FOURTH SET'S LESSON. Rejection sampling without a bound
-- hangs; with a bound and no report, it emits an illegal item; with a bound and a report, it
-- tells you the truth. The third of those is the only acceptable behaviour and it is the one
-- that costs an extra output port.
--
-- WHAT VHDL ADDS HERE. The constraint set is an ENUMERATION and the item is a RECORD, so a set
-- added to the type without a clause in `satisfies` is an analysis error rather than a silently
-- unconstrained run -- which is the same argument Chapter 16.4 made about its fault codes. And
-- `satisfies` is a pure function over the item, so the constraint is a readable object that can
-- be reasoned about on its own rather than a behaviour spread across a draw.
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
package spi_seq_pkg is
constant GDW : natural := 32;
-- The four constraint sets, as a type. A set added here without a clause in `satisfies`
-- below is a case-statement error at analysis time, not an unconstrained run.
type cset_t is (CS_LOOSE, CS_SPEC, CS_OVER, CS_BAD);
type spi_item_t is record
mode : natural; -- {cpol, cpha} as 0..3, the SLAVE's mode
m_mode : natural; -- the MASTER's, derived unless a mismatch is asked for
width_c : natural; -- 0 = 1 bit, 1 = 8 bits, 2 = 32 bits
order_c : natural; -- 0 = MSB-first, 1 = LSB-first
gap_c : natural; -- 0 = min, 1 = mid, 2 = long
data : std_logic_vector(GDW - 1 downto 0);
end record;
constant ITEM_ZERO : spi_item_t := (0, 0, 0, 0, 0, (others => '0'));
-- THE DEVICE SPECIFICATION plus whatever the selected set adds. Every set honours the
-- specification -- including the loose one, because the point of the loose set is bad
-- distributions, not illegal traffic.
function satisfies (cs : cset_t; m : natural; w : natural) return boolean;
end package spi_seq_pkg;
package body spi_seq_pkg is
function satisfies (cs : cset_t; m : natural; w : natural) return boolean is
begin
-- No 32-bit frame in mode 3. This is the sentence in the datasheet.
if m = 3 and w = 2 then
return false;
end if;
case cs is
when CS_LOOSE => return true;
when CS_SPEC => return true;
-- CS_OVER's extra clause. It looks like the line above and it is in no datasheet.
when CS_OVER => return not (m = 0 and w = 2);
-- CS_BAD: in mode 2 the width must be 32 AND must be 1. No value satisfies both.
when CS_BAD => if m = 2 then return (w = 2) and (w = 0); else return true; end if;
end case;
end function satisfies;
end package body spi_seq_pkg;
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
use work.spi_seq_pkg.all;
entity spi_seq_gen is
generic (
-- NAMED `MAX_TRIES` RATHER THAN `TRIES`, AND THAT IS NOT A STYLE CHOICE.
--
-- VHDL is CASE-INSENSITIVE, so a process variable called `tries` shadows a generic called
-- `TRIES`. The loop bound `tries < TRIES` then reads `tries < tries`, which is false at
-- entry, so the rejection loop never executed once -- and the generator reported failures
-- on candidates it had never attempted to fix.
--
-- The symptom was a pair of numbers that cannot both be true: 125 reported failures and
-- ZERO redraws. Neither number alone looked wrong. That is the argument for reporting the
-- work a solver did alongside the result it reached.
MAX_TRIES : natural := 32 -- the rejection-sampling bound
);
port (
clk : in std_logic;
rst_n : in std_logic;
cset : in cset_t;
seed : in unsigned(31 downto 0);
reseed : in std_logic;
req : in std_logic;
item_valid : out std_logic;
item : out spi_item_t;
-- THE PORT THAT MAKES AN INCONSISTENT SET REPORTABLE. Without it the only honest options
-- are to hang or to lie.
solve_fail : out std_logic;
n_items : out natural;
n_fail : out natural;
n_redraws : out natural
);
end entity spi_seq_gen;
architecture rtl of spi_seq_gen is
signal iv_r : std_logic := '0';
signal it_r : spi_item_t := ITEM_ZERO;
signal sf_r : std_logic := '0';
signal ni_r : natural := 0;
signal nf_r : natural := 0;
signal nr_r : natural := 0;
begin
item_valid <= iv_r;
item <= it_r;
solve_fail <= sf_r;
n_items <= ni_r;
n_fail <= nf_r;
n_redraws <= nr_r;
process (clk, rst_n) is
variable rng : unsigned(31 downto 0) := x"00000001";
variable cand_m : natural;
variable cand_w : natural;
variable tries : natural;
procedure step_rng is
begin
rng := rng xor shift_left(rng, 13);
rng := rng xor shift_right(rng, 17);
rng := rng xor shift_left(rng, 5);
end procedure step_rng;
-- The width draw. CS_LOOSE is uniform over the RANGE 1..32 and mapped to a class
-- afterwards, which is how a generator is usually written because the ranges are what the
-- datasheet lists -- and it makes each extreme one draw in sixteen. Every other set
-- weights the classes, because the interesting widths are the extremes and the byte.
impure function draw_width (cs : cset_t) return natural is
variable q : natural;
begin
q := to_integer(rng(19 downto 16));
if cs = CS_LOOSE then
if q = 0 then return 0;
elsif q = 15 then return 2;
else return 1;
end if;
elsif q < 4 then return 0;
elsif q < 12 then return 1;
else return 2;
end if;
end function draw_width;
-- The gap draw. CS_LOOSE is uniform over 0..255 cycles, so the minimum arrives once in
-- 256 draws -- the rate at which a `>= MIN` requirement is actually tested by a uniform
-- generator. The others put deliberate weight on the minimum.
impure function draw_gap (cs : cset_t) return natural is
variable q : natural;
variable wide : natural;
begin
q := to_integer(rng(27 downto 24));
wide := to_integer(rng(27 downto 20));
if cs = CS_LOOSE then
if wide = 0 then return 0;
elsif wide < 128 then return 1;
else return 2;
end if;
elsif q < 6 then return 0;
elsif q < 12 then return 1;
else return 2;
end if;
end function draw_gap;
begin
if rst_n = '0' then
rng := x"00000001";
iv_r <= '0';
sf_r <= '0';
it_r <= ITEM_ZERO;
ni_r <= 0;
nf_r <= 0;
nr_r <= 0;
elsif rising_edge(clk) then
iv_r <= '0';
sf_r <= '0';
if reseed = '1' then
-- A ZERO SEED IS REFUSED, because zero is a fixed point of xorshift and a
-- generator emitting a constant looks exactly like a suite that ran.
if seed = 0 then rng := x"12345678"; else rng := seed; end if;
elsif req = '1' then
step_rng;
cand_m := to_integer(rng(1 downto 0));
cand_w := draw_width(cset);
tries := 0;
-- REJECTION SAMPLING WITH A BOUND, and only the CONSTRAINED variable is redrawn.
-- Redrawing the mode as well would make an inconsistent set look solvable -- and
-- would stop the generator sampling the distribution it was asked for, because
-- the antecedent of the constraint would be chosen to suit the constraint. A
-- solver may reorder its variables; a rejection sampler that redraws its
-- antecedent is answering a different question.
while not satisfies(cset, cand_m, cand_w) and tries < MAX_TRIES loop
step_rng;
cand_w := draw_width(cset);
tries := tries + 1;
end loop;
nr_r <= nr_r + tries;
if not satisfies(cset, cand_m, cand_w) then
-- THE HONEST FAILURE. No item is emitted; the caller is told.
sf_r <= '1';
nf_r <= nf_r + 1;
else
it_r.mode <= cand_m;
it_r.width_c <= cand_w;
it_r.order_c <= to_integer(rng(30 downto 30));
it_r.gap_c <= draw_gap(cset);
it_r.data <= std_logic_vector(rng(15 downto 0) & rng(31 downto 16));
-- THE MASTER'S MODE IS DERIVED, not drawn. Chapter 16.2 measured why: two
-- independent bits disagree three times in four, so an independently drawn
-- master mode makes the MISMATCH case 75% of the suite and the agreeing
-- case -- the one every real transfer uses -- the corner case.
if to_integer(rng(29 downto 28)) = 0 then
-- Flip the phase bit. VHDL has no `xor` on NATURAL, so the
-- flip is arithmetic: an even mode becomes the next one up
-- and an odd mode the next one down.
if (cand_m mod 2) = 0 then
it_r.m_mode <= cand_m + 1;
else
it_r.m_mode <= cand_m - 1;
end if;
else
it_r.m_mode <= cand_m;
end if;
iv_r <= '1';
ni_r <= ni_r + 1;
end if;
end if;
end if;
end process;
end architecture rtl;The Bench
Azvya Education Pvt. Ltd.VLSI Mentor
spi_seq_gen_tb.sv — two legal sets fifty draws apart, a silent stall, and an inconsistent set that refuses to invent a solution
// spi_seq_gen_tb.sv
//
// FOUR CONSTRAINT SETS THROUGH ONE GENERATOR AND CHAPTER 17.3'S COVERAGE MODEL, plus a sequence
// library measured for order and for reproducibility.
//
// FIVE MEASUREMENTS.
//
// 1. THE LOOSE SET UNDER-SAMPLES THE BOUNDARY. Both sets are LEGAL -- the difference is not
// illegal traffic, it is distribution. The loose set draws the frame width as a number in
// 1..32 and the gap as a cycle count in 0..255 and maps each to a class afterwards, which is
// how a generator is usually written because the ranges are what the datasheet lists. The
// extremes then arrive at their share of the range: one width in sixteen, one gap in 256.
// Measured as the draw count at which each set closes the curated coverage model.
//
// 2. THE SPEC SET CLOSES SOONER, and the difference is entirely in the distributions -- same
// axes, same bins, same generator, different weights. This is the measurement that justifies
// writing `dist` clauses instead of `inside` ranges.
//
// 3. OVER-CONSTRAINING IS SILENT. CS_OVER adds one clause that is not in any datasheet. Every
// transaction is legal, the generator reports ZERO failures, and one curated bin is never
// reached in any number of transactions. The symptom is indistinguishable from "run longer",
// and the diagnosis needs the constraint set rather than the coverage report.
//
// 4. AN INCONSISTENT SET MUST FAIL LOUDLY. CS_BAD has no solution when the mode is 2. The
// generator emits NO item and raises `solve_fail`, and the measurement is twofold: the
// failure count is non-zero, AND no emitted item ever violates the specification. A
// generator that gave up and emitted its last draw would satisfy the first half.
//
// 5. SEQUENCES ARE ORDERED AND REPRODUCIBLE. A directed sequence -- configure, then a burst,
// then idle -- is measured to deliver its items in order, and the same seed is measured to
// deliver the identical item stream twice. A random sequence that cannot be replayed is a
// failure nobody can debug.
`timescale 1ns/1ps
module spi_seq_gen_tb;
localparam int DW = 32;
localparam int LEN_W = 6;
localparam int CNT_W = 16;
localparam [1:0] CS_LOOSE = 2'd0, CS_SPEC = 2'd1, CS_OVER = 2'd2, CS_BAD = 2'd3;
reg clk = 1'b0;
always #5 clk = ~clk;
reg rst_n = 1'b1;
reg [1:0] cset = CS_SPEC;
reg [31:0] seed = 32'h1234_5678;
reg reseed = 1'b0;
reg req = 1'b0;
wire item_valid, order_c, solve_fail;
wire [1:0] mode, m_mode, width_c, gap_c;
wire [DW-1:0] data;
wire [CNT_W-1:0] n_items, n_fail, n_redraws;
spi_seq_gen #(.DW(DW), .LEN_W(LEN_W), .CNT_W(CNT_W)) u_g (
.clk(clk), .rst_n(rst_n),
.cset(cset), .seed(seed), .reseed(reseed), .req(req),
.item_valid(item_valid), .mode(mode), .m_mode(m_mode),
.width_c(width_c), .order_c(order_c), .gap_c(gap_c), .data(data),
.solve_fail(solve_fail), .n_items(n_items), .n_fail(n_fail), .n_redraws(n_redraws)
);
// Chapter 17.3's coverage model, unmodified. A coverage model is a reusable component and
// this is what reusing one looks like: the axes and the illegal set come with it, so a
// generator cannot quietly redefine what counts as covered.
reg cov_clr = 1'b0;
wire [CNT_W-1:0] full_hit, full_total, full_reachable, full_informative, full_noise_hit;
wire [CNT_W-1:0] cur_hit, cur_total, cur_reachable, illegal_hits;
spi_cov_model #(.CNT_W(CNT_W)) u_c (
.clk(clk), .rst_n(rst_n),
.sample(item_valid), .mode(mode), .width_c(width_c), .order_c(order_c), .gap_c(gap_c),
.clr(cov_clr),
.full_hit(full_hit), .full_total(full_total), .full_reachable(full_reachable),
.full_informative(full_informative), .full_noise_hit(full_noise_hit),
.cur_hit(cur_hit), .cur_total(cur_total), .cur_reachable(cur_reachable),
.illegal_hits(illegal_hits)
);
integer errors = 0;
initial begin
#4_000_000;
$display("FAIL: the simulation did not finish within its time limit");
$finish;
end
// Records every emitted item, so order and reproducibility can be checked.
localparam int LOG_N = 64;
reg [DW-1:0] log_data [0:LOG_N-1];
reg [1:0] log_mode [0:LOG_N-1];
reg [1:0] log_wid [0:LOG_N-1];
integer log_n;
reg logging;
// And a check that runs on every emitted item, for measurement 4: no item the generator
// emits may violate the device specification, whatever the constraint set asked for.
integer spec_violations;
always @(posedge clk) if (rst_n) begin
if (item_valid) begin
if (mode == 2'd3 && width_c == 2'd2) spec_violations = spec_violations + 1;
if (logging && log_n < LOG_N) begin
log_data[log_n] = data;
log_mode[log_n] = mode;
log_wid[log_n] = width_c;
end
if (logging) log_n = log_n + 1;
end
end
task automatic set_cset(input [1:0] cs, input [31:0] sd);
begin
@(negedge clk);
cset = cs;
seed = sd;
reseed = 1'b1;
@(negedge clk);
reseed = 1'b0;
@(negedge clk);
end
endtask
task automatic draw(input integer n);
integer i;
begin
for (i = 0; i < n; i = i + 1) begin
@(negedge clk);
req = 1'b1;
@(negedge clk);
req = 1'b0;
@(negedge clk);
end
end
endtask
task automatic clear_cov;
begin
@(negedge clk); cov_clr = 1'b1;
@(negedge clk); cov_clr = 1'b0;
@(negedge clk);
end
endtask
// Runs a constraint set until the curated model closes, or until `limit` draws.
task automatic close_run(input [1:0] cs, input integer limit,
output integer closed_at, output integer hit, output integer fails);
integer i;
begin
set_cset(cs, 32'h1234_5678);
clear_cov();
closed_at = 0;
for (i = 1; i <= limit; i = i + 1) begin
draw(1);
if (closed_at == 0 && cur_hit == cur_reachable) closed_at = i;
end
hit = cur_hit;
fails = n_fail;
end
endtask
integer loose_at, loose_hit, loose_fail;
integer spec_at, spec_hit, spec_fail;
integer over_at, over_hit, over_fail;
integer bad_items, bad_fail, bad_redraws;
integer b_items, b_fail, b_redraws;
integer i, j;
integer seq_order_bad, replay_bad;
reg [DW-1:0] first_run [0:31];
initial begin
spec_violations = 0;
log_n = 0;
logging = 1'b0;
rst_n = 1'b1;
@(negedge clk);
rst_n = 1'b0;
repeat (4) @(negedge clk);
rst_n = 1'b1;
repeat (4) @(negedge clk);
// ============================================================
// 1 + 2. LOOSE AGAINST SPEC, SAME AXES, SAME BINS.
// ============================================================
// The loose set is given more than six times the budget, so that the comparison is
// about the RATE it closes at rather than about whether the run was long enough.
close_run(CS_LOOSE, 8000, loose_at, loose_hit, loose_fail);
close_run(CS_SPEC, 1200, spec_at, spec_hit, spec_fail);
$display(" constraint set curated bins closed at draw solver failures");
$display(" loose %10d %14d %15d", loose_hit, loose_at, loose_fail);
$display(" spec %10d %14d %15d", spec_hit, spec_at, spec_fail);
if (spec_hit != cur_reachable || spec_at == 0) begin
$display(" FAIL: the spec-weighted set did not close the curated model (%0d of %0d at draw %0d)",
spec_hit, cur_reachable, spec_at);
errors = errors + 1;
end
if (loose_at == 0) begin
$display(" FAIL: the uniform set never closed even in 8000 draws, so the comparison has no ratio to report");
errors = errors + 1;
end
if (loose_at < 10 * spec_at) begin
$display(" FAIL: the uniform set closed within an order of magnitude of the weighted one (%0d vs %0d), so this stimulus does not demonstrate the cost of a uniform distribution",
loose_at, spec_at);
errors = errors + 1;
end
$display(" 1 + 2. both sets are LEGAL and both eventually close all %0d reachable curated bins. The spec-weighted set closed at draw %0d; the uniform set needed draw %0d -- %0dx as many. Same axes, same bins, same generator: the entire difference is the DISTRIBUTION. A uniform draw over a range reaches that range's extremes at their share of it, so a gap minimum drawn from 0..255 arrives once in 256 draws and has to be crossed with four modes before the bin closes. Nobody chose that rate; it is what writing `inside` instead of `dist` chooses for you",
cur_reachable, spec_at, loose_at, loose_at / spec_at);
// ============================================================
// 3. OVER-CONSTRAINING IS SILENT.
// ============================================================
close_run(CS_OVER, 1200, over_at, over_hit, over_fail);
$display(" over %10d %14d %15d", over_hit, over_at, over_fail);
if (over_hit >= cur_reachable) begin
$display(" FAIL: the over-constrained set closed the model, so its extra clause is not actually blocking a bin");
errors = errors + 1;
end
if (over_fail != 0) begin
$display(" FAIL: the over-constrained set reported %0d solver failures; over-constraining is supposed to be SILENT",
over_fail);
errors = errors + 1;
end
$display(" 3. the over-constrained set reached %0d of %0d curated bins in 1200 draws and reported ZERO solver failures. One clause that is not in any datasheet -- `never a 32-bit frame in mode 0` -- and the coverage report says only that a bin is missing. That is indistinguishable from needing a longer run, and no number of draws will change it: the diagnosis has to come from reading the CONSTRAINT SET, not the coverage report",
over_hit, cur_reachable);
// ============================================================
// 4. AN INCONSISTENT SET MUST FAIL LOUDLY.
// ============================================================
spec_violations = 0;
set_cset(CS_BAD, 32'h1234_5678);
clear_cov();
b_items = n_items;
b_fail = n_fail;
b_redraws = n_redraws;
draw(400);
// The generator's counters are cumulative across the whole run, so the phase is measured
// as a delta. A counter read absolutely after four earlier phases reports the run, not
// the experiment.
bad_items = n_items - b_items;
bad_fail = n_fail - b_fail;
bad_redraws = n_redraws - b_redraws;
$display(" the inconsistent set, 400 draws:");
$display(" items emitted ..................... %0d", bad_items);
$display(" solver failures reported .......... %0d", bad_fail);
$display(" redraws spent ..................... %0d", bad_redraws);
$display(" emitted items violating the spec .. %0d", spec_violations);
if (bad_fail == 0) begin
$display(" FAIL: the inconsistent set reported no failures, so it either found a solution that does not exist or emitted something without saying so");
errors = errors + 1;
end
if (spec_violations != 0) begin
$display(" FAIL: %0d emitted items violated the device specification; a generator that gives up and emits its last draw is worse than one that hangs",
spec_violations);
errors = errors + 1;
end
$display(" 4. the inconsistent set produced %0d reported failures across 400 draws and emitted ZERO items that violate the specification. Both halves are the measurement: a generator that gave up and emitted its last candidate would also report failures, and it would be the component whose job is to enforce the specification producing the violation. Unbounded rejection sampling hangs; bounded and silent, it lies; bounded and reported, it tells the truth -- and the only difference between the last two is one output port",
bad_fail);
// ============================================================
// 5. SEQUENCES: ORDER AND REPRODUCIBILITY.
// ============================================================
// A directed sequence built from the same item type: a configure item, a burst of four,
// and an idle item. The measurement is that they arrive in that order -- which is what a
// sequence gives you over a pile of random items.
seq_order_bad = 0;
set_cset(CS_SPEC, 32'h0BAD_C0DE);
logging = 1'b1;
log_n = 0;
draw(6);
logging = 1'b0;
if (log_n != 6) begin
$display(" FAIL: the directed sequence emitted %0d items where 6 were requested", log_n);
errors = errors + 1;
seq_order_bad = seq_order_bad + 1;
end
// Reproducibility: the same seed must produce the identical stream.
for (i = 0; i < 6; i = i + 1) first_run[i] = log_data[i];
set_cset(CS_SPEC, 32'h0BAD_C0DE);
logging = 1'b1;
log_n = 0;
draw(6);
logging = 1'b0;
replay_bad = 0;
for (i = 0; i < 6; i = i + 1)
if (log_data[i] !== first_run[i]) replay_bad = replay_bad + 1;
$display(" the sequence: %0d items requested, %0d delivered; replay mismatches %0d of 6",
6, log_n, replay_bad);
if (replay_bad != 0) begin
$display(" FAIL: the same seed produced a different item stream in %0d of 6 positions",
replay_bad);
errors = errors + 1;
end
$display(" 5. the same seed produced the IDENTICAL six-item stream twice. A random sequence that cannot be replayed is a failure nobody can debug -- and reproducibility is also what lets the three language versions of this bench be three measurements of one experiment rather than three unrelated numbers");
if (errors == 0)
$display("PASS: a constraint set is a specification written in a solver's language, and it can be wrong in two directions that a coverage report cannot tell apart. Measured against Chapter 17.3's coverage model, both the uniform and the spec-weighted set are LEGAL and both eventually close all %0d reachable curated bins -- the weighted one at draw %0d and the uniform one at draw %0d, %0d times as many -- so the entire difference between them is the DISTRIBUTION. A gap minimum drawn uniformly from 0..255 arrives once in 256 draws and must then be crossed with four modes before its bin closes; nobody chose that rate, it is what writing a legal range instead of a weighted clause chooses for you. Then one extra clause that appears in no datasheet left the over-constrained set at %0d of %0d bins with ZERO reported failures -- a silent stall indistinguishable from needing a longer run, diagnosable only by reading the constraint set. And an inconsistent set, with no solution at all when the mode is 2, produced %0d reported failures in 400 draws and emitted ZERO items violating the specification: unbounded rejection sampling hangs, bounded and silent it lies, bounded and reported it tells the truth, and the difference between the last two is one output port. Finally the same seed produced the identical six-item stream twice, because a random sequence that cannot be replayed is a failure nobody can debug",
cur_reachable, spec_at, loose_at, loose_at / spec_at, over_hit, cur_reachable, bad_fail);
else
$display("FAIL: %0d error(s)", errors);
$finish;
end
endmoduleAzvya Education Pvt. Ltd.VLSI Mentor
spi_seq_gen_tb.v — the same bench in Verilog-2001
// spi_seq_gen_tb.v
//
// FOUR CONSTRAINT SETS THROUGH ONE GENERATOR AND CHAPTER 17.3'S COVERAGE MODEL, plus a sequence
// library measured for order and for reproducibility.
//
// FIVE MEASUREMENTS.
//
// 1. THE LOOSE SET UNDER-SAMPLES THE BOUNDARY. Both sets are LEGAL -- the difference is not
// illegal traffic, it is distribution. The loose set draws the frame width as a number in
// 1..32 and the gap as a cycle count in 0..255 and maps each to a class afterwards, which is
// how a generator is usually written because the ranges are what the datasheet lists. The
// extremes then arrive at their share of the range: one width in sixteen, one gap in 256.
// Measured as the draw count at which each set closes the curated coverage model.
//
// 2. THE SPEC SET CLOSES SOONER, and the difference is entirely in the distributions -- same
// axes, same bins, same generator, different weights. This is the measurement that justifies
// writing `dist` clauses instead of `inside` ranges.
//
// 3. OVER-CONSTRAINING IS SILENT. CS_OVER adds one clause that is not in any datasheet. Every
// transaction is legal, the generator reports ZERO failures, and one curated bin is never
// reached in any number of transactions. The symptom is indistinguishable from "run longer",
// and the diagnosis needs the constraint set rather than the coverage report.
//
// 4. AN INCONSISTENT SET MUST FAIL LOUDLY. CS_BAD has no solution when the mode is 2. The
// generator emits NO item and raises `solve_fail`, and the measurement is twofold: the
// failure count is non-zero, AND no emitted item ever violates the specification. A
// generator that gave up and emitted its last draw would satisfy the first half.
//
// 5. SEQUENCES ARE ORDERED AND REPRODUCIBLE. A directed sequence -- configure, then a burst,
// then idle -- is measured to deliver its items in order, and the same seed is measured to
// deliver the identical item stream twice. A random sequence that cannot be replayed is a
// failure nobody can debug.
`timescale 1ns/1ps
module spi_seq_gen_tb;
localparam DW = 32;
localparam LEN_W = 6;
localparam CNT_W = 16;
localparam [1:0] CS_LOOSE = 2'd0, CS_SPEC = 2'd1, CS_OVER = 2'd2, CS_BAD = 2'd3;
reg clk;
always #5 clk = ~clk;
reg rst_n;
reg [1:0] cset;
reg [31:0] seed;
reg reseed;
reg req;
wire item_valid, order_c, solve_fail;
wire [1:0] mode, m_mode, width_c, gap_c;
wire [DW-1:0] data;
wire [CNT_W-1:0] n_items, n_fail, n_redraws;
spi_seq_gen #(.DW(DW), .LEN_W(LEN_W), .CNT_W(CNT_W)) u_g (
.clk(clk), .rst_n(rst_n),
.cset(cset), .seed(seed), .reseed(reseed), .req(req),
.item_valid(item_valid), .mode(mode), .m_mode(m_mode),
.width_c(width_c), .order_c(order_c), .gap_c(gap_c), .data(data),
.solve_fail(solve_fail), .n_items(n_items), .n_fail(n_fail), .n_redraws(n_redraws)
);
// Chapter 17.3's coverage model, unmodified. A coverage model is a reusable component and
// this is what reusing one looks like: the axes and the illegal set come with it, so a
// generator cannot quietly redefine what counts as covered.
reg cov_clr;
wire [CNT_W-1:0] full_hit, full_total, full_reachable, full_informative, full_noise_hit;
wire [CNT_W-1:0] cur_hit, cur_total, cur_reachable, illegal_hits;
spi_cov_model #(.CNT_W(CNT_W)) u_c (
.clk(clk), .rst_n(rst_n),
.sample(item_valid), .mode(mode), .width_c(width_c), .order_c(order_c), .gap_c(gap_c),
.clr(cov_clr),
.full_hit(full_hit), .full_total(full_total), .full_reachable(full_reachable),
.full_informative(full_informative), .full_noise_hit(full_noise_hit),
.cur_hit(cur_hit), .cur_total(cur_total), .cur_reachable(cur_reachable),
.illegal_hits(illegal_hits)
);
integer errors;
initial begin
#4_000_000;
$display("FAIL: the simulation did not finish within its time limit");
$finish;
end
// Records every emitted item, so order and reproducibility can be checked.
localparam LOG_N = 64;
reg [DW-1:0] log_data [0:LOG_N-1];
reg [1:0] log_mode [0:LOG_N-1];
reg [1:0] log_wid [0:LOG_N-1];
integer log_n;
reg logging;
// And a check that runs on every emitted item, for measurement 4: no item the generator
// emits may violate the device specification, whatever the constraint set asked for.
integer spec_violations;
always @(posedge clk) if (rst_n) begin
if (item_valid) begin
if (mode == 2'd3 && width_c == 2'd2) spec_violations = spec_violations + 1;
if (logging && log_n < LOG_N) begin
log_data[log_n] = data;
log_mode[log_n] = mode;
log_wid[log_n] = width_c;
end
if (logging) log_n = log_n + 1;
end
end
task set_cset;
input [1:0] cs;
input [31:0] sd;
begin
@(negedge clk);
cset = cs;
seed = sd;
reseed = 1'b1;
@(negedge clk);
reseed = 1'b0;
@(negedge clk);
end
endtask
task draw;
input integer n;
integer i;
begin
for (i = 0; i < n; i = i + 1) begin
@(negedge clk);
req = 1'b1;
@(negedge clk);
req = 1'b0;
@(negedge clk);
end
end
endtask
task clear_cov;
begin
@(negedge clk); cov_clr = 1'b1;
@(negedge clk); cov_clr = 1'b0;
@(negedge clk);
end
endtask
// Runs a constraint set until the curated model closes, or until `limit` draws.
task close_run;
input [1:0] cs;
input integer limit;
output integer closed_at;
output integer hit;
output integer fails;
integer i;
begin
set_cset(cs, 32'h1234_5678);
clear_cov();
closed_at = 0;
for (i = 1; i <= limit; i = i + 1) begin
draw(1);
if (closed_at == 0 && cur_hit == cur_reachable) closed_at = i;
end
hit = cur_hit;
fails = n_fail;
end
endtask
integer loose_at, loose_hit, loose_fail;
integer spec_at, spec_hit, spec_fail;
integer over_at, over_hit, over_fail;
integer bad_items, bad_fail, bad_redraws;
integer b_items, b_fail, b_redraws;
integer i, j;
integer seq_order_bad, replay_bad;
reg [DW-1:0] first_run [0:31];
initial begin
spec_violations = 0;
log_n = 0;
logging = 1'b0;
rst_n = 1'b1;
@(negedge clk);
rst_n = 1'b0;
repeat (4) @(negedge clk);
rst_n = 1'b1;
repeat (4) @(negedge clk);
// ============================================================
// 1 + 2. LOOSE AGAINST SPEC, SAME AXES, SAME BINS.
// ============================================================
// The loose set is given more than six times the budget, so that the comparison is
// about the RATE it closes at rather than about whether the run was long enough.
close_run(CS_LOOSE, 8000, loose_at, loose_hit, loose_fail);
close_run(CS_SPEC, 1200, spec_at, spec_hit, spec_fail);
$display(" constraint set curated bins closed at draw solver failures");
$display(" loose %10d %14d %15d", loose_hit, loose_at, loose_fail);
$display(" spec %10d %14d %15d", spec_hit, spec_at, spec_fail);
if (spec_hit != cur_reachable || spec_at == 0) begin
$display(" FAIL: the spec-weighted set did not close the curated model (%0d of %0d at draw %0d)",
spec_hit, cur_reachable, spec_at);
errors = errors + 1;
end
if (loose_at == 0) begin
$display(" FAIL: the uniform set never closed even in 8000 draws, so the comparison has no ratio to report");
errors = errors + 1;
end
if (loose_at < 10 * spec_at) begin
$display(" FAIL: the uniform set closed within an order of magnitude of the weighted one (%0d vs %0d), so this stimulus does not demonstrate the cost of a uniform distribution",
loose_at, spec_at);
errors = errors + 1;
end
$display(" 1 + 2. both sets are LEGAL and both eventually close all %0d reachable curated bins. The spec-weighted set closed at draw %0d; the uniform set needed draw %0d -- %0dx as many. Same axes, same bins, same generator: the entire difference is the DISTRIBUTION. A uniform draw over a range reaches that range's extremes at their share of it, so a gap minimum drawn from 0..255 arrives once in 256 draws and has to be crossed with four modes before the bin closes. Nobody chose that rate; it is what writing `inside` instead of `dist` chooses for you",
cur_reachable, spec_at, loose_at, loose_at / spec_at);
// ============================================================
// 3. OVER-CONSTRAINING IS SILENT.
// ============================================================
close_run(CS_OVER, 1200, over_at, over_hit, over_fail);
$display(" over %10d %14d %15d", over_hit, over_at, over_fail);
if (over_hit >= cur_reachable) begin
$display(" FAIL: the over-constrained set closed the model, so its extra clause is not actually blocking a bin");
errors = errors + 1;
end
if (over_fail != 0) begin
$display(" FAIL: the over-constrained set reported %0d solver failures; over-constraining is supposed to be SILENT",
over_fail);
errors = errors + 1;
end
$display(" 3. the over-constrained set reached %0d of %0d curated bins in 1200 draws and reported ZERO solver failures. One clause that is not in any datasheet -- `never a 32-bit frame in mode 0` -- and the coverage report says only that a bin is missing. That is indistinguishable from needing a longer run, and no number of draws will change it: the diagnosis has to come from reading the CONSTRAINT SET, not the coverage report",
over_hit, cur_reachable);
// ============================================================
// 4. AN INCONSISTENT SET MUST FAIL LOUDLY.
// ============================================================
spec_violations = 0;
set_cset(CS_BAD, 32'h1234_5678);
clear_cov();
b_items = n_items;
b_fail = n_fail;
b_redraws = n_redraws;
draw(400);
// The generator's counters are cumulative across the whole run, so the phase is measured
// as a delta. A counter read absolutely after four earlier phases reports the run, not
// the experiment.
bad_items = n_items - b_items;
bad_fail = n_fail - b_fail;
bad_redraws = n_redraws - b_redraws;
$display(" the inconsistent set, 400 draws:");
$display(" items emitted ..................... %0d", bad_items);
$display(" solver failures reported .......... %0d", bad_fail);
$display(" redraws spent ..................... %0d", bad_redraws);
$display(" emitted items violating the spec .. %0d", spec_violations);
if (bad_fail == 0) begin
$display(" FAIL: the inconsistent set reported no failures, so it either found a solution that does not exist or emitted something without saying so");
errors = errors + 1;
end
if (spec_violations != 0) begin
$display(" FAIL: %0d emitted items violated the device specification; a generator that gives up and emits its last draw is worse than one that hangs",
spec_violations);
errors = errors + 1;
end
$display(" 4. the inconsistent set produced %0d reported failures across 400 draws and emitted ZERO items that violate the specification. Both halves are the measurement: a generator that gave up and emitted its last candidate would also report failures, and it would be the component whose job is to enforce the specification producing the violation. Unbounded rejection sampling hangs; bounded and silent, it lies; bounded and reported, it tells the truth -- and the only difference between the last two is one output port",
bad_fail);
// ============================================================
// 5. SEQUENCES: ORDER AND REPRODUCIBILITY.
// ============================================================
// A directed sequence built from the same item type: a configure item, a burst of four,
// and an idle item. The measurement is that they arrive in that order -- which is what a
// sequence gives you over a pile of random items.
seq_order_bad = 0;
set_cset(CS_SPEC, 32'h0BAD_C0DE);
logging = 1'b1;
log_n = 0;
draw(6);
logging = 1'b0;
if (log_n != 6) begin
$display(" FAIL: the directed sequence emitted %0d items where 6 were requested", log_n);
errors = errors + 1;
seq_order_bad = seq_order_bad + 1;
end
// Reproducibility: the same seed must produce the identical stream.
for (i = 0; i < 6; i = i + 1) first_run[i] = log_data[i];
set_cset(CS_SPEC, 32'h0BAD_C0DE);
logging = 1'b1;
log_n = 0;
draw(6);
logging = 1'b0;
replay_bad = 0;
for (i = 0; i < 6; i = i + 1)
if (log_data[i] !== first_run[i]) replay_bad = replay_bad + 1;
$display(" the sequence: %0d items requested, %0d delivered; replay mismatches %0d of 6",
6, log_n, replay_bad);
if (replay_bad != 0) begin
$display(" FAIL: the same seed produced a different item stream in %0d of 6 positions",
replay_bad);
errors = errors + 1;
end
$display(" 5. the same seed produced the IDENTICAL six-item stream twice. A random sequence that cannot be replayed is a failure nobody can debug -- and reproducibility is also what lets the three language versions of this bench be three measurements of one experiment rather than three unrelated numbers");
if (errors == 0)
$display("PASS: a constraint set is a specification written in a solver's language, and it can be wrong in two directions that a coverage report cannot tell apart. Measured against Chapter 17.3's coverage model, both the uniform and the spec-weighted set are LEGAL and both eventually close all %0d reachable curated bins -- the weighted one at draw %0d and the uniform one at draw %0d, %0d times as many -- so the entire difference between them is the DISTRIBUTION. A gap minimum drawn uniformly from 0..255 arrives once in 256 draws and must then be crossed with four modes before its bin closes; nobody chose that rate, it is what writing a legal range instead of a weighted clause chooses for you. Then one extra clause that appears in no datasheet left the over-constrained set at %0d of %0d bins with ZERO reported failures -- a silent stall indistinguishable from needing a longer run, diagnosable only by reading the constraint set. And an inconsistent set, with no solution at all when the mode is 2, produced %0d reported failures in 400 draws and emitted ZERO items violating the specification: unbounded rejection sampling hangs, bounded and silent it lies, bounded and reported it tells the truth, and the difference between the last two is one output port. Finally the same seed produced the identical six-item stream twice, because a random sequence that cannot be replayed is a failure nobody can debug",
cur_reachable, spec_at, loose_at, loose_at / spec_at, over_hit, cur_reachable, bad_fail);
else
$display("FAIL: %0d error(s)", errors);
$finish;
end
initial begin
clk = 1'b0;
rst_n = 1'b1;
cset = CS_SPEC;
seed = 32'h1234_5678;
reseed = 1'b0;
req = 1'b0;
cov_clr = 1'b0;
errors = 0;
end
endmoduleAzvya Education Pvt. Ltd.VLSI Mentor
spi_seq_gen_tb.vhd — the same bench in VHDL
-- spi_seq_gen_tb.vhd
--
-- FOUR CONSTRAINT SETS THROUGH ONE GENERATOR AND CHAPTER 17.3'S COVERAGE MODEL, plus a sequence
-- library measured for order and for reproducibility.
--
-- FIVE MEASUREMENTS.
--
-- 1. THE LOOSE SET UNDER-SAMPLES THE BOUNDARY. Both sets are LEGAL -- the difference is not
-- illegal traffic, it is distribution. The loose set draws the frame width as a number in
-- 1..32 and the gap as a cycle count in 0..255 and maps each to a class afterwards, which is
-- how a generator is usually written because the ranges are what the datasheet lists. The
-- extremes then arrive at their share of the range: one width in sixteen, one gap in 256.
-- Measured as the draw count at which each set closes the curated coverage model.
--
-- 2. THE SPEC SET CLOSES SOONER, and the difference is entirely in the distributions -- same
-- axes, same bins, same generator, different weights. This is the measurement that justifies
-- writing `dist` clauses instead of `inside` ranges.
--
-- 3. OVER-CONSTRAINING IS SILENT. CS_OVER adds one clause that is not in any datasheet. Every
-- transaction is legal, the generator reports ZERO failures, and one curated bin is never
-- reached in any number of transactions. The symptom is indistinguishable from "run longer",
-- and the diagnosis needs the constraint set rather than the coverage report.
--
-- 4. AN INCONSISTENT SET MUST FAIL LOUDLY. CS_BAD has no solution when the mode is 2. The
-- generator emits NO item and raises `solve_fail`, and the measurement is twofold: the
-- failure count is non-zero, AND no emitted item ever violates the specification. A
-- generator that gave up and emitted its last draw would satisfy the first half.
--
-- 5. SEQUENCES ARE ORDERED AND REPRODUCIBLE. A directed sequence -- configure, then a burst,
-- then idle -- is measured to deliver its items in order, and the same seed is measured to
-- deliver the identical item stream twice. A random sequence that cannot be replayed is a
-- failure nobody can debug.
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
use work.spi_seq_pkg.all;
use work.spi_cov_pkg.all;
entity spi_seq_gen_tb is
end entity spi_seq_gen_tb;
architecture tb of spi_seq_gen_tb is
constant HALF_T : time := 5 ns;
signal clk : std_logic := '0';
signal rst_n : std_logic := '1';
signal done_sim : boolean := false;
signal cset : cset_t := CS_SPEC;
signal seed : unsigned(31 downto 0) := x"12345678";
signal reseed : std_logic := '0';
signal req : std_logic := '0';
signal item_valid, solve_fail : std_logic;
signal item : spi_item_t;
signal n_items, n_fail, n_redraws : natural;
-- Chapter 17.3's coverage model, unmodified. A coverage model is a reusable component and
-- this is what reusing one looks like: the axes and the illegal set come with it, so a
-- generator cannot quietly redefine what counts as covered.
shared variable cov : spi_cov_t;
signal errors : integer := 0;
begin
clk_gen : process is
begin
while not done_sim loop
wait for HALF_T;
clk <= not clk;
end loop;
wait;
end process clk_gen;
u_g : entity work.spi_seq_gen
generic map (MAX_TRIES => 32)
port map (clk => clk, rst_n => rst_n,
cset => cset, seed => seed, reseed => reseed, req => req,
item_valid => item_valid, item => item,
solve_fail => solve_fail, n_items => n_items,
n_fail => n_fail, n_redraws => n_redraws);
main : process is
procedure idle_n (n : natural) is
begin
for i in 1 to n loop wait until falling_edge(clk); end loop;
end procedure idle_n;
procedure set_cset (cs : cset_t; sd : unsigned(31 downto 0)) is
begin
wait until falling_edge(clk);
cset <= cs;
seed <= sd;
reseed <= '1';
wait until falling_edge(clk);
reseed <= '0';
wait until falling_edge(clk);
end procedure set_cset;
-- Sampling happens HERE rather than in a concurrent process, because a protected type's
-- methods must be called from one place to keep the coverage model's order of arrival
-- identical to the generator's order of emission.
procedure draw (n : natural) is
begin
for i in 1 to n loop
wait until falling_edge(clk);
req <= '1';
wait until falling_edge(clk);
req <= '0';
if item_valid = '1' then
cov.sample(item.mode, item.width_c, item.order_c, item.gap_c);
end if;
wait until falling_edge(clk);
end loop;
end procedure draw;
-- Runs a constraint set until the curated model closes, or until `limit` draws.
procedure close_run (cs : cset_t; limit : natural;
closed_at : out natural; hit : out natural; fails : out natural) is
variable base_fail : natural;
begin
set_cset(cs, x"12345678");
cov.clear;
base_fail := n_fail;
closed_at := 0;
for i in 1 to limit loop
draw(1);
if closed_at = 0 and cov.cur_hit = cov.cur_reachable then
closed_at := i;
end if;
end loop;
hit := cov.cur_hit;
fails := n_fail - base_fail;
end procedure close_run;
variable loose_at, loose_hit, loose_fail : natural;
variable spec_at, spec_hit, spec_fail : natural;
variable over_at, over_hit, over_fail : natural;
variable bad_items, bad_fail, bad_redraws : natural;
variable b_items, b_fail, b_redraws : natural;
variable spec_violations : natural := 0;
variable curr : natural;
variable replay_bad : natural := 0;
type data_arr_t is array (0 to 7) of std_logic_vector(GDW - 1 downto 0);
variable first_run, second_run : data_arr_t;
variable k : natural;
-- Emits `n` items and records their data words, for the reproducibility measurement.
procedure draw_log (n : natural; variable log : out data_arr_t; variable cnt : out natural) is
variable c : natural := 0;
begin
c := 0;
for i in 1 to n loop
wait until falling_edge(clk);
req <= '1';
wait until falling_edge(clk);
req <= '0';
-- `item_valid` is a one-cycle pulse and it is already high here, one negedge
-- after the posedge that accepted the request. Waiting a further cycle before
-- looking, as an earlier version did, misses every item and reports a sequence
-- of length zero -- which the replay check then compares against itself and
-- passes.
if item_valid = '1' and c < 8 then
log(c) := item.data;
c := c + 1;
end if;
wait until falling_edge(clk);
end loop;
cnt := c;
end procedure draw_log;
begin
rst_n <= '1';
idle_n(1);
rst_n <= '0';
idle_n(4);
rst_n <= '1';
idle_n(4);
curr := cov.cur_reachable;
-- ==============================================================
-- 1 + 2. LOOSE AGAINST SPEC, SAME AXES, SAME BINS.
-- ==============================================================
-- The loose set is given more than six times the budget, so that the comparison is about
-- the RATE it closes at rather than about whether the run was long enough.
close_run(CS_LOOSE, 8000, loose_at, loose_hit, loose_fail);
close_run(CS_SPEC, 1200, spec_at, spec_hit, spec_fail);
report " constraint set curated bins closed at draw solver failures";
report " loose " & integer'image(loose_hit) & " " &
integer'image(loose_at) & " " & integer'image(loose_fail);
report " spec " & integer'image(spec_hit) & " " &
integer'image(spec_at) & " " & integer'image(spec_fail);
if spec_hit /= curr or spec_at = 0 then
report " FAIL: the spec-weighted set did not close the curated model";
errors <= errors + 1; wait for 1 ns;
end if;
if loose_at = 0 then
report " FAIL: the uniform set never closed even in 8000 draws, so the comparison has no ratio to report";
errors <= errors + 1; wait for 1 ns;
end if;
if loose_at < 10 * spec_at then
report " FAIL: the uniform set closed within an order of magnitude of the weighted one, so this stimulus does not demonstrate the cost of a uniform distribution";
errors <= errors + 1; wait for 1 ns;
end if;
report " 1 + 2. both sets are LEGAL and both eventually close all " & integer'image(curr) &
" reachable curated bins. The spec-weighted set closed at draw " &
integer'image(spec_at) & "; the uniform set needed draw " & integer'image(loose_at) &
" -- " & integer'image(loose_at / spec_at) &
"x as many. Same axes, same bins, same generator: the entire difference is the DISTRIBUTION. A uniform draw over a range reaches that range's extremes at their share of it, so a gap minimum drawn from 0..255 arrives once in 256 draws and has to be crossed with four modes before the bin closes. Nobody chose that rate; it is what writing a legal range instead of a weighted clause chooses for you";
-- ==============================================================
-- 3. OVER-CONSTRAINING IS SILENT.
-- ==============================================================
close_run(CS_OVER, 1200, over_at, over_hit, over_fail);
report " over " & integer'image(over_hit) & " " &
integer'image(over_at) & " " & integer'image(over_fail);
if over_hit >= curr then
report " FAIL: the over-constrained set closed the model, so its extra clause is not actually blocking a bin";
errors <= errors + 1; wait for 1 ns;
end if;
if over_fail /= 0 then
report " FAIL: the over-constrained set reported solver failures; over-constraining is supposed to be SILENT";
errors <= errors + 1; wait for 1 ns;
end if;
report " 3. the over-constrained set reached " & integer'image(over_hit) & " of " &
integer'image(curr) &
" curated bins in 1200 draws and reported ZERO solver failures. One clause that is not in any datasheet -- `never a 32-bit frame in mode 0` -- and the coverage report says only that a bin is missing. That is indistinguishable from needing a longer run, and no number of draws will change it: the diagnosis has to come from reading the CONSTRAINT SET, not the coverage report";
-- ==============================================================
-- 4. AN INCONSISTENT SET MUST FAIL LOUDLY.
-- ==============================================================
set_cset(CS_BAD, x"12345678");
cov.clear;
b_items := n_items;
b_fail := n_fail;
b_redraws := n_redraws;
spec_violations := 0;
for i in 1 to 400 loop
wait until falling_edge(clk);
req <= '1';
wait until falling_edge(clk);
req <= '0';
wait until falling_edge(clk);
-- Every emitted item is checked against the DEVICE SPECIFICATION, whatever the
-- constraint set asked for.
if item_valid = '1' and item.mode = 3 and item.width_c = 2 then
spec_violations := spec_violations + 1;
end if;
end loop;
-- The generator's counters are cumulative across the whole run, so the phase is measured
-- as a delta. A counter read absolutely after three earlier phases reports the run, not
-- the experiment.
bad_items := n_items - b_items;
bad_fail := n_fail - b_fail;
bad_redraws := n_redraws - b_redraws;
report " the inconsistent set, 400 draws:";
report " items emitted ..................... " & integer'image(bad_items);
report " solver failures reported .......... " & integer'image(bad_fail);
report " redraws spent ..................... " & integer'image(bad_redraws);
report " emitted items violating the spec .. " & integer'image(spec_violations);
if bad_fail = 0 then
report " FAIL: the inconsistent set reported no failures, so it either found a solution that does not exist or emitted something without saying so";
errors <= errors + 1; wait for 1 ns;
end if;
if spec_violations /= 0 then
report " FAIL: emitted items violated the device specification; a generator that gives up and emits its last draw is worse than one that hangs";
errors <= errors + 1; wait for 1 ns;
end if;
report " 4. the inconsistent set produced " & integer'image(bad_fail) &
" reported failures across 400 draws and emitted ZERO items that violate the specification. Both halves are the measurement: a generator that gave up and emitted its last candidate would also report failures, and it would be the component whose job is to enforce the specification producing the violation. Unbounded rejection sampling hangs; bounded and silent, it lies; bounded and reported, it tells the truth -- and the only difference between the last two is one output port";
-- ==============================================================
-- 5. SEQUENCES: ORDER AND REPRODUCIBILITY.
-- ==============================================================
set_cset(CS_SPEC, x"0BADC0DE");
draw_log(6, first_run, k);
set_cset(CS_SPEC, x"0BADC0DE");
draw_log(6, second_run, replay_bad);
replay_bad := 0;
for i in 0 to 5 loop
if first_run(i) /= second_run(i) then replay_bad := replay_bad + 1; end if;
end loop;
report " the sequence: 6 items requested, " & integer'image(k) &
" delivered; replay mismatches " & integer'image(replay_bad) & " of 6";
if replay_bad /= 0 then
report " FAIL: the same seed produced a different item stream";
errors <= errors + 1; wait for 1 ns;
end if;
report " 5. the same seed produced the IDENTICAL six-item stream twice. A random sequence that cannot be replayed is a failure nobody can debug -- and reproducibility is also what lets the three language versions of this bench be three measurements of one experiment rather than three unrelated numbers";
wait for 1 ns;
if errors = 0 then
report "PASS: a constraint set is a specification written in a solver's language, and it can be wrong in two directions that a coverage report cannot tell apart. Measured against Chapter 17.3's coverage model, both the uniform and the spec-weighted set are LEGAL and both eventually close all " &
integer'image(curr) &
" reachable curated bins -- the weighted one at draw " & integer'image(spec_at) &
" and the uniform one at draw " & integer'image(loose_at) & ", " &
integer'image(loose_at / spec_at) &
" times as many -- so the entire difference between them is the DISTRIBUTION. A gap minimum drawn uniformly from 0..255 arrives once in 256 draws and must then be crossed with four modes before its bin closes; nobody chose that rate, it is what writing a legal range instead of a weighted clause chooses for you. Then one extra clause that appears in no datasheet left the over-constrained set at " &
integer'image(over_hit) & " of " & integer'image(curr) &
" bins with ZERO reported failures -- a silent stall indistinguishable from needing a longer run, diagnosable only by reading the constraint set. And an inconsistent set, with no solution at all when the mode is 2, produced " &
integer'image(bad_fail) &
" reported failures in 400 draws and emitted ZERO items violating the specification: unbounded rejection sampling hangs, bounded and silent it lies, bounded and reported it tells the truth, and the difference between the last two is one output port. Finally the same seed produced the identical six-item stream twice, because a random sequence that cannot be replayed is a failure nobody can debug"
severity note;
else
report "FAIL: " & integer'image(errors) & " error(s)" severity error;
end if;
done_sim <= true;
wait for 100 ns;
std.env.stop;
end process main;
end architecture tb;9. The Same Sets As SystemVerilog Constraints
Reviewed code, per Chapter 16.3's toolchain note. A solver removes the rejection loop; it does not remove either failure mode.
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
class spi_item extends uvm_sequence_item;
`uvm_object_utils(spi_item)
rand bit [31:0] data;
rand int nbits;
rand bit lsb_first;
rand bit cpol, cpha;
rand bit m_cpol, m_cpha;
rand int lead, half, lag, gap;
// THE DEVICE SPECIFICATION. One constraint, named after the sentence in the datasheet it
// encodes, so that a reviewer can check it against the document rather than against intent.
constraint c_spec_no_wide_in_mode3 {
!({cpol, cpha} == 2'b11 && nbits == 32);
}
// DISTRIBUTIONS, NOT RANGES, and this is the whole of measurement 4. `inside {[1:32]}` is legal
// and reaches each extreme one draw in thirty-two; `dist` puts the mass where the requirements
// are.
constraint c_width_dist {
nbits inside {[1:32]};
nbits dist { 1 := 20, 8 := 40, 32 := 20, [2:31] := 20 };
}
// THE BOUNDARY, WEIGHTED DELIBERATELY. A `>= MIN` requirement is distinguished from a buggy
// `> MIN` only by the value MIN, and a uniform draw over a wide legal range reaches it at the
// rate the range width dictates.
constraint c_timing_dist {
lead >= LEAD_MIN; half >= HALF_MIN; lag >= LAG_MIN; gap >= GAP_MIN;
gap dist { GAP_MIN := 40, [GAP_MIN+1 : GAP_MIN+15] := 40, [GAP_MIN+16 : GAP_MIN+240] := 20 };
lead dist { LEAD_MIN := 40, [LEAD_MIN+1 : LEAD_MIN+15] := 60 };
half dist { HALF_MIN := 40, [HALF_MIN+1 : HALF_MIN+15] := 60 };
}
// THE MASTER'S MODE IS DERIVED. Two independent random bits disagree 75% of the time, which
// would make the mismatch case the common one. `solve ... before` is not what fixes this -- an
// implication is.
constraint c_master_mode { {m_cpol, m_cpha} == {cpol, cpha}; }
endclass
// OVER-CONSTRAINING, as it actually appears: a derived test that adds one clause for a local
// reason. Nothing fails. A bin goes unreached forever, in every test that inherits this class.
class spi_item_over extends spi_item;
`uvm_object_utils(spi_item_over)
constraint c_local { !({cpol, cpha} == 2'b00 && nbits == 32); } // in no datasheet
endclass
// AND THE INCONSISTENT SET. A solver reports this -- and the report is only useful if somebody
// checks the return value.
class spi_item_bad extends spi_item;
`uvm_object_utils(spi_item_bad)
constraint c_a { ({cpol, cpha} == 2'b10) -> (nbits == 32); }
constraint c_b { ({cpol, cpha} == 2'b10) -> (nbits == 1); }
endclass
// In a sequence, `randomize()` RETURNS A STATUS and ignoring it is the class-based spelling of
// "bounded and silent". This is the single line that distinguishes a generator that lies from one
// that tells the truth.
task body();
repeat (N) begin
req = spi_item::type_id::create("req");
start_item(req);
if (!req.randomize())
`uvm_fatal("SPI_SOLVE", "the constraint set has no solution; no item was sent")
finish_item(req);
end
endtaskif (!req.randomize()) is the whole of section 6 in one line. A sequence that writes req.randomize(); and moves on has a generator that emits whatever the solver left in the object.
10. Why a Verification Engineer Cares
Because the over-constrained stall is a genuine schedule risk and its symptom points at the wrong remedy.
The diagnostic sequence is short and worth memorising. Change the seed: a hole that moves is a distribution problem, so tune the weights. A hole that does not move is structural, and then there are exactly two candidates — the transaction object has no field that can reach the bin (Chapter 16.2), or a clause forbids it. Both are found by reading, not by running.
The second habit is one line: check the return value of randomize(). Ignoring it is the class-based spelling of bounded and silent, and it converts an unsatisfiable constraint set into an item that violates the specification, produced by the component whose job is to enforce it.
And the third is about where constraints live. CS_OVER's extra clause is realistic precisely because it was added in a derived class for a local reason — and every test that inherits it loses a bin. A constraint that narrows the space belongs in the test that needs it, as a randomize() with, not in the item.
11. Why an FPGA or ASIC Engineer Cares
Because the distribution measurement decides whether the numbers in your datasheet were ever really tested.
A suite whose gaps were drawn uniformly from 0..255 cycles tested your published minimum once in 256 transactions, and then only in whichever mode happened to come up. The weighted set reached the same bin fifty times sooner. When a customer's master sits exactly at your published minimum and the transfer fails, the question is whether that combination was ever driven — and the answer is a property of the distribution, not of the run length.
And the inconsistent-set result is worth asking about directly: if two clauses of your device's specification cannot both be satisfied in some mode, a silent generator will produce traffic that violates one of them, and it will be indistinguishable from legal stimulus. That is a specification bug, found by the verification environment refusing to invent a solution.
12. Failure Signature — A Coverage Bin That Twenty Thousand Transactions Cannot Reach
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
Symptom one bin will not close. The regression is extended to 20,000
transactions and reseeded a dozen times. The bin stays empty
and the generator reports zero failures throughout.
What happened a constraint added in a derived class for a local reason
forbids the combination. Every transaction is legal, so
nothing in the environment objects.
What would have a seed sweep -- three runs -- to establish that the hole is
caught it structural, and then reading the constraint set rather than
the coverage report. The two structural causes are a missing
FIELD and a forbidding CLAUSE, and both are found by reading.
The tell zero solver failures alongside a permanently empty bin. An
over-constrained set is perfectly satisfiable -- that is what
makes it silent. If the set were inconsistent you would be
looking at failures instead, which is a much easier bug.13. Common Misconceptions
"Both sets are legal, so they are equivalent." They close the same bins fifty times apart. Legality says nothing about where the probability mass is, and the boundary values requirements are written against are single points in a range.
"inside {[1:32]} covers the widths." It reaches each extreme one draw in thirty-two. A dist clause puts the mass where the requirements are, and the measurement here is the cost of not writing one.
"Randomising more fields gives better stimulus." Randomising the master's mode independently of the slave's makes the mismatch case 75% of the suite and the agreeing case rare. Fields that should be derived must be derived.
"An over-constrained set will show up as a solver failure." It will not. An over-constrained set is perfectly satisfiable — that is exactly why it is silent. Only an inconsistent set fails, and an inconsistent set is the easier bug.
"randomize() throws if it cannot solve." It returns a status. Ignoring it leaves the item holding whatever the solver last put there, and the component whose job is to enforce the specification becomes the one producing the violation.
"Rejection sampling is equivalent to a solver." Only if it is bounded and reports. And a rejection loop that redraws the constraint's antecedent to escape the constraint is no longer sampling the requested distribution — it is answering a different question quietly.
14. Reason It Through
Both legal sets close the same 29 bins. Why does the uniform one need 2,120 draws?
Because its gap minimum is one value out of 0..255, so it arrives once in 256 draws — and the bin that needs it is a cross with four modes, so it needs the minimum to coincide with each mode in turn. The product of a rare value and a cross is the closure time, and neither factor is visible in the coverage report.
An over-constrained set reports zero solver failures. Explain why that follows from what over-constrained means.
Over-constrained means the set is satisfiable and narrower than intended. Every draw succeeds, so there is nothing for the generator to report. Failures come from inconsistency, which is the opposite problem and the easier one.
Why must the rejection loop redraw only the constrained variable?
Because redrawing the antecedent escapes the constraint rather than satisfying it — an inconsistent set then looks solvable, and the generator stops sampling the distribution it was asked for since the antecedent is now chosen to suit the constraint. A solver may reorder variables; a rejection sampler that moves its antecedent is answering a different question.
A VHDL generator reported 125 failures and 0 redraws. Why can those two numbers not both be true, and what was the cause?
A failure is only reachable after the loop has exhausted its bound, so failures imply redraws. The cause was case-insensitivity: a variable tries shadowed the generic TRIES, the bound read tries < tries, and the loop never ran. The pair of numbers is what exposed it — which is the argument for reporting the work a solver did alongside its result.
You have an empty bin and a clean regression. Give the three-step diagnosis in order.
Change the seed two or three times: if the hole moves, it is a distribution problem and the weights need tuning. If it does not move, read the transaction object for a field that could reach the bin (Chapter 16.2); if the field exists, read the constraint set for a clause that forbids it. The first step costs three runs and eliminates the most common wrong remedy.
15. Understanding Check
16. Summary
A constraint set is a specification written in a solver's language, and it can be wrong in two directions that a coverage report cannot tell apart. Measured against Chapter 17.3's model, both the uniform and the spec-weighted set are legal and both eventually close all 29 reachable curated bins — the weighted one at draw 42 and the uniform one at draw 2,120, fifty times as many — so the entire difference is the distribution: a gap minimum drawn uniformly from 0..255 arrives once in 256 draws and must then be crossed with four modes before its bin closes. Then one extra clause appearing in no datasheet left the over-constrained set at 28 of 29 bins with zero reported failures — a silent stall indistinguishable from needing a longer run, diagnosable only by reading the constraint set. And an inconsistent set produced 95 reported failures in 400 draws and emitted zero items violating the specification: unbounded rejection sampling hangs, bounded and silent it lies, bounded and reported it tells the truth, and the difference between the last two is one output port — or, in a class-based environment, checking the return value of randomize(). Finally the same seed produced the identical six-item stream twice, because a random sequence that cannot be replayed is a failure nobody can debug.
17. What Comes Next
The stimulus is right. Chapter 17.5 assembles the whole environment into a reusable agent, and measures the part of a sequencer that surprises people: three arbitration policies that deliver identical coverage and completely different traffic.
Continue learning
Related tutorials
- Related topic
Full-Duplex Exchange
Every SPI transfer moves a bit in both directions on every edge, whether the software wanted it to or not. Where dummy bytes come from, why bytes received during a command phase exist but mean nothing, and why read and write are interpretations rather than modes.
- Related topic
Extracting Protocol Rules and the Verification Plan
Eight pin-observable SPI rules, each with a checker and an exercised counter, because a checker alone cannot tell never-broken from never-reached. Legal traffic violates nothing and exercises all eight; eight injected faults produce a diagonal violation matrix; and one plan row is proved to have no checker at all.
- Related topic
Transaction Modelling and Stimulus
Two generators, the same 400 transactions, and a 9-versus-16 coverage result, because the stimulus space is not the data space. The master's mode is derived from the slave's, illegal traffic is a request rather than an accident, and a zero seed is refused.
- Related topic
Driver Architecture
A driver owns every timing number in the protocol, so it is the one component that must be checked against something not written to agree with it. Thirty-two legal transactions violate nothing and exercise all eight rules; seven injected faults fire exactly the rules predicted.