SPI · Module 17
Functional Coverage and Meaningful Crosses
A four-way cross of 72 bins has two defects pulling in opposite directions: six bins the specification forbids, so it tops out at 91.7 percent, and twelve it can fill that carry no information. A curated set of three two-way crosses closes at draw 52 where the full cross plateaus at 372.
Four axes, so cross all four and let the tool report what is missing. It is one line, it is exhaustive, and it produces a number that can never reach 100% — after which the number stops being information and becomes a standing item on a status call.
A cross is a claim that some interaction matters. A cross whose bins cannot be traced to a design mechanism is a claim nobody made.
1. The Four Axes
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
mode CPOL/CPHA 4 bins
width 1, 8, 32 bits 3 bins
order MSB-first, LSB 2 bins
gap min, mid, long 3 binsThe full cross is 4 × 3 × 2 × 3 = 72 bins. Two groups of them can never be filled usefully, and the two groups are unfillable for different reasons — which is the distinction the chapter is about.
2. Two Kinds Of Unfillable, And They Are Not The Same Kind
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
ILLEGAL BY SPECIFICATION (6 bins)
---------------------------------------------------------------------------
This device does not support 32-bit frames in mode 3. That is a sentence in a
datasheet. The generator is constrained never to produce it, and a bin that is
REACHED would be a BUG IN THE TESTBENCH.
SystemVerilog spells this `illegal_bins`, and the spelling matters: an illegal
bin that is merely EXCLUDED is a bin nobody notices when the constraint breaks.
MEANINGLESS BY CONSTRUCTION (12 bins)
---------------------------------------------------------------------------
A one-bit frame has no bit ORDER. MSB-first and LSB-first produce identical
traffic. Both bins fill, neither corresponds to anything a design could get
wrong, and the second is a copy of the first wearing a different label.
SystemVerilog spells this `ignore_bins`. Confusing it with the previous
category is how a real constraint failure gets filed as a known gap.So the full cross has two defects in one number, and they pull in opposite directions:
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
72 bins total
6 unreachable by constraint -> the report tops out at 66/72 = 91.7%,
forever
12 reachable but uninformative -> of the 66 it does fill, 18% carry no
information
54 informativeA report that cannot reach 100% becomes a standing agenda item. A report whose filled bins include copies of each other is worse, because it moves and the movement means nothing.
3. The Curated Alternative
Three two-way crosses, each naming an interaction somebody can point at in the design:
| Cross | Bins | The interaction it claims matters |
|---|---|---|
| mode × width | 12 | the launch-edge arithmetic depends on both (14.3, 14.6) |
| width × order | 6 | the bit-index mapping depends on both (14.3) |
| mode × gap | 12 | the idle level and the gap interact at the select (16.4) |
Thirty bins, and a number that can reach full. What it gives up is the four-way combinations — and the argument for giving them up is not that they are worthless, it is that nobody can name a design mechanism that depends on all four.
4. The Measurement
Every number below is derived from the bin definitions by the model itself rather than written down, so the arithmetic in section 2 is checked by the run.
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
the four-way cross: 72 bins total
unreachable: illegal by specification (mode 3, 32-bit frame) ... 6
reachable .................................................... 66
of which uninformative (a one-bit frame has no bit order) ..... 12
informative .................................................. 54
the curated set: mode x width (12) + width x order (6) + mode x gap (12) = 30 bins,
of which reachable ........................................... 29
after 4000 transactions:
curated set ..... 29 of 29 reachable (30 declared) closed at transaction 52
full cross ...... 66 of 72 last new bin at transaction 372
of the full cross's 66 filled bins, 12 are duplicates of other filled binsThe curated set closed at transaction 52. The full cross reached 66 of 72 — 91% — and its last new bin arrived at transaction 372, after which it did not move for the remaining 3,628 transactions. It is not converging slowly. It is finished, below full.
And 12 of the bins it did fill are copies of other filled bins, so 18% of its apparent progress carried no information at all.
5. The Illegal Set Is A Check, And It Fires
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
illegal-bin hits in 400 transactions:
with the specification's constraint applied ..... 0
with it deliberately removed ................... 22Both halves are the measurement. An illegal bin that has never fired is indistinguishable from an ignored one — and the difference between those two categories is the difference between a testbench bug and a known gap.
This is the same discipline Chapter 16.1 applied to rules and Chapter 17.1 to properties: a check that has never been observed to fire has been assumed, not verified.
6. Building It — Three HDLs
Azvya Education Pvt. Ltd.VLSI Mentor
spi_cov_model.sv — the coverage model, deriving its own reachable and informative counts
// spi_cov_model.sv
//
// Chapter 17.3 -- functional coverage, and the difference between a cross that earns its runtime
// and a cross that spends the project explaining itself.
//
// THE TEMPTATION IS THE FULL CROSS. Four axes, so cross all four, and let the tool tell you what
// is missing. It is one line, it is exhaustive, and it produces a number that can never reach
// 100% -- after which the number stops being information and becomes a standing item on a status
// call.
//
// THE FOUR AXES.
//
// mode CPOL/CPHA 4 bins
// width 1, 8, 32 bits 3 bins
// order MSB-first, LSB 2 bins
// gap min, mid, long 3 bins
//
// The full cross is 4 x 3 x 2 x 3 = 72 bins. Two groups of them can never be filled, and the two
// groups are unfillable for DIFFERENT reasons, which is the distinction the chapter is about.
//
// ILLEGAL BY SPECIFICATION (6 bins). This device does not support 32-bit frames in mode 3.
// That is a sentence in a datasheet, the generator is constrained never to produce it, and a
// bin that is reached would be a BUG IN THE TESTBENCH. SystemVerilog spells this
// `illegal_bins`, and the spelling matters: an illegal bin that is merely excluded is a bin
// nobody notices when the constraint breaks.
//
// MEANINGLESS BY CONSTRUCTION (12 bins). A one-bit frame has no bit ORDER -- MSB-first and
// LSB-first produce identical traffic. Both bins fill, neither corresponds to anything a
// design could get wrong, and the second one is a copy of the first wearing a different label.
// SystemVerilog spells this `ignore_bins`, and confusing it with the previous category is how
// a real constraint failure gets filed as a known gap.
//
// SO THE FULL CROSS HAS TWO DEFECTS IN ONE NUMBER, and they pull in opposite directions:
//
// 72 bins total
// 6 unreachable by constraint -> the report tops out at 66/72 = 91.7%, forever
// 12 reachable but uninformative -> of the 66 it does fill, 18% carry no information
// 54 informative
//
// A report that cannot reach 100% becomes a standing agenda item. A report whose filled bins
// include copies of each other is worse, because it moves and the movement means nothing.
//
// AND THE ILLEGAL COMBINATION IS IN THE CURATED SET TOO. `mode x width` contains mode 3 crossed
// with a 32-bit frame, so 30 curated bins are 29 reachable ones. An `illegal_bins` declaration
// is needed wherever the axes meet, not once per coverage model -- which is the kind of detail
// that makes a curated set look like more work than a full cross until the first time somebody
// has to explain 8.3%.
//
// THE CURATED SET IS THREE TWO-WAY CROSSES, chosen because each names an interaction somebody
// can point at in the design:
//
// mode x width 12 bins the launch-edge arithmetic depends on both (14.3, 14.6)
// width x order 6 bins the bit-index mapping depends on both (14.3)
// mode x gap 12 bins the idle level and the gap interact at the select (16.4)
//
// Thirty bins, all reachable, and a number that can reach 100%. What it gives up is the
// four-way combinations -- and the argument for giving them up is not that they are worthless,
// it is that NOBODY CAN NAME A DESIGN MECHANISM THAT DEPENDS ON ALL FOUR. A cross whose bins
// cannot be traced to an interaction is a cross that generates runtime and status meetings in
// equal measure.
//
// THIS MODULE IMPLEMENTS BOTH so that they can be measured against the same stimulus, and it
// implements the illegal set as a CHECK rather than an exclusion, so that a constraint failure
// is reported rather than quietly absorbed.
`timescale 1ns/1ps
module spi_cov_model #(
parameter int CNT_W = 16
) (
input wire clk,
input wire rst_n,
// One sampled transaction.
input wire sample,
input wire [1:0] mode, // {cpol, cpha}
input wire [1:0] width_c, // 0 = 1 bit, 1 = 8 bits, 2 = 32 bits
input wire order_c, // 0 = MSB-first, 1 = LSB-first
input wire [1:0] gap_c, // 0 = min, 1 = mid, 2 = long
input wire clr,
// The full four-way cross.
output wire [CNT_W-1:0] full_hit, // bins of the full cross that were reached
output wire [CNT_W-1:0] full_total, // 72
output wire [CNT_W-1:0] full_reachable, // 66 -- 72 less the 6 the constraint forbids
output wire [CNT_W-1:0] full_informative, // 54 -- 66 less the 12 that duplicate others
output wire [CNT_W-1:0] full_noise_hit, // how many of the FILLED bins are duplicates
// The curated two-way crosses.
output wire [CNT_W-1:0] cur_hit,
output wire [CNT_W-1:0] cur_total, // 30
output wire [CNT_W-1:0] cur_reachable, // 29 -- the illegal pair lives here too
// The check, not an exclusion: a transaction that lands in an illegal bin.
output reg [CNT_W-1:0] illegal_hits
);
localparam int N_FULL = 72; // 4 * 3 * 2 * 3
localparam int N_MW = 12; // mode x width
localparam int N_WO = 6; // width x order
localparam int N_MG = 12; // mode x gap
reg full_bin [0:N_FULL-1];
reg mw_bin [0:N_MW-1];
reg wo_bin [0:N_WO-1];
reg mg_bin [0:N_MG-1];
wire [6:0] full_idx = {2'b0, mode} * 7'd18 + {5'b0, width_c} * 7'd6
+ {6'b0, order_c} * 7'd3 + {5'b0, gap_c};
wire [3:0] mw_idx = {2'b0, mode} * 4'd3 + {2'b0, width_c};
wire [2:0] wo_idx = {1'b0, width_c} * 3'd2 + {2'b0, order_c};
wire [3:0] mg_idx = {2'b0, mode} * 4'd3 + {2'b0, gap_c};
// ILLEGAL BY SPECIFICATION: 32-bit frames are not supported in mode 3.
wire is_illegal = (mode == 2'd3) && (width_c == 2'd2);
// MEANINGLESS BY CONSTRUCTION: a one-bit frame has no bit order, so the LSB-first half of
// every one-bit row duplicates the MSB-first half and can never add information.
function automatic integer meaningless(input integer idx);
integer m, w, o;
begin
m = idx / 18;
w = (idx % 18) / 6;
o = (idx % 6) / 3;
meaningless = (w == 0 && o == 1) ? 1 : 0;
end
endfunction
function automatic integer illegal_bin(input integer idx);
integer m, w;
begin
m = idx / 18;
w = (idx % 18) / 6;
illegal_bin = (m == 3 && w == 2) ? 1 : 0;
end
endfunction
integer i;
reg [CNT_W-1:0] fh, ch, fr, fi, fn, cr;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
for (i = 0; i < N_FULL; i = i + 1) full_bin[i] <= 1'b0;
for (i = 0; i < N_MW; i = i + 1) mw_bin[i] <= 1'b0;
for (i = 0; i < N_WO; i = i + 1) wo_bin[i] <= 1'b0;
for (i = 0; i < N_MG; i = i + 1) mg_bin[i] <= 1'b0;
illegal_hits <= {CNT_W{1'b0}};
end else begin
if (clr) begin
for (i = 0; i < N_FULL; i = i + 1) full_bin[i] <= 1'b0;
for (i = 0; i < N_MW; i = i + 1) mw_bin[i] <= 1'b0;
for (i = 0; i < N_WO; i = i + 1) wo_bin[i] <= 1'b0;
for (i = 0; i < N_MG; i = i + 1) mg_bin[i] <= 1'b0;
illegal_hits <= {CNT_W{1'b0}};
end else if (sample) begin
full_bin[full_idx] <= 1'b1;
mw_bin[mw_idx] <= 1'b1;
wo_bin[wo_idx] <= 1'b1;
mg_bin[mg_idx] <= 1'b1;
// THE ILLEGAL SET IS A CHECK. An excluded bin is invisible when the constraint
// that was supposed to prevent it breaks; a counted one is a test failure.
if (is_illegal) illegal_hits <= illegal_hits + 1'b1;
end
end
end
// Combinational tallies, so the bench can read coverage at any instant. Every number is
// DERIVED from the bin definitions rather than written down, so the arithmetic in the header
// is checked by the run instead of being trusted.
always @* begin
fh = {CNT_W{1'b0}};
fr = {CNT_W{1'b0}};
fi = {CNT_W{1'b0}};
fn = {CNT_W{1'b0}};
for (i = 0; i < N_FULL; i = i + 1) begin
if (full_bin[i]) fh = fh + 1'b1;
if (illegal_bin(i) == 0) fr = fr + 1'b1;
if (illegal_bin(i) == 0 && meaningless(i) == 0) fi = fi + 1'b1;
if (full_bin[i] && meaningless(i) == 1) fn = fn + 1'b1;
end
ch = {CNT_W{1'b0}};
cr = {CNT_W{1'b0}};
for (i = 0; i < N_MW; i = i + 1) begin
if (mw_bin[i]) ch = ch + 1'b1;
// The illegal pair is mode 3 crossed with the 32-bit width, which in this cross is
// index 3*3 + 2.
if (i != (3 * 3 + 2)) cr = cr + 1'b1;
end
for (i = 0; i < N_WO; i = i + 1) begin
if (wo_bin[i]) ch = ch + 1'b1;
cr = cr + 1'b1;
end
for (i = 0; i < N_MG; i = i + 1) begin
if (mg_bin[i]) ch = ch + 1'b1;
cr = cr + 1'b1;
end
end
assign full_hit = fh;
assign full_total = N_FULL[CNT_W-1:0];
assign full_reachable = fr;
assign full_informative = fi;
assign full_noise_hit = fn;
assign cur_hit = ch;
assign cur_total = (N_MW + N_WO + N_MG);
assign cur_reachable = cr;
endmoduleAzvya Education Pvt. Ltd.VLSI Mentor
spi_cov_model.v — the same design in Verilog-2001
// spi_cov_model.v
//
// Chapter 17.3 -- functional coverage, and the difference between a cross that earns its runtime
// and a cross that spends the project explaining itself.
//
// THE TEMPTATION IS THE FULL CROSS. Four axes, so cross all four, and let the tool tell you what
// is missing. It is one line, it is exhaustive, and it produces a number that can never reach
// 100% -- after which the number stops being information and becomes a standing item on a status
// call.
//
// THE FOUR AXES.
//
// mode CPOL/CPHA 4 bins
// width 1, 8, 32 bits 3 bins
// order MSB-first, LSB 2 bins
// gap min, mid, long 3 bins
//
// The full cross is 4 x 3 x 2 x 3 = 72 bins. Two groups of them can never be filled, and the two
// groups are unfillable for DIFFERENT reasons, which is the distinction the chapter is about.
//
// ILLEGAL BY SPECIFICATION (6 bins). This device does not support 32-bit frames in mode 3.
// That is a sentence in a datasheet, the generator is constrained never to produce it, and a
// bin that is reached would be a BUG IN THE TESTBENCH. SystemVerilog spells this
// `illegal_bins`, and the spelling matters: an illegal bin that is merely excluded is a bin
// nobody notices when the constraint breaks.
//
// MEANINGLESS BY CONSTRUCTION (12 bins). A one-bit frame has no bit ORDER -- MSB-first and
// LSB-first produce identical traffic. Both bins fill, neither corresponds to anything a
// design could get wrong, and the second one is a copy of the first wearing a different label.
// SystemVerilog spells this `ignore_bins`, and confusing it with the previous category is how
// a real constraint failure gets filed as a known gap.
//
// SO THE FULL CROSS HAS TWO DEFECTS IN ONE NUMBER, and they pull in opposite directions:
//
// 72 bins total
// 6 unreachable by constraint -> the report tops out at 66/72 = 91.7%, forever
// 12 reachable but uninformative -> of the 66 it does fill, 18% carry no information
// 54 informative
//
// A report that cannot reach 100% becomes a standing agenda item. A report whose filled bins
// include copies of each other is worse, because it moves and the movement means nothing.
//
// AND THE ILLEGAL COMBINATION IS IN THE CURATED SET TOO. `mode x width` contains mode 3 crossed
// with a 32-bit frame, so 30 curated bins are 29 reachable ones. An `illegal_bins` declaration
// is needed wherever the axes meet, not once per coverage model -- which is the kind of detail
// that makes a curated set look like more work than a full cross until the first time somebody
// has to explain 8.3%.
//
// THE CURATED SET IS THREE TWO-WAY CROSSES, chosen because each names an interaction somebody
// can point at in the design:
//
// mode x width 12 bins the launch-edge arithmetic depends on both (14.3, 14.6)
// width x order 6 bins the bit-index mapping depends on both (14.3)
// mode x gap 12 bins the idle level and the gap interact at the select (16.4)
//
// Thirty bins, all reachable, and a number that can reach 100%. What it gives up is the
// four-way combinations -- and the argument for giving them up is not that they are worthless,
// it is that NOBODY CAN NAME A DESIGN MECHANISM THAT DEPENDS ON ALL FOUR. A cross whose bins
// cannot be traced to an interaction is a cross that generates runtime and status meetings in
// equal measure.
//
// THIS MODULE IMPLEMENTS BOTH so that they can be measured against the same stimulus, and it
// implements the illegal set as a CHECK rather than an exclusion, so that a constraint failure
// is reported rather than quietly absorbed.
`timescale 1ns/1ps
module spi_cov_model #(
parameter CNT_W = 16
) (
input wire clk,
input wire rst_n,
// One sampled transaction.
input wire sample,
input wire [1:0] mode, // {cpol, cpha}
input wire [1:0] width_c, // 0 = 1 bit, 1 = 8 bits, 2 = 32 bits
input wire order_c, // 0 = MSB-first, 1 = LSB-first
input wire [1:0] gap_c, // 0 = min, 1 = mid, 2 = long
input wire clr,
// The full four-way cross.
output wire [CNT_W-1:0] full_hit, // bins of the full cross that were reached
output wire [CNT_W-1:0] full_total, // 72
output wire [CNT_W-1:0] full_reachable, // 66 -- 72 less the 6 the constraint forbids
output wire [CNT_W-1:0] full_informative, // 54 -- 66 less the 12 that duplicate others
output wire [CNT_W-1:0] full_noise_hit, // how many of the FILLED bins are duplicates
// The curated two-way crosses.
output wire [CNT_W-1:0] cur_hit,
output wire [CNT_W-1:0] cur_total, // 30
output wire [CNT_W-1:0] cur_reachable, // 29 -- the illegal pair lives here too
// The check, not an exclusion: a transaction that lands in an illegal bin.
output reg [CNT_W-1:0] illegal_hits
);
localparam N_FULL = 72; // 4 * 3 * 2 * 3
localparam N_MW = 12; // mode x width
localparam N_WO = 6; // width x order
localparam N_MG = 12; // mode x gap
reg full_bin [0:N_FULL-1];
reg mw_bin [0:N_MW-1];
reg wo_bin [0:N_WO-1];
reg mg_bin [0:N_MG-1];
wire [6:0] full_idx = {2'b0, mode} * 7'd18 + {5'b0, width_c} * 7'd6
+ {6'b0, order_c} * 7'd3 + {5'b0, gap_c};
wire [3:0] mw_idx = {2'b0, mode} * 4'd3 + {2'b0, width_c};
wire [2:0] wo_idx = {1'b0, width_c} * 3'd2 + {2'b0, order_c};
wire [3:0] mg_idx = {2'b0, mode} * 4'd3 + {2'b0, gap_c};
// ILLEGAL BY SPECIFICATION: 32-bit frames are not supported in mode 3.
wire is_illegal = (mode == 2'd3) && (width_c == 2'd2);
// MEANINGLESS BY CONSTRUCTION: a one-bit frame has no bit order, so the LSB-first half of
// every one-bit row duplicates the MSB-first half and can never add information.
function integer meaningless;
input integer idx;
integer m, w, o;
begin
m = idx / 18;
w = (idx % 18) / 6;
o = (idx % 6) / 3;
meaningless = (w == 0 && o == 1) ? 1 : 0;
end
endfunction
function integer illegal_bin;
input integer idx;
integer m, w;
begin
m = idx / 18;
w = (idx % 18) / 6;
illegal_bin = (m == 3 && w == 2) ? 1 : 0;
end
endfunction
integer i;
reg [CNT_W-1:0] fh, ch, fr, fi, fn, cr;
always @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
for (i = 0; i < N_FULL; i = i + 1) full_bin[i] <= 1'b0;
for (i = 0; i < N_MW; i = i + 1) mw_bin[i] <= 1'b0;
for (i = 0; i < N_WO; i = i + 1) wo_bin[i] <= 1'b0;
for (i = 0; i < N_MG; i = i + 1) mg_bin[i] <= 1'b0;
illegal_hits <= {CNT_W{1'b0}};
end else begin
if (clr) begin
for (i = 0; i < N_FULL; i = i + 1) full_bin[i] <= 1'b0;
for (i = 0; i < N_MW; i = i + 1) mw_bin[i] <= 1'b0;
for (i = 0; i < N_WO; i = i + 1) wo_bin[i] <= 1'b0;
for (i = 0; i < N_MG; i = i + 1) mg_bin[i] <= 1'b0;
illegal_hits <= {CNT_W{1'b0}};
end else if (sample) begin
full_bin[full_idx] <= 1'b1;
mw_bin[mw_idx] <= 1'b1;
wo_bin[wo_idx] <= 1'b1;
mg_bin[mg_idx] <= 1'b1;
// THE ILLEGAL SET IS A CHECK. An excluded bin is invisible when the constraint
// that was supposed to prevent it breaks; a counted one is a test failure.
if (is_illegal) illegal_hits <= illegal_hits + 1'b1;
end
end
end
// Combinational tallies, so the bench can read coverage at any instant. Every number is
// DERIVED from the bin definitions rather than written down, so the arithmetic in the header
// is checked by the run instead of being trusted.
always @* begin
fh = {CNT_W{1'b0}};
fr = {CNT_W{1'b0}};
fi = {CNT_W{1'b0}};
fn = {CNT_W{1'b0}};
for (i = 0; i < N_FULL; i = i + 1) begin
if (full_bin[i]) fh = fh + 1'b1;
if (illegal_bin(i) == 0) fr = fr + 1'b1;
if (illegal_bin(i) == 0 && meaningless(i) == 0) fi = fi + 1'b1;
if (full_bin[i] && meaningless(i) == 1) fn = fn + 1'b1;
end
ch = {CNT_W{1'b0}};
cr = {CNT_W{1'b0}};
for (i = 0; i < N_MW; i = i + 1) begin
if (mw_bin[i]) ch = ch + 1'b1;
// The illegal pair is mode 3 crossed with the 32-bit width, which in this cross is
// index 3*3 + 2.
if (i != (3 * 3 + 2)) cr = cr + 1'b1;
end
for (i = 0; i < N_WO; i = i + 1) begin
if (wo_bin[i]) ch = ch + 1'b1;
cr = cr + 1'b1;
end
for (i = 0; i < N_MG; i = i + 1) begin
if (mg_bin[i]) ch = ch + 1'b1;
cr = cr + 1'b1;
end
end
assign full_hit = fh;
assign full_total = N_FULL[CNT_W-1:0];
assign full_reachable = fr;
assign full_informative = fi;
assign full_noise_hit = fn;
assign cur_hit = ch;
assign cur_total = (N_MW + N_WO + N_MG);
assign cur_reachable = cr;
endmoduleAzvya Education Pvt. Ltd.VLSI Mentor
spi_cov_model.vhd — the same design in VHDL
-- spi_cov_model.vhd
--
-- Chapter 17.3 -- functional coverage, and the difference between a cross that earns its runtime
-- and a cross that spends the project explaining itself.
--
-- THE TEMPTATION IS THE FULL CROSS. Four axes, so cross all four, and let the tool tell you what
-- is missing. It is one line, it is exhaustive, and it produces a number that can never reach
-- 100% -- after which the number stops being information and becomes a standing item on a status
-- call.
--
-- THE FOUR AXES.
--
-- mode CPOL/CPHA 4 bins
-- width 1, 8, 32 bits 3 bins
-- order MSB-first, LSB 2 bins
-- gap min, mid, long 3 bins
--
-- The full cross is 4 x 3 x 2 x 3 = 72 bins. Two groups of them can never be filled, and the two
-- groups are unfillable for DIFFERENT reasons, which is the distinction the chapter is about.
--
-- ILLEGAL BY SPECIFICATION (6 bins). This device does not support 32-bit frames in mode 3.
-- That is a sentence in a datasheet, the generator is constrained never to produce it, and a
-- bin that is reached would be a BUG IN THE TESTBENCH. SystemVerilog spells this
-- `illegal_bins`, and the spelling matters: an illegal bin that is merely excluded is a bin
-- nobody notices when the constraint breaks.
--
-- MEANINGLESS BY CONSTRUCTION (12 bins). A one-bit frame has no bit ORDER -- MSB-first and
-- LSB-first produce identical traffic. Both bins fill, neither corresponds to anything a
-- design could get wrong, and the second one is a copy of the first wearing a different label.
-- SystemVerilog spells this `ignore_bins`, and confusing it with the previous category is how
-- a real constraint failure gets filed as a known gap.
--
-- SO THE FULL CROSS HAS TWO DEFECTS IN ONE NUMBER, and they pull in opposite directions:
--
-- 72 bins total
-- 6 unreachable by constraint -> the report tops out at 66/72 = 91.7%, forever
-- 12 reachable but uninformative -> of the 66 it does fill, 18% carry no information
-- 54 informative
--
-- A report that cannot reach 100% becomes a standing agenda item. A report whose filled bins
-- include copies of each other is worse, because it moves and the movement means nothing.
--
-- AND THE ILLEGAL COMBINATION IS IN THE CURATED SET TOO. `mode x width` contains mode 3 crossed
-- with a 32-bit frame, so 30 curated bins are 29 reachable ones. An `illegal_bins` declaration
-- is needed wherever the axes meet, not once per coverage model -- which is the kind of detail
-- that makes a curated set look like more work than a full cross until the first time somebody
-- has to explain 8.3%.
--
-- THE CURATED SET IS THREE TWO-WAY CROSSES, chosen because each names an interaction somebody
-- can point at in the design:
--
-- mode x width 12 bins the launch-edge arithmetic depends on both (14.3, 14.6)
-- width x order 6 bins the bit-index mapping depends on both (14.3)
-- mode x gap 12 bins the idle level and the gap interact at the select (16.4)
--
-- Thirty bins, all reachable, and a number that can reach 100%. What it gives up is the
-- four-way combinations -- and the argument for giving them up is not that they are worthless,
-- it is that NOBODY CAN NAME A DESIGN MECHANISM THAT DEPENDS ON ALL FOUR. A cross whose bins
-- cannot be traced to an interaction is a cross that generates runtime and status meetings in
-- equal measure.
--
-- THIS MODULE IMPLEMENTS BOTH so that they can be measured against the same stimulus, and it
-- implements the illegal set as a CHECK rather than an exclusion, so that a constraint failure
-- is reported rather than quietly absorbed.
--
-- WHAT VHDL ADDS HERE: the coverage model is a PROTECTED TYPE, so the bin arrays are private
-- state reachable only through its own methods, and two processes sampling it cannot interleave
-- into each other. The SystemVerilog and Verilog versions expose their tallies as ports because
-- the simulator they are verified in has no class in this position -- and a coverage model whose
-- bins any process can write is a coverage model that will one day be credited for a transaction
-- that never happened.
--
-- The tallies are also methods rather than signals, so the arithmetic in the header is recomputed
-- from the bin definitions every time it is read. Nothing in this file states 66 or 54; both are
-- counted.
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
package spi_cov_pkg is
constant N_MODE : natural := 4;
constant N_WIDTH : natural := 3;
constant N_ORDER : natural := 2;
constant N_GAP : natural := 3;
constant N_FULL : natural := N_MODE * N_WIDTH * N_ORDER * N_GAP; -- 72
constant N_MW : natural := N_MODE * N_WIDTH; -- 12
constant N_WO : natural := N_WIDTH * N_ORDER; -- 6
constant N_MG : natural := N_MODE * N_GAP; -- 12
-- ILLEGAL BY SPECIFICATION: this device does not support 32-bit frames in mode 3.
constant ILLEGAL_MODE : natural := 3;
constant ILLEGAL_WIDTH : natural := 2;
type spi_cov_t is protected
procedure clear;
procedure sample (m : natural; w : natural; o : natural; g : natural);
impure function full_hit return natural;
impure function full_total return natural;
impure function full_reachable return natural;
impure function full_informative return natural;
impure function full_noise_hit return natural;
impure function cur_hit return natural;
impure function cur_total return natural;
impure function cur_reachable return natural;
impure function illegal_hits return natural;
end protected spi_cov_t;
end package spi_cov_pkg;
-- The protected BODY belongs in the package body, not the package declaration: declaring
-- both in the same place makes the type name doubly visible and the analyser refuses it.
package body spi_cov_pkg is
type spi_cov_t is protected body
type bool_arr_t is array (natural range <>) of boolean;
variable full_bin : bool_arr_t(0 to N_FULL - 1) := (others => false);
variable mw_bin : bool_arr_t(0 to N_MW - 1) := (others => false);
variable wo_bin : bool_arr_t(0 to N_WO - 1) := (others => false);
variable mg_bin : bool_arr_t(0 to N_MG - 1) := (others => false);
variable ill : natural := 0;
-- Index arithmetic, in one place, so the decoders below cannot drift from the encoders.
impure function full_index (m, w, o, g : natural) return natural is
begin
return ((m * N_WIDTH + w) * N_ORDER + o) * N_GAP + g;
end function;
impure function dec_mode (idx : natural) return natural is begin return idx / 18; end function;
impure function dec_width (idx : natural) return natural is begin return (idx mod 18) / 6; end function;
impure function dec_order (idx : natural) return natural is begin return (idx mod 6) / 3; end function;
-- Unreachable because the specification forbids the combination. A hit here is a
-- TESTBENCH BUG.
impure function is_illegal_bin (idx : natural) return boolean is
begin
return dec_mode(idx) = ILLEGAL_MODE and dec_width(idx) = ILLEGAL_WIDTH;
end function;
-- Reachable, and carrying no information: a one-bit frame has no bit order, so its
-- LSB-first bin is a copy of its MSB-first one.
impure function is_noise_bin (idx : natural) return boolean is
begin
return dec_width(idx) = 0 and dec_order(idx) = 1;
end function;
procedure clear is
begin
full_bin := (others => false);
mw_bin := (others => false);
wo_bin := (others => false);
mg_bin := (others => false);
ill := 0;
end procedure clear;
procedure sample (m : natural; w : natural; o : natural; g : natural) is
begin
full_bin(full_index(m, w, o, g)) := true;
mw_bin(m * N_WIDTH + w) := true;
wo_bin(w * N_ORDER + o) := true;
mg_bin(m * N_GAP + g) := true;
-- The illegal set is a CHECK. An excluded bin is invisible when the constraint that
-- was supposed to prevent it breaks; a counted one is a test failure.
if m = ILLEGAL_MODE and w = ILLEGAL_WIDTH then
ill := ill + 1;
end if;
end procedure sample;
impure function full_hit return natural is
variable n : natural := 0;
begin
for i in full_bin'range loop
if full_bin(i) then n := n + 1; end if;
end loop;
return n;
end function;
impure function full_total return natural is
begin
return N_FULL;
end function;
impure function full_reachable return natural is
variable n : natural := 0;
begin
for i in full_bin'range loop
if not is_illegal_bin(i) then n := n + 1; end if;
end loop;
return n;
end function;
impure function full_informative return natural is
variable n : natural := 0;
begin
for i in full_bin'range loop
if not is_illegal_bin(i) and not is_noise_bin(i) then n := n + 1; end if;
end loop;
return n;
end function;
impure function full_noise_hit return natural is
variable n : natural := 0;
begin
for i in full_bin'range loop
if full_bin(i) and is_noise_bin(i) then n := n + 1; end if;
end loop;
return n;
end function;
impure function cur_hit return natural is
variable n : natural := 0;
begin
for i in mw_bin'range loop if mw_bin(i) then n := n + 1; end if; end loop;
for i in wo_bin'range loop if wo_bin(i) then n := n + 1; end if; end loop;
for i in mg_bin'range loop if mg_bin(i) then n := n + 1; end if; end loop;
return n;
end function;
impure function cur_total return natural is
begin
return N_MW + N_WO + N_MG;
end function;
-- The illegal pair lives in `mode x width` too, so the curated set has one unreachable
-- bin of its own. An `illegal_bins` declaration belongs wherever the axes meet.
impure function cur_reachable return natural is
begin
return (N_MW - 1) + N_WO + N_MG;
end function;
impure function illegal_hits return natural is
begin
return ill;
end function;
end protected body spi_cov_t;
end package body spi_cov_pkg;The Bench
Azvya Education Pvt. Ltd.VLSI Mentor
spi_cov_model_tb.sv — one stimulus through two coverage models, and an illegal set proved to fire
// spi_cov_model_tb.sv
//
// ONE STIMULUS, TWO COVERAGE MODELS, AND A NUMBER THAT CANNOT REACH 100%.
//
// FOUR MEASUREMENTS.
//
// 1. THE ARITHMETIC OF THE FULL CROSS, DERIVED RATHER THAN ASSERTED. 72 bins, 6 unreachable
// because the specification forbids the combination, and 12 reachable but uninformative
// because a one-bit frame has no bit order and its second order bin is a copy of the first.
// 66 reachable, 54 informative -- and the model computes all three from its own bin
// definitions, so the header's arithmetic is checked by the run.
//
// 2. THE FULL CROSS PLATEAUS AND THE CURATED SET CLOSES. The curated set reaches all 29 of its
// reachable bins. The full cross reaches 66 of 72 and stops -- at 91.7% -- and stays there.
// Of the 66 it filled, 12 are duplicates, so 18% of its apparent progress carried no
// information. Two defects in one number, pulling in opposite directions.
//
// 3. CLOSURE COSTS DIFFERENT AMOUNTS. The transaction count at which each model closes is
// measured. The curated set is the cheaper one, and the difference is the runtime a full
// cross spends on combinations nobody can trace to a design mechanism.
//
// 4. THE ILLEGAL SET IS A CHECK AND IT FIRES. With the constraint in place, zero transactions
// land in an illegal bin. With the constraint deliberately removed, the count is non-zero.
// Both halves are required: an `illegal_bins` that has never fired is indistinguishable
// from an `ignore_bins`, and the difference between those two is the difference between a
// testbench bug and a known gap.
`timescale 1ns/1ps
module spi_cov_model_tb;
localparam int CNT_W = 16;
reg clk = 1'b0;
always #5 clk = ~clk;
reg rst_n = 1'b1;
reg sample = 1'b0;
reg [1:0] mode = 2'd0;
reg [1:0] width_c = 2'd0;
reg order_c = 1'b0;
reg [1:0] gap_c = 2'd0;
reg clr = 1'b0;
wire [CNT_W-1:0] full_hit, full_total, full_reachable, full_informative, full_noise_hit;
wire [CNT_W-1:0] cur_hit, cur_total, cur_reachable, illegal_hits;
spi_cov_model #(.CNT_W(CNT_W)) u_c (
.clk(clk), .rst_n(rst_n),
.sample(sample), .mode(mode), .width_c(width_c), .order_c(order_c), .gap_c(gap_c),
.clr(clr),
.full_hit(full_hit), .full_total(full_total), .full_reachable(full_reachable),
.full_informative(full_informative), .full_noise_hit(full_noise_hit),
.cur_hit(cur_hit), .cur_total(cur_total), .cur_reachable(cur_reachable),
.illegal_hits(illegal_hits)
);
integer errors = 0;
initial begin
#2_000_000;
$display("FAIL: the simulation did not finish within its time limit");
$finish;
end
// ------------------------------------------------------------------
// The generator. A seeded 32-bit xorshift, as in Chapter 16.2, so the stimulus is
// reproducible and the three language versions of this bench measure ONE experiment rather
// than three unrelated ones.
// ------------------------------------------------------------------
reg [31:0] rng = 32'h1234_5678;
function automatic [31:0] nxt(input [31:0] s);
reg [31:0] x;
begin
x = s;
x = x ^ (x << 13);
x = x ^ (x >> 17);
x = x ^ (x << 5);
nxt = x;
end
endfunction
// `constrained` selects whether the specification's restriction is applied. The illegal
// combination is mode 3 with a 32-bit frame.
task automatic gen_txn(input integer constrained);
integer tries;
begin
tries = 0;
rng = nxt(rng);
mode = rng[1:0];
width_c = (rng[9:8] == 2'd3) ? 2'd1 : rng[9:8];
order_c = rng[16];
gap_c = (rng[21:20] == 2'd3) ? 2'd1 : rng[21:20];
// REJECTION SAMPLING, with a bound. A constraint applied by redrawing needs a limit,
// or an over-constrained corner turns into a hang instead of a reported failure --
// which is Chapter 17.4's subject.
while (constrained != 0 && (mode == 2'd3) && (width_c == 2'd2) && tries < 32) begin
rng = nxt(rng);
width_c = (rng[9:8] == 2'd3) ? 2'd1 : rng[9:8];
tries = tries + 1;
end
@(negedge clk);
sample = 1'b1;
@(negedge clk);
sample = 1'b0;
end
endtask
task automatic clear_cov;
begin
@(negedge clk); clr = 1'b1;
@(negedge clk); clr = 1'b0;
@(negedge clk);
end
endtask
integer n, cur_closed_at, full_closed_at, full_plateau_at;
integer prev_full;
integer reach, tot, curt, info, curr;
integer illegal_with, illegal_without;
initial begin
rst_n = 1'b1;
@(negedge clk);
rst_n = 1'b0;
repeat (4) @(negedge clk);
rst_n = 1'b1;
repeat (4) @(negedge clk);
// ============================================================
// 1. THE ARITHMETIC.
// ============================================================
tot = full_total;
reach = full_reachable;
info = full_informative;
curt = cur_total;
curr = cur_reachable;
$display(" the four-way cross: %0d bins total", tot);
$display(" unreachable: illegal by specification (mode 3, 32-bit frame) ... %0d", tot - reach);
$display(" reachable .................................................... %0d", reach);
$display(" of which uninformative (a one-bit frame has no bit order) ..... %0d", reach - info);
$display(" informative .................................................. %0d", info);
$display(" the curated set: mode x width (12) + width x order (6) + mode x gap (12) = %0d bins,",
curt);
$display(" of which reachable ........................................... %0d (the illegal pair is in mode x width too)",
curr);
if (tot != 72 || reach != 66 || info != 54 || curt != 30 || curr != 29) begin
$display(" FAIL: the bin arithmetic is wrong -- total %0d, reachable %0d, informative %0d, curated %0d/%0d",
tot, reach, info, curr, curt);
errors = errors + 1;
end
$display(" 1. every one of those numbers is DERIVED from the bin definitions rather than written down. The full cross tops out at %0d of %0d -- %0d%% -- and only %0d of its bins can tell you anything, so its ceiling is %0d%% and its useful content is %0d%%. Two different defects, reported as one percentage",
reach, tot, (reach * 100) / tot, info, (reach * 100) / tot, (info * 100) / tot);
// ============================================================
// 2 + 3. RUN, AND MEASURE WHERE EACH MODEL CLOSES.
// ============================================================
clear_cov();
cur_closed_at = 0;
full_closed_at = 0;
full_plateau_at = 0;
prev_full = 0;
for (n = 1; n <= 4000; n = n + 1) begin
gen_txn(1);
if (cur_closed_at == 0 && cur_hit == cur_reachable) cur_closed_at = n;
if (full_hit != prev_full) begin
prev_full = full_hit;
full_plateau_at = n;
end
if (full_closed_at == 0 && full_hit == full_total) full_closed_at = n;
// Stop once the curated set has closed and the full cross has stopped moving for a
// long stretch -- there is nothing further to learn from a plateau.
if (cur_closed_at != 0 && (n - full_plateau_at) > 1500) n = 4000;
end
$display(" after %0d transactions:", 4000);
$display(" curated set ..... %0d of %0d reachable (%0d declared) closed at transaction %0d",
cur_hit, cur_reachable, cur_total, cur_closed_at);
$display(" full cross ...... %0d of %0d last new bin at transaction %0d",
full_hit, full_total, full_plateau_at);
$display(" of the full cross's %0d filled bins, %0d are duplicates of other filled bins",
full_hit, full_noise_hit);
if (cur_hit != cur_reachable) begin
$display(" FAIL: the curated set did not close (%0d of %0d reachable)", cur_hit, cur_reachable);
errors = errors + 1;
end
if (full_hit != reach) begin
$display(" FAIL: the full cross reached %0d bins where %0d are reachable", full_hit, reach);
errors = errors + 1;
end
if (full_noise_hit == 0) begin
$display(" FAIL: none of the uninformative bins were filled, so the 'progress that means nothing' claim is unmeasured");
errors = errors + 1;
end
if (full_closed_at != 0) begin
$display(" FAIL: the full cross reported 100%%, which means the illegal or meaningless bins are reachable after all");
errors = errors + 1;
end
$display(" 2. the curated set closed at %0d of %0d reachable. The full cross reached %0d of %0d -- %0d%% -- and its last new bin arrived at transaction %0d, after which it did not move for the remaining %0d transactions. It is not converging slowly; it is finished, below full. And %0d of the bins it did fill are copies of other filled bins, so %0d%% of its apparent progress carried no information at all",
cur_hit, cur_reachable, full_hit, full_total, (full_hit * 100) / full_total,
full_plateau_at, 4000 - full_plateau_at, full_noise_hit,
(full_noise_hit * 100) / full_hit);
$display(" 3. closure cost: the curated set at transaction %0d, the reachable part of the full cross at transaction %0d. The gap is runtime spent on four-way combinations that no design mechanism depends on -- and the argument for dropping them is not that they are worthless, it is that nobody can name the interaction they would exercise",
cur_closed_at, full_plateau_at);
// ============================================================
// 4. THE ILLEGAL SET IS A CHECK, AND IT FIRES.
// ============================================================
clear_cov();
for (n = 0; n < 400; n = n + 1) gen_txn(1);
illegal_with = illegal_hits;
clear_cov();
for (n = 0; n < 400; n = n + 1) gen_txn(0); // the constraint removed on purpose
illegal_without = illegal_hits;
$display(" illegal-bin hits in 400 transactions:");
$display(" with the specification's constraint applied ..... %0d", illegal_with);
$display(" with it deliberately removed ................... %0d", illegal_without);
if (illegal_with != 0) begin
$display(" FAIL: %0d transactions landed in an illegal bin while the constraint was applied",
illegal_with);
errors = errors + 1;
end
if (illegal_without == 0) begin
$display(" FAIL: removing the constraint produced no illegal transactions, so the illegal check has never been shown to fire and is indistinguishable from an exclusion");
errors = errors + 1;
end
$display(" 4. zero illegal hits with the constraint applied and %0d with it removed. Both halves are required: an illegal bin that has never fired is indistinguishable from an ignored one, and the difference between those two is the difference between a TESTBENCH BUG and a known gap",
illegal_without);
if (errors == 0)
$display("PASS: a cross is a claim that some interaction matters, and a cross whose bins cannot be traced to a design mechanism is a claim nobody made. The four-way cross of mode, width, order and gap has %0d bins, and it has TWO defects that pull in opposite directions. Six bins are illegal by specification -- this device does not support 32-bit frames in mode 3 -- so the report tops out at %0d and can never read full; and twelve of the bins it CAN fill are uninformative, because a one-bit frame has no bit order and its second order bin is a copy of the first, leaving %0d that can tell you anything. Measured: the cross reached %0d%% and stopped, with %0d of its filled bins duplicates of other filled bins -- %0d%% of its apparent progress carrying no information -- and its last new bin arriving at transaction %0d, after which the remaining %0d transactions added nothing at all. The curated alternative is three TWO-way crosses, each naming an interaction somebody can point at in the design -- mode x width for the launch-edge arithmetic, width x order for the bit-index mapping, mode x gap for the idle level at the select -- and it closed at %0d of %0d reachable bins out of %0d declared, at transaction %0d. Note that the illegal pair appears in the curated set too: an `illegal_bins` declaration belongs wherever the axes meet, not once per model. And the illegal set is implemented as a CHECK rather than an exclusion: zero hits with the specification's constraint applied and %0d with it deliberately removed, because an illegal bin that has never been seen to fire is indistinguishable from an ignored one, and those two categories differ by whether a hit is a testbench bug or a known gap",
tot, reach, info, (reach * 100) / tot, full_noise_hit,
(full_noise_hit * 100) / full_hit, full_plateau_at, 4000 - full_plateau_at,
cur_hit, cur_reachable, cur_total, cur_closed_at, illegal_without);
else
$display("FAIL: %0d error(s)", errors);
$finish;
end
endmoduleAzvya Education Pvt. Ltd.VLSI Mentor
spi_cov_model_tb.v — the same bench in Verilog-2001
// spi_cov_model_tb.v
//
// ONE STIMULUS, TWO COVERAGE MODELS, AND A NUMBER THAT CANNOT REACH 100%.
//
// FOUR MEASUREMENTS.
//
// 1. THE ARITHMETIC OF THE FULL CROSS, DERIVED RATHER THAN ASSERTED. 72 bins, 6 unreachable
// because the specification forbids the combination, and 12 reachable but uninformative
// because a one-bit frame has no bit order and its second order bin is a copy of the first.
// 66 reachable, 54 informative -- and the model computes all three from its own bin
// definitions, so the header's arithmetic is checked by the run.
//
// 2. THE FULL CROSS PLATEAUS AND THE CURATED SET CLOSES. The curated set reaches all 29 of its
// reachable bins. The full cross reaches 66 of 72 and stops -- at 91.7% -- and stays there.
// Of the 66 it filled, 12 are duplicates, so 18% of its apparent progress carried no
// information. Two defects in one number, pulling in opposite directions.
//
// 3. CLOSURE COSTS DIFFERENT AMOUNTS. The transaction count at which each model closes is
// measured. The curated set is the cheaper one, and the difference is the runtime a full
// cross spends on combinations nobody can trace to a design mechanism.
//
// 4. THE ILLEGAL SET IS A CHECK AND IT FIRES. With the constraint in place, zero transactions
// land in an illegal bin. With the constraint deliberately removed, the count is non-zero.
// Both halves are required: an `illegal_bins` that has never fired is indistinguishable
// from an `ignore_bins`, and the difference between those two is the difference between a
// testbench bug and a known gap.
`timescale 1ns/1ps
module spi_cov_model_tb;
localparam CNT_W = 16;
reg clk;
always #5 clk = ~clk;
reg rst_n;
reg sample;
reg [1:0] mode;
reg [1:0] width_c;
reg order_c;
reg [1:0] gap_c;
reg clr;
wire [CNT_W-1:0] full_hit, full_total, full_reachable, full_informative, full_noise_hit;
wire [CNT_W-1:0] cur_hit, cur_total, cur_reachable, illegal_hits;
spi_cov_model #(.CNT_W(CNT_W)) u_c (
.clk(clk), .rst_n(rst_n),
.sample(sample), .mode(mode), .width_c(width_c), .order_c(order_c), .gap_c(gap_c),
.clr(clr),
.full_hit(full_hit), .full_total(full_total), .full_reachable(full_reachable),
.full_informative(full_informative), .full_noise_hit(full_noise_hit),
.cur_hit(cur_hit), .cur_total(cur_total), .cur_reachable(cur_reachable),
.illegal_hits(illegal_hits)
);
integer errors;
initial begin
#2_000_000;
$display("FAIL: the simulation did not finish within its time limit");
$finish;
end
// ------------------------------------------------------------------
// The generator. A seeded 32-bit xorshift, as in Chapter 16.2, so the stimulus is
// reproducible and the three language versions of this bench measure ONE experiment rather
// than three unrelated ones.
// ------------------------------------------------------------------
reg [31:0] rng;
function [31:0] nxt;
input [31:0] s;
reg [31:0] x;
begin
x = s;
x = x ^ (x << 13);
x = x ^ (x >> 17);
x = x ^ (x << 5);
nxt = x;
end
endfunction
// `constrained` selects whether the specification's restriction is applied. The illegal
// combination is mode 3 with a 32-bit frame.
task gen_txn;
input integer constrained;
integer tries;
begin
tries = 0;
rng = nxt(rng);
mode = rng[1:0];
width_c = (rng[9:8] == 2'd3) ? 2'd1 : rng[9:8];
order_c = rng[16];
gap_c = (rng[21:20] == 2'd3) ? 2'd1 : rng[21:20];
// REJECTION SAMPLING, with a bound. A constraint applied by redrawing needs a limit,
// or an over-constrained corner turns into a hang instead of a reported failure --
// which is Chapter 17.4's subject.
while (constrained != 0 && (mode == 2'd3) && (width_c == 2'd2) && tries < 32) begin
rng = nxt(rng);
width_c = (rng[9:8] == 2'd3) ? 2'd1 : rng[9:8];
tries = tries + 1;
end
@(negedge clk);
sample = 1'b1;
@(negedge clk);
sample = 1'b0;
end
endtask
task clear_cov;
begin
@(negedge clk); clr = 1'b1;
@(negedge clk); clr = 1'b0;
@(negedge clk);
end
endtask
integer n, cur_closed_at, full_closed_at, full_plateau_at;
integer prev_full;
integer reach, tot, curt, info, curr;
integer illegal_with, illegal_without;
initial begin
rst_n = 1'b1;
@(negedge clk);
rst_n = 1'b0;
repeat (4) @(negedge clk);
rst_n = 1'b1;
repeat (4) @(negedge clk);
// ============================================================
// 1. THE ARITHMETIC.
// ============================================================
tot = full_total;
reach = full_reachable;
info = full_informative;
curt = cur_total;
curr = cur_reachable;
$display(" the four-way cross: %0d bins total", tot);
$display(" unreachable: illegal by specification (mode 3, 32-bit frame) ... %0d", tot - reach);
$display(" reachable .................................................... %0d", reach);
$display(" of which uninformative (a one-bit frame has no bit order) ..... %0d", reach - info);
$display(" informative .................................................. %0d", info);
$display(" the curated set: mode x width (12) + width x order (6) + mode x gap (12) = %0d bins,",
curt);
$display(" of which reachable ........................................... %0d (the illegal pair is in mode x width too)",
curr);
if (tot != 72 || reach != 66 || info != 54 || curt != 30 || curr != 29) begin
$display(" FAIL: the bin arithmetic is wrong -- total %0d, reachable %0d, informative %0d, curated %0d/%0d",
tot, reach, info, curr, curt);
errors = errors + 1;
end
$display(" 1. every one of those numbers is DERIVED from the bin definitions rather than written down. The full cross tops out at %0d of %0d -- %0d%% -- and only %0d of its bins can tell you anything, so its ceiling is %0d%% and its useful content is %0d%%. Two different defects, reported as one percentage",
reach, tot, (reach * 100) / tot, info, (reach * 100) / tot, (info * 100) / tot);
// ============================================================
// 2 + 3. RUN, AND MEASURE WHERE EACH MODEL CLOSES.
// ============================================================
clear_cov();
cur_closed_at = 0;
full_closed_at = 0;
full_plateau_at = 0;
prev_full = 0;
for (n = 1; n <= 4000; n = n + 1) begin
gen_txn(1);
if (cur_closed_at == 0 && cur_hit == cur_reachable) cur_closed_at = n;
if (full_hit != prev_full) begin
prev_full = full_hit;
full_plateau_at = n;
end
if (full_closed_at == 0 && full_hit == full_total) full_closed_at = n;
// Stop once the curated set has closed and the full cross has stopped moving for a
// long stretch -- there is nothing further to learn from a plateau.
if (cur_closed_at != 0 && (n - full_plateau_at) > 1500) n = 4000;
end
$display(" after %0d transactions:", 4000);
$display(" curated set ..... %0d of %0d reachable (%0d declared) closed at transaction %0d",
cur_hit, cur_reachable, cur_total, cur_closed_at);
$display(" full cross ...... %0d of %0d last new bin at transaction %0d",
full_hit, full_total, full_plateau_at);
$display(" of the full cross's %0d filled bins, %0d are duplicates of other filled bins",
full_hit, full_noise_hit);
if (cur_hit != cur_reachable) begin
$display(" FAIL: the curated set did not close (%0d of %0d reachable)", cur_hit, cur_reachable);
errors = errors + 1;
end
if (full_hit != reach) begin
$display(" FAIL: the full cross reached %0d bins where %0d are reachable", full_hit, reach);
errors = errors + 1;
end
if (full_noise_hit == 0) begin
$display(" FAIL: none of the uninformative bins were filled, so the 'progress that means nothing' claim is unmeasured");
errors = errors + 1;
end
if (full_closed_at != 0) begin
$display(" FAIL: the full cross reported 100%%, which means the illegal or meaningless bins are reachable after all");
errors = errors + 1;
end
$display(" 2. the curated set closed at %0d of %0d reachable. The full cross reached %0d of %0d -- %0d%% -- and its last new bin arrived at transaction %0d, after which it did not move for the remaining %0d transactions. It is not converging slowly; it is finished, below full. And %0d of the bins it did fill are copies of other filled bins, so %0d%% of its apparent progress carried no information at all",
cur_hit, cur_reachable, full_hit, full_total, (full_hit * 100) / full_total,
full_plateau_at, 4000 - full_plateau_at, full_noise_hit,
(full_noise_hit * 100) / full_hit);
$display(" 3. closure cost: the curated set at transaction %0d, the reachable part of the full cross at transaction %0d. The gap is runtime spent on four-way combinations that no design mechanism depends on -- and the argument for dropping them is not that they are worthless, it is that nobody can name the interaction they would exercise",
cur_closed_at, full_plateau_at);
// ============================================================
// 4. THE ILLEGAL SET IS A CHECK, AND IT FIRES.
// ============================================================
clear_cov();
for (n = 0; n < 400; n = n + 1) gen_txn(1);
illegal_with = illegal_hits;
clear_cov();
for (n = 0; n < 400; n = n + 1) gen_txn(0); // the constraint removed on purpose
illegal_without = illegal_hits;
$display(" illegal-bin hits in 400 transactions:");
$display(" with the specification's constraint applied ..... %0d", illegal_with);
$display(" with it deliberately removed ................... %0d", illegal_without);
if (illegal_with != 0) begin
$display(" FAIL: %0d transactions landed in an illegal bin while the constraint was applied",
illegal_with);
errors = errors + 1;
end
if (illegal_without == 0) begin
$display(" FAIL: removing the constraint produced no illegal transactions, so the illegal check has never been shown to fire and is indistinguishable from an exclusion");
errors = errors + 1;
end
$display(" 4. zero illegal hits with the constraint applied and %0d with it removed. Both halves are required: an illegal bin that has never fired is indistinguishable from an ignored one, and the difference between those two is the difference between a TESTBENCH BUG and a known gap",
illegal_without);
if (errors == 0)
$display("PASS: a cross is a claim that some interaction matters, and a cross whose bins cannot be traced to a design mechanism is a claim nobody made. The four-way cross of mode, width, order and gap has %0d bins, and it has TWO defects that pull in opposite directions. Six bins are illegal by specification -- this device does not support 32-bit frames in mode 3 -- so the report tops out at %0d and can never read full; and twelve of the bins it CAN fill are uninformative, because a one-bit frame has no bit order and its second order bin is a copy of the first, leaving %0d that can tell you anything. Measured: the cross reached %0d%% and stopped, with %0d of its filled bins duplicates of other filled bins -- %0d%% of its apparent progress carrying no information -- and its last new bin arriving at transaction %0d, after which the remaining %0d transactions added nothing at all. The curated alternative is three TWO-way crosses, each naming an interaction somebody can point at in the design -- mode x width for the launch-edge arithmetic, width x order for the bit-index mapping, mode x gap for the idle level at the select -- and it closed at %0d of %0d reachable bins out of %0d declared, at transaction %0d. Note that the illegal pair appears in the curated set too: an `illegal_bins` declaration belongs wherever the axes meet, not once per model. And the illegal set is implemented as a CHECK rather than an exclusion: zero hits with the specification's constraint applied and %0d with it deliberately removed, because an illegal bin that has never been seen to fire is indistinguishable from an ignored one, and those two categories differ by whether a hit is a testbench bug or a known gap",
tot, reach, info, (reach * 100) / tot, full_noise_hit,
(full_noise_hit * 100) / full_hit, full_plateau_at, 4000 - full_plateau_at,
cur_hit, cur_reachable, cur_total, cur_closed_at, illegal_without);
else
$display("FAIL: %0d error(s)", errors);
$finish;
end
initial begin
clk = 1'b0;
rst_n = 1'b1;
sample = 1'b0;
mode = 2'd0;
width_c = 2'd0;
order_c = 1'b0;
gap_c = 2'd0;
clr = 1'b0;
errors = 0;
rng = 32'h1234_5678;
end
endmoduleAzvya Education Pvt. Ltd.VLSI Mentor
spi_cov_model_tb.vhd — the same bench in VHDL
-- spi_cov_model_tb.vhd
--
-- ONE STIMULUS, TWO COVERAGE MODELS, AND A NUMBER THAT CANNOT REACH 100%.
--
-- FOUR MEASUREMENTS.
--
-- 1. THE ARITHMETIC OF THE FULL CROSS, DERIVED RATHER THAN ASSERTED. 72 bins, 6 unreachable
-- because the specification forbids the combination, and 12 reachable but uninformative
-- because a one-bit frame has no bit order and its second order bin is a copy of the first.
-- 66 reachable, 54 informative -- and the model computes all three from its own bin
-- definitions, so the header's arithmetic is checked by the run.
--
-- 2. THE FULL CROSS PLATEAUS AND THE CURATED SET CLOSES. The curated set reaches all 29 of its
-- reachable bins. The full cross reaches 66 of 72 and stops -- at 91.7% -- and stays there.
-- Of the 66 it filled, 12 are duplicates, so 18% of its apparent progress carried no
-- information. Two defects in one number, pulling in opposite directions.
--
-- 3. CLOSURE COSTS DIFFERENT AMOUNTS. The transaction count at which each model closes is
-- measured. The curated set is the cheaper one, and the difference is the runtime a full
-- cross spends on combinations nobody can trace to a design mechanism.
--
-- 4. THE ILLEGAL SET IS A CHECK AND IT FIRES. With the constraint in place, zero transactions
-- land in an illegal bin. With the constraint deliberately removed, the count is non-zero.
-- Both halves are required: an `illegal_bins` that has never fired is indistinguishable
-- from an `ignore_bins`, and the difference between those two is the difference between a
-- testbench bug and a known gap.
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
use work.spi_cov_pkg.all;
entity spi_cov_model_tb is
end entity spi_cov_model_tb;
architecture tb of spi_cov_model_tb is
shared variable cov : spi_cov_t;
signal errors : integer := 0;
begin
main : process is
-- The generator. A seeded 32-bit xorshift, as in Chapter 16.2, so the stimulus is
-- reproducible and the three language versions of this bench measure ONE experiment
-- rather than three unrelated ones.
variable rng : unsigned(31 downto 0) := x"12345678";
procedure step_rng is
begin
rng := rng xor shift_left(rng, 13);
rng := rng xor shift_right(rng, 17);
rng := rng xor shift_left(rng, 5);
end procedure step_rng;
variable m, w, o, g : natural;
-- `constrained` selects whether the specification's restriction is applied.
procedure gen_txn (constrained : boolean) is
variable tries : natural := 0;
begin
tries := 0;
step_rng;
m := to_integer(rng(1 downto 0));
if to_integer(rng(9 downto 8)) = 3 then w := 1;
else w := to_integer(rng(9 downto 8));
end if;
o := to_integer(rng(16 downto 16));
if to_integer(rng(21 downto 20)) = 3 then g := 1;
else g := to_integer(rng(21 downto 20));
end if;
-- REJECTION SAMPLING, with a bound. A constraint applied by redrawing needs a limit,
-- or an over-constrained corner turns into a hang instead of a reported failure --
-- which is Chapter 17.4's subject.
while constrained and m = ILLEGAL_MODE and w = ILLEGAL_WIDTH and tries < 32 loop
step_rng;
if to_integer(rng(9 downto 8)) = 3 then w := 1;
else w := to_integer(rng(9 downto 8));
end if;
tries := tries + 1;
end loop;
cov.sample(m, w, o, g);
end procedure gen_txn;
variable tot, reach, info, curt, curr : natural;
variable n : natural;
variable cur_closed_at, full_closed_at : natural := 0;
variable full_plateau_at, prev_full : natural := 0;
variable illegal_with, illegal_without : natural;
variable fh, fn, ch : natural;
constant NTXN : natural := 4000;
begin
-- ==============================================================
-- 1. THE ARITHMETIC.
-- ==============================================================
tot := cov.full_total;
reach := cov.full_reachable;
info := cov.full_informative;
curt := cov.cur_total;
curr := cov.cur_reachable;
report " the four-way cross: " & integer'image(tot) & " bins total";
report " unreachable: illegal by specification (mode 3, 32-bit frame) ... " &
integer'image(tot - reach);
report " reachable .................................................... " &
integer'image(reach);
report " of which uninformative (a one-bit frame has no bit order) ..... " &
integer'image(reach - info);
report " informative .................................................. " &
integer'image(info);
report " the curated set: mode x width (12) + width x order (6) + mode x gap (12) = " &
integer'image(curt) & " bins,";
report " of which reachable ........................................... " &
integer'image(curr) & " (the illegal pair is in mode x width too)";
if tot /= 72 or reach /= 66 or info /= 54 or curt /= 30 or curr /= 29 then
report " FAIL: the bin arithmetic is wrong";
errors <= errors + 1;
wait for 1 ns;
end if;
report " 1. every one of those numbers is DERIVED from the bin definitions rather than written down. The full cross tops out at " &
integer'image(reach) & " of " & integer'image(tot) & " -- " &
integer'image((reach * 100) / tot) & "% -- and only " & integer'image(info) &
" of its bins can tell you anything, so its ceiling is " &
integer'image((reach * 100) / tot) & "% and its useful content is " &
integer'image((info * 100) / tot) &
"%. Two different defects, reported as one percentage";
-- ==============================================================
-- 2 + 3. RUN, AND MEASURE WHERE EACH MODEL CLOSES.
-- ==============================================================
cov.clear;
n := 1;
while n <= NTXN loop
gen_txn(true);
if cur_closed_at = 0 and cov.cur_hit = curr then
cur_closed_at := n;
end if;
if cov.full_hit /= prev_full then
prev_full := cov.full_hit;
full_plateau_at := n;
end if;
if full_closed_at = 0 and cov.full_hit = tot then
full_closed_at := n;
end if;
-- Stop once the curated set has closed and the full cross has stopped moving for a
-- long stretch -- there is nothing further to learn from a plateau.
if cur_closed_at /= 0 and (n - full_plateau_at) > 1500 then
n := NTXN;
end if;
n := n + 1;
end loop;
fh := cov.full_hit;
fn := cov.full_noise_hit;
ch := cov.cur_hit;
report " after " & integer'image(NTXN) & " transactions:";
report " curated set ..... " & integer'image(ch) & " of " & integer'image(curr) &
" reachable (" & integer'image(curt) & " declared) closed at transaction " &
integer'image(cur_closed_at);
report " full cross ...... " & integer'image(fh) & " of " & integer'image(tot) &
" last new bin at transaction " &
integer'image(full_plateau_at);
report " of the full cross's " & integer'image(fh) & " filled bins, " &
integer'image(fn) & " are duplicates of other filled bins";
if ch /= curr then
report " FAIL: the curated set did not close";
errors <= errors + 1; wait for 1 ns;
end if;
if fh /= reach then
report " FAIL: the full cross reached " & integer'image(fh) &
" bins where " & integer'image(reach) & " are reachable";
errors <= errors + 1; wait for 1 ns;
end if;
if fn = 0 then
report " FAIL: none of the uninformative bins were filled, so the 'progress that means nothing' claim is unmeasured";
errors <= errors + 1; wait for 1 ns;
end if;
if full_closed_at /= 0 then
report " FAIL: the full cross reported 100%, which means the illegal bins are reachable after all";
errors <= errors + 1; wait for 1 ns;
end if;
report " 2. the curated set closed at " & integer'image(ch) & " of " &
integer'image(curr) & " reachable. The full cross reached " & integer'image(fh) &
" of " & integer'image(tot) & " -- " & integer'image((fh * 100) / tot) &
"% -- and its last new bin arrived at transaction " &
integer'image(full_plateau_at) & ", after which it did not move for the remaining " &
integer'image(NTXN - full_plateau_at) &
" transactions. It is not converging slowly; it is finished, below full. And " &
integer'image(fn) & " of the bins it did fill are copies of other filled bins, so " &
integer'image((fn * 100) / fh) &
"% of its apparent progress carried no information at all";
report " 3. closure cost: the curated set at transaction " &
integer'image(cur_closed_at) & ", the reachable part of the full cross at transaction " &
integer'image(full_plateau_at) &
". The gap is runtime spent on four-way combinations that no design mechanism depends on -- and the argument for dropping them is not that they are worthless, it is that nobody can name the interaction they would exercise";
-- ==============================================================
-- 4. THE ILLEGAL SET IS A CHECK, AND IT FIRES.
-- ==============================================================
cov.clear;
for i in 1 to 400 loop gen_txn(true); end loop;
illegal_with := cov.illegal_hits;
cov.clear;
for i in 1 to 400 loop gen_txn(false); end loop; -- the constraint removed on purpose
illegal_without := cov.illegal_hits;
report " illegal-bin hits in 400 transactions:";
report " with the specification's constraint applied ..... " & integer'image(illegal_with);
report " with it deliberately removed ................... " & integer'image(illegal_without);
if illegal_with /= 0 then
report " FAIL: transactions landed in an illegal bin while the constraint was applied";
errors <= errors + 1; wait for 1 ns;
end if;
if illegal_without = 0 then
report " FAIL: removing the constraint produced no illegal transactions, so the illegal check has never been shown to fire and is indistinguishable from an exclusion";
errors <= errors + 1; wait for 1 ns;
end if;
report " 4. zero illegal hits with the constraint applied and " &
integer'image(illegal_without) &
" with it removed. Both halves are required: an illegal bin that has never fired is indistinguishable from an ignored one, and the difference between those two is the difference between a TESTBENCH BUG and a known gap";
wait for 1 ns;
if errors = 0 then
report "PASS: a cross is a claim that some interaction matters, and a cross whose bins cannot be traced to a design mechanism is a claim nobody made. The four-way cross of mode, width, order and gap has " &
integer'image(tot) &
" bins, and it has TWO defects that pull in opposite directions. Six bins are illegal by specification -- this device does not support 32-bit frames in mode 3 -- so the report tops out at " &
integer'image(reach) &
" and can never read full; and twelve of the bins it CAN fill are uninformative, because a one-bit frame has no bit order and its second order bin is a copy of the first, leaving " &
integer'image(info) &
" that can tell you anything. Measured: the cross reached " &
integer'image((fh * 100) / tot) & "% and stopped, with " & integer'image(fn) &
" of its filled bins duplicates of other filled bins -- " &
integer'image((fn * 100) / fh) &
"% of its apparent progress carrying no information -- and its last new bin arriving at transaction " &
integer'image(full_plateau_at) & ", after which the remaining " &
integer'image(NTXN - full_plateau_at) &
" transactions added nothing at all. The curated alternative is three TWO-way crosses, each naming an interaction somebody can point at in the design -- mode x width for the launch-edge arithmetic, width x order for the bit-index mapping, mode x gap for the idle level at the select -- and it closed at " &
integer'image(ch) & " of " & integer'image(curr) & " reachable bins out of " &
integer'image(curt) & " declared, at transaction " & integer'image(cur_closed_at) &
". Note that the illegal pair appears in the curated set too: an `illegal_bins` declaration belongs wherever the axes meet, not once per model. And the illegal set is implemented as a CHECK rather than an exclusion: zero hits with the constraint applied and " &
integer'image(illegal_without) &
" with it deliberately removed, because an illegal bin that has never been seen to fire is indistinguishable from an ignored one"
severity note;
else
report "FAIL: " & integer'image(errors) & " error(s)" severity error;
end if;
wait for 10 ns;
std.env.stop;
end process main;
end architecture tb;7. The Same Model As A SystemVerilog Covergroup
Reviewed code, per Chapter 16.3's toolchain note. The interesting part is not the coverpoints — it is the three declarations that carry the distinctions section 2 is about.
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
class spi_cov extends uvm_subscriber #(spi_item);
`uvm_component_utils(spi_cov)
// THE CURATED MODEL. Three two-way crosses, each traceable to a design mechanism, and no
// four-way cross at all.
covergroup cg with function sample(spi_item t);
option.per_instance = 1;
// A name, so a coverage report names the claim rather than the coordinates.
option.name = "spi_curated";
cp_mode : coverpoint {t.cpol, t.cpha} { bins m[] = {[0:3]}; }
cp_width : coverpoint t.nbits {
bins one = {1};
bins byte_w = {8};
bins full = {32};
// No `default` bin. A default bin absorbs values nobody predicted and reports
// them as covered, which is the opposite of what a coverage model is for.
}
cp_order : coverpoint t.lsb_first { bins msb = {0}; bins lsb = {1}; }
cp_gap : coverpoint t.gap {
bins at_min = {GAP_MIN};
bins mid = {[GAP_MIN+1 : GAP_MIN+15]};
bins long = {[GAP_MIN+16 : $]};
}
// THE THREE CROSSES, each with the exclusions the axes make necessary.
x_mode_width : cross cp_mode, cp_width {
// ILLEGAL BY SPECIFICATION: a hit here is a TESTBENCH BUG, and `illegal_bins` makes it an
// error rather than a silently excluded square. This is the declaration whose absence lets
// a broken constraint pass unnoticed.
illegal_bins unsupported = binsof(cp_mode.m[3]) && binsof(cp_width.full);
}
x_width_order : cross cp_width, cp_order {
// MEANINGLESS BY CONSTRUCTION: a one-bit frame has no bit order, so its LSB-first square
// duplicates its MSB-first one. `ignore_bins` removes it from the denominator WITHOUT
// making a hit an error -- which is the whole difference from the line above.
ignore_bins no_order_at_one = binsof(cp_width.one);
}
x_mode_gap : cross cp_mode, cp_gap;
endgroup
function new(string name, uvm_component parent);
super.new(name, parent);
cg = new();
endfunction
// Sampled from the MONITOR's analysis port, not the sequencer's. A bin credited to a
// transaction that was generated but never reached the pins is a bin that lies -- Chapter
// 16.2's result, and the reason this class extends `uvm_subscriber` rather than hooking the
// sequencer.
function void write(spi_item t);
cg.sample(t);
endfunction
// AND THE SIGNOFF QUESTION, which a percentage cannot answer. `get_coverage()` already excludes
// the ignored bins and counts the illegal ones as errors, so a clean 100% here means something
// that a 91.7% on a full cross never could.
function void report_phase(uvm_phase phase);
`uvm_info("COV", $sformatf("curated coverage %.1f%%", cg.get_coverage()), UVM_LOW)
if (cg.get_coverage() < 100.0)
`uvm_warning("COV_OPEN", "the curated model did not close; every bin in it is reachable")
endfunction
endclassThe three declarations to read twice are illegal_bins, ignore_bins, and the absent default bin. The first makes a hit an error. The second removes a square from the denominator without making a hit an error. The third would have absorbed every value nobody predicted and reported it as covered.
8. Why a Verification Engineer Cares
Because a coverage number that cannot reach full destroys the usefulness of every coverage number in the project.
The practical test to apply to any cross before writing it: name the design mechanism whose failure this cross would expose. mode × width has one — the launch-edge arithmetic reads both. mode × width × order × gap does not, and the runtime it costs is real.
The second test is about what the model is allowed to absorb. A default bin reports unpredicted values as covered, which is the opposite of a coverage model's purpose; an illegal_bins that has never fired is indistinguishable from an ignore_bins; and an exclusion file that turns 40% into 100% has moved the argument somewhere nobody reads.
And the closure numbers here are worth remembering as an order of magnitude: 52 transactions against 372, for the same axes and the same generator. The difference is entirely in how many bins were asked for.
9. Why an FPGA or ASIC Engineer Cares
Because the illegal bin in this model is a line in your datasheet, and whether the verification team implemented it as illegal_bins or as an exclusion decides whether you find out when a constraint breaks.
If your device genuinely does not support a combination, say so in a form that makes a hit an error. A combination merely excluded from a coverage report is a combination the suite may be driving at your silicon while reporting clean.
And the meaningless-bin category is worth checking against your own specification. A one-bit frame having no bit order is obvious once stated; the equivalent in a wider protocol — a field that is don't-care in one mode, a strobe that has no meaning at one width — is the same shape and much less obvious, and it inflates a denominator nobody can ever fill.
10. Failure Signature — A Coverage Number That Has Been 91% For Six Months
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
Symptom coverage sits at 91.7%. The regression is extended, the
constraints are tuned, and a seed sweep is run. It stays at
91.7%. The missing bins are reported and discussed monthly.
What happened the missing 8.3% is a four-way cross's illegal squares --
combinations the specification forbids and the generator is
correctly constrained never to produce. The number cannot
move.
What would have declaring them as `illegal_bins` at the time the cross was
caught it written, which removes them from the denominator AND makes a
hit an error. The two effects are the point: the number can
then reach full, and a broken constraint is still caught.
The tell the unreachable bins are all in ONE cross and they share a
coordinate. Bins missing because of unlucky randomisation
are scattered and move between seeds; bins missing because
of a constraint share a value on some axis and never move.
Changing the seed is the one-run diagnostic.11. Common Misconceptions
"Cross everything and let the tool tell you what is missing." The tool will tell you, every month, that the same 8.3% is missing. A cross is a claim that an interaction matters; crossing everything claims every interaction matters and defends none of them.
"Unreachable and uninformative are the same problem." They are opposite problems. Unreachable bins cap the number below full; uninformative bins fill, which makes the number move for no reason. This chapter's model measures both, and the first version of it conflated them and predicted the plateau in the wrong place.
"illegal_bins and ignore_bins are both ways to exclude a square." illegal_bins makes a hit an error; ignore_bins merely removes the square from the denominator. Using the second where the first was meant is how a broken constraint drives illegal traffic at a device while the report stays clean.
"A default bin catches the cases we did not think of." It reports them as covered. A coverage model exists to say what was reached against a list somebody defended; a default bin absorbs everything outside that list and credits it.
"More bins is more rigour." More bins is more runtime and a lower number. Rigour is whether each bin traces to a mechanism — and the curated model here closed in 52 transactions where the exhaustive one was still reporting the same figure after four thousand.
"An exclusion file is bookkeeping." An exclusion that turns 40% into 100% is the entire coverage argument, relocated into a file that is reviewed once. Exclusions need a category, a reason, and — for the waived ones — a date and an owner.
12. Reason It Through
The full cross plateaus at 66 of 72 and 12 of the 66 are duplicates. Which of those two numbers is the more dangerous, and why?
The 12. The 6 unreachable bins make the number obviously wrong and prompt an investigation. The 12 duplicates make the number move — coverage climbs as they fill — so progress is reported that corresponds to nothing a design could get wrong, and nobody investigates a rising number.
Predict what a seed change does to each of the two missing groups.
Nothing, in both cases — but for different reasons, and that is why the diagnostic works on a different group. The 6 illegal bins never fill under any seed because a constraint forbids them. Bins missing from unlucky randomisation do move between seeds. So a hole that does not move across several seeds is structural — either a constraint or an absent field — and a hole that moves is a distribution problem.
Why does illegal_bins have to make a hit an error rather than simply excluding the square?
Because the constraint that prevents it can break. If the square is merely excluded, a generator that starts producing the forbidden combination drives illegal traffic at the device and the coverage report is unaffected. Making a hit an error turns a broken constraint into a test failure — which is exactly what this chapter measures by removing the constraint on purpose and requiring 22 hits.
The curated set has 30 declared bins and 29 reachable ones. Where is the missing one and what does that tell you about where exclusions belong?
In mode × width: mode 3 crossed with the 32-bit frame, the same combination the specification forbids. Exclusions belong wherever the axes meet, not once per coverage model — a curated set does not escape the declaration just because it is smaller.
Give the one-sentence test to apply before adding any cross.
Name the design mechanism whose failure this cross would expose; if you cannot, the cross is runtime and a status-meeting agenda item rather than coverage.
13. Understanding Check
14. Summary
A four-way cross of mode, width, order and gap has 72 bins and two defects pulling in opposite directions. Six bins are illegal by specification — this device does not support 32-bit frames in mode 3 — so the report tops out at 66 and can never read full; and twelve of the bins it can fill are uninformative, because a one-bit frame has no bit order and its second order bin is a copy of the first, leaving 54 that can tell you anything. Measured: the cross reached 91% and stopped, with 12 of its filled bins duplicates of other filled bins — 18% of its apparent progress carrying no information — and its last new bin arriving at transaction 372, after which 3,628 further transactions added nothing. The curated alternative is three two-way crosses, each naming an interaction somebody can point at in the design, and it closed 29 of 29 reachable bins at transaction 52. The illegal pair appears in the curated set too: an illegal_bins declaration belongs wherever the axes meet. And the illegal set is implemented as a check rather than an exclusion — zero hits with the constraint applied and 22 with it deliberately removed — because an illegal bin that has never been seen to fire is indistinguishable from an ignored one, and those two categories differ by whether a hit is a testbench bug or a known gap.
15. What Comes Next
The coverage model says what was reached. Chapter 17.4 turns to what reaches it, and measures two ways a constraint set fails that a coverage report cannot tell apart.
Continue learning
Related tutorials
- Related topic
Full-Duplex Exchange
Every SPI transfer moves a bit in both directions on every edge, whether the software wanted it to or not. Where dummy bytes come from, why bytes received during a command phase exist but mean nothing, and why read and write are interpretations rather than modes.
- Related topic
Slave Output Valid Timing
How long after a clock edge a peripheral may take before MISO is trustworthy. What clock-to-output includes, why the datasheet number is conditional on a load your board probably exceeds, and why it dominates the return-path budget.
- Related topic
CPHA — Clock Phase
The second mode bit: which logical edge carries the sample role, why that is independent of polarity, and why one of its values forces a transmitter to place its first bit before any clock edge exists.
- Related topic
Mode 1 (CPOL=0, CPHA=1)
The same clock as Mode 0 with the phase flipped, and every implementation difficulty removed. Why the edge accounting balances exactly, what that means for the last bit, and where Mode 1 is genuinely used.