UART · Module 15
Functional Coverage and Configuration Crosses
A coverage model built from requirements rather than convenience, why the cross matters and the marginals do not, reporting holes instead of percentages, and a real defect that survived a full suite because one bin was empty.
Coverage answers exactly one question: was this situation ever reached? It does not say whether anything checked the outcome, and the two get conflated constantly — usually by a number on a slide.
This chapter builds the model, shows why the cross is the only part worth tracking, replaces the percentage with a list of holes, and ends with a defect that survived sixty-two passing checks and four thousand clocks of random traffic because one bin was never hit.
1. Bins Come From Requirements, Not From Convenience
The easy way to build a coverage model is to bin whatever is already a variable. That produces a model that measures the testbench rather than the design.
Every axis here is one that Chapter 14.1's requirement list makes behaviour depend on:
| Axis | Bins | The requirement it serves |
|---|---|---|
| data width | 5, 6, 7, 8, 9 | R1 — the frame is defined per width |
| parity mode | none, even, odd, mark | R2 — parity is defined per mode |
| stop length | 1, 1.5, 2 bits | R1 — a legal configuration axis |
| injected fault | none, parity, stop, short stop, glitch, break | R4, R5, R7 — each needs its own injection |
| baud offset | −10% … +10% in 1% steps | the one continuous axis with a budget |
| FIFO level | empty, 1, middle, DEPTH−1, full | the boundaries, not the interior |
The FIFO level row is the model in miniature. Binning the level 0–16 as sixteen bins would say almost nothing: the interesting values are the ones where behaviour changes, and everything from 2 to 14 behaves identically. Five bins, four of which are boundaries.
2. The Cross Is the Model; the Marginals Are Decoration
Five widths, four parity modes and three stop lengths is sixty distinct frame formats. Hitting every width and every parity mode separately takes five frames and tells you nothing about whether nine-bit-with-mark-parity was ever sent.
The suite demonstrates this deliberately:
// Six samples. Every marginal axis is fully covered.
cov.cov_sample(5, 0, 2, 0, 10);
cov.cov_sample(6, 1, 3, 0, 10);
cov.cov_sample(7, 2, 4, 0, 10);
cov.cov_sample(8, 3, 2, 0, 10);
cov.cov_sample(9, 0, 3, 0, 10);
cov.cov_sample(9, 1, 4, 0, 10); PASS every width was exercised
PASS and every parity mode
PASS and every stop length
PASS yet 54 of the 60 CROSS bins were never reachedEvery marginal bin full. Ninety percent of the cross empty. A report tracking only the marginals would call that closed.
3. The Model
//===========================================================================
// uart_cov_v — the functional coverage model of Chapter 15.3,
// in Verilog-2001
//
// NOT SYNTHESIZABLE. A coverage collector records WHAT WAS EXERCISED. It
// checks nothing, and confusing the two is the most common mistake in this
// area: a full bin says the condition was reached, not that anything
// verified the outcome.
//
// VERILOG-2001 HAS NO COVERGROUPS. SystemVerilog writes
//
// covergroup cg @(posedge clk);
// cp_parity : coverpoint parity { bins none = {0}; ... }
// x_fmt : cross cp_width, cp_parity, cp_stop;
// endgroup
//
// and gets bin management, crosses and a percentage for free. Here the bins
// are arrays and the cross is a three-dimensional one. The arithmetic is the
// same; what is lost is the tool's report, which is why the report task
// below has to be written by hand -- and, as it turns out, why it ends up
// better. See the note on holes.
//
// THE BINS ARE CHOSEN FROM REQUIREMENTS, NOT FROM CONVENIENCE. Every axis
// here is one that Chapter 14.1's requirement list makes behaviour depend
// on: the frame format (R1, R2), the injected fault (R4, R5, R7), and the
// baud offset, which is the one continuous axis the design has a budget for.
//===========================================================================
`timescale 1ns/1ps
module uart_cov_v #(
parameter NBAUD = 21, // baud-offset bins, -10% .. +10%
parameter DEPTH = 16 // FIFO depth, for the level bins
) ();
//---- the axes ---------------------------------------------------------
integer cov_width [5:9]; // data bits
integer cov_parity [0:3]; // none / even / odd / mark
integer cov_stop [2:4]; // stop length in half-bits
integer cov_err [0:5]; // ERR_NONE .. ERR_BREAK
integer cov_baud [0:NBAUD-1];
integer cov_lvl [0:4]; // empty / 1 / mid / DEPTH-1 / full
//---- the cross that actually matters ----------------------------------
// Marginal coverage is nearly free and nearly useless: hitting every
// width and every parity mode separately says nothing about whether
// 9-bit-with-mark-parity was ever sent. Only the cross does.
integer cov_fmt [5:9][0:3][2:4];
integer n_samples;
task cov_reset;
integer a, b, c;
begin
for (a = 5; a <= 9; a = a + 1) cov_width[a] = 0;
for (a = 0; a <= 3; a = a + 1) cov_parity[a] = 0;
for (a = 2; a <= 4; a = a + 1) cov_stop[a] = 0;
for (a = 0; a <= 5; a = a + 1) cov_err[a] = 0;
for (a = 0; a < NBAUD; a = a + 1) cov_baud[a] = 0;
for (a = 0; a <= 4; a = a + 1) cov_lvl[a] = 0;
for (a = 5; a <= 9; a = a + 1)
for (b = 0; b <= 3; b = b + 1)
for (c = 2; c <= 4; c = c + 1) cov_fmt[a][b][c] = 0;
n_samples = 0;
end
endtask
// Sample one frame. Called from the testbench with the description the
// stimulus used -- NOT with anything read back out of the design. A
// coverage model fed from the DUT records what the DUT did, which is a
// different and much less useful thing.
task cov_sample;
input integer nb; // 5..9
input integer pm; // 0..3
input integer sh; // 2..4
input integer er; // 0..5
input integer baud_idx; // 0..NBAUD-1
begin
cov_width[nb] = cov_width[nb] + 1;
cov_parity[pm] = cov_parity[pm] + 1;
cov_stop[sh] = cov_stop[sh] + 1;
cov_err[er] = cov_err[er] + 1;
if (baud_idx >= 0 && baud_idx < NBAUD)
cov_baud[baud_idx] = cov_baud[baud_idx] + 1;
cov_fmt[nb][pm][sh] = cov_fmt[nb][pm][sh] + 1;
n_samples = n_samples + 1;
end
endtask
// FIFO occupancy, sampled separately because it is not a per-frame fact.
task cov_sample_level;
input integer lvl;
begin
if (lvl == 0) cov_lvl[0] = cov_lvl[0] + 1;
else if (lvl == 1) cov_lvl[1] = cov_lvl[1] + 1;
else if (lvl == DEPTH) cov_lvl[4] = cov_lvl[4] + 1;
else if (lvl == DEPTH-1) cov_lvl[3] = cov_lvl[3] + 1;
else cov_lvl[2] = cov_lvl[2] + 1;
end
endtask
//---- the report -------------------------------------------------------
// It prints HOLES, not a percentage, and the difference is not cosmetic.
// "87% format coverage" is a number for a slide. "width=9 parity=MARK
// stop=1.5 never driven" is a line of stimulus somebody can write this
// afternoon.
integer fmt_holes, axis_holes;
task cov_report;
integer a, b, c;
begin
fmt_holes = 0;
axis_holes = 0;
for (a = 5; a <= 9; a = a + 1)
if (cov_width[a] == 0) begin
axis_holes = axis_holes + 1;
$display(" HOLE: data width %0d never driven", a);
end
for (a = 0; a <= 3; a = a + 1)
if (cov_parity[a] == 0) begin
axis_holes = axis_holes + 1;
$display(" HOLE: parity mode %0d never driven", a);
end
for (a = 2; a <= 4; a = a + 1)
if (cov_stop[a] == 0) begin
axis_holes = axis_holes + 1;
$display(" HOLE: stop length %0d half-bits never driven", a);
end
for (a = 0; a <= 5; a = a + 1)
if (cov_err[a] == 0) begin
axis_holes = axis_holes + 1;
$display(" HOLE: error kind %0d never injected", a);
end
for (a = 0; a <= 4; a = a + 1)
if (cov_lvl[a] == 0) begin
axis_holes = axis_holes + 1;
$display(" HOLE: FIFO level bin %0d never reached", a);
end
for (a = 5; a <= 9; a = a + 1)
for (b = 0; b <= 3; b = b + 1)
for (c = 2; c <= 4; c = c + 1)
if (cov_fmt[a][b][c] == 0) begin
fmt_holes = fmt_holes + 1;
$display(" HOLE: width=%0d parity=%0d stop=%0d never driven",
a, b, c);
end
$display(" coverage: %0d samples, format cross %0d of 60 bins hit, %0d holes",
n_samples, 60 - fmt_holes, fmt_holes);
end
endtask
initial cov_reset;
endmoduleSystemVerilog would express all of this as a covergroup with coverpoint and cross, and get bin management and a percentage for free. Icarus Verilog 13.0 does not implement covergroups, so the SystemVerilog model is the same arrays with int and logic — confirmed by trying, not assumed.
//===========================================================================
// uart_cov — the functional coverage model of Chapter 15.3,
// in SystemVerilog
//
// NOT SYNTHESIZABLE. A coverage collector records WHAT WAS EXERCISED. It
// checks nothing, and confusing the two is the most common mistake in this
// area: a full bin says the condition was reached, not that anything
// verified the outcome.
//
// SYSTEMSYSTEMVERILOG HAS COVERGROUPS, AND THIS FILE CANNOT USE THEM. SystemVerilog writes
//
// covergroup cg @(posedge clk);
// cp_parity : coverpoint parity { bins none = {0}; ... }
// x_fmt : cross cp_width, cp_parity, cp_stop;
// endgroup
//
// and gets bin management, crosses and a percentage for free. Here the bins
// are arrays and the cross is a three-dimensional one. The arithmetic is the
// same; what is lost is the tool's report, which is why the report task
// below has to be written by hand -- and, as it turns out, why it ends up
// better. See the note on holes.
//
// THE BINS ARE CHOSEN FROM REQUIREMENTS, NOT FROM CONVENIENCE. Every axis
// here is one that Chapter 14.1's requirement list makes behaviour depend
// on: the frame format (R1, R2), the injected fault (R4, R5, R7), and the
// baud offset, which is the one continuous axis the design has a budget for.
//===========================================================================
`timescale 1ns/1ps
module uart_cov #(
parameter NBAUD = 21, // baud-offset bins, -10% .. +10%
parameter DEPTH = 16 // FIFO depth, for the level bins
) ();
//---- the axes ---------------------------------------------------------
int cov_width [5:9]; // data bits
int cov_parity [0:3]; // none / even / odd / mark
int cov_stop [2:4]; // stop length in half-bits
int cov_err [0:5]; // ERR_NONE .. ERR_BREAK
int cov_baud [0:NBAUD-1];
int cov_lvl [0:4]; // empty / 1 / mid / DEPTH-1 / full
//---- the cross that actually matters ----------------------------------
// Marginal coverage is nearly free and nearly useless: hitting every
// width and every parity mode separately says nothing about whether
// 9-bit-with-mark-parity was ever sent. Only the cross does.
int cov_fmt [5:9][0:3][2:4];
int n_samples;
task cov_reset;
int a, b, c;
begin
for (a = 5; a <= 9; a = a + 1) cov_width[a] = 0;
for (a = 0; a <= 3; a = a + 1) cov_parity[a] = 0;
for (a = 2; a <= 4; a = a + 1) cov_stop[a] = 0;
for (a = 0; a <= 5; a = a + 1) cov_err[a] = 0;
for (a = 0; a < NBAUD; a = a + 1) cov_baud[a] = 0;
for (a = 0; a <= 4; a = a + 1) cov_lvl[a] = 0;
for (a = 5; a <= 9; a = a + 1)
for (b = 0; b <= 3; b = b + 1)
for (c = 2; c <= 4; c = c + 1) cov_fmt[a][b][c] = 0;
n_samples = 0;
end
endtask
// Sample one frame. Called from the testbench with the description the
// stimulus used -- NOT with anything read back out of the design. A
// coverage model fed from the DUT records what the DUT did, which is a
// different and much less useful thing.
task cov_sample;
input int nb; // 5..9
input int pm; // 0..3
input int sh; // 2..4
input int er; // 0..5
input int baud_idx; // 0..NBAUD-1
begin
cov_width[nb] = cov_width[nb] + 1;
cov_parity[pm] = cov_parity[pm] + 1;
cov_stop[sh] = cov_stop[sh] + 1;
cov_err[er] = cov_err[er] + 1;
if (baud_idx >= 0 && baud_idx < NBAUD)
cov_baud[baud_idx] = cov_baud[baud_idx] + 1;
cov_fmt[nb][pm][sh] = cov_fmt[nb][pm][sh] + 1;
n_samples++;
end
endtask
// FIFO occupancy, sampled separately because it is not a per-frame fact.
task cov_sample_level;
input int lvl;
begin
if (lvl == 0) cov_lvl[0] = cov_lvl[0] + 1;
else if (lvl == 1) cov_lvl[1] = cov_lvl[1] + 1;
else if (lvl == DEPTH) cov_lvl[4] = cov_lvl[4] + 1;
else if (lvl == DEPTH-1) cov_lvl[3] = cov_lvl[3] + 1;
else cov_lvl[2] = cov_lvl[2] + 1;
end
endtask
//---- the report -------------------------------------------------------
// It prints HOLES, not a percentage, and the difference is not cosmetic.
// "87% format coverage" is a number for a slide. "width=9 parity=MARK
// stop=1.5 never driven" is a line of stimulus somebody can write this
// afternoon.
int fmt_holes, axis_holes;
task cov_report;
int a, b, c;
begin
fmt_holes = 0;
axis_holes = 0;
for (a = 5; a <= 9; a = a + 1)
if (cov_width[a] == 0) begin
axis_holes++;
$display(" HOLE: data width %0d never driven", a);
end
for (a = 0; a <= 3; a = a + 1)
if (cov_parity[a] == 0) begin
axis_holes++;
$display(" HOLE: parity mode %0d never driven", a);
end
for (a = 2; a <= 4; a = a + 1)
if (cov_stop[a] == 0) begin
axis_holes++;
$display(" HOLE: stop length %0d half-bits never driven", a);
end
for (a = 0; a <= 5; a = a + 1)
if (cov_err[a] == 0) begin
axis_holes++;
$display(" HOLE: error kind %0d never injected", a);
end
for (a = 0; a <= 4; a = a + 1)
if (cov_lvl[a] == 0) begin
axis_holes++;
$display(" HOLE: FIFO level bin %0d never reached", a);
end
for (a = 5; a <= 9; a = a + 1)
for (b = 0; b <= 3; b = b + 1)
for (c = 2; c <= 4; c = c + 1)
if (cov_fmt[a][b][c] == 0) begin
fmt_holes++;
$display(" HOLE: width=%0d parity=%0d stop=%0d never driven",
a, b, c);
end
$display(" coverage: %0d samples, format cross %0d of 60 bins hit, %0d holes",
n_samples, 60 - fmt_holes, fmt_holes);
end
endtask
initial cov_reset;
endmoduleVHDL has no covergroups either, so the bins are arrays again — but VHDL is the one language here that gets a second, complementary mechanism, and it is not in the model at all. PSL cover directives, which NVC runs, cover sequences:
-- In uart_fifo_assert.vhd. This is not a value being binned -- it is a
-- temporal shape: "the FIFO went from full to empty at least once".
-- No array of counters expresses it.
-- psl c_fill_drain : cover {full_i = '1'; (full_i = '0')[*]; empty_i = '1'}
-- report "COVER: the FIFO went from full to empty";The two are complementary: arrays for the configuration cross, PSL cover for the temporal shapes. A model built only from arrays cannot say whether a fill-then-drain ever happened, only how many times each level was observed.
--===========================================================================
-- uart_cov_pkg — the functional coverage model of Chapter 15.3,
-- in VHDL-2008
--
-- NOT SYNTHESIZABLE. A coverage collector records WHAT WAS EXERCISED. It
-- checks nothing, and confusing the two is the commonest mistake in this
-- area: a full bin says the condition was reached, not that anything
-- verified the outcome.
--
-- VHDL HAS NO COVERGROUPS EITHER, so the bins are arrays -- the same shape
-- as the Verilog-2001 model. What VHDL does have, and neither Verilog file
-- can use on this toolchain, is PSL `cover`:
--
-- -- psl c_fill_drain : cover {full = '1'; (full = '0')[*]; empty = '1'};
--
-- That is a SEQUENCE being covered, not a value -- "the FIFO went from full
-- to empty at least once" -- and no array of counters expresses it. The two
-- mechanisms are complementary: arrays for the configuration cross, PSL
-- cover for the temporal shapes. See uart_fifo_assert.vhd for the latter.
--
-- THE BINS ARE CHOSEN FROM REQUIREMENTS, NOT FROM CONVENIENCE. Every axis
-- is one that Chapter 14.1's requirement list makes behaviour depend on.
--
-- It is a PACKAGE with a shared record rather than an entity, because a
-- coverage model is called from wherever stimulus is generated -- VHDL has
-- no cross-entity subprogram call, and an entity would force every sample
-- through a port.
--===========================================================================
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
package uart_cov_pkg is
constant NBAUD : positive := 21; -- baud-offset bins, -10% .. +10%
constant DEPTH : positive := 16;
type width_arr is array (5 to 9) of natural;
type parity_arr is array (0 to 3) of natural;
type stop_arr is array (2 to 4) of natural;
type err_arr is array (0 to 5) of natural;
type baud_arr is array (0 to NBAUD-1) of natural;
type lvl_arr is array (0 to 4) of natural;
-- The cross that actually matters. Marginal coverage is nearly free and
-- nearly useless: hitting every width and every parity mode separately
-- says nothing about whether 9-bit-with-mark-parity was ever sent.
type fmt_stop_arr is array (2 to 4) of natural;
type fmt_parity_arr is array (0 to 3) of fmt_stop_arr;
type fmt_arr is array (5 to 9) of fmt_parity_arr;
type cov_t is record
width : width_arr;
parity : parity_arr;
stop : stop_arr;
err : err_arr;
baud : baud_arr;
lvl : lvl_arr;
fmt : fmt_arr;
samples : natural;
end record cov_t;
procedure cov_reset(variable c : inout cov_t);
-- Sample one frame, with the description the STIMULUS used -- never with
-- anything read back out of the design. A coverage model fed from the DUT
-- records what the DUT did, which is a different and much less useful
-- thing.
procedure cov_sample(variable c : inout cov_t;
constant nb, pm, sh, er, baud_idx : in natural);
procedure cov_sample_level(variable c : inout cov_t; constant lvl : in natural);
-- Prints HOLES, not a percentage. "87% format coverage" is a number for a
-- slide; "width=9 parity=MARK stop=1.5 never driven" is a line of
-- stimulus somebody can write this afternoon.
procedure cov_report(variable c : inout cov_t;
variable fmt_holes : out natural;
variable axis_holes : out natural);
end package uart_cov_pkg;
package body uart_cov_pkg is
procedure cov_reset(variable c : inout cov_t) is
begin
c.width := (others => 0);
c.parity := (others => 0);
c.stop := (others => 0);
c.err := (others => 0);
c.baud := (others => 0);
c.lvl := (others => 0);
c.fmt := (others => (others => (others => 0)));
c.samples := 0;
end procedure cov_reset;
procedure cov_sample(variable c : inout cov_t;
constant nb, pm, sh, er, baud_idx : in natural) is
begin
c.width(nb) := c.width(nb) + 1;
c.parity(pm) := c.parity(pm) + 1;
c.stop(sh) := c.stop(sh) + 1;
c.err(er) := c.err(er) + 1;
if baud_idx < NBAUD then
c.baud(baud_idx) := c.baud(baud_idx) + 1;
end if;
c.fmt(nb)(pm)(sh) := c.fmt(nb)(pm)(sh) + 1;
c.samples := c.samples + 1;
end procedure cov_sample;
procedure cov_sample_level(variable c : inout cov_t; constant lvl : in natural) is
begin
if lvl = 0 then c.lvl(0) := c.lvl(0) + 1;
elsif lvl = 1 then c.lvl(1) := c.lvl(1) + 1;
elsif lvl = DEPTH then c.lvl(4) := c.lvl(4) + 1;
elsif lvl = DEPTH - 1 then c.lvl(3) := c.lvl(3) + 1;
else c.lvl(2) := c.lvl(2) + 1;
end if;
end procedure cov_sample_level;
procedure cov_report(variable c : inout cov_t;
variable fmt_holes : out natural;
variable axis_holes : out natural) is
variable fh, ah : natural := 0;
begin
fh := 0; ah := 0;
for a in 5 to 9 loop
if c.width(a) = 0 then
ah := ah + 1;
report " HOLE: data width " & integer'image(a) & " never driven"
severity note;
end if;
end loop;
for a in 0 to 3 loop
if c.parity(a) = 0 then
ah := ah + 1;
report " HOLE: parity mode " & integer'image(a) & " never driven"
severity note;
end if;
end loop;
for a in 2 to 4 loop
if c.stop(a) = 0 then
ah := ah + 1;
report " HOLE: stop length " & integer'image(a)
& " half-bits never driven" severity note;
end if;
end loop;
for a in 0 to 5 loop
if c.err(a) = 0 then
ah := ah + 1;
report " HOLE: error kind " & integer'image(a) & " never injected"
severity note;
end if;
end loop;
for a in 0 to 4 loop
if c.lvl(a) = 0 then
ah := ah + 1;
report " HOLE: FIFO level bin " & integer'image(a) & " never reached"
severity note;
end if;
end loop;
for a in 5 to 9 loop
for b in 0 to 3 loop
for d in 2 to 4 loop
if c.fmt(a)(b)(d) = 0 then
fh := fh + 1;
report " HOLE: width=" & integer'image(a)
& " parity=" & integer'image(b)
& " stop=" & integer'image(d) & " never driven"
severity note;
end if;
end loop;
end loop;
end loop;
report " coverage: " & integer'image(c.samples) & " samples, format cross "
& integer'image(60 - fh) & " of 60 bins hit, "
& integer'image(fh) & " holes" severity note;
fmt_holes := fh;
axis_holes := ah;
end procedure cov_report;
end package body uart_cov_pkg;4. Report Holes, Not a Percentage
HOLE: width=9 parity=3 stop=3 never driven
HOLE: width=9 parity=3 stop=4 never driven
HOLE: error kind 5 never injected
coverage: 6 samples, format cross 6 of 60 bins hit, 54 holes"Ninety percent format coverage" is a number a manager can put on a slide and nobody can act on. "width=9 parity=MARK stop=1.5 never driven" is a line of stimulus somebody can write this afternoon.
The report is the deliverable. The counters are how it is produced.
5. Calibrating the Instrument
A coverage model is a measuring instrument, and one that has never been calibrated produces numbers nobody should act on. The suite checks three things:
- it counts what happened — a sample lands in the bins it should, and in no others;
- it finds what did not — with a bin unreached, the report names it;
- it is not fooled by the marginals — §2's case, constructed on purpose.
//===========================================================================
// tb_uart_cov_v — self-checking Verilog-2001 testbench
//
// A coverage model is a measuring instrument, and an instrument that has
// never been calibrated reports numbers nobody should act on.
//
// The checks below are of three kinds:
//
// IT COUNTS WHAT HAPPENED. A sample lands in the bins it should and in
// no others.
// IT FINDS WHAT DID NOT. With a bin unreached, the report NAMES it.
// This is the half people skip, and it is the
// half the report exists for.
// IT IS NOT FOOLED BY THE Every marginal bin can be full while the
// MARGINALS. cross is full of holes. The suite constructs
// exactly that case, because it is the one that
// makes a coverage number lie.
//===========================================================================
`timescale 1ns/1ps
module tb_uart_cov_v;
localparam NBAUD = 21;
localparam DEPTH = 16;
uart_cov_v #(.NBAUD(NBAUD), .DEPTH(DEPTH)) cov ();
integer checks = 0, failures = 0;
task check;
input cond;
input [8*80-1:0] name;
begin
checks = checks + 1;
if (cond) $display(" PASS %0s", name);
else begin failures = failures + 1; $display(" FAIL %0s", name); end
end
endtask
integer nb, pm, sh, i, tot;
initial begin
#10_000_000;
$display(" FAIL watchdog: simulation did not finish");
$display("== %0d checks, %0d failures ==", checks+1, failures+1);
$display(" RESULT: VERILOG COVERAGE TESTS FAILED (timeout)");
$finish;
end
initial begin
$display("== uart_cov_v : self-checking Verilog testbench ==");
//=== it starts empty ================================================
cov.cov_reset;
check(cov.n_samples == 0, "reset leaves the model with no samples");
tot = 0;
for (nb = 5; nb <= 9; nb = nb + 1) tot = tot + cov.cov_width[nb];
check(tot == 0, "and every width bin at zero");
//=== one sample lands where it should ===============================
cov.cov_sample(8, 1, 2, 0, 10); // 8 bits, even, 1 stop, clean, nominal
check(cov.n_samples == 1, "one sample is counted once");
check(cov.cov_width[8] == 1, "and lands in the width-8 bin");
check(cov.cov_parity[1] == 1, "and the parity-EVEN bin");
check(cov.cov_stop[2] == 1, "and the 1-stop-bit bin");
check(cov.cov_err[0] == 1, "and the no-error bin");
check(cov.cov_baud[10] == 1, "and the nominal-baud bin");
check(cov.cov_fmt[8][1][2] == 1, "and the CROSS bin for that format");
check(cov.cov_width[7] == 0 && cov.cov_parity[0] == 0,
"and nowhere else");
//=== an out-of-range baud index is ignored, not a crash =============
cov.cov_sample(8, 0, 2, 0, 99);
check(cov.n_samples == 2, "an out-of-range baud index still counts the sample");
check(cov.cov_baud[10] == 1, "but does not land in a baud bin");
//=== the FIFO level bins, at their boundaries =======================
cov.cov_reset;
cov.cov_sample_level(0);
cov.cov_sample_level(1);
cov.cov_sample_level(8);
cov.cov_sample_level(DEPTH-1);
cov.cov_sample_level(DEPTH);
check(cov.cov_lvl[0] == 1 && cov.cov_lvl[1] == 1 && cov.cov_lvl[2] == 1
&& cov.cov_lvl[3] == 1 && cov.cov_lvl[4] == 1,
"the five FIFO level bins each take their boundary value");
cov.cov_sample_level(2);
cov.cov_sample_level(DEPTH-2);
check(cov.cov_lvl[2] == 3,
"and everything between the boundaries falls in the middle bin");
//=== an empty model is ALL holes ====================================
cov.cov_reset;
cov.cov_report;
check(cov.fmt_holes == 60, "an unexercised model reports all 60 cross holes");
check(cov.axis_holes == 5 + 4 + 3 + 6 + 5,
"and every marginal bin as a hole too");
//=== the full cross closes it =======================================
cov.cov_reset;
for (nb = 5; nb <= 9; nb = nb + 1)
for (pm = 0; pm <= 3; pm = pm + 1)
for (sh = 2; sh <= 4; sh = sh + 1)
cov.cov_sample(nb, pm, sh, 0, 10);
cov.cov_report;
check(cov.n_samples == 60, "the full cross is 60 samples");
check(cov.fmt_holes == 0, "and leaves no cross holes");
//=== THE case that makes a coverage number lie ======================
// Every marginal axis fully covered, and the cross barely touched.
// A report that only tracked the marginals would call this closed.
cov.cov_reset;
cov.cov_sample(5, 0, 2, 0, 10);
cov.cov_sample(6, 1, 3, 0, 10);
cov.cov_sample(7, 2, 4, 0, 10);
cov.cov_sample(8, 3, 2, 0, 10);
cov.cov_sample(9, 0, 3, 0, 10);
cov.cov_sample(9, 1, 4, 0, 10);
cov.cov_report;
tot = 0;
for (nb = 5; nb <= 9; nb = nb + 1) if (cov.cov_width[nb] > 0) tot = tot + 1;
check(tot == 5, "every width was exercised");
tot = 0;
for (pm = 0; pm <= 3; pm = pm + 1) if (cov.cov_parity[pm] > 0) tot = tot + 1;
check(tot == 4, "and every parity mode");
tot = 0;
for (sh = 2; sh <= 4; sh = sh + 1) if (cov.cov_stop[sh] > 0) tot = tot + 1;
check(tot == 3, "and every stop length");
check(cov.fmt_holes == 54,
"yet 54 of the 60 CROSS bins were never reached");
//=== the error axis =================================================
cov.cov_reset;
for (i = 0; i <= 5; i = i + 1) cov.cov_sample(8, 0, 2, i, 10);
cov.cov_report;
tot = 0;
for (i = 0; i <= 5; i = i + 1) if (cov.cov_err[i] > 0) tot = tot + 1;
check(tot == 6, "all six error kinds recorded when all six are injected");
cov.cov_reset;
for (i = 0; i <= 4; i = i + 1) cov.cov_sample(8, 0, 2, i, 10);
cov.cov_report;
check(cov.cov_err[5] == 0,
"and the one that was skipped is reported as a hole by name");
$display("== %0d checks, %0d failures ==", checks, failures);
if (failures == 0) $display(" RESULT: ALL VERILOG COVERAGE TESTS PASSED");
else $display(" RESULT: VERILOG COVERAGE TESTS FAILED");
$finish;
end
endmodule//===========================================================================
// tb_uart_cov — self-checking SystemVerilog testbench
//
// A coverage model is a measuring instrument, and an instrument that has
// never been calibrated reports numbers nobody should act on.
//
// The checks below are of three kinds:
//
// IT COUNTS WHAT HAPPENED. A sample lands in the bins it should and in
// no others.
// IT FINDS WHAT DID NOT. With a bin unreached, the report NAMES it.
// This is the half people skip, and it is the
// half the report exists for.
// IT IS NOT FOOLED BY THE Every marginal bin can be full while the
// MARGINALS. cross is full of holes. The suite constructs
// exactly that case, because it is the one that
// makes a coverage number lie.
//===========================================================================
`timescale 1ns/1ps
module tb_uart_cov;
localparam NBAUD = 21;
localparam DEPTH = 16;
uart_cov #(.NBAUD(NBAUD), .DEPTH(DEPTH)) cov ();
int checks = 0, failures = 0;
task automatic check(input logic cond, input string name);
checks++;
if (cond) $display(" PASS %0s", name);
else begin failures++; $display(" FAIL %0s", name); end
endtask
int nb, pm, sh, i, tot;
initial begin
#10_000_000;
$display(" FAIL watchdog: simulation did not finish");
$display("== %0d checks, %0d failures ==", checks+1, failures+1);
$display(" RESULT: SYSTEMVERILOG COVERAGE TESTS FAILED (timeout)");
$finish;
end
initial begin
$display("== uart_cov : self-checking Verilog testbench ==");
//=== it starts empty ================================================
cov.cov_reset;
check(cov.n_samples == 0, "reset leaves the model with no samples");
tot = 0;
for (nb = 5; nb <= 9; nb = nb + 1) tot = tot + cov.cov_width[nb];
check(tot == 0, "and every width bin at zero");
//=== one sample lands where it should ===============================
cov.cov_sample(8, 1, 2, 0, 10); // 8 bits, even, 1 stop, clean, nominal
check(cov.n_samples == 1, "one sample is counted once");
check(cov.cov_width[8] == 1, "and lands in the width-8 bin");
check(cov.cov_parity[1] == 1, "and the parity-EVEN bin");
check(cov.cov_stop[2] == 1, "and the 1-stop-bit bin");
check(cov.cov_err[0] == 1, "and the no-error bin");
check(cov.cov_baud[10] == 1, "and the nominal-baud bin");
check(cov.cov_fmt[8][1][2] == 1, "and the CROSS bin for that format");
check(cov.cov_width[7] == 0 && cov.cov_parity[0] == 0,
"and nowhere else");
//=== an out-of-range baud index is ignored, not a crash =============
cov.cov_sample(8, 0, 2, 0, 99);
check(cov.n_samples == 2, "an out-of-range baud index still counts the sample");
check(cov.cov_baud[10] == 1, "but does not land in a baud bin");
//=== the FIFO level bins, at their boundaries =======================
cov.cov_reset;
cov.cov_sample_level(0);
cov.cov_sample_level(1);
cov.cov_sample_level(8);
cov.cov_sample_level(DEPTH-1);
cov.cov_sample_level(DEPTH);
check(cov.cov_lvl[0] == 1 && cov.cov_lvl[1] == 1 && cov.cov_lvl[2] == 1
&& cov.cov_lvl[3] == 1 && cov.cov_lvl[4] == 1,
"the five FIFO level bins each take their boundary value");
cov.cov_sample_level(2);
cov.cov_sample_level(DEPTH-2);
check(cov.cov_lvl[2] == 3,
"and everything between the boundaries falls in the middle bin");
//=== an empty model is ALL holes ====================================
cov.cov_reset;
cov.cov_report;
check(cov.fmt_holes == 60, "an unexercised model reports all 60 cross holes");
check(cov.axis_holes == 5 + 4 + 3 + 6 + 5,
"and every marginal bin as a hole too");
//=== the full cross closes it =======================================
cov.cov_reset;
for (nb = 5; nb <= 9; nb = nb + 1)
for (pm = 0; pm <= 3; pm = pm + 1)
for (sh = 2; sh <= 4; sh = sh + 1)
cov.cov_sample(nb, pm, sh, 0, 10);
cov.cov_report;
check(cov.n_samples == 60, "the full cross is 60 samples");
check(cov.fmt_holes == 0, "and leaves no cross holes");
//=== THE case that makes a coverage number lie ======================
// Every marginal axis fully covered, and the cross barely touched.
// A report that only tracked the marginals would call this closed.
cov.cov_reset;
cov.cov_sample(5, 0, 2, 0, 10);
cov.cov_sample(6, 1, 3, 0, 10);
cov.cov_sample(7, 2, 4, 0, 10);
cov.cov_sample(8, 3, 2, 0, 10);
cov.cov_sample(9, 0, 3, 0, 10);
cov.cov_sample(9, 1, 4, 0, 10);
cov.cov_report;
tot = 0;
for (nb = 5; nb <= 9; nb = nb + 1) if (cov.cov_width[nb] > 0) tot++;
check(tot == 5, "every width was exercised");
tot = 0;
for (pm = 0; pm <= 3; pm = pm + 1) if (cov.cov_parity[pm] > 0) tot++;
check(tot == 4, "and every parity mode");
tot = 0;
for (sh = 2; sh <= 4; sh = sh + 1) if (cov.cov_stop[sh] > 0) tot++;
check(tot == 3, "and every stop length");
check(cov.fmt_holes == 54,
"yet 54 of the 60 CROSS bins were never reached");
//=== the error axis =================================================
cov.cov_reset;
for (i = 0; i <= 5; i = i + 1) cov.cov_sample(8, 0, 2, i, 10);
cov.cov_report;
tot = 0;
for (i = 0; i <= 5; i = i + 1) if (cov.cov_err[i] > 0) tot++;
check(tot == 6, "all six error kinds recorded when all six are injected");
cov.cov_reset;
for (i = 0; i <= 4; i = i + 1) cov.cov_sample(8, 0, 2, i, 10);
cov.cov_report;
check(cov.cov_err[5] == 0,
"and the one that was skipped is reported as a hole by name");
$display("== %0d checks, %0d failures ==", checks, failures);
if (failures == 0) $display(" RESULT: ALL SYSTEMVERILOG COVERAGE TESTS PASSED");
else $display(" RESULT: SYSTEMVERILOG COVERAGE TESTS FAILED");
$finish;
end
endmodule--===========================================================================
-- tb_uart_cov — self-checking VHDL-2008 testbench
--
-- A coverage model is a measuring instrument, and an instrument that has
-- never been calibrated reports numbers nobody should act on.
--
-- The checks are of three kinds:
-- IT COUNTS WHAT HAPPENED. A sample lands in the bins it should and in
-- no others.
-- IT FINDS WHAT DID NOT. With a bin unreached, the report NAMES it.
-- IT IS NOT FOOLED BY THE Every marginal bin can be full while the
-- MARGINALS. cross is full of holes. The suite constructs
-- exactly that case, because it is the one that
-- makes a coverage number lie.
--
-- Same 24 counted checks as the Verilog and SystemVerilog twins.
--===========================================================================
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
use work.uart_cov_pkg.all;
entity tb_uart_cov is
end entity tb_uart_cov;
architecture sim of tb_uart_cov is
begin
stim : process
variable checks, failures : natural := 0;
variable c : cov_t;
variable fh, ah, tot : natural;
procedure check(cond : boolean; name : string) is
begin
checks := checks + 1;
if cond then
report " PASS " & name severity note;
else
failures := failures + 1;
report " FAIL " & name severity error;
end if;
end procedure check;
begin
report "== uart_cov : self-checking VHDL testbench ==" severity note;
--=== it starts empty ================================================
cov_reset(c);
check(c.samples = 0, "reset leaves the model with no samples");
tot := 0;
for a in 5 to 9 loop tot := tot + c.width(a); end loop;
check(tot = 0, "and every width bin at zero");
--=== one sample lands where it should ===============================
cov_sample(c, 8, 1, 2, 0, 10);
check(c.samples = 1, "one sample is counted once");
check(c.width(8) = 1, "and lands in the width-8 bin");
check(c.parity(1) = 1, "and the parity-EVEN bin");
check(c.stop(2) = 1, "and the 1-stop-bit bin");
check(c.err(0) = 1, "and the no-error bin");
check(c.baud(10) = 1, "and the nominal-baud bin");
check(c.fmt(8)(1)(2) = 1,"and the CROSS bin for that format");
check(c.width(7) = 0 and c.parity(0) = 0, "and nowhere else");
--=== an out-of-range baud index is ignored, not a crash =============
cov_sample(c, 8, 0, 2, 0, 99);
check(c.samples = 2, "an out-of-range baud index still counts the sample");
check(c.baud(10) = 1, "but does not land in a baud bin");
--=== the FIFO level bins, at their boundaries =======================
cov_reset(c);
cov_sample_level(c, 0);
cov_sample_level(c, 1);
cov_sample_level(c, 8);
cov_sample_level(c, DEPTH-1);
cov_sample_level(c, DEPTH);
check(c.lvl(0) = 1 and c.lvl(1) = 1 and c.lvl(2) = 1
and c.lvl(3) = 1 and c.lvl(4) = 1,
"the five FIFO level bins each take their boundary value");
cov_sample_level(c, 2);
cov_sample_level(c, DEPTH-2);
check(c.lvl(2) = 3,
"and everything between the boundaries falls in the middle bin");
--=== an empty model is ALL holes ====================================
cov_reset(c);
cov_report(c, fh, ah);
check(fh = 60, "an unexercised model reports all 60 cross holes");
check(ah = 5 + 4 + 3 + 6 + 5, "and every marginal bin as a hole too");
--=== the full cross closes it =======================================
cov_reset(c);
for nb in 5 to 9 loop
for pm in 0 to 3 loop
for sh in 2 to 4 loop
cov_sample(c, nb, pm, sh, 0, 10);
end loop;
end loop;
end loop;
cov_report(c, fh, ah);
check(c.samples = 60, "the full cross is 60 samples");
check(fh = 0, "and leaves no cross holes");
--=== THE case that makes a coverage number lie ======================
cov_reset(c);
cov_sample(c, 5, 0, 2, 0, 10);
cov_sample(c, 6, 1, 3, 0, 10);
cov_sample(c, 7, 2, 4, 0, 10);
cov_sample(c, 8, 3, 2, 0, 10);
cov_sample(c, 9, 0, 3, 0, 10);
cov_sample(c, 9, 1, 4, 0, 10);
cov_report(c, fh, ah);
tot := 0;
for nb in 5 to 9 loop if c.width(nb) > 0 then tot := tot + 1; end if; end loop;
check(tot = 5, "every width was exercised");
tot := 0;
for pm in 0 to 3 loop if c.parity(pm) > 0 then tot := tot + 1; end if; end loop;
check(tot = 4, "and every parity mode");
tot := 0;
for sh in 2 to 4 loop if c.stop(sh) > 0 then tot := tot + 1; end if; end loop;
check(tot = 3, "and every stop length");
check(fh = 54, "yet 54 of the 60 CROSS bins were never reached");
--=== the error axis =================================================
cov_reset(c);
for i in 0 to 5 loop cov_sample(c, 8, 0, 2, i, 10); end loop;
cov_report(c, fh, ah);
tot := 0;
for i in 0 to 5 loop if c.err(i) > 0 then tot := tot + 1; end if; end loop;
check(tot = 6, "all six error kinds recorded when all six are injected");
cov_reset(c);
for i in 0 to 4 loop cov_sample(c, 8, 0, 2, i, 10); end loop;
cov_report(c, fh, ah);
check(c.err(5) = 0,
"and the one that was skipped is reported as a hole by name");
report "== " & integer'image(checks) & " checks, "
& integer'image(failures) & " failures ==" severity note;
if failures = 0 then
report " RESULT: ALL VHDL COVERAGE TESTS PASSED" severity note;
else
report " RESULT: VHDL COVERAGE TESTS FAILED" severity error;
end if;
wait;
end process stim;
end architecture sim;Twenty-four checks, identical in all three languages:
PASS one sample is counted once
PASS and lands in the width-8 bin
PASS and the CROSS bin for that format
PASS and nowhere else
PASS the five FIFO level bins each take their boundary value
PASS an unexercised model reports all 60 cross holes
PASS the full cross is 60 samples
PASS every width was exercised
PASS yet 54 of the 60 CROSS bins were never reached
PASS and the one that was skipped is reported as a hole by name
== 24 checks, 0 failures ==
Verilog-2001 : 24 checks, 0 failures
SystemVerilog : 24 checks, 0 failures
VHDL-2008 : 24 checks, 0 failures6. The Defect That Survived Everything
This is the result the chapter exists for.
The FIFO property checker of Chapter 15.2 was mutated twelve ways. Eleven mutants died. One did not:
M4 fifo acceptance rule simplified to push && !full *** SURVIVED ***That mutation replaces the FIFO's real acceptance rule — a push into a full FIFO is accepted when a pop frees an entry on the same cycle — with the naive one. It is a genuine defect, it is the exact bug that produced 41 false failures earlier in this module's work, and it survived:
- 62 passing checks across three languages,
- 4,079 clocks of random push/pop traffic,
- the fill test, the drain test, the overflow test and the underflow test.
Why? One line of instrumentation answers it:
clocks=4079 push+pop while FULL occurred 0 timesZero. The single cycle on which the two rules disagree never happened. Not rarely — never.
7. Verification
Calibrate the model before quoting it. §5. A coverage model is a program and nobody writes a correct program first time — including the program whose job is to tell you what you missed.
Track the cross, not the marginals. §2's six samples fill every marginal axis and 10% of the cross.
Report holes by name. A percentage cannot be acted on; width=9 parity=MARK stop=1.5 can.
Assert that the sample count is non-zero. A coverage model that was never connected reports no holes at all in some implementations and all holes in others; neither is a useful signal on its own.
Treat an empty bin as a task, not a statistic. §6's bin was empty for a reason — the stimulus could not reach it — and the fix was five lines of directed traffic.
And do not let coverage stand in for checking. A full bin with no checker behind it is worse than an empty one, because it looks like progress.
8. Debugging
9. Understanding Check
10. Summary
Coverage answers one question — was this reached — and never says whether anything checked it.
Bins come from requirements. Every axis here is one Chapter 14.1's requirement list makes behaviour depend on; the data value gets no axis because the design does not distinguish values.
The cross is the model and the marginals are decoration: six frames fill every marginal axis and leave 54 of 60 cross bins empty.
Report holes by name, not a percentage. One can be acted on this afternoon; the other goes on a slide.
The FIFO level gets five bins, four of them boundaries, because the interior behaves identically.
VHDL gets a second mechanism the others do not: PSL cover on sequences, which bins a temporal shape rather than a value.
And the headline: a real defect survived 62 checks and 4,079 clocks of random traffic because one bin was empty — zero occurrences, not rare ones. Five lines of directed stimulus killed it and took the campaign to twelve of twelve.
11. What Comes Next
Chapter 15.4 drives the error axis of this model for real — six injected faults, and what detects each one.
Browse the full path on the UART tutorials index. For the properties whose coverage this measures, read back to Chapter 15.2.
Continue learning
Related tutorials
- Related topic
Sampling Centres and the Timing Margin Budget
Half a bit period separates an interval's centre from its boundary. That half-bit is a budget spent by origin uncertainty, interval construction, accumulated drift and the decision mechanism — and the last interval of a frame is where it runs out first.
- Related topic
Parity Generation, Checking and Error Detection
One interval, one XOR reduction, and a detection guarantee with a sharp edge: parity catches every corruption that flips an odd number of protected bits and provably misses every even-numbered one — demonstrated, not asserted.
- Related topic
Stop Bits, Frame Boundaries and Back-to-Back Frames
The stop interval is a required mark condition the receiver checks at a known position — not a pause, not recovery time, and not a resynchronisation. What a framing error reports, what it cannot tell you, and why two frames may follow with no idle between them.
- Related topic
Frame Configurations: 8N1 and the Configuration Space
8N1 names three of the four choices a UART link depends on and omits the one most likely to be wrong. Reading the shorthand, computing what each configuration costs in intervals and line time, and why a longer frame spends timing margin as well as throughput.
Where this fits
Part of the UART curriculum.
