Random Seed Configuration

A directed test does the same thing every time. A constrained-random test picks its stimulus from a random stream, so the seed that starts that stream decides what actually gets exercised. Run the same test at seed 1 and seed 42 and you have run two different tests.

That gives the seed two jobs, pulling in opposite directions. It has to vary, so that many runs of one test cover more than one run could. And it has to be recorded exactly, so that when a run fails you can get that precise failure back.

Both jobs are easy to plumb and easy to get subtly wrong. The plumbing takes one section. The rest of the chapter is the ways it goes wrong while looking fine.

1. Passing the Seed Through

Mechanically this is 6.1 again with a different variable name. The seed is a run-time value, so it travels as a plusarg and never reaches the compiler:

TEST ?= axi_smoke
SEED ?= 1

RUNDIR := $(BUILD)/$(TEST)/$(SEED)
RUNLOG := $(RUNDIR)/run.log
STATUS := $(RUNDIR)/passed.stamp

$(STATUS): $(SIMV)
	@mkdir -p $(@D)
	@echo "  [run] $(TEST) seed=$(SEED) -> $(RUNLOG)"
	@vvp $(SIMV) +UVM_TESTNAME=$(TEST) +ntb_random_seed=$(SEED) > $(RUNLOG)
	@grep -q "^UVM_ERROR :  0" $(RUNLOG) || { echo "  [FAIL] $(TEST) seed=$(SEED)"; exit 1; }
	@touch $@
	@echo "  [PASS] $(TEST) seed=$(SEED)"

The seed appears in the path as well as the plusarg, which is Chapter 5.3's per-run directory: build/axi_burst/42/ and build/axi_burst/7/ are two different runs of one test and both sets of results survive.

Different tools spell the seed option differently, and this is one of the few places where the Makefile cannot be tool-agnostic:

./simv +UVM_TESTNAME=axi_smoke_test +ntb_random_seed=42    # VCS
xmsim tb_top +UVM_TESTNAME=axi_smoke_test -svseed 42        # Xcelium
vsim -c tb_top +UVM_TESTNAME=axi_smoke_test -sv_seed 42     # Questa

Putting the whole option in a variable is the usual answer, so the per-tool difference lives in one place — Chapter 7.2 is where simulator selection is built properly. For this chapter one tool is enough.

So far, so ordinary. Now the interesting part.

2. Does the Seed Actually Do Anything?

Here is a testbench that reads the seed, prints it, and never applies it. The bug is a missing line, which is why it is so easy to ship:

integer seed, i;
initial begin
  if (!$value$plusargs("ntb_random_seed=%d", seed)) seed = 1;
  $display("TB: seed=%0d", seed);
  for (i=0;i<5;i=i+1) $display("  stim=%0d", $urandom_range(0,99));
end

Run it at four very different seeds:

SEED=1    ->  57 25 1 19 52
SEED=42   ->  57 25 1 19 52
SEED=7    ->  57 25 1 19 52
SEED=999  ->  57 25 1 19 52

Identical. Every single run produced the same five numbers. And every log faithfully printed a different seed at the top, so a report built from those logs would show 1, 42, 7 and 999 as four distinct runs.

Add the one missing line:

void'($urandom(seed));     // <- apply the seed to the generator

SEED=1    ->  11 76 17 90 88
SEED=42   ->  65 50 27 54 35
SEED=7    ->  77 67 68 24 81
SEED=999  ->  61 79 24 35 87

Four seeds, four different streams.

3. Is the Seed in the Log the Seed You Ran?

Now a subtler one. Print the seed after applying it:

if (!$value$plusargs("ntb_random_seed=%d", seed)) seed = 1;
keep = seed;
$display("before seeding : seed=%0d", seed);
void'($urandom(seed));
$display("after  seeding : seed=%0d   (kept copy=%0d)", seed, keep);

+ntb_random_seed=42
before seeding : seed=42
after  seeding : seed=2900899   (kept copy=42)
-- stimulus --
   65  50  27  54

The variable changed. $urandom(seed) takes its seed argument as an inout — it seeds the generator and writes the generator's new state back into your variable. $random(seed) does the same, every call:

$random: seed before=42
   after call 0: seed=2900899
   after call 1: seed=-1501269880
   after call 2: seed=-2108881687

So a testbench that prints the seed after seeding writes 2900899 into its log. Now try to reproduce that run using the number the log gave you:

--- the original run, seed 42 ---
   65  50  27  54
--- re-run with 2900899, the number the log reported ---
   50  27  54  35

Look carefully. 50 27 54 appears in both. The reproduction is the original stream shifted by exactly one draw.

That is a genuinely nasty failure mode, and worse than a clean mismatch would be. A completely different stream tells you immediately that the seed is wrong. A stream that is almost identical tells you that the design is non-deterministic, or that the failure is intermittent, or that something is racing — and you can lose a day to that. Meanwhile the original failure is still sitting there at seed 42, perfectly reproducible, under a number nobody wrote down.

The fix is two lines:

if (!$value$plusargs("ntb_random_seed=%d", seed)) seed = 1;
seed_used = seed;              // keep a copy BEFORE seeding
void'($urandom(seed));         // this mutates `seed`
$display("UVM_INFO @ 0: reporter [SEED] seed=%0d", seed_used);

+ntb_random_seed=1   -> UVM_INFO @ 0: reporter [SEED] seed=1
+ntb_random_seed=42  -> UVM_INFO @ 0: reporter [SEED] seed=42
+ntb_random_seed=7   -> UVM_INFO @ 0: reporter [SEED] seed=7

Print the seed before you use it, or keep a copy. The rule sounds trivial until you have chased the off-by-one-draw version of it.

4. Reproducibility, Measured

With both bugs fixed, the property the whole chapter exists for is easy to check. Same test, same seed, three times:

  run 1:  65 50 27 54 35
  run 2:  65 50 27 54 35
  run 3:  65 50 27 54 35

Identical, three for three. And through the Makefile, on the stand-in testbench of 6.1:

  run 1: UVM_INFO @ 55000: uvm_test_top [TEST] addr 1 ok   UVM_ERROR :  0
  run 2: UVM_INFO @ 55000: uvm_test_top [TEST] addr 1 ok   UVM_ERROR :  0
  run 3: UVM_INFO @ 55000: uvm_test_top [TEST] addr 1 ok   UVM_ERROR :  0

This is also why Chapter 5.3 §6's caching is legitimate. When Make says Nothing to be done for a test and seed it has already run against the current image, it is right: the run is deterministic, so repeating it would produce a byte-identical log.

5. Why Sweep Seeds At All

Because one seed only ever exercises one path through the random space. The stand-in testbench has a test, axi_flaky, that draws an address and fails when the address lands outside the decoded range — the shape of a real seed-dependent bug. Over twenty consecutive seeds:

  seed 1   addr=0  PASS      seed 11  addr=1  PASS
  seed 2   addr=1  PASS      seed 12  addr=2  PASS
  seed 3   addr=2  PASS      seed 13  addr=3  PASS
  seed 4   addr=3  PASS      seed 14  addr=4  FAIL
  seed 5   addr=4  FAIL      seed 15  addr=5  FAIL
  seed 6   addr=5  FAIL      seed 16  addr=6  FAIL
  seed 7   addr=6  FAIL      seed 17  addr=7  FAIL
  seed 8   addr=7  FAIL      seed 18  addr=7  FAIL
  seed 9   addr=7  FAIL      seed 19  addr=0  PASS
  seed 10  addr=0  PASS      seed 20  addr=1  PASS
  ---- 10 passed / 10 failed out of 20 ----

Run this test once at seed 1 and you ship. Run it at twenty seeds and half of them fail. That is the argument for sweeping, in one table.

And because the seed is recorded and applied correctly, every one of those failures is a command you can type to get it back: make run TEST=axi_flaky SEED=5.

6. Consecutive Seeds Are Not Independent

Look at that addr column again. It goes 0, 1, 2, 3, 4, 5, 6, 7, 7, 0, 1, 2, 3, 4, 5, 6, 7, 7, 0, 1. That is not random. The first draw after seeding is tracking the seed almost exactly.

Extending it to 48 consecutive seeds makes it unmistakable:

0 1 2 3 4 5 6 7 7 0 1 2 3 4 5 6 7 7 0 1 2 3 4 5 6 7 7 0 1 2 3 4 5 6 7 7 0 1 2 3 4 5 6 6 7 0 1 2

A cycle of period nine. The obvious response is to spread the seeds out, so try multiples of a prime:

6 5 3 2 1 7 6 5 3 2 1 7 6 4 3 2 0 7 6 4 3 2 0 7 5 4 3 1 0 7 5 4 3 1 0 6 5 4 2 1 0 6 5 4 2 1 0 6

Still structured — a descending cycle instead of an ascending one. Spreading the seeds changed the pattern without removing it.

Now the part that matters most. Count how often each value appears, over 400 seeds of each kind:

  addr 0 : 49    addr 4 : 48
  addr 1 : 49    addr 5 : 52
  addr 2 : 49    addr 6 : 53
  addr 3 : 48    addr 7 : 52

  addr 0 : 50    addr 4 : 48
  addr 1 : 50    addr 5 : 49
  addr 2 : 52    addr 6 : 50
  addr 3 : 50    addr 7 : 51

Both essentially uniform. Fifty per bucket out of 400, in both cases. A uniformity test passes on the perfectly cyclic data. If you had written a check that the seed sweep produces a flat distribution, it would have reported everything healthy on a sequence that literally counts 0, 1, 2, 3, 4, 5, 6, 7.

So the honest conclusion is not "use widely separated seeds" — that was measured and it does not fix the structure. It is this: the first value drawn after seeding is correlated with the seed, and a histogram cannot detect it. Which leads to the practical rules:

• Do not judge a seed sweep by the distribution of its first draw. It will look fine either way.
• The value of a sweep comes from the depth of each test, not from arithmetic on the seed. A test that draws thousands of values gets well past the correlated first few; a test whose whole behaviour hinges on its first draw is sampling the seed, not the random space.
• Prefer more seeds over cleverly chosen seeds. Both patterns above covered all eight values roughly equally over 400 runs; neither needed the seeds to be special.

7. Choosing the Seed in the Makefile

For a regression you often want a fresh seed per run rather than a hand-typed one. The obvious way is a shell command — and the obvious way to write it is wrong:

# RECURSIVE assignment: re-expanded at EVERY use
SEED_BAD   = $(shell od -An -N2 -tu2 /dev/urandom | tr -d ' ')

# SIMPLE assignment: expanded once, when the Makefile is read
SEED_GOOD := $(shell od -An -N2 -tu2 /dev/urandom | tr -d ' ')

Use each one three times in a recipe:

  dir in the path : build/bad/6851
  plusarg value   : +ntb_random_seed=23515
  and once more   : +ntb_random_seed=56385

  dir in the path : build/good/29747
  plusarg value   : +ntb_random_seed=29747
  and once more   : +ntb_random_seed=29747

Three expansions, three different seeds. = is recursive, so every $(SEED_BAD) re-runs the command and draws again — Chapter 3.1's distinction, with real consequences.

In a real recipe it breaks loudly, which is lucky:

  recipe thinks the run dir is: build/axi_random/10032
/bin/sh: build/axi_random/22052/run.log: No such file or directory

mkdir -p $(RUNDIR) created one directory and the output redirect expanded $(RUNDIR) again, drawing a different seed and naming a directory that does not exist. The run died before it started.

The quiet version is worse. Any arrangement where the directory already exists, or where the path is formed only once but the plusarg is expanded separately, gives you a log stored under one seed and generated with another — exactly the unreproducible run §3 warned about, arriving by a different route.

So: := for anything derived from a command, always. And if you want a random seed recorded for later:

# := so the draw happens once, when the Makefile is read.
SEED ?= $(shell od -An -N2 -tu2 /dev/urandom | tr -d ' ')

8. Recording the Seed Where You Will Look For It

A seed is only useful if you can find it after the run. Three places, all cheap:

$(STATUS): $(SIMV)
	@mkdir -p $(@D)
	@echo "$(SEED)" > $(@D)/seed.txt
	@echo "  [run] $(TEST) seed=$(SEED) -> $(RUNLOG)"
	@vvp $(SIMV) +UVM_TESTNAME=$(TEST) +ntb_random_seed=$(SEED) > $(RUNLOG)
	@grep -q "^UVM_ERROR :  0" $(RUNLOG) || { \
	   echo "  [FAIL] reproduce with: make run TEST=$(TEST) SEED=$(SEED)"; exit 1; }
	@touch $@

In the path (build/axi_flaky/5/) so the directory listing is a record of what ran. In the log, printed by the testbench from the copy it kept before seeding. In the failure message, as the exact command to type — which is the one place a tired engineer will actually read it.

That last line is worth more than it looks. A failure that tells you how to reproduce it converts a debugging session's first ten minutes into a copy and paste.

9. Common Mistakes

A testbench that reads the seed and never applies it. Measured: four very different seeds, byte-identical stimulus, and four logs each reporting its own seed (§2).

Logging the seed after seeding. $urandom(seed) and $random(seed) mutate their argument, so the logged number reproduces the run one draw late — which looks like non-determinism rather than a wrong seed (§3).

Judging a sweep by a histogram. Both a perfectly cyclic sequence and a spread-out one gave a uniform distribution over 400 seeds (§6).

Choosing clever seeds instead of more seeds. Multiplying by a prime changed the pattern and did not remove it (§6).

SEED = $(shell ...) with a recursive assignment. Every expansion draws again: measured 6851 in the path and 23515 in the plusarg (§7).

Assuming ?= fixes that. ?= stores a recursive value, so it has the same problem; draw into a := variable first (§7).

Not recording the seed anywhere. A failure you cannot reproduce is a failure you cannot fix (§8).

Treating the seed as a compile-time define. Then every seed costs a rebuild, and a 200-seed sweep is 200 compiles (Chapter 5.2 §2).

10. Check Your Understanding

Question 1. A regression runs one test at 200 seeds. Every log reports a different seed, every run passes, and coverage barely moves. What would you check first, and with what command?

Question 2. A failing run's log says seed=2900899. Re-running with that seed gives a stream that shares most of its values with the original but is not identical, and the failure does not reproduce. What happened?

Question 3. A seed sweep's first drawn value is checked for a flat distribution across 400 seeds and the check passes. Does that establish that the seeds are giving independent stimulus?

Question 4. Why does SEED ?= $(shell date +%s) not behave like a value drawn once, and what is the two-line fix?

11. Exercise

Use 6.1's flow and the stand-in testbench. Each transcript is real output.

Step 1. Confirm the seed reaches the testbench and is logged as given:

make run TEST=axi_random SEED=42
grep SEED build/axi_random/42/run.log

  [compile] -> build/simv
  [run] axi_random seed=42 -> build/axi_random/42/run.log
  [PASS] axi_random seed=42
UVM_INFO @ 0: reporter [SEED] seed=42

Step 2. Prove the seed changes the stimulus. Strip the seed line and diff:

vvp build/simv +UVM_TESTNAME=axi_random +ntb_random_seed=1   | grep -v SEED > /tmp/a
vvp build/simv +UVM_TESTNAME=axi_random +ntb_random_seed=999 | grep -v SEED > /tmp/b
diff /tmp/a /tmp/b > /dev/null && echo "seed does nothing" || echo "seed matters"

seed matters

Step 3. Prove reproducibility. Run the same test and seed three times and compare:

for i in 1 2 3; do vvp build/simv +UVM_TESTNAME=axi_random +ntb_random_seed=42 | md5 -q; done

f9a54850a730e042f270a1a92009b9ed
f9a54850a730e042f270a1a92009b9ed
f9a54850a730e042f270a1a92009b9ed

Step 4. Find the seed-dependent failure. Sweep axi_flaky by hand over six seeds:

for s in 1 2 3 4 5 6; do
  printf "seed %s: " "$s"
  vvp build/simv +UVM_TESTNAME=axi_flaky +ntb_random_seed=$s | grep -c '^UVM_ERROR :  0' \
    | sed 's/^1$/PASS/;s/^0$/FAIL/'
done

seed 1: PASS
seed 2: PASS
seed 3: PASS
seed 4: PASS
seed 5: FAIL
seed 6: FAIL

Step 5. Reproduce one of the failures from its seed alone:

make run TEST=axi_flaky SEED=5

  [run] axi_flaky seed=5 -> build/axi_flaky/5/run.log
  [FAIL] axi_flaky seed=5
make: *** [build/axi_flaky/5/passed.stamp] Error 1

Then break it on purpose, twice, and predict each result first. Remove the void'($urandom(seed)); line from the testbench and repeat step 2 — it should now print seed does nothing. Put it back, then move the $display of the seed to after the $urandom(seed) call and repeat step 1; the logged number will no longer be 42, and re-running with the number it prints will give you a stream shifted by one draw rather than the original. Both are failures that no validator, no exit status and no histogram would have told you about.

12. Remember

• The same test at two seeds is two tests. That is why the seed is a first-class knob and why it travels as a plusarg, never a define.
• The seed belongs in the path as well as the plusarg, so two seeds of one test keep separate results.
• A seed that is never applied gives identical stimulus for every value — measured on four seeds — while every log still reports its own seed. Check it with a two-run diff.
• $urandom(seed) and $random(seed) mutate their argument. Measured: 42 became 2900899.
• So log the seed before you apply it, or keep a copy. A seed logged afterwards reproduces the run one draw late, which looks like non-determinism and costs far more than a clean mismatch.
• Reproducibility is the payoff: same test, same seed, three identical runs — and the reason Make's caching of a completed run is sound.
• Sweeping finds what one seed cannot: axi_flaky passed at seed 1 and failed at 10 of 20 seeds.
• Consecutive seeds are not independent, and neither are seeds spread by a prime — both first-draw sequences were strictly cyclic.
• A uniform histogram cannot detect that. Both patterns looked flat over 400 seeds, so never certify a sweep with a distribution check.
• Prefer more seeds and deeper tests over clever seed arithmetic.
• Derive a seed with :=, never = or ?= — a recursive value re-draws at every expansion: measured 6851 in the path and 23515 in the plusarg from one assignment.
• Print the reproduction command in the failure message. make run TEST=axi_flaky SEED=5 in the log saves the next person ten minutes.

One test, one seed, reproducible, with its results in a directory of its own. The flow is now correct for a single run — and a verification project needs hundreds of them, over a list of tests and a range of seeds, finished by morning and summarised in one line. Chapter 6.3 turns the test list into a regression, with a single pattern rule that covers every test and seed combination, and measures what one failing test does to the other ninety-nine.