Linux & Bash · Part II · Module 27 · Invoking Simulators and EDA Tools from the Shell

Passing Test Names, Plusargs and Seeds

The compiled simulation is one artifact and every run of it is different — a test name, a seed, a verbosity, a dump switch. Those arrive as plusargs, and a plusarg is the only part of a tool invocation where getting the name wrong produces no error at all: the run proceeds with the default and reports success.

Intermediate22 min readLinuxPart IIPlusargsSeedsReproducibility

Part II · Module 27 · Chapter 27.4 · Passing Test Names, Plusargs and Seeds

One compile, many runs. That is the arrangement every verification flow wants, and it works because the things that vary between runs are not compiled in — they are read at run time from the command line. Chapter 27.3 gave each run its own directory; this chapter fills in what distinguishes them.

1. The Mechanism

An argument beginning with + is a plusarg. The simulator does not interpret it; it keeps it and offers it to the design, which asks for it by name.

There are two forms of asking, and the difference matters:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

// does this plusarg exist? -> a switch
if ($test$plusargs("DUMP")) begin
  $dumpfile("dump.vcd");
  $dumpvars(0, tb_arb);
end

// does this plusarg exist, and what is its value? -> a parameter
if (!$value$plusargs("TESTNAME=%s", testname)) testname = "smoke";
if (!$value$plusargs("SEED=%d", seed))         seed = 1;

$test$plusargs returns true if the name is present and ignores any value. $value$plusargs takes a format with a conversion in it, matches the literal prefix, parses the rest with that conversion, and returns true only if it found and parsed something.

Both return a value you are free to ignore, and that is the root of everything in this chapter. Nothing forces the design to check the result, and nothing anywhere reports an unrecognised plusarg.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ vvp build.vvp +TESTNAME=fairness
TB  test=fairness seed=1 cycles=40 verbosity=LOW
TB  TEST PASSED  test=fairness checks=6 errors=0

That is the mechanism working. The TB line is the testbench echoing what it read, and the presence of that echo is not decoration — it is the only way the command line and the behaviour can be compared.

2. One Underscore

The testbench above defaults TESTNAME to smoke when the plusarg is absent, which is the overwhelmingly common pattern. Now four invocations:

command	test that ran	checks	verdict
`vvp build.vvp +TESTNAME=fairness`	`fairness`	6	PASSED
`vvp build.vvp +TEST_NAME=fairness`	`smoke`	3	PASSED
`vvp build.vvp +UVM_TESTNAME=fairness`	`smoke`	3	PASSED
`vvp build.vvp`	`smoke`	3	PASSED

Rows two and three are the finding. An underscore in the wrong place, or a plusarg spelled for a different vendor's testbench, and the simulation ran a different test than the one you named and reported success. There is no warning, no mention of the unrecognised argument, and no difference in exit status.

The only visible evidence is in the testbench's own echo: test=smoke where you asked for fairness, and checks=3 where fairness performs 6. Which means the echo is the mechanism that makes the defect detectable, and a testbench that does not echo its configuration cannot be checked from outside at all.

3. The Fix Belongs in the Testbench, Not the Wrapper

A wrapper cannot fix this. The wrapper is the thing that got the spelling wrong, and no amount of checking on the shell side verifies that the design recognised the name. The check has to be where the knowledge is:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

// STRICT: a required plusarg that is absent is a failure, not a default.
if (!$value$plusargs("TESTNAME=%s", testname)) begin
  if ($test$plusargs("STRICT")) begin
    $display("TB  FATAL +TESTNAME is required and was not given");
    $display("TB  TEST FAILED  test=<none> checks=0 errors=1");
    $fatal(1, "missing +TESTNAME");
  end
  testname = "smoke";
end

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ vvp build.vvp +TESTNAME=fairness +STRICT
TB  TEST PASSED  test=fairness checks=6 errors=0        # rc=0
$ vvp build.vvp +TEST_NAME=fairness +STRICT
TB  FATAL +TESTNAME is required and was not given       # rc=1
$ vvp build.vvp +UVM_TESTNAME=fairness +STRICT
TB  FATAL +TESTNAME is required and was not given       # rc=1
$ vvp build.vvp +STRICT
TB  FATAL +TESTNAME is required and was not given       # rc=1

All three wrong invocations now fail, loudly, with a status a wrapper can see. The +STRICT switch exists so an interactive run can still rely on defaults while every automated run cannot — the regression driver always passes it.

4. The One-Word Difference Between a Status and No Status

The $fatal in that snippet is doing something a $finish cannot, and the difference is worth measuring because the two look interchangeable in source.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

module t;
  initial begin
    if ($test$plusargs("FATAL1")) begin $display("via $fatal(1)");  $fatal(1, "msg"); end
    if ($test$plusargs("FIN1"))   begin $display("via $finish(1)"); $finish(1);       end
    $display("fell through"); $finish;
  end
endmodule

Seven invocations of that, with the exit status after each:

call	exit status
`$fatal(0, "msg")`	1
`$fatal(1, "msg")`	1
`$fatal(2, "msg")`	1
`$finish(0)`	0
`$finish(1)`	0
`$finish(2)`	0
fell through to plain `$finish`	0

Every $fatal gives 1. Every $finish gives 0. The argument to either one is a diagnostic verbosity level — how much statistics the simulator prints on its way out — and it is not an exit status, in spite of reading exactly like one. $finish(1) is the single most convincing-looking way to write "exit with failure" in SystemVerilog, and it exits 0.

5. Seeds: The Argument That Must Be Written Down

A randomised test is a different test on every seed, so the seed is not a tuning knob — it is part of the test's identity.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ vvp build.vvp +TESTNAME=random +SEED=7  +CYCLES=20
TB  TEST PASSED  test=random checks=6 errors=0
$ vvp build.vvp +TESTNAME=random +SEED=99 +CYCLES=20
TB  TEST PASSED  test=random checks=7 errors=0
$ vvp build.vvp +TESTNAME=random +SEED=7  +CYCLES=20
TB  TEST PASSED  test=random checks=6 errors=0

Two facts in three commands. The seed changes the stimulus — six checks against seven, for the same test and the same cycle count. And the same seed reproduces exactly — the third run matches the first.

That reproducibility is the entire reason to pass seeds explicitly rather than letting the testbench pick one:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

# wrong: the seed is invented inside the run and never leaves it
vvp build.vvp +TESTNAME=random

# right: the seed is chosen by the caller, so it is in the command, in the
# run directory's name, and in the record
for s in 1 2 3 4 5; do
  rundir="$RUNS/random_s${s}"
  mkdir -p "$rundir" || exit 2
  ( cd "$rundir" && exec vvp "$simexe" +TESTNAME=random +SEED="$s" +STRICT ) \
      > "$rundir/run.log" 2>&1
done

A failure found at seed 3 is reproducible because 3 is written in three places. A failure found by a testbench that seeded itself from the clock is a story, not a bug report.

A seed from the clock is not a seed

+SEED=$(date +%s) is a popular way to "get variety". It gives variety and destroys reproducibility, because the value exists only in that one process and nothing records it.

If you want variety and reproducibility, generate the seeds in the caller — where they can be printed, stored and replayed — and pass them in:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

# variety chosen once, recorded, and reusable
seeds=$(awk -v n=20 'BEGIN{srand(12345); for(i=0;i<n;i++) printf "%d\n", rand()*100000}')
printf '%s\n' "$seeds" > "$RUNS/seeds.txt"

Now the sweep is a file, the sweep is repeatable, and seeds.txt is the thing you hand to somebody who has to reproduce last Tuesday's failure. Chapter 31.3 makes the recording of that seed a rule rather than a habit, and Chapter 31.4 builds the whole thing out into a run matrix.

6. Passing Them From the Shell Without Losing Them

Everything in Chapter 27.2 applies, with one addition specific to plusargs: a plusarg's value is parsed by the design, with a format string, and that parser is more permissive than you expect.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

# one argv element containing spaces -- %s takes the whole thing
$ vvp build.vvp "+TESTNAME=random +SEED=7 +CYCLES=20"
TB  test=random +SEED=7 +CYCLES=20 seed=1 cycles=40 verbosity=LOW
TB  TEST FAILED  test=random +SEED=7 +CYCLES=20 checks=0 errors=1

%s matched everything to the end of the argument, so the test name became a sentence, the seed and cycle count fell back to defaults, and nothing was tested. In this case the testbench happened to reject the unknown name — but only because it checks; with a default-to-smoke policy and no strict mode it would have been another silent pass.

The safe assembly, which keeps each plusarg its own argument:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

rargs=(+TESTNAME="$test" +SEED="$seed" +STRICT)
[ "$dump" = 1 ] && rargs+=(+DUMP +DUMP_FILE=dump.vcd)
( cd "$rundir" && exec vvp "$simexe" ${rargs[@]+"${rargs[@]}"} ) > "$rundir/run.log" 2>&1

Two rules to keep in view when choosing plusarg names for a testbench you control:

Do not put a value with a space in a plusarg. A %s conversion will take it and the argument boundary will not protect you if anything later joins the arguments into a string — a log line, a report, a replay built from text. Use an underscore, or pass a path to a file.

Use one unmistakable spelling per concept. The whole of section 2 is what happens when two spellings exist in an organisation. If a flow is being ported from a different vendor, the old spellings should fail, not quietly fall through to defaults — which is what strict mode buys.

7. What the Wrapper Should Echo

A run's identity is the combination of the arguments that varied. Writing them down at the top of the log costs one line and settles every later question about what ran:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

# in the wrapper, before the tool runs
{ printf 'test     %s\n' "$test"
  printf 'seed     %s\n' "$seed"
  printf 'dump     %s\n' "$dump"
  printf 'plusargs %s\n' "$(printf '%s ' ${rargs[@]+"${rargs[@]}"})"
} > "$rundir/config.txt"

And then — this is the part that is easy to skip — compare it against what the testbench echoed:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

# the wrapper said one thing; did the design agree?
want=$test
got=$(sed -n 's/^TB  test=\([^ ]*\).*/\1/p' "$rundir/run.log" | head -1)
if [ "$want" != "$got" ]; then
  printf 'MISMATCH wrapper asked for %s, testbench ran %s\n' "$want" "${got:-<nothing>}" >&2
  exit 1
fi

That five-line check is the only thing in this chapter that catches a misspelled plusarg without the testbench's cooperation. It works because it compares two independent statements of the same fact: what the shell intended, and what the design reported. One measurement cannot validate itself; two can disagree.

Practice

Run the fixture with +TESTNAME=fairness and note the check count. Then run it with +TEST_NAME=fairness and compare.
Record the exit status of all four invocations in the section 2 table. Confirm they are all 0.
Find the only field in the output that reveals which test actually ran.
Add a strict mode to a testbench of your own and prove it rejects an absent required plusarg.
Measure $finish(1) against $fatal(1) yourself, and record the exit status of each.
Then try $fatal(0) and $finish(2) and decide what the argument actually controls.
Run the same seed twice and confirm the check counts are identical; then change the seed and confirm they are not.
Replace an explicit seed with $(date +%s), run it, and then try to reproduce the exact run. Write down what you would need.
Generate twenty seeds from a fixed srand, store them, and confirm regenerating gives the same twenty.
Pass three plusargs as one quoted argument and find checks=0 in the result.
Write the wrapper-versus-testbench comparison and prove it fires on a deliberately misspelled plusarg name.

Check Your Understanding

What is the difference between $test$plusargs and $value$plusargs?
Why does no tool report an unrecognised plusarg?
What did +TEST_NAME=fairness cause the simulation to do, and what was the exit status?
Why can a shell wrapper not detect a misspelled plusarg on its own?
Where does the fix for a missing required plusarg have to live, and why?
What is the exit status of $finish(1)? Of $fatal(0)?
What does the argument to $finish and $fatal actually control?
Give two reasons an explicit seed beats a self-chosen one.
What is wrong with +SEED=$(date +%s) beyond style?
Why does comparing the wrapper's config.txt against the log's echo catch something neither can catch alone?

Answers

$test$plusargs only asks whether a name is present and ignores any value; $value$plusargs matches a literal prefix, parses the remainder with a conversion, and reports whether it succeeded.
Because a plusarg is not the simulator's business — it holds the argument and offers it to the design on request. Nothing is positioned to know which names are meaningful.
It ran smoke — the default — performing 3 checks instead of fairness's 6, and exited 0 with TEST PASSED.
Because the wrapper is the thing that got the spelling wrong, and no shell-side check can confirm that the design recognised the name.
In the testbench, because that is the only place that knows which plusargs are required. A $fatal there turns a silent default into a visible failure.
$finish(1) exits 0. $fatal(0) exits 1. Every $fatal argument gives 1 and every $finish argument gives 0.
A diagnostic verbosity level — how much end-of-run statistics the simulator prints — not an exit status.
It is reproducible, because the value is in the command, the directory name and the record; and it is reportable, so a failure at that seed can be handed to somebody else.
The value exists only inside that one process and is never recorded, so a failure it finds cannot be reproduced by anyone, including you.
Because they are two independent statements of the same fact — what the shell intended and what the design reported — and a single measurement cannot contradict itself.

8. Remember

A plusarg is held by the simulator and interpreted by the design, so nothing is in a position to report an unrecognised one.
A misspelled plusarg runs the default and exits 0. One underscore ran a different test and reported PASSED.
This is the worst failure mode available in a tool invocation, because unlike a bad option or a missing file it enters your results instead of stopping your flow.
A testbench must echo its configuration, or the defect is invisible from outside.
The fix lives in the testbench: make an absent required plusarg fatal, gated by a strict switch the automated flow always passes.
Every $fatal exits 1; every $finish exits 0. The argument is a diagnostic level, not a status.
Pass seeds explicitly, and generate variety in the caller where it can be recorded.
Compare what the wrapper asked for against what the log says ran — two independent statements, which is the only arrangement that can disagree.

9. What Is Next

Plusargs and seeds are the general mechanism. Chapter 27.5 applies it to the specific case everybody meets first — launching one UVM test with its verbosity — and sorts out which of those arguments the simulator consumes and which the testbench reads.

Where this fits

Part of the Linux & Bash curriculum.