Where the Shell Sits in an RTL and Verification Flow

Part II · Module 22 · Chapter 22.2 · Where the Shell Sits in an RTL and Verification Flow

Ask an engineer what they use to run a simulation and the answer is a tool name. Watch what they actually type and it is a shell command, which runs a script, which builds a command line, which starts a tool, which writes a log, which another script reads.

The shell is not around the edges of that. It is the connective tissue, and in most teams it is the flow.

1. The Shell's Four Jobs in a Flow

Four of those six steps are the shell's, and the two that are not still communicate only in things the shell understands — an exit status, a file, a stream. Chapter 22.1 called that assumption two; this chapter is what it looks like in practice.

2. A Real Flow, Two Stages

Everything below was run with Icarus Verilog 13.0, which is a real compiler and a real simulator. A four-bit counter and a self-checking testbench:

module counter (input clk, input rst_n, output reg [3:0] q);
  always @(posedge clk or negedge rst_n)
    if (!rst_n) q <= 4'd0;
    else        q <= q + 4'd1;
endmodule

module tb_counter;
  reg clk = 0, rst_n = 0;
  wire [3:0] q;
  integer errors = 0;
  counter dut (.clk(clk), .rst_n(rst_n), .q(q));
  always #5 clk = ~clk;
  initial begin
    $display("TB: starting");
    @(negedge clk); rst_n = 1;
    repeat (4) @(posedge clk);
    #1;
    if (q !== 4'd4) begin
      $display("ERROR: expected q=4, got q=%0d", q);
      errors = errors + 1;
    end else $display("TB: q=%0d as expected", q);
    if (errors == 0) $display("TEST PASSED");
    else             $display("TEST FAILED with %0d error(s)", errors);
    $finish;
  end
endmodule

Stage one turns source into something runnable:

$ iverilog -o sim.vvp counter.v tb_counter.v
$ echo $?
0
$ ls
compile.log  counter.v  sim.vvp  tb_counter.v

Stage two runs it:

$ vvp sim.vvp
TB: starting
TB: q=4 as expected
TEST PASSED
tb_counter.v:18: $finish called at 46 (1s)
$ echo $?
0

Two commands, two statuses, one artifact passed between them by name. That is the whole structure of every EDA flow you will ever meet, however many tools and however many stages: a sequence of programs, each reading files and writing files, each returning a number, joined by a shell.

A commercial flow has more stages — separate compile, elaborate and run steps, with vendor-specific names and a work library on disk rather than a single file — but the shape does not change, and neither does what the shell has to do about it.

3. The Measurement That Defines Part II

Now break the design. One character: the counter increments by two.

    else        q <= q + 4'd2;

Compile it, run it, and read both statuses:

$ iverilog -o sim.vvp counter.v tb_counter.v
$ echo $?
0
$ vvp sim.vvp
TB: starting
ERROR: expected q=4, got q=8
TEST FAILED with 1 error(s)
tb_counter.v:18: $finish called at 46 (1s)
$ echo $?
0

Zero. The design is wrong, the testbench detected it, the log says TEST FAILED in plain English — and the simulator told the shell that everything was fine.

For contrast, break the source instead, so the tool itself cannot proceed:

$ cat broken.v
module broken  // no semicolon
endmodule
$ iverilog -o b.vvp broken.v
broken.v:2: syntax error
I give up.
$ echo $?
2

Now there is a status. Which gives the asymmetry that the whole of Part II is organised around:

A tool failing gives you a number. A test failing gives you nothing but text. Everything in Part II about logs, regressions and defensive automation exists because of that one row.

4. How a Testbench Ends Decides What the Shell Is Told

The status is not fixed by the simulator. It is chosen by whichever system task ends the simulation. Measured, every row run:

Two of those rows deserve to be read twice.

$finish(2) does not exit 2. The argument to $finish controls how much diagnostic information the simulator prints on its way out — nothing more. If you have ever written $finish(1) believing it signalled failure, it did not:

$ vvp s.vvp
TB: starting
TEST FAILED
tb.v:4: $finish(2) called at 0 (1s)
$ echo $?
0

And $fatal does set a non-zero status, which makes it the mechanism rather than a last resort:

$ vvp s.vvp
TB: starting
TEST FAILED
FATAL: tb.v:4: bad
       Time: 0  Scope: tb
$ echo $?
1

5. The Testbench Side of the Contract

Given §4, a self-checking testbench has one more job than most people give it: convert its error count into an exit status.

module tb2;
  integer errors = 2;
  initial begin
    $display("TEST FAILED with %0d error(s)", errors);
    if (errors != 0) $fatal(1, "%0d error(s)", errors);
    $finish;
  end
endmodule

$ vvp s2.vvp
TEST FAILED with 2 error(s)
FATAL: tb2.v:5: 2 error(s)
       Time: 0  Scope: tb2
$ echo $?
1

Now the shell can do its job with no log parsing at all:

iverilog -o sim.vvp -f files.f && vvp sim.vvp && echo "PASS" || echo "FAIL"

That is Chapter 11.3's chaining doing exactly what it was taught to do, and it only works because the testbench kept its half of Chapter 21.4's contract.

6. The Hang, and Why < /dev/null Belongs on Every Batch Command

One row of §4's table is worse than a wrong status, and it is the row that catches people moving work from a terminal into a queue.

$stop suspends simulation and drops into the simulator's interactive debugger. With standard input closed, it has nothing to read, gives up, and continues:

$ vvp s.vvp < /dev/null
TB: starting
TEST FAILED
tb.v:5: $stop called at 0 (1s)
** VVP Stop(0) **
** Flushing output streams.
** Current simulation time is 0 ticks.
** Continue **
$ echo $?
0

With standard input inherited from something that is open but will never send anything — a pipe, a scheduler's job wrapper, a ssh session with no terminal — it waits:

$ vvp s.vvp > o2.log 2>&1 &
[1] 41204
$ sleep 3; kill -0 %1 && echo "STILL RUNNING after 3s"
STILL RUNNING after 3s
$ cat o2.log
TB: starting
TEST FAILED
tb.v:5: $stop called at 0 (1s)
** VVP Stop(0) **
** Flushing output streams.

It printed up to the flush and stopped there. Not an error, not an exit, not a log line to grep for — a job that holds its slot and its licence until someone notices. Note also that the output ends mid-sequence, so even the log looks like a crash rather than a prompt.

The defence is one redirect on every tool invocation in a script:

vvp sim.vvp < /dev/null > "$log" 2>&1

A batch tool should never have a usable stdin. If it asks a question, you want it to fail immediately rather than wait forever — and < /dev/null is how you say "there is nobody here to answer". Chapter 12.7 introduced /dev/null as a place to send output; this is the same file used as an empty input, which is the use that matters in automation.

7. Why the Flow Itself Is Written in Shell

There is a fair objection to all of this: if the flow is this fiddly, why is it a shell script rather than a program?

Three reasons, and they are the reasons this course exists:

The interface to every tool is a command line. Whatever language you use, the operation is "build an argument list, start a process, collect its status and its files". That is the shell's native data model and everything else's foreign one. Chapter 19.5 is the chapter that made this safe.

The environment is the tool's configuration. Which simulator version runs is decided by PATH, and which licence server by LM_LICENSE_FILE — both environment variables, inherited by children. Chapter 10.1 and Module 25 are the two halves of that story.

And a flow is mostly sequencing and decisions, not computation: run this, check that, if it failed collect these files, otherwise go on. A hundred lines of &&, if and case is the honest expression of that, and it is readable by the next engineer at three in the morning.

The place where shell stops being the right answer is also worth stating plainly: when the job becomes parsing structured data and computing summaries, a shell pipeline turns into line noise and Python is better. The industry convention is close to the honest boundary — shell to run tools and move files, Python to parse results and produce reports — and Chapter 13.9 §7 drew the same line for pipelines in general.

8. Where the Shell Appears in a Day's Work

To make the position concrete, the places a verification engineer touches the shell between morning and evening, each a forward reference to a module that takes it seriously:

Every row is a Part I technique meeting a tool that does not quite behave. None of it is a new language.

Practice

1. Install Icarus Verilog if you can (brew install icarus-verilog, or your distribution's package) and reproduce §2 end to end. Confirm both statuses are 0.
2. Break the counter to + 4'd2 and run again. Confirm vvp still exits 0 while the log says TEST FAILED. This is the single most important thing in this chapter — see it yourself.
3. Break the syntax instead and confirm iverilog exits non-zero. Note the number.
4. Write a testbench ending in $finish(2) and confirm the exit status is 0, not 2.
5. Change it to $fatal(1, "…") and confirm the status is 1. Then write the if (errors != 0) $fatal(...) form and check both the pass and fail cases.
6. Write run-one from §5. Test it four ways: a passing test, a failing test, a compile error, and a log with no verdict line. Confirm you get 0, 1, 2, 2.
7. Then delete the "no verdict" branch so the function falls through, and explain what a truncated log would now report.
8. Run a $stop testbench with < /dev/null and then without, from a pipe. Confirm one continues and the other hangs. Kill the hung one and note what status wait reports.
9. Add < /dev/null to every tool invocation in a script you already have. For each one, say what question the tool might have asked.
10. Write the watchdog from §6 with a two-second timeout against a deliberately hung simulation. Confirm the status you get is 143, and explain the number.
11. For each row of §8's table, write down what you currently do in that moment, and whether it is repeatable by someone else.

Check Your Understanding

1. Which steps of a simulation flow belong to the shell and which to the tools?
2. A design is wrong and the testbench detects it. What exit status does the simulator give?
3. What status does a compile error give, and why is that different?
4. What does the argument to $finish control?
5. Which system task does set a non-zero exit status?
6. Why does $stop sometimes exit 0 and sometimes hang forever?
7. Why should a batch tool be run with < /dev/null?
8. Why is "no verdict found in the log" a distinct outcome from "the test failed"?

Chapter 27.5 works through the invocation most verification engineers meet first, sorting its arguments by which layer actually consumes each one.

9. Remember

• The shell is the flow, not a wrapper around it: it assembles, sequences, decides and reports.
• Tools talk to the shell in three things only — an exit status, files, and streams.
• A tool failing gives you a number; a test failing gives you text. This asymmetry is what Part II is about.
• Measured: a broken design gave vvp exit 0 with TEST FAILED in the log. A syntax error gave iverilog exit 2.
• That is not a simulator bug. Correctness is the testbench's question, and its answer goes to $display.
• $finish(N) is a verbosity level, not an exit code. $finish(2) exits 0.
• $fatal exits non-zero — so if (errors != 0) $fatal(1, …) is how a testbench keeps the contract.
• When you cannot change the testbench, decide the verdict once and convert it to a status immediately — and give "no verdict" its own code.
• $stop exits 0 with < /dev/null and hangs with an inherited pipe. Run every batch tool with < /dev/null.
• A hung job holds its slot and its licence, and its log looks truncated rather than stuck.
• A terminated process reports 128 + N, so a watchdog kill shows up as 143, never as a test result.
• Shell to run tools and move files; Python to parse results and compute summaries.

10. What Is Next

Chapter 22.3 takes assumption three apart: the workstation you log into, the machines your jobs run on, and the one filesystem both are supposed to see at the same path — including the measured reason a path can be correct and still mean two different things.