Compile, Elaborate, Run: Three Stages, Three Failure Modes

Part II · Module 27 · Chapter 27.1 · Compile, Elaborate, Run: Three Stages, Three Failure Modes

"The simulation is broken" is not a report anybody can act on. "Elaboration failed" is, and so is "it compiled, elaborated and ran, and the answer was wrong." The difference between those three sentences is the subject of this chapter, and it is the first thing Module 26's finished filelist runs into: the list is an input to a stage, and which stage consumes it decides what a failure means.

1. Three Jobs Wearing One Name

Every simulator does the same three things, whatever it calls them.

Analysis, or compilation, reads source files one at a time and turns each one into something the tool stores in a library. It is a per-file job: it can tell you that line 12 of tb_arb.vhd is not legal, and it cannot tell you anything about how your modules fit together, because it has not looked at more than one of them.

Elaboration builds the design. It starts at the top, resolves every instance to an actual module or entity, propagates parameters and generics, binds ports, and produces one connected structure. This is the stage that can say "you instantiated something that does not exist" or "that port does not exist on that module" — statements that are about the design, not about a file.

Execution runs the elaborated structure against time. Here the failures are the interesting ones: a wrong value, an assertion, a hang, a testbench that proves the design wrong.

Some tools expose all three as separate commands. Some fuse the first two. Both arrangements are on this machine, which is what makes the next two sections measurements rather than claims.

2. A Tool With Three Literal Commands

nvc is a VHDL simulator whose command-line interface is the three stages, one letter each.

$ nvc -a vhdl/arb.vhd vhdl/tb_arb.vhd    # analyse into the work library
$ echo $?
0
$ nvc -e tb_arb                          # elaborate the design rooted at tb_arb
$ echo $?
0
$ nvc -r tb_arb                          # run it
** Note: 85ns+1: TEST PASSED checks=3
** Note: 86ns+0: FINISH called
$ echo $?
0

Three commands, three exit statuses, and a library directory that persists between them. That persistence is the point: -e reads what -a stored, and -r reads what -e produced. Each stage is a real artifact on disk, which is why each can be rerun without redoing the one before it.

Now the inverse, which tells you what the stage boundaries are actually made of. Each stage gets exactly one deliberate fault, in its own library, with nothing else changed:

# a syntax fault: "end architecture" renamed so the trailing label will not match
$ nvc --work=work_a -a vhdl/arb.vhd faults/tb_a.vhd
** Error: expected trailing architecture body label to match SIM
  -a rc=1    -e not reached    -r not reached

A file-level complaint from the stage that reads files. Exactly as advertised, and notice the stage stopped the flow — there is nothing to elaborate, so attempting -e would be a second, less informative error.

3. The Stage Boundary Is Not Where I Expected

The second fault was supposed to be an elaboration error. I changed the testbench to instantiate work.arbiter, an entity that does not exist, and expected analysis to pass and elaboration to fail. It did not happen:

$ nvc --work=work_e -a vhdl/arb.vhd faults/tb_e.vhd
** Error: design unit ARBITER not found in library WORK
  -a rc=1    -e not reached

Analysis caught it. The reason is specific and worth knowing: a direct entity instantiation — dut : entity work.arb port map (…) — names a library unit, and nvc resolves that name while analysing the file, because the name is right there in the file it is reading. The stage boundary follows what is knowable from one file, and a direct entity instantiation makes the dependency knowable from one file.

To get an error that genuinely cannot be found until elaboration, the dependency has to be invisible to the file. A VHDL component does that: the component declaration is a self-contained promise about ports, and binding it to a real entity is deferred.

# tb_e2.vhd declares "component arb" and instantiates the component.
# arb.vhd is NOT analysed, so there is nothing for it to bind to.
$ nvc --work=work_u -a faults/tb_e2.vhd
  -a rc=0                                  # analysis is happy: the promise is well formed
$ nvc --work=work_u -e tb_arb
** Warning: (init): cannot find entity for component ARB without binding indication
  -e rc=0    0 errors, 1 warning

4. All Three Stages Green, and No Design Under Test

Look again at that elaboration result. Zero errors, one warning, exit status 0. The flow continues. So what does the run do?

$ nvc --work=work_u -r tb_arb
** Warning: 45ns+1: ERROR gnt mismatch
** Warning: 65ns+1: ERROR gnt mismatch
** Warning: 85ns+1: ERROR gnt mismatch
** Note:    85ns+1: TEST FAILED errors=3
$ echo $?
0

Read the three exit statuses together:

Every stage returned success, and the design under test was never in the design. The testbench drove an output nobody was driving, read back a value nobody produced, and correctly reported that it did not match. The one piece of evidence that the DUT was missing was a warning during a stage that reported success, and the one piece of evidence the run failed was a line of text in a run that reported success.

This is the single most important fact in Part II, arriving here because this is the first chapter with three stages to misread:

5. The Other Arrangement: Two Commands

Icarus Verilog fuses analysis and elaboration. iverilog reads the sources and builds the design, emitting one executable; vvp runs it.

$ iverilog -o build.vvp -f filelists/sim.f
$ echo $?
0
$ vvp build.vvp +TESTNAME=smoke
TB  test=smoke seed=1 cycles=40 verbosity=LOW
TB  TEST PASSED  test=smoke checks=3 errors=0

Two commands for three jobs, and that costs you something real. Compare the two tools' failure messages for the same class of problem — a filelist naming a file that is not there:

$ iverilog -o build.vvp -f filelists/sim.f
rtl/arb.v: No such file or directory
No top level modules, and no -s option.

Two messages from one command, and the second is a consequence of the first. No top level modules is an elaboration complaint — nothing could be elaborated because nothing was read — and it is printed by the command you were thinking of as the compile step. On a real design the first message scrolls away and the second one is the one you see, which is how an engineer ends up investigating a top-level naming problem that does not exist.

The practical consequence for a wrapper is a small one with a large payoff: log each stage separately even when the tool does not separate them, so the first message of the first stage is still findable.

# iverilog writes both the per-file and the design-level complaints to one
# stream, so capture it under the stage name you will look for later.
iverilog -o "$rundir/sim.vvp" -f filelists/sim.f > "$rundir/compile.log" 2>&1
crc=$?
printf 'compile rc=%d\n' "$crc" >> "$rundir/stages.txt"
[ "$crc" -ne 0 ] && { echo "compile failed, see $rundir/compile.log" >&2; exit "$crc"; }

vvp "$rundir/sim.vvp" +TESTNAME="$test" > "$rundir/run.log" 2>&1
printf 'run rc=%d\n' "$?" >> "$rundir/stages.txt"

Two files and a two-line summary. When somebody asks what broke, stages.txt answers in one cat and names the log to read.

6. A Stage Report Somebody Can Read

Here is the pattern generalised: a stage runner that records the stage name, its status and its log, and refuses to continue past a failure.

#!/bin/bash
# stage NAME RUNDIR -- CMD...   Run one stage, record it, stop the flow on failure.
set -u
name=${1:?stage: need a stage name}
rundir=${2:?stage: need a run directory}
shift 2
[ "${1:-}" = "--" ] && shift

mkdir -p "$rundir" || exit 2
log="$rundir/$name.log"

"$@" > "$log" 2>&1
rc=$?
printf '%-12s rc=%-3d %s\n' "$name" "$rc" "$log" >> "$rundir/stages.txt"

if [ "$rc" -ne 0 ]; then
  printf 'stage %s FAILED (rc=%d)\n' "$name" "$rc" >&2
  # the first line of a tool's output is almost always the originating error;
  # the rest is consequences. Module 29 makes this a rule.
  head -3 "$log" | sed 's/^/  /' >&2
  exit "$rc"
fi

Driving the three-stage tool with it:

$ stage analyse   runs/r1 -- nvc --work=runs/r1/work -a vhdl/arb.vhd vhdl/tb_arb.vhd
$ stage elaborate runs/r1 -- nvc --work=runs/r1/work -e tb_arb
$ stage run       runs/r1 -- nvc --work=runs/r1/work -r tb_arb
$ cat runs/r1/stages.txt
analyse      rc=0   runs/r1/analyse.log
elaborate    rc=0   runs/r1/elaborate.log
run          rc=0   runs/r1/run.log

And on the broken one, the flow stops where the problem is, and says so:

$ stage analyse runs/r2 -- nvc --work=runs/r2/work -a vhdl/arb.vhd faults/tb_a.vhd
stage analyse FAILED (rc=1)
  ** Error: expected trailing architecture body label to match SIM
$ echo $?
1

Two details carry their weight here. set -u means a missing argument is an error rather than an empty string, which is Chapter 20.3's rule. And exit "$rc" passes the tool's status outward rather than inventing one — the exit-code contract from Chapter 21.4, which matters because the caller is going to be a script.

What the runner deliberately does not do is decide whether the work was correct. Section 4 is why: at this layer the only honest claim is "the stage finished with status N". The verdict needs the log, and that is Chapter 27.9's job.

Practice

1. Run the three stages by hand on any VHDL pair you have, and confirm the library directory persists between them. Then rerun only -r and verify it works without re-running -a or -e.
2. Break the trailing label of an architecture and confirm the error names the stage's own concern (a file, a line) and not the design.
3. Reproduce the stage-boundary surprise. Instantiate a non-existent entity directly, and confirm analysis — not elaboration — rejects it.
4. Reproduce the unbound component. Declare a component, instantiate it, and analyse only the testbench. Record the exit status of all three stages.
5. Then predict, before running: what will the run report? Write down your answer, run it, and compare.
6. Count the evidence. For that unbound run, how many of the three exit statuses indicate a problem? How many lines of output do?
7. Give Icarus a filelist naming a missing file and read both messages. Which one would you have investigated if you had only seen the last line?
8. Write the stage runner and drive a full three-stage flow with it. Confirm stages.txt has three lines.
9. Prove your runner can fail by giving it a command that does not exist, and check that the exit status propagates rather than becoming 0 or 127-masked-as-success.
10. Add a fourth stage of your own — a lint pass, a filelist validation from Chapter 26.5 — and confirm it appears in stages.txt with its own log.
11. Deliberately gate on exit status only. Write the two-line wrapper that says PASS when the run command succeeds, run it on the unbound-component design, and watch it print PASS.

Check Your Understanding

1. What question can analysis answer that elaboration cannot, and the other way round?
2. Why did a missing entity turn out to be an analysis error rather than an elaboration error?
3. What kind of instantiation defers the binding to elaboration, and why?
4. In the unbound-component measurement, what were the three exit statuses?
5. What was the only evidence during elaboration that the design was incomplete?
6. Why is No top level modules printed by a command you think of as the compiler?
7. Why should a wrapper log each stage separately even for a two-command tool?
8. What does exit "$rc" preserve that exit 1 destroys?
9. Why does the stage runner refuse to decide pass or fail?
10. A colleague's flow reports PASS. What two things would you check before believing it?

7. Remember

• Three jobs, not one: analysis reads files, elaboration builds the design, execution runs it against time. Each can only fail in its own terms.
• A stage diagnoses what is knowable at that stage. Whether a missing module is an analysis or an elaboration error depends on how the instantiation was written.
• A direct entity instantiation is resolved during analysis; a component instantiation defers binding to elaboration, and an unbound component is a warning.
• All three stages returned 0 with no design under test in the design. This is the measurement to remember from Module 27.
• Exit status answers "did the tool finish", never "was the work correct." The two had opposite answers three times here.
• Log every stage separately, even when the tool gives you one command for two stages, so the originating error stays findable.
• Propagate the tool's status (exit "$rc"), and keep the verdict out of the stage layer.

Chapter 29.7 returns to these three logs and measures which of them names the cause — it is exactly one, and the other two are louder.

• Whether a flow is steps or a dependency graph decides how these stages are driven. Chapter 35.3 measures both, and what each one reimplements badly inside the other.

8. What Is Next

The stages are separate and each one's status is recorded. Chapter 27.2 builds the thing you hand them: a long tool command line assembled in an array, where one wrong expansion silently changes which test ran.