Simulator Crashes and How to Localize Them

Part II · Module 25 · Chapter 25.8 · Simulator Crashes and How to Localize Them

1. What You Are Handed

Measured, a run that faults inside a function the simulator called:

  exit: 139
  output lines: 0

Nothing else. No message, no file, no line — and, importantly, three $display statements had already executed.

So the starting position is worse than it looks: not only is there no diagnosis, the evidence of how far the run got is also gone. Chapter 25.1 §5 measured why, and §2 is the first method.

2. Method One: Make the Output Survive

Cost: one environment setting. No rebuild.

stdbuf -o0 vvp -M. -mmylib tb.vvp

Measured, the same crash:

  exit: 139
  [SV] initialising the table with 0 entries
  [SV] init returned 0
  [SV] about to read entry 3

The run reached the call that killed it, and now says so. That alone localises most crashes to a transaction.

And once the output survives, a trace line inside the C becomes worth adding. Measured, with one vpi_printf at the top of the faulting function:

  [C] table_get(3) with TABLE=0x0 N=0

That names the bug outright: the table pointer is null, the count is zero, and index 3 was requested. No debugger was involved, and the fix is visible from that one line.

The general form is Chapter 23.6's: print what you were handed before you use it. A crash is the case where that habit pays for itself immediately, because the last successful print is the last thing that happened.

3. Method Two: A Backtrace Across Both Worlds

Cost: one command, plus a build with debug information.

Measured:

  stop reason = EXC_BAD_ACCESS (code=1, address=0xc)
  frame #0: crash_vpi.vpi`table_get(i=…) at crash_vpi.c:13:37 [inlined]
  frame #1: crash_vpi.vpi`get_call(u=…) at crash_vpi.c:22:12
  frame #2: vvp`vpip_execute_vpi_call(vthread_s*, __vpiHandle*) + 124
  frame #3: vvp`of_VPI_CALL(vthread_s*, vvp_code_s*) + 24
  frame #4: vvp`vthread_run(vthread_s*) + 80
  frame #5: vvp`schedule_simulate() + 768
  frame #6: vvp`vvp_run + 964
  frame #7: dyld`start + 6992

Read it from the top and the structure is exactly Chapter 23.4's:

• Frames 0 and 1 are yours — the faulting line and the glue that called it, with file names and line numbers.
• The boundary is between frames 1 and 2.
• Frames 2 to 6 are the simulator's — its VPI dispatcher, its thread runner, its scheduler, its entry point.

That is "one thread with two stacks" made literal, and it answers the question people usually ask first: whose code faulted. The frames above the boundary are the answer; the frames below are the callers.

It also means a crash in your C takes the whole simulation down, because there is no isolation — your function is running on the simulator's own thread, which Chapter 23.4 §4's callout established.

4. Method Three: Read the Faulting Address

Cost: nothing. It is already in the report.

  EXC_BAD_ACCESS (code=1, address=0xc)

0xc is 12. The element type is int, which is 4 bytes here, and 12 ÷ 4 = 3 — the index that was requested, from a null base.

A small faulting address is almost always a null pointer plus an offset, and the offset names what was being accessed:

• 0x0 — a null pointer dereferenced directly, or a member at offset 0
• a small multiple of an element size — a null array indexed at that position
• a small odd number — a member of a struct through a null pointer, at that byte offset

That takes seconds and often confirms what method one suggested, which is why it is worth knowing even though methods one and two are more thorough.

5. The Method, in Order

One — unbuffer and re-run. One setting. You now know how far it got, and often what it was given.

Two — check it is a crash at all. Chapter 25.7 §4 measured an exit 139 that was a load failure, not a fault in your code: a library with an unresolved dependency under lazy resolution. Loading once with immediate resolution distinguishes them and costs one run.

Three — add a trace line at the top of the suspect function, printing its arguments and any pointer it is about to use. Measured, that produced TABLE=0x0 N=0 and ended the investigation.

Four — if it is still not located, take a backtrace. Frames above the boundary are yours; read the topmost one with a file and line.

Five — read the faulting address as an offset from null, and check it against the index or member you suspect.

Six — then look at Chapter 25.5's lifetime table, because a large or wild address usually means freed or expired storage rather than a null, and that is a different chapter's problem.

6. The Misconception: The Simulator Crashed, So It Is a Simulator Bug

"The segmentation fault is inside the simulator process. That is a tool problem."

The process is the simulator's; the frame is not.

Measured, frames 0 and 1 are in crash_vpi.vpi — the loaded library — at crash_vpi.c:13 and crash_vpi.c:22. Frames 2 and below are the simulator's, and they are callers, not culprits.

Chapter 23.4 §4 established why a fault in your C brings the whole thing down: one thread, two stacks, no isolation. Your function runs on the simulator's own thread, so there is nothing to catch the signal and nothing to contain the damage. The simulator being the process that dies is a consequence of the architecture, not evidence about the cause.

The honest other half: sometimes it genuinely is the tool, and the backtrace says so too — frames of the simulator's own code with none of yours above them, in an operation you did not trigger. That is a real finding and worth reporting, with the backtrace attached.

What distinguishes the two is one line of output, which is why the backtrace is worth acquiring before the conversation rather than during it.

The accurate position:

A crash inside the simulator process tells you nothing about whose code faulted; the top frames tell you everything. Get them first — they cost one command — and the question answers itself.

7. What You Should Be Able to Say Now

1. The starting symptom is exit 139 with no output — measured, 0 lines with three $display statements already executed.
2. Method one: unbuffer. One setting, no rebuild — measured, all three lines survive.
3. Then add a trace line inside the C — measured, TABLE=0x0 N=0 named the bug without a debugger.
4. Method two: a backtrace, which crosses the boundary — frames 0 and 1 in your C with file and line, frames 2 onward the simulator's scheduler.
5. The boundary is between frames 1 and 2 — Chapter 23.4's two stacks on one thread, made visible.
6. Method three: read the faulting address — measured, 0xc = 12 = index 3 × 4 from a null base.
7. A small address is a null plus an offset, and the offset names what was accessed.
8. Check it is a crash at all — Chapter 25.7 §4 measured an exit 139 that was a load failure.
9. A large or wild address means expired storage, which is Chapter 25.5's table.
10. Bisection is a poor first move: runs are expensive, history-dependent faults move, and the information was already in the failing run.
11. Bisect when the backtrace has no frames of yours — then the smallest provoking stimulus is the useful artefact.
12. A crash in the simulator process says nothing about whose code faulted — the top frames do.

8. Exercises

1. Crash a C function called from a simulation and record the exit code and the number of output lines.
2. Re-run with unbuffered output. How many lines now, and what was the last thing that happened?
3. Add a trace line printing the arguments and pointers at the top of the suspect function. Does it name the fault?
4. Take a backtrace. Which frames are yours, and where exactly is the boundary?
5. Read the faulting address as an offset from zero. What index or member does it correspond to?
6. Produce an exit 139 that is not a crash in your code, using Chapter 25.7 §4's method. How would you tell the two apart?
7. Estimate the cost of bisecting a test that fails after twenty minutes, against the cost of the method in §5.
8. Construct a backtrace that would justify reporting a tool bug. What makes it different from this chapter's?

9. Summary

• Start: exit 139, 0 lines — measured, with three $display statements already run.
• Method 1, one setting: unbuffer — all three lines survive, plus any trace in the C (TABLE=0x0 N=0).
• Method 2, one command: a backtrace spanning both worlds — frames 0 and 1 yours with file and line, the rest the simulator's scheduler.
• The boundary is between frames 1 and 2 — 23.4's one thread, two stacks.
• Method 3, free: the faulting address — 0xc = 12 = index 3 × 4 from null.
• Small address = null plus offset; large or wild = expired storage (25.5).
• Confirm it is a crash: 25.7 §4 measured an exit 139 that was a load failure.
• Bisect last, and only when the backtrace holds none of your frames.
• The process dying is the simulator's; the faulting frame is yours — the top frames settle it.

A crash at least announces itself. The remaining failures in this module return cleanly, and the first of them is the one where the data that arrives is not wrong so much as diminished.