Debugging Habits That Prevent DPI Bugs

Part II · Module 25 · Chapter 25.11 · Debugging Habits That Prevent DPI Bugs

1. Judged by What Never Arrives

A habit is hard to justify because its entire output is an absence. The build that did not fail, the week not spent on a wrong number, the regression that did not pass while flattening every unknown it was given.

That absence is why habits get dropped. It is also why the only honest way to argue for one is to attach it to a failure that was actually measured — and this module measured nine of them, each with a number.

So nothing below is general advice. Every habit here is an instrument, built for one failure from one chapter, and it comes with the measurement that shows it works. A reader who disagrees with a habit can go and look at what it caught.

2. The Twelve

They sort cleanly by when they run, which is also the order of what they cost.

Before the run

List the symbols in the built library. One nm command. Chapter 25.6 measured the whole value of it: _crc8_step present means a build problem elsewhere, __Z9crc8_stephh means the file was compiled as C++, and absent means the translation unit never made it in. The link error alone cannot tell you which.

Make the callee count its own arguments. Three lines, measured in 25.2 returning 1, 2 and 4 for calls that declared one, two and four. It converts a wrong argument count — which 25.2 measured as exit 134, an assertion inside the simulator — into a message of your own with your own context.

Resolve symbols at load, not lazily. Chapter 25.7's finding, and it is the cheapest decision in the module: an unresolved dependency is named under immediate resolution and produces a silent exit 139 under lazy. The same defect, one configuration flag apart, and one of the two outcomes is unreadable.

Make one confirming call with a known answer. Chapter 25.2's closing point was that "it builds" says remarkably little. The strong form is a single call with an unmistakable result, run before anything else. The model in this chapter publishes one: 0x00 in gives 0xE1F0 out, measured, and any build that fails that is broken in a way no amount of compiling would have revealed.

In every run

Count the calls at each site. Chapter 25.10's whole instrument. Measured there: 10 against an expected 5, and 1 where four writes happened.

Check the unknown-bit word before using the value. One comparison, measured in 25.9 as 0, 4 and 0x20 for a clean value, one X and one Z. Without it, 25.9 measured 0xF0 arriving as 0xD0.

Carry a self-describing payload. Chapter 25.4 turned four indistinguishable array faults into four readable signatures with it — and measured the failure mode of getting it slightly wrong, where a 0x00 in the pattern made two of the four look identical.

Record every exit code. Chapter 25.2 measured 0, 1, 3 and 134; Chapter 25.7 and Chapter 25.8 both measured 139, from entirely different causes. Five distinct values, each meaning something specific, and all of them free.

Only when it fails

Unbuffer the output. One setting. Chapter 25.8 measured it recovering three $display lines that a crash had swallowed, plus the C-side trace that identified the fault.

Take a backtrace. One command, and 25.8 measured what it gives: frames 0 and 1 yours with file and line, the rest the simulator's scheduler, the boundary visible between them.

Turn on the stack-use-after-return detector. Chapter 25.5 measured it as the difference between 0 reports and 1 for the same defect, on this platform where it is off by default.

Drive each field's halfway value and maximum. Chapter 25.3's method: the smallest disagreeing input is always a power of two, so 0x80 names an 8-bit field and 0x8000 a 16-bit one. Realistic stimulus hides both.

3. The Four That Cost Something

The build-time habits are paid once and the failure-time ones are free until needed. The always-on four are the group anyone will push back on, so they were timed.

Two VPI functions doing identical work on the same argument, one bare and one running all four habits on every call: a per-site count, an argument-count check, an unknown-bit check, and a re-verification of the published check value. Two million calls each.

  trivial model          bare 1.59 s  ->  checked 2.22 s    delta 0.63 s
  model doing 64x work   bare 2.18 s  ->  checked 2.66 s    delta 0.48 s

That is roughly 0.24 to 0.32 microseconds per call, and the second line is the one worth reading twice: the delta did not grow when the model got sixty-four times heavier. The checks are a constant per call. Their share fell from about 40% of the trivial call to about 22% of the heavier one, purely because the denominator grew.

So the honest answer to "what do the checks cost?" is: a fixed amount per call, which means you cannot know the percentage without measuring your own model — and if your model does real work, the percentage is small and falling.

Two caveats, stated rather than buried. These are minimum-of-eleven-runs figures; this machine's run-to-run spread was wide — individual bare runs ranged 1.59 to 2.84 seconds and checked runs 2.22 to 6.03 — so the minima are the defensible numbers and the deltas are consistent across both model weights, which is why they are quoted that way.

And the checked run reported this at the end:

      [C] checked site: 2000000 calls, 0 with X/Z, 0 with wrong arity

Two million checks, nothing found. That is the normal case, and it is precisely why the cost question gets asked — the checks spend their entire life being right and the only evidence they are worth anything is the day one of them is not.

4. What the Always-On Four Look Like

Small enough to show in full, which is most of the argument for them.

static long calls, unknown, bad_arity;

/* habit: one counter per call site, not one global */
calls++;

/* habit: check what the caller actually passed */
if (argument_count() != 1) { bad_arity++; report("wrong arity"); return; }

/* habit: check for X or Z before using the value */
if (v.bval != 0) { unknown++; report("input contains X or Z"); return; }

/* habit: the published check value, re-verified on every call */
if (model(0x00) != 0xE1F0u) report("SELF-CHECK FAILED");

Three properties make this the right shape.

Each counter is per-site. A single global would have conflated Chapter 25.10's experiments into one meaningless total, and conflation is how the second caller stayed hidden in the first place.

Each check reports rather than asserting. A report keeps the run going and accumulates a count; Chapter 23.6 measured that a count is what distinguishes a check that is working from one that quietly stopped running. An assertion gives you the first instance and nothing about frequency.

The counts are printed at the end, always. 0 with X/Z is a result. A check whose output you only see when it fires cannot be told from a check that was compiled out.

5. The Habit Behind the Habits

One of the twelve is different in kind, and it is the one most often broken by people who know it.

Record the exit code, and never read it through a pipe.

simulator args > log.txt 2>&1 ; rc=$?       # rc is the simulator's
simulator args 2>&1 | head -20 ; rc=$?      # rc is head's

The second line looks like the first. It is not: $? after a pipeline is the last command's status, so a simulator that died reports whatever head thought of its day — invariably 0.

6. The Misconception: This Is Just Good Practice

"These are standard engineering hygiene. A generic checklist — validate your inputs, check return values, handle errors — would cover the same ground."

It would not, and the twelve habits are the counter-example.

Take the generic list seriously and apply it. "Validate your inputs" does not tell you that the unknown-bit word exists, that it must be read before the integer access, or that ignoring it turns 0xF0 into 0xD0. "Check return values" does not tell you that $? after a pipe is the wrong process's, or that 139 from the loader and 139 from a crash are different problems. "Handle errors" does not tell you that a missing symbol is named under one resolution mode and silent under the other.

Each habit here encodes a specific, measured, non-obvious fact — where the check must go, which instrument reports it, and what number it gives when it fires. A generic list has none of those, because a generic list is written before anyone knows what the failures are.

And the ordering is the real difference:

The measurement came first in every case. The habit is the residue of a failure that was observed, not a precaution that seemed sensible.

Which is also the honest test for anything you add to this list later. If you cannot name the failure it would have caught and the number it would have printed, it is not yet a habit — it is a preference, and preferences are what get dropped first when a schedule tightens.

7. Module 25 in One Page

Eleven chapters, and the structure is the same throughout: a symptom, an instrument, a measurement proving the instrument distinguishes what the obvious check cannot.

The sentence in the bottom bar is not a slogan. It was arrived at six separate times across this part of the track, each time from a measurement, and each time it was the reason an experienced engineer's reasonable check had failed to find anything:

• A test using only 8-bit elements confirms the layout — and cannot distinguish a correct stride from a wrong one, because they coincide at width one.
• A [0:7] range confirms the indexing — and cannot distinguish ascending from descending, because the two agree there.
• A symmetric matrix confirms the dimension order — and cannot distinguish row-major from column-major, because a symmetric matrix is its own transpose.
• A round trip confirms the encoder — and cannot distinguish a correct codec from a decoder matched to a wrong encoder.
• A single stream confirms the model's state handling — and cannot distinguish per-stream state from one shared accumulator, because with one stream running all three designs returned 0x52 (22.7).
• Two independently written models agree on 256 of 256 single samples from reset — and cannot be distinguished there at all, because a single sample never reaches the bound where they differ; on the design that mattered the same two models measured 51 mismatches against 0 (24.3, 24.9).

In every case the fix was the same in shape and different in content: a second measurement chosen specifically so the two candidates diverge. That is what every right-hand cell in Figure 2 is.

8. What You Should Be Able to Say Now

1. A habit's output is an absence, which is why it needs a measured failure attached rather than an argument from principle.
2. Four run before the run: list the symbols · make the callee count its arguments · resolve at load · one confirming call with a known answer.
3. Four run in every run: count calls per site · check the unknown-bit word · carry a self-describing payload · record every exit code.
4. Four cost nothing until needed: unbuffer · backtrace · the stack-use-after-return detector · each field's halfway value and maximum.
5. Measured cost of the always-on four: 2,000,000 calls, 1.59 s → 2.22 s for a trivial model and 2.18 s → 2.66 s for one doing 64× the work.
6. The delta is constant — about 0.3 µs per call — so the share falls as the model gets heavier: ~40% to ~22%.
7. Minimum-of-eleven, because the run-to-run spread on this machine was wide; the deltas agreed across both model weights.
8. The checks found nothing in two million calls, and that is the normal case.
9. A timing harness measures what the compiler produced — a "heavier" model that ran faster had 63 of 64 rounds elided.
10. $? after a pipe is the wrong process's status — broken twice in this module, both times reporting 0 for a real failure.
11. Per-site counters, reports rather than assertions, and counts printed always — 0 with X/Z is a result.
12. A generic checklist covers none of this, because it is written before anyone knows what the failures are.
13. The test for a new habit: name the failure it catches and the number it prints. If you cannot, it is a preference.
14. The module's thesis: the check that confirms a thing is usually the check that cannot distinguish it from its most likely failure — arrived at six times, from six measurements.

9. Exercises

1. Take the four always-on habits and add them to a model you have. Print the counts at the end of a run. What do they say?
2. Time your own model with and without them. What is the per-call delta, and what share of a call is it?
3. Make your model publish a check value. Which builds would it have caught that compiling did not?
4. Find a script of yours that reads $? after a pipeline. What status is it actually recording?
5. Run nm on your built library. Is every name you expect present, and is any of them mangled?
6. Which resolution mode does your flow use? What would an unresolved dependency look like under it?
7. Set up a backtrace before you need one. How long did it take, and how long would it take during an incident?
8. §6 says a generic checklist covers none of the twelve. Pick three habits and say what the generic form fails to specify.
9. Propose a thirteenth habit. Name the failure it catches and the number it prints — or discard it.
10. §7 lists six instances of one pattern. Find a seventh in your own work: a check you trust that cannot distinguish the thing it confirms from its most likely failure.

10. Summary

• A habit is judged by what never arrives, so each one here carries the measured failure it came from.
• Twelve, in three groups: before the run (symbols · arity · resolve at load · a confirming call), every run (per-site counts · the unknown-bit word · a self-describing payload · exit codes), when it fails (unbuffer · backtrace · the stack detector · boundary inputs).
• Measured cost, 2,000,000 calls, minimum of 11: 1.59 → 2.22 s trivial, 2.18 → 2.66 s at 64× the work.
• A constant ~0.3 µs per call — share falls from ~40% to ~22% as the model gets heavier, so measure your own.
• Two million checks, nothing found — the normal case, and the reason the cost question exists.
• A "heavier" model that ran faster had 63 of 64 rounds elided: a harness measures what the compiler produced.
• $? after a pipe is the wrong process's — broken twice here, reporting 0 for real failures both times.
• Per-site counters · report, do not assert · print the counts always.
• Generic advice covers none of it — the measurement came first in every case.
• The thesis, six times over: the check that confirms a thing is usually the check that cannot distinguish it from its most likely failure.

11. The End of the Track

This is chapter 25.11, and with it Module 25 is complete — and so is the C track: 217 chapters across 25 modules and two parts.

Part I built the language: representation, pointers, memory, the standard library, undefined behaviour, and the discipline of knowing which of the four categories any given behaviour belongs to. Part II took that language to the place verification engineers actually meet it — the boundary between a simulation and a program, where the compiler sees one side, the simulator sees the other, and nothing at all sees both.

That last fact is what Part II kept running into, and it is the honest summary of the whole second half: at this boundary the usual sources of truth are absent. There is no type checker spanning the two sides, no warning for a layout that disagrees, no error for a value that was quietly flattened, no report for a model that ran twice as often as its author believed. What remains is the thing this module spent eleven chapters on — a measurement, chosen so that the right answer and the likely wrong one give different numbers.

That habit transfers. It is not about C, and not about DPI.