Reference Models: What They Are and What They Are Not

Part II · Module 18 · Chapter 18.2 · Reference Models: What They Are and What They Are Not

1. What It Is For

A reference model exists so that a disagreement means something.

The design produces an output. The model produces an output. A scoreboard compares them, and if they differ, somebody looks. The entire value of the arrangement rests on one property: the two were arrived at independently, so that a disagreement is evidence rather than noise.

That property is not automatic, and it is not visible from the outside. A model that agrees with the design looks identical whether the agreement is a vindication or a tautology — and this chapter measures both.

There are two ways to get it wrong, and they fail in opposite directions:

• Too close to the design — the model reproduces the design's mistakes and catches nothing.
• Too much of the design — the model reproduces things the specification never required, and reports failures against correct hardware.

Both are common. The first is more dangerous and the second is more annoying, which is why the second gets fixed and the first does not.

2. Too Close: The Model That Catches Nothing

A packet header carries a payload length. The specification is explicit: bits 11:0, in words; payload bytes are words × 4.

The design has a bug — a 10-bit mask where the specification says 12:

static uint32_t dut_payload_bytes(uint32_t hdr) {
    uint32_t words = (hdr >> 0) & 0x3FFu;      /* BUG: 0x3FF should be 0xFFF */
    return words * 4u;
}

Two models of the same thing. One written from the specification:

static uint32_t model_a(uint32_t hdr) {
    uint32_t words = hdr & 0xFFFu;             /* 12 bits, per the spec */
    return words * 4u;
}

And one written the way models frequently are written — by opening the RTL and transcribing what it does:

static uint32_t model_b(uint32_t hdr) {
    uint32_t words = (hdr >> 0) & 0x3FFu;      /* copied from the design */
    return words * 4u;
}

Both run against the design over the entire 12-bit input space:

headers exercised          : 4096
model A (from the spec)    : 3072 mismatches -> the bug is FOUND
model B (from the design)  : 0 mismatches    -> the bug is INVISIBLE

first divergence at hdr=0x400: dut=0  spec=4096

Model B passed every single test. Not most of them — all 4096, against a design that computes the wrong answer for three quarters of its input space. A regression built on model B is green, has full input coverage, and tells you nothing at all.

Model A found the bug on 3072 of 4096 inputs, which is every header with bit 10 or bit 11 set.

The mechanism is not subtle and that is the point: model B shares the design's assumption, so it cannot disagree with it. The shared expression is the bug, and copying it copied the bug.

3. Too Much: The Model That Cries Wolf

The opposite failure, and it is the one this chapter's purpose names specifically — timing, microarchitecture, and the design's internal structure.

A FIFO. The specification says one thing about ordering: bytes come out in the order they went in. It says nothing about how deep the output pipeline is, because that is the designer's business.

Two models. One checks what the specification says:

/* reproduces ONLY order */
static int om_expect(ordmodel_t *m, uint8_t got) {
    return m->head < m->tail && m->q[m->head++] == got;
}

The other also reproduces the pipeline depth, because the model's author knew the design had three stages and encoded it:

/* also asserts WHEN, not just WHAT */
static int tm_check(timedmodel_t *m, unsigned cycle_of_first_output) {
    return cycle_of_first_output == m->depth;    /* depth was 3 when written */
}

Run against two revisions of the design — the original three-stage pipeline, and a later revision with four stages, which is a legitimate change the specification permits:

design rev with pipeline depth 3:
   bytes out        : 4   order model says PASS
   first byte at cyc: 3   timing model says PASS

design rev with pipeline depth 4:
   bytes out        : 4   order model says PASS
   first byte at cyc: 4   timing model says FAIL

The depth-4 design is correct. Every byte came out, in order, exactly as specified. The order model says so. The timing model reports a failure against hardware that does everything the specification requires.

What happens next is the real cost. Somebody investigates the failure, discovers the design is fine, and updates the model to expect four. The model is now coupled to a microarchitectural detail that will change again — and worse, the team has learned that failures from this model are often spurious, which is precisely the habit that lets a real failure be dismissed.

4. The Question That Separates Them

Both failures are answered by one question, asked of every line of the model:

Is this in the specification?

If yes, the model should reproduce it, and a disagreement is a bug in the design.

If no, the model must not reproduce it, and encoding it converts the model from an oracle into a second implementation that has to be maintained in lockstep.

Applied to the three things the chapter's purpose names:

Timing. Unless the specification states a cycle count or a latency bound, the model must not predict when. It predicts what, and the testbench's job is to collect outputs and compare them in order. Where the specification does give a bound — "the result is available within 8 cycles" — model the bound, not the design's actual latency.

Microarchitecture. Pipeline depth, buffer sizes, arbitration internals, state encodings. None of these appear in a specification that was written properly, and a model that knows them is a model that breaks when the designer does their job.

The design's internal structure. A model should not have the same module hierarchy as the RTL. If it does, it almost certainly came from reading the RTL, which is §2's failure — the two are related and they often arrive together.

5. What a Model Looks Like When It Is Right

Three properties, all of which follow from §4's question.

It is written in terms of the specification's vocabulary. If the document talks about packets, fields and transactions, the model's functions are named for those — not for the design's modules. A model whose function names match the RTL's module names came from the RTL.

It is a function of inputs to outputs, with explicit state where the specification has state. Not a cycle-by-cycle simulation. Chapter 16.1's peripheral model is a reasonable example: it implements the documented access type of each register, and it does not model a bus, a clock, or how many cycles a read takes, because the datasheet's access-type column says nothing about those.

Its state is the specification's state. A protocol with three defined states gets three states. If your model has eleven, you have modelled an implementation.

And one practical habit, which is Chapter 16.8 §7's discipline applied here:

/* Payload length. Datasheet rev C §4.2: bits 11:0, in words.
   Bytes = words * 4. Reserved bits 31:12 are not part of the length. */
static uint32_t model_payload_bytes(uint32_t hdr) {
    return (hdr & 0xFFFu) * 4u;
}

The comment is not decoration. It makes the review question answerable: a reviewer can check that line against §4.2 without reading any RTL — which is exactly the check §2's callout said was being skipped.

6. What a Model Is Not

Three things it is explicitly not, each of which someone will ask for.

Not a performance model. Predicting throughput or latency is a different job with different fidelity requirements, and conflating it with a functional reference is how you end up with §3's failure.

Not an implementation. If the model is as complex as the design, it has the same probability of being wrong, and you have doubled the work while halving the independence. A model should be simpler than the design — that is most of where its reliability comes from.

Not a specification. When the model and the document disagree, the document wins, or you fix the document. A model that has quietly become the authority is one nobody can check, which is §7's subject.

7. The Misconception: Agreement Means Correct

"The model and the design agree across the full regression, so the design is correct."

Measured false in the most direct way available: model B agreed with a broken design on all 4096 inputs.

The correction is one sentence and it is worth holding exactly:

Agreement is evidence in proportion to independence. Two implementations that share an assumption cannot disagree about it, so their agreement says nothing about that assumption.

Which makes "do the model and the design agree?" the wrong question to ask of a verification environment. The useful questions are about provenance:

• Where did the model come from — the specification, or the design?
• Who wrote it, relative to who wrote the RTL?
• When model and design last disagreed, which one was changed?

That third question is the most diagnostic and the least asked. A model that has been updated every time it disagreed has been systematically converged onto the design, and the project has been paying for a model that can no longer produce the signal it was bought for.

And this is where the term "golden model" earns a warning. It is used for two different things. One is an architectural reference — the independently-derived model this chapter has been describing, often maintained by an architecture team and shared across projects. The other is "the model we have decided to trust", which is a social fact rather than a technical one. A model that became golden by surviving a few projects, being patched each time it disagreed, is exactly model B with a better reputation.

When someone calls a model golden, the question to ask is the same one: where did it come from, and what was changed the last time it disagreed with the hardware?

8. What You Should Be Able to Say Now

1. A reference model exists so a disagreement means something — which requires the model and the design to have been arrived at independently.
2. Two opposite failures: too close to the design (catches nothing) and reproducing too much of it (false failures).
3. Measured, too close: a design-derived model found 0 mismatches in 4096 inputs against a design with a 10-bit mask where the spec says 12. The spec-derived model found 3072.
4. It arrives by reasonable steps — the RTL exists and is unambiguous, each "fix" to the model looks correct, and review checks the model against the RTL rather than the specification.
5. Measured, too much: a model that encoded the pipeline depth failed a correct design when the depth went from 3 to 4. The order-only model passed both.
6. The real cost of a false failure is that the team learns to discount this model's failures.
7. One question per line: is this in the specification? If yes, model it; if no, do not.
8. Timing, microarchitecture and internal structure are the three to exclude — model a specified bound, never the design's actual latency.
9. A model should be simpler than the design. If it is as complex, you have doubled the work and halved the independence.
10. Agreement is evidence in proportion to independence. Ask where the model came from and what was changed last time it disagreed.

9. Exercises

1. Reproduce §2's three functions and confirm 3072 against 0. Then change the DUT's mask to 0x7FF and predict the new mismatch count before running.
2. Write model B's "fix" that a developer would apply after the first disagreement — update it to match the design — and state exactly what the project has lost.
3. Reproduce §3's FIFO demonstration. Then add a specification clause — "the first byte is available within 5 cycles" — and write the model that checks that rather than the depth. Confirm it passes both revisions.
4. For a block you know, list three things in its RTL that a reference model must not reproduce. For each, say what the specification does say instead.
5. Take §5's commented model function and write the equivalent for a field of your choice, including the document reference. Then hand it to someone and ask whether they can check it without the RTL.
6. A model and a design disagree. Write the three questions you would ask, in order, before anyone changes either.
7. Someone describes a model as golden. Write the two questions from §7 and say what answers would reassure you.

10. Summary

• A model exists so disagreement means something — and that requires independence, which is invisible from the outside.
• Two opposite failures: too close (catches nothing) · too much (false failures).
• Measured too close: 0 of 4096 mismatches against a broken design, versus 3072 for the spec-derived model.
• Measured too much: a depth-encoding model failed a correct design when the pipeline went 3 → 4.
• The question per line: is this in the specification?
• Exclude timing, microarchitecture and internal structure. Model a specified bound, not an observed latency.
• Simpler than the design, in the specification's vocabulary, with the document reference in the comment.
• Not a performance model, not an implementation, not a specification.
• Agreement is evidence in proportion to independence. "What was changed last time it disagreed?" is the diagnostic question — and the one that tells you whether a "golden" model is architectural or merely venerable.

Independence is easiest to achieve when the model did not come from your project at all. For anything that implements a published standard — a checksum, a cipher, a compression format — an implementation already exists that thousands of people have exercised and that owes nothing to your team's reading of the specification, which makes adopting one the cheapest independence available and the subject of the next chapter.