Connecting a C Model to a Scoreboard

Part II · Module 24 · Chapter 24.4 · Connecting a C Model to a Scoreboard

1. Correct Model, Correct Design, Wrong Answer

Chapter 24.3 produced an independent model. Chapter 24.2 drew the arrows. This chapter is what each arrow turns out to cost.

The setup is deliberately the easiest case imaginable: the model is right and the design is right. sat_acc_lat.v implements SPEC.txt exactly — the same arithmetic as the clean design from Chapter 24.2, with two cycles of output latency and a result_valid flag to say when a result is ready, which is how a pipelined block normally behaves.

Nothing is wrong with either side. Measured, thirty transactions:

  compared at the drive cycle  : 30 checks, 16 mismatch(es), first at 1
  compared when the DUT says so: 30 checks,  0 mismatch(es)

Sixteen mismatches from a design with no bug in it. The model was right every time, the design was right every time, and the comparison was wrong.

2. Pairing

The first and largest decision: which expected result goes with which actual one.

The naive version reads the design's output at the moment the sample was driven:

@(negedge clk); sample = s; valid = 1;
@(negedge clk); valid = 0;
t = $m_check(0, s, r_lat);          // compare now

That is a decision, made implicitly. It assumes the result for a sample is available at the cycle the sample was accepted — true for the combinational-plus-register design in Chapter 24.2, false for anything pipelined.

The paired version waits for the design to say so:

@(negedge clk); sample = s; valid = 1;
@(negedge clk); valid = 0;
while (rv !== 1'b1) @(negedge clk);  // the design's own ready signal
t = $m_check(1, s, r_lat);

Measured, 0 mismatches in 30.

The difference is not cleverness, it is whose knowledge is used. The naive version encodes the testbench author's belief about latency; the paired version asks the design. Only one of those stays correct when the pipeline depth changes.

3. Why Sixteen and Not Thirty

The number matters more than it looks, and it is the reason this failure survives.

A pairing error compares each expected value against a stale actual one. With the stimulus used here — a constant sample driving the accumulator toward its bound — the expected sequence eventually stops changing: once the accumulator saturates, consecutive expected outputs are equal, and a stale value is then indistinguishable from a fresh one.

So the mismatch rate is partial. Sixteen of thirty.

That is exactly what an intermittent design bug looks like from the outside: most transactions fine, a cluster wrong, no obvious pattern. The investigation goes to the design, where there is nothing to find — and Chapter 24.9 is about how long that takes when nobody suspects the comparison.

Had the rate been 30 of 30, somebody would have questioned the harness in the first five minutes.

4. The Other Three Decisions

Ordering. The model must be stepped once per transaction, in the order the design accepted them — not in the order the testbench generated them, and not in the order a monitor happened to observe them. Chapter 24.1 §4 measured why this is not pedantry: the same forty samples in reverse order produced outputs coinciding in 2 of 40 places. For a stateful model the order is the stimulus.

The practical form: step the model at the point the design accepts a transaction, from the same event, so there is no second notion of "next".

State ownership. Who resets the model, and when. Measured in Chapter 24.1 §4: a model continuing from a previous run's state matched 0 of 40 outputs against a fresh run. The model's reset must be driven by the same event as the design's — if the design is reset mid-test, so is the model, from the same line of code.

That is also why the model's state belongs behind a per-instance handle rather than in a file-scope variable: Chapter 22.7 measured three designs agreeing at 0x52 with one stream and separating immediately with two, and a scoreboard with two agents is exactly the two-stream case.

Data ownership. Whose buffer the payload is in. Chapter 21.7's rule is unchanged — an argument is valid for the duration of the call, so anything the model keeps must be copied on arrival, into storage the model owns. A scoreboard that stores a pointer and compares it later is Chapter 21.7 §5's stale-pointer problem with a scoreboard's name on it.

5. The Misconception: A Mismatch Means the Design Is Wrong

"The scoreboard fired. The model is independent and well reviewed, so the design is broken."

Measured false, directly: 16 mismatches from a design that meets its specification exactly, produced entirely by comparing at the wrong moment.

The three candidates for any mismatch are the design, the model, and the comparison — and the third is the one that is usually left off the list, because it feels like plumbing rather than logic.

What makes it expensive is not the hours spent looking at the wrong thing. It is the second-order effect:

False mismatches teach a team to disbelieve the scoreboard. After a few investigations that end in "the harness was wrong", the next mismatch is triaged as probably-the-harness, and the first real bug is triaged the same way. A scoreboard that cries wolf is worse than no scoreboard, because no scoreboard at least does not create confidence it then destroys.

So the practical order when a mismatch appears, before any RTL is opened:

• Does the mismatch rate make sense? 16 of 30 with no pattern is a harness smell; Chapter 24.2's 2 of 70 is what a genuine narrow bug looks like.
• Are the two values from the same transaction? The report should answer this, per §4's callout.
• Did the model and the design see the same reset? Chapter 24.1 §4's 0 of 40.
• Then look at the design.

The accurate position:

A mismatch means the design, the model and the comparison do not all agree. Which of the three is wrong is a question, and the comparison is the cheapest to rule out — Chapter 24.9 is the full treatment.

6. What You Should Be Able to Say Now

1. A correct model and a correct design still need a correct comparison — four decisions stand between them.
2. Pairing is the largest: measured, 16 mismatches in 30 checks from a spec-conformant design, and 0 when the design's own ready signal decides when to compare.
3. The paired version asks the design rather than encoding the testbench author's belief about latency.
4. Sixteen and not thirty — a stale value sometimes coincides, so a pairing error presents as a partial rate, which is what an intermittent design bug looks like.
5. Ordering: step the model once per transaction in the design's order — 24.1 measured 2 of 40 coinciding in reverse.
6. State ownership: the design's reset resets the model, from the same event — 24.1 measured 0 of 40 without one.
7. And the model's state belongs behind a per-instance handle (22.7), because a scoreboard with two agents is the two-stream case.
8. Data ownership: valid for the call, so copy on arrival (21.7).
9. Report what was compared, not just whether it matched: check number, time, stimulus, both values — and both transaction indices when there is latency.
10. A mismatch means the three do not all agree — design, model, comparison — and the third is the cheapest to rule out.
11. False mismatches cost belief, and a scoreboard nobody believes is worse than none.

7. Exercises

1. Take a spec-conformant design with output latency and compare at the drive cycle. How many mismatches, and how many transactions did you drive?
2. Change the comparison to wait for the design's ready signal. What is the new count?
3. Why is the mismatch rate partial rather than total? Construct a stimulus that would make it total, and say why that version would be easier to debug.
4. Step a stateful model in a different order from the one the design accepted. Predict the mismatch rate before measuring it.
5. Reset the design mid-test without resetting the model. What does the scoreboard report, and which of the three candidates is actually wrong?
6. Write the mismatch report line for a scoreboard you have. Which of §4's fields are missing, and which bug would each have caught?
7. Add both transaction indices to a mismatch report. Construct a pairing error and show that the report now identifies it without opening the design.
8. §5 says false mismatches cost belief. Describe the sequence by which a team stops trusting a scoreboard, and what it costs when a real bug arrives.

8. Summary

• Four decisions between a correct model and a correct comparison: pairing · ordering · state ownership · data ownership.
• Measured: 16 mismatches in 30 comparing at the drive cycle, 0 comparing when the design says the result is ready — on a design with no bug.
• Ask the design about latency; do not encode a belief about it.
• 16 and not 30 — a partial rate, which reads as an intermittent design bug.
• Order is the stimulus (2 of 40 reversed) · the reset is one event for both sides (0 of 40 without) · a payload is valid for the call (21.7).
• Model state goes behind a per-instance handle (22.7) — two agents is the two-stream case.
• Report what was compared: check, time, stimulus, both values, both transaction indices.
• A mismatch means design, model and comparison do not all agree — rule out the comparison first.
• False mismatches cost the team's belief in the scoreboard.

The comparison path is wired and the decisions are written down. What flows through it has been a single arithmetic result from a block invented for this module — which is enough to demonstrate the architecture and not enough to demonstrate that a model can be trusted. The next three chapters build models of things that already have right answers published somewhere.