Tcl · Module 29

When a Flow Succeeds and the Result Is Wrong

Every failure in this chapter exits zero with a plausible log. They are not bugs in the tool or in Tcl — they are questions nobody asked, and each one has an assertion that would have asked it.

Advanced22 min readtcledapart-iidebuggingassertionssignoff

Part II · Module 29 · Chapter 29.1 · When a Flow Succeeds and the Result Is Wrong

Module 14 covered errors: how Tcl raises them, how to catch them, what a stack trace says. This module assumes all of that and is about the failures that never raise anything.

They are the ones that reach silicon.

1. The shape

Figure 1 — the same fault, with and without an assertion. Above: a query matches nothing, which is not an error, so every step succeeds, the flow exits zero and a wrong result is signed off. Below: an assertion states what the step was expected to produce, so the flow stops there and exits non-zero — which converts a wrong sign-off into a broken run, and a broken run is something somebody fixes.

Every failure in this chapter has the same structure: a correct mechanism, applied to state nobody verified, producing a confident result. Nothing raises, because nothing that happened was illegal.

The cure is never a better error message — there is no error. It is an assertion: a statement of what the step was supposed to produce, checked against what it did.

2. The catalogue

This is not a new observation; it is the thread running through the whole of Part II. Collected, with where each was demonstrated:

The fault	What it produces	Where it was measured
A query matched nothing	an ECO that changed nothing	18.8, 23.6 §5
A floorplan command on an impossible input	111.6% utilisation, command succeeded	23.1 §5
Power straps in one direction only	connected, and no mesh	23.2 §3
A regex that matched the wrong thing	slack `-0.0` instead of `-0.0730`	24.1 §7
Units dropped from a value	leakage 98.2% of power instead of 5.2%	24.3 §2
A parser given an empty file	"PASS" on a zero-byte report	24.7 §2
A failing test's exit status	`$finish` exits 0 after `TEST FAILED`	25.4 §2
A testbench reading a DUT output	defect caught on 0 of 8 seeds	25.5 §5
A half-applied scenario	one corner's number under another's name	26.5 §3
Aggregating what exists, not what was planned	a verdict over 3 of 4 scenarios	26.6 §3
A pipe between the tool and the shell	exit 3 reported as 0	28.1 §2

Eleven instances, eleven chapters, one class. Naming it as a class is the point: the next one you meet will not be on this list, and recognising the shape is what lets you ask the right question about it.

3. Three faults in one flow

The mock below has three faults that can be switched on individually. Each one is one of the three the registry names.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

# fault 1: a query that matched nothing — an empty collection is not an error
proc size_up {pattern} {
    set hits [get_cells $pattern]
    foreach c $hits { }   ;# the real work would happen here
    return [llength $hits]
}
# fault 2: a constraint file never sourced — and nothing checks the return
proc source_sdc {path} {
    if {[fault no_sdc]} { return 0 }
    if {![file isfile $path]} { return 0 }
    set ::sim::constrained 1
    return 1
}
# fault 3: a report written before the data existed
proc report_timing {} {
    if {!$::sim::constrained} { return {} }
    if {!$::sim::updated}     { return {} }
    return {{u_mem/u_and -0.0730} {u_io/u_sync -0.0210}}
}

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

=== the same flow with NO assertions ===
    CASE             WHAT IS WRONG                                RESULT
    ok               nothing wrong                                cells 4  sized 2  viol 2  FAIL
    typo-in-pattern  a query that matched nothing                 cells 4  sized 0  viol 2  FAIL
    no-sdc           a constraint file never sourced              cells 4  sized 2  viol 0  PASS
    no-update        a report written before the data existed     cells 4  sized 2  viol 0  PASS

The design has two real violations. The first row finds them. Two of the three faults turn that into PASS.

no-sdc and no-update are the familiar shape: with no constraints there is nothing to violate, and with no timing graph there is nothing to report, so zero violations is the honest output of a flow that analysed nothing.

4. The one that does not change the verdict

Look at typo-in-pattern again.

sized 0, and the verdict is still FAIL — exactly the same as the correct run.

The pattern was u_corre/* instead of u_core/*. It matched nothing, so the sizing did nothing. And because sizing was not what produced the violations, the verdict is unchanged.

That makes it a different and worse shape than the other two:

no-sdc and no-update produce an implausibly clean result. A design that has always had violations suddenly passing is something an experienced engineer squints at.
typo-in-pattern produces exactly the result you expected. The ECO you just wrote did nothing at all, and the only evidence is a 0 in a column most people do not read.

So the two kinds need different defences. An implausible result can be caught by a reviewer; a result that is correct-but-for-the-wrong-reason cannot be caught by anybody, ever, without an assertion.

And note the asymmetry in cost: no-sdc wastes a run. typo-in-pattern means the fix you believe you applied is not in the database — so the next run shows the same violation, somebody re-applies the same ECO, and the loop continues until someone reads the pattern character by character. Chapter 23.6 §5 demonstrated exactly this on the ECO side.

5. The assertion each one needs

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

proc run_flow_checked {pattern sdc} {
    set n [load_design]
    if {$n == 0} { error "the design has no cells" {} {FLOW NODESIGN} }

    set hits [get_cells $pattern]
    if {[llength $hits] == 0} {
        error "pattern '$pattern' matched no cells" {} [list FLOW NOMATCH $pattern]
    }

    if {![source_sdc $sdc]} {
        error "no constraints were applied from '$sdc'" {} [list FLOW NOSDC $sdc]
    }
    if {!$::sim::constrained} { error "constraints did not take effect" {} {FLOW UNCONSTRAINED} }

    if {![update_timing]} { error "the timing graph was not built" {} {FLOW NOUPDATE} }

    set rows [report_timing]
    if {[llength $rows] == 0} {
        error "the timing report is empty — nothing was analysed" {} {FLOW EMPTYREPORT}
    }
    ...
}

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

=== and with the assertion each fault needs ===
    CASE             RESULT
    ok               cells 4  sized 2  viol 2  FAIL
    typo-in-pattern  REFUSED: pattern 'u_corre/*' matched no cells
    no-sdc           REFUSED: no constraints were applied from 'constraints/c.sdc'
    no-update        REFUSED: the timing graph was not built

The correct run still produces the correct answer, and each fault is refused by name. That first row matters as much as the other three: an assertion set that rejects the good case too is one somebody will switch off (Chapter 23.1 §6).

Four properties make these assertions useful rather than decorative.

Each states what was expected, not what went wrong. "pattern matched no cells" tells you the pattern is the problem. "operation failed" would not.

Each carries the offending value. 'u_corre/*' is in the message, so the typo is visible without opening the script — and in a 40,000-line log that is the difference between a minute and an afternoon.

Each has a machine-readable code. FLOW NOMATCH, FLOW NOSDC, FLOW NOUPDATE — so a driver can distinguish them and Chapter 29.4 can route them, per Chapter 14.2.

They check the thing the step was for, not an incidental side effect. source_sdc returning 1 is checked and $::sim::constrained is checked afterwards — because a return value says the call was made and the state says it took effect, and those are different claims.

6. Where the assertion has to be

And the complementary rule: assert what you can afford to assert. A check per cell in a two-million-instance design is not free, and the useful assertions are almost always about counts and existence — did this match anything, did that take effect, is this report non-empty — which are cheap because the tool already knows them.

7. What assertions cannot catch

Being honest, because a chapter about assertions invites the belief that enough of them is sufficient.

A plausible wrong number. Chapter 24.1 §7's -0.0 passes every existence and count check: a path was found, a number was parsed, the report is non-empty. Catching that needed arithmetic cross-checked against a total the report itself stated (Chapter 24.2 §4) — a different kind of check entirely.

A correct result from the wrong inputs. A perfectly executed analysis of last week's netlist asserts cleanly at every step. That is Chapter 29.2's subject and it needs identity, not assertions.

A scenario nobody ran. Assertions check what happened; they are silent about what was never attempted (Chapter 26.6 §4).

A fault nobody imagined. The three in §3 are in the mock because I put them there. The real list is open-ended, which is why §1's shape matters more than §2's list: when you meet a new one, the question to ask is "what did this step claim to produce, and did it?"

8. Exercises

Exercise 1. What structure do all the failures in this chapter share?

Exercise 2. Why is a better error message never the cure?

Exercise 3. The design has two real violations. Which faults turn that into PASS, and why does each?

Exercise 4. typo-in-pattern keeps the verdict at FAIL. Why is that worse than flipping it?

Exercise 5. Compare the cost of no-sdc with the cost of typo-in-pattern.

Exercise 6. Why does the checked flow's first row matter as much as the other three?

Exercise 7. Why is source_sdc's return value checked and the state checked afterwards?

Exercise 8. Why must an assertion carry the offending value?

Exercise 9. Why place an assertion at the step rather than in a final validation pass?

Exercise 10. Which assertions are cheap enough to be worth having everywhere?

Exercise 11. Name three things assertions cannot catch.

Answers.

1. A correct mechanism applied to state nobody verified, producing a confident result. Nothing raises because nothing illegal happened.

2. Because there is no error. The fix is an assertion — a statement of what the step was supposed to produce, checked against what it did.

3. no-sdc and no-update. With no constraints there is nothing to violate, and with no timing graph there is nothing to report, so zero violations is the honest output of a flow that analysed nothing.

4. Because it produces exactly the result you expected, so there is nothing to notice. An implausibly clean result can be caught by a reviewer; a correct-looking one cannot be caught by anybody without an assertion.

5. no-sdc wastes a run. typo-in-pattern means the fix you believe you applied is not in the database — so the violation recurs, somebody re-applies the same ECO, and the loop continues until the pattern is read character by character.

6. Because an assertion set that rejects the good case is one somebody will switch off.

7. Because they are different claims: the return value says the call was made, and the state says it took effect.

8. Because it makes the fault visible without opening the script — in a 40,000-line log, the difference between a minute and an afternoon.

9. Because a late check reports a symptom three steps from the cause, and the log shows every step succeeding. At the step, "the report is empty" becomes "no constraints were applied from …", which is a fix rather than an investigation.

10. Counts and existence — did this match anything, did that take effect, is the report non-empty. The tool already knows them, so they are nearly free.

11. Any three of: a plausible wrong number (needs an arithmetic cross-check), a correct result from stale inputs (needs identity), a scenario nobody ran, and a fault nobody imagined.

Remember.

Every failure here is a correct mechanism applied to unverified state, producing a confident result with nothing raised.
The cure is an assertion, never a better message — there is no error to improve.
Eleven instances across Modules 18 to 28 are one class. Recognising the shape matters more than memorising the list.
Two of the three faults turn a failing design into PASS.
The third does not change the verdict at all, which makes it the one nobody finds — and the one whose fix silently is not in the database.
An implausibly clean result can be caught by a reviewer; a correct-looking one cannot.
Keep the good case passing, or the assertions get switched off.
Check the return value and the resulting state — "the call was made" and "it took effect" are different claims.
Put the offending value in the message, and give every error a machine-readable code.
Assert at the step, not in a final pass. A late assertion reports a symptom three steps from the cause.
Counts and existence are the cheap assertions and almost always the useful ones.
Assertions cannot catch a plausible wrong number, stale inputs, a scenario never run, or a fault nobody imagined.

Where this fits

Part of the Tcl curriculum.