Reviewing a Synthesis Script

Part II · Module 21 · Chapter 21.10 · Reviewing a Synthesis Script

Every phase in this module can fail quietly. A missing source file becomes a black box. A mistyped port leaves the design unconstrained. A report command that produced nothing leaves a zero-byte file.

Any of those can happen in a run that completes all six phases and exits zero. This chapter is the check that notices — and it runs over the run directory, not the script, which means anybody can run it afterwards without a tool.

1. What a signable run leaves behind

From Chapter 21.1 §7, made checkable:

• the run directory exists,
• every expected report is present and non-empty,
• a netlist was written,
• a provenance record sits beside it,
• and the quality numbers are actually in the quality report.

Each is a file-system fact, which is why this is Part I work.

2. The run that finished and produced nothing

Here is the target. A run directory containing a reports/ and netlist/ directory, and a timing.rpt that was created and never filled:

=== a run that finished and produced nothing you can sign off ===
  ok      run directory present
  ERROR   3 report(s) missing: area.rpt, power.rpt, qor.rpt
  ERROR   1 report(s) empty: timing.rpt
  ERROR   no netlist written
  ERROR   no provenance record beside the netlist
  ---
  4 problem(s)

3. The review

proc review_run {rundir expected_reports} {
    set problems 0
    proc _bad {msg} { upvar 1 problems p ; puts stderr "  ERROR   $msg" ; incr p }
    proc _ok  {msg} { puts "  ok      $msg" }

    if {![file isdirectory $rundir]} { _bad "run directory '$rundir' does not exist" ; return $problems }
    _ok "run directory present"

    set miss {} ; set empty {}
    foreach r $expected_reports {
        set p [file join $rundir reports $r]
        if {![file isfile $p]}   { lappend miss $r ; continue }
        if {[file size $p] == 0} { lappend empty $r }
    }
    if {[llength $miss]}  { _bad "[llength $miss] report(s) missing: [join $miss {, }]" } \
    else                  { _ok "all [llength $expected_reports] report(s) present" }
    if {[llength $empty]} { _bad "[llength $empty] report(s) empty: [join $empty {, }]" }

    set nl [glob -nocomplain [file join $rundir netlist *.v]]
    if {[llength $nl] == 0} { _bad "no netlist written" } \
    else { _ok "netlist written: [file tail [lindex $nl 0]]" }

    if {![file isfile [file join $rundir netlist PROVENANCE.txt]]} {
        _bad "no provenance record beside the netlist"
    } else { _ok "provenance recorded" }

    set qor [file join $rundir reports qor.rpt]
    if {[file isfile $qor]} {
        set fp [open $qor r] ; set txt [read $fp] ; close $fp
        if {![regexp {wns\s+(\S+)} $txt -> wns]} { _bad "qor.rpt has no wns line" } \
        else { _ok "worst negative slack recorded: $wns" }
    }
    puts "  ---"
    puts "  $problems problem(s)"
    return $problems
}

Against a complete run:

=== a complete run ===
  ok      run directory present
  ok      all 4 report(s) present
  ok      netlist written: cpu_top_mapped.v
  ok      provenance recorded
  ok      worst negative slack recorded: 0.142
  ---
  0 problem(s)

Five ok lines and zero problems — and notice the last one reports the actual number, so the review also tells you what the run achieved, not just that it finished.

4. The Part I decisions in it

Every one of these has appeared before in the track, which is the point:

• Problems to stderr, progress to stdout (Chapter 14.9 §2).
• The ok cases reported too, so you can tell the check ran (Chapter 20.8 §3).
• Offenders named, not just counted.
• A count returned, never exit called (Chapter 15.4 §3).
• Checked against a declared expected list (Chapter 21.8 §4).
• Size checked, not just presence — the zero-byte case.
• glob -nocomplain so a missing netlist directory is an empty list rather than a raise (Chapter 21.2 §2).

5. It checks the artefacts, not the script

Worth being explicit, because it decides where this check is useful.

A review of the script tells you whether the right commands are present. A review of the run directory tells you whether they worked. This is the second.

That has a practical consequence: it needs no tool and no licence. It is a Tcl script over a directory, so it can run

• at the end of the flow, deciding the exit status;
• as a gate when a block is handed over, run by the receiver;
• across an archive of old runs, to find which are still usable.

A source review cannot do any of those.

6. Wiring it into the exit status

The registry purpose's last clause:

set problems [review_run $RUN {timing.rpt area.rpt power.rpt qor.rpt}]
if {$problems > 0} {
    puts stderr "synthesis run incomplete: $problems problem(s)"
    exit 1
}
puts "synthesis run complete"
exit 0

That is the line that makes the whole module's checking matter. Without it, every check in Chapters 21.2 through 21.9 produces a message in a log, and the surrounding automation proceeds regardless.

With it, a run that produced nothing signable stops the flow — which is Chapter 15.4 §6's argument in the place it is worth the most.

7. What the review cannot tell you

The honest half, and the same shape as Chapter 20.8 §6.

It can check: the artefacts exist, are non-empty, and contain the lines they should.

It cannot check:

• Whether the numbers are good. It reports worst slack; whether 0.142 is acceptable is a project decision.
• Whether the design is complete. A black box (Chapter 21.5 §2) produces a perfectly well-formed netlist and a clean report set.
• Whether the constraints were right. Chapter 20.8 reviews that, and even it cannot tell whether a period matches the specification.
• Whether the right libraries were used. The provenance records them; a human decides whether they were the intended ones.

So add two checks to the script when you can state them: a threshold on the quality number, and a sanity bound on the instance count from Chapter 21.5 §5. A run whose instance count dropped by an order of magnitude is a black box, and that is mechanically detectable.

8. Exercises

Exercise 1. What does a check of "did the tool exit zero" miss?

Exercise 2. Name the five things a signable run leaves behind.

Exercise 3. Why check report size rather than presence?

Exercise 4. Why does the review return a count instead of exiting?

Exercise 5. Why does checking the run directory rather than the script make the review more useful?

Exercise 6. Which single line makes all of this module's checking matter?

Exercise 7. Name two things the review cannot tell you, and one mechanical check that would catch a black box.