A Physical Design Report Collector

Part II · Module 30 · Chapter 30.5 · A Physical Design Report Collector

An implementation run leaves a directory of reports behind, and somebody has to turn it into one line that says whether the block is good. Parsing the metrics is Module 24, already done. The hard part is the registry's last clause: dealing with the step that did not run.

That is not an edge case. It is the normal failure of a long flow, and a collector built on pass-or-fail cannot express it.

1. Three outcomes, not two

The top row is the collector everybody writes. The bottom row is the chapter.

2. The plan is the driver

# The PLAN — what the run was supposed to contain. Driving the collection
# from this rather than from the filesystem is what makes a missing step
# visible (26.6 section 4).
set ::PLAN {
    {floorplan  utilisation  {utilisation %}        85.0  below}
    {place      congestion   {worst overflow %}      2.0  below}
    {cts        skew         {clock skew ns}        0.120 below}
    {route      drc          {drc violations}       0     equal}
    {route      wirelength   {total wirelength um}  0     report}
}

Five planned metrics, each with the step that produces it, the key to look for, a label for printing, a limit, and a comparison rule. Three rules, because not every metric is a pass-or-fail: below for a budget, equal for a count that must be zero, and report for a number you want recorded and will not judge.

The critical property is the direction of the loop. The collector iterates the plan, not the filesystem. A collector that walks the run directory can only report on what it finds, which means a step that produced nothing produces no row — it is not reported as missing, it is simply absent from a table nobody counts. Driving from the plan is what makes absence visible, and it is Chapter 26.6 §4's rule and Chapter 23.5's: detect absence by looping over what you expected.

proc parse_metric {path key} {
    if {![file isfile $path]} { return "" }
    set fh [open $path r] ; set val ""
    while {[gets $fh line] >= 0} {
        if {[regexp "^$key\\s*:\\s*(-?\\d+\\.?\\d*)" $line -> v]} { set val $v }
    }
    close $fh
    return $val
}

proc verdict {value limit rule} {
    if {$value eq ""} { return NOT-RUN }
    switch -- $rule {
        below  { return [expr {$value <  $limit ? "pass" : "FAIL"}] }
        equal  { return [expr {$value == $limit ? "pass" : "FAIL"}] }
        report { return "info" }
    }
    return "?"
}

proc collect {rundir} {
    set rows {}
    foreach item $::PLAN {
        lassign $item step key label limit rule
        set path [file join $rundir $step report.txt]
        set v [parse_metric $path $key]
        lappend rows [dict create step $step key $key label $label \
                          limit $limit rule $rule value $v \
                          verdict [verdict $v $limit $rule]]
    }
    return $rows
}

parse_metric returns the empty string for a file that does not exist — not an error, because a missing report is a legitimate state of a run in progress, and not a zero, because zero is a real and often good value for skew, congestion and DRC count. Conflating "no measurement" with "measured zero" is how a collector reports a perfect clock tree for a clock tree that was never built.

verdict therefore tests for the empty string first, before any comparison. That ordering is the entire three-outcome design in two lines.

3. A complete run

proc summarise {rows} {
    puts [format "  %-10s %-22s %10s %8s  %s" STEP METRIC VALUE LIMIT VERDICT]
    foreach r $rows {
        set v [dict get $r value]
        puts [format "  %-10s %-22s %10s %8s  %s" \
                  [dict get $r step] [dict get $r label] \
                  [expr {$v eq "" ? "-" : $v}] \
                  [expr {[dict get $r rule] eq "report" ? "-" : [dict get $r limit]}] \
                  [dict get $r verdict]]
    }
    # THREE counts, not two. A step that did not run is its own outcome.
    set pass 0 ; set fail 0 ; set notrun 0 ; set info 0
    foreach r $rows {
        switch -- [dict get $r verdict] {
            pass    { incr pass }
            FAIL    { incr fail }
            NOT-RUN { incr notrun }
            info    { incr info }
        }
    }
    puts ""
    puts [format "  pass %d   FAIL %d   NOT-RUN %d   info %d   of %d planned" \
              $pass $fail $notrun $info [llength $rows]]
    if {$notrun > 0} {
        puts "  REFUSING a verdict: $notrun planned metric(s) were never produced"
        foreach r $rows {
            if {[dict get $r verdict] eq "NOT-RUN"} {
                puts "      [dict get $r step]/[dict get $r key]"
            }
        }
        return 3
    }
    return [expr {$fail > 0 ? 1 : 0}]
}

=== a complete run ===
  STEP       METRIC                      VALUE    LIMIT  VERDICT
  floorplan  utilisation %                72.4     85.0  pass
  place      worst overflow %              1.4      2.0  pass
  cts        clock skew ns               0.072     0.12  pass
  route      drc violations                  0        0  pass
  route      total wirelength um       1844219        -  info

  pass 4   FAIL 0   NOT-RUN 0   info 1   of 5 planned
  exit would be 0

of 5 planned is the denominator, and it comes from the plan rather than from the rows — so the line cannot say 4 of 4 on a run that produced four reports instead of five. Wirelength is info: recorded, not judged, with a - where a limit would be. A number with a fabricated limit is worse than a number with none, because somebody will act on the limit.

4. The step that did not run

Delete the CTS directory and run the identical collector:

=== the same collector, with CTS never run ===
  STEP       METRIC                      VALUE    LIMIT  VERDICT
  floorplan  utilisation %                72.4     85.0  pass
  place      worst overflow %              1.4      2.0  pass
  cts        clock skew ns                   -     0.12  NOT-RUN
  route      drc violations                  0        0  pass
  route      total wirelength um       1844219        -  info

  pass 3   FAIL 0   NOT-RUN 1   info 1   of 5 planned
  REFUSING a verdict: 1 planned metric(s) were never produced
      cts/skew
  exit would be 3

Four things happen that a two-outcome collector cannot do. The row still appears, with a - for the value and its limit intact. The count line reports NOT-RUN 1 as its own category. The collector refuses a verdict and says so in words. And it names what is missing — cts/skew — so the next action is obvious rather than requiring a hunt.

Look at FAIL 0. Nothing failed. Every metric that was measured was within its limit. That is exactly why this is hard to catch in review: the collector that gets this wrong is not mis-counting anything.

5. The same data, two collectors

=== the SAME run directory, with CTS never run, through two collectors ===
  two outcomes : 0 failure(s) -> PASS
  three        : 0 failure(s), 1 never produced -> NO VERDICT

  both are looking at the same 5 metrics and the same 4 files.
  the clock tree was never built, and the first one says PASS.

PASS. Same directory, same plan, same parser, same limits. The clock tree does not exist and the block is signed off.

And the bug is one word:

=== and where the difference comes from ===
  the two-outcome collector's 'continue' is the bug: it treats a metric
  it could not read as a metric it does not need to judge

foreach r $rows {
    set v [dict get $r value]
    if {$v eq ""} { continue }          ;# <- here
    if {[dict get $r verdict] eq "FAIL"} { incr fail }
}
return [list $fail [expr {$fail == 0 ? "PASS" : "FAIL"}]]

That continue is not careless. It is the considerate thing to write — you cannot compare an empty string to a number, so skipping it avoids an error. What it actually encodes is "a metric I could not read is a metric I do not need to judge", and the final line then converts an absence of failures into a positive claim.

6. Three statuses, three exit codes, all three seen to fire

A third status is only worth having if the caller can act on it, which means a third exit code — and all three paths have to be exercised or the third one is decoration.

=== and a run that genuinely failed its limit ===
  STEP       METRIC                      VALUE    LIMIT  VERDICT
  floorplan  utilisation %                91.8     85.0  FAIL
  place      worst overflow %              1.4      2.0  pass
  cts        clock skew ns               0.072     0.12  pass
  route      drc violations                  0        0  pass
  route      total wirelength um       1844219        -  info

  pass 3   FAIL 1   NOT-RUN 0   info 1   of 5 planned
  exit would be 1

3 and 1 are genuinely different instructions. A 1 says the block is not good enough; a 3 says you do not yet know, and treating the second as the first is how a missing step becomes a waiver.

The FAIL block above was added because the FAIL branch had never once fired in any demonstration — the same unproven-branch problem as 30.1 §8's archive gate, 30.2 §6's histogram marker, 30.3 §4's uncertainty floor and 30.4 §5's row-count check. Four chapters, five dead branches, one habit.

7. A demonstration that was correct exactly once

Worth publishing because it is a trap in any script that demonstrates a failure by causing it.

The second block above deletes the CTS directory. The first version of this script did not build the run directory first — it used whatever was on disk. So the first run was correct, the delete happened, and every subsequent run showed NOT-RUN in both blocks, including the one captured for this chapter. The output looked plausible: two blocks, both refusing, both naming cts/skew. Nothing in it is impossible, which is why it survived a read-through.

# A demo that MUTATES the tree must build the tree first, or it is correct
# exactly once. The second run of the version without this showed NOT-RUN in
# both blocks, because the first run had already deleted the CTS directory.
# See Chapter 30.5 section 7.
proc build_fixture {run} {
    file delete -force $run
    foreach {step body} {
        floorplan {utilisation : 72.4
die area um2 : 42100.0}
        place {congestion : 1.4
placed instances : 16233}
        cts {skew : 0.072
insertion delay : 0.412}
        route {drc : 0
wirelength : 1844219}
    } {
        file mkdir [file join $run $step]
        set fh [open [file join $run $step report.txt] w]
        puts $fh $body
        close $fh
    }
    return $run
}

Building the fixture is four lines and makes the run idempotent — verified by running it twice and diffing. Two consequences worth keeping:

• A demonstration that mutates state must create that state. Otherwise it is a function of how many times it has been run, which is not a property anyone tracks.
• Fixing it exposed a second bug. The first fixture wrote worst overflow : 1.4 while the plan greps for congestion, so three metrics came back NOT-RUN on a tree that was complete. Two mistakes in the same four lines, both found by running it twice rather than once.

8. What this collector still cannot say

• It cannot tell you the plan is right. Five metrics is a judgement about what matters. A metric nobody planned cannot be missing.
• It cannot tell you a report is from this run. A stale cts/report.txt from last week parses perfectly and reports pass. That is Chapter 29.2's problem — and 30.1 §8 measured the same thing wearing an archive's clothes. Identity, not counting.
• It cannot distinguish "never ran" from "ran and crashed before writing". Both leave no file. They need different actions, and separating them needs the step's own exit status recorded, which is Chapter 29.5's record.
• A report metric is never wrong. info rows cannot fail by construction, so a wirelength that doubled passes silently. That is the correct trade for a metric with no agreed limit, and it means info rows need a human looking at trends.

Exercises

1. Make parse_metric return 0 instead of "" for a missing file. Which row changes, and which verdict does it become?
2. Add a sixth planned metric that no step produces. What is the exit status, and is that right?
3. The two-outcome collector's continue is removed. What happens, and is the resulting error better or worse than PASS?
4. A step runs, writes its report, but writes skew : N/A. What does the collector report, and is it correct?
5. Change of 5 planned to count rows instead of the plan. Construct a run where the line becomes misleading.
6. Two steps never ran. Does exit 3 still convey enough?

Answers

1. cts becomes pass — because 0 is below the 0.120 limit, so a clock tree that was never built reports the best possible skew. That is the single most dangerous one-character change in the script, and it is why parse_metric returns the empty string: zero is a real measurement for every metric in this plan.
2. 3, and yes. The metric is planned and never produced, which is precisely the condition. It is also the cheap way to test the mechanism: a plan entry for a step that does not exist should refuse a verdict, and if it does not, the refusal is not driven by the plan.
3. It raises can't use empty string as operand of "<" and the script dies with no summary. Better than PASS — a crash is a signal and a wrong verdict is not — and still much worse than the third status, because it tells you a comparison broke rather than that a step did not run. The hierarchy is: wrong answer, then crash, then an honest refusal.
4. It reports NOT-RUN, because the regex wants a number and finds none, so parse_metric returns the empty string. Correct by luck rather than by design — the mechanism that handles a missing file happens to handle an unparseable value, and the two cases deserve different messages. The same shape as 30.3 §5's missing create_clock reported as a name mismatch: the right outcome with the wrong explanation.
5. Delete the CTS and place directories. Row-counting would print of 3 planned, the two surviving passes would be 2 of 3, and the summary would look like a mostly-successful small run instead of a run that lost two steps. The denominator has to come from the plan, or it shrinks to fit whatever happened.
6. Yes for the exit status — the caller's action is the same, find out why — but only because the collector names them. 3 is "you do not know"; the cts/skew list is what makes it actionable. An exit code without the names means the caller has to re-derive what was missing, and will guess.