Perl Part II · Module 18 · Log Parsing and Text Processing

A Small State Machine for a Sectioned Report

A synthesis or timing report is several reports in one file, and they reuse each other's field names. The same label — total cell area, slack, a cell count — appears in the preamble, in the report that owns it, and again in a summary that restates it, with a different value each time. A parser that simply looks for the label finds all of them and has to choose, and this chapter measures what happens when it chooses the obvious way: taking the first match and taking the last match both report a wrong number. The fix is one variable holding which section you are in, which turns the parser into a small state machine worth drawing.

Advanced11 min readPerlPart IILog Parsing

Part II · Module 18 · Chapter 18.7 · A Small State Machine for a Sectioned Report

1. The Same Field, Three Sections

Azvya Education Pvt. Ltd.VLSI Mentor

section-state.pl

use strict;
use warnings;

# a SAMPLE report in the shape a synthesis report has. "Total cell area"
# appears THREE times: once in the preamble, once in the area report where
# it means the design's area, and once in the timing report where it does not.
my @log = split /\n/, <<'LOG';
Information: Building model 'alu' (HDL-193)
  Total cell area:          0.00
Report : area
Design : alu
  Combinational area:     12345.67
  Total cell area:        21246.90
Report : qor
Design : alu
  Critical Path Slack:    -0.034
Report : timing
Design : alu
  slack (VIOLATED)        -0.034
  Total cell area:         9999.99
LOG

my $AREA = qr/^\s+Total cell area:\s+([\d.]+)/;
my $TRUTH = "21246.90";                 # the value in the AREA report

# ---- A: no state. Find every match and pick one.
my @all;
for my $l (@log) { push @all, $1 if $l =~ $AREA }
printf "A. no state tracking: %d match(es) -> %s\n", scalar @all, join(", ", @all);
printf "   first-wins reports %-10s %s\n", $all[0],
    ($all[0] eq $TRUTH ? "right" : "WRONG");
printf "   last-wins  reports %-10s %s\n", $all[-1],
    ($all[-1] eq $TRUTH ? "right" : "WRONG");

# ---- B: one variable holding which report we are inside
my ($state, $area) = ("preamble", undef);
for my $l (@log) {
    if ($l =~ /^Report\s*:\s*(\w+)/) { $state = $1; next }   # a marker moves state
    next unless $state eq "area";                            # the guard
    $area = $1 if $l =~ $AREA;
}
printf "\nB. section-scoped:    reports %-10s %s\n", $area // "(none)",
    (($area // "") eq $TRUTH ? "right" : "WRONG");

print "\nthe design's total cell area is the one in the AREA report.\n";
print "both naive strategies pick a different section's value.\n";

Azvya Education Pvt. Ltd.VLSI Mentor

Captured output

A. no state tracking: 3 match(es) -> 0.00, 21246.90, 9999.99
   first-wins reports 0.00       WRONG
   last-wins  reports 9999.99    WRONG

B. section-scoped:    reports 21246.90   right

the design's total cell area is the one in the AREA report.
both naive strategies pick a different section's value.

Three matches, three different values, and both obvious strategies are wrong.

first-wins reports 0.00 — the preamble's placeholder, printed before anything was synthesised. last-wins reports 9999.99 — a figure from the timing report that is not the design's area at all. The right answer, 21246.90, is the middle one, and nothing about its position identifies it.

The section-scoped version gets it right by construction rather than by luck: the field is only read while the parser is inside the area report.

2. The State Variable

The whole mechanism is two lines inside the loop:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

if ($l =~ /^Report\s*:\s*(\w+)/) { $state = $1; next }   # a marker moves the state
next unless $state eq "area";                            # a guard uses it

A marker line changes the state and is consumed. Every other line is interpreted in the current state. That is a state machine with one variable, and drawing it is worth the space once there are more than two states.

Figure 18.7.1 — The parser's state while reading a sectioned report

fsm

The parser begins in the preamble, where field lines are present and must be ignored because nothing has been synthesised yet. Each Report line moves it into the named section, and only while it is in the area section does the total-cell-area pattern get to run. The self-loops are the ordinary field lines, which do not change the state — the preamble's loop discards them and the area section's loop is where the value is actually captured.

Note that the figure is wrapped in a DiagramContainer. FSMDiagram takes states, transitions, ariaLabel, width, height and stateRadius — and has no title or caption prop of its own, so a figure caption has to come from the container.

3. The Four Rules of a Parsing State Machine

One variable, named for what it holds. $state or $section, not $flag. If you find yourself with $in_area and $in_timing and $in_qor as separate booleans, you have a state machine with an invalid state — all three true at once — and the single variable makes that unrepresentable.

Markers move the state and are consumed. The next after setting $state matters: a marker line is not also a field line, and falling through means a pattern gets a chance to match the marker.

Every field pattern is guarded by the state. This is the point of the whole exercise. A pattern that is valid in one section must be unable to run in another, and next unless $state eq "..." is how.

An unknown marker is a state too. A tool release that adds Report : power must not leave the parser in area:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

if ($l =~ /^Report\s*:\s*(\w+)/) {
    my $s = $1;
    warn "unknown report section: $s\n" unless $KNOWN{$s};
    $state = $s;                      # move anyway -- do NOT stay in the old one
    next;
}

Moving into an unknown state is deliberately safer than staying: in the unknown state every guard fails and nothing is captured, which is the right behaviour. Staying in area would make the new section's fields look like area fields — Chapter 18.8's subject.

Use a table rather than a chain of elsif once there are more than about three sections, because the chain is where the guards drift out of step with the markers.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

my %FIELD = (
    area   => { total => qr/^\s+Total cell area:\s+([\d.]+)/,
                comb  => qr/^\s+Combinational area:\s+([\d.]+)/ },
    timing => { slack => qr/^\s+slack\s+\((?:VIOLATED|MET)\)\s+(-?[\d.]+)/ },
    qor    => { cps   => qr/^\s+Critical Path Slack:\s+(-?[\d.]+)/ },
);
...
for my $field (keys %{ $FIELD{$state} || {} }) {
    $value{$state}{$field} = $1 if $l =~ $FIELD{$state}{$field};
}

Three properties that gives you. A field can only be captured in the section that declares it, structurally. Adding a section is one table entry rather than an elsif plus a guard in two places. And $FIELD{$state} being empty for an unknown section means the unknown-state behaviour from above is automatic.

That is Chapter 7.8's hash of hashes, and the keys %{ ... || {} } guards against an unknown state without a separate test.

4. When One Variable Is Not Enough

One variable handles a report whose sections do not nest. Two situations need more.

A record inside a section. An STA report has sections, and inside the timing section it has path blocks. The section state and the record accumulator from Chapter 18.6 are independent — one says which report you are in, the other holds the record in progress. Keep them as two variables and a record that is open when a section marker arrives gets flushed:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

if ($l =~ /^Report\s*:\s*(\w+)/) {
    push @rec, { %cur } if %cur;      # 18.6's flush, at a section boundary
    %cur  = ();
    $state = $1;
    next;
}

Forgetting that flush is the last-record bug from Chapter 18.6 with a different trigger.

Genuinely nested sections. A hierarchical report with sub-reports needs a stack, not a variable:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

push @section, $1 if $l =~ /^Begin\s+(\w+)/;
pop  @section    if $l =~ /^End\s+(\w+)/;
my $here = join ".", @section;         # "timing.path.element"

And then the guard is on the joined path rather than a single name. This is rare in tool output — most reports are flat sections — and when you meet it, it is worth asking whether the format is really a language rather than a report, which is Chapter 18.11's distinction.

One last practical note: print the state when you report an anomaly. A warning that says "unexpected line" is much less useful than one that says "unexpected line while in the timing section" — and since you have the variable, including it is free.

5. What To Remember

A tool report is several reports in one file, and they reuse each other's field names.
Measured: Total cell area appeared three times with three values — 0.00, 21246.90, 9999.99.
Measured: first-wins reports 0.00 and last-wins reports 9999.99. Both are WRONG.
The right value is the middle one, and nothing about its position identifies it.
A demonstration whose sides agree proves nothing — my first sample had the values coincide and had to be rebuilt.
The mechanism is two lines: a marker sets the state and is consumed; a guard tests it.
FSMDiagram has no title or caption prop — wrap it in DiagramContainer. Its states take explicit cx/cy.
One variable, named for what it holds — separate booleans allow an invalid state.
Markers move the state and next, so a marker is not also read as a field.
Guard every field pattern by the state. That is the entire point.
An unknown marker moves into an unknown state, where every guard fails and nothing is captured — safer than staying in the previous section.
Use a hash-of-hashes field table past three sections, so a field can only be captured in its own section and an unknown section is automatically inert.
A record accumulator is a separate variable from the section state — and a section boundary must flush the open record, or you get 18.6's last-record bug with a new trigger.
Genuinely nested sections need a stack, and should prompt the question in Chapter 18.11.
Include the state in every anomaly message. It is free and it is the difference between a useful warning and a useless one.

Chapter 18.8 takes the problem this chapter's unknown-state rule was anticipating. A parser outlives the tool version it was written against, and the field it is looking for gets renamed — so the question is how a parser notices that instead of returning zero.

Where this fits

Part of the Perl curriculum.