Why Legacy Flows Still Use Perl

Part II · Module 16 · Chapter 16.2 · Why Legacy Flows Still Use Perl

1. The Trade, Stated Plainly

A rewrite has a cost and a benefit, and for flow infrastructure the numbers are unusually lopsided.

The benefit is real but modest: code that is easier for the current team to read, in a language they use elsewhere, with tests.

The cost is not the rewriting. It is the risk that the new version differs from the old one in a way nobody notices until a regression passes that should have failed. On a flow feeding a tapeout, that is the only cost anyone is weighing.

So the decision is not "Perl or Python". It is "known-correct or probably-correct", and in the weeks before a deadline there is no version of that argument where "probably" wins.

Which means the useful question is not should we rewrite this. It is how much do we not know about it — and that can be measured.

2. Measure the Interface, Not the Size

Here is a sample script of the kind every established flow has. It is written for this chapter — it is not taken from anyone's repository — but it is built from the patterns you will actually meet.

#!/usr/bin/perl
# run_regress.pl -- nightly regression driver.
# original author unknown; last substantive change undated.
use strict;
my $ROOT = $ENV{PROJ_ROOT} || "/proj/chipA";
my $SCRATCH = $ENV{SCRATCH} || "/tmp";
my $LIST = shift || "$ROOT/lists/nightly.list";
my $SEED = $ENV{SEED} || 1;
my $TOOL = $ENV{SIM} || "vcs";
open(L, "<$LIST") || die "no list";
my @t;
while (<L>) { chomp; next if /^#/; next unless /\S/; push @t, $_ }
close L;
my $fail = 0;
foreach my $t (@t) {
    my $d = "$SCRATCH/$t.$$";
    mkdir $d;
    my $cmd = "$TOOL -f $ROOT/rtl/files.f +seed=$SEED -l $d/sim.log $t";
    system($cmd);
    if ($? >> 8) { $fail++; warn "FAIL $t\n"; next }
    open(G, "grep -c UVM_ERROR $d/sim.log |");
    my $e = <G>; close G; chomp $e;
    $fail++ if $e > 0;
    system("rm -rf $d") if $e == 0 && !$ENV{KEEP};
}
print "done: $fail failures of ", scalar(@t), "\n";
exit($fail ? 1 : 0);

Now the tool. It does not judge the script; it counts the things you would have to get right.

use strict;
use warnings;
use File::Temp qw(tempdir);
use File::Spec;

# write the sample script shown above into a temp tree, then analyse it
my $dir = tempdir(CLEANUP => 1);
my $script = File::Spec->catfile($dir, "run_regress.pl");
open my $w, ">", $script or die "cannot write: $!\n";
print $w <<'LEGACY';
#!/usr/bin/perl
use strict;
my $ROOT = $ENV{PROJ_ROOT} || "/proj/chipA";
my $SCRATCH = $ENV{SCRATCH} || "/tmp";
my $LIST = shift || "$ROOT/lists/nightly.list";
my $SEED = $ENV{SEED} || 1;
my $TOOL = $ENV{SIM} || "vcs";
open(L, "<$LIST") || die "no list";
my @t;
while (<L>) { chomp; next if /^#/; next unless /\S/; push @t, $_ }
close L;
my $fail = 0;
foreach my $t (@t) {
    my $d = "$SCRATCH/$t.$$";
    mkdir $d;
    my $cmd = "$TOOL -f $ROOT/rtl/files.f +seed=$SEED -l $d/sim.log $t";
    system($cmd);
    if ($? >> 8) { $fail++; warn "FAIL $t\n"; next }
    open(G, "grep -c UVM_ERROR $d/sim.log |");
    my $e = <G>; close G; chomp $e;
    $fail++ if $e > 0;
    system("rm -rf $d") if $e == 0 && !$ENV{KEEP};
}
print "done: $fail failures of ", scalar(@t), "\n";
exit($fail ? 1 : 0);
LEGACY
close $w;

open my $r, "<", $script or die "cannot read: $!\n";
my @line = <$r>;
close $r;
my $src = join "", @line;

sub uniq_list {
    my ($re) = @_;
    my %seen;
    $seen{$1} = 1 while $src =~ /$re/g;
    return join(", ", sort keys %seen);
}

my @row = (
  [ "lines of code",        scalar grep { /\S/ && !/^\s*#/ } @line ],
  [ "branches",             scalar(() = $src =~ /\b(?:if|unless|elsif)\b/g)   ],
  [ "environment reads",    uniq_list(qr/\$ENV\{(\w+)\}/)                     ],
  # a command run through a VARIABLE is marked, because the real name is
  # whatever that variable held -- which is the point, not a tool defect
  [ "external commands",    do {
        my %seen;
        while ($src =~ /(?:system|exec)\s*\(?\s*["']?(\$?)(\w+)/g) {
            $seen{ $1 ? "\$$2 (a variable!)" : $2 } = 1;
        }
        $seen{$1} = 1 while $src =~ /open\s*\([A-Z]+\s*,\s*["'](\w+)[^"']*\|/g;
        join(", ", sort keys %seen);
    } ],
  [ "paths hard-coded",     uniq_list(qr{["'](/[\w\/]{4,})})                   ],
  [ "bareword filehandles", uniq_list(qr/open\s*\(\s*([A-Z]\w*)\s*,/)          ],
  [ "exit codes produced",  uniq_list(qr/exit\s*\(?\s*([^);\n]+)/)             ],
  [ "use warnings",         ($src =~ /use warnings/ ? "present" : "MISSING")   ],
  [ "two-argument open",    ($src =~ /open\s*\([A-Z]+\s*,\s*["'][<>]/
                              ? "YES - mode glued to the name" : "no")         ],
);

for my $r (@row) {
    my ($k, $v) = @$r;
    $v = "(none)" if !defined $v || $v eq "" || $v eq "0";
    $v =~ s/^,\s*|,\s*$//g;
    printf "  %-22s %s\n", $k, $v;
}

my $envs = scalar(() = $row[2][1] =~ /\w+/g);
printf "\n  %d lines of code, but %d environment variables decide what it does\n",
    $row[0][1], $envs;

  lines of code          24
  branches               5
  environment reads      KEEP, PROJ_ROOT, SCRATCH, SEED, SIM
  external commands      $cmd (a variable!), grep, rm
  paths hard-coded       /proj/chipA
  bareword filehandles   G, L
  exit codes produced    $fail ? 1 : 0
  use warnings           MISSING
  two-argument open      YES - mode glued to the name

  24 lines of code, but 5 environment variables decide what it does

Twenty-four lines of code. And to reproduce its behaviour you would have to know, and get right:

• five environment variables — PROJ_ROOT, SCRATCH, SEED, SIM, KEEP — each with a default that is itself a decision
• three external commands — grep, rm, and whatever $SIM happens to hold. The analyser marks that last one $cmd (a variable!) rather than guessing, because the real command name is not in the source at all
• one hard-coded path, /proj/chipA, which is the default when PROJ_ROOT is unset
• two bareword filehandles and a two-argument open, whose behaviour on a filename with a leading space or a > differs from the three-argument form (Chapter 13.1)
• one exit convention, $fail ? 1 : 0

That list is the script's real interface, and it is nowhere written down. Every item is something a caller might depend on, and some caller probably does.

3. The Four Reasons, Each With Its Row

Now the four reasons from the chapter's purpose, each tied to something the measurement shows.

It is on the critical path to a tapeout. This script decides whether the nightly regression passed. Its exit status gates everything downstream. A rewrite that is subtly wrong does not produce an error; it produces a regression that reports success, which is Chapter 14.1's third kind of failure at the scale of a project.

Its behaviour was never written down. The five environment variables are the proof. Nothing in the file explains what KEEP is for, why SEED defaults to 1, or whether SIM=xcelium was ever tested. The defaults are the specification, and they are the specification by accident.

There is no owner. The header says the author is unknown. In practice this means no one can answer "is this behaviour deliberate?" — and the measurement is full of behaviours you would have to ask about:

• open(L, "<$LIST") || die "no list" discards $!, so a permissions problem and a missing file report identically (Chapter 13.5 measured that exact loss).
• system($cmd) passes one string, so the simulator name and the test name go through a shell. A test name with a space in it becomes two arguments (Chapter 15.10 measured this).
• grep -c UVM_ERROR is read through a pipe whose status is never checked, and grep -c exits non-zero when it finds nothing — so "no errors" and "grep failed" are the same observation here.
• rm -rf $d runs only when $e == 0, so a failed test leaves its directory behind. Deliberate, probably. Documented, no.

There is no test suite. This is the one that settles it. To rewrite safely you need to know the current answer for every combination that matters, and there is nothing that will tell you. With five environment variables and five branches, the space of behaviours is large enough that "we tested it" means "we tried the one case we had".

4. What To Do Instead

Nothing above argues for leaving it alone. It argues against a rewrite as the first move.

Write down what it does, from the measurement. The analyser's output is most of a specification. Put it in a comment block at the top of the existing script, as a first commit that changes no behaviour at all. This is the highest-value thing you can do and it takes an afternoon.

Add the checks Part I taught, one at a time. Each is small, each is independently reviewable, and none changes the happy path:

use warnings;                                   # was MISSING
open(my $fh, "<", $LIST) or die "cannot read $LIST: $!\n";   # 13.1, 13.5
system($TOOL, "-f", "$ROOT/rtl/files.f", ...) == 0 or ...;   # 15.10 list form
my $root = $ENV{PROJ_ROOT} // "/proj/chipA";    # 15.5: // not ||

Note the last one. || replaces an empty PROJ_ROOT with the default; // keeps it. Chapter 15.5 measured that difference, and here it means a deliberately-blanked variable silently gets the hard-coded path.

Build the test harness before the rewrite. Capture the current script's exit status and output for a set of real inputs. That recording is the oracle. Without it a rewrite cannot be verified; with it the rewrite becomes ordinary work.

Then replace it in pieces, not at once. Pull one function out, call it from the old script, prove the output is identical, move on. The old script shrinks until there is nothing left — and at every single commit, the flow still works. Every tool in the rest of Part II is built to be a piece you can adopt this way, which is what Chapter 16.10's contract is for.

5. What To Remember

• The decision is "known-correct or probably-correct", not Perl or Python.
• The cost of a rewrite is not the rewriting — it is a difference nobody notices until a regression wrongly passes.
• Ask how much you do not know, not whether to rewrite. That is measurable.
• Measured: 24 lines of code, 5 environment variables, 3 external commands, 1 hard-coded path, 2 bareword handles, 1 exit convention.
• Size is not the risk; interface size is. A 600-line formatter reading one file is a safer rewrite than this 24-line driver.
• Compare a script's line count against its interface count — that ratio is the useful signal.
• The analyser is crude on purpose — a checklist to verify by reading, not an answer.
• Four reasons, each visible in the measurement: it gates a tapeout, its behaviour is undocumented, nobody can confirm intent, and nothing can verify a rewrite.
• The defaults are the specification, by accident.
• Treat the old bugs as the spec at first. Change the language or the behaviour, never both in one commit.
• Document from the measurement first — a comment block that changes no behaviour.
• Add Part I's checks one at a time: use warnings, three-argument open with $!, the list form of system, // instead of ||.
• Record the current exit status and output as an oracle before changing anything.
• Replace in pieces, so the flow works at every commit.

Chapter 16.3 narrows this to the practical question for someone joining a team: given that the Perl is there, where exactly will you meet it, and which of those places will you be expected to work in during your first month?