Perl Part II · Module 28 · Legacy Perl and Python Migration
Why Immediate Rewrites Usually Cost More
The script is ugly, it has no tests, nobody likes it, and it could be rewritten in Python in an afternoon. All four statements are usually true, and they still do not add up to a reason to rewrite it this week. This chapter does the experiment instead of the argument. It writes a genuinely good Python replacement from the script's own usage comment and runs it through the eleven probes of Chapter 28.2. Then it runs the flow's real consumers against both versions. The rewrite fixes four real bugs. It also behaves differently in every one of the eleven scenarios, crashes on two inputs the original survived, and leaves the dashboard showing yesterday's numbers without a single error.
Intermediate26 min readPerlPart IIEDA Flows
Part II · Module 28 · Chapter 28.3 · Why Immediate Rewrites Usually Cost More
1. The Temptation, Stated Fairly
Look at sta_summary.pl from Chapter 28.1 through the eyes of an engineer who has just been asked to own it. No use strict. Globals everywhere. A bareword filehandle. An open ... || next that hides failures. A worst-slack calculation that is simply wrong for a corner that meets timing. The whole thing is 59 lines, and the replacement in Python would be cleaner, tested and shorter to explain.
None of that is false. The case for rewriting is not made by foolish people:
- the script does have bugs, and Chapter 28.2 found several;
- a modern rewrite would be easier to change next year;
- the team does know Python better than Perl;
- and 59 lines is small.
So the question this chapter asks is not "is the old code bad?". It is: what does replacing it actually change, for the people and programs that use it, and what does finding that out cost? That is a question with a measurable answer, because Chapter 28.2 left us an inventory and a probe harness.
2. The Experiment: A Good Rewrite
Here is the rewrite. It is written the way rewrites are usually written: from the original's usage comment ("prints one line per corner: paths, violations, WNS and TNS") and a fair understanding of what the script is for. Its author has read the old code but has not taken an inventory. It is deliberately a good piece of work, not a straw man.
#!/usr/bin/env python3
"""sta_summary_v2.py - summarise per-corner timing reports.
A clean REWRITE of sta_summary.pl, written the usual way: from its usage
comment and a fair idea of what the script is for, WITHOUT the inventory.
usage: sta_summary_v2.py [-v] [report_dir]
"""
import argparse
import pathlib
import re
import sys
ENDPOINT = re.compile(r"^Endpoint\s*:\s*(\S+)")
SLACK = re.compile(r"slack \((MET|VIOLATED)\)\s+(-?\d+(?:\.\d+)?)")
def parse(path):
"""Return paths, violations, TNS, WNS and violating endpoints."""
paths, viol, tns, wns, eps = 0, 0, 0.0, None, []
endpoint = None
for line in path.read_text().splitlines():
m = ENDPOINT.match(line)
if m:
endpoint = m.group(1)
m = SLACK.search(line)
if m:
status, slack = m.group(1), float(m.group(2))
paths += 1
if status == "VIOLATED":
viol += 1
tns += slack
eps.append(endpoint)
wns = slack if wns is None else min(wns, slack) # the TRUE worst slack
return {"paths": paths, "viol": viol, "tns": tns, "wns": wns, "eps": eps}
def main():
ap = argparse.ArgumentParser(description="Summarise per-corner timing reports.")
ap.add_argument("-v", "--verbose", action="store_true", help="list violating endpoints")
ap.add_argument("report_dir", nargs="?", default=".")
args = ap.parse_args()
d = pathlib.Path(args.report_dir)
if not d.is_dir():
print(f"error: {d} is not a directory", file=sys.stderr)
return 2
reports = sorted(d.glob("*.rpt"))
if not reports:
print(f"error: no reports in {d}", file=sys.stderr)
return 2
failing = 0
for rpt in reports:
r = parse(rpt)
print(f"{rpt.stem:<16} paths={r['paths']:<3} viol={r['viol']:<3} "
f"wns={r['wns']:.3f} tns={r['tns']:.3f}")
if args.verbose:
for ep in r["eps"]:
print(f" - {ep}")
if r["viol"]:
failing += 1
return 1 if failing else 0 # a flag, not a count
if __name__ == "__main__":
sys.exit(main())It makes sensible choices everywhere: argparse with a help message, a check that the directory exists, errors on STDERR, the true worst slack (min over every path), -v output as a readable bullet list, and an exit status that is a flag (1 for "something failed") instead of a count. It does not write tmp.csv, because nothing in the usage comment mentions it, and a summary tool that drops a stray file in the caller's directory is a bug in anyone's book.
The original, unchanged, with its two sample reports:
#!/usr/bin/perl
# sta_summary.pl - summarise per-corner timing reports
# usage: sta_summary.pl [-v] [report_dir]
# prints one line per corner: paths, violations, WNS and TNS
#
# (A SAMPLE legacy script written for this module, in the style of
# scripts that have run unchanged in a flow for a decade.)
use Getopt::Long;
$verbose = 0;
GetOptions("v" => \$verbose);
$dir = shift @ARGV;
$dir = "." unless $dir;
opendir(D, $dir) || die "cannot open $dir: $!\n";
@files = grep(/\.rpt$/, readdir(D));
closedir(D);
if (!@files) {
print "No reports found in $dir\n";
exit 0;
}
foreach $f (@files) {
($corner = $f) =~ s/\.rpt$//;
open(F, "$dir/$f") || next;
while (<F>) {
if (/^Endpoint\s*:\s*(\S+)/) {
$ep = $1;
}
if (/slack \((\w+)\)\s+(\S+)/) {
$paths{$corner}++;
if ($1 eq "VIOLATED") {
$viol{$corner}++;
$tns{$corner} += $2;
push(@{$eps{$corner}}, $ep);
}
$wns{$corner} = $2 if $2 < $wns{$corner};
}
}
close(F);
}
open(CSV, ">tmp.csv");
$bad = 0;
foreach $c (sort keys %paths) {
printf("%-16s paths=%-3d viol=%-3d wns=%.3f tns=%.3f\n",
$c, $paths{$c}, $viol{$c}, $wns{$c}, $tns{$c});
print CSV "$c,$viol{$c},$wns{$c}\n";
if ($verbose) {
foreach $e (@{$eps{$c}}) { print " $e\n"; }
}
$bad++ if $viol{$c};
}
close(CSV);
exit $bad;Report : timing (synthetic teaching format)
Design : chip_top
Corner : ss_0p72v_125c
Startpoint: u_core/r_reg[3]
Endpoint : u_core/acc_reg[7]
slack (VIOLATED) -0.043
Startpoint: u_core/r_reg[1]
Endpoint : u_core/acc_reg[2]
slack (VIOLATED) -0.011
Startpoint: u_core/cnt_reg[0]
Endpoint : u_core/cnt_reg[5]
slack (MET) 0.120
End of reportReport : timing (synthetic teaching format)
Design : chip_top
Corner : tt_0p80v_25c
Startpoint: u_core/r_reg[3]
Endpoint : u_core/acc_reg[7]
slack (MET) 0.087
Startpoint: u_core/cnt_reg[0]
Endpoint : u_core/cnt_reg[5]
slack (MET) 0.240
End of report3. The Same Eleven Probes
probe.pl from Chapter 28.2 runs a .py script under python3, so the rewrite can be taken through exactly the scenarios the original went through. (The harness itself is repeated in section 10 so that this page runs on its own.) compare_inv.pl reads two inventories and reports, for each scenario, the exit status of both and whether each of the other three channels matched:
use strict;
use warnings;
# compare_inv.pl - channel-by-channel comparison of two inventories.
# usage: perl compare_inv.pl old.inv new.inv
sub load {
my ($file) = @_;
my (%inv, $id, $ch);
open my $fh, '<', $file or die "cannot read $file: $!\n";
while (my $line = <$fh>) {
if ($line =~ /^(S\d+)\s+(.*?)\s+argv:/) { $id = $1; $inv{$id}{name} = $2; next }
if ($line =~ /^ (\w+)\s*: (.*)$/) { $ch = $1; $inv{$id}{$ch} = "$2\n"; next }
$inv{$id}{$ch} .= $line if defined $ch; # a continuation line
}
return \%inv;
}
my ($old, $new) = map { load($_) } @ARGV;
printf "%-30s %-9s %-7s %-7s %s\n", 'scenario', 'exit', 'stdout', 'stderr', 'files';
my $same_all = 0;
for my $id (sort { substr($a, 1) <=> substr($b, 1) } keys %$old) {
my ($o, $n) = ($old->{$id}, $new->{$id});
my @verdict = map { $o->{$_} eq $n->{$_} ? 'same' : 'DIFF' } qw(stdout stderr files);
chomp(my $eo = $o->{exit}); chomp(my $en = $n->{exit});
$same_all++ if $eo eq $en && !grep { $_ eq 'DIFF' } @verdict;
printf "%-30s %-9s %-7s %-7s %s\n", sprintf("%-4s %s", $id, $o->{name}), "$eo -> $en", @verdict;
}
printf "\nscenarios identical on all four channels: %d of %d\n", $same_all, scalar keys %$old;#!/bin/bash
perl probe.pl sta_summary.pl > original.inv
perl probe.pl sta_summary_v2.py > rewrite.inv
perl compare_inv.pl original.inv rewrite.invscenario exit stdout stderr files
S1 two reports 1 -> 1 DIFF same DIFF
S2 two reports, -v 1 -> 1 DIFF same DIFF
S3 no argument at all 1 -> 1 DIFF same DIFF
S4 empty dir, old tmp.csv 0 -> 2 DIFF DIFF same
S5 missing dir 2 -> 2 same DIFF same
S6 dir named 0 0 -> 1 DIFF same same
S7 unreadable report 1 -> 1 same DIFF DIFF
S8 report with no slack 1 -> 1 DIFF DIFF DIFF
S9 slack with 4 decimals 1 -> 1 same same DIFF
S10 unknown option -x 1 -> 2 DIFF DIFF DIFF
S11 256 failing corners 0 -> 1 same same DIFF
scenarios identical on all four channels: 0 of 11The exit status agrees in seven of eleven scenarios. The behaviour agrees in none. The exit-status column is the one people check first, and on its own it would suggest a mostly faithful replacement. The other three columns show a different program.
4. Behaviour by Behaviour
Map the differences back onto Chapter 28.2's inventory, and sort them by what they mean for whoever relies on the script.
| ID | Original behaviour | Rewrite | Effect |
|---|---|---|---|
| B1 | fixed-format summary line | same format | reproduced (for corners with a violation) |
| B2 | -v endpoints with a 4-space indent | - bullets | changed: a consumer parses this |
| B3 | no argument means . | same | reproduced |
| B4 | a clean corner prints wns=0.000 | prints 0.087 | bug fixed; also changes the summary |
| B5 | tmp.csv written in the working directory | never written | changed: a consumer reads this file |
| B6 | CSV fields are captured text, empty when clean | no CSV | changed (with B5) |
| B7 | empty directory: STDOUT, exit 0, old tmp.csv kept | STDERR, exit 2 | changed: a nightly gate now stops |
| B8 | missing directory: errno message, exit 2 | own message, exit 2 | partly reproduced: same code, other text |
| B9 | a directory named 0 becomes . | reads 0 | bug fixed |
| B10 | an unreadable or empty report is dropped silently | crashes | new failure |
| B11 | unknown option: warning, then ignored | usage error, exit 2 | bug fixed |
| B12 | exit = count of failing corners (256 reads 0) | exit 1 | bug fixed |
Tally it: two behaviours reproduced, one partly, four bugs fixed, three interface changes that a known consumer depends on, and one new failure. That new failure deserves its own run, because it is the kind of difference that only shows up on a bad night.
#!/bin/bash
# One corner's timing run died before writing any paths. What does each
# implementation do with the other two, perfectly good, reports?
work=$(mktemp -d)
cp -R reports sta_summary.pl sta_summary_v2.py "$work"/
cd "$work" || exit 1
printf 'Report : timing\nError: license unavailable\n' > reports/ff_0p88v_m40c.rpt
for impl in "perl sta_summary.pl" "python3 sta_summary_v2.py"; do
echo "=== $impl"
$impl reports 2> err.txt | sed 's/^/ /'
status=${PIPESTATUS[0]} # read it at once: the next command resets it
last=$(tail -1 err.txt)
echo " exit $status; last line of STDERR: ${last:-(none)}"
done=== perl sta_summary.pl
ss_0p72v_125c paths=3 viol=2 wns=-0.043 tns=-0.054
tt_0p80v_25c paths=2 viol=0 wns=0.000 tns=0.000
exit 1; last line of STDERR: (none)
=== python3 sta_summary_v2.py
exit 1; last line of STDERR: TypeError: unsupported format string passed to NoneType.__format__When one corner's timing run dies early and leaves a report with no paths, the original prints the two good corners and says nothing about the third. That is B10, and it is a defect. The rewrite does worse. It formats wns=None with :.3f, raises TypeError, and prints nothing at all: not the two good corners, not a summary. And its exit status, 1, is the same number the flow already reads as "timing violated", so the Makefile's message points the engineer at timing when the real problem is a dead run.
The unreadable report of S7 fails the same way, through PermissionError: [Errno 13] Permission denied. There the good corner happens to sort before the unreadable one, so one line of summary is printed before the crash, and which corners survive depends on their names. Both are fixable in a few lines. Both are also the kind of bug that the original's author fixed, or papered over, years ago with that || next, and that a rewrite from the usage comment reintroduces. The old script's ugliest line was encoding a decision about failure handling. The rewrite had no way to know.
5. What the Consumers See on the Night of the Cutover
Chapter 28.2 found three consumers in the sample flow. Run each of them against both implementations, in a run directory that still holds yesterday's tmp.csv, because a real run directory does:
#!/bin/bash
# post_wns.sh (SAMPLE) - feed per-corner WNS to the team dashboard.
# Reads the CSV that sta_summary.pl leaves in the run directory.
while IFS=, read -r corner viol wns; do
echo "dashboard: $corner wns=${wns:-clean}"
done < "${1:-run}/tmp.csv"#!/bin/bash
# Tonight's run, as each consumer from Chapter 28.2 sees it, under the
# original script and under the rewrite. Yesterday's tmp.csv is still in
# the run directory, as it would be in a real one.
here=$(pwd)
for impl in "perl sta_summary.pl" "python3 sta_summary_v2.py"; do
work=$(mktemp -d)
cp -R reports sta_summary.pl sta_summary_v2.py flow "$work"/
printf 'ss_0p72v_125c,5,-0.210\ntt_0p80v_25c,1,-0.005\n' > "$work/tmp.csv"
cd "$work" || exit 1
echo "=== $impl"
$impl reports > sta_summary.txt
status=$?
echo " Makefile gate : exit $status -> make would $([ $status -eq 0 ] && echo CONTINUE || echo STOP)"
bash flow/bin/post_wns.sh . | sed 's/^/ /'
echo " weekly digest : $($impl -v reports | grep -c '^ [^ ]') failing endpoints"
cd "$here"
done=== perl sta_summary.pl
Makefile gate : exit 1 -> make would STOP
dashboard: ss_0p72v_125c wns=-0.043
dashboard: tt_0p80v_25c wns=clean
weekly digest : 2 failing endpoints
=== python3 sta_summary_v2.py
Makefile gate : exit 1 -> make would STOP
dashboard: ss_0p72v_125c wns=-0.210
dashboard: tt_0p80v_25c wns=-0.005
weekly digest : 0 failing endpointsRead the two blocks side by side.
The Makefile gate behaves identically. Both versions exit 1 because a corner violates, and Make stops. If the cutover were checked by running the flow once and seeing it stop, it would pass.
The dashboard is silently wrong. The rewrite does not write tmp.csv, so post_wns.sh reads the file that is still there from yesterday and publishes -0.210 and -0.005, numbers from a different run, for both corners. No step fails and no message appears. The first person to notice will be someone asking why a fixed corner still shows a violation.
The weekly digest reports zero failing endpoints. It counts lines that start with four spaces, and the rewrite indents its bullets by two. The count drops from 2 to 0, and the mail says everything is fine.
None of the three is a bug in the rewrite in isolation. Each is a difference between two reasonable programs, and each lands on somebody who was not in the room when the rewrite was written.
6. Where the Cost Actually Is
The rewrite took a short time to write, and that is the number people quote when they propose one. It is the smallest number in the whole exercise. The real cost is in the work that follows, and it can be counted from the table in section 4: every row that is not "reproduced" is a decision somebody has to make.
| Difference | Who has to be asked | What has to happen |
|---|---|---|
B2 -v format | owner of weekly_digest.pl | change the digest, or keep the old indent |
| B4 true WNS | whoever reads the summary and the dashboard | agree that the numbers will move, and announce it |
B5/B6 no tmp.csv | owner of post_wns.sh, and anyone else reading the file | provide the file, or migrate them all to a new output |
| B7 empty dir exits 2 | owner of the Makefile | decide whether an empty run should stop signoff (it probably should) |
| B8 new message | anyone matching the old text in a log scanner | usually nobody; confirm |
| B9, B11, B12 fixes | owner of the Makefile | confirm nothing relied on the old behaviour |
| B10 crash | the rewrite's author | fix it, which means deciding what a dead run should report |
That ledger is the actual cost of the rewrite. Each row is a conversation, a decision, a change in somebody else's code or a deliberate "keep it", and a test that proves the decision was implemented. For a script with three consumers it is a few days of coordination. For the scripts Chapter 16.7 found at the bottom of every migration (a regression driver, a queue submitter), the consumers are other teams, and the ledger is the migration.
There is an old name for the trap underneath this. Fred Brooks, in The Mythical Man-Month, called it the second-system effect: the replacement is designed with all the improvements the first version lacked, and those improvements are where the new problems come from. The rewrite above is a small, honest example. Every one of its fixes is an improvement, and every one is also a change that someone has to absorb.
7. When Leaving It Alone Is the Professional Answer
Put the measurements together and the cases where a working script should simply be left alone become clear. Leave it alone when most of these hold:
- It works for the people who use it, including its known defects, which have known workarounds.
- It rarely changes. A script edited once a year costs very little to keep, as long as each edit is small, careful and checked against the inventory.
- It has consumers you do not own, so every behaviour difference is a negotiation with another team.
- There is no inventory and no test. Without them, a rewrite cannot even measure how different it is.
- Nothing forces the change. No interpreter removal, no new tool format, no requirement the script cannot meet.
In that situation, the professional answer to "shall we rewrite it?" is "not now", followed by the cheap things that make a later change safe: take the inventory (Chapter 28.2), write down the consumers, and if it must be touched, wrap it rather than replace it. Chapter 28.4 turns these conditions into an explicit choice between four strategies.
This is not timidity. A senior engineer's job includes not spending a week of coordination to make a working script prettier, especially in the weeks before a tapeout, when a quiet regression in a timing summary is about the most expensive kind there is.
8. When a Rewrite Is Right
The experiment is not an argument that rewrites are always wrong. A rewrite is the right call when the conditions flip:
- The script must change substantially anyway. If a new tool version changes the report format, every line of the parser is being rewritten regardless, and the question becomes which language it is rewritten in.
- The inventory exists and the consumers are known. Then the ledger in section 6 can be written before the rewrite, and every difference becomes a planned decision rather than a discovery.
- The script is small and has few consumers, so the ledger is short. A 40-line formatter whose output only a person reads is a fine afternoon's rewrite.
- The current behaviour is actively harmful, like B12's gate that passes 256 failing corners, and the safest fix is a small, well-understood replacement with its own tests.
Notice what all four have in common: the rewrite starts from the inventory, not from the usage comment. That one change of starting point is the difference between this chapter's rewrite and the port in Chapter 28.8, which reproduces all twelve behaviours, including the ones it would rather not, and then changes them one at a time, on purpose.
9. Mistakes Behind "Just Rewrite It"
Counting lines of code. The original is 59 lines, so the job looks small. Its cost to replace is set by its twelve behaviours and three consumers, as Chapter 16.2 measured for a different script.
Writing the replacement from the usage comment. The comment described the intent. The probes in section 3 measured the program, and none of the eleven scenarios agreed.
Validating by exit status. Seven of eleven matched, and that check alone would have shipped a stale dashboard.
Treating fixes as free. Four of the rewrite's differences are genuine bug fixes. Each still changes what a consumer sees, and must be announced like any other change.
Discarding the ugly code's knowledge. The || next that looked careless was handling a dead run. The rewrite crashed on the same input.
10. The Harness, Repeated
This is probe.pl from Chapter 28.2, unchanged, so that every run on this page works in a fresh directory:
use strict;
use warnings;
use File::Temp qw(tempdir);
use File::Find;
use File::Copy qw(copy);
use File::Spec;
use Digest::MD5;
# probe.pl - record what a script DOES, one scenario at a time.
# usage: perl probe.pl SCRIPT (a .py script is run with python3)
# Every scenario runs in its own fresh scratch directory. For each one we
# record the four channels a caller can observe: STDOUT, STDERR, the exit
# status, and every file created, changed or deleted in that directory.
my $script = File::Spec->rel2abs(shift // die "usage: probe.pl SCRIPT\n");
my @run = $script =~ /\.py\z/ ? ('python3', $script) : ($^X, $script);
my $fixtures = File::Spec->rel2abs('reports'); # the two sample reports
sub put { # write a file, making dirs
my ($path, $text) = @_;
my ($vol, $dirs) = File::Spec->splitpath($path);
mkdir $dirs unless -d $dirs;
open my $fh, '>', $path or die "cannot write $path: $!\n";
print {$fh} $text;
close $fh;
}
sub sample_reports { # copy both samples into $to
my ($to) = @_;
mkdir $to;
copy("$fixtures/$_", "$to/$_") or die "copy $_: $!\n"
for qw(ss_0p72v_125c.rpt tt_0p80v_25c.rpt);
}
sub one_path { # a minimal one-path report
my ($slack, $status) = @_;
return "Report : timing (synthetic teaching format)\n\n"
. "Startpoint: u_core/a_reg\nEndpoint : u_core/b_reg\n"
. " slack ($status) $slack\n\nEnd of report\n";
}
# name, setup(sub receiving the scratch dir), arguments
my @scenarios = (
[ 'S1 two reports', sub { sample_reports("$_[0]/reports") }, 'reports' ],
[ 'S2 two reports, -v', sub { sample_reports("$_[0]/reports") }, '-v', 'reports' ],
[ 'S3 no argument at all', sub { sample_reports($_[0]) } ],
[ 'S4 empty dir, old tmp.csv', sub { mkdir "$_[0]/empty"; put("$_[0]/tmp.csv", "old,9,-9.999\n") }, 'empty' ],
[ 'S5 missing dir', sub { }, 'nosuch' ],
[ 'S6 dir named 0', sub { sample_reports("$_[0]/0") }, '0' ],
[ 'S7 unreadable report', sub { sample_reports("$_[0]/r"); chmod 0, "$_[0]/r/tt_0p80v_25c.rpt" }, 'r' ],
[ 'S8 report with no slack', sub { sample_reports("$_[0]/r");
put("$_[0]/r/ff_0p88v_m40c.rpt", "Report : timing\nError: license unavailable\n") }, 'r' ],
[ 'S9 slack with 4 decimals', sub { put("$_[0]/r/ss_x.rpt", one_path('-0.0430', 'VIOLATED')) }, 'r' ],
[ 'S10 unknown option -x', sub { sample_reports("$_[0]/reports") }, '-x', 'reports' ],
[ 'S11 256 failing corners', sub { put(sprintf("%s/r/c%03d.rpt", $_[0], $_), one_path('-0.010', 'VIOLATED'))
for 1 .. 256 }, 'r' ],
);
sub snapshot { # relative path => md5
my ($root) = @_;
my %s;
find({ no_chdir => 1, wanted => sub {
return unless -f $_;
my $rel = File::Spec->abs2rel($_, $root);
if (open my $fh, '<', $_) { binmode $fh; $s{$rel} = Digest::MD5->new->addfile($fh)->hexdigest }
else { $s{$rel} = 'unreadable' }
} }, $root);
return \%s;
}
sub slurp { open my $fh, '<', $_[0] or return ''; local $/; return scalar <$fh> }
sub show { # print a channel, briefly
my ($label, $text) = @_;
my @l = split /\n/, $text;
if (!@l) { printf " %-7s: (nothing)\n", $label; return }
my @shown = @l > 4 ? (@l[0 .. 1], sprintf "... %d lines in all", scalar @l) : @l;
printf " %-7s: %s\n", $label, shift @shown;
printf " %-7s %s\n", '', $_ for @shown;
}
for my $sc (@scenarios) {
my ($name, $setup, @argv) = @$sc;
my $dir = tempdir(CLEANUP => 1);
my $cap = tempdir(CLEANUP => 1); # captures live OUTSIDE $dir
$setup->($dir);
my $before = snapshot($dir);
my $pid = fork() // die "fork: $!\n";
if ($pid == 0) {
open STDOUT, '>', "$cap/out" or die;
open STDERR, '>', "$cap/err" or die;
chdir $dir or die "chdir: $!\n";
exec @run, @argv or die "exec: $!\n";
}
waitpid $pid, 0;
my ($code, $sig) = ($? >> 8, $? & 127);
my $after = snapshot($dir);
my @files;
for my $f (sort keys %{{ %$before, %$after }}) {
my ($was, $now) = ($before->{$f}, $after->{$f});
my $what = !defined $was ? 'created'
: !defined $now ? 'deleted'
: $was ne $now ? 'changed'
: '';
next unless $what;
my @body = $what eq 'deleted' ? () : split /\n/, slurp("$dir/$f");
push @files, @body > 3 ? sprintf("%s %s: %d lines", $what, $f, scalar @body)
: ("$what $f:", map { " $_" } @body);
}
push @files, "tmp.csv untouched" if exists $before->{'tmp.csv'} && !grep /tmp\.csv/, @files;
print "$name argv: ", (@argv ? "@argv" : '(none)'), "\n";
printf " %-7s: %d%s\n", 'exit', $code, $sig ? " (signal $sig)" : '';
show('stdout', slurp("$cap/out"));
show('stderr', slurp("$cap/err"));
show('files', join "\n", @files);
chmod 0644, glob("$dir/*/*"); # let CLEANUP remove S7's file
}11. Exercises
1. Make the rewrite faithful. Change sta_summary_v2.py until compare_inv.pl reports 11 of 11 scenarios identical. Keep a list of every change you made against your better judgement (writing tmp.csv into the working directory, for example). That list is the set of decisions a migration must hand to the consumers' owners.
2. Fix B10 properly. Decide what a summary should print for a report with no paths: nothing, a line with paths=0, or an explicit error. Implement your choice in the rewrite, and write the inventory row that describes the new behaviour. Which consumer from section 5 is affected by your choice, and how?
3. Find a check that would have caught the dashboard. Propose one automatic check, runnable in the nightly flow, that would have caught post_wns.sh publishing yesterday's numbers under either implementation. (Hint: Chapter 13.6's file tests, and what a consumer can learn from a file's age.)
4. Price your own rewrite. Pick a script in your flow that someone wants to rewrite. Without writing any code, produce the section 6 ledger for it: the behaviours you know of, who consumes each, and who would have to be asked. Count the conversations. Then decide whether the rewrite is still worth it this quarter.
5. The seven. Explain, for each of the seven scenarios where the exit statuses matched, which channel would have revealed the difference, and which consumer would have been the first to notice it.
12. What To Remember
- The case for a rewrite is usually made honestly: the old script is buggy, ugly and untested. That is not the same as a reason to replace it now.
- A competent rewrite written from the usage comment behaved identically in 0 of 11 scenarios (measured).
- Exit status matched in 7 of 11, the most misleading number on the page.
- Of 12 inventoried behaviours, the rewrite reproduced 2, fixed 4 real bugs, changed 3 that a consumer depends on, and added a crash on a dead run.
- On the night of the cutover the gate behaved, the dashboard published yesterday's numbers, and the digest reported zero (measured).
- The cost of a rewrite is its difference ledger: one decision, one owner and one test per changed behaviour.
- Ugly code often encodes a decision.
|| nextwas handling a dead run. - Leave a working script alone when it is stable, rarely changed, consumed by others, untested, and nothing forces the change.
- Rewrite when the change is coming anyway, the inventory exists, the consumers are few, or the current behaviour is harmful.
- Start any rewrite from the inventory, not from the comment. That is what turns surprises into decisions.
Chapter 28.4 makes the choice explicit. Leaving a script alone and rewriting it are two of four strategies, and the right one for a given script can be read off four facts about it: its size, its tests, its change rate and how far its outputs reach.
Where this fits
Part of the Perl curriculum.