Where Each Language Belongs

Part II · Module 26 · Chapter 26.9 · Where Each Language Belongs

1. Four Languages, Four Jobs

The canonical example for this chapter is four comment lines, and they are the whole rule:

# bash  : orchestration and entry points
# tcl   : inside the tool
# perl  : text out, structured data in
# python: anything new

Each line is a job, not a preference, and each job has a reason that follows from where the language runs and what it is good at:

Bash runs first and decides the order. It is what a CI system, a queue and a Makefile call. Its strengths are starting programs, passing their arguments and environment, and reacting to their exit status — exactly the things Chapters 26.1, 26.2 and 26.8 measured. It is weak at anything involving data structures or arithmetic, and its quoting rules are a constant hazard.

Tcl runs inside the tool. It is the only language that can talk to the tool's in-memory design — query objects, apply constraints, request reports — because the tool embeds the interpreter. Chapter 26.4 reviewed it from Perl's side. Its job ends when the tool has written what it was asked to write.

Perl runs after the tool and turns its text into data. Reports and logs written for people become tables written for programs: Chapters 26.3 and 26.5. Perl's regular expressions, its line-at-a-time reading and its fast start-up are why it took this job in the first place (Chapter 16.2), and why it still holds it in most established flows.

Python takes on new work, especially anything that consumes the structured data — dashboards, trend analysis, database loading — and new tooling that has no Perl history to preserve. Section 5 is precise about what that does and does not mean.

2. Mistake One: Heavy Text Processing in the Tool's Tcl

Tcl can read files and match regular expressions, so it is tempting to finish the job inside the tool: write the report, then read it back in the same Tcl script and compute the summary. For a few lines that is harmless. For a large report, it puts work that belongs after the tool inside it.

The measurement below runs the same scan — count the violations and find the worst slack — over a generated report of twenty thousand paths in this module's synthetic format, once in Perl and once in a plain tclsh. The two scripts are written to be as alike as the languages allow.

use strict;
use warnings;

my ($violations, $worst) = (0, undef);
open my $fh, "<", "big.rpt" or die "big.rpt: $!\n";
while (my $line = <$fh>) {
    next unless $line =~ /^\s+slack \((\w+)\)\s+(-?[\d.]+)/;
    $violations++ if $1 eq "VIOLATED";
    $worst = $2 if !defined $worst || $2 < $worst;
}
close $fh;
print "violations=$violations worst=$worst\n";

set violations 0
set worst ""
set fh [open big.rpt r]
while {[gets $fh line] >= 0} {
    if {![regexp {^\s+slack \((\w+)\)\s+(-?[\d.]+)} $line -> status value]} continue
    if {$status eq "VIOLATED"} { incr violations }
    if {$worst eq "" || $value < $worst} { set worst $value }
}
close $fh
puts "violations=$violations worst=$worst"

use strict;
use warnings;
use Time::HiRes qw(time);

# A synthetic report: 20,000 paths, 80,000 lines, in this module's format.
open my $out, ">", "big.rpt" or die "big.rpt: $!\n";
for my $i (1 .. 20_000) {
    my $v = $i % 97 == 0;
    printf {$out} "Startpoint: u_core/r%05d_reg\nEndpoint  : u_core/q%05d_reg\n  slack %s%.3f\n\n",
        $i, $i, $v ? "(VIOLATED)   " : "(MET)         ",
        $v ? -0.010 - ($i % 7) / 1000 : 0.050 + ($i % 50) / 1000;
}
close $out;

my $p = `perl scan.pl`;
my $t = `tclsh scan.tcl`;
print "perl : $p", "tcl  : $t";
print "same answer: ", ($p eq $t ? "yes" : "NO"), "\n";

# Publish a verdict against a fixed threshold, never a raw time.
my $slower = 0;
for (1 .. 3) {
    my $t0 = time; system("perl scan.pl > /dev/null");  my $tp = time - $t0;
    $t0    = time; system("tclsh scan.tcl > /dev/null"); my $tt = time - $t0;
    $slower++ if $tt >= 3 * $tp;
}
print "Tcl took at least 3 times as long as Perl in $slower of 3 rounds\n";

perl : violations=206 worst=-0.016
tcl  : violations=206 worst=-0.016
same answer: yes
Tcl took at least 3 times as long as Perl in 3 of 3 rounds

The answers are identical, and the Tcl scan took at least three times as long in every round. Measured separately across eleven rounds on this machine, the ratio ranged from about 6 to about 10. The page publishes only the threshold verdict because raw timings change with every machine and every run, and a number that cannot be reproduced does not belong in a published table.

The speed is the smaller half of the argument, and it should not be over-read: this is a plain tclsh 8.5.9, and the Tcl inside a current tool may be a newer, faster version. The stronger reasons are about where the work runs:

• Inside the tool, every second is a licensed second. The tool holds its licence and usually the whole design in memory while the Tcl reads a text file that the tool has already finished writing.
• A parser inside the tool can only be re-run inside the tool. When the summary is wrong, fixing and re-running a Perl parser needs only the report file. Re-running a Tcl parser that lives in the tool script means starting the tool again — a design load, a licence checkout, often a queue wait.
• The parser cannot be tested without the tool. Chapter 26.5's collector was tested against six damaged directories in seconds. The same logic in tool Tcl could only be tested by producing damaged reports from inside a tool session.

The boundary rule that follows: Tcl writes the report and the manifest, and stops. Anything that reads a finished file belongs after the tool exits.

3. Mistake Two: Perl Driving a Tool That Tcl Already Controls

The opposite mistake starts from the Perl side. A Perl script needs reports for several corners, so it loops over the corners and starts the tool once for each, generating a small Tcl script per corner. Each session loads the design, analyses one corner, writes one report and exits.

It works, and it produces the same reports. The cost is invisible in the reports and enormous in the run: every session loads the design again. Chapter 26.4's teaching mock counts its design loads in a file, which makes the cost visible. (In the mock, a load is instant. In a real timing tool, loading a full design with its parasitics is usually the most expensive single step of the run.)

use strict;
use warnings;

# The mistake: Perl starts the tool once per corner.
my @corners = qw(ss_0p72v_125c tt_0p80v_25c);
unlink "loads.log";
mkdir "perl_reports";
for my $c (@corners) {
    open my $tcl, ">", "one_corner.tcl" or die "one_corner.tcl: $!\n";
    print {$tcl} "read_design chip_top\n",
                 "set_corner $c\n",
                 "report_timing -max_paths 2 > perl_reports/$c.rpt\n";
    close $tcl;
    system("tclsh", "mock_sta.tcl", "-f", "one_corner.tcl") == 0
        or die "corner $c failed\n";
}

#!/bin/bash
echo "--- Perl starts the tool once per corner"
perl per_corner_from_perl.pl | sed 's/^/  /'
echo "  design loads: $(wc -l < loads.log | tr -d ' ')"

echo "--- one tool session, the corner loop in Tcl (26.4)"
rm -f loads.log
export CORNERS="ss_0p72v_125c tt_0p80v_25c"
tclsh mock_sta.tcl -f sta_corners_v2.tcl | sed 's/^/  /'
echo "  design loads: $(wc -l < loads.log | tr -d ' ')"

echo "--- are the reports the same?"
for c in ss_0p72v_125c tt_0p80v_25c; do
    cmp -s "perl_reports/$c.rpt" "reports/$c.rpt" && echo "  $c: identical" || echo "  $c: DIFFERENT"
done

--- Perl starts the tool once per corner
  mock_sta: design chip_top loaded
  mock_sta: design chip_top loaded
  design loads: 2
--- one tool session, the corner loop in Tcl (26.4)
  mock_sta: design chip_top loaded
  design loads: 1
--- are the reports the same?
  ss_0p72v_125c: identical
  tt_0p80v_25c: identical

Two corners, two design loads from Perl; one from the Tcl loop — and the reports are byte-identical. The count grows with the corners: a sign-off run with a dozen corners loads the design a dozen times from the Perl loop and once from the Tcl one. Each session is also a separate licence checkout and a separate tool start-up.

The Perl version has a second, quieter cost. It generates Tcl to send one command at a time, so every value it pastes into that Tcl — here, corner names — is subject to everything Chapter 26.6 measured. With safe corner names the risk is invisible; with a port list it is not.

There is one legitimate reason to run one session per corner: running the corners in parallel on separate machines to finish sooner. That is a deliberate trade — more licences and more loads for less wall-clock time — and when it is made, the orchestration belongs to the queue or the Bash entry point, each session's corner work still belongs to that session's Tcl, and the decision should be written down where the next engineer will find it. The mistake is doing it serially, from Perl, by habit.

The boundary rule: when the tool can loop, let its Tcl loop. Perl's job is to prepare data the Tcl reads (Chapter 26.6) and to read what the Tcl wrote (Chapter 26.5) — not to stand between them sending commands.

4. The Quieter One: Parsing in Bash

Bash scripts drift into parsing by small steps: one grep -c to count violations, then an awk to pull out the worst slack, then a sed to clean the corner name. Each step is a line, and the result is a parser with no data structures, no error handling, and the exit-status rules of Chapter 26.1 applied to every stage of every pipeline.

The warning signs are concrete: grep, awk, sed or cut reading a report or a log; a variable holding a number extracted from text; a pipeline whose status the script acts on. Any one of them is fine in an interactive shell. In a flow's entry point, each one is a parser that Chapter 26.5's six damaged directories have never been run against.

The boundary rule: Bash decides what runs and in what order; it does not read what the tools wrote. When a Bash script needs a number from a report, it calls a Perl stage that prints it — or, better, reads the CSV that the Perl stage already wrote.

5. Where Python Fits, Honestly

"Python: anything new" is the line most likely to be misread, in both directions.

It does not mean "rewrite the Perl". A Perl stage that parses a tool's reports correctly, has been tested against the tool's real failure modes and has run every night for years is an asset. Rewriting it converts tested code into untested code and buys nothing the consumers can see. Chapter 16.4 drew the honest boundary between the two languages; nothing in this module changes it.

It does mean "new work starts where it will be maintained". A new dashboard, a new trend analysis, a new consumer of the CSV and JSON this module's Perl produces — those are naturally Python today, because that is where most teams' new code and most new engineers already are, and because a consumer of a documented schema (Chapter 26.3) does not care what language produced it.

That last point is the useful one. A documented hand-off is what makes the language choice reversible. If the Perl stage writes a schema'd CSV, its consumers can be in any language, and it can itself be replaced one day without the consumers noticing. If it writes prose that a Python script scrapes, both are locked to each other's exact text.

6. Reviewing a Flow's Boundaries With a Script

The three mistakes above leave recognisable traces in source code, which means a short script can point at the places worth reviewing. The sample flow below is written for this chapter: six small files, three of which contain one of the mistakes and three of which are clean. The tool names in them are only strings that the review script matches; nothing here runs a vendor tool.

#!/bin/bash
set -euo pipefail
pt_shell -f sta.tcl > sta.log
viol=$(grep -c VIOLATED reports/*.rpt | awk -F: '{s += $2} END {print s}')
echo "violations: $viol"

foreach corner $::env(CORNERS) {
    set_corner $corner
    report_timing -max_paths 2 > reports/$corner.rpt
}

set fh [open reports/ss.rpt r]
while {[gets $fh line] >= 0} {
    if {[regexp {slack \(VIOLATED\)\s+(\S+)} $line -> v]} { lappend vs $v }
}
close $fh

foreach p [get_ports *] {
    if {[regexp {^scan_} [get_object_name $p]]} { lappend scan $p }
}

for my $c (qw(ss tt ff)) {
    write_corner_script($c);
    system("pt_shell", "-f", "corner_$c.tcl");
}

for my $rpt (sort glob "reports/*.rpt") {
    open my $fh, "<", $rpt or die "$rpt: $!\n";
    while (<$fh>) { print if /slack/ }
}

The review script applies one deliberately narrow test per mistake and prints the evidence for each finding:

use strict;
use warnings;
use File::Glob qw(bsd_glob);

my $TOOL_SHELL = qr/\b(?:pt_shell|dc_shell|innovus|genus|tempus|vcs|xrun|vsim)\b/;
my @findings;

for my $f (sort(bsd_glob("flow/*"))) {
    open my $fh, "<", $f or die "$f: $!\n";
    my @lines = <$fh>;
    close $fh;
    my $text = join "", @lines;

    # Tcl reading a file line by line AND pattern-matching: parsing in the tool.
    if ($f =~ /\.tcl$/ && $text =~ /\bgets\b/ && $text =~ /\bregexp\b/) {
        push @findings, [$f, "text processing inside the tool's Tcl", grep_lines(\@lines, qr/\bgets\b|\bregexp\b/)];
    }
    # Perl starting a tool shell inside a loop: driving the tool from Perl.
    if ($f =~ /\.pl$/) {
        for my $i (0 .. $#lines) {
            next unless $lines[$i] =~ /^\s*(?:for|foreach|while)\b/;
            my @body = @lines[$i .. ($i + 5 > $#lines ? $#lines : $i + 5)];
            if (grep { /\b(?:system|exec|qx)\b|`/ && $_ =~ $TOOL_SHELL } @body) {
                push @findings, [$f, "Perl starts a tool shell inside a loop", "line " . ($i + 1) . ": $lines[$i]"];
            }
        }
    }
    # Bash running a text tool over reports or logs: parsing in the entry point.
    if ($f =~ /\.sh$/) {
        my @hits = grep_lines(\@lines, qr/\b(?:grep|awk|sed|cut)\b.*\.(?:rpt|log)\b/);
        push @findings, [$f, "Bash parses tool output", @hits] if @hits;
    }
}

sub grep_lines {
    my ($lines, $re) = @_;
    return map { "line " . ($_ + 1) . ": $lines->[$_]" } grep { $lines->[$_] =~ $re } 0 .. $#$lines;
}

for my $f (@findings) {
    my ($file, $what, @evidence) = @$f;
    print "$file: $what\n";
    print "    $_" for @evidence;
}
my %flagged = map { $_->[0] => 1 } @findings;
my @clean = grep { !$flagged{$_} } sort(bsd_glob("flow/*"));
printf "\n%d finding(s); not flagged: %s\n", scalar @findings, join(", ", @clean);

flow/drive_corners.pl: Perl starts a tool shell inside a loop
    line 1: for my $c (qw(ss tt ff)) {
flow/run_flow.sh: Bash parses tool output
    line 4: viol=$(grep -c VIOLATED reports/*.rpt | awk -F: '{s += $2} END {print s}')
flow/summarise.tcl: text processing inside the tool's Tcl
    line 2: while {[gets $fh line] >= 0} {
    line 3:     if {[regexp {slack \(VIOLATED\)\s+(\S+)} $line -> v]} { lappend vs $v }

3 finding(s); not flagged: flow/collect.pl, flow/pick_ports.tcl, flow/sta.tcl

Three findings, one per mistake, each with the lines that triggered it — and the three clean files were not flagged. The one that matters most for trusting the script is pick_ports.tcl: it uses regexp, which a cruder test ("Tcl that uses regexp") would have flagged. It is matching object names inside the tool, which is exactly Tcl's job, and the review does not fire on it because it requires a file being read with gets as well.

That is also the honest limit of the script. Each test is a heuristic about what code usually looks like, not an analysis of what it does:

• It will miss mistakes written differently — a Perl tool launch hidden in a subroutine called from the loop, a Tcl file read with read instead of gets, a parser in Bash that uses while read.
• It can fire on code that is fine — a tiny gets loop reading a three-line configuration file in Tcl is not a performance problem.
• Its findings are places to look, not verdicts. The point is to make the review of a large flow start from twenty specific lines instead of two thousand files.

Section 3's parallel-corner exception is a good example: the script would flag a Perl loop that launches sessions on purpose, and the right response is a comment in the code explaining the trade, not a change.

7. Exercises

1. Classify the stages. For a flow you know — or the one in this module — list every stage and the language it is written in. Mark each one that is in its natural place, and for each that is not, name which of the three mistakes it is and what moving it would cost.

2. Defeat the review. Rewrite drive_corners.pl so that it makes the same mistake but boundary_review.pl no longer flags it. Then improve the review so it catches your version without starting to flag collect.pl. What does each round of this teach about heuristic checks?

3. The parallel case, written down. A team runs twelve corners as twelve parallel tool sessions to meet an overnight deadline. Write the comment that should sit at the top of the script that launches them: the trade being made, the numbers that justify it, and the condition under which it should be revisited.

4. Move a parser out of the tool. Take summarise.tcl and move its work to a Perl stage. Write down exactly which file the Tcl must now produce, what the Perl must check before trusting it (Chapter 26.5's table), and what the tool session no longer needs to do.

5. One hand-off, two languages. The Perl collector from Chapter 26.5 writes corners.csv. Write a ten-line Python consumer that reads it by column name, and show that it keeps working if the collector is changed to add a new column at the end. Which property of the hand-off made that true?

8. What To Remember

• Bash: entry points, sequencing, exit statuses. Tcl: inside the tool. Perl: tool text into structured data. Python: new work.
• Each language's job follows from where it runs — Tcl is the only one that can touch the tool's in-memory design.
• Mistake one: heavy text processing in the tool's Tcl. Measured: identical answers, and Tcl took at least 3x as long in 3 of 3 rounds (6x to 10x over 11 rounds here).
• The bigger costs are licensed time, re-running only inside the tool, and testing only inside the tool. Tcl writes the report and stops.
• Mistake two: Perl driving a tool Tcl already controls. Measured: 2 design loads for 2 corners from Perl, 1 from the Tcl loop, byte-identical reports.
• Running sessions in parallel on purpose is a trade, not a mistake — write it down, and orchestrate it from the queue or Bash.
• The quieter mistake: parsing in Bash. Bash decides what runs; it does not read what the tools wrote.
• "Python: anything new" does not mean rewriting tested Perl. It means new work starts where it will be maintained.
• A documented hand-off makes the language choice reversible.
• A review script points at places to look; it does not deliver verdicts. Prove it does not fire on the clean cases, and know what it misses.

That closes Module 26. You can now place a Perl stage inside a multi-language flow and make every boundary around it hold: a Bash entry point that forwards arguments with "$@" and returns the parser's status unchanged instead of an echo's zero; values passed across the Bash/Perl boundary as data rather than spliced into code, with the empty variables, dash-leading names and newline-bearing paths that break the obvious forms; a documented CSV or JSON hand-off that a consumer in any language can check and refuse; a review of the tool-side Tcl that insists on a manifest, atomic reports and an honest exit status; a collector that reads the manifest before it trusts the directory and gives every corner exactly one status; generated Tcl that delivers every value exactly, as data rather than code; a tool launch judged by four witnesses rather than an exit status that a real simulator set to zero after failing; a tool environment set for one child and recorded in a run file; and a deliberate decision about which language owns each stage.