Reading and Rewriting Tool Configuration Files

Part II · Module 17 · Chapter 17.7 · Reading and Rewriting Tool Configuration Files

1. Why Parse-and-Rewrite Destroys the File

The obvious design is to read the file into a hash, change the key, and write the hash back out. Here is what that costs.

use strict;
use warnings;
use File::Temp qw(tempdir);
use File::Spec;

my $dir = tempdir(CLEANUP => 1);
my $orig = File::Spec->catfile($dir, "setup.tcl");

# ---- a SAMPLE setup file, in the shape a real one has
open my $w, ">", $orig or die "cannot write: $!\n";
print $w <<'SETUP';
# .synopsys_dc.setup -- project defaults
# DO NOT reorder: the library search path must precede link_library
set search_path   [list . /proj/chipA/libs/std /proj/chipA/libs/io]
set target_library "tcbn05_ss.db"          ;# slow corner for signoff

# link_library must include * so DC finds already-loaded cells
set link_library  "* tcbn05_ss.db io_ss.db"

set hdlin_enable_hier_map true             ;# added for the fifo bug
# set hdlin_check_no_latch true            ;# disabled 2019, too noisy
SETUP
close $w;

my $want = "tcbn05_ff.db";                   # the one change we came to make

# ---- A: parse into a hash, write the hash back
open my $r, "<", $orig or die "cannot read: $!\n";
my %set;
while (my $l = <$r>) {
    $set{$1} = $2 if $l =~ /^\s*set\s+(\w+)\s+(.+?)\s*(?:;#.*)?$/;
}
close $r;
$set{target_library} = "\"$want\"";
my $naive = File::Spec->catfile($dir, "naive.tcl");
open my $nw, ">", $naive or die "cannot write: $!\n";
print $nw "set $_ $set{$_}\n" for sort keys %set;
close $nw;

# ---- B: line-oriented. Touch the matching line, copy every other byte.
open my $r2, "<", $orig or die "cannot read: $!\n";
my (@out, $changed);
while (my $l = <$r2>) {
    if ($l =~ /^(\s*set\s+target_library\s+)"[^"]*"(.*)$/) {
        push @out, "$1\"$want\"$2\n";
        $changed++;
    }
    else { push @out, $l }
}
close $r2;
my $lineop = File::Spec->catfile($dir, "lineop.tcl");
open my $lw, ">", $lineop or die "cannot write: $!\n";
print $lw @out;
close $lw;

# ---- compare what survived
sub survey {
    my ($f) = @_;
    open my $h, "<", $f or die "cannot read: $!\n";
    my @l = <$h>;
    close $h;
    return (scalar @l,
            scalar(grep { /^\s*#/ } @l),
            scalar(grep { /;#/ } @l));
}
printf "%-12s %6s %9s %12s\n", "FILE", "lines", "comments", "trailing ;#";
for my $f ($orig, $naive, $lineop) {
    printf "%-12s %6d %9d %12d\n", (File::Spec->splitpath($f))[2], survey($f);
}
printf "\nthe line-oriented edit changed %d line(s)\n\n", $changed;

print "what the naive rewrite produced, in full:\n";
open my $n, "<", $naive or die "cannot read: $!\n";
print "    $_" while defined($_ = <$n>);
close $n;

print "\nand the one line the line-oriented edit changed:\n";
open my $lo, "<", $lineop or die "cannot read: $!\n";
while (my $l = <$lo>) { print "    $l" if $l =~ /target_library/ }
close $lo;

FILE          lines  comments  trailing ;#
setup.tcl        10         4            3
naive.tcl         4         0            0
lineop.tcl       10         4            3

the line-oriented edit changed 1 line(s)

what the naive rewrite produced, in full:
    set hdlin_enable_hier_map true
    set link_library "* tcbn05_ss.db io_ss.db"
    set search_path [list . /proj/chipA/libs/std /proj/chipA/libs/io]
    set target_library "tcbn05_ff.db"

and the one line the line-oriented edit changed:
    set target_library "tcbn05_ff.db"          ;# slow corner for signoff

Ten lines and four comments became four lines and zero comments. The naive rewrite did make the change it was asked to make, and in doing so it destroyed four separate things:

All the comment lines, including DO NOT reorder: the library search path must precede link_library.

All three trailing ;# annotations, including the note recording which bug a setting was added for.

The commented-out setting. # set hdlin_check_no_latch true is not a setting — it is the record that somebody turned it off deliberately in 2019 and why. The parser skipped it, so the rewrite dropped it, so the next person to wonder about latches has no idea it was ever considered.

And then it reordered the file, alphabetically, moving link_library ahead of search_path — which is precisely what the comment it had just deleted was warning against.

The line-oriented version changed one line and left the other nine byte-for-byte identical.

And look closely at that one line in the output. The value is now tcbn05_ff.db — the fast library — and the trailing comment still says slow corner for signoff. The edit preserved a comment that its own change made wrong.

That is not a defect in the approach; it is the honest limit of it. A line-oriented edit is faithful, which means it is faithful to comments you have invalidated as well as to ones you have not. A script that changes a value whose comment describes it should say so:

if ($matched_line =~ /;#/) {
    print STDERR "note: the comment on this line may now be stale:\n  $matched_line";
}

One line on standard error, per Chapter 15.12, and the person running it can decide. Silently leaving a wrong comment is how a setup file ends up with annotations that actively mislead the next reader — which is worse than having none.

2. The Line-Oriented Edit

The pattern is three lines of logic:

while (my $line = <$in>) {
    if ($line =~ /^(\s*set\s+target_library\s+)"[^"]*"(.*)$/) {
        push @out, "$1\"$want\"$2\n";      # rebuild from the captures
    }
    else { push @out, $line }              # byte-for-byte
}

Four details make it safe.

Capture the parts you are keeping. $1 is the leading whitespace and the set target_library text; $2 is everything after the closing quote, which is where the trailing ;# comment lives. Rebuilding from the captures preserves the original spacing and the comment without you having to think about either.

Anchor the pattern. ^ means a line mentioning target_library inside a comment is not matched. Chapter 11.8's anchors, and here the consequence is not matching # target_library was changed by....

Everything unmatched is copied unchanged, including blank lines, indentation and the lines you do not understand. That last category is the important one: a real setup file contains settings whose purpose nobody currently knows, and the correct action is to leave them exactly as they are.

Count the changes and check the count. This is the step people skip:

if ($changed == 0) {
    print STDERR "error: no 'set target_library' line found in $file\n";
    exit 3;                               # 15.4: could not reach a verdict
}
if ($changed > 1) {
    print STDERR "error: $changed 'set target_library' lines; refusing to guess\n";
    exit 3;
}

Zero matches means the file is not what you expected and writing it back would be a silent no-op. More than one means the file sets the value twice — which happens, and which of them wins is a question about the tool, not about your script. Both deserve a refusal rather than a guess.

3. Reading a Value Without Rewriting

Most scripts only need to read a setting, and that is simpler — but it has its own trap.

sub read_setting {
    my ($file, $name) = @_;
    open my $fh, "<", $file or die "cannot read $file: $!\n";
    my $value;
    while (my $l = <$fh>) {
        next if $l =~ /^\s*#/;                              # skip comments
        next unless $l =~ /^\s*set\s+\Q$name\E\s+(.+?)\s*(?:;#.*)?$/;
        $value = $1;                                        # LAST wins
    }
    close $fh;
    return $value;
}

Three things in there are deliberate:

next if /^\s*#/ first. Without it, the commented-out # set hdlin_check_no_latch true is read as a live setting. That is the same information the rewrite lost, now being misread rather than deleted.

\Q$name\E so a setting name is matched literally. Without it a name containing a . or + becomes a pattern.

The last match wins, not the first. A Tcl setup file is a script: if it sets a value twice, the second assignment is the one in effect. Returning the first match gives you the value the tool is not using. This is the single most common bug in config readers.

And one honest limitation: that function reads a literal. It does not evaluate [list ...], does not expand $env(PROJ_ROOT), and does not know about if blocks. For the common case of a quoted string or a bare word, it is right; beyond that, see section 4.

4. When the File Is Not Line-Oriented

Everything above assumes one setting per line. Three cases break that, and the honest answer differs for each.

A continued line. Tcl continues with a trailing backslash:

set search_path [list . \
                 /proj/chipA/libs/std \
                 /proj/chipA/libs/io]

A line loop sees three lines. The fix is a small join before matching — accumulate while the line ends in a backslash — and it is worth doing because continued lines are common in real setup files.

A value computed by the tool's own language. set target_library [get_lib_name $corner]. There is no string to substitute, and a script that rewrites it is changing a program. The correct response is to refuse: match only a literal value, and report that the line is computed. Chapter 15.4's "could not reach a verdict" exit code exists for exactly this.

A structured format — YAML, JSON, XML. Here the parse-and-rewrite approach is correct, because the format has a defined model and comments are either preserved by the library or not part of the format. Use a real parser. For JSON that is JSON::PP, which is core — Chapter 12.9's list. For YAML it is not core, which puts you back in Chapter 12.10's dependency problem on a locked-down machine.

5. What To Remember

• A setup file is a record of decisions, most of them in the comments beside the settings.
• Measured: parse-and-rewrite turned 10 lines and 4 comments into 4 lines and 0 comments.
• It destroyed a DO NOT reorder comment and then reordered the file — alphabetically, moving link_library ahead of search_path.
• It dropped the commented-out setting, losing the record that it was disabled deliberately.
• The line-oriented edit changed 1 line and left the other nine byte-for-byte identical.
• Capture the parts you keep — leading text in $1, everything after the value in $2, so spacing and trailing ;# survive.
• Anchor the pattern with ^, or a comment mentioning the setting matches.
• Copy every unmatched line unchanged, including the ones you do not understand.
• A faithful edit preserves comments your change invalidated. Measured: the value became tcbn05_ff.db while the comment still read slow corner for signoff. Warn on stderr when the edited line carries a ;#.
• Count the changes: zero means the file is not what you expected; more than one means refusing to guess. Exit 3 for both.
• Write to a temporary file and rename. Opening with > truncates immediately, and a truncated setup file is a broken project.
• Put the temporary beside the target — rename is only atomic within one filesystem — and include $$ in its name.
• When reading, skip comment lines first, or a disabled setting reads as live.
• Use \Q$name\E so a setting name is matched literally.
• The LAST match wins, because the file is a script — returning the first gives the value the tool is not using.
• A continued line needs joining before matching; backslash continuation is common.
• A computed value cannot be rewritten — match only literals and refuse the rest.
• For a structured format, use a real parser — JSON::PP is core; YAML is not.
• Tcl and shell: edit line by line. JSON: parse and regenerate. For your own config, choose JSON.

Chapter 17.8 is rated Critical and is about something that sounds trivial. Sorting output feels like a formatting preference until you measure what two unsorted runs do to a diff — and the answer is that the diff stops carrying any information at all.