Reading and Writing Delimited Text

Part I · Module 13 · Chapter 13.14 · Reading and Writing Delimited Text

1. Read a Row, Name the Fields

use strict;
use warnings;
use File::Temp qw(tempdir);

my $dir = tempdir(CLEANUP => 1);
open my $w, ">", "$dir/t.csv" or die $!;
print $w "pin,check,slack,unit\n";
print $w "u_alu/A,setup,-0.12,ns\n";
print $w "u_fifo/B,hold,0.44,ns\n";
close $w;

open my $fh, "<", "$dir/t.csv" or die $!;
chomp(my $header = <$fh>);
my @cols = split /,/, $header;
printf "columns : %s\n", join(" | ", @cols);

while (my $line = <$fh>) {
    chomp $line;
    my @f = split /,/, $line;
    # Build a hash so the code reads by NAME, not by position.
    my %row; @row{@cols} = @f;
    printf "  %-10s %-6s %6s %s\n", $row{pin}, $row{check}, $row{slack}, $row{unit};
}
close $fh;

columns : pin | check | slack | unit
  u_alu/A    setup   -0.12 ns
  u_fifo/B   hold     0.44 ns

The important line is the short one:

my %row; @row{@cols} = @f;

That is a hash slice from Module 6, and it is the single most useful idiom for delimited data. It takes the column names from the header and the values from the row and pairs them up, so the rest of your code says $row{slack} instead of $f[2].

The difference matters when the file changes. A tool that adds a column in the middle breaks every $f[2] in your script, silently, because index 2 is now a different field. With named access the same change breaks nothing — and if a column you need is removed, $row{slack} is undef rather than quietly the wrong value.

Reading the header separately, before the loop, is what makes this work. Note the chomp on it too: without that, the last column name would be unit\n and $row{unit} would never match.

2. Two Things split Does That You Need To Know

use strict;
use warnings;

# split DROPS trailing empty fields unless you ask it not to.
my $row = "u_alu/A,setup,,";

my @default = split /,/, $row;
my @kept    = split /,/, $row, -1;

printf "row              : [%s]\n", $row;
printf "split /,/        : %d field(s)\n", scalar @default;
printf "split /,/ , -1   : %d field(s)\n", scalar @kept;
print  "\nwithout -1 you lose empty columns at the END of the row\n";

# A LIMIT also keeps a delimiter inside the last field.
my $note = "u_alu/A,note,contains,commas";
my @three = split /,/, $note, 3;
printf "\nsplit with limit 3 : %s\n", join(" | ", @three);

row              : [u_alu/A,setup,,]
split /,/        : 2 field(s)
split /,/ , -1   : 4 field(s)

without -1 you lose empty columns at the END of the row

split with limit 3 : u_alu/A | note | contains,commas

A row with four columns, two of them empty at the end. split returned 2 fields.

The second half of the output shows the same third argument used positively. split /,/, $note, 3 stops after three fields and leaves the rest of the line — delimiters included — in the last one. That is how you handle a format where the final field is free text that may contain the delimiter: split /,/, $line, 4 for a four-column format whose last column is a comment.

3. Writing Rows Out

use strict;
use warnings;
use File::Temp qw(tempdir);

my $dir = tempdir(CLEANUP => 1);
my @rows = (
    { pin => "u_alu/A",  check => "setup", slack => -0.12 },
    { pin => "u_fifo/B", check => "hold",  slack =>  0.44 },
);
my @cols = qw(pin check slack);

open my $w, ">", "$dir/out.csv" or die $!;
print $w join(",", @cols), "\n";
for my $r (@rows) {
    print $w join(",", map { $r->{$_} } @cols), "\n";
}
close $w or die $!;

open my $fh, "<", "$dir/out.csv" or die $!;
print while <$fh>;
close $fh;

pin,check,slack
u_alu/A,setup,-0.12
u_fifo/B,hold,0.44

Writing is the mirror of reading, and the shape is worth copying.

One list of column names, used twice. @cols writes the header and drives the map for every row, so the header can never disagree with the data. Writing the header as a literal string and the rows from a hash is how a file ends up with columns in the wrong order.

join(",", map { $r->{$_} } @cols) builds each row in the column order, by name. Add a column by adding it to @cols in one place.

Two cautions when you write:

• undef becomes an empty string with a warning. If a field may be missing, write $r->{$_} // "" to say so deliberately.
• A field containing your delimiter will corrupt the file. If an instance name or comment can contain a comma, either choose a delimiter it cannot contain — a tab is usually safe for this industry's data — or quote properly, which is the next section.

Tab-separated output is worth preferring where you can. join("\t", ...) is read correctly by every spreadsheet and by cut, awk and sort, and tool-generated names very rarely contain tabs.

4. Where a Plain split Stops Working

use strict;
use warnings;

# A field that legitimately contains the delimiter, in quotes.
my $row = 'u_alu/A,"setup, hold",-0.12';

my @naive = split /,/, $row;
printf "plain split        : %d field(s)  %s\n", scalar @naive, join(" | ", @naive);

my @aware = $row =~ /("[^"]*"|[^,]+)/g;
printf "quote-aware regex  : %d field(s)  %s\n", scalar @aware, join(" | ", @aware);

# ...and the same regex on a DOUBLED quote, the standard CSV escape.
my $hard = 'a,"b "" c",d';
my @hard = $hard =~ /("[^"]*"|[^,]+)/g;
printf "\non a doubled quote : %d field(s)  %s\n", scalar @hard, join(" | ", @hard);
printf "correct answer     : 3 fields, the middle one being: b \" c\n";

printf "\nText::CSV installed here : %s\n",
    (eval { require Text::CSV; 1 } ? "yes" : "no");

plain split        : 4 field(s)  u_alu/A | "setup |  hold" | -0.12
quote-aware regex  : 3 field(s)  u_alu/A | "setup, hold" | -0.12

on a doubled quote : 4 field(s)  a | "b " | " c" | d
correct answer     : 3 fields, the middle one being: b " c

Text::CSV installed here : no

Three results, and they form a ladder.

A plain split gave 4 fields where there are 3, because it cut the quoted field at its internal comma.

A quote-aware pattern gave the correct 3. It prefers a quoted run over a bare run — Chapter 11.9's alternation — and it fixes this row.

The same pattern gave 4 on the next row, which uses the standard CSV convention of doubling a quote to mean a literal one. You can patch the pattern for that. Then a field will contain a newline, and you will patch it again.

That is the signal, and it is worth stating as a rule: if you have patched the same parsing pattern twice, stop patching. The input has structure your pattern is not modelling, and the third case is already on its way. Chapter 11.22 made the same argument from the regex side; this is its file-handling home.

The proper answer is a real parser — Text::CSV, which handles quoting, doubled quotes, embedded newlines and the rest, and has been handling them for twenty years.

5. What To Remember

• split to read a row, join to write one.
• Read the header separately and use a hash slice: my %row; @row{@cols} = @f;. Then your code says $row{slack}, not $f[2], and a new column in the middle breaks nothing.
• chomp the header too, or the last column name carries a newline.
• split drops trailing empty fields. Measured: a 4-column row with two blank columns at the end gave 2 fields.
• Use split /,/, $line, -1 for delimited data, always.
• A positive limit keeps delimiters in the final field — split /,/, $line, 4 for a format whose last column is free text.
• Write from one list of column names, used for both the header and the rows, so they cannot disagree.
• Write $r->{$_} // "" for a field that may be missing.
• A plain split breaks on a quoted delimiter. Measured: 4 fields where 3 are right.
• Two patches to the same pattern means stop patching and use a real parser.
• Text::CSV is the parser, and it is not core — check with perl -MText::CSV -e 1.
• If you control the format, use tabs. split /\t/, $line, -1 and the problem never arises.

That completes Module 13. Fourteen chapters: opening and closing files safely, the read loop and why it is written as it is, the choice between one line and the whole file, writing and appending, checking for failure, asking about a path, the diamond operator, line numbers, directories, recursive traversal, path handling, encoding and binary data, temporary files and atomic writes, and delimited text. Module 14 turns to what happens when something goes wrong: errors, warnings, and how to find out why a script did not do what you expected.