Perl Part II · Module 24 · Coverage and Regression Reporting
Writing CSV Correctly
Writing CSV is four lines, and the four lines are easy to get wrong in ways that produce a file every tool opens and one tool misreads. This chapter builds the writer, then builds the test that proves it — and the test is the interesting part, because the first version of it passed two of the four deliberately broken writers it was pointed at, and the fourth broken writer loses no data at all.
Intermediate12 min readPerlPart IIRegression Reporting
Part II · Module 24 · Chapter 24.8 · Writing CSV Correctly
1. The Writer, And A Round Trip
The field writer is the registry's own sketch, and it is as short as it looks:
sub csv_field {
my ($v) = @_;
$v = '' unless defined $v;
return $v =~ /[",\n]/ ? '"' . ($v =~ s/"/""/gr) . '"' : $v;
}Three decisions in four lines: an undefined value becomes empty, a field containing a quote, a comma or a newline gets wrapped in quotes, and a quote inside becomes two quotes. The /r modifier (11.14) returns the modified copy without touching $v.
use strict;
use warnings;
# ---- the field writer from the registry's sketch
sub csv_field {
my ($v) = @_;
$v = '' unless defined $v;
return $v =~ /[",\n]/ ? '"' . ($v =~ s/"/""/gr) . '"' : $v;
}
sub csv_row { return join(",", map { csv_field($_) } @_) . "\n" }
# ---- a correct reader, so the writer can be checked by round trip.
# \G with /gc walks the line field by field, which is what makes a TRAILING
# empty field survive -- the first version of this dropped it.
sub csv_parse {
my ($line) = @_;
$line =~ s/\r?\n\z//;
my @f;
pos($line) = 0;
while (1) {
if ($line =~ /\G"((?:[^"]|"")*)"/gc) {
my $v = $1;
$v =~ s/""/"/g;
push @f, $v;
} elsif ($line =~ /\G([^,]*)/gc) {
push @f, $1;
}
last unless $line =~ /\G,/gc;
}
return @f;
}
# ---- 1. the hostile field set. These are TEST VECTORS, not log data:
# a writer is only trustworthy if it survives the values a real message,
# path or tool banner can contain.
my @VECTORS = (
["plain", "arb_smoke"],
["a comma", "granted 2, expected 0"],
["a double quote", 'expected "GNT_VEC"'],
["both", 'got "2,0" not "0,2"'],
["a newline", "line one\nline two"],
["leading space", " indented"],
["trailing space", "trailing "],
["empty string", ""],
["undef", undef],
["a lone quote", '"'],
["only a comma", ","],
["looks like maths", "=1+1"],
["leading zero", "007"],
["a big number", "12345678901234567890"],
["a CR", "has\rcarriage"],
);
# A STRICT reader: RFC 4180 forbids a bare quote inside an unquoted field,
# so a lenient reader hides a writer that forgot to quote one.
sub csv_parse_strict {
my ($line) = @_;
$line =~ s/\r?\n\z//;
my (@f, @err);
pos($line) = 0;
while (1) {
if ($line =~ /\G"((?:[^"]|"")*)"/gc) {
my $v = $1;
$v =~ s/""/"/g;
push @f, $v;
} elsif ($line =~ /\G([^,]*)/gc) {
push @err, "bare quote in an unquoted field" if $1 =~ /"/;
push @f, $1;
}
last unless $line =~ /\G,/gc;
}
return (\@f, \@err);
}
print "==== round trip, one ROW at a time ====\n";
print "(a field on its own cannot show a newline breaking the record, so each\n";
print " value is written into a three-field row and the row is read back)\n";
printf "%-18s %-24s %-6s %-7s %s\n", "CASE", "VALUE", "LINES", "FIELDS", "VERDICT";
my ($ok, $bad, $multi) = (0, 0, 0);
for my $vec (@VECTORS) {
my ($name, $in) = @$vec;
my $row = csv_row("start", $in, "end");
my @rowlines = split /\n/, $row, -1;
pop @rowlines if @rowlines && $rowlines[-1] eq "";
my ($f, $err) = csv_parse_strict($rowlines[0]);
my $want = defined $in ? $in : "";
# three outcomes, not two. A quoted newline makes the RECORD span two
# lines, which is valid CSV that a line-oriented reader cannot consume --
# that is a reader limitation, not the writer losing data.
my ($verdict, @why);
if (@rowlines > 1) {
$verdict = "multi-line record";
$multi++;
} else {
push @why, @$err;
push @why, sprintf("%d fields, expected 3", scalar @$f) if @$f != 3;
push @why, "value changed"
if @$f == 3 && (!defined $f->[1] || $f->[1] ne $want);
$verdict = @why ? "LOST: " . join("; ", @why) : "ok";
@why ? $bad++ : $ok++;
}
my $show = defined $in ? $in : "(undef)";
$show =~ s/\n/\\n/g;
$show =~ s/\r/\\r/g;
printf "%-18s %-24s %-6s %-7s %s\n", $name, $show,
scalar @rowlines, scalar @$f, $verdict;
}
printf "\n%d read back intact, %d LOST by the writer, %d valid multi-line record(s)\n",
$ok, $bad, $multi;
print "a value containing a newline is written correctly and still cannot be\n";
print "read by anything that works a line at a time, so a pipeline with a\n";
print "line-based consumer should replace newlines before writing, not after\n"
if $multi;
# ---- 1b. the test has to be able to fail, so run it against writers that
# are deliberately wrong. A __WARN__ handler counts warnings in-process,
# because a writer can lose nothing and still be broken.
print "\n==== the same test against four broken writers ====\n";
my %WRITERS = (
"correct" => \&csv_field,
"no quote doubling" => sub { my ($v) = @_; $v = '' unless defined $v;
return $v =~ /[",\n]/ ? '"' . $v . '"' : $v },
"quotes only on ','" => sub { my ($v) = @_; $v = '' unless defined $v;
return $v =~ /[,]/
? '"' . ($v =~ s/"/""/gr) . '"' : $v },
"no quoting at all" => sub { my ($v) = @_; $v = '' unless defined $v;
return $v },
"no undef guard" => sub { my ($v) = @_;
return $v =~ /[",\n]/
? '"' . ($v =~ s/"/""/gr) . '"' : $v },
);
printf "%-20s %-7s %-7s %-7s %-10s %s\n",
"WRITER", "INTACT", "LOST", "MULTI", "WARNINGS", "CAUGHT BY";
for my $name ("correct", "no quote doubling", "quotes only on ','",
"no quoting at all", "no undef guard") {
my $w = $WRITERS{$name};
my ($i, $l, $m, $warn) = (0, 0, 0, 0);
local $SIG{__WARN__} = sub { $warn++ };
for my $vec (@VECTORS) {
my (undef, $in) = @$vec;
my $row = join(",", map { $w->($_) } ("start", $in, "end")) . "\n";
my @rl = split /\n/, $row, -1;
pop @rl if @rl && $rl[-1] eq "";
if (@rl > 1) { $m++; next }
my ($f, $err) = csv_parse_strict($rl[0]);
my $want = defined $in ? $in : "";
(@$err || @$f != 3 || !defined $f->[1] || $f->[1] ne $want)
? $l++ : $i++;
}
my @by;
push @by, "lost data" if $l;
push @by, "warnings" if $warn;
printf "%-20s %-7d %-7d %-7d %-10d %s\n", $name, $i, $l, $m, $warn,
$name eq "correct" ? "-- the golden writer" :
(@by ? join(" + ", @by) : "NOTHING -- a survivor");
}
print "\nthree of the four are caught by data loss. The fourth loses nothing at\n";
print "all -- its only signal is the warnings -- so a round-trip check that\n";
print "ignores stderr passes it unchanged.\n";
print "The gate needs BOTH: no lost fields AND no warnings.\n";
# ---- 2. a naive split cannot read a correct file
print "\n==== what a naive split does to a correct row ====\n";
my @fields = ("arb_random", 7, "FAIL", 'got "2,0" not "0,2"', "");
my $row = csv_row(@fields);
printf "fields written : %d\n", scalar @fields;
print "the row : $row";
my @naive = split /,/, $row;
chomp @naive;
printf "split /,/ gives: %d field(s) %s\n", scalar @naive,
join " | ", @naive;
my @proper = csv_parse($row);
printf "a real parser : %d field(s) %s\n", scalar @proper,
join " | ", map { defined $_ ? $_ : "(undef)" } @proper;
printf "so the file is correct and the reader is wrong -- %d against %d fields\n",
scalar @naive, scalar @fields;
# ---- 3. empty against missing
print "\n==== an empty value and a missing one ====\n";
my $r_empty = csv_row("arb_trunc", 7, "UNKNOWN", 8, "");
my $r_undef = csv_row("arb_trunc", 7, "UNKNOWN", 8, undef);
printf "simtime = '' -> %s", $r_empty;
printf "simtime = undef -> %s", $r_undef;
printf "the two rows are %s\n", $r_empty eq $r_undef ? "IDENTICAL" : "different";
print "so CSV cannot carry the difference between 'measured as nothing' and\n";
print "'never measured'. A spreadsheet reads both as blank, and a chart reads\n";
print "blank as zero.\n";
my $r_marked = csv_row("arb_trunc", 7, "UNKNOWN", 8, "NA");
printf "with an explicit marker -> %s", $r_marked;
# ---- 4. the column order has to be fixed, not taken from a hash
print "\n==== the column order ====\n";
my %rec = (test => "arb_light", seed => 7, verdict => "FAIL",
errors => 1, simtime => 325);
# the actual order is masked here, because printing it would make this page
# different on every run -- which is exactly the problem being measured
printf "keys %%rec in this process : %d key(s) in an order chosen per process\n",
scalar keys %rec;
my @COLS = qw(test seed verdict errors simtime);
printf "the declared column order : %s\n", join ",", @COLS;
if (!$ENV{CSV_CHILD}) {
my %orders;
for (1 .. 5) {
local $ENV{CSV_CHILD} = 1;
my $out = `perl "$0" 2>/dev/null`;
$orders{$1}++ if $out =~ /^CHILD KEY ORDER: (.*)$/m;
}
printf "5 separate processes: %s\n", keys(%orders) > 1
? "MORE THAN ONE distinct key order" : "one key order";
print "the declared order : one, in every process, by construction\n";
print "so a CSV header taken from keys %hash can change between two runs\n";
print "of the same script, and every consumer keyed on column position\n";
print "silently reads the wrong column\n";
} else {
printf "CHILD KEY ORDER: %s\n", join ",", keys %rec;
exit 0;
}
# ---- 5. the real file
print "\n==== the regression as CSV ====\n";
my %RANK = (FAIL => 0, TIMEOUT => 1, UNKNOWN => 2, PASS => 3);
sub read_log {
my ($path) = @_;
my %r = (verdict => "UNKNOWN", errors => 0, complete => 0, budget => 0);
open my $fh, "<", $path or return \%r;
while (my $l = <$fh>) {
$r{test} = $1 if $l =~ /^TEST:\s+(\S+)/;
$r{seed} = $1 if $l =~ /^SEED:\s+(\d+)/;
$r{simtime} = $1 if $l =~ /^SIMTIME:\s+(\d+)/;
$r{budget} = 1 if $l =~ /^UVM_FATAL\s.*\[BUDGET\]/;
$r{msg}++ if $l =~ /^UVM_ERROR\s+\S+\(\d+\)/;
$r{complete} = 1 if $l =~ /^\*\* Report counts by severity/;
$r{sum} = $1 if $l =~ /^UVM_ERROR\s*:\s*(\d+)/;
$r{verdict} = "PASS" if $l =~ /^TEST PASSED/;
$r{verdict} = "FAIL" if $l =~ /^TEST FAILED/;
}
close $fh;
$r{errors} = $r{complete} ? $r{sum} : ($r{msg} || 0);
$r{verdict} = "TIMEOUT" if $r{budget} && $r{verdict} eq "UNKNOWN";
return \%r;
}
my @recs = grep { defined $_->{test} } map { read_log($_) } sort glob("logs/*.log");
open my $out, ">", "results.csv" or die $!;
print {$out} csv_row(@COLS);
for my $r (sort { $RANK{$a->{verdict}} <=> $RANK{$b->{verdict}}
or $a->{test} cmp $b->{test}
or $a->{seed} <=> $b->{seed} } @recs) {
print {$out} csv_row(map { defined $r->{$_} ? $r->{$_} : "NA" } @COLS);
}
close $out;
printf "rows written: %d data + 1 header\n", scalar @recs;
open my $in, "<", "results.csv" or die $!;
my @all = <$in>; close $in;
print "the first four lines:\n";
print " $_" for @all[0 .. 3];
# every row must parse back to the same number of fields as the header
my $want = scalar @COLS;
my @wrongwidth = grep { my @f = csv_parse($all[$_]); @f != $want } 0 .. $#all;
printf "rows whose field count differs from the header: %d of %d\n",
scalar @wrongwidth, scalar @all;==== round trip, one ROW at a time ====
(a field on its own cannot show a newline breaking the record, so each
value is written into a three-field row and the row is read back)
CASE VALUE LINES FIELDS VERDICT
plain arb_smoke 1 3 ok
a comma granted 2, expected 0 1 3 ok
a double quote expected "GNT_VEC" 1 3 ok
both got "2,0" not "0,2" 1 3 ok
a newline line one\nline two 2 2 multi-line record
leading space indented 1 3 ok
trailing space trailing 1 3 ok
empty string 1 3 ok
undef (undef) 1 3 ok
a lone quote " 1 3 ok
only a comma , 1 3 ok
looks like maths =1+1 1 3 ok
leading zero 007 1 3 ok
a big number 12345678901234567890 1 3 ok
a CR has\rcarriage 1 3 ok
14 read back intact, 0 LOST by the writer, 1 valid multi-line record(s)
a value containing a newline is written correctly and still cannot be
read by anything that works a line at a time, so a pipeline with a
line-based consumer should replace newlines before writing, not after
==== the same test against four broken writers ====
WRITER INTACT LOST MULTI WARNINGS CAUGHT BY
correct 14 0 1 0 -- the golden writer
no quote doubling 11 3 1 0 lost data
quotes only on ',' 12 2 1 0 lost data
no quoting at all 9 5 1 0 lost data
no undef guard 14 0 1 2 warnings
three of the four are caught by data loss. The fourth loses nothing at
all -- its only signal is the warnings -- so a round-trip check that
ignores stderr passes it unchanged.
The gate needs BOTH: no lost fields AND no warnings.
==== what a naive split does to a correct row ====
fields written : 5
the row : arb_random,7,FAIL,"got ""2,0"" not ""0,2""",
split /,/ gives: 7 field(s) arb_random | 7 | FAIL | "got ""2 | 0"" not ""0 | 2""" |
a real parser : 5 field(s) arb_random | 7 | FAIL | got "2,0" not "0,2" |
so the file is correct and the reader is wrong -- 7 against 5 fields
==== an empty value and a missing one ====
simtime = '' -> arb_trunc,7,UNKNOWN,8,
simtime = undef -> arb_trunc,7,UNKNOWN,8,
the two rows are IDENTICAL
so CSV cannot carry the difference between 'measured as nothing' and
'never measured'. A spreadsheet reads both as blank, and a chart reads
blank as zero.
with an explicit marker -> arb_trunc,7,UNKNOWN,8,NA
==== the column order ====
keys %rec in this process : 5 key(s) in an order chosen per process
the declared column order : test,seed,verdict,errors,simtime
5 separate processes: MORE THAN ONE distinct key order
the declared order : one, in every process, by construction
so a CSV header taken from keys %hash can change between two runs
of the same script, and every consumer keyed on column position
silently reads the wrong column
==== the regression as CSV ====
rows written: 36 data + 1 header
the first four lines:
test,seed,verdict,errors,simtime
arb_light,7,FAIL,1,325
arb_random,1,FAIL,55,4025
arb_random,7,FAIL,62,4025
rows whose field count differs from the header: 0 of 37The test writes each value into a three-field row rather than testing the field alone, and that was not the first design. The first version round-tripped each field on its own — which cannot see a newline splitting the record in two, because there is no record. Section 2 shows what that cost.
One result in the table is not a failure:
a newline line one\nline two 2 2 multi-line recordThe writer is correct here. A quoted newline makes the CSV record span two lines, which RFC 4180 permits and every spreadsheet handles. What cannot handle it is anything reading a line at a time — which is most of what consumes a regression CSV. So if the consumer is line-based, replace newlines before writing, and say so in the header or the column name. Fixing it after the fact is impossible: by then the record boundary is gone.
2. The Test Against Four Broken Writers
A round-trip test that only ever sees a correct writer proves nothing, so the script points the same vectors at four writers that are deliberately wrong:
WRITER INTACT LOST MULTI WARNINGS CAUGHT BY
correct 14 0 1 0 -- the golden writer
no quote doubling 11 3 1 0 lost data
quotes only on ',' 12 2 1 0 lost data
no quoting at all 9 5 1 0 lost data
no undef guard 14 0 1 2 warningsTwo things to take from this table.
Three writers are caught by losing data — three, two and five of the fifteen values respectively. That is what a round-trip test is for.
The fourth loses nothing. Dropping the $v = '' unless defined $v guard changes no output byte; it just makes Perl warn twice while producing it. A round-trip check that only compares values passes it, and the warnings go to stderr where a nightly cron job sends them nowhere.
So the gate is two conditions:
no lost fields AND no warningsAnd it is worth knowing how the warnings were counted, because 2>/dev/null is the usual habit:
local $SIG{__WARN__} = sub { $warn++ };A __WARN__ handler counts them in-process (14.2), which means the check works the same way inside a test harness as it does at a terminal.
There is also a reader-side lesson buried in the earlier version of this test. It used a lenient parser, which happily read a bare quote inside an unquoted field — so it reported success for the writer that never quotes anything. RFC 4180 forbids that, and once the reader enforced it, two more mutants fell. A lenient reader hides a broken writer.
3. A Correct File Defeats A Naive Split
fields written : 5
the row : arb_random,7,FAIL,"got ""2,0"" not ""0,2""",
split /,/ gives: 7 field(s) arb_random | 7 | FAIL | "got ""2 | 0"" not ""0 | 2""" |
a real parser : 5 field(s) arb_random | 7 | FAIL | got "2,0" not "0,2" |Seven fields from a five-field row. The file is correct; split /,/ is wrong. This is the harder direction to debug, because the producer is innocent and the symptom appears in somebody else's script — usually as a column shifted by two on a handful of rows.
A real parser walks the line field by field with \G and /gc (11.20):
if ($line =~ /\G"((?:[^"]|"")*)"/gc) { ... }
elsif ($line =~ /\G([^,]*)/gc) { ... }
last unless $line =~ /\G,/gc;That final last unless is what preserves a trailing empty field. My first version of this parser lost it — it returned four fields for the five-field row above — and the symptom was a silently short row rather than an error. For production, Text::CSV exists and is better than either version; the point of writing one here is that the round-trip test needs a reader it can trust, and a reader from CPAN cannot be mutated to prove the test discriminates.
4. Empty Against Missing
simtime = '' -> arb_trunc,7,UNKNOWN,8,
simtime = undef -> arb_trunc,7,UNKNOWN,8,
the two rows are IDENTICAL
with an explicit marker -> arb_trunc,7,UNKNOWN,8,NACSV has one representation for both. A field that was measured as nothing and a field that was never measured come out as the same bytes, and the distinction matters in exactly the case this corpus contains: arb_trunc never reached the line that reports its simulated time, so it has no measurement — not a measurement of zero.
What happens downstream makes it worse. A spreadsheet shows a blank; a chart reads blank as zero; an average over the column quietly includes a run that produced no number. Chapter 24.3 found the same thing in coverage, where "never measured" must not merge in as zeros.
So write a marker — NA, or -, or anything the consumer is told about — and keep it out of the numeric columns' arithmetic. The real file in the script does exactly that:
csv_row(map { defined $r->{$_} ? $r->{$_} : "NA" } @COLS);5. The Column Order
keys %rec in this process : 5 key(s) in an order chosen per process
the declared column order : test,seed,verdict,errors,simtime
5 separate processes: MORE THAN ONE distinct key order
the declared order : one, in every process, by constructionA header built from keys %hash can change between two runs of the same script (6.6). The file stays self-consistent — the header matches its rows — so nothing looks broken. Every consumer that reads by column position rather than by name then reads the wrong column, and a consumer that reads by name works fine, which is how this survives for months before anyone notices.
Declare the columns once, as a list, and use that list for both the header and every row:
my @COLS = qw(test seed verdict errors simtime);The declared order is also the only thing that makes the file diffable night over night, which Chapter 24.11 depends on.
6. The Real File
rows written: 36 data + 1 header
test,seed,verdict,errors,simtime
arb_light,7,FAIL,1,325
rows whose field count differs from the header: 0 of 37That last line is the cheapest possible self-check and worth keeping in the writer: every row must parse back to the same number of fields as the header. It costs one pass, it catches an unquoted comma immediately, and it is the check that would have caught every one of the three data-losing writers in section 2.
7. What To Remember
csv_field: undef becomes empty, quote a field containing",,or newline, and double an inner quote./rreturns the modified copy (11.14).- Round-trip a ROW, not a field. A field on its own cannot show a newline breaking the record — the field-level version passed two of four broken writers.
- Measured: 14 of 15 vectors intact, 0 lost, 1 valid multi-line record.
- A quoted newline is correct CSV that a line-oriented reader cannot consume — replace newlines before writing if the consumer reads lines.
- Measured: three broken writers lost 3, 2 and 5 fields; the fourth lost nothing and warned twice.
- The gate is two conditions: no lost fields AND no warnings.
- Count warnings with
local $SIG{__WARN__} = sub { $warn++ }(14.2) —2>/dev/nullis how the fourth writer survives. - A lenient reader hides a broken writer — enforce RFC 4180's ban on a bare quote in an unquoted field.
- Measured:
split /,/returned 7 fields from a correct 5-field row. The file is right and the reader is wrong. - Walk the line with
\Gand/gc(11.20), and keep thelast unless /\G,/gcthat preserves a trailing empty field. - Use
Text::CSVin production; write one here only so the test has a reader it can mutate. - Measured:
''andundefproduce identical rows. CSV cannot carry "empty" against "missing". - Write an explicit marker (
NA) and keep it out of the numeric arithmetic (24.3). - Measured:
keys %hashgave more than one column order across 5 processes (6.6) — declare@COLSand use it for the header and every row. - Self-check the file: every row must parse back to the header's field count. One pass, and it catches every data-losing writer above.
Chapter 24.9 assembles the table a nightly regression actually publishes. The columns are decided, the verdicts are decided, and what remains is the part that makes a table worth reading at all: a row order that does not change when nothing changed.
Where this fits
Part of the Perl curriculum.