Splitting and Joining

Part I · Module 4 · Chapter 4.7 · Splitting and Joining

1. The Ordinary Case

use strict;
use warnings;

my $line = "alu,smoke,PASS,184";

my @fields = split /,/, $line;

print scalar(@fields), " fields\n";
print join(" | ", @fields), "\n";

4 fields
alu | smoke | PASS | 184

split takes a separator and a string, and returns the pieces between the separators. join takes a separator and a list, and returns one string with that separator between the pieces.

They are opposites, and on tidy data the round trip is exact:

use strict;
use warnings;

my $line = "alu,smoke,PASS,184";

my @fields = split /,/, $line;
my $back   = join ",", @fields;

print "original : $line\n";
print "rebuilt  : $back\n";
print "identical: ", ($back eq $line ? "yes" : "no"), "\n";

original : alu,smoke,PASS,184
rebuilt  : alu,smoke,PASS,184
identical: yes

That is the shape of nearly every parsing script: split the line, work with the fields, join them back. The rest of this chapter is about the cases where the round trip is not exact.

2. Trailing Empty Fields Are Discarded

Here is the behaviour to know.

use strict;
use warnings;

my $line = "a,b,,";

my @default = split /,/, $line;
my @kept    = split /,/, $line, -1;

printf "no limit : %d fields -> [%s]\n", scalar(@default), join("][", @default);
printf "limit -1 : %d fields -> [%s]\n", scalar(@kept),    join("][", @kept);

no limit : 2 fields -> [a][b]
limit -1 : 4 fields -> [a][b][][]

The line "a,b,," has four fields: a, b, an empty one, and another empty one. The default split returns two.

The two empty fields at the end were discarded. Passing -1 as a third argument — the limit — keeps them.

Which breaks the round trip

use strict;
use warnings;

my $line = "a,b,,";

my @fields = split /,/, $line;
my $back   = join ",", @fields;

print "original : [$line]\n";
print "rebuilt  : [$back]\n";
print "identical: ", ($back eq $line ? "yes" : "no"), "\n";

original : [a,b,,]
rebuilt  : [a,b]
identical: no

The rebuilt line is two characters shorter. Without the brackets around each value — and without the identical line — that difference is easy to miss entirely, which is why both are in the script.

In a real script neither is printed, and the shortened line simply goes into the output file.

3. A Leading Empty Field Is Kept

The behaviour is not symmetric, and knowing that stops the other case looking like a bug:

use strict;
use warnings;

my @fields = split /,/, ",a,b";

printf "%d fields -> [%s]\n", scalar(@fields), join("][", @fields);

3 fields -> [][a][b]

Three fields, the first one empty. So the rule is specifically about the end of the string: leading and interior empty fields are preserved, trailing ones are dropped unless you ask for them.

That asymmetry is deliberate rather than accidental — it makes the common case of a line with a trailing separator behave tidily — but it is still a surprise the first time a column count comes up short.

4. The Round Trip, Drawn

5. split " " Is Not split / /

The second trap, and the two forms are a single character apart in source.

use strict;
use warnings;

my $line = "  alu   smoke  PASS ";

my @magic   = split " ",  $line;
my @literal = split / /,  $line;

printf "split \" \" : %d fields -> [%s]\n", scalar(@magic),   join("][", @magic);
printf "split / /  : %d fields -> [%s]\n", scalar(@literal), join("][", @literal);

split " " : 3 fields -> [alu][smoke][PASS]
split / /  : 8 fields -> [][][alu][][][smoke][][PASS]

Three fields against eight, from the same line.

• split " " — the separator written as a quoted string containing one space — is a documented special case. It means "split on runs of whitespace, and ignore any whitespace at the start of the line". It is the right tool for whitespace-separated text with irregular spacing, which is most tool output.
• split / / — the separator written as a pattern matching one space — means exactly that. Three consecutive spaces are three separators with empty fields between them, and leading spaces produce leading empty fields.

So for a line of whitespace-aligned columns, split " " gives you the fields and split / / gives you the fields plus a lot of empty strings. The difference is invisible at a glance and it is why the quoted form exists.

Note that split /\s+/ behaves like split / / regarding the leading whitespace — it collapses runs but still gives you an empty first field when the line starts with a space. Only the literal " " form discards the leading whitespace. That is the detail that makes " " worth remembering as its own thing rather than as shorthand for a pattern.

6. The Limit Argument Has a Second Use

A positive limit stops splitting after that many fields and leaves the remainder intact:

use strict;
use warnings;

my $line = "alu,smoke,PASS,184";

my @two = split /,/, $line, 2;

printf "%d fields -> [%s]\n", scalar(@two), join("][", @two);

2 fields -> [alu][smoke,PASS,184]

That is genuinely useful: a log line of the form key: everything else, including: colons splits correctly with a limit of 2, where an unlimited split would break the value apart at every colon.

To summarise the three forms of the third argument:

7. Two More Forms Worth Knowing

An empty pattern splits into characters:

use strict;
use warnings;

print join("|", split //, "alu"), "\n";

a|l|u

join takes any separator, including an empty one or a multi-character one:

use strict;
use warnings;

my @parts = ("alu", "smoke", "01");

print join("_",    @parts), "\n";
print join("",     @parts), "\n";
print join(" -> ", @parts), "\n";

alu_smoke_01
alusmoke01
alu -> smoke -> 01

join is also the readable way to build a separator-delimited line, and it is better than concatenating in a loop because there is no trailing-separator problem to handle.

8. A Worked Example

Reading a report row, changing one field, writing it back — done faithfully:

use strict;
use warnings;

my $row = "alu_smoke_01,PASS,184,";

my @fields = split /,/, $row, -1;

printf "read %d fields\n", scalar(@fields);

$fields[1] = "FAIL";

my $out = join ",", @fields;

print "in  : [$row]\n";
print "out : [$out]\n";
printf "same field count: %s\n", (scalar(@fields) == 4 ? "yes" : "no");

read 4 fields
in  : [alu_smoke_01,PASS,184,]
out : [alu_smoke_01,FAIL,184,]
same field count: yes

Four fields read, four written, and the trailing empty column survived. The -1 is the only thing making that true — drop it and the same script reads three fields and writes alu_smoke_01,FAIL,184, with the final comma gone.

Indexing into the array with $fields[1] is Module 5's syntax; it appears here because the read-modify-write shape is the whole point of the chapter and cannot be shown without it.

9. Why This Matters in Engineering Work

Parsing a delimited report is the single most common scripting task in this trade. A regression summary, a timing report exported as CSV, a coverage table — all of them are split on input.

Read-modify-write is where the default bites. Any script that rewrites a file it read is at risk, and the risk is concentrated on exactly the rows where a value was missing. A timing report whose slack column is empty for unconstrained paths will lose that column from precisely those rows.

Irregular whitespace is the normal case in tool output. Columns aligned by padding, not by a delimiter. split " " handles it; split / / produces a list full of empty strings that then fail to compare against anything. This is a frequent cause of "field 3 is empty and I don't know why".

Separators that are metacharacters. Splitting a version on . or a path on | without escaping gives nonsense, because the pattern argument is a regex. The callout in §1 is the whole warning.

Two habits cover all of it:

• Split with -1 whenever the data will be written back.
• Count the fields and check the count before you index into them. A row with fewer fields than expected is a data problem you want to report, not one you want to discover through an undefined value three functions later.

10. Common Mistakes

Letting the default drop trailing empty fields. Use -1 for any round trip. §2 measures the loss.

Assuming empty fields are dropped generally. Only trailing ones are; leading and interior ones are kept.

Writing split / / for whitespace-separated columns. Use split " ". §5 is three fields against eight.

Expecting split /\s+/ to discard leading whitespace. Only the literal " " form does.

Forgetting the pattern is a regex. split /./ splits on every character. Escape the separator or use \Q.

Indexing a field without checking the count. A short row gives undef, which then interpolates as nothing — Chapter 4.2 §6.

Building a delimited line by concatenating in a loop. join has no trailing-separator problem.

11. Exercise

Exercise 1 — the basic round trip. Split "alu,smoke,PASS,184" on commas, report the field count, join it back and confirm the result is identical to the original.

Exercise 2 — lose the data, then keep it. Run the §2 script. Record both field counts. Then rebuild each list with join and report which one reproduces "a,b,," exactly.

Exercise 3 — the asymmetry. Split ",a,b," with no limit and then with -1. Report both counts and say exactly which empty fields survived in each, then state the rule in one sentence.

Exercise 4 — three against eight. Run the §5 script. Then write one sentence explaining why split / / produced two empty fields at the start, and say which form you would use for a line of space-aligned tool output.

Exercise 5 — the metacharacter. Split "v1.2.3" with split /./ and report how many fields you get. The answer is surprising — explain it using §2's rule about trailing empty fields. Then do it correctly and say what one character you changed.

Exercise 6 — a limit of 2. Given my $line = "ERROR: setup violation on path: U1/U2";, use a limit to split it into exactly two fields: the label and the whole rest of the message. Confirm the second field still contains its colon.

Exercise 7 — the faithful update. Write a script that reads "fifo_wr,PASS,92,", changes the second field to FAIL, and writes the row back out with the trailing empty field intact. Then remove the -1 and record how the output row differs.

12. Summary

• split turns a line into fields; join turns fields into a line. On tidy data the round trip is exact.
• The first argument to split is a regular expression, so a separator that is a metacharacter must be escaped.
• Trailing empty fields are discarded by default. "a,b,," gives two fields, which silently shortens any line you write back.
• A limit of -1 keeps them, and is the right default whenever the data will be rewritten.
• A leading empty field is kept — the rule is about the end of the string only.
• split " " is a special case meaning "runs of whitespace, leading whitespace discarded". split / / is one literal space, and gave eight fields where " " gave three.
• A positive limit returns at most that many fields, with the remainder unsplit in the last one.
• split // splits into characters, and join accepts any separator.
• Count your fields before indexing them.

Chapter 4.8 covers the other half of producing a report — placing values into a template with fixed widths and a set number of decimals, so that the columns this chapter splits can be written out aligned.