chomp and chop

Part I · Module 4 · Chapter 4.9 · chomp and chop

1. The Problem

Reading a line from a file gives you the line and its ending.

use strict;
use warnings;

open my $fh, "<", "data.txt" or die "cannot open data.txt: $!\n";

while (my $line = <$fh>) {
    printf "len=%-3d [%s]  eq \"alu,PASS\": %s\n",
        length($line), $line, ($line eq "alu,PASS" ? "yes" : "no");
}

close $fh;

len=9   [alu,PASS
]  eq "alu,PASS": no
len=10  [fifo,FAIL
]  eq "alu,PASS": no

The first line is nine characters, not eight. The ninth is a newline, which is why the output bracket closes on the next line, and why the comparison against "alu,PASS" is no.

That comparison is correct. The operator is right — eq for text, as Chapter 3.5 established. The spelling is right. It fails anyway, because the value has a character on the end that you cannot see.

open and the while (my $line = <$fh>) loop belong to Module 13. They are here because the newline problem cannot be demonstrated honestly without reading a real file.

2. chomp Removes It

use strict;
use warnings;

open my $fh, "<", "data.txt" or die "cannot open data.txt: $!\n";

while (my $line = <$fh>) {
    chomp $line;

    printf "len=%-3d [%s]  eq \"alu,PASS\": %s\n",
        length($line), $line, ($line eq "alu,PASS" ? "yes" : "no");
}

close $fh;

len=8   [alu,PASS]  eq "alu,PASS": yes
len=9   [fifo,FAIL]  eq "alu,PASS": no

Nine characters became eight, the bracket now closes on the same line, and the comparison is yes.

chomp removes a trailing newline if there is one and does nothing if there is not — which makes it safe to call unconditionally, and that is exactly how it should be used: chomp every line as you read it, before anything else looks at the value.

Strictly, chomp removes the current value of the input record separator, held in the variable $/, which is a newline unless you have changed it. Module 13 covers $/; for now, "removes the trailing newline" is accurate for every script in Part I.

It also only ever removes one:

use strict;
use warnings;

my $line = "alu\n\n";

printf "before: %d\n", length($line);
chomp $line;
printf "after : %d\n", length($line);

before: 5
after : 4

Five to four, not five to three.

3. chomp Returns a Count, Not the String

This is the trap, and it catches people who have used similar functions in other languages.

use strict;
use warnings;

my $with    = "alu\n";
my $without = "alu";

my $r1 = chomp($with);
my $r2 = chomp($without);

printf "chomp(\"alu\\n\") returned [%s], variable is now [%s]\n", $r1, $with;
printf "chomp(\"alu\")   returned [%s], variable is now [%s]\n", $r2, $without;

chomp("alu\n") returned [1], variable is now [alu]
chomp("alu")   returned [0], variable is now [alu]

chomp returned 1 and 0 — the number of characters removed. It modifies the variable in place and hands back a count.

So this is a bug:

use strict;
use warnings;

my $line = "alu\n";

my $clean = chomp($line);

printf "\$clean = [%s]\n", $clean;
printf "\$line  = [%s]\n", $line;

$clean = [1]
$line  = [alu]

$clean holds 1. The chomped text is in $line, where it always was.

4. chop Is a Different Function

The names are one letter apart and the behaviours are not similar.

use strict;
use warnings;

my $a = "alu\n";
my $b = "alu";

my $r1 = chop($a);
my $r2 = chop($b);

printf "chop on \"alu\\n\": returned [%s]  left [%s]\n", ($r1 eq "\n" ? "\\n" : $r1), $a;
printf "chop on \"alu\"  : returned [%s]  left [%s]\n", $r2, $b;

chop on "alu\n": returned [\n]  left [alu]
chop on "alu"  : returned [u]  left [al]

chop removes the last character whatever it is, and returns that character.

On a line that ends in a newline the two look identical. On a line that does not, chop eats a character of real data — alu became al.

That case is not hypothetical. The last line of a file often has no trailing newline, so a loop using chop processes every line correctly except the last one, which arrives one character short. A block name that is wrong by its final letter will not match anything, and the error message will name a block that does not exist.

Use chomp. chop has legitimate uses — removing a known trailing character you put there yourself — but line endings are not one of them.

5. chomp Does Not Fix a Windows Line Ending

This is the version of the problem that wastes the most time, because it survives the fix you have already applied.

use strict;
use warnings;

my $line = "PASS\r\n";

printf "before chomp : len=%d  eq \"PASS\": %s\n",
    length($line), ($line eq "PASS" ? "yes" : "no");

chomp $line;

printf "after chomp  : len=%d  eq \"PASS\": %s\n",
    length($line), ($line eq "PASS" ? "yes" : "no");

my $trimmed = "PASS\r\n";
$trimmed =~ s/\s+$//;

printf "after trim   : len=%d  eq \"PASS\": %s\n",
    length($trimmed), ($trimmed eq "PASS" ? "yes" : "no");

before chomp : len=6  eq "PASS": no
after chomp  : len=5  eq "PASS": no
after trim   : len=4  eq "PASS": yes

Six characters, then five, then four.

A Windows line ending is two characters — carriage return then newline, the \r and \n measured in Chapter 4.3 §2. chomp removed the newline and left the carriage return. The length went down, so something clearly happened; the comparison still fails.

The symptom is maddening in a specific way: you add chomp, the length changes, and the bug does not go away. The natural conclusion is that chomp is not the problem — and the real conclusion is that it only did half the job.

Two fixes:

• $line =~ s/\s+$//; — the trailing half of Chapter 4.6 §3's trim. This removes the newline, the carriage return, and any trailing spaces or tabs, which on parsed data you almost always also want gone.
• $line =~ s/\r?\n$//; — remove an optional carriage return and a newline, and nothing else. Use this when trailing spaces are meaningful data.

For general-purpose parsing in this trade, the first is the right default. The practical rule:

chomp for a Unix file you control. A trailing-whitespace trim for anything that might have crossed platforms.

6. Why This Matters in Engineering Work

It is the first bug in almost every parsing script. A filter for lines whose status is PASS matches nothing, a hash keyed on a block name has keys that look right and do not match, a numeric field compared with == happens to work — because, as Chapter 2.13 showed, "184\n" numifies to 184 without complaint.

That last asymmetry is worth holding onto: numeric comparisons tolerate a trailing newline and textual ones do not. So a script can appear to work on its number columns while every one of its name and status comparisons silently fails.

Logs cross platforms constantly. A tool run on a Windows workstation, a report pulled from a shared drive, a file edited on a laptop. Any of them can carry \r\n, and §5 is the result.

The debugging technique is the one from Chapter 4.6 §5: when a comparison fails and the values look identical, print the length. One extra character is the answer nine times out of ten, and the length tells you so immediately where the printed value never will.

And chomp on input, once. Like normalisation, this belongs at the point you read the line — not at the point you compare. A script that chomps in its read loop never has to think about it again.

7. Common Mistakes

Forgetting to chomp. The comparison is correct and fails. §1.

Assigning chomp's return value. my $clean = chomp($line) puts 1 in $clean. Call it as a statement.

Using chop instead of chomp. It removes the last character unconditionally, which corrupts the last line of a file with no trailing newline.

Assuming chomp handles \r\n. It removes the newline and leaves the carriage return. Trim instead.

Expecting chomp to remove all trailing newlines. It removes one.

Concluding chomp is not the problem because the length changed. It changed by one and needed to change by two.

Chomping at comparison time rather than read time. You will miss a path through the code.

8. Exercise

Exercise 1 — reproduce the failure. Create a two-line file, read it without chomping, and for each line print the length and whether it equals the text you expect. Then add chomp and run again, recording both sets of numbers.

Exercise 2 — the return value. Run the §3 script. Record what chomp returned in each case and what each variable holds afterwards. Then write one sentence explaining why my $clean = chomp($line); is a bug that no diagnostic reports.

Exercise 3 — chop eats data. Apply chop to "fifo\n" and to "fifo", printing the return value and the remaining string for each. Say which call lost real data and describe the file condition that would cause that to happen in a loop.

Exercise 4 — the Windows line. Build the string "FAIL\r\n", then print its length after chomping and after trimming with s/\s+$//. Confirm only one of the two makes it eq "FAIL", and state the length at each of the three stages.

Exercise 5 — the asymmetry. Take the string "184\n". Compare it with 184 using == and with "184" using eq, capturing STDERR. Report which comparison succeeded, which warned, and what that implies about a script whose numeric columns work and whose status column does not.

Exercise 6 — find it by length. You have two values that both print as DONE and are not eq. Write the three lines of debugging output you would add to identify the difference, and say what each line tells you.

Exercise 7 — choose the right cleanup. For each, say whether you would use chomp, s/\s+$// or s/\r?\n$//, and why in one clause: (a) a Unix file your own script wrote; (b) a CSV exported from a Windows tool; (c) a field whose trailing spaces are significant padding you must preserve; (d) a log of unknown origin from a shared drive.

9. Summary

• Reading a line includes its line ending, so "alu,PASS\n" is nine characters and is not eq to "alu,PASS".
• chomp removes one trailing newline — strictly, the value of $/ — and is safe to call when there is none.
• chomp returns the count of characters removed, not the string, and modifies the variable in place. my $clean = chomp($line) puts 1 in $clean, silently.
• Call chomp as a statement, not inside an expression.
• chop removes the last character whatever it is, so it truncates a line with no trailing newline. Do not use it for line endings.
• chomp leaves a carriage return behind on a Windows line ending — the length changes and the comparison still fails.
• Trim with s/\s+$// for data that may have crossed platforms.
• Numeric comparisons tolerate a trailing newline; textual ones do not, so a script can look half-working.
• When two values print the same and compare unequal, print the length.

Chapter 4.10 turns to producing multi-line text in one statement — usage messages, generated files — including the indented form and the Perl version it requires.