The Core Modules Worth Knowing

Part I · Module 12 · Chapter 12.9 · The Core Modules Worth Knowing

1. Paths: File::Basename and File::Spec

Two modules, one job each: take a path apart, and put one together.

use strict;
use warnings;
use File::Basename qw(basename dirname fileparse);
use File::Spec;

my $path = "/proj/reports/timing/setup.rpt";

printf "basename : %s\n", basename($path);
printf "dirname  : %s\n", dirname($path);
my ($name, $dir, $suffix) = fileparse($path, qr/\.[^.]*/);
printf "fileparse: name=%s suffix=%s\n", $name, $suffix;

# File::Spec builds paths without hard-coding a separator
printf "catfile  : %s\n", File::Spec->catfile("proj", "reports", "setup.rpt");
printf "catdir   : %s\n", File::Spec->catdir("proj", "reports");
my ($vol, $dirs, $file) = File::Spec->splitpath($path);
printf "splitpath: dirs=%s file=%s\n", $dirs, $file;

basename : setup.rpt
dirname  : /proj/reports/timing
fileparse: name=setup suffix=.rpt
catfile  : proj/reports/setup.rpt
catdir   : proj/reports
splitpath: dirs=/proj/reports/timing/ file=setup.rpt

basename and dirname are the two you will use most. fileparse is the one worth remembering, because it is the only one that separates the suffix: given a pattern for the extension it returns the stem and the extension separately, which is what you want when turning setup.rpt into setup.csv.

Note that fileparse takes a pattern for the suffix rather than guessing. qr/\.[^.]*/ means "a dot and everything after it that is not a dot", which handles setup.rpt correctly and leaves v1.2.rpt with the stem v1.2. Getting that right by hand is Chapter 11.7's greedy-versus-lazy problem in miniature.

File::Spec is the constructive half, and catfile is its headline. Joining with / by hand works on every machine you are likely to use and is still worth avoiding, for a reason that has nothing to do with Windows: catfile will not produce a doubled separator when a component already ends in one, which is the actual bug that path-joining code has.

2. Walking a Tree: File::Find and File::Path

use strict;
use warnings;
use File::Temp qw(tempdir);
use File::Path qw(make_path remove_tree);
use File::Find qw(find);
use File::Spec;

my $root = tempdir(CLEANUP => 1);

# File::Path makes a whole tree in one call
my $deep = File::Spec->catdir($root, "a", "b", "c");
make_path($deep);
printf "make_path created the tree : %s\n", (-d $deep ? "yes" : "no");

for my $n (1 .. 3) {
    open my $fh, ">", File::Spec->catfile($deep, "r$n.rpt") or die $!;
    print $fh "x\n"; close $fh;
}
open my $other, ">", File::Spec->catfile($root, "notes.txt") or die $!;
print $other "x\n"; close $other;

# File::Find walks it
my @rpt;
find(sub { push @rpt, $File::Find::name if /\.rpt$/ }, $root);
printf "find located .rpt files    : %d\n", scalar @rpt;

# remove_tree deletes it
remove_tree($deep);
printf "remove_tree removed it     : %s\n", (-d $deep ? "no" : "yes");

make_path created the tree : yes
find located .rpt files    : 3
remove_tree removed it     : yes

Three .rpt files found and the notes.txt correctly ignored.

make_path creates every missing intermediate directory, which mkdir does not — that is the whole reason to use it, and it is also idempotent on a directory that already exists. remove_tree is the counterpart and the one to be careful with, for obvious reasons.

File::Find's interface is unusual enough to be worth spelling out, because it is the most common reason people write their own opendir recursion instead. You pass a callback, and inside it:

• $_ is the bare filename, which is why if (/\.rpt$/) works with no variable named — Chapter 11.2's bare pattern against $_.
• $File::Find::name is the full path, which is almost always what you want to keep.
• $File::Find::dir is the directory being scanned.
• The working directory has been changed to that directory, so a bare -s $_ works on the current file.

That last point is the gotcha: because find chdirs, a relative path you captured earlier may not resolve inside the callback. Collect $File::Find::name and do the work afterwards, which is what the script above does.

3. Command Lines: Getopt::Long

use strict;
use warnings;
use Getopt::Long qw(GetOptionsFromArray);

# GetOptionsFromArray lets us demonstrate parsing without a real command line.
my @argv = ("--report", "setup.rpt", "--threshold", "-0.1", "--verbose", "extra");

my ($report, $threshold, $verbose) = ("", 0, 0);
GetOptionsFromArray(\@argv,
    "report=s"    => \$report,
    "threshold=f" => \$threshold,
    "verbose"     => \$verbose,
) or die "bad options\n";

printf "report    : %s\n", $report;
printf "threshold : %s\n", $threshold;
printf "verbose   : %d\n", $verbose;
printf "left over : %s\n", join(",", @argv);

report    : setup.rpt
threshold : -0.1
verbose   : 1
left over : extra

Three options parsed into three variables, with types enforced, and the non-option argument left behind.

The type suffixes are the reason to use this rather than a loop: =s is a required string, =f a float, =i an integer, and a bare name is a boolean flag. threshold=f accepted -0.1 — a negative number that a hand-rolled parser would likely mistake for another option.

That last line is the detail people miss. GetOptions removes the options it recognises and leaves everything else in the array, so after the call @ARGV holds exactly the file arguments. That is how a tool takes both switches and a list of files.

This script uses GetOptionsFromArray rather than GetOptions so the parse can be demonstrated on a list we control. In a real script you call GetOptions(...) with no array and it reads @ARGV.

4. Numbers, Time and Where You Are: POSIX and Cwd

use strict;
use warnings;
use POSIX qw(floor ceil strftime fmod);
use Cwd qw(getcwd abs_path);

printf "floor(-0.12) : %s\n", floor(-0.12);
printf "ceil(-0.12)  : %s\n", ceil(-0.12);
printf "fmod(7, 2.5) : %s\n", fmod(7, 2.5);
printf "strftime     : %s\n", strftime("%Y-%m-%d", gmtime(0));

# Cwd: where am I, and what is this path really
printf "getcwd is absolute : %s\n", (getcwd() =~ m{^/} ? "yes" : "no");
printf "abs_path('.') == getcwd : %s\n", (abs_path(".") eq getcwd() ? "yes" : "no");

floor(-0.12) : -1
ceil(-0.12)  : 0
fmod(7, 2.5) : 2
strftime     : 1970-01-01
getcwd is absolute : yes
abs_path('.') == getcwd : yes

fmod is the floating-point remainder, which % is not — Perl's % operator works on integers, so 7 % 2.5 is not what you want and fmod(7, 2.5) giving 2 is.

strftime formats a time, and the arguments are the nine-element list that gmtime and localtime return in list context. Using gmtime(0) here makes the output reproducible; a real script passes localtime(time).

Cwd answers two different questions. getcwd is where the process is — and Chapter 12.4's use lib trap is entirely about how much that matters. abs_path resolves a relative path, and symbolic links, into a real absolute one, which is the right way to turn a user-supplied path into something you can compare or log.

5. The Two Already Used

Two of the nine have appeared throughout this track and need no re-introduction.

Data::Dumper was Chapter 7.12's subject: it prints a nested structure in a form you can read, and the chapter measured its quirks — that Indent and Sortkeys are package variables you set directly, in the our tradition Chapter 12.7 described, and that its quoting reflects whether a value is a float or an integer rather than its history.

Time::HiRes supplied the time with sub-second resolution behind every timing measurement in Module 11, including the one that overturned the qr// folklore. Its one essential fact is that you must import time explicitly — use Time::HiRes qw(time) — or you get the built-in with one-second resolution and your measurements all read zero.

6. What To Remember

• All nine ship with Perl and were confirmed present on the Perl this page was built with. Writing any of them by hand is a choice, not a necessity.
• File::Basename: basename, dirname, and fileparse for separating the suffix — give fileparse a pattern rather than letting it guess.
• File::Spec->catfile joins path components and will not produce a doubled separator, which is the real bug in hand-written joining.
• File::Path: make_path creates intermediate directories and is idempotent; remove_tree deletes.
• File::Find takes a callback where $_ is the bare name and $File::Find::name is the full path — and it has chdir'd, so collect names and work on them afterwards.
• Getopt::Long enforces types (=s, =i, =f, bare for a flag), handles a negative number as a value, and leaves non-options in the array so a tool can take switches and files.
• floor on a negative number moves away from zero. floor(-0.12) is -1. Use int() to truncate toward zero or sprintf to round.
• fmod is the floating-point remainder; % is integer-only.
• Cwd: getcwd for where the process is, abs_path to resolve a path and its symlinks.
• Time::HiRes needs qw(time) imported explicitly, or you silently get one-second resolution.

Chapter 12.10 covers what to do when the module you need is not one of these — and the constraint that makes that question harder in this industry than anywhere else, which is that you almost certainly cannot write to the Perl installation on a shared compute farm.