Finding RTL Sources With File::Find

Part II · Module 19 · Chapter 19.2 · Finding RTL Sources With File::Find

1. Four Filters, One Tree

The tree holds exactly five files a filelist should list: two packages, two modules, one testbench. It also holds an include header, a second header, a generated module, a .bak copy and an editor lock file — all of which a careless filter will pick up.

use strict;
use warnings;
use File::Find;
use Cwd qw(abs_path);

# The tree holds exactly FIVE files a filelist should list:
#   rtl/core/{chip_pkg,types_pkg,counter,alu}.sv  and  tb/tb_counter.sv
# Everything else that looks like a source is a trap.
my $TRUTH = 5;

sub collect {
    my ($keep) = @_;
    my @hits;
    find({ no_chdir => 1, wanted => sub { push @hits, $File::Find::name if $keep->($File::Find::name) } },
         "rtl", "tb");
    return sort @hits;
}

# ---- A: anything whose name contains ".sv" (the unanchored mistake)
my @a = collect(sub { -f $_[0] && $_[0] =~ /\.sv/ });

# ---- B: anchored at the end
my @b = collect(sub { -f $_[0] && $_[0] =~ /\.sv$/ });

# ---- C: anchored, and prune the generated tree
my @c = collect(sub { -f $_[0] && $_[0] =~ /\.sv$/ && $_[0] !~ m{(^|/)generated/} });

# ---- D: ... and skip dotfiles (editor locks)
my @d = collect(sub {
    my $p = shift;
    return 0 unless -f $p && $p =~ /\.sv$/;
    return 0 if $p =~ m{(^|/)generated/};
    return 0 if (split m{/}, $p)[-1] =~ /^\./;
    return 1;
});

printf "the tree contains exactly %d files a filelist should list\n\n", $TRUTH;
printf "%-42s %-7s %s\n", "FILTER", "FOUND", "verdict";
for my $r (["A  name contains .sv", \@a], ["B  name ENDS .sv", \@b],
           ["C  B + prune generated/", \@c], ["D  C + skip dotfiles", \@d]) {
    my ($label, $list) = @$r;
    printf "%-42s %-7d %s\n", $label, scalar @$list,
        (@$list == $TRUTH ? "correct" : @$list > $TRUTH ? "WRONG (+".(@$list-$TRUTH).")"
                                                        : "WRONG (-".($TRUTH-@$list).")");
}

print "\nwhat each filter wrongly collected:\n";
my %want = map { $_ => 1 } @d;
for my $r (["A", \@a], ["B", \@b], ["C", \@c]) {
    my ($label, $list) = @$r;
    my @extra = grep { !$want{$_} } @$list;
    printf "  %s: %s\n", $label, (@extra ? join(", ", @extra) : "(nothing)");
}

print "\nthe accepted list, as a filelist would carry it:\n";
printf "  %s\n", $_ for @d;

print "\nabsolute paths resolve from any directory:\n";
printf "  relative: %s\n", $d[0];
my $abs = abs_path($d[0]);
printf "  absolute ends with: %s\n", ($abs =~ m{(/rtl/.*)$} ? $1 : "?");
printf "  is the absolute form rooted at /?  %s\n", ($abs =~ m{^/} ? "yes" : "no");

# ---- does each list actually build? (packages first, so order is not the variable)
print "\nhand each list to the compiler (packages first in every case):\n";
sub order_pkgs_first { my @l = @_; return ((grep { /_pkg\.sv$/ } @l), (grep { !/_pkg\.sv$/ } @l)) }
for my $r (["A", \@a], ["B", \@b], ["C", \@c], ["D", \@d]) {
    my ($label, $list) = @$r;
    open my $o, ">", "f_$label.f" or die $!;
    print $o "$_\n" for order_pkgs_first(@$list);
    close $o;
    unlink "o_$label.vvp";
    my $log = `iverilog -g2012 -I rtl/include -o o_$label.vvp -c f_$label.f 2>&1`;
    my $rc  = $? >> 8;
    my ($first) = grep { /\S/ } split /\n/, ($log // "");
    printf "  %s  rc=%-3s %s\n", $label, $rc,
        ($rc == 0 ? "compiles" : ($first // "failed"));
}

the tree contains exactly 5 files a filelist should list

FILTER                                     FOUND   verdict
A  name contains .sv                       10      WRONG (+5)
B  name ENDS .sv                           7       WRONG (+2)
C  B + prune generated/                    6       WRONG (+1)
D  C + skip dotfiles                       5       correct

what each filter wrongly collected:
  A: rtl/core/.#counter.sv, rtl/core/counter.sv.bak, rtl/generated/counter_gen.sv, rtl/include/opcodes.svh, rtl/include/widths.svh
  B: rtl/core/.#counter.sv, rtl/generated/counter_gen.sv
  C: rtl/core/.#counter.sv

the accepted list, as a filelist would carry it:
  rtl/core/alu.sv
  rtl/core/chip_pkg.sv
  rtl/core/counter.sv
  rtl/core/types_pkg.sv
  tb/tb_counter.sv

absolute paths resolve from any directory:
  relative: rtl/core/alu.sv
  absolute ends with: /rtl/core/alu.sv
  is the absolute form rooted at /?  yes

hand each list to the compiler (packages first in every case):
  A  rc=2   rtl/core/counter.sv.bak:2: error: 'counter' has already been declared in this scope.
  B  rc=0   compiles
  C  rc=0   compiles
  D  rc=0   compiles

Ten, seven, six, five. Each filter removes one class of mistake, and the list of what each one wrongly collected names the class exactly.

2. Why Each Trap Is There

.svh headers (caught only by anchoring). /\.sv/ without the $ matches widths.svh, because .svh contains .sv. An include header in the source list is not a harmless extra: it is meant to be found through the include path (Chapter 19.4), and compiling it directly is at best pointless. This is the single most common filelist bug and it comes from a missing anchor — Chapter 11.8's anchors applied to a filename instead of a log line.

counter.sv.bak (also caught by anchoring). It ends .bak, so /\.sv$/ rejects it. The unanchored version keeps it, and because it declares the same module as counter.sv, that is the one wrong list the compiler refuses outright.

rtl/generated/counter_gen.sv (needs a directory rule). A perfectly well-formed .sv file with a perfectly ordinary name. Nothing about the filename says to skip it; only its directory does. This is why a filename filter is not enough and the walk has to prune by path — Chapter 17.4 measured the same thing on a source tree, where a filename filter kept five files and a pruning walk kept two.

rtl/core/.#counter.sv (needs a dotfile rule). An editor's lock file. It ends .sv, it is not in generated/, and it is in the middle of the real source directory. Only the leading dot distinguishes it. Chapter 17.3 noted that glob skips dotfiles while File::Find does not — this is the cost of that difference.

3. The Part That Matters: Two Wrong Lists Compile

Read the last block of the output again, because it is the reason this chapter exists.

The compiler accepts two of the three wrong lists. List B builds module stale — from the editor lock file — and module counter_gen into the design. List C still builds module stale. Neither module is ever instantiated, so nothing in the compile or the simulation complains about them.

And list A is rejected for a reason that has nothing to do with its being too big: the .bak declares a module name that already exists (Chapter 19.1 measured that error). Had the backup been of a module since renamed, A would have compiled too.

So the lesson is not "check that it compiles":

A filelist that compiles is not a correct filelist. Compiling proves the list is sufficient — every name the design needs is defined somewhere. It says nothing about the list being minimal, and the extra files are the ones that cause the problems nobody can reproduce: a stale module that shadows nothing today and shadows something after the next rename, a generated file from an older run, a header compiled as a source.

The only things that catch an over-full list are the filter itself and an explicit count. Which is why the script above prints FOUND against a known truth rather than just printing the list — a number you can assert on, as Chapter 18.4 argued about counting errors in a log.

4. Absolute Paths, And The Walk Itself

The walk is the one from Chapter 13.10, with no_chdir => 1:

find({ no_chdir => 1, wanted => sub { ... $File::Find::name ... } }, "rtl", "tb");

no_chdir => 1 keeps the process in one directory and gives $File::Find::name as a path relative to where the walk started, rather than chdir-ing around and handing you a bare basename in $_. That is what makes the paths usable directly and the filter readable.

Two notes on what the walk gives you:

wanted is called for directories too, so the -f test in every filter is doing real work, not defensive decoration. Without it, rtl/core itself is a candidate.

Several roots in one call. find(..., "rtl", "tb") walks both. Chapter 19.3 is about what happens when two roots overlap — which they always eventually do.

Then the paths are made absolute, anchored at the project root as Chapter 17.1 established:

use FindBin;
use Cwd qw(abs_path);
my $ROOT = $ENV{PROJ_ROOT} // abs_path("$FindBin::Bin/..");
print $out abs_path("$ROOT/$rel"), "\n";

The measured output above confirms the mechanics: the relative form is rtl/core/alu.sv, the absolute form ends with the same tail and is rooted at /. Chapter 13.11's File::Spec is the portable way to join the pieces if the script ever has to run somewhere other than Unix; on a flow machine, string concatenation and abs_path are enough.

5. What To Remember

• The first job is deciding what counts as a source, and the usual failure is collecting too much, not too little.
• Measured, one tree, four filters: 10, 7, 6, 5 — and 5 is correct.
• /\.sv/ unanchored matches .svh headers and .sv.bak — the most common filelist bug, and an anchoring bug.
• An include header does not belong in the source list — it is reached through the include path (19.4).
• A generated file's name looks fine; only its directory is wrong — so filter by path, not filename alone.
• An editor lock file like .#counter.sv ends in .sv and sits in the real source directory — only the leading dot gives it away. glob skips dotfiles; File::Find does not.
• Prune directories rather than filtering their files — and remember prune needs -d $_, with a broken prune failing silently (17.4).
• Allow-list extensions (fails closed), deny-list directories (fails open) — use both.
• Measured: the compiler accepted two of the three wrong lists. B compiled module stale and module counter_gen in; C compiled module stale in.
• A filelist that compiles is not a correct filelist — compiling proves sufficiency, never minimality.
• Unused extra modules never complain, which is why an over-full list is found by counting, not by building.
• Assert the count against a known truth, rather than eyeballing the list.
• readdir order is not alphabetical, so sort before emitting (17.8) — otherwise two runs give two filelists and two designs.
• Use no_chdir => 1 so $File::Find::name is a usable path, and keep the -f test — wanted fires for directories too.
• find takes several roots in one call; overlapping roots are Chapter 19.3's problem.
• abs_path resolves symlinks — right for deduplication, wrong if you meant to follow a current/ view.

Chapter 19.3 takes the next inevitable problem. Two search roots that overlap produce the same file twice, and the same file can arrive under two different spellings — so the fix is a hash, plus one decision about what "the same file" means.