Searching Recursively, and Excluding What You Did Not Mean

Part I · Module 14 · Chapter 14.4 · Searching Recursively

Three chapters of searching one file at a time. Real work happens in a tree.

1. One Flag

$ grep -rn clk_div tree
tree/docs/notes.md:1:clk_div mentioned in docs
tree/rtl/alu.v:2:  clk_div = 2;
tree/build/gen.v:1:clk_div in a build artefact

-r searches every file beneath a directory. With -n for line numbers, each result names the file and the line, which is what makes the output actionable.

grep -rn 'clk_div' .           # from here down
grep -rn 'clk_div' rtl/        # one subtree
grep -rln 'TODO' .             # just the file names — Chapter 14.5

2. Why That Is Not Enough

On the three-file tree above, three matches is a fine answer. On a real repository it is not.

$ grep -rn 'clk_div' . | wc -l
847

A search for one signal returns eight hundred lines, and almost none of them are the definition you wanted. They come from:

• generated output — synthesis reports, elaborated netlists, simulation logs;
• build trees — object files, intermediate Verilog, copies of your own source;
• vendor and third-party code you are not going to change;
• version-control internals — .git holds every version of every file;
• binary files that happen to contain the byte sequence.

3. Naming What You Want

$ grep -rn --include='*.v' clk_div tree
tree/rtl/alu.v:2:  clk_div = 2;
tree/build/gen.v:1:clk_div in a build artefact

--include='PATTERN' searches only files whose names match. The pattern is a glob (Chapter 3.5), not a regular expression — so *.v means what it looks like.

Quote it. Unquoted, the shell expands *.v against the current directory before grep ever sees it (Chapter 9.1), and you end up restricting the search to whatever happened to be in your working directory.

Several are allowed:

grep -rn --include='*.v' --include='*.sv' --include='*.vh' 'clk_div' .

4. Naming What You Do Not

$ grep -rn --exclude-dir=build clk_div tree
tree/docs/notes.md:1:clk_div mentioned in docs
tree/rtl/alu.v:2:  clk_div = 2;

--exclude-dir=NAME skips a directory entirely — grep does not descend into it at all, so it costs nothing.

grep -rn --exclude-dir=.git --exclude-dir=build --exclude-dir=node_modules 'clk_div' .

And --exclude does the same for files:

grep -rn --exclude='*.log' --exclude='*.o' 'clk_div' .

5. Make It a Default

Typing six exclusions every time is how people stop using them. Two ways to avoid it.

An alias or function in ~/.bashrc (Chapter 10.5):

rtlgrep() {
  grep -rn --include='*.sv' --include='*.v' --include='*.vh' \
       --exclude-dir=build --exclude-dir=.git \
       "$@"
}

$ rtlgrep clk_div .

A function rather than an alias, because it needs "$@" — Chapter 10.5's point exactly.

GREP_OPTIONS is not the answer. It was an environment variable that added flags to every invocation; GNU grep deprecated it precisely because it silently changed the behaviour of other people's scripts. If you see it in an old setup file, remove it.

6. Binary Files

$ grep -rn clk_div .
Binary file ./sim/waves.vcd matches

grep detects binary content and refuses to print matching lines, saying so instead — which is right, since dumping a megabyte of control characters into a terminal can leave it unusable.

Two flags control it:

grep -rnI  'clk_div' .                      # skip binary files entirely
grep -rna  'clk_div' .                      # treat them as text and print anyway

-I is the one to want. A "Binary file matches" line is noise in a source search, and -I removes it along with the file.

That message is also a useful signal in itself: a .v file reported as binary usually means it has the wrong line endings, a stray null byte, or is not what you think it is (Chapter 4.7).

7. Searching Many Files Without -r

-r walks a tree. Sometimes you want a specific set of files instead, and there are two ways.

A glob, when the shell can express it:

grep -n 'clk_div' rtl/*.v
grep -n 'clk_div' rtl/**/*.v        # needs shopt -s globstar — Chapter 10.6

A list from find, when it cannot:

find . -name '*.v' -newer last_build -print0 | xargs -0 grep -n 'clk_div'

That second form is Chapter 14.7, and it is what you use when the selection depends on something a glob cannot express — age, size, type, depth.

8. ripgrep

Worth knowing about, because it is now common on engineering machines.

rg clk_div

ripgrep is a grep replacement that respects .gitignore by default, skips binary files and hidden directories automatically, and is substantially faster. For searching a source repository it does the right thing with no flags at all, which is most of this chapter's content made unnecessary.

Three cautions:

• It is not installed by default anywhere, so a script cannot assume it.
• Its options differ — -g '*.v' rather than --include, and its default is already recursive.
• Its defaults can hide things. A file listed in .gitignore is skipped, which is usually right and occasionally exactly wrong — searching for a value in generated output, for instance, needs rg -u.

Chapter 14.1 noted that grep on the verification machine was in fact ugrep, another of these. Know which you are running, and write scripts against grep rather than whichever replacement happens to be installed.

9. In Practice

# the search you will type most
grep -rn --include='*.sv' --include='*.v' 'clk_div' .

# with the usual noise excluded
grep -rnI --exclude-dir=.git --exclude-dir=build --exclude-dir=work 'clk_div' .

# which files, not which lines — Chapter 14.5
grep -rl --include='*.sv' 'uvm_component_utils' .

# case-insensitive across documentation as well
grep -rni --include='*.md' --include='*.txt' 'clock divider' docs/

# a whole-word search, which on an identifier is the important one
grep -rnw --include='*.v' 'clk' rtl/

# count matches per file
grep -rc --include='*.sv' 'assert' . | grep -v ':0$'

# find a signal's declaration rather than every mention
grep -rnE --include='*.v' '^\s*(wire|reg|logic).*\bclk_div\b' .

The last one combines everything so far: recursive, restricted by type, extended syntax from Chapter 14.3, anchored, and with word boundaries. It finds the declaration and not the four hundred uses.

And note grep -v ':0$' on the counting one — grep -rc reports every file including those with zero matches, which is rarely what you want.

Practice

mkdir -p /tmp/rg/{rtl,build,docs,.git} && cd /tmp/rg
printf 'module alu;\n  wire clk_div;\n  assign clk_div = 1;\nendmodule\n' > rtl/alu.v
printf 'module fifo;\n  wire clk_divider;\nendmodule\n' > rtl/fifo.v
printf 'clk_div in a generated artefact\n' > build/gen.v
printf 'clk_div appears in the notes\n' > docs/notes.md
printf 'clk_div in version control data\n' > .git/objects.txt
printf 'binary\000clk_div\000data\n' > sim.vcd

1 — Search everything, and count the noise.

grep -rn clk_div .
grep -rn clk_div . | wc -l

2 — Restrict by file type.

grep -rn --include='*.v' clk_div .

3 — Exclude directories.

grep -rn --exclude-dir=build --exclude-dir=.git clk_div .

4 — Combine both.

grep -rn --include='*.v' --exclude-dir=build clk_div .

5 — Watch the binary message, then suppress it.

grep -rn clk_div . 2>&1 | grep -i binary
grep -rnI clk_div . | wc -l

6 — See why quoting the include matters.

cd rtl
grep -rn --include='*.v' clk_div ..
grep -rn --include=*.v  clk_div ..
cd ..

The second may behave differently depending on what the shell expanded.

7 — Use word boundaries.

grep -rn  --include='*.v' clk_div rtl/
grep -rnw --include='*.v' clk_div rtl/

fifo.v has clk_divider, which only one of those should match.

8 — Make it a function.

rtlgrep() { grep -rn --include='*.v' --include='*.sv' --exclude-dir=build --exclude-dir=.git "$@"; }
rtlgrep clk_div .

9 — Count per file, without the zeros.

grep -rc --include='*.v' clk_div .
grep -rc --include='*.v' clk_div . | grep -v ':0$'

10 — Clean up. cd /tmp && rm -rf /tmp/rg

Check Your Understanding

1. What is the difference between -r and -R?

2. Why does grep -rn pattern . on a repository usually return too much?

3. Why must --include='*.v' be quoted?

4. --exclude-dir=build — does it skip one directory or several?

5. grep -rn pattern . prints "Binary file ./x.v matches". What does that suggest about x.v?

6. Why is grep pattern $(find . -name '*.v') a bad idea?

10. Remember

• -r searches a whole tree; -R additionally follows symbolic links, which can duplicate results. Prefer -r.
• Always give the directory — grep -rn pattern . — because BSD grep requires it.
• -r alone is nearly useless on a repository. The signal is buried in generated output, vendor code and version-control data.
• --include='*.v' is usually the single most effective flag, and it is faster because files are skipped by name without being opened.
• Quote the glob, or the shell expands it first.
• --exclude-dir=build skips a whole subtree and matches the name, at any depth.
• --include, --exclude and --exclude-dir are GNU extensions that BSD grep also supports; a strictly portable script should use find.
• -I skips binary files and removes the "Binary file matches" noise.
• A source file reported as binary is a signal, not just noise.
• Wrap your usual flags in a shell function, not an alias — it needs "$@".
• GREP_OPTIONS is deprecated. Remove it if you find it.
• Never substitute find into an argument list. Use -r, or find -print0 | xargs -0.
• ripgrep does most of this automatically but is not installed by default and can hide ignored files.
• Choosing where to search matters more than choosing the tool — Chapter 24.3 measures an --include form that silently matches nothing, and Chapter 23.5 one whose answer depends on which grep is installed.

You can now search a tree and get a readable answer. The next chapter is about shaping that answer — asking "which files" or "how many" instead of reading every matching line yourself.