Keeping Generated Directories Out of Your Way

Part II · Module 23 · Chapter 23.5 · Keeping Generated Directories Out of Your Way

Chapter 23.3 found three definitions of one module, and one of them was a stale copy inside a run directory. That is the general problem: generated directories contain copies of your source, often several versions old, and a recursive search cannot tell them apart.

1. The Cost, Measured

Searching chip_a for clk across SystemVerilog files, with nothing excluded:

$ /usr/bin/grep -rn clk --include='*.sv' . | wc -l
16
$ /usr/bin/grep -rln clk --include='*.sv' . | sed 's|^\./||; s|/.*||' | sort | uniq -c
   2 ip
   7 rtl
   1 run_2026_10_01
   1 simv.daidir
   5 tb

Two of the sixteen hits are in generated directories, in a repository with one simulation's worth of output. The ratio in a real tree is the other way round — a project with a week of regression results has tens of thousands of generated files and a few thousand source files, so the signal is a few per cent of the output.

Worse than the volume is the plausibility. A stale copy of axi_master.sv in a run directory is valid SystemVerilog defining the right module with slightly wrong contents. Nothing marks it. You open it, read it, and reason about code that is not in the build.

2. The Four Ways, Compared

All four measured on the same repository, searching for clk in *.sv:

Two rows deserve attention, and the first is the registry's own suggested command.

The deny-list missed a directory. --exclude-dir={work,simv.daidir,csrc} removed one hit, leaving fifteen — because run_2026_10_01/ was not on the list:

$ /usr/bin/grep -rln clk --include='*.sv' --exclude-dir={work,simv.daidir,csrc} .
./tb/tests/tb_chip.sv
./run_2026_10_01/axi_master.sv
./rtl/core/chip_top.sv
./rtl/axi/axi_master.sv
./rtl/uart/uart_tx.sv
./ip/vendor_axi/rtl/axi_master.sv

That is the structural flaw: a deny-list excludes what you remembered. Run directories are created with names nobody predicted, a new tool adds xcelium.d next quarter, and the list in your shell alias silently stops matching reality. It is the same argument Chapter 20.4 made about deny-lists in a destructive command, and the consequence here is quieter but the mechanism is identical.

git grep found fourteen, including ip/. That is correct: imported IP is tracked source, which Chapter 23.2 §3 established. git grep excludes exactly the untracked and ignored files, which is a better definition of "not source" than any list you will maintain by hand.

$ git grep -ln clk -- '*.sv'
ip/vendor_axi/rtl/axi_master.sv
rtl/axi/axi_master.sv
rtl/core/chip_top.sv
rtl/uart/uart_tx.sv
tb/tests/tb_chip.sv

Five files, all source, nothing generated — with no list to maintain, because the repository already declares what is generated in its .gitignore.

3. The Discovery: Your grep May Already Be Doing This

Here is the measurement that matters most in this chapter, and it is a reproducibility problem rather than a search problem.

The same command, twice, on the same machine, in the same directory:

$ grep -rl 'module axi_master' --include='*.sv' .
ip/vendor_axi/rtl/axi_master.sv
rtl/axi/axi_master.sv
$ /usr/bin/grep -rl 'module axi_master' --include='*.sv' .
./ip/vendor_axi/rtl/axi_master.sv
./rtl/axi/axi_master.sv
./run_2026_10_01/axi_master.sv

Two files, or three. The difference is which program answered:

$ type -a grep
grep is a shell function from /Users/engineer/.../snapshot-zsh-….sh
grep is /usr/bin/grep
$ grep --version | head -1
ugrep 7.8.4 aarch64-apple-macosx +neon/AArch64; -P:pcre2jit; …
$ /usr/bin/grep --version | head -1
grep (BSD grep, GNU compatible) 2.6.0-FreeBSD

And the cause is provable rather than guessed. Move the .gitignore away and ask ugrep again:

$ mv .gitignore /tmp/ && grep -rl 'module axi_master' --include='*.sv' .
ip/vendor_axi/rtl/axi_master.sv
rtl/axi/axi_master.sv
run_2026_10_01/axi_master.sv
$ mv /tmp/.gitignore . && grep -rl 'module axi_master' --include='*.sv' .
ip/vendor_axi/rtl/axi_master.sv
rtl/axi/axi_master.sv

Three without it, two with it. ugrep honours .gitignore by default; BSD grep does not. The modern search tools behave the same way — ripgrep (rg) and ag both skip ignored files unless told otherwise — so this is the common case rather than an oddity.

4. The --exclude-dir Details That Bite

Three facts about the registry's suggested form, all measured, in the order you meet them.

BSD grep does accept --exclude-dir. It describes itself as "GNU compatible", and the option works:

$ /usr/bin/grep -rn clk --include='*.sv' --exclude-dir=work . | wc -l
15
$ echo $?
0

So the option is portable between GNU and BSD, which is not true of many of its neighbours — Chapter 21.6's table is full of counterexamples.

The braces are the shell's, not grep's. This matters more than it looks:

$ echo --exclude-dir={work,simv.daidir,csrc}
--exclude-dir=work --exclude-dir=simv.daidir --exclude-dir=csrc

Brace expansion produces three separate options, which is why the form works at all. But brace expansion is a Bash feature, so:

$ bash -c 'echo --exclude-dir={a,b}'
--exclude-dir=a --exclude-dir=b
$ /bin/sh -c 'echo --exclude-dir={a,b}'
--exclude-dir=a --exclude-dir=b
$ /bin/dash -c 'echo --exclude-dir={a,b}'
--exclude-dir={a,b}

/bin/sh on macOS expanded them — because it is Bash in POSIX mode, version 3.2.57. dash passed the literal string, and grep then excludes a directory whose name is {a,b}, which matches nothing, with no error.

On Debian and Ubuntu, /bin/sh is dash. So a flow script with #!/bin/sh and a brace form silently searches everything on a Linux build machine and correctly on your Mac — which is Chapter 15.2's warning arriving with a concrete cost. The same applies to a Makefile recipe, where make uses /bin/sh by default.

The portable spelling is to repeat the option, which also reads better once there are more than two:

/usr/bin/grep -rn --include='*.sv' \
  --exclude-dir=work --exclude-dir=simv.daidir --exclude-dir=csrc \
  --exclude-dir=cov --exclude-dir='run_*' "$pat" .

And a pattern works in --exclude-dir, which is the one thing that makes a deny-list survivable: --exclude-dir='run_*' catches every run directory, including the ones that do not exist yet. Quote it, or the shell expands it against the current directory first.

5. A Positive List Is Better Than Any Deny-List

The structural fix, and it is three characters shorter than the problem:

$ /usr/bin/grep -rn clk --include='*.sv' rtl tb | wc -l
12

Name what you want. A new generated directory cannot appear inside rtl/, so the list does not go stale. A new source directory does need adding — but that is a visible failure (you search and find nothing) rather than an invisible one (you search and find too much, including a lie).

This is the same reasoning Chapter 20.4 §7 applied to deletion: a positive rule fails safe, a deny-list fails open. The habit is worth building as a shell function:

# in your shell configuration
rg-src() {
  local roots=(rtl tb ip)
  /usr/bin/grep -rn --include='*.sv' --include='*.v' --include='*.svh' -- "$@" "${roots[@]}"
}

Two details: -- before "$@" so a pattern beginning with a dash is not read as an option — Chapter 1.5 — and the roots in an array expanded last, so the pattern and any extra flags come first. Chapter 19.5 is the array; Chapter 18.6 is where to keep it.

6. Beyond grep: the Same Problem in Four Other Tools

The exclusion has to be stated once per tool, and each one spells it differently.

find — prune rather than filter, which is the difference between not entering a directory and walking it and discarding the results:

find . \( -name work -o -name csrc -o -name 'run_*' -o -name .git \) -prune -o \
       -name '*.sv' -print

The shape is \( conditions \) -prune -o <what you want> -print. Chapter 14.6 covered it, and the detail worth repeating is that -prune must come before the -o, and the final -print is required because any action switches off the implicit one.

du, when you want to know how big the source is:

du -sk --exclude=work --exclude=csrc . 2>/dev/null   # GNU
du -sk -I work -I csrc .                              # BSD

Different flags on each platform — a command -v-style feature test, or git ls-files | xargs du -ck | tail -1 which sidesteps the question entirely by asking git what the source is.

tar, where getting it wrong means shipping a hundred gigabytes:

tar --exclude='work' --exclude='csrc' --exclude='run_*' -czf src.tgz .

Or, far better in a git repository:

git archive --format=tar.gz -o src.tgz HEAD

git archive ships exactly the tracked files at a commit, which is both correct and reproducible. It is the right answer whenever the question is "send someone the source".

And rsync, which has the useful --filter form:

rsync -a --filter=':- .gitignore' src/ dest/

:- .gitignore tells rsync to read each directory's .gitignore as an exclusion list, which gives it the same behaviour as git grep and ugrep — one declaration, honoured by several tools.

7. The Search Script

Everything above, as the thing to put in your bin:

#!/usr/bin/env bash
# dsearch — search the design, not the build output
#
# usage: dsearch [-a] [-t] <pattern> [path...]
#   -a   include imported IP (default: rtl and tb only)
#   -t   search tracked files with git grep instead of the filesystem
# exit: 0 matches found · 1 none · 2 bad usage
set -o nounset

GREP=${GREP:-/usr/bin/grep}
all=0; tracked=0
while getopts ':at' o; do
  case $o in
    a) all=1 ;;
    t) tracked=1 ;;
    \?) printf 'dsearch: unknown option -%s\n' "$OPTARG" >&2; exit 2 ;;
  esac
done
shift $(( OPTIND - 1 ))

[ $# -ge 1 ] || { printf 'usage: dsearch [-a] [-t] <pattern> [path...]\n' >&2; exit 2; }
pat=$1; shift

exts=(--include='*.sv' --include='*.svh' --include='*.v' --include='*.vh')

if [ "$tracked" -eq 1 ]; then
  git rev-parse --is-inside-work-tree > /dev/null 2>&1 ||
    { printf 'dsearch: -t needs a git repository\n' >&2; exit 2; }
  git grep -n -- "$pat" -- '*.sv' '*.svh' '*.v' '*.vh' ${1+"$@"}
  exit $?
fi

roots=()
if [ $# -gt 0 ]; then roots=("$@")
else
  for d in rtl tb src hdl design verif; do [ -d "$d" ] && roots+=("$d"); done
  [ "$all" -eq 0 ] || for d in ip lib vendor third_party; do [ -d "$d" ] && roots+=("$d"); done
fi

[ "${#roots[@]}" -gt 0 ] || { printf 'dsearch: no source directories found here\n' >&2; exit 2; }

"$GREP" -rn "${exts[@]}" \
  --exclude-dir=work --exclude-dir=csrc --exclude-dir=cov \
  --exclude-dir=simv.daidir --exclude-dir='run_*' --exclude-dir=.git \
  -- "$pat" "${roots[@]}"

$ ./dsearch clk
rtl/axi/axi_master.sv:2:module axi_master (input logic clk, input logic rst_n, output logic [7:0] stat);
rtl/axi/axi_master.sv:3:  always_ff @(posedge clk) if (!rst_n) stat <= 8'h00; else stat <= 8'hA1;
rtl/core/chip_top.sv:2:module chip_top (input logic clk, input logic rst_n, output logic [7:0] status);
...
$ ./dsearch clk | wc -l
12
$ ./dsearch -a clk | wc -l
14
$ ./dsearch -t clk | wc -l
14

Twelve with the default roots, fourteen with IP included, fourteen via git grep — and git grep's fourteen are the same fourteen, which is the two methods agreeing that Chapter 23.3 §3 asked for.

Six decisions:

Both a deny-list and a positive list. The roots are positive, and the --exclude-dir options are belt and braces for the case where somebody has created rtl/work/. Redundant protection is cheap here.

--exclude-dir='run_*' is quoted, so grep receives the pattern rather than whatever the shell expanded it to.

The roots are discovered, trying the six commonest names and keeping the ones that exist — so the script works in a repository that calls its design src/ without being told.

${1+"$@"} in the git grep line, which expands to nothing when there are no extra arguments and to the quoted arguments when there are. Plain "$@" would be correct here too, but the ${1+…} form is the one that is also safe under set -u in older shells — Chapter 8.2's alternate-value expansion.

An empty roots array is exit 2, not an empty search. A search that silently examines nothing and reports no matches is the failure mode Chapter 21.8 §4 is about — it looks exactly like "no matches".

And GREP is pinned and overridable, for §3's measured reason.

Practice

1. Count hits for a common signal name in your repository with no exclusions, then with the generated directories excluded. What is the ratio?
2. Run type -a grep and grep --version. Which grep do you have, and does it honour .gitignore?
3. If it does, prove it: move .gitignore aside, repeat a search, and put it back.
4. Run the registry's --exclude-dir={work,simv.daidir,csrc} form in your tree. Which generated directory does it miss?
5. Run /bin/dash -c 'echo --exclude-dir={a,b}'. Then find a script in your flow with #!/bin/sh and a brace expansion in it.
6. Replace a deny-list search of your own with a positive list. Which directories did you have to name?
7. Add --exclude-dir='run_*' unquoted, in a directory that contains a run_2026_10_01. What does grep actually receive?
8. Compare git grep -n <pat> -- '*.sv' with your best grep invocation. Do they agree? Explain any difference.
9. Write the .gitignore completeness check from §6 and run it. Does it print anything?
10. Build a source tarball two ways — tar --exclude=... and git archive — and compare the sizes.
11. Write dsearch, then run it in a repository whose design directory is called something it does not know. Fix that by passing a path.
12. Make roots empty on purpose and confirm dsearch exits 2 rather than reporting no matches.

Check Your Understanding

1. Why is a stale copy of a source file in a run directory worse than a missing file?
2. What is the structural flaw in --exclude-dir={...}?
3. Why did git grep include ip/, and is that right?
4. The same grep command returned two files and three files on one machine. Why?
5. How was that cause proven rather than assumed?
6. In which direction does the interactive-versus-batch difference fail, and why is that the bad direction?
7. Who expands the braces in --exclude-dir={a,b}, and where does that break?
8. Why is a positive list safer than a deny-list?
9. Which tools honour .gitignore without being asked?
10. In dsearch, why is an empty roots array an error rather than an empty search?

8. Remember

• Generated directories hold copies of your source, often stale, and always valid-looking.
• Measured: 16 hits with nothing excluded, 12 with a positive list — and in a real tree the signal is a few per cent.
• A deny-list excludes what you remembered. --exclude-dir={work,simv.daidir,csrc} still matched run_2026_10_01/.
• Name what you want, not what you do not. A positive list fails safe.
• git grep lets .gitignore decide, which is the one definition of "generated" that the repository maintains for you.
• The same grep command gave 2 or 3 files on one machine: ugrep honours .gitignore, BSD grep does not.
• That difference does not survive leaving your interactive shell, and it fails in the dangerous direction.
• Pin the tool in a script: GREP=${GREP:-/usr/bin/grep}, or use git grep.
• BSD grep does accept --exclude-dir — but the braces are the shell's, and dash does not expand them.
• /bin/sh is dash on Debian and is what make uses, so the brace form breaks there silently.
• --exclude-dir='run_*' accepts a pattern — quote it.
• find needs -prune before -o, and a final -print.
• git archive is the right way to ship source.
• Check that .gitignore is complete; one line there improves git grep, ugrep and rsync at once.
• A search over zero directories reports "no matches". Make it an error.

9. What Is Next

Chapter 23.6 assembles the whole module into a routine: the first hour in a repository you have never seen, in an order that answers the most per command — and where the real documentation turns out to be.