Context, File Lists and Counts

Part I · Module 14 · Chapter 14.5 · Context, File Lists and Counts

Chapter 14.4 made a tree searchable. This chapter is about what to ask it for.

1. Three Questions

A search answers one of three questions, and the default — print every matching line — only suits the first:

Reaching for the default when you wanted one of the others is why people end up reading four hundred lines of output by eye.

2. Context — What Is Around It

$ grep -A2 -n 'ERROR' run.log
3:[00:03] ERROR: timeout after 1203 cycles
4-[00:04] info: retrying
5-[00:05] Error: assertion failed

-A2 prints the match and the two lines after it.

$ grep -B1 -n 'TEST FAILED' run.log
5-[00:05] Error: assertion failed
6:[00:06] TEST FAILED

-B1 prints the line before. And -C does both:

$ grep -C1 -n 'rror' run.log
4-[00:04] info: retrying
5:[00:05] Error: assertion failed
6-[00:06] TEST FAILED

Why this is the flag for log reading

$ grep ERROR run.log
[00:03] ERROR: timeout after 1203 cycles

That tells you a timeout happened. It does not tell you what was running, what it was waiting for, or what happened next — and those are in the surrounding lines.

grep -B5 -A10 'ERROR' run.log

Five lines of what led up to it and ten of the consequences. For a first look at a failing run this is almost always the right command, and it is the one that replaces opening the log in an editor and searching by eye.

A useful asymmetry: errors usually need more lines after than before, because the cause appears in the error and the cascade follows. A deadlock or a resource problem is the reverse — the evidence is in what was happening beforehand.

3. Which Files

$ grep -rl clk_div tree
tree/docs/notes.md
tree/rtl/alu.v
tree/build/gen.v

-l prints only the names of files that contain a match, one per line, with no line content at all.

This changes the shape of the answer completely. "Which modules reference this signal?" is a question about files, and a list of twelve names is readable where four hundred matching lines is not.

$ grep -rL clk_div tree

-L is the inverse — files that contain no match. It answers questions nothing else does:

grep -rL --include='*.sv' 'uvm_component_utils' tb/     # classes missing registration
grep -rL --include='*.v'  'timescale' rtl/              # files with no timescale
grep -L  'Copyright' src/*.c                            # files missing a header

That last shape — "which of these is missing the thing they should all have?" — is a genuinely common review task and -L is the whole of it.

4. How Many

$ grep -c FAIL results.txt
12

-c gives a count instead of the lines, and Chapter 14.1 covered the single-file case. Recursively it reports per file:

$ grep -rc clk_div tree
tree/docs/notes.md:1
tree/rtl/alu.v:1
tree/build/gen.v:1

Totals

-c counts per file and grep has no flag for a grand total. Two ways:

grep -rh --include='*.v' assert . | wc -l            # count the matching lines
grep -roh --include='*.v' assert . | wc -l           # count the occurrences

-h suppresses the filename prefix, which matters here: without it each line carries a path and wc -l still works, but anything doing arithmetic downstream would be parsing the wrong thing.

The difference between those two is Chapter 14.1's: the first counts lines, the second counts matches, and they differ whenever a line matches twice.

5. Shaping the Output

Four more flags that change the form rather than the content.

grep -rn --color=always pattern . | less -R    # keep colour through a pager
grep -rn --no-filename pattern .               # same as -h
grep -rn --with-filename pattern file          # same as -H, forces it for one file
grep -rn -Z pattern .                          # NUL after each filename

--color=always plus less -R is worth knowing. grep turns colour off when its output is not a terminal, so a plain grep --color ... | less loses the highlighting; forcing it and telling less to interpret the codes keeps it.

-Z pairs with Chapter 14.7:

grep -rlZ 'old_macro' . | xargs -0 sed -i.bak 's/old_macro/new_macro/g'

-lZ emits the file names separated by null bytes, and xargs -0 reads them safely — which is the only correct way to do this when any file name might contain a space.

6. Choosing the Shape

7. Putting It Together

A first look at a failed regression, in four commands:

$ grep -rl 'TEST FAILED' runs/*/sim.log | wc -l
7

Seven runs failed. Which ones:

$ grep -rl 'TEST FAILED' runs/*/sim.log
runs/run_alu_seed3/sim.log
runs/run_alu_seed11/sim.log
...

What the failures were, grouped — the Chapter 13.4 pipeline:

$ grep -rho 'TEST FAILED: .*' runs/*/sim.log | sed 's/[0-9][0-9]*/N/g' | sort | uniq -c | sort -rn
   4 TEST FAILED: timeout after N cycles
   3 TEST FAILED: assertion at alu.sv:N

And the context around one of them:

$ grep -B5 -A10 'TEST FAILED' runs/run_alu_seed3/sim.log

Four questions, four different flags, and only the last one prints log lines. That progression — how many, which, what kind, then why — is how an investigation actually goes, and each step narrows the next.

# which test files have no assertions at all
grep -rL --include='*.sv' 'assert' tb/

# the files most worth reviewing, by assertion density
grep -rc --include='*.sv' 'assert' tb/ | grep -v ':0$' | sort -t: -k2,2 -rn | head

# every file mentioning a macro, ready to be edited
grep -rlZ --include='*.v' 'OLD_MACRO' . | xargs -0 sed -i.bak 's/OLD_MACRO/NEW_MACRO/g'

Practice

mkdir -p /tmp/ctx/{rtl,tb} && cd /tmp/ctx
cat > run.log <<'EOF'
[00:01] starting simulation
[00:02] WARNING: clock divider not set
[00:03] ERROR: timeout after 1203 cycles
[00:04] info: retrying
[00:05] Error: assertion failed
[00:06] TEST FAILED
EOF
printf 'module alu;\n  assert (1);\n  assert (2);\n  assert (3);\nendmodule\n' > rtl/alu.v
printf 'module fifo;\nendmodule\n' > rtl/fifo.v
printf 'class t1; `uvm_component_utils(t1)\nendclass\n' > tb/t1.sv
printf 'class t2;\nendclass\n' > tb/t2.sv

1 — Context, three ways.

grep -A2 -n ERROR run.log
grep -B1 -n 'TEST FAILED' run.log
grep -C1 -n rror run.log

2 — Read the separators.

grep -C1 -n rror run.log

Note which lines have a colon and which a hyphen.

3 — The log-reading command.

grep -B2 -A3 'ERROR' run.log

4 — Which files.

grep -rl assert .
grep -rL assert rtl/

The second finds the module with no assertions.

5 — Find what is missing.

grep -rL --include='*.sv' 'uvm_component_utils' tb/

6 — Count, and remove the zeros.

grep -rc --include='*.v' assert .
grep -rc --include='*.v' assert . | grep -v ':0$'

7 — Rank by count.

grep -rc --include='*.v' assert . | grep -v ':0$' | sort -t: -k2,2 -rn

8 — Lines versus occurrences.

printf 'assert assert assert\n' > rtl/triple.v
grep -rc  --include='*.v' assert . | grep -v ':0$'
grep -rho --include='*.v' assert . | wc -l

9 — Keep colour through a pager.

grep -rn --color=always assert . | less -R

Press q.

10 — Clean up. cd /tmp && rm -rf /tmp/ctx

Check Your Understanding

1. What do the colon and hyphen after line numbers mean in context output?

2. Which flags would you use for a first look at a failing log, and why that asymmetry?

3. What question does -L answer that nothing else does?

4. grep -rc pattern . produces mostly useless output on a real tree. Why, and what is the fix?

5. Why does grep -rc ... | sort -k2 -rn need -t:?

6. Why is -l faster than plain -r on a large tree?

8. Remember

• Three questions, three shapes of answer: context, file names, counts. The default suits none of them well.
• -A, -B, -C print surrounding lines, and a matching line alone usually says too little.
• A colon marks a match; a hyphen marks context. Groups are separated by a -- line.
• -B5 -A10 is the first command to run on a failing log. More after than before, because errors cascade.
• -l lists the files that match, which changes a four-hundred-line answer into a twelve-name one.
• -L lists the files that do not — the only tool for "which of these is missing the thing they should all have?"
• -l is faster, because it stops at the first match in each file.
• -rc reports zeros too. Filter with grep -v ':0$'.
• Rank counts with sort -t: -k2,2 -rn — the colon separator and the numeric flag both matter.
• -h suppresses the filename prefix when you are counting or feeding another tool.
• --color=always | less -R keeps highlighting through a pager.
• -lZ with xargs -0 is the safe way to feed matched file names to another command.
• A declaration is found by anchoring, not by hoping: Chapter 23.4 walks a hierarchy path back to a file and a line using exactly these anchored patterns.

Both halves of this module so far have asked "which files contain something". The next chapter asks the other question entirely — which files exist, matched by name, age, size or type.