grep: Searching Text for a Pattern

Part I · Module 14 · Chapter 14.1 · grep: Searching Text for a Pattern

Module 13 transformed streams. This module finds the text in the first place, and Chapter 14.8 closes it by settling which of the two modules' tools answers which question.

1. Matching Lines

$ grep timeout run.log
[00:03] ERROR: timeout after 1203 cycles

grep PATTERN FILE prints every line containing the pattern. That is the whole core of it, and the rest of this chapter is the options that make the answer more useful than a list of lines.

grep reads standard input when given no file, so it works as a pipeline stage without any special arrangement:

grep ERROR run.log
ps -ef | grep vcs
cat *.log | grep ERROR          # works, but see 13.9 — grep opens files itself

2. The Six Options

-n — where is it?

$ grep -n timeout run.log
3:[00:03] ERROR: timeout after 1203 cycles

Line 3. On a forty-thousand-line log that number is the difference between "it is in here somewhere" and being able to open the file at the right place. Use -n by default when reading a log by hand.

-i — case, which machine output is inconsistent about

$ grep -c ERROR run.log
1

$ grep -ic error run.log
2

One match versus two, on the same file, because the log contains both ERROR: and Error:. That is extremely common — different components of the same tool disagree about capitalisation — and a count taken without -i is quietly low.

-v — everything except

$ grep -vc info run.log
5

Invert the match. It is how you filter noise out rather than filtering signal in:

grep -v '^#' config.txt          # drop comments
grep -v '^$' config.txt          # drop blank lines
./tool 2>&1 | grep -v '^INFO'    # drop the chatter, keep everything else

Chaining two is the common idiom for "neither of these":

grep -v '^#' config.txt | grep -v '^$'

-c — how many?

$ grep -c FAIL results.txt
12

One number instead of twelve lines. Prefer it to grep … | wc -l: one process instead of two, and Chapter 12.8's pipeline-status problem does not arise.

The one difference worth knowing: grep -c counts matching lines, not matches. A line containing the word three times counts once. For occurrences, use grep -o PATTERN file | wc -l.

-w — whole words

$ cat w.txt
clk
clk_div
my_clk

$ grep -c clk w.txt
3

$ grep -wc clk w.txt
1

Three matches become one. -w requires the match to be bounded by non-word characters, so clk_div and my_clk no longer count — underscore counts as a word character, which is exactly right for identifiers.

This is the most under-used option in the list. Searching an RTL tree for a signal called clk without -w returns every clk_div, clk_en, pclk and clk_gate in the design.

-x is the stricter version — the whole line must match:

$ grep -xc clk w.txt
1

-q — just answer the question

$ grep -q timeout run.log && echo "the run timed out"
the run timed out

That form — the search command used directly as a condition — is the idiom Chapter 16.7 is built on, and it is measurably cheaper than capturing the output and testing it afterwards.

No output; the exit status is the answer. It also stops at the first match rather than reading the rest of the file, which on a large log is a real saving.

This is the form for a script. Chapter 11.3 used it repeatedly, and it is why: grep -q asks a yes-or-no question and if reads the answer.

3. The Exit Status Is the Point

$ grep -q timeout run.log; echo "found     -> $?"
found     -> 0
$ grep -q zzzzz run.log;   echo "not found -> $?"
not found -> 1
$ grep -q x nofile 2>/dev/null; echo "bad file  -> $?"
bad file  -> 2

Chapter 11.1 introduced this three-way split and it is worth restating here, because it is what makes grep a predicate rather than a report:

1 is an answer. The file was searched successfully and contained nothing. That distinction is why grep reserves 2 for actual problems, and why a careful script can tell "no errors in the log" from "could not open the log" — which are very different situations and look identical if you only check for non-zero.

if grep -q 'TEST PASSED' sim.log; then
  echo "pass"
elif [ $? -eq 1 ]; then
  echo "fail — no pass line found"
else
  echo "could not read sim.log" >&2
fi

4. Several Files

$ grep -c TEST run.log run2.log
run.log:1
run2.log:1

With two or more files, every line is prefixed with the filename. With one file it is not. That is convenient when reading and a nuisance when parsing, because the output shape changes depending on how many files matched the glob.

Two flags control it explicitly:

$ grep -hc TEST run.log run2.log      # -h: never show the name
1
1

$ grep -Hn TEST run.log               # -H: always show it, even for one file
run.log:6:[00:06] TEST FAILED

Use -H in a script. A pipeline written against two files and then run against one silently loses a column, and -H makes the shape constant.

5. Two More Worth Knowing

-o — just the matching part

$ grep -o 'after [0-9]*' run.log
after 1203

Prints only what matched, not the whole line. It is how you extract a value rather than find a line, and it pairs naturally with the counting pipeline from Chapter 13.4:

grep -o 'ERROR: [a-z ]*' *.log | sort | uniq -c | sort -rn

One match per output line, so a line matching three times produces three lines — which is what makes grep -o … | wc -l count occurrences rather than lines.

-m — stop early

$ grep -m1 -n 'E' run.log
3:[00:03] ERROR: timeout after 1203 cycles

Stops after the first match. On a huge file this is grep … | head -1 without the pipe — and therefore without Chapter 12.8's SIGPIPE complication under pipefail.

6. The Full Working Set

The last three need a word each.

7. When the Pattern Should Not Be a Pattern

$ printf 'a.c\nabc\n' > dots.txt

$ grep -c 'a.c' dots.txt
2

$ grep -Fc 'a.c' dots.txt
1

-F turns off the pattern language and matches the characters literally. The dot stops meaning "any character".

Use it whenever the thing you are searching for is data rather than a pattern — a file path, a version string, an IP address, a value from a variable:

grep -F "$user_input" file          # cannot be a pattern injection
grep -F '/opt/tools/2024.09' setup.sh
grep -F 'a[0]' rtl.v                # brackets are literal

That first line matters for the same reason Chapter 9.4 warned against eval: a value you did not write should not be interpreted as code, and a regular expression is code.

-e and -f

grep -e '-v' file                   # search FOR "-v", not the option
grep -e ERROR -e WARN run.log       # either pattern
grep -f patterns.txt run.log        # one pattern per line, from a file

-e is how you search for something beginning with a hyphen, which grep would otherwise read as an option — the same -- problem from Chapter 9.6, with its own flag.

-f with a file of patterns is the efficient way to search for many strings at once; combined with -F it is substantially faster than a long alternation.

8. Which grep Are You Running?

Worth checking once, because it affects what the rest of this module says.

$ grep --version
ugrep 7.8.4 aarch64-apple-macosx

$ /usr/bin/grep --version
grep (BSD grep, GNU compatible) 2.6.0-FreeBSD

On the machine these chapters were verified against, grep was not the system grep at all — a faster drop-in replacement was earlier on PATH. Every measured figure in this module was therefore re-run against /usr/bin/grep, and all of them were identical.

9. In Practice

# did this run pass?
grep -q 'TEST PASSED' sim.log && echo pass || echo fail

# how many failures, case-insensitively
grep -ic 'fail' results.txt

# the config without comments or blanks
grep -v '^#' setup.cfg | grep -v '^$'

# find a signal without matching every name containing it
grep -wn 'clk' rtl/alu.v

# extract the cycle counts and total them
grep -o 'after [0-9]*' run.log | grep -o '[0-9]*' | awk '{s+=$1} END {print s}'

# search for a literal path, safely
grep -Fn '/opt/tools/2024.09' setup.sh

# first match only, on a huge file
grep -m1 -n 'BEGIN SIMULATION' enormous.log

The fourth is the one to internalise. grep -w on an RTL tree is the difference between four matches and four hundred.

Practice

mkdir -p /tmp/gr && cd /tmp/gr
cat > run.log <<'EOF'
[00:01] starting simulation
[00:02] WARNING: clock divider not set
[00:03] ERROR: timeout after 1203 cycles
[00:04] info: retrying
[00:05] Error: assertion failed
[00:06] TEST FAILED
EOF
printf 'clk\nclk_div\nmy_clk\n' > w.txt
printf 'a.c\nabc\n' > dots.txt
cp run.log run2.log

1 — The six options.

grep -n timeout run.log
grep -c ERROR run.log; grep -ic error run.log
grep -vc info run.log
grep -c clk w.txt; grep -wc clk w.txt; grep -xc clk w.txt
grep -q timeout run.log && echo "found"

2 — Read the three exit statuses.

grep -q timeout run.log;        echo "found     -> $?"
grep -q zzzzz  run.log;         echo "not found -> $?"
grep -q x nofile 2>/dev/null;   echo "bad file  -> $?"

3 — Make set -e trip, then fix it.

( set -e; grep zzzzz run.log; echo "NOT REACHED" ); echo "  exited $?"
( set -e; grep zzzzz run.log || true; echo "reached" )

4 — Watch the filename prefix appear.

grep -c TEST run.log
grep -c TEST run.log run2.log
grep -hc TEST run.log run2.log
grep -Hn TEST run.log

5 — Extract rather than find.

grep -o 'after [0-9]*' run.log
grep -o '[0-9][0-9]*' run.log | head -4

6 — Turn the pattern off.

grep -c  'a.c' dots.txt
grep -Fc 'a.c' dots.txt

7 — Search for something starting with a hyphen.

printf 'use -v for verbose\n' > opts.txt
grep '-v' opts.txt
grep -e '-v' opts.txt

The first may be read as an option.

8 — Find out which grep you have.

type -a grep
grep --version | head -1
/usr/bin/grep --version | head -1

9 — Clean up. cd /tmp && rm -rf /tmp/gr

Check Your Understanding

1. grep -c ERROR run.log says 1 but you can see two error lines. Why?

2. What do exit statuses 0, 1 and 2 each mean, and why does the distinction matter?

3. A script under set -e stops at a grep on a clean log. Why, and what is the fix?

4. You search an RTL tree for clk and get four hundred matches. Which flag?

5. grep -c PATTERN file returns 5 but you expected 8 occurrences. What is it counting?

6. Why should grep -F be used when the pattern comes from a variable?

Chapter 29.1 puts this to work on a four-thousand-line simulation log, including the line whose leading dashes stop grep from running at all.

10. Remember

• grep PATTERN FILE prints matching lines, and reads standard input when given no file.
• Six options cover most work: -n line numbers, -i ignore case, -v invert, -c count, -w whole words, -q status only.
• -w is the most under-used. On an identifier it is the difference between four matches and four hundred.
• The exit status is three-way: 0 matched, 1 matched nothing — a result, 2 a real error.
• Status 1 is what trips set -e on a clean log. || true is the fix.
• -q stops at the first match and is the form for a script.
• -c counts matching lines, not matches. Use -o … | wc -l for occurrences.
• The filename prefix appears only with two or more files. Use -H in a script so the shape is constant.
• -o prints just the matching part, which turns grep into an extractor.
• -m1 is | head -1 without the pipe, and without the SIGPIPE complication.
• -F turns the pattern language off — essential when the pattern is data rather than a pattern.
• -e lets you search for something starting with a hyphen.
• Check which grep you have. ripgrep and ugrep are common on PATH ahead of the real one, and the edge options differ.

Every pattern in this chapter has been literal text. The next one is the language they were quietly part of.