awk vs grep

Part I · Module 1 · Chapter 1.3 · awk vs grep

1. The Question This Chapter Answers

If you have used a terminal at all, you have probably used grep. It finds things in files, and it is usually the first tool anyone reaches for. Chapter 1.1 then introduced awk, which also finds things in files. A reasonable reader is left wondering whether awk is just a more complicated grep.

It is not, and the difference is worth getting exactly right, because the two tools fail differently. Choosing awk when grep would do costs you a slightly longer command. Choosing grep when you needed awk can cost you a wrong number that nothing complains about.

One sentence, which the rest of the chapter earns:

grep decides which lines you keep. awk decides which part of a line you keep, and what you compute from it.

Everything else follows from that.

2. What grep Actually Does

Start with a small file. Four test names and their results — the same shape Chapter 1.1 used, so nothing here is new input to learn.

test_a PASS
test_b FAIL
test_c PASS
test_d FAIL

grep's job is to read lines and print the ones that match a pattern:

grep 'FAIL' results.txt

test_b FAIL
test_d FAIL

That is the whole model: for each line, does this pattern appear anywhere in it, and if so, print the line.

Before any comparison, it is worth seeing how much grep can do inside that model, because the usual version of this chapter undersells it badly. grep can count the matching lines:

grep -c 'FAIL' results.txt

2

It can invert the test and show you everything that did not match:

grep -v 'PASS' results.txt

test_b FAIL
test_d FAIL

It can require that the pattern be a whole word rather than part of one, with -w. It can number the lines it prints with -n. It can show lines around a match with -A and -B, which is how you read the context of an error. It understands regular expressions, and -E turns on the extended syntax. And with -o it prints only the matching text rather than the whole line:

grep -o '9[0-9]' scores.txt

Given the file of names and scores used in section 6, that prints 91 — the matching text on its own, not the line it came from.

So grep is not limited to printing whole lines, it can count, it can extract, and it does regular expressions. Any comparison that pretends otherwise is one you should distrust.

3. What awk Does Differently

awk reads lines too. The difference is what it has already done to the line by the time your instruction runs.

Chapter 1.1 covered this, so one short reminder is enough: awk splits each line into fields on whitespace, and gives them names. $1 is the first field, $2 the second, $NF the last, and $0 the whole line. That splitting is free and automatic.

This is why the two tools answer different questions about the same file:

awk '$2 == "FAIL" { print $1 }' results.txt

test_b
test_d

Compare that with the grep from section 2. grep printed test_b FAIL; awk printed test_b. Three distinct things changed:

• grep asked "does FAIL appear anywhere in this line". awk asked "is the second field exactly FAIL".
• grep printed the whole line, because the line is its unit. awk printed field 1, because it can address parts.
• grep compared text. awk can compare text or numbers, which section 8 is about.

4. Where Text Matching Goes Wrong: Position

So far the two tools agreed about which rows mattered. Here is the first case where they genuinely disagree, and it is the one that costs engineers real time.

A UVM simulation log carries a severity in a fixed column. But a message is free text, and free text can mention anything — including the name of a severity:

100 UVM_INFO Test started
250 UVM_INFO Checking UVM_ERROR recovery
420 UVM_ERROR Data mismatch
500 UVM_INFO Test finished

Ask grep for the errors:

grep 'UVM_ERROR' sim.log

250 UVM_INFO Checking UVM_ERROR recovery
420 UVM_ERROR Data mismatch

Two lines, but only one is an error. The first is an informational message that happens to contain the text.

The natural reaction is to reach for -w, which requires the pattern to be a whole word. It does not help:

grep -w 'UVM_ERROR' sim.log

250 UVM_INFO Checking UVM_ERROR recovery
420 UVM_ERROR Data mismatch

Still two lines. UVM_ERROR is a whole word in that message, so -w has nothing to object to. The problem was never about word boundaries. It is that grep has no way to say which column the word must be in, because grep has no columns.

awk does:

awk '$2 == "UVM_ERROR" { print }' sim.log

420 UVM_ERROR Data mismatch

One line, correctly. $2 names the severity column, and == demands the whole field be exactly that value.

5. Counting: Lines Against Records

The disagreement in section 4 gets worse when you stop reading the output and start reporting a number.

grep -c 'UVM_ERROR' sim.log

2

awk '$2 == "UVM_ERROR" { n++ } END { print n + 0 }' sim.log

1

Two against one, on a four-line file. Now imagine the log is 200,000 lines and the number lands in a nightly email. Nobody reads 200,000 lines to check it. The grep figure is simply wrong, and it is wrong in the direction that wastes an engineer's morning.

There is a second, subtler point about grep -c. It counts matching lines, not matches. A line containing the pattern three times counts once. If you want matches, you have to say so differently — grep -o then count the output lines. awk's counter has the same property for the same reason: n++ runs once per record, not once per occurrence. Neither tool is wrong; both are counting records, and you have to know that is what the number means.

6. Extracting Columns

Section 3 showed awk printing $1. It is worth being precise about what grep can and cannot do here, because -o is genuinely an extraction option.

Here is a file of names and scores:

alice 72
bob 91
carol 58
dave 9
erin 100

grep -o prints the matching text instead of the line:

grep -o '9[0-9]' scores.txt

91

That really did extract a value from inside a line. But look at what you had to do to get it: you described the value by its shape — a 9 followed by one digit. It found 91 and nothing else, because 9 and 100 do not have that shape.

awk addresses the column instead:

awk '{ print $2 }' scores.txt

72
91
58
9
100

Every score, regardless of how many digits it has, because $2 means "the second field" and not "text that looks like this".

That is the distinction. grep -o extracts text by pattern. awk extracts by position. When the thing you want is "the second column", position is the correct description of it and a pattern is a guess that happens to work on your sample.

Printing two columns shows the same point again:

awk '{ print $1, $2 }' scores.txt

alice 72
bob 91
carol 58
dave 9
erin 100

The comma puts a space between them. There is no grep option that reorders or recombines columns, because grep does not know they exist.

7. The Line grep Cannot Cross: Arithmetic

Everything so far has been about text. This section is the one place where the boundary is absolute rather than a matter of convenience.

Ask for the scores above 80. awk compares numbers:

awk '$2 > 80 { print $1, $2 }' scores.txt

bob 91
erin 100

Correct: 91 and 100 are the two scores above 80.

Now try to ask grep the same question. grep has no numeric comparison, so the only available move is to describe the answer as text. A first attempt might be to look for a 9:

grep '9' scores.txt

bob 91
dave 9

This is wrong in both directions. It includes dave 9, whose score is 9 and nowhere near 80. And it misses erin 100, whose score is above 80 but contains no 9. No cleverer pattern fixes this in general, because "greater than 80" is not a statement about which characters appear — it is arithmetic, and grep does not do arithmetic.

The same thing in engineering form. Chapter 1.2 introduced slack: a negative value means the path failed its timing check.

path_a -0.12
path_b 0.08
path_c -1.20

awk '$2 + 0 < 0 { print $1, $2 }' slack.txt

path_a -0.12
path_c -1.20

Both violations. Now the text attempt — a negative number starts with a minus sign, so match that:

grep -- '-0\.' slack.txt

path_a -0.12

One row, and it is the less severe of the two violations. path_c -1.20 is missed because the pattern assumed the number starts -0., and -1.20 does not. The worst-timed path in the file is invisible, with no error and exit status 0.

8. A Semiconductor Example: Triaging One Log

Put it together on the log from section 4. A realistic first-pass triage asks three things: did anything fail, how many real errors, and what were they.

grep is the right tool for the first question. You want to know whether the text appears at all, you do not care about position yet, and it is the shortest thing to type:

grep -c 'UVM_ERROR' sim.log

2

Non-zero, so something is worth looking at. That is a perfectly good use of grep — as a screen, not as a measurement.

For the number that goes in a report, the severity column is what you mean:

awk '$2 == "UVM_ERROR" { n++ } END { print n + 0 }' sim.log

1

And to see the errors with their simulation time, so you know when in the test it happened:

awk '$2 == "UVM_ERROR" { print $1, $3, $4 }' sim.log

420 Data mismatch

$1 is the time, $3 and $4 are the first two words of the message. That last command is a good illustration of awk's limit as well as its reach: it printed the message only because this message is two words long. Addressing "everything after field 2" needs tools from Module 3.

9. Choosing the Simpler Tool

The honest default is: if grep answers your question, use grep. It is shorter, every engineer reads it instantly, and there is no field-counting to get wrong.

grep is the better choice when:

• You want to know whether something appears at all.
• You want to see the matching lines to read them yourself.
• You want surrounding context, with -A or -B.
• The thing you are looking for is distinctive text, not a column.
• You are screening many files for which ones are worth opening.

awk is the better choice when:

• The question is about a specific column, as in section 4.
• You need to print part of a line, or reorder columns.
• The question involves arithmetic — a comparison with a number, a total, a worst case.
• You need a count that respects position rather than text, as in section 5.

And they combine. A pipeline can let grep do the cheap screening and awk do the field work:

grep 'UVM_ERROR' sim.log | awk '$2 == "UVM_ERROR" { print $1 }'

420

That is a legitimate shape, and it is also worth saying that here it is redundant: the awk program alone would do the same job, and the grep stage only narrows what awk reads. Chapter 1.2's pipeline section and Module 12 take this up. The rule of thumb is to add a grep stage when it genuinely removes most of the input, and not out of habit.

10. Common Mistakes

Using grep for a numeric question. The subject of section 7, and the one that produces wrong numbers rather than errors. If the words "greater than", "negative", "total" or "worst" are in your question, grep is not the tool.

Trusting -w to mean "this column". Section 4. -w constrains word boundaries, not position. Only a field test constrains position.

Reading grep -c as a count of occurrences. It counts matching lines. A line with three matches contributes one.

Forgetting -- before a pattern that starts with a minus. grep '-0\.' makes grep try to read -0\. as options. Write grep -- '-0\.'.

Quoting the awk program with double quotes. Chapter 1.1 covered this and it remains the most common beginner failure. In double quotes the shell expands $2 to nothing before awk sees it, so awk "{ print $2 }" prints whole lines and reports no error. Use single quotes.

Counting fields from zero. $1 is the first field. $0 is the whole record, not the first field.

11. Portability

The commands here were verified with the awk and grep that ship with macOS — awk version 20200816, which is POSIX awk, and BSD grep 2.6.0. Every grep command was also run under a second grep implementation and produced identical output.

Two cautions for work on other machines. The options used here — -c, -v, -o, -w, -n, -E, -A, -B — are widely available, but -o, -A and -B are extensions rather than the oldest POSIX core, so a minimal environment such as a container built on BusyBox may differ. And error message wording differs between implementations, so if you are matching on an error string rather than an exit status, that is a portability hazard in itself. Chapter 8.6 covers awk's own POSIX-versus-gawk split properly.

12. Exercises

Create the files as shown and compare your output against the expected result.

Exercise 1 — Which lines, and which part

Using results.txt:

test_a PASS
test_b FAIL
test_c PASS
test_d FAIL

Print the names of the passing tests — the names only, not the whole line.

awk '$2 == "PASS" { print $1 }' results.txt

test_a
test_c

grep could select those lines but could not strip the result column off them, because the line is its unit of output.

Exercise 2 — Modify a demonstrated command

Using scores.txt:

alice 72
bob 91
carol 58
dave 9
erin 100

Print the names of everyone scoring below 60.

awk '$2 < 60 { print $1 }' scores.txt

carol
dave

One character changed from section 7. Note that dave appears correctly here — a text pattern looking for a leading 5 would have found carol and missed dave entirely.

Exercise 3 — Debugging a wrong count

A colleague reports three errors in this log and you believe there is one. Their command was grep -c 'UVM_ERROR' sim.log. Explain the disagreement, then produce the correct count.

100 UVM_INFO Test started
250 UVM_INFO Checking UVM_ERROR recovery
420 UVM_ERROR Data mismatch
500 UVM_INFO Test finished

awk '$2 == "UVM_ERROR" { n++ } END { print n + 0 }' sim.log

1

grep counts every line containing the text, and the INFO message on line 2 contains it. So grep reports 2 where the severity column contains it once. If your colleague reported three, their log has another line mentioning the text — which is itself worth looking at, and is a good reason to print the matches rather than only count them.

Exercise 4 — An engineering decision

You have 400 simulation logs in one directory. You want to know which files are worth opening, then how many real errors are in those files.

Name the right tool for each step and say why.

The first step is grep. You are asking a yes-or-no question about whole files, you do not care about position, and the answer is a list of filenames — grep -l 'UVM_ERROR' across the directory does exactly that, and it is the cheapest way to discard the files that are clean.

The second step is awk, because "real errors" means the severity column rather than the text, as section 5 showed. Reaching for grep again at this step is how the number ends up wrong.

The general shape: grep to decide where to look, awk to measure what you found. Running awk across all 400 files directly is also correct and often simpler — Chapter 7.5 covers multi-file processing, and Module 12 covers building this as a pipeline.

Exercise 5 — Challenge: pick the tool

For each question, say whether grep or awk is the right first choice, and why. Answers below.

1. Does this log mention a timeout anywhere?
2. Which paths have slack worse than minus 0.5?
3. Show me the three lines before each fatal error.
4. What is the total cell count in this report?
5. Which tests are recorded as FAIL?

Answers. 1 — grep: an existence question about distinctive text, no position or arithmetic involved. 2 — awk: a numeric comparison; no text pattern expresses "worse than minus 0.5". 3 — grep, with -B 3; awk has no equivalent of context lines at this stage of the curriculum. 4 — awk: a total is arithmetic. 5 — either, and grep is fine if you want to read the lines, but awk if FAIL could appear in another column or inside a message, or if you want the names alone.

If you picked awk for question 1 or 3, re-read section 9. Reaching for the more capable tool by default is a real habit worth resisting.

13. Summary

• grep decides which lines you keep. awk decides which part of a line you keep, and what it computes. That is the whole distinction.
• grep is more capable than the usual comparison admits. It counts with -c, inverts with -v, extracts matching text with -o, matches whole words with -w, shows context with -A and -B, and understands regular expressions.
• What grep has no notion of is field position. -w constrains word boundaries, not columns, which is why an INFO message mentioning UVM_ERROR defeats both.
• grep cannot do arithmetic. grep '9' for scores above 80 included a 9 and missed a 100; grep -- '-0\.' for negative slack missed the worst path in the file. Both returned plausible output and exit status 0.
• grep -c counts matching lines, not matches, and awk's counter counts records. Know which your number is.
• If grep answers your question, use grep. It is shorter and instantly readable. Escalate to awk for a column, a part of a line, or a quantity.
• The failure mode is a wrong number, not an error. This is why the tool choice is an engineering decision rather than a style preference.

You can now tell the two tools apart by the question rather than by habit, which is the judgement the rest of Part I keeps relying on.

That leaves a third command that often appears in the same breath as these two. sed also reads text line by line and also transforms it — but what it produces is text of the same shape, not values extracted from it. Chapter 1.4 — awk vs sed draws that boundary, and it is a different one from this chapter's.