Testing a Command Instead of a Value

Part I · Module 16 · Chapter 16.7 · Testing a Command Instead of a Value

Two ways to ask whether a log contains an error.

# one
out=$(grep "UVM_ERROR" run.log)
if [ -n "$out" ]; then
  echo "errors present"
fi

# two
if grep -q "UVM_ERROR" run.log; then
  echo "errors present"
fi

Both work. The second is better, and not merely because it is shorter — the first does measurably more work, and the gap can be unbounded.

1. What the Capture Costs

$ printf 'start\nUVM_ERROR : mismatch at 120ns\nUVM_ERROR : mismatch at 300ns\nTEST DONE\n' > run.log

$ out=$(grep "UVM_ERROR" run.log)
$ echo "captured ${#out} characters"
captured 59 characters

Fifty-nine characters, built in memory, never read. On a four-line log that is nothing. On a real simulation log with ten thousand error lines it is a multi-megabyte string, assembled so that a single -n test can ask whether it is empty.

And the work does not stop at the allocation. grep without -q must read the entire file, because it has to print every match. grep -q is permitted to stop at the first one — and that is not a small optimisation.

2. The Measurement

Timing a 200,000-line file gives you noise. So here is a difference that cannot be argued with — an input that never ends:

$ time (yes ERROR | grep -q ERROR)

real    0m0.012s
user    0m0.002s
sys     0m0.007s

yes emits the word ERROR forever. grep -q found a match on the first line, stopped reading, and exited. Four milliseconds.

Now the counting version:

$ yes ERROR | grep -c ERROR
^C

It never finishes. It cannot: a count is not known until the input ends, and this input does not end. Left running for three seconds it had printed nothing at all, and it would still be running next week.

That is the difference between "does this contain X" and "how many X are there", made absolute. -q asks the question you actually have. A count is a different, strictly harder question, and asking it to answer a yes/no costs you everything between the first match and the end of the input.

The same argument applies to the capture form, which also has to reach the end:

if [ "$(grep -c UVM_ERROR run.log)" -gt 0 ]; then   # reads the whole file
if grep -q UVM_ERROR run.log; then                  # may stop on line 1

Both are correct. Only one of them is free.

3. grep as a Condition

The form, with its negation:

$ if grep -q "UVM_ERROR" run.log; then echo "errors present"; fi
errors present

$ if ! grep -q "UVM_FATAL" run.log; then echo "no fatals"; fi
no fatals

-q means quiet: find it, print nothing, report the answer in the status. It is the single most useful flag in this chapter.

Two things to remember about it, both from earlier modules:

-q does not silence errors. It suppresses matches, not diagnostics:

$ if grep -q "x" /no/such/file; then echo found; fi
grep: /no/such/file: No such file or directory

That is stderr, and Module 12's rules apply: add 2>/dev/null if you want it gone — but think first, because Chapter 16.3 showed that a missing file makes grep exit 2, and a condition cannot tell 1 from 2. If the difference matters, capture $? and use a case.

A pipeline works as a condition, and its status is the last command's:

$ if grep "UVM_ERROR" run.log | grep -q "300ns"; then
>   echo "an error at 300ns"
> fi
an error at 300ns

That is Chapter 12.8's pipeline status rule, used deliberately: the first grep's status is discarded and only the final -q decides. Which is what you want here, and a trap anywhere you needed the first command's failure to matter.

4. command -v: Is the Tool Even Installed

The standard guard at the top of any script that shells out to a tool:

$ if command -v grep > /dev/null 2>&1; then echo "grep available"; fi
grep available

$ if ! command -v vcs > /dev/null 2>&1; then echo "vcs not installed"; fi
vcs not installed

command -v prints where a command would be found and returns 0, or prints nothing and returns 1:

$ command -v grep
/usr/bin/grep
$ command -v cd
cd
$ command -v nosuchtool
$ echo $?
1

Note cd: a builtin has no path, so command -v just echoes the name. That is correct and it is the reason for > /dev/null — you are using this for the status, not the output.

5. Other Commands Worth Using as Conditions

diff -q — are two files the same:

$ printf 'a\nb\n' > x; printf 'a\nb\n' > y; printf 'a\nc\n' > z

$ if diff -q x y > /dev/null; then echo "identical"; fi
identical

$ if ! diff -q x z > /dev/null; then echo "they differ"; fi
they differ

diff returns 0 for identical, 1 for different and 2 for trouble — the same three-way shape as grep, and the reason a golden-output check is one line rather than a hash comparison.

mkdir — create a directory, and find out whether you were the one who created it:

$ if mkdir newdir 2>/dev/null; then echo "we created it"; else echo "already existed"; fi
we created it
$ if mkdir newdir 2>/dev/null; then echo "we created it"; else echo "already existed"; fi
already existed

This is better than [ -d newdir ] || mkdir newdir and the reason is Chapter 16.4's closing callout. mkdir is atomic: the check and the creation are one operation inside the kernel, so two scripts racing for the same directory cannot both believe they created it. The test-then-create version has a window between the test and the mkdir in which the other script can win, and then one of them fails for a reason it was specifically written to avoid. This is the standard way to take a lock in a shell script.

kill -0 — does a process exist:

$ if kill -0 $$ 2>/dev/null; then echo "I exist"; fi
I exist
$ if ! kill -0 999999 2>/dev/null; then echo "no such process"; fi
no such process

Chapter 7.6's signal zero: it sends nothing and performs only the permission-and-existence check. Chapter 16.5's lock-file script used exactly this to tell a stale lock from a live one.

Assorted others you will meet:

The last one is the hinge into Chapter 17.5. read returns 0 while it successfully read a line and non-zero at end of input, which is precisely what makes while IFS= read -r line the canonical way to walk a file — a condition that is a command, driving a loop.

6. When a Bracket Really Is Better

The argument has limits, and knowing them is the difference between idiom and dogma.

When you need the output anyway. Already covered: pay for the count once and test it.

When the command is expensive and you need the answer twice:

# bad: runs the simulator twice
if run_sim; then ...
if run_sim; then ...

# good: run it once, keep the status
run_sim
status=$?
if [ "$status" -eq 0 ]; then ...

When you need to distinguish failure modes. A condition collapses every non-zero to "no", so a script that must treat exit 1 and exit 2 differently has to capture $? and case on it — Chapter 16.6's form.

When the question is about a value, not an action. [ "$count" -gt 100 ] is a comparison of two numbers; there is no command that already knows the answer, and inventing one would be absurd. Use the bracket.

And never reach for an external process to do what Bash can do:

if [ "$(echo "$name" | grep -c "^run_")" -gt 0 ]; then   # a pipe and two processes
if [[ $name == run_* ]]; then                            # no processes at all

Both answer the same question. The second forks nothing, and in a loop over ten thousand filenames that is ten thousand process launches you did not need. Chapter 16.2's [[ ]] pattern matching exists for exactly this.

7. A Complete Script

The module, assembled. A regression triage tool that uses a command as its condition everywhere one is available, and a bracket everywhere one is not:

#!/usr/bin/env bash
# triage.sh — classify every log in a results directory
# usage: triage.sh <resultsdir>
#
# exit: 0 all passed · 1 at least one failure · 2 unusable arguments
#       4 the environment is not set up

set -o nounset

dir=${1:-}

# a value question: a bracket
if [ -z "$dir" ]; then
  echo "usage: ${0##*/} <resultsdir>" >&2
  exit 2
fi
if ! [ -d "$dir" ] || ! [ -r "$dir" ]; then
  echo "${0##*/}: $dir is not a readable directory" >&2
  exit 2
fi

# an action question: let the command answer it
if ! command -v grep > /dev/null 2>&1; then
  echo "${0##*/}: grep is not on PATH — something is very wrong" >&2
  exit 4
fi

failed=0
seen=0

for log in "$dir"/*.log; do
  # the glob matched nothing and expanded to itself — Module 9
  [ -e "$log" ] || continue
  seen=$(( seen + 1 ))

  name=${log##*/}
  name=${name%.log}

  # order is severity: the most serious answer wins
  if ! [ -s "$log" ]; then
    verdict="EMPTY"
  elif grep -q "UVM_FATAL" "$log"; then
    verdict="FATAL"
  elif grep -q "UVM_ERROR" "$log"; then
    verdict="ERRORS"
  elif grep -q "Error-\[" "$log"; then
    verdict="COMPILE"
  elif ! grep -q "TEST PASSED" "$log"; then
    verdict="NO-VERDICT"
  else
    verdict="PASS"
  fi

  case $verdict in
    PASS) ;;
    *)    failed=$(( failed + 1 )) ;;
  esac

  printf '%-10s %s\n' "$verdict" "$name"
done

if [ "$seen" -eq 0 ]; then
  echo "${0##*/}: no logs found in $dir" >&2
  exit 2
fi

printf -- '-- %d logs, %d not passing\n' "$seen" "$failed"
[ "$failed" -eq 0 ]

$ mkdir results
$ printf 'UVM_FATAL : bad config\n'                    > results/alu_rand.log
$ printf 'UVM_ERROR : mismatch\nUVM_ERROR : again\n'   > results/fifo_full.log
$ printf 'Error-[SE] Syntax error\n'                   > results/top_build.log
$ printf 'running\n'                                   > results/uart_tx.log
$ : >                                                    results/spi_mode3.log
$ printf 'running\nTEST PASSED\n'                      > results/reset_seq.log

$ ./triage.sh results
FATAL      alu_rand
ERRORS     fifo_full
PASS       reset_seq
EMPTY      spi_mode3
COMPILE    top_build
NO-VERDICT uart_tx
-- 6 logs, 5 not passing
$ echo $?
1

$ rm results/*.log && printf 'TEST PASSED\n' > results/ok.log
$ ./triage.sh results
PASS       ok
-- 1 logs, 0 not passing
$ echo $?
0

$ ./triage.sh /etc
triage.sh: no logs found in /etc
$ echo $?
2

$ ./triage.sh /nope
triage.sh: /nope is not a readable directory
$ echo $?
2

The verdicts come out in alphabetical order because "$dir"/*.log is a glob, and Chapter 3.5 established that filename expansion sorts its results. The script does not sort anything; it simply walks the list the shell handed it.

Six things in that script are worth naming, because each is a decision from a different chapter of this module:

[ -z "$dir" ] is a bracket because "is this string empty" is a question about a value. No command knows it.

command -v grep is a command because "is this tool installed" is a question about the world, and command -v is the thing that knows.

[ -e "$log" ] || continue catches the non-matching glob. When "$dir"/*.log matches nothing it expands to the literal text — Chapter 9.6 — so the loop body runs once with a filename that does not exist. This one line is why the script does not report a phantom log.

The elif ladder is ordered by severity. UVM_FATAL before UVM_ERROR before a compile error, because a broken run usually produces several of these and the most serious one is the useful answer. Chapter 16.1's point about ordering being a policy decision, applied.

! [ -s "$log" ] comes first of all — before any grep — because Chapter 16.4 established that an empty log is a tool that never ran, and grepping it would report a confident, meaningless "no errors found".

The last line is [ "$failed" -eq 0 ]. No exit, no if. A script's exit status is the status of its last command (Chapter 15.6), so that single test is the return value: zero when nothing failed, one otherwise. It is the most compact correct ending a script of this shape can have, and it only reads as obvious once you believe Chapter 16.1 — that a test is a command, and a command's answer is a number.

Practice

1. Run time (yes ERROR | grep -q ERROR). Then run yes ERROR | grep -c ERROR and interrupt it. Explain in one sentence why the second cannot finish.
2. Build a 200,000-line file whose only match is on the first line. Compare grep -q against grep -c on it with time. Then move the match to the last line and compare again. Which measurement is the honest one?
3. Rewrite if [ -n "$(grep UVM_ERROR run.log)" ] as a command condition, then explain the two separate costs you removed.
4. Run command -v grep, command -v cd and command -v nosuchtool, printing $? each time. Why does the cd case justify the > /dev/null?
5. Find out whether which exists on your system, and whether its exit status for a missing command is reliable. Then write the command -v guard you would actually ship.
6. Use mkdir as a lock. Run your script twice concurrently with & and confirm exactly one instance proceeds. Then do it with [ -d ] || mkdir and try to produce the race.
7. Replace if [ "$(echo "$f" | grep -c '^run_')" -gt 0 ] with a [[ ]] pattern. Count the processes each version launches for a thousand filenames.
8. Delete the [ -e "$log" ] || continue line from triage.sh and run it on an empty directory. Explain the output.
9. Change triage.sh's last line from [ "$failed" -eq 0 ] to exit 0 and find the caller that is now misled.
10. Add a --strict option to triage.sh that makes NO-VERDICT a failure but leaves EMPTY a warning. You will need Chapter 16.6's option loop.

Check Your Understanding

1. What does grep -q do that grep does not, and why does it matter beyond tidiness?
2. yes ERROR | grep -c ERROR never finishes. What does that prove about using a count to answer a yes/no?
3. Why command -v rather than which?
4. grep -q x /no/such/file prints an error. Why did -q not suppress it?
5. Why is if mkdir d better than [ -d d ] || mkdir d?
6. Name two situations where a bracket is the right answer and a command is not.
7. triage.sh ends with a bare [ "$failed" -eq 0 ]. What does that accomplish?

8. Remember

• The condition slot takes a command, so the best condition is usually the command that already knows the answer.
• grep -q suppresses matches and may stop at the first one. yes ERROR | grep -q ERROR returns instantly; grep -c on the same stream never returns.
• Do not compute a count, a list or a string purely to test it once. Compute what you will use.
• -q does not silence stderr. A missing file still reports, and still exits 2 — which a condition cannot distinguish from 1.
• A pipeline is a valid condition and its status is the last command's.
• command -v tool > /dev/null 2>&1 is the installed-tool guard. Never which.
• type -t is the interactive tool — file, builtin, keyword, function, alias.
• if mkdir d is an atomic lock. [ -d d ] || mkdir d has a race in the gap.
• kill -0 pid tests existence without signalling — how you tell a stale lock from a live job.
• read returns 0 while a line was read, which is what drives the canonical file-reading loop.
• Use a bracket for questions about values, and [[ $x == pat* ]] rather than an echo | grep pipeline.
• A script's last command sets its status, so a final bare test can be the whole return value.

9. The Module, in One Page

Seven chapters, and they reduce to one sentence and its consequences.

if runs a command and reads the number it returned. That is Chapter 16.1, and everything else is downstream of it:

• [ is a command with a punctuation name, which is why its spaces are mandatory and why its contents obey every expansion rule in Module 9. [[ ]] is syntax, which is why it needs no quoting and can offer patterns and regexes (16.2).
• The operator picks the comparison, because the operands carry no type. Words for numbers, symbols for strings, and the wrong choice is silent (16.3).
• A file can exist and be unreadable, exist and be empty, or be a link to nothing. Those are four questions and four operators (16.4).
• Join tests from outside, with && and || or inside [[ ]] — never with -a and -o, which misparse your data as their own syntax (16.5).
• case matches patterns, reusing Chapter 3.5's globs, and an unmatched case reports success — so always write *) (16.6).
• And most conditions should contain no brackets at all, because the command you were going to run already computed the answer (16.7).

A script can now decide. What it cannot yet do is decide the same thing repeatedly — walk a list of files, iterate a testlist, retry until something succeeds, read a log line by line.

Module 17 is loops: for, while, until, and the reading loop that read's exit status at the end of Section 5 was built for. Every condition in this module works unchanged inside one, which is the point — you have already learned the hard half.