Linux & Bash · Part I · Module 14 · Finding Files and Searching Their Contents

Basic vs Extended Regex, and grep -E

A pattern that works perfectly in one tool fails in another with no useful error, and the difference turns out to be a backslash. There are two dialects of the same language, they differ over six characters, and the rule for converting between them fits in one sentence.

Intermediate16 min readLinuxPart IFundamentalsSearchingRegex

Part I · Module 14 · Chapter 14.3 · Basic vs Extended Regex

Every pattern in Chapter 14.2 needed backslashes on its braces, groups and alternations. This chapter is why, and how to stop.

1. The Same Pattern, Twice

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ grep -c 'ab\{2,3\}' q.txt
2

$ grep -E -c 'ab{2,3}' q.txt
2

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ grep -c '\(FAIL\|ERROR\)' run.log
2

$ grep -E -c '(FAIL|ERROR)' run.log
2

Identical results, different escaping. These are the two dialects:

Basic regular expressions — the older one, what plain grep and plain sed use.
Extended regular expressions — what grep -E, sed -E and awk use.

They express exactly the same things. Neither is more capable. They differ over which side the backslash goes on, for six characters.

Figure 1 — why the same pattern needs different backslashes in different tools. In the older basic syntax, grouping and alternation are ordinary characters until a backslash makes them special, so a forgotten backslash matches the character literally and the pattern silently finds nothing. In the extended syntax the rule is exactly reversed — they are special already, so most patterns need no backslashes at all.

2. The Six Characters

Meaning	Basic	Extended
Group	`$` `$`	`(` `)`
Interval	`\{n,m\}`	`{n,m}`
One or more	`\+`	`+`
Zero or one	`\?`	`?`
Alternation	`\|`	`\|` → written `\

Everything else is the same in both:

Piece	Both dialects
`.` any character	`.`
`*` zero or more	`*`
`^` `$` anchors	`^` `$`
`[…]` `[^…]` classes	same
`[[:digit:]]` named classes	same
`\1` backreference	same

So the anchors, the dot, the star and the classes — most of a typical pattern — never change. Only those six.

3. What a Missing Backslash Does

The failure is quiet, which is what makes it expensive.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ printf 'ab\nab+\nab?\nab|\n' > meta.txt

$ grep -c 'ab+' meta.txt
1

$ grep -Ec 'ab+' meta.txt
4

In the basic dialect, ab+ means "a, b, then a literal plus sign" — so it matched the one line actually containing ab+. In the extended dialect it means "a, then one or more b" — matching all four.

No error either way. Both are valid patterns; they just mean different things.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ grep -c '(FAIL|ERROR)' run.log
0

Zero. In the basic dialect that pattern means a literal (FAIL|ERROR) — parentheses, pipe and all — which appears nowhere. You searched for the wrong thing and grep told you, correctly, that it was not there.

The symptom is no matches, not an error

A pattern written in the wrong dialect usually matches nothing rather than failing. So the message you get is the same one you would get from a correct pattern on a file that genuinely does not contain it.

The diagnostic: if an alternation or a group returns nothing, try the other dialect before doubting the data.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

grep '(FAIL|ERROR)' run.log      # nothing
grep -E '(FAIL|ERROR)' run.log   # there it is

Ten seconds, and it is the first thing to try.

4. Which Tool Speaks Which

This table is the practical content of the chapter. The syntax is easy; remembering which tool defaults to which is the part that costs time.

Tool	Default	Extended with
`grep`	Basic	`-E`
`sed`	Basic	`-E`
`awk`	Extended	always
`egrep`	Extended	— deprecated, use `grep -E`
`expr`	Basic	—
Bash `[[ =~ ]]`	Extended	always
`find -regex`	Basic	`-E` before the path on BSD
`vim`	Its own variant	—

Three of those are worth calling out:

awk is always extended. So a pattern that works in awk needs backslashes added to work in plain grep, which is the single most common direction of this confusion.

Bash's own [[ string =~ pattern ]] is extended. A pattern moved from a script's if into a grep will silently stop working.

egrep is deprecated. It still works everywhere, and GNU grep now prints a warning. Write grep -E.

Just use -E

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

grep -E '(FAIL|ERROR)' run.log
sed -E 's/run_([0-9]+)/test \1/' list.txt

Default to the extended dialect in anything you write. Three reasons:

It is what everyone else means. Online examples, other languages, and awk are all extended; the basic dialect is the odd one out.
Fewer backslashes is fewer mistakes. $FAIL\|ERROR$ versus (FAIL|ERROR) — the second is readable at a glance.
It matches Bash's own =~, so a pattern can move between a script and a command line unchanged.

-E is supported by both GNU and BSD versions of grep and sed, so there is no portability cost. The only reason to use the basic dialect is when reading somebody else's pattern.

5. Converting a Pattern

Mechanical, in both directions.

Basic to extended — remove backslashes from the six:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

\(FAIL\|ERROR\)        →   (FAIL|ERROR)
ab\{2,3\}              →   ab{2,3}
[0-9]\+                →   [0-9]+
colou\?r               →   colou?r

Extended to basic — add them:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

(ERROR|FATAL)          →   \(ERROR\|FATAL\)
[a-z]+                 →   [a-z]\+
run_[0-9]{3}           →   run_[0-9]\{3\}

And the reverse case, which people forget: to match those characters literally, the escaping also flips.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

grep    'a+b'    file     # basic: a literal plus
grep -E 'a\+b'   file     # extended: a literal plus
grep    '(x)'    file     # basic: literal parentheses
grep -E '\(x\)'  file     # extended: literal parentheses

A pattern searching for a literal ( in a log — a function call, a bus signal like data(3) — needs a backslash in the extended dialect and must not have one in the basic.

6. The Extensions

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ grep -c 'ab\+' q.txt
3

\+ and \? in the basic dialect are GNU extensions, not POSIX. They work in GNU grep, and on the BSD grep these chapters were verified against — which documents itself as "GNU compatible" — but they are not guaranteed.

Three more GNU-only pieces worth recognising:

Written	Means	Portable?
`\+` `\?` in basic	One-or-more, zero-or-one	GNU extension
`\<` `\>`	Word boundaries	GNU extension
`\b`	Either word boundary	GNU extension
`\w` `\s`	Word character, whitespace	GNU extension
`-P`	Full Perl-compatible expressions	GNU only, and often not compiled in

grep -P is the one to be careful with. It enables \d, \w, lookahead, lookbehind and non-greedy *?. It is absent from BSD grep entirely, and even on Linux it is sometimes built without PCRE support — in which case it fails at run time with a message about the option being unsupported.

For anything portable, use the POSIX pieces: [[:digit:]] rather than \d, [[:space:]] rather than \s, grep -w rather than \b, and [^X]* rather than .*?.

7. A Third Layer: the Shell

A pattern passes through the shell before the tool sees it, which adds its own escaping — and this is the part that turns a two-backslash problem into a four-backslash one.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

grep -E '(FAIL|ERROR)' run.log      # correct
grep -E "(FAIL|ERROR)" run.log      # also works here, but...

Chapter 9.3's rule applies exactly: the pattern belongs to grep, so single-quote it. Double quotes let the shell expand $, backticks and ! first:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

grep -E "^$user" file        # the SHELL substitutes $user
grep -E '^\$[0-9]' file      # a literal dollar — in double quotes this needs doubling

And an unquoted pattern is worse still, because *, ? and [ are also glob characters (Chapter 9.1):

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

grep [0-9] file              # the shell may expand this against filenames
grep '[0-9]' file            # correct

Three languages, one line

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

grep -E '\.log$' filelist.txt

That single line involves three separate escaping systems:

the shell, which the single quotes switch off entirely;
the extended regex dialect, where \. is a literal dot and $ is an anchor;
nothing else — because the quotes kept the shell out of it.

Remove the quotes and $ becomes a shell variable reference and the whole thing changes meaning. Single-quote every pattern, and two of the three systems stop being your problem.

When the pattern genuinely must contain a shell value, build it deliberately:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

grep -E "^${prefix}_[0-9]+" file

Double quotes so $prefix expands — and now you must trust $prefix not to contain regex metacharacters. If it comes from outside, use grep -F instead (Chapter 14.1).

8. In Practice

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

# default to -E and stop counting backslashes
grep -E '(ERROR|FATAL|FAIL)' run.log
grep -E '^(module|endmodule)' rtl.v
grep -E 'seed[0-9]{1,4}' testlist.txt
grep -E '\.(sv|v|vh)$' filelist.f

# the same patterns in sed, same flag
sed -E 's/run_([0-9]+)_seed([0-9]+)/test \1 seed \2/' list.txt

# awk needs no flag — it is extended already
awk '/^(ERROR|FATAL)/' run.log

# a literal plus sign, both ways
grep    'C+' notes.txt
grep -E 'C\+' notes.txt

# when the pattern is data, not a pattern
grep -F "$path" setup.sh

The fourth is a good example of why -E is worth defaulting to. In the basic dialect it would be '\.$sv\|v\|vh$$', which is the same pattern wearing five extra backslashes.

Practice

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

mkdir -p /tmp/bre && cd /tmp/bre
printf 'a\nab\nabb\nabbb\n' > q.txt
printf 'ab\nab+\nab?\nab|\n' > meta.txt
cat > run.log <<'EOF'
[00:03] ERROR: timeout after 1203 cycles
[00:05] Error: assertion failed
[00:06] TEST FAILED
EOF

1 — The same pattern in both dialects.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

grep -c  'ab\{2,3\}' q.txt
grep -Ec 'ab{2,3}'   q.txt
grep -c  '\(FAIL\|ERROR\)' run.log
grep -Ec '(FAIL|ERROR)'    run.log

2 — Watch a missing backslash change the meaning.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

grep -c  'ab+' meta.txt
grep -Ec 'ab+' meta.txt

One versus four, no error either way.

3 — The silent zero.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

grep -c  '(FAIL|ERROR)' run.log
grep -Ec '(FAIL|ERROR)' run.log

4 — Convert a pattern by hand. Write the basic form of each, then check:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

grep -E '(ERROR|FATAL)'  run.log
grep -E '[0-9]+'         run.log
grep -E 'run_[0-9]{3,}'  run.log

5 — Escape a literal, both ways.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

printf 'C+\nC\n' > plus.txt
grep -c  'C+'  plus.txt
grep -Ec 'C+'  plus.txt
grep -Ec 'C\+' plus.txt

6 — Confirm awk is always extended.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

awk '/(FAIL|ERROR)/' run.log
awk '/\(FAIL\|ERROR\)/' run.log

The second finds nothing — in awk those backslashes make the characters literal.

7 — Confirm Bash's =~ is extended.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

s="run_042"
[[ $s =~ ^run_[0-9]+$ ]] && echo "matched with extended syntax"
[[ $s =~ ^run_[0-9]\+$ ]] && echo "matched" || echo "did not match"

8 — See the shell interfere.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

grep -E '^\$[0-9]' run.log
grep -E "^\$[0-9]" run.log
prefix=ERROR; grep -E "^.*(${prefix})" run.log

9 — Check for -P.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

echo 'abc123' | grep -P '\d+' 2>&1 | head -1
echo 'abc123' | grep -oE '[[:digit:]]+'

The first may fail on your system; the second works everywhere.

10 — Clean up. cd /tmp && rm -rf /tmp/bre

Check Your Understanding

1. State the difference between the two dialects in one sentence.

2. grep '(FAIL|ERROR)' run.log returns nothing on a file that clearly contains both. Why?

3. Which dialect does awk use, and what does that imply for moving a pattern to grep?

4. How do you match a literal plus sign in each dialect?

5. Why is grep -P '\d+' a poor choice for a portable script?

6. Why should a pattern always be single-quoted?

Answers

1. In the basic dialect ( ) { } + ? | are ordinary characters that a backslash makes special; in the extended dialect they are special and a backslash makes them ordinary. The rule is exactly reversed, and nothing else differs.

2. Because plain grep uses the basic dialect, where that pattern means a literal (FAIL|ERROR) — parentheses and pipe included — which appears nowhere in the file. Add -E. The symptom is "no matches" rather than an error, which is what makes it expensive.

3. Extended, always. So a pattern that works in awk needs backslashes added to work in plain grep — or, better, give grep the -E flag so no conversion is needed. This is the most common direction of the confusion.

4. Basic: 'a+' — it is already literal. Extended: 'a\+' — the backslash switches off its special meaning. The escaping flips along with everything else.

5. Because -P is GNU-only and often not compiled in even on Linux, and absent from BSD grep entirely. \d is a Perl extension rather than POSIX. Use [[:digit:]], which works everywhere.

6. Because the pattern belongs to the tool, not the shell. Single quotes stop the shell expanding $, backticks and !, and stop *, ? and [ being treated as filename globs. It removes one of the three escaping systems from the problem entirely.

9. Remember

Two dialects, one language. Basic and extended express exactly the same things.
The rule is reversed for six characters: ( ) { } + ? |. Basic needs a backslash to make them special; extended needs one to make them ordinary.
Everything else is identical — the dot, the star, anchors, classes, backreferences.
A wrong-dialect pattern matches nothing rather than erroring, which is why it costs time. Try the other dialect first.
grep and sed default to basic; awk and Bash's =~ are always extended.
Default to -E. Fewer backslashes, matches what everyone else writes, and supported by both GNU and BSD.
egrep is deprecated — write grep -E.
Escaping a literal flips too: 'a+' in basic, 'a\+' in extended.
\+, \?, \<, \>, \b, \w are GNU extensions, not POSIX.
grep -P is GNU-only and often unavailable. Use [[:digit:]] not \d.
Single-quote every pattern, so the shell's own escaping stops being part of the problem.
When the pattern is data rather than a pattern, use -F.
An HDL construct is a shape, not a string — Chapter 24.2 measures a pattern that found two instantiations out of three because a negated class cannot span nested parentheses.

You can now write a pattern and know which tool will understand it. The next chapter points one at a whole directory tree — where the problem stops being the pattern and becomes everything it matches that you did not want.

Where this fits

Part of the Linux & Bash curriculum.

Meaning	Basic	Extended
Group	`\(` `\)`	`(` `)`
Interval	`\{n,m\}`	`{n,m}`
One or more	`\+`	`+`
Zero or one	`\?`	`?`
Alternation	`\|`	`\|` → written `\