Matching a Kind of Character: . \d \w \s

Part II · Module 19 · Chapter 19.4 · Matching a Kind of Character: . \d \w \s

1. From Spelling to Describing

Up to this point a pattern has been a literal: r"drv\.sv" matches the six characters it spells. That is useful and limited — it cannot find the line number in drv.sv(88) without knowing in advance that it is 88.

A character class matches a kind of character rather than a specific one:

import re

LINE = "UVM_ERROR drv.sv(88) @ 1420: bad handshake"

print("a digit anywhere :", re.search(r"\d", LINE).group())
print("where            :", re.search(r"\d", LINE).span())
print("a word character :", re.search(r"\w", LINE).group())
print("whitespace       :", repr(re.search(r"\s", LINE).group()))
print("literally anything:", re.search(r".", LINE).group())

a digit anywhere : 8
where            : (17, 18)
a word character : U
whitespace       : ' '
literally anything: U

The pattern no longer says what it matches. It says what kind of thing it matches, and the match object reports which one it found.

This is the point where a pattern becomes unreadable to somebody who has not learned the notation, and it is worth being aware of that cost rather than discovering it in review.

2. The Four Shorthands

Executed, so the boundaries are visible rather than asserted:

import re

SAMPLE = "aB_1 -./"

print("\\d :", re.findall(r"\d", SAMPLE))
print("\\w :", re.findall(r"\w", SAMPLE))
print("\\s :", re.findall(r"\s", SAMPLE))
print(".  :", re.findall(r".", SAMPLE))

\d : ['1']
\w : ['a', 'B', '_', '1']
\s : [' ']
.  : ['a', 'B', '_', '1', ' ', '-', '.', '/']

Read \w carefully: it took a, B, _ and 1, and left -, . and / behind. That one line is the source of most surprises in this chapter.

3. The Underscore Is In, the Dot and Hyphen Are Out

\w is designed for programming identifiers, which is exactly right for VLSI work and exactly wrong in one common case:

import re

for subject in ["reg_q", "u_core", "drv.sv", "test-name", "a/b"]:
    chars = re.findall(r"\w", subject)
    print(f"{subject:<12} \\w kept {''.join(chars):<10} dropped {''.join(c for c in subject if c not in chars)}")

reg_q        \w kept reg_q      dropped 
u_core       \w kept u_core     dropped 
drv.sv       \w kept drvsv      dropped .
test-name    \w kept testname   dropped -
a/b          \w kept ab         dropped /

reg_q and u_core survive intact — the underscore is a word character, which is why \w is the natural tool for a signal name or an instance name.

drv.sv loses its dot. That matters the moment a quantifier is added, because the match will stop at the dot rather than spanning it:

import re

print("one \\w from drv.sv   :", re.search(r"\w", "drv.sv").group())
print("a run of \\w stops    :", re.search(r"\w+", "drv.sv").group())
print("spanning it needs more:", re.search(r"\w+\.\w+", "drv.sv").group())

one \w from drv.sv   : d
a run of \w stops    : drv
spanning it needs more: drv.sv

\w+ returned drv, not drv.sv. Nothing failed — the pattern matched exactly what it described, and the author's intent was elsewhere. A parser extracting a filename with \w+ silently drops the extension.

Chapter 19.6 covers the +; it appears here only because the one-character behaviour of a class is impossible to appreciate without it.

4. \d Matches More Digits Than You Expect

The surprise that catches everybody:

import re

print("ASCII digits     :", re.findall(r"\d", "1420"))
print("mixed with text  :", re.findall(r"\d", "drv.sv(88)"))
print("Arabic-Indic too :", re.findall(r"\d", "٣7"))
print("count            :", len(re.findall(r"\d", "٣7")))
print("ASCII-only class :", re.findall(r"[0-9]", "٣7"))

ASCII digits     : ['1', '4', '2', '0']
mixed with text  : ['8', '8']
Arabic-Indic too : ['٣', '7']
count            : 2
ASCII-only class : ['7']

\d matched the Arabic-Indic digit. By default Python's re operates on Unicode, and \d means any character the Unicode standard classifies as a decimal digit — which is several hundred characters across many scripts.

For log parsing this is almost never a problem, because tool output is ASCII. It becomes one the moment a value reaches int():

print("int accepts it too:", int("٣7"))
print("but a C tool reading your report would not")

int accepts it too: 37
but a C tool reading your report would not

Where it genuinely matters — a value that must be ASCII because something downstream requires it — use [0-9], which is Chapter 19.5's custom class and says exactly what it means.

The same applies to \w, which includes accented letters and other scripts. [A-Za-z0-9_] is the ASCII-only spelling.

5. A Class Matches Exactly One Character

The single most common beginner mistake in the module:

import re

LINE = "UVM_ERROR drv.sv(88) @ 1420: bad handshake"

m = re.search(r"\d", LINE)
print("expected 88 or 1420, got:", m.group())
print("span                    :", m.span())
print("length of the match     :", len(m.group()))

expected 88 or 1420, got: 8
span                    : (17, 18)
length of the match     : 1

\d is one digit. It is not "a number" and not "all the digits" — it matches a single character and stops.

Getting the whole number requires a quantifier, which is the next chapter:

import re

LINE = "UVM_ERROR drv.sv(88) @ 1420: bad handshake"

print("one digit  :", re.search(r"\d", LINE).group())
print("one or more:", re.search(r"\d+", LINE).group())
print("all of them:", re.findall(r"\d+", LINE))

one digit  : 8
one or more: 88
all of them: ['88', '1420']

Hold the distinction clearly: a class says what kind, a quantifier says how many. They are separate decisions and this chapter is only the first.

6. The Negated Shorthands

Each shorthand has an uppercase inverse:

import re

SAMPLE = "ab 12"

print("\\d :", re.findall(r"\d", SAMPLE), " \\D :", re.findall(r"\D", SAMPLE))
print("\\w :", re.findall(r"\w", SAMPLE), " \\W :", re.findall(r"\W", SAMPLE))
print("\\s :", re.findall(r"\s", SAMPLE), " \\S :", re.findall(r"\S", SAMPLE))

\d : ['1', '2']  \D : ['a', 'b', ' ']
\w : ['a', 'b', '1', '2']  \W : [' ']
\s : [' ']  \S : ['a', 'b', '1', '2']

\D, \W and \S match everything the lowercase version does not, including characters you were not thinking about — \D above matched the space as well as the letters.

They are occasionally the right tool and usually not. "Everything up to the comma" is better said with a negated custom class, [^,], which states the delimiter explicitly rather than inverting a category. Chapter 19.5 covers that, and it is the form you will reach for far more often.

7. The Dot, and the Newline

The dot met Chapter 19.3 as a character to escape; here it is a class to use, and it has one boundary worth knowing:

import re

print("dot matches a letter    :", re.search(r"a.c", "abc").group())
print("dot matches a symbol    :", re.search(r"a.c", "a-c").group())
print("dot matches a space     :", re.search(r"a.c", "a c").group())
print("dot matches a dot       :", re.search(r"a.c", "a.c").group())
print("dot does NOT cross a newline:", re.search(r"a.c", "a\nc"))
print("findall over a newline  :", re.findall(r".", "a\nb"))

dot matches a letter    : abc
dot matches a symbol    : a-c
dot matches a space     : a c
dot matches a dot       : a.c
dot does NOT cross a newline: None
findall over a newline  : ['a', 'b']

The dot matches anything except a newline. That default is deliberate and usually what you want, because log parsing is line-oriented — a pattern should not accidentally run from one line into the next.

Two chapters depend on it. Chapter 19.14's DOTALL flag removes the exception, for the rare case where a field genuinely spans lines. And Chapter 19.13's greedy matching is about how far a .* runs — the answer being "to the end of the line", precisely because of this rule.

8. Predict, Run, Observe

import re

LINE = "slack -0.142ns at u_core/reg_q"

print("A:", re.findall(r"\d", LINE))
print("B:", re.search(r"\w", LINE).group())
print("C:", re.findall(r"\w+", LINE))
print("D:", re.search(r"\s", LINE).span())
print("E:", re.findall(r"\D", "0.142")) 
print("F:", bool(re.search(r"\w", "-")))
print("G:", re.search(r"-\d", LINE).group())

A: ['0', '1', '4', '2']
B: s
C: ['slack', '0', '142ns', 'at', 'u_core', 'reg_q']
D: (5, 6)
E: ['.']
F: False
G: -0

C is the line to study. \w+ broke -0.142ns into 0 and 142ns, because neither the hyphen nor the dot is a word character — so a run of \w stops at each. It also split u_core/reg_q into two pieces at the slash, while keeping each underscore.

A collected four separate digits and lost all structure; the value -0.142 cannot be reassembled from it.

E — within 0.142 the only non-digit is the dot, confirming \D is the exact inverse.

F — the hyphen is not a word character, which is why C split where it did and why a signed number needs the sign matched explicitly, as G does.

The lesson from C and G together: a number in tool output is rarely just \d. A signed decimal with a unit needs the sign, the digits, the dot and the unit each described, which is exactly the pattern Chapter 19.12 builds step by step.

9. Exercises

Exercise 1 — the four shorthands. Run each of \d, \w, \s and . over a mixed string with findall. Tabulate what each kept.

Exercise 2 — the boundaries. For drv.sv, test-name and u_core/reg_q, list which characters \w drops.

Exercise 3 — one character. Search a line containing 1420 with \d and confirm the match is one character long.

Exercise 4 — the stopped run. Use \w+ on a filename with an extension and show it drops the extension. Fix it.

Exercise 5 — Unicode digits. Confirm \d matches a non-ASCII digit and [0-9] does not. Say when the difference would matter in a flow.

Exercise 6 — negated. Use \D on a decimal number and explain what it returns. Then do the same job with a negated custom class.

Exercise 7 — the newline. Show that . does not cross a newline. Say which chapter changes that and why the default is right for logs.

Exercise 8 — narrowest class. Take a pattern of your own built with dots and replace each with the narrowest class that still matches. Find an input the original wrongly accepted.

10. Summary

• A character class matches a kind of character rather than a specific one — the point where a pattern starts describing instead of spelling.
• \d is a digit, \w a letter, digit or underscore, \s whitespace, . almost anything.
• \w includes the underscore, which makes it right for signal and instance names.
• \w excludes the dot, hyphen and slash, so \w+ on drv.sv returns drv and silently drops the extension.
• \d matches Unicode digits, not only ASCII; [0-9] is the ASCII-only form, and the same applies to \w versus [A-Za-z0-9_].
• A class matches exactly one character — \d gives you 8, not 88.
• A class says what kind; a quantifier says how many, and they are separate decisions.
• \D, \W and \S are exact inverses and match more than you usually intend; a negated custom class is clearer.
• The dot does not match a newline, which keeps line-oriented parsing safe and underlies both DOTALL and greedy matching.
• Prefer the narrowest class that works — patterns built from dots accept almost anything.

The next chapter builds your own classes, including the negated form that grabs everything up to a delimiter.