Character Classes: . \d \w \s and []

Part I · Module 11 · Chapter 11.5 · Character Classes: . \d \w \s and []

1. The Three Shorthands, Measured

Rather than describe what \d, \w and \s mean, here they are tested one character at a time. The anchors ^ and $ in each pattern make the test apply to the whole single-character string — Chapter 11.8 explains them properly.

use strict;
use warnings;

my @chars = ("a", "Z", "5", "_", "-", ".", " ", "\t");

printf "%-6s %-5s %-5s %s\n", "CHAR", "\\d", "\\w", "\\s";
for my $c (@chars) {
    my $label = $c eq " " ? "space" : $c eq "\t" ? "tab" : $c;
    printf "%-6s %-5s %-5s %s\n", $label,
        ($c =~ /^\d$/ ? "yes" : "-"),
        ($c =~ /^\w$/ ? "yes" : "-"),
        ($c =~ /^\s$/ ? "yes" : "-");
}

CHAR   \d    \w    \s
a      -     yes   -
Z      -     yes   -
5      yes   yes   -
_      -     yes   -
-      -     -     -
.      -     -     -
space  -     -     yes
tab    -     -     yes

The table is the definition, and three rows in it are worth stating in words.

5 is both \d and \w. The classes overlap — \w includes digits, so a pattern of \w+ happily matches 42. If you mean digits only, \d is not merely shorter, it is different.

The hyphen and the dot belong to none of the three. They are not word characters and not whitespace. This is why u-alu and 0.42 need explicit handling in any set you build.

Tab is \s. So is a newline and a carriage return. \s is whitespace in general, not the space character — which matters when you parse tool output that is tab-aligned in some places and space-aligned in others. Writing \s+ between fields instead of a literal space is the habit that makes a parser survive a tool upgrade.

2. The Underscore Is a Word Character

This looks like a footnote and is in fact one of the more consequential facts in the module.

use strict;
use warnings;

# \w is "word character": letter, digit, OR UNDERSCORE
printf "\\w on '_'  : %s\n", ("_" =~ /^\w$/ ? "yes" : "no");
printf "\\w on '-'  : %s\n", ("-" =~ /^\w$/ ? "yes" : "no");

# Which is why this matters for a whole-token test
for my $tok ("u_alu", "u-alu", "NO_ERROR") {
    printf "%-10s is all \\w : %s\n", $tok, ($tok =~ /^\w+$/ ? "yes" : "no");
}

\w on '_'  : yes
\w on '-'  : no
u_alu      is all \w : yes
u-alu      is all \w : no
NO_ERROR   is all \w : yes

\w is best read as "the characters an identifier is made of": letters, digits, and the underscore. That makes /^\w+$/ an exact description of a legal Verilog or SystemVerilog identifier, which is a genuinely useful thing to have.

It also has a consequence that bites. Since _ is a word character, the string NO_ERROR is one unbroken word as far as the regex engine is concerned — there is no boundary between NO_ and ERROR. A pattern using \b to look for the word ERROR therefore will not find one inside NO_ERROR. That is usually what you want, and it is the subject of Chapter 11.8; the point here is that the behaviour comes from this table rather than from anything special about \b.

3. The Complements: \D \W \S

Each shorthand has an upper-case form meaning "any character that is not in this class".

use strict;
use warnings;

my @chars = ("7", "x", " ");

printf "%-7s %-6s %-6s %-6s %-6s %-6s %s\n",
    "CHAR", "\\d", "\\D", "\\w", "\\W", "\\s", "\\S";
for my $c (@chars) {
    my $label = $c eq " " ? "space" : $c;
    printf "%-7s %-6s %-6s %-6s %-6s %-6s %s\n", $label,
        ($c =~ /^\d$/ ? "yes" : "-"), ($c =~ /^\D$/ ? "yes" : "-"),
        ($c =~ /^\w$/ ? "yes" : "-"), ($c =~ /^\W$/ ? "yes" : "-"),
        ($c =~ /^\s$/ ? "yes" : "-"), ($c =~ /^\S$/ ? "yes" : "-");
}

CHAR    \d     \D     \w     \W     \s     \S
7       yes    -      yes    -      -      yes
x       -      yes    yes    -      -      yes
space   -      yes    -      yes    yes    -

Each pair is exactly complementary — no character is both \d and \D, and none is neither.

The row to read carefully is x. It is \D and \w. \D means "not a digit", which includes letters, punctuation, and whitespace. People reach for \D intending "a letter" and get something much broader. If you mean a letter, write [a-zA-Z] or [[:alpha:]].

\S is the most useful of the three in practice. A field in tool output is usually "a run of non-whitespace", which is exactly \S+, and it does not care whether the field is a number, a name, a path, or N/A.

4. The Dot Stops at a Newline

Chapter 11.4 introduced the dot as "any character". The measurement is slightly narrower.

use strict;
use warnings;

my $s = "a\nb";

printf "subject is 'a', newline, 'b'\n";
printf "/a.b/   : %s   <- the dot does NOT match a newline\n",
    ($s =~ /a.b/  ? "match" : "no match");
printf "/a.b/s  : %s   <- /s makes the dot match everything\n",
    ($s =~ /a.b/s ? "match" : "no match");
printf "/a\\s b/ : %s   (with \\s, no /s needed)\n",
    ($s =~ /a\sb/ ? "match" : "no match");

subject is 'a', newline, 'b'
/a.b/   : no match   <- the dot does NOT match a newline
/a.b/s  : match   <- /s makes the dot match everything
/a\s b/ : match   (with \s, no /s needed)

So the dot is "any character except a newline".

This default is right for line-at-a-time processing, which is most of what you do — a .* cannot run past the end of a line into the next one, because there is no next one in the string. It becomes a problem the moment you read a whole file into one scalar, which is a common thing to do with a small report. Then a pattern written for a single line sees a string full of newlines, and a .* that worked before now stops at the first one.

Two ways out, and they are not equivalent. The /s modifier makes the dot match newlines as well — Chapter 11.16 covers it along with /m. Or use \s, which already includes newlines, as the third line shows. Prefer the explicit class when you specifically mean "whitespace including line breaks", and /s when you genuinely mean "any character at all".

5. Your Own Sets With [ ]

Square brackets list the characters allowed in one position. A ^ as the first character inside them inverts the set.

use strict;
use warnings;

# Your own set: any ONE of the listed characters
my @verdicts = ("setup", "fifo", "trace", "hold");

printf "%-8s %-12s %s\n", "WORD", "starts [sf]", "starts [^sf]";
for my $w (@verdicts) {
    printf "%-8s %-12s %s\n", $w,
        ($w =~ /^[sf]/  ? "yes" : "-"),
        ($w =~ /^[^sf]/ ? "yes" : "-");
}

WORD     starts [sf]  starts [^sf]
setup    yes          -
fifo     yes          -
trace    -            yes
hold     -            yes

[sf] means "one character, either s or f". The two columns are exact complements, as they should be.

The thing to keep straight is that [sf] matches one character, not the word sf. A set is a choice of character, never a choice of string — that is alternation, and it is Chapter 11.9. A beginner pattern like /[setup|hold]/ does not mean "setup or hold"; it means "one character from the set s, e, t, u, p, |, h, o, l, d", which matches almost any word in the language.

Note also where the ^ sits. Inside the brackets and first, it negates. Outside the brackets it is the start-of-string anchor, which is how it is being used in /^[sf]/ here — that pattern means "starts with s or f". Both ^ characters appear in /^[^sf]/, doing two different jobs.

6. Ranges

A hyphen between two characters inside brackets means everything between them.

use strict;
use warnings;

my @chars = ("b", "B", "3", "_");

printf "%-6s %-10s %-10s %-14s %s\n",
    "CHAR", "[a-z]", "[A-Z]", "[a-zA-Z0-9]", "[0-9]";
for my $c (@chars) {
    printf "%-6s %-10s %-10s %-14s %s\n", $c,
        ($c =~ /^[a-z]$/        ? "yes" : "-"),
        ($c =~ /^[A-Z]$/        ? "yes" : "-"),
        ($c =~ /^[a-zA-Z0-9]$/  ? "yes" : "-"),
        ($c =~ /^[0-9]$/        ? "yes" : "-");
}
print "\n";
printf "[a-zA-Z0-9_] and \\w agree on '_' : %s vs %s\n",
    ("_" =~ /^[a-zA-Z0-9_]$/ ? "yes" : "no"),
    ("_" =~ /^\w$/           ? "yes" : "no");

CHAR   [a-z]      [A-Z]      [a-zA-Z0-9]    [0-9]
b      yes        -          yes            -
B      -          yes        yes            -
3      -          -          yes            yes
_      -          -          -              -

[a-zA-Z0-9_] and \w agree on '_' : yes vs yes

Ranges are case-sensitive, so [a-z] rejects B. Several ranges can be combined in one set, as [a-zA-Z0-9] shows.

The _ row is the useful one: it is in none of those four sets. Add it explicitly and [a-zA-Z0-9_] becomes the long way of writing \w, which the last line confirms. Use \w — it is shorter, it is what readers expect, and it does not invite a typo in the ranges.

Remember from Chapter 11.4 that a hyphen you want literally must go last, as in [0-9.-], or be escaped. A set like [.-9] is a range from the dot to the nine and will not do what you hoped.

7. Mixing Shorthands Into Sets

The shorthands work inside brackets, which is how you build a set for a real field.

use strict;
use warnings;

# Shorthands work INSIDE brackets, and combine
my @subjects = ("0.42", "-0.42", "+0.42", "0,42", "abc");

printf "%-8s %s\n", "SUBJECT", "all [-+0-9.] characters?";
for my $s (@subjects) {
    printf "%-8s %s\n", $s, ($s =~ /^[-+0-9.]+$/ ? "yes" : "no");
}
print "\n";
# A set mixing a shorthand and literals
for my $s ("u_alu/A", "u alu/A") {
    printf "%-10s all [\\w/] : %s\n", $s, ($s =~ m{^[\w/]+$} ? "yes" : "no");
}

SUBJECT  all [-+0-9.] characters?
0.42     yes
-0.42    yes
+0.42    yes
0,42     no
abc      no

u_alu/A    all [\w/] : yes
u alu/A    all [\w/] : no

[-+0-9.] accepts a signed decimal's characters and rejects a comma-decimal and a word. The hyphen is first, so it is literal rather than a range — the rule from the previous chapter.

[\w/] is "a word character or a slash", which describes a hierarchical pin name exactly, and it correctly rejects the version with a space in it.

8. Putting It Together on a Real Line

With classes alone — plus + to mean "one or more", which is the next chapter's business — a field-shaped line test becomes short and readable.

use strict;
use warnings;

# Build up a field test from character classes alone (no quantifier subtlety)
my @lines = (
    "u_alu/A   0.42  ns",
    "u_alu/A   -0.12 ns",
    "u_alu/A   N/A   ns",
    "u_alu/A   0.42  ps",
);

for my $l (@lines) {
    my $ok = ($l =~ m{^[\w/]+\s+[-\d.]+\s+ns$}) ? "accept" : "reject";
    printf "%-22s %s\n", $l, $ok;
}

u_alu/A   0.42  ns     accept
u_alu/A   -0.12 ns     accept
u_alu/A   N/A   ns     reject
u_alu/A   0.42  ps     reject

Four lines, four correct verdicts. The two real rows are accepted including the negative value, N/A is rejected because A is not in [-\d.], and the picoseconds row is rejected because the pattern ends in a literal ns anchored to the end of the line.

Notice what the classes bought: \s+ between the fields means the alignment does not matter, so this same pattern works whether the tool separates columns with two spaces, five spaces, or a tab. That is the single most valuable habit in this chapter.

9. What To Remember

• \d digit, \w letter/digit/underscore, \s whitespace — and a digit is both \d and \w.
• The underscore is a word character. /^\w+$/ is exactly a Verilog identifier, and NO_ERROR is one unbroken word — which is why Chapter 11.8's \b behaves as it does.
• A hyphen is not a word character. It breaks a word; an underscore does not.
• \D \W \S are strict complements. \D means "not a digit", which includes letters and punctuation and whitespace — it does not mean "a letter".
• \S+ is the right way to grab an unknown field, whatever it contains.
• The dot matches any character except a newline. Fine for line-at-a-time work, a trap on a whole file read into one scalar. Use \s or the /s modifier.
• [ ] is a choice of one character, never a choice of strings. [setup|hold] is a set of ten characters, not two words.
• A leading ^ inside brackets negates; outside brackets it anchors.
• \s+ between fields rather than a literal space is what makes a parser survive a tool's formatting change.
• A character class cannot express order. It validates the alphabet, not the shape.

Chapter 11.6 supplies what is missing: quantifiers. * for none or more, + for one or more, ? for optional, and {n,m} for a counted range — which is how you match a number of unknown length without guessing how long it might be.