Pattern Syntax: Classes, Anchors, Quantifiers

Part I · Module 13 · Chapter 13.3 · Pattern Syntax: Classes, Anchors, Quantifiers

1. Three questions

Every pattern answers some combination of:

• What? — character classes: \d, \w, [A-Z], .
• Where? — anchors: ^, $, \y
• How many? — quantifiers: *, +, ?, {2,3}

{^\d{4}-\d{2}-\d{2}$} is all three: digits (what), bounded counts (how many), pinned to the whole string (where).

2. What: character classes

foreach {pattern subject} {
    {^\d+$}        "12345"
    {^\w+$}        "ab_12"
    {^[A-Z]+$}     "ABC"
    {^[A-Za-z]+$}  "Mixed"
    {^[0-9a-f]+$}  "3fa9"
} {
    puts [format "  %-14s on %-10s -> %d" $pattern "'$subject'" [regexp $pattern $subject]]
}

  ^\d+$          on '12345'    -> 1
  ^\w+$          on 'ab_12'    -> 1
  ^[A-Z]+$       on 'ABC'      -> 1
  ^[A-Za-z]+$    on 'Mixed'    -> 1
  ^[0-9a-f]+$    on '3fa9'     -> 1

The shorthands:

And a bracket expression [...] lists characters or ranges directly; [^...] negates it:

puts [regexp {^[aeiou]+$} "aeiou"]
puts [regexp {^[^aeiou]+$} "xyz"]

1
1

Tcl also accepts POSIX class names inside brackets, which some people find more readable:

puts [regexp {^[[:digit:]]+$} "123"]
puts [regexp {^[[:alpha:]]+$} "abc"]

1
1

3. The unescaped dot

. is not a full stop — it matches any character:

puts "escaped   : [regexp {^\d+\.\d+$} "1.5"]  [regexp {^\d+\.\d+$} "1x5"]"
puts "unescaped : [regexp {^\d+.\d+$} "1.5"]  [regexp {^\d+.\d+$} "1x5"]"

escaped   : 1  0
unescaped : 1  1

The unescaped version accepts 1x5, which was almost certainly not intended. Any pattern meant to match a literal dot — a decimal point, a filename extension, an IP address — needs \..

4. Where: anchors

set s "max_width = 8"

puts "^max      : [regexp {^max} $s]"
puts "^width    : [regexp {^width} $s]"
puts "8\$        : [regexp {8$} $s]"
puts "^.*8\$     : [regexp {^.*8$} $s]"

^max      : 1
^width    : 0
8$        : 1
^.*8$     : 1

^ means the start of the string and $ the end. Together they describe the whole of it, which is Chapter 13.2 §3's validation shape.

5. Tcl's word boundary is \y

Most languages spell this \b. Tcl's regex engine uses \y:

set s "max_width = 8"

puts "plain width   : [regexp {width} $s]"
puts "\\ywidth\\y     : [regexp {\ywidth\y} $s]"
puts "on 'width = 8': [regexp {\ywidth\y} "width = 8"]"

plain width   : 1
\ywidth\y     : 0
on 'width = 8': 1

The plain pattern matched inside max_width, which is the classic "matched the wrong thing" failure. \y requires a word boundary on each side, so it rejects the embedded occurrence and accepts the standalone one.

6. How many: quantifiers

foreach {pattern subject expected} {
    {^a*$}      ""      "zero or more"
    {^a+$}      ""      "one or more"
    {^ab?c$}    "ac"    "optional b"
    {^a{3}$}    "aaa"   "exactly 3"
    {^a{2,3}$}  "aaa"   "2 to 3"
    {^a{2,}$}   "aaaa"  "2 or more"
} {
    puts [format "  %-12s on %-6s -> %d   (%s)" $pattern "'$subject'" [regexp $pattern $subject] $expected]
}

  ^a*$         on ''     -> 1   (zero or more)
  ^a+$         on ''     -> 0   (one or more)
  ^ab?c$       on 'ac'   -> 1   (optional b)
  ^a{3}$       on 'aaa'  -> 1   (exactly 3)
  ^a{2,3}$     on 'aaa'  -> 1   (2 to 3)
  ^a{2,}$      on 'aaaa' -> 1   (2 or more)

The difference between * and + on an empty string is the one that bites: * accepts nothing at all, so {^\d*$} considers the empty string a valid number.

7. Groups and alternation

Parentheses group, and | offers alternatives:

puts [regexp {^(abc)+$} "abcabc"]
puts [regexp {^(cat|dog)$} "dog"]
puts [regexp {^(cat|dog)$} "cow"]
puts [regexp {^\d+(\.\d+)?$} "42"]
puts [regexp {^\d+(\.\d+)?$} "42.5"]

1
1
0
1
1

The last pair is the common "optional decimal part" idiom: a group with ? after it. Parentheses also capture, which is Chapter 13.4.

8. Reading a pattern you did not write

This is the skill the chapter is for. Take it left to right:

set pattern {^\s*(\w+)\s*=\s*(.+?)\s*$}

foreach line {"width = 32" "  mode=fast  " "nodelimiter" "x = "} {
    if {[regexp {^\s*(\w+)\s*=\s*(.+?)\s*$} $line -> k v]} {
        puts [format "  %-18s key='%s' value='%s'" "'$line'" $k $v]
    } else {
        puts [format "  %-18s no match" "'$line'"]
    }
}

  'width = 32'       key='width' value='32'
  '  mode=fast  '    key='mode' value='fast'
  'nodelimiter'      no match
  'x = '             key='x' value=' '

The last line is the one to look at. x = matched, and the value came out as a single space rather than empty — because (.+?) demands at least one character, and the trailing \s* is free to match nothing and leave that character to the capture.

Whether that is right depends on whether an empty value is legal in your format. If it is not, the pattern should reject the line; if it is, the value should be empty rather than a space. Either way the pattern as written does neither — and noticing that is exactly what reading it piece by piece buys you.

9. Building one deliberately

Work outward from the thing you are sure of. To match clk_div2[3]:

set tests {clk_div2[3] clk[0] data_bus[31] bad[x] noindex}

# 1. the name
puts "names only  : [regexp {^[A-Za-z_][A-Za-z0-9_]*} "clk_div2[3]"]"

# 2. add the bracketed index
set p {^[A-Za-z_][A-Za-z0-9_]*\[\d+\]$}

foreach t $tests {
    puts [format "  %-14s -> %d" "'$t'" [regexp $p $t]]
}

names only  : 1
  'clk_div2[3]'  -> 1
  'clk[0]'       -> 1
  'data_bus[31]' -> 1
  'bad[x]'       -> 0
  'noindex'      -> 0

The brackets are escaped as \[ and \] because unescaped they would start a bracket expression. Test against things that should not match — bad[x] and noindex — which is Chapter 13.9's discipline arriving early.

10. Exercises

Exercise 1. What are the three questions every pattern answers, and which syntax answers each?

Exercise 2. Why does {^\d+.\d+$} match 1x5, and how do you fix it?

Exercise 3. Write a pattern matching a 4-digit year, a dash, a 2-digit month, a dash and a 2-digit day, anchored.

Exercise 4. Why does regexp {width} "max_width = 8" return 1, and what pattern rejects it?

Exercise 5. What is Tcl's word-boundary escape, and what does \b mean instead?

Exercise 6. Read {^#\s*(\w+)\b?} aloud piece by piece. Which part is wrong for Tcl?

Answers

1. What (character classes: \d, \w, [...], .), where (anchors: ^, $, \y), and how many (quantifiers: *, +, ?, {n,m}).

2. Because . matches any character, not a literal dot — so x satisfies it. Escape it as \., or put it in a bracket expression as [.].

3. {^\d{4}-\d{2}-\d{2}$}.

4. Because matching is unanchored and width occurs inside max_width. {\ywidth\y} requires word boundaries on both sides and returns 0 for that string.

5. \y. \b is the backspace character in Tcl, not a word boundary — a pattern borrowed from another language that uses \b will fail silently.

6. ^ start of string, # a literal hash, \s* optional whitespace, (\w+) capture one or more word characters, \b? — and that last part is wrong: \b is a backspace in Tcl, so \b? means an optional backspace character rather than an optional word boundary. It should be \y.

11. Remember

• Three questions: what, where, how many — classes, anchors, quantifiers.
• \d, \w, \s and their negated \D, \W, \S; bracket expressions [...] and [^...]; POSIX names like [[:digit:]].
• . matches any character. A literal dot needs \. — the commonest silent regex bug.
• ^ and $ describe the whole string; without them the pattern only has to occur somewhere.
• Tcl's word boundary is \y, not \b — \b is the backspace character, so a borrowed pattern fails quietly.
• * accepts nothing, so {^\d*$} considers the empty string a valid number; + requires at least one.
• (...) groups and captures; | alternates; (...)? is the optional-part idiom.
• Read a pattern left to right, piece by piece — and test it against input that should not match.