regexp: Matching a Pattern

Part I · Module 13 · Chapter 13.2 · regexp: Matching a Pattern

1. The command and its result

puts [regexp {\d+} "abc123"]
puts [regexp {\d+} "abcdef"]

1
0

regexp pattern string returns 1 if the pattern matches and 0 if it does not. That makes it directly usable as a condition:

set field "12345"

if {[regexp {^\d+$} $field]} {
    puts "'$field' is all digits"
} else {
    puts "'$field' is not"
}

'12345' is all digits

Note the pattern is in braces. Every pattern in this module is, for reasons Chapter 13.5 makes concrete — for now, treat braces around a pattern as part of how regexp is spelled.

2. Matching is unanchored

This is the first default to internalise. The pattern does not have to describe the whole string — it only has to be found somewhere in it:

puts [regexp {\d+} "v2 has 32 bits"]
puts [regexp {abc} "xxabcxx"]
puts [regexp {^abc} "xxabcxx"]

1
1
0

The first two found their pattern inside a larger string. The third did not, because ^ requires the match to start at the beginning.

3. Anchoring to the whole string

^ and $ pin the match to the start and end:

foreach s {"12345" "v2 has 32 bits" "123 45" " 123"} {
    puts [format "  %-16s unanchored=%d  anchored=%d" \
        "'$s'" [regexp {\d+} $s] [regexp {^\d+$} $s]]
}

  '12345'          unanchored=1  anchored=1
  'v2 has 32 bits' unanchored=1  anchored=0
  '123 45'         unanchored=1  anchored=0
  ' 123'           unanchored=1  anchored=0

The unanchored column says "contains digits" and is 1 for everything. The anchored column says "is entirely digits" and is 1 only for the first. For validation you almost always want the second.

4. The leftmost match

When more than one position could match, the engine takes the leftmost one. A third argument to regexp captures what actually matched:

regexp {\d+} "v2 has 32 bits" matched
puts "the pattern matched: '$matched'"

the pattern matched: '2'

Not 32 — the 2 in v2, because that comes first. The pattern asked for "one or more digits" and the earliest place satisfying that is position 1.

This is why an unanchored pattern so often appears to work and then returns something unexpected: it did match, just not where you were looking.

foreach s {"v2 has 32 bits" "has 32 bits" "32 bits"} {
    regexp {\d+} $s m
    puts [format "  %-16s -> '%s'" "'$s'" $m]
}

  'v2 has 32 bits' -> '2'
  'has 32 bits'    -> '32'
  '32 bits'        -> '32'

Same pattern, three answers, all correct and only one of them probably what was wanted.

5. The third argument captures the whole match

Even with no parentheses in the pattern:

regexp {\d+} "abc123def" whole
puts "whole match : '$whole'"

regexp {[A-Za-z]+} "abc123def" word
puts "first word  : '$word'"

whole match : '123'
first word  : 'abc'

Parentheses add more variables after this one, which is Chapter 13.4. The first variable is always the whole match.

6. A pattern beginning with a dash

A trap worth meeting now rather than later. regexp takes switches, so a pattern starting with - is read as one:

regexp {-?\d+} "-42"

bad switch "-?\d+": must be -all, -about, -indices, -inline, -expanded, -line, -linestop, -lineanchor, -nocase, -start, or --

-- ends the switches, exactly as it does for switch in Chapter 8.5 §5:

puts [regexp -- {-?\d+} "-42"]

1

Escaping the dash also works, since a backslashed - is not a switch:

puts [regexp {\-?\d+} "-42"]

1

7. An invalid pattern raises

A malformed pattern is an error, and it happens when the line runs, not when the script loads:

regexp {[} "x"

couldn't compile regular expression pattern: brackets [] not balanced

That matters because a pattern in a rarely-taken branch is not checked until that branch executes — the same late-binding behaviour as a mistyped command name in Chapter 10.1 §5.

regexp -about reports what a pattern will do without matching anything, which is a quick way to check one is well-formed and count its groups:

puts [regexp -about {(\w+)=(\d+)}]

2 {REG_UNONPOSIX REG_ULOCALE}

The 2 is the number of capture groups.

8. A worked example

Validating fields, with every pattern anchored on purpose:

proc classify {value} {
    if {[regexp {^-?\d+$} $value]}                  { return "integer" }
    if {[regexp {^-?\d+\.\d+$} $value]}             { return "decimal" }
    if {[regexp {^[A-Za-z_][A-Za-z0-9_]*$} $value]} { return "identifier" }
    if {[regexp {^\s*$} $value]}                    { return "blank" }
    return "other"
}

foreach v {42 -42 3.14 clk_div2 "" "  " "v2 has 32" "12abc"} {
    puts [format "  %-12s %s" "'$v'" [classify $v]]
}

  '42'         integer
  '-42'        integer
  '3.14'       decimal
  'clk_div2'   identifier
  ''           blank
  '  '         blank
  'v2 has 32'  other
  '12abc'      other

Three things are worth pointing at.

v2 has 32 is other, not integer. Every pattern is anchored, so "contains digits" is not enough — this is section 3's distinction doing real work.

12abc is other too. The identifier pattern requires a letter or underscore first, and the integer pattern requires digits all the way to $.

The order of the tests matters. 42 satisfies both the integer pattern and nothing else, but if the identifier test came first and were written loosely — say {^\w+$}, which matches digits — then 42 would be classified as an identifier. Anchoring protects against the string; ordering protects against the patterns overlapping with each other.

10. Exercises

Exercise 1. What does regexp return, and how is that used in an if?

Exercise 2. Why does regexp {\d+} "v2 has 32 bits" return 1, and what does the whole match contain?

Exercise 3. Write a pattern testing that a string is entirely digits.

Exercise 4. What error does regexp {-?\d+} "-42" give, and what are two fixes?

Exercise 5. When is an invalid pattern detected?

Exercise 6. A validator built on regexp {\d+} accepts abc123. What is wrong and how do you fix it?

Answers

1. 1 for a match and 0 for no match, so it can be used directly: if {[regexp {^\d+$} $field]} { ... }.

2. Because matching is unanchored — the pattern only has to be found somewhere in the string. The whole match is 2, the digits in v2, because the engine takes the leftmost position that satisfies the pattern.

3. {^\d+$} — ^ pins the start, $ pins the end, so the digits must be the entire string.

4. bad switch "-?\d+": must be -all, -about, ... — the pattern begins with a dash and is read as a switch. Fix it with regexp -- {-?\d+} "-42" or by escaping the dash as {\-?\d+}.

5. When the regexp call executes, not when the script is loaded. A bad pattern in a branch that never runs is never reported.

6. It is unanchored, so it means "contains digits" rather than "is digits". Anchor it: regexp {^\d+$}.

11. Remember

• regexp pattern string returns 1 or 0, usable directly as a condition.
• Matching is unanchored — the pattern need only occur somewhere in the string. Use ^ and $ to describe the whole of it.
• The engine takes the leftmost match, which is why {\d+} on v2 has 32 bits gives 2 rather than 32.
• Leftmost bugs survive testing, because test data puts the interesting thing first.
• A third argument captures the whole match, even with no parentheses.
• A pattern starting with - is read as a switch — use -- or escape the dash.
• The bad switch message lists every switch regexp accepts.
• An invalid pattern raises when the line runs, not when the script loads.
• Always brace a pattern — Chapter 13.5 shows what happens otherwise.