How Tcl Parses One Command

Part I · Module 2 · Chapter 2.2 · How Tcl Parses One Command

1. The Question

Chapter 2.1 showed that a line is a command name and its words. It left a gap: what happens between the text you typed and the command actually running?

That gap is where $name becomes a value, where [...] runs, and where quoting matters. This chapter fills it in.

2. The Stages

Tcl handles one command at a time. For each command it does this:

One clarification, so the picture stays honest. Tcl does not make a full pass over the line to find words and then a second full pass to substitute. It works through the line once, and as it builds each word it performs the substitutions that word allows.

The order in the figure is still exactly right, and it is the order that matters: where a word begins and ends is settled by the source text, before any substituted value is put into it.

3. A Line, Traced

Take a line that uses a variable:

set name "Ravi"
puts "Hello, $name"

Hello, Ravi

Here is the second line, stage by stage:

Source text:
    puts "Hello, $name"

Stage 2 — split into words:
    word 1:  puts
    word 2:  "Hello, $name"      (the quotes group it into one word)

Stage 3 — substitute inside word 2:
    $name  →  Ravi

Stage 4 — final values:
    word 1:  puts
    word 2:  Hello, Ravi

Stage 5 — run:
    the command puts, given one argument: Hello, Ravi

The quotes did not survive into stage 4. They were instructions about grouping, not characters in the message. That is why the output has no quotation marks in it.

4. The Rule That Follows

Now the important part. Look again at stage 2 and stage 3.

The words were counted in stage 2. The value of $name was not known until stage 3. So Tcl decided there were two words before it knew what $name held.

That gives us the rule:

What a substitution returns becomes part of a word. It never changes how many words there are.

This is not a quirk. It is the defining property of Tcl's evaluation, and it is the reason Tcl is predictable in a way shell scripting is not. Let us prove it three ways.

5. Proof One: A Value With Spaces Stays One Word

If a substituted value could split a word, then a variable holding two words would arrive as two arguments. Use the show tool from Chapter 2.1:

proc show {args} { puts "received [llength $args] word(s)" }

set msg "two words"

show $msg
show "two words"
show two words

received 1 word(s)
received 1 word(s)
received 2 word(s)

The first line passed $msg, with no quotes at all around it, and the command still received one argument. The space inside the value did nothing, because the word boundaries had already been decided.

Only the third line, where the space is in the source text, produced two words.

6. Proof Two: A Result Containing $ Is Not Substituted Again

If Tcl re-read substituted values, a value containing a dollar sign would be substituted a second time:

set a 5
set b {$a}

puts "b is: $b"

b is: $a

$b was substituted once, and what came back was the two characters $a. Tcl then stopped. It did not look at that result and think "this contains a variable reference".

If it had, the output would have been b is: 5.

7. Proof Three: A Result Containing [ ] Does Not Run

The same applies to command substitution, and here the consequence is easier to feel:

set danger {[exit 1]}

puts "danger is: $danger"
puts "still running"

danger is: [exit 1]
still running

$danger held the text [exit 1]. If Tcl re-read substituted values as source, that would have ended the script. It printed instead, and the next line ran.

This is worth remembering beyond Module 2. It means a value that came from a file, a user, or another program cannot turn into executable Tcl just by being used. It becomes executable only if you hand it to a command that is meant to run it — which is Chapter 2.11.

8. The One Exception

There is exactly one thing in Tcl that changes word boundaries after substitution, and it is deliberate. It is written {*}:

proc show {args} { puts "received [llength $args] word(s)" }

set pair "alpha beta"

show $pair
show {*}$pair

received 1 word(s)
received 2 word(s)

{*} tells Tcl to take the value and use its elements as separate words. It exists because the rule is otherwise absolute, so there has to be a way to opt out when you genuinely want one.

You do not need it yet. It is here so that the rule in §4 is true as stated, with its exception named, rather than true most of the time.

9. Common Mistakes

Quoting a variable to protect it. Writing show "$msg" instead of show $msg changes nothing about word count. It is harmless, but it shows a shell habit rather than a Tcl reason.

Expecting a value to be re-read. Building a command as text and passing it around does not run it. §7.

Expecting braces to hide a value from a command. Braces affect what the word contains, not whether the command gets it. The command always gets its words.

Thinking the stages are separate passes. Tcl works through the line once. §2 says so, and the distinction matters when you read Tcl's own documentation.

10. Exercises

Exercise 1 — predict the word count. For each line, say how many words the command receives, then check with show:

set file "my report.txt"

show $file
show $file backup
show "$file backup"
show {$file backup}

Answers: 1, 2, 1, 1. The last two differ in content, not in count — one holds my report.txt backup, the other holds the literal text $file backup. If that is not obvious yet, Chapter 2.8 explains it.

Exercise 2 — disprove the shell habit. Set a variable to a value with two spaces in it and pass it to show with no quotes. Predict the count first. If you predicted anything other than 1, re-read §5.

Exercise 3 — break the no-rescan rule, and fail. Try to make this print 5 by changing only the puts line, without using eval:

set a 5
set b {$a}
puts "b is: $b"

You will not manage it, and that is the point. Write one sentence saying why.

Exercise 4 — trace it on paper. Write out the five stages from Figure 1 for this line, before running it:

set n 3
puts "n doubled is [expr {$n * 2}]"

Then run it and check your stage 4 against the output.

Exercise 5 — use the exception. Using {*}, make a single variable holding a b c arrive as three separate words. Then explain why this does not contradict §4.

11. Remember

• Tcl handles one command at a time, in the order shown in Figure 1.
• Word boundaries are decided by the source text, before any substitution happens.
• Therefore a substituted value never changes the word count. A variable holding spaces is still one argument.
• A substituted result is not read as Tcl again. Text containing $ is not substituted twice; text containing [ ] does not run.
• {*} is the one documented exception, and you have to ask for it.
• This is not shell behaviour. Do not carry defensive-quoting habits across.

Next: the thing stage 2 is actually looking at — the spaces.