lmap: Building a List While You Iterate

Part I · Module 9 · Chapter 9.8 · lmap: Building a List While You Iterate

1. Before anything else: the version

lmap was added in Tcl 8.6. On 8.5 and earlier it does not exist:

puts [info patchlevel]
lmap x {1 2 3} {expr {$x * 2}}

8.5.9
invalid command name "lmap"

That matters more for Tcl than for most languages, because Tcl interpreters are often embedded in other tools and you do not always get to choose the version. Check before you depend on it:

if {[llength [info commands lmap]] == 0} {
    puts "this interpreter has no lmap"
}

this interpreter has no lmap

Section 7 gives a drop-in replacement for exactly this case.

2. What it replaces

The accumulator pattern, which you have now written several times:

set names {clk rst data}

set upper {}
foreach n $names {
    lappend upper [string toupper $n]
}
puts $upper

CLK RST DATA

Four lines, of which one is the actual transformation. lmap is the same loop with the collecting built in:

set upper [lmap n $names {string toupper $n}]
puts $upper

CLK RST DATA

The body's result on each pass becomes the next element. There is no lappend and no accumulator to initialise, so there is no way to forget either.

3. It is foreach, with a result

Everything you learned about foreach applies — the same argument shapes, the same padding rules. Several lists in parallel (Chapter 9.2):

set sums [lmap a {1 2 3} b {10 20 30} {expr {$a + $b}}]
puts $sums

set same {}
foreach a {1 2 3} b {10 20 30} { lappend same [expr {$a + $b}] }
puts $same

11 22 33
11 22 33

Several variables per pass (Chapter 9.3):

set pairs [lmap {k v} {mode fast runs 3} {format "%s=%s" $k $v}]
puts $pairs

mode=fast runs=3

And the padding rule is unchanged:

puts [lmap a {1 2 3} b {10} {list $a $b}]

{1 10} {2 {}} {3 {}}

4. Every pass contributes exactly one element

Including when the body produces nothing:

puts "length: [llength [lmap x {1 2} {set _ ""}]]"
puts "value : '[lmap x {1 2} {set _ ""}]'"

length: 2
value : '{} {}'

Two empty elements, not an empty list. This is the behaviour from Chapter 8.7 §7 — a list of empty strings is not an empty list — and it is the thing people get wrong when they try to use lmap as a filter.

5. Filtering needs continue

To drop elements rather than transform them, use continue — a pass that continues contributes nothing at all:

set evens [lmap x {1 2 3 4} {
    if {$x % 2} { continue }
    expr {$x * 10}
}]
puts $evens

20 40

Two elements from four passes. Compare the foreach version, which runs natively:

set same {}
foreach x {1 2 3 4} {
    if {$x % 2} { continue }
    lappend same [expr {$x * 10}]
}
puts $same

20 40

break stops the loop and returns what has been collected so far:

puts [lmap x {1 2 3 4} {
    if {$x == 3} { break }
    expr {$x * 10}
}]

10 20

6. When not to use it

lmap is for building one list from one iteration. The moment the loop is doing something else as well, the accumulator version is clearer:

# accumulating two things — lmap can only return one
set total 0
set wide {}
foreach {name width} {clk 1 data 32 valid 1} {
    incr total $width
    if {$width > 1} { lappend wide $name }
}
puts "total=$total wide=$wide"

total=34 wide=data

Trying to express that with lmap means either two passes over the data or a side effect inside the body, and both are worse than four honest lines. Chapter 9.9 is about exactly this case.

Use lmap when the loop body is a single expression whose value is the point. Use foreach when the body does work.

7. A compatibility shim for Tcl 8.5

If you have to run on 8.5 and want the shape anyway, define lmap when it is missing:

if {[llength [info commands lmap]] == 0} {
    namespace eval ::lmapc { variable acc; variable n 0 }

    proc lmap args {
        if {[llength $args] < 3 || [llength $args] % 2 == 0} {
            return -code error \
                "wrong # args: should be \"lmap varList list ?varList list ...? command\""
        }
        set body  [lindex $args end]
        set pairs [lrange $args 0 end-1]

        set id [incr ::lmapc::n]
        set ::lmapc::acc($id) {}

        set driver [linsert $pairs 0 foreach]
        lappend driver [string map [list @ID@ $id @BODY@ $body] {
            set ::lmapc::code [catch {@BODY@} ::lmapc::res]
            switch -- $::lmapc::code {
                0 { lappend ::lmapc::acc(@ID@) $::lmapc::res }
                4 { }
                3 { break }
                default { return -code $::lmapc::code $::lmapc::res }
            }
        }]

        set rc [catch {uplevel 1 $driver} r]
        set out $::lmapc::acc($id)
        unset ::lmapc::acc($id)
        if {$rc != 0} { return -code $rc $r }
        return $out
    }
}

puts [lmap n {1 2 3} {expr {$n * 2}}]

2 4 6

It delegates the iteration to foreach, so the padding and grouping rules are inherited rather than reimplemented, and it maps the body's completion code onto the documented behaviour: normal results are collected, continue contributes nothing, break stops, and errors propagate.

8. A worked example

Turning a table of records into a report, both ways:

set ports {{clk 1} {rst 1} {data 32} {valid 1}}

# with lmap
set lines [lmap p $ports {
    lassign $p name width
    format "  %-6s %3d bits" $name $width
}]
puts [join $lines "\n"]

  clk      1 bits
  rst      1 bits
  data    32 bits
  valid    1 bits

# portable to 8.5 without a shim
set lines {}
foreach p $ports {
    lassign $p name width
    lappend lines [format "  %-6s %3d bits" $name $width]
}
puts [join $lines "\n"]

  clk      1 bits
  rst      1 bits
  data    32 bits
  valid    1 bits

The second is one line longer and runs everywhere. That is the trade, and for a script you will hand to someone else it is usually worth taking.

9. Exercises

Exercise 1. Which Tcl version added lmap, and what exactly happens if you use it on an older one?

Exercise 2. Write one line doubling every element of {1 2 3}, and the foreach equivalent.

Exercise 3. What does lmap x {1 2} {set _ ""} return, and why is it not an empty list?

Exercise 4. How do you drop elements rather than transform them, and what does returning "" do instead?

Exercise 5. Write a check that reports whether the current interpreter has lmap.

Exercise 6. Give a loop that should not be written as lmap, and say why.

Answers

1. Tcl 8.6. On 8.5 and earlier the command does not exist and you get invalid command name "lmap" at the moment the line runs — not when the script loads, so a rarely-taken branch can hide the problem until production.

2.

set doubled [lmap n {1 2 3} {expr {$n * 2}}]

set doubled {}
foreach n {1 2 3} { lappend doubled [expr {$n * 2}] }

Both give 2 4 6.

3. A two-element list, both elements the empty string — {} {}. Every pass contributes exactly one element, and an empty result is still a result. An empty list would have length 0; this has length 2.

4. Use continue, which makes the pass contribute nothing. Returning "" keeps the element and makes it blank.

5.

if {[llength [info commands lmap]] == 0} {
    puts "no lmap on [info patchlevel]"
} else {
    puts "lmap available on [info patchlevel]"
}

On a plain 8.5.9 interpreter, with the shim from section 7 not loaded:

no lmap on 8.5.9

Load the shim first and the same check reports it as available — which is the point of guarding the shim on info commands rather than on a version number.

6. Any loop that accumulates more than one thing, or whose body does work rather than computing a value — for example a loop that totals widths and collects the names of the wide ports. lmap returns a single list, so expressing that needs either two passes or a side effect in the body, and the plain foreach with two accumulators is clearer than both.

10. Remember

• lmap is foreach that collects its body's results into a list, replacing the initialise-loop-append pattern.
• It was added in Tcl 8.6. On 8.5 it is invalid command name "lmap" — check with info commands lmap before relying on it.
• All of foreach's shapes work: several lists in parallel, several variables per pass, and the same empty-string padding.
• Every pass contributes exactly one element, so an empty body result gives an empty element, not a dropped one.
• continue drops an element; break stops and returns what was collected.
• Returning "" does not filter — it leaves a blank element behind.
• Use it when the body is one expression whose value is the point; use foreach when the body does work or accumulates more than one thing.
• string cat is also 8.6+ — use format when writing for 8.5.
• A guarded shim gives you the shape on 8.5 without redefining the real command where it exists.