Accumulating Results Across an Iteration

Part I · Module 9 · Chapter 9.9 · Accumulating Results Across an Iteration

1. One data set, four questions

set rows {clk 3 rst 0 din -2 dout 7 en -1}

A flat list of name-value pairs, walked with foreach {name value} from Chapter 9.3. The four questions:

• how many entries are there?
• what do the values add up to?
• which names have a negative value?
• how do the names group by sign?

Each needs an accumulator: a variable that exists before the loop, is updated inside it, and is read after.

2. A count

set rows {clk 3 rst 0 din -2 dout 7 en -1}

set n 0
foreach {name value} $rows {
    incr n
}
puts "entries: $n"

entries: 5

Counting a condition is the more useful version:

set negatives 0
foreach {name value} $rows {
    if {$value < 0} { incr negatives }
}
puts "negative: $negatives"

negative: 2

3. A total

set total 0
foreach {name value} $rows {
    incr total $value
}
puts "total: $total"

total: 7

incr takes an optional amount, so incr total $value adds rather than counting. It only works on integers — for anything that might be fractional, use expr:

set sum 0.0
foreach v {1.5 2.25 3.0} {
    set sum [expr {$sum + $v}]
}
puts "sum: $sum"

sum: 6.75

4. A filtered list

set failures {}
foreach {name value} $rows {
    if {$value < 0} {
        lappend failures $name
    }
}
puts "failures: $failures"
puts "count   : [llength $failures]"

failures: din en
count   : 2

lappend is the right command here rather than set x "$x $name", because it handles the quoting — Chapter 6.6 made that case.

5. A dictionary of groups

The shape that turns a flat list into structure:

set bysign [dict create]
foreach {name value} $rows {
    set key [expr {$value < 0 ? "negative" : "nonneg"}]
    dict lappend bysign $key $name
}

dict for {k names} $bysign {
    puts "$k: $names"
}

nonneg: clk rst dout
negative: din en

dict lappend appends to the list stored under a key, creating both the key and the list if they are not there — which is why no initialisation of the inner lists is needed.

Counting per group is dict incr:

set tally [dict create]
foreach w {clk rst clk en clk} {
    dict incr tally $w
}
puts $tally

clk 3 rst 1 en 1

Again no initialisation: dict incr starts a missing key at zero.

6. Most accumulator bugs are initialisation bugs

Here is where the four shapes actually go wrong.

Not initialised at all. Tcl is inconsistent about this, and the inconsistency is the trap:

unset -nocomplain total
foreach v {1 2 3} { set total [expr {$total + $v}] }

can't read "total": no such variable

But these two are fine:

unset -nocomplain n items
foreach v {1 2 3} {
    incr n
    lappend items $v
}
puts "n=$n items=$items"

n=3 items=1 2 3

incr creates a missing variable at 0 (Chapter 3.9) and lappend creates a missing variable as an empty list (Chapter 6.6). set total [expr {$total + ...}] does not. So three of the four shapes forgive a missing initialisation and one does not — which is exactly the sort of inconsistency that lets a bug sit until the one code path that uses expr runs.

Initialised outside a loop that should reset it. The commonest real bug of the four:

set groups {{a 1} {b 2}  {c 3} {d 4}}

# WRONG — total is never reset, so each group includes the previous ones
set total 0
foreach g $groups {
    lassign $g name value
    incr total $value
    puts "$name running total: $total"
}

a running total: 1
b running total: 3
c running total: 6
d running total: 10

That is correct if you wanted a running total and wrong if you wanted a per-group one. The fix is to move the initialisation inside:

foreach g $groups {
    lassign $g name value
    set total 0
    incr total $value
    puts "$name alone: $total"
}

a alone: 1
b alone: 2
c alone: 3
d alone: 4

Nothing in the language tells you which you meant. Where the set goes is the entire specification.

Read before the loop can have run. An accumulator over an empty list keeps its initial value, which is usually right — except when the code after the loop assumes at least one pass:

set values {}
set total 0
foreach v $values { incr total $v }
puts "total: $total"
puts "average: [expr {[llength $values] ? $total / double([llength $values]) : "n/a"}]"

total: 0
average: n/a

7. All four at once

set rows {clk 3 rst 0 din -2 dout 7 en -1}

set n        0
set total    0
set failures {}
set bysign   [dict create]

foreach {name value} $rows {
    incr n
    incr total $value
    if {$value < 0} { lappend failures $name }
    dict lappend bysign [expr {$value < 0 ? "negative" : "nonneg"}] $name
}

puts "entries  : $n"
puts "total    : $total"
puts "average  : [format %.2f [expr {$total / double($n)}]]"
puts "failures : [join $failures {, }]"
dict for {k names} $bysign {
    puts [format "  %-9s %s" $k $names]
}

entries  : 5
total    : 7
average  : 1.40
failures : din, en
  nonneg    clk rst dout
  negative  din en

One pass, four accumulators, all initialised on four adjacent lines where a reader can check them together.

8. Exercises

Exercise 1. Name the four accumulator shapes and the correct empty value for each.

Exercise 2. Given {1 2 3 4 5}, write a loop producing both the count of even numbers and their total.

Exercise 3. Why does foreach v {1 2} { incr n } work without set n 0, while foreach v {1 2} { set t [expr {$t + $v}] } does not?

Exercise 4. What is the difference between putting set total 0 before the loop and inside it?

Exercise 5. Write a loop grouping {clk in rst in data out} into a dictionary keyed by direction.

Exercise 6. A script computes an average inside a loop over a list that is sometimes empty. Name the two things that can go wrong and how to prevent each.

Answers

1. A count (0), a total (0, or 0.0 for fractions), a filtered list ({}), and a dictionary of groups ([dict create]).

2.

set count 0
set total 0
foreach v {1 2 3 4 5} {
    if {$v % 2 == 0} {
        incr count
        incr total $v
    }
}
puts "count=$count total=$total"

count=2 total=6

3. incr creates a missing variable and starts it at 0. expr has no such behaviour — reading $t when t does not exist is an error, so the set t [expr {$t + $v}] form fails on the first pass.

4. Before the loop, it accumulates across all passes — a running total. Inside, it is reset on every pass, so each pass reports only its own value. Which one is correct is entirely determined by where the line goes; the language cannot tell you.

5.

set bydir [dict create]
foreach {name dir} {clk in rst in data out} {
    dict lappend bydir $dir $name
}
dict for {d names} $bydir {
    puts "$d: $names"
}

in: clk rst
out: data

6. First, dividing by [llength $values] when it is zero — guard with a conditional before dividing. Second, integer division truncating the result — force a float with double(...) or 1.0 *.

9. Remember

• Four shapes cover nearly everything: a count, a total, a filtered list, and a dictionary of groups.
• Each needs an explicit empty value: 0, 0, {}, [dict create].
• incr counts and totals — incr total $value adds rather than counting by one.
• lappend builds filtered lists and handles quoting; dict lappend and dict incr build groups and per-key counts without initialising the inner values.
• Most accumulator bugs are initialisation bugs, not loop bugs.
• incr, lappend, dict incr and dict lappend all create a missing variable; set x [expr {$x + ...}] does not. Initialise explicitly so the behaviour does not depend on which you used.
• Where the set goes is the specification — before the loop means running, inside means per-pass.
• An average needs double() and a guard against an empty list.