Accumulating a Result in a Loop

Part I · Module 9 · Chapter 9.9 · Accumulating a Result in a Loop

1. The Three Patterns

use strict;
use warnings;

my @rows = (
    { blk => "alu",  slack => -0.12, corner => "ss" },
    { blk => "fifo", slack =>  0.34, corner => "ff" },
    { blk => "regf", slack => -0.05, corner => "ss" },
);

my $count = 0;          # 1. a running count
my @kept;               # 2. a list of items kept
my %per_corner;         # 3. a hash of counts per key

for my $r (@rows) {
    $count++                      if $r->{slack} < 0;
    push @kept, $r->{blk}         if $r->{slack} < 0;
    $per_corner{ $r->{corner} }++;
}

printf "count      : %d\n", $count;
printf "kept       : %s\n", join(" ", @kept);
printf "per corner : %s\n", join("  ", map { "$_=$per_corner{$_}" } sort keys %per_corner);

count      : 2
kept       : alu regf
per corner : ff=1  ss=2

Three details in that code are load-bearing:

• my $count = 0; is initialised; my @kept; and my %per_corner; are not. An array and a hash start empty and usable. A scalar starts undef, and $count++ on undef happens to work — Chapter 3.4 measured that ++ and += are exempt from the uninitialised warning — but writing = 0 says the intent and costs nothing.
• $per_corner{$k}++ needs no setup. It creates the key on first use, which is Chapter 6.8's counting idiom and Chapter 7.9's mechanism.
• sort keys %per_corner in the output. Without it the corner order varies between runs — Chapter 6.5's rule.

The three combine freely. A per-key list rather than a count is push @{ $by_corner{$k} }, $blk; — Chapter 7.7's pattern, which is the fourth shape and the one that turns a flat log into grouped data.

2. Sum, Mean and a Running Worst

use strict;
use warnings;

my @slacks = (-0.12, 0.34, -0.05);

my $sum = 0;
my $worst;

for my $v (@slacks) {
    $sum += $v;
    $worst = $v if !defined $worst || $v < $worst;
}

printf "sum=%+.2f  mean=%+.3f  worst=%+.2f\n",
    $sum, $sum / scalar(@slacks), $worst;

sum=+0.17  mean=+0.057  worst=-0.12

The worst accumulator is the one with a subtlety worth naming. The condition is:

$worst = $v if !defined $worst || $v < $worst;

!defined $worst has to be tested first so the very first value is taken unconditionally. Short-circuit evaluation from Chapter 8.5 §1 guarantees that $v < $worst is never reached while $worst is still undef, which is what keeps STDERR clean.

The tempting alternative is to seed it:

my $worst = 0;          # WRONG — 0 beats every positive slack
my $worst = 9**9**9;    # infinity — works, and reads oddly

my $worst = 0; is the mistake, and on this data it produces -0.12 anyway so a test would pass. Give it a list of all-positive slacks and it reports 0 as the worst — a value that was never in the input.

List::Util does it properly and is core:

use List::Util qw(min max sum0);

my $worst = min @slacks;
my $total = sum0 @slacks;     # sum0 returns 0 for an empty list; sum returns undef

Chapter 5.11 covers List::Util. Note sum0 rather than sum — the difference is exactly §3's subject.

3. What Each One Does With No Input

This is the section that matters, because an empty input set is the normal good outcome.

use strict;
use warnings;

my @none;

my $count = 0;
my @kept;
my %per_key;
my $sum = 0;
my $worst;

for my $r (@none) {
    $count++;
    push @kept, $r;
    $per_key{$r}++;
    $sum += $r;
    $worst = $r;
}

printf "count=%d  kept=%d  keys=%d  sum=%s  worst=%s\n",
    $count, scalar(@kept), scalar(keys %per_key), $sum,
    (defined $worst ? $worst : "undef");

if (@none) {
    printf "mean=%.3f\n", $sum / scalar(@none);
}
else {
    print "mean: skipped, there is nothing to divide by\n";
}

count=0  kept=0  keys=0  sum=0  worst=undef
mean: skipped, there is nothing to divide by

The loop body never ran, and each accumulator holds its initial value:

4. When an Expression Is Better Than a Loop

Two of the three patterns have a one-line form.

use strict;
use warnings;

my @rows = (
    { blk => "alu",  slack => -0.12 },
    { blk => "fifo", slack =>  0.34 },
    { blk => "regf", slack => -0.05 },
);

# the loop version
my @kept_loop;
for my $r (@rows) { push @kept_loop, $r->{blk} if $r->{slack} < 0 }

# the expression version
my @kept_expr = map { $_->{blk} } grep { $_->{slack} < 0 } @rows;

my $count_expr = grep { $_->{slack} < 0 } @rows;

printf "loop : %s\n", join(" ", @kept_loop);
printf "expr : %s\n", join(" ", @kept_expr);
printf "count via grep in scalar context : %d\n", $count_expr;

loop : alu regf
expr : alu regf
count via grep in scalar context : 2

Identical results, so this is a readability choice rather than a correctness one.

• grep in scalar context is the count — Chapter 5.10 §5's finding. my $n = grep { COND } @list; replaces a counting loop entirely.
• map { } grep { } is the kept list, read right to left: filter, then extract.
• There is no expression form for the per-key hash. A tally needs a loop, or a map into a list of pairs that you then assign to a hash — which is longer and harder to read than the loop.

The strongest case for the loop is §1's: three accumulators in one pass. The expression form would mean three traversals of @rows and three separate statements, and the loop says "one pass over the data, building these three things" directly.

5. Accumulating Into a Nested Structure

The fourth shape, and the one that most real scripts end up at.

use strict;
use warnings;

my @rows = (
    { blk => "alu",  slack => -0.12, corner => "ss" },
    { blk => "fifo", slack =>  0.34, corner => "ff" },
    { blk => "regf", slack => -0.05, corner => "ss" },
    { blk => "pad",  slack => -0.40, corner => "ff" },
);

my %failing_by_corner;
my %worst_by_corner;

for my $r (@rows) {
    next unless $r->{slack} < 0;

    push @{ $failing_by_corner{ $r->{corner} } }, $r->{blk};

    my $w = $worst_by_corner{ $r->{corner} };
    $worst_by_corner{ $r->{corner} } = $r->{slack}
        if !defined $w || $r->{slack} < $w;
}

for my $corner (sort keys %failing_by_corner) {
    printf "%-3s %d failing [%s]  worst %+.2f\n",
        $corner,
        scalar(@{ $failing_by_corner{$corner} }),
        join(" ", @{ $failing_by_corner{$corner} }),
        $worst_by_corner{$corner};
}

ff  1 failing [pad]  worst -0.40
ss  2 failing [alu regf]  worst -0.12

Two accumulators, one pass, both keyed by corner. push @{ $h{$k} }, $v is Chapter 7.7's idiom — it creates the inner array on first use, so there is no setup. The per-corner worst uses §2's !defined guard, now once per key rather than once overall.

Note the next unless guard at the top rather than wrapping the body in an if. That is Chapter 8.3's flat-guard shape, and it keeps both accumulators at the same indentation level.

6. Why This Matters in Engineering Work

These three patterns are what a report script is. Count the violations, list the blocks that have them, tally them per corner. A tool that parses output and summarises it is a loop with two or three accumulators and a sorted print.

One pass is worth preferring. Three grep expressions over a hundred-thousand-row parsed report traverse it three times. The loop version does it once, and more importantly it keeps the three related questions in one place where a reader can see they are about the same data.

And the empty case is the one to design for. A clean run produces an empty list. A script that dies with Illegal division by zero while printing the all-clear, or that reports a worst slack of +0.00 that was never measured, has turned the good outcome into a failure or a lie. Guarding a mean and a minimum costs one if each.

7. Module 9, Closed

Loops are where a script spends its time, and this module has been one argument: the shape you choose determines which mistakes are available to you.

foreach over a list cannot hang, has no bound to get wrong, and no index to desynchronise from the array — which is why Chapter 9.1 makes it the default and the counting for the exception. while is for work whose length the data decides, and Chapter 9.3 showed Perl quietly inserting a defined into the commonest one, in exactly one situation and nowhere else.

The traps in this module share the property that made Module 8's worth collecting — none of them produces an error:

• An assignment to the loop variable rewrites the array, because it is an alias and not a copy.
• while (my $job = shift @queue) stops on a 0, leaving work undone and exiting 0.
• 0 .. scalar(@a) runs one iteration too far, and one whole iteration too far on an empty array.
• splice inside a counting loop skips elements — three of five visited, exit 0.
• A subroutine that assigns to $_ rewrites the caller's array — [CLOBBERED|CLOBBERED], empty STDERR.
• A retry counter shared across items makes a healthy item fail, and reordering the input changes which one.
• An unbounded wait hangs and prints nothing at all.

Two of them are not even silent so much as invisible to the tools: last inside a do { } while and a misspelled loop label are both runtime errors that perl -c reports as syntax OK, and both live naturally on error paths that tests do not reach.

What the module suggests as a working habit is small. Prefer foreach. Name the variable. Bound every wait. And ask what the loop does when the list is empty — because in this trade the empty list is usually the result you were hoping for, and it is the input least likely to have been tested.

Module 10 turns to subroutines: how arguments actually arrive, why they are aliases too, and what a Perl subroutine returns when you are not careful about context.