When a Loop Reads Better Than a Comprehension

Part I · Module 12 · Chapter 12.7 · When a Loop Reads Better Than a Comprehension

1. Where a Comprehension Genuinely Wins

Start with the case for it, because it is the common one:

results = [
    {"test": "axi_smoke", "status": "PASS"},
    {"test": "axi_burst", "status": "FAIL"},
    {"test": "reset_seq", "status": "FAIL"},
]

fails = [r for r in results if r["status"] != "PASS"]

print(len(fails))
print([r["test"] for r in fails])

2
['axi_burst', 'reset_seq']

One line, one idea, nothing hidden. The loop version is four lines of which three are machinery — Chapter 12.1 made that argument and it stands. For a single filter or a single transformation, the comprehension is better, and the rest of this chapter is not an argument against it.

2. What Compression Costs

A comprehension puts the whole operation on one line. That is the benefit and it is also the mechanism of the problem: a reviewer scanning a diff reads one line at normal speed, where the same logic as a loop occupies four lines with the condition on its own.

Nothing about that matters when the condition is r["status"] != "PASS". It matters a great deal when the condition is long.

3. The Compound Filter That Hides a Bug

results = [
    {"test": "a", "status": "PASS", "runtime_s": 100, "forced": False},
    {"test": "b", "status": "FAIL", "runtime_s": 900, "forced": True},
    {"test": "c", "status": "FAIL", "runtime_s": 50,  "forced": False},
]

picked = [r["test"] for r in results
          if r["status"] == "FAIL" and r["runtime_s"] < 600 or r["forced"]]

print(picked)

['b', 'c']

Read the condition as a reviewer would: failing and fast, or forced. Now check b against it. b failed, but took 900 seconds — so it is not fast. It appears anyway.

The reason is Chapter 9.3 Section 3's precedence rule: and binds more tightly than or, so the condition is (FAIL and fast) or forced. For b the first half is False and forced is True, so it is selected on the strength of forced alone.

That may be intended. It may also be a missing pair of parentheses, and the comprehension gives a reviewer no place to notice the question. The same logic as a loop puts the condition on its own line:

results = [
    {"test": "a", "status": "PASS", "runtime_s": 100, "forced": False},
    {"test": "b", "status": "FAIL", "runtime_s": 900, "forced": True},
    {"test": "c", "status": "FAIL", "runtime_s": 50,  "forced": False},
]

picked = []
for r in results:
    is_quick_failure = r["status"] == "FAIL" and r["runtime_s"] < 600
    if is_quick_failure or r["forced"]:
        picked.append(r["test"])

print(picked)

['b', 'c']

Same answer — the loop does not fix the logic, and it was never supposed to. What it does is name the halves. is_quick_failure or r["forced"] states the grouping that the precedence rule was applying silently, and a reviewer can now agree or disagree with it.

4. A Comprehension Written for a Side Effect

The second misuse, and the one with a visible tell:

results = [{"test": "a"}, {"test": "b"}]

out = [print("submitting", r["test"]) for r in results]

print("what it built:", out)

submitting a
submitting b
what it built: [None, None]

The submissions happened. A list of two Nones was also built, held in memory, and thrown away — because print returns None (Chapter 11.8, Section 3) and a comprehension's job is to produce a list.

If you are discarding the result, the comprehension was the wrong construct. It is a loop with a list allocated around it for no reason, and it reads as though the list matters, which it does not.

results = [{"test": "a"}, {"test": "b"}]

for r in results:
    print("submitting", r["test"])

submitting a
submitting b

Two lines, no wasted list, and the shape says do this to each rather than build something.

The tell is simple: a comprehension whose result is not assigned to anything. A bare [...] on its own line is always this mistake.

5. State Carried Between Items

A comprehension evaluates its output expression independently for each item. Anything that depends on what came before — a running total, a maximum and the name that goes with it, a count of consecutive failures — cannot be expressed that way.

Chapter 10.10's patterns are loop patterns:

results = [
    {"test": "a", "runtime_s": 120},
    {"test": "b", "runtime_s": 900},
    {"test": "c", "runtime_s": 45},
]

total = 0
slowest_name = ""
slowest_time = 0

for r in results:
    total += r["runtime_s"]
    if r["runtime_s"] > slowest_time:
        slowest_time = r["runtime_s"]
        slowest_name = r["test"]

print("total  :", total)
print("slowest:", slowest_name, slowest_time)

total  : 1065
slowest: b 900

total accumulates across items and the two slowest-tracking names update together — Chapter 10.10 Section 8's paired accumulators. Neither is expressible as a comprehension, and attempting it produces something far worse than the loop.

A comprehension can still feed such a loop, which is often the cleanest arrangement:

results = [
    {"test": "a", "runtime_s": 120, "status": "FAIL"},
    {"test": "b", "runtime_s": 900, "status": "PASS"},
    {"test": "c", "runtime_s": 45,  "status": "FAIL"},
]

failures = [r for r in results if r["status"] == "FAIL"]

total = 0
for r in failures:
    total += r["runtime_s"]

print(len(failures), "failures totalling", total, "seconds")

2 failures totalling 165 seconds

The comprehension selects; the loop accumulates. Each construct does the job it is for.

6. Four Questions

Applied to any comprehension you are about to write or review:

1. Is the result used? If it is discarded, write a loop. (Section 4)
2. Does the filter have more than one condition joined by and/or? If so, either parenthesise it explicitly or name the halves in a loop. (Section 3)
3. Does anything need to be carried between items? A total, a maximum, a previous value — that is a loop. (Section 5)
4. Can you read it aloud as one sentence? If you have to stop and re-parse, so will a reviewer. (Chapters 12.4 and 12.6)

7. The Opposite Mistake

Over-correcting is its own failure. A four-line accumulator loop that only filters is noise, and writing one to look careful is as much a readability problem as an unreadable one-liner:

results = [{"test": "a", "status": "FAIL"}, {"test": "b", "status": "PASS"}]

names = []
for r in results:
    if r["status"] == "FAIL":
        names.append(r["test"])

print(names)

['a']

results = [{"test": "a", "status": "FAIL"}, {"test": "b", "status": "PASS"}]

print([r["test"] for r in results if r["status"] == "FAIL"])

['a']

The second is better, and it is better for the reason Chapter 12.1 gave: three of the four lines above are machinery supporting one idea. The default is the comprehension, and the questions in Section 6 are the specific cases where it is not.

8. Predict, Run, Observe

rows = [{"n": "a", "v": 1}, {"n": "b", "v": 9}]

# A
x = [r["n"] for r in rows if r["v"] > 5]
print("A:", x)

# B
y = [print("seen", r["n"]) for r in rows]
print("B:", y)

# C
t = 0
for r in rows:
    t += r["v"]
print("C:", t)

# D
print("D:", [r["n"] for r in rows if r["v"] > 5 or r["v"] < 2])

The verified output:

A: ['b']
seen a
seen b
B: [None, None]
C: 10
D: ['a', 'b']

A — a comprehension used properly: the result is assigned and used.

B — the side-effect misuse. Note the printing happens before the B: line, because the comprehension is evaluated as an argument before print runs — Chapter 11.3 Section 5.

C — an accumulator, which has to be a loop.

D — a two-condition filter where both rows match for different reasons. Short enough to read, which is why no parentheses are needed here and were needed in Section 3.

9. Exercises

Exercise 1 — apply the questions. For each, run Section 6's four questions and say comprehension or loop: (a) collect failing test names (b) print each test name (c) total the runtimes (d) collect names of tests that failed and ran under a minute, or were forced (e) uppercase every signal name.

Exercise 2 — find the precedence bug. Take Section 3's comprehension and add parentheses two different ways. Give the output of each and say which matches failing and fast, or forced.

Exercise 3 — name the halves. Rewrite Section 3's comprehension as a loop with two named intermediates. Say what a reviewer can now check that they could not before.

Exercise 4 — the discarded list. Write a comprehension that calls print for each item, then print its result. Explain the [None, None] in two sentences, naming the chapter that says why.

Exercise 5 — spot the tell. Write a bare [...] comprehension on its own line with no assignment. Say what makes this always a mistake.

Exercise 6 — carry state. Try to write a running total as a comprehension. Describe what goes wrong, then write the loop.

Exercise 7 — feed the loop. Using a comprehension to select and a loop to accumulate, report the number of failures and their combined runtime. Say which construct did which job.

Exercise 8 — over-corrected. Find a four-line loop in your Module 10 work that only filters, and fold it. Say why the shorter version is better here and not in Section 3.

10. Summary

• For one filter or one transformation, the comprehension is better — that is the default, not the exception.
• Compression is the benefit and the hazard. One line is read at one line's speed.
• A compound filter mixing and and or hides its grouping. A and B or C selects on C alone, and a reviewer scanning a line will not see it.
• Rewriting as a loop fixes nothing by itself — it creates somewhere to put a name, and a named intermediate is a checkable claim about intent.
• A comprehension written for a side effect builds a list of None and discards it. If the result is thrown away, the construct was wrong.
• A bare [...] with no assignment is always that mistake.
• State carried between items cannot be a comprehension — totals, maxima and counters are loop patterns from Chapter 10.10.
• A comprehension can feed a loop, which is usually the cleanest split: select with one, accumulate with the other.
• Four questions: is the result used, is the filter compound, is state carried, can you read it aloud.
• Over-correcting into loops everywhere is its own readability failure.

That closes Module 12. You can fold an accumulator loop into a comprehension and unfold one you did not write, build lists, dictionaries and sets the same way, tell a filter from a conditional expression by the count alone, flatten or preserve a nested structure, and decide — with four questions rather than a preference — which of the two forms a given job wants.

Everything so far has lived in one file. The next module is about what happens when a script grows past that, and about the single statement that lets one file use what another one defines.