Nested Comprehensions, and Their Limit

Part I · Module 12 · Chapter 12.6 · Nested Comprehensions, and Their Limit

1. Two Different Things Called Nesting

[cell for row in table for cell in row]      TWO SOURCES   -> one flat list
[[cell for cell in row] for row in table]    NESTED OUTPUT -> a list of lists

The first has two for clauses and produces a single flat list. The second has a comprehension inside its output expression and produces a list whose items are lists — the original shape, with each cell transformed.

They differ by one pair of brackets and they are not variations of one feature. This chapter never says "nested" without saying which.

The working data is Chapter 5.7's nested list:

table = [["axi_smoke", "axi_burst"], ["reset_seq"], ["cdc_check", "power_seq"]]

print(len(table))
print(table[0])

3
['axi_smoke', 'axi_burst']

Three rows holding five names in total.

2. Two Sources: Flattening

table = [["axi_smoke", "axi_burst"], ["reset_seq"], ["cdc_check", "power_seq"]]

flat = [cell for row in table for cell in row]

print(flat)
print(len(table), "rows ->", len(flat), "items")

['axi_smoke', 'axi_burst', 'reset_seq', 'cdc_check', 'power_seq']
3 rows -> 5 items

One list, five names, rows gone. This is Chapter 10.11's nested loop folded exactly as Chapter 12.1 folded a single one:

table = [["axi_smoke", "axi_burst"], ["reset_seq"], ["cdc_check", "power_seq"]]

flat = []
for row in table:
    for cell in row:
        flat.append(cell)

print(flat)

['axi_smoke', 'axi_burst', 'reset_seq', 'cdc_check', 'power_seq']

The two for clauses are the two loop headers, in the same order, with the append argument moved to the front.

3. The Clause Order, and the Error That Enforces It

Here is the rule that makes the order memorable: the clauses read left to right in exactly the order you would write the nested loops. Outer first, inner second.

Reverse them and Python refuses:

table = [["a", "b"], ["c"]]

flat = [cell for cell in row for row in table]

print(flat)

NameError: name 'row' is not defined. Did you mean: 'pow'?

The first source clause says for cell in row — and at that point row does not exist, because the clause that creates it has not been reached. The second clause's source must already be bound by the first.

That is why the order is not arbitrary and does not need memorising as a rule: it is forced by what has been defined yet, exactly as the nested loops are.

4. The Count Is the Product

Chapter 10.11 established that a nested loop's body runs len(outer) × len(inner) times, and a two-source comprehension is that loop:

tests = ["axi_smoke", "axi_burst"]
seeds = [4210, 4211, 4212]

jobs = [(t, s) for t in tests for s in seeds]

print(len(jobs), "=", len(tests), "x", len(seeds))
print(jobs[0], jobs[-1])

6 = 2 x 3
('axi_smoke', 4210) ('axi_burst', 4212)

Two sources over independent collections produce every combination — the regression grid from Chapter 10.11 Section 2, in one line. The warning from that chapter applies unchanged: one extra clause multiplies the work, and len(a) * len(b) is worth printing before submitting anything.

A filter may follow either clause, and it applies where it is written:

tests = ["axi_smoke", "axi_burst"]
seeds = [4210, 4211, 4212]

evens = [(t, s) for t in tests for s in seeds if s % 2 == 0]

print(len(evens))
print(evens)

4
[('axi_smoke', 4210), ('axi_smoke', 4212), ('axi_burst', 4210), ('axi_burst', 4212)]

5. Ragged and Empty Rows Need No Guard

Flattening handles uneven data without a special case:

table = [["a", "b"], [], ["c"], []]

flat = [cell for row in table for cell in row]

print(len(table), "rows ->", len(flat), "items")
print(flat)

4 rows -> 3 items
['a', 'b', 'c']

Four rows, three items. An empty row contributes nothing because its inner loop runs zero times, which is Chapter 10.1 Section 8's rule. No if row: guard is needed, and adding one would change nothing.

6. Nested Output: Keeping the Shape

The other form puts a whole comprehension in the output expression:

table = [["axi_smoke", "axi_burst"], ["reset_seq"]]

upper = [[cell.upper() for cell in row] for row in table]

print(upper)
print(len(table), "rows ->", len(upper), "rows")

[['AXI_SMOKE', 'AXI_BURST'], ['RESET_SEQ']]
2 rows -> 2 rows

Two rows in, two rows out. The structure survives; only the cells changed.

Put the two side by side on the same data:

table = [["a", "b"], ["c"]]

flattened = [cell for row in table for cell in row]
reshaped = [[cell for cell in row] for row in table]

print("flattened:", flattened, len(flattened))
print("reshaped :", reshaped, len(reshaped))

flattened: ['a', 'b', 'c'] 3
reshaped : [['a', 'b'], ['c']] 2

Three against two. One pair of brackets, and the result is a different shape entirely. As in Chapters 12.3 and 12.4, the length is what tells you which you wrote.

7. The Limit

The canonical guidance is explicit: two levels is the practical limit. Here is the three-level version, which runs and should not be written:

runs = [
    [["a", "b"], ["c"]],
    [["d"]],
]

flat = [cell for group in runs for row in group for cell in row]

print(flat)

['a', 'b', 'c', 'd']

It is correct. It is also four clauses on one line, and working out which name belongs to which level means counting fors. Add a filter and it stops being reviewable.

The loop version is three lines longer and immediately readable:

runs = [
    [["a", "b"], ["c"]],
    [["d"]],
]

flat = []
for group in runs:
    for row in group:
        for cell in row:
            flat.append(cell)

print(flat)

['a', 'b', 'c', 'd']

The rule worth adopting:

• Two source clauses and at most one filter — a comprehension is fine.
• Three or more sources, or two sources plus a conditional expression — write the loop.
• Any nesting you cannot read aloud in one breath — write the loop.

8. Predict, Run, Observe

t = [[1, 2], [3], [4, 5, 6]]

# A
print("A:", [x for row in t for x in row])

# B
print("B:", [[x for x in row] for row in t])

# C
print("C:", len([x for row in t for x in row]), len([[x for x in row] for row in t]))

# D
print("D:", [x for row in t for x in row if x > 3])

The verified output:

A: [1, 2, 3, 4, 5, 6]
B: [[1, 2], [3], [4, 5, 6]]
C: 6 3
D: [4, 5, 6]

A — two sources: flattened to six.

B — nested output: the three rows preserved.

C — six against three, from the same data. One pair of brackets.

D — the filter follows the innermost source, so it tests each cell.

9. Exercises

Exercise 1 — tell them apart. For each, say flatten or keep-shape and give the output length for [[1,2],[3]]: (a) [x for r in t for x in r] (b) [[x for x in r] for r in t] (c) [len(r) for r in t].

Exercise 2 — flatten. Flatten a three-row table of test names and print the total. Then write the equivalent nested loop and confirm the results are ==.

Exercise 3 — the clause order. Write the two source clauses in the wrong order and record the exact error. Explain in two sentences what was not yet defined.

Exercise 4 — keep the shape. Uppercase every cell of a ragged table without flattening it. Print the number of rows before and after.

Exercise 5 — the grid. Build every test-and-seed pair from a 3-test list and a 4-seed range. Print the count and confirm it equals the product.

Exercise 6 — empty rows. Flatten a table containing two empty rows. Say how many items you got and why no guard was needed.

Exercise 7 — the filter's position. For [x for r in t for x in r if x > 2], say which collection the filter tests. Then write a version whose filter tests the rows instead, and give both results.

Exercise 8 — past the limit. Write a three-source flatten, then its loop. Count the for keywords in each and say which you would put in a review.

10. Summary

• Two different things are called nesting: two source clauses, and a comprehension inside the output.
• Two sources flatten a list of lists into one list; a nested output preserves the rows and transforms the cells.
• They differ by one pair of brackets and produce different-length results — the count tells you which you wrote.
• Source clauses read left to right in nested-loop order, outer first.
• Reversing them is a NameError, because the second clause's source is not bound yet — the order is forced, not conventional.
• Everything after the output expression reads in execution order, including filters.
• Two sources over independent collections give every combination, and the count is the product — Chapter 10.11's multiplication.
• Empty rows contribute nothing and need no guard.
• Two source clauses and one filter is the limit. Three sources, or two plus a conditional expression, belongs in a loop.
• More than three for/if keywords in one comprehension is the signal to stop.

The last chapter of the module gives the general version of that judgement.