Python Part II · Module 27 · Parallel Regression
Why Parallelism, and What It Cannot Fix
Module 26 built a regression that runs four hundred jobs correctly, one after another, and a regression that takes five days is a regression nobody runs. Running the jobs at the same time is the obvious answer and it is the right one — the jobs were made independent four chapters ago precisely so that it would be. What is worth establishing before writing any of it is which problems concurrency actually solves, because the famous theoretical limit turns out not to apply here and the two real limits are not about speed at all.
Advanced18 min readPythonPart IIAutomationRegressionVerification Flow
Part II · Module 27 · Chapter 27.1 · Why Parallelism, and What It Cannot Fix
1. A Regression Nobody Runs
JOBS, MINUTES_EACH = 600, 12
serial_minutes = JOBS * MINUTES_EACH
print("jobs :", JOBS)
print("minutes each :", MINUTES_EACH)
print("serial total :", serial_minutes, "minutes")
print(" =", f"{serial_minutes / 60:.0f} hours")
print(" =", f"{serial_minutes / 1440:.1f} days")
print()
print("runs per week :", f"{7 / (serial_minutes / 1440):.1f}")
print("feedback delay :", "a change made on Monday is tested by Saturday")jobs : 600
minutes each : 12
serial total : 7200 minutes
= 120 hours
= 5.0 days
runs per week : 1.4
feedback delay : a change made on Monday is tested by Saturday1.4 runs per week. Everything else in this module follows from that number: a regression slower than the rate at which the design changes is not a safety net, it is an archive.
Every figure in this chapter is arithmetic over configured inputs — a job count, an assumed per-job duration, a worker count. Nothing here is measured, and Chapter 27.2 is specifically about why you must measure your own rather than trust a number like 12.
2. The Annotation
# 600 jobs x 12 min serial = 5 days
# 32 at a time = ~4 hoursJOBS, MINUTES_EACH = 600, 12
serial = JOBS * MINUTES_EACH
print(f"{'workers':>8} {'minutes':>9} {'hours':>7} {'speedup':>8}")
for workers in (1, 8, 16, 32, 64):
parallel = serial / workers
print(f"{workers:>8} {parallel:>9.0f} {parallel / 60:>7.2f} {serial / parallel:>7.0f}x")
print()
print("the annotation's claim:", "32 at a time is about 4 hours")
print("the arithmetic says :", f"{serial / 32 / 60:.2f} hours")
print("'~4 hours' is honest :", 3.5 <= serial / 32 / 60 <= 4.5) workers minutes hours speedup
1 7200 120.00 1x
8 900 15.00 8x
16 450 7.50 16x
32 225 3.75 32x
64 112 1.88 64x
the annotation's claim: 32 at a time is about 4 hours
the arithmetic says : 3.75 hours
'~4 hours' is honest : TrueFive days to under four hours, and the speedup column is perfectly linear, because the model assumes the jobs are independent and nothing is shared.
That assumption is doing an enormous amount of work, and it is the one Module 26 spent nine chapters earning. Chapter 26.4 gave every job its own directory, Chapter 26.2 fixed the job list up front, Chapter 26.6 reconciled against it rather than against a process. The jobs are already independent, which is why this module is five chapters rather than twenty.
3. The Famous Limit
def amdahl(serial_fraction, workers):
return 1 / (serial_fraction + (1 - serial_fraction) / workers)
print(f"{'serial part':>12} {'8':>8} {'32':>8} {'128':>8} {'ceiling':>9}")
for s in (0.00, 0.01, 0.05, 0.20):
row = " ".join(f"{amdahl(s, n):>8.1f}" for n in (8, 32, 128))
ceiling = "unbounded" if s == 0 else f"{1 / s:.0f}x"
print(f"{s:>12.0%} {row} {ceiling:>9}")
print()
print("at 5% serial, 128 workers buy:", f"{amdahl(0.05, 128):.1f}x")
print("and the ceiling is :", f"{1 / 0.05:.0f}x")
print("so the last 96 workers buy :", f"{amdahl(0.05, 128) - amdahl(0.05, 32):.1f}x more") serial part 8 32 128 ceiling
0% 8.0 32.0 128.0 unbounded
1% 7.5 24.4 56.4 100x
5% 5.9 12.5 17.4 20x
20% 3.3 4.4 4.8 5x
at 5% serial, 128 workers buy: 17.4x
and the ceiling is : 20x
so the last 96 workers buy : 4.9x moreAmdahl's law: whatever fraction of the work cannot be parallelised sets a ceiling no number of workers can exceed. At twenty percent serial, a hundred and twenty-eight workers buy 4.8×.
It is the right tool for asking the question. It is also, for a verification regression, usually the wrong answer:
PARTS = [("expand the job list", 1, "Chapter 26.2, once"),
("compile the design", 1, "Chapter 25.3 §7, shared by every job"),
("run the jobs", 600, "independent"),
("aggregate and report", 1, "Chapter 26.6, once")]
total = sum(units for _, units, _ in PARTS)
serial = sum(units for name, units, _ in PARTS if name != "run the jobs")
fraction = serial / total
for name, units, note in PARTS:
kind = "parallel" if name == "run the jobs" else "serial"
print(f"{name:<22} {units:>4} unit(s) {kind:<9} {note}")
print()
print("serial units :", serial, "of", total)
print("serial fraction :", f"{fraction:.4f}")
print("Amdahl ceiling :", f"{1 / fraction:.0f}x")
print("workers anybody runs:", "tens, not hundreds")expand the job list 1 unit(s) serial Chapter 26.2, once
compile the design 1 unit(s) serial Chapter 25.3 §7, shared by every job
run the jobs 600 unit(s) parallel independent
aggregate and report 1 unit(s) serial Chapter 26.6, once
serial units : 3 of 603
serial fraction : 0.0050
Amdahl ceiling : 201x
workers anybody runs: tens, not hundredsA theoretical ceiling of 201× and nobody runs more than a few dozen workers. Amdahl's law does not bind a verification regression, and saying so plainly is more useful than reciting it as though it did.
Where it does bind is the diagnosis, and that is the form worth keeping:
If your measured speedup is far below your worker count, something has been serialised by accident.
A compile that is not shared is the usual culprit — remove Chapter 25.3 Section 7's cache and every job compiles, which moves the serial fraction from half a percent to most of the run. A shared lock, a single licence, one NFS directory every job writes to: each of those converts the parallel part back into a serial one, and Amdahl is how you put a number on what it cost.
4. The Limit That Does Bind
JOBS = [("smoke", 12)] * 560 + [("burst", 45)] * 35 + [("gate_soak", 360)] * 5
total = sum(m for _, m in JOBS)
longest = max(m for _, m in JOBS)
print("jobs :", len(JOBS))
print("total work :", total, "minutes")
print("longest single job :", longest, "minutes")
print()
print(f"{'workers':>8} {'ideal':>8} {'floor':>8} {'efficiency':>11}")
for w in (8, 32, 128, 600):
ideal = total / w
floor = max(ideal, longest)
print(f"{w:>8} {ideal:>8.0f} {floor:>8.0f} {100 * ideal / floor:>10.1f}%")
print()
print("maximum useful speedup:", f"{total / longest:.1f}x")
print("Amdahl said :", "201x")
print("which one binds :", "the longest job")jobs : 600
total work : 10095 minutes
longest single job : 360 minutes
workers ideal floor efficiency
8 1262 1262 100.0%
32 315 360 87.6%
128 79 360 21.9%
600 17 360 4.7%
maximum useful speedup: 28.0x
Amdahl said : 201x
which one binds : the longest jobA regression cannot finish faster than its longest single job, however many workers are available. Five gate-level soaks at six hours each set a floor of six hours, and the other 595 jobs are irrelevant to it.
That is the limit Section 3 was looking for and did not find. Amdahl's law measured the wrong thing — it asks what fraction of the work is serial, and the answer was half a percent — while the binding constraint is the largest indivisible unit, which gives a real cap of 28× rather than a theoretical 201×.
Where it starts to bite is worth locating precisely:
JOBS = [("smoke", 12)] * 560 + [("burst", 45)] * 35 + [("gate_soak", 360)] * 5
total = sum(m for _, m in JOBS)
longest = max(m for _, m in JOBS)
print(f"{'workers':>8} {'ideal':>7} binding constraint")
for w in (8, 16, 24, 32, 64):
ideal = total / w
print(f"{w:>8} {ideal:>7.0f} {'the long job' if longest > ideal else 'the total work'}")
crossover = total / longest
print()
print("the crossover is at:", f"{crossover:.0f} workers")
print("below it, more workers help:", True)
print("above it, more workers buy :", "nothing at all") workers ideal binding constraint
8 1262 the total work
16 631 the total work
24 421 the total work
32 315 the long job
64 158 the long job
the crossover is at: 28 workers
below it, more workers help: True
above it, more workers buy : nothing at allTwenty-eight workers, and the twenty-ninth buys nothing. Chapter 27.5's resource calculation will probably land below that anyway — twelve licences, say — but knowing where the ceiling is changes what buying more licences is worth.
Two consequences follow, and both are about the job list rather than about the pool.
Start the long jobs first. A scheduler that happens to run the six-hour soak last finishes six hours after everything else is done. Sorting the job list by expected duration, longest first, is the standard greedy heuristic and it costs one sorted call — Chapter 26.2 Section 4 established that the clause order decides the job order, and this is the second reason to care about it.
Split the long job, or accept the floor. A six-hour gate-level soak that could be four ninety-minute runs at different seeds has a floor of ninety minutes instead of six hours, for the same total work. That is a verification decision rather than a flow one, and it is the only thing that actually moves this limit.
5. What It Actually Buys
BUYS = [
("faster feedback", "a change is tested within hours, not days"),
("more seeds in the same window", "Chapter 26.3's sweep can be wider"),
("a regression that runs nightly at all", "5 days does not fit in a night"),
]
DOES_NOT_BUY = [
("correctness", "the same jobs produce the same results"),
("reliability", "Chapter 25.7's non-determinism rate is unchanged"),
("fewer failures", "it finds the same bugs, sooner"),
("less compute", "the total work is identical"),
]
print("what concurrency buys:")
for what, why in BUYS:
print(f" {what:<32} {why}")
print()
print("what it does not:")
for what, why in DOES_NOT_BUY:
print(f" {what:<32} {why}")
print()
print("the second list is longer:", len(DOES_NOT_BUY) > len(BUYS))
print("and the first item on it is:", DOES_NOT_BUY[0][0])what concurrency buys:
faster feedback a change is tested within hours, not days
more seeds in the same window Chapter 26.3's sweep can be wider
a regression that runs nightly at all 5 days does not fit in a night
what it does not:
correctness the same jobs produce the same results
reliability Chapter 25.7's non-determinism rate is unchanged
fewer failures it finds the same bugs, sooner
less compute the total work is identical
the second list is longer: True
and the first item on it is: correctnessThe second entry in the first list is the underrated one. The same window now fits more seeds, so a nightly that managed one seed per test can afford five — and Chapter 26.3 Section 7 showed that distinct findings come from seed count rather than from test count. Parallelism's real contribution to quality is indirect: it buys coverage, not correctness.
The fourth entry in the second list is the one people forget when costing it. Thirty-two workers for four hours is the same compute as one worker for five days, plus overhead. Concurrency is not cheaper; it is sooner.
6. It Does Not Fix an Unreliable Job
JOBS = 600
print(f"{'non-determinism':>17} {'serial wrong':>14} {'parallel wrong':>16}")
for rate in (0.00, 0.01, 0.02, 0.05):
wrong = rate * JOBS
print(f"{rate:>17.0%} {wrong:>14.0f} {wrong:>16.0f}")
print()
print("the rate is a property of:", "the job, not the scheduling")
print("what parallelism changes :", "how quickly the wrong answer arrives")
print()
print("a 2% rate over 600 jobs :", f"{0.02 * JOBS:.0f} wrong results")
print("delivered serially in :", "5 days")
print("delivered in parallel in :", "4 hours")
print("improvement in accuracy :", "none") non-determinism serial wrong parallel wrong
0% 0 0
1% 6 6
2% 12 12
5% 30 30
the rate is a property of: the job, not the scheduling
what parallelism changes : how quickly the wrong answer arrives
a 2% rate over 600 jobs : 12 wrong results
delivered serially in : 5 days
delivered in parallel in : 4 hours
improvement in accuracy : noneThe columns are identical. Chapter 25.7 established that a non-determinism rate is a property of the machinery, measured by running one test many times at one seed; nothing about running six hundred different jobs at once changes it.
This is why Chapter 25.7 came before this module rather than after. The soak drives the rate to zero first, and parallelism then multiplies a trustworthy result. Applied in the other order it multiplies an untrustworthy one — twelve wrong results per night instead of twelve per five nights, which is a worse situation arriving faster.
7. And It Can Make Things Worse
HAZARDS = [
("two jobs in one directory", "unreachable", "reachable", "Chapter 26.4 §2"),
("a shared log file", "unreachable", "reachable", "Chapter 26.4 §2"),
("licence pool exhausted", "impossible with 1 job", "routine", "Chapter 27.5"),
("memory exhausted", "one job's footprint", "n jobs' footprint", "Chapter 27.5"),
("an exception inside a worker", "propagates", "can be swallowed", "Chapter 27.4"),
("results arriving out of order", "never", "always", "Chapter 27.4"),
]
print(f"{'hazard':<30} {'serial':<22} {'parallel':<20} where")
for hazard, serial, parallel, where in HAZARDS:
print(f"{hazard:<30} {serial:<22} {parallel:<20} {where}")
print()
newly = sum(1 for _, s, _, _ in HAZARDS if s in ("unreachable", "never", "impossible with 1 job"))
print("hazards that did not exist serially:", newly, "of", len(HAZARDS))
print("hazards parallelism removes :", 0)hazard serial parallel where
two jobs in one directory unreachable reachable Chapter 26.4 §2
a shared log file unreachable reachable Chapter 26.4 §2
licence pool exhausted impossible with 1 job routine Chapter 27.5
memory exhausted one job's footprint n jobs' footprint Chapter 27.5
an exception inside a worker propagates can be swallowed Chapter 27.4
results arriving out of order never always Chapter 27.4
hazards that did not exist serially: 4 of 6
hazards parallelism removes : 0Four new hazards, none removed. That is the honest ledger, and it is why Chapter 27.4 exists as a chapter of its own.
The first two are Chapter 26.4's, and the relationship is worth stating precisely: Module 26 fixed them before they could happen. A serial regression with a shared directory works perfectly, so the bug is latent — it is created by the directory naming and activated by concurrency. Chapter 26.4 was marked Critical for a reason that only becomes visible here.
The last row is the one that surprises people. Results arrive in completion order, not submission order, so anything that assumed the job list's order — a progress log, a comparison against yesterday by position, a report that walks results and tests in parallel — is now wrong. Chapter 26.6's aggregation survives because it reconciles by job identity rather than by position, which was not an accident.
8. The Decision
def should_parallelise(serial_days, window_hours, non_determinism_rate,
directories_isolated):
reasons = []
if serial_days * 24 <= window_hours:
reasons.append("it already fits in the window")
if non_determinism_rate > 0:
reasons.append(f"non-determinism is {non_determinism_rate:.0%} — fix that first (25.7)")
if not directories_isolated:
reasons.append("jobs are not directory-isolated (26.4)")
return (not reasons), reasons
CASES = [
("ready", 5.0, 10, 0.00, True),
("already fits", 0.3, 10, 0.00, True),
("unreliable", 5.0, 10, 0.02, True),
("not isolated", 5.0, 10, 0.00, False),
]
for name, days, window, rate, isolated in CASES:
ok, reasons = should_parallelise(days, window, rate, isolated)
print(f"{name:<14} parallelise={str(ok):<6} {reasons if reasons else ''}")ready parallelise=True
already fits parallelise=False ['it already fits in the window']
unreliable parallelise=False ['non-determinism is 2% — fix that first (25.7)']
not isolated parallelise=False ['jobs are not directory-isolated (26.4)']Three preconditions, and the middle one is the one teams skip.
It must not already fit. A regression that finishes in seven hours against a ten-hour window gains nothing from concurrency except Chapter 27.4's hazards. Speed is worth having when it changes what is possible — nightly instead of weekly — and not otherwise.
The non-determinism rate must be zero, by Chapter 25.7's soak. Parallelising first means every later investigation has two candidate explanations, and the farm is always the more comfortable one to blame.
The jobs must be directory-isolated, which Module 26 already did. If they are not, Chapter 26.4 is the prerequisite and not an optimisation.
The function returns reasons rather than a boolean alone, which is Chapter 24.5's pattern: a decision that explains itself can be argued with, and one that does not gets overridden silently.
9. Predict, Run, Observe
def amdahl(s, n):
return 1 / (s + (1 - s) / n)
J, M = 600, 12
serial = J * M
# A
print("A:", serial, f"{serial / 1440:.1f}")
# B
print("B:", f"{serial / 32 / 60:.2f}")
# C
print("C:", f"{amdahl(0.0, 32):.1f}")
# D
print("D:", f"{amdahl(0.05, 32):.1f}", f"{amdahl(0.05, 128):.1f}")
# E
print("E:", f"{1 / 0.05:.0f}")
# F
print("F:", f"{3 / 603:.4f}", f"{603 / 3:.0f}")
# G
print("G:", 0.02 * J, 0.02 * J)
# H
print("H:", 32 * 3.75, 1 * 120)The verified output:
A: 7200 5.0
B: 3.75
C: 32.0
D: 12.5 17.4
E: 20
F: 0.0050 201
G: 12.0 12.0
H: 120.0 120A — the serial total, in minutes and days.
B — thirty-two workers, under four hours.
C — with nothing serial, speedup equals the worker count exactly.
D — with five percent serial, quadrupling the workers buys less than five times.
E — and the ceiling is twenty whatever you do.
F — a regression's actual serial fraction, and its ceiling: not the binding constraint.
G — the wrong-result count, serial and parallel. Identical.
H — and the total compute is identical too: concurrency is sooner, not cheaper.
10. Exercises
Exercise 1 — your own number. Compute your regression's serial duration from its job count and a typical per-job time. Report runs per week.
Exercise 2 — the archive. Say at what point a regression stops being a safety net for your team's change rate.
Exercise 3 — the table. Produce the worker/hours/speedup table for your own job count.
Exercise 4 — the assumption. Name the Module 26 chapters that make the linear speedup column plausible.
Exercise 5 — Amdahl. Compute the speedup at 8, 32 and 128 workers for serial fractions of 1%, 5% and 20%.
Exercise 6 — your serial part. List what in your flow runs once, and compute the fraction.
Exercise 7 — the ceiling. Report your Amdahl ceiling and compare it with the worker count you would actually use.
Exercise 8 — the diagnosis. State the useful form of Amdahl's law for a regression, in one sentence.
Exercise 9 — accidentally serialised. Name three things that could serialise your parallel part, and say what each would do to the fraction.
Exercise 10 — the floor. Find your suite's longest single job and compute the maximum useful speedup as total over longest.
Exercise 11 — the crossover. Work out the worker count above which more workers buy nothing.
Exercise 12 — two caps. Compare your Amdahl ceiling with your makespan floor and say which binds.
Exercise 13 — longest first. Sort your job list by expected duration and say what it saves, citing Chapter 26.2.
Exercise 14 — split it. Identify a long job that could be several shorter ones and compute the new floor.
Exercise 15 — mean or max. Say which of the two sets the floor and which you budget with.
Exercise 16 — two lists. Write out what concurrency buys and does not buy for your flow.
Exercise 11 — coverage, not correctness. Explain parallelism's indirect contribution to quality, citing Chapter 26.3.
Exercise 12 — not cheaper. Compute the total compute for your serial and parallel plans and compare.
Exercise 13 — the identical columns. Show that a non-determinism rate produces the same number of wrong results either way.
Exercise 14 — order of operations. Say why Chapter 25.7's soak must come before this module.
Exercise 15 — the comfortable explanation. Describe how twelve spurious failures a night get attributed to the farm.
Exercise 16 — the ledger. List the hazards concurrency adds for your flow and the ones it removes.
Exercise 17 — latent, then activated. Explain why a serial regression with a shared directory works perfectly.
Exercise 18 — out of order. Find something in your flow that assumes the job list's order.
Exercise 19 — three preconditions. Check all three for your regression and report which fail.
Exercise 20 — reasons, not a boolean. Say why the decision function returns its reasons.
11. Summary
- A five-day regression runs 1.4 times a week, which is an archive rather than a safety net.
- Every figure here is arithmetic over configured inputs; Chapter 27.2 is about measuring your own.
- The annotation's arithmetic holds: 600 × 12 minutes is five days, and 32 at a time is 3.75 hours.
- The speedup column is linear because the jobs are independent, which Module 26 spent nine chapters making true.
- Amdahl's law caps speedup at the reciprocal of the serial fraction, regardless of worker count.
- It does not bind a verification regression: a serial fraction of 0.5% gives a ceiling of 201×, and nobody runs that many workers.
- Its useful form is a diagnosis — a measured speedup far below the worker count means something was serialised by accident, usually an unshared compile.
- A regression cannot finish faster than its longest single job, whatever the worker count.
- That floor is the limit that actually binds — a real cap of 28× against Amdahl's theoretical 201×.
- Above the crossover, more workers buy nothing at all, which changes what more licences are worth.
- Start the long jobs first: a six-hour job scheduled last finishes six hours after everything else.
- Splitting a long job into several shorter ones is the only thing that moves the floor, and it is a verification decision rather than a flow one.
- The mean per-job duration is what you budget with; the maximum is what sets the floor.
- Concurrency buys faster feedback and more seeds in the same window, and Chapter 26.3 showed that seeds are where distinct findings come from.
- It does not buy correctness, reliability, fewer failures or less compute. It is sooner, not cheaper.
- A non-determinism rate is unchanged by parallelism — twelve wrong results either way, delivered in four hours instead of five days.
- Chapter 25.7's soak must come first, or every later investigation has the farm as a comfortable alternative explanation.
- Concurrency adds four hazards and removes none.
- Chapter 26.4's directory collision was latent and is activated here — a serial regression with a shared directory works perfectly.
- Results arrive in completion order, so anything that assumed the job list's order is now wrong; Chapter 26.6 survives because it reconciles by identity.
- Three preconditions: it must not already fit the window, the non-determinism rate must be zero, and the jobs must be directory-isolated.
- Return the reasons, not just the verdict, so the decision can be argued with.
The next chapter measures the serial regression properly, so that every later comparison has something to be compared against.
Where this fits
Part of the Python curriculum.