Python Part II · Module 27 · Parallel Regression
Cores, Memory and Licences
Chapter 27.3 chose a worker count from four limits and Chapter 27.4 showed what happens when it is chosen too high: a worker is killed by the kernel, the pool breaks, and dozens of healthy jobs are reclassified as failures. So the number matters more than it looks, and the one-line expression everybody writes for it is the right shape built on three inputs that are each harder to obtain correctly than they appear — one of which reports a number about the wrong machine entirely.
Advanced19 min readPythonPart IIAutomationRegressionVerification Flow
Part II · Module 27 · Chapter 27.5 · Cores, Memory and Licences
1. Cores Are Almost Never the Limit
LIMITS = {"cores": 64, "memory": 32, "licences": 12, "disk bandwidth": 16}
for name, value in sorted(LIMITS.items(), key=lambda kv: kv[1]):
print(f"{name:<16} allows {value:>3} workers")
binding = min(LIMITS, key=LIMITS.get)
print()
print("workers :", min(LIMITS.values()))
print("binding constraint:", binding)
print("cores would allow :", LIMITS["cores"])
print("ratio :", f"{LIMITS['cores'] / min(LIMITS.values()):.1f}x too many")licences allows 12 workers
disk bandwidth allows 16 workers
memory allows 32 workers
cores allows 64 workers
workers : 12
binding constraint: licences
cores would allow : 64
ratio : 5.3x too manyChapter 27.3 Section 8's table, sorted so the answer is on the first line. Cores are the largest of the four and therefore never the answer — which is awkward, because os.cpu_count() is the number everybody reaches for and it is the one number in the list that cannot be the binding constraint on a well-provisioned machine.
All four figures are configured inputs. Where each actually comes from is the rest of this chapter.
2. The Annotation
workers = min(os.cpu_count(), licences_available, memory_gb // 8)The shape is right: a min over independent limits, each expressed in workers. Three things about the inputs are not.
import os
import sys
print("python :", sys.version.split()[0])
print("os.cpu_count exists :", os.cpu_count() is not None)
print("os.process_cpu_count exists :", hasattr(os, "process_cpu_count"))
print("os.sched_getaffinity exists :", hasattr(os, "sched_getaffinity"))
print()
if hasattr(os, "process_cpu_count"):
print("the two agree on this machine:", os.cpu_count() == os.process_cpu_count())
print()
print("the values themselves are not printed:", "they differ on every machine")python : 3.13.5
os.cpu_count exists : True
os.process_cpu_count exists : True
os.sched_getaffinity exists : False
the two agree on this machine: True
the values themselves are not printed: they differ on every machineos.cpu_count() reports the machine; os.process_cpu_count() reports what this process may use. They agree here because nothing is restricting this process — and they disagree in exactly the environments a regression runs in:
SITUATIONS = [
("a bare workstation", "agree", "nothing restricts the process"),
("inside a container with a CPU quota", "cpu_count over-reports", "it sees the host"),
("under taskset or an affinity mask", "cpu_count over-reports", "it sees all cores"),
("an LSF slot with 4 CPUs on a 64-core host", "cpu_count over-reports", "by 16x"),
]
print(f"{'situation':<44} {'behaviour':<24} why")
for where, behaviour, why in SITUATIONS:
print(f"{where:<44} {behaviour:<24} {why}")
print()
over = sum(1 for _, b, _ in SITUATIONS if b != "agree")
print("situations where cpu_count over-reports:", over, "of", len(SITUATIONS))
print("and those are :", "the ones a regression runs in")situation behaviour why
a bare workstation agree nothing restricts the process
inside a container with a CPU quota cpu_count over-reports it sees the host
under taskset or an affinity mask cpu_count over-reports it sees all cores
an LSF slot with 4 CPUs on a 64-core host cpu_count over-reports by 16x
situations where cpu_count over-reports: 3 of 4
and those are : the ones a regression runs inThe last row is the one that matters here. Chapter 26.9 hands jobs to a scheduler that allocates a slot — four CPUs, say — and os.cpu_count() inside that slot cheerfully reports sixty-four. A pool sized from it runs sixteen times more workers than the slot was granted, and the scheduler's response is to throttle or kill them.
import os
def available_cpus():
"""What this process may actually use, with a portable fallback."""
if hasattr(os, "process_cpu_count"):
return os.process_cpu_count()
if hasattr(os, "sched_getaffinity"):
return len(os.sched_getaffinity(0))
return os.cpu_count() or 1
print("available_cpus() returns an int:", isinstance(available_cpus(), int))
print("and it is at least 1 :", available_cpus() >= 1)
print("never None :", available_cpus() is not None)
print()
print("the fallback chain:")
print(" os.process_cpu_count() 3.13+, the right answer")
print(" len(os.sched_getaffinity(0)) Linux, pre-3.13")
print(" os.cpu_count() or 1 anywhere, over-reports")available_cpus() returns an int: True
and it is at least 1 : True
never None : True
the fallback chain:
os.process_cpu_count() 3.13+, the right answer
len(os.sched_getaffinity(0)) Linux, pre-3.13
os.cpu_count() or 1 anywhere, over-reportsThe or 1 on the last line is not decoration: os.cpu_count() can return None, and a None flowing into a min raises a TypeError comparing int and NoneType — far from the cause, which is the recurring shape of every defect in this chapter.
3. A Zero Limit Is Silent
from concurrent.futures import ProcessPoolExecutor
def annotation_form(cpus, licences, memory_gb):
return min(cpus, licences, memory_gb // 8)
CASES = [(64, 12, 256), (64, 100, 32), (8, 100, 256), (64, 0, 256), (64, 12, 4)]
for cpus, lic, mem in CASES:
n = annotation_form(cpus, lic, mem)
print(f"cpus={cpus:<4} licences={lic:<4} memory={mem:<4} -> workers={n}")
print()
print("min() complained about the zero:", False)
try:
ProcessPoolExecutor(max_workers=0)
except ValueError as exc:
print("the pool did :", exc)
print()
print("where the error points:", "the pool construction")
print("where the cause is :", "the licence server returned 0, three functions earlier")cpus=64 licences=12 memory=256 -> workers=12
cpus=64 licences=100 memory=32 -> workers=4
cpus=8 licences=100 memory=256 -> workers=8
cpus=64 licences=0 memory=256 -> workers=0
cpus=64 licences=12 memory=4 -> workers=0
min() complained about the zero: False
the pool did : max_workers must be greater than 0
where the error points: the pool construction
where the cause is : the licence server returned 0, three functions earlierTwo of five configurations give zero workers and min is perfectly happy with both.
The fourth row is a licence server that reported nothing available — unreachable, or genuinely exhausted. The fifth is a machine with four gigabytes, which at eight per job allows nothing. Neither is an unreasonable thing to happen, and both produce a ValueError from ProcessPoolExecutor whose message says nothing about licences or memory.
This is Chapter 24.5's argument in a new place: check each input where its meaning is known, not after it has been collapsed into one number.
4. Per-Job Memory Is Not a Constant
TOTAL_GB = 256
FOOTPRINTS = [("axi_smoke", 2.1), ("axi_burst", 3.4), ("gate_soak", 28.0)]
for name, gb in FOOTPRINTS:
print(f"{name:<11} {gb:>5.1f} GB -> {int(TOTAL_GB // gb):>3} workers")
worst = max(gb for _, gb in FOOTPRINTS)
mean = sum(gb for _, gb in FOOTPRINTS) / len(FOOTPRINTS)
print()
print("sizing on the worst case:", int(TOTAL_GB // worst), "workers")
print("sizing on the mean :", int(TOTAL_GB // mean), "workers")
print("oversubscription :", f"{int(TOTAL_GB // mean) / int(TOTAL_GB // worst):.1f}x")
print()
print("what oversubscription causes:", "the OOM killer")
print("what the OOM killer causes :", "BrokenProcessPool for every outstanding job")axi_smoke 2.1 GB -> 121 workers
axi_burst 3.4 GB -> 75 workers
gate_soak 28.0 GB -> 9 workers
sizing on the worst case: 9 workers
sizing on the mean : 22 workers
oversubscription : 2.4x
what oversubscription causes: the OOM killer
what the OOM killer causes : BrokenProcessPool for every outstanding jobThirteen-fold variation between the smallest and largest test, and the annotation's constant 8 is somewhere in the middle of it.
Sizing on the mean oversubscribes by 2.4×, which is fine right up until several gate-level jobs happen to be running at once — and then it is Chapter 27.4 Section 4's kill, which does not fail one job but breaks the pool.
Three ways to handle it, in increasing order of effort:
TOTAL_GB = 256
FOOTPRINTS = {"axi_smoke": 2.1, "axi_burst": 3.4, "gate_soak": 28.0}
STRATEGIES = [
("worst case", int(TOTAL_GB // max(FOOTPRINTS.values())),
"safe, and wastes the machine on small tests"),
("per-group pools", None, "a big-job pool of 9 and a small-job pool of 70"),
("a memory budget", None, "admit jobs while the sum of footprints fits"),
]
print(f"{'strategy':<18} {'workers':<9} note")
for name, n, note in STRATEGIES:
print(f"{name:<18} {str(n) if n else 'varies':<9} {note}")
print()
budget, admitted = TOTAL_GB, []
for name, gb in sorted(FOOTPRINTS.items(), key=lambda kv: -kv[1]):
while budget >= gb:
admitted.append(name)
budget -= gb
print("a greedy budget admits:", len(admitted), "jobs")
print("largest first :", admitted[0])
print("memory left :", f"{budget:.1f} GB")strategy workers note
worst case 9 safe, and wastes the machine on small tests
per-group pools varies a big-job pool of 9 and a small-job pool of 70
a memory budget varies admit jobs while the sum of footprints fits
a greedy budget admits: 10 jobs
largest first : gate_soak
memory left : 0.6 GBWorst-case sizing is the right default and the other two are what to do when it costs too much.
The greedy budget admits ten jobs: nine gate-level soaks at 28 GB fill 252 of the 256, one 3.4 GB job fits in the remainder, and the 2.1 GB jobs do not — 0.6 GB is left stranded. That is more than the nine that worst-case sizing allows and far fewer than the machine could hold if the big jobs ran separately, which is the honest summary of a greedy packer: better than a constant, worse than scheduling the big jobs apart.
It also shows the catch. A budget scheduler needs each job's footprint before running it, and that number comes from measurement rather than from the suite file. Chapter 25.7's soak is where it is obtained, with resource monitoring added.
5. Licences Are a Shared, Moving Number
PROBLEMS = [
("the count is a snapshot", "another team's regression starts at the same moment"),
("your own jobs hold them", "n workers hold n licences for the whole run"),
("checkout is not instant", "a job can wait minutes before failing"),
("some tools hold extras", "a waveform viewer or a debug feature takes its own"),
]
for what, why in PROBLEMS:
print(f"{what:<28} {why}")
print()
TOTAL, OTHERS, MARGIN = 20, 5, 2
usable = TOTAL - OTHERS - MARGIN
print("licences in the pool :", TOTAL)
print("held by others right now:", OTHERS)
print("safety margin :", MARGIN)
print("workers we may use :", usable)
print()
print("without the margin :", TOTAL - OTHERS)
print("what the margin is for :", "an interactive debug session somebody starts at 10pm")the count is a snapshot another team's regression starts at the same moment
your own jobs hold them n workers hold n licences for the whole run
checkout is not instant a job can wait minutes before failing
some tools hold extras a waveform viewer or a debug feature takes its own
licences in the pool : 20
held by others right now: 5
safety margin : 2
workers we may use : 13
without the margin : 15
what the margin is for : an interactive debug session somebody starts at 10pmA licence count is a snapshot of a shared resource, which makes it unlike the other three inputs: cores and memory belong to the machine, and licences belong to everybody.
The margin is the practical consequence. A regression that consumes every free licence is a regression that stops a colleague debugging, and the social cost of that is higher than the few minutes the extra workers would have saved. Leaving two is cheap insurance.
What happens when the pool runs out mid-run is the more interesting question:
OUTCOMES = [
("the tool waits and eventually succeeds", "slow", "PASS", "fine"),
("the tool waits and times out", "slow", "NO_FINISH / TIMEOUT", "retryable — 26.5"),
("the tool exits immediately", "fast", "ERROR", "retryable — 26.5"),
("the tool runs without the feature", "fast", "PASS", "the dangerous one"),
]
print(f"{'what the tool does':<40} {'speed':<7} {'status':<22} verdict")
for what, speed, status, verdict in OUTCOMES:
print(f"{what:<40} {speed:<7} {status:<22} {verdict}")
print()
print("statuses that are honest :", 3)
print("the dangerous one :", OUTCOMES[-1][0])
print("why :", "a PASS from a run that did not do what you asked")what the tool does speed status verdict
the tool waits and eventually succeeds slow PASS fine
the tool waits and times out slow NO_FINISH / TIMEOUT retryable — 26.5
the tool exits immediately fast ERROR retryable — 26.5
the tool runs without the feature fast PASS the dangerous one
statuses that are honest : 3
the dangerous one : the tool runs without the feature
why : a PASS from a run that did not do what you askedThree of four outcomes are honest and Chapter 26.5's retry policy already handles two of them — a licence ERROR or TIMEOUT is exactly what it is willing to retry.
The fourth is the one to defend against. A tool that cannot check out an optional feature licence and silently proceeds without it produces a passing run with, say, no assertions enabled or no coverage collected. Nothing in Chapter 25.6's classification can see it: the return code is zero, the log has no errors, the end banner is there. The defences are Chapter 25.6's artefact check — if coverage was requested, a coverage database must exist — and reading the tool's own licence warnings out of the log.
6. Choosing the Number
import os
class ResourceError(Exception):
"""No worker count satisfies the constraints."""
def available_cpus():
if hasattr(os, "process_cpu_count"):
return os.process_cpu_count()
if hasattr(os, "sched_getaffinity"):
return len(os.sched_getaffinity(0))
return os.cpu_count() or 1
def choose_workers(*, cpus, licences_free, licence_margin, memory_gb, worst_job_gb,
disk_mb_s, job_disk_mb_s, requested=None):
limits = {
"cpu": cpus,
"licences": licences_free - licence_margin,
"memory": int(memory_gb // worst_job_gb),
"disk": int(disk_mb_s // job_disk_mb_s),
}
if requested is not None:
limits["requested"] = requested
empty = sorted(k for k, v in limits.items() if v < 1)
if empty:
raise ResourceError(
f"{', '.join(empty)} allow(s) no workers "
f"({ {k: limits[k] for k in empty} }) — nothing can run")
n = min(limits.values())
binding = sorted(k for k, v in limits.items() if v == n)
return n, binding, limits
SPEC = dict(cpus=64, licences_free=20, licence_margin=2, memory_gb=256,
worst_job_gb=28.0, disk_mb_s=400, job_disk_mb_s=25)
n, binding, limits = choose_workers(**SPEC)
print("limits :", dict(sorted(limits.items())))
print("workers :", n)
print("binding :", binding)
print("in a real flow, cpus comes from available_cpus():", callable(available_cpus))
print()
for label, kwargs in (("licence server down", {"licences_free": 0, "licence_margin": 0}),
("tiny machine", {"memory_gb": 4, "worst_job_gb": 28.0})):
args = dict(SPEC)
args.update(kwargs)
try:
choose_workers(**args)
except ResourceError as exc:
print(f"{label:<21} ResourceError: {exc}")limits : {'cpu': 64, 'disk': 16, 'licences': 18, 'memory': 9}
workers : 9
binding : ['memory']
in a real flow, cpus comes from available_cpus(): True
licence server down ResourceError: licences allow(s) no workers ({'licences': 0}) — nothing can run
tiny machine ResourceError: memory allow(s) no workers ({'memory': 0}) — nothing can runFive decisions, and the second is the one that turns Section 3's silent zero into a message.
Every limit is named, so the function can say which one bound. Chapter 27.3 Section 8's point: the number says what to type and the name says what to buy.
A limit below one raises, naming it. ResourceError: licences allow(s) no workers arrives at the point the licence count was consulted, not three functions later from ProcessPoolExecutor.
binding is a list, because two limits can tie — and a tie is worth seeing, since raising either one alone buys nothing.
cpus is a parameter, not a call. In a real flow it is available_cpus(); passing it in keeps the function testable with a fixed value, which is the same argument Chapter 25.1 Section 9 made for a runner that takes its inputs rather than discovering them.
requested is a limit like any other, so a user asking for --workers 4 on a machine that allows 9 gets 4, and the binding constraint is reported as requested — which stops somebody concluding the licence pool is small.
7. Failing Clearly When It Runs Out
from collections import Counter
RESULTS = [("axi_smoke", "PASS", None), ("axi_burst", "ERROR", "no licence"),
("axi_mem", "ERROR", "no licence"), ("gate_soak", "ERROR", "no licence"),
("axi_cfg", "PASS", None), ("axi_x", "ERROR", "no licence")]
counts = Counter(status for _, status, _ in RESULTS)
licence_errors = sum(1 for _, s, r in RESULTS if s == "ERROR" and r == "no licence")
print("jobs :", len(RESULTS))
print("statuses :", dict(sorted(counts.items())))
print("licence failures:", licence_errors)
print("as a fraction :", f"{licence_errors / len(RESULTS):.0%}")
print()
if licence_errors > len(RESULTS) * 0.1:
print("DIAGNOSIS: the worker count exceeded the licence pool")
print("ACTION : reduce workers, do not retry")
else:
print("DIAGNOSIS: incidental licence contention")
print("ACTION : retry is appropriate — Chapter 26.5")jobs : 6
statuses : {'ERROR': 4, 'PASS': 2}
licence failures: 4
as a fraction : 67%
DIAGNOSIS: the worker count exceeded the licence pool
ACTION : reduce workers, do not retryTwo thirds of the jobs failed for a licence and retrying them is the wrong response.
This is Chapter 26.5's retry policy meeting its limit. A licence ERROR is retryable in principle — the design was never exercised — and a systemic licence shortage is not an incidental failure at all: retrying sixty-seven percent of the jobs against a pool that is too small produces the same sixty-seven percent again, having spent the retry budget. Chapter 26.5 Section 5's global cap is what stops that becoming four hundred wasted runs, and hitting it is the diagnosis.
The threshold turns a pile of individual failures into one statement about configuration, which is the same move Chapter 26.7 made with buckets: a cause is visible in the proportion and invisible in the rows.
8. The Whole Decision
import os
class ResourceError(Exception):
"""No worker count satisfies the constraints."""
def available_cpus():
if hasattr(os, "process_cpu_count"):
return os.process_cpu_count()
if hasattr(os, "sched_getaffinity"):
return len(os.sched_getaffinity(0))
return os.cpu_count() or 1
def plan_workers(spec):
limits = {"cpu": spec["cpus"],
"licences": spec["licences_free"] - spec["licence_margin"],
"memory": int(spec["memory_gb"] // spec["worst_job_gb"]),
"disk": int(spec["disk_mb_s"] // spec["job_disk_mb_s"])}
if spec.get("requested"):
limits["requested"] = spec["requested"]
empty = sorted(k for k, v in limits.items() if v < 1)
if empty:
raise ResourceError(f"no workers possible: {empty} allow(s) none")
n = min(limits.values())
return {"workers": n,
"binding": sorted(k for k, v in limits.items() if v == n),
"headroom": {k: v - n for k, v in sorted(limits.items())},
"limits_named": sorted(limits)}
SPEC = {"cpus": 64, "licences_free": 20, "licence_margin": 2, "memory_gb": 256,
"worst_job_gb": 28.0, "disk_mb_s": 400, "job_disk_mb_s": 25}
plan = plan_workers(SPEC)
for key in ("workers", "binding", "headroom", "limits_named"):
print(f"{key:<14} {plan[key]}")
print()
asked = plan_workers(dict(SPEC, requested=4))
print("with --workers 4:")
print(" workers:", asked["workers"], "| binding:", asked["binding"])
print()
print("report line:", f"running {plan['workers']} workers, limited by {plan['binding'][0]}")workers 9
binding ['memory']
headroom {'cpu': 55, 'disk': 7, 'licences': 9, 'memory': 0}
limits_named ['cpu', 'disk', 'licences', 'memory']
with --workers 4:
workers: 4 | binding: ['requested']
report line: running 9 workers, limited by memoryFour properties worth copying, and the third is the one nobody writes.
The report line is the output. running 9 workers, limited by memory is one line in a regression log that answers, forever, why last Tuesday's run used nine — and it costs nothing.
requested participates as a limit, so an explicit --workers 4 is honoured and reported as the binding constraint. Without that, somebody reading workers: 4 concludes the machine is small.
headroom shows how much slack each limit has. Memory has zero — it is binding — and licences have nine. That is the number that answers "if we bought more memory, how far could we go?": up to the next limit, which the headroom column names without any further calculation.
available_cpus() supplies spec["cpus"] in a real flow and the examples pass a constant, so every figure in this chapter is reproducible on any machine. That separation — compute the environment-dependent value at the edge, pass it as data — is what makes the planner testable at all.
9. Predict, Run, Observe
import os
from concurrent.futures import ProcessPoolExecutor
# A
print("A:", hasattr(os, "process_cpu_count"), hasattr(os, "sched_getaffinity"))
# B
print("B:", min(64, 12, 256 // 8))
# C
print("C:", min(64, 0, 32))
# D
try:
ProcessPoolExecutor(max_workers=0)
except ValueError as exc:
print("D:", exc)
# E
print("E:", int(256 // 28.0), int(256 // ((2.1 + 3.4 + 28.0) / 3)))
# F
limits = {"cpu": 64, "licences": 18, "memory": 9, "disk": 16}
n = min(limits.values())
print("F:", n, sorted(k for k, v in limits.items() if v == n))
# G
print("G:", sorted(k for k, v in {"a": 0, "b": 3}.items() if v < 1))
# H
print("H:", {k: v - n for k, v in sorted(limits.items())})The verified output:
A: True False
B: 12
C: 0
D: max_workers must be greater than 0
E: 9 22
F: 9 ['memory']
G: ['a']
H: {'cpu': 55, 'disk': 7, 'licences': 9, 'memory': 0}A — process_cpu_count exists on 3.13 and sched_getaffinity is Linux-only, which is why the fallback chain has three steps.
B — the annotation's form, on a configuration where it is right.
C — and on one where it silently yields zero.
D — the error that eventually results, naming nothing useful.
E — worst-case sizing against mean sizing: nine workers against twenty-two.
F — the minimum, and the limit that produced it.
G — isolating the limits that allow nothing, which is what the guard reports.
H — the headroom, with the binding limit at zero by construction.
10. Exercises
Exercise 1 — four limits. Compute all four for your machine and sort them. Report which is binding.
Exercise 2 — never cores. Say why the largest of the four can never be the answer.
Exercise 3 — two counts. Report whether os.cpu_count and os.process_cpu_count agree on your machine.
Exercise 4 — the slot. Describe what os.cpu_count() returns inside a four-CPU scheduler slot on a sixty-four-core host.
Exercise 5 — the fallback chain. Implement available_cpus with all three steps and say which applies on your platform.
Exercise 6 — or 1. Say what None from os.cpu_count() does inside a min.
Exercise 7 — the silent zero. Produce a configuration where min yields zero and record where the error finally appears.
Exercise 8 — check at the source. Rewrite so each limit is validated where its meaning is known.
Exercise 9 — measure three footprints. Get the peak memory of three tests from your flow and report the ratio.
Exercise 10 — mean versus worst. Compute both worker counts and the oversubscription factor.
Exercise 11 — what oversubscription causes. Trace it to Chapter 27.4's failure and say how many jobs it affects.
Exercise 12 — the drift. Name two changes that would invalidate a six-month-old footprint measurement.
Exercise 13 — three strategies. Choose between worst-case, per-group pools and a memory budget for your flow, and justify it.
Exercise 14 — the budget's catch. Say what a budget scheduler needs that a fixed worker count does not.
Exercise 15 — the margin. Decide your licence margin and say what it protects.
Exercise 16 — four outcomes. For your simulator, determine which of the four it does when a licence is unavailable.
Exercise 17 — the dangerous one. Describe how a silently degraded run passes every check in Chapter 25.6, and name two defences.
Exercise 18 — the proportion. Set a threshold above which licence errors mean a configuration problem rather than contention.
Exercise 19 — do not retry. Say what retrying a systemic licence shortage costs, citing Chapter 26.5.
Exercise 20 — the report line. Add it to your regression log and say what question it answers six months later.
Exercise 21 — requested as a limit. Honour --workers n and report it as the binding constraint. Say what that prevents somebody concluding.
Exercise 22 — headroom. Compute it and say which number answers "what if we bought more memory?".
11. Summary
- Cores are the largest of the four limits and therefore never the answer, yet
os.cpu_count()is the number everybody reaches for. os.cpu_count()reports the machine;os.process_cpu_count()reports the process — and they diverge under containers, affinity masks and scheduler slots, which is where regressions run.- Use a fallback chain:
process_cpu_count, thensched_getaffinityon Linux, thencpu_count() or 1. os.cpu_count()can returnNone, and aNonein aminraises far from the cause.- A zero limit propagates silently.
min(64, 0, 32)is zero, and the failure arrives asmax_workers must be greater than 0from a line that mentions neither licences nor memory. - Validate each limit where its meaning is known, and name it in the error.
- Per-job memory is not a constant — thirteen-fold variation here — and sizing on the mean oversubscribed by 2.4×.
- Oversubscription causes the OOM killer, which causes
BrokenProcessPool, which breaks the pool rather than one job. - Worst-case sizing is the right default; per-group pools and a memory budget are what to do when it costs too much.
- A footprint is a measurement and it drifts — waves roughly double it — so treat recurring
BrokenProcessPoolas the signal to re-measure. - A licence count is a snapshot of a shared resource, unlike cores and memory, so leave a margin for the colleague debugging at 10pm.
- Three of four licence-exhaustion outcomes are honest; the dangerous one is a tool that proceeds without the feature and passes.
- A high proportion of licence errors is a configuration diagnosis, not a retry candidate — retrying reproduces it and spends the budget.
- Report the binding constraint by name: the number says what to type, the name says what to buy.
- Let
requestedparticipate as a limit, so an explicit worker count is honoured and reported as the reason. - Publish headroom, which answers "how far could we go if we fixed this one?" without further calculation.
- The worker count is a result computed at startup, not a setting in a file.
That completes Module 27. A regression now runs its already-isolated jobs concurrently, against a worker count derived from the resource that actually binds, with a baseline to compare the results against, a handler that tells a failed job from a broken pool, and a report that explains the number it chose. Module 28 turns from running simulations to reading what the implementation tools say about the design.
Where this fits
Part of the Python curriculum.