Python Part II · Module 34 · From Script to Maintainable Tool
Changing a Tool Other People Depend On
The tool is now portable, documented and tested, which means people use it — and the moment somebody writes a wrapper around it, its flags, its output format and its exit codes stop being implementation and become an interface that was never designed. This chapter is about changing one of those anyway. The standard advice is to emit a deprecation warning, and the measured behaviour of that warning is narrower than the advice implies: written the obvious way, nobody will ever see it.
Advanced24 min readPythonPart IIAutomationInterfacesDeprecation
Part II · Module 34 · Chapter 34.7 · Changing a Tool Other People Depend On
1. What Became an Interface
SURFACE = [
("the flag names", "run_regress.py --tests smoke", "yes", "loud"),
("the output file names", "summary.csv", "yes", "loud"),
("the CSV column names", "status", "yes", "loud or silent — §6"),
("the CSV column ORDER", "test,seed,status", "yes", "silent"),
("the exit codes", "0 / 1 / 2 / 3 / 4", "yes", "silent"),
("the env var names", "FLOW_RUN_ROOT", "yes", "silent"),
("the printed wording", "3/4 passed", "no — 34.6 said so", "n/a"),
("the internal function names", "parse_sim_log", "no", "n/a"),
]
print(f"{'surface':<26} {'example':<30} {'interface':<22} breaks")
for what, example, iface, how in SURFACE:
print(f"{what:<26} {example:<30} {iface:<22} {how}")
print()
iface = sum(1 for _, _, i, _ in SURFACE if i.startswith("yes"))
silent = sum(1 for _, _, i, h in SURFACE if i.startswith("yes")
and "silent" in h)
print("surfaces that are an interface:", iface)
print("of those, breaking SILENTLY :", silent)
print()
print("the ones you can change freely:", 2)
print("and 34.6 is why :", "the docs said the wording is not promised")surface example interface breaks
the flag names run_regress.py --tests smoke yes loud
the output file names summary.csv yes loud
the CSV column names status yes loud or silent — §6
the CSV column ORDER test,seed,status yes silent
the exit codes 0 / 1 / 2 / 3 / 4 yes silent
the env var names FLOW_RUN_ROOT yes silent
the printed wording 3/4 passed no — 34.6 said so n/a
the internal function names parse_sim_log no n/a
surfaces that are an interface: 6
of those, breaking SILENTLY : 4
the ones you can change freely: 2
and 34.6 is why : the docs said the wording is not promisedSix surfaces are an interface and four of them can break silently. The two
reliably loud ones are the easy cases: a renamed flag produces unrecognized arguments, a renamed output file produces a missing-file error, and somebody
tells you within the hour.
The others are the subject of Sections 6 and 7. A reordered CSV column, a reused exit code and a renamed environment variable all let the wrapper keep running and start being wrong, which is Chapter 30.7's confidently-wrong value arriving through an interface change. The column-name row is marked both ways because it is genuinely both: Section 6 measures a rename breaking one kind of consumer loudly and the other kind silently, in the same change.
Note the last two rows. Chapter 34.6's documentation explicitly declined to promise the printed wording, which is what makes it changeable — a promise you never made is the only kind you can withdraw.
2. The Annotation
# --tests became --suite: keep --tests working, warn for one releaseThree decisions in one line, and all three are load-bearing. The new name exists, so new callers get the better flag. The old name keeps working, so no wrapper breaks today. And the overlap has an end — one release — so the shim does not become permanent.
PLANS = [
("rename it and tell people", "every wrapper breaks at once", 0),
("rename it and keep the old one forever", "nothing breaks, ever",
"unbounded"),
("keep the old one, warn, remove after one release", "nothing breaks now",
1),
]
print(f"{'plan':<48} {'immediate effect':<30} releases of overlap")
for plan, effect, overlap in PLANS:
print(f"{plan:<48} {effect:<30} {overlap}")
print()
print("the first plan's problem :", "a flag day across every team")
print("the second plan's problem:", "two names forever, both documented")
print("the annotation's plan :", "a deprecation with a deadline")
print()
print("what makes the third work:", "the warning is actually SEEN — §5")plan immediate effect releases of overlap
rename it and tell people every wrapper breaks at once 0
rename it and keep the old one forever nothing breaks, ever unbounded
keep the old one, warn, remove after one release nothing breaks now 1
the first plan's problem : a flag day across every team
the second plan's problem: two names forever, both documented
the annotation's plan : a deprecation with a deadline
what makes the third work: the warning is actually SEEN — §5The second plan is the one teams drift into, and its cost is not maintenance — it is that Chapter 34.6's documentation now has to describe two flags that do the same thing, and a new user has to choose. The deadline is what the annotation adds over "be backwards compatible".
3. The Shim
import argparse
import contextlib
import io
def build_parser():
p = argparse.ArgumentParser(prog="run_regress.py")
p.add_argument("--suite", dest="suite", default="smoke",
help="a suite name from flow.toml (default: %(default)s)")
# the shim: the OLD flag, same dest, hidden from --help
p.add_argument("--tests", dest="suite", help=argparse.SUPPRESS)
return p
parser = build_parser()
print(f"{'invocation':<30} {'parsed suite':<14} note")
for argv in ([], ["--suite", "nightly"], ["--tests", "nightly"],
["--tests", "a", "--suite", "b"], ["--suite", "b", "--tests", "a"]):
ns = parser.parse_args(argv)
note = ("default" if not argv else
"last one wins" if len(argv) == 4 else "")
print(f"{str(argv):<30} {ns.suite!r:<14} {note}")
buf = io.StringIO()
with contextlib.redirect_stdout(buf):
parser.print_help()
help_text = buf.getvalue()
print()
print("--tests appears in --help:", "--tests" in help_text)
print("--suite appears in --help:", "--suite" in help_text)
print("because the old flag uses:", "help=argparse.SUPPRESS")
print()
print("both flags write to :", "the same dest")
print("so the body of main sees :", "one name only")invocation parsed suite note
[] 'smoke' default
['--suite', 'nightly'] 'nightly'
['--tests', 'nightly'] 'nightly'
['--tests', 'a', '--suite', 'b'] 'b' last one wins
['--suite', 'b', '--tests', 'a'] 'a' last one wins
--tests appears in --help: False
--suite appears in --help: True
because the old flag uses: help=argparse.SUPPRESS
both flags write to : the same dest
so the body of main sees : one name onlyTwo flags, one dest, and the body of main never learns there was a
rename. That is what makes the shim cheap: args.suite is the only name in
the code, so the deprecation lives entirely in the parser and deleting it later
touches one line.
help=argparse.SUPPRESS hides the old flag from --help, which is the right
default: the shim exists for wrappers that already use it, not to be discovered
by new users. Chapter 34.6's documentation should mention it once, in a
deprecations line, with the removal release.
The last two rows are worth noticing because they are a real hazard. When both
flags are given, the last one wins — argparse simply assigns to dest in
order — so a wrapper that passes --tests a while a site default adds
--suite b gets b, silently. If that matters, the shim needs to detect the
conflict explicitly rather than rely on ordering.
4. argparse Can Do the Warning Itself
import argparse
import contextlib
import io
import sys
print("python", ".".join(str(n) for n in sys.version_info[:2]))
print()
parser = argparse.ArgumentParser(prog="run_regress.py")
parser.add_argument("--suite", dest="suite", default="smoke")
parser.add_argument("--tests", dest="suite", deprecated=True,
help=argparse.SUPPRESS)
for argv in (["--suite", "nightly"], ["--tests", "nightly"]):
err = io.StringIO()
with contextlib.redirect_stderr(err):
ns = parser.parse_args(argv)
lines = err.getvalue().strip().splitlines()
print(f" {str(argv):<24} suite={ns.suite!r:<12} "
f"stderr lines={len(lines)}")
for line in lines:
print(f" {line}")
print()
print("it uses the warnings module:", False)
print("it writes straight to :", "stderr")
print("so it is always visible :", True)
print("and it does not affect :", "the exit code or stdout")python 3.13
['--suite', 'nightly'] suite='nightly' stderr lines=0
['--tests', 'nightly'] suite='nightly' stderr lines=1
run_regress.py: warning: option '--tests' is deprecated
it uses the warnings module: False
it writes straight to : stderr
so it is always visible : True
and it does not affect : the exit code or stdoutOne line on stderr, and only when the old flag is used. Python 3.13's
add_argument(..., deprecated=True) is the whole deprecation mechanism for a
flag, and it deliberately bypasses the warnings module — which, as Section 5
measures, is the right call.
Three properties make it the correct tool here. It goes to stderr, so
Chapter 33.8 Section 6's clean-stdout rule holds and a wrapper doing
run_regress.py > summary.csv is unaffected. It does not change the exit
code. And it is unconditional — no interpreter flag, no filter
configuration, no stacklevel subtlety.
For a deprecated function in flow.parse there is no argparse to help, and
that is where Section 5 applies.
5. The Warning Nobody Sees
import subprocess
import sys
from pathlib import Path
Path("flow").mkdir(exist_ok=True)
(Path("flow") / "__init__.py").write_text("", encoding="utf-8")
(Path("flow") / "api.py").write_text(
"import warnings\n"
"\n"
"\n"
"def no_stacklevel(suite):\n"
' warnings.warn("no_stacklevel() is deprecated; use run_suite()",\n'
" DeprecationWarning)\n"
" return suite\n"
"\n"
"\n"
"def with_stacklevel(suite):\n"
' warnings.warn("with_stacklevel() is deprecated; use run_suite()",\n'
" DeprecationWarning, stacklevel=2)\n"
" return suite\n", encoding="utf-8")
Path("user_script.py").write_text(
"from flow.api import no_stacklevel, with_stacklevel\n"
"\n"
'no_stacklevel("smoke")\n'
'with_stacklevel("smoke")\n'
'print("done")\n', encoding="utf-8")
r = subprocess.run([sys.executable, "user_script.py"],
stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
print("a user runs their own script, which calls two deprecated functions")
print()
print(" no_stacklevel warning shown:", "no_stacklevel() is deprecated"
in r.stderr)
print(" with_stacklevel warning shown:", "with_stacklevel() is deprecated"
in r.stderr)
print(" stderr lines :", len(r.stderr.strip().splitlines()))
print()
for line in r.stderr.splitlines():
if "DeprecationWarning" in line:
print(" attributed to:", line.split("/")[-1].strip()[:60])
print()
print("the rule: a DeprecationWarning is shown only when attributed to:",
"__main__")
print("and stacklevel=2 attributes it to :", "the CALLER")a user runs their own script, which calls two deprecated functions
no_stacklevel warning shown: False
with_stacklevel warning shown: True
stderr lines : 2
attributed to: user_script.py:4: DeprecationWarning: with_stacklevel() is d
the rule: a DeprecationWarning is shown only when attributed to: __main__
and stacklevel=2 attributes it to : the CALLEROne of the two warnings was shown. The code is identical apart from
stacklevel=2, and that one argument is the difference between a deprecation
every user sees and one nobody ever will.
The mechanism is worth stating precisely, because the usual summary —
"DeprecationWarning is hidden by default" — is wrong in a way that leads to
the wrong fix. The default warning filters do show DeprecationWarning,
but only when it is attributed to __main__. A warnings.warn call with no
stacklevel is attributed to the file containing the call — flow/api.py —
which is not __main__, so it is filtered out. stacklevel=2 says blame my
caller, and the caller here is the user's own script, which is __main__.
import subprocess
import sys
from pathlib import Path
Path("flow").mkdir(exist_ok=True)
(Path("flow") / "__init__.py").write_text("", encoding="utf-8")
(Path("flow") / "api.py").write_text(
"import warnings\n"
"\n"
"\n"
"def deprecated_dep(suite):\n"
' warnings.warn("use run_suite()", DeprecationWarning)\n'
" return suite\n"
"\n"
"\n"
"def deprecated_future(suite):\n"
' warnings.warn("use run_suite()", FutureWarning)\n'
" return suite\n", encoding="utf-8")
Path("u.py").write_text(
"from flow.api import deprecated_dep, deprecated_future\n"
'deprecated_dep("a")\n'
'deprecated_future("a")\n', encoding="utf-8")
r = subprocess.run([sys.executable, "u.py"], stdout=subprocess.PIPE,
stderr=subprocess.PIPE, text=True)
print("both raised from the library, NEITHER with stacklevel:")
print(" DeprecationWarning shown:", "DeprecationWarning" in r.stderr)
print(" FutureWarning shown :", "FutureWarning" in r.stderr)
print()
CATEGORIES = [
("DeprecationWarning", "for other developers", "filtered unless __main__"),
("FutureWarning", "for END USERS", "always shown"),
("UserWarning", "the default category", "always shown"),
]
print(f" {'category':<22} {'intended audience':<22} default visibility")
for cat, who, vis in CATEGORIES:
print(f" {cat:<22} {who:<22} {vis}")
print()
print("so for a tool whose users are not its developers:", "FutureWarning")
print("and for a library API used by other code :",
"DeprecationWarning + stacklevel=2")both raised from the library, NEITHER with stacklevel:
DeprecationWarning shown: False
FutureWarning shown : True
category intended audience default visibility
DeprecationWarning for other developers filtered unless __main__
FutureWarning for END USERS always shown
UserWarning the default category always shown
so for a tool whose users are not its developers: FutureWarning
and for a library API used by other code : DeprecationWarning + stacklevel=2FutureWarning is shown from anywhere; DeprecationWarning is not. That is
the category distinction doing exactly what it was designed for, and it answers
the question the previous measurement raises. DeprecationWarning means this
is addressed to whoever maintains the calling code, so Python filters it out of
ordinary runs; FutureWarning means this is addressed to the person running
the program.
A flow tool's callers are usually verification engineers running wrappers, not
library maintainers — so FutureWarning is the honest category for a changed
flag or output format, and DeprecationWarning with stacklevel=2 is right
for flow.parse's public functions.
6. Changing the Output Format
import csv
import io
VARIANTS = {
"unchanged": "test,seed,status\naxi_burst,4121,FAIL\n",
"a column added at the END": "test,seed,status,errors\n"
"axi_burst,4121,FAIL,2\n",
"a column added in the MIDDLE": "test,seed,errors,status\n"
"axi_burst,4121,2,FAIL\n",
"columns reordered": "status,test,seed\nFAIL,axi_burst,4121\n",
"the column RENAMED": "test,seed,verdict\naxi_burst,4121,FAIL\n",
}
def positional_reader(text):
"""A wrapper written in a hurry: awk-style, by column index."""
return text.splitlines()[1].split(",")[2]
def named_reader(text):
"""A wrapper written carefully: by column name."""
try:
return next(csv.DictReader(io.StringIO(text)))["status"]
except KeyError:
return "KeyError"
print(f"{'change':<30} {'row[2]':<10} {'by name':<10} who breaks")
for label, text in VARIANTS.items():
a, b = positional_reader(text), named_reader(text)
if a == "FAIL" and b == "FAIL":
who = "nobody"
elif a != "FAIL" and b == "FAIL":
who = "the careless one"
elif a == "FAIL" and b != "FAIL":
who = "the CAREFUL one"
else:
who = "both"
print(f"{label:<30} {a:<10} {b:<10} {who}")
print()
print("the only change that breaks nobody:", "adding a column at the end")
print("the change that breaks only the ")
print(" careful consumer :", "renaming a column")change row[2] by name who breaks
unchanged FAIL FAIL nobody
a column added at the END FAIL FAIL nobody
a column added in the MIDDLE 2 FAIL the careless one
columns reordered 4121 FAIL the careless one
the column RENAMED FAIL KeyError the CAREFUL one
the only change that breaks nobody: adding a column at the end
the change that breaks only the
careful consumer : renaming a columnAdding a column at the end is the only change that breaks nobody, and the last row is the finding worth carrying: renaming a column breaks the consumer who did it right and spares the one who indexed by position.
That inverts the usual intuition and it has a practical consequence. A rename
produces a KeyError in every carefully written wrapper and silence in every
hasty one, so the hasty wrappers keep running and keep reading a column whose
name no longer describes it. The careless consumers are the ones you will not
hear from, and they are the ones now reporting the wrong field.
The two middle rows are the mirror image and are the reason column order is an
interface at all: a positional reader gets 2 where it wanted FAIL, which is
a string that is not a status and may or may not raise downstream.
RULES = [
("add a column", "at the end only", "nobody breaks"),
("remove a column", "never without a release of deprecation",
"both break"),
("rename a column", "emit both for one release", "careful ones break"),
("reorder columns", "never — there is no benefit", "careless ones break"),
("change a value's meaning", "never — rename the column instead",
"everybody is silently wrong"),
]
print(f"{'change':<26} {'the rule':<42} if you ignore it")
for change, rule, cost in RULES:
print(f"{change:<26} {rule:<42} {cost}")
print()
print("changes that are always safe:", 1)
print("changes that are never safe :", 2)
print("the worst one :", "changing what a value MEANS")
print("because :", "no reader can detect it")change the rule if you ignore it
add a column at the end only nobody breaks
remove a column never without a release of deprecation both break
rename a column emit both for one release careful ones break
reorder columns never — there is no benefit careless ones break
change a value's meaning never — rename the column instead everybody is silently wrong
changes that are always safe: 1
changes that are never safe : 2
the worst one : changing what a value MEANS
because : no reader can detect itEmitting both columns for one release is the CSV equivalent of the flag
shim: write status and verdict with the same value, document the removal
release, then drop status. It costs one column and it turns a KeyError in
somebody else's code into a warning in yours.
7. The Exit Code Is the Worst Thing to Change
OLD = {0: "every job passed", 1: "at least one job failed",
2: "the command line was wrong"}
NEW = {0: "every job passed", 1: "the tool itself crashed",
2: "the command line was wrong", 3: "at least one job failed"}
WRAPPER = """if run_regress.py --suite smoke; then
publish_results
else
open_bug_report
fi"""
print("the wrapper, unchanged:")
for line in WRAPPER.splitlines():
print(" ", line)
print()
print(f"{'exit':>5} {'old meaning':<26} {'new meaning':<26} wrapper does")
for code in sorted(set(OLD) | set(NEW)):
old = OLD.get(code, "—")
new = NEW.get(code, "—")
action = "publish" if code == 0 else "open a bug"
print(f"{code:>5} {old:<26} {new:<26} {action}")
print()
print("codes whose meaning changed:", sum(1 for c in set(OLD) & set(NEW)
if OLD[c] != NEW[c]))
print("the wrapper still runs :", True)
print("it still branches correctly :", False)
print()
print("a failing suite used to exit:", 1, "->", "open a bug (right)")
print("a failing suite now exits :", 3, "->", "open a bug (right)")
print("a CRASH now exits :", 1, "->", "open a bug (wrong kind)")
print()
print("so the breakage is :", "a misclassified bug report")
print("and nothing anywhere :", "errors or warns")the wrapper, unchanged:
if run_regress.py --suite smoke; then
publish_results
else
open_bug_report
fi
exit old meaning new meaning wrapper does
0 every job passed every job passed publish
1 at least one job failed the tool itself crashed open a bug
2 the command line was wrong the command line was wrong open a bug
3 — at least one job failed open a bug
codes whose meaning changed: 1
the wrapper still runs : True
it still branches correctly : False
a failing suite used to exit: 1 -> open a bug (right)
a failing suite now exits : 3 -> open a bug (right)
a CRASH now exits : 1 -> open a bug (wrong kind)
so the breakage is : a misclassified bug report
and nothing anywhere : errors or warnsOne code's meaning changed and the wrapper kept working. A pass/fail wrapper that only distinguishes zero from non-zero is unaffected here, which is why this change looks safe — and a wrapper that distinguishes kinds of failure now files design bugs for tool crashes.
That is the reason exit codes are the one surface to treat as permanently
frozen. There is no shim: a process has one integer to say everything with, you
cannot emit both the old and the new value, and there is nowhere to put a
warning that a wrapper reading $? will see.
STRATEGIES = [
("add a new code for a new condition", "yes",
"old codes keep their meaning"),
("reuse a retired code for something else", "no",
"silent misclassification"),
("renumber for tidiness", "no", "there is no benefit at all"),
("widen 'failed' to include crashes", "no",
"destroys 30.1's design/infra split"),
("narrow 'failed' and add a crash code", "yes",
"old wrappers over-report, never under-report"),
]
print(f"{'strategy':<42} {'safe':<6} why")
for s, safe, why in STRATEGIES:
print(f"{s:<42} {safe:<6} {why}")
print()
print("safe strategies:", 2)
print("the shared property of both:", "an old code never changes meaning")
print()
print("and the asymmetry that makes the last one safe:")
print(" an old wrapper seeing a NEW code :", "falls into its else branch")
print(" an old wrapper seeing a REUSED one:", "takes the wrong branch")strategy safe why
add a new code for a new condition yes old codes keep their meaning
reuse a retired code for something else no silent misclassification
renumber for tidiness no there is no benefit at all
widen 'failed' to include crashes no destroys 30.1's design/infra split
narrow 'failed' and add a crash code yes old wrappers over-report, never under-report
safe strategies: 2
the shared property of both: an old code never changes meaning
and the asymmetry that makes the last one safe:
an old wrapper seeing a NEW code : falls into its else branch
an old wrapper seeing a REUSED one: takes the wrong branchOnly two strategies are safe and both preserve every old code's meaning.
The asymmetry in the last three lines is the whole rule: an unknown code lands
in a wrapper's else branch, which is conservative — it treats something new
as a failure. A reused code lands in a branch that was written for a
different meaning, which is not conservative at all.
8. A Deprecation With an End
import argparse
import contextlib
import io
RELEASES = [
("1.4", "--tests only", "none", "the original"),
("1.5", "--suite added, --tests deprecated", "on --tests",
"the annotation's release"),
("1.6", "--suite only", "n/a", "--tests removed"),
]
print(f"{'release':<9} {'flags accepted':<36} {'warning':<12} note")
for rel, flags, warn, note in RELEASES:
print(f"{rel:<9} {flags:<36} {warn:<12} {note}")
print()
print("releases of overlap :", 1)
print("what must be written down :", "the removal release, in --help and docs")
print()
DEPRECATIONS = [
("--tests", "--suite", "1.5", "1.6", "argparse deprecated=True"),
("status column", "verdict column", "1.5", "1.6", "emit both"),
("flow.api.old_entry", "flow.api.run_suite", "1.5", "1.7",
"FutureWarning, stacklevel=2"),
]
print("the deprecations table, which lives in the repository:")
print(f" {'old':<22} {'new':<22} {'since':<7} {'removed':<9} mechanism")
for old, new, since, removed, how in DEPRECATIONS:
print(f" {old:<22} {new:<22} {since:<7} {removed:<9} {how}")
print()
print("entries with a removal release:", len(DEPRECATIONS))
print("entries without one :", 0)
print("why that matters :",
"a deprecation with no deadline is a second interface")release flags accepted warning note
1.4 --tests only none the original
1.5 --suite added, --tests deprecated on --tests the annotation's release
1.6 --suite only n/a --tests removed
releases of overlap : 1
what must be written down : the removal release, in --help and docs
the deprecations table, which lives in the repository:
old new since removed mechanism
--tests --suite 1.5 1.6 argparse deprecated=True
status column verdict column 1.5 1.6 emit both
flow.api.old_entry flow.api.run_suite 1.5 1.7 FutureWarning, stacklevel=2
entries with a removal release: 3
entries without one : 0
why that matters : a deprecation with no deadline is a second interfaceEvery entry has a removal release. That column is what separates the
annotation's plan from "be backwards compatible", and the table belongs in the
repository next to Chapter 34.5's ALLOWED table and Chapter 34.6's docstring —
three small files that are executable or checkable and therefore cannot drift.
Note the three different mechanisms for three kinds of surface, each chosen
from the measurements above: deprecated=True for a flag because it always
shows; emitting both columns because there is no warning channel in a CSV; and
FutureWarning with stacklevel=2 for a library function because a plain
DeprecationWarning would be invisible.
9. What the Specimen Became
STAGES = [
("34.1", "measured it", 166, 3, 0, 1, False),
("34.2", "split into modules", 166, 9, 5, 1, True),
("34.3", "three layers, one direction", 170, 11, 7, 1, True),
("34.4", "no class needed; a frozen Result", 174, 12, 8, 1, True),
("34.5", "config, 7 facts removed", 196, 15, 10, 6, True),
("34.6", "documented, 5 sections", 219, 15, 10, 6, True),
("34.7", "a deprecations table", 224, 16, 11, 6, True),
]
print(f"{'ch':<6} {'what changed':<34} {'lines':>6} {'funcs':>6} "
f"{'testable':>9} {'audiences':>10} importable")
for ch, what, lines, funcs, testable, audiences, imp in STAGES:
print(f"{ch:<6} {what:<34} {lines:>6} {funcs:>6} {testable:>9} "
f"{audiences:>10} {imp}")
first, last = STAGES[0], STAGES[-1]
print()
print("lines :", first[2], "->", last[2],
f"(+{round(100 * (last[2] - first[2]) / first[2])}%)")
print("testable functions :", first[4], "->", last[4])
print("audiences who can run:", first[5], "->", last[5])
print()
print("the parsing logic rewritten:", "never")
print("the one defect found :", "34.1 §6's empty report")
print("and it was found by :", "measuring, not by reading")ch what changed lines funcs testable audiences importable
34.1 measured it 166 3 0 1 False
34.2 split into modules 166 9 5 1 True
34.3 three layers, one direction 170 11 7 1 True
34.4 no class needed; a frozen Result 174 12 8 1 True
34.5 config, 7 facts removed 196 15 10 6 True
34.6 documented, 5 sections 219 15 10 6 True
34.7 a deprecations table 224 16 11 6 True
lines : 166 -> 224 (+35%)
testable functions : 0 -> 11
audiences who can run: 1 -> 6
the parsing logic rewritten: never
the one defect found : 34.1 §6's empty report
and it was found by : measuring, not by readingThirty-five per cent more lines, zero to eleven testable functions, one audience to six. That is the honest accounting for the module: the tool got bigger, and what grew was configuration handling, documentation and an entry point — not the logic, which was never rewritten.
The last two lines are the module's thesis. One real defect was found in 166
lines of working code, and it was found by ast counting references to a
global, not by anybody reading carefully. Chapter 34.1's RESULTS had been
read many times.
10. Predict, Run, Observe
import argparse
import contextlib
import csv
import io
p = argparse.ArgumentParser(prog="t")
p.add_argument("--suite", dest="suite", default="smoke")
p.add_argument("--tests", dest="suite", deprecated=True,
help=argparse.SUPPRESS)
# A
err = io.StringIO()
with contextlib.redirect_stderr(err):
ns = p.parse_args(["--tests", "nightly"])
print("A:", ns.suite, len(err.getvalue().strip().splitlines()))
# B
err = io.StringIO()
with contextlib.redirect_stderr(err):
ns = p.parse_args(["--tests", "a", "--suite", "b"])
print("B:", ns.suite)
# C
RENAMED = "test,seed,verdict\naxi_burst,4121,FAIL\n"
print("C:", RENAMED.splitlines()[1].split(",")[2],
"status" in (csv.DictReader(io.StringIO(RENAMED)).fieldnames or []))
# D
buf = io.StringIO()
with contextlib.redirect_stdout(buf):
p.print_help()
print("D:", "--tests" in buf.getvalue())The verified output:
A: nightly 1
B: b
C: FAIL False
D: FalseA — the deprecated flag still works and produces exactly one line on stderr.
B — both flags given, and --suite wins because it came last. Section 3's
hazard: the shim does not detect a conflict, it resolves it by position.
C — after a rename, the positional reader still returns FAIL while the
field name status is gone. The careless consumer is fine and the careful one
is about to raise KeyError — Section 6's inversion in two values.
D — and the deprecated flag is absent from --help, so a new user cannot
discover it. That is argparse.SUPPRESS doing its job, and it is why the
deprecations table in Section 8 has to exist somewhere a reader can find.
11. Exercises
Exercise 1 — list your surfaces. Write down every flag, file name, column, exit code and env var your tool exposes.
Exercise 2 — loud or silent. Mark which of them would break loudly.
Exercise 3 — what you never promised. Find something you can change freely because the docs declined to promise it.
Exercise 4 — three plans. For a rename you want, cost out all three.
Exercise 5 — the deadline. Explain what the annotation adds over "be backwards compatible".
Exercise 6 — build the shim. Add a second flag writing the same dest.
Exercise 7 — one name in the body. Confirm main never learns about the
old flag.
Exercise 8 — SUPPRESS it. Hide the old flag from --help and say where it
is documented instead.
Exercise 9 — both flags given. Check which one wins, and decide whether to detect the conflict.
Exercise 10 — deprecated=True. Use it and record the exact stderr line.
Exercise 11 — three properties. Say why stderr, no exit-code change, and unconditional visibility all matter.
Exercise 12 — no stacklevel. Raise a DeprecationWarning from a module and
run a script that calls it.
Exercise 13 — add stacklevel. Add stacklevel=2 and compare.
Exercise 14 — the real rule. State precisely when a DeprecationWarning is
shown.
Exercise 15 — why the usual summary misleads. Explain what fix "hidden by default" would lead you to.
Exercise 16 — the category. Decide between FutureWarning and
DeprecationWarning for your own users, and justify it.
Exercise 17 — five variants. Run the output-format table on your own CSV and both reader styles.
Exercise 18 — the only safe change. Add a column at the end and confirm nobody breaks.
Exercise 19 — the inversion. Explain why renaming a column breaks the careful consumer.
Exercise 20 — who you will not hear from. Say which consumers keep running and are now wrong.
Exercise 21 — emit both. Implement a one-release column rename.
Exercise 22 — the worst change. Explain why changing a value's meaning is undetectable by any reader.
Exercise 23 — the pass/fail wrapper. Show an exit-code change that such a wrapper survives and a kind-aware one does not.
Exercise 24 — no shim exists. Explain why an exit code cannot be deprecated the way a flag can.
Exercise 25 — two safe strategies. Name them and the property they share.
Exercise 26 — the asymmetry. Explain why an unknown code is safer than a reused one.
Exercise 27 — the table. Write your deprecations table with a removal release for every row.
Exercise 28 — three mechanisms. Justify a different mechanism for a flag, a column and a function.
Exercise 29 — account for the growth. Measure your tool before and after and say what grew.
Exercise 30 — what was never rewritten. Confirm your core logic is unchanged, and say what that proves.
12. Summary
- Six surfaces are an interface and four can break silently — column order, exit codes, env var names, and a column rename for half its consumers — all let a wrapper keep running and start being wrong.
- A promise you never made is the only one you can withdraw, which is why Chapter 34.6 explicitly declined to promise the printed wording.
- The annotation makes three decisions: the new name exists, the old one keeps working, and the overlap has an end.
- Two flags, one
dest, so the body ofmainnever learns about the rename and removal later touches one line. Hide the old one withargparse.SUPPRESS. - When both flags are given the last one wins — the shim resolves a conflict by position rather than detecting it.
- Python 3.13's
add_argument(..., deprecated=True)prints one line to stderr, bypasses thewarningsmodule, leaves the exit code alone, and is therefore always visible. - A
DeprecationWarningraised inside a library with nostacklevelis never shown. The default filters do show it — but only when it is attributed to__main__, and a barewarnings.warnis attributed to the library's own file. stacklevel=2decides whether the warning exists as far as the user is concerned, which is far more than the line-number improvement it is usually introduced as.FutureWarningis shown from anywhere, so it is the honest category when your users run wrappers rather than maintain libraries.- Adding a column at the end is the only output change that breaks nobody.
- Renaming a column breaks the careful consumer and spares the careless
one — a
KeyErrorin every wrapper that read by name, and silence in every wrapper that read by index and is now reporting the wrong field. - Changing what a value means is the worst change of all, because no reader can detect it.
- An exit code cannot be deprecated. One integer, no second channel, and
nowhere to put a warning a
$?reader would see. - Only two exit-code strategies are safe, and both preserve every old
code's meaning: an unknown code lands in a wrapper's
elsebranch, a reused one lands in the wrong branch. - Every deprecation needs a removal release, written in a table in the repository — one with no deadline is simply a second interface.
- Thirty-five per cent more lines, zero to eleven testable functions, one audience to six — and the parsing logic was never rewritten. The one real defect in the original was found by counting references to a global, not by reading.
That closes Module 34. The script that nobody wanted to touch is a tool with a contract, a test suite, a configuration, a handover document and a plan for changing it — and the next module puts all of it to work on four complete case studies.
Where this fits
Part of the Python curriculum.