Python Part II · Module 34 · From Script to Maintainable Tool

Removing Your Own Project's Assumptions

Chapter 34.1 measured seven facts about one machine baked into the specimen — a username, a project, a block, a scratch root, a tool vendor, a tool version and a licence server — and none of the three refactorings since has removed any of them. This chapter does, and the interesting part is not the mechanics of reading an environment variable but deciding which of the four possible homes each fact belongs in. The canonical idiom for the first one has a hole in it that a wrapper script will find.

Advanced23 min readPythonPart IIAutomationConfigurationPortability

Part II · Module 34 · Chapter 34.5 · Removing Your Own Project's Assumptions

1. Seven Facts, Four Homes

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

FACTS = [
    ("the block name", "axi_fifo", "per project"),
    ("the project", "chip_a", "per project"),
    ("the test list", "4 tests", "per project"),
    ("the scratch root", "/proj/.../users/azhar", "per user"),
    ("the licence server", "5280@lic-server-3", "per site"),
    ("the simulator", "xrun 19.09", "per site"),
    ("the seeds for this run", "4121, 7734, 9001", "per run"),
]

HOMES = {"per run": "a command-line flag",
         "per user": "an environment variable",
         "per site": "an environment variable",
         "per project": "a file in the repository"}

print(f"{'fact':<24} {'current value':<24} {'varies':<12} belongs in")
for fact, value, varies in FACTS:
    print(f"{fact:<24} {value:<24} {varies:<12} {HOMES[varies]}")

print()
print("distinct homes needed:", len(set(HOMES.values())))
print("the question for each fact:", "who changes it, and how often?")
print("NOT:", "is it a path?")

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

fact                     current value            varies       belongs in
the block name           axi_fifo                 per project  a file in the repository
the project              chip_a                   per project  a file in the repository
the test list            4 tests                  per project  a file in the repository
the scratch root         /proj/.../users/azhar    per user     an environment variable
the licence server       5280@lic-server-3        per site     an environment variable
the simulator            xrun 19.09               per site     an environment variable
the seeds for this run   4121, 7734, 9001         per run      a command-line flag

The question is who varies a fact, not what type it is. Four of these seven are paths and they do not all belong in the same place: the scratch root changes per user, the simulator per site, and the filelist path is derivable from the project root and belongs nowhere — Section 6 is about that fourth home.

2. The Annotation

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

RUN_ROOT = Path(os.environ.get("FLOW_RUN_ROOT", ROOT / "runs"))

That line does four things right. It names the variable with a prefix, so FLOW_RUN_ROOT cannot collide with another tool's RUN_ROOT. It has a default, so the tool works with nothing set. The default is derived from ROOT rather than written out, which is the composition Chapter 34.1 Section 5 found missing. And it wraps the result in Path, so the rest of the code gets one type whichever branch was taken.

It is worth checking the type question first, because the line looks like it has a bug and does not:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

import os
from pathlib import Path

ROOT = Path("/opt/flow")

print("with FLOW_RUN_ROOT unset:")
value = os.environ.get("FLOW_RUN_ROOT", ROOT / "runs")
print("  os.environ.get returned type:", type(value).__name__)
print("  wrapping it in Path() works :", Path(value) == ROOT / "runs")

os.environ["FLOW_RUN_ROOT"] = "/scratch/runs"
print()
print("with FLOW_RUN_ROOT set:")
value = os.environ.get("FLOW_RUN_ROOT", ROOT / "runs")
print("  os.environ.get returned type:", type(value).__name__)
print("  wrapping it in Path() works :", Path(value) == Path("/scratch/runs"))
del os.environ["FLOW_RUN_ROOT"]

print()
print("so the mixed str/Path default is:", "fine — Path() accepts both")
print("the hole is elsewhere           :", "§3")

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

with FLOW_RUN_ROOT unset:
  os.environ.get returned type: PosixPath
  wrapping it in Path() works : True

with FLOW_RUN_ROOT set:
  os.environ.get returned type: str
  wrapping it in Path() works : True

so the mixed str/Path default is: fine — Path() accepts both
the hole is elsewhere           : §3

The two branches return different types and it does not matter, because Path accepts a str or another Path. That is worth confirming rather than assuming, since a default of a different type from the value is usually a defect — here the Path(...) wrapper is exactly what makes it safe.

3. Set But Empty

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

import os
from pathlib import Path

ROOT = Path("/opt/flow")


def run_root():
    """The annotation, verbatim."""
    return Path(os.environ.get("FLOW_RUN_ROOT", ROOT / "runs"))


CASES = [
    ("unset", None),
    ("an absolute path", "/scratch/azhar/runs"),
    ("set but EMPTY", ""),
    ("a relative path", "runs"),
    ("a tilde path", "~/runs"),
    ("three spaces", "   "),
]

print(f"{'FLOW_RUN_ROOT':<18} {'resolves to':<24} {'absolute':<10} default used")
for label, val in CASES:
    os.environ.pop("FLOW_RUN_ROOT", None)
    if val is not None:
        os.environ["FLOW_RUN_ROOT"] = val
    p = run_root()
    print(f"{label:<18} {str(p)!r:<24} {str(p.is_absolute()):<10} "
          f"{p == ROOT / 'runs'}")

os.environ.pop("FLOW_RUN_ROOT", None)
print()
print("environments tried          :", len(CASES))
print("that give an absolute path  :", 2)
print("that give a RELATIVE path   :", 4)
print()
print("os.environ.get treats set-but-empty as:", "SET")
print("so the default is not used            :", True)
print("and Path('') is                       :", repr(str(Path(""))))

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

FLOW_RUN_ROOT      resolves to              absolute   default used
unset              '/opt/flow/runs'         True       True
an absolute path   '/scratch/azhar/runs'    True       False
set but EMPTY      '.'                      False      False
a relative path    'runs'                   False      False
a tilde path       '~/runs'                 False      False
three spaces       '   '                    False      False

environments tried          : 6
that give an absolute path  : 2
that give a RELATIVE path   : 4

os.environ.get treats set-but-empty as: SET
so the default is not used            : True
and Path('') is                       : '.'

Four of six environments give a relative path, and the third row is the dangerous one. os.environ.get returns the empty string because the variable is set — emptiness is not absence — so the default never applies, and Path("") is Path("."). The run root becomes the current working directory.

That is not a contrived input. It is what a wrapper produces:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

import os
from pathlib import Path

WRAPPERS = [
    ("export FLOW_RUN_ROOT=$SCRATCH", "SCRATCH is unset", ""),
    ("FLOW_RUN_ROOT=\"$(cat site.conf)\"", "site.conf is empty", ""),
    ('FLOW_RUN_ROOT="$ROOT/runs"', "ROOT is unset", "/runs"),
    ("export FLOW_RUN_ROOT=", "a trailing equals", ""),
]

print(f"{'what the wrapper did':<36} {'why':<24} value")
for line, why, val in WRAPPERS:
    print(f"{line:<36} {why:<24} {val!r}")

print()
print("wrappers producing an empty value:",
      sum(1 for _, _, v in WRAPPERS if v == ""))
print("and the one producing '/runs'    :",
      "writes to the filesystem root — if it can")
print()
print("Chapter 24.5 called this:", "the unset-variable hazard")
print("33.11 §4 measured       :", "Path('') is Path('.')")
print("this is the same fact   :", "arriving through an environment variable")

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

what the wrapper did                 why                      value
export FLOW_RUN_ROOT=$SCRATCH        SCRATCH is unset         ''
FLOW_RUN_ROOT="$(cat site.conf)"     site.conf is empty       ''
FLOW_RUN_ROOT="$ROOT/runs"           ROOT is unset            '/runs'
export FLOW_RUN_ROOT=                a trailing equals        ''

wrappers producing an empty value: 3
and the one producing '/runs'    : writes to the filesystem root — if it can

Chapter 24.5 called this: the unset-variable hazard
33.11 §4 measured       : Path('') is Path('.')
this is the same fact   : arriving through an environment variable

Three of four ordinary wrapper lines produce an empty value, and the fourth produces /runs at the filesystem root. Chapter 24.5's unset-variable hazard and Chapter 33.11 Section 4's Path("") finding are the same fact meeting here, and the symptom is a sweep whose results are not where anyone expects.

The fix is to treat empty as absent, and to make the result absolute:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

import os
from pathlib import Path

ROOT = Path("/opt/flow")


def env_path(name, default):
    """Empty or whitespace-only is ABSENT. The result is always absolute."""
    raw = os.environ.get(name, "")
    if not raw.strip():
        return Path(default)
    return Path(raw.strip()).expanduser().resolve()


CASES = [("unset", None), ("absolute", "/tmp/runs"), ("empty", ""),
         ("three spaces", "   "), ("relative", "runs"), ("tilde", "~/runs")]

print(f"{'FLOW_RUN_ROOT':<16} {'absolute':<10} {'default used':<14} "
      f"outside the cwd")
for label, val in CASES:
    os.environ.pop("FLOW_RUN_ROOT", None)
    if val is not None:
        os.environ["FLOW_RUN_ROOT"] = val
    p = env_path("FLOW_RUN_ROOT", ROOT / "runs")
    print(f"{label:<16} {str(p.is_absolute()):<10} "
          f"{str(p == ROOT / 'runs'):<14} {not str(p).startswith(str(Path.cwd()))}")

os.environ.pop("FLOW_RUN_ROOT", None)
print()
print("environments giving a relative path now:", 0)
print("empty and whitespace fall back to the default:", True)
print("tilde is expanded                            :", True)
print("a relative value is resolved against the cwd :", True)

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

FLOW_RUN_ROOT    absolute   default used   outside the cwd
unset            True       True           True
absolute         True       False          True
empty            True       True           True
three spaces     True       True           True
relative         True       False          False
tilde            True       False          True

environments giving a relative path now: 0
empty and whitespace fall back to the default: True
tilde is expanded                            : True
a relative value is resolved against the cwd : True

Zero relative paths, and empty, whitespace and unset all reach the default. Four lines, and the last row of the table is the remaining honest surprise: a genuinely relative value like runs is resolved against the current directory, so it is absolute but it still depends on where the job started. That is the caller's choice and now at least it is recorded as an absolute path, which Chapter 33.14 Section 2 needs for cmd.sh.

4. The Resolution Chain

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

import os
from pathlib import Path


def resolve(name, cli=None, env=None, file_value=None, default=None):
    """Highest priority first. Returns (value, where it came from)."""
    if cli is not None:
        return cli, "command line"
    raw = os.environ.get(env or "", "")
    if raw.strip():
        return raw.strip(), f"environment ({env})"
    if file_value is not None:
        return file_value, "config file"
    return default, "default"


os.environ["FLOW_SIM"] = "iverilog"

CASES = [
    ("all four present", "vcs", "FLOW_SIM", "xrun", "iverilog"),
    ("no cli flag", None, "FLOW_SIM", "xrun", "iverilog"),
    ("no cli, no env", None, "FLOW_NOPE", "xrun", "iverilog"),
    ("only the default", None, "FLOW_NOPE", None, "iverilog"),
]

print(f"{'situation':<20} {'value':<12} came from")
for label, cli, env, fv, dflt in CASES:
    val, src = resolve("sim", cli, env, fv, dflt)
    print(f"{label:<20} {val!r:<12} {src}")

del os.environ["FLOW_SIM"]
print()
print("layers                 :", 4)
print("the order is fixed     :", "cli > env > file > default")
print("why cli is highest     :", "it is the most specific statement of intent")
print("why file beats default :", "the project said so deliberately")
print()
print("and every resolved value:", "records WHERE it came from")
print("which 33.14 needs for   :", "cmd.sh and the run record")

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

situation            value        came from
all four present     'vcs'        command line
no cli flag          'iverilog'   environment (FLOW_SIM)
no cli, no env       'xrun'       config file
only the default     'iverilog'   default

layers                 : 4
the order is fixed     : cli > env > file > default
why cli is highest     : it is the most specific statement of intent
why file beats default : the project said so deliberately

and every resolved value: records WHERE it came from
which 33.14 needs for   : cmd.sh and the run record

Four layers, one fixed order, and every value knows its source. That second return value is the part people omit and the part that pays for itself: when a sweep runs with the wrong simulator, the question is never "what was the value" — the printed command shows that — it is "why", and the answer is a layer name.

Note that resolve applies the same empty-is-absent rule as Section 3. A config system with four layers has four chances to make the Section 3 mistake, so the check belongs in the one function all four go through.

5. Where Each Fact Belongs

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

PLACEMENT = [
    ("the seeds", "per run", "--seed, repeatable", "cli",
     "changes every invocation"),
    ("the test list", "per project", "flow.toml in the repo", "file",
     "reviewed, versioned with the RTL"),
    ("the block name", "per project", "flow.toml", "file", "same"),
    ("the scratch root", "per user", "FLOW_RUN_ROOT", "env",
     "differs per person, not per project"),
    ("the simulator", "per site", "FLOW_SIM", "env",
     "differs per machine, not per project"),
    ("the licence server", "per site", "LM_LICENSE_FILE", "env",
     "NEVER in the repo — §6 callout"),
    ("the filelist path", "derivable", "root / 'rtl' / 'files.f'", "derived",
     "a config option nobody should set"),
]

print(f"{'fact':<20} {'varies':<12} {'how it arrives':<28} {'home':<8} why")
for fact, varies, how, home, why in PLACEMENT:
    print(f"{fact:<20} {varies:<12} {how:<28} {home:<8} {why}")

from collections import Counter
print()
for home, n in sorted(Counter(p[3] for p in PLACEMENT).items()):
    print(f"  {home:<8} {n}")
print()
print("the fourth home is  :", "derived — not configuration at all")
print("the test for it     :", "could the tool work it out itself?")
print("what a wrong option costs:", "a way for someone to set it inconsistently")

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

fact                 varies       how it arrives               home     why
the seeds            per run      --seed, repeatable           cli      changes every invocation
the test list        per project  flow.toml in the repo        file     reviewed, versioned with the RTL
the block name       per project  flow.toml                    file     same
the scratch root     per user     FLOW_RUN_ROOT                env      differs per person, not per project
the simulator        per site     FLOW_SIM                     env      differs per machine, not per project
the licence server   per site     LM_LICENSE_FILE              env      NEVER in the repo — §6 callout
the filelist path    derivable    root / 'rtl' / 'files.f'     derived  a config option nobody should set

  cli      1
  derived  1
  env      3
  file     2

the fourth home is  : derived — not configuration at all
the test for it     : could the tool work it out itself?
what a wrong option costs: a way for someone to set it inconsistently

The fourth home is the one people forget, and it is the best one. The filelist lives at rtl/files.f under the project root in every project this tool will ever see, so making it configurable adds a knob whose only purpose is to be set wrong — and then a run where --filelist and --root disagree is a new failure mode that configuration created.

Chapter 33.12's pre-flight becomes much simpler when a value is derived: there is nothing to validate beyond "does the derived path exist", and no way for two settings to contradict each other.

6. A Config File, and Which Format

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

import json
import sys
import tomllib
from pathlib import Path

TOML = """# flow.toml - checked into the RTL repository
[project]
name = "chip_a"
block = "axi_fifo"

[regression]
tests = ["axi_burst_test", "axi_lite_test", "axi_wrap_test", "axi_err_test"]
default_seeds = [4121, 7734, 9001]
"""

Path("flow.toml").write_text(TOML, encoding="utf-8")
with open("flow.toml", "rb") as fh:          # tomllib wants BINARY mode
    cfg = tomllib.load(fh)

print("tomllib is in the standard library:", True, "| python",
      ".".join(str(n) for n in sys.version_info[:2]))
print()
print("parsed:")
for section, body in cfg.items():
    print(f"  [{section}]")
    for k, v in body.items():
        print(f"    {k:<15} {v!r}")

print()
print("tomllib can WRITE toml:", hasattr(tomllib, "dump") or
      hasattr(tomllib, "dumps"))
print("so a generated config must be:", "json, or a template you own")

cfg_json = json.dumps(cfg, indent=2, sort_keys=True)
print()
print("json round-trips exactly   :", json.loads(cfg_json) == cfg)
print("and byte-identically       :",
      json.dumps(json.loads(cfg_json), indent=2, sort_keys=True) == cfg_json)
print("which 31.7 required for    :", "a generated file a diff can check")

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

tomllib is in the standard library: True | python 3.13

parsed:
  [project]
    name            'chip_a'
    block           'axi_fifo'
  [regression]
    tests           ['axi_burst_test', 'axi_lite_test', 'axi_wrap_test', 'axi_err_test']
    default_seeds   [4121, 7734, 9001]

tomllib can WRITE toml: False
so a generated config must be: json, or a template you own

json round-trips exactly   : True
and byte-identically       : True
which 31.7 required for    : a generated file a diff can check

tomllib reads and cannot write. That is a deliberate design decision in the standard library and it settles the format question by splitting it: TOML for the file a human maintains — comments, sections, no quoting of keys — and JSON for anything the tool generates, because sort_keys=True makes the output byte-stable and a generated file has to be diffable, which is Chapter 31.7's rule.

Two mechanical details worth carrying. tomllib.load requires a file opened in binary mode; passing a text handle raises TypeError, and the error does not mention binary. And the config file is checked into the RTL repository, not into the tool's repository — it describes the project, so it travels with the project and is reviewed by the people who own the test list.

7. The Resolved Config, Validated Once

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

import os
from dataclasses import dataclass
from pathlib import Path


@dataclass(frozen=True)
class Config:
    """Everything the tool needs, resolved, validated, and immutable."""
    root: Path
    run_root: Path
    sim: str
    tests: tuple
    seeds: tuple
    block: str

    @property
    def filelist(self):
        """Derived — §5's fourth home. Not settable."""
        return self.root / "rtl" / "files.f"


def env_path(name, default):
    raw = os.environ.get(name, "")
    return (Path(raw.strip()).expanduser().resolve() if raw.strip()
            else Path(default))


def env_str(name, default):
    raw = os.environ.get(name, "")
    return raw.strip() or default


def build_config(root, file_cfg, cli_seeds=None):
    root = Path(root).expanduser().resolve()
    return Config(
        root=root,
        run_root=env_path("FLOW_RUN_ROOT", root / "runs"),
        sim=env_str("FLOW_SIM", "iverilog"),
        tests=tuple(file_cfg["regression"]["tests"]),
        seeds=tuple(cli_seeds or file_cfg["regression"]["default_seeds"]),
        block=file_cfg["project"]["block"],
    )


def validate(cfg):
    problems = []
    if not cfg.root.is_dir():
        problems.append(f"root is not a directory: {cfg.root}")
    if not cfg.tests:
        problems.append("no tests configured")
    if not cfg.seeds:
        problems.append("no seeds given")
    if any(not isinstance(s, int) for s in cfg.seeds):
        problems.append(f"non-integer seed in {cfg.seeds}")
    if not cfg.block:
        problems.append("no block name configured")
    return problems


FILE_CFG = {"project": {"name": "chip_a", "block": "axi_fifo"},
            "regression": {"tests": ["axi_burst_test", "axi_lite_test"],
                           "default_seeds": [4121, 7734]}}

Path("proj").mkdir(exist_ok=True)
os.environ.pop("FLOW_RUN_ROOT", None)
os.environ["FLOW_SIM"] = "iverilog"

cfg = build_config("proj", FILE_CFG)
print("resolved config")
for field in ("root", "run_root", "sim", "tests", "seeds", "block"):
    val = getattr(cfg, field)
    shown = val.name if isinstance(val, Path) else val
    print(f"  {field:<10} {shown!r}")
print(f"  {'filelist':<10} {str(cfg.filelist.relative_to(cfg.root))!r} (derived)")

print()
print("validation problems:", validate(cfg) or "none")
print()
bad = build_config("nosuch", {"project": {"block": ""},
                              "regression": {"tests": [],
                                             "default_seeds": ["x"]}})
print("a deliberately broken config:")
for p in validate(bad):
    print("  -", p)
print()
print("problems reported in one pass:", len(validate(bad)))
print("the config is frozen          :", True)
print("so a later stage cannot       :", "quietly change the run root")
del os.environ["FLOW_SIM"]

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

resolved config
  root       'proj'
  run_root   'runs'
  sim        'iverilog'
  tests      ('axi_burst_test', 'axi_lite_test')
  seeds      (4121, 7734)
  block      'axi_fifo'
  filelist   'rtl/files.f' (derived)

validation problems: none

a deliberately broken config:
  - root is not a directory: nosuch
  - no tests configured
  - non-integer seed in ('x',)
  - no block name configured

problems reported in one pass: 4
the config is frozen          : True
so a later stage cannot       : quietly change the run root

Four problems from four independent checks, in one pass. The good config reports none and the broken one reports everything wrong with it, which is the pair Chapter 33.12 Section 8 insisted on: a validator that has only ever been seen to pass is not known to work.

The first bullet reads root is not a directory: nosuch rather than an absolute path, and that is worth a word because build_config calls .resolve(). It did resolve — to an absolute path inside the temporary directory these examples run in — and what you are seeing is that path with the directory prefix removed, exactly as Chapter 33.14 Section 3 required of anything published. The resolved value is absolute; only its displayed form is trimmed.

Three properties of that Config are worth carrying forward. It is frozen, so no later stage can change the run root after cmd.sh has recorded it. filelist is a property, so the derived value cannot be set and cannot disagree with root. And validation happens once, on the resolved object, reporting everything in one pass — Chapter 33.11 Section 7's collect-then-decide applied to configuration.

8. The Tool Another Team Can Run

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

BEFORE = [
    ("the author", "yes", "it is their path"),
    ("the author's teammate", "no", "/proj/.../users/azhar"),
    ("another block, same project", "no", "axi_fifo hard-coded"),
    ("another project", "no", "chip_a hard-coded"),
    ("another site", "no", "licence server, tool version"),
    ("CI, no licence", "no", "xrun only"),
]
AFTER = [
    ("the author", "yes", "defaults"),
    ("the author's teammate", "yes", "FLOW_RUN_ROOT"),
    ("another block, same project", "yes", "flow.toml"),
    ("another project", "yes", "flow.toml + its own root"),
    ("another site", "yes", "FLOW_SIM, LM_LICENSE_FILE"),
    ("CI, no licence", "yes", "FLOW_SIM=iverilog"),
]

print(f"{'who can run it':<28} {'34.1':<6} {'34.5':<6} what makes it work")
for (who, before, _), (_, after, how) in zip(BEFORE, AFTER):
    print(f"{who:<28} {before:<6} {after:<6} {how}")

print()
print("audiences before:", sum(1 for _, b, _ in BEFORE if b == "yes"))
print("audiences after :", sum(1 for _, a, _ in AFTER if a == "yes"))
print()
print("lines of parsing logic changed:", 0)
print("facts moved out of the source :", 7)
print("new ways to misconfigure it   :", "1 — the empty env var, now handled")

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

who can run it               34.1   34.5   what makes it work
the author                   yes    yes    defaults
the author's teammate        no     yes    FLOW_RUN_ROOT
another block, same project  no     yes    flow.toml
another project              no     yes    flow.toml + its own root
another site                 no     yes    FLOW_SIM, LM_LICENSE_FILE
CI, no licence               no     yes    FLOW_SIM=iverilog

audiences before: 1
audiences after : 6

lines of parsing logic changed: 0
facts moved out of the source : 7
new ways to misconfigure it   : 1 — the empty env var, now handled

One audience became six, and the parsing logic did not change. That is the same control as Chapter 34.2 Section 10: nothing about what the tool does was touched, only what it assumes.

The last row is the honest cost. Configuration adds failure modes — a value can now be empty, contradictory, or absent — which is why Section 3's four-line env_path and Section 7's one-pass validate are not optional extras. A tool that is configurable and does not validate its configuration has traded seven assumptions for an unbounded number of them.

9. Predict, Run, Observe

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

import os
from pathlib import Path

ROOT = Path("/opt/flow")

# A
os.environ["FLOW_X"] = ""
print("A:", repr(os.environ.get("FLOW_X", "DEFAULT")))

# B
print("B:", repr(str(Path(os.environ.get("FLOW_X", ROOT / "runs")))))

# C
raw = os.environ.get("FLOW_X", "")
print("C:", repr(str(Path(raw.strip()) if raw.strip() else ROOT / "runs")))

# D
os.environ["FLOW_X"] = "~/runs"
print("D:", str(Path(os.environ["FLOW_X"]).expanduser()).startswith("~"))
del os.environ["FLOW_X"]

The verified output:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

A: ''
B: '.'
C: '/opt/flow/runs'
D: False

A — the default is not used. os.environ.get found the key, and an empty value is a value.

B — so the annotation resolves to ., the current directory. Section 3 in one line.

C — the four-line fix: raw.strip() is falsy, so the default applies and the path is the intended absolute one.

D — expanduser() removed the tilde, so the result does not start with ~. Chapter 33.11 Section 4 measured the version without it, where a path a user can cat in their shell reads as missing.

10. Exercises

Exercise 1 — who varies it. For every constant in your own tool, write down who changes it and how often.

Exercise 2 — not by type. Find two paths in your code that belong in different homes.

Exercise 3 — four things right. Name the four good decisions in the annotation.

Exercise 4 — the type question. Confirm for yourself that a Path default and a str value are both safe inside Path(...).

Exercise 5 — set but empty. Run your own resolver with the variable set to the empty string.

Exercise 6 — count the relative outcomes. Try six environments and count how many give a relative path.

Exercise 7 — the wrapper. Find a line in your own shell wrappers that could export an empty value.

Exercise 8 — treat empty as absent. Write env_path and confirm all six cases become absolute.

Exercise 9 — the remaining surprise. Explain what resolve() does to a genuinely relative value, and why that still has to be recorded.

Exercise 10 — four layers. Implement the resolution chain for one setting.

Exercise 11 — where it came from. Return the source alongside the value and say which question that answers.

Exercise 12 — one function, one check. Explain why the empty-is-absent rule belongs in the shared resolver.

Exercise 13 — place your facts. Assign each of your constants to cli, env, file or derived.

Exercise 14 — the fourth home. Find a value that is currently configurable and should be derived.

Exercise 15 — the knob set wrong. Describe the failure when two settings that should agree do not.

Exercise 16 — credentials. Confirm no licence server, host or token is committed in your repository.

Exercise 17 — the vendor's own name. Explain why inventing FLOW_LICENCE would be worse than using LM_LICENSE_FILE.

Exercise 18 — TOML by hand. Write a flow.toml for your project with comments.

Exercise 19 — binary mode. Pass tomllib.load a text handle and read the error.

Exercise 20 — no writer. Confirm tomllib has no dump, and decide what your generated config uses.

Exercise 21 — byte-stable JSON. Confirm sort_keys=True round-trips identically.

Exercise 22 — which repository. Decide where your config file lives and who reviews it.

Exercise 23 — a frozen config. Build one and try to change a field after construction.

Exercise 24 — a derived property. Make one value a property and confirm it cannot be set.

Exercise 25 — validate once. Write a validate that reports every problem in one pass.

Exercise 26 — prove it can fail. Build a deliberately broken config and confirm every check in your validate fires.

Exercise 27 — count the audiences. List who could run your tool before and after.

Exercise 28 — the honest cost. Name the failure modes configuration added, and the check that covers each.

11. Summary

A fact's home follows from who varies it — per run, per user, per site, per project — not from whether it is a path.
The annotation does four things right: a prefixed name, a default, a default derived from ROOT, and a Path wrapper that normalises both branches.
The mixed Path/str default is safe, which is worth confirming because it looks like a defect.
os.environ.get treats set-but-empty as set, so the default never applies and Path("") is Path(".") — the run root silently becomes the current directory.
Four of six environments give a relative path, and three of four ordinary wrapper lines produce an empty value.
Treating empty and whitespace as absent, expanding ~, and resolving takes four lines and removes every relative outcome.
A relative value still depends on where the job started, so the resolved absolute path has to be recorded — Chapter 33.14's cmd.sh.
Four layers in a fixed order — cli, env, file, default — and every value records its source, because the question after a wrong run is "why", not "what".
The fourth home is "derived", and it is the best one: a value the tool can work out itself needs no option, and an option exists only to be set inconsistently.
The licence server belongs in the environment and never in the repository, under the vendor's own variable name rather than an invented one.
tomllib is stdlib and read-only. TOML for the file a human maintains, JSON with sort_keys=True for anything generated — and tomllib.load needs a binary handle.
The config file belongs in the RTL repository, because it describes the project and is reviewed by the people who own the test list.
Freeze the config, derive what you can as a property, and validate once on the resolved object, reporting everything in one pass.
Check the validator against a deliberately broken config: four problems from four independent checks, where the good config reports none.
One audience became six with no change to the parsing logic, and the honest cost is that configuration adds failure modes, which is why the resolver and the validator are not optional.

The next chapter writes down what a new owner needs: the contract, the exit codes, the configuration, and what the tool deliberately does not do.

Where this fits

Part of the Python curriculum.