Safe Cleanup of Run and Scratch Directories

Part II · Module 18 · Chapter 18.9 · Safe Cleanup of Run and Scratch Directories

1. Why This Is Different

Chapter 15.9 established that deletion has no undo, no prompt, and runs unattended. Everything in this chapter follows from the third of those: a cleanup step executes at 3am against paths assembled from a config somebody edited on Tuesday, and there is nobody to notice that one of them is wrong.

The specific risk is not that rmtree is dangerous — it does exactly what it says. It is that the path handed to it is computed, and Chapter 18.2 Section 4 showed that a computed path can land somewhere other than intended without anything raising.

2. Delete by Ownership, Not by Pattern

The instinct is to describe what to remove:

from pathlib import Path

ROOT = Path(".").resolve()
(ROOT / "runs" / "smoke_1234").mkdir(parents=True)
(ROOT / "runs" / "smoke_1234" / "sim.log").write_text("generated\n")
(ROOT / "rtl").mkdir()
(ROOT / "rtl" / "lint.log").write_text("CHECKED IN -- somebody's reference output\n")

by_pattern = sorted(ROOT.rglob("*.log"))
print("a pattern finds:", [str(p.relative_to(ROOT)) for p in by_pattern])

RUNS = ROOT / "runs"
by_ownership = sorted(p for p in RUNS.rglob("*.log"))
print("ownership finds:", [str(p.relative_to(ROOT)) for p in by_ownership])

a pattern finds: ['rtl/lint.log', 'runs/smoke_1234/sim.log']
ownership finds: ['runs/smoke_1234/sim.log']

The pattern found a checked-in file. *.log is a statement about names, and a name tells you nothing about who created it — rtl/lint.log is somebody's committed reference output and deleting it is a git checkout away from being noticed, which is to say several days.

Ownership is a statement about the tree: Chapter 18.4 Section 2 divided directories into committed inputs and generated outputs, and the rule falls straight out of it.

A script may delete only inside directories it created. Everything else is somebody else's.

That rule is checkable, it does not depend on naming conventions, and it survives a colleague adding a file type nobody anticipated.

3. The Containment Check

The canonical form:

assert scratch.is_relative_to(RUNS)   # never delete outside runs/
shutil.rmtree(scratch, ignore_errors=True)

is_relative_to (Chapter 18.3) answers whether one path lies inside another, and it catches the cases a computed path produces:

from pathlib import Path

ROOT = Path(".").resolve()
RUNS = ROOT / "runs"
RUNS.mkdir()
(ROOT / "rtl").mkdir()

candidates = ["smoke_1234", "../rtl", "/etc", "smoke_1234/../../rtl"]

for c in candidates:
    target = (RUNS / c).resolve()
    print(f"{c!r:<24} inside runs/: {target.is_relative_to(RUNS)}")

'smoke_1234'             inside runs/: True
'../rtl'                 inside runs/: False
'/etc'                   inside runs/: False
'smoke_1234/../../rtl'   inside runs/: False

Three of the four are refused, including the absolute component from Chapter 18.2 Section 4 that silently discarded its prefix, and the .. escape that a naive startswith check would miss.

4. assert Is the Wrong Statement for This

The canonical annotation writes the check with assert, and Chapter 14.7 Section 7 established — with an executed demonstration — that python3 -O removes every assert from a program.

The two cannot both be right, and the resolution is not subtle:

import sys

print("this program's asserts are active:", __debug__)
print("under python3 -O, __debug__ is False and every assert is removed")
print()
print("a check that disappears under a flag somebody else sets")
print("is not a safety check -- it is a comment that occasionally runs")

this program's asserts are active: True
under python3 -O, __debug__ is False and every assert is removed

a check that disappears under a flag somebody else sets
is not a safety check -- it is a comment that occasionally runs

The check in the annotation is exactly right; the statement is not. A wrapper script, a deployment image or a colleague's habit can add -O, and the protection silently stops existing while the rmtree on the next line does not.

So carry the intent and change the mechanism — Chapter 14.7's rule was assert for invariants you believe cannot fail, raise for anything depending on input, and a path built from configuration is input:

import shutil
from pathlib import Path

ROOT = Path(".").resolve()
RUNS = ROOT / "runs"
(RUNS / "smoke_1234").mkdir(parents=True)
(RUNS / "smoke_1234" / "sim.log").write_text("x\n")
(ROOT / "rtl").mkdir()


def remove_run(target):
    """Remove one run directory. Refuses anything outside RUNS."""
    t = (RUNS / target).resolve()
    if not t.is_relative_to(RUNS):
        raise ValueError(f"refusing to delete outside runs/: {target!r}")
    shutil.rmtree(t)
    return f"removed {t.relative_to(ROOT)}"


print(remove_run("smoke_1234"))

for bad in ["../rtl", "/etc"]:
    try:
        remove_run(bad)
    except ValueError as e:
        print("refused:", e)

print("rtl survived:", (ROOT / "rtl").is_dir())

removed runs/smoke_1234
refused: refusing to delete outside runs/: '../rtl'
refused: refusing to delete outside runs/: '/etc'
rtl survived: True

5. Containment Is Not Sufficient

The hole in the check, and it is a real one:

from pathlib import Path

ROOT = Path(".").resolve()
RUNS = ROOT / "runs"
RUNS.mkdir()

configured = ""
target = (RUNS / configured).resolve()

print("configured value   :", repr(configured))
print("target             :", target.relative_to(ROOT))
print("is it inside RUNS? :", target.is_relative_to(RUNS))
print("is it RUNS itself? :", target == RUNS)
print()
print("the containment check PASSES and the target is every run you have")

configured value   : ''
target             : runs
is it inside RUNS? : True
is it RUNS itself? : True

the containment check PASSES and the target is every run you have

An empty configured value produces the runs root, and is_relative_to reports True because a path is trivially relative to itself. The check passes, and rmtree would remove every run directory in the project.

An empty string is not exotic. It is what a missing environment variable, an unset config key or a stripped blank line produces, and Chapter 1.7's hidden-inputs section is exactly about values arriving from places nobody is watching.

So the rule needs a second clause:

import shutil
from pathlib import Path

ROOT = Path(".").resolve()
RUNS = ROOT / "runs"
for name in ["smoke_1234", "cdc_7"]:
    (RUNS / name).mkdir(parents=True)


def remove_run(target):
    t = (RUNS / target).resolve()
    if not t.is_relative_to(RUNS):
        raise ValueError(f"outside runs/: {target!r}")
    if t == RUNS:
        raise ValueError(f"refusing to delete the runs root itself: {target!r}")
    if not t.is_dir():
        return f"nothing to remove at {target!r}"
    shutil.rmtree(t)
    return f"removed {t.relative_to(ROOT)}"


for target in ["smoke_1234", "", ".", "missing_run"]:
    try:
        print(f"{target!r:<14} -> {remove_run(target)}")
    except ValueError as e:
        print(f"{target!r:<14} -> REFUSED: {e}")

print()
print("remaining:", sorted(p.name for p in RUNS.iterdir()))

'smoke_1234'   -> removed runs/smoke_1234
''             -> REFUSED: refusing to delete the runs root itself: ''
'.'            -> REFUSED: refusing to delete the runs root itself: '.'
'missing_run'  -> nothing to remove at 'missing_run'

remaining: ['cdc_7']

Both "" and "." resolve to the root and are refused. One run was removed, one survives, and a target that does not exist is reported rather than raising — which is Chapter 15.9's missing_ok reasoning applied to a directory.

The complete rule is three clauses: inside the owned root, not the root itself, and actually there.

6. Resolve Before Checking

The check in Sections 4 and 5 calls .resolve() first, and that is load-bearing:

from pathlib import Path

ROOT = Path(".").resolve()
RUNS = ROOT / "runs"
RUNS.mkdir()
(ROOT / "rtl").mkdir()

sneaky = RUNS / "smoke_1234" / ".." / ".." / "rtl"

print("unresolved parts    :", sneaky.parts[-4:])
print("unresolved check    :", sneaky.is_relative_to(RUNS))
print("resolved check      :", sneaky.resolve().is_relative_to(RUNS))
print("where it really goes:", sneaky.resolve().relative_to(ROOT))

unresolved parts    : ('smoke_1234', '..', '..', 'rtl')
unresolved check    : True
resolved check      : False
where it really goes: rtl

The unresolved check passes. is_relative_to is a string-structure comparison — it sees a path beginning with runs/ and says yes, because .. is still a literal component (Chapter 18.1 Section 6 and Chapter 18.4 Section 7 both noted that .. survives in parts).

Only after resolve() does the path become what the filesystem will actually use, and the check becomes meaningful. Resolve first, then check, then act — and resolve exactly once, storing the result, so the value you checked is the value you delete.

7. Prefer Moving Aside, and Beware ignore_errors

Two closing refinements.

Chapter 15.9 Section 7 recommended moving aside rather than removing where a flow allows it, and a run directory is a good candidate — a quarantined run is recoverable until somebody confirms it is not needed:

import shutil
from pathlib import Path

ROOT = Path(".").resolve()
RUNS = ROOT / "runs"
TRASH = ROOT / "runs" / ".trash"
(RUNS / "smoke_1234").mkdir(parents=True)
(RUNS / "smoke_1234" / "sim.log").write_text("evidence\n")
TRASH.mkdir()


def retire(name):
    src = (RUNS / name).resolve()
    if not src.is_relative_to(RUNS) or src == RUNS:
        raise ValueError(name)
    dest = TRASH / name
    if dest.exists():
        shutil.rmtree(dest)
    src.rename(dest)
    return dest


d = retire("smoke_1234")
print("retired to      :", d.relative_to(ROOT))
print("evidence intact :", (d / "sim.log").read_text().strip())
print("original gone   :", not (RUNS / "smoke_1234").exists())

retired to      : runs/.trash/smoke_1234
evidence intact : evidence
original gone   : True

And the flag in the canonical annotation deserves a caveat:

import shutil
from pathlib import Path

missing = Path("runs/never_existed")

shutil.rmtree(missing, ignore_errors=True)
print("ignore_errors on a missing path: no error")

try:
    shutil.rmtree(missing)
except FileNotFoundError as e:
    print("without it                     :", type(e).__name__)

ignore_errors on a missing path: no error
without it                     : FileNotFoundError

ignore_errors=True lets a cleanup loop finish past an already-absent directory, which is what it is for. It also swallows permission errors and partial failures, so a directory that could not be removed is reported as success — Chapter 14.9's silent handler in a flag. Where it matters, check afterwards that the directory is actually gone.

8. Predict, Run, Observe

from pathlib import Path

ROOT = Path(".").resolve()
RUNS = ROOT / "runs"
RUNS.mkdir()
(RUNS / "smoke_1234").mkdir()

for value in ["smoke_1234", "", ".", "..", "/tmp", "smoke_1234/.."]:
    t = (RUNS / value).resolve()
    inside = t.is_relative_to(RUNS)
    is_root = t == RUNS
    print(f"{value!r:<16} inside={str(inside):<5} is_root={str(is_root):<5} allowed={inside and not is_root}")

'smoke_1234'     inside=True  is_root=False allowed=True
''               inside=True  is_root=True  allowed=False
'.'              inside=True  is_root=True  allowed=False
'..'             inside=False is_root=False allowed=False
'/tmp'           inside=False is_root=False allowed=False
'smoke_1234/..'  inside=True  is_root=True  allowed=False

Only the first is allowed, and the five refusals come from two different clauses — which is the argument for having both.

'', '.' and 'smoke_1234/..' all resolve to the root. Three different-looking inputs, one dangerous target, and only the is_root clause catches any of them. The last is the one worth noting: a path that starts inside and climbs back out to exactly the root still passes containment.

'..' and '/tmp' escape entirely and are caught by containment.

Six inputs, one allowed, and no deletion in this table happened at all — the check runs before rmtree, which is the only ordering that helps.

9. Exercises

Exercise 1 — pattern versus ownership. Create a .log inside and outside a run directory. Show a pattern finds both and an ownership scan finds one.

Exercise 2 — the four escapes. Run the containment check against a relative name, a .., an absolute path and a nested escape. Record which are refused.

Exercise 3 — the optimiser. Write the check with assert, run the script with python3 -O, and confirm the check is gone. Then convert it to raise.

Exercise 4 — the empty string. Show that an empty configured value passes containment and targets the root. Add the second clause.

Exercise 5 — three spellings of the root. Find three inputs that resolve to the runs root. Confirm your guard catches all three.

Exercise 6 — resolve first. Run the containment check on an unresolved path containing .. and show it passes. Then resolve and re-check.

Exercise 7 — retire instead. Implement the move-aside version. Say how you would eventually empty the quarantine, and who decides.

Exercise 8 — what ignore_errors hides. Make a directory unremovable and call rmtree with the flag. Add the check that notices it is still there.

10. Summary

• A cleanup script runs unattended against computed paths, which is what makes it different from any other deletion.
• Delete by ownership, not by pattern — a name says nothing about who created the file, and a pattern will find checked-in output.
• The rule is: only inside directories the script created, which follows from Chapter 18.4's committed-versus-generated split.
• is_relative_to catches .. escapes and absolute components that silently discarded their prefix.
• Do not write the check with assert — python3 -O removes it, and a check that vanishes under a flag is not a check.
• Containment is necessary and not sufficient: an empty configured value resolves to the root, which is trivially relative to itself.
• The complete rule is three clauses: inside the owned root, not the root itself, and actually present.
• Resolve before checking, because .. survives in parts and an unresolved containment check passes.
• Resolve once and store it, so the value you checked is the value you delete.
• ignore_errors=True also swallows permission failures, so verify the directory is gone where it matters.

The last chapter of this module makes every path in a script independent of where it was launched from.