Asking About a Path: exists, is_file, is_dir

Part I · Module 15 · Chapter 15.8 · Asking About a Path: exists, is_file, is_dir

1. Asking Instead of Opening

Chapter 15.2 found out a file was missing by trying to open it and catching FileNotFoundError. That is often right. But sometimes the absence is not an error — it is information:

• a run directory that does not exist yet means the job has not started;
• a missing sim.log means the tool never produced output;
• a files.f that is not there means an earlier step failed.

In each case you want to branch, not to fail. For that you ask.

2. The Four Questions

from pathlib import Path

Path("sim.log").write_text("Compile start\n")

p = Path("sim.log")
print("exists   :", p.exists())
print("is_file  :", p.is_file())
print("is_dir   :", p.is_dir())
print("st_size  :", p.stat().st_size)

exists   : True
is_file  : True
is_dir   : False
st_size  : 14

Path comes from pathlib, a standard-library module (Chapter 13.4) that represents a path as an object rather than a string. Module 18 Chapter 18.3 covers it properly; this chapter uses only the four methods above, which are the ones the questions need.

st_size is the size in bytes. For a text file of ASCII characters that is also the character count — Compile start\n is 14.

3. exists() Is Not "Is There a File"

The trap:

import os
from pathlib import Path

Path("sim.log").write_text("Compile start\n")
os.mkdir("runs")

for name in ["sim.log", "runs", "missing.log"]:
    p = Path(name)
    print(f"{name:<12} exists={str(p.exists()):<5} is_file={str(p.is_file()):<5} "
          f"is_dir={p.is_dir()}")

sim.log      exists=True  is_file=True  is_dir=False
runs         exists=True  is_file=False is_dir=True
missing.log  exists=False is_file=False is_dir=False

exists() is True for the directory. It answers "is there anything at this name", which is rarely the question. So this is wrong:

import os
from pathlib import Path

os.mkdir("runs")

p = Path("runs")
if p.exists():
    with open(p) as f:
        print(f.read())

Traceback (most recent call last):
  File "wrong.py", line 8, in <module>
    with open(p) as f:
         ~~~~^^^
IsADirectoryError: [Errno 21] Is a directory: 'runs'

That is Chapter 15.2 Section 7's second failure, and now it has a cause: the guard checked the wrong property.

Use is_file() before opening something for reading. Use is_dir() before treating a name as a directory. Use exists() only when you genuinely mean "anything at all".

Note also that all three are False for a missing path — none of them raises.

4. stat() Does Raise

stat() is the exception, because it has to actually ask the filesystem:

from pathlib import Path

Path("sim.log").write_text("Compile start\n")

for name in ["sim.log", "missing.log"]:
    p = Path(name)
    try:
        print(f"{name:<12} {p.stat().st_size} bytes")
    except FileNotFoundError:
        print(f"{name:<12} no such file")

sim.log      14 bytes
missing.log  no such file

So the safe shape is to gate it:

from pathlib import Path

Path("sim.log").write_text("Compile start\n")
Path("empty.log").write_text("")


def describe(name):
    p = Path(name)
    if not p.exists():
        return "missing"
    if not p.is_file():
        return "not a file"
    return f"{p.stat().st_size} bytes"


for name in ["sim.log", "empty.log", "missing.log"]:
    print(f"{name:<12} {describe(name)}")

sim.log      14 bytes
empty.log    0 bytes
missing.log  missing

The is_file() check before stat() is not decoration. A directory has an st_size too, and its value is a property of the filesystem rather than of anything you care about — it differs between operating systems, so a script that compares it against zero will behave differently on a Mac and on a compute node.

5. Missing Is Not Empty

The second half of this chapter's purpose, and the distinction the whole of Chapter 15.10 is built on:

from pathlib import Path

Path("ran.log").write_text("Compile start\nSimulation complete\n")
Path("died.log").write_text("")


def state(name):
    p = Path(name)
    if not p.exists():
        return "NO_LOG      the tool never ran"
    if p.stat().st_size == 0:
        return "EMPTY_LOG   the tool started and died immediately"
    return f"HAS_CONTENT {p.stat().st_size} bytes to parse"


for name in ["ran.log", "died.log", "never.log"]:
    print(f"{name:<12} {state(name)}")

ran.log      HAS_CONTENT 34 bytes to parse
died.log     EMPTY_LOG   the tool started and died immediately
never.log    NO_LOG      the tool never ran

Three states, three different stories about what happened to the job:

• no file — the launcher failed, the queue rejected it, the path is wrong;
• empty file — the process started, created its log, and died before writing anything: a licence failure, a missing library, a bad argument;
• content — something ran, and now you have parsing to do.

A script that treats the first two the same reports "no errors found" for both, which is Chapter 14.9's green nightly arriving by a different road.

6. The Answer Is Already Out of Date

Every one of these checks describes the filesystem at the moment it ran. By the time you act on it, something else may have changed it:

import os
from pathlib import Path

p = Path("sim.log")
p.write_text("Compile start\n")

if p.is_file():
    # another process deletes it here -- simulated directly
    os.unlink("sim.log")

    with open(p) as f:
        print(f.read())

Traceback (most recent call last):
  File "race.py", line 11, in <module>
    with open(p) as f:
         ~~~~^^^
FileNotFoundError: [Errno 2] No such file or directory: 'sim.log'

The check passed and the open failed. On a compute farm this is not contrived: scratch cleaners delete old run directories, other jobs move files, and a regression that takes four hours is four hours of opportunity.

Ask when you want to branch. Catch when you want to act.

A check does not remove the need for a handler; it lets you make a decision before committing. If the only thing you would do on failure is report it, skip the check and catch the error — it is shorter and it cannot go stale:

from pathlib import Path

Path("sim.log").write_text("Compile start\n")

for name in ["sim.log", "missing.log"]:
    try:
        with open(name) as f:
            print(f"{name:<12} {len(f.read())} characters")
    except FileNotFoundError:
        print(f"{name:<12} not produced by this run")

sim.log      14 characters
missing.log  not produced by this run

7. Naming the Parts of a Path

Two more that are useful in messages, and are the last of pathlib this module needs:

from pathlib import Path

p = Path("runs") / "seed_41" / "sim.log"

print("full  :", p)
print("name  :", p.name)
print("parent:", p.parent)
print("suffix:", p.suffix)
print("stem  :", p.stem)

full  : runs/seed_41/sim.log
name  : sim.log
parent: runs/seed_41
suffix: .log
stem  : sim

The / operator joins path components, and it is the reason pathlib exists: it produces the right separator on any operating system, where an f-string with a hard-coded / does not. Module 18 Chapter 18.2 is about why paths are not strings; for now, note that p.name in an error message is much more readable than the whole path.

8. A Broken Link Does Not Exist

Run directories on a compute farm are routinely symbolic links — latest pointing at last night's run, a scratch path pointing somewhere with space. A link whose target has been deleted is a case worth knowing, because exists() answers about the target, not the link:

import os
from pathlib import Path

Path("real.log").write_text("x\n")
os.symlink("real.log", "good_link.log")
os.symlink("gone.log", "broken_link.log")

for name in ["good_link.log", "broken_link.log"]:
    p = Path(name)
    print(f"{name:<18} exists={str(p.exists()):<5} "
          f"is_file={str(p.is_file()):<5} is_symlink={p.is_symlink()}")

good_link.log      exists=True  is_file=True  is_symlink=True
broken_link.log    exists=False is_file=False is_symlink=True

broken_link.log reports exists=False while plainly being there. The name is in the directory — is_symlink() proves it — but the thing it points at is gone, and exists() follows the link before answering.

For a reader that is the right answer: there is nothing to read, so treat it as missing. For a cleanup script it is dangerously wrong, because a loop that removes "paths that exist" will skip every broken link and leave the directory full of them.

exists() answers about the target. is_symlink() answers about the name.

The combination that is almost always a problem worth reporting is is_symlink() and not exists() — a dangling pointer, usually meaning the run directory it referenced has been cleaned up while something still refers to it.

9. Predict, Run, Observe

import os
from pathlib import Path

Path("a.log").write_text("x\n")
Path("b.log").write_text("")
os.mkdir("c")

for name in ["a.log", "b.log", "c", "d.log"]:
    p = Path(name)
    size = p.stat().st_size if p.is_file() else None
    print(f"{name:<6} exists={str(p.exists()):<5} is_file={str(p.is_file()):<5} size={size}")

print("---")
print("A:", Path("b.log").exists())
print("B:", bool(Path("b.log").read_text()))
print("C:", Path("d.log").is_file())

a.log  exists=True  is_file=True  size=2
b.log  exists=True  is_file=True  size=0
c      exists=True  is_file=False size=None
d.log  exists=False is_file=False size=None
---
A: True
B: False
C: False

A and B together are the point. b.log exists, and its contents are falsy. exists() told you nothing about whether there is anything to parse — only st_size, or reading it, does. A guard written as if p.exists(): will happily hand an empty file to a parser that then finds no errors in it.

c shows size=None because the is_file() guard skipped stat(). That is Section 4's rule doing its job: the directory never had its size taken, so nothing machine-dependent reached the output.

d.log is False for every question and raised nothing — which is why these three are safe to call on a path you know nothing about.

a.log is 2 bytes for one character and one newline, the separator Chapter 15.3 made visible.

10. Exercises

Exercise 1 — the four questions. Create a file, a directory and a missing name. Tabulate exists, is_file, is_dir for all three.

Exercise 2 — the wrong guard. Write if p.exists(): open(p) against a directory and record the error. Fix it with one word.

Exercise 3 — stat raises. Call .stat() on a missing path. Then write the gated version that cannot raise.

Exercise 4 — three states. Implement Section 5's state() and test it against a real log, an empty file and a missing name.

Exercise 5 — why they differ. Write two sentences a flow engineer would accept explaining why NO_LOG and EMPTY_LOG get different treatment.

Exercise 6 — the race. Reproduce Section 6. Then rewrite it so the failure is handled rather than prevented, and say which version you would ship.

Exercise 7 — branch or catch. For each, choose: (a) a log that is optional (b) a filelist the compile requires (c) a run directory you will create if absent (d) a config whose absence means "use defaults".

Exercise 8 — the parts. Build a path three levels deep with / and print every component. Say which one you would put in an error message and why.

11. Summary

• Ask about a path when absence is information, not an error.
• Path represents a path as an object; Module 18 covers pathlib properly.
• exists() is True for a directory, so it is not the check you want before opening.
• Use is_file() before reading and is_dir() before treating a name as a directory.
• exists, is_file and is_dir never raise; stat() does, so gate it.
• A directory's st_size is machine-dependent — never compare it against zero.
• Missing and empty are different verdicts: one means the tool never ran, the other that it died at startup.
• Every answer is out of date immediately — the file can change between the check and the open.
• Ask when you want to branch; catch when you want to act.
• exists() follows symlinks, so a broken link reports False while the name is still in the directory; is_symlink() answers about the name.
• / joins path components portably, and .name makes a readable message.

The next chapter stops asking and starts changing things — creating, renaming and deleting, which is where an unattended script can do real damage.