Opening and Reading a File Safely

Part I · Module 15 · Chapter 15.2 · Opening and Reading a File Safely

1. The Whole Thing

Compile start
UVM_INFO top.sv(42) smoke begin
UVM_ERROR drv.sv(88) bad handshake
Simulation complete

with open("sim.log") as f:
    text = f.read()

print(type(text).__name__, len(text))
print(text)

str 101
Compile start
UVM_INFO top.sv(42) smoke begin
UVM_ERROR drv.sv(88) bad handshake
Simulation complete

Three things happened, and it is worth naming each:

1. open("sim.log") located the file and returned a file object.
2. f.read() pulled its entire contents back as one string.
3. with closed the file when the block ended.

Note str 101: read() returns a single string of 101 characters, not a list of lines. Turning it into lines is a separate step, and Chapter 15.4 gives you three ways to do it.

2. The File Object Is Not the Contents

A common early confusion:

with open("sim.log") as f:
    print(len(f))

Compile start
Simulation complete

Traceback (most recent call last):
  File "confuse.py", line 2, in <module>
    print(len(f))
          ~~~^^^
TypeError: object of type '_io.TextIOWrapper' has no len()

f is a handle — a thing you can ask for data — not the data itself. Its type name, TextIOWrapper, says what it is: a wrapper that turns bytes on a disk into text on demand. Chapter 15.3 explains the wrapping.

You get data by calling a method on the handle. f.read() is the one that returns everything.

3. with Is the Form You Always Write

Every example in this module opens a file like this:

with open("sim.log") as f:
    text = f.read()

with guarantees the file is closed when the block ends — whether the block finished normally or raised. Chapter 15.5 is entirely about what that guarantee is and why it matters, so this chapter will simply use it.

For now, one rule, no exceptions:

Never call open() outside a with statement.

The alternative exists and Chapter 15.5 shows it, along with the specific damage it does on a shared compute machine.

4. The Path Is Just Text

open takes a path, and a path is an ordinary string that the operating system knows how to interpret:

name = "sim"
path = f"{name}.log"
print("opening:", path)

with open(path) as f:
    first_20 = f.read()[:20]

print(repr(first_20))

Compile start
UVM_INFO top.sv(42) smoke begin

opening: sim.log
'Compile start\nUVM_IN'

Building the path with an f-string works because it is just a string. Module 18 introduces pathlib, which treats paths as objects and is better for anything non-trivial; for a single filename, text is fine.

Note the repr output again: the newline between the two lines is a character inside the string, sitting between start and UVM. It is not a list boundary. That is Chapter 15.3's subject.

5. Reading a Real Log

Putting Chapter 15.1's verdict function onto an actual file — the substitution that chapter promised:

Chronologic VCS simulator copyright 1991-2024
UVM_INFO @ 0: reporter [RNTST] Running test smoke_test...
UVM_ERROR drv.sv(88) @ 1420: uvm_test_top.env.agt.drv [DRV] bad handshake
UVM_INFO @ 9180: reporter [FINISH] *** TEST FAILED ***

def verdict(lines):
    for line in lines:
        if "TEST FAILED" in line:
            return "FAIL"
        if "TEST PASSED" in line:
            return "PASS"
    return "NO_VERDICT"


with open("run.log") as f:
    text = f.read()

lines = text.splitlines()
print("lines:", len(lines))
print("verdict:", verdict(lines))

lines: 4
verdict: FAIL

The function is character-for-character the one from Chapter 15.1. Only its input changed, from a literal list to a file. That is the entire point of the module, and everything after this is refinement.

Note splitlines() rather than split("\n"), for the reason Chapter 15.1 Section 8 measured.

6. Where Does It Look?

This is the detail that produces "it works on my machine".

open("sim.log") does not look next to your script. It looks in the working directory — the directory your shell was in when the script started:

with open("sim.log") as f:
    print("found it:", len(f.read()), "characters")

Compile start

found it: 14 characters

That run worked because the harness ran the script from the directory holding both files. Run the identical script from one level up and it fails, because sim.log is resolved from wherever you are:

with open("sim.log") as f:
    print(len(f.read()))

Traceback (most recent call last):
  File "missing.py", line 1, in <module>
    with open("sim.log") as f:
         ~~~~^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'sim.log'

This is the third time this fact has appeared: Chapter 13.5 Section 6 found it for imports, Chapter 14.3 Section 7 named it as the usual cause of FileNotFoundError, and here it is with a file in hand.

7. The Two Failures Worth Recognising

Compile start

import os

os.mkdir("runs")

for path in ["sim.log", "missing.log", "runs"]:
    try:
        with open(path) as f:
            print(f"{path:<12} read {len(f.read())} characters")
    except FileNotFoundError:
        print(f"{path:<12} no such file")
    except IsADirectoryError:
        print(f"{path:<12} that is a directory, not a file")

sim.log      read 14 characters
missing.log  no such file
runs         that is a directory, not a file

Both are OSError subclasses, and both are routinely mishandled as "the file is bad". They are different diagnoses:

• FileNotFoundError — the tool never produced it, or you are in the wrong directory.
• IsADirectoryError — the name exists but is the wrong kind of thing, usually a path built with a missing final component.

Chapter 15.8 asks these questions without opening anything, which is often what you want. (os.mkdir appears here only to create something to fail on; Module 18 covers directory creation properly.)

8. An Empty File Reads Successfully

A file with no content is not an error, and the difference between "no file" and "no content" is one your parser has to make:

with open("empty.log") as f:
    text = f.read()

print("read ok    :", repr(text))
print("length     :", len(text))
print("falsy      :", not text)
print("splitlines :", text.splitlines())

read ok    : ''
length     : 0
falsy      : True
splitlines : []

Nothing raised. read() returned an empty string, splitlines() returned an empty list, and a for loop over the file would run zero times.

That is the correct behaviour and it is also a trap, because a parser written as "count the errors and report PASS if there are none" returns PASS for a file with nothing in it at all:

def verdict(path):
    errors = 0
    with open(path) as f:
        for line in f:
            if "UVM_ERROR" in line:
                errors += 1
    return "FAIL" if errors else "PASS"


print("empty log verdict:", verdict("empty.log"))

empty log verdict: PASS

A simulator that died at startup — licence refused, library missing — leaves exactly this file. The three states a real script must separate are missing, empty and has content, and Chapters 15.8 and 15.10 are about making that distinction properly. For now, note that open() succeeding tells you the file exists, and nothing more.

9. Predict, Run, Observe

alpha
beta
gamma

with open("four.log") as f:
    text = f.read()

print("A:", repr(text))
print("B:", len(text))
print("C:", len(text.splitlines()))
print("D:", text[0], text[5])

with open("four.log") as f:
    a = f.read()
    b = f.read()
print("E:", len(a), len(b))

A: 'alpha\nbeta\ngamma\n'
B: 17
C: 3
D: a 

E: 17 0

A — one string, with the separators visible. Three lines of 5, 4 and 5 characters plus three newlines is 17.

D prints a and then a newline character, because text[5] is the separator — which is why the output has a stray blank line in it. A file's newlines are ordinary characters at ordinary indexes.

E is the one that surprises people. The second f.read() returned an empty string. A file object remembers its position; the first read consumed everything, so there was nothing left. Reading twice does not give you the file twice.

That position is also why Chapter 15.7's line-by-line iteration works at all.

10. Exercises

Exercise 1 — read one. Create a three-line file and read it with f.read(). Print its type, its length, and its repr. Account for every character in the length.

Exercise 2 — handle versus data. Call len() on the file object and record the error. Say in one sentence what f actually is.

Exercise 3 — substitute the source. Take any function you wrote in Module 14 that processed a list of strings and feed it a real file via splitlines(). Confirm you changed nothing inside the function.

Exercise 4 — the working directory. Put a script and its data file in a folder. Run the script from inside the folder, then from its parent. Record both outcomes and explain the difference.

Exercise 5 — two failures. Produce FileNotFoundError and IsADirectoryError deliberately. Write the one-sentence diagnosis you would put in a handler for each.

Exercise 6 — the exhausted handle. Read a file twice inside one with block. Explain the second result, then find a way to get the contents twice.

Exercise 7 — build a path. Use an f-string to build a filename from a test name and a seed, and open it. Say what Module 18 would do differently.

Exercise 8 — count the separators. For a file of n lines that ends with a newline, write a formula for len(text) in terms of the line lengths. Verify it on a file you create.

11. Summary

• open(path) returns a file object, a handle — not the file's contents.
• f.read() returns the whole file as one string, not a list of lines.
• Always open inside with; Chapter 15.5 explains what it guarantees.
• A path is ordinary text, and can be built with an f-string.
• A relative path is resolved from the working directory, not from the script's own directory — the same rule imports follow.
• FileNotFoundError and IsADirectoryError are different diagnoses: one means it is not there, the other means it is the wrong kind of thing.
• A file object remembers its position, so a second read() returns an empty string.
• Newlines are characters inside the string, at ordinary indexes.
• An empty file reads successfully, returning '' — which is a different state from a missing file, and one a naive parser reports as a pass.
• Use splitlines() to turn file text into lines.

The next chapter looks at what open did to the bytes on the way in — the decoding step, and the newline you have been seeing in every repr.