read, readline, readlines — and Which to Use

Part I · Module 15 · Chapter 15.4 · read, readline, readlines — and Which to Use

1. The Three

text  = f.read()        # whole file, as one string
line  = f.readline()    # one line, as a string
lines = f.readlines()   # list of every line

All three read from the same file object and all three advance the same position. The difference is the quantity each returns.

Compile start
UVM_INFO top.sv(42) smoke begin
UVM_ERROR drv.sv(88) bad handshake
Simulation complete

with open("sim.log") as f:
    text = f.read()

with open("sim.log") as f:
    line = f.readline()

with open("sim.log") as f:
    lines = f.readlines()

print("read()      ->", type(text).__name__, len(text), "characters")
print("readline()  ->", type(line).__name__, repr(line))
print("readlines() ->", type(lines).__name__, len(lines), "items")
print("            ->", repr(lines[-1]))

read()      -> str 101 characters
readline()  -> str 'Compile start\n'
readlines() -> list 4 items
            -> 'Simulation complete\n'

Note each file is opened fresh. That is deliberate, and Section 3 explains why it has to be.

2. What Each One Is For

Every line from readline() and readlines() carries its newline, as Chapter 15.3 established. read() gives you the separators too, inside the one string.

3. One Cursor, Shared

A file object remembers where it is, and all three methods move the same marker:

with open("sim.log") as f:
    first = f.readline()
    rest = f.read()

print("first:", repr(first))
print("rest begins:", repr(rest[:24]))
print("rest lines:", len(rest.splitlines()))

Compile start
UVM_INFO top.sv(42) smoke begin
UVM_ERROR drv.sv(88) bad handshake
Simulation complete

first: 'Compile start\n'
rest begins: 'UVM_INFO top.sv(42) smok'
rest lines: 3

read() returned everything after the line readline() already consumed. That is why Section 1 reopened the file each time — a second read() on an exhausted handle returns '', as Chapter 15.2 Section 8 showed.

Mixing them deliberately is a real technique: take a header line, then process the remainder:

test,status,runtime_s
smoke,PASS,182.4
cdc,FAIL,204.1

with open("report.csv") as f:
    header = f.readline().strip().split(",")
    body = f.readlines()

print("columns:", header)
print("rows:", len(body))
for row in body:
    print("  ", row.strip().split(","))

columns: ['test', 'status', 'runtime_s']
rows: 2
   ['smoke', 'PASS', '182.4']
   ['cdc', 'FAIL', '204.1']

(Module 21 does CSV properly, with a module that handles quoting. This is the mechanism underneath.)

4. How readline() Says "No More"

At the end of the file, readline() returns an empty string — not None, not an error:

with open("two.log", "w") as f:
    f.write("alpha\n\nbeta\n")

with open("two.log") as f:
    while True:
        line = f.readline()
        if line == "":
            break
        print("got:", repr(line))
print("done")

got: 'alpha\n'
got: '\n'
got: 'beta\n'
done

The file has a blank line in the middle, and this loop handled it correctly. That is the whole reason the test is written if line == "" and not if not line... except that those are equivalent here, and the real trap is the opposite one:

with open("two.log", "w") as f:
    f.write("alpha\n\nbeta\n")

with open("two.log") as f:
    kept = []
    for raw in f:
        line = raw.strip()
        if not line:
            break
        kept.append(line)

print("kept:", kept)

kept: ['alpha']

One line kept out of three. After .strip(), the blank line became "", which is falsy — so if not line: break stopped at it. The author meant "stop at end of file" and wrote "stop at the first empty line".

A blank line is "\n", which is truthy. End of file is "", which is falsy. They become the same thing the moment you .strip().

This is Chapter 9.5's truthiness rule meeting file data, and it is a common way to silently process a fraction of a log.

5. readlines() Is Not the Cheap One

The widespread belief is that read() is the memory-hungry one and readlines() is the line-based, careful alternative. Both statements are wrong:

import sys

with open("big.log", "w") as f:
    for i in range(50000):
        f.write(f"UVM_INFO seq.sv({i}) transaction {i} complete\n")

with open("big.log") as f:
    whole = f.read()

with open("big.log") as f:
    lines = f.readlines()

print("read()      string bytes:", sys.getsizeof(whole))
print("readlines() list bytes  :", sys.getsizeof(lines), "(plus every line)")
print("lines held               :", len(lines))

read()      string bytes: 2477821
readlines() list bytes  : 444376 (plus every line)
lines held               : 50000

The list object looks smaller, and that number is misleading: 444376 is the cost of 50,000 pointers. Every one of those pointers refers to a separate string that is also in memory. readlines() holds the entire file content plus a list to index it — strictly more than read(), not less.

Both are O(file size). Neither is suitable for a log you cannot afford to hold.

6. The Number That Decides It

So what is cheap? Not one of the three:

import sys

with open("big.log", "w") as f:
    for i in range(50000):
        f.write(f"UVM_INFO seq.sv({i}) transaction {i} complete\n")

with open("big.log") as f:
    whole = f.read()

longest = 0
with open("big.log") as f:
    for line in f:
        if sys.getsizeof(line) > longest:
            longest = sys.getsizeof(line)

print("whole file held at once :", sys.getsizeof(whole), "bytes")
print("largest single line     :", longest, "bytes")
print("ratio                   :", sys.getsizeof(whole) // longest)

whole file held at once : 2477821 bytes
largest single line     : 91 bytes
ratio                   : 27228

27,228 times less memory, for a log of only 50,000 lines — about 2.4 MB, a rounding error by simulation standards. Scale the file to 20 GB and the whole file figure scales with it while the largest line does not move.

Iterating the file directly — for line in f — holds one line at a time. It is not read, readline or readlines; it is the fourth option, it is the right default, and Chapter 15.7 is entirely about it.

7. The Decision Rule

The last row is the one that matters. "Small" means a file whose size you control or know. A simulation log is neither — it is whatever the run produced, and the pathological case is exactly the failing run you most need to parse.

8. Predict, Run, Observe

alpha
beta
gamma

with open("p.log") as f:
    a = f.readline()
    b = f.readlines()
    c = f.read()
    d = f.readline()

print("A:", repr(a))
print("B:", b)
print("C:", repr(c))
print("D:", repr(d))
print("E:", len(b))

A: 'alpha\n'
B: ['beta\n', 'gamma\n']
C: ''
D: ''
E: 2

A took one line and moved the cursor past it.

B took everything remaining — two lines, not three. readlines() is not "all lines", it is "all lines from here".

C and D are both empty. The handle is exhausted; read() returns '' and so does readline(). Neither raises, which is why an exhausted-handle bug is usually discovered as missing data rather than as an error.

E confirms it: the list has two entries, and the first line of the file is not among them. Every one of these methods is relative to the cursor.

9. Exercises

Exercise 1 — the three. Read one file three ways and print the type, length and first element of each result.

Exercise 2 — the shared cursor. Call readline() then read() on one handle. Explain the second result without re-reading Section 3.

Exercise 3 — header and body. Write a file with a header line and three data rows. Parse the header with readline() and the rest with readlines().

Exercise 4 — the blank-line trap. Reproduce Section 4's trap.py and confirm it keeps one line. Fix it so all three are kept, and say which chapter's rule you applied.

Exercise 5 — EOF versus blank. Print the repr of a blank line and of what readline() returns at end of file. State the difference in one sentence.

Exercise 6 — the cost. Run Section 5's measurement on a file of 200,000 lines. Record both figures and say which one changed.

Exercise 7 — the ratio. Explain in two sentences why the "largest single line" figure barely moves as the file grows.

Exercise 8 — pick one. For each, choose a method and justify it: (a) a 10-line YAML config (b) a 14 GB UVM log (c) the first line of a CSV (d) a filelist you will iterate twice (e) a log of unknown size from a failed run.

10. Summary

• read() returns the whole file as one string; readline() returns one line; readlines() returns a list of every line.
• All three share one cursor and return content from the current position, not from the start.
• A second read on an exhausted handle returns '' and does not raise.
• readline() signals end of file with '', which is falsy — while a blank line is '\n', which is truthy.
• .strip() collapses that distinction, which is how a loop silently stops at the first blank line.
• readlines() is not the memory-efficient option — it holds the whole content plus a list of pointers into it.
• Iterating the file is the fourth option and the right default: one line at a time, measured here at 27,228× less memory than holding the file.
• "Small" means a file whose size you control. A simulation log is not one.
• Make iteration the reflex; reach for read() deliberately.

The next chapter explains the keyword every example has used and none has justified — and what it guarantees when the code inside it fails.