Reading a File Line by Line

Part I · Module 12 · Chapter 12.2 · Reading a File Line by Line

1. The loop

set fp [open data.txt r]

while {[gets $fp line] >= 0} {
    puts "read: $line"
}

close $fp

Given a file containing three lines:

read: alpha
read: beta
read: gamma

Three parts: open for reading, a while whose condition both reads a line and decides whether to continue, and close. That is the whole thing.

2. What gets actually returns

The condition looks odd until you know what gets gives back. In its two-argument form it stores the line in the variable and returns the line's length:

set fp [open data.txt r]

puts "call 1 returned: [gets $fp line]  (line = '$line')"
puts "call 2 returned: [gets $fp line]  (line = '$line')"
puts "call 3 returned: [gets $fp line]  (line = '$line')"
puts "call 4 returned: [gets $fp line]  (line = '$line')"

close $fp

call 1 returned: 5  (line = 'alpha')
call 2 returned: 4  (line = 'beta')
call 3 returned: 5  (line = 'gamma')
call 4 returned: -1  (line = '')

Five characters, four, five — then -1 once there is nothing left, with the variable set to the empty string. The loop condition is simply "did I get a line?"

3. Why the condition is >= 0 and not something else

Because a blank line has length 0, and 0 is a real result:

set fp [open withblank.txt w]
puts $fp "first"
puts $fp ""
puts $fp "third"
close $fp

set fp [open withblank.txt r]
set n 0
while {[gets $fp line] >= 0} {
    incr n
    puts "line $n: '$line' (length [string length $line])"
}
close $fp

line 1: 'first' (length 5)
line 2: '' (length 0)
line 3: 'third' (length 5)

Three lines, as there should be. Now watch what a truthiness-style condition does:

set fp [open withblank.txt r]
set n 0
while {[gets $fp line]} {
    incr n
}
close $fp
puts "the wrong condition read $n lines"

the wrong condition read 1 lines

It stopped at the blank line, because 0 is false. The file has three lines and it found one.

4. It works on a file of any size

The loop holds exactly one line in memory at a time. That is what makes it the form to memorise — the same five lines read a two-line file and a two-gigabyte one:

set fp [open big.txt w]
for {set i 1} {$i <= 50000} {incr i} {
    puts $fp "line $i"
}
close $fp

set fp [open big.txt r]
set count 0
set chars 0
while {[gets $fp line] >= 0} {
    incr count
    incr chars [string length $line]
}
close $fp

puts "$count lines, $chars characters, [file size big.txt] bytes on disk"

50000 lines, 488894 characters, 538894 bytes on disk

Memory use did not grow with the file. Chapter 12.3 covers the alternative that does, and when it is the better choice anyway.

5. Counting, filtering, accumulating

Everything from Chapter 9.9 applies — the loop body is an ordinary loop body:

set fp [open log.txt w]
foreach l {"INFO started" "WARN slow" "ERROR failed" "INFO done" "WARN again"} {
    puts $fp $l
}
close $fp

set total 0
set problems {}
set bylevel [dict create]

set fp [open log.txt r]
while {[gets $fp line] >= 0} {
    incr total
    set level [lindex $line 0]
    dict incr bylevel $level
    if {$level in {WARN ERROR}} {
        lappend problems $line
    }
}
close $fp

puts "lines    : $total"
puts "by level : $bylevel"
puts "problems : [llength $problems]"

lines    : 5
by level : INFO 2 WARN 2 ERROR 1
problems : 3

6. The one-argument form, and why to avoid it

gets with only a channel returns the line itself rather than its length:

set fp [open data.txt r]
puts "first line: '[gets $fp]'"
close $fp

first line: 'alpha'

Convenient, and it cannot be used as a loop condition — because at end of file it returns the empty string, which is exactly what a blank line returns too:

set fp [open withblank.txt r]
puts "line 1 : '[gets $fp]'"
puts "line 2 : '[gets $fp]'   <- a real blank line"
puts "line 3 : '[gets $fp]'"
puts "line 4 : '[gets $fp]'   <- end of file"
close $fp

line 1 : 'first'
line 2 : ''   <- a real blank line
line 3 : 'third'
line 4 : ''   <- end of file

Lines 2 and 4 are indistinguishable. The two-argument form exists precisely so the return value can carry that information, which is why it is the one to use.

7. A worked example

A line-numbered listing that skips blanks and comments, closing safely:

set fp [open config.txt w]
foreach l {"# settings" "" "width = 32" "  " "depth = 16" "# end"} {
    puts $fp $l
}
close $fp

proc show_settings {path} {
    set fp [open $path r]
    set code [catch {
        set n 0
        set shown 0
        while {[gets $fp line] >= 0} {
            incr n
            set t [string trim $line]
            if {$t eq ""}               continue
            if {[string match "#*" $t]} continue
            incr shown
            puts [format "  %2d: %s" $n $t]
        }
        set _ "$shown of $n lines were settings"
    } result]
    close $fp
    if {$code} { return -code error $result }
    return $result
}

puts [show_settings config.txt]

   3: width = 32
   5: depth = 16
2 of 6 lines were settings

The line numbers come from counting every line while only printing some — which is why n is incremented before the guards and shown after them. The guards are Chapter 9.6 §3's pattern, and the catch around the body is Chapter 12.1 §6's.

8. Exercises

Exercise 1. Write the canonical reading loop from memory.

Exercise 2. What does gets $fp line return, and what does it return at end of file?

Exercise 3. Why is the condition >= 0 rather than a bare [gets $fp line]?

Exercise 4. What does this print for a file whose second line is blank?

while {[gets $fp line]} { incr n }

Exercise 5. Write a procedure returning the number of lines in a file.

Exercise 6. When is the one-argument gets $fp acceptable?

Answers

1.

set fp [open data.txt r]
while {[gets $fp line] >= 0} {
    # work with $line
}
close $fp

2. It stores the line in line and returns the line's length, with the terminating newline stripped. At end of file it returns -1 and sets the variable to the empty string.

3. Because a blank line has length 0, and 0 is false. A bare truth test stops at the first empty line and silently reads a fraction of the file.

4. 1. The loop counts the first line, then reaches the blank one, gets 0 back, treats it as false and stops.

5.

proc count_lines {path} {
    set fp [open $path r]
    set n 0
    while {[gets $fp line] >= 0} { incr n }
    close $fp
    return $n
}
puts [count_lines data.txt]

3

6. When you want exactly one line and already know it is there — a single-line file, or stripping a header before the main loop. It cannot be used as a loop condition because end of file and a blank line both come back as the empty string.

9. Remember

• The canonical loop is open / while {[gets $fp line] >= 0} / close. Memorise it once.
• gets $fp var stores the line and returns its length, with the newline stripped; -1 at end of file.
• The condition is >= 0 because a blank line returns 0, and a bare truth test would stop there.
• That bug silently truncates the file at its first empty line, with no error.
• The loop holds one line at a time, so it works on a file of any size.
• The body is an ordinary loop body — counting, filtering and accumulating all apply.
• The one-argument gets $fp returns the line itself and cannot distinguish a blank line from end of file. Use it only for a single known line.
• Wrap the body in catch so close always runs.