read: Taking the Whole File at Once

Part I · Module 12 · Chapter 12.3 · read: Taking the Whole File at Once

1. The whole file as one string

set fp [open data.txt r]
set content [read $fp]
close $fp

puts "[string length $content] characters"
puts "---"
puts -nonewline $content
puts "---"

17 characters
---
alpha
beta
gamma
---

read with just a channel consumes everything remaining and returns it as one string — newlines and all. Note puts -nonewline: the content already ends with a newline, so adding another would leave a blank line.

2. Turning it into lines, and the extra element

The obvious next step has a surprise in it:

set fp [open data.txt r]
set lines [split [read $fp] "\n"]
close $fp

puts "llength says : [llength $lines]"
puts "the elements : $lines"
puts "the last one : '[lindex $lines end]'"

llength says : 4
the elements : alpha beta gamma {}
the last one : ''

Four elements for a three-line file. The last is empty.

This is not a bug and not special to files — it is Chapter 5.13's split doing exactly what it always does. The file's text is alpha\nbeta\ngamma\n, with a newline after the last line, so splitting on \n produces a fourth, empty piece after that final separator:

puts [llength [split "a\nb\n" "\n"]]
puts [llength [split "a\nb"   "\n"]]

3
2

A trailing separator always produces a trailing empty element. Text files conventionally end with a newline, so this happens to almost every file you will read.

3. The fix: read -nonewline

set fp [open data.txt r]
set lines [split [read -nonewline $fp] "\n"]
close $fp

puts "llength says : [llength $lines]"
puts "the last one : '[lindex $lines end]'"

llength says : 3
the last one : 'gamma'

-nonewline discards exactly one trailing newline if there is one, which is precisely the right amount of correction. This is the form to use:

set fp [open $path r]
set lines [split [read -nonewline $fp] "\n"]
close $fp

4. The empty file, and the one it cannot be told apart from

Worth knowing because it is the case your test data will not have:

set fp [open empty.txt w]
close $fp

set fp [open empty.txt r]
set content [read -nonewline $fp]
close $fp

puts "content length : [string length $content]"
puts "split gives    : [llength [split $content "\n"]] element(s)"

content length : 0
split gives    : 0 element(s)

Zero elements, which is what you want — split on the empty string gives an empty list, so a foreach over it simply does not run.

Now the part to be careful about. A file containing a single blank line is indistinguishable from an empty one once -nonewline has done its work:

set fp [open oneblank.txt w] ; puts $fp "" ; close $fp
set fp [open reallyempty.txt w] ;            close $fp

puts "one blank line : [file size oneblank.txt] byte(s) on disk"
puts "truly empty    : [file size reallyempty.txt] byte(s) on disk"

foreach path {oneblank.txt reallyempty.txt} {
    set fp [open $path r]
    set c [read -nonewline $fp]
    close $fp
    puts "  $path -> [llength [split $c "\n"]] element(s), content eq '' : [expr {$c eq ""}]"
}

one blank line : 1 byte(s) on disk
truly empty    : 0 byte(s) on disk
  oneblank.txt -> 0 element(s), content eq '' : 1
  reallyempty.txt -> 0 element(s), content eq '' : 1

The files differ on disk — one byte against zero — and after read -nonewline both are the empty string. That is -nonewline doing exactly its job: it removed the only newline there was.

Two or more blank lines are a different matter, because only the last newline is stripped:

set fp [open twoblank.txt w] ; puts $fp "" ; puts $fp "" ; close $fp
set fp [open twoblank.txt r] ; set c [read -nonewline $fp] ; close $fp
puts "two blank lines -> [llength [split $c "\n"]] element(s)"

two blank lines -> 2 element(s)

5. When to use this and when not to

read holds the entire file in memory. For a configuration file or a short report that is irrelevant; for a log it may not be.

set fp [open big.txt w]
for {set i 1} {$i <= 50000} {incr i} { puts $fp "line $i" }
close $fp

puts "the file is [file size big.txt] bytes"

set fp [open big.txt r]
set all [read $fp]
close $fp
puts "read into one string of [string length $all] characters"

the file is 538894 bytes
read into one string of 538894 characters

Half a megabyte in one variable, and then the split makes a 50,000-element list beside it. For this size that is fine on any modern machine. The point is that the cost scales with the file, and Chapter 12.2's loop does not.

A rule of thumb that holds up:

6. Reading a fixed number of characters

read takes an optional count:

set fp [open data.txt r]
set first5 [read $fp 5]
set rest   [read $fp]
close $fp

puts "first 5 : '$first5'"
puts "rest    : [string length $rest] characters"

first 5 : 'alpha'
rest    : 12 characters

The two calls continued from where the previous one stopped — a channel has a position, and every read advances it. That is also why reading the same channel twice without rewinding gives you nothing the second time.

7. A worked example

Reading a settings file whole, because it is small and we want to report on it as a unit:

set fp [open config.txt w]
foreach l {"# run settings" "width = 32" "" "depth = 16" "mode = fast"} {
    puts $fp $l
}
close $fp

proc load_settings {path} {
    set fp [open $path r]
    set code [catch {read -nonewline $fp} content]
    close $fp
    if {$code} { return -code error $content }

    set settings [dict create]
    set skipped 0

    foreach line [split $content "\n"] {
        set t [string trim $line]
        if {$t eq "" || [string match "#*" $t]} { incr skipped ; continue }
        set i [string first "=" $t]
        if {$i < 0} { incr skipped ; continue }
        dict set settings \
            [string trim [string range $t 0 [expr {$i - 1}]]] \
            [string trim [string range $t [expr {$i + 1}] end]]
    }

    return [list $settings $skipped]
}

lassign [load_settings config.txt] settings skipped
puts "parsed  : $settings"
puts "skipped : $skipped line(s)"
puts "width   : [dict get $settings width]"

parsed  : width 32 depth 16 mode fast
skipped : 2 line(s)
width   : 32

read -nonewline means the foreach sees three real settings and two skippable lines — five, which is the number of lines in the file. Without the flag it would have been six.

8. Exercises

Exercise 1. What does read $fp return, and does it include the final newline?

Exercise 2. Why does split [read $fp] "\n" give four elements for a three-line file?

Exercise 3. What is the fix, and why is it safe even when the file does not end with a newline?

Exercise 4. What does llength [split [read -nonewline $fp] "\n"] give for a completely empty file, and which other file gives the same answer?

Exercise 5. Write a procedure returning the last line of a file, read whole.

Exercise 6. Give two situations where the gets loop is the better choice.

Answers

1. The entire remaining content of the channel as a single string, including the final newline if the file has one.

2. Because the file's text ends with a newline, and split produces an empty element after a trailing separator. The file is alpha\nbeta\ngamma\n, so splitting on \n yields the three lines plus an empty fourth piece after the last separator.

3. read -nonewline $fp, which discards exactly one trailing newline. It is safe either way: if the file ends with a newline it removes the phantom element, and if it does not there is nothing to remove.

4. 0 — splitting the empty string gives an empty list, so a foreach over it does not run. A file containing a single blank line gives the same answer, because -nonewline strips its only newline and leaves the empty string. The two differ on disk (one byte against zero) but not after reading this way.

5.

proc last_line {path} {
    set fp [open $path r]
    set code [catch {read -nonewline $fp} content]
    close $fp
    if {$code} { return -code error $content }
    if {$content eq ""} { return "" }
    return [lindex [split $content "\n"] end]
}
puts [last_line data.txt]

gamma

6. When the file's size is not something you control, and when you do not know how large it might become. Both are the same concern: read holds the whole file in memory, so its cost scales with the input, while the loop's does not.

9. Remember

• read $fp returns the whole remaining content as one string, final newline included.
• split [read $fp] "\n" gives one element too many — an empty one, because the file ends with a separator.
• read -nonewline $fp is the fix, and is safe whether or not the file ends with a newline. Treat it as one token.
• The phantom element is silent — a loop runs one extra time, a count is one too high, lindex ... end returns empty.
• An empty file splits into an empty list (0 elements), so a foreach over it simply does not run — but a file holding one blank line reads identically. Use file size if that distinction matters.
• read $fp N reads N characters and advances the channel's position.
• Use whole-file reading for small, bounded inputs; use the gets loop whenever the size is not yours to control.