flush and close: Why Output Can Go Missing

Part I · Module 12 · Chapter 12.6 · flush and close: Why Output Can Go Missing

1. Watching the buffer

The whole chapter is in one sequence. Open a file, write to it, and ask the filesystem how big it is at each step:

set fp [open buffered.txt w]

puts $fp "first line"
puts $fp "second line"
puts "after two writes : [file size buffered.txt] bytes"

flush $fp
puts "after flush      : [file size buffered.txt] bytes"

puts $fp "third line"
puts "after one more   : [file size buffered.txt] bytes"

close $fp
puts "after close      : [file size buffered.txt] bytes"

after two writes : 0 bytes
after flush      : 23 bytes
after one more   : 23 bytes
after close      : 34 bytes

Four measurements, and each one says something:

• 0 bytes after two writes. The data is in memory. The file exists — open ... w created it — and it is empty.
• 23 bytes after flush. The buffer was sent on. Those two lines are now on disk.
• 23 bytes after a third write. flush is not a mode; it acts once. The third line is in the buffer now.
• 34 bytes after close. Closing flushes whatever is left.

2. close flushes — so normal endings are safe

A reasonable worry at this point is whether a script that forgets close loses its output. Usually not:

set fp [open ending.txt w]
puts $fp "written but never explicitly closed"
# no close, and the script ends here

Running that as a complete script and then checking the file:

36 bytes

Tcl flushes and closes open channels when the interpreter exits normally. So the common case — a script that writes, forgets close, and reaches its end — does not lose data.

That is worth knowing because it means buffering is not a reason to panic. It also means the failure, when it comes, is not the one people expect.

3. Where output really does go missing

The data is lost when the process ends abnormally, before the buffer is sent. A crash, a kill, a power loss, or the machine being rebooted out from under a long run.

Here is that happening, using a child interpreter that is killed mid-write:

set child {
    set fp [open killed.txt w]
    puts $fp "line one"
    puts $fp "line two"
    flush stdout
    after 5000          ;# wait, holding the buffer
    close $fp
}
set f [open script.tcl w] ; puts $f $child ; close $f

set pid [exec tclsh script.tcl &]
after 500
exec kill -9 $pid
after 200

puts "after the child was killed : [file size killed.txt] bytes"

after the child was killed : 0 bytes

Two lines written, nothing on disk. The process died with the buffer still in memory and the data went with it.

4. What flush is for

Three situations, all the same underlying reason — something other than your script needs to see the data now.

A log you might read while the script is still running:

proc note {fp text} {
    puts $fp $text
    flush $fp
}

set log [open progress.log w]
note $log "step 1 complete"
note $log "step 2 complete"
puts "while still open : [file size progress.log] bytes"
close $log

while still open : 32 bytes

Before something slow or risky, so the record of having started it survives:

set log [open risky.log w]
puts $log "about to do the dangerous thing"
flush $log
# ... the dangerous thing ...
puts $log "survived"
close $log

Before handing control to another program that will read the file.

5. The per-call open shape flushes for free

Chapter 12.5 §5's logger opens, writes and closes on every call:

proc log_line {path text} {
    set fp [open $path a]
    puts $fp $text
    close $fp
}

file delete -force per_call.log
log_line per_call.log "first"
puts "after one call : [file size per_call.log] bytes"
log_line per_call.log "second"
puts "after two      : [file size per_call.log] bytes"

after one call : 6 bytes
after two      : 13 bytes

Each line is on disk the moment the call returns, because each close flushes. That is the simplest way to get durable logging, and it is why the shape is so common despite the per-call cost.

6. stdout is buffered too

The same applies to your terminal output, which matters when output is redirected to a file or read by another program:

puts "this is on its way"
flush stdout
puts stderr "this goes to a different stream"

this is on its way
this goes to a different stream

On a terminal, stdout is usually line-buffered and you see each line immediately. Redirect it to a file and it typically becomes block-buffered — so a script whose output is piped into something else can appear to produce nothing for a long time, then everything at once.

flush stdout is the fix when interleaving matters — for example, printing progress before launching a subprocess whose output would otherwise appear first.

7. A worked example

A long-ish job that keeps a readable log without flushing every line:

file delete -force job.log

set log [open job.log w]
set flush_every 3

for {set step 1} {$step <= 7} {incr step} {
    puts $log "step $step complete"
    if {$step % $flush_every == 0} {
        flush $log
        puts "  (flushed at step $step: [file size job.log] bytes on disk)"
    }
}

puts "before close : [file size job.log] bytes"
close $log
puts "after close  : [file size job.log] bytes"

  (flushed at step 3: 48 bytes on disk)
  (flushed at step 6: 96 bytes on disk)
before close : 96 bytes
after close  : 112 bytes

The log is never more than two steps behind, and the script paid for three flushes rather than seven. The last step's line arrived at close.

8. Exercises

Exercise 1. After two puts $fp calls and no flush, how many bytes are on disk?

Exercise 2. Does flush make subsequent writes unbuffered?

Exercise 3. Is output lost if a script writes to a file and reaches its end without calling close?

Exercise 4. When is buffered output genuinely lost?

Exercise 5. Give three situations where flush is the right call.

Exercise 6. Why does the open-append-close-per-line logger not need flush?

Answers

1. Usually zero. The data is in the channel's buffer in memory; the file exists but is empty until the buffer is sent on.

2. No. flush acts once, on the data currently buffered. The next write is buffered again.

3. No. Tcl flushes and closes open channels when the interpreter exits normally, so the data reaches the file. Forgetting close is still bad practice — it leaks the channel while the script runs — but it does not lose output on a normal exit.

4. When the process ends abnormally: a crash, a kill, a power failure. The buffer is in the process's memory and dies with it.

5. A log you might read while the script is still running; immediately before a slow or risky operation, so the record of starting it survives; and before handing control to another program that will read the file.

6. Because close flushes, and it closes after every line. Each line is durable by the time the call returns — at the cost of an open and close per line.

9. Remember

• Writes are buffered. A file can sit at zero bytes while your script has correctly written to it.
• flush $fp sends the buffer on now, and acts once — it does not change the channel's mode.
• close flushes, and so does a normal interpreter exit, so forgetting close does not lose data on a clean ending.
• Output is lost when the process dies abnormally — a crash or a kill, with the buffer still in memory.
• That is why a running script's log can be empty, and why killing the job destroys the evidence.
• Flush when something external needs the data now: a readable log, before a risky step, before handing off to another program.
• Do not flush every line in a hot loop — flush every N lines instead.
• The open-append-close-per-line logger is durable for free, because every close flushes.
• stdout is buffered too, and becomes block-buffered when redirected.