Splitting a Script Into Reusable Files

Part I · Module 15 · Chapter 15.7 · Splitting a Script Into Reusable Files

Chapter 15.1 showed how to load another file. This chapter is about deciding what goes in it.

The useful split is not "one file per topic". It is between files that define things and files that do things. Get that line right and everything else — testing, reuse, handing the program to someone else — becomes easy. Get it wrong and the files are just a longer program in more places.

1. The two kinds of file

A library file contains proc definitions and namespace eval blocks that declare defaults. Nothing else. Sourcing it changes what commands exist and does no work.

A program file reads argv, calls the library, writes output and exits. There is exactly one of these, and nothing else sources it.

The test from 15.1 §7 still applies, and it is the whole rule: if I source this file and nothing happens, it is a library.

2. The layout

rt/
├── bin/
│   └── report-tool.tcl      the program: argv, exit status, output
├── lib/
│   ├── parse.tcl            reading and validating a readings file
│   └── report.tcl           turning rows into a summary
└── data/
    ├── readings.txt         sample input
    └── bad.txt              a deliberately malformed one, for testing

Three directories, and the names carry the meaning. bin/ is what you run. lib/ is what it uses. data/ is not code.

Someone who has never seen this program can guess where to look for the thing they want to change, which is most of what a layout is for.

3. What a library file looks like

# lib/parse.tcl — turn a readings file into a list of {name value} pairs.
# Procedures only. Sourcing this file must not do any work.

namespace eval ::report {}

# Read $path and return a list of {name value} pairs.
# Blank lines and lines starting with # are skipped.
proc ::report::parse_file {path} {
    set fp [open $path r]
    set rows {}
    set lineno 0
    while {[gets $fp line] >= 0} {
        incr lineno
        set line [string trim $line]
        if {$line eq "" || [string index $line 0] eq "#"} { continue }
        if {[llength $line] != 2} {
            close $fp
            error "$path:$lineno: expected 'name value', got '$line'" {} \
                  [list REPORT BADLINE $lineno]
        }
        lassign $line name value
        if {![string is double -strict $value]} {
            close $fp
            error "$path:$lineno: '$value' is not a number" {} \
                  [list REPORT BADVALUE $value]
        }
        lappend rows [list $name $value]
    }
    close $fp
    return $rows
}

Four decisions in that file, each one a rule you can reuse:

• It raises; it never prints and never exits. A library does not know whether its caller wants the message on stderr, in a log, or suppressed. It reports the problem and lets the program decide — Chapter 15.4 §3.
• The error names the file and line. data/bad.txt:2: 'warm' is not a number is a diagnosis. bad value is not.
• It carries a structured -errorcode. REPORT BADVALUE lets a caller distinguish a malformed file from a missing one without matching message text — Chapter 14.5 §7.
• Everything lives in a namespace. ::report::parse_file, not parse_file. Two libraries that both wanted a procedure called parse would otherwise silently overwrite each other, as Module 11 covered.

lib/report.tcl follows the same rules for the summarising half:

# lib/report.tcl — turn parsed rows into a summary. Procedures only.

namespace eval ::report {}

proc ::report::summarise {rows limit} {
    set count [llength $rows]
    if {$count == 0} { error "no readings to summarise" {} {REPORT EMPTY} }
    set total 0.0
    set over {}
    set worst [lindex [lindex $rows 0] 1]
    foreach row $rows {
        lassign $row name value
        set total [expr {$total + $value}]
        if {$value > $worst} { set worst $value }
        if {$value > $limit} { lappend over $name }
    }
    return [dict create \
        count $count \
        mean  [expr {$total / $count}] \
        max   $worst \
        over  $over]
}

Note that summarise takes limit as an argument rather than reading the namespace variable. That keeps it a pure function of its inputs: you can call it with any limit, in any order, and get a predictable answer. Configuration is the program's job; computing is the library's.

4. The payoff: sourcing the library runs nothing

This is the property the whole split exists for, and it is easy to check.

# test-parse.tcl
source rt/lib/parse.tcl
source rt/lib/report.tcl

puts "sourcing the library printed nothing and ran no program."
set rows [::report::parse_file rt/data/readings.txt]
puts "parse_file returned [llength $rows] rows"
puts "first row          : [lindex $rows 0]"
set s [::report::summarise $rows 30]
puts "summarise mean     : [format %.2f [dict get $s mean]]"
puts "summarise over     : [dict get $s over]"
puts ""
puts "and a deliberately bad input:"
puts "  [catch {::report::parse_file rt/data/bad.txt} e] / $e"

sourcing the library printed nothing and ran no program.
parse_file returned 4 rows
first row          : temp-inlet 21.5
summarise mean     : 51.48
summarise over     : temp-outlet pressure-1 temp-case

and a deliberately bad input:
  1 / rt/data/bad.txt:2: 'warm' is not a number

That short file is a test. No framework, no library, no dependency — it loads the logic, calls it with known input, and prints what came back. You can run it after every change in under a second, and you can call summarise with a limit of 30 without touching the command line or the environment.

This is what the registry purpose means by tested without running the whole program.

5. What happens if you source the program file instead

Now the contrast. Same source, different file:

# try-source-bin.tcl
puts "about to source bin/report-tool.tcl ..."
source rt/bin/report-tool.tcl
puts "THIS LINE NEVER RUNS"

$ tclsh try-source-bin.tcl
about to source bin/report-tool.tcl ...
error: no input file given
usage: try-source-bin.tcl [-verbose] [-limit N] [-out FILE] INPUT...
[exit status 2]

6. The program file

With the logic elsewhere, the program file is mostly about the outside world:

#!/usr/bin/env tclsh
# bin/report-tool.tcl — summarise one or more readings files.
#
#   usage: report-tool.tcl [-verbose] [-limit N] [-out FILE] INPUT...
#
# exit status: 0 all files summarised, 1 a file failed, 2 usage error.

set here [file dirname [file normalize [info script]]]
source [file join $here .. lib parse.tcl]
source [file join $here .. lib report.tcl]

namespace eval ::report {
    variable verbose 0
    variable limit   100.0
    variable out     ""
}

Three things to notice at the top of that file:

• The usage line and the exit-status convention are in a comment, at the top, where someone opening the file finds them first. This is the cheapest documentation in the project and the one most often missing.
• $here comes from [file dirname [file normalize [info script]]] — the idiom from 15.1 §6 — so bin/ finds lib/ no matter where the program is run from.
• [file join $here .. lib parse.tcl] navigates up one level and back down. It reads as the directory tree, and file join handles the separators.

Then main does the sequence, and the only exit in the project is the last line:

proc ::report::main {argv} { ... ; return [expr {$failures > 0 ? 1 : 0}] }

exit [::report::main $argv]

main is an ordinary procedure returning an ordinary number. It could be called from a test script just like parse_file was. Only that final line converts its result into a process status.

7. Does it run from anywhere?

The acceptance test for the $here idiom. Three different working directories, same program:

$ cd /private/tmp/rt15 ; tclsh rt/bin/report-tool.tcl rt/data/readings.txt
report for rt/data/readings.txt
  readings : 4
  mean     : 51.48
  max      : 101.30
  over limit: pressure-1
$ cd /private/tmp/rt15/rt/bin ; tclsh report-tool.tcl ../data/readings.txt
report for ../data/readings.txt
  readings : 4
  mean     : 51.48
  max      : 101.30
  over limit: pressure-1
$ cd / ; tclsh /private/tmp/rt15/rt/bin/report-tool.tcl /private/tmp/rt15/rt/data/readings.txt
report for /private/tmp/rt15/rt/data/readings.txt
  readings : 4
  mean     : 51.48
  max      : 101.30
  over limit: pressure-1

From the project root, from inside bin/, and from the filesystem root. The source lines found lib/ every time.

Replace $here with a plain relative path and the third case fails immediately with couldn't read file "./lib/parse.tcl" — which is 15.1 §5's trap, now affecting a program that otherwise works perfectly.

8. Deciding where a procedure goes

The questions worth asking, in order:

1. Does it touch argv, env, exit, or print a user-facing message? Then it belongs in the program file. Those are decisions about this invocation.
2. Is it a function of its arguments? Then it belongs in a library, and it should take what it needs as parameters rather than reading a namespace variable.
3. Would a second program want it? Then a library, definitely.
4. Is it only meaningful to one other procedure? Then leave it next to that procedure rather than promoting it.

The honest version of rule 2 is that most of the useful logic in any program is a function of its arguments, and the parts that are not — configuration, output, status — are a thin shell around it. Writing that shell last, in its own file, is what keeps the rest testable.

9. Exercises

Exercise 1. State the one-sentence test for whether a file is a library.

Exercise 2. Why does a library procedure error rather than print to stderr and exit?

Exercise 3. summarise takes limit as an argument instead of reading the namespace variable. Why is that better?

Exercise 4. You source the program file by mistake. Name three distinct things that go wrong.

Exercise 5. In the sourcing experiment the usage line said try-source-bin.tcl. Why?

Exercise 6. Write the two lines at the top of bin/report-tool.tcl that let it find lib/ from any working directory.

Exercise 7. Where does a procedure that reads $::env(REPORT_LIMIT) belong, and why?