package require and package provide

Part I · Module 15 · Chapter 15.8 · package require and package provide

Chapter 15.7 put the procedures in lib/ and loaded them with source [file join $here .. lib parse.tcl]. That works, and for a program with its own private library it is the right answer.

It stops being the right answer when a second program wants the same library. Now both have to know where it lives, and moving it means editing both. Packages replace where it is with what it is called.

1. The difference in one line

source /some/path/lib/reportutil.tcl        ;# where it is
package require reportutil 1.0              ;# what it is called

The second line says nothing about the filesystem. Tcl works out where the package lives, loads it if it has not already been loaded, and gives you an error if it cannot find a suitable version.

2. What a package is

Three pieces. None of them is complicated on its own.

1. A library file that declares its own name and version — package provide.
2. An index file that tells Tcl how to load it — pkgIndex.tcl.
3. A search path Tcl looks along — auto_path.

3. The library declares itself

# reportutil/reportutil.tcl
package provide reportutil 1.0

namespace eval ::reportutil {}

proc ::reportutil::mean {values} {
    if {[llength $values] == 0} { error "mean needs at least one value" }
    set total 0.0
    foreach v $values { set total [expr {$total + $v}] }
    return [expr {$total / [llength $values]}]
}

package provide reportutil 1.0 is the whole declaration: this file supplies the package reportutil, version 1.0. It goes at the top, and everything else is the ordinary library file from 15.7 §3 — procedures in a namespace, no side effects.

The namespace and the package name are separate things that conventionally match. The package name is what package require asks for; the namespace is what the procedures are called. Keeping them the same saves confusion later.

4. The index tells Tcl how to load it

# reportutil/pkgIndex.tcl
package ifneeded reportutil 1.0 [list source [file join $dir reportutil.tcl]]

Read package ifneeded as: if anyone needs reportutil 1.0, here is the script that provides it. The script here is simply source of the real file.

$dir is set by Tcl to the directory containing pkgIndex.tcl while the index is being read. That is what makes the package relocatable — move the directory and the index still points at the right file, with no absolute path written anywhere.

Tcl can generate this file for you with pkg_mkIndex, which scans a directory and writes the index. For a one-file package, writing it by hand is clearer and no slower.

5. auto_path is where Tcl looks

puts "before adding our directory:"
puts "  [catch {package require reportutil 1.0} e] / $e"

lappend auto_path [file dirname [file normalize [info script]]]

puts "after lappend auto_path <this directory>:"
set v [package require reportutil 1.0]
puts "  package require returned: $v"
puts "  mean {10 20 30}         : [::reportutil::mean {10 20 30}]"
puts "  package versions        : [package versions reportutil]"

before adding our directory:
  1 / can't find package reportutil 1.0
after lappend auto_path <this directory>:
  package require returned: 1.0
  mean {10 20 30}         : 20.0
  package versions        : 1.0

auto_path is a global list of directories. For each one, Tcl looks in its subdirectories for a pkgIndex.tcl and reads what it finds. Adding our directory is what made reportutil findable.

Three details in that output:

• can't find package reportutil 1.0 is the error you will actually meet, and it almost always means auto_path rather than a missing file. Print $auto_path first.
• package require returns the version it loaded — 1.0. Worth capturing when you care which one you got.
• The directory added is the parent of reportutil/. Tcl searches one level down, so you point auto_path at the directory containing package directories, not at the package itself.

6. What a version number means

This is the part most people guess wrong. Each request below ran in a fresh interpreter, so nothing was cached from a previous load.

With only version 1.0 installed:

only version 1.0 installed:
  package require reportutil               -> got 1.0
  package require reportutil 1.0           -> got 1.0
  package require reportutil 1.1           -> ERROR: can't find package reportutil 1.1
  package require reportutil 2.0           -> ERROR: can't find package reportutil 2.0
  package require -exact reportutil 1.0    -> got 1.0

Now with only version 1.1 installed:

only version 1.1 installed:
  package require reportutil 1.0           -> got 1.1
  package require reportutil 2.0           -> ERROR: can't find package reportutil 2.0

So when you write package provide, the number is a promise:

• Bump the minor number (1.0 → 1.1) when you add something. Existing callers keep working.
• Bump the major number (1.1 → 2.0) when you change or remove something. Callers asking for 1.x will now be refused instead of silently breaking.

7. Using it from a program

From the caller's side it is one line, and nothing in the program knows where the library is:

package require reportutil 1.0
puts [::reportutil::mean {10 20 30}]

Compare that with the source version, which has to know the path, resolve it against the script's own directory, and be edited if the library ever moves.

The trade is real but small: a package needs the index file and a findable auto_path, and in exchange the caller stops caring about the filesystem. For a library used by one program, keep source. For one used by five, package it.

8. Two errors you will meet

can't find package X — Tcl searched every pkgIndex.tcl under every auto_path entry and nothing provided a satisfying version. Check, in order: is auto_path pointing at the parent of the package directory; does pkgIndex.tcl exist; does the version requested match what is provided.

version conflict for package "X" — something already loaded a different version than the one now being required. This happens when two libraries each require the same package at incompatible versions. The fix is to agree on a version, not to force one.

For diagnosis, three commands pay for themselves:

puts $auto_path                       ;# where Tcl is looking
puts [package names]                  ;# what it knows about
puts [package versions reportutil]    ;# which versions of one

This is the same introspection habit as Chapter 14.10 — ask the interpreter rather than guessing.

9. What this looks like in Part II

You will meet package require long before you write one. A vendor tool's Tcl often ships packages for its own facilities, and site scripts are commonly shipped as packages so that many flows can share them.

When you see an unfamiliar package require something 2.1 at the top of a script, you now know exactly what it does: look along auto_path for a pkgIndex.tcl providing something at 2.1 or any later 2.x, run the stored loading script, and fail loudly if there is none.

10. Exercises

Exercise 1. What are the three pieces of a Tcl package?

Exercise 2. What does $dir mean inside pkgIndex.tcl, and why does it matter?

Exercise 3. You add the package's own directory to auto_path and still get can't find package. What is probably wrong?

Exercise 4. Only version 1.1 is installed. What does package require reportutil 1.0 do?

Exercise 5. Why does package require reportutil 2.0 fail when 1.1 is installed?

Exercise 6. Write the line that demands exactly version 1.0.

Exercise 7. A file is sourced three times and required three times. How many times does each actually load?