file normalize, and Paths That Survive

Part I · Module 12 · Chapter 12.11 · file normalize, and Paths That Survive

1. What it does

puts "cwd       : [pwd]"
puts "relative  : data/run.csv"
puts "normalized: [file normalize data/run.csv]"

cwd       : /tmp/work
relative  : data/run.csv
normalized: /tmp/work/data/run.csv

It resolves the path against the current directory and returns an absolute one. It also collapses . and ..:

puts [file normalize "a/b/../c"]
puts [file normalize "./x/./y"]

/tmp/work/a/c
/tmp/work/x/y

The exact prefix depends on where you run it; the shape does not.

2. It does not touch the filesystem

file normalize is a question about the path, not about what is there:

puts [file normalize "no/such/place.txt"]
puts "does it exist? [file exists [file normalize "no/such/place.txt"]]"

/tmp/work/no/such/place.txt
does it exist? 0

Nothing was created and nothing was checked. That makes it safe to call on a path you are about to create, and it means a normalized path is still only a name — Chapter 12.8's rules about checking still apply.

3. The hazard: a relative path outlives its directory

Here is why the chapter exists. Record a relative path, change directory, and it now means somewhere else:

proc read_first {path} {
    set fp [open $path r]
    set line [gets $fp]
    close $fp
    return $line
}

file mkdir sub
set fp [open data.txt w] ; puts $fp "original" ; close $fp
set fp [open sub/data.txt w] ; puts $fp "the one in sub" ; close $fp

set remembered "data.txt"
set normalized [file normalize "data.txt"]

puts "before cd:"
puts "  relative   -> [read_first $remembered]"
puts "  normalized -> [read_first $normalized]"

cd sub

puts "after cd into sub:"
puts "  relative   -> [read_first $remembered]"
puts "  normalized -> [read_first $normalized]"

before cd:
  relative   -> original
  normalized -> original
after cd into sub:
  relative   -> the one in sub
  normalized -> original

The relative path silently started referring to a different file. The normalized one did not.

4. One location, one string

Different spellings of the same place normalize to the same result, which is what makes paths comparable:

set a [file normalize "data/run.csv"]
set b [file normalize "./data/run.csv"]
set c [file normalize "data/../data/run.csv"]
set d [file normalize "data/./run.csv"]

puts "all four identical? [expr {$a eq $b && $b eq $c && $c eq $d}]"
puts "the canonical form: $a"

all four identical? 1
the canonical form: /tmp/work/data/run.csv

Without normalizing, comparing paths as strings is unreliable:

puts "raw comparison : [expr {"data/run.csv" eq "./data/run.csv"}]"

raw comparison : 0

Two strings, one file. Any code that de-duplicates a list of paths, or checks whether an output would overwrite an input, has to normalize first.

5. Deduplicating a list of paths

proc unique_paths {paths} {
    set seen [dict create]
    set out {}
    foreach p $paths {
        set key [file normalize $p]
        if {[dict exists $seen $key]} continue
        dict set seen $key 1
        lappend out $key
    }
    return $out
}

set inputs {data/run.csv ./data/run.csv data/other.csv data/../data/run.csv}
puts "given  : [llength $inputs] path(s)"
puts "unique : [llength [unique_paths $inputs]] path(s)"
foreach p [unique_paths $inputs] { puts "  [file tail $p]" }

given  : 4 path(s)
unique : 2 path(s)
  run.csv
  other.csv

Four spellings, two files. The dictionary-as-a-set idiom is Chapter 7.3's.

6. Where it matters in practice

Before storing a path anywhere. A path written into a report, a log or a state file is read back later, possibly by something standing somewhere else.

Before comparing two paths, for the reason in section 4.

Before changing directory, if you have paths in hand — normalize them first, as section 3 showed.

When a path comes from a user or a configuration file, where ~, . and .. are all likely:

foreach raw {"./out" "../sibling" "out/../out/final"} {
    puts [format "  %-20s -> %s" $raw [file normalize $raw]]
}

  ./out                -> /tmp/work/out
  ../sibling           -> /tmp/sibling
  out/../out/final     -> /tmp/work/out/final

7. A worked example

A procedure that records inputs so they stay valid wherever the script goes next:

proc collect_inputs {paths} {
    set records {}
    foreach p $paths {
        set abs [file normalize $p]
        lappend records [dict create \
            given $p \
            path  $abs \
            name  [file tail $abs] \
            found [file exists $abs]]
    }
    return $records
}

set fp [open data.txt w] ; puts $fp "x" ; close $fp

set records [collect_inputs {data.txt ./data.txt missing.txt}]

cd sub   ;# the script moves on

foreach r $records {
    puts [format "  %-14s found=%s  name=%s" \
        [dict get $r given] [file exists [dict get $r path]] [dict get $r name]]
}

  data.txt       found=1  name=data.txt
  ./data.txt     found=1  name=data.txt
  missing.txt    found=0  name=missing.txt

The cd happened between recording and checking, and both spellings of data.txt still resolve — because what was stored was the normalized form, not what the caller typed.

8. Exercises

Exercise 1. What does file normalize do to a relative path, and to .. in the middle of one?

Exercise 2. Does it require the path to exist?

Exercise 3. Why does a relative path stored before a cd refer to a different file afterwards?

Exercise 4. Why is "data/run.csv" eq "./data/run.csv" false, and how do you compare them properly?

Exercise 5. Write a procedure returning the unique files from a list of possibly-duplicated path spellings.

Exercise 6. Give three moments at which normalizing is the right thing to do.

Answers

1. It resolves the path against the current directory and returns an absolute one, collapsing . and .. along the way — and expanding a leading ~.

2. No. It is a question about the path, not about the filesystem, so it works on a path that does not exist and creates nothing.

3. Because a relative path is interpreted against the current directory, and cd changes that. The same string now names a different location — and the bug is silent, because the file it finds usually exists.

4. Because they are different strings naming the same file, and eq compares strings. Normalize both first; they then produce the identical canonical form.

5.

proc unique_paths {paths} {
    set seen [dict create]
    set out {}
    foreach p $paths {
        set key [file normalize $p]
        if {[dict exists $seen $key]} continue
        dict set seen $key 1
        lappend out $key
    }
    return $out
}

6. When receiving a path from outside the script; before storing one for later use; and before comparing two paths. All three are the same underlying reason — a relative path is only meaningful alongside the directory it was written against.

9. Remember

• file normalize returns one absolute, canonical form, collapsing . and .. and expanding ~.
• It does not touch the filesystem and works on paths that do not exist.
• It is idempotent, so normalizing twice is harmless.
• A relative path stored before a cd silently means something else afterwards. Normalize when you receive a path, not when you use it.
• Different spellings of one location normalize to the same string, which is what makes paths comparable and de-duplicable.
• Comparing raw path strings is unreliable — data/run.csv and ./data/run.csv are not equal as strings.
• Normalize before storing, before comparing, and before changing directory.