Temporary Files Done Right: mktemp

Part I · Module 20 · Chapter 20.6 · Temporary Files Done Right: mktemp

Chapter 20.5's scripts all began the same way:

tmp=$(mktemp -d)
trap 'rm -rf -- "$tmp"' EXIT

The second line has been explained. This chapter is the first.

1. What People Write Instead

tmp="/tmp/mybuild.$$"
mkdir -p "$tmp"

$$ is the process id (Chapter 7.1), so the name is different for each run. That looks like it solves the problem, and it has three faults.

It is predictable. Anyone on the machine can work out the name — $$ is visible in ps — and /tmp is world-writable. So another user can create /tmp/mybuild.12345 as a symlink to something they want you to overwrite, before your script gets there. Your > "$tmp/output" then writes through the symlink. That is the classic temp-file attack, and it needs no privileges beyond being logged in.

It is not unique enough. Process ids are recycled, and $$ in a subshell is the parent's pid (Chapter 17.7's note), so two things you thought were distinct can agree. More simply: a script that builds two temporary paths from $$ and forgets to distinguish them collides with itself.

And it is world-readable. Files you create inherit your umask (Chapter 5.6), which on most systems leaves them readable by everyone:

$ umask
0022

A build log in /tmp with mode 644 is readable by every account on a shared machine. On an EDA farm that may include the contents of a design you are not allowed to share.

2. mktemp Creates It, and That Is the Point

$ t=$(mktemp)
$ echo "$t"
/var/folders/7g/…/T/tmp.8R6Lz0ajRM
$ ls -l "$t"
-rw-------  1 engineer  engineer  0 Oct  8 19:02 /var/folders/…/tmp.8R6Lz0ajRM

$ d=$(mktemp -d)
$ ls -ld "$d"
drwx------  2 engineer  engineer  64 Oct  8 19:02 /var/folders/…/tmp.sELOvBccj1

Two things to read off that.

The mode is 0600 for a file and 0700 for a directory. Not your umask; mktemp sets them explicitly. Nobody else can read the file or even list the directory.

And the thing already exists. mktemp did not suggest a name for you to create — it created it and told you what it called it. That is the whole mechanism, and it is why the symlink attack does not apply: the create happened atomically, with O_EXCL, so there was no window between "this name is free" and "this name is mine". If the name had existed — as a file, a directory or a symlink — the create would have failed and mktemp would have tried another.

Successive calls never agree:

$ for i in 1 2 3; do mktemp -u -t m20test; done
/var/folders/…/T/m20test.VRG07RUJQt
/var/folders/…/T/m20test.Kgj8Cpw37y
/var/folders/…/T/m20test.WX2qfhVfXq

(-u means "give me the name without creating it", which is only safe for a demonstration — never use it in a script, because it reintroduces exactly the window mktemp exists to close.)

3. The Options Worth Knowing

$TMPDIR is not /tmp on macOS:

$ echo "$TMPDIR"
/var/folders/7g/8jhn933x6cg_pq8gzx8z9b_r0000gn/T/
$ TMPDIR=/tmp mktemp -u -t m20
/tmp/m20.9Pf39Ghgu3

A per-user directory, which is a better default than a shared /tmp — and a reason to use mktemp rather than hardcoding /tmp even when you are not worried about collisions. Honour TMPDIR if it is set; it is how a user or a job scheduler tells you where scratch space lives, and on a compute farm that is often a fast local disk rather than a network mount.

A template must end in Xs, and how many is platform-dependent:

$ mktemp /tmp/m20.XXXXXX
/tmp/m20.HtNayR
$ mktemp /tmp/m20.XXX
/tmp/m20.LpJ
$ mktemp /tmp/m20.X
/tmp/m20.i

BSD mktemp on this machine accepted a single X. GNU mktemp requires at least three, and more is better regardless — six is conventional and gives enough randomness that a collision is not a practical concern. Write six. Relying on one working is relying on which mktemp you happen to have.

4. Always Pair It With a Trap

A temporary directory with no cleanup is worse than no temporary directory, because the debris accumulates silently until a disk fills:

tmp=$(mktemp -d)
trap 'rm -rf -- "$tmp"' EXIT

That is Chapter 20.5 §1 and §2, and the pairing is so standard it is worth treating as a single idiom: the trap goes on the line after the mktemp, every time.

$ cat mk.sh
#!/usr/bin/env bash
set -euo pipefail
tmp=$(mktemp -d)
trap 'rm -rf -- "$tmp"' EXIT
printf 'scratch\n' > "$tmp/work.txt"
echo "  created $tmp, contains: $(ls "$tmp")"

$ ./mk.sh
  created /…/tmp.BgbZjbLA5q, contains: work.txt
$ ls -d /…/tmp.BgbZjbLA5q
ls: /…/tmp.BgbZjbLA5q: No such file or directory

And Chapter 20.5 §5's full version covers the interrupt as well, which matters most here — a long build interrupted at hour three is exactly when a multi-gigabyte scratch directory gets abandoned.

5. A Complete Script

A filelist-driven build, with one scratch directory, readable names inside it, and cleanup on every path:

#!/usr/bin/env bash
# compile.sh — collect sources, write a filelist, run a compile
# usage: compile.sh [-k] <srcdir>
#        -k  keep the scratch directory
#
# exit: 0 compiled · 1 compile failed · 2 unusable arguments · 130 interrupted

set -o errexit
set -o nounset
set -o pipefail
shopt -s nullglob

note() { printf '%s: %s\n' "${0##*/}" "$*" >&2; }
die()  { note "$*"; exit 2; }

keep=0
case ${1:-} in -k) keep=1; shift ;; esac

src=${1:-}
[ -n "$src" ] || die "usage: ${0##*/} [-k] <srcdir>"
[ -d "$src" ] || die "not a directory: $src"

# ---- one scratch directory, and its handler ----------------------------

tmp=

cleanup() {
  local st=$?
  if [ "$keep" -eq 1 ] && [ -n "$tmp" ]; then
    note "keeping scratch: $tmp"
  else
    rm -rf -- "${tmp:-}"
  fi
  return "$st"
}
trap cleanup EXIT
trap 'cleanup; trap - INT EXIT; kill -INT $$' INT
trap 'cleanup; trap - TERM EXIT; kill -TERM $$' TERM

tmp=$(mktemp -d "${TMPDIR:-/tmp}/compile.XXXXXX") || die "cannot create a scratch directory"

# readable names inside a 0700 directory
filelist="$tmp/files.f"
buildlog="$tmp/build.log"

note "scratch: $tmp"

# ---- collect ------------------------------------------------------------

declare -a sources=()
while IFS= read -r -d '' f; do
  sources+=("$f")
done < <(find "$src" -type f -name '*.sv' -print0)

[ "${#sources[@]}" -gt 0 ] || die "no .sv files under $src"

printf '%s\n' "${sources[@]}" > "$filelist"
note "filelist: ${#sources[@]} source(s)"

# ---- compile ------------------------------------------------------------

# stand-in for the real tool: fails if any source mentions SYNTAX_ERROR
if grep -l SYNTAX_ERROR "${sources[@]}" > "$buildlog" 2>&1; then
  note "compile failed; offending file(s):"
  sed 's/^/      /' "$buildlog" >&2
  exit 1
fi

printf 'compiled %d source(s) from %s\n' "${#sources[@]}" "$src"

$ mkdir -p rtl/sub
$ printf 'module top;\nendmodule\n'        > rtl/top.sv
$ printf 'module alu;\nendmodule\n'        > 'rtl/my alu.sv'
$ printf 'module deep;\nendmodule\n'       > rtl/sub/deep.sv

$ ./compile.sh rtl
compile.sh: scratch: /…/T/compile.SHdQ3m
compile.sh: filelist: 3 source(s)
compiled 3 source(s) from rtl
$ echo $?
0

$ printf 'module bad;\nSYNTAX_ERROR\nendmodule\n' > rtl/bad.sv
$ ./compile.sh rtl
compile.sh: scratch: /…/T/compile.7Kp2Rx
compile.sh: filelist: 4 source(s)
compile.sh: compile failed; offending file(s):
      rtl/bad.sv
$ echo $?
1

$ ./compile.sh -k rtl
compile.sh: scratch: /…/T/compile.9Lm4Tz
compile.sh: filelist: 4 source(s)
compile.sh: compile failed; offending file(s):
      rtl/bad.sv
compile.sh: keeping scratch: /…/T/compile.9Lm4Tz
$ ls /…/T/compile.9Lm4Tz
build.log  files.f

$ mkdir bare && ./compile.sh bare
compile.sh: scratch: /…/T/compile.2Qw8Er
compile.sh: no .sv files under bare
$ echo $?
2

Five decisions:

One directory, two named files inside it. files.f and build.log are readable in the -k listing and in a trace; two separate mktemp calls would have given two opaque names and two cleanup obligations.

mktemp -d "${TMPDIR:-/tmp}/compile.XXXXXX" honours TMPDIR with a sensible fallback, puts the script's name in the path so a stray directory is identifiable, and uses six Xs rather than relying on this platform accepting fewer.

|| die on the mktemp. It can fail — a full disk, an unwritable TMPDIR, a bad template — and under set -e a bare failure would abort with no message of your own. Measured:

$ TMPDIR=/ ./compile.sh rtl
mktemp: mkdtemp failed on //compile.yU02TT: Read-only file system
compile.sh: cannot create a scratch directory
$ echo $?
2

Two lines: mktemp's own diagnostic, which names the real cause, and yours, which names what the script was trying to do. Both are useful and neither alone is sufficient.

tmp= is initialised empty before the traps, so a failure between registration and assignment does not break the handler under set -u. Chapter 20.5 §6's rule.

The bare run shows the scratch message and then exits 2 — the directory was created before the source count was known, so it had to be cleaned up, and it was. Validating earlier would have avoided creating it at all; here the count depends on a find that needs no scratch space, so moving that check above the mktemp would be a genuine improvement. Chapter 20.3 §3's ordering rule is easy to state and easy to drift from.

Practice

1. Run t=$(mktemp) and ls -l "$t". Then run touch /tmp/mine.$$ and compare the modes. Which is readable by other users?
2. Print your umask and explain the mode you got on the hand-made file.
3. Run mktemp -u -t demo three times. Then explain why -u must not appear in a real script.
4. Print $TMPDIR. Is it /tmp? Then run TMPDIR=/tmp mktemp -u -t demo and note the difference.
5. Try templates with six, three and one X. Which does your mktemp accept? Then say why you should write six regardless.
6. Write a script with tmp=$(mktemp -d) and no trap. Run it ten times and count what is left in $TMPDIR.
7. Add the trap and confirm the directory is gone after the script exits. Then make the script fail halfway and confirm it is still gone.
8. Convert a script that calls mktemp three times into one that calls mktemp -d once. Count the cleanup lines before and after.
9. Give compile.sh an unwritable TMPDIR (TMPDIR=/ ./compile.sh rtl) and confirm the || die message rather than a bare abort.
10. Move compile.sh's source-count check above the mktemp. Run it on an empty directory and confirm no scratch directory is created at all.

Check Your Understanding

1. What three things are wrong with tmp=/tmp/myscript.$$?
2. What mode does mktemp create a file with, and why does that matter?
3. What does mktemp do that [ -e "$f" ] || touch "$f" does not?
4. Why must -u not appear in a script?
5. Is $TMPDIR the same as /tmp?
6. How many Xs should a template have?
7. Why one directory rather than several files?

A run directory is the long-lived counterpart of a temporary one: Chapter 27.3 measures what happens to four concurrent simulations that share one.

For the long-lived equivalent, Chapter 28.5 keeps 4 KB of a 60 MB run and proves it is enough by deleting the original and reproducing the failure.

6. Remember

• mktemp creates the thing and tells you its name — it does not suggest a name for you to create. That atomicity is the point.
• A predictable name in /tmp is a security bug, not just a collision risk: another user can pre-create it as a symlink.
• $$ is not unique enough. Process ids are recycled, and $$ in a subshell is the parent's.
• mktemp sets 0600 for a file and 0700 for a directory, regardless of your umask.
• Never -u in a script. It hands back a name and reopens the window.
• Honour $TMPDIR — it is not /tmp on macOS, and on a farm it is often a fast local disk.
• Write six Xs. BSD accepted one here and GNU wants three; six removes the question.
• || die on the mktemp itself. It can fail, and under set -e a bare failure aborts with no message.
• The trap goes on the line after the mktemp, every time.
• One directory with readable names beats several opaque files — one cleanup obligation, and files.f is a better thing to see in a trace than tmp.8R6Lz0ajRM.
• Create scratch space as late as possible. A check that does not need it belongs above the mktemp.

Chapter 20.7 is about the lines a script prints rather than the files it writes: which of its two output channels each one belongs on, and why a wrapper that gets this wrong cannot be used inside a pipeline at all.