Capturing an Environment for Reproduction and Hand-Off

Part II · Module 25 · Chapter 25.8 · Capturing an Environment for Reproduction and Hand-Off

A run happens in an environment that exists for a few minutes. Afterwards the licence variable has changed, the tool directory has a new current symlink, someone has edited the setup script, and the question "what produced this result" has no answer.

The fix costs four files and about a second per run.

1. What env | sort Gives You, and What It Misses

The registry's suggestion is the right starting point:

env | sort > "$rundir/env.txt"

It captures every exported variable, which is most of what matters. Four things it misses, and each has cost somebody a day:

Anything not exported. A shell variable set in a setup script without export affects that shell's behaviour and is invisible to env — and to the run's children, which is usually the bug.

Facts that are not variables: the host, the physical working directory, the umask, ulimit -n, the shell version, the operating system. None appears in env and all of them change results.

Which tool a name resolved to. PATH is captured; the resolution is not. Two machines with the same PATH and different installations behind it produce different runs with identical env.txt.

And the source revision, with whether the tree was clean.

So the capture is four files, not one.

2. The Four Files

The path.txt split is the one worth explaining. A PATH with twenty entries is one enormously long line; change one entry and diff reports one changed line containing two thousand characters, which tells you nothing. Split and numbered, the same change is one short line — and the number matters because Chapter 25.2 §4 established that position decides which tool wins.

3. The Script

#!/usr/bin/env bash
# capture-env — write everything needed to reproduce this run into a directory
# usage: capture-env <dir> [tool...]
# exit:  0 written · 2 unusable
set -o nounset

[ $# -ge 1 ] || { printf 'usage: capture-env <dir> [tool...]\n' >&2; exit 2; }
d=$1; shift
mkdir -p -- "$d" || exit 2
d=$(cd -- "$d" && pwd -P) || exit 2

tool_version() {                      # [Chapter 25.7](/linux-bash/tool-version-checks) §2, with its 2>&1
  local t=$1 f out
  for f in -version --version -V -ID -v; do
    out=$("$t" "$f" 2>&1) || continue
    out=$(printf '%s\n' "$out" | head -1)
    [ -n "$out" ] || continue
    printf '%s\n' "$out"; return 0
  done
  return 1
}

# 1. the full environment, sorted, with the volatile parts removed
env | sort | grep -vE '^(_|SHLVL|OLDPWD|RANDOM|SECONDS|LINENO|PPID)=' > "$d/env.txt"

# 2. PATH as a numbered list
printf '%s\n' "$PATH" | tr ':' '\n' | grep -n . > "$d/path.txt"

# 3. the facts that are not in the environment
{
  printf 'host        %s\n' "$(hostname)"
  printf 'user        %s\n' "$(id -un)"
  printf 'cwd         %s\n' "$(pwd -P)"
  printf 'date        %s\n' "$(date -u '+%Y-%m-%dT%H:%M:%SZ')"
  printf 'shell       %s\n' "${BASH_VERSION:-unknown}"
  printf 'uname       %s\n' "$(uname -srm)"
  printf 'locale      LANG=%s LC_ALL=%s\n' "${LANG:-unset}" "${LC_ALL:-unset}"
  printf 'umask       %s\n' "$(umask)"
  printf 'ulimit -n   %s\n' "$(ulimit -n)"
} > "$d/run.txt"

# 4. source revision, and whether it was clean
if git rev-parse --is-inside-work-tree > /dev/null 2>&1; then
  rev=$(git rev-parse --short HEAD 2>/dev/null || echo unknown)
  n=$(git status --porcelain 2>/dev/null | grep -c . || true)
  [ "$n" -eq 0 ] || rev="$rev+$n-modified"
  printf 'source      %s\n' "$rev" >> "$d/run.txt"
  [ "$n" -eq 0 ] || git diff > "$d/uncommitted.patch" 2>/dev/null
else
  printf 'source      not a git repository\n' >> "$d/run.txt"
fi

# 5. every tool named, with its path and version
: > "$d/tools.txt"
for t in "$@"; do
  p=$(command -v -- "$t" 2>/dev/null) || { printf '%-12s NOT FOUND\n' "$t" >> "$d/tools.txt"; continue; }
  v=$(tool_version "$t") || v='(no version)'
  printf '%-12s %s\n' "$t" "$p" >> "$d/tools.txt"
  printf '%-12s %s\n' '' "$v" >> "$d/tools.txt"
done

printf 'capture-env: wrote %s\n' "$d"
ls -1 "$d"

$ ./capture-env run_x/env simtool iverilog nvc vcs
capture-env: wrote /home/engineer/eda/run_x/env
env.txt
path.txt
run.txt
tools.txt
$ cat run_x/env/run.txt
host        compute-07
user        engineer
cwd         /home/engineer/eda
date        2026-10-08T19:56:52Z
shell       3.2.57(1)-release
uname       Darwin 25.5.0 arm64
locale      LANG=C.UTF-8 LC_ALL=unset
umask       0022
ulimit -n   1048576
source      not a git repository
$ cat run_x/env/tools.txt
simtool      /home/engineer/eda/tools/simtool-2024.09/bin/simtool
             simtool 2024.09  (simlib 1.4.2)
iverilog     /opt/homebrew/bin/iverilog
             Icarus Verilog version 13.0 (stable) (v13_0)
nvc          /opt/homebrew/bin/nvc
             nvc 1.23.0 (Using LLVM 23.1.1)
vcs          NOT FOUND
$ wc -l < run_x/env/env.txt
61

vcs NOT FOUND is recorded, not skipped. A tool the flow was told to use and could not find is one of the most valuable things in the file, and a capture that silently omitted it would hide the cause of a run that fell back to something else.

Eight decisions:

mkdir -p then cd … && pwd -P. The directory is created, and then its physical path is recorded — so the header names somewhere another machine can find, per Chapter 22.3 §4.

The volatile filter is explicit and short. _ changes on every command, SHLVL and PPID differ between a shell and a script, and RANDOM/SECONDS/LINENO change as you read them. Filtering more than that starts hiding real differences; filtering less makes every diff noisy.

grep -n . for path.txt, which numbers the lines and drops empty ones — so a trailing colon in PATH does not produce a blank numbered entry, and the numbering reflects real entries. (An empty PATH element means the current directory, which Chapter 10.2 §7 measured; it is worth noticing in env.txt rather than silently numbering here.)

date -u in ISO form, because a local timestamp from a farm spanning time zones cannot be ordered, and because %Y-%m-%dT%H:%M:%SZ sorts lexicographically.

uname -srm — kernel name, release and machine — which is the minimum that distinguishes two compute nodes, and is the thing a GLIBC_2.34' not found error from Chapter 25.3 §5 refers to.

The revision carries +N-modified when the tree is dirty, and the actual diff is saved alongside it. Chapter 25.7 §5 argued for the marker; saving uncommitted.patch is what makes the run genuinely reproducible rather than merely honestly labelled.

Both path and version are recorded per tool, on two lines. Two installations can share a version string, and one installation can be reached by two paths.

And it prints the directory and lists what it wrote, so a flow log shows the capture happened. A silent capture is one nobody notices is missing.

4. The Payoff: a Diff That Says Exactly What Differed

A capture is only worth writing if comparing two of them is cheap. Two runs were captured in deliberately different environments — a different tool version, a locale, a licence variable removed:

$ diff run_x/env/run.txt run_x/env2/run.txt
7c7
< locale      LANG=C.UTF-8 LC_ALL=unset
---
> locale      LANG=C.UTF-8 LC_ALL=C

One line. Host, user, shell, umask and limits all matched, so they are eliminated.

$ diff run_x/env/tools.txt run_x/env2/tools.txt
1,2c1,2
< simtool      /home/engineer/eda/tools/simtool-2024.09/bin/simtool
<              simtool 2024.09  (simlib 1.4.2)
---
> simtool      /home/engineer/eda/tools/simtool-2025.03/bin/simtool
>              simtool 2025.03  (simlib 1.6.0)
7d6
< vcs          NOT FOUND

The tool and its library both changed, which is Chapter 25.2 §1's measurement showing up in a record rather than in a terminal.

$ diff run_x/env/env.txt run_x/env2/env.txt | grep -cE '^[<>]'
5
$ diff run_x/env/env.txt run_x/env2/env.txt | grep -E '^[<>]' | cut -c1-72
> LC_ALL=C
< PATH=/home/engineer/eda/tools/simtool-2024.09/bin:/opt/homebrew/bin…
> PATH=/home/engineer/eda/tools/simtool-2025.03/bin:/opt/homebrew/bin…
< SIMTOOL_LICENSE_FILE=/home/engineer/eda/licenses/site.lic
> SIMTOOL_VERSION=2025.03

Five changed lines out of 61, and every one is a real difference: the locale, the two halves of the PATH change, the removed licence variable, and the added version variable. Nothing spurious.

That is the whole argument for sorting and filtering before writing. An unsorted env diffs as dozens of moved lines; an unfiltered one diffs on _ every time. The capture is designed for the diff, not for reading.

5. What Not to Capture

Secrets. env includes everything exported, and a token or password in the environment ends up in a file that gets copied into a bug report. Chapter 10.1 measured that the environment is readable by other processes; writing it to a file in a shared results area makes it readable by everyone, permanently.

env | sort | grep -vE '^(.*_TOKEN|.*_SECRET|.*PASSWORD|.*_KEY)=' > "$d/env.txt"

A deny-list, with Chapter 23.5's warning attached: it excludes what you remembered. The better answer is not to put secrets in the environment at all, and the filter is a second line of defence.

Enormous variables. A PATH of fifty entries is fine; a variable holding a serialised blob of several megabytes — which some tool wrappers do — makes every capture large and every diff useless. Truncate with a note rather than dropping it silently:

env | sort | awk 'length($0) > 4000 { printf "%s=<%d bytes, truncated>\n", substr($0,1,index($0,"=")-1), length($0); next } {print}'

And the whole module list output, if a module system is present — capture it, but as its own file, because it is the authoritative statement of what was loaded and it diffs much more cleanly than the PATH it produced. Chapter 25.6 §5's point that module list says what you asked for applies: capture both it and tools.txt, since the gap between them is exactly the kind of problem a capture exists to find.

6. Where the Capture Lives

In the run directory, next to the log — not in a central database, and not in the project area:

run_2026_10_09_axi_burst/
├── env/
│   ├── env.txt
│   ├── path.txt
│   ├── run.txt
│   ├── tools.txt
│   └── uncommitted.patch      (only when the tree was dirty)
├── axi_burst_test.log
└── compile.log

Three reasons it belongs there:

It is copied with the results. Whatever mechanism moves logs out of scratch moves the capture too, which is the only way it survives Chapter 22.4's nightly purge.

It is found by whoever is looking at the log. A capture in a separate system is a capture nobody consults.

And it is small. Four files of a few kilobytes against a log of hundreds of megabytes — so there is never a reason to leave it out, which is the argument that actually gets it adopted.

The one addition worth making at the flow level: write the capture before the first tool runs, and never overwrite it. A capture rewritten at the end of a run records the environment the run left behind, which is not the one it used.

7. Module 25 in One Page

• 25.1 — five preconditions, and measured statuses 3, 4, 5, 6 and 134 for five distinct refusals. -version needs no licence and halves the search.
• 25.2 — the same command gave 2024.09 and 2025.03 from PATH order alone, library included. type -a sees functions; which cannot.
• 25.3 — the loader speaks, not the tool; status 134 is SIGABRT; macOS strips DYLD_* for system binaries, so the fix vanishes through a #!/bin/sh wrapper.
• 25.4 — a licence source is an ordered search list; four failures, four statuses; and a mid-run loss is the one licence message that appears at the end of a healthy log.
• 25.5 — require what you cannot guess, default what you can, derive the rest; existence is not identity.
• 25.6 — sourced twice, PATH doubles; the obvious guard matches /opt/t inside /opt/tools; module is a shell function that evals printed commands.
• 25.7 — no portable version flag; classify by status, not text; and vvp -V writes to stderr while iverilog -V writes to stdout.
• 25.8 — four files per run, designed for the diff.

Three things recurred in every chapter, and they are the module's real content.

The message often comes from something other than the tool — the shell, the dynamic loader, the licence daemon — and knowing who is speaking eliminates most of the search space before you read a word of the message.

The exit status is an interface and the message text is not. That produced the version-flag table in 25.7, the signal check in 25.1, and the four-way licence split in 25.4; and reading text instead of status is what made three of my own probes wrong.

And a check that cannot fail, or cannot abstain, is worse than none. 25.1's toolcheck reported a tool that could not start as healthy; 25.4's licheck reported a healthy licence as broken. Both were found by running them against a known state — broken in one case, known-good in the other — which is the discipline Chapter 21.8 established and which this module needed eight times.

Module 26 moves from the environment to the inputs: what a filelist is, how it goes stale, and why the list of files a tool compiles is harder to pin down than it looks.

Practice

1. Run env | sort | wc -l on your machine. How many variables, and how many would you recognise?
2. Find a variable your setup script sets without export. Is it invisible to env?
3. List four facts that affect a run and are not in env.
4. Write capture-env and run it. Read all four files.
5. Capture twice from the same shell with no changes and diff. Is the diff empty? If not, add to the volatile filter.
6. Change your locale and capture again. How many lines differ in each file?
7. Switch tool version and capture. Does tools.txt show the library change too?
8. Name a tool that is not installed and confirm NOT FOUND is recorded.
9. Make your tree dirty, capture, and confirm the revision carries +N-modified and the patch is saved.
10. Put a long PATH on one line and diff it against a changed version. Then do it with path.txt. Compare the readability.
11. Export a variable called FAKE_TOKEN and confirm the secret filter removes it. Then name one it would miss.
12. Export a variable holding 10 kB of text and confirm the truncation note.
13. Capture at the start of a flow and again at the end. What changed, and which is the one to keep?

Check Your Understanding

1. What four things does env | sort miss?
2. Why is PATH captured as a separate numbered file?
3. Why record both the path and the version of each tool?
4. Why is a tool recorded as NOT FOUND rather than skipped?
5. Why date -u in ISO form?
6. What does +N-modified mean, and why save the patch as well?
7. Which variables are filtered as volatile, and what is the cost of filtering more?
8. In what order should two captures be compared, and why is env.txt last?
9. Why must two captures be from comparable points in a run?
10. Why is a secret deny-list only a second line of defence?
11. Why does the capture live in the run directory?
12. Why write it before the first tool runs?

Chapter 27.9 puts a capture beside every run and then does the part that proves it sufficient — replaying from it, and detecting when the tool on disk has changed underneath.

8. Remember

• env | sort misses four things: unexported variables, non-variable facts, name resolution, and the source revision.
• Four files: env.txt, path.txt, run.txt, tools.txt — plus uncommitted.patch when the tree was dirty.
• Split PATH into numbered lines, or one change diffs as one enormous line.
• Record each tool's path and version. Neither alone identifies what ran.
• Record a missing tool as NOT FOUND. It is often the most useful line in the file.
• date -u in ISO form, so timestamps order across a farm.
• +N-modified on the revision, and save the patch. A bare revision over uncommitted changes is a false claim.
• Filter only the genuinely volatile — _, SHLVL, PPID, RANDOM, SECONDS, LINENO, OLDPWD.
• The capture is designed for the diff, not for reading: sorted and filtered.
• Measured: 5 changed lines out of 61, all real, between two deliberately different environments.
• Compare in order: tools, run facts, path, then environment.
• Captures must come from comparable points. Write it before the first tool runs, and never overwrite it.
• No secrets. The filter is a second line of defence, not the answer.
• Keep it in the run directory — copied with the results, found by whoever reads the log, and small enough that there is no excuse.

Chapter 30.5 reproduces a scheduler's environment with env -i so a submission can be tested before it is ever submitted.

9. What Is Next

Writing the environment down is what makes a failure reproducible later — and Chapter 32.6 measures the one field that matters most: restoring the source files did not restore the compiled image, and only a recorded digest noticed.

The environment is settled: the tool starts, its libraries resolve, its licence is granted, the project variables are right, and the whole thing is written down. Module 26 turns to what you hand the tool — the filelist — and to the several ways a list of files turns out to be harder to pin down than it looks.