PATH, Tool Homes, Licences and the Environment You Pass

Part II · Module 24 · Chapter 24.1 · PATH, Tool Homes, Licences and the Environment You Pass

1. The Same Command, Two Different Tools

Two directories, each holding an executable called xrun, each a different release:

import os
import subprocess
from pathlib import Path

for version, directory in (("2023.12", "toolsA"), ("2024.09", "toolsB")):
    Path(directory).mkdir(exist_ok=True)
    tool = Path(directory, "xrun")
    tool.write_text(f"#!/bin/sh\necho 'xrun version {version}'\n", encoding="utf-8")
    tool.chmod(0o755)

for directory in ("toolsA", "toolsB"):
    env = {**os.environ, "PATH": str(Path(directory).resolve()) + os.pathsep + os.environ["PATH"]}
    cp = subprocess.run(["xrun"], env=env, capture_output=True, text=True)
    print(f"PATH begins with {directory}:  {cp.stdout.strip()}")

PATH begins with toolsA:  xrun version 2023.12
PATH begins with toolsB:  xrun version 2024.09

The command is byte-for-byte identical in both runs. ["xrun"] — same list, same arguments, same working directory. Two different programs ran.

This is the whole reason Module 24 exists. Everything Chapter 23 built — the list form, the return code, the log, the timeout, the recorded cmd.sh — describes a run precisely and still does not tell you which tool it was. A regression that passed last month and fails today with an identical cmd.sh is very often this, and it looks like a design regression until somebody checks.

import shutil
from pathlib import Path

for directory in ("toolsA", "toolsB"):
    Path(directory).mkdir(exist_ok=True)
    tool = Path(directory, "xrun")
    tool.write_text("#!/bin/sh\necho hi\n", encoding="utf-8")
    tool.chmod(0o755)

a = shutil.which("xrun", path=str(Path("toolsA").resolve()))
b = shutil.which("xrun", path=str(Path("toolsB").resolve()))

print("both resolve         :", a is not None and b is not None)
print("to the same file     :", a == b)
print("same basename though :", Path(a).name == Path(b).name)
print("the name you typed   :", Path(a).name)

both resolve         : True
to the same file     : False
same basename though : True
the name you typed   : xrun

shutil.which from Chapter 23.9 takes an explicit path=, which makes this measurable: two different files, one name. The name is what your command carries, and the name is the part that does not identify anything.

2. Order Decides, Not Membership

import os
import subprocess
from pathlib import Path

for version, directory in (("2023.12", "toolsA"), ("2024.09", "toolsB")):
    Path(directory).mkdir(exist_ok=True)
    tool = Path(directory, "xrun")
    tool.write_text(f"#!/bin/sh\necho 'xrun version {version}'\n", encoding="utf-8")
    tool.chmod(0o755)

a, b = str(Path("toolsA").resolve()), str(Path("toolsB").resolve())

for label, prefix in (("A then B", a + os.pathsep + b), ("B then A", b + os.pathsep + a)):
    env = {**os.environ, "PATH": prefix + os.pathsep + os.environ["PATH"]}
    cp = subprocess.run(["xrun"], env=env, capture_output=True, text=True)
    print(f"{label}:  {cp.stdout.strip()}")

print()
print("both directories are on PATH in both runs — only the order changed")
print("the separator on this platform:", repr(os.pathsep))

A then B:  xrun version 2023.12
B then A:  xrun version 2024.09

both directories are on PATH in both runs — only the order changed
the separator on this platform: ':'

Both installations were on PATH every time. Membership decided nothing; position decided everything.

That is worth stating as a rule, because the usual mental model is the wrong one:

PATH is not a set of places the tool might be. It is an ordered search, and the first match wins.

The practical consequence is that adding a tool directory to PATH is not a safe operation. Prepending shadows anything later with the same name; appending is shadowed by anything earlier. A setup script that prepends a 2024.09 installation changes which simulator every later command gets, including ones nobody was thinking about.

os.pathsep is the separator — : on Unix-likes and ; on Windows. Use the constant rather than typing ":", for the same reason Chapter 18.x used Path over string joining.

3. env= Replaces; It Does Not Merge

import os
import subprocess
import sys

cp = subprocess.run([sys.executable, "-c", "import os; print('PATH' in os.environ)"],
                    env={"MY_VAR": "1"}, capture_output=True, text=True)

print("child has PATH :", cp.stdout.strip())
print("parent has PATH:", "PATH" in os.environ)

child has PATH : False
parent has PATH: True

env= is the complete environment the child gets, not an addition to yours. Passing env={"MY_VAR": "1"} does not mean "and also everything else" — it means "this, and nothing else".

That is the single most common mistake with this parameter, and its symptom is confusing: the tool launches and then fails somewhere deep inside, complaining about a library or a licence, because a variable it has always had is suddenly absent.

It gets worse when the thing missing is PATH itself:

import os
import subprocess
import sys
from pathlib import Path

Path("toolsA").mkdir(exist_ok=True)
tool = Path("toolsA", "xrun")
tool.write_text("#!/bin/sh\necho 'xrun ran'\n", encoding="utf-8")
tool.chmod(0o755)

try:
    subprocess.run(["xrun"], env={}, capture_output=True)
except FileNotFoundError as exc:
    print("bare name, env={} :", type(exc).__name__, "-", exc.filename)

cp = subprocess.run([str(tool.resolve())], env={}, capture_output=True, text=True)
print("absolute, env={}  :", cp.returncode, repr(cp.stdout.strip()))

bare name, env={} : FileNotFoundError - xrun
absolute, env={}  : 0 'xrun ran'

Those two lines together are the cleanest statement of what PATH is. With no PATH, the bare name raises Chapter 23.9's FileNotFoundError — reached here by a completely different route, since the tool exists and is executable and there is simply nowhere to look. The absolute path still runs, because an absolute path needs no search.

So PATH is a lookup mechanism and nothing more. It does not grant access, it does not configure anything; it answers one question, where is the file called this?, and it answers it in order.

And env={} is not quite empty:

import subprocess
import sys

COUNT = "import os; print(len(os.environ))"
empty = int(subprocess.run([sys.executable, "-c", COUNT], env={},
                           capture_output=True, text=True).stdout)
inherited = int(subprocess.run([sys.executable, "-c", COUNT],
                               capture_output=True, text=True).stdout)

print("env={} gives the child zero keys :", empty == 0)
print("it gives a small handful         :", 0 < empty < 5)
print("far fewer than inheriting        :", empty < inherited // 3)

env={} gives the child zero keys : False
it gives a small handful         : True
far fewer than inheriting        : True

The system injects a couple of keys of its own whatever you pass. The exact set is platform-specific, which is why this prints comparisons rather than the counts — the counts would be a fact about this machine, not about Python.

4. Build It From a Copy

import os
import subprocess
import sys

lic = "27000@licsrv-a.example.com"

env = {**os.environ, "SNPSLMD_LICENSE_FILE": lic}

CHECK = ("import os; print('PATH present:', 'PATH' in os.environ);"
         " print('licence present:', 'SNPSLMD_LICENSE_FILE' in os.environ);"
         " print('licence value length:', len(os.environ.get('SNPSLMD_LICENSE_FILE', '')))")

cp = subprocess.run([sys.executable, "-c", CHECK], env=env, capture_output=True, text=True)
print(cp.stdout.strip())
print()
print("parent unaffected:", "SNPSLMD_LICENSE_FILE" in os.environ)

PATH present: True
licence present: True
licence value length: 26

parent unaffected: False

That is the canonical annotation:

env = {**os.environ, "SNPSLMD_LICENSE_FILE": lic}
subprocess.run(cmd, cwd=run_dir, env=env)

{**os.environ, "KEY": value} builds a new dictionary: every inherited key, then your overrides on top. Later keys win, so naming a key that already exists replaces it.

Two properties make this the right idiom rather than merely a working one.

It does not modify os.environ. The parent is untouched — the last line proves it — so a loop that gives four hundred tests four hundred different environments leaves the script itself exactly as it started. Compare Chapter 23.7's argument for cwd= over os.chdir: same shape, same reason, and here too there is no state to restore.

It is explicit about what changed. The one line names every variable this run overrode, which is both the code and the documentation. A setup script that exports eleven variables somewhere else gives you neither.

Notice also what the child printed: the licence variable's presence and the length of its value, never the value. That is deliberate and Section 8 is about why.

5. Where the Decision Lives

Read it right to left and the point is sharper. The binary that runs is determined by the dictionary; the dictionary is determined by your code; therefore which tool runs is a decision your script makes — or, if you never write that code, a decision somebody's shell made for you.

6. Two Rules About Values

import os
import subprocess
import sys

try:
    subprocess.run([sys.executable, "-c", "pass"],
                   env={**os.environ, "SEED": 1839274})
except TypeError as exc:
    print("TypeError:", exc)

cp = subprocess.run([sys.executable, "-c", "import os; print(os.environ['SEED'])"],
                    env={**os.environ, "SEED": str(1839274)},
                    capture_output=True, text=True)
print("as a string:", cp.stdout.strip())

TypeError: expected str, bytes or os.PathLike object, not int
as a string: 1839274

Every environment value must be a string. An environment is a mapping of strings to strings at the operating-system level, and Python declines to guess how you want a number spelled. A seed, a timeout or a thread count is exactly the realistic way to hit this, and str() is the whole fix.

The second rule is Chapter 23.1 Section 4's one-way boundary, in this new setting:

import os
import subprocess
import sys

subprocess.run([sys.executable, "-c", "import os; os.environ['CHILD_SET'] = 'yes'"])
print("parent sees CHILD_SET  :", "CHILD_SET" in os.environ)

os.environ["SET_BY_PARENT"] = "yes"
cp = subprocess.run([sys.executable, "-c",
                     "import os; print(os.environ.get('SET_BY_PARENT'))"],
                    capture_output=True, text=True)
print("child sees SET_BY_PARENT:", cp.stdout.strip())

GRAND = ("import subprocess, sys; subprocess.run([sys.executable, '-c',"
         " \"import os; print(os.environ.get('SET_BY_PARENT'))\"])")
cp = subprocess.run([sys.executable, "-c", GRAND], capture_output=True, text=True)
print("grandchild sees it too  :", cp.stdout.strip())

parent sees CHILD_SET  : False
child sees SET_BY_PARENT: yes
grandchild sees it too  : yes

Environment flows downward only, and it flows all the way down. A child cannot hand a variable back — which is why source setup.sh as its own subprocess.run accomplishes nothing, the point Chapter 23.1 Section 4 made and Chapter 23.3 Section 5 answered.

The third line is the half people forget: a variable reaches the grandchild too. An EDA tool that launches a compiler, which launches a linker, passes everything down the whole tree. That is what makes the environment powerful and also what makes a stray variable so hard to trace.

7. The Variables an EDA Flow Actually Cares About

import os

TOOL_ENV = {
    "PATH": "where every bare tool name is looked up, in order",
    "LD_LIBRARY_PATH": "where the tool's shared libraries are found at run time",
    "VCS_HOME": "the Synopsys VCS installation root",
    "XCELIUM_HOME": "the Cadence Xcelium installation root",
    "UVM_HOME": "the UVM library the testbench compiles against",
    "SNPSLMD_LICENSE_FILE": "Synopsys licence server (a port@host, or a file)",
    "LM_LICENSE_FILE": "the generic FlexLM licence setting",
    "DISPLAY": "needed by a GUI tool, and a classic batch-run failure",
    "TMPDIR": "where the tool puts large temporary data",
}

print(f"{'variable':<22} {'set here':>8}   what it decides")
for name, meaning in TOOL_ENV.items():
    print(f"{name:<22} {str(name in os.environ):>8}   {meaning}")
print()
print("set on this machine:", sum(1 for k in TOOL_ENV if k in os.environ), "of", len(TOOL_ENV))

variable               set here   what it decides
PATH                       True   where every bare tool name is looked up, in order
LD_LIBRARY_PATH           False   where the tool's shared libraries are found at run time
VCS_HOME                  False   the Synopsys VCS installation root
XCELIUM_HOME              False   the Cadence Xcelium installation root
UVM_HOME                  False   the UVM library the testbench compiles against
SNPSLMD_LICENSE_FILE      False   Synopsys licence server (a port@host, or a file)
LM_LICENSE_FILE           False   the generic FlexLM licence setting
DISPLAY                   False   needed by a GUI tool, and a classic batch-run failure
TMPDIR                     True   where the tool puts large temporary data

set on this machine: 2 of 9

Only PATH and TMPDIR are set here, because no EDA tool is installed on this machine — the standing convention from Chapter 23.1 Section 1. On a real workstation most of the rest would be populated, and the column that matters is still the boolean, not the value.

Three of those deserve a sentence each.

LD_LIBRARY_PATH is PATH for shared libraries, with the same ordered-search semantics and the same shadowing hazard. A tool that launches and then dies complaining about a missing or mismatched .so is usually this rather than a broken install.

DISPLAY is the classic batch failure. A tool that works interactively and hangs or dies under a scheduler is very often trying to open a window. The fix is a batch flag — Chapter 23.11's vsim -c is exactly that — not a DISPLAY value.

The licence variables are the ones that carry something sensitive, and that is the next section.

8. Never Print the Value

import os

SENSITIVE = ("LICENSE", "LICENCE", "TOKEN", "SECRET", "KEY", "PASSWORD", "CREDENTIAL")


def redact(mapping, keys):
    """Report what is set, never what it is set to."""
    out = {}
    for key in keys:
        if key not in mapping:
            out[key] = {"set": False}
            continue
        value = mapping[key]
        sensitive = any(marker in key.upper() for marker in SENSITIVE)
        out[key] = {"set": True, "sensitive": sensitive,
                    "length": len(value),
                    "value": "<redacted>" if sensitive else value}
    return out


pretend = {"UVM_HOME": "/tools/uvm-1.2", "SNPSLMD_LICENSE_FILE": "27000@licsrv-a",
           "ARTIFACTORY_TOKEN": "not-a-real-token-abcdef"}

for key, info in redact(pretend, ["UVM_HOME", "SNPSLMD_LICENSE_FILE",
                                  "ARTIFACTORY_TOKEN", "VCS_HOME"]).items():
    print(f"{key:<22} {info}")

UVM_HOME               {'set': True, 'sensitive': False, 'length': 14, 'value': '/tools/uvm-1.2'}
SNPSLMD_LICENSE_FILE   {'set': True, 'sensitive': True, 'length': 14, 'value': '<redacted>'}
ARTIFACTORY_TOKEN      {'set': True, 'sensitive': True, 'length': 23, 'value': '<redacted>'}
VCS_HOME               {'set': False}

A run record is written into a run directory, and a run directory is shared, archived, attached to tickets and copied into bug reports. An environment dump in a run record is a credential leak with a long half-life, and it is an easy one to create by accident, because json.dumps(dict(os.environ)) is one obvious line.

Three rules make the record useful without making it dangerous.

Record an allow-list, never the whole environment. The variables that decide a run are a short named list — Section 7's table is most of it. Everything else is noise, and some of it is secret.

Default to redacting, and let the allow-list opt a value in. The helper above keys off the name, so a variable nobody anticipated with TOKEN in it is redacted without anyone having thought about it. A list of things to hide fails on the first unanticipated name; a rule that hides by default does not.

A length is usually enough to debug with. "Set, 26 characters" distinguishes unset, set to the empty string and set to something plausible, which covers nearly every environment question you actually have. Chapter 24.6 builds the full record on exactly this.

9. One Deliberate Environment

import os
import subprocess
import sys
from pathlib import Path

REQUIRED = ("PATH",)
TOOL_HOMES = {"UVM_HOME": "uvm-1.2"}


def build_env(tool_dir, licence=None, extra=None):
    """Return the complete environment for one run, built from a copy."""
    env = {**os.environ}

    tool_dir = str(Path(tool_dir).resolve())
    env["PATH"] = tool_dir + os.pathsep + env.get("PATH", "")

    for name, value in TOOL_HOMES.items():
        env[name] = str(Path(value).resolve())

    if licence is not None:
        env["SNPSLMD_LICENSE_FILE"] = licence

    for name, value in (extra or {}).items():
        env[name] = str(value)

    missing = [name for name in REQUIRED if not env.get(name)]
    if missing:
        raise ValueError(f"refusing to launch with empty {', '.join(missing)}")
    return env


Path("toolsB").mkdir(exist_ok=True)
tool = Path("toolsB", "xrun")
tool.write_text("#!/bin/sh\necho 'xrun version 2024.09'\n", encoding="utf-8")
tool.chmod(0o755)

env = build_env("toolsB", licence="27000@licsrv-a", extra={"SEED": 1839274})

cp = subprocess.run(["xrun"], env=env, capture_output=True, text=True)
print("tool said          :", cp.stdout.strip())
print("PATH was prepended :", env["PATH"].split(os.pathsep)[0].endswith("toolsB"))
print("SEED is a string   :", isinstance(env["SEED"], str), repr(env["SEED"]))
print("keys we overrode   :", sorted(k for k in ("PATH", "UVM_HOME", "SNPSLMD_LICENSE_FILE", "SEED")))
print("parent untouched   :", "SEED" not in os.environ)

try:
    build_env("toolsB", extra={"PATH": ""})
except ValueError as exc:
    print("guard              :", exc)

tool said          : xrun version 2024.09
PATH was prepended : True
SEED is a string   : True '1839274'
keys we overrode   : ['PATH', 'SEED', 'SNPSLMD_LICENSE_FILE', 'UVM_HOME']
parent untouched   : True
guard              : refusing to launch with empty PATH

Four decisions in that function are worth copying.

It returns an environment; it does not run anything. Same rule as Chapter 23.11 Section 8's adapters — a function that builds a value can be tested by asserting on the value, with no tool installed.

It prepends rather than replaces PATH. Replacing it outright would be more deterministic and would also break every ordinary command the tool itself shells out to. Prepending puts your choice first, which Section 2 showed is what actually decides.

It coerces every extra value with str(), so Section 6's TypeError cannot reach a caller who passed a seed as an integer.

It refuses to launch with an empty PATH. An empty string is not a missing key — env.get("PATH") returns "", which is truthy-adjacent enough to slip through a in env test and produces Section 3's confusing FileNotFoundError much later. Checking for empty rather than absent is the version that catches it.

10. Predict, Run, Observe

import os
import shutil
import subprocess
import sys
from pathlib import Path

Path("toolsA").mkdir(exist_ok=True)
t = Path("toolsA", "mytool")
t.write_text("#!/bin/sh\necho ran\n", encoding="utf-8")
t.chmod(0o755)
A = str(Path("toolsA").resolve())

# A
print("A:", shutil.which("mytool"))

# B
print("B:", shutil.which("mytool", path=A) is not None)

# C
cp = subprocess.run([sys.executable, "-c", "import os; print('PATH' in os.environ)"],
                    env={"X": "1"}, capture_output=True, text=True)
print("C:", cp.stdout.strip())

# D
try:
    subprocess.run(["mytool"], env={}, capture_output=True)
except OSError as exc:
    print("D:", type(exc).__name__)

# E
cp = subprocess.run([str(t.resolve())], env={}, capture_output=True, text=True)
print("E:", cp.returncode, repr(cp.stdout.strip()))

# F
try:
    subprocess.run([sys.executable, "-c", "pass"], env={**os.environ, "N": 7})
except TypeError:
    print("F: TypeError")

# G
env = {**os.environ, "PATH": A + os.pathsep + os.environ["PATH"]}
cp = subprocess.run(["mytool"], env=env, capture_output=True, text=True)
print("G:", cp.stdout.strip(), "| parent changed:", "mytool" in os.environ.get("PATH", ""))

# H
print("H:", repr(os.pathsep))

The verified output:

A: None
B: True
C: False
D: FileNotFoundError
E: 0 'ran'
F: TypeError
G: ran | parent changed: False
H: ':'

A — the tool is not on the inherited PATH, so which says None.

B — pointed at the directory explicitly, it is found. The tool never moved.

C — env= replaced the environment, so the child has no PATH.

D — a bare name with no PATH is Chapter 23.9's FileNotFoundError.

E — the same tool by absolute path runs fine with the same empty environment.

F — an int value is a TypeError.

G — prepending the directory makes the bare name work, and the parent's own PATH is untouched.

H — use os.pathsep, not a literal.

11. Exercises

Exercise 1 — two tools, one name. Build two directories with the same tool name and different output. Run the identical command twice and get both.

Exercise 2 — order, not membership. Put both on PATH and swap only the order. Record what changed.

Exercise 3 — state the rule. Write the one-sentence rule about PATH in your own words, and say what it implies about adding a directory to a shared setup script.

Exercise 4 — which, explicitly. Use shutil.which(..., path=...) to resolve both installations and show the paths differ while the basenames match.

Exercise 5 — replacement. Pass env={"X": "1"} and show the child has no PATH. Say what symptom a real tool would show.

Exercise 6 — the lookup proof. With env={}, run the same tool by bare name and by absolute path. Explain the two results in one sentence each.

Exercise 7 — not empty. Show that env={} does not give the child zero variables, and say why your check should be a comparison rather than a count.

Exercise 8 — the merge. Use {**os.environ, ...} and confirm both that the override took effect and that os.environ did not change.

Exercise 9 — the parallel. Say why {**os.environ, ...} is to os.environ[...] = ... as Chapter 23.7's cwd= is to os.chdir.

Exercise 10 — strings only. Pass a seed as an int and record the exception. Fix it.

Exercise 11 — downward only. Show a variable reaching a grandchild, and show a child's own variable not reaching the parent.

Exercise 12 — your flow's variables. List every environment variable your flow depends on and what each decides. Mark which are set on your machine.

Exercise 13 — the library path. Explain why a tool that launches and then dies about a missing .so is usually LD_LIBRARY_PATH.

Exercise 14 — the batch failure. Describe a tool working interactively and failing under a scheduler, and say why setting DISPLAY is the wrong fix.

Exercise 15 — redact. Write a redactor that hides by name and reports a length. Say why hiding by default beats a list of things to hide.

Exercise 16 — the second leak. Name the non-print route by which a secret reaches a log file, and say what follows for where you store such values.

Exercise 17 — build it. Write a build_env that prepends PATH, coerces values, and refuses an empty PATH. Say why empty rather than absent is the right check.

Exercise 18 — testable. Write a test for build_env that needs no EDA tool installed.

12. Summary

• The same command can run a different binary, decided entirely by the environment — which is the gap Chapter 23.10 Section 7 named.
• PATH is an ordered search, not a set of places. The first match wins, so order decides and membership does not.
• Adding a directory to PATH is not a safe operation; prepending shadows every later tool of the same name.
• Use os.pathsep, not a literal ":".
• env= replaces the whole environment — it is what the child gets, not an addition to what you have.
• With no PATH, a bare name raises FileNotFoundError and an absolute path still runs, which is the cleanest proof that PATH is only a lookup.
• env={} is not zero variables; the system injects a couple, and the exact set is platform-specific.
• Build the environment as {**os.environ, "KEY": value} — a copy plus overrides, which leaves os.environ untouched exactly as cwd= leaves the working directory untouched.
• Every value must be a string; an int raises TypeError, and a seed is the realistic way to meet it.
• Environment flows downward only, and all the way down — a child cannot hand one back, and a grandchild inherits it.
• LD_LIBRARY_PATH is PATH for shared libraries, with the same shadowing hazard.
• DISPLAY is the classic interactive-works, batch-fails cause, and the fix is a batch flag rather than a value.
• Never record an environment value you have not deliberately allowed. A run directory is shared and archived; json.dumps(dict(os.environ)) is a one-line credential leak.
• Redact by name and report a length — hiding by default survives a variable nobody anticipated, and a length answers nearly every real question.
• A secret reaches a log through a traceback as easily as a print, so keep it out of your own data structures and pass it through env=.
• Build the environment in a function that returns it and runs nothing, prepend rather than replace, coerce with str(), and refuse an empty PATH rather than an absent one.

The next chapter records which tool version that environment actually produced, so the question this chapter opened with can be answered from the run record alone.