Linux & Bash · Part II · Module 25 · EDA Environment and Tool Access

Setup Scripts: Sourcing, Modules, and Idempotence

Every project has one script everyone sources, and almost every one of them is not safe to source twice. This chapter is why sourcing is required rather than preferred, what an environment-module system adds, and the measured failure of the guard that most people write to make a setup script idempotent.

Intermediate22 min readLinuxPart IISourcingPATHIdempotence

Part II · Module 25 · Chapter 25.6 · Setup Scripts: Sourcing, Modules, and Idempotence

source /proj/env/setup.sh is the first thing a new engineer is told to type and the last thing anyone reads. It sets PATH, licence variables, project variables and often several hundred others — and it is sourced again every time someone opens a terminal, changes project, or runs a script that sources it defensively.

1. Why Sourcing, and Not Running

What sourcing does that running cannot

Running a script creates a child process: it inherits your environment, changes its own copy, and the copy is destroyed when it exits. Sourcing executes the same lines in your own shell, so the changes persist. That is not a preference — a child cannot modify its parent's environment at all, so a setup script that is run instead of sourced appears to do nothing and reports no error.

Chapter 10.4 measured this, and the fact that makes it non-negotiable is Chapter 7.4's: no child process can modify its parent's environment. There is no mechanism. So a setup script that is executed rather than sourced is not merely less effective — it has no effect at all, and it exits 0.

Which is why a setup script should refuse to be executed, as Chapter 25.5 §7's fragment does:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

case ${0##*/} in
  (setup.sh) printf 'setup.sh must be sourced, not executed\n' >&2; exit 2 ;;
esac

Executed, $0 is setup.sh and the guard fires. Sourced, $0 is bash or -bash and it does not. Three lines, and it replaces the most common ten minutes of confusion in this module.

2. Sourced Twice, and PATH Doubles

Here is the problem this chapter exists for. The obvious way to add a tool to PATH:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

export PATH=/opt/t/bin:$PATH

Source the script containing it twice — which happens when you open a terminal that sources it from .bashrc and then run a flow that sources it again — and:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ bash -c 'P=/opt/t/bin
> export PATH=$P:$PATH
> export PATH=$P:$PATH
> echo "$PATH" | tr ":" "\n" | grep -c "^/opt/t/bin$"'
2

Two copies. Five sources give five, and a real PATH after a day of switching projects reaches fifty entries:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ bash -c 'P=/usr/bin:/bin
> for i in 1 2 3 4 5; do P=/opt/t/bin:$P; done
> echo "$P" | tr ":" "\n" | wc -l'
7

Seven entries where two were needed. Three costs follow, in increasing order of seriousness:

Every command lookup is slower, because the shell stats each directory in turn. Measurable only at scale, and genuinely visible in a flow that runs ten thousand short commands.

The list becomes unreadable, so nobody can tell which tool version is actually first.

And the order becomes unpredictable when two different setup scripts each prepend a different version. That is the real cost: a duplicate is harmless, but the pattern that creates duplicates is the same pattern that silently changes which version wins.

3. The Guard, and Its Bug

The standard fix is to add only if absent:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ bash -c 'add() { case ":$PATH:" in *":$1:"*) ;; *) PATH=$1:$PATH;; esac; }
> add /opt/t/bin; add /opt/t/bin
> echo "$PATH" | tr ":" "\n" | grep -c "^/opt/t/bin$"'
1

One copy, sourced twice. And the obvious version of that guard is wrong, in a way that is silent and specific:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ bash -c 'PATH=/opt/tools/bin:/usr/bin
> case "$PATH" in *"/opt/t"*) echo "MATCHED — would skip wrongly";; *) echo no;; esac'
MATCHED — would skip wrongly

/opt/t is a substring of /opt/tools/bin, so a guard written without delimiters decides the directory is already present and does not add it. The tool is then not on PATH, and the setup script reports success.

The colons fix it:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ bash -c 'PATH=/opt/tools/bin:/usr/bin
> case ":$PATH:" in *":/opt/t:"*) echo MATCHED;; *) echo "correctly did not match";; esac'
correctly did not match

Both sides need wrapping. ":$PATH:" guarantees every entry is bounded by colons, including the first and last, and *":$1:"* requires the match to be a whole entry. Leave the colon off either side and the first or last entry escapes the check.

Three more things the guard has to decide

Prepend or append? Prepending wins over the system tools, which is what you want for a tool version. Appending is for a fallback. A path_add that always prepends is fine; one that silently appends when you expected prepend is the source of "I set it and got the old version".

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

path_prepend() { case ":${PATH}:" in (*":$1:"*) ;; (*) PATH=$1${PATH:+:$PATH} ;; esac; }
path_append()  { case ":${PATH}:" in (*":$1:"*) ;; (*) PATH=${PATH:+$PATH:}$1 ;; esac; }

${PATH:+:$PATH} rather than :$PATH — so that if PATH is somehow empty, you get /opt/t/bin and not /opt/t/bin:, and a trailing colon means the current directory, which is a security problem and a source of baffling behaviour. Chapter 10.2 §7 measured that an empty PATH element is ..

Should it move an existing entry to the front? The guard above does not: if the directory is already present at position twelve, it stays there. For a version switch that is wrong — you wanted the new version first. The honest version removes and re-adds:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

path_force_front() {
  path_remove "$1"
  PATH=$1${PATH:+:$PATH}
}
path_remove() {
  local out= d
  IFS=: ; for d in $PATH; do [ "$d" = "$1" ] || out=${out:+$out:}$d ; done ; unset IFS
  PATH=$out
}

IFS=: with an unquoted $PATH is deliberate word splitting — the one case Chapter 9.2 calls legitimate — and it needs a comment saying so, because every reviewer will flag it.

And should it check the directory exists? Usually yes, because a non-existent PATH entry is pure cost: every lookup stats it and fails. A setup script that has accumulated ten stale entries from retired tool versions is slowing every command for no benefit.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

path_prepend() {
  [ -d "$1" ] || { printf 'path_prepend: no such directory: %s\n' "$1" >&2; return 1; }
  case ":${PATH}:" in (*":$1:"*) ;; (*) PATH=$1${PATH:+:$PATH} ;; esac
}

Returning non-zero rather than silently skipping is what lets a setup script notice that a tool installation has been removed — which is the real event behind "it worked last week".

4. Deduplicating a PATH That Is Already Doubled

Two forms, both measured, and the choice is about whether you mind forking:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ bash -c 'P=/a:/b:/a:/c:/b
> echo "$P" | tr ":" "\n" | awk "!seen[\$0]++" | paste -sd: -'
/a:/b:/c

awk '!seen[$0]++' is the canonical first-occurrence filter: the array entry is 0 (false) the first time, !0 is true so the line prints, and the ++ makes it 1 thereafter. Chapter 13.7 is awk's arrays; this is the one idiom worth memorising outright.

And in pure shell, which forks nothing and is what belongs in a sourced setup script:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ bash -c '
> P=/a:/b:/a:/c:/b
> out=
> IFS=:
> for d in $P; do
>   case ":$out:" in (*":$d:"*) continue ;; esac
>   out=${out:+$out:}$d
> done
> unset IFS
> echo "$out"'
/a:/b:/c

Both give /a:/b:/c — first occurrence kept, order otherwise preserved. Keeping the first is the right policy: it is the one that was winning, so deduplication does not change which tool you get.

Three details in the shell version: IFS=: then unset IFS rather than IFS=: read, because this is a for loop over the whole value; ${out:+$out:} to avoid a leading colon on the first iteration; and the parenthesised case pattern, for the Bash 3.2 reason Chapter 25.1's callout measured — which matters here because a setup script is frequently sourced from inside a $( ).

Never experiment on PATH — measured

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ bash -c 'PATH=/a:/b:/a; tr --version'
bash: tr: command not found

Assigning a test value to PATH removes every external command, including the ones you were about to use to check the result. Interactively this looks like the shell has lost ls, grep and cat at once.

So in this chapter's own examples the variable is P, not PATH, and the assignment to PATH happens only when the value is known good. The other safe habit is a subshell:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

( export PATH=/opt/t/bin:$PATH; command -v simtool )

The damage cannot outlive the parentheses — Chapter 11.4. And the recovery, if you do it anyway, is worth memorising because you will not be able to look it up:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

PATH=/usr/bin:/bin

Enough to get env, ls and an editor back, and then re-source the setup script. Chapter 25.2 §4 made the same point from the diagnostic side; it is worth making twice because this is the variable where a mistake removes the means of repair.

5. Environment-Module Systems

Large sites do not hand-write PATH manipulation. They use an environment-module system — environment-modules or Lmod — and the interface is a small set of commands:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

module avail                 # what is installed
module load vcs/2023.03      # put a version on PATH and set its variables
module list                  # what is loaded now
module unload vcs            # undo it
module switch vcs/2023.03 vcs/2024.09
module show vcs/2023.03      # what it WOULD do, without doing it

None of these is installed here:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ for t in module modulecmd lmod; do command -v $t > /dev/null || echo "$t absent"; done
module absent
modulecmd absent
lmod absent
$ type module
bash: type: module: not found

So the commands above are quoted from their documentation, not demonstrated. What is worth knowing without a system to try it on is the mechanism, because it explains the one thing about module that confuses everybody:

module is a shell function, not a program. It has to be, for §1's reason: a program cannot change your PATH. What the function does is run a helper that prints shell commands, and then eval them:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

module() { eval "$(modulecmd bash "$@")"; }

Three consequences follow directly, and all three are real support questions:

module does not work in a script that did not source the init file, because the function does not exist there. The fix is to source /usr/share/Modules/init/bash (or the site's equivalent) at the top of the script — the same requirement as any other function library, which is Chapter 18.6.

module load in a subshell has no effect on you. $(module load vcs) runs the function in a child, and the child's PATH dies with it — §1 again, one level down.

And module show is the diagnostic to reach for, because it prints the variable changes without applying them. That is the closest thing in this module to a dry run, and Chapter 21.5's argument applies: a tool that can tell you what it would do is one you can trust with your environment.

What a module system buys, and what it does not

It buys: a single vocabulary across every tool at a site; unload and switch, which a hand-written setup script almost never implements; conflict declarations, so loading two versions of one tool is refused rather than silently resolved by PATH order; and a record of what is loaded, which module list prints and which is exactly what Chapter 25.8 wants to capture.

It does not buy idempotence of your scripts. A flow that sources a project setup script which module loads a tool still has every problem in §2 and §3 — the module system is idempotent about modules, and knows nothing about the PATH entry your own script prepended.

Nor does it remove the need for Chapter 25.2's verification. module load vcs/2023.03 succeeding is not proof that command -v vcs resolves to that version: a hand-edited PATH entry earlier in the list still wins, and the module system cannot see it. module list tells you what you asked for; command -v and -version tell you what you got, and when they disagree the second pair is right.

6. A Setup Script That Is Safe to Source Twice

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

# setup.sh — sourced, not executed. Idempotent: sourcing it twice changes nothing.

# 0. refuse execution
case ${0##*/} in
  (setup.sh) printf 'setup.sh must be sourced: use ". setup.sh"\n' >&2; exit 2 ;;
esac

# 1. path helpers, defined every time (cheap, and survives being unset)
_path_has()     { case ":${PATH}:" in (*":$1:"*) return 0 ;; esac; return 1; }
_path_prepend() {
  [ -d "$1" ] || { printf 'setup: no such directory: %s\n' "$1" >&2; return 1; }
  _path_has "$1" && return 0
  PATH=$1${PATH:+:$PATH}
}

# 2. project variables ([Chapter 25.5](/linux-bash/project-environment-variables))
_here=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P) || return 2
PROJECT_ROOT=${PROJECT_ROOT:-$(cd -- "$_here/.." && pwd -P)}
[ -d "$PROJECT_ROOT" ] || { printf 'setup: no project root: %s\n' "$PROJECT_ROOT" >&2; return 2; }
PROJECT_NAME=${PROJECT_NAME:-${PROJECT_ROOT##*/}}
SCRATCH_ROOT=${SCRATCH_ROOT:-${TMPDIR:-/tmp}}

# 3. the tool, by version, with a check
SIMTOOL_VERSION=${SIMTOOL_VERSION:-2024.09}
SIMTOOL_HOME=$PROJECT_ROOT/tools/simtool-$SIMTOOL_VERSION
if [ ! -x "$SIMTOOL_HOME/bin/simtool" ]; then
  printf 'setup: simtool %s is not installed at %s\n' "$SIMTOOL_VERSION" "$SIMTOOL_HOME" >&2
  return 2
fi
_path_prepend "$SIMTOOL_HOME/bin" || return 2
SIMTOOL_BIN=$SIMTOOL_HOME/bin/simtool

# 4. the licence, if the site has not already set it
SIMTOOL_LICENSE_FILE=${SIMTOOL_LICENSE_FILE:-$PROJECT_ROOT/licenses/site.lic}

export PROJECT_ROOT PROJECT_NAME SCRATCH_ROOT
export SIMTOOL_VERSION SIMTOOL_HOME SIMTOOL_BIN SIMTOOL_LICENSE_FILE
unset _here

printf 'setup: %s  simtool %s  (%s)\n' \
  "$PROJECT_NAME" "$SIMTOOL_VERSION" "$("$SIMTOOL_BIN" -version 2>/dev/null | head -1)"

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ . ./setup.sh
setup: eda  simtool 2024.09  (simtool 2024.09  (simlib 1.4.2))
$ echo "$PATH" | tr ':' '\n' | grep -c 'simtool-2024.09/bin'
1
$ . ./setup.sh
setup: eda  simtool 2024.09  (simtool 2024.09  (simlib 1.4.2))
$ echo "$PATH" | tr ':' '\n' | grep -c 'simtool-2024.09/bin'
1

Sourced twice, one entry. And switching version works because the new directory is prepended and the old one is still further down — which is a deliberate choice, not an oversight: a _path_force_front would be needed to guarantee order, and §3's callout has it.

Seven decisions:

exit in the execution guard, return everywhere else. The guard runs only when the script was executed, where exit is correct; every other error path runs when sourced, where exit would close the user's terminal — Chapter 18.3.

The helpers are redefined every source. Guarding their definition with command -v _path_has || ... would save nothing measurable and would mean a stale definition survives an edit to the script, which is worse.

_path_has returns a status and prints nothing, so it composes: _path_has x && .... A function that printed "yes" would be unusable in a condition — Chapter 18.4's output-versus-status distinction.

The tool check is -x on the executable, not -d on the directory. A half-extracted installation has the directory and not the binary, and that is a real state after an interrupted copy.

SIMTOOL_LICENSE_FILE uses :-, so the site's value wins. A setup script that unconditionally sets a licence source overrides a site configuration that was probably more correct than the project's.

SIMTOOL_BIN is exported so the flow can use an absolute path — Chapter 25.2 §5 — and so Chapter 25.8 can record which executable ran.

And the one printed line includes the tool's own version string, which turns the setup script into a verification: if that parenthesis is empty, the tool did not run, and you know before you start work rather than after.

Practice

Find your project's setup script. Does it guard against being executed?
Execute it rather than sourcing it. What happens, and what is the exit status?
Source it twice and count a PATH entry before and after.
Write the naive export PATH=$P:$PATH twice and confirm you get two copies.
Write the guard without colons and construct a directory pair where it skips wrongly.
Add the colons and confirm the same pair now works.
Remove the colon from one side only and find the entry that escapes the check.
Set PATH to a single directory in a subshell and try to run ls. Then leave the subshell.
Deduplicate a doubled PATH with awk '!seen[$0]++' and with the pure-shell loop. Do they agree?
Count the non-existent entries in your PATH. How many retired tool versions?
Write setup.sh and source it twice. Then set SIMTOOL_VERSION to a version that does not exist and confirm it returns 2 without changing PATH.
If you have a module system, run module show for a tool and compare it with module load followed by env | diff.
Run module list and command -v for the same tool. Do they agree?

Check Your Understanding

Why must a setup script be sourced rather than run?
How does case ${0##*/} distinguish the two?
What happens to PATH when a naive setup script is sourced twice, and what are the three costs?
What is wrong with case "$PATH" in *"/opt/t"*)?
Why must both sides be wrapped in colons?
Why ${PATH:+:$PATH} rather than :$PATH?
Why does the standard guard not move an existing entry to the front, and when does that matter?
Why check that a PATH directory exists?
Why is module a shell function rather than a program?
What does module load in a $( ) achieve?
What does module list tell you that command -v does not, and which is authoritative?
Why does the setup script use exit in one place and return everywhere else?

Answers

1. Because no child process can modify its parent's environment, so an executed setup script changes a copy that is destroyed when it exits — and exits 0 while doing nothing. 2. Executed, $0 is the script's own file name and the pattern matches; sourced, $0 is the shell (bash or -bash) and it does not. 3. It gets two copies of each entry — measured — and five sources give five. The costs are slower command lookup, an unreadable list, and unpredictable order when two setup scripts prepend different versions. 4. /opt/t is a substring of /opt/tools/bin, so the guard decides the directory is already present and does not add it — measured. 5. So that every entry, including the first and the last, is bounded by colons; otherwise the first or last entry escapes the whole-entry match. 6. Because with an empty PATH the simpler form leaves a trailing colon, and an empty PATH element means the current directory. 7. Because it only adds when absent, so an entry already at position twelve stays there — which is wrong for a version switch, where you wanted the new version first. 8. Because a non-existent entry costs a failed stat on every command lookup and buys nothing; accumulated stale entries slow every command. 9. Because it has to change the current shell's environment, which a separate process cannot do. It runs a helper that prints shell commands and evals them. 10. Nothing useful: the function runs in a child and the child's PATH dies with it. 11. module list says what you asked for; command -v and -version say what you got. The second pair is authoritative, because a hand-edited PATH entry earlier in the list wins and the module system cannot see it. 12. The guard only runs when the script was executed, where exit is right; every other path runs when sourced, where exit would close the user's terminal.

7. Remember

A setup script must be sourced. A child cannot change its parent's environment, so running it does nothing and exits 0.
Guard against execution with case ${0##*/}, and use exit only there.
Measured: sourced twice, PATH doubles. Five sources, five copies; the base-2 list became 7 entries.
The cost is not the duplicate — it is that the same pattern silently changes which version wins.
case "$PATH" in *"/opt/t"* matches inside /opt/tools and skips wrongly. Wrap both sides: case ":$PATH:" in (*":$1:"*).
${PATH:+:$PATH}, or an empty PATH gains a trailing colon, which means the current directory.
The guard does not reorder. For a version switch you need remove-then-prepend.
Check the directory exists and return non-zero — stale entries slow every lookup.
awk '!seen[$0]++' deduplicates keeping first occurrence; the pure-shell IFS=: loop does the same with no forks. Both gave /a:/b:/c.
Never assign a test value to PATH. Recovery is PATH=/usr/bin:/bin.
module is a shell function that evals printed commands — so it does not exist in a script that did not source its init file, and does nothing in a subshell.
module show is the dry run.
module list is what you asked for; command -v is what you got.
A setup script should print one line including the tool's own version — an empty version means it did not run.

8. What Is Next

Chapter 25.7 takes the verification half seriously: recording and checking exact tool versions, measured across four tools that accept four different version flags and return four different statuses for the ones they reject.

Where this fits

Part of the Linux & Bash curriculum.