Linux & Bash · Part II · Module 25 · EDA Environment and Tool Access

Project Variables and Conventions

A flow script with an absolute path in it works for exactly one person. The fix is a handful of agreed variables and a discipline about which of them a script may default and which it must refuse to guess — which is a smaller and more interesting decision than it sounds.

Intermediate20 min readLinuxPart IIEnvironmentConventionsScripting

Part II · Module 25 · Chapter 25.5 · Project Variables and Conventions

The previous four chapters were about variables a tool reads. This one is about variables a project agrees on, and the problem it solves is the single most common defect in home-grown flow scripts.

1. The Defect

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

cd /proj/soc_a/verif/axi
iverilog -f /proj/soc_a/filelists/axi.f -o /scratch/asha/work/sim.vvp

Three absolute paths. That script works on one machine, for one person, until the project moves — and then it fails with "No such file or directory", naming a path that is obviously correct to whoever wrote it.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ [ -d /proj/soc_a ] && echo exists || echo "does not exist here"
does not exist here

That is not a hypothetical. The path above does not exist on the machine these measurements were taken on, which is exactly the position a colleague is in when they clone your flow.

2. The Agreed Set

Six variables cover almost everything, and the value is in the agreement rather than the names:

Variable	Means	Who sets it
`PROJECT_ROOT`	the top of the project tree	the setup script
`PROJECT_NAME`	a short identifier, for file names and reports	the setup script
`SCRATCH_ROOT`	the purged, fast area for run output	the site, or defaulted
`WORK_DIR`	this run's directory under scratch	derived per run
`TOOL_HOME` / `TOOL_VERSION`	which installation is in use	the setup script
`TMPDIR`	per-process temporary space	the site, or defaulted

Only TMPDIR is standard; the rest are conventions, which is why a script must not assume they exist. Two of them — PROJECT_ROOT and TOOL_VERSION — are the ones a flow genuinely cannot guess.

3. What May Be Defaulted, and What May Not

This is the chapter's one real decision, and the shell has distinct syntax for each side of it:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

# required: fail loudly, naming the variable and what it is for
: "${PROJECT_ROOT:?set PROJECT_ROOT to the project area}"

# optional: a sane default
scratch=${SCRATCH_ROOT:-/tmp}

# derived: built from the others, never set by hand
work=${WORK_DIR:-$scratch/$USER/run_$$}

Measured, with set -o nounset and PROJECT_ROOT unset:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ bash -c 'set -o nounset
> : "${PROJECT_ROOT:?set PROJECT_ROOT to the project area}"'
bash: line 1: PROJECT_ROOT: set PROJECT_ROOT to the project area

And with it set:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

$ bash -c 'set -o nounset
> PROJECT_ROOT=/proj/soc_a
> : "${PROJECT_ROOT:?}"
> scratch=${SCRATCH_ROOT:-/tmp}
> work=${WORK_DIR:-$scratch/$USER/run_$$}
> printf "PROJECT_ROOT=%s\nscratch=%s\nwork=%s\n" "$PROJECT_ROOT" "$scratch" "$work"'
PROJECT_ROOT=/proj/soc_a
scratch=/tmp
work=/tmp/engineer/run_21392

One required, one defaulted, one derived, and the derived one incorporates $$ so two concurrent runs cannot collide. Chapter 8.2 is the expansion forms; what Part II adds is the policy about which to use where.

The rule, stated once:

Require what you cannot guess. PROJECT_ROOT, a tool version, a licence source. Guessing produces a run against the wrong tree, which is worse than not running.

Default what you can. Scratch space, verbosity, a job count. A default that is wrong costs a little; a refusal to start costs a lot.

Derive everything else. If WORK_DIR can be computed from SCRATCH_ROOT, USER and the run id, it should be — because a variable a user can set is a variable a user can set inconsistently.

The null-command idiom for a required variable, and one caveat

${X:?message} expands to the value of X, or prints the message and exits if X is unset or empty. To use it purely as a check, the expansion needs somewhere to go, and : — the null command — is that somewhere:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

: "${PROJECT_ROOT:?set PROJECT_ROOT to the project area}"

Three details:

The quotes matter. Without them a value containing spaces would be word-split and : would receive several arguments — harmless here, and a habit worth keeping because the same expansion often appears where splitting is not harmless.

:? triggers on unset or empty. ${X-...} and ${X?...} without the colon trigger only on unset, so an exported-but-empty PROJECT_ROOT would pass. Empty is the commoner failure — a setup script that ran and did not find the project — so the colon forms are almost always what you want. Chapter 20.4 measured what an empty value does to rm -rf "$dir/".

And the exit status is not 2. Measured in Chapter 23.4 §5: a :? failure exits 1 inside a script, and 127 under bash -c. So if your script documents "exit 2 means a usage error", :? does not deliver it:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

[ -n "${PROJECT_ROOT:-}" ] || { printf 'flow: set PROJECT_ROOT\n' >&2; exit 2; }

Four more lines, and the contract in Chapter 21.4 holds. Use :? for interactive convenience and an explicit check wherever the status is part of an interface.

4. Validating a Root, Not Just Reading It

A set variable is not a correct one. Three checks, cheapest first:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

: "${PROJECT_ROOT:?set PROJECT_ROOT to the project area}"

# 1. does it exist and is it a directory?
[ -d "$PROJECT_ROOT" ] || { printf 'flow: not a directory: %s\n' "$PROJECT_ROOT" >&2; exit 2; }

# 2. resolve it, so everything recorded later is link-free
PROJECT_ROOT=$(cd -- "$PROJECT_ROOT" && pwd -P) || exit 2

# 3. does it look like the right tree? a marker file is the cheapest proof
for marker in filelists rtl .project-root; do
  [ -e "$PROJECT_ROOT/$marker" ] && break
done || { printf 'flow: %s does not look like a project root\n' "$PROJECT_ROOT" >&2; exit 2; }
export PROJECT_ROOT

Check three is the one people leave out, and it is the one that catches the expensive mistake: PROJECT_ROOT set to a parent of the real root, or to last quarter's branch. A flow that validates only existence will happily run against the wrong tree and produce results that look fine.

cd … && pwd -P is Chapter 22.3 §4's resolution, and it matters here more than usual: a PROJECT_ROOT recorded in a log is read on another machine, so it must be physical.

A marker file is a cheap contract

.project-root as an empty committed file at the top of the tree costs nothing and turns "is this the right directory" from a guess into a test. The same trick works for:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

[ -e "$PROJECT_ROOT/.project-root" ]       # is this a project root at all?
[ -e "$WORK_DIR/.run-in-progress" ]        # is a run already using this directory?
[ -e "$SCRATCH_ROOT/.purged-nightly" ]     # is this area safe to write large files into?

The third is the interesting one. Chapter 22.4 established that scratch is purged and project areas are not, and that writing a hundred gigabytes into the wrong one is a real failure. A marker file placed by whoever administers the area lets a script check the policy rather than trusting a variable's name — and a variable called SCRATCH_ROOT pointing at a backed-up project area is exactly the kind of thing that happens when someone copies a setup script.

Markers are also how a flow becomes safe to run twice: .run-in-progress with a trap to remove it is Chapter 21.5's idempotence applied to a directory rather than to a command.

5. Relative to the Script, Not to the Caller

The complement to a project root: a script that needs its own directory should compute it, not be told.

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

here=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)
PROJECT_ROOT=${PROJECT_ROOT:-$(cd -- "$here/.." && pwd -P)}

That gives a flow a sensible default: if the script lives in $PROJECT_ROOT/scripts, then .. is the root. A caller can still override it, and most of the time nobody needs to.

Two notes:

${BASH_SOURCE[0]}, not $0. $0 is the invoked name, which for a sourced script is the shell itself. Chapter 10.4 measured the difference, and a setup script is by definition sourced.

And the default is computed inside ${...:-}, so the cd only runs when the variable is unset. A $(...) outside the expansion would run every time and cost a subshell per invocation — trivial here, less trivial in a loop over five hundred tests.

6. Where These Belong, and Where They Do Not

Put it in	Things like
The setup script that everyone sources	`PROJECT_ROOT`, `TOOL_VERSION`, licence variables, `PATH`
The flow script, derived	`WORK_DIR`, output file names, log paths
The command line	the test name, the seed, the verbosity
Nowhere — compute it	anything derivable from the above

The fourth row is the discipline. Every variable a user can set is a variable that can be set inconsistently, and every one you add is a thing the next person must discover. A flow with six variables and clear derivation is easier to hand over than one with thirty.

And one thing that belongs in none of them:

Never put a secret in the environment. Chapter 10.1's measurement that the environment is readable by other processes is not a theoretical concern on a shared compute server — anyone who can run ps can often read it. A licence source is fine; a password is not.

7. The Setup Fragment

Everything above, as the part of a setup script that deals with project variables:

Azvya Education Pvt. Ltd.VLSI Mentor

Snippet

# project.sh — sourced, not executed. Sets the project variables and validates them.
# Safe to source more than once.

# 0. refuse to be executed; this must be sourced to have any effect
case ${0##*/} in
  (project.sh) printf 'project.sh must be sourced, not executed\n' >&2; exit 2 ;;
esac

# 1. where am I? (BASH_SOURCE, because $0 is the shell when sourced)
_here=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P) || return 2

# 2. required, with a default derived from the script's own location
PROJECT_ROOT=${PROJECT_ROOT:-$(cd -- "$_here/.." && pwd -P)}
[ -d "$PROJECT_ROOT" ] || { printf 'project.sh: not a directory: %s\n' "$PROJECT_ROOT" >&2; return 2; }
PROJECT_ROOT=$(cd -- "$PROJECT_ROOT" && pwd -P) || return 2

# 3. does it look like the right tree?
_ok=0
for _m in filelists rtl .project-root; do
  [ -e "$PROJECT_ROOT/$_m" ] && { _ok=1; break; }
done
[ "$_ok" -eq 1 ] || { printf 'project.sh: %s has no filelists/, rtl/ or .project-root\n' "$PROJECT_ROOT" >&2; return 2; }

# 4. derived, and overridable
PROJECT_NAME=${PROJECT_NAME:-${PROJECT_ROOT##*/}}
SCRATCH_ROOT=${SCRATCH_ROOT:-${TMPDIR:-/tmp}}
[ -w "$SCRATCH_ROOT" ] || { printf 'project.sh: not writable: %s\n' "$SCRATCH_ROOT" >&2; return 2; }

export PROJECT_ROOT PROJECT_NAME SCRATCH_ROOT
unset _here _ok _m

printf 'project: %s at %s (scratch %s)\n' "$PROJECT_NAME" "$PROJECT_ROOT" "$SCRATCH_ROOT"

Seven decisions:

return, not exit. A sourced script runs in your shell, so exit closes your terminal. Chapter 18.3 measured exactly that, and it is the single most common defect in a setup script.

The case on ${0##*/} refuses execution. Run rather than sourced, $0 is the script's own name and the guard fires; sourced, $0 is bash or -bash and it does not. That inverts Chapter 10.4's observation into a check, and it saves the next person the ten minutes of wondering why ./project.sh did nothing.

${PROJECT_ROOT##*/} for a default name — the last path component, with no basename process. Chapter 8.3.

SCRATCH_ROOT falls back through TMPDIR to /tmp, which is a chain of two defaults and is worth writing as one expansion rather than an if.

It is writable-checked, because a scratch root that is not writable produces a failure a hundred lines later in the flow, where nothing points back here.

The underscore-prefixed locals are unset at the end. A sourced script has no scope of its own — every variable it creates persists in your interactive shell — so a setup script that leaves _m and _ok behind is polluting your environment. Chapter 18.5 is about local in functions; this is the case where local is not available because there is no function, and the discipline has to be manual.

And it prints one line. A setup script that is silent leaves you unsure whether it worked; one that prints twenty lines gets skipped. One line, naming what was set.

Practice

Find a flow script in a project you work on and count the absolute paths in it.
Check whether each of those paths exists on a machine other than the author's.
Write the three-way :? / :- / derived block and run it with the required variable unset.
Export the required variable as the empty string and confirm :? still fires — then change it to ${X?} without the colon and confirm it does not.
Check the exit status of a :? failure in a script. Does it match what your script documents?
Add the marker-file check to a script and point PROJECT_ROOT at the parent of the real root. Does it catch it?
Put a .project-root file at the top of a tree and commit it. What does that let a script assert?
Compute a script's own directory with ${BASH_SOURCE[0]} and then with $0, once executed and once sourced. Compare all four results.
Write project.sh and source it twice. Does anything change on the second source?
Then execute it and confirm the guard fires.
Replace a return with exit in it, source it, and see what happens to your shell. (Open a new terminal first.)
Source it and then run set | grep '^_'. Did it leave anything behind?

Check Your Understanding

What is wrong with an absolute path in a flow script?
Which two project variables can a flow genuinely not guess?
What is the difference between ${X:?} and ${X?}?
Why is : "${X:?msg}" written with a : and quotes?
Why does a :? failure not satisfy a documented exit code of 2?
Why is "does it exist" an insufficient check on PROJECT_ROOT?
What does a marker file buy over trusting a variable's name?
Why ${BASH_SOURCE[0]} rather than $0?
Why must a sourced script use return and not exit?
How does the case ${0##*/} guard distinguish sourced from executed?
Why does a sourced script have to unset its own temporaries?

Answers

1. It resolves on exactly one machine for one person. When the project moves it fails with a message naming a path the author believes is correct. 2. PROJECT_ROOT and the tool version (and a licence source). Guessing them runs against the wrong tree or the wrong tool, which is worse than refusing to run. 3. The colon forms trigger on unset or empty; without the colon only on unset. Empty is the commoner failure — a setup script that ran and found nothing. 4. : is the null command, which gives the expansion somewhere harmless to go; the quotes stop a value containing spaces from being word-split. 5. Because a :? failure exits 1 in a script (and 127 under bash -c), not 2 — so a documented contract needs an explicit check and exit 2. 6. Because a path can exist and be the wrong tree — a parent of the real root, or last quarter's branch. Existence does not establish identity. 7. It lets a script test the policy rather than trust a name: a variable called SCRATCH_ROOT pointing at a backed-up project area is a real and common mistake. 8. Because $0 is the invoked name, which for a sourced script is the shell itself rather than the script. 9. Because a sourced script runs in your interactive shell, so exit closes your terminal. 10. Executed, $0 is the script's own file name and the pattern matches; sourced, $0 is bash or -bash and it does not. 11. Because there is no scope: every variable it creates persists in the shell that sourced it, and local is unavailable outside a function.

8. Remember

An absolute path in a flow script works for one person. Measured: /proj/soc_a does not exist here.
Six variables cover it: PROJECT_ROOT, PROJECT_NAME, SCRATCH_ROOT, WORK_DIR, tool home and version, TMPDIR. Only TMPDIR is standard.
Require what you cannot guess, default what you can, derive everything else.
:? for required, :- for optional, and the colon matters: without it, empty passes.
A :? failure exits 1, not 2. Use an explicit check where the status is an interface.
Existence is not identity. Check a marker file, or a flow runs against the wrong tree and looks fine.
Resolve a root with cd … && pwd -P before recording it anywhere.
A marker file lets a script test policy, not a variable's name — including whether an area is really scratch.
${BASH_SOURCE[0]}, not $0, in anything sourced.
A sourced script uses return. exit closes the user's terminal.
Guard against execution with case ${0##*/}.
Unset your temporaries — a sourced script has no scope.
Print one line. Silent is untrustworthy; twenty lines gets skipped.
Never put a secret in the environment. ps can read it.

9. What Is Next

Chapter 25.6 is the script that sets all of this: how sourcing really works for tool setup, what an environment-module system does, and the measured reason a setup script sourced twice doubles your PATH — along with the guard that prevents it and the subtle bug in the obvious version of that guard.

Where this fits

Part of the Linux & Bash curriculum.