Structure, Comments and a Script Someone Else Can Read

Part I · Module 15 · Chapter 15.5 · Structure, Comments and a Script Someone Else Can Read

Four chapters on making a script work. This one is on making it survive.

1. A Script That Works and Cannot Be Read

#!/bin/bash
cd /work/proj/build
rm -rf out
mkdir out
vcs -full64 -sverilog -f ../rtl.f -o out/simv > out/c.log 2>&1
if [ $? -ne 0 ]; then echo "fail"; exit 1; fi
./out/simv +UVM_TESTNAME=$1 > out/s.log 2>&1
grep -q "TEST PASSED" out/s.log && echo ok || echo bad

Eight lines, and it probably does what its author wanted. It is also close to unmaintainable:

• No statement of purpose. Somebody finding this in a repository has to read it to find out what it is.
• A hard-coded path, so it works on exactly one machine.
• No usage. Running it with no argument produces a confusing simulator error rather than a message.
• $1 unquoted, so a test name with a space breaks it (Chapter 9.2).
• rm -rf out with no check, which after a failed cd deletes something else (Chapter 11.2).
• No meaningful exit status. It prints ok or bad and returns 0 either way (Chapter 15.6).
• Nothing says why any of it is as it is.

The rest of this chapter is that script, fixed.

2. A Fixed Order of Sections

Scripts vary; their shape should not. Five sections, always in the same order:

The value is predictability. A reader opening any script knows the purpose is at the top, the inputs are a third of the way down, and the work is at the bottom — so finding one of those takes no searching.

3. The Header

#!/usr/bin/env bash
#
# run_sim.sh — compile and run one UVM test, logging everything.
#
# usage:  run_sim.sh <test-name> [seed]
#         run_sim.sh --help
#
# exit:   0  the test passed
#         1  the test failed
#         2  bad usage, or the run directory could not be made
#         3  the simulator is not on PATH
#
# Requires PROJECT_ROOT, or defaults to the script's own directory.

Five things, and none of them optional in a script anybody else will run:

• What it is — one line, in plain words.
• How to call it — every form, including --help.
• What it returns — the documented codes from Chapter 11.1.
• What it needs — required variables and tools.
• Nothing else. Not a change log, not an author, not a date. Version control has those and they go stale here.

This is the single highest-value thing in the chapter. Ten lines of header save every future reader the work of inferring all of it from the code — and they are what makes --help possible without writing the text twice:

usage() { sed -n '3,12p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'; }

Chapter 13.6's line-range selection, reading the script's own header. The help text cannot drift from the documentation because it is the documentation.

4. Comments Say Why

# Increment the counter.
count=$(( count + 1 ))

That comment is worse than none: it occupies a line, says what the code already says, and will not be updated when the code changes.

# The licence server rejects more than four concurrent checkouts,
# so the pool size is a hard limit rather than a tuning parameter.
max_parallel=4

That one cannot be inferred from the code at all.

Three that earn their place:

# Unquoted on purpose: EXTRA_FLAGS is a list of separate arguments.
./run_sim.sh $EXTRA_FLAGS

# vcs exits 0 after a fatal elaboration error, so the log must be checked too.
grep -q 'Error-\[' "$logdir/compile.log" && exit 1

# Sorting with the C locale so the output is byte-identical between machines.
LC_ALL=C sort -o "$manifest" "$manifest"

Each records something a reader could not work out, and the first is Chapter 9.2's requirement that deliberate splitting be labelled.

Section markers

# ---------------------------------------------------------------- constants

A one-line marker between sections costs nothing and makes a two-hundred-line script navigable. Keep them short and consistent; a box of hashes around every section is noise.

5. Naming

d=/work/proj                 # what is d?
project_root=/work/proj      # obvious six months later

Lowercase with underscores for your own variables, uppercase for exported ones — Chapter 8.1's convention, and the reason for it is that a lowercase name cannot collide with something the system already uses.

The exception is a loop variable with a short life:

for f in *.log; do
  echo "$f"
done

f is fine there — it exists for two lines and its meaning is visible in both. A f that lives for forty lines is not.

6. Making the Steps Visible

main() {
  check_prerequisites
  prepare_run_directory
  compile_design
  run_simulation
  report_results
}

main "$@"

The bottom of the script is a table of contents. Somebody asking "what does this do?" reads five lines; somebody asking "how does it compile?" goes to one function.

main "$@" passes the script's arguments through — Chapter 15.4's quoted form — so the functions see the same arguments the script did.

7. The Script, Rewritten

#!/usr/bin/env bash
#
# run_sim.sh — compile and run one UVM test, logging everything.
#
# usage:  run_sim.sh <test-name> [seed]
#         run_sim.sh --help
#
# exit:   0  the test passed
#         1  the test failed
#         2  bad usage, or the run directory could not be made
#         3  the simulator is not on PATH
#
# Uses PROJECT_ROOT if set, otherwise the script's own directory.

# ------------------------------------------------------------------ options
set -euo pipefail

# ---------------------------------------------------------------- constants
script_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
project_root="${PROJECT_ROOT:-$script_dir}"
filelist="$project_root/rtl.f"

# ---------------------------------------------------------------- functions
usage() {
  sed -n '3,12p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'
}

die() { echo "error: $*" >&2; exit "${2:-1}"; }

check_prerequisites() {
  command -v vcs >/dev/null 2>&1 || die "vcs is not on PATH — source the tool setup" 3
  [ -r "$filelist" ]             || die "cannot read $filelist" 2
}

prepare_run_directory() {
  # Belt and braces: an unset run_dir here would make this rm -rf /* .
  rm -rf "${run_dir:?run_dir is not set}"
  mkdir -p "$run_dir" || die "cannot create $run_dir" 2
}

compile_design() {
  vcs -full64 -sverilog -f "$filelist" -o "$run_dir/simv" \
    > "$run_dir/compile.log" 2>&1 \
    || die "compile failed — see $run_dir/compile.log"

  # vcs exits 0 after some fatal elaboration errors, so check the log as well.
  ! grep -q 'Error-\[' "$run_dir/compile.log" \
    || die "elaboration error — see $run_dir/compile.log"
}

run_simulation() {
  "$run_dir/simv" +UVM_TESTNAME="$test_name" +ntb_random_seed="$seed" \
    > "$run_dir/sim.log" 2>&1 || true   # a failing test is a result, not an error
}

report_results() {
  if grep -q 'TEST PASSED' "$run_dir/sim.log"; then
    echo "PASS  $test_name (seed $seed)"
    return 0
  fi
  echo "FAIL  $test_name (seed $seed) — see $run_dir/sim.log" >&2
  return 1
}

# --------------------------------------------------------------------- main
main() {
  if [ "$#" -ge 1 ] && [ "$1" = "--help" ]; then
    usage
    exit 0
  fi

  test_name="${1:?usage: $0 <test-name> [seed]}"
  seed="${2:-0}"
  run_dir="$project_root/runs/${test_name}_seed${seed}"

  check_prerequisites
  prepare_run_directory
  compile_design
  run_simulation
  report_results
}

main "$@"

Sixty lines where there were eight, and every addition prevents something specific:

None of it is clever. It is the same eight commands with the failures that were always possible now handled, and a layout that says so.

8. A Checklist

Before a script leaves your machine:

And one command:

shellcheck myscript.sh

shellcheck finds unquoted expansions, useless cats, unreachable code and about two hundred other things, including several this course has warned about. It is not installed by default and is worth asking for. Module 20 returns to it.

Practice

1 — Read the bad version and list its faults. Copy §1's script into a file and write down every problem you can see before reading §1's list.

2 — Write a header for something you already have. Take any script of your own and add purpose, usage and exit codes. Note how often you have to decide what the exit codes are, rather than describe them.

3 — Build the self-documenting help.

mkdir -p /tmp/str && cd /tmp/str
cat > demo.sh <<'EOF'
#!/bin/bash
#
# demo.sh — a script that documents itself.
#
# usage:  demo.sh <name>
#         demo.sh --help
#
# exit:   0 fine · 2 bad usage

usage() { sed -n '3,8p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'; }

if [ "$#" -ge 1 ] && [ "$1" = "--help" ]; then usage; exit 0; fi
name="${1:?usage: $0 <name>}"
echo "hello $name"
EOF
chmod +x demo.sh
./demo.sh --help
./demo.sh world
./demo.sh; echo "status $?"

4 — Sort comments into why and what. For each, decide whether it earns its place:

# Loop over the files.
# Skip the build directory because it contains copies of the sources.
# Set x to 5.
# 5 is the licence limit, not a tuning parameter.
# Call the function.
# Must run before compile_design, which reads the filelist this writes.

5 — Rename for readability. Rewrite with better names:

d=/work/proj
f=$d/rtl.f
t=$1
o=$d/runs/$t

6 — Add the destructive guard.

mkdir -p /tmp/str/scratch
( unset target; echo "would run: rm -rf ${target}/*" )
( unset target; echo "would run: rm -rf ${target:?target is not set}/*" ) || echo "  guard fired"

7 — Split a flat script into functions. Take a script of yours longer than fifty lines and give it a main() whose body is five named steps.

8 — Run shellcheck if you have it.

command -v shellcheck >/dev/null 2>&1 && shellcheck demo.sh || echo "shellcheck not installed"

9 — Clean up. cd /tmp && rm -rf /tmp/str

Check Your Understanding

1. What five sections should a script have, and in what order?

2. Why must functions be defined before they are called?

3. What belongs in a header, and what does not?

4. Which of these earns its place: "Increment the counter" or "Four is the licence limit, not a tuning parameter"?

5. Why does rm -rf "${run_dir:?}" keep the guard even when run_dir was set two lines earlier?

6. Why must a || true always carry a comment?

9. Remember

• Five sections, always in the same order: header, options, constants, functions, main.
• The header is the highest-value ten lines in the file — purpose, usage, exit codes, requirements.
• Generate --help from the header so the documentation and the help text cannot diverge.
• The code says what; the comment says why.
• Comment a value chosen for an external reason, something that looks wrong and is not, a non-obvious ordering, or a workaround.
• Do not comment the obvious, the author, the date, or commented-out code. Delete that last one.
• Lowercase with underscores for your own names, uppercase for exported. Short names only for short lives.
• Define functions before use, which the section order enforces.
• main() plus main "$@" makes the bottom of the script a table of contents.
• Functions start paying at about fifty lines, or when one step exceeds ten.
• Keep the destructive guard on the destructive line, even when it looks redundant.
• A || true without a comment will be removed by somebody, and should be.
• Run shellcheck.

A script is now readable. One thing remains: making sure that whatever runs it can tell whether it worked.