Option Parsing with getopts

Part I · Module 21 · Chapter 21.2 · Option Parsing with getopts

The hand-rolled loop works. It is also where every tool's option handling quietly diverges from every other tool's.

1. The Shape

while getopts ':t:s:v' opt; do
  case $opt in
    t) test=$OPTARG ;;
    s) seeds=$OPTARG ;;
    v) verbose=1 ;;
    \?) printf 'unknown option: -%s\n' "$OPTARG" >&2; exit 2 ;;
    :)  printf 'option -%s needs an argument\n' "$OPTARG" >&2; exit 2 ;;
  esac
done
shift $(( OPTIND - 1 ))

The optstring ':t:s:v' is the whole specification:

• a letter means a flag — v;
• a letter followed by a colon means it takes a value — t: and s:;
• and a leading colon turns on silent error reporting, which §3 shows is not optional.

getopts sets three things for you:

And shift $(( OPTIND - 1 )) discards everything it consumed, leaving the positional arguments in "$@".

2. What It Handles That Your Loop Did Not

A probe script that reports everything it parsed:

#!/usr/bin/env bash
verbose=0; test=; seeds=
while getopts ':t:s:v' opt; do
  case $opt in
    t) test=$OPTARG ;;
    s) seeds=$OPTARG ;;
    v) verbose=1 ;;
    \?) printf 'unknown option: -%s\n' "$OPTARG" >&2; exit 2 ;;
    :)  printf 'option -%s needs an argument\n' "$OPTARG" >&2; exit 2 ;;
  esac
done
shift $(( OPTIND - 1 ))
printf 'test=[%s] seeds=[%s] verbose=%s OPTIND=%s remaining=%s\n' \
  "$test" "$seeds" "$verbose" "$OPTIND" "$#"
printf '  arg: [%s]\n' "$@"

Separate options, then positionals:

$ ./g1.sh -t smoke -s 3 -v extra1 extra2
test=[smoke] seeds=[3] verbose=1 OPTIND=6 remaining=2
  arg: [extra1]
  arg: [extra2]

Clustered flags, with the last one taking a value:

$ ./g1.sh -vt smoke file.sv
test=[smoke] seeds=[] verbose=1 OPTIND=3 remaining=1
  arg: [file.sv]

-vt smoke is -v and -t smoke. The hand-rolled case $1 in -v) … loop in Chapter 21.1 would have reported -vt as an unknown option.

A value attached to its letter:

$ ./g1.sh -tsmoke -s3
test=[smoke] seeds=[3] verbose=0 OPTIND=3 remaining=0
  arg: []

-tsmoke and -s3, with no space. Users type this, tar and gcc accept it, and a hand-rolled loop does not. (The arg: [] line is printf running its format once with zero arguments — Chapter 15.4.)

-- ends the options:

$ ./g1.sh -v -- -notanoption
test=[] seeds=[] verbose=1 OPTIND=3 remaining=1
  arg: [-notanoption]

So Chapter 21.1 §5's convention is free — getopts implements it, and a file named -notanoption arrives intact.

And the two caller mistakes:

$ ./g1.sh -z
unknown option: -z
$ echo $?
2

$ ./g1.sh -t
option -t needs an argument
$ echo $?
2

Four behaviours, all conventional, all for the price of the optstring.

3. The Leading Colon Is Not Optional

This is the detail that decides whether your error messages are usable. Compare the same two mistakes with and without it:

$ cat v.sh          # optstring 't:v' — no leading colon
while getopts 't:v' opt; do
  case $opt in t) :;; v) :;; \?) echo "  caught \?: OPTARG=[$OPTARG]";; esac
done

$ bash v.sh -z
v.sh: illegal option -- z
  caught ?: OPTARG=[]

$ bash v.sh -t
v.sh: option requires an argument -- t
  caught ?: OPTARG=[]

Two problems. Bash printed its own message, which you cannot phrase, translate or suppress. And OPTARG is empty, so your handler cannot say which option was wrong — both mistakes arrive as an indistinguishable \?.

With the leading colon:

$ cat s.sh          # optstring ':t:v'
while getopts ':t:v' opt; do
  case $opt in
    t) :;; v) :;;
    \?) echo "  caught \?: OPTARG=[$OPTARG]";;
    :)  echo "  caught :: OPTARG=[$OPTARG]";;
  esac
done

$ bash s.sh -z
  caught ?: OPTARG=[z]

$ bash s.sh -t
  caught :: OPTARG=[t]

Bash says nothing, the two mistakes arrive separately, and OPTARG names the letter. Now your handler can produce exactly the message Chapter 21.1 §6 wanted:

\?) die "unknown option: -$OPTARG" ;;
:)  die "option -$OPTARG needs a value" ;;

Always write the leading colon, and always handle both \? and :. A getopts loop with no : arm silently treats a missing value as an unknown option, which is a worse message for a more common mistake.

(The \? needs the backslash because ? is a case pattern metacharacter — Chapter 16.6's glob patterns. '?' quoted works too.)

4. Limit One: No Long Options

Measured, because this is the limit people most want not to be true:

$ ./g1.sh --verbose
unknown option: --
$ echo $?
2

getopts read --verbose, took the first character after the dash — which is another dash — found it is not in the optstring, and reported \? with OPTARG set to -.

getopts cannot parse --long options. There is no flag for it and no workaround inside getopts. Your choices are:

Accept only short options, which is what most internal tools do and is perfectly respectable. -v, -n, -s 3.

Handle the long forms before the loop, for the two or three that matter most:

# translate the long forms people will actually type, then let getopts work
args=()
for a in "$@"; do
  case $a in
    --verbose) args+=(-v) ;;
    --dry-run) args+=(-n) ;;
    --help)    args+=(-h) ;;
    *)         args+=("$a") ;;
  esac
done
set -- "${args[@]}"

while getopts ':vnhs:' opt; do ...

That is Chapter 19.5's array-of-arguments and Chapter 19.3's set -- "${a[@]}", and it handles --verbose and --dry-run without pretending to be a general long-option parser. It does not handle --seeds=3; if you need that, you are writing the parser by hand.

Or hand-roll the whole loop, as Chapter 21.1 did — which is the honest answer when long options with values are a requirement. There is no getopt to fall back to either: the external GNU getopt(1) does support long options, and it is not installed by default on macOS or in minimal containers, so a script depending on it is a script that fails on arrival.

5. Limit Two: OPTIND Is Not Reset

The one that produces a genuinely baffling bug. OPTIND is a normal shell variable, and getopts leaves it where it finished:

parse() {
  local opt v=0
  while getopts ':v' opt; do case $opt in v) v=1 ;; esac; done
  echo "  parse() saw v=$v OPTIND=$OPTIND"
}
parse -v
parse -v

  parse() saw v=1 OPTIND=2
  parse() saw v=0 OPTIND=2

The same call, twice, with different results. The second parse -v saw v=0, because OPTIND was still 2 from the first call and getopts therefore started past the one argument it was given.

One word fixes it:

parse2() {
  local opt v=0 OPTIND=1
  while getopts ':v' opt; do case $opt in v) v=1 ;; esac; done
  echo "  parse2() saw v=$v OPTIND=$OPTIND"
}

  parse2() saw v=1 OPTIND=2
  parse2() saw v=1 OPTIND=2

local OPTIND=1 in any function that calls getopts. It is both the reset and the scoping — Chapter 18.5's local doing two jobs at once, and without it a function that parses options is correct exactly once per shell.

At the top level of a script this never bites, because the script runs once. It bites in a library function (Chapter 18.6), in a function called in a loop, and in anything sourced into an interactive shell — the three places a bug is hardest to reproduce.

6. A Complete Script

Chapter 21.1's run-suite.sh, with the option loop replaced:

#!/usr/bin/env bash
# run-suite.sh — run every test in a testlist, once per seed
#
# usage: run-suite.sh [-v] [-n] [-s SEEDS] [-o OUTDIR] <testlist>
#
# exit: 0 all passed · 1 at least one failed · 2 bad invocation

set -o errexit
set -o nounset
set -o pipefail

note()  { printf '%s: %s\n' "${0##*/}" "$*" >&2; }
usage() { printf 'usage: %s [-v] [-n] [-s SEEDS] [-o OUTDIR] <testlist>\n' "${0##*/}" >&2; }
die()   { note "$*"; usage; exit 2; }

help_text() {
  usage 2>&1
  cat <<'EOF'

  -v           verbose
  -n           dry run: print what would run, then exit 0
  -s SEEDS     seeds per test (default 1, maximum 100)
  -o OUTDIR    where logs go (default ./results)
  -h, --help   this message, on stdout

Exit status: 0 all passed · 1 a failure · 2 bad invocation
EOF
}

# ---- interface -----------------------------------------------------------

verbose=0
dry_run=0
seeds=1
outdir=./results

# long forms worth translating, before getopts sees them
declare -a argv=()
for a in "$@"; do
  case $a in
    --verbose) argv+=(-v) ;;
    --dry-run) argv+=(-n) ;;
    --help)    argv+=(-h) ;;
    *)         argv+=("$a") ;;
  esac
done
[ "${#argv[@]}" -eq 0 ] || set -- "${argv[@]}"

while getopts ':vnhs:o:' opt; do
  case $opt in
    v)  verbose=1 ;;
    n)  dry_run=1 ;;
    h)  help_text; exit 0 ;;
    s)  seeds=$OPTARG ;;
    o)  outdir=$OPTARG ;;
    \?) die "unknown option: -$OPTARG" ;;
    :)  die "option -$OPTARG needs a value" ;;
  esac
done
shift $(( OPTIND - 1 ))

[ $# -eq 1 ] || { usage; exit 2; }
testlist=$1

[ "$seeds" -ge 1 ] 2>/dev/null   || die "seeds must be a positive integer, got: $seeds"
[ "$seeds" -le 100 ] 2>/dev/null || die "seeds must be 100 or fewer, got: $seeds"
[ -r "$testlist" ]               || die "cannot read testlist: $testlist"

# ---- the work ------------------------------------------------------------

declare -a tests=()
while IFS= read -r line || [ -n "$line" ]; do
  case $line in ''|\#*) continue ;; esac
  tests+=("$line")
done < "$testlist"

[ "${#tests[@]}" -gt 0 ] || die "no tests in $testlist"

if [ "$dry_run" -eq 1 ]; then
  printf 'would run %d test(s) x %d seed(s) into %s\n' "${#tests[@]}" "$seeds" "$outdir"
  exit 0
fi

mkdir -p -- "$outdir" || die "cannot create $outdir"

failed=0
for t in "${tests[@]}"; do
  for (( s = 1; s <= seeds; s++ )); do
    [ "$verbose" -eq 0 ] || note "starting $t seed $s"
    if [ "$t" = known_bad ]; then
      printf 'FAIL  %-16s seed %d\n' "$t" "$s"; failed=$(( failed + 1 ))
    else
      printf 'pass  %-16s seed %d\n' "$t" "$s"
    fi
  done
done

printf -- '-- %d test(s) x %d seed(s), %d failure(s)\n' "${#tests[@]}" "$seeds" "$failed"
[ "$failed" -eq 0 ]

$ printf '# the suite\n\nsmoke\ncache coherency\nknown_bad\n' > tests.txt

$ ./run-suite.sh -vn -s 2 tests.txt
would run 3 test(s) x 2 seed(s) into ./results
$ echo $?
0

$ ./run-suite.sh -s2 tests.txt | tail -1
-- 3 test(s) x 2 seed(s), 2 failure(s)

$ ./run-suite.sh --dry-run --verbose tests.txt
would run 3 test(s) x 1 seed(s) into ./results

$ ./run-suite.sh --help | head -1
usage: run-suite.sh [-v] [-n] [-s SEEDS] [-o OUTDIR] <testlist>
$ echo $?
0

$ ./run-suite.sh -z tests.txt
run-suite.sh: unknown option: -z
usage: run-suite.sh [-v] [-n] [-s SEEDS] [-o OUTDIR] <testlist>
$ echo $?
2

$ ./run-suite.sh -s
run-suite.sh: option -s needs a value
usage: run-suite.sh [-v] [-n] [-s SEEDS] [-o OUTDIR] <testlist>
$ echo $?
2

$ ./run-suite.sh --bogus tests.txt
run-suite.sh: unknown option: --
usage: run-suite.sh [-v] [-n] [-s SEEDS] [-o OUTDIR] <testlist>
$ echo $?
2

$ ./run-suite.sh
usage: run-suite.sh [-v] [-n] [-s SEEDS] [-o OUTDIR] <testlist>
$ echo $?
2

Five decisions:

-vn and -s2 both work without a line of code for either. That is the whole argument for getopts: a caller gets conventional behaviour because the builtin is conventional.

The long-form translation is three lines and two options. It handles the forms people type and makes no claim to be a general parser — --seeds=3 is not supported, and the usage line does not offer it.

[ "${#argv[@]}" -eq 0 ] || set -- "${argv[@]}" guards the set --, because set -- "${argv[@]}" on an empty array under set -u is Chapter 19.2 §5's abort. With no arguments at all, argv is empty and the guard skips the set.

--bogus is reported as unknown option: --, which is getopts's honest answer and is slightly unhelpful. The usage line underneath it carries the real information, which is why die prints both.

shift $(( OPTIND - 1 )) then [ $# -eq 1 ]. The count check is on what is left after the options, which is the only point at which the number of positional arguments is knowable.

Practice

1. Write the §2 probe script. Run it with separate options, clustered options, attached values, and a -- followed by a dash-leading filename. Record OPTIND and the remaining count each time.
2. Call it with -vt smoke and then with -v -t smoke. Confirm they agree, and say which one Chapter 21.1's loop would have rejected.
3. Build the same loop twice — optstring 't:v' and ':t:v' — and pass -z then -t to each. Record what Bash printed, what $opt was, and what OPTARG was, in all four cases.
4. Remove the :) arm from a silent-mode loop and pass -t with no value. What message does the user get, and why is it the wrong one?
5. Run -s 3 tests.txt, then -s tests.txt, then -s alone, printing $seeds and $# each time. Explain why only the third reports a missing value, then add the value check that catches the second.
6. Pass --verbose to a getopts loop and print OPTARG in the \? arm. Explain the value you see.
7. Write the three-line long-form translation. Make it handle --verbose and --dry-run, then try --seeds=3 and describe what happens.
8. Write a function that calls getopts, call it twice with the same arguments, and print the result each time. Then add local OPTIND=1 and repeat.
9. Put that function in a file, source it into an interactive shell, and call it twice. Explain why this is the hardest place to notice the bug.
10. In run-suite.sh, remove the [ "${#argv[@]}" -eq 0 ] || guard and run it with no arguments. Which chapter predicted the result?
11. Add a -j JOBS option to run-suite.sh and count the lines you had to change. Compare with the same exercise in Chapter 21.1.

Check Your Understanding

1. What do the three parts of ':t:s:v' mean?
2. What three variables does getopts set, and what does each hold?
3. Name three things getopts handles that a hand-rolled case/shift loop does not.
4. What does the leading colon change? Give both effects.
5. What happens if you omit the :) arm? And when does the :) arm not fire even though a value is missing in spirit?
6. Why can't getopts parse --verbose, and what does OPTARG contain when you pass it?
7. A function parses options correctly the first time and not the second. Why, and what is the one-word fix?

7. Remember

• getopts ':t:s:v' opt is the whole specification: leading colon for silent errors, bare letter for a flag, trailing colon for a value.
• It sets $opt, $OPTARG and $OPTIND, and shift $(( OPTIND - 1 )) discards what it consumed.
• It handles clustering, attached values and -- for free — all three of which a hand-rolled loop rejects.
• Always write the leading colon. Without it Bash prints its own message and OPTARG is empty, so you cannot name the bad option.
• Always handle both \? and :. Omitting the : arm reports a missing value as an unknown option.
• The : arm fires only when the option is last on the line. -s tests.txt takes the testlist as the value — getopts checks the shape, you check the meaning.
• \? needs the backslash, because ? is a case glob metacharacter.
• getopts cannot do long options. --verbose arrives as \? with OPTARG set to -.
• Translate the two or three long forms you care about into short ones before the loop, and do not pretend to support --opt=value.
• Do not depend on external getopt(1) — it is absent on macOS and in minimal containers.
• local OPTIND=1 in every function that calls getopts, or it is correct exactly once per shell.
• Check the positional count after shift, which is the only point at which it is knowable.

Chapter 21.3 handles the other half of configuration. Flags are one source of a setting; a config file and the environment are two more, and a tool that does not state which wins behaves differently for every person who runs it.