while and until

Part I · Module 17 · Chapter 17.4 · while and until

Both loops so far needed the count in advance — a list, or a range. Neither can express keep going until something changes, and that is most of what automation waits on.

1. while Takes a Command

$ i=0
$ while [ "$i" -lt 3 ]; do printf '%s ' "$i"; i=$(( i + 1 )); done; echo
0 1 2

The important word in that line is not while. It is that [ "$i" -lt 3 ] is a command — Chapter 16.2's [, the real program — and while does exactly what Chapter 16.1's if does: runs it, reads the status, and continues while that status is zero.

So every condition form from Module 16 works here unchanged, for the same reason and with the same spellings:

$ i=0; while (( i < 3 ));          do printf '%s ' "$i"; i=$(( i + 1 )); done; echo
0 1 2
$ i=0; while [[ $i -lt 3 ]];       do printf '%s ' "$i"; i=$(( i + 1 )); done; echo
0 1 2

And so does a condition that is not a test at all — Chapter 16.7's form, which is where while earns its keep:

$ printf 'a\nb\n' | while read -r l; do echo "got $l"; done
got a
got b

read returns 0 while it successfully read a line and non-zero at end of input, so while read means keep going while there is more input. That is Chapter 17.5, in full, and it is the most-used while loop in existence.

2. until Is while With the Test Inverted

$ i=0; until [ "$i" -ge 3 ]; do printf '%s ' "$i"; i=$(( i + 1 )); done; echo
0 1 2

until runs its body while the command fails — it continues until the command succeeds. The two loops above produce identical output, with -lt 3 and -ge 3 being the inverse tests.

There is no mechanical reason to prefer either; until X is exactly while ! X. Choose whichever makes the sentence read correctly:

until [ -f "$lockfile" ]; do sleep 1; done        # until the lock appears
while [ -f "$lockfile" ]; do sleep 1; done        # while the lock is held

Both are clear. What is not clear is a negated until — until ! [ -f x ] makes a reader stop and work it out, so if you find yourself writing one, switch to while.

3. The Deliberate Infinite Loop

$ n=0
$ while true; do n=$(( n + 1 )); [ "$n" -ge 3 ] && break; done; echo "n=$n"
n=3

while true is an infinite loop, and it is spelled with a command because that is all a condition can be — true is the builtin from Chapter 16.1 whose only job is to return zero. until false is the same thing, and for (( ; ; )) from Chapter 17.3 is a third spelling. while true is the one to write; it reads as English.

An infinite loop is a legitimate structure, not an admission of defeat. It is the right shape when the exit condition is awkward to state up front or when there are several ways out:

while true; do
  if [ -f "$done_flag" ]; then break; fi
  if [ -f "$abort_flag" ]; then echo "aborted" >&2; exit 1; fi
  if (( SECONDS > 600 )); then echo "timed out" >&2; exit 1; fi
  sleep 5
done

Three exits, each with its own message and status, and the condition slot stays empty because none of them belongs there. break and the loop's status are Chapter 17.6.

4. A Poll Without a Pause Is a Core at Full Load

This is the measurement of the chapter, and it is the thing people get wrong in production.

Here is the canonical wait — another job will create a flag file in two seconds:

$ ( sleep 2; touch /tmp/m17.flag ) &
$ /usr/bin/time -p bash -c 'while [ ! -f /tmp/m17.flag ]; do :; done'
real 2.01
user 0.90
sys 0.46

It worked. It also spent 1.36 seconds of CPU to wait 2 seconds of wall time, because : does nothing and the loop therefore re-tests as fast as the machine allows. How fast is that?

$ ( sleep 2; touch /tmp/m17.flag ) &
$ bash -c 'n=0; while [ ! -f /tmp/m17.flag ]; do n=$(( n + 1 )); done; echo "$n tests"'
107635 tests

One hundred and seven thousand checks of the same file, 107,634 of which were certain to say no. Now add one line:

$ ( sleep 2; touch /tmp/m17.flag ) &
$ /usr/bin/time -p bash -c 'while [ ! -f /tmp/m17.flag ]; do sleep 0.2; done'
real 2.06
user 0.02
sys 0.05

$ ( sleep 2; touch /tmp/m17.flag ) &
$ bash -c 'n=0; while [ ! -f /tmp/m17.flag ]; do n=$(( n + 1 )); sleep 0.2; done; echo "$n tests"'
9 tests

Nine tests. 0.07 seconds of CPU. The same wall time — 2.06 against 2.01, a difference you cannot notice.

Never poll without a pause. On a shared compute farm a spin-wait is worse than slow — it occupies a slot the scheduler could have given to real work, and a wrapper that spins while waiting for a licence can starve the very job it is waiting for.

Choosing the interval is a trade: the answer arrives, on average, half an interval late. For something that takes minutes, sleep 5 is right and costs you 2.5 seconds of latency. For something that takes hours, sleep 60. Sub-second intervals are almost never justified in a shell script — if you genuinely need to react in milliseconds, a shell poll is the wrong tool.

5. Always Bound the Wait

A poll with no limit is a script that hangs forever when the thing it waits for never arrives — and it hangs silently, which is the worst way for automation to fail. Bound it.

By attempts:

tries=0
while [ ! -f "$flag" ]; do
  tries=$(( tries + 1 ))
  if [ "$tries" -ge 60 ]; then
    echo "${0##*/}: $flag did not appear after $tries checks" >&2
    exit 1
  fi
  sleep 5
done

By deadline, which is what you want when the body's own duration varies:

$ end=$(( $(date +%s) + 1 ))
$ n=0
$ while [ ! -f /tmp/never ]; do
>   n=$(( n + 1 ))
>   if [ "$(date +%s)" -ge "$end" ]; then echo "timed out after $n polls"; break; fi
>   sleep 0.1
> done
timed out after 3 polls

A deadline is honest about what you actually promised — this will not take more than five minutes — whereas an attempt count only approximates it, and stops approximating as soon as anything in the body gets slower.

6. The Exit Status

Both loops report the status of the last body iteration, exactly as for does — and both report 0 when the body never ran:

$ while false; do echo BODY; done; echo "status=$?"
status=0
$ until true;  do echo BODY; done; echo "status=$?"
status=0
$ i=0; while [ "$i" -lt 2 ]; do i=$(( i + 1 )); false; done; echo "status=$?"
status=1

The first two are the same trap Chapter 17.1 §5 described: a loop that did nothing is indistinguishable from a loop that worked. For a polling loop that is usually what you want — the condition was already satisfied, so there was nothing to wait for. For a processing loop it usually is not, and you count.

7. A Complete Script

A licence-wait wrapper, which is where these loops really live:

#!/usr/bin/env bash
# wait-for-licence.sh — wait for a licence to free up, then run the job
# usage: wait-for-licence.sh <timeout-seconds> <command> [args...]
#
# exit: the command's status · 2 bad usage · 3 timed out waiting

set -o nounset

timeout=${1:-}
shift || true

if [ -z "$timeout" ] || [ $# -eq 0 ]; then
  echo "usage: ${0##*/} <timeout-seconds> <command> [args...]" >&2
  exit 2
fi
if ! [ "$timeout" -ge 1 ] 2>/dev/null; then
  echo "${0##*/}: timeout must be a positive integer, got: $timeout" >&2
  exit 2
fi

# stand-in for `lmstat -a | grep -c "licences available"`
licence_free() {
  [ -f /tmp/licence.free ]
}

start=$SECONDS
polls=0
interval=2

until licence_free; do
  polls=$(( polls + 1 ))
  elapsed=$(( SECONDS - start ))

  if (( elapsed >= timeout )); then
    printf '%s: no licence after %ds (%d polls) — giving up\n' \
      "${0##*/}" "$elapsed" "$polls" >&2
    exit 3
  fi

  # tell the user once, not every poll
  if (( polls == 1 )); then
    printf '%s: waiting for a licence (up to %ds)\n' "${0##*/}" "$timeout" >&2
  fi

  sleep "$interval"
done

if (( polls > 0 )); then
  printf '%s: licence free after %ds\n' "${0##*/}" "$(( SECONDS - start ))" >&2
fi

exec "$@"

$ touch /tmp/licence.free
$ ./wait-for-licence.sh 30 echo "running the job"
running the job
$ echo $?
0

$ rm /tmp/licence.free
$ ( sleep 5; touch /tmp/licence.free ) &
$ ./wait-for-licence.sh 30 echo "running the job"
wait-for-licence.sh: waiting for a licence (up to 30s)
wait-for-licence.sh: licence free after 6s
running the job
$ echo $?
0

$ rm -f /tmp/licence.free
$ ./wait-for-licence.sh 5 echo "running the job"
wait-for-licence.sh: waiting for a licence (up to 5s)
wait-for-licence.sh: no licence after 6s (4 polls) — giving up
$ echo $?
3

$ ./wait-for-licence.sh 0 true
wait-for-licence.sh: timeout must be a positive integer, got: 0
$ echo $?
2

$ touch /tmp/licence.free
$ ./wait-for-licence.sh 10 bash -c 'exit 7'
$ echo $?
7

That last one is exec doing its job: the command's status is the script's status, with no shell in between to flatten or reinterpret it.

Six decisions worth naming:

until licence_free reads as a sentence. The condition is a function (Chapter 18.1), which is Chapter 16.7's point taken to its conclusion: the clearest condition is often one you named yourself.

The wait is bounded by a deadline, not a count, using SECONDS. If the licence server gets slow to answer, an attempt count would silently stretch the promised timeout; elapsed time cannot.

sleep "$interval" is there, and §4 is the reason. Without it this script would hold a core for the entire wait — on a farm, possibly starving the job holding the licence it wants.

The "waiting" message prints once, on the first poll, not every two seconds. A wrapper that logs a line per poll makes a thirty-minute wait unreadable.

It exits 3 for a timeout, distinct from whatever the command would have returned. The caller can tell "your job failed" from "your job never started", which Chapter 15.6 argued is the most valuable distinction a script can offer.

exec "$@" replaces the shell rather than running the command as a child. The wrapper has nothing left to do, so there is no reason to keep a shell alive holding a process slot — and the command inherits the real process, so signals and exit status pass through with nothing in the middle.

Practice

1. Write the 0 1 2 loop three ways: while [ ], while (( )) and until [ ]. Then delete the increment from one and describe exactly what happens before you run it.
2. Reproduce the §4 measurement. Start a background job that touches a flag after 2 seconds, then time a spin-wait and a sleep 0.2 wait. Report user and sys for both, and the ratio.
3. Count the tests each version performs. Explain why the spin's count varies between runs while the sleep version's barely does.
4. Change the sleep from 0.2 to 5 in a wait for a 2-second event. What is the wall time now, and what does that tell you about choosing an interval?
5. Write a poll bounded by attempts, then the same one bounded by SECONDS. Make the body slower with sleep 1 and show which bound still honours its promise.
6. Replace while true; do ... break; done with an until loop that has the condition in the condition slot. Which version is clearer for a single exit, and which for three?
7. Start a background job with & and wait for it two ways — a polling loop on a flag file, and wait "$!". Which gives you the job's exit status?
8. Run wait-for-licence.sh with the licence already free and confirm no waiting message appears. Then explain what (( polls > 0 )) is protecting against.
9. Replace exec "$@" with "$@" in wait-for-licence.sh. Use ps during a long command to show the difference, and say why exec is preferable here.

Check Your Understanding

1. What does while actually test?
2. until X is equivalent to what?
3. Why is while true spelled with a command rather than a keyword?
4. A two-second spin-wait cost 1.36 seconds of CPU and performed 107,635 tests. What did the one-line fix cost, and what did it save?
5. Why bound a polling loop, and why is a deadline better than an attempt count?
6. You started the job yourself with &. Why should you not poll for it?
7. while false; do echo hi; done — what is the exit status?

8. Remember

• while tests a command's exit status, like if. Every Module 16 condition form works unchanged, including read.
• Nothing updates the condition for you. If you cannot point at the line that will make it false, it is an infinite loop.
• until X is while ! X. Choose for readability; never write a negated until.
• while true is the idiomatic infinite loop, and a legitimate structure when there are several ways out.
• Never poll without a pause. Measured: a 2-second wait costs 1.36s of CPU and 107,635 tests bare, against 0.07s and 9 tests with sleep 0.2 — at the same wall time.
• The interval costs you half its length in latency, on average. Seconds for minutes-long waits; sub-second almost never.
• Always bound a wait, and prefer a deadline (SECONDS) over an attempt count.
• Exit with a distinct status for a timeout so the caller can tell "it failed" from "it never started".
• If you started the job, use wait, not a poll. It is free and it gives you the status.
• Both loops report the last body's status, and 0 if the body never ran.

Chapter 17.5 is the while loop you will write more than all the others combined — reading a file one line at a time. It has three separate traps, and the default spelling walks into all three.