Working on a Machine You Share

Part II · Module 22 · Chapter 22.5 · Working on a Machine You Share

Module 7's processes were yours. On a shared machine they are not: your sixteen-thread compile is why a colleague's editor is unresponsive, and their stray simulation from Tuesday is why your regression is slow today.

Nothing here is new Bash. It is four habits, and the commands to support them — each of which has a portability trap worth knowing about before you depend on it in a script.

1. Look Before You Start

The first question is whether the machine has room, and the command is uptime:

$ uptime
22:37  up 35 days,  9:39, 1 user, load averages: 4.07 4.33 5.14

Three numbers: the average number of processes wanting to run, over the last 1, 5 and 15 minutes. Chapter 7.3 introduced them; what changes on a shared machine is that they are now a question about other people. They are also the most misread numbers in Unix, for one reason:

A load average means nothing until you divide it by the core count. A load of 4 is idle on a 64-core server and a crisis on a 2-core one.

$ sysctl -n hw.ncpu
8

So 4.07 on eight cores is 0.51 per core — half-busy, with room to work. The rule of thumb that follows:

And the three numbers together tell you a direction: 4.07 4.33 5.14 is falling — the 15-minute average is highest — so the machine is recovering from something. Rising numbers mean the opposite, and a job started into a rising load is a job that will take far longer than you expect.

2. Memory Decides More Often Than CPU

For EDA work this is the correction most people need. A simulation that is slow because the machine is busy finishes late; a simulation that exhausts memory dies, or worse, pushes the machine into swap and takes everyone else down with it.

Elaborating a large design is a memory-hungry operation, and the ceiling is hard. So the question before launching four jobs is not "are there four cores" but "is there four times the memory".

$ sysctl -n hw.memsize | awk '{printf "%.1f GB\n", $1/1073741824}'
8.0 GB

The GNU tool everyone reaches for is not here:

$ command -v free
$ free -g
bash: free: command not found

free is part of GNU procps and exists on Linux only. The BSD equivalent is vm_stat, which reports in pages and tells you the page size in its own header:

$ vm_stat | head -4
Mach Virtual Memory Statistics: (page size of 16384 bytes)
Pages free:                                     3304.
Pages active:                                  69195.
Pages inactive:                                65808.

3304 free pages at 16384 bytes is about 54 MB genuinely free — which sounds alarming and is normal, because inactive pages are reclaimable. The number to watch on either platform is swap activity, not free memory: a machine using swap heavily is a machine where everyone's interactive session has become unusable.

# the portable-enough memory line for a pre-flight check
if command -v free > /dev/null; then
  free -g | awk '/^Mem:/ { printf "RAM %s GB total, %s GB available\n", $2, $7 }'
else
  sysctl -n hw.memsize | awk '{ printf "RAM %.1f GB total\n", $1/1073741824 }'
fi

command -v free > /dev/null as the test rather than checking the OS name — you care whether the tool is there, not what the kernel is called. Chapter 10.2 is the lookup, and Chapter 21.6 §1 the principle.

3. Finding the Process That Is Costing Everyone

This chapter's own curriculum entry suggests:

ps -eo user,pcpu,rss,cmd --sort=-pcpu | head

That is the GNU spelling, and on this machine it does not run:

$ ps -eo user,pcpu,rss,cmd --sort=-pcpu | head
ps: cmd: keyword not found
ps: illegal option -- -
usage: ps [-AaCcEefhjlMmrSTvwXx] [-O fmt | -o fmt] [-G gid[,gid...]]

Two separate rejections: cmd is not a recognised column name here (BSD calls it command or comm), and --sort is a GNU long option that BSD ps does not have. The BSD form that works uses -r to sort by CPU — Chapter 7.2 taught the -o format; this is the same flag meeting a vendor difference:

$ ps -axo user,pcpu,rss,comm -r | head -4
USER              %CPU    RSS COMM
engineer          95.8    640 grep
_windowserver     42.8  67616 /System/.../WindowServer
engineer          19.1 267936 /Users/engineer/.../claude

RSS is in kilobytes, which is the column people misread by a factor of a thousand. Resident set size is the physical memory the process currently holds, and converting it where it is displayed is worth the extra awk:

$ ps -axo pcpu,rss,comm -r | awk 'NR>1 && NR<=4 { printf "%5.1f%% %7.1f MB  %s\n", $1, $2/1024, $3 }'
 92.6%     0.6 MB  grep
 42.8%    67.0 MB  WindowServer
 19.1%   261.5 MB  claude

So the portable wrapper, which is the form worth keeping in your own bin:

#!/usr/bin/env bash
# top-procs — the heaviest processes, on GNU or BSD
set -o nounset
n=${1:-10}

if ps -eo user,pcpu,rss,comm --sort=-pcpu > /dev/null 2>&1; then
  ps -eo user,pcpu,rss,comm --sort=-pcpu        # GNU
else
  ps -axo user,pcpu,rss,comm -r                # BSD
fi | awk -v n="$n" 'NR==1 { print; next }
                    NR<=n+1 { printf "%-16s %5s %8.1f MB  %s\n", $1, $2, $3/1024, $4 }'

The feature test is the command itself with its output discarded — if the GNU spelling runs, use it; otherwise fall back. That is better than checking uname, because it tests the thing you actually depend on. And note comm rather than cmd, which is the column name both accept.

4. nice: Asking to Matter Less

nice runs a command at a lower scheduling priority, so it yields CPU to everything else. On a machine shared with people doing interactive work, this is the single most considerate thing you can do with a long compile.

$ nice -n 19 sh -c 'ps -o nice= -p $$'
19

The niceness scale runs from −20 (most favoured) to 19 (least), and the default is 0. 19 means "run me only when nothing else wants the CPU", which for a batch job is usually exactly right.

renice adjusts a process that is already running, and it is the apology you make after starting something at the wrong priority:

renice -n 19 -p "$pid"          # one process
renice -n 19 -u "$(id -un)"     # everything you own

The second form is worth remembering as the thing to run when you realise your regression is making the machine unusable — it affects only your own processes, so it is always safe.

5. Leave Nothing Behind

The last habit, and the one that earns the most goodwill. Three mechanisms, all from Part I, applied here.

A trap that cleans up, so an interrupted job does not leave a scratch directory full of waveforms:

work=$(mktemp -d "${TMPDIR:-/tmp}/run.XXXXXX") || exit 2
trap 'rm -rf -- "$work"' EXIT

Chapter 20.5, and it fires on INT and TERM as well as on normal exit — which is the case that matters when someone has to stop your job.

Background work that survives your logout only when you meant it to. A long job started with & and then disowned keeps running after you log out; one you forgot about keeps running for weeks. Chapter 7.5 and Chapter 7.8 are the mechanics; the habit is to write down what you started, and to check:

ps -u "$(id -un)" -o pid,etime,pcpu,comm   # everything of mine, with ages

etime is the column that finds the forgotten ones — anything measured in days that you cannot immediately account for.

And a final check before you log off, which takes five seconds:

#!/usr/bin/env bash
# mine — what am I still running, and how old is it
set -o nounset
ps -u "$(id -un)" -o pid,etime,pcpu,rss,comm 2>/dev/null |
  awk 'NR==1 { print; next }
       { if ($2 ~ /-/ || $2 ~ /^[0-9]+:[0-9]+:/) print "OLD  " $0; else print "     " $0 }'

The awk condition flags anything whose elapsed time contains a dash (days) or has three colon-separated parts (hours) — ps writes etime as [[dd-]hh:]mm:ss, so those two patterns are exactly "older than an hour". Formatting rather than filtering, so you still see everything.

Practice

1. Run uptime and sysctl -n hw.ncpu (or nproc). Compute load per core and say whether you would start a four-job build.
2. Compare your uptime output with the two formats in §1. Which does your machine print?
3. Write the naive comma-splitting parse and run it on your machine. If it returns the whole string, explain why nothing reported an error.
4. Then write the portable parse and confirm it extracts one number.
5. Try free -g. If it is absent, find the equivalent on your system and work out genuinely-free memory in MB.
6. Run the GNU ps spelling from §3. If it fails, record the exact two errors and name the BSD column that replaces cmd.
7. Write top-procs and run it. Confirm the feature test picks the right branch by temporarily breaking the GNU one.
8. Run nice -n 19 ps -o nice= -p $$ and then the sh -c form. Explain the difference in one sentence about expansion order.
9. Run nice -n -5 sh -c 'ps -o nice= -p $$' twice: once normally, once with 2>/dev/null. Describe what the second one looks like to someone debugging.
10. Start a long job, renice it to 19, and confirm with ps -o nice=.
11. Run mine and account for every process it lists. Anything you cannot explain is the point of the exercise.
12. Add etime to a ps format and find the oldest process you own.

Check Your Understanding

1. Why is a load average meaningless on its own?
2. What does 4.07 4.33 5.14 tell you beyond the current load?
3. Why does memory matter more than CPU for a large elaboration?
4. Why does the naive uptime parse fail on macOS without reporting an error?
5. What two things are wrong with ps -eo user,pcpu,rss,cmd --sort=-pcpu on BSD?
6. What units is RSS in?
7. Why does nice -n 19 ps -o nice= -p $$ print 0?
8. What happens when an ordinary user runs nice -n -5?
9. Which ps column finds forgotten processes?

6. Remember

• Divide the load average by the core count. Below ~0.7 per core, start; above 1.0, find another machine.
• The three numbers give a direction — falling means recovering, rising means do not start now.
• Memory is the hard ceiling, CPU is only slow. Check RAM before launching parallel elaborations.
• uptime's format differs: load averages: + spaces on BSD, load average: + commas on Linux. Normalise commas to spaces before parsing.
• A wrong parse returns a valid string, not an error — which is why it has to be run.
• free is GNU-only; BSD has vm_stat, in pages, with the page size in its header.
• ps -eo … --sort= is GNU; BSD is ps -axo … -r, and the column is comm, not cmd.
• RSS is in kilobytes.
• Feature-test the command, not the OS name: run the GNU form with output discarded and fall back.
• nice -n 19 for batch work. $$ in the argument reports the shell — quote it for the child.
• An ordinary user cannot go below nice 0: it warns on stderr and runs at 0, so a pipeline hides it.
• renice -n 19 -u "$(id -un)" is the safe apology — it touches only your own processes.
• A trap … EXIT on a scratch directory, and etime to find what you forgot.
• Never kill a colleague's job. Find out whose it is and ask.

7. What Is Next

Chapter 22.6 closes the module with the question every chapter in it has been circling: why does an identical command produce two different results for two engineers — and what, specifically, to record so the answer takes minutes rather than days.