Getting a Value Out: Output vs Status

Part I · Module 18 · Chapter 18.4 · Getting a Value Out: Output vs Status

Chapter 18.3 established what the status is for: how it went. Which means it cannot also be what came back. So how does a function hand you a filename, a count, a version string?

1. It Prints, and You Capture

The same way every other command does:

$ derive() { printf '%s\n' "${1%.sv}_tb.sv"; }
$ n=$(derive top.sv)
$ echo "$n"
top_tb.sv

This is the idiomatic answer and the one to reach for by default. It works for any value, it needs no shared names, and it makes the function usable in a pipeline, in a here-string, or by a different script entirely — because it behaves like basename or date.

2. The Capture Is a Subshell

Chapter 17.7's rule, arriving for the third time and now about functions:

$ f() { result=INSIDE; printf 'out\n'; }
$ result=before
$ v=$(f)
$ echo "v=$v result=$result"
v=out result=before

result is still before. The function did assign it — in the child process that $( ) created, which then exited.

Call the same function directly and the assignment sticks:

$ result=before
$ f > /dev/null
$ echo "result=$result"
result=INSIDE

So a function cannot both be captured and set variables for its caller. If it needs to do both, it must not be captured — which is §5's convention.

3. Three Things Command Substitution Does to the Text

It strips every trailing newline.

$ f() { printf 'a\n\n\n'; }
$ v=$(f); printf 'v=[%s]\n' "$v"
v=[a]
$ f() { printf 'a\nb\n'; }
$ v=$(f); printf 'v=[%s]\n' "$v"
v=[a
b]

All three trailing newlines went; the interior one survived. This is usually a convenience — it is why n=$(wc -l < f) gives you a number and not a number-plus-newline — and it is a problem only if trailing blank lines are data, in which case you need a sentinel.

It does not strip leading or interior whitespace, which is why $(wc -l < f) on BSD still carries the leading padding Chapter 16.4 hit.

And it captures stdout only — which is the sharp edge.

4. A Log Line in the Wrong Stream Ends Up Inside Your Value

$ bad()  { echo "LOG: working";      printf '%s\n' RESULT; }
$ v=$(bad);  printf 'v=[%s]\n' "$v"
v=[LOG: working
RESULT]

The log is now part of the value. Nothing errored. $v is two lines, the first of which is diagnostic text, and every later use of $v is wrong in a way that will be blamed on something else.

One character fixes it:

$ good() { echo "LOG: working" >&2; printf '%s\n' RESULT; }
$ v=$(good); printf 'v=[%s]\n' "$v"
LOG: working
v=[RESULT]

The log still appears on your terminal — it went to stderr, which $( ) does not capture — and $v is exactly the answer.

5. The Status Survives, Even Though the Variables Do Not

A genuinely useful asymmetry:

$ f() { echo out; return 3; }
$ v=$(f); echo "v=$v status=$?"
v=out status=3

The subshell's exit status becomes the command substitution's status. So the standard guard works:

name=$(derive_tb "$src") || die "could not derive a name from $src"

That is what made Chapter 18.3 §7's fix — name=$(derive_name "") || exit 1 — possible even though the die inside could not kill the script.

And here is the one-word mistake that throws it away:

$ f() { echo out; return 3; }
$ g() { local v=$(f); echo "local v=\$(f)  -> status $?"; }; g
local v=$(f)  -> status 0

$ g() { local v; v=$(f); echo "local v; v=\$(f) -> status $?"; }; g
local v; v=$(f) -> status 3

local v=$(f) reports 0 regardless. The status you get is local's own — and local almost always succeeds. The command substitution's status is discarded before you can look at it.

Chapter 11.1 measured this for local; it is identical for declare, export, readonly and typeset. It matters far more in a function-heavy script, because local x=$(...) is such a natural thing to write:

# BROKEN — never detects a failure
load_version() {
  local v=$(cat "$1/VERSION")        # file missing? status is still 0
  [ -n "$v" ] || return 1            # only catches it because the value is empty
  printf '%s\n' "$v"
}

# correct — declare, then assign
load_version() {
  local v
  v=$(cat "$1/VERSION") || return 1
  printf '%s\n' "$v"
}

The broken version limps along because an empty value happens to correlate with failure. It stops limping the first time a command fails and prints something — which is most of them.

Declare on one line, assign on the next. Every time. It is two extra characters and it is the difference between a function that detects errors and one that cannot.

6. Or Have It Assign an Agreed Variable

The other mechanism. The function sets a variable; the caller reads it:

$ derive2() { REPLY_OUT="${1%.sv}_tb.sv"; }
$ derive2 top.sv
$ echo "$REPLY_OUT"
top_tb.sv

No subshell, no fork, no capture — so assignments survive, a cd survives, and in a loop over ten thousand items it costs nothing where $( ) would cost ten thousand processes.

The price is a shared name, which is a contract both sides must honour and which is invisible at the call site. derive2 top.sv does not look like it produces anything. Three ways to make it bearable, in increasing sophistication:

A documented convention. Bash's own read does this — it sets REPLY when you give it no variable name — so an all-caps REPLY_-prefixed name is idiomatic and recognisable. Document it in the function's comment and never reuse the name elsewhere.

Let the caller name the variable, via printf -v:

$ setvar() { printf -v "$1" '%s' "$2"; }
$ setvar myname hello
$ echo "$myname"
hello

printf -v NAME writes into the variable named by $NAME instead of printing. So a function can take the output variable's name as its first argument:

# derive_into <varname> <source.sv>
derive_into() {
  [ $# -eq 2 ] || return 2
  printf -v "$1" '%s_tb.sv' "${2%.sv}"
}

$ derive_into tb top.sv
$ echo "$tb"
top_tb.sv

The call site now shows which variable gets written, and the function forks nothing. This is the best option available on Bash 3.2.

And the thing you will see in older code, which you should not copy:

$ setvar_eval() { eval "$1=\"\$2\""; }
$ setvar_eval myname hello
$ echo "$myname"
hello

eval works and is a command-injection hole the moment either argument can be influenced by anything outside your script. printf -v does the same job with no evaluation at all. Use printf -v.

7. More Than One Value

A function returning several values is where output capture shows its limits.

One value per line, read back with a loop — correct, including for values with spaces:

$ list() { printf '%s\n' a "b c" d; }
$ while IFS= read -r x; do printf '[%s]\n' "$x"; done < <(list)
[a]
[b c]
[d]

Three items, b c intact. That is Chapter 17.5's loop fed by Chapter 17.7's process substitution, and it is the portable answer.

Capturing into one string and re-splitting — broken, for exactly the reason Chapter 17.2 gave:

$ v=$(list); for x in $v; do printf '[%s]\n' "$x"; done
[a]
[b]
[c]
[d]

Four items. The newline boundaries were destroyed by word-splitting, and b c became two values.

The real answer is an array, which is Chapter 19.2 — and the reason Module 19 exists at all is that a string genuinely cannot represent a list. For now: if a function must return several values, print one per line and have the caller read them in a loop. Never capture-and-resplit.

8. Choosing Between Them

Default to print-and-capture. It is what the rest of Unix does, it keeps the two sides independent, and the fork cost is irrelevant for anything called tens or hundreds of times.

Switch to an assigned variable when the function is called in a tight loop, when it must also change state the caller sees, or when the value could contain trailing newlines that matter.

9. A Complete Script

Both mechanisms, each where it belongs:

#!/usr/bin/env bash
# name-logs.sh — derive output names for a list of sources
# usage: name-logs.sh <srcdir> <outdir>
#
# exit: 0 all named · 1 at least one source was unusable · 2 bad usage

set -o nounset

# every message goes to stderr, so no caller can pollute a capture
note() { printf '%s: %s\n' "${0##*/}" "$*" >&2; }
die()  { note "$*"; exit 2; }

# derive_tb <source> -> PRINTS the testbench name; 1 if not a source file
# captured by the caller, so nothing but the answer goes to stdout
derive_tb() {
  [ $# -eq 1 ] || return 2
  case $1 in
    *.sv|*.v) ;;
    *) note "not an HDL source: $1"; return 1 ;;
  esac
  printf '%s_tb%s\n' "${1%.*}" ".${1##*.}"
}

# count_lines <file> -> PRINTS the line count; status says whether it read it
count_lines() {
  [ $# -eq 1 ] || return 2
  local n
  n=$(wc -l < "$1") || return 1      # declare, then assign — §5
  printf '%s\n' "$(( n ))"           # arithmetic strips BSD wc's padding
}

# tally_into <varname> <file>... -> ASSIGNS the total; no fork per file
# used in a loop, so it avoids one subshell per source
tally_into() {
  [ $# -ge 1 ] || return 2
  local var=$1; shift
  local total=0 n
  local f
  for f in "$@"; do
    n=$(count_lines "$f") || continue
    total=$(( total + n ))
  done
  printf -v "$var" '%s' "$total"
}

# ---- top level -----------------------------------------------------------

srcdir=${1:-}
outdir=${2:-}

[ -n "$srcdir" ] && [ -n "$outdir" ] || die "usage: ${0##*/} <srcdir> <outdir>"
[ -d "$srcdir" ] || die "$srcdir is not a directory"
mkdir -p "$outdir" || die "cannot create $outdir"

bad=0
sources=""

for f in "$srcdir"/*; do
  [ -e "$f" ] || continue
  [ -f "$f" ] || continue

  base=${f##*/}

  tb=$(derive_tb "$base") || { bad=$(( bad + 1 )); continue; }

  lines=$(count_lines "$f") || { note "cannot read $f"; bad=$(( bad + 1 )); continue; }

  printf '%-16s -> %-20s %4s lines\n' "$base" "$tb" "$lines"
  sources="$sources $f"
done

tally_into grand_total $sources
printf -- '-- %s lines across all sources, %d skipped\n' "$grand_total" "$bad"

[ "$bad" -eq 0 ]

$ mkdir src
$ printf 'module top;\nendmodule\n'                     > src/top.sv
$ printf 'module alu;\nwire x;\nendmodule\n'            > src/alu.v
$ printf 'notes\n'                                      > src/README.md
$ ./name-logs.sh src out
name-logs.sh: not an HDL source: README.md
alu.v            -> alu_tb.v                3 lines
top.sv           -> top_tb.sv               2 lines
-- 5 lines across all sources, 1 skipped
$ echo $?
1

$ rm src/README.md
$ ./name-logs.sh src out
alu.v            -> alu_tb.v                3 lines
top.sv           -> top_tb.sv               2 lines
-- 5 lines across all sources, 0 skipped
$ echo $?
0

$ ./name-logs.sh
name-logs.sh: usage: name-logs.sh <srcdir> <outdir>
$ echo $?
2

(Alphabetical order, because "$srcdir"/* is a glob and Chapter 3.5 established that filename expansion sorts.)

Now add a source you cannot read, and watch the §5 rule earn its place:

$ printf 'x\n' > src/locked.sv && chmod 000 src/locked.sv
$ ./name-logs.sh src out
alu.v            -> alu_tb.v                3 lines
./name-logs.sh: line 16: src/locked.sv: Permission denied
name-logs.sh: cannot read src/locked.sv
top.sv           -> top_tb.sv               2 lines
-- 5 lines across all sources, 1 skipped
$ echo $?
1

Detected, reported, counted, and the script exits 1. Now collapse count_lines' two lines into one — local n=$(wc -l < "$1") || return 1 — and run the identical directory:

$ ./bad.sh src out
alu.v            -> alu_tb.v                3 lines
./bad.sh: line 15: src/locked.sv: Permission denied
locked.sv        -> locked_tb.sv            0 lines
top.sv           -> top_tb.sv               2 lines
-- 5 lines across all sources, 0 skipped
$ echo $?
0

The unreadable file is now reported as 0 lines, the skip count is 0, and the script exits success. The only surviving trace is wc's own Permission denied on stderr — which in a CI log scrolls past above a green summary. One word, local, moved from its own line onto the assignment, and the script stopped being able to detect failure.

Five decisions, each priced by this chapter:

derive_tb and count_lines print. They are pure functions of their argument, their callers want the value, and printing keeps them usable from anywhere. Both send every non-answer to stderr via note.

tally_into assigns. It is the one that loops, so print-and-capture would have cost a subshell per call on top of the ones count_lines already needs. It takes the variable name as its first argument, so the call site says where the answer lands.

local n then n=$(wc -l < "$1") — on two lines, so || return 1 can see wc's status. Written as local n=$(wc -l < "$1") the function would report success for a file it could not read.

$(( n )) strips BSD wc's leading padding, which Chapter 16.4 measured.

note "not an HDL source: $1" is inside derive_tb, which is captured — and it still works, because note writes to stderr. That is the §4 callout paying for itself: the message appears on the terminal, and $tb contains nothing but the name.

Practice

1. Write f() { result=INSIDE; printf 'out\n'; }. Capture it with $( ) and print $result; then call it directly and print $result. Name the mechanism that explains the difference.
2. Capture a function that prints a followed by three newlines. How many newlines are in the value? Then find a value where that stripping loses real data.
3. Write bad() that logs to stdout and good() that logs to stderr, both printing a result. Capture each and print the value in brackets. Then explain why putting >&2 inside note protects every call site at once.
4. Write f() { echo out; return 3; }. Capture it and print $?. Then do it as local v=$(f) inside a function and print $? again. Which is 0, and why?
5. Take the broken load_version from §5, point it at a missing file, and show that it does not detect the failure. Fix it and show that it does.
6. Write derive_into <varname> <src> with printf -v. Then write the eval version and say precisely what input would make the eval one dangerous.
7. Run f() { local -n r=$1; r=X; }; f out on your machine. If it fails, say what that rules out for a script that must run on macOS.
8. Write list() printing a, b c, d. Read it with a while read loop over process substitution, then capture-and-resplit. Count the items each way.
9. In name-logs.sh, change local n / n=$(...) into local n=$(...). Make a source file unreadable with chmod 000 and show the script no longer notices.
10. Convert tally_into to the print-and-capture style. Use /usr/bin/time -p on a directory of 500 files to measure what the change costs.

Check Your Understanding

1. Why can't a Bash function return a value?
2. A function sets a variable and you capture it with $( ). Is the variable set afterwards?
3. What does command substitution do to trailing newlines?
4. A captured function's log line appeared inside the value. What went wrong?
5. Does a function's exit status survive $( )?
6. What does local v=$(f) report, and what should you write instead?
7. When is assigning an agreed variable better than printing?

10. Remember

• A function cannot return a value. Print it, or assign an agreed variable.
• Print-and-capture is the default. It matches the rest of Unix and keeps both sides independent.
• $( ) is a subshell, so nothing the function assigned comes back — but the exit status does.
• Command substitution strips every trailing newline and captures stdout only.
• A log line on stdout lands inside your value. Put >&2 inside the logging helper so every call site is safe.
• local v=$(f) reports local's status, not the function's. Declare on one line, assign on the next — same for declare, export, readonly.
• printf -v "$1" lets the caller name the output variable with no eval and no fork. local -n is Bash 4.3 and absent on macOS.
• Never eval "$1=..." when printf -v does the same job without evaluating anything.
• For several values, print one per line and read them in a loop. Capture-and-resplit destroys any value containing a space.
• Switch to an assigned variable for tight loops, for functions that must also change state, or when trailing newlines matter.

Chapter 18.5 is the hazard that makes the assigned-variable convention risky in the first place: in Bash, a variable a function assigns is shared by default, and two functions that happen to pick the same short name will corrupt each other.