Passing Variables With -v

Part II · Module 8 · Chapter 8.2 · Passing Variables With -v

1. Why This Matters

Chapter 8.1 moved the program into a file, which immediately raises a question: the program no longer sits inside the shell script, so how does it learn the shell's values?

The answer is one option — -v name=value — and the reason this chapter exists is that the obvious alternative is worse than it looks. Building the value into the program text with a double-quoted string is the first thing most people try, it works on the values you test with, and section 6 shows what it does with a value you did not choose.

2. -v name=value

alu 0.5
fifo -0.2
uart 0.1

awk -v limit=0 '$2 + 0 < limit { print $1, $2 }' timing.rpt

fifo -0.2

limit is an ordinary awk variable holding 0. Everything Chapter 5.1 said about variables applies to it.

Several -v options are allowed:

awk -v lo=-1 -v hi=0 'BEGIN { print "window:", lo, "to", hi }' </dev/null

window: -1 to 0

3. -v Is Visible in BEGIN

The property that makes -v the right choice for a threshold:

awk -v limit=-0.05 'BEGIN { print "BEGIN sees limit =", limit }' </dev/null

BEGIN sees limit = -0.05

A -v assignment happens before BEGIN runs, so the value can be validated, defaulted or used to configure FS there.

That matters because BEGIN is where Chapter 8.3's input validation goes, and a threshold that is not yet set cannot be checked.

4. A file=value Operand Is Not

awk also accepts assignments among the filenames, and they behave completely differently:

awk 'BEGIN { print "BEGIN sees limit = [" limit "]" } { print "record:", limit }' limit=7 timing.rpt

BEGIN sees limit = []
record: 7
record: 7
record: 7

Empty in BEGIN. The assignment is processed when awk reaches that point in the operand list, which is after BEGIN has already run.

Its real use is that it takes effect between files, so a parameter can change per input:

awk '{ print FILENAME, "limit=" limit }' timing.rpt limit=99 timing.rpt

timing.rpt limit=
timing.rpt limit=
timing.rpt limit=
timing.rpt limit=99
timing.rpt limit=99
timing.rpt limit=99

The same file twice, with limit unset for the first pass and 99 for the second. That is genuinely useful — a per-file threshold, or a label identifying which run a file came from — and it is the only thing this form does that -v cannot.

5. Which to Use

-v for anything the program needs before reading — thresholds, modes, labels, FS. This is almost always what you want.

A file=value operand only when the value must differ per input file.

ENVIRON when the value contains backslashes, which section 8 explains.

6. Do Not Build the Value Into the Program

Here is the alternative most people reach for first. Switch to double quotes and let the shell substitute:

L=0
awk "\$2 + 0 < $L { print \$1 }" timing.rpt

fifo

It works. The $2 had to be escaped so the shell would not eat it, which is already a warning sign, but the output is correct.

Now the same command with a value nobody chose deliberately:

L='0 { print "INJECTED" } #'
awk "\$2 + 0 < $L { print \$1 }" timing.rpt

INJECTED

The variable's contents ran as awk code. The program awk actually received was $2 + 0 < 0 { print "INJECTED" } # { print $1 } — a complete, valid program that does something else entirely, with the intended action commented out.

The same value passed with -v is inert:

L='0 { print "INJECTED" } #'
awk -v L="$L" '$2 + 0 < L { print $1 }' timing.rpt

fifo

L is a string. $2 + 0 < L compares a number against it, finds -0.2 is less, and prints fifo — the original intent, unaffected.

7. -v Values Are Numeric Strings

A -v value that looks like a number is treated as one in comparisons:

awk -v x=10 'BEGIN { print "x < 9 ?", (x < 9 ? "yes" : "no") }' </dev/null

x < 9 ? no

Numeric comparison, correctly — 10 is not less than 9.

That is worth stating because a string constant behaves differently, which Chapter 8.5 covers in full: "10" < 9 is true, because a quoted constant forces a text comparison. A -v value sits with fields on the numeric-string side of that rule, which is almost always what you want.

A non-numeric value converts to 0 in arithmetic, per Chapter 5.3:

awk -v x=abc 'BEGIN { print "x + 0 =", x + 0 }' </dev/null

x + 0 = 0

So a mistyped threshold becomes 0 silently — which is why Chapter 8.3 validates -v values in BEGIN rather than trusting them.

8. -v Processes Escape Sequences

The one way a -v value is still interpreted. Backslash sequences are expanded as if the value were a string literal:

awk -v s='a\tb' 'BEGIN { print "[" s "]"; print "length:", length(s) }' </dev/null | cat -et

[a^Ib]$
length: 3$

Three characters, and the middle one is a real tab. The shell passed four characters — a, \, t, b — and awk turned \t into a tab.

That is convenient for -v OFS='\t' and destructive for anything else containing a backslash:

awk -v p='C:\temp\new' 'BEGIN { print "[" p "]" }' </dev/null | cat -et

[C:^Iemp$
ew]$

The path is gone. \t became a tab and \n became a newline, so a single-line Windows path arrived as two lines containing a tab.

A regex suffers the same way, and this one is easy to miss because the result is still a valid pattern:

awk -v re='^u_\w+' 'BEGIN { print "[" re "]" }' </dev/null

[^u_w+]

\w silently became w, so the pattern now matches a literal w repeated rather than a word character. Doubling the backslash preserves it:

awk -v re='^u_\\w+' 'BEGIN { print "[" re "]" }' </dev/null

[^u_\w+]

9. The Three Channels

10. Position Matters

-v must come before the program text. After it, it is not an option:

awk 'BEGIN { print "v = [" v "]" }' -v v=5 </dev/null

v = []

Empty. awk took the program from the first non-option argument, so -v and v=5 became operands rather than an option — and v=5 would then be processed as a file=value assignment, after BEGIN.

The habit: options, then the program, then the files.

11. Common Mistakes

Splicing a shell variable into the program text. Its contents become code — section 6.

Escaping \$1 to protect fields from the shell. That means you are in double quotes; switch to -v.

Expecting a file=value operand in BEGIN. It is processed later — section 4.

Passing a path or regex with -v. Escape processing destroys it — section 8.

Trusting a -v threshold without validating it. A typo becomes 0 silently — section 7.

Putting -v after the program. It is not recognised there — section 10.

Walking ARGV assuming every element is a filename. An operand assignment is not — section 4.

Forgetting to export an ENVIRON value. Set it inline on the command or export it.

12. Exercises

Exercise 1 — Predict the output

awk -v a=2 -v b=3 'BEGIN { print a * b }' </dev/null

6

Two -v options, both assigned before BEGIN.

Exercise 2 — Pass a threshold

Report paths whose slack is below a limit supplied from outside.

alu 0.5
fifo -0.2
uart 0.1

awk -v limit=0.2 '$2 + 0 < limit { print $1, $2 }' timing.rpt

fifo -0.2
uart 0.1

Two paths below 0.2.

Exercise 3 — BEGIN visibility

Predict both, then explain the difference.

awk -v limit=7 'BEGIN { print "[" limit "]" }' </dev/null

[7]

awk 'BEGIN { print "[" limit "]" }' limit=7 /dev/null

[]

A -v assignment precedes BEGIN; an operand assignment is processed when awk reaches it in the operand list, which is later.

Exercise 4 — Debug an injected program

A shell script's threshold came from a config file and the awk command printed something nobody wrote.

L='0 { print "INJECTED" } #'
awk "\$2 + 0 < $L { print \$1 }" timing.rpt

INJECTED

The shell substituted L into the program text, so awk received $2 + 0 < 0 { print "INJECTED" } # { print $1 } — a valid program with the intended action commented out.

L='0 { print "INJECTED" } #'
awk -v L="$L" '$2 + 0 < L { print $1 }' timing.rpt

fifo

With -v the value is data. L is a string, $2 + 0 < L compares against it, and fifo prints as intended.

Exercise 5 — Debug a destroyed path

A script passes an output directory with -v and the filename it builds is wrong.

awk -v p='C:\temp\new' 'BEGIN { print "[" p "]" }' </dev/null | cat -et

[C:^Iemp$
ew]$

-v expanded \t to a tab and \n to a newline.

MYPATH='C:\temp\new' awk 'BEGIN { print "[" ENVIRON["MYPATH"] "]" }' </dev/null | cat -et

[C:\temp\new]$

ENVIRON does no escape processing, so the path arrives intact.

Exercise 6 — Engineering application

Extend Chapter 8.1's signoff script to take its coverage threshold from outside, validate it in BEGIN, and refuse a bad value before reading any input.

# signoff2.awk — merge timing and coverage, with an external threshold.
#
# Usage:  awk -v min_cov=80 -f signoff2.awk timing.csv cov.csv
# Exit:   0 all blocks complete and at or above min_cov
#         1 at least one block incomplete or below min_cov
#         2 could not evaluate (bad threshold, or empty input)

BEGIN {
    FS = ","
    # Validate the threshold before any input is read. A -v value that is
    # not numeric converts to 0 silently, which would pass everything.
    if (min_cov == "") { print "ERROR: min_cov not set (use -v min_cov=N)" > "/dev/stderr"; bail = 2; exit 2 }
    if (min_cov !~ /^[0-9]+(\.[0-9]+)?$/) { printf "ERROR: min_cov is not a number: [%s]\n", min_cov > "/dev/stderr"; bail = 2; exit 2 }
    if (min_cov + 0 > 100) { printf "ERROR: min_cov %s exceeds 100\n", min_cov > "/dev/stderr"; bail = 2; exit 2 }
}

FILENAME == ARGV[1] { slack[$1] = $2; key[$1]; next }
{ cov[$1] = $2; key[$1] }

END {
    if (bail) exit bail
    if (!NR) { print "ERROR: no records in either input" > "/dev/stderr"; exit 2 }
    for (k in key) {
        n++
        have_t = (k in slack); have_c = (k in cov)
        if (!have_t || !have_c) { incomplete++; st = "INCOMPLETE" }
        else if (cov[k] + 0 < min_cov + 0) { low++; st = "LOW_COV" }
        else st = "ok"
        printf "%-6s %8s %6s  %s\n", k, (have_t ? slack[k] : "-"), (have_c ? cov[k] : "-"), st | "sort"
    }
    close("sort")
    printf "(%d block(s); %d incomplete, %d below %s%%)\n", n, incomplete + 0, low + 0, min_cov
    if (incomplete || low) exit 1
}

alu,0.5
fifo,-0.2
uart,0.1

alu,95
uart,88
ddr,70

awk -v min_cov=90 -f signoff2.awk timing.csv cov.csv
echo "status: $?"

alu         0.5     95  ok
ddr           -     70  INCOMPLETE
fifo       -0.2      -  INCOMPLETE
uart        0.1     88  LOW_COV
(4 block(s); 2 incomplete, 1 below 90%)
status: 1

uart at 88% is now LOW_COV against a threshold of 90, which it was not against 80 — the threshold is genuinely external.

The three BEGIN guards each refuse a different bad call. Unset:

awk -f signoff2.awk timing.csv cov.csv 2>&1
echo "status: $?"

ERROR: min_cov not set (use -v min_cov=N)
status: 2

Non-numeric, which is the one that matters most — section 7 showed it would otherwise become 0 and pass everything:

awk -v min_cov=ninety -f signoff2.awk timing.csv cov.csv 2>&1
echo "status: $?"

ERROR: min_cov is not a number: [ninety]
status: 2

Out of range:

awk -v min_cov=150 -f signoff2.awk timing.csv cov.csv 2>&1
echo "status: $?"

ERROR: min_cov 150 exceeds 100
status: 2

Note the bail flag on every guard. exit in BEGIN still runs END — Chapter 5.6 section 7 — so without it each refusal would be followed by a table or a second error, which Chapter 7.5's final exercise demonstrated.

And the validation is a positive check (!~ /^[0-9]+(\.[0-9]+)?$/) rather than a test for known-bad values, per Chapter 5.3 section 11.

Exercise 7 — Challenge: prove the value is data, not code

Write a check that demonstrates -v cannot alter the program's behaviour, using values designed to if they could.

for v in '0' '0 { print "X" } #' '/./ { print "Y" } #' ; do
  printf 'value=%-22s -> ' "$v"
  awk -v L="$v" 'BEGIN { got = "" } $2 + 0 < L + 0 { got = got $1 " " } END { printf "matched: [%s]\n", got }' timing.rpt
done

value=0                      -> matched: [fifo ]
value=0 { print "X" } #      -> matched: [fifo ]
value=/./ { print "Y" } #    -> matched: [fifo ]

Three values — two of them complete awk programs — and the output is identical every time. Each was converted to a number by L + 0, which Chapter 5.3's rule makes 0 for all three, so every run filtered on < 0 and found fifo. Neither print "X" nor print "Y" ever appears.

The same two values spliced into the program behave differently:

for v in '0' '0 { print "X" } #' ; do
  printf 'value=%-22s -> ' "$v"
  awk "\$2 + 0 < $v { print \$1 }" timing.rpt 2>&1 | paste -sd',' -
done

value=0                      -> fifo
value=0 { print "X" } #      -> X

The second run printed X instead of fifo — the injected rule replaced the intended action. It fired once because the condition it was attached to, $2 + 0 < 0, is true for exactly one record. Choosing a value whose condition matches everything makes the takeover complete:

for v in '0' '999 { print "X" } #' ; do
  printf 'value=%-22s -> ' "$v"
  awk "\$2 + 0 < $v { print \$1 }" timing.rpt 2>&1 | paste -sd',' -
done

value=0                      -> fifo
value=999 { print "X" } #    -> X,X,X

Three records, three X — comma-joined by paste so the line has no trailing space. The -v loop could not be made to misbehave and the spliced loop could — which is the property worth having, and it holds for every value rather than for the ones tested.

Note L + 0 is doing real work in the first loop. Without it, $2 + 0 < L would compare a number against the raw string and Chapter 8.5's comparison rules would decide the result from the value's text. Forcing the conversion makes any nonsense threshold behave as 0 — which Exercise 6's BEGIN validation then reports rather than silently accepting.

13. Summary

• -v name=value assigns before BEGIN runs, which is what a threshold needs.
• A name=value operand is not visible in BEGIN and takes effect between files — its only unique use.
• An operand assignment occupies an ARGV slot and is not a filename.
• Never splice a shell variable into the program text. Its contents become code, and the failure needs no malice.
• A -v value is data — the same hostile string is inert.
• A -v value is a numeric string, so it compares numerically like a field, unlike a quoted constant.
• A non-numeric -v value becomes 0 in arithmetic, so validate it in BEGIN with a positive pattern.
• -v processes escape sequences, which destroys a path and silently weakens a regex.
• ENVIRON does no escape processing and is POSIX — use it for paths and regexes.
• Options, then the program, then the files. -v after the program is not an option.

Values can now reach the program safely. Chapter 8.3 — Validating Input turns to the data itself: checking that the file looks the way the program assumes, so a changed report format produces a stated error instead of a confidently wrong zero.