User-Defined Variables

Part II · Module 5 · Chapter 5.1 · User-Defined Variables

1. Why This Matters

This is the first chapter of Part II, and the shift is worth naming.

Part I taught you to select. Four modules of patterns, fields, separators and conditions, all answering one question: which records matter? Every example ended by printing something about the records it had chosen.

Part II is about computing. Totals, rates, worst cases, classifications — the numbers a report actually carries. And all of it rests on one fact Chapter 2.1 established and then set aside:

awk holds one record at a time and discards it. A variable is the only thing that survives.

That is why every summary you have seen looked the same — a value that grows on each record, and a print at the end. This chapter is about the value that grows.

2. What a Variable Is

A variable is a name you choose that holds a value. You create one by assigning to it:

awk 'BEGIN { count = 0; count = count + 1; print count }'

1

Three steps in one line. count = 0 creates the variable and gives it a value. count = count + 1 reads the old value, adds one, and stores the result back under the same name. print count reads it again.

The = is assignment, not comparison. Chapter 4.2 warned about confusing it with ==, and the difference is exactly this: = stores, == asks.

Note that nothing declared count or said what kind of thing it holds. awk creates a variable the first moment you use it.

3. An Unset Variable Is Empty and Zero

awk gives a variable you have never assigned a value that behaves as both the empty string and zero, depending on how you use it:

awk 'BEGIN { print "[" nope "]", nope + 0, length(nope) }'

[] 0 0

In a string context it is empty — the brackets show nothing between them. In arithmetic it is 0. Its length is 0.

That dual nature is convenient and it is the source of two recurring problems in this curriculum: a counter that was never incremented prints as blank rather than 0 (section 6), and a comparison against an unset variable matches nothing (Chapter 4.2). Both come from this one rule.

4. A Variable Survives Between Records

Here is the property that makes summaries possible. A synthetic regression summary:

alu PASS
fifo FAIL
uart PASS
spi FAIL

awk '$2 == "FAIL" { failures++ } END { print failures + 0 }' results.txt

2

failures++ adds one to failures. Chapter 5.3 covers ++ properly; for now read it as "increase this by one".

The important part is that failures is the same variable on every record. awk reads alu PASS, the pattern does not match, nothing happens. It reads fifo FAIL, the pattern matches, and failures becomes 1. The record is then discarded — but failures is not. It is still 1 when uart arrives.

If the variable were reset for each record, this command could only ever print 0 or 1.

5. The Counter, Traced

Rather than assert what the counter does, here is a command that prints its own state — capturing the value before the counting rule and after it:

awk '{ before = failures + 0 } $2 == "FAIL" { failures++ } { print $1, $2, before, failures + 0 }' results.txt

alu PASS 0 0
fifo FAIL 0 1
uart PASS 1 1
spi FAIL 1 2

That table is awk's own output, not a hand-drawn illustration.

Read the third rule as the evidence: it has no pattern, so it runs on every record, and it reports what the counter held both before and after the second rule had its chance. On uart the counter is 1 going in and 1 coming out — unchanged, but carried. That carrying is the whole point.

The three rules fire in source order on each record, which is Chapter 2.3's model doing the work.

6. Why + 0 Matters

Every count in this curriculum has been printed as n + 0 rather than n, and section 3 is the reason. On a regression where nothing failed, the counter is never created:

awk '$2 == "FAIL" { failures++ } END { print "[" failures "]" }' clean.txt

alu PASS
uart PASS

[]

Empty brackets. The variable was never assigned, so in a string context it is the empty string, and print emits nothing between the brackets.

awk '$2 == "FAIL" { failures++ } END { print "[" failures + 0 "]" }' clean.txt

[0]

+ 0 forces a numeric context, and the number is 0.

7. Accumulators: a Running Sum

A counter adds one; an accumulator adds a value. The += operator does both jobs:

u_alu comb 1820
u_regfile seq 2410
u_io pad 640

awk '{ sum += $NF } END { print sum }' area.rpt

4870

sum += $NF means sum = sum + $NF. On the first record sum is unset, so it behaves as 0 and becomes 1820; then 4230; then 4870.

This is the registry's own example for the chapter, and it is the shape of nearly every awk summary: an unpatterned rule that accumulates, and an END that reports.

The caveat Chapter 1.2 established still holds and is worth repeating because this is where the number gets made: 4870 is the arithmetic sum of one column. It is the design's total area only if the report lists every instance exactly once at one level of hierarchy — a real hierarchical report lists a parent and its children, and summing every row double-counts.

8. Remembering a Record, Not Just a Number

A variable can hold text, which lets you remember which record something happened on:

awk '$2 == "FAIL" { last = $1 } END { print "last failure:", last }' results.txt

last failure: spi

Each failing record overwrites last, so at the end it holds the most recent one. This is how a summary names a culprit rather than only counting them.

The same idea with two variables keeps a value and its label together:

awk '$3 > max { max = $3; which = $1 } END { print which, max }' area.rpt

u_regfile 2410

The largest area and the instance it belongs to. Both are updated in the same action, so they cannot drift apart — and that pairing is exactly what the next section is about to break.

9. The Seeding Bug in a Worst-Case Tracker

Section 8's maximum worked. Here is the same pattern failing silently, and it is the most important thing in this chapter.

A synthetic timing report where every slack is negative — which is the normal state of a design that does not yet meet timing:

path_a -0.124
path_b -0.081
path_c -1.200

Ask for the best slack — the least negative — with the same idiom as section 8:

awk '$2 > max { max = $2; which = $1 } END { print "best slack:", "[" which "]", "[" max "]" }' all_negative.rpt

best slack: [] []

Nothing. Both variables are empty.

The reason is section 3. max starts unset, which in a numeric comparison is 0. Every slack in the file is less than 0, so $2 > max is never true, so the action never runs, so which is never assigned either.

The mirror case fails the same way. A naive minimum over all-positive values:

awk '$2 < min { min = $2; which = $1 } END { print "min:", "[" which "]", "[" min "]" }' all_positive.rpt

path_a 0.124
path_b 0.081
path_c 1.200

min: [] []

The fix is to seed from the first record rather than from an implicit zero:

awk 'NR == 1 { max = $2 + 0; which = $1 } $2 + 0 > max { max = $2 + 0; which = $1 } END { print "best slack:", which, max }' all_negative.rpt

best slack: path_b -0.081

Correct: -0.081 is the least negative of the three. NR == 1 is a pattern that matches only the first record — NR is a built-in variable and Chapter 5.2's subject — so the tracker starts from a value that is actually in the data.

Reporting the count alongside, so an empty answer is impossible to misread:

awk 'NR == 1 { max = $2 + 0; which = $1 } $2 + 0 > max { max = $2 + 0; which = $1 } END { if (NR == 0) print "no records"; else print "best slack", max, "at", which, "of", NR, "paths" }' all_negative.rpt

best slack -0.081 at path_b of 3 paths

10. One Variable, Numbers and Strings

awk does not require a variable to keep holding the same kind of value:

awk 'BEGIN { x = 42; print x + 1; x = "hello"; print x "!" }'

43
hello!

The same name held a number and then a string. There is no declaration to contradict and no error.

This flexibility is why awk programs are short, and it is why Chapter 4.2's comparison rule matters so much: since a variable does not announce its type, awk decides numeric-or-text comparison from the values at the moment of comparison. A variable holding "10" from a file and one holding the number 10 are not quite interchangeable in a comparison, which is the trap Chapter 4.2 demonstrated.

The practical habit: keep one variable for one purpose. Reusing x for a count and then for a name is legal and is the kind of economy that makes a script unreadable six months later.

11. The Three Phases, and What a Variable Does in Each

Chapter 2.4 established that an awk program has three phases. A variable behaves differently in each, and the difference is worth stating together:

A variable set in BEGIN is visible in every rule and in END. One built up in the rule loop is readable in END. There is no scope boundary between the phases at all — it is one set of variables for the whole program.

That is why BEGIN { n = 0 } is optional, as Chapter 2.4 noted: the variable would be created on first use anyway. What the explicit assignment buys is documentation, and a printable 0 rather than a blank if no record ever matched.

12. Naming

awk accepts names made of letters, digits and underscores, not starting with a digit. Beyond that the rules are about readability.

Two real hazards. A name you choose can collide with a built-in: assigning to NF, NR, FS or OFS changes awk's behaviour rather than storing a private value, which Chapters 3.3 and 3.6 both demonstrated. Chapter 5.2 lists the names to avoid.

And a single letter is fine for a loop counter and poor for anything that lives longer. In a one-liner you will read once, n and s are clear enough. In a script somebody maintains, failures and total_area save more time than they cost to type.

13. Common Mistakes

Printing a counter without + 0. Section 6. A blank where a zero belongs.

Using a naive maximum or minimum. Section 9. Seed from the first record.

Expecting a variable to reset each record. It does not — that is the entire mechanism. If you want a per-record value, assign it fresh in the rule.

Writing = where == was meant. $2 = "FAIL" assigns to field 2, is always true, and rebuilds the record (Chapter 3.6). awk will not stop you.

Assuming an unset variable is an error. It is empty and zero. Quiet, and occasionally exactly the bug.

Writing a bare if (v) on a value read from a file. A field containing 0 is false, so a real measurement of zero is indistinguishable from a missing one. Exercise 6 has the full rule; write v != "" for presence or v + 0 != 0 for non-zero.

Assigning to a built-in by accident. Section 12. NF = 2 truncates the record.

Reusing one variable for two purposes. Legal, and unreadable.

Reading an accumulator inside the rule loop and calling it a total. It holds the running value. Only in END is it final — the same point Chapter 2.1 made about NR.

14. Exercises

Exercise 1 — Predict the output

awk 'BEGIN { x = 5; y = x; x = 10; print x, y }'

10 5

y = x copied the value 5, so changing x afterwards leaves y alone. awk has no references between simple variables — the assignment took a snapshot.

Exercise 2 — Modify a counter

Using the regression summary, count the passing tests instead of the failing ones, and print a numeric zero if there are none.

alu PASS
fifo FAIL
uart PASS
spi FAIL

awk '$2 == "PASS" { passes++ } END { print passes + 0 }' results.txt

2

Exercise 3 — Trace the state

Predict the four output lines before running this, then check. Pay attention to which rule runs first.

awk '{ n++ } { print NR, n, (n == NR ? "in step" : "DIVERGED") }' results.txt

1 1 in step
2 2 in step
3 3 in step
4 4 in step

Both rules have no pattern, so both run on every record, and the first increments n before the second reads it. So n tracks NR exactly.

Move the increment after the print and they diverge by one — which is Chapter 2.3's source-order lesson, and worth trying to confirm you believe it.

Exercise 4 — Debug a silent worst case

An engineer asks for the worst slack in the report below and gets no output. Their command is given. Explain it and fix it.

path_a -0.124
path_b -0.081
path_c -1.200

awk '$2 > max { max = $2; which = $1 } END { print which, max }' all_negative.rpt | cat -e

 $

cat -e marks each line end with $, so that is the whole output: one space, then the end of the line. print with two unset variables still emits the output separator between them and the terminating newline, and nothing else. On a bare terminal it looks like a blank line, which is why cat -e is worth reaching for when output is suspiciously empty.

max begins unset, which compares as 0. Every slack is below 0, so the action never ran, so which was never assigned either. Note the command did not fail: exit status 0, no error message.

awk 'NR == 1 { max = $2 + 0; which = $1 } $2 + 0 > max { max = $2 + 0; which = $1 } END { print which, max }' all_negative.rpt

path_b -0.081

Seeding from record 1 starts the comparison inside the data. The + 0 is Chapter 4.2's habit, protecting against a value that carries a unit suffix.

Exercise 5 — Engineering application

From the synthetic regression summary, report the failure count, the total test count, and the name of the last failing test — naming none when nothing failed.

awk '{ total++ } $2 == "FAIL" { f++; last = $1 } END { print f + 0, "of", total, "failed; last was", (f ? last : "none") }' results.txt

2 of 4 failed; last was spi

awk '{ total++ } $2 == "FAIL" { f++; last = $1 } END { print f + 0, "of", total, "failed; last was", (f ? last : "none") }' clean.txt

0 of 2 failed; last was none

(f ? last : "none") is a conditional expression — it yields last when f is true and "none" when it is not. Chapter 5.4 covers conditions properly; what matters here is that it reads a variable's truth, which is a question section 3 already half-answered: unset is empty and zero, and both are false.

And the engineering caveat Chapter 1.2 established: 2 of 4 failed counts recorded results, not distinct RTL bugs. One root cause can fail several tests, and a test that crashed before writing its row does not appear in this file at all.

Exercise 6 — Challenge: when is a variable true?

Exercise 5 relied on f being false before anything failed. That rule has a sharp edge. Predict all four of these, then run them.

awk 'BEGIN { s = "0"; n = 0; print (s ? "s TRUE" : "s false"), (n ? "n TRUE" : "n false") }'

s TRUE n false

The string "0" is true. The number 0 is false. They print identically and they behave oppositely.

The reason is that awk tests a string for emptiness and a number for zero. "0" is a one-character string, so it is not empty, so it is true.

Now the same value arriving from a file:

zero 0

awk '{ v = $2; print (v ? "field-0 TRUE" : "field-0 false") }' z.txt

field-0 false

False — the opposite of the string constant, for the same visible character. A field that looks numeric is a numeric string: awk remembers it came from input and could be read as a number, so it is tested as the number 0. This is the same rule that decided numeric-or-text comparison in Chapter 4.2, reappearing as truth rather than ordering.

And it is reversible, which is the part that catches people:

awk '{ v = $2 ""; print (v ? "concat-0 TRUE" : "concat-0 false") }' z.txt

concat-0 TRUE

Concatenating an empty string strips the numeric-string status. v is now an ordinary string "0", non-empty, true.

Exercise 7 — Challenge: a tracker that cannot lie

Write a worst-case tracker that reports the worst slack, the path, and how many paths it compared — and that says something sensible on an empty file.

awk 'NR == 1 { worst = $2 + 0; which = $1 } $2 + 0 < worst { worst = $2 + 0; which = $1 } END { if (NR == 0) { print "ERROR: no records"; exit 2 } printf "worst %s at %s, over %d paths\n", worst, which, NR }' all_negative.rpt
echo "status: $?"

worst -1.2 at path_c, over 3 paths
status: 0

: > empty.rpt
awk 'NR == 1 { worst = $2 + 0; which = $1 } $2 + 0 < worst { worst = $2 + 0; which = $1 } END { if (NR == 0) { print "ERROR: no records"; exit 2 } printf "worst %s at %s, over %d paths\n", worst, which, NR }' empty.rpt
echo "status: $?"

ERROR: no records
status: 2

Three defences in one command. Seeding from record 1 means the answer always comes from the data. Reporting over 3 paths means a reader can tell a real comparison from a vacuous one. And the NR == 0 guard with a non-zero exit means a calling script learns that the file was empty rather than receiving a confident blank — which is Chapter 2.4's phase model and Chapter 1.6's exit-status warning working together.

Note worst -1.2 rather than -1.200: + 0 made it a number and a trailing zero is not part of a number's value.

15. Summary

• A variable is the only thing in awk that outlives a record. Every total, count and worst case depends on it.
• Assignment creates the variable. No declaration, no type.
• An unset variable is the empty string and zero, depending on context — which is why counters need + 0 and why comparisons against unset variables match nothing.
• A counter keeps its value between records, which section 5's awk-generated trace shows record by record.
• + 0 on every printed counter. A blank where a zero belongs is indistinguishable from a crash, an empty file or a typo.
• += accumulates a value where ++ adds one. Both rely on an unset variable behaving as 0 on the first record.
• A variable can hold text, which is how a summary names the culprit rather than only counting.
• A string "0" is true; a field containing 0 is false. awk tests a string for emptiness and a number for zero, so a bare if (v) on input asks a different question than you probably meant.
• A naive maximum or minimum fails silently when every value sits on one side of zero — all-negative slack being the exact case you needed it for. Seed from the first record.
• One set of variables spans all three phases. BEGIN for thresholds, the rule loop for accumulating, END for reading.
• Do not assign to a built-in name — it changes awk's behaviour rather than storing your value.

You can now carry a value across records deliberately, which is the foundation of everything in Part II.

What you have been using without introduction is the other kind of variable — the ones awk maintains itself. NR seeded section 9's tracker, NF has been in nearly every command since Module 2, and FILENAME named the file in Chapter 4.7. Chapter 5.2 — Built-In Variables collects them, says exactly when awk updates each one, and answers the question that matters once you know variables are writable: which of them are you allowed to assign?