BEGIN, END, and Execution Order

Part I · Module 2 · Chapter 2.4 · BEGIN, END, and Execution Order

1. Why This Matters

Module 2 has been building one model. Chapter 2.1 established that awk reads one record at a time. Chapter 2.2 showed how each record becomes fields. Chapter 2.3 showed that every rule is tried against every record, in source order.

What none of them explained is the thing they all used: a block labelled END that ran once, at the end, and reported a total. It appeared in Chapter 1.1, in 2.1's record counts, in 2.2's warnings and in 2.3's triage, always on trust.

This chapter pays that off and completes the model:

BEGIN runs once before any input is read. The rule loop runs once per record. END runs once after the last record.

Get this and you can predict the output of any program in this curriculum.

2. The Three Phases

awk 'BEGIN { print "Start" } { print $1 } END { print "Finished" }' scores.txt

Given the familiar file:

alice 85
bob 72
carol 91

Start
alice
bob
carol
Finished

Start once, before anything was read. Then the ordinary rule, once per record. Then Finished once, after the last one. The order is never in doubt and never varies.

3. BEGIN: Before Any Input

BEGIN is for whatever has to be true before the first record arrives. The most common use is printing a header:

awk 'BEGIN { print "name score" } { print $1, $2 }' scores.txt

name score
alice 85
bob 72
carol 91

The header appears once, at the top, because BEGIN ran before any record was read.

The other common use is initialising a variable. You will see programs that open with BEGIN { n = 0 }, and it is worth being precise about why that is optional. awk creates a variable on first use and treats it as 0 in arithmetic and the empty string in text, so n++ works on a variable that was never declared. Writing n = 0 explicitly does not change the arithmetic — it documents the intent, and it makes print n emit 0 rather than an empty line if no record ever matched. Chapter 1.6's + 0 trick solves the same problem at the other end.

4. END: After the Last Record

END is for whatever can only be known once the input is exhausted. Totals, counts, worst cases, averages — every summary in this curriculum.

awk 'END { print "NR=" NR }' scores.txt

NR=3

NR holds 3 because three records were read. Inside the rule loop NR is the current record's number; only in END is it the total. Chapter 2.1 made this point and this is the mechanism behind it.

5. Neither Is a Record Pattern

BEGIN and END are written exactly where a pattern goes, which makes them look like patterns. They are not, and the empty file proves it.

: > empty.txt
awk 'BEGIN { print "B" } { print "rule" } END { print "E, NR=" NR }' empty.txt

B
E, NR=0

Zero records were read. The ordinary rule never ran — there was no record for it to run on, and rule does not appear in the output. But B and E both printed, and NR is 0.

No record-matching pattern could behave that way. With no records, a pattern has nothing to be tested against and can never be true. BEGIN and END ran anyway, because they are not conditions about records — they name phases of the program.

6. A BEGIN-Only Program Reads Nothing

If a program has only a BEGIN block, awk does not read input at all:

awk 'BEGIN { print "no input needed" }'

no input needed

No filename was given, and the command returned immediately rather than waiting for you to type something. awk worked out that no ordinary rule and no END block exists, so nothing could ever need a record, and skipped reading entirely.

This is why awk 'BEGIN { print 2 + 2 }' works as a pocket calculator, and it is a genuinely useful habit for checking what an awk expression does without inventing a file.

7. Several BEGIN or END Blocks

A program may have more than one of either. They run in the order written, as a group, in their phase:

awk 'BEGIN { print "B1" } BEGIN { print "B2" } END { print "E1" } END { print "E2" }' scores.txt

B1
B2
E1
E2

Both BEGIN blocks ran before any record; both END blocks ran after the last. The interleaving of the source — BEGIN, BEGIN, END, END — had no effect on the phase each belongs to. You will rarely write this deliberately, but it explains what happens when two parts of a longer script each add their own setup.

8. What END Can See

END can see every variable the rule loop built, which is the entire point. It can also see NR, as section 4 showed.

It can usually see $0 as well, holding the last record read:

awk 'END { print "last record was: [" $0 "]" }' scores.txt

last record was: [carol 91]

9. exit: Leaving Early

exit ends the input processing immediately — and then END still runs:

awk 'NR == 2 { print "exiting at", NR; exit } END { print "END still ran, NR=" NR }' scores.txt

exiting at 2
END still ran, NR=2

Two things to notice. END ran, so exit does not mean "stop the program" — it means "stop reading input and go to the end". And NR is 2, not 3: it holds however many records were actually read, so a summary printed after an early exit describes a partial pass.

exit inside BEGIN is stronger. It skips the record loop entirely, and END still runs:

awk 'BEGIN { print "B"; exit } { print "rule ran" } END { print "E" }' scores.txt

B
E

rule ran never appears — no record was processed — yet E did. This is the phase model again: exit short-circuits the loop, not the phases.

exit can also carry a status, which becomes the command's exit status and survives END:

awk 'BEGIN { exit 3 } END { print "E" }'

E

That command exits with status 3. This is how an awk program tells a shell script something, and it is the proper answer to Chapter 1.6's warning that a zero exit status proves only that awk ran. Chapter 5.6 covers exit and next together.

10. An Engineering Example

A synthetic regression summary: a test name, its recorded result, and its runtime in seconds.

alu_test PASS 120
fifo_test FAIL 180
reset_test PASS 95

The three-phase shape every summary script uses:

awk 'BEGIN { print "test result" } $2 == "FAIL" { n++; print $1, $2 } END { print n + 0, "failures of", NR, "tests" }' regress.log

test result
fifo_test FAIL
1 failures of 3 tests

BEGIN wrote the header before any record. The ordinary rule fired on the one failing record, printing it and incrementing a counter. END reported the counter and NR together, which is the pair a reader actually needs — one failure out of three is information, while 1 failure alone is not.

Now the version that also survives an empty input, which is what makes it fit to run unattended:

: > empty.log
awk 'END { if (NR == 0) { print "ERROR: no test records found"; exit 2 } print n + 0, "failures of", NR, "tests" }' empty.log
echo "exit status: $?"

ERROR: no test records found
exit status: 2

END ran despite there being no records, detected NR == 0, said so, and set a non-zero exit status a calling script can test. Compare that with the same program on real input, where the check passes silently.

The engineering caveats are unchanged from Chapter 1.2 and remain the point. 1 failures of 3 tests counts records whose second field is FAIL — the regression system's recorded verdict, not an independent check of the logs. A test recorded PASS whose log is full of unclassified warnings still counts as a pass here, and a test that timed out without writing a row does not appear at all. NR of 3 is a count of rows in this file, not proof that three tests ran.

11. The Whole Model, in Order

Everything Module 2 has established, as one sequence:

1. awk parses the program.
2. All BEGIN blocks run, in source order. No input has been read.
3. If there is any ordinary rule or END block, awk reads a record.
4. NR increases. The record is split into fields, giving $0, $1 through $NF, and NF.
5. Every ordinary rule is tried against that record, in source order.
6. Each matching rule's action runs. Several may run for one record.
7. next in an action skips the remaining rules for this record.
8. Steps 3 to 7 repeat until the input is exhausted — or exit cuts it short.
9. All END blocks run, in source order.
10. awk exits, with the status from exit if one was given.

Chapters 2.1, 2.2 and 2.3 are steps 3 to 7. This chapter is steps 2, 9 and 10, plus the fact that 2 and 9 happen regardless of what step 3 found.

12. Common Mistakes

Treating BEGIN or END as a pattern. They are phases. Section 5's empty file is the proof.

Expecting END to be skipped when the input is empty. It runs. That is what makes it the right place for a count of zero.

Reading NR in a rule and calling it a total. In the loop it is the current record. Only in END is it the total — and after an exit it is the partial count, as section 9 showed.

Relying on $0 in END. Save it yourself with { last = $0 }. Section 8.

An awk command that seems to hang. It is waiting on standard input because no filename was given and the program contains an ordinary rule or an END block. Section 6.

Thinking exit skips END. It does not, from either a rule or BEGIN.

Putting a header inside the rule loop. { print "name score" } with no pattern prints the header once per record. It belongs in BEGIN.

Assuming an awk exit status of 0 means the data was fine. Chapter 1.6's warning, and section 9 is the tool that fixes it: give exit a status deliberately.

13. Exercises

Exercise 1 — Predict the output

Write down the exact output before running it.

alice 85
bob 72
carol 91

awk 'BEGIN { print "first" } { print NR } END { print "last" }' scores.txt

first
1
2
3
last

Five lines. BEGIN once, the rule three times, END once. If you expected first and last to appear beside each record, the phase model is the thing to re-read.

Exercise 2 — Modify a command

Add a header and a record count to awk '{ print $1 }' scores.txt, without changing what the ordinary rule prints.

awk 'BEGIN { print "names:" } { print $1 } END { print NR, "names total" }' scores.txt

names:
alice
bob
carol
3 names total

The header goes in BEGIN so it prints once, and the count in END so NR has finished growing.

Exercise 3 — Trace the execution

Predict this, paying attention to when the comparison happens.

awk 'BEGIN { max = 0 } $2 > max { max = $2 } END { print "highest:", max }' scores.txt

highest: 91

The trace. BEGIN sets max to 0. Record 1: 85 is above 0, so max becomes 85. Record 2: 72 is not above 85, so nothing changes. Record 3: 91 is above 85, so max becomes 91. END prints it.

Note that the pattern itself does the work of deciding, and the action only records the new value — the same structure Chapter 2.3's running-maximum exercise used. Starting max at 0 in BEGIN is safe here because scores are positive; for a column that can be negative, such as slack, starting at 0 would be a bug, and Chapter 6.4 returns to it.

Exercise 4 — Debug an empty-input script

An engineer's nightly check is the command below. On a night when the regression produced an empty summary file, it printed 0 failures and the pipeline reported success. Explain why that is dangerous and fix it.

: > empty.log
awk '$2 == "FAIL" { n++ } END { print n + 0, "failures" }' empty.log

0 failures

The output is identical to a night where every test genuinely passed. END ran, n was never created so n + 0 gave 0, and awk exited 0. Nothing distinguishes "no tests failed" from "no tests were recorded".

: > empty.log
awk '$2 == "FAIL" { n++ } END { if (NR == 0) { print "ERROR: empty summary"; exit 2 } print n + 0, "failures of", NR }' empty.log
echo "status: $?"

ERROR: empty summary
status: 2

Two fixes together: NR == 0 distinguishes the cases, and exit 2 gives the calling script something to test. Reporting the denominator alongside the count would have made the problem visible even without the check.

Exercise 5 — Engineering application

Using the synthetic regression summary, report the pass count, the fail count, and the total runtime, with a header.

alu_test PASS 120
fifo_test FAIL 180
reset_test PASS 95

awk 'BEGIN { print "regression summary" } $2 == "PASS" { p++ } $2 == "FAIL" { f++ } { t += $3 } END { print "pass:", p + 0, "fail:", f + 0, "runtime:", t + 0 }' regress.log

regression summary
pass: 2 fail: 1 runtime: 395

Four rules across three phases. Note the third rule has no pattern, so it accumulates runtime for every record including the failing one — which is correct here, since a failed test still consumed time. That is a deliberate choice, and it is the kind of choice rule order and patterns let you state precisely.

runtime: 395 is the sum of the third column in whatever unit the file uses. It is not a wall-clock duration for the regression, because tests may have run in parallel — a good example of a number that is arithmetically right and needs a sentence of context before anyone acts on it.

Exercise 6 — Challenge: order across phases

Predict the output of this program, then explain which lines come from which phase and why n has the value it does.

awk 'END { print "end sees n =", n + 0 } BEGIN { print "begin sees n =", n + 0 } { n++ }' scores.txt

begin sees n = 0
end sees n = 3

The END block is written first in the program and runs last, because phase beats position. BEGIN sees n as 0 — it has never been assigned, and + 0 turns the empty string into a printable zero. The unpatterned rule then increments n once per record, so END sees 3.

This is the cleanest demonstration that BEGIN and END are not ordinary rules: if they were, source order would decide when they ran, and it does not. Within a phase order matters, as Chapter 2.3 showed; between phases it does not.

14. Summary

• Three phases, always in this order: all BEGIN blocks, then the rule loop once per record, then all END blocks.
• BEGIN and END are phases, not patterns. The empty file proves it: zero records, no ordinary rule can fire, and both still run with NR of 0.
• That is what makes END the right place for a count — including a count of zero, which is a measurement rather than silence.
• BEGIN is for what must be true first: a header, or an explicit initialisation that documents intent.
• A BEGIN-only program reads no input and returns at once, which is why awk works as a calculator. Add an ordinary rule or an END block and awk must read — so a command with no filename will appear to hang.
• Several BEGIN or END blocks run in source order within their phase, wherever they sit in the program.
• END sees every variable the loop built, and NR. It usually sees $0 too, but that is not safe to rely on — save the last record yourself.
• exit stops the input, not the program. END still runs, from a rule or from BEGIN, and NR then holds the partial count. exit with a status propagates out through END.
• Phase beats position; within a phase, position decides.

That completes the execution model. You can now take any awk program in this curriculum, walk it phase by phase and record by record, and say what it will print before running it — which is what the rest of the track assumes you can do.

Every chapter in this module has been working with awk's defaults: a newline ends a record, and whitespace splits it into fields. Chapter 2.2 kept flagging the limits of that — a single-character separator behaves differently from the generous whitespace rule, and a report whose logical records span several lines cannot be read one line at a time. Module 3, Fields, Records, and Separators, is where those defaults become settings you control.