Record Separators: RS and ORS

Part I · Module 3 · Chapter 3.4 · Record Separators: RS and ORS

1. Why This Matters

Chapter 2.1 made a careful distinction that has not paid off until now. A record is what awk processes as a unit, and a line is what happens to be a record by default. The chapter insisted on the first word and promised the two would come apart.

This is where they do:

RS decides what counts as a record. Change it, and every rule you have written keeps working — in terms of records, which are now something else.

That is the whole value of the vocabulary. You do not relearn patterns, fields, NF or END. They all still apply, to a record that is no longer a line.

2. The Default: One Line, One Record

RS holds the record separator, and its default is a newline. A report of seven lines is seven records:

Startpoint: u_alu/a
Endpoint: u_regfile/d
slack: -0.124

Startpoint: u_io/pad
Endpoint: u_core/in
slack: 0.081

awk 'END { print "records:", NR }' paths.rpt

records: 7

Seven, counting the blank line in the middle — Chapter 2.1 established that a blank line is a record.

And this is exactly the problem. The report contains two paths, and awk sees seven records. The startpoint of path one is in record 1 and its slack is in record 3, and no field expression can reach from one to the other, because fields only exist within a record.

3. RS = "": Paragraph Mode

Set RS to the empty string and awk switches into paragraph mode: records are separated by blank lines rather than by newlines.

awk 'BEGIN { RS = "" } END { print "records:", NR }' paths.rpt

records: 2

Two records — one per path. The blank line became the boundary, and the newlines inside each block stopped being boundaries.

awk 'BEGIN { RS = "" } /Startpoint/ { print NR }' paths.rpt

1
2

Each block is a record, and each contains a Startpoint, so both match.

4. The Special Rule That Makes It Useful

Paragraph mode on its own would be a curiosity. A three-line block as one record, split on whitespace, would be... well, let awk answer:

awk 'BEGIN { RS = "" } { print "rec", NR, "has", NF, "fields" }' paths.rpt

rec 1 has 6 fields
rec 2 has 6 fields

Six fields. Each block holds three lines of two words, and all six words are separate fields.

That happens because of a rule specific to paragraph mode:

When RS is the empty string, a newline is always a field separator, in addition to whatever FS says.

Without it, the embedded newlines would not split anything and each block would be one enormous field — technically a record, practically useless. With it, every word in the block is addressable.

The record itself still contains its newlines:

awk 'BEGIN { RS = "" } NR == 1 { print "[" $0 "]" }' paths.rpt

[Startpoint: u_alu/a
Endpoint: u_regfile/d
slack: -0.124]

So $0 is the whole block, newlines and all, while the fields are the individual words. Both views are available at once.

5. Reaching Across Lines

This is the payoff. Field 2 is on the block's first line and field 6 is on its third:

awk 'BEGIN { RS = "" } { print $2, $6 }' paths.rpt

u_alu/a -0.124
u_io/pad 0.081

A startpoint and its slack, on one output line, pulled from two different input lines of the same block.

Nothing before this chapter could do that. Chapter 1.2 said a real timing report "spans many lines, with the startpoint, the endpoint, a delay table and the slack figure on separate lines" and that "splitting that on whitespace does not give you a row per path". It does now — the splitting just had to happen inside a bigger record.

6. Paragraph Mode Is Forgiving About Blank Lines

Real files are padded inconsistently — a trailing newline here, two blank lines there. Paragraph mode handles that deliberately.

Startpoint: x
slack: -1


Startpoint: y
slack: 2

awk 'BEGIN { RS = "" } END { print "records:", NR }' padded_paths.rpt

records: 2

Still two records. Three things happened: leading blank lines were skipped, a run of blank lines counted as one separator, and trailing blank lines did not produce an empty final record.

awk 'BEGIN { RS = "" } { print NR ":" $1 }' padded_paths.rpt

1:Startpoint:
2:Startpoint:

Two clean records with no empty ones between them. This is a genuine convenience: the equivalent logic written by hand in line mode would need explicit blank-line handling.

7. RS as a Single Character

RS can be an ordinary character, which is useful for data delimited by something other than lines:

A fence cannot show you a missing final newline, and here it matters, so build the file with printf:

printf 'a;b;c' > semi.txt
awk 'BEGIN { RS = ";" } { print NR, $0 }' semi.txt

1 a
2 b
3 c

Three records from one line. The semicolons were the boundaries, and a newline would now be ordinary text rather than a separator.

That last point has a consequence worth seeing. Had the file ended with a newline — as text files conventionally do — the third record would be c followed by that newline, and printing it would emit an extra blank line:

printf 'a;b;c\n' > semi_nl.txt
awk 'BEGIN { RS = ";" } { print NR, "[" $0 "]" }' semi_nl.txt

1 [a]
2 [b]
3 [c
]

The brackets make it visible: the final newline is inside record 3. Once RS is not a newline, newlines stop being special and become data like any other character.

8. ORS: The Output Record Separator

ORS is what awk puts after each print. Its default is a newline, which is why every print so far ended a line.

awk 'BEGIN { ORS = " | " } { print $1 }' three.txt

a 1
b 2
c 3

a | b | c |

The three values are now separated by | rather than newlines — including a trailing one after the last, because ORS goes after every record without exception.

Set it to nothing and everything runs together:

awk 'BEGIN { ORS = "" } { print $1 }' three.txt

abc

ORS and OFS are easy to confuse, and the distinction is exactly parallel to RS and FS:

9. An Engineering Example

The two settings together, converting a block-structured report into one row per path — which is the shape every other chapter in this curriculum has assumed it already had.

awk 'BEGIN { RS = ""; OFS = "," } { print $2, $4, $6 }' paths.rpt

u_alu/a,u_regfile/d,-0.124
u_io/pad,u_core/in,0.081

Three lines of report became one comma-separated row per path. RS = "" made the block a record, the paragraph-mode newline rule made all six words fields, and OFS joined the three chosen ones with commas.

And with ORS used to put a visible separator between paths instead:

awk 'BEGIN { RS = ""; ORS = "\n---\n" } { print $2 }' paths.rpt

u_alu/a
---
u_io/pad
---

The honest caveats, which matter more here than anywhere else in the module. This file was written so that each block is exactly three lines of exactly two words, and the field positions in section 5's table depend on that completely. A real timing report has a delay table of a dozen rows inside each block, so $6 would be somewhere in the middle of it rather than the slack.

So what this chapter gives you is the mechanism — a block can be one record — not a parser for any real report. Matching on the labels (slack: and the word after it) rather than counting to six is what makes it survive contact, and that needs regular expressions and the match function from Module 4. Chapter 10.7 is where the durable version is built.

10. Common Mistakes

Expecting RS = "" to mean "no separator". It names paragraph mode, with its own rules about blank lines. Section 6.

Forgetting that a newline is also a field separator in paragraph mode. Section 4. Without knowing this, NF of 6 on a three-line block is baffling.

Setting RS outside BEGIN. Exactly the same hazard as FS in Chapter 3.3 section 10: the first record has already been read using the old separator.

Counting field positions across a block and assuming they are stable. Section 5's callout. A block layout change shifts everything, and by more than one position.

Confusing ORS with OFS. OFS goes between values in one print; ORS goes after the whole print. Section 8's table.

Forgetting that ORS applies after the last record too. Section 8's output has a trailing |.

Using a regular expression as RS. A gawk extension, not POSIX. Section 7.

Assuming paragraph mode fixes a report. It makes a block addressable. It does not make the addresses durable.

11. Exercises

Exercise 1 — Predict the record count

Startpoint: u_alu/a
Endpoint: u_regfile/d
slack: -0.124

Startpoint: u_io/pad
Endpoint: u_core/in
slack: 0.081

awk 'BEGIN { RS = "" } END { print NR }' paths.rpt

2

Two records, one per block. If you said 6 you were counting non-blank lines; if you said 7 you were in default mode. The blank line is the separator in paragraph mode rather than a record of its own.

Exercise 2 — Modify a command

Print the endpoint of every path — the second word of each block's second line.

awk 'BEGIN { RS = "" } { print $4 }' paths.rpt

u_regfile/d
u_core/in

Field 4, because the block's six fields run across its three lines: $3 is the label Endpoint: and $4 is its value.

Exercise 3 — Trace the field mapping

Without running it, write down what each of $1 through $6 holds on the first block, then check.

awk 'BEGIN { RS = "" } NR == 1 { print $1, "/", $2, "/", $3, "/", $4, "/", $5, "/", $6 }' paths.rpt

Startpoint: / u_alu/a / Endpoint: / u_regfile/d / slack: / -0.124

The labels are fields too — they are words separated by whitespace like anything else. That is why the values are at the even positions and why inserting a line anywhere in the block would move all of them.

Exercise 4 — Debug a record count

An engineer reads a report with blank-line-padded blocks and expects 2 records but gets 5. Their command is below. Explain the discrepancy and fix it.

Startpoint: x
slack: -1


Startpoint: y
slack: 2

awk 'END { print NR }' padded_paths.rpt

8

In default mode every line is a record, including all the blank ones, so a padded file counts far more than the data suggests — eight lines here for two paths' worth of data.

awk 'BEGIN { RS = "" } END { print NR }' padded_paths.rpt

2

Paragraph mode gives the answer they wanted, and section 6's forgiveness is what makes it 2 rather than something sensitive to how many blank lines happen to be between blocks.

Exercise 5 — Engineering application

Convert the two-path report into a table of startpoint, endpoint and slack, pipe-separated, with a header.

awk 'BEGIN { RS = ""; OFS = "|"; print "startpoint|endpoint|slack" } { print $2, $4, $6 }' paths.rpt

startpoint|endpoint|slack
u_alu/a|u_regfile/d|-0.124
u_io/pad|u_core/in|0.081

RS for the blocks, OFS for the output delimiter, BEGIN for the header — three of this module's settings in one command, which is the practical shape of a report converter.

The caveat from section 9 applies in full: this works because every block in this synthetic file has the same six-field layout, and a real report does not promise that.

Exercise 6 — Challenge: survive one extra line

The report gains a Path Group: line in every block. Predict what the command from Exercise 5 produces, then write a version that still finds the slack.

Startpoint: u_alu/a
Endpoint: u_regfile/d
Path Group: clk_main
slack: -0.124

Startpoint: u_io/pad
Endpoint: u_core/in
Path Group: clk_io
slack: 0.081

awk 'BEGIN { RS = ""; OFS = "|" } { print $2, $4, $6 }' paths_v2.rpt

u_alu/a|u_regfile/d|Group:
u_io/pad|u_core/in|Group:

The slack column now contains the word Group:. The inserted line added three fields — Path, Group: and the clock name — so $6 landed in the middle of the new line. This is Chapter 3.1's shift, three times as wide.

Counting from the end survives it, because slack stayed last in the block:

awk 'BEGIN { RS = ""; OFS = "|" } { print $2, $4, $NF }' paths_v2.rpt

u_alu/a|u_regfile/d|-0.124
u_io/pad|u_core/in|0.081

awk 'BEGIN { RS = ""; OFS = "|" } { print $2, $4, $NF }' paths.rpt

u_alu/a|u_regfile/d|-0.124
u_io/pad|u_core/in|0.081

One command, both layouts — the same "which end is stable" reasoning Chapter 3.2 settled, now applied to blocks instead of lines. And the same limitation: it works only while slack is the last thing in the block, which a report that appends a verdict line would break. Matching the slack: label itself is the durable answer, and that is Module 4's territory.

12. Summary

• RS decides what counts as a record. Change it and every pattern, field and counter you know keeps working, on a record that is no longer a line.
• Its default is a newline, which is why a seven-line file is seven records — blank line included.
• RS = "" is paragraph mode: blank lines separate records, so a block-structured report becomes one record per block.
• In paragraph mode a newline is also a field separator, on top of FS. That is what makes the mode useful — a three-line block of two words each has six fields.
• $0 keeps the block's newlines while the fields are the individual words, so both views are available at once.
• Paragraph mode is forgiving: leading and trailing blank lines are skipped and runs of them collapse. That forgiveness belongs to RS = "" specifically, not to a single-character RS.
• A single-character RS is POSIX; a regular-expression RS is a gawk extension.
• ORS separates one print from the next, and applies after the last one too. OFS separates values within one print.
• Paragraph mode makes a block addressable, not durable. A block layout change shifts fields by more than one position, and label matching rather than position counting is the real defence.

You can now read a report whose logical records span several lines, which was the last shape of input this curriculum could not touch.

What every chapter in this module has had in common is that the record arrived, you read from it, and you printed something derived from it. The record itself was never altered — Chapter 3.3 made a point of the fact that OFS cannot change $0. That turns out to be only half true, and the other half is a genuinely useful tool for cleaning a report as it passes through. Chapter 3.5 — Custom Delimiters first works through the delimiters you actually meet, and then Chapter 3.6 — Rebuilding Records shows what happens when you assign to a field.