Pattern Space vs Hold Space

Part II · Module 6 · Chapter 6.1 · Pattern Space vs Hold Space

1. The Second Buffer

Chapter 2.2 described one buffer: the pattern space, holding the current line, cleared at the end of every cycle.

There is another, called the hold space. Two properties define it:

• It starts empty — before the first line is read, it contains nothing.
• It is never cleared by the cycle. Whatever you put there is still there on the next line, and the line after that, all the way to the end of the input.

No command reads or matches the hold space directly. You cannot run s on it or address a line by its contents. The only thing you can do is move text between it and the pattern space — and then work on the pattern space as usual.

2. Proving It Starts Empty

G appends the hold space to the pattern space. Run it with nothing else, on a two-line file:

printf 'a\nb\n' | sed 'G' | cat -et

a$
$
b$
$

cat -et marks each line end with $, so what you are looking at is four lines: a, a blank, b, a blank. The blanks are the hold space — empty, appended after each line, producing an empty line each time.

That is the whole proof, and it is also the standard way to double-space a file. More usefully, it tells you something to watch for: a G or g before you have put anything in the hold space gives you emptiness, not an error.

3. The Five Commands

Lowercase replaces; uppercase appends. That is the only pattern you need to remember.

Reading the letters: h and H move text to the hold space; g and G bring it back. The lowercase pair overwrite; the uppercase pair accumulate.

A minimal h and g demonstration — save line 1, then stamp it over lines 2 and 3:

printf 'KEEP\nx\ny\n' | sed '1h; 2,3g'

KEEP
KEEP
KEEP

Line 1 copied itself into the hold space and printed normally. Lines 2 and 3 each replaced their own content with the hold space, so all three lines came out as KEEP. One line's text reached two other lines, which is exactly what Parts I and II could not do.

4. x Swaps, and Can Lose Your Last Line

x exchanges the buffers. On a three-line file with nothing else in the script:

printf 'one\ntwo\nthree\n' | sed 'x'

one
two

Three lines out, and read them carefully because there are two separate lessons here.

The first output line is blank. On line 1, x swapped one into the hold space and the empty hold space into the pattern space — so what got auto-printed was emptiness. Section 2's fact again, from the other side.

three is missing. On the last line, x put three into the hold space and brought two back out, which is what got printed. Nothing ever retrieved three, and when sed reached the end of input the hold space was simply discarded.

5. Accumulating With H

H appends, so running it on every line builds up the whole file in the hold space. Bring it back on the last line:

printf 'a\nb\nc\n' | sed -n 'H; ${ x; p; }' | cat -et

$
a$
b$
c$

Four lines from a three-line file. The leading blank is, once again, the hold space having been empty when the first H appended to it — H puts a newline between the old contents and the new, and the old contents were nothing, so the result begins with a newline.

Make it visible by turning the newlines into pipes:

printf 'a\nb\nc\n' | sed -n 'H; ${ x; s/\n/|/g; p; }'

|a|b|c

The leading | is that empty first element. The standard way to avoid it is to use h on the first line and H thereafter:

   1h; 1!H      on line 1 replace; on every other line append

Notice also that the pattern space now holds three lines at once — a, b and c with newlines between them. That is a multi-line pattern space, and it is why s/\n/|/g had newlines to match. Chapter 6.2 is about the commands that create and read one deliberately.

6. A Real Job: Attaching a Heading to Its Value

Here is the registered example, and it is worth working through because the output order is not what most people predict.

A timing report, where each path's identity is on one line and its slack is four lines later:

cat -n sta/timing.rpt

     1	Startpoint: u_alu/rd_data_reg[3]
     2	Endpoint:   u_fifo/wr_ptr_reg[0]
     3	  data arrival time                        1.842
     4	  data required time                       2.100
     5	  slack (MET)                              0.258
     6	Startpoint: u_fifo/rd_ptr_reg[1]
     7	Endpoint:   u_alu/op_reg[2]
     8	  data arrival time                        2.410
     9	  slack (VIOLATED)                        -0.110

The goal is a summary pairing each slack with the path it belongs to:

sed -n '/^Startpoint/{ h; d; }; /^[[:space:]]*slack/{ G; p; }' sta/timing.rpt

  slack (MET)                              0.258
Startpoint: u_alu/rd_data_reg[3]
  slack (VIOLATED)                        -0.110
Startpoint: u_fifo/rd_ptr_reg[1]

Four lines — the two slacks, each followed by its startpoint. Trace it:

1. Line 1 matches /^Startpoint/. h saves it to the hold space; d deletes the pattern space and ends the cycle, so nothing is printed and nothing else in the script runs for this line.
2. Lines 2–4 match neither address. With -n in force, they produce no output at all.
3. Line 5 matches /slack/. G appends the hold space, so the pattern space becomes the slack line, a newline, then the saved startpoint. p prints both lines.
4. Line 6 is another Startpoint: h overwrites the hold space with the new path. The old one is gone, which is correct — it has already been used.
5. Line 9 pairs the second slack with it.

The slack comes first because G appends. It puts the hold space after the pattern space. If you want the startpoint on top, you have to swap the roles — hold the slack line and append it to the startpoint instead, or use x to reverse the two before printing. The commands are symmetric; the output order is a consequence of which buffer you append to which.

7. The Classic: Reversing a File

This is the idiom everyone meets first and few can explain, which makes it a good test of the model. It is tac, written in sed:

sed -n '1!G; h; $p' filelists/top.f

./rtl/fifo.sv
./rtl/alu.sv
+incdir+/proj/rtl
+define+DEBUG

The input was those four lines in the opposite order. Three commands do it, and the trick is that the hold space always contains everything seen so far, reversed.

Trace all four cycles. 1!G means "on every line except the first, append the hold space".

Each cycle puts the new line on top of everything before it, then saves the result. On the last line $p prints the pattern space, which by then is the whole file upside down.

Two details that make it work, both of which the rest of the chapter has already established:

• 1!G rather than G — on line 1 the hold space is empty, so a plain G would append a blank and leave a trailing empty line in the output. Section 2 is why.
• $p with -n — nothing prints until the end, because an unfinished reversal is not useful. Section 4's warning is why the explicit $ address is needed at all.

8. Where This Sits

Module 6 is canonical advanced depth, not a prerequisite. Parts III and IV — in-place safety, shell composition, filelist and timing-report work — do not assume any of it. You can stop at Module 5 and do real work.

What the hold space is genuinely good for is a narrow set of jobs where the alternative is worse: pairing a heading with a value several lines later, as section 6 did; squashing repeated blank lines; and carrying one piece of context through a block. For anything involving arithmetic, counting, or more than one piece of state at a time, awk exists and is clearer.

Everything in this chapter also belongs in a script file rather than a command line — a three-command hold-space script is already at the edge of what is reviewable as a one-liner, and Chapter 6.4 is about moving it.

9. Think It Through

1. Why does sed 'G' file double-space a file, and what does sed 'G;G' file do?

G appends the hold space, which is empty, so each line gains a trailing blank line. G;G appends it twice and gives you two blank lines after each input line — triple spacing. Neither needs the hold space to contain anything, which is the joke: the most common use of a hold-space command involves never putting anything in it.

2. You write sed -n 'H' file to collect the file and get no output. What is missing?

The retrieval. H fills the hold space and -n suppresses auto-print, so nothing is ever printed and the hold space is discarded at end of input. Add ${ x; p; } — on the last line, swap the accumulation into the pattern space and print it. Section 4's callout is this exact bug.

3. In section 6's script, why is d necessary after h?

Without it the Startpoint line would fall through to the rest of the script and, more importantly, would be considered by the auto-print — except -n is in force, so actually nothing would print. What d genuinely buys here is clarity and cost: it ends the cycle immediately, so no later command is even tested on a line that has already been dealt with. Chapter 5.2 covers that behaviour; this is a clean use of it.

4. Predict the output of printf '1\n2\n3\n' | sed -n '1h; 1!H; ${ g; p; }'.

All three lines, in order: 1, 2, 3. Line 1 replaces the hold space; lines 2 and 3 append to it; on the last line g copies the accumulation back and p prints it. This is the leading-blank fix from section 5 — using h on line 1 instead of H is what keeps the output from starting with an empty line.

5. Why does the reverse idiom use h rather than H?

Because the pattern space already contains the full reversed accumulation after G, so the hold space needs replacing with it, not appending to it. H would append the whole accumulation to itself and the buffer would double in size every line — four lines in would be fifteen lines of hold space. Replacing is correct precisely because the pattern space is already the answer so far.

6. What happens to the hold space between two files given on one command line?

Nothing — it carries straight across. Chapter 2.4 established that several file operands are one stream, so there is no per-file boundary for the hold space to be reset at. A script that accumulates and flushes on $ will flush once, at the end of the last file. If you want per-file behaviour, run sed once per file with a shell loop, exactly as Chapter 2.4 recommended.

10. Summary

• The hold space is a second buffer. It starts empty and is never cleared by the cycle.
• You cannot match or edit it directly — you move text to the pattern space and work there.
• It starts empty, provably: sed 'G' appends a blank line to every line, which is why it double-spaces a file.
• Lowercase replaces, uppercase appends. h/H go to the hold space; g/G come back; x exchanges.
• The hold space is discarded at end of input. sed 'x' lost the last line of a three-line file, and an accumulate script with no $ flush prints nothing.
• H on an empty hold space leaves a leading newline — the |a|b|c result. Use 1h; 1!H instead.
• G appends the hold space after the pattern space, which is why the slack line came out above its startpoint. Output order follows from which buffer you append to which.
• The reverse-a-file idiom works because the hold space always holds everything so far, reversed. Use tac or tail -r in real life.
• Module 6 is optional depth. Parts III and IV do not depend on it, and awk is usually the better answer when a job needs real state.

Section 5 produced a pattern space holding three lines at once, almost by accident. Doing that deliberately — so a pattern can match across a line break, and a tool message split over two lines can be rejoined — needs three more commands and one guard that is easy to forget. Chapter 6.2 — Multiline Processing covers N, P and D, including the measured case where forgetting the guard silently discards the last line of your file.