sed in Semiconductor Engineering

Part I · Module 1 · Chapter 1.2 · sed in Semiconductor Engineering

1. Where sed Fits in a Chip Project

Almost nothing a chip flow produces is binary. A build script writes a list of source files. A simulator writes a log. Synthesis and timing write reports. A run script writes a config the next tool reads. All of it is lines of text, produced by a machine, in a format that stays the same from run to run.

That last property is what makes it sed's territory. Text a machine generated in a predictable shape is text a short command can correct predictably. Text a person wrote, in a language with rules sed knows nothing about, is not.

2. Which File Is It?

Before choosing an expression, classify the file. This is the judgement the rest of the chapter rests on, and it is the part that does not appear in any syntax reference.

Authored source. Hand-written SystemVerilog, VHDL, C, a testbench, a constraint file somebody reasoned about. These have an author, a review history, and meaning that lives in a language sed cannot read. A wrong substitution here is a defect introduced into the design, and it may survive until silicon. Treat these as off limits for casual substitution; Chapter 9.3 is devoted to where the line falls.

Generated text. A filelist a script emitted, a config a flow wrote, an expanded command file. These have no author to disagree with, a shape fixed by the generator, and a cheap recovery path — you can usually regenerate them. This is where sed earns its keep.

Tool output. Logs and reports. Nobody consumes these as input to a build, so transforming a copy of one risks almost nothing. The risk here is not corruption, it is misreading: a transformation that makes a log easier to scan can also make it easier to draw a conclusion the log does not support.

Derived artifacts. Intermediate files in a build directory. Usually safe to transform and trivial to recreate, and usually better regenerated than patched.

The ordering is deliberate. Risk falls as you go down the list, and the question "what kind of file is this?" answers "how careful must I be?" before you have written a single character of the command.

3. Look Before You Change Anything

Everything below follows the same three steps: find out what is there, preview the change, then compare the preview with the original. Chapter 1.1 established that a plain substitution prints its result and leaves the input alone, which is what makes this possible at all.

Here is a generated filelist, written by a build script into a scratch directory:

+incdir+/proj/scratch/build/rtl
/proj/scratch/build/rtl/top.sv
/proj/scratch/build/rtl/fifo.sv
/proj/scratch/build/tb/top_tb.sv

The +incdir+ marks a directory the compiler should search for included files; the other lines are sources to compile. The whole file was generated, so it is in the second category from section 2.

Step one — find out how much is in scope. Before transforming anything, count what you would touch:

grep -c '/proj/scratch/build/' build/files.f

4

Four lines. That number is the thing to be surprised by: if a filelist you believed had a dozen entries reports four hundred, stop and find out why before going further.

4. A Generated Filelist

The scratch build is being promoted to a release area, so every path has to move from /proj/scratch/build/ to /proj/release/.

Step two — preview it. This is just the command from 1.1, with a path-safe separator:

sed 's|/proj/scratch/build/|/proj/release/|g' build/files.f

+incdir+/proj/release/rtl
/proj/release/rtl/top.sv
/proj/release/rtl/fifo.sv
/proj/release/tb/top_tb.sv

Four lines in, four lines out, each one corrected, and the +incdir+ line handled identically to the source lines because sed has no idea they are different kinds of entry. That is exactly the behaviour you want here and exactly the behaviour section 8 warns about elsewhere.

Step three — confirm the file is untouched.

cat build/files.f

+incdir+/proj/scratch/build/rtl
/proj/scratch/build/rtl/top.sv
/proj/scratch/build/rtl/fifo.sv
/proj/scratch/build/tb/top_tb.sv

Unchanged, as 1.1 promised. What you have is a proposal, not an edit.

Reviewing the proposal. On four lines you can read the output directly. On four hundred you cannot, and the habit that scales is to diff the preview against the original:

sed 's|/proj/scratch/build/|/proj/release/|g' build/files.f | diff build/files.f -

1,4c1,4
< +incdir+/proj/scratch/build/rtl
< /proj/scratch/build/rtl/top.sv
< /proj/scratch/build/rtl/fifo.sv
< /proj/scratch/build/tb/top_tb.sv
---
> +incdir+/proj/release/rtl
> /proj/release/rtl/top.sv
> /proj/release/rtl/fifo.sv
> /proj/release/tb/top_tb.sv

The - in diff build/files.f - is not an option: it is the second filename, and it means read this one from the pipe. So you are comparing the file on disk against the proposed version sed just produced. The < lines are what you have, the > lines are what you would get. What makes this worth doing is that a diff reports only the parts that differ, so your attention goes to the change rather than to the file. Its size tracks the number and spread of those changes, not the length of the input: a one-line correction in a four-hundred-line filelist is a few lines of diff, while a prefix change that touches every line is a diff as long as the file. Either way the question you are asking is the same — is every line in here one I meant to change? — and an unexpected line in that output is the cheapest warning you will ever get. Writing this result back to disk is a separate decision with its own risks, and Module 7 covers it.

5. A Simulation Log

A log is tool output: nothing downstream compiles it, so transforming a copy is low-risk. Here is a simplified one.

UVM_INFO @ 0: reporter [RNTST] Running test apb_smoke_test
UVM_INFO @ 1200: uvm_test_top [SEQ] starting sequence
UVM_ERROR @ 4500: uvm_test_top.env.sb [MISCMP] addr=0x40 exp=0xA5 act=0x5A
UVM_INFO @ 9000: uvm_test_top [SEQ] sequence done
UVM_FATAL @ 9100: uvm_test_top [TIMEOUT] no response

Selecting the lines that matter. Using -n with p from 1.1:

sed -n '/UVM_ERROR/p' sim.log

UVM_ERROR @ 4500: uvm_test_top.env.sb [MISCMP] addr=0x40 exp=0xA5 act=0x5A

To catch two severities at once you need an alternation, and that requires extended regular expressions — the -E flag:

sed -n -E '/UVM_(ERROR|FATAL)/p' sim.log

UVM_ERROR @ 4500: uvm_test_top.env.sb [MISCMP] addr=0x40 exp=0xA5 act=0x5A
UVM_FATAL @ 9100: uvm_test_top [TIMEOUT] no response

Both forms work the same way on GNU and BSD sed. Section 7 shows a third form that does not, and the way it fails is worth seeing.

Making two runs comparable. Timestamps differ on every run, so a plain diff of two logs is useless. Replacing them with a placeholder makes the real differences visible:

sed 's/@ [0-9]*:/@ <t>:/' sim.log

UVM_INFO @ <t>: reporter [RNTST] Running test apb_smoke_test
UVM_INFO @ <t>: uvm_test_top [SEQ] starting sequence
UVM_ERROR @ <t>: uvm_test_top.env.sb [MISCMP] addr=0x40 exp=0xA5 act=0x5A
UVM_INFO @ <t>: uvm_test_top [SEQ] sequence done
UVM_FATAL @ <t>: uvm_test_top [TIMEOUT] no response

Every timestamp is gone and nothing else moved. Two logs normalised this way can be diffed against each other, and what remains is behaviour rather than timing.

Pulling out a region. A range address takes everything between two matching lines:

sed -n '/RNTST/,/MISCMP/p' sim.log

UVM_INFO @ 0: reporter [RNTST] Running test apb_smoke_test
UVM_INFO @ 1200: uvm_test_top [SEQ] starting sequence
UVM_ERROR @ 4500: uvm_test_top.env.sb [MISCMP] addr=0x40 exp=0xA5 act=0x5A

Three lines: from the start marker to the first mismatch, inclusive. Addresses are Chapter 4.4's subject; this is a first look at why they matter.

6. A Generated Configuration

Flows often read a small generated file to decide what to run.

SIM_ROOT = /proj/scratch/build
TEST = apb_smoke_test
SEED = 1

To point the next run at a different test, replace the whole line rather than the test name, anchoring on the key at the start of the line:

sed 's/^TEST = .*/TEST = apb_burst_test/' run.cfg

SIM_ROOT = /proj/scratch/build
TEST = apb_burst_test
SEED = 1

Two things make this safe. ^TEST = anchors the match to a line that begins with that key, so a test name appearing anywhere else is untouched. And replacing the whole line means the result does not depend on what the old value was. The other two lines pass through unchanged, which is sed's default behaviour from 1.1.

This is still a generated file. Making the flow emit the right value is better than correcting it afterwards — patch the output when you cannot reach the generator, not as a habit.

7. Failure Modes — and One That Is Not

Three of the four situations below are genuine failures, and none of them announces itself: two print plausible output, one prints nothing at all, and all three exit zero. The fourth is not a failure — it is a correct command that is hard to review — and it is here precisely so you can tell the two kinds of risk apart.

The substitution was broader than the path. Replacing a bare word instead of a delimited path:

sed 's/build/release/g' build/files.f

+incdir+/proj/scratch/release/rtl
/proj/scratch/release/rtl/top.sv
/proj/scratch/release/rtl/fifo.sv
/proj/scratch/release/tb/top_tb.sv

Every line changed, so it looks like it worked. It did not: the intent was to replace the prefix /proj/scratch/build/ with /proj/release/, and what happened was that the word build was rewritten wherever it appeared, leaving /proj/scratch/release/ — a path that does not exist. Including the surrounding slashes in the pattern is what made section 4 correct.

The expression was correct and hard to review. This one is not a failure, and it is here to be told apart from the others. / is a perfectly good delimiter; it just means every / inside the pattern and the replacement has to be escaped, because otherwise sed would read it as the end of a part. Written that way, the section 4 command looks like this:

sed 's/\/proj\/scratch\/build\//\/proj\/release\//g' build/files.f

+incdir+/proj/release/rtl
/proj/release/rtl/top.sv
/proj/release/rtl/fifo.sv
/proj/release/tb/top_tb.sv

Identical output to section 4, because it is the same instruction correctly escaped. The problem is not the result, it is the reading: a reviewer has to check nine backslashes to agree the command is right. Get one wrong and the outcome depends on where it falls — sed may reject the expression outright, or it may accept a different expression than you intended and transform the wrong text. Choosing a delimiter your text does not contain, as section 4 does, removes the escaping from the problem entirely. That is a reviewability decision, and the correctness risk it removes is the risk of a mistake nobody spots.

The pattern was valid somewhere else. GNU sed accepts alternation in a basic regular expression; BSD sed does not. On the sed used throughout this chapter:

sed -n '/UVM_ERROR\|UVM_FATAL/p' sim.log

No output, and exit status 0. On GNU sed the same command prints both lines. Nothing warns you: the pattern is not invalid, it simply matches nothing here, and empty output from a filter is indistinguishable from "there was nothing to find". The portable form is the -E version in section 5.

The input format moved. Every pattern above encodes an assumption about layout — that the timestamp is followed by a colon, that the key is at the start of the line, that the path begins with a particular prefix. A tool upgrade that changes any of those turns a working command into one that silently matches nothing.

8. Deciding Whether sed Is the Right Tool

Two questions settle most cases, and they are asked in this order.

Is the change mechanical and describable? If you can say "find this exact thing, put that exact thing in its place" and be confident about every place it occurs, a substitution fits. If answering needs context — which module this is, what the name refers to, whether this occurrence means the same as that one — then the change depends on meaning, and text matching cannot supply it.

What class of file is it? Section 2's ordering is the answer: generated text and tool output are sed's natural home; derived artifacts are usually better regenerated; authored source is where you stop.

That second answer is worth stating plainly, because it is the one that costs real time when it is got wrong. sed matches characters. It has no model of SystemVerilog or VHDL syntax, of elaboration, of macro expansion, of module connectivity, of timing constraints, or of UVM class relationships. A pattern matches inside a comment, inside a string literal, and inside a longer identifier that merely begins the same way, with no indication that it has done so. Renaming a signal correctly everywhere it genuinely is that signal, and nowhere it is not, is refactoring — it needs a tool that has parsed the language, plus a compile and a review to confirm the result. Chapter 9.3 takes that boundary up properly.

When the job is outside sed's range, the alternatives differ in what they are good at rather than in power: the shell for composing steps and selecting files, awk for anything that reasons about columns, Python for structure and logic that deserves its own tests, Tcl for work inside an EDA tool that already embeds it. Chapter 1.3 compares those directly; the purpose here is only to recognise when you have left sed's territory.

9. Think It Through

A colleague needs the release path change from section 4 applied across every filelist in a build directory, and sends you this:

sed 's/scratch/release/g' build/files.f

Work out your answer before reading on.

What it does. It replaces the word scratch wherever it appears, producing /proj/release/build/rtl/top.sv — the build component is still in the path and the prefix was never /proj/release/. It will print four changed lines and exit zero, so at a glance it looks right.

What is wrong with it. The pattern describes a word, not the path prefix the task is actually about. It is the same mistake as the first failure in section 7, and the fix is the same: match the delimited prefix, s|/proj/scratch/build/|/proj/release/|g.

The question worth more than the fix. Before running anything, ask which files are in that directory. If it holds only generated filelists, this is routine work. If a hand-maintained file has been kept there too, the same command reaches it, and section 2 says that is a different category of risk entirely. Deciding what the command will touch is a separate act from deciding what it should do, and this chapter has argued that the first one comes first.

10. Summary

• A chip flow writes text at every stage — filelists, logs, reports, configs — and that generated text is sed's territory.
• The risk of a substitution is a property of the file, not the command. Classify the artifact before writing the expression: authored source, generated text, tool output, derived artifact.
• Authored RTL and constraints are where casual substitution stops. Generated text and logs are where it earns its keep.
• Inspect, preview, then diff. Count what is in scope first; read the preview; on anything larger than a screen, diff it against the original.
• A plain substitution still changes nothing on disk — the behaviour from 1.1 is what makes previewing possible.
• Anchor and delimit the pattern. A bare word matches inside longer strings; including the surrounding slashes is what made the filelist command correct.
• Portability is part of correctness. Plain substitution, ranges and -E behave the same on GNU and BSD sed; alternation in a basic regular expression is GNU-only and fails by matching nothing.
• Every failure in section 7 exited zero. A clean exit says the instruction was well formed, nothing more.
• sed does not understand HDL — not syntax, elaboration, macros, connectivity, constraints or UVM classes. Meaning-dependent renaming is refactoring and needs a different tool.

You can now look at a file a flow produced and say whether a substitution belongs anywhere near it, preview one safely, and name the failure mode when the output looks wrong.

What that leaves open is the comparison itself. grep, awk and Python all operate on the same text, and the boundaries between them are sharper than they first appear. Chapter 1.3 — sed vs grep, awk, and Python draws them, so that choosing the tool stops being a matter of habit.