End-to-End Filelist Migration

Part IV · Module 12 · Chapter 12.1 · End-to-End Filelist Migration

1. The Task

The RTL tree has moved from /proj/old to /proj/new. Every filelist that points at it has to follow, and the change has to be reviewable by somebody who was not there when it was made.

cat -n filelists/top.f

     1	+define+SYNTHESIS
     2	+incdir+/proj/old/rtl
     3	/proj/old/rtl/alu.sv
     4	/proj/old/rtl/fifo.sv
     5	# /proj/old/rtl/disabled.sv
     6	/proj/old/tb/env.sv

cat -n filelists/tb.f

     1	+incdir+/proj/old/tb
     2	/proj/old/tb/test.sv

Two files. Chapter 9.1 covered the substitution; this chapter is everything around it.

2. The Procedure

3. Step 1 — Inventory

Start from what is there, not from what you remember. The registered inventory pulls every project path out of every filelist:

sed -n 's|.*\(/proj/[^[:space:]]*\).*|\1|p' filelists/*.f | sort -u

/proj/old/rtl
/proj/old/rtl/alu.sv
/proj/old/rtl/disabled.sv
/proj/old/rtl/fifo.sv
/proj/old/tb
/proj/old/tb/env.sv
/proj/old/tb/test.sv

Seven distinct paths, all under /proj/old, so this is one migration rather than several tangled together.

Look at the third entry. /proj/old/rtl/disabled.sv is the commented-out line. The inventory found it because the pattern does not care about the #, and that is useful: you now know, before writing anything, that the migration will touch a disabled entry and that somebody should decide whether it should.

4. Step 2 — Predict

grep -c '/proj/old' filelists/*.f

filelists/tb.f:2
filelists/top.f:5

Two plus five is seven lines. Write that number down. It is the only thing that can contradict git in step 5, and a migration with no predicted count cannot be verified — only inspected, which is weaker and slower.

Note this is a line count and the inventory was a path count; they agree at seven here by coincidence of layout, not by rule. Chapter 11.1 measured a case where they differed, and the difference was the signal that a line needed /g.

5. Step 3 — Dry Run, and Read It

sed 's|/proj/old/|/proj/new/|g' filelists/top.f | diff -u filelists/top.f -

--- filelists/top.f	2026-10-12 23:42:05
+++ -	2026-10-12 23:43:28
@@ -1,6 +1,6 @@
 +define+SYNTHESIS
-+incdir+/proj/old/rtl
-/proj/old/rtl/alu.sv
-/proj/old/rtl/fifo.sv
-# /proj/old/rtl/disabled.sv
-/proj/old/tb/env.sv
++incdir+/proj/new/rtl
+/proj/new/rtl/alu.sv
+/proj/new/rtl/fifo.sv
+# /proj/new/rtl/disabled.sv
+/proj/new/tb/env.sv

Five lines on this file, and +define+SYNTHESIS shown with a leading space — present, considered, unchanged.

The fourth changed line is the commented-out entry. This is the moment to decide about it, and the diff is what makes the decision possible: a reviewer sees # /proj/new/rtl/disabled.sv and can ask whether a disabled file should be repointed at a tree where it may not exist. Chapter 9.1 showed the anchored alternative that leaves it alone.

Here the decision is to migrate it — the file moved with the rest of the tree, and leaving the comment pointing at the old path would be its own small lie. What matters is that it was a decision rather than a side effect.

6. Step 4 — Apply

sed -i.bak 's|/proj/old/|/proj/new/|g' filelists/*.f
ls -1 filelists/

tb.f
tb.f.bak
top.f
top.f.bak

Two edits, two backups. On BSD/macOS the suffix is attached the same way; the bare -i forms differ, which is Chapter 7.1.

The .bak files are belt-and-braces for this single run. The real safety net is the commit in step 6 — Chapter 7.2 measured why: run -i.bak a second time and the backup is overwritten with the first run's output, so it protects you against exactly one mistake. Delete them before committing.

7. Step 5 — Verify Against the Prediction

git diff --stat

 filelists/tb.f  |  4 ++--
 filelists/top.f | 10 +++++-----
 2 files changed, 7 insertions(+), 7 deletions(-)

Seven insertions, seven deletions — the number predicted in step 2.

The two numbers come from different places — yours from grep -c before the edit, git's from comparing the trees afterwards — so they are not two views of one computation, and agreement means something.

It is evidence, not proof. Seven matching seven does not establish that the correct seven lines changed, that the new paths exist, that the filelist still compiles, or that it is still semantically valid for the tool that reads it. A pattern that was wrong in a uniform way would produce exactly this agreement.

So the count sits third in a hierarchy, not at the top of one:

Steps 1 and 2 are the ones that catch a wrong edit; 3 and 4 catch an incomplete one. Step 6 is the only one that proves the result works, and it was not performed here — no simulator or compiler consumed this filelist, because none is installed on the machine these examples were verified on. Chapter 9.2 is where a compiler was available and used; this chapter stops at step 5.

If git had said nine, three things could be true — the pattern was wider than you thought, a file you did not inventory was in the glob, or your grep -c counted lines where the substitution changed several occurrences on some of them. All three are found by reading the diff, and all three are cheaper to find now than after the commit.

Then the stale check:

grep -rn '/proj/old' filelists/ --include='*.f' || echo 'no stale references'

no stale references

The --include matters: without it the .bak files are searched too, and they correctly contain the old paths, so the check would fail on its own backups.

8. Step 6 — Commit, and Knowing Which Recovery You Need

   rm -f filelists/*.bak
   git add filelists/
   git commit -m "filelists: migrate /proj/old -> /proj/new (7 lines, 2 files)"

Putting the count in the message is a small habit worth having: it records the prediction alongside the change, so a later reader can check the same equality you did.

Recovery is the other half, and it is worth being precise because the command people reach for first does not do what they think. These were measured in a disposable repository on git 2.50.1.

If the edit is still unstaged — you ran sed, you have not run git add:

   git restore filelists/            # modern spelling
   git checkout -- filelists/        # older spelling, same effect here

Both restore the tracked files in that directory from the index. This is the case Chapter 12.3's rollback uses, and it is the common one.

If you already staged it — the distinction that catches people:

   git restore --source=HEAD --staged --worktree filelists/

A bare git checkout -- filelists/ here restores the file from the index, which is the staged version — that is, your migration, not the original. Measured: with /proj/STAGED staged and /proj/THIRD in the worktree, git checkout -- produced /proj/STAGED, while HEAD still held /proj/old. The --source=HEAD form is what actually goes back.

If you already committed it — neither of the above does anything at all:

   git revert --no-edit <sha>        # a NEW commit that undoes it; history preserved
   git reset --hard <sha>            # moves the branch; DISCARDS commits — shared branches, no

Measured: after committing the migration, git checkout -- top.f left the file migrated and git status clean, because the worktree already matched. git revert --no-edit HEAD restored the content and added a Revert "migrate paths" commit on top.

9. Think It Through

1. Why predict a line count before the edit rather than just reading the diff after?

Because a prediction can be wrong in a way you notice. Reading a diff tells you what happened; comparing it against a number you committed to beforehand tells you whether what happened is what you meant. The two numbers also come from different tools, so their agreement is evidence rather than a tautology.

2. The inventory found a commented-out path. What do you do with it?

Decide, visibly. Either migrate it — the file moved, so the comment should point at where it now lives — or leave it with Chapter 9.1's anchored pattern. What you should not do is let the dry run scroll past without noticing, because then the decision was made by a regex.

3. Why does the stale check need --include='*.f'?

Because the .bak files are in the same directory and correctly contain the old paths. Without the filter the check fails on your own backups, and a check that cries wolf is one you will start ignoring.

4. git diff --stat says 9 lines; you predicted 7. Name three possible causes.

The pattern matched something you did not inventory; the glob included a file you were not thinking about; or your grep -c counted lines while the substitution changed more than one occurrence on some of them. All three are found by reading the diff, and all three are cheaper to find now than after the commit.

5. You ran the migration on a dirty tree and it was wrong. What are your options?

Restoring the directory would discard the unrelated unstaged work as well, so that is not free. The .bak files help if you used -i.bak and have not re-run — Chapter 7.2 measured the second run destroying them. Otherwise you are separating your change from somebody else's by hand. This is the argument for git status --short as step zero: it is the difference between one command and an afternoon.

5b. The migration is already committed and wrong. Now what?

git revert --no-edit <sha>, which adds a commit undoing it and leaves history intact — measured in section 8. git reset --hard also works and discards commits, which is wrong on any branch somebody else has pulled. What does not work is git checkout -- filelists/: after a commit the worktree already matches, so it does nothing.

6. What would make this procedure wrong for a tree of 400 filelists rather than 2?

The glob. At that scale filelists/*.f is both fragile and liable to exceed the argument limit, and the inventory becomes something to skim rather than read. Chapter 8.3's find … -print0 | xargs -0 is the replacement, with the target list reviewed as its own artifact — and the prediction becomes more valuable, not less, because nobody can eyeball 400 diffs.

10. Summary

• The substitution is one line of a six-step procedure, and the other five are what make the change reviewable.
• Inventory first. Seven distinct paths, including a commented-out entry the migration would touch — found before anything was written.
• Predict a line count: grep -c gave 2 + 5 = 7.
• Dry-run as a diff, so the decision about the commented-out line is visible rather than incidental.
• Apply with -i.bak for the single run; the backup protects against one mistake only (Chapter 7.2).
• Verify against the prediction: git diff --stat reported 7 insertions, 7 deletions. Agreement between a number you made up beforehand and one git computed afterwards is real evidence.
• Filter the stale check with --include='*.f', or it fails on your own .bak files.
• Commit with the count in the message, so a later reader can repeat the check.
• Three states, three recoveries — measured: unstaged takes git restore path; staged needs git restore --source=HEAD --staged --worktree path, because a bare checkout -- restores the staged version; and once committed only git revert <sha> undoes it, while checkout -- does nothing.
• All of them act on tracked files only, and restoring a directory discards unrelated unstaged work in it — which is why git status --short comes before step 1.

That is the procedure on an artifact with no semantics. The next case study applies the same six steps to something executable, where the edit is a multi-rule upgrade rather than a single substitution and the script itself has to be reviewed. Chapter 12.2 — Simulator Upgrade Automation moves a directory of run scripts from one tool release to the next.