Why Semiconductor Work Happens on Linux

Part II · Module 22 · Chapter 22.1 · Why Semiconductor Work Happens on Linux

Part I taught the language. It never had to say where the language runs, because none of it depended on that.

Part II does. Every chapter from here assumes a workstation you log into, compute servers your jobs run on, a shared filesystem both can see, and tools that cost more than the machines. That environment is almost universally Linux, and the reason is not culture or inertia. It is four specific assumptions the tools make, each of which is checkable in a few seconds.

1. The Answer in One Table

The rest of this chapter measures the first one, because it is the one that bites a real team on a real afternoon, and then establishes the other three as the ground Part II stands on.

2. Assumption One: The Filesystem Is Case-Sensitive

Verilog is case-sensitive. Top and top are different identifiers, and by near-universal convention a module lives in a file named after it. So an RTL tree may legitimately contain both Top.v and top.v.

On Linux, ext4 and xfs are case-sensitive: those are two files. Here, they are not.

$ echo "module Top; endmodule" > Top.v
$ [ -f top.v ] && echo "exists" || echo "does not exist"
exists

The file was created as Top.v and is answering to top.v. Writing to the lowercase name does not create a second file — it overwrites the first:

$ echo "module top; endmodule" > top.v
$ ls -1 *.v
Top.v
$ cat Top.v
module top; endmodule

Read that carefully. ls reports Top.v — the filesystem is case-preserving, so it keeps the name it was given — and the contents are now the lowercase module. One of the two source files is simply gone, and the directory listing gives no hint that anything happened.

The test that settles it in any script, portably, is identity rather than naming — the same device:inode pair reached by two names:

$ stat -f "%d %i  %N" Top.v top.v
16777230 83530510  Top.v
16777230 83530510  top.v

3. What That Does to a Real Repository

The overwrite above is what happens when you create both files. A team hits this differently: the repository already contains both, authored on Linux, and you clone it.

Here is that reproduced end to end. The repository genuinely tracks two files:

$ git ls-files
Top.v
top.v

The working tree contains one:

$ ls -1
Top.v

And git status says something that takes a minute to believe:

$ git status --short
 M top.v

Git reports an uncommitted modification to the file that is not on disk. It compared the index entry for top.v against what it found by opening top.v — which, on this filesystem, is the bytes of Top.v — saw a difference, and reported a change you never made. The file it names is the one you cannot see.

Cloning is where the warning finally appears:

$ git clone repo clone
Cloning into 'clone'...
warning: the following paths have collided (e.g. case-sensitive paths
on a case-insensitive filesystem) and only one from the same
colliding group is in the working tree:
  'Top.v'

$ cd clone && ls -1
top.v
$ git ls-files
Top.v
top.v
$ cat top.v
module top;
  // a DIFFERENT module, lowercase
endmodule
$ git status --short
 M Top.v

Note that the clone kept the other one. Which name survives depends on checkout order, so two engineers on two Macs can end up with different halves of the same collision — and both will have a git status that reports a modification neither of them made.

The underlying setting is not subtle, and git knows the answer:

$ git config core.ignorecase
true

4. Detecting It Before It Costs You a Day

One line over the index, and it belongs in any repository that must survive being cloned onto a Mac:

#!/usr/bin/env bash
# check-case-collisions — fail if two tracked paths differ only in case
set -o nounset

git ls-files | awk '
  { lower = tolower($0)
    if (lower in seen && seen[lower] != $0)
      { printf "COLLISION: %s  <->  %s\n", seen[lower], $0; bad++ }
    else seen[lower] = $0 }
  END { exit (bad ? 1 : 0) }
'

$ ./check-case-collisions
COLLISION: Top.v  <->  top.v
$ echo $?
1

Three things about that script, all of them Part I:

It reads git ls-files, not the working tree. The working tree has already lost the information — there is one file there. The index still knows about both. Chapter 14.6's find would be the wrong tool here for exactly that reason.

The comparison is seen[lower] != $0. Two identical paths are not a collision; the same path cannot appear twice in the index anyway. What is being detected is two different spellings that fold together.

And the END block converts the finding into a status, so it works in a quality gate — Chapter 21.4's contract, and Chapter 21.8's rule that a gate you have never watched fail is not yet evidence. Break it deliberately: run it on a repository with no collision and confirm it exits 0, then on this one and confirm 1.

5. The Same Assumption, One Layer Up: Locale

Case is a filesystem property. There is a second, subtler one in the same family, and it is the shell rather than the disk: what a character range means depends on the locale.

$ LC_ALL=C bash -c 'case B in [a-z]) echo yes;; *) echo no;; esac'
no
$ LC_ALL=en_US.UTF-8 bash -c 'case B in [a-z]) echo yes;; *) echo no;; esac'
yes

Same shell, same pattern, same input, opposite answers. Under LC_ALL=C the range is the eight-bit code points from a to z; under a UTF-8 locale it is a collation range, and B sorts inside it.

A filelist filter written as [a-z]*.v therefore selects a different set of files for two engineers whose shells were configured differently. Chapter 3.5 taught the glob; this is the same glob with the environment changing its meaning underneath, and Chapter 22.6 is the chapter that makes a habit of hunting causes like it.

The fix is to say what you mean: [[:lower:]] for letters as the locale defines them, an explicit [abcdefghijklmnopqrstuvwxyz] when you mean exactly those, or LC_ALL=C at the top of a script when you want byte semantics everywhere.

6. Assumption Two: POSIX Processes and Exit Status

This is the assumption Part I spent two modules on without naming it. A flow is a tree of processes: a shell starts a compiler, which exits with a number; the shell reads that number and decides whether to start the simulator.

Nothing in that sentence is available in a GUI-first operating system, and everything in it was Part I's subject — Chapter 11.1 for the number, Chapter 7.1 for the process, Chapter 11.3 for && deciding what runs next, Chapter 15.6 for a script of your own joining the same contract.

What Part II adds is that the tools do not always keep their half of it. That is Chapter 22.2's subject, measured with a real simulator, and it is the most important fact in this half of the course.

7. Assumption Three: Shared Compute, One Filesystem

An EDA regression is not one long job. It is five hundred short ones, dispatched across whatever machines are free, all of them reading the same source tree and writing into the same results area.

That only works if every machine sees the same filesystem at the same path. /proj/chip_a/rtl/top.v must mean the identical file whether the job landed on cmp014 or cmp031. The mechanism is a network filesystem — NFS, or a clustered equivalent — exported from a file server and mounted at the same mount point everywhere.

Chapter 22.3 is where that is taken apart, including what to do when the path is not the same in both places, which is the single most common cause of "it works for me".

8. Assumption Four: Licences Are Floating, and on the Network

A commercial simulator will not run without checking out a licence from a server, usually over TCP to a daemon named in an environment variable such as LM_LICENSE_FILE.

This has three consequences a shell user feels directly, and all three are why licensing appears in a Linux course at all:

A run can fail for a reason that has nothing to do with your design. No licence available means the tool exits — and whether it exits with a useful status is tool-specific, which is Module 34's problem — its opening chapter is called "Tools That Exit Zero on Failure", and it is named that for a reason.

Parallelism is bounded by licences, not by cores. A regression that launches two hundred jobs with thirty licences does not run two hundred jobs. Chapter 31.5 is where that becomes a scheduling question.

And a licence is held for the life of the process. A job that hangs holds its licence while it hangs — so one stuck simulation can idle a team. Chapter 22.5 is about not being the person who did that.

9. The Limits a Tool Actually Runs Into

The last thing worth measuring is the set of resource limits an EDA tool meets. These are per-process ceilings the kernel enforces, and an elaborator opening a few thousand files or recursing through a deep hierarchy meets them for real.

$ ulimit -n      # open files, soft limit
1048576
$ ulimit -Hn     # open files, hard limit
unlimited
$ ulimit -s      # stack, in KB
8176
$ ulimit -u      # processes per user
1333
$ getconf ARG_MAX
1048576

Those are generous numbers on this machine, and they are not the numbers on a shared compute server, where a soft ulimit -n of 1024 is still common. Two of them matter enough to name:

ulimit -n is why a large filelist can fail where a small one works. A compiler that keeps every included file open hits a descriptor ceiling, and the error it reports — "too many open files", or something far less clear — says nothing about filelists.

ARG_MAX is why *.v can fail on a big directory. The glob expands to a command line, and a command line has a maximum size. Chapter 14.7 solved this with xargs and find -exec +; Module 26's filelists are the reason the problem shows up in this industry at all.

A probe worth keeping, since the answer differs on every machine you will use:

#!/usr/bin/env bash
# env-limits — the per-process ceilings a tool will meet here
set -o nounset
printf '%-22s %s\n' \
  "open files (soft)" "$(ulimit -n)" \
  "open files (hard)" "$(ulimit -Hn)" \
  "stack KB"          "$(ulimit -s)" \
  "processes"         "$(ulimit -u)" \
  "ARG_MAX"           "$(getconf ARG_MAX)" \
  "cores"             "$( (nproc 2>/dev/null || sysctl -n hw.ncpu) )"

printf is given one format and many arguments rather than being called in a loop: it reuses the format until the arguments run out, which is what makes that four lines instead of twelve. And nproc 2>/dev/null || sysctl -n hw.ncpu because nproc is GNU and absent on BSD, which is Chapter 21.6 arriving on line six of the first script of Part II.

Practice

1. On your own machine, run the Top.v / top.v experiment from §2. Which behaviour do you get? Record it — you will need to know.
2. Run stat -f "%d %i %N" (BSD) or stat -c "%d %i %n" (GNU) on two names you suspect are the same file. Explain why %d is needed as well as %i.
3. Write check-case-collisions and run it on a repository with no collision. Confirm it exits 0 — a check you have only ever seen pass is not yet evidence.
4. Then create a collision deliberately (use git update-index --cacheinfo, as §3 did, so the filesystem cannot stop you) and confirm the script exits 1 and names both paths.
5. Run git status --short in a repository with a case collision. Explain, in one sentence, why git reports a modification you did not make.
6. Run the case B in [a-z]) test under LC_ALL=C and under a UTF-8 locale. Then rewrite the pattern two ways so that it means the same thing in both.
7. Run env-limits on your machine, then on any shared server you have access to. Which number differs most, and which tool would notice?
8. getconf ARG_MAX, then find a directory large enough that ls * fails. What exactly is the error, and which command does it come from?
9. Find out whether your own / is case-sensitive without creating any files, using only mount, diskutil or df.
10. For each of the four assumptions in §1, name one thing in your current environment that would break if it were false.

Check Your Understanding

1. What are the four assumptions EDA tools make about the operating system?
2. Why can a case collision not be detected by comparing file names?
3. On a case-insensitive filesystem, what does ls show after echo x > Top.v; echo y > top.v, and what are the contents?
4. Why does git status report a modification to a file that is not in the working tree?
5. Why is the failure mode of a case collision worse than a missing file?
6. What does LC_ALL have to do with which files a glob selects?
7. Why does a hung simulation affect your colleagues, and not only you?

10. Remember

• Four assumptions: case-sensitive filesystem, POSIX processes and exit status, shared compute on one filesystem, floating network licences.
• Top.v and top.v are the same file on macOS by default. The second write wins, ls shows the first name, and nothing warns you.
• Detect it with device:inode, never with names. Both names pass [ -f ].
• git ls-files still knows about both when the working tree has only one — which is why the check reads the index.
• git status reporting a change you did not make to a file you cannot see is the signature of a case collision. git clone warns; git pull into an existing tree may not.
• The failure is a plausible wrong answer, not an error — the filelist resolves because something is there.
• A bracket range depends on the locale. [a-z] matched B under en_US.UTF-8 and not under LC_ALL=C. Use [[:lower:]], or set LC_ALL=C deliberately.
• A flow is a process tree reporting numbers, which is the whole of Part I's exit-status material becoming load-bearing.
• One filesystem at the same path on every machine is what makes a regression possible at all.
• Licences bound parallelism, not cores — and a hung job holds its licence.
• ulimit -n and ARG_MAX are real ceilings that filelists and recursive elaboration meet, and they differ on every machine.
• Never show a transcript for a tool you do not have. Describe and attribute instead.

11. What Is Next

Chapter 22.2 takes assumption two seriously and measures it with a real simulator: where the shell actually sits in a compile-and-simulate flow, and the single most consequential fact in Part II — that a tool failing and a test failing reach the shell as two completely different things.