Creating Files and Directories

Part I · Module 3 · Chapter 3.1 · Creating Files and Directories

1. From Looking to Changing

Module 2 gave you a complete picture of the tree and a notation for pointing into it. Every command in it was read-only. ls looked, pwd reported, cd moved you — and if you got a path wrong, you saw an error and tried again.

That safety ends here. The commands in this module add, overwrite and remove things, and two facts from Module 2 stop being merely useful and become load-bearing:

• Your working directory decides what a relative path means. A command that creates something in the wrong place is a command run from the wrong place.
• There is no undo. Chapter 3.3 is entirely about that, and it is not an exaggeration.

So the habit Chapter 2.4 introduced — pwd before anything that matters — stops being tidy practice and starts being the thing that keeps your work intact.

2. mkdir — Make a Directory

$ pwd
/home/engineer/project

$ mkdir build

$ ls
build  docs  rtl  scripts  sim

One new entry. Chapter 2.1 described a directory as a container for entries, and that is literally what happened: mkdir added an entry named build to project, and that entry points at a new, empty directory.

The argument is a path like any other, so every form from Chapter 2.2 works:

$ mkdir sim/run1          # relative — resolved from the working directory
$ mkdir /tmp/scratch      # absolute — same meaning from anywhere
$ mkdir ../spare          # relative, using the parent
$ mkdir ~/notes           # the shell expands ~ first, per Chapter 2.3

Several at once, in one command:

$ pwd
/home/engineer/project

$ mkdir reports waves coverage

$ ls
build  coverage  docs  reports  rtl  scripts  sim  waves

That is Chapter 1.5's grammar in action: mkdir takes any number of operands and creates one directory per operand.

3. The First Refusal: It Already Exists

$ mkdir build
mkdir: cannot create directory 'build': File exists

Read it with Chapter 1.3's rule: the message begins with mkdir:, so the program ran and declined. It did not overwrite the existing directory, and it did not delete anything inside it.

That refusal is a safety feature. Overwriting a directory you forgot about would destroy whatever was in it, so mkdir simply will not do it.

It also returns a non-zero exit status, which matters more than it looks:

$ mkdir build
mkdir: cannot create directory 'build': File exists

$ echo $?
1

$? is the status of the last command, which Module 11 teaches in full. The useful part now: a failure here is reported, not silent — unlike some commands later in this module.

4. The Second Refusal: the Parent Is Missing

Try to create two levels at once:

$ pwd
/home/engineer/project

$ mkdir sim/run2/logs
mkdir: cannot create directory 'sim/run2/logs': No such file or directory

This one confuses people, because the message names the whole path and the real problem is in the middle of it.

Work it out with Chapter 2.2's rule. The path resolves to /home/engineer/project/sim/run2/logs. For mkdir to create logs, the directory that will contain it — run2 — has to exist already. It does not. mkdir creates one directory, in a parent that is already there, and it will not invent the missing links in a chain.

The long way works:

$ mkdir sim/run2
$ mkdir sim/run2/logs

Two commands, each creating one level in a parent that exists by the time it runs.

5. mkdir -p — Make the Whole Chain

The option that does it in one step:

$ pwd
/home/engineer/project

$ mkdir -p sim/run2/logs

$ ls sim/run2
logs

-p is for parents: create any missing parent directories along the way. Chapter 1.5's grammar applies — -p is a short option, and mkdir -p is one option and one operand.

The lines in Figure 1 mean contains, exactly as in Chapter 2.1 — read it top-down, and the labels rather than the colour tell you which two are new.

The other half of -p, which matters more

-p does a second thing that is easy to miss and ends up being the reason it is used so widely:

$ mkdir -p sim/run2/logs

$ mkdir -p sim/run2/logs

$ echo $?
0

It does not complain when the directory already exists. Run it twice and the second run succeeds quietly, having done nothing.

Compare that with plain mkdir, which fails the second time. The difference is the whole reason mkdir -p appears in nearly every setup script ever written: it expresses "make sure this exists" rather than "create this new thing", and only the first of those is safe to run again.

6. touch — Make an Empty File

$ pwd
/home/engineer/project

$ touch notes.txt

$ ls -l notes.txt
-rw-r--r-- 1 engineer engineer 0 Mar 20 11:04 notes.txt

Read that listing with Chapter 2.5: the leading - says ordinary file, and the size is 0. The file exists and is completely empty.

Several at once, as usual:

$ touch a.txt b.txt c.txt

And with any path form:

$ touch sim/run1/notes.txt      # relative
$ touch ~/scratch.txt           # the shell expands ~ first

One limit worth knowing immediately, because it is the same lesson as §4:

$ touch sim/run9/notes.txt
touch: cannot touch 'sim/run9/notes.txt': No such file or directory

touch creates a file, not the directories above it. If run9 does not exist, there is nowhere to put the file. There is no -p for touch; make the directory first.

7. What touch Does to a File That Already Exists

This is the part people guess wrong, and the guess is expensive.

A reasonable-sounding assumption is that touch on an existing file empties it, or resets it, or recreates it. It does none of those. Watch:

$ printf 'hello\n' > notes.txt

$ ls -l notes.txt
-rw-r--r-- 1 engineer engineer 6 Mar 20 11:04 notes.txt

$ touch notes.txt

$ ls -l notes.txt
-rw-r--r-- 1 engineer engineer 6 Mar 20 11:09 notes.txt

$ cat notes.txt
hello

Compare the two listings field by field, using Chapter 2.5:

• the size is still 6 — not a byte was changed;
• the timestamp moved from 11:04 to 11:09;
• the contents are intact, as cat confirms.

So the honest description of touch is: update a file's timestamp, and create the file if it is not there. The creating is the side effect people use it for; updating the timestamp is its actual job, and the name says so.

The printf 'hello\n' > notes.txt line above used that same redirection to put content in a file. It is shown so the demonstration is complete, not taught — the > and what it does belong to Module 12.

8. Checking What You Created

Three habits, each one command, worth using every time.

Confirm the location before creating. The silent half of every relative path:

$ pwd
/home/engineer/project

$ mkdir -p sim/run3/logs

Confirm the shape afterwards. ls on the thing you just made:

$ ls sim/run3
logs

Confirm it is the right kind of thing. Chapter 2.5's -ld describes the entry itself rather than listing its contents:

$ ls -ld sim/run3
drwxr-xr-x 3 engineer engineer 4096 Mar 20 11:12 sim/run3

The leading d confirms a directory, and the owner column confirms it belongs to you. That last check matters on a shared machine, where a directory created under the wrong account is a problem you want to find now rather than an hour later.

9. When Creating Fails

Three failures, three different causes. The message tells you which.

$ mkdir build
mkdir: cannot create directory 'build': File exists

Something of that name is already there. Nothing was changed. If you expected an empty directory, ls build will tell you what is actually in it.

$ mkdir sim/run9/logs
mkdir: cannot create directory 'sim/run9/logs': No such file or directory

A directory in the middle of the path is missing. Either create the chain with -p, or check the path — because this message also appears when you have simply mistyped a directory name.

$ mkdir /usr/local/mytool
mkdir: cannot create directory '/usr/local/mytool': Permission denied

The path is fine and you are not allowed to write there. Chapter 2.5 showed where that permission is recorded; Module 5 explains it. On a shared machine this is normal and correct — most of the tree is not yours to modify, and your home directory is.

10. Why This Matters in Engineering Work

Two connections, neither requiring any semiconductor knowledge.

A directory per run. Work that produces output — a build, a simulation, any long job — is far easier to live with when each run writes into its own directory. Creating it is exactly §5:

$ mkdir -p sim/run_042/logs

Why it matters: two runs writing into the same directory overwrite each other's results, and when something looks wrong you cannot tell which run produced which file. One directory per run makes every result traceable to the thing that made it. Part II builds whole flows on this idea; the command is the one you just learned.

-p because setup runs more than once. A sequence of commands that prepares a work area gets run repeatedly — on a new machine, after a cleanup, by a colleague. With plain mkdir the second run fails on the first directory that already exists. With -p it succeeds and changes nothing. That is the difference between a setup you can re-run and one that needs a human to decide whether its failure mattered.

11. Practice

Work these out before trying them. The starting tree is:

/home/engineer/project
├── docs
├── rtl
├── scripts
└── sim
    └── logs

Question 1. Your working directory is /home/engineer/project. What does mkdir sim/run1/data do, and why?

Question 2. Same starting point. Write the single command that creates /home/engineer/project/sim/run1/data.

Question 3. You run mkdir -p docs and get no output and no error, even though docs already existed. Did anything change? What would plain mkdir docs have done instead?

Question 4. notes.txt contains three lines. You run touch notes.txt. How many lines does it contain afterwards, and which field of ls -l changed?

Question 5. Your working directory is /home/engineer/project/sim. You run mkdir ../reports. Where is the new directory, written as an absolute path?

Question 6. touch sim/run5/out.log fails with No such file or directory, but sim definitely exists. What is missing, and what are the two commands that fix it?

Question 7. You are writing a sequence of commands that will be run every time a new work area is set up. Which of mkdir and mkdir -p should it use, and what goes wrong with the other one on the second run?

12. Remember

• This module changes things. pwd before a command that creates, copies or deletes is no longer tidiness — it is what keeps the result where you meant it.
• mkdir creates one directory in a parent that already exists. A nested path fails with No such file or directory, naming the whole path while the real gap is in the middle.
• mkdir -p creates every missing level on the way down.
• -p also makes "already there" a success, not a failure — which is why it belongs in anything that may run twice.
• mkdir refuses to overwrite an existing directory, and reports a non-zero status when it refuses. That refusal protects whatever was inside.
• touch creates an empty file, and takes no -p — the directories above it must already exist.
• On an existing file, touch changes only the timestamp. The size and the contents are untouched. It is not a way to empty a file.
• Read the failure: File exists, No such file or directory and Permission denied are three different problems with three different fixes.

You can now add to the tree. The next chapter copies and moves what is in it — and introduces the first command in this course that destroys an existing file without asking you first.