YAML for Configuration Humans Edit

Part II · Module 21 · Chapter 21.5 · YAML for Configuration Humans Edit

1. What a Person Needs From a Config File

Chapter 21.4 Section 7 listed four edits an engineer makes to a regression config and showed all four failing to parse. The requirements that follow are short:

• Comments, so the reason a timeout is 7200 lives next to the 7200.
• Forgiving punctuation, so deleting the last entry of a list does not leave a trailing comma that stops the whole run.
• Readable nesting, so a per-test override is visibly inside its test.

JSON provides none of the three. That is not a flaw in JSON — it was designed for machines to exchange — and it is a real problem for the one file in a flow that engineers open weekly.

2. What YAML Looks Like

The same config as Chapter 21.4's, in YAML:

# The simulator the whole flow uses. Changing this needs a clean build.
sim: xcelium

# Raised from 3600 for the gate-level runs — see VLSI-4412.
timeout_s: 7200

waves: false

tests:
  axi_smoke:
    seeds: 4
  axi_burst:
    seeds: 16       # 16 since the arbiter rewrite

Three comments, no braces, no commas, and the nesting is indentation. Deleting the axi_burst block leaves a file that still parses. Adding a note above a value is one line.

This block is shown, not executed. The interpreter this curriculum is verified on does not have a YAML library installed, which is the subject of the next section rather than an omission — every executed example in this chapter runs, and every YAML example is source text.

3. The Cost, on This Machine

print("import json:")
import json
print("   fine, standard library")

print()
print("import tomllib:")
import tomllib
print("   fine, standard library since 3.11")

print()
print("import yaml:")
try:
    import yaml
    print("   available, version", yaml.__version__)
except ModuleNotFoundError as exc:
    print("   ModuleNotFoundError:", exc)

import json:
   fine, standard library

import tomllib:
   fine, standard library since 3.11

import yaml:
   ModuleNotFoundError: No module named 'yaml'

That is the cost, and it is worth seeing rather than being told. A script using YAML does not run on this interpreter until somebody installs something.

In a VLSI flow that sentence has more weight than it does in web development. The machine that runs your regression at 2am is frequently not the machine you wrote the script on: it is a farm node with a site-managed Python, a module system that pins versions, and no network access to a package index. "Just pip install it" is a conversation with whoever owns that image, and on a shared farm it is a conversation that takes a week.

So the question the canonical purpose asks — where does YAML earn an extra dependency — has a concrete answer: where the file is edited by people often enough that comments and forgiving syntax are worth that conversation, and where the environment is yours to change.

4. safe_load, Never load

The canonical annotation carries its own warning:

cfg = yaml.safe_load(Path("regress.yaml").read_text())   # never yaml.load

The reason is a documented property of the library rather than a style preference. YAML's full specification can describe arbitrary objects, and PyYAML's load will construct them — which means a YAML file can cause a program to be run as a side effect of being parsed. safe_load restricts the document to plain data: strings, numbers, booleans, null, lists and dicts.

A regression config comes from a repository, a web form, a user's home directory, or a colleague's email. Treating it as data rather than as code is not paranoia; it is the same reasoning that made eval unacceptable for reading a CSV cell in Chapter 21.1 Section 5.

The rule is short and absolute: safe_load for reading, safe_dump for writing, every time. Modern PyYAML has made load safer than it once was, and the rule stands anyway — because it costs four characters and removes the need to know which version is installed.

This section is attributed rather than demonstrated. Verifying it would need the dependency, and this chapter is explicitly about running without it.

5. The Standard Library Already Has One

Most flows reach for YAML without noticing that Python ships a human-editable config format:

# The simulator the whole flow uses. Changing this needs a clean build.
sim = "xcelium"

# Raised from 3600 for the gate-level runs — see VLSI-4412.
timeout_s = 7200
waves = false

[tests.axi_smoke]
seeds = 4

[tests.axi_burst]
seeds = 16        # 16 since the arbiter rewrite

import json
import tomllib
from pathlib import Path

cfg = tomllib.loads(Path("regress.toml").read_text(encoding="utf-8"))

print("sim      :", cfg["sim"])
print("timeout_s:", cfg["timeout_s"], type(cfg["timeout_s"]).__name__)
print("waves    :", cfg["waves"], type(cfg["waves"]).__name__)
print("tests    :", sorted(cfg["tests"]))
print("seeds    :", cfg["tests"]["axi_burst"]["seeds"])
print()
print("as json  :", json.dumps(cfg, sort_keys=True))

sim      : xcelium
timeout_s: 7200 int
waves    : False bool
tests    : ['axi_burst', 'axi_smoke']
seeds    : 16

as json  : {"sim": "xcelium", "tests": {"axi_burst": {"seeds": 16}, "axi_smoke": {"seeds": 4}}, "timeout_s": 7200, "waves": false}

Comments at the top and at the end of a line, nesting through [tests.axi_burst] headers, booleans as false, and the result is an ordinary dict — the same dict JSON would have produced, which is the point of the last line.

tomllib has been in the standard library since Python 3.11. On an interpreter at least that new, a commented, human-editable config costs nothing to install.

6. What tomllib Will Not Do

One limitation decides where it fits:

import tomllib

print("tomllib exports:", [name for name in dir(tomllib) if not name.startswith("_")])
print("can it write?  :", hasattr(tomllib, "dumps"), hasattr(tomllib, "dump"))

tomllib exports: ['TOMLDecodeError', 'load', 'loads']
can it write?  : False False

Read only. Three names: two readers and an exception. There is no dumps, by design — the standard library took the position that reading configuration is a universal need and writing TOML is not.

That settles the division of labour rather than complicating it. A config is written by a person and read by a program, so a read-only parser is all it needs. A result is written by a program and read by a program, which is Chapter 21.4's JSON and Chapter 21.3's CSV. The absence of a writer is a statement about what TOML is for.

If a flow genuinely needs to write TOML — regenerating a config from a UI, say — that needs a third-party package, and the dependency conversation from Section 3 starts again. Writing JSON instead is almost always the better answer.

7. Three Formats, Three Jobs

Reading that table as a decision:

A table of results — CSV. Chapter 21.1's record is scalars-only for exactly this reason, and a spreadsheet opens it.

A structured result, or anything nested — JSON. Chapter 21.4.

A config a person edits — TOML if your interpreter is 3.11 or newer, because it costs nothing. YAML if your project already depends on it, if the config needs YAML-specific structure, or if the team's other tooling reads YAML and a second format would be worse than a dependency.

A config generated by a program — JSON, whatever the hand-edited ones use. Nobody reads a generated file by hand, and generating JSON needs no writer you do not have.

The failure this table is meant to prevent is the common one: adopting YAML for a config, then needing to write one from a script, then adding a second dependency for that, then having two formats in the flow because the results are still JSON. Start from what has to be written by a program, and only the hand-edited files are a free choice.

8. A Loader That Degrades Honestly

If a project does use YAML, the script that reads it should fail in a way somebody can act on:

sim = "xcelium"
timeout_s = 7200

import json
import tomllib
from pathlib import Path


def load_config(path):
    """Read a config by its suffix. YAML needs PyYAML; the others do not."""
    text = Path(path).read_text(encoding="utf-8")
    suffix = Path(path).suffix.lower()

    if suffix == ".json":
        return json.loads(text)
    if suffix == ".toml":
        return tomllib.loads(text)
    if suffix in (".yaml", ".yml"):
        try:
            import yaml
        except ModuleNotFoundError:
            raise RuntimeError(
                f"{path}: reading YAML needs PyYAML, which is not installed. "
                f"Install it, or convert the config to .toml or .json."
            ) from None
        return yaml.safe_load(text)

    raise ValueError(f"{path}: unknown config format {suffix!r}")


print("toml   :", load_config("regress.toml"))

Path("regress.json").write_text('{"sim": "vcs"}', encoding="utf-8")
print("json   :", load_config("regress.json"))

Path("regress.yaml").write_text("sim: vcs\n", encoding="utf-8")
try:
    load_config("regress.yaml")
except RuntimeError as exc:
    print("yaml   :", exc)

Path("regress.ini").write_text("sim = vcs\n", encoding="utf-8")
try:
    load_config("regress.ini")
except ValueError as exc:
    print("unknown:", exc)

toml   : {'sim': 'xcelium', 'timeout_s': 7200}
json   : {'sim': 'vcs'}
yaml   : regress.yaml: reading YAML needs PyYAML, which is not installed. Install it, or convert the config to .toml or .json.
unknown: regress.ini: unknown config format '.ini'

Four behaviours, and the third is the one that matters. The message names the file, says what is missing, and gives two ways out — which is the difference between a colleague fixing it themselves in a minute and filing a bug against your script.

Three details are deliberate.

The import is inside the branch. A top-level import yaml would stop the script from running at all, even for the JSON config it was given — so a flow that mostly uses JSON would be broken by the mere possibility of YAML.

from None suppresses the chained traceback. Raising inside an except block normally shows both exceptions, joined by During handling of the above exception, another exception occurred — and here a reader seeing ModuleNotFoundError followed by the real message has to read twice, when the ModuleNotFoundError adds nothing the new message does not already say. from None drops the original from the report.

The unknown suffix raises too. Silently defaulting to JSON for a .ini file would produce a parse error about the file's contents rather than about its format.

9. Predict, Run, Observe

import tomllib

TOML = """
sim = "xcelium"      # trailing comment
timeout_s = 7200
waves = false
ratio = 0.5
names = ["a", "b"]

[tests.smoke]
seeds = 4
"""

cfg = tomllib.loads(TOML)

# A
print("A:", sorted(cfg))

# B
print("B:", type(cfg["timeout_s"]).__name__, type(cfg["ratio"]).__name__)

# C
print("C:", cfg["waves"], type(cfg["waves"]).__name__)

# D
print("D:", cfg["names"], type(cfg["names"]).__name__)

# E
print("E:", cfg["tests"]["smoke"]["seeds"])

# F
try:
    tomllib.loads('sim = "xcelium"\ntimeout_s = \n')
except tomllib.TOMLDecodeError as exc:
    print("F:", type(exc).__name__)

# G
try:
    import yaml
    print("G: yaml available")
except ModuleNotFoundError:
    print("G: no yaml")

The verified output:

A: ['names', 'ratio', 'sim', 'tests', 'timeout_s', 'waves']
B: int float
C: False bool
D: ['a', 'b'] list
E: 4
F: TOMLDecodeError
G: no yaml

A — the comments are gone from the parsed data. A config format's comments are for the file, not for the program.

B — TOML distinguishes integers from floats, which JSON does too and CSV does not.

C — false becomes Python's False, a real boolean rather than Chapter 21.1's 'True' string.

D — a list, nested inside a config, with no quoting rules to get wrong.

E — [tests.smoke] produced a nested dict, the same shape JSON's braces would have.

F — a malformed value raises a named exception with a position, like Chapter 21.4's JSONDecodeError. Chapter 21.7 is about catching these at load time.

G — still not installed, which is the chapter.

10. Exercises

Exercise 1 — the three requirements. Take a JSON config from your own flow and list every place a comment would help. Count them.

Exercise 2 — the cost. Run import yaml on the interpreter your regression actually uses — not your laptop. Record what happens and who you would have to ask to change it.

Exercise 3 — write the YAML. Convert a config of yours to YAML by hand, with a comment on every non-obvious value. Say what the file gained.

Exercise 4 — safe_load. State in two sentences why yaml.load is not interchangeable with yaml.safe_load, and name the chapter where the same reasoning rejected eval.

Exercise 5 — the stdlib one. Convert the same config to TOML and read it with tomllib. Confirm the comments are absent from the parsed dict.

Exercise 6 — types. For one config, compare what TOML, JSON and CSV each give you for an integer, a float and a boolean.

Exercise 7 — read only. List tomllib's exports and say what that implies about which files in your flow it can own.

Exercise 8 — fill in the table. Complete Section 7's table for your own flow: which file is written by whom, read by whom, and what it costs to install.

Exercise 9 — the two-dependency trap. Describe the sequence by which a flow that adopts YAML for configs ends up with two config formats and two dependencies.

Exercise 10 — the loader. Write load_config dispatching on the suffix. Confirm the YAML branch produces an actionable message on an interpreter without PyYAML.

Exercise 11 — import placement. Move the import yaml to the top of the file and run the loader on a JSON config. Record what happens and why the branch-local import is correct.

Exercise 12 — from None. Remove the from None and compare the two tracebacks. Say which one a colleague reads faster.

11. Summary

• A config a person edits needs comments, forgiving punctuation and readable nesting. JSON has none of the three, by design.
• YAML provides all three and costs a dependency. On the interpreter this curriculum is verified on, import yaml raises ModuleNotFoundError.
• That cost is larger in a VLSI flow than elsewhere — the farm node that runs your regression at 2am has a site-managed Python and no package index.
• YAML earns the dependency where the file is edited often enough for comments to matter and the environment is yours to change.
• safe_load for reading and safe_dump for writing, always. YAML's full specification can construct arbitrary objects, so a config file is data and must be parsed as data.
• tomllib is in the standard library from Python 3.11, has comments, nests, and gives back an ordinary dict.
• tomllib is read-only — load, loads and TOMLDecodeError, with no writer — which is a statement about what TOML is for.
• Match the format to the direction of travel: CSV for result tables, JSON for structured results and anything a program writes, TOML or YAML for hand-edited configs.
• Start from what a program has to write. Only the hand-edited files are a free choice.
• Three formats with three audiences is not confusion; one format for everything means either configs with no comments or tables a spreadsheet cannot open.
• Import the optional dependency inside the branch that needs it, so a flow using JSON is not broken by the possibility of YAML.
• Make the failure actionable — name the file, name what is missing, and give a way out.
• raise ... from None keeps the message a colleague reads to one.

The next chapter takes whichever format you chose and answers the question every real flow asks of it: what happens when a project default, a block override and a per-test setting all specify the same thing.