Layering Defaults, Per-Block and Per-Test Settings

Part II · Module 21 · Chapter 21.6 · Layering Defaults, Per-Block and Per-Test Settings

1. Why There Is More Than One File

A regression config that is one file is a config that either repeats itself or cannot express what a project needs:

• Project defaults — the simulator, the UVM verbosity, the house timeout. One file, owned by the methodology team, changed rarely.
• Per-block overrides — the memory block's tests take four times as long as the CPU's. One file per block, owned by that block's team.
• Per-test overrides — three tests need waves on and one needs a different seed count. A section inside the block file, or a file of its own.
• Tonight — a flag on the command line, because somebody is debugging.

The runner does not want four files. It wants one dictionary — the effective config — and the whole of this chapter is about producing it predictably.

2. The One-Liner

DEFAULTS = {"sim": "xcelium", "timeout_s": 3600, "waves": False, "seeds": 1}
BLOCK = {"timeout_s": 7200}
TEST = {"waves": True, "seeds": 16}

effective = {**DEFAULTS, **BLOCK, **TEST}

for key in sorted(effective):
    print(f"  {key:<10} {effective[key]!r}")

  seeds      16
  sim        'xcelium'
  timeout_s  7200
  waves      True

That is the canonical annotation:

effective = {**defaults, **block_cfg, **test_cfg}

Later wins. timeout_s came from the block, waves and seeds from the test, sim from the defaults because nothing overrode it. Chapter 8.6's dict unpacking, and the left-to-right order in the expression is the precedence.

That property is worth stating as a rule, because it is the whole interface: the order of the layers in the expression is the order of increasing authority. A reader who knows that can predict any value without running anything.

3. The Merge Is Shallow

Here is the same expression on a config shaped the way a real one is:

DEFAULTS = {
    "sim": "xcelium",
    "timeout_s": 3600,
    "plusargs": {"UVM_VERBOSITY": "UVM_LOW", "UVM_MAX_QUIT_COUNT": "1"},
}

BLOCK = {
    "timeout_s": 7200,
    "plusargs": {"UVM_VERBOSITY": "UVM_MEDIUM"},
}

effective = {**DEFAULTS, **BLOCK}

print("defaults plusargs :", DEFAULTS["plusargs"])
print("block    plusargs :", BLOCK["plusargs"])
print("effective plusargs:", effective["plusargs"])
print()
lost = sorted(set(DEFAULTS["plusargs"]) - set(effective["plusargs"]))
print("silently dropped  :", lost)

defaults plusargs : {'UVM_VERBOSITY': 'UVM_LOW', 'UVM_MAX_QUIT_COUNT': '1'}
block    plusargs : {'UVM_VERBOSITY': 'UVM_MEDIUM'}
effective plusargs: {'UVM_VERBOSITY': 'UVM_MEDIUM'}

silently dropped  : ['UVM_MAX_QUIT_COUNT']

The block wanted one thing: raise the verbosity. What it got was the whole plusargs section replaced, and UVM_MAX_QUIT_COUNT — the plusarg that stops UVM after the first error instead of letting a broken test log for two hours — is gone.

** merges one level deep. A value in a later layer replaces the value in an earlier one, and a dict is a value like any other. Nothing raises, nothing warns, and the effective config is a perfectly plausible one that nobody wrote.

This is the single most common config bug in a flow, and it has a recognisable symptom: a setting works until somebody overrides a sibling of it. The memory block's tests start running for the full timeout after a change that only touched verbosity, and the change looks unrelated.

4. The Deep Merge

def deep_merge(base, override):
    """Layer `override` onto `base`, merging dicts instead of replacing them."""
    out = dict(base)
    for key, value in override.items():
        if isinstance(value, dict) and isinstance(out.get(key), dict):
            out[key] = deep_merge(out[key], value)
        else:
            out[key] = value
    return out


DEFAULTS = {
    "sim": "xcelium",
    "timeout_s": 3600,
    "plusargs": {"UVM_VERBOSITY": "UVM_LOW", "UVM_MAX_QUIT_COUNT": "1"},
}
BLOCK = {"timeout_s": 7200, "plusargs": {"UVM_VERBOSITY": "UVM_MEDIUM"}}
TEST = {"waves": True, "plusargs": {"UVM_TIMEOUT": "200us"}}

shallow = {**DEFAULTS, **BLOCK, **TEST}
deep = deep_merge(deep_merge(DEFAULTS, BLOCK), TEST)

print("shallow plusargs:", shallow["plusargs"])
print("deep    plusargs:", dict(sorted(deep["plusargs"].items())))
print()
print("shallow keys    :", sorted(shallow))
print("deep    keys    :", sorted(deep))

shallow plusargs: {'UVM_TIMEOUT': '200us'}
deep    plusargs: {'UVM_MAX_QUIT_COUNT': '1', 'UVM_TIMEOUT': '200us', 'UVM_VERBOSITY': 'UVM_MEDIUM'}

shallow keys    : ['plusargs', 'sim', 'timeout_s', 'waves']
deep    keys    : ['plusargs', 'sim', 'timeout_s', 'waves']

The shallow result has one plusarg — the last layer's — and the deep result has all three, with UVM_VERBOSITY carrying the block's value because the block layer came after the defaults.

Four details in deep_merge are deliberate.

out = dict(base) copies, so the function never mutates the layer it was given. A merge that modifies DEFAULTS corrupts the defaults for every later test in the same process — Chapter 5.8's aliasing, in a config loader.

isinstance(value, dict) and isinstance(out.get(key), dict) requires both sides to be dicts. If the defaults have a scalar where the override has a dict, the override replaces it, which is the only sensible reading.

out.get(key) rather than out[key], because the key may not be in the base at all.

The recursion handles nesting to any depth, which a config two levels deep will eventually have.

The copy is still shallow one level down, though — a list value is shared between the merged result and the layer it came from. For a config that is read and never mutated that is harmless, and it is worth knowing before somebody appends to effective["tests"].

5. Deep Is Not Automatically Better

The obvious conclusion from Section 3 is to always merge deeply. That trades one bug for its mirror image:

def deep_merge(base, override):
    out = dict(base)
    for key, value in override.items():
        if isinstance(value, dict) and isinstance(out.get(key), dict):
            out[key] = deep_merge(out[key], value)
        else:
            out[key] = value
    return out


DEFAULTS = {"plusargs": {"UVM_VERBOSITY": "UVM_LOW", "UVM_MAX_QUIT_COUNT": "1"}}

# "this test needs a completely clean plusarg set"
TEST = {"plusargs": {"UVM_VERBOSITY": "UVM_DEBUG"}}

print("deep   :", dict(sorted(deep_merge(DEFAULTS, TEST)["plusargs"].items())))
print("shallow:", {**DEFAULTS, **TEST}["plusargs"])

deep   : {'UVM_MAX_QUIT_COUNT': '1', 'UVM_VERBOSITY': 'UVM_DEBUG'}
shallow: {'UVM_VERBOSITY': 'UVM_DEBUG'}

With a deep merge, a later layer cannot remove anything. A test that genuinely needs a clean plusarg set — a soak test that must not quit after one error — has no way to say so. The defaults' UVM_MAX_QUIT_COUNT follows it everywhere.

So the choice is not which is correct, it is which question the section is answering:

Most configs want deep for plusargs-shaped maps of independent settings, and shallow for a section that is a single coherent unit — a coverage block where turning it on needs all four of its keys together.

Two ways to resolve it, and either is fine as long as it is the written-down one:

One rule for the whole config. Simplest to explain, and the sections that wanted the other behaviour have to be flattened — plusargs_UVM_VERBOSITY rather than a nested dict.

Per-section, declared. A set of keys that merge deeply, and everything else shallow:

DEEP_KEYS = {"plusargs", "env"}


def merge_layer(base, override):
    """Deep for the declared sections, shallow for everything else."""
    out = dict(base)
    for key, value in override.items():
        if key in DEEP_KEYS and isinstance(value, dict) and isinstance(out.get(key), dict):
            out[key] = {**out[key], **value}
        else:
            out[key] = value
    return out


DEFAULTS = {"plusargs": {"A": "1", "B": "2"}, "coverage": {"line": True, "toggle": True}}
TEST = {"plusargs": {"A": "9"}, "coverage": {"line": False}}

effective = merge_layer(DEFAULTS, TEST)
print("plusargs (deep)   :", dict(sorted(effective["plusargs"].items())))
print("coverage (shallow):", effective["coverage"])

plusargs (deep)   : {'A': '9', 'B': '2'}
coverage (shallow): {'line': False}

plusargs kept B; coverage was replaced wholesale, so toggle is gone — deliberately, because DEEP_KEYS says coverage is a unit.

DEEP_KEYS is the precedence rule written down, which is what the canonical purpose asks for. A reader can answer "what happens if I override one coverage key" by reading one set, instead of by running the flow.

6. Which Layer Set This?

The question asked every single time an effective config surprises somebody, and it costs one extra dict:

LAYERS = [
    ("defaults", {"sim": "xcelium", "timeout_s": 3600, "waves": False, "seeds": 1}),
    ("block", {"timeout_s": 7200}),
    ("test", {"waves": True, "seeds": 16}),
    ("command line", {"waves": False}),
]


def resolve(layers):
    """The effective config, and where each value came from."""
    effective = {}
    source = {}
    for name, layer in layers:
        for key, value in layer.items():
            effective[key] = value
            source[key] = name
    return effective, source


effective, source = resolve(LAYERS)

print(f"{'key':<12} {'value':<10} from")
for key in sorted(effective):
    print(f"{key:<12} {effective[key]!r:<10} {source[key]}")

key          value      from
seeds        16         test
sim          'xcelium'  defaults
timeout_s    7200       block
waves        False      command line

Four layers, and every value traceable to the one that set it. waves is False because somebody passed a flag tonight, which is a completely different situation from waves being False because nobody ever turned it on — and the effective config alone cannot tell you which.

This is cheap enough that there is no good reason not to have it. Print it when the runner starts, or behind a --show-config, and most config questions stop being questions.

Note that resolve builds the effective config with the same loop rather than with {**a, **b}. That is what makes the provenance possible — the one-liner has nowhere to record where a value came from.

7. What a None in a Later Layer Means

One ambiguity the merge cannot resolve for you:

DEFAULTS = {"waves": True, "wave_file": "dump.fsdb"}
TEST = {"wave_file": None}

print("merged:", {**DEFAULTS, **TEST})

merged: {'waves': True, 'wave_file': None}

Did that test mean do not write a wave file or use the default wave file? The merge produced None either way, and the two readings lead to different behaviour in the runner.

There is no right answer, only a chosen one. The three conventions in use:

None is a value. It means "explicitly nothing" and the runner must handle it. Simple and the most common.

None means remove the key. Then a layer can genuinely unset something, which Section 5 said a deep merge otherwise prevents:

def merge_with_removal(base, override):
    """A None in the override removes the key entirely."""
    out = dict(base)
    for key, value in override.items():
        if value is None:
            out.pop(key, None)
        else:
            out[key] = value
    return out


DEFAULTS = {"waves": True, "wave_file": "dump.fsdb"}
TEST = {"wave_file": None}

print("as a value :", {**DEFAULTS, **TEST})
print("as removal :", merge_with_removal(DEFAULTS, TEST))

as a value : {'waves': True, 'wave_file': None}
as removal : {'waves': True}

Neither — forbid it. The loader rejects a None in any layer and the config uses an explicit "none" string or a boolean for the absent case. The most verbose and the least surprising.

Whichever you pick, the thing that matters is that every consumer agrees, which is why this belongs next to DEEP_KEYS in the written-down rules rather than being decided independently by each reader. Chapter 21.7 can then check it.

8. The Resolver

Everything together, as the function a runner would call:

{
  "sim": "xcelium",
  "timeout_s": 3600,
  "waves": false,
  "plusargs": {"UVM_VERBOSITY": "UVM_LOW", "UVM_MAX_QUIT_COUNT": "1"}
}

{
  "timeout_s": 7200,
  "plusargs": {"UVM_VERBOSITY": "UVM_MEDIUM"}
}

{
  "axi_smoke": {"seeds": 4},
  "axi_soak": {"seeds": 1, "timeout_s": 28800, "plusargs": {"UVM_MAX_QUIT_COUNT": "0"}}
}

import json
from pathlib import Path

DEEP_KEYS = {"plusargs", "env"}


def merge_layer(base, override):
    out = dict(base)
    for key, value in override.items():
        if key in DEEP_KEYS and isinstance(value, dict) and isinstance(out.get(key), dict):
            out[key] = {**out[key], **value}
        else:
            out[key] = value
    return out


def resolve(test_name, cli=None):
    """The effective config for one test, and where each value came from."""
    layers = [
        ("defaults", json.loads(Path("defaults.json").read_text(encoding="utf-8"))),
        ("block", json.loads(Path("block.json").read_text(encoding="utf-8"))),
        ("test", json.loads(Path("tests.json").read_text(encoding="utf-8")).get(test_name, {})),
        ("command line", cli or {}),
    ]
    effective = {}
    source = {}
    for name, layer in layers:
        before = dict(effective)
        effective = merge_layer(effective, layer)
        for key in effective:
            if key not in before or effective[key] != before[key]:
                source[key] = name
    return effective, source


for test in ["axi_smoke", "axi_soak"]:
    effective, source = resolve(test, cli={"waves": True})
    print(f"--- {test}")
    for key in sorted(effective):
        value = effective[key]
        shown = dict(sorted(value.items())) if isinstance(value, dict) else value
        print(f"    {key:<10} {shown!r:<58} {source.get(key, '-')}")

--- axi_smoke
    plusargs   {'UVM_MAX_QUIT_COUNT': '1', 'UVM_VERBOSITY': 'UVM_MEDIUM'} block
    seeds      4                                                          test
    sim        'xcelium'                                                  defaults
    timeout_s  7200                                                       block
    waves      True                                                       command line
--- axi_soak
    plusargs   {'UVM_MAX_QUIT_COUNT': '0', 'UVM_VERBOSITY': 'UVM_MEDIUM'} test
    seeds      1                                                          test
    sim        'xcelium'                                                  defaults
    timeout_s  28800                                                      test
    waves      True                                                       command line

Two tests, one set of files, and every value explained.

axi_soak is the case the whole chapter is for. It needs a very long timeout and UVM_MAX_QUIT_COUNT of 0 so it does not stop at the first error — and because plusargs is a deep key, setting that one plusarg kept the block's verbosity instead of deleting it. With Section 3's shallow merge, UVM_VERBOSITY would have vanished and the soak log would be missing everything the methodology team expected to see.

The provenance loop compares against before so a key only re-attributes when its value actually changes — a layer that sets a key to the value it already had does not claim credit for it. That is a small distinction and it is the difference between a table that helps and one that says command line next to every row.

9. Testing the Precedence

Because the resolver takes layers and returns a dict, its tests need no files — Chapter 20.11 Section 4's property, and Chapter 19.18's sample list:

DEEP_KEYS = {"plusargs"}


def merge_layer(base, override):
    out = dict(base)
    for key, value in override.items():
        if key in DEEP_KEYS and isinstance(value, dict) and isinstance(out.get(key), dict):
            out[key] = {**out[key], **value}
        else:
            out[key] = value
    return out


def resolve(layers):
    effective = {}
    for _, layer in layers:
        effective = merge_layer(effective, layer)
    return effective


CASES = [
    ("later layer wins",
     [("a", {"timeout_s": 3600}), ("b", {"timeout_s": 7200})], "timeout_s", 7200),
    ("unset key falls through",
     [("a", {"sim": "xcelium"}), ("b", {"timeout_s": 1})], "sim", "xcelium"),
    ("deep key merges",
     [("a", {"plusargs": {"A": "1", "B": "2"}}), ("b", {"plusargs": {"A": "9"}})],
     "plusargs", {"A": "9", "B": "2"}),
    ("shallow key replaces",
     [("a", {"coverage": {"line": True, "toggle": True}}), ("b", {"coverage": {"line": False}})],
     "coverage", {"line": False}),
]

failures = 0
for name, layers, key, want in CASES:
    got = resolve(layers).get(key)
    ok = got == want
    failures += 0 if ok else 1
    print(f"{'ok  ' if ok else 'FAIL'} {name:<26} {key} = {got!r}")

print("failures:", failures)

ok   later layer wins           timeout_s = 7200
ok   unset key falls through    sim = 'xcelium'
ok   deep key merges            plusargs = {'A': '9', 'B': '2'}
ok   shallow key replaces       coverage = {'line': False}
failures: 0

Four cases, no files, and between them they pin down every rule this chapter stated. That is what "testable rather than implied" means in the canonical purpose — the precedence is not a paragraph in a wiki, it is four assertions that fail if somebody changes the merge.

The fourth case is the one to keep. It is the only thing standing between DEEP_KEYS and somebody "simplifying" the resolver into a universal deep merge, which would pass the first three.

10. Predict, Run, Observe

A = {"x": 1, "y": {"p": 1, "q": 2}}
B = {"y": {"p": 9}}
C = {"x": None}


def deep_merge(base, override):
    out = dict(base)
    for key, value in override.items():
        if isinstance(value, dict) and isinstance(out.get(key), dict):
            out[key] = deep_merge(out[key], value)
        else:
            out[key] = value
    return out


# A
print("A:", {**A, **B}["y"])

# B
print("B:", deep_merge(A, B)["y"])

# C
print("C:", {**A, **B, **C}["x"])

# D
print("D:", sorted({**A, **B, **C}))

# E
merged = deep_merge(A, B)
print("E:", A["y"]["p"], merged["y"]["p"])

# F
print("F:", {**{"a": 1}, **{}} == {"a": 1}, {**{}, **{"a": 1}} == {"a": 1})

# G
print("G:", deep_merge({"y": 5}, {"y": {"p": 1}})["y"])

The verified output:

A: {'p': 9}
B: {'p': 9, 'q': 2}
C: None
D: ['x', 'y']
E: 1 9
F: True True
G: {'p': 1}

A — the shallow merge replaced the nested dict and q is gone. Section 3.

B — the deep merge kept it.

C — a None in the last layer wins like any other value, and the merge cannot tell you whether that meant "unset".

D — merging never removes a key, whatever the values. Section 7's removal needs code.

E — the original A is unchanged, because deep_merge copies. A merge that mutated its input would have corrupted the defaults for every later call.

F — an empty layer is a no-op in either position, which is why cli or {} in Section 8 is safe.

G — a dict overriding a scalar replaces it, because the isinstance check requires both sides to be dicts.

11. Exercises

Exercise 1 — name your layers. List the config layers in your own flow, in precedence order, and say who owns each file.

Exercise 2 — the one-liner. Merge three layers with {**a, **b, **c} and confirm which layer each value came from by reading the expression.

Exercise 3 — the shallow trap. Build a config with a nested plusargs and override one plusarg in a later layer. Report which settings disappeared.

Exercise 4 — why it hides. Name a setting in your own flow that would be silently dropped by Exercise 3 and say how long it would take anyone to notice.

Exercise 5 — deep merge. Write deep_merge and confirm it keeps the sibling settings. Then show that it does not mutate the layer it was given.

Exercise 6 — the mirror bug. Using your deep merge, try to remove a default setting from a later layer. Say what a soak test would need instead.

Exercise 7 — declare the deep keys. Write DEEP_KEYS for your config and justify each entry in one phrase.

Exercise 8 — provenance. Build the effective config and a source map in the same loop. Print an aligned table.

Exercise 9 — a dead layer. Construct layers where one contributes nothing, and show that the provenance table reveals it.

Exercise 10 — the None question. Choose one of the three conventions for None and write the one sentence you would put in your config documentation.

Exercise 11 — removal. Implement merge_with_removal and show the same layers producing a different effective config under each convention.

Exercise 12 — four cases. Write Section 9's four tests for your own resolver, with no files. Then replace your resolver with a universal deep merge and confirm exactly one of them fails.

12. Summary

• A real flow has several config layers — project defaults, per-block, per-test, and the command line — and the runner wants one effective dict.
• {**defaults, **block, **test} resolves them, and later wins. The order in the expression is the order of increasing authority.
• The merge is shallow: a nested dict in a later layer replaces the earlier one entirely rather than merging into it.
• That silently deletes sibling settings — a block raising UVM_VERBOSITY drops UVM_MAX_QUIT_COUNT, and nothing raises.
• It hides because the dropped setting usually does nothing on a passing run, so it is found weeks later by somebody else.
• deep_merge recurses when both sides are dicts, copies rather than mutating, and uses .get() because the key may be new.
• Deep is not automatically right: with a deep merge a later layer cannot remove anything, so a test needing a clean section has no way to say so.
• Choose per section and write it down. DEEP_KEYS is the precedence rule as code.
• Deep for a map of independent settings; shallow for a section that is one coherent unit.
• Record where each value came from in the same loop as the merge — the one-liner has nowhere to put it.
• Attribute a key only when its value changes, or every row credits the last layer.
• Provenance also exposes a dead layer, which is how a config file that stopped mattering finally gets deleted.
• A None in a later layer is ambiguous — a value, a removal, or forbidden — and every consumer must agree on which.
• Test the precedence with layers in memory and no files. Four cases pin down every rule, and the shallow-key case is the one that stops somebody simplifying the resolver into a universal deep merge.

The last chapter of this module checks the config that comes out of this resolver, before a regression spends four hours discovering it was wrong.