Thinking Before You Write a Script

Part II · Module 17 · Chapter 17.5 · Thinking Before You Write a Script

1. Two Different Sets of Seven

Chapter 1.4 gave seven questions, and this chapter gives seven more. They are not the same seven and it is worth being explicit about the difference before going further.

Chapter 1.4's seven decide whether to build anything at all. Is the task repetitive, is the input well defined, can the output be checked, who maintains it. They are a filter, and the possible outcomes include do not write this script.

This chapter's seven are asked after that filter returns yes. They decide what exactly is being built. The outcome is never "do not build it" — it is a specification precise enough that writing the code becomes transcription.

One is a gate. The other is a design. Running the second set on a task that failed the first is how teams end up with a beautifully specified script that should not exist.

2. The Seven Questions

# written before any code:
# in   : one simulation log
# out  : PASS/FAIL plus an error count
# fail : log missing, log truncated, tool never ran

That comment block is the compressed form of seven answers:

1. What exactly comes in? Not "logs" — one file, a directory, a list of paths, a stream? Produced by what, and in whose format?
2. What exactly must come out? For a human, and separately for the next program.
3. What decision does it make? The rule, in one sentence, in the team's words.
4. What can go wrong with the input, and what should happen for each case?
5. What must it not destroy? Which files it writes, which it must leave alone, and what a half-finished run leaves behind.
6. Who or what consumes the result? A person reading a terminal, a Makefile, a CI job, another script.
7. How does the consumer learn the answer? Printed text, an exit status, a file, all three.

3. Working Them on a Real Task

The task: tell me whether last night's smoke test passed.

Q1 — what comes in. One file, sim.log, written by VCS, path given as the script's argument. Not a directory, not a glob, not "the latest one" — those are three different scripts and choosing now avoids writing two of them.

Q2 — what comes out. For a human: a verdict and an error count. For the next program: an exit status. Deciding this now is what stops the script from being rewritten the first time somebody puts it in a Makefile.

Q3 — the decision. The run passed if the simulator finished and reported no errors. One sentence, in the team's language, containing no code.

Q4 — what can go wrong. The log may be missing because the job never launched. It may exist and be empty because the tool died at startup. It may stop mid-line because the queue killed it at the timeout. It may be last week's log because the run directory was not cleaned.

Q5 — what it must not destroy. It reads only. It writes nothing, which means the whole category of half-written output does not apply — and knowing that is itself a design decision worth recording.

Q6 — who consumes it. This morning, an engineer at a terminal. By next month, a nightly wrapper that aggregates two hundred of these.

Q7 — how they learn the answer. The engineer reads the printed line. The wrapper reads the exit status, which it must because it is not going to parse English.

4. Question 4 Is the One That Gets Skipped

Six of the seven can be answered by imagining the script working. The fourth requires imagining it failing, and that is a different and less comfortable exercise — which is why in practice it is the one left blank.

It is also the question that decides whether you have written a script or a tool. Consider what a parser that skipped Q4 does with each of the four cases from Section 3:

TRUNCATED = [
    "Chronologic VCS simulator",
    "UVM_INFO @ 0: reporter [RNTST] Running test smoke_test",
    "UVM_INFO @ 1200: starting sequence",
]

EMPTY = []


def verdict_no_q4(lines):
    """Written by somebody who only imagined the happy path."""
    for line in lines:
        if "UVM_ERROR" in line:
            return "FAIL"
    return "PASS"


print("truncated log ->", verdict_no_q4(TRUNCATED))
print("empty log     ->", verdict_no_q4(EMPTY))

truncated log -> PASS
empty log     -> PASS

Both report PASS. A run that was killed halfway and a run that never started are both indistinguishable, to this function, from a clean pass — because the absence of an error line is the only thing it looks at, and absence is exactly what both failures produce.

Nothing about that function is badly written. The loop is correct, the names are fine, it would pass review from anybody reading it as code. The defect was introduced before the code existed, by not asking what the input could be.

Answering Q4 turns the same function into one that distinguishes its inputs:

TRUNCATED = [
    "Chronologic VCS simulator",
    "UVM_INFO @ 0: reporter [RNTST] Running test smoke_test",
]
CLEAN = ["UVM_INFO @ 0: Running test", "Simulation complete"]
FAILED = ["UVM_ERROR drv.sv(88) bad handshake", "Simulation complete"]
EMPTY = []

END_MARKER = "Simulation complete"


def verdict_with_q4(lines):
    if not lines:
        return "EMPTY_LOG"
    errors = 0
    finished = False
    for line in lines:
        if "UVM_ERROR" in line:
            errors += 1
        if END_MARKER in line:
            finished = True
    if not finished:
        return "NO_FINISH"
    return "PASS" if errors == 0 else "FAIL"


for name, log in [("clean", CLEAN), ("failed", FAILED),
                  ("truncated", TRUNCATED), ("empty", EMPTY)]:
    print(f"{name:<10} -> {verdict_with_q4(log)}")

clean      -> PASS
failed     -> FAIL
truncated  -> NO_FINISH
empty      -> EMPTY_LOG

Four inputs, four distinct answers. The difference between the two functions is six lines of code and one question asked ten minutes earlier.

5. The Answers Become the File

When all seven are answered, the comment block writes itself and the code follows it closely enough that most of the thinking is already done:

# problem : did last night's smoke test pass?
# in      : one sim.log, path from the caller
# out     : verdict + error count for a human; exit status for a wrapper
# decision: passed = finished AND zero errors
# fail    : missing -> NO_LOG | empty -> EMPTY_LOG | no marker -> NO_FINISH
# writes  : nothing

LOG = ["UVM_INFO Running test", "UVM_ERROR drv.sv(88)", "Simulation complete"]
END_MARKER = "Simulation complete"


def extract_facts(lines):
    errors = 0
    finished = False
    for line in lines:
        if "UVM_ERROR" in line:
            errors += 1
        if END_MARKER in line:
            finished = True
    return errors, finished


def verdict(errors, finished):
    if not finished:
        return "NO_FINISH"
    return "PASS" if errors == 0 else "FAIL"


errors, finished = extract_facts(LOG)
print(f"{verdict(errors, finished)} ({errors} errors)")

FAIL (1 errors)

Read the comment block and the code together. Every line of the block has a visible counterpart below it, and nothing in the code is doing something the block did not anticipate. That is what "the code becomes transcription" means — not that writing it is trivial, but that the decisions were all made before the typing started, so the typing is about expression rather than discovery.

The one thing the block promises and this program does not yet deliver is the exit status. That is Chapter 1.9, and the gap is deliberate: it is the single most commonly omitted line in flow scripting.

6. When Ten Minutes Is Too Long

The proportionality caution again, because this chapter is easy to over-apply.

For a throwaway — a five-line script you will run once this afternoon and delete — the seven questions are not worth asking, and pretending otherwise makes the advice easy to dismiss wholesale. Write the five lines.

The threshold is the same one Chapter 1.4 used: will this run when you are not watching? A script that only ever runs under your supervision, with inputs you just looked at, has no bad night. One that runs at 3am against whatever the queue produced does, and the ten minutes is cheap against a single wrong nightly verdict.

Between those, a reasonable rule: answer Q1, Q2 and Q4 for anything you will run more than twice. The other four can wait until somebody else depends on it.

7. Where the Answers Come From

The seven questions are easy to list and the honest difficulty is that several of them cannot be answered by thinking harder. They are answered by asking somebody, and knowing which somebody is part of the skill.

Question 1, about the input, is usually answered by the tool's owner. Whether a log is written incrementally or in one flush at the end, whether a header is always present, whether the format changed at the last upgrade — these are facts about a tool you did not write, and the person who configured it knows them. Ten minutes of their time is cheaper than a week of inferring the format from three examples, which is what most parsers are built on.

Question 4, about failure, is answered by history rather than imagination. The most productive version of this question is not "what could go wrong?" but "what went wrong last time?" Ask a colleague what the last three infrastructure incidents looked like and you will get four failure states that are specific to your flow, already prioritised by how often they happen. Imagination produces exotic cases; history produces the ones that recur.

Questions 6 and 7, about the consumer, are answered by the consumer. If a wrapper will call your script, its author knows what they need — probably an exit status, possibly a machine-readable file, almost certainly not your English summary. Guessing here is how a script acquires an output format nobody wanted and a missing one somebody needed.

Questions 2, 3 and 5 are genuinely yours. What must come out, what rule applies, and what must not be destroyed are design decisions, and the fact that they are the only three you can answer alone is worth noticing — it means a script written entirely in isolation has four unanswered questions in it by construction.

There is a practical consequence for how the ten minutes is spent. If three of the seven need somebody else, ask early, because the answers change the design rather than decorating it. Discovering on Thursday that the consumer needed a CSV is a different conversation from discovering it after the script is written in a shape that cannot produce one.

And when an answer genuinely is not available — the tool's owner has left, nobody remembers the format — that is itself an answer. It means the script must check rather than assume, which is a decision, and it belongs in the written specification alongside the others.

8. Exercises

Reasoning, with one short program.

Exercise 1 — Seven answers for a real task

Pick a script you are about to write or recently wrote. Answer all seven questions in one line each. Note which one took longest.

Exercise 2 — Answer Question 4 properly

For the same task, list every way the input can be wrong. Aim for at least four. For each, write what the script should do — not what it currently does.

Exercise 3 — Reproduce the false pass

Write a parser that returns PASS for a truncated log, as in Section 4. Then fix it, and count the lines the fix took.

Exercise 4 — Which questions does your script answer?

Take an inherited script from your flow and try to recover all seven answers by reading it. The ones you cannot recover are the ones its author did not ask.

Exercise 5 — Find the threshold

Name one script in your flow for which the seven questions would have been a waste of time, and one for which skipping them caused a problem. State what distinguishes them.

9. Related Tutorials

What the bad inputs actually look like:

• GLS in Regression — where truncated and missing logs come from in a real nightly, and why they are routine rather than exotic.
• First-Mismatch Triage — the shape of the evidence a parser should be extracting.

The output format you are parsing:

• UVM Report Server — what produces UVM_ERROR, and the fact that its format is configurable, which is a Question 1 problem.
• Reporting and Debug Techniques — how much a run chooses to print, and why that changes between projects.

10. Summary

• Chapter 1.4's seven questions are a gate; these seven are a design, asked only once the gate returns yes.
• The seven: what comes in, what must come out, what decision, what can fail, what must not be destroyed, who consumes it, how they learn the answer.
• Q1 deserves more precision than it usually gets — one file, a directory and a stream are three different scripts.
• Q4 is the one that gets skipped, because it requires imagining failure rather than success.
• Almost every tool failure produces less output, not different output, so a parser looking for bad news reports good news for the whole category.
• A parser that skips Q4 reports PASS for a truncated log and an empty one — and nothing about its code looks wrong.
• The fix is six lines of code and one question asked ten minutes earlier.
• When all seven are answered the comment block writes itself, and each line has a visible counterpart in the code.
• Transcription does not mean trivial — it means the decisions were made before the typing.
• Several questions cannot be answered by thinking — Q1 belongs to the tool's owner, Q4 to history, Q6 and Q7 to the consumer.
• Ask "what went wrong last time" rather than "what could go wrong"; imagination produces exotic cases and history produces recurring ones.
• Only Q2, Q3 and Q5 are yours alone, so a script written in isolation has four unanswered questions by construction.
• An unavailable answer is still an answer: it means the script must check rather than assume.
• The threshold is whether it runs unattended. A script you watch has no bad night; answer Q1, Q2 and Q4 for anything you run more than twice.

The next chapter takes the first two questions on their own and makes them precise, because "a log goes in and a verdict comes out" is not yet a specification.