The Script Design Model: Problem, Input, Processing, Decision, Output

Part II · Module 17 · Chapter 17.4 · The Script Design Model: Problem, Input, Processing, Decision, Output

1. The Engineering Problem

Ask an engineer what their script does and you usually get a description of its code. "It loops over the logs and greps for UVM_ERROR and prints a summary." That is an account of the implementation, and it is the wrong answer to the question — it describes how the thing works without ever saying what it is for, what it needs, or what it does when the inputs are not what it expected.

The symptom shows up later. A script described that way cannot be handed over, because the handover consists of reading the code. It cannot be tested, because nobody can state what the correct output is. And it cannot be changed safely, because the rule it applies and the data it reads are tangled in the same loop.

The fix is to be able to answer five questions about a script before writing it, and to keep answering them as it grows.

2. The Five Parts

# problem    : is this run clean?
# input      : one sim.log
# processing : count UVM_ERROR lines
# decision   : zero errors AND finished
# output     : verdict + exit status

Five lines, written in a comment block or a notebook or on paper, and the script that follows is largely transcription. Each line answers a different question:

• Problem — what question is this script answering for a human? Stated as a question, not a task.
• Input — exactly what it receives, and from where.
• Processing — the facts it extracts. Measurements, not conclusions.
• Decision — the rule that turns those facts into a verdict. Policy, not measurement.
• Output — what it emits, for whom, and in what form the next tool can read.

3. Worked on a Real Script

Take the most common script in any verification flow: decide whether one simulation passed.

Problem. Is this run clean? Note the phrasing. Not "parse the log" — that is a task, and tasks do not tell you when you are finished. A question does: you are finished when the question is answered.

Input. One sim.log, produced by the simulator, path supplied by the caller. Already this is more specific than most scripts manage, and the specificity pays: it rules out "a directory of logs" and "whatever is in the current directory", both of which are different scripts.

Processing. Count the lines containing UVM_ERROR. Find whether the end-of-run marker is present. These are facts about the file and two different people would extract identical values.

Decision. The run is clean when the error count is zero and the end marker is present. This is a choice. A different team might count UVM_WARNING too, or tolerate three known failures, or require a coverage threshold. The facts would be unchanged; the verdict would differ.

Output. A verdict a human reads, and an exit status a Makefile reads. Chapter 1.9 is entirely about the second half, which is the part scripts routinely omit.

Here is the whole thing, with the five parts marked:

LOG = [
    "Chronologic VCS simulator",
    "UVM_INFO @ 0: reporter [RNTST] Running test smoke_test",
    "UVM_ERROR drv.sv(88) @ 1420: bad handshake",
    "Simulation complete",
]

END_MARKER = "Simulation complete"


def extract_facts(lines):            # PROCESSING -- measurements only
    errors = 0
    finished = False
    for line in lines:
        if "UVM_ERROR" in line:
            errors += 1
        if END_MARKER in line:
            finished = True
    return errors, finished


def verdict(errors, finished):       # DECISION -- policy only
    if not finished:
        return "NO_FINISH"
    return "PASS" if errors == 0 else "FAIL"


errors, finished = extract_facts(LOG)
print("facts   :", errors, "errors, finished =", finished)
print("verdict :", verdict(errors, finished))

facts   : 1 errors, finished = True
verdict : FAIL

Two functions, and the line between them is the point.

4. Why Processing and Decision Are Separate

The version almost everybody writes first fuses them:

LOG = [
    "UVM_INFO @ 0: Running test smoke_test",
    "UVM_ERROR drv.sv(88) @ 1420: bad handshake",
    "Simulation complete",
]


def check(lines):
    for line in lines:
        if "UVM_ERROR" in line:
            print("FAIL")
            return
    print("PASS")


check(LOG)

FAIL

It produces the right answer, it is shorter, and it is a dead end. Four things it cannot do:

It cannot be tested. check returns nothing and prints. To test it you must capture standard output, which is the sort of thing that makes people decide testing is not worth it. The split version returns values, and a test is one assert.

It cannot report how bad. The loop returns at the first error, so the error count — the single most useful number in a triage summary — is thrown away. A reviewer asking "how many?" gets a second script.

Its policy cannot be changed without touching its parsing. Tolerating three known failures means editing the loop that reads the lines. In the split version it is one line of verdict, and extract_facts is untouched.

It never noticed the run did not finish. A truncated log contains no UVM_ERROR, so this function prints PASS. The split version has finished as a fact and a decision that uses it — and Chapter 1.8 is about enumerating exactly those cases.

5. Facts Are Not Verdicts

The cleanest way to hold the distinction: processing produces things two engineers would agree on; decision produces things they might argue about.

The right-hand column is the test. When you find yourself unable to say whether something is processing or decision, ask whether a reasonable colleague could want it to come out differently. If yes, it is policy, and policy belongs somewhere a reader can find it and change it in one place.

This matters beyond tidiness. Policy that is distributed through a parsing loop is policy nobody can audit, and "what exactly does our regression consider a failure?" becomes a question answered by reading four hundred lines.

6. The Model Is a Planning Tool, Not an Architecture

A caution, because five parts invites five functions and that is not the claim.

A forty-line script can answer all five questions and implement them in one function. The model does not say write five functions; it says have five answers. For a small script the answers live in a comment block at the top and the code below is twenty lines. For a regression summariser they become modules.

What is not optional is having the answers. A script whose author cannot state its decision rule in one sentence has a decision rule anyway — it is just scattered, undocumented and unchangeable.

The proportionality rule from Chapter 1.4 applies here too: the planning should cost minutes for a small script and an afternoon for something a team will depend on. Writing five lines of comment for a ten-line script is sensible. Drawing an architecture for it is not.

7. Where the Remaining Modules Fit

The model is also a map of what is coming, which is worth seeing now.

Input is Modules 15 and 18 — reading files, and the paths they live at. It is also Chapter 1.7, which is about saying precisely what the input is before any of that.

Processing is most of Part I: strings, lists, dictionaries, loops and functions, now in service of extracting facts.

Decision is Module 9's conditionals and Chapter 1.8's enumeration of failure states, which is the hardest part of the model and the one with its own chapter.

Output is Module 21's reports and Chapter 1.9's exit status — the line that makes a verdict visible to something other than a person reading a terminal.

Problem has no module, because it is not a programming skill. It is the thing you do before deciding any of the rest, and Chapter 1.6 turns it into seven questions.

8. A Second Example: a Script That Writes

The worked example above only reads, which makes it the easy case. Apply the model to a script that produces a file and a sixth consideration appears that the five parts do not name on their own.

The task: produce the morning summary of last night's regression.

Problem. What happened overnight, in a form somebody can read in thirty seconds? Note how different this is from the earlier problem despite using the same inputs. "Is this run clean" is a question about one run; this is a question about a set, and the difference decides almost everything else.

Input. The verdicts of four hundred runs, each already classified by the earlier script. Critically, this script's input is another script's output — which means the earlier script's output specification is now this one's input specification, and a change to one is a change to both.

Processing. Count by verdict. Find the slowest. Identify which tests are newly failing compared with yesterday, which requires yesterday's summary as a second input — and noticing that while writing the model rather than halfway through the code is exactly the saving the model exists for.

Decision. Which failures are worth putting at the top. A summary that lists four hundred results in submission order is not a summary; the decision is what deserves the reader's first ten seconds.

Output. A text file somebody reads, and — because this is a producer as well as a consumer — a machine-readable form for whatever comes next.

The sixth consideration is what a half-finished run leaves behind. The reading script could be killed at any moment and the worst outcome is that no answer appears. This one can be killed halfway through writing its summary, and the worst outcome is a file that looks like a summary, is readable, and is missing the bottom two hundred rows. A consumer cannot tell.

That is not a new part of the model so much as a question the Output part must answer whenever the output is a file: what does a reader see if I stop in the middle? The standard answer — write to a temporary name and rename into place when complete — belongs to Module 15, and the reason to raise it here is that it is a design decision, not an implementation detail. Deciding it after the script works means retrofitting it.

Two scripts, the same five questions, and the second one has an answer the first did not need.

9. Exercises

Reasoning, with one short program.

Exercise 1 — Five lines for a script you already have

Take a script you or your team runs regularly and write its five parts in five lines. Expect the decision line to be the hardest; if you cannot write it in one sentence, that is the finding.

Exercise 2 — Find the fusion

In the same script, find a place where a measurement and a judgment happen on the same line. Describe what you would have to change to tolerate one known failure.

Exercise 3 — Facts or policy

Classify each as processing or decision: the number of assertions fired; whether the run is a regression blocker; the wall-clock runtime; whether that runtime is too slow; the absence of a coverage database.

Exercise 4 — Split one

Take Section 4's fused check function and split it into extract_facts and verdict. Then add a count to the output without touching the parsing, and note how long it took.

Exercise 5 — The question, not the task

Rewrite these as questions rather than tasks: "parse the coverage report"; "clean up old run directories"; "check the filelist". Say what each question tells you about when the script is finished.

10. Related Tutorials

The model applied to real flow problems:

• Automating GLS Triage — a worked example of separating what the log says from what the team decides it means.
• First-Mismatch Triage — why the first failure is the fact worth extracting, and what a decision layer then does with it.

Where the output half leads:

• UVM Report Server — the mechanism producing the UVM_ERROR lines this chapter counts, and why their format is a contract.
• Verbosity Control — how the input to your parser changes when somebody changes a simulation switch.

11. Summary

• Five questions before any code: problem, input, processing, decision, output.
• State the problem as a question, not a task — a question tells you when you are finished.
• Processing produces facts; two engineers would extract identical values.
• Decision produces a verdict; a reasonable colleague could want it different.
• The test for which is which: could someone disagree with this statement?
• Fusing the two is the most common structural mistake, and it costs testability, detail, changeability and the ability to notice a truncated input.
• None of those four problems is fixable by writing the loop more carefully — they follow from a decision made before the loop existed.
• The model is a planning tool, not an architecture: five answers, not necessarily five functions.
• A script whose author cannot state its decision rule in one sentence has one anyway — scattered and unauditable.
• A script that writes must also answer what a half-finished run leaves behind — a design decision, not an implementation detail.
• When one script consumes another's output, the producer's output specification is the consumer's input specification, and changing one changes both.
• The five parts map the rest of the curriculum, and the decision part is the one with its own chapter.

The next chapter turns the first two parts into something you can actually do at a desk: seven questions, answered on paper, before the editor opens.