Looping Over Tests, Seeds and Log Lines

Part I · Module 10 · Chapter 10.2 · Looping Over Tests, Seeds and Log Lines

1. One Shape

Every loop in this chapter is this shape:

for <name> in <collection>:
    <body>

Three slots. The collection decides how many passes happen and what each one hands you. The name is bound to one item per pass. The body is what you want done to each.

The rest of the chapter changes only the first slot.

2. A List of Tests

The simplest collection is one you wrote down:

tests = ["axi_smoke", "axi_burst", "reset_seq"]

for test in tests:
    print("running", test)

print("submitted", len(tests), "tests")

running axi_smoke
running axi_burst
running reset_seq
submitted 3 tests

The body can do anything a block can, including making a decision — Module 9 and Module 10 compose with no new rules:

tests = ["axi_smoke", "axi_burst", "reset_seq"]

for test in tests:
    if test.startswith("axi_"):
        print(test, "-> AXI suite")
    else:
        print(test, "-> other")

axi_smoke -> AXI suite
axi_burst -> AXI suite
reset_seq -> other

One if per pass, evaluated fresh each time, because Chapter 9.7 established that a condition is evaluated when execution reaches it — and here execution reaches it three times.

3. A Sweep of Seeds

The second collection is one you do not write down, because it has a hundred entries and they follow a rule. range() generates them:

for seed in range(3):
    print("seed", seed)

seed 0
seed 1
seed 2

range(3) supplies 0, 1, 2 — three values, starting at zero and stopping before three. Given a starting seed, the sweep reads naturally:

base = 4210

for seed in range(base, base + 3):
    print("running axi_smoke with seed", seed)

running axi_smoke with seed 4210
running axi_smoke with seed 4211
running axi_smoke with seed 4212

4. Every Line of a Log

The third collection is the one that makes loops worth learning. A simulation log is a sequence of lines, and the loop over it looks exactly like the loop over three test names:

log = [
    "UVM_INFO  @0: reset asserted",
    "UVM_ERROR @120: crc mismatch on beat 3",
    "UVM_INFO  @200: burst complete",
    "UVM_ERROR @410: unexpected BRESP",
]

for line in log:
    if "UVM_ERROR" in line:
        print(line)

UVM_ERROR @120: crc mismatch on beat 3
UVM_ERROR @410: unexpected BRESP

Four lines here; the identical loop handles four million. Nothing in the code depends on the size, which is the property that makes this worth writing instead of reading the log by eye.

Because each line is a string, every string method from Module 4 is available per pass:

log = ["  UVM_ERROR @120: crc mismatch  ", "UVM_INFO @200: done"]

for line in log:
    clean = line.strip()
    if clean.startswith("UVM_ERROR"):
        print("severity:", clean.split()[0])
        print("message :", clean.split(":", 1)[1].strip())

severity: UVM_ERROR
message : crc mismatch

5. Building a Result as You Go

A loop that only prints is a loop whose answer you cannot use. More often you want to collect something:

tests = ["axi_smoke", "axi_burst", "reset_seq"]

axi_tests = []

for test in tests:
    if test.startswith("axi_"):
        axi_tests.append(test)

print(axi_tests)
print(len(axi_tests), "of", len(tests), "are AXI tests")

['axi_smoke', 'axi_burst']
2 of 3 are AXI tests

Three parts, and the order of them is the whole pattern: the empty list is created before the loop, .append() runs inside it, and the result is used after. Put the axi_tests = [] line inside the loop and it is emptied on every pass — Chapter 10.10 is built on exactly this, and Chapter 10.12 shows the bug it causes.

The same shape builds a job list for a sweep:

jobs = []

for seed in range(4210, 4213):
    jobs.append(("axi_smoke", seed))

print(jobs)
print("jobs:", len(jobs))

[('axi_smoke', 4210), ('axi_smoke', 4211), ('axi_smoke', 4212)]
jobs: 3

Each job is a tuple of test and seed — a fixed two-field record, which is what Chapter 6.1 said tuples are for. Running every test at every seed needs one loop inside another, and that is Chapter 10.11.

6. Looping Over What You Just Built

A collection built by one loop is an ordinary collection, so a second loop can walk it:

log = [
    "UVM_ERROR @120: crc mismatch",
    "UVM_INFO  @200: burst complete",
    "UVM_ERROR @410: unexpected BRESP",
]

errors = []

for line in log:
    if "UVM_ERROR" in line:
        errors.append(line)

print("found", len(errors), "errors")

for error in errors:
    print(" -", error.split(":", 1)[1].strip())

found 2 errors
 - crc mismatch
 - unexpected BRESP

Two loops, one after the other, each with its own job. That is usually clearer than one loop doing two things.

7. The Collection Is an Expression

The thing after in does not have to be a name. It is an expression, evaluated once, before the first pass:

for test in ["smoke", "reset"]:
    print(test)

print("---")

status = {"reset": "FAIL", "smoke": "PASS"}

for key in sorted(status):
    print(key, status[key])

smoke
reset
---
reset FAIL
smoke PASS

sorted(status) is evaluated first, producing a list, and the loop walks that list. This is the ordinary way to get a dependable order out of a dictionary or a set, and it is not special loop syntax — it is Chapter 5.5's sorted() in the slot where a collection goes.

That the expression is evaluated once matters:

tests = ["smoke", "reset"]

for test in tests:
    print(test)
    tests = ["replaced", "entirely"]

print("tests is now:", tests)

smoke
reset
tests is now: ['replaced', 'entirely']

Rebinding the name tests inside the loop did not redirect the loop. It had already taken hold of the original list before the first pass. (Changing the contents of the list being looped over is a different and more dangerous matter — Chapter 10.12.)

8. What Changed, and What Did Not

Three settings, side by side:

What is identical across all three: the for ... in ...: line, the body being a block, one pass per item, the loop variable being re-bound each pass, and the collection being left untouched.

The loop is not the interesting part of any of these programs. What differs between a suite runner and a log checker is the body — and that is made of Modules 1 to 9, which you already have.

9. Predict, Run, Observe

# A
for n in range(4):
    print(n)

# B
names = ["clk", "rst_n", "data"]
long_names = []
for name in names:
    if len(name) > 3:
        long_names.append(name)
print(long_names)

# C
log = ["INFO ok", "ERROR bad", "INFO fine"]
count = 0
for line in log:
    if line.startswith("ERROR"):
        count = count + 1
print("errors:", count)

# D
for word in "axi smoke pass".split():
    print(word.upper())

The verified output:

0
1
2
3
['rst_n', 'data']
errors: 1
AXI
SMOKE
PASS

A — range(4) supplies four values beginning at zero, so the last is 3.

B — "clk" has length 3, which is not greater than 3, so it is excluded. The boundary matters exactly as Chapter 9.1 said.

C — one line starts with ERROR; "ERROR" appearing elsewhere in a line would not have counted, because .startswith() anchors at the beginning.

D — .split() produces a list of three words and the loop walks it. The collection slot held a method call, evaluated once.

10. Exercises

Exercise 1 — three loops, one shape. Write three loops that each print one line per item: over the list ["smoke", "reset"], over range(2), and over "hi". Say in one sentence what is identical about all three.

Exercise 2 — filter a suite. Given tests = ["axi_smoke", "ahb_reset", "axi_burst", "apb_rw"], build a list of only the AXI tests and print both the list and how many there are.

Exercise 3 — a seed sweep. Print the ten seeds from 4210 to 4219 inclusive, one per line. State what you passed to range() and why the second argument is not 4219.

Exercise 4 — count severities. For the log list in Section 4, count the UVM_INFO lines and the UVM_ERROR lines separately and print both counts. Use two counters and one loop.

Exercise 5 — extract a field. Each line of ["axi_smoke 4210 PASS", "axi_burst 4211 FAIL"] has three space-separated fields. Loop over it and print just the test name and the status, skipping the seed.

Exercise 6 — build then walk. Build a list of the failing test names from Exercise 5's data in one loop, then print them in a second loop with a leading FAILED: . Say why two loops may be clearer than one.

Exercise 7 — the expression slot. Predict the output of for x in sorted([3, 1, 2]): and of for k in sorted({"b": 1, "a": 2}):, then verify. Explain in one sentence what the loop actually walked in each case.

Exercise 8 — evaluated once. Run Section 7's rebinding program. In two sentences, explain why the loop printed the original two items, and what tests refers to afterwards.

11. Summary

• There is one loop shape: for <name> in <collection>: followed by an indented body.
• A test list, a seed sweep and a log are the same loop — only the collection differs.
• range(a, b) generates the integers from a up to but not including b; Chapter 10.4 covers it properly.
• A log is a sequence of lines, and the loop over it does not depend on how many there are. Reading one from a file is Module 15; the loop is unchanged.
• Loops and conditionals compose with no new rules. An if inside a body is evaluated once per pass.
• To collect a result: create the empty container before the loop, append inside it, use it after.
• A collection built by one loop can be walked by the next.
• The thing after in is an expression, evaluated once before the first pass — so sorted(...), a literal, or a method call all belong there.
• Rebinding the collection's name inside the loop does not redirect the loop.

The next chapter asks what each kind of collection actually hands you on each pass, because for one of them the answer surprises people.