fork-join

Module 14 · Page 14.1

fork-join is the fundamental tool for running multiple threads in parallel and waiting for all of them to finish. It turns a sequential testbench into a concurrent one — cutting simulation time and enabling stimulus patterns that are simply impossible without parallelism. The trap is the loop-variable capture bug: silently produces wrong results in every parallel loop a junior writes. This page walks the basic semantics, the three join variants, the variable-scope rules, the automatic-task discipline, and the canonical patterns.

1. Engineering Problem — Why fork-join Exists

A real DUT has 4 AXI channels (AW / W / B / AR / R), 8 PCIe lanes, 16 DDR banks, and at least one debug interface — all alive at the same instant. A testbench that drives them sequentially does not exercise the design; it drives one channel, idles 3, drives the next, idles 3, and so on. Coverage stays at single digits; cross-channel arbitration is never exercised; protocol races between concurrent channels never fire.

Without fork, you have two options, both bad:

• One initial block per channel — works but cannot coordinate end-of-test, cannot share IPC, cannot launch dynamically based on the test scenario.
• An always block per channel — works for steady-state stimulus but cannot model finite-length scenarios, cannot return, cannot be disabled cleanly.

fork-join is the IEEE 1800 answer: spawn an arbitrary set of named threads from inside a procedural block, run them all at the same simulation time, and block the parent until every one of them finishes. The total wall-clock time of the parent becomes max(thread_durations), not their sum.

2. Mental Model — Barrier-Launch, Barrier-Wait

The picture every engineer carries:

fork is a barrier-launch: every statement inside it springs into a parallel thread at the same simulation instant. join is a barrier-wait: the parent thread suspends and waits for every spawned thread to finish before crossing the barrier and resuming.

Three invariants this picture preserves:

• Same start, different finish. All threads begin at the same simulation time. They run independently — different delays, different events, different external signals — and finish whenever their own work completes.
• Parent is paused, not killed. The parent suspends at join and consumes no scheduling work while waiting. When the last thread finishes, the parent resumes in the same simulation time the last thread completed in.
• Local variables are shared. Every thread sees the same enclosing-scope variables. This is the source of the loop-capture bug in §6 — the rule that catches every junior writing their first parallel loop.

3. Visual Explanation — Three Threads, One Barrier

The figure captures the canonical pattern. All three threads start at the same instant (the fork bar); each runs independently; the parent waits at the join bar until every thread crosses it. The JOIN variant is the strictest of the three — join_any (next page 14.2) and join_none (page 14.3) relax that wait in different ways but share the same fork-bar semantics.

4. Syntax & Semantics — fork, join, Named Blocks

4.1 The basic form

// ── Syntax ────────────────────────────────────────────────────
fork
    statement_or_block_1 ;
    statement_or_block_2 ;
    // ... any number of threads ...
join
// parent resumes HERE — after ALL threads are done
 
// ── Three concurrent tasks, parent waits for all ──────────────
$display("[MAIN] Launching 3 threads at t=%0t", $time);
fork
    begin : thread_a
        #20;
        $display("[A] done at t=%0t", $time);
    end
    begin : thread_b
        #30;
        $display("[B] done at t=%0t", $time);
    end
    begin : thread_c
        #10;
        $display("[C] done at t=%0t", $time);
    end
join
$display("[MAIN] All threads done at t=%0t", $time);
// Output:
//   [MAIN] Launching 3 threads at t=0
//   [C] done at t=10
//   [A] done at t=20
//   [B] done at t=30
//   [MAIN] All threads done at t=30   ← resumes after the LAST thread (B)

4.2 The three join variants

SystemVerilog has three forms — they differ only in when the parent resumes. The other two are covered in their own pages.

4.3 Single-statement vs multi-statement threads, and named blocks

// ── Single-statement threads: no begin-end needed ─────────────
fork
    drive_reset();          // task call — one thread
    init_memory();          // task call — another thread, starts at the same time
    #100 check_clocks();    // statement with delay — starts after 100 ns
join
// All three run in parallel; parent waits for all three.
 
// ── Multi-statement blocks ────────────────────────────────────
fork
    begin
        $display("[CH0] Start");
        drive_channel_0();
        $display("[CH0] Done");
    end
    begin
        $display("[CH1] Start");
        drive_channel_1();
        $display("[CH1] Done");
    end
join
 
// ── Named blocks — required for selective disable ─────────────
fork
    begin : blk_a
        @(posedge clk); drive_a();
    end
    begin : blk_b
        @(posedge clk); @(posedge clk); drive_b();
    end
join
// Named blocks (blk_a, blk_b) allow `disable blk_a` to kill just one thread

Name every thread you might want to disable, debug, or reference from wait fork — the names show up in simulator logs and in Verdi's thread browser, and they make disable targeted instead of nuclear.

5. Simulation View — How the Scheduler Handles fork-join

At the fork keyword, the simulator:

1. Spawns each statement inside the block as a separate process, sharing the parent's local variables.
2. Schedules all spawned threads at the current simulation time in the Active region. Their relative order in the Active region is not specified by IEEE 1800 — see the event regions lesson for the consequences.
3. Suspends the parent process at the join keyword. The parent consumes no scheduler work while suspended.
4. Resumes the parent in the Active region of the time slot in which the last thread completes. "Last" is by $time, not by source order.

A fork-join with no threads (fork join) is a legal no-op — the parent passes through without suspending. A fork-join with one thread blocks the parent for that thread's duration; semantically equivalent to the thread body inline, but with a separate scheduler frame.

disable fork issued from outside the parent's call tree terminates every thread spawned by the parent — including ones in nested forks. disable <named_block> terminates exactly the named thread.

6. Waveform — Parent Blocked Until the Last Thread

The waveform below shows the three-thread example from §4.1. Threads A, B, C all start at cycle 0. C finishes at cycle 10, A at cycle 20, B at cycle 30. The parent is blocked through the entire window — it consumes no scheduler work, fires no $display, drives no signals — until cycle 30, the instant B finishes. The parent then resumes in the same time slot.

7. Synthesis — Not Applicable

fork-join is a procedural simulation construct. It has no hardware footprint and no place in synthesisable RTL — synthesis tools reject it outright. This section is intentionally omitted; the topic does not warrant it.

The corresponding hardware idiom for "do these N things in parallel" is just an always @(*) or always_ff per concurrent block — the synthesisable equivalent is built into the RTL semantics, not a procedural primitive. fork-join is a testbench-only tool.

8. Verification View — Where fork-join Earns Its Keep

8.1 The loop-variable capture bug — the canonical first-fork failure

When you fork inside a loop, the spawned threads are scheduled to run concurrently — they do not necessarily execute immediately. The loop variable i is a static storage location by default. By the time the threads execute, the loop has incremented i to its final value; every thread reads that same final value.

for (int i = 0; i < 4; i++) begin
    fork
        begin
            #10;
            $display("Thread %0d done", i);
        end
    join_none
end
// Output:
//   Thread 4 done
//   Thread 4 done
//   Thread 4 done
//   Thread 4 done
// Every thread reads i=4 (the post-loop value).

for (int i = 0; i < 4; i++) begin
    automatic int idx = i;   // NEW per iteration
    fork
        begin
            #10;
            $display("Thread %0d done", idx);
        end
    join_none
end
// Output:
//   Thread 0 done
//   Thread 1 done
//   Thread 2 done
//   Thread 3 done

8.2 The automatic task discipline

When a forked thread calls a task, the task's local variables must be automatic if the same task may be called from multiple concurrent threads. Without automatic, all concurrent calls share the same storage — a race.

// ── WRONG: static task — all concurrent calls share local variables
task drive_channel(int ch);     // static by default in a module
    int count = 0;              // SHARED by ALL concurrent calls!
    repeat (10) begin
        @(posedge clk);
        count++;
        $display("ch=%0d count=%0d", ch, count);
    end
endtask
 
// ── CORRECT: automatic — each call has its own locals ─────────
task automatic drive_channel(int ch);
    int count = 0;              // PRIVATE to this call instance
    repeat (10) begin
        @(posedge clk);
        count++;
        $display("ch=%0d count=%0d", ch, count);
    end
endtask
 
initial fork
    drive_channel(0);   // ch=0, count is private
    drive_channel(1);   // ch=1, count is private
    drive_channel(2);   // ch=2, count is private
join

8.3 Nested fork-join for hierarchical parallelism

A fork-join block can appear inside another fork-join. Inner threads must all complete before the inner join releases control back to the outer thread that launched it.

fork
    begin : channel_a_init
        fork
            configure_phy_a();
            configure_mac_a();
        join                       // inner join: wait for both PHY and MAC
        enable_channel_a();        // runs only after both sub-tasks complete
    end
    begin : channel_b_init
        fork
            configure_phy_b();
            configure_mac_b();
        join
        enable_channel_b();
    end
join                               // outer join: wait for both channels
$display("[MAIN] Both channels initialised");

Both channels initialise in parallel. Within each channel, PHY and MAC configure simultaneously. enable_channel_* runs only after both sub-tasks of that channel finish. The outer parent waits until both channels are fully enabled.

8.4 Parallel multi-channel stimulus — the canonical pattern

parameter N_CH = 8;
 
initial begin
    for (int i = 0; i < N_CH; i++) begin
        automatic int ch = i;          // MUST be automatic
        fork
            drive_channel(ch);
        join_none                       // spawn without waiting
    end
 
    // ... other test setup ...
 
    wait fork;                         // wait for ALL spawned threads here
    $display("[TB] All channels done at t=%0t", $time);
end

fork ... join_none inside the loop spawns each channel-driving thread and returns immediately. wait fork later in the parent waits for every previously-spawned thread to finish — even threads that were join_none-ed earlier.

9. Industry Usage — Where fork-join Lands in Real Verification

• UVM phase machinery — uvm_test::run_phase() runs as a thread; each agent's run_phase() spawns its driver / monitor / sequencer threads via fork-join_none internally; the test waits on phase.raise_objection / drop_objection rather than join directly, but the underlying spawn mechanism is the same.
• UVM uvm_sequence::do_pre_post — pre / post / body fork patterns inside the sequencer use fork-join_any and fork-join_none heavily.
• Multi-protocol VIPs — every AXI / AHB / APB / DDR / PCIe / Ethernet agent spawns separate driver / monitor threads; the test's run_phase fork-joins the whole agent tree.
• Reset / clock-gating tests — the test's fork-join runs stimulus, reset injector, and gating sequencer in parallel; the test waits for all three to complete before scoring.
• Coverage-closure tests — multiple seeds run as parallel fork-join_none threads inside a single simulation when the verification environment supports it.
• Regression infrastructure — when one simulation runs N tests sequentially (e.g. checkpoint / restart flows), each test is a fork ... join block with a watchdog timeout in the parent.

10. Design Review Notes — What a Senior Will Flag

The single highest-value rule: automatic is not optional for fork-loops. Every shop's coding-style guide mandates automatic int idx = i; (or for (automatic int i = ...)) — and most ship a lint check for the violation.

11. Debugging Guide — Real Failures, Real Fixes

for (int i = 0; i < 4; i++) begin
    fork
        begin
            drive_channel(i);    // reads i when it runs, not when forked
        end
    join_none
end

task drive_channel(int ch);          // static by default!
    int count = 0;                   // SHARED by all concurrent calls
    repeat (10) begin
        @(posedge clk);
        count++;
        $display("ch=%0d count=%0d", ch, count);
    end
endtask
 
initial fork
    drive_channel(0);
    drive_channel(1);
    drive_channel(2);
join

initial begin
    for (int i = 0; i < 8; i++) begin
        automatic int ch = i;
        fork
            drive_channel(ch);       // spawned, not waited on
        join_none
    end
    #1000;                           // arbitrary
    $finish;                         // kills every still-running thread
end

initial begin
    fork
        drive_stimulus();
        wait_for_dut_done();        // never asserts if DUT is broken
        sample_coverage();
    join                            // blocks forever on wait_for_dut_done
end

Txn shared_txn = new();
fork
    begin
        shared_txn.randomize();
        drive_to_dut(shared_txn);     // reads shared_txn
    end
    begin
        shared_txn.randomize();       // ALSO writes shared_txn concurrently!
        push_to_scoreboard(shared_txn);
    end
join

12. Interview Insights — What Interviewers Actually Probe

13. Exercises

1. Design — N parallel channel drivers (Foundation)
Write a for loop over N_CHANNELS = 8 that spawns one drive_channel(ch) thread per channel. Use the correct automatic capture pattern. Add a wait fork; $finish; at the end of the parent to ensure no channel is killed mid-flight.

2. Debug — the channel-4-only bug (Intermediate)
A teammate's parallel-channel test only drives stimulus on channel 4 of 4 — channels 0–3 never see traffic. The code is for (int i = 0; i < 4; i++) fork drive(i); join_none end. Walk through the diagnostic, name the bug, and write the one-line fix.

3. Code review — the static task (Intermediate)
A teammate submits a task drive_protocol(int ch); with a local int seq_num = 0; and uses it as initial fork drive_protocol(0); drive_protocol(1); drive_protocol(2); join. Identify the bug, predict the symptom, and write the fix.

4. Trade-off — fork-join everywhere vs fork-join_none + wait fork (Advanced)
A junior teammate argues "always use fork-join — it's the safest because the parent blocks until everything finishes; you never lose work." Argue the case against the blanket rule. When does fork-join_none + wait fork legitimately beat the all-join style, and what specifically does it enable that pure join cannot?

14. Summary

fork spawns parallel threads from inside a procedural block; join blocks the parent until every spawned thread completes. The parent's wall-clock duration is the longest single thread, not the sum. The single failure mode every junior ships once is the loop-variable capture bug — fix with automatic int idx = i; before the fork. Tasks called from concurrent fork branches must be declared automatic. Always name every fork branch you might want to disable or debug.

Defaults to memorise. fork ... join waits for all. automatic is mandatory for fork-loops and for tasks called from forks. wait fork is the parent-side counterpart to fork ... join_none — without it, threads get killed mid-flight by $finish. Wrap the test-level fork-join in fork ... join_any with a watchdog to prevent silent hangs.

Next up: 14.2 — fork-join_any — the variant where the parent resumes as soon as any one thread completes. The basis for the canonical timeout pattern and the race-detect pattern.