RTL Design Patterns · Chapter 8 · Pipelining
Latency vs Throughput
Pipelining makes a block faster, but that word hides two different measurements engineers must never blur together. Latency is how many cycles pass from an input arriving to its own result coming out; a depth-N pipeline has N-cycle latency, and pipelining deeper makes latency worse. Throughput is how many results the block delivers per cycle, and a full pipeline delivers one every cycle no matter how deep, so pipelining keeps throughput high while raising the clock. That is the trade: for a long stream of independent work you win a result every cycle at a higher clock, but for one lonely or latency-critical operation the extra stages are pure added delay. Worse, inside a feedback loop the added latency makes the recurrence compute on stale data. This lesson separates the two metrics, quantifies fill and drain overhead, and measures both a depth-N pipeline and the feedback-loop trap in SystemVerilog, Verilog, and VHDL.
Intermediate12 min readRTL Design PatternsPipeliningLatencyThroughputFmaxFeedback Loop
Chapter 8 · Section 8.2 · Pipelining
1. The Engineering Problem
You inherit a datapath that computes a 32-bit result from a 32-bit input through a long chain of combinational logic — a multiply, then an add, then a saturate. Static timing says the whole chain takes 9 ns, so at the target 300 MHz (a 3.33 ns period) it fails timing badly. You already learned the fix in 8.1: cut the chain into three registered stages of roughly 3 ns each, and now every stage meets the 3.33 ns period. Fmax jumps, timing closes, and you write in the report that the block is now three times faster.
Then two things happen that the word faster did not prepare you for. First, the verification lead points out that a single computed result now appears three cycles after its input, not one — a control routine that reads this block's result and immediately acts on it is now three cycles slower to respond, and it got worse, not better. Second, a DSP colleague who streams a million samples through the block reports it is wonderful: once it warms up it produces one finished sample every cycle, at the new higher clock, so a million samples fly through in about a million cycles. Same block, same change — one engineer says it got slower, the other says it got faster. They are both right, because they are measuring two different things.
This is the confusion this section exists to kill. "Faster" is not one number. It is at least two: latency — the delay from an input to its own result — and throughput — how many results come out per unit time. Pipelining moves these two in opposite directions: it lengthens latency (more stages to walk through) while preserving throughput (still one result per cycle) at a higher clock. Whether that trade is a win or a loss depends entirely on the shape of your workload — a long independent stream, or a single latency-critical operation. And in one workload shape, a feedback loop, the extra latency is not just a cost to weigh; it silently breaks the math.
// The SAME depth-3 pipeline, measured two ways:
// LATENCY — cycles from an input to ITS result:
// input at cycle T -> its result at cycle T+3 => latency = 3 cycles
// (pipelining made this WORSE: it was 1 combinational eval before)
// THROUGHPUT — results delivered per cycle, once full:
// fed a new input every cycle -> one result out every cycle => 1 / cycle
// (pipelining KEPT this at 1/cycle, and did it at a HIGHER clock)
// A stream of 1000 items therefore takes 1000 + 3 cycles, NOT 1000 * 3.
// fill = 3 cycles to first result, then 1/cycle for the rest.2. Mental Model
3. Pattern Anatomy
The whole trade is captured by two numbers and one overhead term. Get these three straight and every pipelining decision becomes arithmetic.
Latency — in cycles, and in time. Latency is the number of clock cycles from an input entering the pipeline to its result emerging. For a straight depth-N pipeline it is exactly N cycles: the datum is captured by register 1 on the first clock, register 2 on the second, and appears at the output after N. Latency in time is latency_cycles * clock_period — and this is where the subtlety lives: pipelining shortens the clock period (raises Fmax) but multiplies the cycle count, so the latency in time can go up, down, or sideways depending on how the two move. A single op through a deeper pipeline usually gets worse in absolute time; only a stream benefits, because a stream cares about the exit rate, not one car's traversal.
Throughput — results per cycle when kept full. Throughput is results out per cycle. For a full depth-N pipeline it is 1 result per cycle, independent of N — this is the property that makes pipelining pay. In time, throughput is 1 / clock_period results per second, so a higher Fmax raises throughput even though the per-result exit rate in cycles is unchanged. This is the exact opposite of latency: latency scales with N, throughput does not.
Fill and drain — the one-time overhead. The pipeline is not full on cycle 1. The first result appears only after N cycles of fill (the latency), and after the last input, the tail takes N cycles of drain to flush the final results out. For a run of items fed one per cycle, total cycles = items + N - 1 (fill overlaps the stream). The headline number is that N is paid once: for 1000 items through a depth-4 pipeline the cost is about 1000 + 3 cycles, not 4000. The fill/drain overhead is negligible for a long stream and dominant for a short burst — a 4-item burst through a depth-4 pipeline is nearly half overhead.
Latency scales with depth; throughput does not — one input, one result per cycle once full
data flowWhen deeper pipelining HELPS vs HURTS. The trade resolves entirely on workload shape. It helps a throughput-bound stream — DSP filters, video/image pipelines, packet processing, crypto over a byte stream — where thousands of independent items flow and you care only about the one-per-cycle exit rate at the highest clock; the N-cycle fill is amortized to nothing. It hurts a latency-critical path — a tight control or feedback loop, a request-response turnaround, an interrupt-to-action path, or an occasional single op — where the answer is needed as fast as possible in absolute time and there is no stream to amortize the extra stages; there the added latency is pure loss. The single most dangerous case, deferred to §6 and §7, is a feedback/recurrence loop, where a result must feed the next computation: added latency there is not just wasted time, it makes the block read stale data and compute the wrong value — or forces it to accept a new input only once every N cycles, so effective throughput actually drops.
4. Real RTL Implementation
This is the core of the page. We build two things and show each in SystemVerilog, Verilog, and VHDL with an RTL block and a self-checking clocked testbench:
- (a) A parameterized depth-N pipeline whose testbench measures latency (cycles from a marked input to its result) and measures throughput (results per cycle when fed every cycle), and self-checks that latency
== DEPTHand throughput== 1/cycle. - (b) The pipelined-feedback-loop bug vs the feed-forward-only fix — a recurrence
acc = acc + x. The naive "pipelined" accumulator inserts a register inside the loop, so the adder reads a stale ACC and the running sum is wrong; the fix keeps the recurrence single-cycle (no latency inside the loop) so ACC is always current.
One discipline runs through every block: latency lives in the feed-forward path, never inside a recurrence. A depth-N feed-forward pipeline is free to be as deep as timing needs; the moment a register sits between an output and its own next input, that register's latency corrupts the loop.
4a. Depth-N pipeline — measure latency and throughput
A depth-N pipeline is just N registers in series carrying the data, plus a parallel N-deep valid shift so we know which output cycle corresponds to a real input. The valid bit walking the pipeline alongside the data is what lets a testbench measure both metrics: latency is the delay from asserting valid_in to seeing valid_out; throughput is how many valid_out cycles you get per input cycle.
module pipe_dn #(
parameter int WIDTH = 8, // datapath width
parameter int DEPTH = 4 // number of registered stages == LATENCY
)(
input logic clk,
input logic rst, // synchronous, active-high
input logic valid_in, // a real datum is present this cycle
input logic [WIDTH-1:0] data_in,
output logic valid_out,// the result this cycle is real
output logic [WIDTH-1:0] result // data_in delayed by DEPTH cycles
);
// DEPTH data registers and DEPTH valid registers, in series. The data
// just walks down the stages (stand-in for real per-stage logic); the
// valid bit walks alongside so we always know which output is real.
logic [WIDTH-1:0] stage_data [DEPTH];
logic stage_valid [DEPTH];
always_ff @(posedge clk) begin
if (rst) begin
for (int i = 0; i < DEPTH; i++) begin
stage_data[i] <= '0;
stage_valid[i] <= 1'b0;
end
end else begin
stage_data[0] <= data_in; // stage 0 captures the input
stage_valid[0] <= valid_in;
for (int i = 1; i < DEPTH; i++) begin
stage_data[i] <= stage_data[i-1]; // shift one stage/cycle
stage_valid[i] <= stage_valid[i-1];
end
end
end
// The last stage is the output: DEPTH cycles after the input -> LATENCY.
assign result = stage_data[DEPTH-1];
assign valid_out = stage_valid[DEPTH-1];
endmoduleThe SystemVerilog testbench measures latency by tagging one input cycle and counting clocks until valid_out rises, then measures throughput by feeding a valid input every cycle for a window and counting how many valid_out cycles come back. It asserts latency equals DEPTH and that a fully-fed window yields one result per input (throughput == 1/cycle).
module pipe_dn_tb;
localparam int WIDTH = 8;
localparam int DEPTH = 4;
logic clk = 0, rst = 1, valid_in = 0, valid_out;
logic [WIDTH-1:0] data_in = 0, result;
int errors = 0;
pipe_dn #(.WIDTH(WIDTH), .DEPTH(DEPTH)) dut (.*);
always #5 clk = ~clk; // 100 MHz
// ---- latency measurement ----
int launch_cyc, arrive_cyc, measured_latency;
// ---- throughput measurement ----
int fed = 0, produced = 0, cyc = 0;
always @(posedge clk) if (!rst) cyc++; // global cycle counter
initial begin
repeat (2) @(posedge clk); rst = 0; // release reset
// MEASURE LATENCY: launch ONE tagged input, count clocks to its result.
@(negedge clk); valid_in = 1; data_in = 8'hA5; launch_cyc = cyc + 1;
@(negedge clk); valid_in = 0; data_in = 0;
// wait for the single result to emerge
do @(posedge clk); while (!valid_out);
arrive_cyc = cyc;
measured_latency = arrive_cyc - launch_cyc;
assert (measured_latency == DEPTH)
else begin $error("latency=%0d exp DEPTH=%0d", measured_latency, DEPTH); errors++; end
assert (result == 8'hA5)
else begin $error("result=%h exp A5", result); errors++; end
// drain any tail before the throughput window
repeat (DEPTH+2) @(posedge clk);
// MEASURE THROUGHPUT: feed a valid input EVERY cycle for 100 cycles,
// then count valid_out over an equal steady window. Full pipe => 1/cycle.
fed = 0; produced = 0;
fork
begin // feeder: one input per cycle
repeat (100) begin
@(negedge clk); valid_in = 1; data_in = fed[7:0]; fed++;
end
@(negedge clk); valid_in = 0;
end
begin // counter: count results after the pipe has filled
repeat (DEPTH) @(posedge clk); // skip the fill
repeat (100) @(posedge clk) if (valid_out) produced++;
end
join
// In steady state every fed cycle yields exactly one result: 1/cycle.
assert (produced == 100)
else begin $error("throughput: produced=%0d/100 (bubbles?)", produced); errors++; end
if (errors == 0)
$display("PASS: latency==DEPTH(%0d) and throughput==1/cycle", DEPTH);
else
$display("FAIL: %0d mismatches", errors);
$finish;
end
endmoduleThe Verilog form is the same pipeline with reg arrays and always @(posedge clk), and a testbench that measures the same two numbers using plain counters and if (...) $display("FAIL") self-checks. Note the throughput window counts valid_out after DEPTH fill cycles — the fill is paid once and skipped, exactly as the anatomy predicts.
module pipe_dn #(
parameter WIDTH = 8,
parameter DEPTH = 4
)(
input wire clk,
input wire rst,
input wire valid_in,
input wire [WIDTH-1:0] data_in,
output wire valid_out,
output wire [WIDTH-1:0] result
);
reg [WIDTH-1:0] stage_data [0:DEPTH-1];
reg stage_valid [0:DEPTH-1];
integer i;
always @(posedge clk) begin
if (rst) begin
for (i = 0; i < DEPTH; i = i + 1) begin
stage_data[i] <= {WIDTH{1'b0}};
stage_valid[i] <= 1'b0;
end
end else begin
stage_data[0] <= data_in; // capture input
stage_valid[0] <= valid_in;
for (i = 1; i < DEPTH; i = i + 1) begin
stage_data[i] <= stage_data[i-1]; // one stage per cycle
stage_valid[i] <= stage_valid[i-1];
end
end
end
assign result = stage_data[DEPTH-1]; // DEPTH-cycle latency
assign valid_out = stage_valid[DEPTH-1];
endmodule module pipe_dn_tb;
parameter WIDTH = 8;
parameter DEPTH = 4;
reg clk, rst, valid_in;
reg [WIDTH-1:0] data_in;
wire valid_out;
wire [WIDTH-1:0] result;
integer cyc, launch_cyc, measured_latency;
integer fed, produced, k, errors;
pipe_dn #(.WIDTH(WIDTH), .DEPTH(DEPTH)) dut
(.clk(clk), .rst(rst), .valid_in(valid_in), .data_in(data_in),
.valid_out(valid_out), .result(result));
initial clk = 0;
always #5 clk = ~clk;
always @(posedge clk) if (!rst) cyc = cyc + 1; // cycle counter
initial begin
errors = 0; cyc = 0; valid_in = 0; data_in = 0; rst = 1;
@(posedge clk); @(posedge clk); rst = 0;
// MEASURE LATENCY: one tagged input, count clocks to valid_out.
@(negedge clk); valid_in = 1; data_in = 8'hA5; launch_cyc = cyc + 1;
@(negedge clk); valid_in = 0; data_in = 0;
@(posedge clk);
while (!valid_out) @(posedge clk);
measured_latency = cyc - launch_cyc;
if (measured_latency !== DEPTH)
begin $display("FAIL: latency=%0d exp=%0d", measured_latency, DEPTH); errors = errors + 1; end
if (result !== 8'hA5)
begin $display("FAIL: result=%h exp=A5", result); errors = errors + 1; end
repeat (DEPTH+2) @(posedge clk); // drain the tail
// MEASURE THROUGHPUT: feed every cycle, count results after fill.
fed = 0; produced = 0;
for (k = 0; k < 100 + DEPTH; k = k + 1) begin
@(negedge clk);
if (fed < 100) begin valid_in = 1; data_in = fed[7:0]; fed = fed + 1; end
else valid_in = 0;
@(posedge clk);
if (k >= DEPTH && k < 100 + DEPTH && valid_out) produced = produced + 1;
end
if (produced !== 100)
begin $display("FAIL: throughput produced=%0d/100 (bubbles)", produced); errors = errors + 1; end
if (errors == 0)
$display("PASS: latency==DEPTH(%0d), throughput==1/cycle", DEPTH);
else
$display("FAIL: %0d mismatches", errors);
$finish;
end
endmoduleIn VHDL the pipeline is an array of vectors plus a valid vector, shifted in a clocked process; numeric_std handles the counting in the testbench. The self-check uses assert ... severity error, and the throughput count skips the DEPTH fill cycles just like the other two.
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
entity pipe_dn is
generic (
WIDTH : positive := 8;
DEPTH : positive := 4 -- stages == LATENCY
);
port (
clk : in std_logic;
rst : in std_logic; -- sync, active-high
valid_in : in std_logic;
data_in : in std_logic_vector(WIDTH-1 downto 0);
valid_out : out std_logic;
result : out std_logic_vector(WIDTH-1 downto 0)
);
end entity;
architecture rtl of pipe_dn is
type slv_array is array (natural range <>) of std_logic_vector(WIDTH-1 downto 0);
signal stage_data : slv_array(0 to DEPTH-1);
signal stage_valid : std_logic_vector(0 to DEPTH-1);
begin
process (clk)
begin
if rising_edge(clk) then
if rst = '1' then
stage_data <= (others => (others => '0'));
stage_valid <= (others => '0');
else
stage_data(0) <= data_in; -- capture input
stage_valid(0) <= valid_in;
for i in 1 to DEPTH-1 loop
stage_data(i) <= stage_data(i-1); -- one stage per cycle
stage_valid(i) <= stage_valid(i-1);
end loop;
end if;
end if;
end process;
result <= stage_data(DEPTH-1); -- DEPTH-cycle latency
valid_out <= stage_valid(DEPTH-1);
end architecture; library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
entity pipe_dn_tb is
end entity;
architecture sim of pipe_dn_tb is
constant WIDTH : positive := 8;
constant DEPTH : positive := 4;
constant PERIOD : time := 10 ns;
signal clk : std_logic := '0';
signal rst : std_logic := '1';
signal valid_in : std_logic := '0';
signal data_in : std_logic_vector(WIDTH-1 downto 0) := (others => '0');
signal valid_out : std_logic;
signal result : std_logic_vector(WIDTH-1 downto 0);
begin
dut : entity work.pipe_dn
generic map (WIDTH => WIDTH, DEPTH => DEPTH)
port map (clk => clk, rst => rst, valid_in => valid_in, data_in => data_in,
valid_out => valid_out, result => result);
clk <= not clk after PERIOD/2;
stim : process
variable measured_latency : integer := 0;
variable produced : integer := 0;
begin
wait until rising_edge(clk); wait until rising_edge(clk);
rst <= '0';
-- MEASURE LATENCY: launch one tagged input, count clocks to valid_out.
wait until falling_edge(clk);
valid_in <= '1'; data_in <= x"A5";
wait until falling_edge(clk);
valid_in <= '0'; data_in <= (others => '0');
-- count rising edges until the result appears
measured_latency := 0;
loop
wait until rising_edge(clk);
measured_latency := measured_latency + 1;
exit when valid_out = '1';
end loop;
assert measured_latency = DEPTH
report "latency /= DEPTH" severity error;
assert result = x"A5"
report "result /= A5" severity error;
for d in 0 to DEPTH+1 loop wait until rising_edge(clk); end loop; -- drain
-- MEASURE THROUGHPUT: feed every cycle, count results after DEPTH fill.
produced := 0;
for k in 0 to 100 + DEPTH - 1 loop
wait until falling_edge(clk);
if k < 100 then valid_in <= '1'; data_in <= std_logic_vector(to_unsigned(k mod 256, WIDTH));
else valid_in <= '0'; end if;
wait until rising_edge(clk);
if k >= DEPTH and valid_out = '1' then produced := produced + 1; end if;
end loop;
assert produced = 100
report "throughput /= 1/cycle (bubbles present)" severity error;
report "pipe_dn self-check complete: latency=DEPTH, throughput=1/cycle" severity note;
wait;
end process;
end architecture;4b. Pipelined feedback loop — BUGGY vs feed-forward-only FIX
Now the trap. The recurrence is acc = acc + x: each cycle add the new input to the running sum. The adder has a long carry chain, so someone "pipelines" it to raise Fmax by inserting a register inside the loop, between the adder output and the ACC that feeds the adder's next input. That register adds one cycle of latency inside the recurrence — so the adder now adds x to an ACC that is one cycle stale, and the running sum is wrong. The fix is to accept that a recurrence has no feed-forward slack to pipeline: keep the loop single-cycle so ACC is always current. (If Fmax truly demands it, you restructure the recurrence — unroll, or split the carry — but you never simply drop a latency register into the loop.)
// BUGGY: a register sits BETWEEN the adder and the ACC it feeds. The adder
// reads a ONE-CYCLE-STALE acc, so the running sum is wrong. It looks
// 'higher Fmax' but the recurrence is broken.
module acc_bad #(parameter int WIDTH = 16) (
input logic clk, rst, valid_in,
input logic [WIDTH-1:0] x,
output logic [WIDTH-1:0] acc
);
logic [WIDTH-1:0] sum_q; // extra pipeline reg INSIDE the loop
always_ff @(posedge clk) begin
if (rst) begin acc <= '0; sum_q <= '0; end
else if (valid_in) begin
sum_q <= acc + x; // stage 1: add current acc
acc <= sum_q; // stage 2: acc lags by one cycle => STALE feedback
end
end
endmodule
// FIXED: the recurrence is single-cycle. acc is updated directly from acc + x,
// so the feedback is ALWAYS current. No latency inside the loop.
module acc_good #(parameter int WIDTH = 16) (
input logic clk, rst, valid_in,
input logic [WIDTH-1:0] x,
output logic [WIDTH-1:0] acc
);
always_ff @(posedge clk) begin
if (rst) acc <= '0;
else if (valid_in) acc <= acc + x; // current acc feeds itself, one cycle
end
endmoduleThe testbench feeds a known stream every cycle and compares acc against a software running sum. The FIXED module tracks the reference exactly; the BUGGY module diverges because its feedback is stale — the assertion fires and prints FAIL. Bind to acc_good to PASS.
module acc_bug_tb;
localparam int WIDTH = 16;
logic clk = 0, rst = 1, valid_in = 0;
logic [WIDTH-1:0] x, acc;
int errors = 0, ref_sum = 0, i;
// Bind to acc_good to PASS; to acc_bad to watch the stale feedback diverge.
acc_good #(.WIDTH(WIDTH)) dut (.clk(clk), .rst(rst), .valid_in(valid_in), .x(x), .acc(acc));
always #5 clk = ~clk;
initial begin
repeat (2) @(posedge clk); rst = 0;
for (i = 1; i <= 8; i++) begin
@(negedge clk); valid_in = 1; x = i[WIDTH-1:0]; // stream 1,2,...,8
ref_sum += i; // software reference
@(posedge clk); // acc updates this edge
#1;
// In a correct single-cycle loop acc == running sum every cycle.
assert (acc === ref_sum[WIDTH-1:0])
else begin $error("cycle %0d: acc=%0d exp=%0d (stale feedback?)", i, acc, ref_sum); errors++; end
end
@(negedge clk); valid_in = 0;
if (errors == 0) $display("PASS: acc tracks running sum (loop has NO added latency)");
else $display("FAIL: %0d mismatches (pipelined inside the feedback loop)", errors);
$finish;
end
endmoduleThe Verilog pair is the same recurrence: the BUGGY module carries an extra sum_q register inside the loop so ACC lags one cycle; the FIXED module updates acc directly. The testbench streams a known sequence and flags any cycle where acc does not equal the software running sum.
// BUGGY: sum_q inside the loop makes acc lag one cycle => stale feedback.
module acc_bad #(parameter WIDTH = 16) (
input wire clk, rst, valid_in,
input wire [WIDTH-1:0] x,
output reg [WIDTH-1:0] acc
);
reg [WIDTH-1:0] sum_q;
always @(posedge clk) begin
if (rst) begin acc <= {WIDTH{1'b0}}; sum_q <= {WIDTH{1'b0}}; end
else if (valid_in) begin
sum_q <= acc + x; // adds STALE acc
acc <= sum_q; // acc one cycle behind
end
end
endmodule
// FIXED: single-cycle recurrence, acc always current.
module acc_good #(parameter WIDTH = 16) (
input wire clk, rst, valid_in,
input wire [WIDTH-1:0] x,
output reg [WIDTH-1:0] acc
);
always @(posedge clk) begin
if (rst) acc <= {WIDTH{1'b0}};
else if (valid_in) acc <= acc + x; // no latency inside the loop
end
endmodule module acc_bug_tb;
parameter WIDTH = 16;
reg clk, rst, valid_in;
reg [WIDTH-1:0] x;
wire [WIDTH-1:0] acc;
integer i, ref_sum, errors;
// Bind to acc_good to PASS; to acc_bad to observe the stale-feedback divergence.
acc_good #(.WIDTH(WIDTH)) dut (.clk(clk), .rst(rst), .valid_in(valid_in), .x(x), .acc(acc));
initial clk = 0;
always #5 clk = ~clk;
initial begin
errors = 0; ref_sum = 0; valid_in = 0; x = 0; rst = 1;
@(posedge clk); @(posedge clk); rst = 0;
for (i = 1; i <= 8; i = i + 1) begin
@(negedge clk); valid_in = 1; x = i[WIDTH-1:0]; // stream 1..8
ref_sum = ref_sum + i; // reference sum
@(posedge clk);
#1;
if (acc !== ref_sum[WIDTH-1:0]) begin
$display("FAIL: cycle %0d acc=%0d exp=%0d (stale feedback)", i, acc, ref_sum);
errors = errors + 1;
end
end
@(negedge clk); valid_in = 0;
if (errors == 0) $display("PASS: acc tracks running sum (no latency in loop)");
else $display("FAIL: %0d mismatches (pipelined feedback loop)", errors);
$finish;
end
endmoduleThe VHDL pair mirrors it with numeric_std arithmetic on unsigned. The BUGGY architecture carries an intermediate sum_q register in the loop (stale ACC); the FIXED architecture updates acc in one cycle. The self-check is assert acc = ref severity error.
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
-- BUGGY: sum_q inside the loop => acc lags one cycle => STALE feedback.
entity acc_bad is
generic ( WIDTH : positive := 16 );
port ( clk, rst, valid_in : in std_logic;
x : in std_logic_vector(WIDTH-1 downto 0);
acc : out std_logic_vector(WIDTH-1 downto 0) );
end entity;
architecture rtl of acc_bad is
signal acc_q, sum_q : unsigned(WIDTH-1 downto 0) := (others => '0');
begin
process (clk)
begin
if rising_edge(clk) then
if rst = '1' then acc_q <= (others => '0'); sum_q <= (others => '0');
elsif valid_in = '1' then
sum_q <= acc_q + unsigned(x); -- adds stale acc_q
acc_q <= sum_q; -- acc one cycle behind
end if;
end if;
end process;
acc <= std_logic_vector(acc_q);
end architecture;
-- FIXED: single-cycle recurrence, acc always current.
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
entity acc_good is
generic ( WIDTH : positive := 16 );
port ( clk, rst, valid_in : in std_logic;
x : in std_logic_vector(WIDTH-1 downto 0);
acc : out std_logic_vector(WIDTH-1 downto 0) );
end entity;
architecture rtl of acc_good is
signal acc_q : unsigned(WIDTH-1 downto 0) := (others => '0');
begin
process (clk)
begin
if rising_edge(clk) then
if rst = '1' then acc_q <= (others => '0');
elsif valid_in = '1' then acc_q <= acc_q + unsigned(x); -- no loop latency
end if;
end if;
end process;
acc <= std_logic_vector(acc_q);
end architecture; library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
entity acc_bug_tb is
end entity;
architecture sim of acc_bug_tb is
constant WIDTH : positive := 16;
constant PERIOD : time := 10 ns;
signal clk : std_logic := '0';
signal rst : std_logic := '1';
signal valid_in : std_logic := '0';
signal x : std_logic_vector(WIDTH-1 downto 0) := (others => '0');
signal acc : std_logic_vector(WIDTH-1 downto 0);
begin
-- Bind to acc_good to PASS; to acc_bad to observe the stale-feedback divergence.
dut : entity work.acc_good
generic map (WIDTH => WIDTH)
port map (clk => clk, rst => rst, valid_in => valid_in, x => x, acc => acc);
clk <= not clk after PERIOD/2;
stim : process
variable ref_sum : integer := 0;
begin
wait until rising_edge(clk); wait until rising_edge(clk);
rst <= '0';
for i in 1 to 8 loop
wait until falling_edge(clk);
valid_in <= '1';
x <= std_logic_vector(to_unsigned(i, WIDTH)); -- stream 1..8
ref_sum := ref_sum + i; -- reference
wait until rising_edge(clk);
wait for 1 ns;
assert to_integer(unsigned(acc)) = ref_sum
report "acc /= running sum (stale feedback in loop)" severity error;
end loop;
wait until falling_edge(clk); valid_in <= '0';
report "acc self-check complete: acc tracks running sum" severity note;
wait;
end process;
end architecture;Across all three languages the lesson is one sentence: depth is free in a feed-forward pipeline and poison in a recurrence — latency you happily pay on a stream corrupts a loop that must read its own last result.
5. Verification Strategy
Latency and throughput are measured, not asserted by inspection — the testbench must instrument the block and read the two numbers off the waveform. The single invariant that governs a correctly pipelined block is:
A depth-N pipeline delivers each input's result exactly N cycles later (latency == N) and, when fed a valid input every cycle, one result per cycle (throughput == 1/cycle); a bubble is the only thing that lowers throughput, and a recurrence must show NO added latency inside its loop.
Everything below is that invariant made checkable, and it maps directly onto the testbenches in §4.
- Measure latency from a marked input. Tag exactly one input cycle (assert
valid_in, note the cycle), then count clocks untilvalid_outrises; assert the count equalsDEPTH. This is what all threepipe_dntestbenches do — SVassert (measured_latency == DEPTH), Verilogif (measured_latency !== DEPTH) $display("FAIL"), VHDLassert measured_latency = DEPTH severity error. Also check the value arrives intact (result == A5), so you are measuring the right datum's latency, not a neighbour's. - Measure throughput over a full-stream window. After skipping the
DEPTHfill cycles, feed a valid input every cycle for a fixed window (100 cycles here) and countvalid_outcycles; assert the count equals the window (one result per input). A count below the window means bubbles crept in. This confirms throughput is exactly 1/cycle when the pipe is kept full. - Confirm a single op pays the full latency. The single-input latency test is this check: one lonely operation costs the whole
DEPTHcycles with no stream to amortize it — the concrete demonstration of why a latency-critical single op is exactly the workload pipelining hurts. - Confirm bubbles drop throughput. As a directed case, deliberately gap the feed (drop
valid_infor a few cycles) and confirmproduced < windowby exactly the number of gapped cycles — proving throughput is 1/cycle only when kept full, and that a bubble costs precisely one result. This closes the "assumed throughput without keeping the pipe full" mistake of §6. - The recurrence check — acc must equal the running sum every cycle. For the feedback block, stream a known sequence and compare
accagainst a software running sum on every cycle (not just at the end). The FIXED single-cycle loop matches every cycle; the BUGGY pipelined-loop version diverges the moment its stale feedback is read — the assertion catches it immediately. This is the check that distinguishes "higher Fmax" from "correct," and it is the whole point: a block can pass a feed-forward latency/throughput test and still be wrong in a loop. - Parameter-generic verification (DEPTH = 1, 2, 4, 8; WIDTH varied). A parameterized pipeline must be verified generic. Re-run the latency measurement for several
DEPTHvalues and confirm the measured latency tracksDEPTHeach time (latency = 1 at DEPTH = 1, = 8 at DEPTH = 8), and that throughput stays 1/cycle regardless of depth — the direct demonstration that latency scales with depth while throughput does not.
6. Common Mistakes
Quoting Fmax/throughput as 'faster' while latency got worse. The classic conflation, and the reason this section exists. A deeper pipeline is higher-throughput and higher-Fmax but higher-latency — those are different metrics moving in opposite directions. Reporting a deep-pipelined block as simply 3x faster is wrong for anyone who cares about the response time of a single operation: their latency in absolute time may have gone up. Always report both numbers, and always name which one the consumer of the block actually needs — a stream cares about throughput, a control loop cares about latency.
Pipelining a latency-critical path. Adding stages to a path whose real metric is response time (a feedback loop, a request-response turnaround, an interrupt-to-action path, an occasional single op) makes the metric that matters worse while improving one nobody in that context cares about. Higher throughput is useless to a control loop that runs one operation and waits for its answer; the extra stages are pure added delay. Pipeline throughput-bound streams; leave latency-critical paths shallow (or restructure them, not pipeline them).
Assuming throughput without keeping the pipeline full (bubbles). Throughput is 1/cycle only when a valid input arrives every cycle. Every stall, every gap in the data, every cycle you fail to feed is a bubble — an empty stage that walks through and produces no result. Quoting the ideal 1/cycle for a block that is starved half the time overstates real throughput by 2x. Real throughput is valid results / total cycles, and it is below the ideal by exactly the bubble count — which is why the next chapters build a valid bit and back-pressure to keep the pipe full.
Ignoring fill/drain for short bursts. The N-cycle fill (and N-cycle drain) is negligible for a long stream but dominant for a short burst. A depth-8 pipeline processing a 4-item burst spends more cycles filling and draining than doing useful work — a 4-item burst costs about 4 + 8 cycles for 4 results, so the effective per-result cost is ~3 cycles, not 1. For bursty or short-run workloads, a deep pipeline can be slower end-to-end than a shallow one; size the depth to the run length, not just to Fmax.
Dropping a register inside a feedback loop to hit timing. The most dangerous mistake and the subject of §7. Adding a pipeline register inside a recurrence (between an output and its own next input) buys Fmax but makes the loop read N-cycle-stale data, so the recurrence computes the wrong value — or, if you also throttle the input to once every N cycles to keep the math right, effective throughput actually drops to 1/N. A recurrence has no feed-forward slack to pipeline; deepen the feed-forward path around it, or restructure/unroll the recurrence, but never simply insert latency into the loop.
7. DebugLab
The accumulator that got a higher Fmax and the wrong answer
The engineering lesson: latency and throughput are different metrics, and pipelining buys throughput and Fmax at the cost of latency — a bargain on a stream and a disaster inside a feedback loop. A pipeline register in a feed-forward path is invisible to correctness (the stream amortizes it) but a register inside a recurrence delays the block's own result back to itself, so the loop reads stale data and computes the wrong value — or you throttle the input to compensate and effective throughput collapses. The tell is history-dependent wrongness: a block that passes a feed-forward test and fails only when its previous output feeds its next input. Two habits make it impossible, and they hold in SystemVerilog, Verilog, and VHDL alike: never insert latency inside a recurrence — pipeline the feed-forward path only, and restructure (unroll / split) a loop that fails timing, and choose the metric your workload actually needs before you deepen anything — throughput for a stream, latency for a loop or a single op.
8. Interview Q&A
9. Exercises
Design and reasoning exercises — no syntax drills. Each rehearses the "which metric does this workload actually need?" habit.
Exercise 1 — Name the metric for the workload
For each workload, state whether latency or throughput is the metric that matters, and whether deepening the pipeline helps or hurts, in one line each: (a) a 512-tap FIR filter streaming a continuous audio sample stream; (b) a branch-target computation whose result the fetch unit needs this cycle to decide the next PC; (c) a block-cipher core encrypting a multi-gigabyte file; (d) an accumulator computing acc = acc + x over an input stream. (Hint: stream vs single-shot vs feedback loop.)
Exercise 2 — Do the fill/drain arithmetic
A depth-6 pipeline runs at 400 MHz (2.5 ns period). (i) What is the latency of one operation, in cycles and in nanoseconds? (ii) How many cycles does a stream of 1000 independent items take, fed one per cycle? (iii) How many cycles does a burst of 4 items take, and what fraction of that is fill/drain overhead? (iv) The un-pipelined version was one combinational evaluation at 120 MHz — for a single operation, which is faster in absolute time, and by how much? Explain what your four answers say about when this pipeline is worth it.
Exercise 3 — Diagnose the stale accumulator
An engineer pipelines the adder inside acc = acc + x to raise Fmax, inserting one register between the adder output and ACC. (i) Streaming x = 1,2,3,4,5,... one per cycle, write out what the correct running sum is versus what the stale-feedback version produces for the first five cycles. (ii) Explain precisely why a feed-forward smoke test can pass while this is broken. (iii) Give the two legitimate ways to hit timing without breaking the recurrence, and state the cost each introduces (throughput, area, or control). (iv) Write the one assertion you would add to the testbench to catch the bug on the first diverging cycle.
Exercise 4 — Bubbles and real throughput
A depth-4 pipeline processes 100 items, but the upstream source stalls (drops valid_in) for 10 scattered cycles during the run. (i) How many bubble cycles are there, and what is the real throughput in results per cycle over the run? (ii) By what factor does that undershoot the ideal 1/cycle? (iii) Name the two mechanisms a real pipeline uses to keep itself full and thus keep real throughput near the ideal, and say which upcoming chapter each belongs to. (Hint: a bit that walks with the data, and a handshake that lets the consumer say 'wait'.)
10. Related Tutorials
Continue in RTL Design Patterns (forward links unlock as they ship):
- Why Pipeline? — Chapter 8.1; the section that raised Fmax by cutting a long path into stages — the trade this page names and splits into its two metrics.
- The Valid Bit — Chapter 8.3; the bit that walks with the data so the output knows which cycles are real — exactly the mechanism this page uses to measure throughput and count bubbles.
- Pipeline Stalls and Flush — Chapter 8; where bubbles come from and how a stall holds the pipe — the throughput-below-1/cycle case in full.
- Pipeline Hazards — Chapter 8; the data/control hazards that force stalls and the feedback dependencies that make added latency dangerous.
- A Pipelined Datapath — Chapter 8; a full multi-stage datapath where latency, throughput, and fill/drain are all live at once.
- Valid-Ready Handshake — Chapter 8; the back-pressure protocol that keeps a pipeline full so real throughput stays near the 1/cycle ideal.
Backward / method and building blocks:
- The Register Pattern (D-FF) — Chapter 2.1; the single register that is a pipeline stage's atom — one stage of latency, the unit this whole trade is counted in.
- Adders and Subtractors — Chapter 5; the carry-chain path whose length motivates pipelining, and the recurrence (
acc = acc + x) whose loop this page shows must stay single-cycle. - Up/Down Counters — Chapter 3; a counter is a recurrence (
count = count + 1), the simplest feedback loop and the same reason you cannot pipeline latency into it. - Synchronous FIFO Architecture — Chapter 6; the buffer that absorbs bursts and rate mismatches so a pipeline can be kept full and throughput held at the ideal.
- FSM + Datapath — Chapter 4; the control loop whose response time is latency-critical — the workload shape where pipelining hurts.
Verilog v1 prerequisites (the grammar this pattern transcribes into):
- Arithmetic Operators — the
+carry chain whose delay motivates pipelining and forms the recurrence in §4b. - Blocking and Non-Blocking Assignments — why every pipeline stage register uses non-blocking (
<=) so all stages shift on the same edge. - If-Else Statements — the reset / valid control around each clocked stage in §4.
- Physical Data Types (wire, reg) — the
regstage registers andwireoutputs that carry the data down the pipeline.
11. Summary
- Latency and throughput are different metrics. Latency is the cycles from an input to its own result — for a depth-N pipeline it equals N. Throughput is results per cycle — for a full pipeline it is 1/cycle regardless of N. They live on different axes: latency scales with depth, throughput does not.
- Pipelining trades one for the other. It raises latency (more stages to walk) and Fmax while holding throughput at 1/cycle. "Faster" is never one number — always report both, and name which one the block's consumer actually needs.
- A stream wins; a single op loses. A stream of
itemscostsitems + N - 1cycles — the N-cycle fill is paid once and amortized (1000 items through a depth-4 pipe is ~1003 cycles, not 4000), so throughput-bound work (DSP, packets, crypto) loves depth. One lonely, latency-critical operation pays the full N-cycle traversal with nothing to amortize it, so it loses. - Bubbles and fill/drain are the fine print. Throughput is 1/cycle only when kept full; every bubble (a starved cycle) costs exactly one result, so real throughput is
valid results / total cycles. The N-cycle fill/drain is negligible for a long stream and dominant for a short burst. - Latency is poison inside a feedback loop. A register in a feed-forward path is free (the stream hides it); a register inside a recurrence (
acc = acc + x) delays the block's result back to itself, so the loop reads N-cycle-stale data and computes the wrong value — or you throttle the input and throughput collapses. Keep recurrences single-cycle; restructure (unroll / split), never pipeline, a loop that fails timing. Verify by measuring latency (== DEPTH) and throughput (== 1/cycle when full) and assertingacc == running sumevery cycle — self-checking clocked testbenches shown here in SystemVerilog, Verilog, and VHDL.