FPGA Design Challenges

Chapter 1.4 made the case for a common interface at the level of an ecosystem. This chapter makes it on a single board.

An FPGA SoC is the cheapest place in engineering to experience on-chip integration honestly. A soft CPU, some block RAM and half a dozen peripherals is a week of work, the tools tell you the truth about area and timing, and every problem in this chapter shows up at a scale one person can hold in their head. It is also where most engineers meet a bus for the first time, usually by building something that is not quite one.

The question this chapter answers: why does connecting six blocks that each work perfectly turn out to be harder than writing any of them?

1. The System

A realistic small FPGA SoC — the shape of a LiteX or OpenRISC build, or of any RISC-V soft-core project on a mid-range part.

The map, extending Chapter 1.2's and again illustrative rather than prescribed by anything:

Read the latency column, because it is the load-bearing one. Five targets, four different latencies, spanning zero cycles to tens. Any design in which the initiator must know the target's latency has just acquired five special cases and a maintenance hazard — this is Chapter 1.3's timer adapter, now five times over.

2. RTL 1 — The Decoder, and the Properties It Must Have

Address decode is the fabric's first job. It is also where the most damaging FPGA SoC bugs live, because a decoder can be wrong in ways that produce no error anywhere.

// ─────────────────────────────────────────────────────────────────────────
// Address decoder for the system above.
//
// Emits a ONE-HOT target select. Exactly one bit is high for every address
// in the space — including addresses no peripheral owns, which select the
// default target. That totality is what makes "unmapped" a defined outcome
// rather than a hang, and it is the property the assertions below check.
//
// Parameterised on the region size so the comparison width is derived once
// rather than hand-written per target.
// ─────────────────────────────────────────────────────────────────────────
module soc_decoder #(
  parameter int unsigned AW         = 32,
  // 4 KiB peripheral windows → the low 12 bits are the target's own offset.
  parameter int unsigned PERIPH_LSB = 12,
  // 64 KiB RAM window → the low 16 bits belong to the RAM.
  parameter int unsigned RAM_LSB    = 16
) (
  input  logic [AW-1:0] addr,
  output logic [5:0]    target_sel,   // {default, spi, timer, uart, gpio, ram}
  output logic          unmapped
);
  // Index constants make the one-hot vector readable at every use site and
  // keep the bit order in one place.
  localparam int unsigned SEL_RAM   = 0;
  localparam int unsigned SEL_GPIO  = 1;
  localparam int unsigned SEL_UART  = 2;
  localparam int unsigned SEL_TIMER = 3;
  localparam int unsigned SEL_SPI   = 4;
  localparam int unsigned SEL_DFLT  = 5;

  // Region tags: the address bits ABOVE each region's offset field.
  localparam logic [AW-RAM_LSB-1:0]    TAG_RAM   = (AW-RAM_LSB)'('h2000);
  localparam logic [AW-PERIPH_LSB-1:0] TAG_GPIO  = (AW-PERIPH_LSB)'('h40000);
  localparam logic [AW-PERIPH_LSB-1:0] TAG_UART  = (AW-PERIPH_LSB)'('h40001);
  localparam logic [AW-PERIPH_LSB-1:0] TAG_TIMER = (AW-PERIPH_LSB)'('h40002);
  localparam logic [AW-PERIPH_LSB-1:0] TAG_SPI   = (AW-PERIPH_LSB)'('h40003);

  logic hit_ram, hit_gpio, hit_uart, hit_timer, hit_spi;

  // Each hit compares the FULL tag — every address bit above the offset
  // field participates. A decoder that compares fewer bits still "works"
  // and silently aliases the target across the space; Section 6 is what
  // that costs.
  assign hit_ram   = (addr[AW-1:RAM_LSB]    == TAG_RAM);
  assign hit_gpio  = (addr[AW-1:PERIPH_LSB] == TAG_GPIO);
  assign hit_uart  = (addr[AW-1:PERIPH_LSB] == TAG_UART);
  assign hit_timer = (addr[AW-1:PERIPH_LSB] == TAG_TIMER);
  assign hit_spi   = (addr[AW-1:PERIPH_LSB] == TAG_SPI);

  assign unmapped = ~(hit_ram | hit_gpio | hit_uart | hit_timer | hit_spi);

  // The default target is selected by the ABSENCE of every other hit, which
  // is what makes the vector total: there is no address for which
  // target_sel is zero.
  always_comb begin
    target_sel              = '0;
    target_sel[SEL_RAM]     = hit_ram;
    target_sel[SEL_GPIO]    = hit_gpio;
    target_sel[SEL_UART]    = hit_uart;
    target_sel[SEL_TIMER]   = hit_timer;
    target_sel[SEL_SPI]     = hit_spi;
    target_sel[SEL_DFLT]    = unmapped;
  end
endmodule

What this module teaches

The engineering problem. Turn a 32-bit address into exactly one target selection, for every address, with no gaps and no overlaps. "Every address" is the hard part: the map in Section 1 leaves the overwhelming majority of the space unclaimed, and a decoder that simply produces no select for those addresses has created the hang Chapter 1.2 warned about.

Why the regions are compared as tags rather than as ranges. A range comparison — greater-than-or-equal to base and less-than base-plus-size — needs two magnitude comparators per target, and a magnitude comparator is a carry chain. An equality comparison on the address bits above the offset field is a wide XNOR-and-reduce, which an FPGA implements in a couple of LUT levels. This is why peripheral regions are power-of-two sized and naturally aligned: it converts arithmetic into an equality test, and that conversion is worth far more than the address space it wastes.

The combinational structure. Five independent equality comparisons, one OR-reduction for unmapped, and an assembly into a 6-bit vector. All parallel, no priority. The always_comb block assigns target_sel = '0 first so every bit has a definite value on every path — the habit that prevents inferred latches, even though this particular block assigns all six bits unconditionally afterwards.

No sequential behaviour at all. The decoder is pure combinational logic, deliberately. Registering the select would add a cycle to every access and put the decoder and the multiplexer in different cycles, which complicates the completion path for no benefit at this size. In a larger fabric that decision reverses, and Module 12 is where it gets made properly.

What it does not do. No byte enables, no protection checks, no support for regions of differing size within one comparison group, and no error injection. It also assumes the peripheral windows are all 4 KiB — which is why PERIPH_LSB is a parameter, and also why mixing window sizes inside one decoder is more than a parameter change.

How it could fail. Three ways, in increasing order of nastiness: a wrong tag constant sends accesses to the wrong target; two tags that are equal make two targets answer at once; and a comparison that omits high address bits aliases a target across the space. The first fails loudly. The other two are Section 3's subject.

3. Verification — The Two Properties That Matter

A decoder is unusually well suited to formal-style assertion, because its correctness is a property of a single cycle with no state. Two properties cover the failures that matter.

// ─────────────────────────────────────────────────────────────────────────
// Decoder properties. These check the ARCHITECTURAL claims this chapter
// makes about address decoding — not any bus protocol, which does not exist
// yet in this curriculum. Bind this to soc_decoder in simulation or hand it
// to a formal tool, where the address is unconstrained and the proof is
// exhaustive over all 2^32 addresses rather than the ones a test happened
// to generate.
// ─────────────────────────────────────────────────────────────────────────
module soc_decoder_checker #(
  parameter int unsigned AW = 32
) (
  input logic          clk,
  input logic          rst_n,
  input logic [AW-1:0] addr,
  input logic [5:0]    target_sel,
  input logic          unmapped
);
  default disable iff (!rst_n);

  // P1 — MUTUAL EXCLUSION plus TOTALITY, in one operator.
  //
  // $onehot0 permits zero or one bit, which is NOT what this design wants:
  // a zero vector is the hang. $onehot requires exactly one, which is the
  // real requirement, and it is only achievable because the default target
  // claims everything the others do not.
  property p_exactly_one_target;
    @(posedge clk) $onehot(target_sel);
  endproperty
  a_exactly_one_target : assert property (p_exactly_one_target)
    else $error("target_sel = %b — zero targets hangs, two corrupts", target_sel);

  // P2 — the unmapped flag and the default select agree. Two signals that
  //      encode the same fact must never disagree, or debug becomes a
  //      question of which one to believe.
  property p_unmapped_consistent;
    @(posedge clk) unmapped == target_sel[5];
  endproperty
  a_unmapped_consistent : assert property (p_unmapped_consistent)
    else $error("unmapped=%b but default select=%b", unmapped, target_sel[5]);

  // P3 — decode is a pure function of the address. If the address did not
  //      change, the selection must not either. Catches a decoder that has
  //      accidentally acquired state, which is exactly what happens when
  //      somebody "optimises" it with a registered intermediate signal.
  property p_stable_decode;
    @(posedge clk) $stable(addr) |-> $stable(target_sel);
  endproperty
  a_stable_decode : assert property (p_stable_decode)
    else $error("target_sel changed while addr was stable");
endmodule

Why $onehot and not $onehot0. This distinction is worth the paragraph. $onehot0 asserts at most one bit is set, which catches overlapping regions and permits the all-zero vector. $onehot asserts exactly one, which additionally catches the gap.

The all-zero vector is not a benign state. It means no target was selected, so nothing will respond, so the initiator waits for a completion that never arrives, and a stray pointer in software takes the whole system down. A decoder that is $onehot0-correct but not $onehot-correct has the more dangerous of the two bugs, and the only reason $onehot is achievable here is that the default target was designed in from the start.

If your fabric has no default target, $onehot0 is the strongest property available — and that is a signal about the fabric, not about the assertion.

Why P3 exists. It looks redundant against a combinational block, and it is, today. It stops being redundant the first time somebody registers an intermediate signal to close timing — which Section 5 is about, and which is exactly the kind of change that gets made without re-reading the decoder's contract.

What is not asserted, and why. Nothing about response, completion or data, because those are protocol facts and this curriculum has not defined a protocol. Nothing about arbitration. Both belong to later modules, and asserting them here would be checking rules the reader has not been given.

4. RTL 2 — The Read Multiplexer, Where the Cost Actually Lives

The decoder is cheap. The return path is not, and it is the structure that grows with every target you add.

// ─────────────────────────────────────────────────────────────────────────
// Response merge: select one target's read data and completion using the
// one-hot vector from the decoder.
//
// Written as an AND-OR reduction rather than a case statement, because the
// one-hot encoding is already available and this form maps directly onto
// FPGA LUTs without the synthesiser having to re-derive mutual exclusion.
// The comment matters: this structure is only correct BECAUSE of assertion
// P1. If two selects were ever high, the OR would merge two targets' data
// into a value belonging to neither, silently.
// ─────────────────────────────────────────────────────────────────────────
module soc_response_mux #(
  parameter int unsigned DW      = 32,
  parameter int unsigned NTARGET = 6
) (
  input  logic [NTARGET-1:0]      target_sel,
  input  logic [NTARGET-1:0][DW-1:0] t_rdata,
  input  logic [NTARGET-1:0]      t_done,
  output logic [DW-1:0]           rdata,
  output logic                    done
);
  always_comb begin
    rdata = '0;
    done  = 1'b0;
    for (int unsigned i = 0; i < NTARGET; i++) begin
      rdata |= {DW{target_sel[i]}} & t_rdata[i];
      done  |= target_sel[i] & t_done[i];
    end
  end
endmodule

What this costs, and why it is the scaling problem

The structure. For each of DW bits, an NTARGET-input OR of NTARGET two-input ANDs. On an FPGA with 6-input LUTs, six targets fit in roughly one LUT level per data bit; a seventh forces a second level for every one of the 32 bits.

That is the whole scaling story in one sentence: the decoder grows by one comparator per target, which is nothing, and the response path grows by one input on DW parallel multiplexers, which eventually is not.

The LUT counts are approximations and depend on the device family and the synthesiser — they are here to show the shape, not to be quoted. The shape is what matters: cost is flat until the LUT input count is exceeded, then steps.

Where the critical path goes. The path that closes last in a small SoC fabric is almost never inside a peripheral. It is: initiator address register → decoder comparators → one-hot select → response multiplexer → initiator read-data register. Every target added lengthens the middle of that path, and the peripherals themselves are untouched. This is the specific sense in which integration is harder than the blocks.

Two consequences worth carrying.

Registering the response breaks the path and costs a cycle. That trade is only payable if the initiator can tolerate a variable-latency response — which is precisely what a completion handshake provides, and precisely what the ad-hoc interfaces of Chapter 1.3 did not.

A hierarchical fabric is the other answer. Split the targets into groups, decode to a group first, then within it. The path becomes two small multiplexers instead of one wide one, and the region sizes stop being arbitrary — this is why real address maps cluster peripherals into a contiguous band, as Chapter 1.2's did at 0x4000_0000. Module 18 is where that becomes the subject.

5. Where the Difficulty Actually Concentrates

The decoder and the multiplexer are the visible part. Four other pressures make a six-block FPGA SoC harder than it looks, and none of them is about any individual peripheral.

Latency disagreement, which is the expensive one. The table in Section 1 has four distinct read latencies. If the fabric has no completion signal, the initiator must know each one — five special cases in the initiator, a maintenance hazard on every revision, and a design that cannot absorb a register slice added for timing closure. With a completion signal it is one case: wait. This single property is what makes a fabric composable, and it is the clearest thing a standard buys.

Backpressure. The SPI controller is busy for tens of cycles. Without a way to say not yet, the system's only options are to poll a status register before every access — which pushes the problem into software and costs a full access to find out — or to size everything for the worst case. A target that can stall is not a complication; it is what lets a slow block share a bus with a fast one.

Reset, which is the one that bites in the lab. Section 1's six blocks plausibly arrive with three different reset conventions, as Chapter 1.3's three did. On an FPGA the danger is specific: block RAM and registers come up from the bitstream with defined contents, so a block that is never reset appears to work in a way it would not in simulation or in an ASIC. The failure surfaces later, on the second run without reconfiguration, and looks like anything but a reset problem.

Debug visibility, which is the one nobody plans for. When an access misbehaves you need to see the address, the select, the response and the completion, all on the same waveform. With one interface that is one integrated logic analyser instance with one trigger condition, reusable across every target. With six private interfaces it is six probe sets and six mental models, and the instrumentation itself consumes block RAM that the design wanted.

6. Overlapping Regions, Concretely

Section 2 mentioned that an incomplete comparison aliases a target. It is worth doing the arithmetic, because this is the FPGA SoC bug that survives longest.

Suppose the UART decode were written lazily, comparing only the bits that distinguish it from its neighbours:

  // WRONG — compares only bits 15:12 instead of the full tag 31:12.
  assign hit_uart = (addr[15:12] == 4'h1);

The UART now answers at 0x4000_1000, as intended — and also at 0x0000_1000, 0x2000_1000, 0xDEAD_1000, and roughly sixteen million other addresses.

What breaks, and when. Nothing, for a while. Software uses the documented address and everything works. Then one of three things happens:

• Something else is added at an aliased address, and now two targets answer. With the AND-OR mux of Section 4, the read data is the bitwise OR of both targets' values — a number belonging to neither, which is exceptionally confusing to look at.
• A stray pointer lands on an aliased address and silently writes a UART register instead of faulting. The symptom appears in the UART, arbitrarily later.
• Someone writes a driver against an alias because it happened to work, and the map is now undocumentedly load-bearing at an address nobody intended.

Why it survives so long: every test passes, because every test uses the documented address. The bug is in the addresses nobody tests, which is all of them.

What catches it. Assertion P1, driven with an unconstrained address in a formal tool, finds it immediately and exhaustively: the checker simply asks whether any address exists for which two selects are high. A directed simulation will not, because it would have to guess the alias. This is the case that most justifies formal for a decoder — the property is simple, stateless, and the interesting inputs are the ones no human would think to write.

7. Common Misconceptions

"The fabric is trivial compared to the peripherals."

The wrong model: decode and mux are a few lines, so they are a small part of the problem.

What it costs: an integration where nobody owns the fabric, the critical path ends up in it, and the timing failure is treated as a mysterious tool problem.

The corrected model: the peripherals are independent and the fabric is where they interact, so it carries all the coupling. It is also on the critical path of every access and grows with every target, while the peripherals do not.

"Adding a peripheral is a small, local change."

The wrong model: one more block, one more decode line.

What it costs: an eighth target that pushes the read multiplexer to a second LUT level and fails timing on a design that met it comfortably at seven — a change with no functional content that breaks the build.

The corrected model: a target adds a comparator, an input to every bit of the response mux, a completion to merge, and possibly a different latency and reset convention. Only the first of those is local.

"Unmapped accesses will fault."

The wrong model: the system has a sensible default.

What it costs: a hang from a stray pointer, on a system that had no default target because nobody decided it needed one.

The corrected model: the default target is a design element you build. Its absence is what makes the all-zero select vector reachable, which is why the strongest decoder property degrades from $onehot to $onehot0.

"It met timing, so the fabric is fine."

The wrong model: timing closure is a property of the design as built.

What it costs: a design with no headroom, where the next peripheral, the next clock-rate bump or the next synthesiser version turns a working build into a failing one with no functional change.

The corrected model: look at where the critical path is, not only at whether it closed. A fabric path at seven targets tells you what the eighth will do.

"Wishbone would define the address map for us."

The wrong model: adopting a bus standard settles the system's layout.

What it costs: looking for something in the specification that is not there, and an address map that ends up undocumented because everyone assumed it was inherited.

The corrected model: a bus standard defines the interface. The map, the region sizes, the decode, the default target and the arbitration policy are all yours, in every system, exactly as Chapter 1.2 said.

8. Interview Reasoning

9. Understanding Check

10. What's Next

This chapter built the fabric. The decode had to be exhaustive and mutually exclusive, and one assertion checks both. The response multiplexer, not the peripherals, carried the critical path and the scaling cost. Four different read latencies across five targets made the case for a completion signal on structural grounds rather than aesthetic ones. And every structural fix for a timing problem turned out to require the same property.

Everything so far has been derived from engineering pressure alone. A team that has never heard of Wishbone, working carefully, arrives at exactly these conclusions — which raises a historical question rather than a technical one.

When an ecosystem of independently written, freely available hardware blocks actually came into existence, what did its participants discover they needed, and what did they build in response?

Chapter 1.6 — OpenCores Origins answers it with the history, separating what is documented from what is interpretation, and shows why an ecosystem of independent IP authors reached for a common interconnection convention. The full path is on the Wishbone curriculum index.