Interconnect RTL

This is the last chapter of the build module, and it assembles everything before it: 23.1's masters, 23.2's slaves, 23.5's decoder, and the conformance monitor that has been watching all of them.

It also settles a question this curriculum has circled since Module 16, and the answer is not the one the usual explanation gives.

"Use a crossbar when your masters talk to different slaves" is the standard advice. It is true, it is incomplete, and the measurement below shows a crossbar winning decisively on traffic where both masters hammer the same slave and no transfer overlaps any other.

1. Neither Word Is In The Specification

B3 does not contain "crossbar". It does not contain "shared bus". What it has is INTERCONN, described in terms of what must arrive at a slave's pins, never how it gets there.

That absence is the chapter's central fact. A topology is a cost and throughput decision, not a protocol decision, and the only way to demonstrate that rather than assert it is to build both from one module and change nothing else:

  parameter int unsigned TOPOLOGY = 0,       // 0 shared bus, 1 crossbar

The whole difference is nine lines:

  always_comb begin
    if (TOPOLOGY == 0) begin
      // shared bus: whoever owns it, and ownership only changes when the
      // owner has no cycle open (Chapter 17.5's rule, reproduced)
      m0_go = m0_cyc_i && (owner_q == 1'b0);
      m1_go = m1_cyc_i && (owner_q == 1'b1);
    end else begin
      // crossbar: both go unless they collide on one slave, and then
      // master 0 wins
      m0_go = m0_cyc_i;
      m1_go = m1_cyc_i && !(same_slave && m0_cyc_i);
    end
  end

2. Routing The Whole Context, Or None Of It

RULE 3.60 qualifies [ADR_O], [DAT_O()], [SEL_O()] and [WE_O] with [STB_O]. An interconnect that routes the address from one master and the write data from another has produced a transfer that neither master requested, and nothing downstream can detect it — the slave sees a perfectly well-formed request.

So every field takes the same decision, deliberately:

// Every field takes the SAME `from_m0` decision, which makes field
// mixing structurally impossible rather than merely absent. Chapter
// 18.3's defect took ADR from one master and DAT from another; there is
// no way to express that here without editing four lines in step.
  assign s0_adr_o = s_from_m0[0] ? m0_adr_i : m1_adr_i;
  assign s0_dat_o = s_from_m0[0] ? m0_dat_i : m1_dat_i;
  assign s0_sel_o = s_from_m0[0] ? m0_sel_i : m1_sel_i;
  assign s_we_o[0] = s_from_m0[0] ? m0_we_i : m1_we_i;

"Structurally impossible rather than merely absent" is the design claim. A reviewer can verify it by checking that all four lines test the same expression, which is a far cheaper review than reasoning about whether a mixing case can arise.

And RULE 3.30 is enforced once, at the point where a slave's [CYC_I] is generated:

  // RULE 3.30: a slave's CYC_I is asserted only when it is selected by a
  // master that has a cycle open.
  assign s_cyc_o = s_from_m0 | s_from_m1;
  assign s_stb_o = s_cyc_o;

3. The Return Path, And The Rule That Makes It Easy

The forward route is a fan-out: one master's context reaches one slave. The return is a fan-in, and it is where an interconnect normally needs to remember something.

  assign m0_dat_o = s_from_m0[1] ? s1_dat_i :
                    s_from_m0[2] ? s2_dat_i :
                    s_from_m0[3] ? s3_dat_i : s0_dat_i;

No state. No transaction tag. No reorder buffer. The mux is driven by the same select vector that routed the request out, in the same clock.

That is RULE 3.35 paying for itself:

"The cycle termination signals [ACK_O], [ERR_O], and [RTY_O] must be generated in response to the logical AND of [CYC_I] and [STB_I]."

One outstanding phase per master means the answer can only belong to the request currently being made. Chapter 20.2 measured what AXI spends to relax this — IDs, reorder depth, and the buffering to make them work — and the honest summary is that Wishbone's interconnect is cheap because its protocol forbids the thing that makes interconnects expensive.

The fabric also counts what only it can see:

  output logic [15:0]   concurrent_o,     // clocks both masters progressed
  output logic [15:0]   m0_xfers_o,
  output logic [15:0]   m1_xfers_o,
  output logic [15:0]   unmapped_o        // accesses that hit no region

concurrent_o is the measurement that makes Section 5 falsifiable. Without it, a crossbar finishing sooner would simply be assumed to have overlapped transfers — and Section 5 is precisely the case where that assumption is wrong.

4. The Baseline Nobody Measures

Before comparing topologies, measure one master alone, because every number afterwards is a story about this one:

    BASELINE  (master 0 alone, master 1 idle, shared bus)
      31 clocks for 8 transfers = 3.8 clocks each

      Only TWO of those four are slave work: one clock to
      present the phase and one to wait out WAITS=1. The
      other two are the master FSM retiring the phase and
      accepting the next request, and THE SLAVE IS IDLE FOR
      BOTH OF THEM. One master keeps this slave busy 51%
      of the time. Every number below is a story about that
      idle half.

A single master, with nothing competing for anything, leaves the slave idle half the time. Those idle clocks are the master's own bookkeeping — the DONE and IDLE states Chapter 23.1's clock audit counted — and they are invisible in any measurement that only reports clocks per transfer.

5. Disjoint Traffic: The Expected Result

Master 0 to slave 0, master 1 to slave 1. Eight writes each, one wait state.

    DISJOINT TRAFFIC  (master 0 -> slave 0, master 1 -> slave 1)
      topology      clocks   m0 done  m1 done  concurrent  viol
      shared bus       48        8        8           0     0
      crossbar         31        8        8           8     0

31 clocks against 48, a 35% saving, and 8 clocks on which both masters genuinely completed transfers at once. This is the result the standard advice predicts, and it is real.

It is also paid for. The shared bus has one set of forward muxes and one return mux. The crossbar has a set per slave port — four address muxes, four data muxes, four select muxes, four write-enable muxes, plus a return mux per master. Four times the routing for two masters and four slaves, and it grows as the product of the two counts.

6. Contended Traffic, Which Is Where It Gets Interesting

Now point both masters at slave 0. One slave cannot answer two masters in a clock, so a crossbar should buy nothing at all.

    CONTENDED TRAFFIC (both masters -> slave 0)
      topology      clocks   m0 done  m1 done  concurrent  viol
      shared bus       48        8        8           0     0
      crossbar         33        8        8           0     0

Concurrent is 0 for both. No transfer overlapped any other transfer. And the crossbar still finished 15 clocks sooner.

      SLAVE 0, THE ONLY SLAVE ANYONE WANTED:
        rig            xfers  clocks  clks/xfer  slave busy
        one master        8      31      3.8        51%
        shared bus       16      48      3.0        66%
        crossbar         16      33      2.0        96%

Read the middle row first, because it is the surprising one:

Adding a second competing master made each transfer cheaper — 3.8 clocks each became 3.0. A single master leaves the slave idle half the time doing its own bookkeeping; a second master has bookkeeping of its own to do at the same moment, so the two interleave. Contention filled a hole that was already there.

The crossbar fills it almost completely. Sixteen transfers need two clocks of slave time each — 32 clocks — and it spent 33. The slave was busy 96% of the time.

Why the shared bus only half fills it

      // shared-bus ownership changes only between cycles - a grant that
      // moved mid-phase would break RULE 3.60 for the master that lost
      // it, which Chapter 16.4 measured as the PREEMPT defect.
      if (TOPOLOGY == 0) begin
        if (owner_q == 1'b0 && !m0_cyc_i && m1_cyc_i) owner_q <= 1'b1;
        if (owner_q == 1'b1 && !m1_cyc_i && m0_cyc_i) owner_q <= 1'b0;
      end

Ownership is registered, and it must be. A grant that moved while a phase was open would change [ADR_O] under a slave that had already sampled it — RULE 3.60 violated, and Chapter 16.4 measured exactly that as the PREEMPT defect.

The consequence is that handover costs a clock. The crossbar's per-slave gate is combinational and re-decides every clock, so master 1 is already presenting on the clock master 0 stops.

The limit was never slave bandwidth. It was ownership granularity. A designer who reads the disjoint table as "crossbars help when traffic separates" has taken the wrong lesson from the right number: this traffic does not separate at all, and the crossbar still won.

7. The Fix, And Its Price

The shared bus can close most of that gap with a combinational grant — decide ownership from the current [CYC_O] signals rather than a register.

It is not free, and B3 tells you so in advance. PERMISSION 3.30 permits a combinational [ACK_O], creating a path from [STB_I] to [ACK_O]; a combinational grant extends that path backwards, putting every master's [CYC_O] in the path to every slave's [STB_I]. B3 §4.1:

"...this results in an asynchronous loop from the MASTER, through the INTERCONN to the SLAVE, and then from the SLAVE through the INTERCONN back to the MASTER... In large System-on-Chip devices this routing delay between MASTER and SLAVE is the dominant timing factor."

The specification hands you this trade explicitly, twice, in two adjacent chapters. Chapter 17.5 measured the arbitration half of it.

8. Topology Does Not Touch Conformance

Both master ports were monitored in all four rigs:

      -> ZERO violations in all four rigs. Both masters
         saw a conformant Wishbone port in every case.
         TOPOLOGY IS NOT A PROTOCOL DECISION: B3 does not
         name either word, and neither master can tell
         which one it is attached to except by TIMING.

Four rigs, two topologies, two traffic patterns, zero violations. A master cannot discover which fabric it is attached to by any means the protocol provides — only by measuring how long things take. That is what it means for topology to be outside the specification, and it is why a Wishbone core written against B3 ports cleanly between the two.

9. What This Interconnect Does Not Do

• Two masters and four slaves, fixed. The ports are enumerated, not generated. A real generator emits this from a table; the structure would not change, and the enumeration keeps the routing legible.
• Master 0 always wins a crossbar collision. Fixed priority, and Chapter 17.4 showed what that does to the loser under sustained load. Swapping in Chapter 17.5's round-robin arbiter is a drop-in change.
• No pipelining between the decode and the route. Both are combinational, which is the shortest path and the longest wire.
• No [RTY_O] handling in the fabric. A retry is passed back to the master untouched; the interconnect does not re-issue.
• No burst or [CTI_O()] awareness. Classic only, like everything in this module.

10. Where Module 23 Ends

Six chapters built a master, a slave skeleton, a register-bank generator, a memory controller, an N-way decoder and a two-topology interconnect — 1,331 lines of synthesisable RTL, eleven simulations, and a conformance monitor shown both catching defects and missing them.

The single most transferable result is the negative control, and it is worth restating out of its chapter: the memory controller with REFRESH_PREEMPTS lost twelve of twelve transfers and scored zero protocol violations. A conformance monitor catches its own rules and is blind to everything else.

Next: Module 24 takes Chapter 23.2's deliberately shallow slave skeleton and fills it in — register maps, read and write datapaths, [ACK_O] timing in depth, and the [ERR_O] and [RTY_O] policy this module was careful to leave alone.