Crossbar Concepts

Chapter 18.1 ended with two free slaves, two independent requests, and zero clocks of overlap.

What has to change structurally before two independent transfers can proceed at the same time?

1. The Structural Change, in One Line

Take the shared bus's pipeline and swap two stages.

shared bus   masters -> ONE arbiter -> one path -> ONE decoder -> slaves
crossbar     masters -> A DECODER PER MASTER -> AN ARBITER PER SLAVE -> slaves

Decode first, then arbitrate. Once each master's destination is known before the arbitration, a master asking for S0 and a master asking for S1 are not competitors at all — they are in different queues.

That is not this module's invention. LiteX's Wishbone library builds its crossbar exactly that way, and the whole of the construction is five lines of its Crossbar class:

# decode each master into its access row
for row, master in zip(access, masters):
    self.submodules += Decoder(master, row, register)
# arbitrate each access column onto its slave
for column, bus in zip(zip(*access), busses):
    self.submodules += Arbiter(column, bus, mode=arbiter)

A Decoder per master and an Arbiter per slave. In the same file, InterconnectShared is one Arbiter plus one Decoder. The two topologies are built from the same two components in the opposite order, and Chapter 18.5 returns to that source.

And the specification's own example of a multi-master arbiter is a per-destination one, in the BLOCK section:

if the SLAVE is a shared (dual port) memory, then an arbiter for that memory can determine when one MASTER is done with it so that another can gain access to the memory.

"An arbiter for that memory." Not an arbiter for the bus.

2. RTL — The Crossbar

Everything below is Chapter 18.1's interconnect with the two stages swapped and the arbitration replicated. The return path is the one part that is genuinely new.

// ─────────────────────────────────────────────────────────────────────────
// wb_crossbar2x2 — two masters, two slaves, TWO independent paths.
//
// The structural change from wb_shared_intercon2x2 is small and it is the
// whole chapter: the decode moves in front of the arbitration, and the
// arbitration is replicated per destination.
//
//   shared bus   ONE arbiter  -> one shared path -> ONE decoder -> 2 slaves
//   crossbar     A DECODER PER MASTER -> AN ARBITER PER SLAVE
//
// That second line is not this module's invention. LiteX's Wishbone library
// builds its Crossbar exactly that way, in five lines:
//
//   # decode each master into its access row
//   for row, master in zip(access, masters):
//       self.submodules += Decoder(master, row, register)
//   # arbitrate each access column onto its slave
//   for column, bus in zip(zip(*access), busses):
//       self.submodules += Arbiter(column, bus, mode=arbiter)
//
// ── WHAT A CROSSBAR DOES AND DOES NOT REMOVE ────────────────────────────
// It does not remove arbitration - it has two arbiters where the shared bus
// had one. It does not remove decoding, ownership, routing, provenance or
// contention. What it changes is the SCOPE of contention: a shared bus has
// one contention domain for the whole system; this has one per destination.
// Two masters going to different slaves no longer meet. Two masters going
// to the SAME slave meet exactly as before.
//
// ── THE RETURN PATH IS NOT A LIVE DECODE ────────────────────────────────
// This is the one place the shared-bus reasoning stops working. There, the
// shared address WAS the owner's address, so decoding it live was coherent.
// Here a master that is waiting for slave 0 may have any address on its
// pins, and slave 1 may be answering somebody else on the same clock. The
// return path must therefore be keyed on WHICH SLAVE GRANTED THIS MASTER -
// sown0_q and sown1_q - and not on what the master's address currently
// decodes to. ZipCPU's wbxbar keeps the same state for the same reason,
// storing a per-master slave index "for faster/cheaper logic on the return
// path".
//
// ── THE THREE DEFECTS ───────────────────────────────────────────────────
//   LIVE_RETURN     the return path uses the master's current decode
//   SAME_SLAVE_MIX  a contested slave gets ADR from one master and DAT
//                   from the other
//   BROADCAST       a slave's termination reaches both masters
// ─────────────────────────────────────────────────────────────────────────
module wb_crossbar2x2 #(
  parameter int unsigned AW             = 12,
  parameter int unsigned DW             = 32,
  parameter int unsigned SW             = DW/8,
  parameter int unsigned SEL_BIT        = 11,
  parameter bit          POLICY         = 1'b1,
  parameter bit          LIVE_RETURN    = 1'b0,
  parameter bit          SAME_SLAVE_MIX = 1'b0,
  parameter bit          BROADCAST      = 1'b0
) (
  input  logic          clk_i,
  input  logic          rst_i,
  input  logic          m0_cyc_i, m0_stb_i, m0_we_i,
  input  logic [AW-1:0] m0_adr_i,
  input  logic [DW-1:0] m0_dat_i,
  input  logic [SW-1:0] m0_sel_i,
  output logic [DW-1:0] m0_dat_o,
  output logic          m0_ack_o, m0_err_o,
  input  logic          m1_cyc_i, m1_stb_i, m1_we_i,
  input  logic [AW-1:0] m1_adr_i,
  input  logic [DW-1:0] m1_dat_i,
  input  logic [SW-1:0] m1_sel_i,
  output logic [DW-1:0] m1_dat_o,
  output logic          m1_ack_o, m1_err_o,
  output logic          s0_cyc_o, s0_stb_o, s0_we_o,
  output logic [AW-1:0] s0_adr_o,
  output logic [DW-1:0] s0_dat_o,
  output logic [SW-1:0] s0_sel_o,
  input  logic [DW-1:0] s0_dat_i,
  input  logic          s0_ack_i, s0_err_i,
  output logic          s1_cyc_o, s1_stb_o, s1_we_o,
  output logic [AW-1:0] s1_adr_o,
  output logic [DW-1:0] s1_dat_o,
  output logic [SW-1:0] s1_sel_o,
  input  logic [DW-1:0] s1_dat_i,
  input  logic          s1_ack_i, s1_err_i,
  // -- observation --
  output logic [1:0]    sown0_o, sown1_o,   // 0 NONE, 1 M0, 2 M1
  output logic          m0_dest_o, m1_dest_o,
  output logic          s0_con_o, s1_con_o  // contested this clock
);
  localparam logic [1:0] OWN_NONE = 2'd0;
  localparam logic [1:0] OWN_M0   = 2'd1;
  localparam logic [1:0] OWN_M1   = 2'd2;

  // ── A DECODER PER MASTER ──
  logic m0_dest, m1_dest;
  assign m0_dest   = m0_adr_i[SEL_BIT];
  assign m1_dest   = m1_adr_i[SEL_BIT];
  assign m0_dest_o = m0_dest;
  assign m1_dest_o = m1_dest;

  // A master is eligible AT A DESTINATION, not globally. This one line is
  // the difference between the two topologies.
  logic m0_w0, m0_w1, m1_w0, m1_w1;
  assign m0_w0 = m0_cyc_i && !m0_dest;
  assign m0_w1 = m0_cyc_i &&  m0_dest;
  assign m1_w0 = m1_cyc_i && !m1_dest;
  assign m1_w1 = m1_cyc_i &&  m1_dest;

  assign s0_con_o = m0_w0 && m1_w0;
  assign s1_con_o = m0_w1 && m1_w1;

  // ── AN ARBITER PER SLAVE ──
  logic [1:0] sown0_q, sown1_q;
  assign sown0_o = sown0_q;
  assign sown1_o = sown1_q;

  // Retention: a slave is held by its owner while that owner still wants
  // THIS slave. A master that changes destination releases the old one,
  // which is what keeps a master from holding two slaves at once.
  logic hold0, hold1;
  assign hold0 = (sown0_q == OWN_M0 && m0_w0) || (sown0_q == OWN_M1 && m1_w0);
  assign hold1 = (sown1_q == OWN_M0 && m0_w1) || (sown1_q == OWN_M1 && m1_w1);

  logic ev0, ev1;
  assign ev0 = (sown0_q == OWN_NONE) || !hold0;
  assign ev1 = (sown1_q == OWN_NONE) || !hold1;

  logic p0sel, p0val, p0last, p1sel, p1val, p1last;
  wb_arb_policy #(.NREQ(2), .POLICY(POLICY)) u_arb_s0 (
    .clk_i(clk_i), .rst_i(rst_i), .req_i({m1_w0, m0_w0}), .commit_i(ev0),
    .sel_o(p0sel), .valid_o(p0val), .last_o(p0last));
  wb_arb_policy #(.NREQ(2), .POLICY(POLICY)) u_arb_s1 (
    .clk_i(clk_i), .rst_i(rst_i), .req_i({m1_w1, m0_w1}), .commit_i(ev1),
    .sel_o(p1sel), .valid_o(p1val), .last_o(p1last));

  logic [1:0] sown0_n, sown1_n;
  always_comb begin
    sown0_n = sown0_q;
    if (ev0) begin
      if (!p0val)     sown0_n = OWN_NONE;
      else if (p0sel) sown0_n = OWN_M1;
      else            sown0_n = OWN_M0;
    end
    sown1_n = sown1_q;
    if (ev1) begin
      if (!p1val)     sown1_n = OWN_NONE;
      else if (p1sel) sown1_n = OWN_M1;
      else            sown1_n = OWN_M0;
    end
  end
  always_ff @(posedge clk_i) begin
    if (rst_i) begin sown0_q <= OWN_NONE; sown1_q <= OWN_NONE; end
    else       begin sown0_q <= sown0_n;  sown1_q <= sown1_n;  end
  end

  // ── FORWARD: each slave sees exactly one master's whole context ──
  logic mix0, mix1;
  assign mix0 = SAME_SLAVE_MIX && s0_con_o;
  assign mix1 = SAME_SLAVE_MIX && s1_con_o;

  logic s0o0, s0o1, s1o0, s1o1;
  assign s0o0 = (sown0_q == OWN_M0);
  assign s0o1 = (sown0_q == OWN_M1);
  assign s1o0 = (sown1_q == OWN_M0);
  assign s1o1 = (sown1_q == OWN_M1);

  assign s0_cyc_o = (s0o0 && m0_w0)   || (s0o1 && m1_w0);
  assign s0_stb_o = (s0o0 && m0_stb_i) || (s0o1 && m1_stb_i);
  assign s1_cyc_o = (s1o0 && m0_w1)   || (s1o1 && m1_w1);
  assign s1_stb_o = (s1o0 && m0_stb_i) || (s1o1 && m1_stb_i);

  always_comb begin
    if (mix0) begin
      // DEFECT: address from M0, payload from M1, at a contested slave.
      s0_adr_o = m0_adr_i; s0_we_o = m0_we_i;
      s0_dat_o = m1_dat_i; s0_sel_o = m1_sel_i;
    end else if (s0o0) begin
      s0_adr_o = m0_adr_i; s0_we_o = m0_we_i;
      s0_dat_o = m0_dat_i; s0_sel_o = m0_sel_i;
    end else if (s0o1) begin
      s0_adr_o = m1_adr_i; s0_we_o = m1_we_i;
      s0_dat_o = m1_dat_i; s0_sel_o = m1_sel_i;
    end else begin
      s0_adr_o = '0; s0_we_o = 1'b0; s0_dat_o = '0; s0_sel_o = '0;
    end
    if (mix1) begin
      s1_adr_o = m0_adr_i; s1_we_o = m0_we_i;
      s1_dat_o = m1_dat_i; s1_sel_o = m1_sel_i;
    end else if (s1o0) begin
      s1_adr_o = m0_adr_i; s1_we_o = m0_we_i;
      s1_dat_o = m0_dat_i; s1_sel_o = m0_sel_i;
    end else if (s1o1) begin
      s1_adr_o = m1_adr_i; s1_we_o = m1_we_i;
      s1_dat_o = m1_dat_i; s1_sel_o = m1_sel_i;
    end else begin
      s1_adr_o = '0; s1_we_o = 1'b0; s1_dat_o = '0; s1_sel_o = '0;
    end
  end

  // ── RETURN: keyed on the GRANT, never on the current decode ──
  logic m0_a_ok, m0_a_lr, m1_a_ok, m1_a_lr;
  logic m0_e_ok, m0_e_lr, m1_e_ok, m1_e_lr;
  logic [DW-1:0] m0_d_ok, m0_d_lr, m1_d_ok, m1_d_lr;

  assign m0_a_ok = (s0o0 && s0_ack_i) || (s1o0 && s1_ack_i);
  assign m1_a_ok = (s0o1 && s0_ack_i) || (s1o1 && s1_ack_i);
  assign m0_e_ok = (s0o0 && s0_err_i) || (s1o0 && s1_err_i);
  assign m1_e_ok = (s0o1 && s0_err_i) || (s1o1 && s1_err_i);
  assign m0_d_ok = s1o0 ? s1_dat_i : s0_dat_i;
  assign m1_d_ok = s1o1 ? s1_dat_i : s0_dat_i;

  // DEFECT: the same signals derived from the master's live decode.
  assign m0_a_lr = m0_dest ? s1_ack_i : s0_ack_i;
  assign m1_a_lr = m1_dest ? s1_ack_i : s0_ack_i;
  assign m0_e_lr = m0_dest ? s1_err_i : s0_err_i;
  assign m1_e_lr = m1_dest ? s1_err_i : s0_err_i;
  assign m0_d_lr = m0_dest ? s1_dat_i : s0_dat_i;
  assign m1_d_lr = m1_dest ? s1_dat_i : s0_dat_i;

  logic bcast_a, bcast_e;
  assign bcast_a = s0_ack_i || s1_ack_i;
  assign bcast_e = s0_err_i || s1_err_i;

  always_comb begin
    if (BROADCAST) begin
      m0_ack_o = bcast_a; m1_ack_o = bcast_a;
      m0_err_o = bcast_e; m1_err_o = bcast_e;
      m0_dat_o = m0_d_ok; m1_dat_o = m1_d_ok;
    end else if (LIVE_RETURN) begin
      m0_ack_o = m0_a_lr; m1_ack_o = m1_a_lr;
      m0_err_o = m0_e_lr; m1_err_o = m1_e_lr;
      m0_dat_o = m0_d_lr; m1_dat_o = m1_d_lr;
    end else begin
      m0_ack_o = m0_a_ok; m1_ack_o = m1_a_ok;
      m0_err_o = m0_e_ok; m1_err_o = m1_e_ok;
      m0_dat_o = m0_d_ok; m1_dat_o = m1_d_ok;
    end
  end
endmodule

Reading it

Four lines carry the whole topology:

  assign m0_w0 = m0_cyc_i && !m0_dest;
  assign m0_w1 = m0_cyc_i &&  m0_dest;
  assign m1_w0 = m1_cyc_i && !m1_dest;
  assign m1_w1 = m1_cyc_i &&  m1_dest;

A master is eligible at a destination, not globally. On the shared bus the eligibility vector was {m1_cyc, m0_cyc} and there was one of it. Here there are two vectors, {m1_w0, m0_w0} and {m1_w1, m0_w1}, and a master appears in exactly one of them.

The two wb_arb_policy instances are Chapter 17.1's module, unchanged, instantiated twice. Nothing about arbitration had to change; there is simply more of it.

The retention condition is where destination coherence lives:

  assign hold0 = (sown0_q == OWN_M0 && m0_w0) || (sown0_q == OWN_M1 && m1_w0);

A slave is held by its owner while that owner still wants this slave. A master that changes destination releases the old one automatically, which is what stops a master holding two slaves at once — and the probe checks that it never does.

Now the return path, which is the part that could not have been written by analogy.

On the shared bus the response was selected by decoding the shared address live, and Chapter 18.1 §3 argued that this was safe because the shared address was the owner's address. Here that argument fails. A master waiting for S0 has its own address on its own pins, and S1 may be answering somebody else on the same clock. Decode M0's address live and you will hand it S1's answer.

So the return path is keyed on the grant:

  assign m0_a_ok = (s0o0 && s0_ack_i) || (s1o0 && s1_ack_i);

"Which slave granted me?" — not "what does my address say?" ZipCPU's wbxbar keeps the same state for the same reason, storing a per-master slave index explicitly "for faster/cheaper logic on the return path". LIVE_RETURN is this module with that reasoning removed, and Chapter 18.3 measures what it costs.

Compare this with Chapter 18.1's figure. There, everything narrowed to one arrow. Here there is no single arrow every transfer must cross — and the two arbiter boxes are the two contention domains that replaced the one.

3. Simulation — SIM C: Two Paths at Once

M0 reads 0x011 (S0) and M1 reads 0x813 (S1), on the same clock. Both slaves insert two wait states, so there is room to watch.

  === SIM C - independent destinations, at the same time ===
    M0 reads 0x011 (S0) and M1 reads 0x813 (S1). Both slaves
    insert two wait states.

      clk  S0 active  S0 owner  S1 active  S1 owner    ACK
        2      0        -          0        -       -
        3      0        -          0        -       -
        4      0        -          0        -       -
        5      1        M0         1        M1      -
        6      1        M0         1        M1      -
        7      1        M0         1        M1      S0+S1
        8      0        M0         0        M1      -
        9      0        -          0        -       -

    clocks with BOTH slave paths active  3
    clocks with exactly one active       0
    S0 path active clocks 3   S1 path active clocks 3
    same-destination contention  S0 0   S1 0
    M0 read 0xa0000011   M1 read 0xb1000013
    provenance violations  dest 0  fwd 0  return 0  data 0

Reading it — clock 7

Clocks 5, 6 and 7: S0 active 1, owner M0; S1 active 1, owner M1. Both, on every one of those clocks.

And clock 7's ACK column reads S0+S1. Two slaves acknowledged two different masters on the same clock. That is the decisive crossbar measurement and it is one cell of one table.

clocks with BOTH slave paths active: 3. clocks with exactly one active: 0. In Chapter 18.1's SIM B, on the same question, those numbers were 0 and 2.

same-destination contention: S0 0, S1 0. The two masters were both asking, throughout, and they never contended — because they were never asking for the same thing. On the shared bus they contended for the path regardless of where they were going.

M0 read 0xA0000011 and M1 read 0xB1000013. Each master's own offset, from its own slave's signature, while the other transfer was in flight. All eight provenance counters are zero.

One detail worth pausing on: the two transfers are not synchronised with each other. They happen to start together here because the stimulus made them, but nothing couples them. Two independent paths means two independent transfers, with their own wait states, their own terminations and their own durations — Chapter 18.5's audit shows them finishing on different clocks.

4. Simulation — SIM D: The Same Crossbar, One Destination

Nothing about the hardware changes. M0 reads 0x021 and M1 reads 0x022. Both are in S0.

  === SIM D - the same destination, on the same crossbar ===
    M0 reads 0x021 (S0) and M1 reads 0x022 (S0).

      clk  S0 active  S0 owner  S1 active  S1 owner    ACK
        1      0        -          0        -       -
        2      0        -          0        -       -
        3      0        -          0        -       -
        4      1        M1         0        -       -
        5      1        M1         0        -       -
        6      1        M1         0        -       S0
        7      0        M1         0        -       -
        8      1        M0         0        -       -
        9      1        M0         0        -       -
       10      1        M0         0        -       S0
       11      0        M0         0        -       -
       12      0        -          0        -       -

    clocks with BOTH slave paths active  0
    S0 path active clocks 6   S1 path active clocks 0
    same-destination contention  S0 4   S1 0
    S0 reads 2   S1 reads 0
    M0 read 0xa0000021   M1 read 0xa0000022
    owner moved under an open phase  0
    -> a crossbar relocates contention. It does not remove it.

Reading it

clocks with BOTH slave paths active: 0. S1 path active clocks: 0.

The second path exists, is correct, and is not used, because nothing is addressed to it. M1 owns S0 for clocks 4–6, M0 owns it for 8–10, and the sequence is indistinguishable from the shared bus.

same-destination contention: S0 4. Four clocks on which both masters were asking for S0. On the shared bus the equivalent figure counted clocks on which both were asking at all. The number moved because the question did.

owner moved under an open phase: 0. Chapter 17.5's retention rule, applied per destination and still holding: once S0 granted M1, it kept M1 through all three clocks of an unanswered phase. A crossbar has more places to get retention wrong, not fewer.

Both masters got their own word — 0xA0000021 and 0xA0000022. Same signature, different offsets, correctly separated.

5. Capability Is Not Concurrency

Put SIM C and SIM D side by side and the statement writes itself.

The topology supplies a CAPABILITY. The workload decides whether it is ever used.

A crossbar cannot create concurrency out of a workload that has none, and "we moved to a crossbar and saw no improvement" is, far more often than not, a statement about where the traffic is going rather than about the interconnect. Chapter 18.4 measures that case directly.

The specification's glossary makes a broad claim about crossbars and it is worth quoting exactly, because it is easy to over-read:

Each connection channel can be operated in parallel to other connection channels. This increases the data transfer rate of the entire system by employing parallelism... This makes the crossbar switches inherently faster than traditional bus schemes.

That is a general statement about crossbar switches, in the glossary of a protocol specification. It is not a measurement of any Wishbone system, and B3 contains no timing, area or bandwidth model for any interconnect at all. SIM D is a crossbar that is exactly as fast as a shared bus, and Chapter 18.4's SIM H is a crossbar with an idle second path.

6. Failure Modes and Discriminating Evidence

SYMPTOM — the crossbar shows no concurrency improvement.

Candidates. Both masters target the same slave. The workload has no temporal overlap. A shared internal path still serialises them.

Discriminating evidence. Per-slave active clocks, and the per-destination contention counters. If one slave's active count is zero, it is the workload. If both are non-zero and the both-active count is still zero, there is a shared resource inside the thing you are calling a crossbar — Chapter 18.4's false-parallelism case.

SYMPTOM — a master gets another master's data, only under load.

Candidates. The return path is keyed on the master's live decode instead of its grant. Two grants to one slave. A response mux with no ownership qualifier.

Discriminating evidence. Which slave granted this master on the clock it completed, against which slave produced the data. Chapter 18.3 measures exactly this and names it.

SYMPTOM — a slave receives a transfer that combines two masters' fields.

Candidates. The per-slave forward mux selects fields independently rather than selecting a master.

Discriminating evidence. The slave's ADR, WE and DAT against each master's own, on the same clock. RULE 3.60 names that set as one thing a master qualifies together; keeping it together through an interconnect is unstated in B3 and therefore entirely your problem.

SYMPTOM — a master appears to hold two slaves.

Candidates. The retention condition tests only "does the owner still have CYC_O" and not "does it still want this slave".

Discriminating evidence. Both slaves' owner registers on the same clock. The probe's exclusivity counter is one comparison and it is worth having.

7. Common Mistakes

"A crossbar eliminates arbitration."

Why it is wrong: this one has two arbiters. It eliminates arbitration between masters going to different places, which is a much narrower and much more useful statement.

"A crossbar eliminates contention."

Why it is wrong: SIM D. Four contention clocks at S0, on a crossbar, with an idle second path available.

"Crossbar means every transfer is parallel."

Why it is wrong: parallelism requires two transfers to different destinations at the same time. Two of the three conditions are properties of the workload.

"Crossbar is always faster."

Why it is unsupportable as stated: SIM D is a crossbar performing exactly like a shared bus. The specification's glossary does say crossbars are "inherently faster than traditional bus schemes" — as a general statement about crossbar switches, not as a measurement of a Wishbone system, and this chapter has one that is not.

"A shared bus is obsolete."

Why it is unsupportable: one arbiter, one decoder, one path, one place to look. Chapter 18.4 counts what the crossbar costs structurally, and the answer is not zero.

"Arbitration and routing are the same thing."

Why it is wrong: arbitration chose M1 for S0 in SIM D. Routing is what then carried M1's address to S0 and S0's answer back to M1 — and Chapter 18.3 breaks the second while leaving the first perfect.

8. Interview Reasoning

"What changes in a crossbar?"

The order of two stages. Decode moves in front of arbitration, and arbitration is replicated per destination. Everything else — ownership, retention, routing, provenance — is still there, in more copies.

"Where does arbitration happen in a crossbar?"

At each destination. A master competes only with masters going to the same place. The specification's own multi-master example is a per-memory arbiter, in the BLOCK section.

"Can two masters access two different slaves simultaneously?"

On this topology, yes — SIM C measures three clocks of it, ending with both slaves acknowledging on the same edge. On a shared bus, no, and that is a property of the architecture rather than of Wishbone.

"Can two masters access the same slave simultaneously?"

No, on any topology. SIM D is a crossbar and it serialises them. The destination is the resource, and a second path to somewhere else does not help.

"Why can the return path not simply decode the master's address?"

Because a waiting master's address is not the address of the transfer that is being answered. On a shared bus those coincide; on a crossbar they do not, and the return path must be keyed on which slave granted the master. That is state you have to keep.

"A team reports that their crossbar gave no speed-up. What do you ask?"

Where the traffic went. Per-slave active clocks first, then both-active clocks. If one slave is idle, the interconnect is not the subject of the conversation.

9. Understanding Check

SIM C and SIM D run on identical hardware and produce opposite concurrency results. What is the variable?

The destination addresses. In SIM C the two masters are in different queues; in SIM D they are in the same one. Nothing in the interconnect knows or cares which case it is in.

In SIM D, S1 was idle for the entire run. Is that a defect?

No — it is a correct path with nothing addressed to it. Chapter 18.4's SIM H makes this the whole experiment, because "we have more paths" and "our traffic uses them" are different claims.

Why does a crossbar need per-master destination state that a shared bus does not?

Because a master can be waiting at one slave while a different slave answers a different master. The shared bus has one transfer in flight and one address; the crossbar has up to two, and each master's response has to be found by grant, not by address.

Your crossbar's retention condition is owner still has CYC_O, without checking the destination. What can go wrong?

A master that changes destination mid-CYC keeps holding a slave it is no longer addressing — and can then be granted a second one. The exclusivity check catches it; the fix is one term.

10. What's Next

Two topologies now exist and both route correctly. The routing has been asserted rather than examined.

How do the forward and return paths stay coherent — and what does it look like when one of them does not?

Chapter 18.3 — Routing defines provenance as five things that must agree, measures all five, and then breaks the return path on a system whose every master and every slave still obeys every rule in B3.