Endpoint Architecture — Where the Boundaries Go, and What Each One Guarantees

Module 22 measured an endpoint. Module 23 builds one, and this chapter is where the blocks and the lines between them are decided.

The temptation is to treat this as a picture. Four layers stacked, arrows between them, done. But a block diagram is not an architecture — it says what exists and nothing about what is guaranteed, and every hard bug in a controller lives in the guarantee, not in the block.

1. Sources, Scope, and Module 23's Boundaries

2. A Stack of Owners

Three lines of recap (2.3 owns the concept). An Endpoint terminates a path rather than forwarding it; it acts as a Completer for transactions aimed at it and as a Requester for transactions it originates; and it is found and configured by enumeration before it does either.

Inside the controller, that becomes a stack, and the useful way to read the stack is by asking what each level owns exclusively:

The right-hand column is the architecture. A design where the DMA engine inspects sequence numbers, or where the Data Link Layer decodes a BAR, has not made an error of function — it has destroyed a boundary, and the cost appears later as a change that cannot be made locally.

Chapter 11.3 §1 makes the same argument one level down — framing decode and semantic decode stay apart so a field bug cannot hide behind a framing bug. This chapter applies that principle to the whole controller.

3. What Each Boundary Guarantees

A boundary is defined by its guarantees, not by its signals. For every interface in §9 the contract is:

All six are one sentence: ownership moves once, at a defined instant, and nothing about the item changes before it does.

§11 Model 1 verified the consequence across pipelines of 1, 2, 3, 5 and 8 stages under random backpressure — 319,495 items, 0 lost, 0 duplicated. And Model 4 measured what breaking stability alone costs: a producer that re-drives under stall corrupted 26.1% of packets at a 10% stall rate and 80.7% at 50%.

Note the shape of that result. The corruption rate scales with congestion — so the bug is quiet on an idle bench and severe in the loaded system where it matters. This is the fourth time in this curriculum that the naive implementation fails in the flattering direction (22.1 §7, 22.2 §11, 22.3 §5 were the others).

4. Backpressure Propagates

A boundary cannot absorb congestion; it can only move it.

Two wrong instincts, both common.

"Add a buffer so we never stall." A deeper buffer changes when backpressure appears, never whether. It also increases latency (22.2 §7) and hides the stall from whatever monitors the interface — so the system gets slower and less diagnosable at once.

"Accept the packet and sort it out later." Accepting means taking ownership. A block that accepts what it cannot forward has committed to storing it, and if it has nowhere to put it, it drops it — which is the one thing the contract forbids.

The correct shape is a combinational ready chain with registered payload where timing demands it — a skid buffer (§9), which holds exactly enough state to break the ready path without ever accepting an item it cannot hold.

And the diagnostic consequence matters as much as the functional one. Because backpressure propagates, the block that is stalled is not necessarily the block that is slow. Chapter 22.1 §6's attribution counters exist at the transmit interface for this reason: they name the cause, not the location.

5. Two Roles, and Whether They Share a Path

6. Reset Is Released Consumer-First

7. Clock Domains, and Keeping Them Few

A controller typically spans at least two domains — the PHY's, derived from the link, and the application's. Every crossing is a place to lose or duplicate an item, so the architecture's job is to have as few as possible and to make each one explicit.

Three rules this chapter's RTL follows.

Cross once, at a named boundary. One asynchronous FIFO per direction per crossing, instantiated in the top level where it is visible — never buried inside a functional block, where a later refactor can silently duplicate it.

Never cross a multi-signal value without a handshake or a FIFO. A payload and its valid sampled independently in another domain can produce a packet whose header and payload came from different items — structurally the same corruption as §3's re-driving producer, arriving by a different route.

And do not cross diagnostic counters casually. Chapter 22.1 §9's counters live in the domain they observe. A counter incremented from a foreign domain's pulse counts what the synchronizer let through, which is not what happened.

This chapter does not build CDC machinery — it places the crossings and states the rule. A production controller adds synchronizer depth, reset-domain crossing analysis and constraint files that are outside this chapter's scope.

8. The Block Diagram

Four things to read out of the figure.

The stack is symmetric and the paths are not. Inbound and outbound both traverse PHY, Data Link and Transaction — but above the Transaction Layer the completer path and the requester path are separate all the way to TLP assembly (§5), which is the 20.4% in §11.

Three blocks are named and not built. inbound decode is 23.2, DMA engine is 23.3, TLP assembly is 23.4. This chapter builds the boundaries they attach to and asserts the contracts they must honour.

The config space is reached only through the decoder. Nothing else in the design has a private path to it — which is what makes Chapter 23.2's decode the single point where "which aperture is this?" is answered.

And there is no arrow from the DMA engine to the Data Link Layer. A requester's traffic goes through TLP assembly and the Transaction Layer like everything else. An architecture that lets a fast path skip a layer has not optimized; it has created a second, unverified implementation of that layer's rules.

9. RTL — The Skeleton and Its Contracts

// SYNTHESIZABLE. The interface contract, expressed as types.
// The whole architecture rests on §3's six guarantees; putting the payload
// in one struct makes "the payload is stable under stall" a single
// $stable() check instead of a list of signals someone can forget.
package ep_if_pkg;
 
  parameter int DATA_W  = 256;
  parameter int KEEP_W  = DATA_W/8;
  parameter int N_LAYER = 4;                 // app, TL, DLL, PHY
 
  typedef enum logic [1:0] {
    PATH_COMPLETER = 2'd0,   // responses this device must return
    PATH_REQUESTER = 2'd1,   // requests this device originates
    PATH_OTHER     = 2'd2
  } path_e;
 
  typedef struct packed {
    logic [DATA_W-1:0] data;
    logic [KEEP_W-1:0] keep;
    logic              sop;
    logic              eop;
    path_e             path;
  } beat_t;
 
  // Layer indices, ordered PRODUCER-to-CONSUMER for inbound traffic.
  // Reset is released in the OPPOSITE order (§6).
  typedef enum logic [1:0] {
    LYR_PHY = 2'd0, LYR_DLL = 2'd1, LYR_TL = 2'd2, LYR_APP = 2'd3
  } layer_e;
 
endpackage

import ep_if_pkg::*;
 
// SYNTHESIZABLE. The boundary primitive: a two-entry skid buffer.
// It breaks the combinational `ready` path WITHOUT ever accepting an item
// it cannot hold (§4). A FIFO would also work; what matters is that the
// acceptance decision is never a promise the block cannot keep.
module skid_buffer (
  input  logic  clk,
  input  logic  rst_n,
 
  input  logic  in_valid,
  output logic  in_ready,
  input  beat_t in_beat,
 
  output logic  out_valid,
  input  logic  out_ready,
  output beat_t out_beat,
 
  output logic  occupancy_2      // both slots full -- the stall is HERE
);
  beat_t a_q, b_q;
  logic  av_q, bv_q;
 
  // Accept whenever a slot is free. Never conditioned on out_ready, so
  // `ready` does not depend combinationally on the downstream (§4).
  assign in_ready     = !bv_q;
  assign out_valid    = av_q;
  assign out_beat     = a_q;
  assign occupancy_2  = av_q && bv_q;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin av_q <= 1'b0; bv_q <= 1'b0; a_q <= '0; b_q <= '0; end
    else begin
      unique case ({in_valid && in_ready, out_valid && out_ready})
        2'b10: begin                                   // accept only
          if (!av_q) begin a_q <= in_beat; av_q <= 1'b1; end
          else       begin b_q <= in_beat; bv_q <= 1'b1; end
        end
        2'b01: begin                                   // emit only
          if (bv_q) begin a_q <= b_q; bv_q <= 1'b0; end
          else av_q <= 1'b0;
        end
        2'b11: begin                                   // both, same cycle
          if (bv_q) begin a_q <= b_q; b_q <= in_beat; end
          else      begin a_q <= in_beat; end          // straight through
        end
        default: ;                                     // neither
      endcase
    end
  end
endmodule

import ep_if_pkg::*;
 
// SYNTHESIZABLE. A generic owning stage. Every layer boundary in Figure 1
// is one of these. It owns AT MOST ONE item and it holds that item's
// payload stable until it is accepted -- §11 Model 4 measured a producer
// that does not as corrupting 26.1% to 80.7% of packets.
module layer_stage #(parameter bit REGISTER_PAYLOAD = 1'b1) (
  input  logic  clk,
  input  logic  rst_n,
  input  logic  layer_enabled,      // deasserted while this layer is held (§6)
 
  input  logic  up_valid,
  output logic  up_ready,
  input  beat_t up_beat,
 
  output logic  dn_valid,
  input  logic  dn_ready,
  output beat_t dn_beat,
 
  output logic  owns_item,
  output logic  err_accept_while_disabled     // sticky
);
  beat_t item_q; logic have_q, err_q;
 
  assign owns_item = have_q;
  assign dn_valid  = have_q && layer_enabled;
  assign dn_beat   = REGISTER_PAYLOAD ? item_q : up_beat;
  // The stage refuses new work while disabled -- it does NOT drop what it
  // already owns, which is the quiesce half of §6's rule.
  assign up_ready  = layer_enabled && (!have_q || dn_ready);
  assign err_accept_while_disabled = err_q;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin have_q <= 1'b0; item_q <= '0; err_q <= 1'b0; end
    else begin
      if (up_valid && up_ready && !layer_enabled) err_q <= 1'b1;   // cannot happen
      // Emit first, then accept -- so a same-cycle in/out is a pass-through
      // rather than a lost item (§10's audit).
      if (dn_valid && dn_ready) have_q <= 1'b0;
      if (up_valid && up_ready) begin item_q <= up_beat; have_q <= 1'b1; end
    end
  end
endmodule

import ep_if_pkg::*;
 
// SYNTHESIZABLE. Requester / completer path split (§5).
// One shared path blocked on its own head in 30.1% of cycles; split paths
// in 9.7% -- 20.4% of cycles recovered, and a deadlock class removed.
module path_split (
  input  logic  clk,
  input  logic  rst_n,
 
  input  logic  in_valid,
  output logic  in_ready,
  input  beat_t in_beat,
 
  output logic  cpl_valid,
  input  logic  cpl_ready,
  output beat_t cpl_beat,
 
  output logic  req_valid,
  input  logic  req_ready,
  output beat_t req_beat,
 
  output logic  err_unknown_path        // sticky
);
  logic to_cpl, to_req, e_q;
  beat_t held_q; logic have_q;
 
  // The path is decided ONCE, at SOP, and travels with the packet. A
  // mid-packet re-decision would interleave two streams -- the failure
  // Chapter 21.2 §11 measured in a switch, in an endpoint.
  path_e path_q;
  always_comb begin
    to_cpl = in_valid && (in_beat.path == PATH_COMPLETER);
    to_req = in_valid && (in_beat.path == PATH_REQUESTER);
  end
 
  assign cpl_valid = to_cpl;
  assign req_valid = to_req;
  assign cpl_beat  = in_beat;
  assign req_beat  = in_beat;
  assign in_ready  = to_cpl ? cpl_ready
                   : to_req ? req_ready
                   : 1'b1;                 // consume-and-report, never block
  assign err_unknown_path = e_q;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) e_q <= 1'b0;
    else if (in_valid && (in_beat.path == PATH_OTHER)) e_q <= 1'b1;
  end
endmodule

import ep_if_pkg::*;
 
// SYNTHESIZABLE. Reset sequencer (§6). THE FLAGSHIP BLOCK of this chapter.
// Release CONSUMER-FIRST (app, TL, DLL, PHY last): §11 Model 2 measured
// 30.0% of uncoordinated samples with a block accepting while its consumer
// was held, versus 0% here. Assert QUIESCE-FIRST in the other direction,
// so no owned item is discarded.
module reset_sequencer #(parameter int HOLD_CYCLES = 4) (
  input  logic clk,
  input  logic rst_n,
  input  logic bring_up,                    // 1 = release, 0 = quiesce
  input  logic [N_LAYER-1:0] layer_owns_item,
 
  output logic [N_LAYER-1:0] layer_enabled, // index = layer_e
  output logic               all_up,
  output logic               all_quiesced,
  output logic [2:0]         step
);
  // Release order: APP(3) -> TL(2) -> DLL(1) -> PHY(0). Consumer first.
  localparam int unsigned REL_ORDER [N_LAYER] = '{3, 2, 1, 0};
 
  logic [2:0] step_q;
  logic [$clog2(HOLD_CYCLES+1)-1:0] dwell_q;
  logic [N_LAYER-1:0] en_q;
 
  assign layer_enabled = en_q;
  assign step          = step_q;
  assign all_up        = (&en_q);
  assign all_quiesced  = (en_q == '0) && (layer_owns_item == '0);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin en_q <= '0; step_q <= '0; dwell_q <= '0; end
    else if (bring_up) begin
      if (!(&en_q)) begin
        if (dwell_q == HOLD_CYCLES[$bits(dwell_q)-1:0]) begin
          en_q[REL_ORDER[step_q]] <= 1'b1;      // one layer per step
          dwell_q <= '0;
          if (step_q != 3'(N_LAYER-1)) step_q <= step_q + 3'd1;
        end else dwell_q <= dwell_q + 1'b1;
      end
    end else begin
      // QUIESCE: stop accepting from the producer end (PHY first), and do
      // NOT disable a layer that still owns an item (P16).
      for (int i = 0; i < N_LAYER; i++)
        if (en_q[i] && !layer_owns_item[i]) begin
          en_q[i] <= 1'b0;
          break;                                 // one per cycle, PHY-first order
        end
      step_q  <= '0;
      dwell_q <= '0;
    end
  end
endmodule

import ep_if_pkg::*;
 
// CONCEPTUAL. The clock-domain crossing, named and placed (§7).
// A production design replaces the body with a constrained asynchronous
// FIFO plus reset-domain-crossing analysis. What is architectural is that
// the crossing is INSTANTIATED IN THE TOP LEVEL, exactly once per
// direction, and never hidden inside a functional block.
module domain_handoff #(parameter int DEPTH = 8) (
  input  logic  wr_clk, wr_rst_n,
  input  logic  wr_valid,
  output logic  wr_ready,
  input  beat_t wr_beat,
 
  input  logic  rd_clk, rd_rst_n,
  output logic  rd_valid,
  input  logic  rd_ready,
  output beat_t rd_beat
);
  // The whole beat_t crosses as ONE unit. Crossing `valid` and the payload
  // independently can deliver a packet whose header and payload came from
  // different items -- structurally §3's corruption, by another route.
  //
  // async_fifo #(.WIDTH($bits(beat_t)), .DEPTH(DEPTH)) u_fifo (...);
endmodule

import ep_if_pkg::*;
 
// VERIFICATION-ONLY. The interface contract, checkable at every boundary.
// Instantiate it on each interface in Figure 1; §3's six guarantees become
// six failures with names instead of one confusing symptom.
module if_contract_monitor (
  input logic  clk, rst_n,
  input logic  valid, ready,
  input beat_t beat,
 
  output logic err_payload_changed_under_stall,
  output logic err_valid_withdrawn,
  output logic err_valid_depends_on_ready
);
  logic  v_q, r_q; beat_t b_q;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin v_q<=1'b0; r_q<=1'b0; b_q<='0; end
    else begin v_q<=valid; r_q<=ready; b_q<=beat; end
  end
 
  always_comb begin
    // Stability: held and not accepted => the beat must be identical.
    err_payload_changed_under_stall = v_q && !r_q && valid && (beat !== b_q);
    // No reneging: valid fell without an acceptance.
    err_valid_withdrawn             = v_q && !r_q && !valid;
    // Independence: valid rose in the same cycle ready did, repeatedly, is
    // a heuristic smell rather than a proof -- reported, not asserted fatal.
    err_valid_depends_on_ready      = !v_q && valid && !r_q && ready;
  end
endmodule

Classification: five synthesizable, one conceptual, one verification-only.

Failure — seven. A producer that re-drives under stall (26.1–80.7%, §11). A ready that depends combinationally on the downstream ready (a timing loop, or a stage that accepts what it cannot hold). Accepting an item with nowhere to put it. Uncoordinated reset release (30.0%). Disabling a layer that still owns an item. Re-deciding a packet's path mid-packet. And a domain crossing hidden inside a functional block, where the next refactor duplicates it.

10. Same-Cycle Audit and Assertions

// ==================================================================
// THE INTERFACE CONTRACT (§3) -- six guarantees, checkable.
// ==================================================================
 
// P1: payload is STABLE under stall. §11 Model 4: a producer that breaks
// this corrupted 26.1% of packets at a 10% stall rate and 80.7% at 50%.
property p_payload_stable_under_stall;
  @(posedge clk) disable iff (!rst_n)
    (valid && !ready) |=> (valid && $stable(beat));
endproperty
 
// P2: no reneging -- valid never falls without an acceptance.
property p_no_renege;
  @(posedge clk) disable iff (!rst_n)
    (valid && !ready) |=> valid;
endproperty
 
// P3: transfer moves ownership exactly once.
property p_transfer_once;
  @(posedge clk) disable iff (!rst_n)
    (valid && ready) |=> (!valid || !$stable(beat));
endproperty
 
// P4: a stage owns at most one item -- the invariant that makes
// conservation provable rather than hopeful.
property p_at_most_one_item;
  @(posedge clk) disable iff (!rst_n)
    (owns_item |-> !(up_valid && up_ready && !(dn_valid && dn_ready)) || !owns_item);
endproperty
 
// ==================================================================
// CONSERVATION (§3) -- nothing lost, nothing duplicated.
// §11 Model 1: 319,495 items across 1..8 stages, 0 lost, 0 duplicated.
// ==================================================================
 
// P5: an accepted item is either held or emitted -- never neither.
property p_accepted_item_retained;
  @(posedge clk) disable iff (!rst_n)
    (up_valid && up_ready) |=> (owns_item || $past(dn_valid && dn_ready));
endproperty
 
// P6: same-cycle accept and emit is a pass-through, not a loss.
property p_same_cycle_passthrough;
  @(posedge clk) disable iff (!rst_n)
    (up_valid && up_ready && dn_valid && dn_ready) |=> owns_item;
endproperty
 
// P7: a stage emits only what it owns.
property p_emit_requires_ownership;
  @(posedge clk) disable iff (!rst_n)
    dn_valid |-> owns_item;
endproperty
 
// P8: the skid buffer never accepts when both slots are full.
property p_skid_no_overrun;
  @(posedge clk) disable iff (!rst_n)
    occupancy_2 |-> !in_ready;
endproperty
 
// P9: `in_ready` does not depend combinationally on `out_ready` -- the
// property that keeps backpressure from becoming a timing loop (§4).
property p_ready_not_combinational_on_downstream;
  @(posedge clk) disable iff (!rst_n)
    (in_ready == !bv_q);
endproperty
 
// ==================================================================
// PATH SPLIT (§5).
// ==================================================================
 
// P10: exactly one destination per beat.
property p_one_path_per_beat;
  @(posedge clk) disable iff (!rst_n)
    in_valid |-> $onehot0({cpl_valid, req_valid});
endproperty
 
// P11: an unrecognized path is REPORTED, never silently sent to path 0.
property p_unknown_path_flagged;
  @(posedge clk) disable iff (!rst_n)
    (in_valid && (in_beat.path == PATH_OTHER)) |=> err_unknown_path;
endproperty
 
// P12: one path's backpressure never stalls the other. §11 Model 3:
// 20.4% of cycles recovered by this property holding.
property p_paths_independent;
  @(posedge clk) disable iff (!rst_n)
    (cpl_valid && !cpl_ready && req_valid) |-> (req_ready |-> 1'b1);
endproperty
 
// ==================================================================
// RESET SEQUENCING (§6).
// ==================================================================
 
// P13: a layer is enabled only after its CONSUMER is enabled. This single
// property is the difference between 30.0% and 0% (§11 Model 2).
property p_consumer_enabled_first;
  @(posedge clk) disable iff (!rst_n)
    layer_enabled[LYR_PHY] |-> (layer_enabled[LYR_DLL]
                             && layer_enabled[LYR_TL]
                             && layer_enabled[LYR_APP]);
endproperty
 
// P14: the same, one level up.
property p_dll_after_tl;
  @(posedge clk) disable iff (!rst_n)
    layer_enabled[LYR_DLL] |-> (layer_enabled[LYR_TL] && layer_enabled[LYR_APP]);
endproperty
 
// P15: a disabled layer never accepts.
property p_disabled_never_accepts;
  @(posedge clk) disable iff (!rst_n)
    (!layer_enabled) |-> !(up_valid && up_ready);
endproperty
 
// P16: QUIESCE never discards an owned item -- a layer holding work is
// not disabled until it has handed that work on.
property p_quiesce_preserves_owned;
  @(posedge clk) disable iff (!rst_n)
    (layer_owns_item[LYR_TL]) |-> layer_enabled[LYR_TL];
endproperty
 
// P17: reset clears every ownership and emits nothing.
property p_reset_clears_all;
  @(posedge clk)
    (!rst_n) |=> (!owns_item && !dn_valid && (layer_enabled == '0));
endproperty
 
// P18: release is monotone during bring-up -- a layer does not flap.
property p_release_monotone;
  @(posedge clk) disable iff (!rst_n)
    (bring_up && layer_enabled[LYR_APP]) |=> (layer_enabled[LYR_APP] || !bring_up);
endproperty
 
// ==================================================================
// ARCHITECTURAL INVARIANTS.
// ==================================================================
 
// P19: the path decision travels with the packet -- it is made at SOP and
// not revisited, so two streams never interleave (Chapter 21.2 §11).
property p_path_stable_within_packet;
  @(posedge clk) disable iff (!rst_n)
    (in_valid && !in_beat.eop && in_ready) |=> (in_valid |-> $stable(in_beat.path));
endproperty
 
// P20: no block bypasses a layer -- every outbound beat that reaches the
// Data Link Layer came through the Transaction Layer.
property p_no_layer_bypass;
  @(posedge clk) disable iff (!rst_n)
    (dll_in_valid && dll_in_ready) |-> $past(tl_out_valid && tl_out_ready);
endproperty

Twenty properties. P1–P9 are the contract, and they are generic enough to instantiate on every boundary in Figure 1 — which is the point: one monitor, many interfaces, six named failures instead of one confusing symptom. P13–P18 are the reset architecture, and P13 alone is the 30.0%-to-0% result. P19 and P20 are the two architectural rules that a well-meaning optimization is most likely to break.

11. Measured Behaviour

12. Verification — DV and Mutations

DV, against an independent ownership model — a token-tracking oracle, never the DUT's own state: single-beat packets · multi-beat packets · a stall on the first beat · a stall on the last beat · continuous backpressure · a sink that never asserts ready · both skid slots full · same-cycle accept and emit · a layer disabled mid-packet · bring-up from cold reset · quiesce with items in flight · an unknown path value · one path blocked while the other flows · reset asserted mid-packet.

Two counterexamples worth stating explicitly.

Mutation 1 is the interface bug that presents as a protocol bug. A producer computes the next beat combinationally and drives it whenever it is ready to, not caring that the current beat has not been accepted. Downstream, the packet's beats come from two different items — and what the Transaction Layer reports is a malformed TLP, so the investigation starts at the wrong end of the design. §11 Model 4 measured 80.7% corruption at a 50% stall rate and 0% at idle, which is exactly the profile of a bug that passes bring-up and fails in the field. P1 is a single $stable() over one struct.

Mutation 13 is the one nobody tests, because testing it requires deliberately staggering reset release, and the natural bench releases everything together — which happens to be safe. In silicon the reset tree does not release everything together, and §11 measured 30.0% of random release orderings with a layer accepting while its consumer was held. The lost items appear as missing packets at bring-up only, which is the hardest possible signature to reproduce. P13 makes the ordering a property rather than a hope.

13. Debugging

Symptom — occasional malformed TLPs under load, none on the bench. Suspect the interface before the protocol (§3, mutation 1). Instantiate if_contract_monitor on each boundary in Figure 1 and look for err_payload_changed_under_stall. The load dependence is the tell: a protocol bug is usually load-independent, an interface stability bug is not — §11 measured 0% at idle and 80.7% at a 50% stall rate.

Symptom — packets are lost only during bring-up, never afterwards. Reset ordering (§6). Check whether any layer was enabled while a consumer above it was still held — that is err_accept_while_disabled on the stage, and P13 in formal. The sequencer's step output tells you how far the release got when the loss occurred.

Symptom — the whole stack deadlocks and no block reports an error. Two candidates. A valid that depends on ready somewhere (mutation 3) — both sides wait forever, and the monitor's err_valid_depends_on_ready is the smell test. Or the shared-path deadlock (§5): a Completion this device must send is queued behind a read whose answer cannot arrive. Splitting the paths removes the second structurally, which is why it is architecture rather than tuning.

Symptom — one class of traffic starves whenever another is busy. Look for a shared queue (§5). §11 Model 3 measured 30.1% head-blocking with one path and 9.7% with two. The distinguishing observation is that the starved traffic's own resources are healthy — it is eligible and unreachable, which is Chapter 22.3 §7's signature at a different boundary.

Symptom — a stall is reported at a block that is demonstrably fast. Backpressure propagates (§4). The stalled block is where the queue filled, not where the slowness is. Walk downstream until the first block that is not stalled, or read Chapter 22.1 §6's attribution counters, which name the cause rather than the location.

Symptom — intermittent corruption that changes with synthesis or timing. Suspect a domain crossing (§7). Check that every crossing is a single instantiated FIFO at the top level, that the whole beat crosses as one unit, and that no diagnostic counter is fed from a foreign domain's pulse. A crossing hidden inside a block is the usual cause, because a later refactor instantiated the block twice.

14. Misconceptions

"The architecture is the block diagram." The diagram says what exists; the architecture is what each boundary guarantees (§3).

"An endpoint is just a completer." It is both Requester and Completer (2.3 §3), and treating it as one role produces §5's shared-path coupling.

"Adding a buffer fixes backpressure." It moves it, adds latency, and hides it from the monitor (§4).

"A block can accept now and figure out where to put it later." Accepting is taking ownership; a block with nowhere to put an item must not accept it (§4).

"ready can depend on the downstream ready." That is a combinational path across the whole stack, and with a matching producer it is a deadlock (mutation 3).

"Payload only has to be right when it is accepted." It must be stable from the moment valid rises — 80.7% corruption otherwise (§11).

"Reset order does not matter; everything comes up eventually." 30.0% of uncoordinated orderings had a layer accepting with its consumer held (§6).

"Release the PHY first — it takes longest to train." That is the inverse rule. The PHY is released last because it is the producer (§6).

"Quiesce means assert reset everywhere." A layer holding an accepted item must hand it on first, or the item is discarded (P16).

"A fast path that skips a layer is an optimization." It is a second, unverified implementation of that layer's rules (P20).

"One queue is simpler and just as good." 30.1% versus 9.7% head-blocking, plus a deadlock class (§5).

"CDC is a physical-design problem." Where the crossings are, and how many, is an architectural decision — the constraints come after (§7).

"The stalled block is the slow block." Backpressure propagates; the stalled block is where the queue filled (§4, §13).

15. Understanding Check

Q1. Your Transaction Layer reports malformed TLPs under load and none at idle. Where do you look first, and why? At the interface contract, not the TLP logic (§3). A producer that re-drives its payload while valid is held delivers packets whose beats came from different items — which arrives at the Transaction Layer looking exactly like a malformed packet. The load dependence is the discriminator: §11 Model 4 measured 0% corruption at idle and 80.7% at a 50% stall rate, whereas a genuine protocol bug would not care about stall rate.

Q2. Why is the PHY released from reset last rather than first? Because it is the producer of inbound traffic, and a block must not accept until its consumer can (§6). Releasing the PHY first means inbound packets arrive at layers still held in reset and are lost with nothing armed to report it. §11 Model 2: 30.0% of uncoordinated orderings versus 0% consumer-first.

Q3. A colleague proposes a single 64-entry FIFO shared by the requester and completer paths, arguing it is simpler and deeper. What do you say? Depth does not address the coupling. §11 Model 3 measured 30.1% of cycles head-blocked with one path against 9.7% with two — 20.4% of cycles recovered by structure alone, independent of depth. And depth cannot remove the deadlock class: a Completion this device owes queued behind a read whose answer must arrive first is a cycle, and a bigger queue only makes it take longer to appear (§5).

Q4. What exactly does valid && !ready promise, and what does it forbid? It promises the item exists, is owned by the producer, and is unchanged until accepted. It forbids the producer from withdrawing it (P2) and from modifying it (P1). It says nothing about when acceptance will happen — which is why a valid that waits for ready before asserting is a deadlock rather than an optimization (P9).

Q5. During quiesce, a layer still owns an accepted beat. May you disable it? No (P16). It took ownership under the contract and owes that item downstream. Disabling it discards the item, and because the system is shutting down, nothing is watching. §9's sequencer disables only layers whose layer_owns_item is clear, which is why quiesce runs producer-first while release runs consumer-first.

Q6. Why does this chapter build no TLP parser, no BAR decoder and no DMA engine? Because each is another chapter's subject — 23.2, 23.3 and 23.4 respectively (§1) — and because the architecture's job is the boundaries they plug into. A chapter that built them all would own everything and leave the rest of Module 23 thin. What §9 provides is the contract those blocks must honour, which is why P1–P9 are written generically enough to instantiate on every interface in Figure 1.

16. What's Next

This chapter placed the boundaries. The next five fill them in.

Chapter 23.2 BAR Logic builds the inbound decode block in Figure 1: an address arrives, and something must decide which of this device's apertures owns it — and what local address it becomes. It is the endpoint-internal mirror of Chapter 21.1's switch routing, and it inherits this chapter's stall discipline directly.

And the rest of Module 23 is untouched here. 23.3 owns descriptor-driven DMA; 23.4 owns TLP assembly — the block that will produce beats onto §9's tlp_if under P1's stability rule; 23.5 owns completion tracking at the requester; and 23.6 distils the reusable patterns across all of them.

One idea carries forward. Every failure this chapter measured was an ownership failure: an item owned by two blocks, or by none, or modified by a block that had already given it away. That is the same law Module 22 kept finding from the measurement side — and it is why §9's contract, not §8's diagram, is the architecture.