Layer Interactions

Every layer can be individually correct and the endpoint can still be broken. That is not a paradox — it is the normal condition of integrated hardware, and it is what this chapter is about.

Chapter 4.5 followed one payload through a chain of handshakes. This chapter is the advanced counterpart: what happens when two boundaries, three state machines, several buffers, and asynchronous status updates all operate on the same clock edges. The failures here are not layer bugs. They are composition bugs, and they are the ones that survive unit verification, appear late, reproduce rarely, and consume weeks.

Layer interaction is distributed state-machine composition. Local correctness does not compose automatically.

1. The Endpoint as One Wiring Problem

Put both boundaries on the table at once, with data and status together — the first time this curriculum models both.

Illustrative architecture RTL — not UCIe normative signal naming. FDI and RDI are the specification's boundaries; these reduce them to payload, backpressure, and status.

// ---- Protocol <-> Adapter (conceptually FDI) --------------------------
logic [W-1:0] fdi_data;      // Protocol drives
logic         fdi_valid;     // Protocol drives
logic         fdi_ready;     // Adapter drives
 
// ---- Adapter <-> PHY (conceptually RDI) ------------------------------
logic [W-1:0] rdi_data;      // Adapter drives
logic         rdi_valid;     // Adapter drives
logic         rdi_ready;     // PHY drives
 
// ---- Status, travelling UPWARD ---------------------------------------
logic         phy_link_up;        // PHY -> Adapter
logic         link_operational;   // Adapter -> Protocol

Read the direction of every signal, because the asymmetry is the design. Payload descends and is driven by the upper side. Backpressure ascends and is driven by the lower side. Status ascends and is progressively abstracted — the PHY tells the Adapter about the physical link; the Adapter tells the Protocol Layer about the session, which is not the same fact (Chapter 5.4 §6).

Two boundaries with three drivers each, plus two status paths, is already enough structure for every hazard in this chapter.

2. The Combinational Ready-Chain

Chapter 4.2 warned that threading readiness combinationally creates a long path. At two boundaries the risk sharpens into an actual cycle.

// DANGEROUS COMPOSITION — three plausible lines that form a loop.
assign fdi_ready = rdi_ready && !adapter_block;   // (1) up-ready from down-ready
assign rdi_valid = fdi_valid && fdi_ready;        // (2) forward valid from ready
assign rdi_ready = phy_ready && !rdi_valid;       // (3) "I can accept if you aren't sending"

Each line is individually defensible. Line (1) passes backpressure through. Line (2) forwards a beat the same cycle it is accepted. Line (3) is a receiver optimisation that sounds sensible.

Together they form a dependency cycle:

fdi_ready → rdi_valid → rdi_ready → fdi_ready

fdi_ready depends on rdi_ready; rdi_ready depends on rdi_valid; rdi_valid depends on fdi_ready. The signals depend on each other within the same cycle with no register breaking the loop.

What happens depends on the toolchain and is uniformly bad: elaboration may reject it, synthesis may report a combinational loop, static timing may find an unconstrainable path, and simulation may settle to an arbitrary value or oscillate — meaning it can appear to work in one tool and fail in another. That is worse than a clean failure, because it survives to silicon in the wrong hands.

Note precisely what is not the lesson: ready need not be registered, and combinational ready is legitimate and common. The lesson is that the forward and backward directions must not both derive from each other. Line (3) is the culprit — deriving a receiver's readiness from a sender's valid closes the ring.

3. Breaking It With Elastic Buffering

Storage in the Adapter breaks the cycle by construction, because fdi_ready then depends on local occupancy rather than on rdi_ready directly. Two entries — not the one entry of Chapter 4.5 — buy something specific.

// Two-entry elastic buffer at the Adapter. Breaks the ready dependency and
// absorbs one cycle of late backpressure without losing throughput.
logic [W-1:0] d0_q, d1_q;     // d0 = output stage, d1 = skid stage
logic         v0_q, v1_q;
logic [1:0]   occ_q;          // 0, 1, or 2
 
// Upstream readiness depends ONLY on local state and the downstream ready.
assign fdi_ready = (occ_q != 2'd2) || rdi_ready;
assign rdi_valid = v0_q;
assign rdi_data  = d0_q;
 
always_ff @(posedge clk) begin
  if (!rst_n) begin
    v0_q <= 1'b0; v1_q <= 1'b0; occ_q <= 2'd0;
    d0_q <= '0;   d1_q <= '0;
  end else begin
    unique case ({fdi_valid && fdi_ready, rdi_valid && rdi_ready})
      2'b10: begin                            // fill only
        if (!v0_q) begin v0_q <= 1'b1; d0_q <= fdi_data; end
        else       begin v1_q <= 1'b1; d1_q <= fdi_data; end
        occ_q <= occ_q + 2'd1;
      end
      2'b01: begin                            // drain only
        if (v1_q) begin d0_q <= d1_q; v1_q <= 1'b0; end
        else      begin v0_q <= 1'b0; end
        occ_q <= occ_q - 2'd1;
      end
      2'b11: begin                            // both: advance, occupancy holds
        if (v1_q) begin d0_q <= d1_q; d1_q <= fdi_data; end
        else      begin d0_q <= fdi_data; end
      end
      default: ;                              // neither
    endcase
  end
end

Architecture. Decouples the two boundaries so a PHY stall does not reach the Protocol Layer combinationally in the same cycle.

State. d0_q/v0_q is the beat presented downstream; d1_q/v1_q absorbs one more; occ_q is the occupancy the upstream readiness is computed from.

Cycle behaviour. The 2'b11 arm is the throughput-critical one: filling and draining on the same edge advances the pipeline with occupancy unchanged, so back-to-back transfers sustain full rate. The 2'b10-into-full case cannot occur because fdi_ready is low then — which is exactly what §4 asserts.

Why two entries rather than one. With one entry, fdi_ready = !v0_q || rdi_ready still contains rdi_ready in the common case, so the path is shortened but not removed, and a downstream stall arriving late in a cycle propagates upward immediately. With two, the buffer can accept while full-minus-one regardless of rdi_ready, so one cycle of late backpressure is absorbed rather than forwarded. That is the difference between a design that meets timing and one that does not.

Contract. Upstream: acceptance means retention. Downstream: a presented beat is stable until taken.

Failure/DV. Get the 2'b11 arm wrong — for instance reloading d0_q from fdi_data while v1_q holds an older beat — and payloads reorder. That is a correctness bug the occupancy assertions below will not catch; it needs a scoreboard.

4. Boundary Assertions

// Illustrative integration properties — conceptual, not UCIe normative.
 
// SAFETY: never accept when full and not draining.
property p_no_accept_when_full;
  @(posedge clk) disable iff (!rst_n)
    (occ_q == 2'd2 && !rdi_ready) |-> !fdi_ready;
endproperty
assert property (p_no_accept_when_full);
 
// SAFETY: occupancy stays in range.
property p_occ_bounded;
  @(posedge clk) disable iff (!rst_n) occ_q <= 2'd2;
endproperty
assert property (p_occ_bounded);
 
// SAFETY: never present a beat we do not hold.
property p_no_valid_when_empty;
  @(posedge clk) disable iff (!rst_n) rdi_valid |-> (occ_q != 2'd0);
endproperty
assert property (p_no_valid_when_empty);
 
// SAFETY: payload stable while the downstream stalls.
property p_stable_while_stalled;
  @(posedge clk) disable iff (!rst_n)
    (rdi_valid && !rdi_ready) |=> (rdi_valid && $stable(rdi_data));
endproperty
assert property (p_stable_while_stalled);

These prove the boundary neither loses nor fabricates beats. They do not prove ordering, and they do not prove progress — both of which need different tooling (§9, §10).

5. A Cycle Trace Across Two Stalls

Follow four payloads with the PHY stalling. This is architectural sequencing, not a latency claim.

Two things to extract. Cycle 2 is backpressure reaching the source — and note it took two cycles after the PHY stalled, because the buffer absorbed first. Cycle 3 is the simultaneous case doing real work: a drain, an advance, and a fill on one edge, with occupancy unchanged. A design that mishandles that arm loses half its throughput or reorders.

6. Status Racing Data

Now the advanced hazard, and the one most likely to be missing from a design.

Consider a single cycle in which: link_operational is high, the FDI handshake fires (payload accepted), and the layer below detects a fault in that same cycle.

Ask the question the design must answer: who owns that payload?

The Protocol Layer has already retired it — by the handshake contract, the transfer occurred and it may free its buffer. If the Adapter treats the fault as invalidating everything in flight, the payload exists nowhere. Nothing reports an error; a transaction vanishes during a recovery transition that otherwise looks clean.

The design must therefore define an ordering between data acceptance and status transition, and the safe rule is:

Once an upper-layer handshake fires, the lower layer owns that payload and must either transport it or resolve it through a defined recovery path. A status change may not retroactively un-accept it.

Here is the wrong implementation, and it is a natural thing to write:

// WRONG: link loss silently destroys accepted payloads.
always_ff @(posedge clk) begin
  if (!link_operational)
    buf_valid_q <= 1'b0;        // whatever was accepted just disappeared
  else if (fdi_valid && fdi_ready)
    buf_valid_q <= 1'b1;
end

Why it fails. A payload accepted at cycle N is destroyed at cycle N+1 if the link drops. The Protocol Layer believes it was delivered and is waiting for a completion that will never come — a timeout much later, far from the cause. It is intermittent by nature: it only manifests when a fault lands within a cycle or two of an acceptance, so it survives fault injection that is not deliberately timed.

The corrected shape keeps accepted work visible and resolves it explicitly:

// CORRECTED: accepted payload is retained and resolved through a defined path.
logic accepted_pending_q;
 
always_ff @(posedge clk) begin
  if (!rst_n) begin
    accepted_pending_q <= 1'b0;
  end else if (fdi_valid && fdi_ready) begin
    accepted_pending_q <= 1'b1;               // we own it now
  end else if (transferred_onward || flushed_by_recovery) begin
    accepted_pending_q <= 1'b0;               // resolved, one way or the other
  end
end

The distinction that matters. flushed_by_recovery is not the same as silently clearing: it is an explicit, defined resolution that the recovery path takes responsibility for — and which can be reported upward so the Protocol Layer learns its transaction was discarded rather than waiting forever. Losing a payload and deliberately discarding it with notification are different behaviours, and only the second is acceptable.

What UCIe specifies for this case in each mode belongs to the recovery chapters; the architectural invariant — accepted work must be resolved, never vaporised — is what this chapter asserts.

7. Reset Is Three Different Facts

Chapter 5.4 §6 said bring-up has three owners. Here is what that means cycle by cycle, with all three state machines side by side.

Three distinct facts: flops initialised, physical link usable, negotiated transport usable. They become true at different times, in that order, and a design that conflates any two of them injects traffic into a link that cannot carry it.

// Illustrative properties distinguishing the three.
 
// Protocol must not transact merely because reset was released.
property p_no_fdi_accept_before_adapter_ready;
  @(posedge clk) disable iff (!rst_n)
    (adp_state_q != ADP_READY) |-> !(fdi_valid && fdi_ready);
endproperty
assert property (p_no_fdi_accept_before_adapter_ready);
 
// Adapter must not transmit merely because it left reset.
property p_no_rdi_tx_before_phy_ready;
  @(posedge clk) disable iff (!rst_n)
    !phy_link_up |-> !(rdi_valid && rdi_ready);
endproperty
assert property (p_no_rdi_tx_before_phy_ready);

The pair is worth having together: the first catches Protocol jumping the gun on the Adapter, the second catches the Adapter jumping the gun on the PHY, and either alone leaves half the sequencing unchecked.

8. Status CDC as an Integration Failure

Chapter 5.3 covered CDC mechanics. Here is the system consequence, which is what makes it worth revisiting.

Suppose phy_link_up crosses a domain boundary and is sampled without proper synchronisation. The Adapter's state machine may briefly see it asserted while it is metastable or transiently mis-sampled. What follows is a cascade:

1. The Adapter leaves WAIT_PHY and enters NEGOTIATE.
2. It begins a parameter exchange over a link that is not actually ready.
3. It may capture garbage capability values from a partner that is not responding.
4. It either falls back — or, worse, reaches READY with a nonsense operating point.

The root cause is a single unsynchronised bit at the PHY boundary. The symptom is a rare bring-up failure with corrupted negotiation state, which will be investigated as a negotiation bug or a partner problem, several layers away from the cause.

Two integration rules follow. Synchronise the single-bit status before any state machine consumes it, and — per 5.3 — never independently synchronise a multi-bit capability bus, because bits settling on different cycles produce a value that never existed. Capture such a bus only when a synchronised control bit says it is stable.

// The FSM must consume the synchronised status, never the raw signal.
property p_fsm_uses_synchronised_status;
  @(posedge clk) disable iff (!rst_n)
    (adp_state_q == ADP_NEGOTIATE) |-> phy_link_up_sync_q;
endproperty
assert property (p_fsm_uses_synchronised_status);

As always: this proves the protocol around the crossing, not metastability safety. Structural CDC analysis remains required.

9. Latency Mismatch Between Data and Its Metadata

The same class of bug as 5.2's CRC misalignment, now at a layer boundary.

Suppose an integrator pipelines the data path by two stages for timing, and the valid by one. From that point, valid for beat B arrives alongside data for beat A. Every transfer is now mismatched, and the corruption is systematic rather than occasional.

The defence is structural rather than vigilance:

// Bundle related fields so they cannot be pipelined apart.
typedef struct packed {
  logic [W-1:0] data;
  logic         sop;      // start of transport unit
  logic         eop;      // end of transport unit
} bundle_t;
 
bundle_t bundle_q;
logic    bundle_valid_q;
 
always_ff @(posedge clk) begin
  if (!rst_n) begin
    bundle_valid_q <= 1'b0;
  end else if (stage_ready) begin
    bundle_q       <= bundle_in;     // all fields move together
    bundle_valid_q <= bundle_in_valid;
  end
end

Related metadata must move as one timing object. If data, sop, and eop are separate signals, a later timing fix can pipeline one and not the others. If they are one struct, that mistake is not expressible.

For DV, this is a scoreboard property rather than a neat SVA: each accepted bundle must appear downstream with its own metadata intact. An assertion comparing sop/eop against a reference tag is more reliable than trying to express the relationship temporally.

10. Backpressure Is Resource Exhaustion, Not a Wire

The sharpest formulation this curriculum can offer, and it changes how waveforms are read.

When the far end stalls, fdi_ready does not fall. What happens is that buffers fill, and only when finite storage is exhausted does readiness drop. Trace it concretely with an Adapter FIFO of depth 4 and a six-cycle PHY stall, with the Protocol Layer offering every cycle:

Backpressure is resource exhaustion propagating through time, not a signal propagating through wires.

Three consequences. Bursts are absorbed — a stall shorter than the buffer depth never reaches the source. The delay is proportional to storage, so deeper buffers hide longer stalls and take longer to signal. And the source learns late, which is why a design must not assume it can react to a downstream problem immediately.

For waveform reading: an occupancy trace that rises to full and stays there means the downstream is blocked; one that stays near zero while work is pending means the upstream is not offering. That single distinction resolves most integration stalls.

11. Deadlock Is a Cyclic Dependency

Everything so far has been about single-direction flow. Deadlock needs a cycle, and layered flow control creates them easily.

A generic and realistic shape:

1. The Protocol Layer will not free a request resource until it receives the matching response.
2. Responses arrive on a path whose buffer is full of outbound requests awaiting transmission.
3. The outbound path cannot drain because the remote side is not accepting.
4. The remote side is not accepting because its Protocol Layer is in the same state — waiting for responses.

Draw it as a wait-for graph:

Protocol waits on → Adapter buffer space Adapter waits on → transmit capacity to remote Remote waits on → its own response resource Remote Protocol waits on → responses from us which waits on → our Adapter buffer space

The dependency returns to its origin. Nothing is broken; nothing progresses.

The architectural defence is to break the cycle structurally — commonly by ensuring responses cannot be blocked behind requests, through separate resources or separate paths for the two classes. The specific mechanism a design uses is an architecture decision, and what UCIe defines for it belongs to the flow-control chapters. What belongs here is the diagnostic method:

To find a deadlock, do not look for a broken block. Build the wait-for graph and look for a cycle.

12. Safety Assertions Pass During Deadlock

An observation that surprises people the first time and is worth internalising.

In the deadlock above, every safety property in this chapter still holds. No buffer overflows. No payload changes while stalled. Occupancy stays in range. Nothing is accepted while full. Nothing transmits while the link is down. All green — and the system is doing nothing forever.

That is because safety properties say bad things never happen; deadlock is the absence of good things happening. That is a liveness failure, and safety assertions are structurally incapable of detecting it.

The practical tools are different:

• Progress watchdogs in the design or testbench (§13).
• Bounded progress assertions under explicit assumptions (§14).
• Coverage that work actually completes, not merely that states were reached.
• Formal liveness with fairness assumptions — powerful and requiring care, because an unconstrained eventually is usually either vacuous or unprovable.

A test suite of only safety assertions can pass completely on a design that hangs.

13. A Progress Watchdog

// Conceptual progress watchdog -- illustrative, not UCIe normative behaviour.
parameter int unsigned WD_W = 16;
 
logic [WD_W-1:0] no_progress_q;
logic            work_pending, progress_made;
 
assign work_pending  = (occ_q != '0) || fdi_valid;
assign progress_made = (fdi_valid && fdi_ready)
                    || (rdi_valid && rdi_ready)
                    || state_advanced;
 
always_ff @(posedge clk) begin
  if (!rst_n) begin
    no_progress_q <= '0;
  end else if (progress_made || !work_pending) begin
    no_progress_q <= '0;                       // progress, or nothing to do
  end else if (no_progress_q != {WD_W{1'b1}}) begin
    no_progress_q <= no_progress_q + 1'b1;     // saturate rather than wrap
  end
end
 
assign stalled_alarm = (no_progress_q == {WD_W{1'b1}});

Architecture. Converts a silent hang into an observable event. Without it, "stopped" and "still working" look identical from outside.

State. One counter, cleared by any forward progress or by having no work — the !work_pending term matters, otherwise an idle link raises a false alarm.

Cycle behaviour. Saturates rather than wrapping; wrapping would silently restart the wait, defeating the purpose.

DV. The watchdog is testable in its own right: assert that it clears whenever progress occurs, and that it asserts its alarm after the threshold with work pending and no progress. Those are cheap safety properties about a liveness detector — a useful pattern.

14. Bounded Progress, Stated Honestly

The assertion that catches deadlock must state its assumptions, or it is not meaningful:

// BOUNDED LIVENESS -- valid only under the stated assumptions:
//   * the link stays operational,
//   * the downstream becomes ready at least once within the window,
//   * no persistent error is being handled.
// Without these, failure to progress is legitimate, not a bug.
property p_accepted_work_advances;
  @(posedge clk) disable iff (!rst_n || !link_operational || error_active)
    (occ_q != '0) |-> ##[1:PROGRESS_LIMIT] (rdi_valid && rdi_ready);
endproperty
assert property (p_accepted_work_advances);

Note the disable iff doing real work: it excludes exactly the conditions under which non-progress is correct. An engineer who omits those terms will get failures during legitimate recovery and will disable the assertion — which is worse than not having written it.

Choosing PROGRESS_LIMIT is a design judgement: long enough to cover legitimate stalls, short enough to catch a hang before a test ends. Too tight produces noise; too loose never fires.

15. Integration Debug

A payload never arrived. Walk both boundaries and the status path — the first no localises it:

1. Did the FDI handshake fire — fdi_valid && fdi_ready on the same edge?
2. Did Adapter occupancy increment?
3. Did the payload reach the head of the buffer?
4. Did rdi_valid assert?
5. Did the PHY accept — rdi_ready high on that edge?
6. Did link state change between acceptance and transmission? (§6)
7. If recovery ran, was the accepted payload transported or explicitly flushed — and was the flush reported?
8. Did the receive side accept it?
9. Are data and metadata aligned at every stage? (§9)
10. Did backpressure eventually clear?

The system hangs. Different questions entirely:

1. Which block is waiting?
2. What exact resource is it waiting for — space, credit, a response, a state transition?
3. Who owns that resource?
4. What event returns it?
5. Does the chain of "waits for" return to the start? (§11)

The two lists are not interchangeable, and choosing the right one is most of the skill. Missing acceptance is a data-path question; missing progress is a dependency question — and running the first checklist on a deadlock wastes hours confirming that everything is individually fine.

16. Common Misconceptions

17. Understanding Check

18. Summary — and the End of Module 5

Layer interaction is distributed state-machine composition, and local correctness does not compose automatically. Every failure in this chapter is a disagreement between two correct layers rather than a defect inside one.

The hazards, in order of how often they bite. Combinational ready-chains close into loops when forward and backward directions derive from each other; elastic buffering breaks them by deriving upstream readiness from local occupancy, and two entries rather than one absorb a cycle of late backpressure. Status races data: a fault in the same cycle as an acceptance must not un-accept it — accepted work is transported or explicitly resolved with the discard reported, never vaporised. Reset is three facts — flops initialised, physical link usable, negotiated transport usable — true at different times, in that order. Status CDC failures are system failures: one unsynchronised bit lets a state machine advance early and capture garbage capability, presenting as a rare, corrupted bring-up several layers from its cause. Data and metadata must be bundled, or a later timing fix pairs every beat with the wrong qualifier.

Two formulations are worth keeping permanently. Backpressure is resource exhaustion propagating through time, so occupancy — not ready — is what a waveform should be read for. And deadlock is a cycle in the wait-for graph, not a broken block: every safety assertion stays green while the system does nothing, because safety cannot observe the absence of progress.

That closes Module 5. The stack is now understood as hardware — what each layer owns, and how their state machines and buffers compose cycle by cycle. The next constraint is not logical at all. It is the physical structure the entire link rides on:

• 6.1 — Organic Substrates — what a package substrate physically is, why bump count is not routability, and how package geometry ends up choosing your RTL parameters.

Browse the full path on the UCIe tutorials index.