DLLP Types — The Packets That Are Not Transactions

Module 14 built one Link's reliability and named the packets that carry its control information exactly once, in passing: acknowledgement and retry are exchanged using DLLPs. Then it moved on, because the mechanisms were the subject.

They are not the only such packets, and they are not the only Link-local mechanism.

What is a Data Link Layer Packet, how is it different from a TLP, and what categories of Link-local control information does PCIe exchange using DLLPs?

1. The Verified Taxonomy

2. A DLLP Is Not a Small TLP

The most common misreading, and it is not a matter of size.

3. A DLLP Is Never Forwarded

Chapter 3.2 §2 established that reliability is per-Link. This is that fact seen from the packet side.

Endpoint A ──Link A── Switch Port X │ Switch │ Switch Port Y ──Link B── Root Complex

An acknowledgement DLLP arriving on Link A ends at Switch Port X. It is consumed there, by that port's reliability engine, and it retires that port's retained state for Link A.

If the Switch needs a corresponding action on Link B, it generates Link B's own control — from Link B's own state, at Link B's own timing, with Link B's own sequence identities (Chapter 14.5 §2).

4. The Receive Path

The figure's argument is the missing edge. There is no path from the DLLP classifier to the Transaction Layer engines — not a filtered one, not a disabled one, none. §6 is why that absence is the design rather than an omission.

5. RTL — DLLP Classifier and Dispatch

// SYNTHESIZABLE. Classify a decoded DLLP into a normalized category and
// deliver it to exactly one local control engine.
// The category set: NORMATIVE (section 1). The enum encoding, the port
// names, and the raw-type abstraction: ILLUSTRATIVE normalized metadata —
// NOT PCIe wire Type values.
package dllp_pkg;
 
  typedef enum logic [2:0] {
    DLLP_ACK         = 3'd0,
    DLLP_NAK         = 3'd1,
    DLLP_FC_INIT1    = 3'd2,
    DLLP_FC_INIT2    = 3'd3,
    DLLP_FC_UPDATE   = 3'd4,
    DLLP_PM          = 3'd5,
    DLLP_VENDOR      = 3'd6,
    DLLP_UNSUPPORTED = 3'd7    // not represented by this model
  } dllp_class_e;
 
  // Which local engine consumes each class. Separate from the class itself,
  // because "what kind of packet" and "who handles it" are different
  // questions — Chapter 11.7 section 7's separation, at this layer.
  typedef enum logic [1:0] {
    ENG_RELIABILITY = 2'd0,   // Module 14
    ENG_FLOWCTRL    = 2'd1,   // Chapter 15.2, Module 16
    ENG_POWER       = 2'd2,   // Chapter 15.5
    ENG_ERROR       = 2'd3
  } dllp_engine_e;
 
endpackage

import dllp_pkg::*;
 
module dllp_dispatch #(
  // Opaque decoded payload. The classifier does not interpret it; each
  // engine does, according to its own chapter.
  parameter int PAYLOAD_W = 32
) (
  input  logic clk,
  input  logic rst_n,
 
  // ---- Decoded DLLP in --------------------------------------------------
  input  logic                  in_valid,
  output logic                  in_ready,
  input  dllp_class_e           in_class,      // from the framing decoder
  input  logic [PAYLOAD_W-1:0]  in_payload,
 
  // ---- Local control engines -------------------------------------------
  output logic                  rel_valid,
  input  logic                  rel_ready,
  output logic                  fc_valid,
  input  logic                  fc_ready,
  output logic                  pm_valid,
  input  logic                  pm_ready,
  output logic                  err_valid,
  input  logic                  err_ready,
 
  output dllp_class_e           out_class,
  output logic [PAYLOAD_W-1:0]  out_payload,
  output dllp_engine_e          out_engine
);
 
  // ---- Class -> engine, decided ONCE -----------------------------------
  // A single enum decode. Mutual exclusion is structural: the four valids
  // below are decodes of one value, so no configuration can assert two.
  dllp_engine_e eng;
 
  always_comb begin
    unique case (in_class)
      DLLP_ACK, DLLP_NAK:                          eng = ENG_RELIABILITY;
      DLLP_FC_INIT1, DLLP_FC_INIT2, DLLP_FC_UPDATE: eng = ENG_FLOWCTRL;
      DLLP_PM:                                     eng = ENG_POWER;
      // Vendor-specific content is not interpreted by this model, and an
      // unrepresented class is reported. Neither is a protocol judgement —
      // "not represented by this model" is a statement about the design.
      default:                                     eng = ENG_ERROR;
    endcase
  end
 
  assign out_class   = in_class;
  assign out_payload = in_payload;
  assign out_engine  = eng;
 
  assign rel_valid = in_valid && (eng == ENG_RELIABILITY);
  assign fc_valid  = in_valid && (eng == ENG_FLOWCTRL);
  assign pm_valid  = in_valid && (eng == ENG_POWER);
  assign err_valid = in_valid && (eng == ENG_ERROR);
 
  // Backpressure comes from the SELECTED engine only. ANDing every ready
  // would stall a credit update behind a busy power-management engine that
  // has nothing to do with it.
  always_comb begin
    unique case (eng)
      ENG_RELIABILITY: in_ready = rel_ready;
      ENG_FLOWCTRL:    in_ready = fc_ready;
      ENG_POWER:       in_ready = pm_ready;
      default:         in_ready = err_ready;
    endcase
  end
 
  // NOTE WHAT IS ABSENT. There is no Transaction Layer output port on this
  // module. A DLLP cannot reach the Transaction Layer because there is no
  // wire by which it could — section 6. That is stronger than any filter.
 
endmodule

Classification: synthesizable (package: compile-time).

Architecture. One enum decode driving a one-hot valid. Mutual exclusion is structural — the four valids are decodes of a single value, so no state can assert two.

Contract. The framing decoder guarantees in_class is stable while in_valid is asserted. Each engine relies on receiving a DLLP only when its own valid is high, and on never seeing one another engine also received.

Failure — three. Independent per-engine comparisons rather than one enum decode allow two engines to accept the same packet if the selection ever becomes inconsistent. ANDing all the readies stalls every class behind the busiest engine. And adding a Transaction Layer output — the one the module deliberately does not have — reintroduces §6's failure with a wire.

Deliberately simplified: no framing, no CRC, no payload interpretation; the class arrives already decoded, because the encodings belong to later chapters.

6. Keeping DLLPs Out of the Transaction Layer

A design rule, and the reason it is structural rather than a check.

A DLLP must never enter Transaction Layer request routing.

7. Assertions

// SVA over dllp_dispatch. These assert the LOCAL dispatch contract and the
// layer separation of section 6. They assert nothing about DLLP format
// (Chapters 15.3-15.5), nothing about the acknowledgement algorithm
// (Module 14), and nothing about credit arithmetic (Chapter 15.2).
 
// P1: EXACTLY ONE ENGINE. Neither zero nor two.
property p_exactly_one_engine;
  @(posedge clk) disable iff (!rst_n)
  in_valid |-> ($countones({rel_valid, fc_valid, pm_valid, err_valid}) == 1);
endproperty
a_one_hot : assert property (p_exactly_one_engine);
 
// P2: THE LAYER-SEPARATION PROPERTY. No DLLP ever produces a Transaction
// Layer event. Bound across the boundary, because that is the only place
// section 6's rule is expressible — and a design that wired the two
// together fails here rather than in a debugging session.
property p_no_dllp_reaches_tl;
  @(posedge clk) disable iff (!rst_n)
  in_valid |-> !(tl_mem_valid || tl_cfg_valid || tl_cpl_valid || tl_msg_valid);
endproperty
a_layer_separation : assert property (p_no_dllp_reaches_tl);
 
// P3: class-to-engine mapping is exactly section 1's taxonomy.
property p_mapping_correct;
  @(posedge clk) disable iff (!rst_n)
  in_valid |-> ((in_class inside {DLLP_ACK, DLLP_NAK})       ? rel_valid :
                (in_class inside {DLLP_FC_INIT1, DLLP_FC_INIT2,
                                  DLLP_FC_UPDATE})           ? fc_valid  :
                (in_class == DLLP_PM)                        ? pm_valid  :
                                                               err_valid);
endproperty
a_mapping : assert property (p_mapping_correct);
 
// P4: an unrepresented class reaches ONLY the error path.
property p_unsupported_isolated;
  @(posedge clk) disable iff (!rst_n)
  (in_valid && (out_engine == ENG_ERROR))
    |-> (err_valid && !rel_valid && !fc_valid && !pm_valid);
endproperty
a_unsupported : assert property (p_unsupported_isolated);
 
// P5: dispatch and payload are stable while the selected engine stalls. A
// packet must not migrate between engines mid-offer.
property p_stable_under_stall;
  @(posedge clk) disable iff (!rst_n)
  (in_valid && !in_ready)
    |=> (in_valid && $stable({rel_valid, fc_valid, pm_valid, err_valid,
                              out_class, out_payload}));
endproperty
a_stable : assert property (p_stable_under_stall);
 
// P6: backpressure comes from the SELECTED engine only — an unrelated busy
// engine never blocks.
property p_ready_from_selected_only;
  @(posedge clk) disable iff (!rst_n)
  in_valid |-> (in_ready == ((rel_valid && rel_ready) || (fc_valid && fc_ready)
                          || (pm_valid  && pm_ready)  || (err_valid && err_ready)));
endproperty
a_selected_ready : assert property (p_ready_from_selected_only);
 
// P7: one accepted DLLP produces exactly one local control event.
// (in_count / out_count are testbench counters.)
property p_one_event_per_dllp;
  @(posedge clk) disable iff (!rst_n)
  (in_valid && in_ready) |-> (out_count + 1 <= in_count);
endproperty
a_conserved : assert property (p_one_event_per_dllp);

P2 is the property this chapter exists to make writable. §6's rule is normally taught and never checked — binding it across the layer boundary turns "a DLLP never enters the Transaction Layer" from a paragraph into something that fails a design. And note that with §5's structure it is vacuously true, which is the point: the property confirms the architecture rather than policing a filter.

P1 uses $countones == 1 rather than a mutual-exclusion check because zero is a failure too — a DLLP reaching no engine is dropped, and a dropped credit update stalls a Link permanently with nothing reporting it.

P6 exists because ANDing readies is the natural mistake. It looks conservative and produces a real coupling: an acknowledgement waiting behind a busy power-management engine delays retirement, which fills the replay buffer, which backpressures the Transaction Layer — a Link-wide stall caused by an unrelated engine.

8. Verification

Monitors observe: the decoded DLLP input, all four engine interfaces, and the Transaction Layer engine interfaces (for P2).

The scoreboard holds its own class-to-engine table, written from §1. It must not import dllp_pkg or call the design's decode.

Per category

• Each class in §1's taxonomy: ACK, NAK, InitFC1, InitFC2, UpdateFC, PM, vendor-specific. Verify the engine, the one-hot (P1), and the mapping (P3).
• All three Flow Control classes. Verify they reach the same engine — the class distinction matters to Chapter 15.2, not to dispatch.
• An unrepresented class. Verify the error path only (P4), and that the report says not represented by this model, not malformed per PCIe.

Stress and ordering

• Back-to-back DLLPs of different classes, every ordered pair.
• Alternating ACK and UpdateFC — the realistic steady-state mix.
• Each engine stalled in turn while a packet for it is offered. Verify stability (P5) and that other classes still flow (P6).
• The error path stalled with an unrepresented class offered. Verify it does not block represented ones.
• A long randomised run with independent per-engine backpressure, checking the conservation count (P7).
• Reset mid-offer.

Layer separation

• Drive every DLLP class while a TLP stream runs concurrently. Verify no Transaction Layer engine ever fires on a DLLP (P2).
• Confirm structurally, not only by simulation: the dispatcher has no Transaction Layer port, so review is the check and P2 is the confirmation.

Which mutation which check kills

9. Debugging

An ACK DLLP reaches the Transaction Layer's Completion engine

A layer-separation failure, and the first question is structural.

Does a wire exist from the DLLP path to that engine? If yes, that is the bug and no amount of decode fixing will make it safe (§6). If no, then the packet was classified as a TLP by the framing decoder — the fault is upstream of this chapter entirely, in whatever separates the two kinds.

What it looks like at the Completion engine. A packet with no correlation identity presented to correlation logic. A well-built engine reports an unexpected Completion (Chapter 13.2); a less careful one matches garbage against an outstanding entry and resolves the wrong read.

A Flow Control update is received but the transmit credits never change

Two boundaries, and the classifier is only the first.

Check fc_valid fired. If it did, dispatch worked and the fault is in the flow-control engine's decode of the update — which is Chapter 15.2's subject, and its most common failure is treating a cumulative value as a delta.

If it did not fire, check what out_engine said. A Flow Control class reaching the reliability engine is a mapping bug here (P3); reaching the error path means the class was not recognised at all.

A Switch forwards an ACK DLLP unchanged onto its other Link

A Link-local scope violation (§3), and it is invisible point to point.

The consequence is not subtle. The forwarded acknowledgement names a Link A sequence identity, which on Link B refers to a different packet chosen by coincidence (Chapter 14.5 §2). Link B's retained state is retired arbitrarily.

The observation: capture the same logical TLP on both Links and compare the acknowledgement traffic. Link B's acknowledgements must reference Link B's own sequence values. If they mirror Link A's, the Switch is relaying what it should be terminating.

An unrecognised DLLP wedges the receive path

The error path is not draining, or it does not exist.

Check err_ready. If the error path is stalled and in_ready is derived from it, an unrecognised DLLP blocks every subsequent one, including acknowledgements and credit updates — and the Link stops for a reason that has nothing to do with either.

This is why the error path is a real destination with a real handshake rather than a default that does nothing. A packet with nowhere to go stalls the port.

10. Common Misconceptions

• "A DLLP is a small TLP." Different layer, different scope, and it carries none of a TLP's transaction concepts — no address, no Requester ID, no Tag (§2).
• "DLLPs are routed end to end." They terminate at the port they reach. A Switch consumes one and generates its own (§3).
• "ACK and NAK are Transaction Layer packets." They are Data Link control and never produce a Transaction Layer event (Chapter 14.2 §2, P2).
• "A Flow Control Update carries application data." It carries credit information about a receive buffer (Chapter 15.2). No payload of any transaction is involved.
• "A DLLP has a Tag identifying a Request." A Tag is Transaction Layer correlation (Chapter 11.3 §6). Nothing at this layer identifies a Request.
• "Every DLLP eventually enters the Transaction Layer." None does — that is the layer boundary (§6).
• "DLLP type and TLP type share one semantic namespace." They are separate taxonomies at separate layers, and merging them is what lets a control packet reach transaction machinery (§6).
• "Flow Control DLLPs control replay-buffer occupancy." Replay storage is transmit-side reliability state (Chapter 14.4 §17). Credits describe the remote receive buffer. Different resources, different sides of the Link.
• "An ACK DLLP means a Memory Read completed." It means a neighbour received a packet (Chapter 14.2 §4). A read completes when a Completion returns.
• "Power Management DLLPs are software configuration transactions." They are Link-local control. Configuration accesses are TLPs and are routed by ID (Chapter 11.7 §4).

11. Understanding Check

12. What's Next

This chapter established the taxonomy and the boundary: four categories of Link-local control, none of them routed, none of them reaching the Transaction Layer — enforced by a receive structure with no path between the two domains rather than by a check somewhere downstream.

Chapter 15.2 — Flow Control Updates takes the category this chapter only named: what a credit update actually carries, why its values are cumulative counters rather than deltas, and the modular arithmetic needed to turn one into "how much capacity was just returned" — the same wrap problem Chapter 14.5 solved for sequence identities, on a different counter.

Chapters 15.3 and 15.4 then own the acknowledgement and retry packets' formats and timing, and 15.5 the power-management packets. Module 16 owns the credit system itself — the pools, the consumption, and the transmitter's eligibility decision.

The idea to carry forward: a DLLP is addressed to a neighbour, not to a destination — and the cleanest way to guarantee it never goes further is to build a path that cannot take it there.