Data-Centre Processors

Chapter 22.3 modelled an accelerator package as a bandwidth graph. A server processor breaks that model — because most of what makes it hard is not bandwidth at all.

1. The One-Sentence Model

A data-centre processor is a latency-and-ownership system. Its hard problems are coherent state, tail latency, request identity, ordering, locality, progress, and failure isolation — and for every one of them, adding bandwidth to a die boundary does nothing. What a die boundary changes is where a request's ownership lives and how long it takes to get there.

Which is why 22.3's method does not transfer. There, the question was which edge binds? Here it is: what happens to a request's identity, its ordering guarantees, and its owner when it crosses a die — and what breaks when any of those is mishandled by a few cycles.

2. What This Chapter Owns

Four things are genuinely new here, and none of them is about throughput:

What a die boundary costs a cache miss (§7–§8) — and why throughput can be unchanged while tail latency moves sharply.

Request identity across a boundary (§10–§12). The flagship failure: an ID reused a few cycles too early, and a late response corrupting a different request. This is the most consequential bug class in the chapter and the RTL is short.

Ordering domains (§13–§14) — how to avoid serialising a link to satisfy an ordering rule that was never required.

And failure isolation (§17–§19). A server cannot reset the package to clear one link's problem, so recovery must preserve outstanding semantic work — which is a much stronger requirement than "the link comes back."

3. Sourcing and Evidence Date

4. Claim-vs-Evidence — UCIe and Server Processors

Three readings.

Row 2 is the trap. It is true, well-documented, and says nothing about the interface — and the inference from row 2 to the missing row is the single most common false claim about UCIe in servers.

Rows 3 and 4 are real and are about other things: a test chip and a roadmap statement, neither of which is a server product.

And the honest position is that server processors are the segment where the architectural case for a standard boundary is weakest (§20) — because the most latency-critical boundaries are exactly where co-design wins (22.2 §10).

5. Conceptual Server Partitioning

Three things to read.

The whole diagram is CONCEPTUAL (§3). It is a partitioning a server could have — no vendor topology is depicted.

The llc node owns coherence, and no die boundary changes that (§9). Coherence is a semantic property of the fabric; a link carries messages between the fabric's participants and defines none of their meaning.

And every boundary here has a different latency sensitivity. Compute-to-LLC is on the critical path of every miss (§8); management is on no critical path at all. Standardising them is not one decision — which is 22.2 §11's layering argument applied per boundary.

6. Why Servers Differ From Accelerators

Two readings, and the second is the chapter's organising idea.

Row 3 is the difference in one line. An accelerator that loses bandwidth runs slower; a server that loses latency stalls a thread, and a thread stalled long enough is a service-level failure regardless of aggregate throughput.

And rows 6 and 7 are why the same die boundary is a harder problem here. Coherent state must survive a link event (14.2), and the system cannot take the recovery action — a reset — that would most easily restore the link. §17 is that constraint developed properly.

7. What a Die Boundary Costs

ILLUSTRATIVE latency decomposition. Symbolic terms — no UCIe or vendor
latency figure appears anywhere in this chapter (§3).
 
  MONOLITHIC last-level miss:
    t_core_issue + t_llc_lookup + t_mc + t_mem + return path
 
  PARTITIONED across a die boundary:
    t_core_issue + t_llc_lookup
                 + t_xing        <-- die crossing, request direction
                 + t_mc + t_mem
                 + t_xing        <-- die crossing, response direction
                 + return path
 
  The crossing is paid TWICE per round trip, and it is on the critical
  path of every miss that leaves the tile.

Three properties.

The crossing is paid twice, which is the term most often forgotten when a partitioning is sketched.

It applies to every miss that leaves the tile — so its impact scales with miss rate, not with bandwidth.

And it is a fixed additive term, which means it hurts the fastest paths most. A miss that was already long absorbs it proportionally; a miss that would have been short is affected far more in relative terms — which is why §8's throughput and tail-latency results diverge.

8. Worked — Throughput Unchanged, Tail Latency Worse

ILLUSTRATIVE. All values symbolic and internal to this example (§3).
 
  Monolithic miss latency          L  = 100 units
  Die crossing, each direction     c  =  15 units
  Partitioned miss latency         L' = 100 + 2(15) = 130 units
 
  THROUGHPUT, with enough outstanding requests to cover latency:
    required outstanding  ~=  target_rate x latency        (21.5 §26)
    if the design supports >= 130/100 x the previous outstanding count,
    sustained throughput is UNCHANGED.
    -> a bandwidth benchmark shows no regression.
 
  TAIL LATENCY, for a single dependent chain of 6 misses:
    monolithic    6 x 100 = 600
    partitioned   6 x 130 = 780        <-- +30%
 
  And a thread doing dependent pointer-chasing cannot hide ANY of it:
  there is no second request to overlap with.

Four readings.

The bandwidth benchmark is clean and the workload is 30% slower. 21.5 §15's three metrics: utilisation, delivered payload and operation rate answer different questions, and only the third sees this.

Latency hiding requires independent work. A dependent chain has none — so the outstanding-request argument that rescues throughput does nothing for the tail.

Which workloads care is therefore an architectural input, not a detail. Pointer-chasing, lock acquisition and synchronisation are dependent by nature; streaming and scan workloads are not, and the same partitioning is nearly free for one and expensive for the other.

And this is why §6's row 1 matters: an accelerator team measuring bandwidth would approve this partitioning, and a server team measuring p99 latency would reject it — from the same silicon.

9. Coherence Crosses; the Link Does Not Define It

The coherence protocol owns the semantic state. A die-to-die link transports its messages. Officially published UCIe material describes a transport stack (19.1); nothing I could reach establishes that UCIe defines CPU cache coherence, and this chapter does not claim it.

What the transport must nonetheless guarantee for coherence to remain correct — 22.2 §14's four requirements, restated as they apply here:

And the practical consequence is that a link carrying coherence has a much narrower operating envelope than one carrying bulk data — the same physical link, a much stricter contract above it.

10. Illustrative — the Request Record

// ILLUSTRATIVE ONLY. A request crossing a die boundary. Every field exists
// because omitting it produces a specific failure below.
typedef struct packed {
  logic [ID_W-1:0]    req_id;         // the reusable identity
  logic [GEN_W-1:0]   generation;     // WHICH USE of that identity — §11
  logic [ADDR_W-1:0]  addr_token;     // opaque; not an architectural address
  logic [3:0]         op_class;       // read / write / coherent / sync / ctrl
  logic [DIE_W-1:0]   src_die;        // for return routing AND for debug
  logic [DOM_W-1:0]   order_domain;   // which ordering rules apply — §13
  logic [EPOCH_W-1:0] cfg_epoch;      // 21.6 §10
  logic               requires_ack;   // must this complete? — §9
} server_req_t;

Architecture. Identity, ordering scope, and configuration context on every request, so a response can be attributed without relying on arrival order.

State. One record per outstanding request in the tracker (§15).

Cycle/event behaviour. Built at issue; generation is captured from the per-ID generation counter at that moment (§12).

Contract. req_id is reusable; generation is what makes a specific use of it identifiable. 20.4 §21's lifecycle rule — and §11 is what happens without it.

Failure. Without generation, §11. Without order_domain, either the link is globally serialised (§14) or ordering is violated. Without cfg_epoch, a response from before a reconfiguration is applied under the new rules (21.6 §11).

DV/debug. These fields are exactly what a scoreboard correlates on (20.4) and what a silicon trace event carries (21.7 §16).

11. Wrong RTL — Identity Reuse

// WRONG — an ID is returned to the free pool when the request is CONSIDERED
// done locally, with no generation and no proof the peer has finished with it.
// This is a plausible design, not a cartoon: it is correct whenever responses
// cannot be late.
always_ff @(posedge clk or negedge por_n) begin
  if (!por_n)                 id_free_q <= '1;
  else begin
    if (issue_fire)           id_free_q[issue_id]    <= 1'b0;
    if (local_timeout_fire)   id_free_q[timeout_id]  <= 1'b1;   // <-- the bug
    if (resp_fire)            id_free_q[resp_id]     <= 1'b1;
  end
end

The failure, step by step.

Five properties, and this is the most serious failure in the chapter.

It is silent. No assertion fires, no error is logged, no link event occurs. A core receives a plausible response for a request it never made, and the wrong data enters the coherent domain.

Every component behaved as designed. The timeout freed a resource; the allocator reused it; the tracker matched an ID to a live entry. 21.4 §17's layer-local correctness, producing data corruption.

It requires a late response, which is exactly what a recovery produces. So the bug is dormant until the first real link event (14.2) — and then it appears as memory corruption with no link error to point at.

A server makes it far more likely than an accelerator does, because servers run with large outstanding counts and long-tailed latencies: the ID space wraps quickly and late responses are normal.

And the local timeout is not the root cause — the missing generation is. A timeout is a legitimate mechanism; freeing an identity that a peer may still reference is the defect.

12. Corrected RTL — Generation Tagging

// CORRECTED. The identity may be reused; the GENERATION makes each use
// distinguishable. A late response carrying a stale generation is detectable
// rather than mis-attributed.
logic [GEN_W-1:0] id_gen_q  [NUM_IDS];
logic             id_live_q [NUM_IDS];
 
always_ff @(posedge clk or negedge por_n) begin
  if (!por_n) begin
    for (int i = 0; i < NUM_IDS; i++) begin
      id_gen_q[i]  <= '0;
      id_live_q[i] <= 1'b0;
    end
  end else begin
    // ALLOCATE: bump the generation as the id is handed out, so the new use is
    // distinguishable from every previous one. Wrapping is acceptable ONLY if
    // GEN_W is wide enough that a response cannot outlive a full wrap — that
    // is a design obligation, not an assumption (21.4 §34).
    if (alloc_fire) begin
      id_gen_q[alloc_id]  <= id_gen_q[alloc_id] + GEN_W'(1);
      id_live_q[alloc_id] <= 1'b1;
    end
 
    // RETIRE on an observed response, or on an explicit reconciliation. A
    // timeout may retire the ENTRY but the generation still moves on, so a
    // straggler cannot match.
    if (retire_fire)  id_live_q[retire_id]  <= 1'b0;
    if (timeout_fire) id_live_q[timeout_id] <= 1'b0;
  end
end
 
// Acceptance is gated on BOTH identity and generation. This one expression is
// the whole fix.
logic resp_matches;
assign resp_matches = id_live_q[resp_id] && (resp_gen == id_gen_q[resp_id]);
 
logic resp_stale;
assign resp_stale = resp_fire && !resp_matches;
 
// Stale responses are COUNTED, not silently dropped — the count is what
// distinguishes "a straggler arrived" from "responses are being lost"
// (21.4 §46's rejection-taxonomy argument).
logic [31:0] stale_resp_cnt_q;
always_ff @(posedge clk or negedge por_n) begin
  if (!por_n)          stale_resp_cnt_q <= '0;
  else if (resp_stale) stale_resp_cnt_q <= stale_resp_cnt_q + 32'd1;
end
 
// MANDATORY. English: a response is applied ONLY if its identity is live and
// its generation matches. Fires at the cycle §11's corruption would occur.
a_no_stale_response_applied: assert property (
  @(posedge clk) disable iff (!por_n)
    resp_apply_fire |-> (id_live_q[resp_id] && (resp_gen == id_gen_q[resp_id]))
);

Architecture. A per-ID generation counter, bumped at allocation, with acceptance gated on the pair.

State. NUM_IDS generation counters and live bits.

Cycle/event behaviour. The generation advances at allocation, so it moves on even when a timeout — rather than a response — retired the previous use. That is what makes the straggler detectable.

Contract. GEN_W must be wide enough that a response cannot outlive a full generation wrap. That is derived from the maximum possible response delay and the allocation rate — a design obligation, not a guess (21.4 §34's width argument).

Failure. A generation bumped at retire rather than at allocate leaves a window where a timed-out entry and its reallocation share a generation — the same bug with extra steps.

DV/debug. stale_resp_cnt is the silicon-visible evidence. A nonzero count after a recovery is expected and drains; a count that keeps rising means responses are being generated against identities that no longer exist — which is a different investigation (21.4 §49's stale-rejection reading).

13. Ordering Domains

A server has real ordering requirements and a link that satisfies them globally is a link that has been serialised for nothing.

And the rule is 21.6 §16's: the ordering graph is sparse and defaults to unordered. Only sourced dependencies are edges.

14. Wrong — Global Serialisation

// WRONG — one sequence number for all traffic, completions forced in order.
// It is "safe", it is easy to argue for, and it destroys the link.
assign may_complete = (resp_seq == expected_seq_q);   // GLOBAL order

Three consequences.

One slow response blocks every unrelated completion behind it. A memory access to a slow region stalls independent accesses that were ready — head-of-line blocking (21.5 §45) built into the completion path rather than into a queue.

It converts a latency problem into a throughput problem. Outstanding requests cannot retire, so the outstanding pool fills, so issue stalls — and the measured effect is a bandwidth collapse whose cause is an ordering decision.

And it is usually adopted to satisfy a rule nobody stated. 21.6 §15's false violation report: "B completed before A" is only a violation if a sourced dependency exists — and serialising to make an unfounded report go away is the most expensive possible response.

The correct form is per-domain, exactly as 21.6 §14 writes it: order within order_domain, no assertion across domains.

15. Illustrative — Per-Die Outstanding Tracker

// ILLUSTRATIVE ONLY (§10). Per-die outstanding accounting. Small, passive, and
// it answers "is this die the problem?" without a trace.
module outstanding_tracker #(
  parameter int NUM_DIES = 4,
  parameter int CNT_W    = 16
) (
  input  logic                clk,
  input  logic                por_n,
  input  logic                diag_clear,
  // ACCEPTED, not offered — §16 is the bug this input name guards against
  input  logic                accept_fire,
  input  logic [DIE_W-1:0]    accept_die,
  input  logic                complete_fire,
  input  logic [DIE_W-1:0]    complete_die,
  output logic [CNT_W-1:0]    outstanding_q  [NUM_DIES],
  output logic [CNT_W-1:0]    max_outstanding_q [NUM_DIES],
  output logic [CNT_W-1:0]    completed_q    [NUM_DIES]
);
 
  // Signed next-state, computed per die, so a simultaneous accept and complete
  // on the SAME die nets to zero instead of one update being lost (19.5 §14).
  logic signed [CNT_W:0] next_outstanding [NUM_DIES];
 
  always_comb begin
    for (int d = 0; d < NUM_DIES; d++) begin
      next_outstanding[d] = $signed({1'b0, outstanding_q[d]})
                          + $signed({{CNT_W{1'b0}},
                              (accept_fire   && (accept_die   == DIE_W'(d)))})
                          - $signed({{CNT_W{1'b0}},
                              (complete_fire && (complete_die == DIE_W'(d)))});
    end
  end
 
  always_ff @(posedge clk or negedge por_n) begin
    for (int d = 0; d < NUM_DIES; d++) begin
      if (!por_n || diag_clear) begin
        outstanding_q[d]     <= '0;
        max_outstanding_q[d] <= '0;
        completed_q[d]       <= '0;
      end else begin
        outstanding_q[d] <= next_outstanding[d][CNT_W-1:0];
        if (next_outstanding[d] > $signed({1'b0, max_outstanding_q[d]}))
          max_outstanding_q[d] <= next_outstanding[d][CNT_W-1:0];
        if (complete_fire && (complete_die == DIE_W'(d)))
          completed_q[d] <= completed_q[d] + CNT_W'(1);
      end
    end
  end
 
  // MANDATORY. English: outstanding can never go negative — a completion with
  // no matching acceptance is an invented transaction (21.4 §4).
  a_outstanding_non_negative: assert property (
    @(posedge clk) disable iff (!por_n) (next_outstanding[0] >= 0)
  );
 
  // MANDATORY. English: outstanding never exceeds the architectural limit.
  a_outstanding_bounded: assert property (
    @(posedge clk) disable iff (!por_n)
      (outstanding_q[0] <= CNT_W'(MAX_OUTSTANDING_PER_DIE))
  );
 
endmodule

Architecture. Per-die outstanding count, high-water mark and completion count — three registers per die that localise a stall to a destination.

State. 3 × NUM_DIES counters.

Cycle/event behaviour. Maintained as a signed next-state expression per die, so an accept and a completion landing on the same die in the same cycle net correctly rather than one being lost to two sequential non-blocking assignments.

Contract. The increment is on acceptance, not on valid — §16.

Failure. Two if statements writing outstanding_q[d] in the same cycle is the classic simultaneous-event bug: the last assignment wins and one event vanishes, so the count drifts and the high-water mark is meaningless.

DV/debug. max_outstanding is the cheap silicon instrument (21.7 §22): a die whose high-water never approaches its limit was never the constraint, which eliminates it without any rate analysis. And a rising outstanding with a static completed localises a stall to one die in a single read.

16. Wrong Counter — Increment on Valid

// WRONG — counts offered requests, not accepted ones.
if (req_valid) outstanding_q <= outstanding_q + 1;   // no `&& req_ready`

Worked. A request is held valid for 6 cycles awaiting ready. The counter increments 6 times for one request.

Four properties.

The error scales with backpressure, so the count is most wrong exactly when the system is most loaded — and that is when it is being read.

It never comes back down correctly, because completions decrement once per request. The count drifts upward monotonically and eventually pins at its limit.

Every conclusion drawn from it is wrong in the same direction: the die looks saturated, max_outstanding looks pinned, and an investigation opens into a die that is fine.

And it is 21.5 §20's offered-versus-accepted distinction and 21.6 §23's held-valid monitor bug — the same defect appearing in a performance counter, in a monitor, and in a tracker. The fix is one term: req_valid && req_ready.

17. Failure Isolation

A server cannot reset the package to clear one link's problem. Other tenants, other cores and other devices are still running, and outstanding coherent work must survive — a lost request is a hang, not a retry (§9).

Which makes recovery a much stronger requirement than "the link comes back":

And the fourth row is where the debug chapter and the product requirement coincide: the reset that would most easily restore service is also the action that destroys the evidence — and in a server you frequently cannot take it anyway.

18. Fault and Recovery Domains

Two properties.

The domains are nested but their recovery actions are not. A link event should not escalate to a coherence-domain action, and a design where it does has effectively made the package the only fault domain.

And the coherence row is the constraint that shapes the rest. 14.2's epoch reasoning: a re-established link must either carry the previous agreement's outstanding work forward or explicitly reconcile it (21.4 §23) — silently forgetting it is a hang, not a recovery.

19. Degraded Operation

When a resource degrades rather than fails, a server has four options and they are not equivalent.

And the reporting requirement is the part most often missed. A processor running in a degraded configuration must make that visible, or every performance measurement taken against it is computed with the wrong denominator — 21.7 §38's trace exactly: 96.8% of active read as 58% of requested, and weeks spent on a transfer path that is fine.

This chapter states no vendor's degradation policy (§3); the requirement to expose the active configuration is generic.

20. Where a Standard Boundary Fits — and Where It Does Not

And the pattern is 22.2 §9's unchanged: the decisive argument is an organisational boundary, not a technical one — and in servers the latency-critical boundaries and the organisationally-crossable boundaries are almost disjoint, which is the honest architectural summary of this chapter.

21. What Public Material Cannot Tell You

22. Common Misconceptions

"Every chiplet server CPU uses UCIe." §3, §4: multi-die is widely documented and says nothing about the interface; server die-to-die interfaces predate UCIe.

"UCIe defines CPU coherence." §9: the coherence protocol owns the semantic state; a link transports its messages and defines none of their meaning.

"Bandwidth is the only server metric." §8: throughput can be unchanged while a dependent chain is 30% slower, and only the operation-rate metric sees it.

"More outstanding requests always help." §8: they hide latency only when there is independent work. A dependent chain has none — and more outstanding IDs makes §11's reuse window arrive sooner.

"A remote die is automatically a NUMA node." §21: software-visible locality classes are an architectural choice; they do not map one-to-one onto die boundaries by default.

"Recovery can clear outstanding processor work." §17: a lost coherent request is a hang. Outstanding identities and coherence state must survive, or be explicitly reconciled.

"Global ordering is safer." §14: it converts a latency problem into a throughput collapse, usually to satisfy an ordering rule nobody stated.

"A multi-tile package proves standard interoperability." §4: it proves partitioning. Interoperability requires a die crossing an organisational boundary.

23. Understanding Check

24. Summary

Five things.

A server is a latency-and-ownership system (§1, §6). Bandwidth is rarely the binding constraint; a stalled thread is a service failure regardless of aggregate throughput.

A die crossing is paid twice per round trip, on every miss that leaves the tile (§7) — and it is invisible to throughput benchmarks while costing 30% on dependent chains (§8).

Identity must be generation-tagged (§12). An ID freed by a timeout and reallocated before a late response arrives completes the wrong request with the wrong data, silently — the most serious failure in the chapter, and the fix is one counter and one comparison.

Ordering is per-domain and defaults to unordered (§13–§14). Global serialisation converts a latency problem into a throughput collapse, usually to satisfy a rule nobody stated.

And recovery must preserve outstanding semantic work (§17), because the package reset that would fix the link is unavailable — which makes epoch reconciliation a product requirement rather than a debug nicety.

On the evidence: server processors are widely documented as multi-die, and that establishes nothing about the interface (§4). No shipping server processor with UCIe die-to-die is established from sources I could reach — and architecturally, servers are the segment where the latency-critical boundaries and the organisationally-crossable ones are almost disjoint (§20).