IO Chiplets

The system die in the previous chapter's figure did a lot of quiet work. Every memory request from every compute die passed through it, along with all host and peripheral traffic — and it was drawn as a single box labelled with three words. That box is this chapter's subject, and the label most people reach for is misleading. An I/O chiplet is routinely described as "the die with the PHYs on it", which is roughly as useful as describing a CPU as "the die with the pins". The PHYs are the part you can point at. The architecture is what sits behind them.

1. The Die That Owns the System's Middle

Start from function rather than contents.

A compute chiplet executes work. Something else has to decide where that work's memory requests go, which die receives an incoming host transaction, what order concurrent operations become visible in, and who is told when something fails. In a partitioned system those responsibilities have to live somewhere — and the die holding the external interfaces is frequently where they land, because it is already the point everything passes through.

A compute chiplet executes work; the I/O die usually coordinates how that work reaches memory and the outside world.

That coordination role is what makes the "just PHYs" description wrong. A PHY converts signals; it does not decide anything. The functions that make a collection of dies behave like one machine — routing, address decoding, ordering, arbitration, interrupt delivery, management, and often boot and control — are decisions, and they concentrate here. In many designs the I/O die is the harder die to architect, even though the compute die has more transistors and gets the marketing.

2. What It Might Contain — and Why That Is a Decision

An I/O chiplet may hold external-facing controllers (PCIe, network, storage, display or other peripheral interfaces), the SerDes and PHY circuitry those need, CXL-facing logic where the architecture uses it, memory controllers, the system fabric and its routing, address decode, interrupt and management logic, boot and control functions, and the die-to-die interfaces to every compute and accelerator die.

No I/O chiplet contains all of these, and the specific set is architectural rather than definitional. Two things follow, and they pull in different directions.

The external interfaces are the obvious contents and the least architecturally interesting: they are largely determined by what the product must plug into. The fabric, decode, ordering, and management functions are the interesting part, because each one is a responsibility that was implicit inside a monolithic die and now has an owner and a location. Putting it here is a choice with consequences; putting it elsewhere is a different system.

3. Why Separate I/O From Compute At All

Four distinct arguments, which are worth keeping separate because they apply unevenly.

Different process economics. Compute-dense logic and external interface circuitry do not necessarily gain equally from the same process. Separating them lets each be manufactured where it makes sense rather than where the larger block wanted to be. How much this is worth depends entirely on the design — it is a reason, not a guarantee.

Reuse across compute configurations. A stable I/O die can potentially serve several compute arrangements — one compute die, two, four — without redesign, because its job does not change with how much compute sits behind it.

Independent evolution. External interface standards and compute microarchitecture move on different clocks. A new generation of an external interface does not necessarily imply a new compute design, and vice versa. Separating them lets each be revised on its own schedule — which is only true if the die-to-die interface between them stays stable across those revisions.

Product segmentation. A common compute die paired with different I/O capability produces different products without touching the compute design.

Note what these have in common: none is about size or yield. Those arguments applied to any partition, and Module 1 covered them. These are specific to I/O being the part of the system whose requirements are set by the outside world rather than by the workload.

4. Architectural State Moves With the Boundary

This is where the chapter's real content sits, and it is the part the "PHY collection" framing hides completely.

Depending on the architecture, an I/O die may own:

• Routing and address decode — knowing which target a given address belongs to, and how to reach it. This makes the die the component that knows the system's memory map.
• Memory controller state — outstanding requests, scheduling, refresh and timing management for whatever memory it fronts.
• Ordering points — the place where concurrent operations acquire a defined order. If the architecture promises that two operations become visible in some order, something has to enforce it, and a central die is a natural place.
• Coherency coordination — where the system is coherent across dies, tracking or directing that traffic.
• Interrupt routing and management — delivering events to the right destination, plus telemetry, error aggregation, and control.
• Boot and control — often the die that brings the system up, since it is the one connected to the outside world.

Each of those changes something concrete: the failure domain (what stops working when this die has a problem is the system, not one subsystem), reset sequencing (this die typically has to be alive before others are useful), the verification boundary (system-level properties are now enforced by a specific die), and where bottlenecks appear.

Separating I/O can centralise system responsibility just as easily as it modularises the system.

That sentence is worth sitting with, because it is the opposite of what "modular architecture" suggests. Partitioning made the compute modular. It made the I/O die singular.

5. Where the Memory Controllers Go

Chapter 2.2 raised this from the compute side. Here it is from the other end, because the choice defines what kind of die this is.

Controllers on the compute dies. Each compute die owns its own memory path, so accesses to local memory never cross a link, and memory bandwidth grows as compute dies are added. The costs: controller circuitry is replicated in every compute die, the compute die inherits whatever that interface needs from the process, reaching another die's memory becomes a cross-die operation with non-uniform latency, and the compute die is now harder to reuse because it carries a memory personality.

Controllers on the I/O die. Memory exists once, is reached uniformly by every compute die, and the compute die stays purely about compute — which makes it markedly more reusable. The cost is severe and specific: every memory access now traverses a die-to-die link, putting that link's bandwidth and latency directly into the most performance-critical path most systems have. It also makes the I/O die a bandwidth convergence point for the entire system.

Neither is correct in general. What decides it is the workload's memory behaviour, how much link bandwidth the packaging can actually provide, and how much the design values compute-die reuse. Notice that the second option buys reuse with latency — a trade that is invisible if you think of the I/O die as passive plumbing.

6. One Compute Die, Three Products

A concrete hypothetical. Take one compute die design and pair it with three I/O dies:

• Variant A — a modest external interface set, aimed at a cost-sensitive product.
• Variant B — substantially more host and network connectivity for a system that moves large volumes of external traffic.
• Variant C — a different memory interface capability, for a product whose workloads are memory-capacity bound rather than I/O bound.

Three products, one compute design, no compute redesign. The differentiation moved from how much compute to how the compute is connected:

System differentiation can move from redesigning compute to recombining compute with different connectivity.

State the limits plainly, because this is where the idea is usually oversold. Each pairing is a distinct system that needs its own package design, its own system-level validation, and its own compatibility confirmation — the compute die's die-to-die interface must genuinely satisfy all three I/O dies, including their different bandwidth and ordering demands. And if Variant C's memory capability changes the traffic profile the compute die sees, "unchanged compute die" may still mean "compute die operating outside the envelope it was characterised for". Reuse is a design property that must be maintained, not a consequence of having drawn a boundary.

7. The Bottleneck the Shape Predicts

Look at Figure 1 again. Everything converges on one node. That is the efficiency of the design and also its exposure, and a serious architect reads both from the same picture.

A centralised I/O die can become the system's limit through several distinct mechanisms:

• Memory bandwidth convergence. If it fronts memory, the aggregate demand of every compute die lands on its controllers. Adding compute dies adds demand without adding supply.
• Die-to-die bandwidth. Each link has finite capacity, and traffic from all compute dies must fit through the ones attached here.
• Arbitration. Multiple requesters competing for shared resources need arbitration, and the policy determines whether one heavy client can starve another.
• Internal routing congestion. Its own fabric must carry the crossbar of traffic between every source and destination.
• Ordering and coherence pressure. If it is the ordering point, operations that must be ordered serialise here — and serialisation is a throughput limit that more bandwidth cannot remove.
• Latency. Every access through it pays the crossing, and that latency is in the path of ordinary work.
• External link saturation. Its outward interfaces have their own limits.

Centralisation removes duplication but concentrates traffic and failure responsibility in one die.

That is the honest summary of the I/O chiplet trade, and it is why "centralise the I/O" is a decision to be sized rather than a default to be assumed. The die that makes the system coherent is also the die that can make it slow, and the one whose failure is not survivable.

8. What This Means for Verification

The I/O die concentrates system behaviour, so it concentrates system-level verification. The questions are protocol-agnostic and largely about the die's role as a decision maker:

• Is traffic routed to the correct endpoint, and does the response return to the correct requester?
• Are the architecture's ordering requirements maintained when requests from several compute dies interleave?
• What happens under sustained backpressure — does throttling propagate correctly to every source without deadlock or loss?
• What happens when one compute die resets while others keep issuing traffic?
• What happens when the I/O die itself resets — can anything recover, or is that a system-level event by construction?
• Are interrupts and error reports delivered to the right destination?
• Can one heavy client starve another through the shared fabric or arbitration?
• Can memory traffic and die-to-die traffic deadlock against each other through shared resources?
• Does each supported product configuration behave correctly, including asymmetric or partially populated arrangements?

The fifth question is the one that distinguishes this die from the others. For a compute die, "what if it resets?" has a scoped answer. For the die that owns routing, decode, and ordering, it usually does not.

9. Common Misconceptions

10. Understanding Check

11. Summary and What Comes Next

An I/O chiplet is best understood by what it decides rather than what it connects. Its external interfaces are largely dictated by the product; its routing, address decode, ordering, arbitration, interrupt, and management functions are architecture, and they typically make it the coordination point that lets separate dies behave as one system. The arguments for separating it — differing process economics, reuse across compute configurations, independent evolution from compute, and product segmentation — are specific to I/O being the part of the system whose requirements come from the outside world.

Two consequences dominate. Memory-controller placement is the single choice that most changes the system: on compute dies it gives local, scalable bandwidth at the cost of replication, non-uniform latency, and compute-die reuse; on the I/O die it gives uniformity and reuse at the cost of putting a link in every memory access. And centralisation concentrates: everything converging on one die makes it the most likely bottleneck — through bandwidth convergence, arbitration, congestion, ordering serialisation, and latency — and the die whose failure is a system event rather than a subsystem one.

Memory has now appeared in every argument in this chapter as a destination reached through something. That deserves its own treatment:

• 2.4 — Memory Chiplets — what changes when memory capacity and bandwidth are supplied by separately manufactured dies in the package, and why capacity and bandwidth are not the same product dimension.

Browse the full path on the UCIe tutorials index.