Multi-Die Systems

Monolithic-SoC Limitations ended on a claim rather than a picture: multi-die architecture trades internal monolithic coupling for explicit inter-die boundaries. That sentence is easy to nod at and hard to use, because it does not say what the resulting system actually is. This chapter builds that picture. It is the first concrete architectural lesson of the multi-die world, and everything later in this curriculum — chiplet taxonomy, packaging, the UCIe stack itself — assumes the mental model established here. One idea carries the whole chapter: the moment a connection crosses from one die to another, it stops being wiring and becomes a contract.

1. What Changes When One Die Becomes Several

Start with the two definitions, because the terminology gets used loosely.

• A die is one independently fabricated piece of semiconductor silicon — one object that came off a wafer and was tested as a unit.
• A multi-die system is a product in which two or more dies cooperate to implement one larger logical system. Not two chips that happen to share a board: dies that were partitioned out of what could have been a single design, and that must work together to be the product.

The logical system often barely changes. The same compute, the same accelerator, the same I/O, doing the same work in roughly the same arrangement. What changes is where the physical boundaries fall — and therefore what kind of engineering problem each connection is:

Inside one die, communication is a wiring and interconnect problem. Across dies, communication is an interface problem.

That is the pivot the whole chapter turns on. A wire between two blocks on one die is an implementation detail; the designer controls both ends, the timing environment is shared, and nothing needs to be negotiated. The same connection between two dies has two independently manufactured ends that may have been designed by different teams, built on different processes, and powered up at different times.

This is also why a multi-die system is not merely "several chips in a package". A motherboard carrying a CPU, a GPU, a network controller, and DRAM is physically multi-chip, and that is a different thing: those are separately specified products connected by board-level interfaces. What this chapter describes is package-scale integration — dies placed close together and connected through the package, which permits far shorter and denser connections than a board can offer, at lower energy per bit and higher bandwidth density. Those connections are still fundamentally different from on-die wires. Better than board-level, and not as free as staying on one die.

2. Partitioning — Deciding What Must Cross

Turning a monolithic design into a multi-die system means choosing where the physical boundaries go. Real designs are partitioned along axes such as compute versus I/O, repeated compute tiles, accelerator versus host, separating memory controllers or their PHYs, technology-node boundaries, or boundaries chosen so that a die can be reused across products.

There is no single correct axis — it depends on the system. What is general is the consequence:

Partitioning determines what must cross a die boundary.

And what crosses determines nearly everything else that follows: the bandwidth the interface must sustain, how much latency the design can tolerate on those paths, how complex the protocol has to be, how many physical connections and bumps are needed, how much verification the boundary demands, and what the package must physically route. A partition is not a drawing exercise. It is the decision that sets the requirements for every interface the product will then have to build.

3. Logical Modularity Is Not Physical Partitionability

This is the insight that separates an architect from someone reading a block diagram.

It is tempting to assume a clean RTL module boundary is a natural die boundary. It often is not. A boundary can be immaculate in the source tree and terrible in silicon if what crosses it carries enormous bandwidth, sits inside a tight latency loop, involves fine-grained coherence traffic, or contains many timing-sensitive control signals with tight clock coupling. None of that is visible in the module hierarchy — it lives in the traffic, not the structure.

The converse is also true and less obvious. A boundary that looks untidy in RTL may be perfectly good physically if the traffic across it can be regularised — aggregated into a well-defined, flow-controlled interface with predictable behaviour, even if the internal structure on either side is messy.

Logical modularity and physical partitionability are related, but not equivalent.

The useful question is never "where are the clean module boundaries?" It is "where is the traffic thin, latency-tolerant, and regular enough to survive being turned into an interface?"

4. Why the Boundary Needs an Interface, Not Longer Wires

The naive move is to imagine extending the on-die bus across the gap — same signals, just longer. That does not work, and understanding why is the point.

An on-die connection carries implicit assumptions: both ends share a timing environment, signals arrive within a known window, nothing is lost, both ends powered up together, and if something is not ready the designer knew about it at design time. A die boundary invalidates all of those, because the two ends are now independently manufactured objects.

So the boundary needs a defined interface, and the questions it has to answer are exactly the assumptions that just disappeared:

• How is data represented, and how wide is the link?
• When may the sender transmit — and how does the receiver push back when it cannot accept more?
• How are errors detected, and what happens when one occurs?
• How is ordering preserved, if the system depends on it?
• How are reset and bring-up sequenced across two independent dies?
• What happens if one side simply is not ready yet?

Every one of those had an implicit answer inside a single die. Across a boundary each needs an explicit one, agreed by both sides in advance. That agreement is the interface contract.

5. Why Interfaces Become Protocols

Watch the escalation, because this is how a bundle of wires turns into something with a specification.

A designer connecting two blocks locally thinks in signals: data, address, valid, ready, response. Across a die boundary, physical reality intrudes. There are far fewer connections available than an on-die bus would use, so data may be serialised or mapped onto lanes. Lanes need framing so the receiver knows where a unit of data starts. Independent ends need flow control so the sender does not overrun the receiver. Physical links can have errors, so detection and some form of retry or recovery may be needed. The link must be brought up before it can be used, which implies initialisation and state machines. Some interfaces add lane repair or remapping so a single bad connection does not kill the link, and power-state changes on either side have to be coordinated rather than assumed.

Be careful how strongly to read that list — not every conceivable die-to-die link needs every one of those mechanisms, and the specifics vary enormously with distance, packaging, and requirements. What is general is the reason they show up at all:

Protocols appear because boundaries create failure modes and coordination problems that raw wires do not solve.

Later chapters will show one particular set of answers to these questions. The point here is that the questions are forced by the boundary, before any specification is chosen.

6. Clocking, Reset, and Errors Stop Being Assumptions

Three implicit properties of a single die become explicit engineering across a boundary.

Clocking. A large monolithic die already has many clock domains, so multiple clocks are not new. What is new is that the two sides may have independent clock sources with no fixed phase relationship, so a crossing may be asynchronous or mesochronous and the interface logic must manage that difference. Do not over-generalise — die-to-die links vary, and not all are asynchronous. The durable point is that a die boundary breaks the assumption that both sides share one coordinated clocking environment, and whatever the actual arrangement, it now has to be stated rather than assumed.

Reset and bring-up. On one die, reset sequencing is complex but internal. Across dies it becomes a protocol concern: which die powers up first, which side initiates link bring-up, how one die learns that the other is ready, what happens when one side resets while the other keeps running, and whether the link can recover without resetting the whole package. Bring-up stops being an implementation detail and becomes part of the interface definition.

Errors. An internal on-die signal is generally not treated as a channel that can fail. A package-level link may face transmission errors, a failed lane, an unsuccessful training attempt, transient disturbance, power-state transitions, or a partner that is only partly available. Robust interfaces therefore need explicit detection, reporting, and some recovery or degraded-mode behaviour — again, not identically in every design.

A boundary converts formerly internal implementation assumptions into explicit reliability contracts.

7. What the Split Buys — Process Choice and Real Die Reuse

Two benefits follow directly from the couplings the previous chapter identified.

Process choice becomes per-die. A monolithic die commits every block to one process. Once functions sit on separate dies, each die's process is an independent decision — a compute die, an I/O die, and an accelerator die can each be built where that function makes sense rather than where the largest block wanted to be. What that is worth depends entirely on how heterogeneous the system is, and it is not free: each split adds interfaces, package complexity, and qualification and integration work.

Reuse moves from RTL to silicon. Chapter 1.5 drew the line between reusing a design and reusing a manufactured part. Multi-die architecture is what makes the second possible: a die can potentially become a reusable physical element, appearing in several system products without being re-integrated, re-closed, and re-masked each time. That changes how fixed cost is amortised and how quickly a related product can be derived.

State it carefully, because this is where enthusiasm outruns engineering. A reusable die still requires stable interfaces that later products can rely on, compatible packaging, validation of each combination it appears in, and version and compatibility management as it evolves. Reuse becomes possible; it does not become automatic.

8. The Package Becomes Part of the Interconnect

In a monolithic product, the package protects the die and carries signals out to the board. It is downstream of architecture — important, but not deciding how the system works internally.

That changes completely once the system is partitioned. The connections between dies are physically carried by the package, so the package now participates in decisions that used to be settled on-die: where each die sits and how far apart, how many connections are available and at what density, what routing resources exist between two given dies, and therefore how much bandwidth and how much latency each link has. It also carries power delivery to several dies and governs how heat from each escapes and how they interact thermally.

In a multi-die system, the package becomes part of the interconnect architecture.

The practical consequence is organisational as much as technical: package feasibility becomes an input to partitioning rather than a consequence of it. A partition that the package cannot route is not a partition. (The packaging technologies themselves — what options exist and what each provides — are a later topic; what matters here is that the package has moved upstream into architecture.)

9. Verification and Test Change Shape

A monolithic programme verifies blocks, then subsystems, then the whole SoC. A multi-die system keeps all of that per die and adds a category that did not previously exist: the interfaces are now first-class verification targets, and they carry the integration risk.

New questions have to be verified rather than assumed. What happens when one die resets in the middle of a transaction? When the receiver applies backpressure for a long time? When the link reports an error and recovers? When one die finishes booting much later than the other? When two dies meet in a configuration nobody tested together? None of these existed when both sides were regions of one die with a shared reset and a shared designer.

Test changes in a different way, connecting back to yield and cost. Dies are manufactured separately, so they can — and must — be tested separately before being committed to an expensive package, because assembling a bad die wastes the good ones next to it. That is the known-good die requirement: screening before assembly, then testing the assembled package again.

Independent manufacturing creates independent test responsibility.

10. A Hypothetical Three-Die System

Make it concrete with a deliberately generic example — this is illustrative, not a description of any real product.

Take a large accelerator system partitioned into two compute dies and one I/O die. The compute dies carry replicated processing and cache; the I/O die owns host-facing interfaces, memory connectivity, and management functions.

Trace what the partition actually did:

• The compute-to-I/O path, previously on-die interconnect, is now a die-to-die interface — and every byte the compute dies exchange with the host or memory crosses it, so its bandwidth must be budgeted explicitly rather than assumed.
• That path's latency is now architecturally visible, so anything latency-sensitive that used to sit across it may need restructuring.
• The dies may have independent clocks, so the interface manages the crossing.
• Bring-up must be coordinated: the compute dies and the I/O die power up and initialise as separate objects that then have to find each other.
• The interface becomes its own verification programme, separate from either die's internal verification.
• The I/O die can use a different process from the compute dies, since it no longer shares theirs.
• The compute die may be reused — two of them here, perhaps four in a larger product and one in a smaller one, without redesigning it each time.

Notice that the last two items are the payoff and the first five are the price. That is the shape of every partitioning decision.

11. Multi-Die, Chiplets, and the Road to Standardisation

Two distinctions worth getting right before the next chapter.

Multi-die is the broad concept: a system implemented using more than one die. A chiplet-based system is a more modular form of it, in which dies are deliberately designed as composable subsystem building blocks with interfaces intended for reuse. Every chiplet system is multi-die; not every multi-die system is built from chiplets. A company can split its system into dies that only ever work with each other, using an interface of its own design — that is genuinely multi-die, and not especially modular.

Which leads to why standardisation eventually matters. A proprietary die-to-die interface works fine as long as one organisation designs both ends. The moment the ambition widens — dies from different business units, different vendors, different foundries, dies meant to be reused across products, or an ecosystem where parts are sourced rather than built — compatibility becomes strategic rather than convenient. Two dies must agree electrically, agree on protocol behaviour, agree on how bring-up proceeds, agree on management and reliability expectations, and agree on what the package provides. Every one of those is an opportunity to be incompatible.

That is the pressure that eventually produces a standard. This chapter deliberately stops at naming the pressure.

12. The Balance Sheet, and Why the Partition Decides It

Multi-die architecture is a trade, and both sides are real.

What it can buy: smaller individual dies with potentially better die-level yield, per-die process choice, physically reusable dies, finer-grained product scaling, more independent development schedules and smaller closure scope per die, and the ability to build systems larger than any single manufacturable die.

What it costs: die-to-die interface area and power on every die, added latency across boundaries, package complexity, assembly yield, known-good-die testing, extra verification, interface protocol design, clock and reset coordination, power-state coordination, compatibility and version management, and thermal interaction between neighbouring dies.

Note the hedged verbs on the first list — can, potentially, allows. None of those benefits is automatic, and whether the totals favour partitioning is design-specific.

Which brings the chapter to its sharpest practical point. A badly chosen partition can be worse than the monolithic design it replaced. Put high-volume fine-grained traffic across a boundary and the interface becomes a bottleneck. Break a tight latency loop across dies and performance suffers in a way no amount of bandwidth fixes. Cut into too many small dies and duplicated infrastructure and PHY overhead dominate. Demand more connections than the package can route and the partition is simply not buildable.

A good partition minimises what must cross expensive boundaries while maximising what becomes independently useful.

Those two goals pull against each other, which is what makes partitioning a genuine architectural skill rather than a procedure.

13. Why This Matters in Real Engineering Work

• SoC architects choose the boundaries and then own the budgets across them — bandwidth, latency, and power for every link the partition created.
• RTL engineers implement what the boundary requires: interface logic, buffering, flow control, clock- and reset-domain crossings, and an internal architecture that tolerates the latency the partition introduced.
• Verification engineers own a new class of target — die-to-die transactions, bring-up, error and recovery paths, configuration compatibility, and reset and power corner cases that only exist because the two sides are independent.
• Physical-design engineers get smaller, more tractable dies to close, in exchange for D2D PHY placement and package-facing constraints that a monolithic floorplan never had.
• DFT and test engineers enable die-level screening so dies can be known-good before assembly, then test the assembled package.
• Package engineers turn logical connectivity into physical reality — placement, routing, bump maps, power delivery, and thermal behaviour — and their constraints feed back into whether a partition is viable at all.

14. Common Misconceptions

15. Understanding Check

16. Summary

A multi-die system implements one logical system across two or more independently manufactured dies that cooperate inside a package. The logical arrangement may barely change from the monolithic version; what changes is that some connections now cross between separately made objects — and inside one die communication is a wiring problem, while across dies it is an interface problem. That is also why it is not "several chips in a package": package-scale integration allows much shorter, denser connections than a board, but those connections are still contracts rather than wires.

Partitioning decides what must cross, and therefore sets the bandwidth, latency, protocol complexity, connection count, verification burden, and package requirements for everything that follows. Choosing well requires separating two things that look alike: logical modularity is not physical partitionability. What matters is the traffic across a boundary — its volume, latency sensitivity, coherence behaviour, and clock coupling — not how tidy the module hierarchy looks.

The boundary needs an interface because it destroys the assumptions on-die wires rely on, and interfaces grow into protocols because boundaries introduce failure modes and coordination problems that wires do not solve: serialisation and framing, flow control, error detection and recovery, initialisation and state machines, power-state coordination. Clocking, reset, and error behaviour all move from implicit to specified. In exchange, process choice becomes per-die and a die can become a reusable physical element — both genuine, both conditional on stable interfaces, compatible packaging, and validation of each combination.

Two consequences reach beyond the silicon. The package becomes part of the interconnect architecture, so package feasibility is an input to partitioning rather than a result of it. And independent manufacturing creates independent test responsibility, which is why known-good-die screening exists. Set against the benefits are interface area and power, added latency, package complexity, assembly yield, extra verification, and compatibility management — so partitioning is a trade in both directions, and a good partition minimises what must cross expensive boundaries while maximising what becomes independently useful.

17. What Comes Next

Multi-die architecture solves no interoperability problem by itself. A company can split its own system into its own dies, connect them with an interface of its own design, and never intend any of it to work with anything else. That is a legitimate multi-die product — and it is not modular in any way that helps anyone outside the organisation that built it.

• 1.7 — The Chiplet Revolution — how the industry moved from integrated, proprietary multi-die designs toward dies deliberately built as reusable, composable building blocks, and what had to change for that to be possible.

That is where the interfaces named in this chapter stop being an internal engineering detail and start being the thing an ecosystem has to agree on. Browse the full path on the UCIe tutorials index.