Point-to-Point Links — The Local Building Block

Chapter 2.7 reasoned about whole routes across the fabric. This chapter goes the other way and examines a single edge:

What does it really mean that a PCIe Link joins exactly two components, and what architectural consequences follow?

Module 1 arrived at dedicated links as the answer to a scaling problem, and every chapter since has used them as a given. That is a lot of weight resting on a term nobody has yet examined closely. What follows unpacks it — including the distinction between a Link and a path, which the previous chapter needed and this one owns.

1. Exactly Two, and What That Rules Out

Start with the literal claim, because its consequences are larger than they first appear.

A Link has exactly two attached components. Not "usually two." Not "two plus some monitoring taps." Two. One at each end, and nothing else connected anywhere along it.

Compare this with what Chapter 1.6 described. A multi-drop bus has several devices attached to one set of conductors, each attachment a branch off the main path. Every one of those branches contributes to the signalling environment that every other device must operate in — which is why device count and achievable rate were coupled, and why the architecture stopped scaling.

A two-component connection has no branches, because there is nothing to branch to. That single structural difference is what makes the signalling problem tractable: the conditions on a Link are determined by that Link's own construction and the two components it joins, and not by how many other devices exist in the system.

So the consequence of "exactly two" is not merely tidiness. It is that attaching a device elsewhere in the hierarchy does not change the conditions on this connection. Add a device to another branch, or ten, and nothing about this Link's signalling environment moves. On a shared medium that statement was false by construction.

2. A Link Is Not a Path

This distinction deserves its own section, because conflating the two produces a specific and very common class of error.

A Link is one connection between two adjacent components.

A path is the sequence of Links a transaction traverses between its origin and destination.

A path may consist of one Link, when a device attaches directly to a Root Port. It may consist of several when Switches sit in between, and each Switch on the route means one more Link in the path.

Read Figure 1 carefully and three facts separate cleanly:

• Link B is a resource. It joins the Switch and Endpoint 1. Its capability is a property of that connection.
• The path from the Root Complex to Endpoint 1 is Link A + Link B. Two resources, one forwarding decision between them.
• Link C is unrelated to Link B. They share a neighbour, not a resource. Traffic on one does not occupy the other.

But notice the qualification the figure's caption makes: Endpoints 1 and 2 do share Link A. Their own Links are separate; their paths overlap upstream. That is precisely the fabric-level point from Chapter 2.7, and it is only expressible once Link and path are distinct words.

3. What "Local" Means — and What It Does Not

Locality is the property that makes dedicated connections valuable, and it is routinely overstated. Being precise about it is worth a section.

What is genuinely local to a Link:

• Signalling conditions. The electrical environment is determined by this connection's construction and the two components it joins. Devices elsewhere do not participate in it.
• Capability. What this connection can carry is a property of this connection, developed in §5.
• Operational state. Whether this Link is currently usable is a fact about this Link, developed in §6.
• Diagnosability. Because the above are local, a problem with this connection can be investigated as a problem with this connection.

What is not local, and never was:

• Upstream capacity. Everything this Link carries toward the host continues onto other Links, which are shared with other devices.
• Destinations. System memory is common to everything targeting it.
• System conditions. Power delivery, thermal environment, board construction, and overall system state are shared in ways no topology choice removes.
• Position in the hierarchy. How deep this Link sits, and what else lies behind the same Switches, determines much about the behaviour observed across it.

The accurate summary is therefore narrower than "independent":

A Link provides architectural and electrical locality, not independence from the system.

That distinction matters because the overstated version predicts things that do not happen. A device whose Link is perfect can still be starved by a saturated upstream segment, throttled by system conditions, or limited by a shared destination — none of which is a defect in the Link, and none of which "independent" would lead you to expect.

4. Traffic in Both Directions

A Link is bidirectional, and at architecture level the useful consequence is straightforward: traffic can move in both directions without the two directions taking turns on one shared resource.

That is worth contrasting with what came before. On a shared bus, a transfer occupied the medium — during it, no other transfer proceeded, in any direction. A Link's directions are not in competition with each other in that way: a device can be receiving while it transmits.

Both directions matter in practice, because both are used constantly. Chapter 2.3 established that an Endpoint both responds to host-directed accesses and initiates transactions of its own, typically toward system memory. Those flows run opposite ways along the same Link, and a system in which a device is streaming data to memory while receiving control accesses is entirely ordinary rather than exceptional.

How a Link physically provides directional capacity — the signalling arrangement that makes simultaneous transmission and reception possible — is a physical-layer matter that later modules develop. The architectural fact to carry now is that the two directions are separately usable, so reasoning about a Link means reasoning about two flows rather than one shared turn-taking resource.

5. Capability Belongs to the Connection

Here is the property that answers Module 1's central complaint most directly.

Chapter 1.7 established that on a shared bus, capacity was a property of the medium: width, rate, and electrical capability applied to everyone, so improving one device's transport meant obliging every attachment to keep up, bounded by the least capable participant. Provisioning was necessarily global.

With dedicated Links this inverts. Capability is a property of each connection. Different Links in the same system can differ, and a connection serving a demanding device can be provisioned more generously without requiring anything of a connection serving an undemanding one.

The consequences are worth spelling out, because this is what "scalable" concretely means here:

• Selective provisioning. Capacity can be placed where demand is rather than applied uniformly.
• Mixed systems are ordinary. A hierarchy containing connections of differing capability is normal, not a compromise.
• No least-common-denominator ceiling. One modest device does not constrain what other connections may be.

Recall from Chapter 1.4 that AGP existed precisely because selective provisioning was impossible on a shared bus — a whole separate interface had to be built for one device class. Per-connection capability makes that a general property of the architecture rather than an exception carved out for a favourite.

How capability is actually expressed, and how two components settle on what a given connection will use, involves mechanisms that Module 6 develops properly. The architectural point stands without them: capability is local to the connection.

6. A Link Has Its Own Operational State

One further property is local, and it is easy to overlook because a working system hides it.

A Link is not simply present the moment two components are physically connected. It must become usable as a connection before higher-level traffic can rely on it — the two components have to establish that a working connection exists between them and on what terms. Until that has happened, the Link is not carrying transactions.

Two architectural consequences follow.

State is per-Link. Because each connection establishes itself with its neighbour, its operational state is its own. One Link being unusable does not imply anything about another. In Figure 1, Link C can be perfectly operational while Link B is not.

A path requires all of its Links. A route is usable only if every Link along it is. An Endpoint two hops away is unreachable if either hop is down — and from the host's perspective those two failures can look similar while having entirely different causes and locations.

That second point is where the Link-versus-path distinction earns its keep operationally. "The device is not reachable" is a statement about a path. Localising it means asking which Link along that path is the problem, and the answer is not obvious from the symptom.

How Links are detected, established, and maintained — the states involved and what happens when a connection is disturbed — is a substantial topic that Module 18 owns in full, with the physical mechanisms in Module 17. This chapter deliberately stops at the architectural fact: each Link has its own operational state, and traffic depends on it.

7. Common Misconceptions

8. Real Engineering Relevance

The Link is the natural unit of verification scope and failure localisation, and understanding why makes both activities much more tractable.

A Link has exactly two participants, which bounds the problem. Verifying behaviour at a connection means verifying how one component behaves toward its single neighbour. There is no third party whose activity could explain an observation, which is a substantially simpler situation than anything a shared medium offers. This is why PCIe verification environments reason per-Link even when the transactions under test travel further: the Link is where obligations are checkable.

Neighbour behaviour must be modelled. A device-side design has one neighbour, and verifying it requires standing in for that neighbour convincingly. What is legitimate for the far end to do — and what it may never do — is precisely the boundary Chapter 2.2 framed when distinguishing root-side from device-side responsibilities. A model that is too permissive lets real bugs through; one that is too strict reports failures that would never occur.

Failure localisation follows the Link/path distinction. When something is unreachable or slow, the productive first question is which Link, not which device. Three cases separate cleanly:

• Link-local — a problem with one connection. Affects only traffic traversing it, and its operational state is diagnosable on its own.
• Path-level — every Link along a route is individually fine, but a shared segment is saturated. Presents as several devices behind the same convergence point degrading together.
• Forwarding — Links are healthy and traffic is not arriving where it should, which is a qualitatively different symptom from slowness.

An engineer who reasons in Links asks which of those three applies before investigating any of them. One who reasons only in devices tends to investigate whichever device complained, which — as Chapter 2.4 noted — is frequently not where the problem lives.

9. Understanding Check

10. Summary

A PCIe Link is a dedicated bidirectional connection between exactly two adjacent components. The literal count matters: with no third attachment there are no branches, so conditions on a connection are set by that connection and its two neighbours rather than by how many devices exist elsewhere.

The distinction this chapter owns is Link versus path. A Link is one connection; a path is the sequence of Links a transaction traverses, one longer for each Switch on the route. Keeping them separate is what allows the ordinary situation — every Link healthy, the device still limited by a shared upstream segment — to be stated at all.

Locality is real but narrower than independence. Local: signalling conditions, capability, operational state, diagnosability. Not local: upstream capacity, shared destinations, system-wide conditions, and position in the hierarchy. A Link provides architectural and electrical locality, not independence from the system.

Three further properties belong to the connection. It is bidirectional, so both directions are separately usable rather than taking turns. Its capability is its own, which makes selective provisioning general rather than exceptional — the direct answer to the global-provisioning limit Module 1 identified. And it has its own operational state, so a path is usable only when every Link along it is.

Hold the model: a Link's signalling, capability, and state are local to that connection rather than shared globally across the hierarchy.

11. What Comes Next

Module 2 has now built the system from both directions: Chapter 2.7 reasoned about whole routes, and this chapter about the single edges those routes are made of.

Chapter 2.9 — Switched Architecture closes the module by putting the complete picture against what it replaced. Not Module 1's derivation of why PCIe had to exist, but a structural account of how the finished system behaves differently — where concurrency comes from, where contention moved to, and what still limits scaling — now that every component has been defined.

Revisit The PCIe Fabric for the whole-route view, or Switch for the component that turns several Links into a path. Browse the full path on the PCIe tutorials index.