PCIe · Module 30
Integration Checklist — Two Reasonable Assumptions
The IP is correct, the SoC is correct, and the product writes to the wrong memory. Integration failures are disagreements between individually reasonable assumptions, and every one produces perfectly legal PCIe traffic.
30.3 asked whether the environment could disprove the design. It could — inside its own testbench, where every assumption was written by one team. This gate asks what survives contact with an SoC whose assumptions were written by six.
1. What This Gate Owns
Integration failures are disagreements between individually reasonable assumptions.
Three readings, and the third is why this gate needs to exist separately.
Nothing is broken, and that is the defining property. The PCIe IP meets its specification. The SoC meets its. The driver is correct against its documentation. Every component passes its own review and the product fails — which is 29.6's composition problem moved from the fabric into the chip.
Which means the review technique is different. 30.2 inspected a module against a contract. This gate inspects a seam against two documents, and the finding is usually that the two documents describe the same thing differently — or that one of them never mentions it.
And the characteristic symptom is legal traffic with a wrong outcome. §5's flagship produces perfectly well-formed TLPs that write to the wrong physical memory. No error is signalled anywhere, because at the protocol layer nothing went wrong.
| This gate owns | Owned elsewhere |
|---|---|
| seams — address, reset, interrupt, clock, power, firmware, debug | the subsystem's internal contracts — 30.1 |
| whether the SoC honours the subsystem's stated assumptions | RTL faithfulness — 30.2 |
| whether readiness composes correctly across layers | environment capability — 30.3 |
| system-level debug reachability | performance closure — 30.5 |
2. Contract Boundaries
3. The Integration Contract Matrix
The artefact this gate produces. One row per seam; a blank cell is a finding, not an omission.
| Contract | Producer | Consumer | Owner | Clock domain | Reset scope | Config source | DV evidence | SW-visible | Silicon observable | Disposition |
|---|---|---|---|---|---|---|---|---|---|---|
| DMA address space | driver | DMA engine | must be one owner | core | function | driver + IOMMU | system test with translation active | yes | fault address register | FAIL if unstated |
| BAR → SoC address map | SoC map | config logic | SoC arch | config | function | integration params | enumeration test | yes | BAR readback | PASS |
| subsystem readiness | firmware | driver | firmware | core | see §9 | firmware | readiness-race test | yes | ready register | FAIL if link-only |
| interrupt vector → CPU | subsystem | interrupt controller | SoC arch | async | function | integration params | end-to-end delivery | yes | pending register | PASS |
| reset scope | SoC reset ctrl | subsystem | SoC arch | async | — | reset tree | reset-under-load | partly | reset cause | FAIL if unmapped |
| link readiness | PHY/LTSSM | firmware | PCIe IP | link | recovery | — | link-up test | yes | LTSSM state | PASS |
| error escalation | subsystem | SoC error agg | SoC arch | core | function | policy | injected errors | yes | first fault | PASS |
| completion timeout | subsystem | driver | architecture | core | function | config | timeout test | yes | timeout count | PASS |
| clock enable / gating | power ctrl | subsystem | SoC power | multiple | — | power policy | gate under load | no | activity counter | FAIL if untested |
| debug reach | subsystem | SoC debug | SoC arch | core | — | — | reachability test | no | the registers | PASS |
Three readings.
"Must be one owner" in row 1 is the entire chapter in a cell. An address contract with two owners is an address contract with two answers — and §5 is what that costs.
The "Clock domain" and "Reset scope" columns exist because they are the two dimensions that vanish in a standalone testbench. A subsystem-level environment runs one clock and one reset by construction; the SoC has several of each, and the seams between them are where the interesting cases live.
And "Silicon observable" is the column that gets left blank under schedule pressure. Every blank there is a future debug session with no evidence — 30.1 §11 already required these; this gate checks they survived integration.
4. Address and BAR Integration
Do not re-teach BARs — 9.1, 9.4, 9.5 and 9.6 own the mechanism. Check the agreement.
| Question | The disagreement it finds |
|---|---|
| does the SoC address map match the BAR sizes requested? | a BAR that cannot be placed at its required alignment |
| what does the SoC return for an unimplemented offset inside a BAR? | silent aliasing, which reads as working (25.5) |
| does the driver's register map match the RTL's? | two documents, both maintained, diverging quietly |
| are prefetchable and non-prefetchable regions correctly separated? | speculative reads with side effects |
| which registers are safe to touch while traffic is live? | 30.2 §9's partial-configuration hazard |
| does anything else in the SoC decode the same range? | two decoders, one wins, non-deterministically |
And the single most effective check here is mechanical: diff the driver's header against the RTL's register definitions, automatically, in CI. Two hand-maintained copies of one map will diverge, and the divergence is discovered by a field that reads back wrong months later.
5. The DMA and IOMMU Contract
The highest-severity seam in this chapter, because it produces perfectly legal traffic that lands in the wrong memory.
The question is one sentence and it must have exactly one answer: what address space do the addresses the driver hands the device live in?
| Possibility | Implication |
|---|---|
| physical | the device writes where it is told; no translation |
| IOVA — translated by an IOMMU | the address is meaningless without translation |
| guest-physical under virtualisation | another translation layer |
| restricted / windowed | valid only within a programmed aperture |
Wrong integration — the driver supplies an IOVA and the DMA engine treats it as physical.
| Step | What happens | Who is wrong |
|---|---|---|
| 0 | driver allocates a buffer, maps it, obtains an IOVA | nobody |
| 1 | driver programs the descriptor with the IOVA | correct per its documentation |
| 2 | DMA engine issues a memory write to that address | correct per its documentation |
| 3 | the address is not translated — it goes out as-is | the seam |
| 4 | the write is a perfectly legal PCIe memory write | ✓ well-formed, ✓ routed, ✓ acknowledged |
| 5 | it lands on whatever physical memory that number names | — |
| 6 | no PCIe error, no completion error, no fault | — |
| 7 | symptom: unrelated memory corruption, or nothing at all | arbitrarily far away |
First divergence: step 3, and it is not a step anybody performs — it is a step nobody performs, which is why no code review finds it.
Four readings.
Both documents are correct in isolation. The driver's says "the descriptor takes a DMA address obtained from the mapping API." The RTL's says "the descriptor address is issued as the memory address." Neither is wrong; together they are a silent memory corruptor.
The symptom is the worst available. With an IOMMU actively enforcing, the access is blocked and you get a fault with an address — annoying and diagnosable. Without enforcement, the write succeeds into unrelated memory, and the failure surfaces as instability somewhere else entirely, possibly much later.
Which makes the IOMMU-enabled configuration the easier one to debug, and it should be tested first. A fault register naming the offending address converts this from a multi-week investigation into a one-line finding — so "is the IOMMU-enforcing path in the test matrix?" is a checklist item with unusually high value.
And the durable fix is to make the address space part of the interface, not the prose. Name the field dma_iova rather than dma_addr; state the space in the register description; assert it in the driver. A field whose name states its space cannot be misread by the next integrator.
6. RTL — Aperture Validation and Bounds
§5's failure is silent because nothing checks. The containment is cheap, and it converts an unbounded corruption into a contained, attributed, reported event.
// ILLUSTRATIVE. An aperture guard at the DMA engine's address output. It cannot
// fix an address-space disagreement — nothing in hardware can — but it converts
// a silent write into somebody else's memory into a REPORTED containment event
// with the offending address captured.
localparam int AW = 64;
logic [AW-1:0] ap_base_q, ap_limit_q; // programmed by firmware
logic ap_valid_q; // written
logic ap_checked_q; // validated: limit > base, aligned, mapped
logic [AW-1:0] first_viol_addr_q; // FIRST, not last — evidence survives
logic first_viol_valid_q;
logic [31:0] viol_count_q;
// In range means BOTH bounds and the whole burst — checking only the start
// address lets a transfer begin legally and run off the end, which is the
// subtle version of the same bug.
wire in_range = ap_valid_q && ap_checked_q
&& (dma_addr >= ap_base_q)
&& ((dma_addr + AW'(dma_bytes)) <= ap_limit_q);
// The request is only emitted if it is contained. Containment is a gate, not a
// warning — a violation that is merely counted still corrupts memory.
assign dma_req_valid = dma_req_pending && in_range;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
first_viol_addr_q <= '0; first_viol_valid_q <= 1'b0; viol_count_q <= '0;
end else if (dma_req_pending && !in_range) begin
// First-fault semantics: the first violation is the diagnostic one, and a
// stream of subsequent violations must not overwrite it (30.1 §13 Q40).
if (!first_viol_valid_q) begin
first_viol_addr_q <= dma_addr;
first_viol_valid_q <= 1'b1;
end
viol_count_q <= viol_count_q + 32'd1;
end
end
// MANDATORY. English: no DMA request leaves the subsystem outside the validated
// aperture. This is the property that bounds §5's blast radius.
a_dma_within_aperture: assert property (
@(posedge clk) disable iff (!rst_n)
dma_req_valid |-> in_range
);
// MANDATORY. English: no DMA is issued before the aperture has been both
// programmed AND validated. Catches §8's readiness race at the point of harm.
a_no_dma_before_aperture: assert property (
@(posedge clk) disable iff (!rst_n)
dma_req_valid |-> (ap_valid_q && ap_checked_q)
);
// MANDATORY. English: the first violation address is sticky until cleared.
a_first_viol_sticky: assert property (
@(posedge clk) disable iff (!rst_n || viol_clear)
$rose(first_viol_valid_q) |=> always first_viol_valid_q
);Architecture. A gate, not a monitor. A violation that is counted but still emitted has not been contained — the write already happened, and the counter merely documents it.
State. Aperture bounds, two qualifying bits, and first-fault capture. ap_checked_q is separate from ap_valid_q because programmed and validated are different facts — §9's readiness aggregation requires both for the same reason.
Event. The check runs on every request; the first-violation capture fires once, and viol_count_q counts the rest.
Contract. Firmware must set ap_checked_q only after verifying the aperture against the SoC address map and the IOMMU's mapping — this hardware cannot verify translation, and claiming otherwise would be the same overreach §5 is about. What it can guarantee is containment within a programmed range.
Failure. Checking only dma_addr and not dma_addr + dma_bytes lets a legal-looking transfer run off the end of the aperture, which is the subtle version of §5 and is harder to spot because the start address is correct. Last-violation-wins instead of first destroys the diagnostic in exactly the burst of violations that follows the first one.
DV/debug. first_viol_addr_q is the register that makes §5 diagnosable without an enforcing IOMMU — it names the offending address from inside the subsystem. One register, and it turns a multi-week memory-corruption investigation into a one-line finding, which is the same trade every Module 29 chapter ended on.
7. Reset Ownership Across the Seam
30.1 §6 built the subsystem's reset matrix. This gate asks whether the SoC's reset tree implements it.
| Reset | Who asserts it | What the subsystem expects | The integration failure |
|---|---|---|---|
| SoC cold reset | power sequencer | everything clears | deassertion not synchronised to each domain |
| PCIe block reset | SoC reset controller | per 30.1 §6 | mapped to the wrong scope — clears more or less than intended |
| function-level reset | software, via config | this function's state | siblings affected |
| link Recovery (18.5) | the link | retain live requests | SoC treats it as an error and resets the block |
| software reset | driver | a defined sequence | driver and RTL disagree on what it clears |
| power-domain transition | power controller | state retained or restored | retention list incomplete |
Three readings.
Row 4 is the classic and it is expensive. The SoC's error aggregator sees a link event, classifies it as a fault, and resets the block — while the subsystem's architecture said Recovery retains live requests. The result is 30.2 §7's orphaned-work timeline, triggered by the platform rather than by the RTL.
Row 3 is 29.6 §14's blast radius, one level down. A function-level reset that disturbs a sibling function is the same failure as a subtree reset disturbing a sibling endpoint, and it has the same cause: scope decided without reference to ownership.
And row 1's deassertion synchronisation is the one that produces impossible-looking bugs. Different domains leaving reset on different cycles means the first cycle of operation differs between them — and the symptom is a rare, ratio-dependent startup failure that disappears under instrumentation.
8. Readiness Is a Hierarchy, Not a Bit
The most under-modelled contract in the whole gate: ready means different things at five different layers, and they complete in order.
| Layer | "Ready" means | Completes when |
|---|---|---|
| 1 — physical / link | the link reached L0 (18.6) | training completes |
| 2 — enumerated | the device has been discovered and BARs assigned (7.1) | enumeration finishes |
| 3 — configured | the subsystem's own configuration is committed | firmware programs it |
| 4 — resourced | DMA apertures mapped, IOMMU entries installed, interrupts routed | driver setup completes |
| 5 — functional | the device can actually serve a request | all of the above, and only then |
Exposing layer 1 as though it were layer 5 is the single most common readiness bug, and it is attractive precisely because layer 1 is the easiest to observe.
Wrong integration — firmware publishes readiness at link-up.
| Time | Event | ready bit | Reality |
|---|---|---|---|
| 0 | link reaches L0 | — | layer 1 only |
| 1 ms | firmware sets ready | 1 | layers 3, 4 incomplete |
| 1.1 ms | driver issues a request | 1 | DMA aperture not yet programmed |
| 1.2 ms | device performs a DMA to an unmapped address | 1 | §5's failure, triggered by a race |
| 5 ms | firmware finishes configuration | 1 | too late |
| later | works perfectly on every subsequent boot | 1 | the race is timing-dependent |
Three readings.
The intermittency is the defining feature. A slow boot hides it entirely; a fast one exposes it. So it appears on one machine, in one lab, at one temperature — and the natural conclusion is that the hardware is marginal.
Readiness must be an aggregate whose terms are each individually observable. A single bit tells you it is not ready; five bits tell you which layer is missing, which is the difference between a bug report and a diagnosis.
And the aggregation belongs in hardware where the terms are hardware. §9 is that register. Firmware may add its own term, but it must not be the only term — firmware asserting readiness on its own behalf is precisely the failure above.
9. RTL — Readiness Aggregation
// ILLUSTRATIVE. Readiness as a conjunction of individually observable terms.
// The design goal is that "not ready" is always answerable with "because of
// WHICH term", which is what converts a boot race into a one-read diagnosis.
typedef struct packed {
logic link_l0; // layer 1 — from the LTSSM
logic enumerated; // layer 2 — BARs assigned
logic cfg_committed; // layer 3 — 30.2 §9's commit, not a shadow write
logic dma_mapped; // layer 4 — aperture programmed AND validated
logic irq_routed; // layer 4 — vectors installed
logic fw_ready; // layer 5 — firmware's own term, one of six
} ready_terms_t;
ready_terms_t ready_q;
logic func_ready_q;
// Software reads THIS to find out which term is missing. One register.
assign ready_status = ready_q;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
ready_q <= '0; func_ready_q <= 1'b0;
end else begin
// Each term is set by its own owner and, critically, CLEARED by the event
// that invalidates it. A term that only ever sets is a latch for a
// condition that can go away — which is how a stale ready survives a reset.
ready_q.link_l0 <= ltssm_state_is_l0; // level, not a pulse
ready_q.enumerated <= bars_assigned;
ready_q.cfg_committed <= cfg_valid_q; // 30.2 §9
ready_q.dma_mapped <= dma_aperture_valid && dma_aperture_checked;
ready_q.irq_routed <= irq_vectors_valid;
ready_q.fw_ready <= fw_ready_wr ? fw_ready_data : ready_q.fw_ready;
func_ready_q <= &ready_q; // all six, always
end
end
// MANDATORY. English: no request is accepted unless the function is ready.
// This is the property that makes the race impossible rather than unlikely.
a_no_traffic_before_ready: assert property (
@(posedge clk) disable iff (!rst_n)
req_fire |-> func_ready_q
);
// MANDATORY. English: readiness is not sticky. If a term goes away — link left
// L0, configuration invalidated — readiness must drop in the next cycle.
a_ready_not_sticky: assert property (
@(posedge clk) disable iff (!rst_n)
(!(&ready_q)) |=> !func_ready_q
);Architecture. A conjunction, not a bit. Each term has exactly one owner and each is separately readable, so "not ready" is always answerable with "because of which term."
State. Six terms plus the aggregate. dma_mapped requires _valid && _checked because a programmed aperture that was never validated is the §5 failure waiting for a trigger.
Event. Every term is assigned from a level, not a pulse. A term set by a pulse cannot be cleared by the condition going away, which produces a readiness bit that survives events that should invalidate it — including a link drop.
Contract. Firmware owns exactly one term. The programming model must say so, or the next integrator will find fw_ready and use it as the readiness bit, reproducing §8 exactly.
Failure. Any term latched rather than levelled → stale readiness after a link event. func_ready_q computed from a subset → the race returns, now harder to see because a readiness register exists and looks thorough.
DV/debug. a_no_traffic_before_ready is the property; the negative test is issue traffic with one term deasserted and confirm it fires. In silicon, ready_status turns "the device didn't come up" into "dma_mapped is 0" — one read.
10. Interrupts Across the Seam
| Question | The disagreement |
|---|---|
| does each MSI-X vector reach the intended CPU handler? | vector table programmed by one team, routed by another |
| is data visible before the interrupt is observed? | the producer-consumer inversion, now with an SoC interconnect in between |
| what happens to pending interrupts across each reset? | 30.1 §13 Q34's row, at SoC scope |
| does interrupt moderation interact with the SoC's aggregator? | double moderation, or none |
| is there a path for a lost interrupt to be recovered? | a poll fallback, or a permanent hang |
And the ordering question deserves emphasis because the SoC changes it. 19.2's mechanism is a memory write, which is why it is ordered with respect to data. An SoC interconnect between the subsystem and memory may reorder, buffer or route differently — so the property that held at the subsystem boundary must be re-argued at the SoC boundary, not assumed to carry.
11. Clocks, Power and Gating
| Question | The failure |
|---|---|
| which clocks does the subsystem need, and are they all present before reset deassertion? | a domain starting with no clock latches an undefined state |
| can any clock be gated while traffic is outstanding? | outstanding requests stall forever; the peer times out |
| does a power-domain transition preserve the state 30.1 §6 says must survive? | an incomplete retention list |
| are single-cycle events crossed safely at every SoC boundary? | 30.2 §12, now with domains you do not control |
| is reset deassertion synchronised per domain? | §7 row 1 |
Two readings.
Gating with work outstanding is the highest-severity item here and it is easy to arrive at: the power controller sees an idle clock activity signal, not an idle protocol state. "No transitions on this bus" and "no requests outstanding" are different facts, and only the second is safe to gate on. The subsystem must export an outstanding-work indication to the power controller — which is a contract row, and it is frequently missing.
And an activity counter is the cheap observable that settles it. "Was the clock gated while outstanding_q was non-zero?" is a one-bit sticky answer, and without it a gating-induced timeout is indistinguishable from a link problem.
12. Software and Firmware Contracts
| Contract | Failure when unstated |
|---|---|
| the reset sequence the driver must perform | a driver reset that leaves hardware half-configured |
| which registers are safe to touch while live | 30.2 §9 partial configuration |
| the readiness protocol | §8 |
| the descriptor ownership handoff point | 30.1 §13 Q17 — the producer-consumer class |
| the error-clearing protocol | first fault cleared before it is read |
| the address space of every address the driver supplies | §5 |
| what the driver must do after each reset type | recovery that leaves the device unusable |
And the reviewable artefact is a single document that both teams signed, not two documents that agree today. Every row here is a place where "reasonable" differs between a hardware engineer and a driver author — and neither will discover it by reading their own side.
13. Debug Reachability
| Requirement | Why |
|---|---|
| 30.1 §11's observables are readable from the SoC, in the failing state | a register behind a gated clock is unreadable exactly when needed |
| a first-fault register survives the escalation that follows the fault | 25.1 |
| reset cause is recorded | §7 — otherwise "who reset us?" is unanswerable |
| readiness terms are individually readable | §8, §9 |
| a shared timestamp exists across subsystem, SoC and software events | 29.6 §15: correlation needs a common time base |
| counters survive a functional reset | evidence outliving the event |
The sharpest version of this gate's question: in the failing state, is the evidence readable? A debug register in a powered-down or clock-gated domain is absent precisely when the failure occurs — and that is a design finding, not a lab inconvenience.
14. The Integration Checklist — 46 Questions
Address and BAR
| # | Question | Why | Evidence | FAIL if |
|---|---|---|---|---|
| 1 | Does the SoC address map accommodate every BAR's size and alignment? | 9.4 | the map with BARs placed | "it'll fit" |
| 2 | Does anything else decode the same range? | non-deterministic winner | a decode audit | not checked |
| 3 | What is returned for unimplemented offsets inside a BAR? | silent aliasing (25.5) | a defined response | undefined |
| 4 | Is the driver's register map diffed against the RTL's, automatically? | §4 | the CI check | two hand-maintained copies |
| 5 | Are prefetchable and non-prefetchable regions correctly separated? | speculative reads with side effects | the classification | mixed |
| 6 | Which registers are safe to write while traffic is live? | 30.2 §9 | per-register statement | unstated |
DMA and address translation
| # | Question | Why | Evidence | FAIL if |
|---|---|---|---|---|
| 7 | What address space do driver-supplied addresses live in? | §5 | one stated answer | two documents that never mention it |
| 8 | Does the field name state its space? | §5 | dma_iova vs dma_addr | a generic name |
| 9 | Is the IOMMU-enforcing configuration in the test matrix? | §5 — it is the diagnosable case | the test | only the permissive path tested |
| 10 | Is there a fault-address register readable after a translation fault? | §5 | the register | none |
| 11 | Is the DMA aperture validated, not just programmed? | §9's _checked term | the validation step | programmed only |
| 12 | Does the device ever generate an address outside the aperture? | containment | a bounds check + counter | unchecked |
| 13 | Is coherency assumed anywhere without being stated? | assumption, not property | the explicit statement | assumed |
| 14 | Is scatter-gather alignment/length/termination agreed with the driver? | 20.4 | the joint document | inherited from an example |
| 15 | Where exactly does descriptor ownership transfer? | 30.1 §13 Q17 | one named event | two readings |
Reset
| # | Question | Why | Evidence | FAIL if |
|---|---|---|---|---|
| 16 | Is every reset in 30.1 §6's matrix mapped to an SoC reset? | §7 | the mapping | unmapped rows |
| 17 | Does the SoC treat link Recovery as an error? | §7 row 4 | the error policy | it resets the block |
| 18 | Does a function-level reset disturb a sibling function? | §7 row 3 | the scope test | untested |
| 19 | Is reset deassertion synchronised per clock domain? | §7 row 1 | the synchronisers | shared async deassert |
| 20 | Is reset cause recorded and readable afterwards? | §13 | the register | not recorded |
| 21 | Is reset tested with work outstanding at SoC level? | 30.3 §10 | the test | idle-only |
| 22 | Does the driver's reset sequence match the hardware's expectation? | §12 | the joint sequence | two versions |
Readiness
| # | Question | Why | Evidence | FAIL if |
|---|---|---|---|---|
| 23 | Is readiness an aggregate of all five layers? | §8 | the conjunction | link-up published as ready |
| 24 | Is each term individually readable? | §9 | ready_status | one bit |
| 25 | Are terms driven from levels, not pulses? | §9 | the assignments | latched terms |
| 26 | Does firmware own exactly one term? | §9 | the programming model | firmware owns the bit |
| 27 | Is traffic-before-ready impossible, or merely unlikely? | §9 | a_no_traffic_before_ready | a software convention |
| 28 | Is the boot race tested at varying boot speeds? | §8's intermittency | the timing sweep | one boot path |
Interrupts
| # | Question | Why | Evidence | FAIL if |
|---|---|---|---|---|
| 29 | Does every vector reach its intended handler? | §10 | end-to-end delivery per vector | vector 0 tested only |
| 30 | Is data visible before the interrupt is observed at SoC level? | §10 | the re-argued ordering | assumed to carry |
| 31 | What happens to pending interrupts across each reset? | §10 | the matrix row | blank |
| 32 | Is there a recovery path for a lost interrupt? | §10 | poll fallback or timeout | permanent hang |
| 33 | Does moderation interact correctly with the SoC aggregator? | §10 | the combined behaviour | double moderation |
Clocks and power
| # | Question | Why | Evidence | FAIL if |
|---|---|---|---|---|
| 34 | Are all clocks present before reset deassertion? | §11 | the sequence | unspecified |
| 35 | Can a clock be gated with work outstanding? | §11 | outstanding-work export to the power controller | gated on bus idleness |
| 36 | Is there an activity/violation counter for gating? | §11 | the sticky bit | none |
| 37 | Does the retention list cover everything 30.1 §6 says must survive? | §11 | the list, diffed | partial |
| 38 | Are single-cycle events crossed safely at SoC boundaries? | 30.2 §12 | the scheme | bare pulses |
Errors, software, debug
| # | Question | Why | Evidence | FAIL if |
|---|---|---|---|---|
| 39 | Does the SoC error aggregator classify PCIe events correctly? | §7 row 4 | the classification table | everything is fatal |
| 40 | Does first fault survive the escalation it triggers? | §13 | the retention | cleared by the response |
| 41 | Is the error-clearing protocol agreed with software? | §12 | the joint document | driver clears before reading |
| 42 | Is there one signed document per software contract? | §12 | the document | two agreeing documents |
| 43 | Are the observables readable in the failing state? | §13 | powered/clocked-domain analysis | behind a gated clock |
| 44 | Is there a shared timestamp across subsystem, SoC and software? | 29.6 §15 | the time base | three unrelated clocks |
| 45 | Do counters survive a functional reset? | §13 | the reset row | cleared |
| 46 | Has the full stack been run — real driver, real firmware, real IOMMU? | §1 | the system run | subsystem testbench only |
15. Misconceptions
"The IP is verified and the SoC is verified, so integration is a formality." §1: both are correct and the product writes to the wrong memory. The failures live on the seams.
"A DMA address is a DMA address." §5: physical, IOVA, guest-physical and windowed are four different things, and choosing wrong produces legal PCIe traffic into unrelated memory with no error.
"The IOMMU makes it safer to debug." Correct, and it is the reason to test that path first — an enforcing IOMMU turns a silent corruption into a fault with an address.
"Link up means the device is ready." §8: that is layer 1 of five. Publishing it as readiness is the most common readiness bug, and its symptom is a boot race that appears on one machine.
"It works on every boot." §8: the race is timing-dependent. A slow boot hides it completely.
"The clock is gated when the bus is idle." §11: bus idleness and zero outstanding requests are different facts, and only the second is safe to gate on.
"Recovery is an error, so we reset the block." §7: 30.1 §6 says Recovery retains live requests. The platform reset then orphans them.
"We have debug registers." §13: readable in the failing state, or not at all.
16. Understanding Check
Q1. A driver hands the device an address, the device performs a legal memory write, and unrelated memory is corrupted. What happened?
An address-space disagreement, and nobody did anything wrong (§5). The driver supplied an IOVA obtained from a mapping API — correct per its documentation. The DMA engine issued that number as the memory address — correct per its documentation. The failing step is the one nobody performs: translation. The resulting TLP is well-formed, correctly routed and acknowledged, so there is no PCIe error, no completion error and no fault — which is why no code review and no protocol analyser finds it.
Two things follow. The IOMMU-enforcing configuration is the easier one and should be tested first — enforcement turns silent corruption into a fault with an offending address, converting a multi-week investigation into a one-line finding. And the durable fix is to put the address space into the interface rather than the prose: name the field dma_iova, state the space in the register description, and assert it in the driver. A field whose name states its space cannot be misread by the next integrator.
Q2. Firmware sets a ready bit when the link reaches L0. What is wrong, and why does it pass most testing?
It publishes layer 1 of a five-layer hierarchy as though it were layer 5 (§8). Ready means different things at the link, enumerated, configured, resourced and functional layers, and they complete in order. Publishing at link-up lets the driver issue a request while the DMA aperture is not yet programmed — which triggers §5's failure as a race.
It passes most testing because the race is timing-dependent: a slow boot hides it entirely, a fast one exposes it, so it appears on one machine in one lab and reads as marginal hardware. The fix is an aggregate whose terms are individually observable (§9): six terms, each with one owner, each driven from a level rather than a pulse — a latched term cannot be cleared when its condition goes away, which produces a readiness bit that survives a link drop. Firmware owns exactly one of the six, and the programming model must say so or the next integrator will use fw_ready as the readiness bit and reproduce the original bug. a_no_traffic_before_ready makes the race impossible rather than unlikely, and ready_status turns "it didn't come up" into "dma_mapped is 0" — one read.
Q3. The SoC power controller gates the PCIe clock when the bus is idle. What can go wrong?
Bus idleness and zero outstanding work are different facts, and only the second is safe to gate on (§11). Non-posted requests can be outstanding with no bus transitions at all while their Completions are in flight (10.4) — the peer is working, and nothing is moving locally. Gating there stalls the outstanding requests indefinitely, and the failure surfaces as completion timeouts (25.7) that look exactly like a link problem.
The missing artefact is a contract row: the subsystem must export an outstanding-work indication to the power controller, and §3's matrix is where that absence becomes visible. The cheap observable is a sticky bit — "was the clock ever gated while outstanding_q was non-zero?" — which separates a gating-induced timeout from a genuine link event in one read. Without it the two are indistinguishable.
Q4. Why does this gate exist separately, given that 30.2 and 30.3 both passed?
Because those gates inspect components against contracts, and this one inspects seams against two documents (§1). Every failure in this chapter has the same shape: the IP meets its specification, the SoC meets its, the driver is correct against its documentation, and the product fails — which is 29.6's composition problem moved from the fabric into the chip. The finding is usually that two documents describe one thing differently, or that neither mentions it.
Two dimensions make it structurally invisible earlier. A subsystem testbench runs one clock and one reset by construction; the SoC has several of each, and §3's "Clock domain" and "Reset scope" columns are exactly where the interesting cases live. No amount of subsystem verification reaches them, which is why question 46 — has the full stack been run with real driver, real firmware and a real enforcing IOMMU? — is the one that subsumes many of the others. And the symptom to expect throughout is legal traffic with a wrong outcome, because at the protocol layer nothing went wrong.
17. Module 30 So Far
| Gate | Asks | Passing means |
|---|---|---|
| 30.1 Architecture | is it decisive? | two engineers cannot implement incompatible behaviour |
| 30.2 RTL | is it faithful? | simultaneous legal events do not break ownership |
| 30.3 Verification | can it fail? | every contract has an independent checker seen to fire |
| 30.4 Integration | does it survive the SoC? | the seams agree, and the evidence is readable |
The remaining gates — 30.5 Performance Checklist, 30.6 Debug Checklist and 30.7 Interview Checklist — are planned. 30.5 inherits every performance assumption 30.1 §10 required to be declared, and it can only verify what this module made someone write down.
The through-line across all four. 30.1 fails on blanks, 30.2 on coincidences, 30.3 on circularity, and 30.4 on disagreement. None of them is about whether the design is clever. Every one asks the same underlying question in a different register: is there evidence, and would anyone have noticed if there were not?