Resource Allocation — Where a Function's Regions Live

Chapter 7.7 ended with a function that is discovered, identified, numbered, and whose configuration space reads perfectly.

It still cannot do anything.

After a function is discovered and accessible, how does the system decide where that function's required regions should live?

1. Enumerated Is Not Usable

Six stages were listed in Chapter 7.1. Five of them are now done: the function has been discovered, identified, given a hierarchy identity, and is reachable by configuration access.

What it does not have is anywhere for its operational regions to live.

A function's configuration space is reached by identity (Chapter 7.7) and needs no address assignment. Its actual working registers and data windows — the ones a driver uses to operate the device — are reached by address, and no address has been assigned to them.

A function that has been enumerated but not resourced is present in the system's description and unable to do its job.

This is why "the device shows up but does not work" is such a common report, and why §10 treats it as a distinct failure class rather than a variant of "missing."

2. Requirement Versus Assignment

The distinction that organises the whole chapter:

A function describes what it needs. It does not choose where it lives — it cannot, because it has no knowledge of what else exists or what space is available. The host has that knowledge and makes the placement decision.

Why the separation is necessary. If functions chose their own addresses, two functions could choose the same one, and neither would have the information to detect it. Centralising placement is what makes non-overlap achievable at all.

3. What Gets Allocated

At the level this chapter needs:

Memory address regions. The dominant case. A function needs a range of the system's memory address space through which its registers and data windows are reached.

I/O resources, where a platform and function use them. A separate, historically motivated address space, present for compatibility reasons that Chapter 1.3 explains.

Bridge and switch windows. Not a function's own requirement, but a consequence of the topology: a hierarchy-expanding port must forward accesses aimed at everything below it, and it needs a window describing what that is. §5 is about this.

Interrupt and other system resources are also assigned during platform bring-up; they are outside this chapter, which is about address-space allocation.

4. The Allocation Problem

Put the pieces together and a real constraint-satisfaction problem appears:

• The system's address space is finite.
• Several functions each need a region.
• Each region has an alignment constraint.
• No two regions may overlap.
• Every region must sit within the windows of every parent on its path.

Symbolically, with two devices:

Device A needs a region of size Sa, aligned to Aa
Device B needs a region of size Sb, aligned to Ab
 
Host assigns:  A → [Xa, Xa + Sa)
               B → [Xb, Xb + Sb)
 
Required:      Xa aligned to Aa
               Xb aligned to Ab
               [Xa, Xa+Sa)  and  [Xb, Xb+Sb)  do not intersect
               both within every enclosing parent window

Three separate checks — overlap, alignment, containment — and §§6–8 build one hardware checker for each.

A worked placement

Symbolic sizes, to show how the three constraints interact rather than to model any specific device. A branch has 1 MB of address space available starting at 0x8000_0000, and three functions beneath it:

Placing them naively in the order discovered — A, then B, then C, each at the next free address:

• A → 0x8000_0000, ending at 0x8001_0000. Aligned to 64 KB ✓
• B → next free is 0x8001_0000, but B needs 256 KB alignment and 0x8001_0000 is not a multiple of 0x4_0000. The candidate must move up to 0x8004_0000, ending at 0x8008_0000.
• C → 0x8008_0000, aligned to 16 KB ✓, ending at 0x8008_4000.

Total consumed: 0x8000_0000 to 0x8008_4000 — 528 KB, of which 192 KB is a hole between A's end and B's start, created purely by B's alignment.

Placing the most-aligned first — B, then A, then C:

• B → 0x8000_0000, ending at 0x8004_0000. Aligned ✓
• A → 0x8004_0000, aligned to 64 KB ✓, ending at 0x8005_0000
• C → 0x8005_0000, aligned to 16 KB ✓, ending at 0x8005_4000

Total consumed: 336 KB, no hole at all.

What the example shows. Both placements satisfy every constraint — no overlap, every alignment met, everything inside the branch's 1 MB. Neither is wrong. But the first consumes 57% more space than the second, and in a branch that was tight to begin with, that difference is what makes an allocation fail.

And the parent window has to cover the result. Whichever ordering is used, the branch's window must span from the lowest base to the highest end — so a window sized against the first placement is 528 KB and against the second is 336 KB. A window sized for one ordering and then used with the other is §11's second debugging scenario.

What this chapter does not claim. Ordering by alignment is one heuristic among many, and real allocators balance it against fragmentation, address-space kind, and constraints this model does not represent. The transferable point is narrower: alignment is not a check applied after placement — it changes where things can go, and therefore how much space the set requires.

5. Hierarchical Windows

Allocation is not only about avoiding collisions between peers. It has a containment dimension that comes directly from the topology.

An access aimed at a function's region must be forwarded down the hierarchy to reach it. Every hierarchy-expanding port on the path decides whether to forward based on a window describing the address ranges below it — the address-space counterpart of the bus ranges in Chapter 7.4.

Therefore:

A parent's window must encompass every resource allocated beneath it.

The failure this creates is distinctive. A region can be correctly sized, correctly aligned, and overlapping nothing — and still be unreachable, because a parent window does not cover it. Nothing about the region itself is wrong. The fault is one level up, in a window that was sized before the child was placed, or never widened afterwards.

That is structurally the same problem as Chapter 7.4's subordinate bus number: a parent's extent depends on what is beneath it, which is not known until the subtree has been walked. The same interleaving, the same class of failure.

How windows are represented — the registers involved and their layout — is Module 8's subject. This chapter needs only that they exist and must contain their children.

6. RTL — Overlap Checking, Done Safely

// SYNTHESIZABLE. Overflow-safe intersection test for two address ranges.
// Generic range arithmetic — NOT PCIe resource-programming logic.
module range_overlap_check #(
  parameter int ADDR_W = 64
) (
  input  logic [ADDR_W-1:0] a_base,
  input  logic [ADDR_W-1:0] a_size,
  input  logic [ADDR_W-1:0] b_base,
  input  logic [ADDR_W-1:0] b_size,
 
  output logic              a_legal,
  output logic              b_legal,
  output logic              overlap
);
 
  // THE ARITHMETIC HAZARD THIS MODULE EXISTS TO AVOID:
  //
  //   wire [ADDR_W-1:0] a_end = a_base + a_size - 1;   // DO NOT DO THIS
  //
  // Two ways it breaks. If a_size is zero, the subtraction underflows and
  // a_end becomes a_base - 1, which is enormous — a zero-size range then
  // appears to cover almost the whole space. If a_base + a_size exceeds the
  // address width, the addition wraps and a_end lands BELOW a_base, so the
  // range appears to cover nothing. Both produce a confidently wrong answer.
  //
  // The fix is one extra bit and half-open intervals: compute the EXCLUSIVE
  // end in ADDR_W+1 bits, where the carry-out is visible and no -1 is needed.
  wire [ADDR_W:0] a_end_x = {1'b0, a_base} + {1'b0, a_size};   // exclusive end
  wire [ADDR_W:0] b_end_x = {1'b0, b_base} + {1'b0, b_size};
 
  // A range is legal only if it is non-empty AND does not run off the top of
  // the address space. The carry-out bit is exactly the overflow indication.
  assign a_legal = (a_size != '0) && !a_end_x[ADDR_W];
  assign b_legal = (b_size != '0) && !b_end_x[ADDR_W];
 
  // Half-open intersection: [a_base, a_end_x) meets [b_base, b_end_x) if and
  // only if each starts before the other ends. No -1, no inclusive-end
  // comparisons, no special case for adjacency — two ranges that touch
  // exactly (a_end_x == b_base) correctly do NOT overlap.
  assign overlap = a_legal && b_legal
                && (a_base < b_end_x[ADDR_W-1:0])
                && (b_base < a_end_x[ADDR_W-1:0]);
 
endmodule

Classification: synthesizable.

What it models: the pairwise non-overlap constraint at the heart of allocation.

What it teaches — and this is the most transferable content in the chapter:

1. end = base + size - 1 is a bug pattern, not a formula. It underflows on zero size and wraps at the top of the space, and both failures produce a plausible answer rather than an error. The half-open form removes both.
2. The carry-out bit is the overflow check. Computing in ADDR_W+1 bits makes "this range runs off the end of the address space" directly observable instead of something to reason about.
3. Half-open intervals make adjacency correct for free. Two ranges that touch exactly — one ending where the next begins — do not overlap, and the comparison gets that right with no special case. An inclusive-end formulation needs <= in exactly one place and < in another, which is where off-by-one errors live.

Deliberately simplified: two ranges at a time, so checking N allocations requires N(N−1)/2 invocations or a sequential sweep; one address space, with no distinction between memory and I/O; and no notion of which allocation should yield when a conflict is found.

Production implication: a real allocator must check a candidate against every existing allocation, handle several address spaces separately, apply a placement policy when a conflict is found, and — usually the hard part — work out an ordering that satisfies all alignment constraints simultaneously.

7. RTL — Alignment

// SYNTHESIZABLE. Generic power-of-two alignment test.
// NOT a model of any specific PCIe alignment requirement — Module 9 owns the
// rules governing how a function's regions must be aligned.
module alignment_check #(
  parameter int ADDR_W = 64
) (
  input  logic [ADDR_W-1:0] base,
  input  logic [ADDR_W-1:0] alignment,   // expected to be a power of two
 
  output logic              alignment_legal,
  output logic              aligned
);
 
  // The mask trick below is ONLY valid for powers of two, so legality is
  // checked rather than assumed. An alignment of, say, 24 would make
  // (base & 23) a meaningless test that silently returns plausible answers.
  assign alignment_legal = (alignment != '0)
                        && ((alignment & (alignment - 1'b1)) == '0);
 
  // base is aligned when it has no bits set below the alignment boundary.
  assign aligned = alignment_legal && ((base & (alignment - 1'b1)) == '0);
 
endmodule

Classification: synthesizable.

What it teaches: that base & (alignment - 1) is a conditional identity. It tests alignment correctly only when alignment is a power of two, and produces a well-formed wrong answer otherwise. Checking the precondition costs one comparison and converts a silent mis-test into a visible one.

Deliberately simplified: power-of-two alignment only; no relationship enforced between alignment and size.

Production implication: a real allocator derives each region's alignment requirement from the function's stated requirement (Module 9's subject) rather than receiving it as an input, and must satisfy all of them simultaneously while packing efficiently.

8. RTL — Parent Window Containment

// SYNTHESIZABLE. Does a child range lie entirely within a parent window?
// Generic containment arithmetic — NOT a model of PCIe bridge window
// registers, whose layout is Module 8's subject.
module window_containment #(
  parameter int ADDR_W = 64
) (
  input  logic [ADDR_W-1:0] parent_base,
  input  logic [ADDR_W-1:0] parent_size,
  input  logic [ADDR_W-1:0] child_base,
  input  logic [ADDR_W-1:0] child_size,
 
  output logic              parent_legal,
  output logic              child_legal,
  output logic              contained
);
 
  // Same discipline as §6: exclusive ends in ADDR_W+1 bits.
  wire [ADDR_W:0] p_end_x = {1'b0, parent_base} + {1'b0, parent_size};
  wire [ADDR_W:0] c_end_x = {1'b0, child_base}  + {1'b0, child_size};
 
  assign parent_legal = (parent_size != '0) && !p_end_x[ADDR_W];
  assign child_legal  = (child_size  != '0) && !c_end_x[ADDR_W];
 
  // Compared at full ADDR_W+1 width. Truncating to ADDR_W before comparing
  // would make a child that runs to the very top of the space appear to
  // exceed a parent that also runs to the top.
  assign contained = parent_legal && child_legal
                  && (child_base >= parent_base)
                  && (c_end_x <= p_end_x);
 
endmodule

Classification: synthesizable.

What it teaches: that the containment comparison must be done at the widened width. Both ends are computed in ADDR_W+1 bits; comparing them after truncation reintroduces exactly the wrap the extra bit was added to expose.

Deliberately simplified: one parent and one child per invocation; no walking of a multi-level hierarchy, which a real check must do for every ancestor on the path.

Production implication: a real system must verify containment at every level between a function and the root, and must size each parent window only after its subtree's allocations are known — the same ordering dependency Chapter 7.4 described for subordinate bus numbers.

9. Assertions

// SVA over the three checkers. Implementation invariants for THESE designs —
// not PCIe protocol requirements.
//
// Note on scope: these modules are combinational and define no transaction
// boundary, so every property below is a same-cycle relation over their
// inputs and outputs. Nothing here asserts an ownership or in-flight contract,
// because these interfaces do not define one.
 
// CORRECTNESS — P1: overlap agrees with the half-open definition whenever
// both ranges are legal. The property that catches an inclusive/exclusive
// mix-up, which misreports exactly the adjacent case.
property p_overlap_matches_definition;
  @(posedge clk) disable iff (!rst_n)
  (a_legal && b_legal)
    |-> (overlap == ((a_base < b_end_x[ADDR_W-1:0]) && (b_base < a_end_x[ADDR_W-1:0])));
endproperty
a_overlap_correct : assert property (p_overlap_matches_definition);
 
// SAFETY — P2: exactly adjacent ranges do not overlap. Stated separately
// because it is the single case an inclusive-end implementation gets wrong,
// and because "allocations packed back to back" is the normal situation.
property p_adjacent_not_overlapping;
  @(posedge clk) disable iff (!rst_n)
  (a_legal && b_legal && (a_end_x == {1'b0, b_base})) |-> !overlap;
endproperty
a_adjacent_ok : assert property (p_adjacent_not_overlapping);
 
// SAFETY — P3: a zero-size range is never legal and never reported as
// overlapping. Catches the underflow that `base + size - 1` produces, where a
// zero-size range appears to span almost the entire address space.
property p_zero_size_illegal;
  @(posedge clk) disable iff (!rst_n)
  (a_size == '0) |-> (!a_legal && !overlap);
endproperty
a_zero_size : assert property (p_zero_size_illegal);
 
// SAFETY — P4: a range that runs off the top of the address space is not
// legal. The carry-out is the check; this asserts it is actually consulted.
property p_overflow_illegal;
  @(posedge clk) disable iff (!rst_n)
  a_end_x[ADDR_W] |-> !a_legal;
endproperty
a_overflow_rejected : assert property (p_overflow_illegal);
 
// SAFETY — P5: overlap is never reported for an illegal range. An illegal
// range has no meaningful extent, so any answer about it would be noise
// presented as a result.
property p_no_overlap_when_illegal;
  @(posedge clk) disable iff (!rst_n)
  (!a_legal || !b_legal) |-> !overlap;
endproperty
a_illegal_no_overlap : assert property (p_no_overlap_when_illegal);
 
// LEGALITY — P6: alignment is only tested against a power of two. Catches the
// mask test being applied to a non-power-of-two, which returns a well-formed
// wrong answer.
property p_alignment_precondition;
  @(posedge clk) disable iff (!rst_n)
  aligned |-> alignment_legal;
endproperty
a_alignment_guarded : assert property (p_alignment_precondition);
 
// CORRECTNESS — P7: containment implies the child lies wholly inside the
// parent, compared at full width. Catches a truncated comparison, which
// misjudges ranges at the top of the address space.
property p_containment_is_total;
  @(posedge clk) disable iff (!rst_n)
  contained |-> ((child_base >= parent_base) && (c_end_x <= p_end_x));
endproperty
a_containment_total : assert property (p_containment_is_total);
 
// CONSISTENCY — P8: a contained child cannot extend beyond its parent, so a
// parent and a non-contained child that starts inside it must overlap only
// partially — the case that indicates a window sized before its subtree.
property p_partial_overlap_detected;
  @(posedge clk) disable iff (!rst_n)
  (parent_legal && child_legal && (child_base >= parent_base)
   && (c_end_x > p_end_x)) |-> !contained;
endproperty
a_partial_detected : assert property (p_partial_overlap_detected);

P2 deserves its own property despite looking like a special case of P1. Back-to-back allocations are the normal result of packing an address space efficiently, so the adjacent case is not rare — it is the common one. An inclusive-end implementation reports every adjacent pair as overlapping, which makes the allocator refuse perfectly valid packings and then run out of space.

P3 catches the underflow directly. A zero-size range under base + size - 1 produces base - 1 as its end, which is near the top of the address space. That range then appears to overlap almost everything, and the allocator rejects every candidate. The symptom — "allocation always fails" — points nowhere near a subtraction.

P8 is the containment failure that matters most in practice. A child that starts inside its parent and extends past the end is exactly what happens when a parent window was sized before the subtree beneath it was fully allocated. The child looks correct in isolation; only the relationship is wrong.

10. Verification

Monitors observe: every candidate range with its size and alignment, every existing allocation, each checker's legality and result outputs, and the parent windows in force.

The scoreboard independently computes expected overlap, alignment, and containment using its own arithmetic — ideally with a different formulation, such as unbounded integers in the testbench, so that a shared overflow assumption cannot hide in both. A checker and a scoreboard that both use base + size - 1 agree with each other and are both wrong.

Scenarios:

• One function. A single allocation in an empty space. Baseline.
• Several functions, non-overlapping. Verify no false conflicts.
• Exactly adjacent ranges. One ends where the next begins. Must not overlap (P2) — the case that separates a correct implementation from a plausible one.
• One-unit overlap. Ranges sharing exactly one address. Must be detected — the minimum detectable conflict.
• Full containment. One range entirely inside another. Must overlap.
• Partial overlap in each direction. Candidate starting before and ending inside; starting inside and ending after.
• Candidate entirely before, and entirely after, an existing range. No overlap in either.
• Zero size. Verify illegal and no overlap reported (P3).
• Arithmetic overflow. A base near the top of the space with a size that pushes past it. Verify illegal (P4) rather than a wrapped result. This must be written explicitly; random stimulus in the low part of a 64-bit space will never reach it.
• Alignment satisfied and violated. For several power-of-two alignments.
• Non-power-of-two alignment. Verify alignment_legal clears and aligned does not assert (P6).
• Parent window exactly fitting its children. Child ends exactly at the parent's end — contained, not excluded.
• Parent window one unit too small. The child extends one past the parent. Not contained (P8) — the boundary that catches an off-by-one in containment.
• Nested switch branch. Grandparent, parent, child. Verify containment holds at every level, and that a child contained in its parent but not its grandparent is detected — a check a single-level implementation misses entirely.
• No available space. Every candidate conflicts. Verify the failure is reported rather than an arbitrary placement chosen.

Coverage should include: overlap in each relative position; adjacency on both sides; zero size; overflow at the top of the space; each alignment value exercised aligned and misaligned; non-power-of-two alignment; containment exact-fit and off-by-one on both ends; and hierarchy depth of at least three.

11. Debugging

Symptom: the function enumerates, configuration space reads fine, the driver loads, and its MMIO accesses fail

What the successes establish — and this is a large elimination:

The Link works. The hierarchy forwards. Bus, device, and function identity are correct (Chapters 7.4–7.6). Configuration access works end to end (Chapter 7.7), which means the address-to-identity conversion, forwarding, the function's response logic, and correlation are all functioning.

Why MMIO can still fail after all that. Configuration access is reached by identity; MMIO is reached by address. They are different mechanisms over different address spaces, and everything proven above says nothing about the second.

The candidates, in order:

1. Was a resource assigned at all? A function whose region was never allocated has no address to be reached at. Check whether an assignment exists before anything else.
2. Was the assignment programmed into the function? Allocation is a decision; the function must be told the result. A decision made and not programmed leaves the function decoding a reset value.
3. Does the assigned range overlap another? Two functions claiming the same addresses produce accesses reaching the wrong one, or neither — and which happens can depend on path.
4. Do all parent windows cover it? §5's failure. The region is correct and unreachable because a bridge does not forward accesses aimed at it. This is the one most often missed, because nothing about the endpoint or its region is wrong.
5. Does the function's own decode match its assignment? The function must claim exactly the range it was given.

The measurement that splits fastest. Read back the assignment through configuration space — which works, as established. If no assignment is present, stop at candidate 1 or 2. If one is present, compare it against the parent windows on the path; a region outside any of them is candidate 4 and needs no further endpoint investigation.

Symptom: two devices each work alone, and fail when both are present

Why this is diagnostically strong. Each device working alone eliminates almost everything device-specific: its Link, its identity, its configuration access, its resource programming, and its internal decode all function.

What changes when both are present is not either device — it is the allocation.

Candidates:

• Overlapping assignments. The most direct explanation. With one device present, its range is unambiguous; with two, accesses in the overlapping region may reach either. Symptoms often look like data corruption or one device responding to another's traffic.
• Parent window conflict. Both regions are individually contained, but a window had to be widened to cover both and now also covers something it should not — or was not widened and now excludes one.
• Insufficient parent aperture. The parent window can cover either child but not both, so whichever is placed second falls outside.
• Address space exhaustion. The allocator could not place the second and either failed silently or placed it somewhere invalid.

Why the "works alone" evidence is what makes this tractable. It converts an open-ended two-device failure into a question about one data structure: the allocation table. Dump both assignments and every parent window on both paths, then run §§6–8's three checks by hand. Overlap, alignment, and containment cover the whole candidate list.

12. Common Misconceptions

• "Discovery makes a device usable." Discovery establishes that a function exists and can be identified. Its operational regions are reached by address, and no address exists until one is assigned. A discovered, unresourced function is visible and inert.
• "A function chooses its own addresses." A function states what it needs; the host decides where it goes. Functions cannot place themselves, because none has the system-wide knowledge required to avoid conflicts.
• "Requirement and assignment are the same thing." One is a property of the function, the other a decision by the host. Conflating them makes the allocation problem invisible — and the allocation problem is the entire difficulty.
• "A wider assigned range is always better." Address space is finite and shared. Over-allocating one function reduces what is available for others and can make an otherwise satisfiable set of requirements unsatisfiable.
• "Two devices can share overlapping regions if software only uses one at a time." Overlap is a property of the address decode, not of software intent. Accesses in the overlapping range are ambiguous regardless of which driver believes it owns them, and which device responds may depend on the path.
• "Bridge windows are unrelated to child allocations." A parent forwards based on its window, so a child outside it is unreachable. Windows and allocations are the same problem viewed from two levels, and a parent's extent cannot be finalised until its subtree has been placed.
• "Resource allocation is endpoint RTL." The placement decision is a host and platform responsibility. Endpoint hardware states a requirement, accepts an assignment, and decodes accordingly — three things, none of which is the allocation.
• "A successful configuration read proves MMIO will work." They are different mechanisms over different address spaces. Configuration access is reached by identity and works before any allocation exists; that is precisely why it can succeed while address-reached access fails.
• "Address conflicts are software bugs." The consequences are decoded in hardware. A conflicting assignment produces ambiguous behaviour in the address decode of two components, and the symptom appears as corruption or wrong-target responses rather than as a software error.
• "Allocation policy and the resource-description mechanism are one topic." How a function describes what it needs is Module 9. How the system decides where to put it is this chapter. The same allocation reasoning applies regardless of the description mechanism.

13. Understanding Check

14. Module 7 Complete

Eight chapters taking a system from powered hardware to usable devices.

7.1 Enumeration Overview — the six-stage lifecycle, and the division of labour between a host that orchestrates and hardware that participates.

7.2 Power-Up — enumeration cannot begin until the path and function are initialised; reset release is a dependency chain, not one event.

7.3 Device Discovery — directed probe-and-response, with absence inferred rather than reported, and a late answer that must never be attributed to the next probe.

7.4 Bus Number Assignment — numbering the hierarchy's regions so a port can decide locally whether a target lies behind it.

7.5 Device Number Assignment — a position within one bus, meaningless without it, and mostly unused below a point-to-point Link.

7.6 Function Number Assignment — one device position, several software-visible functions, and isolation that shared hardware makes an active requirement.

7.7 Configuration Access — identity plus offset becoming an access that traverses the hierarchy, and five terminal outcomes that must not be collapsed.

7.8 Resource Allocation — non-conflicting placement in a finite space, within every parent window, turning a discovered function into a usable one.

The through-line: each stage produces something the next one requires, and each fails in a way that looks like the others from the outside. Module 7's real contribution is not any single mechanism but the discipline of asking which stage — because "the device does not work" is produced identically by all eight.

15. What's Next

Module 7 used configuration space constantly and never described it. Every stage read or wrote it — to identify a function, to establish bus numbers, to learn about multifunction devices, to program an assignment — and no chapter said what it is.

Chapter 8.1 — Configuration Mechanism opens Module 8 with that question: what configuration space is as an architectural mechanism, how each function exposes a standardised interface to software, why it is emphatically not the same thing as the MMIO regions this chapter allocated, and how register access behaviour is actually built in RTL.