Fabric Utilisation

Module 15 · Chapter 15.7 · CHI Performance

Project thread — 15.6 reduced snoop traffic. 15.7 asks how evenly the remaining traffic spreads, and closes the module; 16.1 opens the RTL.

1. Learning Outcomes

By the end of this chapter you should be able to:

• Explain that real-workload fabric utilisation is non-uniform — hotspots vs idle links.
• State that throughput is limited by the maximum per-link utilisation, not the average.
• Explain why a low average can hide a link pinned at 100%.
• Describe why aggregate-bandwidth figures mislead the same way.
• Diagnose a design declared healthy on average while a hotspot caps it.
• Implement a representative fabric-utilisation model in SystemVerilog, Verilog-2001, and VHDL.

2. Why Should I Learn This?

Every earlier chapter analyzed one resource — a stage, a channel, a home, a directory. A real fabric is a spatial structure with many links, and traffic does not spread evenly across them. Some links — those toward a popular home or a memory controller — carry far more than their share and saturate, while most links are nearly idle. The fabric's performance is set by that busiest link, not by how it looks on average.

This makes the average utilisation a dangerous metric. A fabric can average 30% and still be fully bottlenecked by one link at 100% — the average is dragged down by all the idle links and hides the constraint. Aggregate-bandwidth numbers mislead the same way, summing capacity that will never all be used at once. An engineer who provisions or signs off on the average will add bandwidth to idle links and wonder why throughput did not move — because the hotspot was untouched. This closing chapter of the performance module is the discipline of looking at the max per-link utilisation, finding the hotspot, and balancing the traffic that creates it.

3. Key Terms

4. Previous Chapter Connection

This chapter is the spatial view of everything the module analyzed. Chapter 15.3 showed that a centralized home flattens scaling; here, even a distributed home set can create a hotspot if one home is more popular than the others, or if a memory controller sits on one edge of the mesh. Chapter 15.3's interleaving is the primary tool for balancing that traffic — this chapter is why you verify the balance with a per-link analysis, not just assume it.

It also ties back to Chapter 15.2's bottleneck idea, moved from channels to links. There, throughput was the min over channels on one link; here it is the min over links across the topology — the same "the busiest resource sets the rate" principle, applied to the fabric's geography. Closing the performance module, this chapter unifies the theme: performance is set by the constraint, and finding the constraint means looking at the maximum, never the average.

5. Core Concept — the hotspot, not the average, sets throughput

Real-workload utilisation is non-uniform; throughput is capped by the maximum-utilisation link (the hotspot), and the average hides it.

• Traffic is uneven. Links toward a popular home or a memory controller carry far more traffic than links in quiet corners of the mesh — a hotspot forms.
• The max sets the rate. Throughput is limited by the busiest link. Once one link hits 100%, it caps the workload regardless of how idle the others are.
• The average misleads. With one link at 100% and many near 0%, the average is low — but the low average is meaningless for throughput, which the hotspot already caps.
• Balance relieves the hotspot. Spreading the hotspot's traffic — interleaving addresses (Chapter 15.3), moving/replicating the hot home or memory controller, routing around the link — is the only fix; adding bandwidth to idle links does nothing.

The synthesis:

Fabric utilisation is non-uniform: a hotspot link (toward a popular home or memory controller) saturates while most links idle. Throughput is capped by the maximum per-link utilisation, not the average — so a fabric averaging 30% can be fully bottlenecked by one link at 100%. The average (and aggregate-bandwidth figures) hide the hotspot. The fix is to find the max-utilisation link and balance its traffic — not add bandwidth to idle links.

6. Engineering Mental Model — one jammed highway in a city

Think of a city's road network (the fabric) with traffic reports.

• Most streets are empty at a given moment, but one highway — the on-ramp to a popular stadium (a hot home node) — is gridlocked. The city's average road utilisation is low.
• Yet everyone trying to reach the stadium is stuck, because their route goes through the one jammed highway. The empty streets elsewhere do not help them.
• A traffic planner who looks at the average ("roads are only 30% used, plenty of capacity") and widens a quiet side street has spent money for zero improvement — the jam is on the stadium highway, untouched.
• The fix is to look at the worst road, find the stadium on-ramp, and balance its load — add exits, spread the destinations (interleaving), or route some traffic around it.

The jammed highway is the hotspot link; the empty streets are the idle links dragging the average down. You must look at the worst road, not the average, and fix that one.

7. Engineering Diagram — uniform vs hotspot

Top: balanced links, the average tells the truth. Bottom: one link at 100% among idle ones — the average reads low but the hotspot caps throughput. The DebugLab trusts the low average on the bottom row and misses the hotspot.

8. Average vs Maximum

The two ways to read fabric utilisation.

The rule to carry: read the maximum, not the mean. Throughput is a min/max property — set by the single most-loaded link — so the mean, which averages the hotspot together with all the idle links, systematically understates the constraint. A per-link histogram or the max utilisation exposes the hotspot immediately; the average and the aggregate sum hide it. Every optimization decision — where to add bandwidth, whether the fabric is healthy — must be made on the max, because that is the number the workload actually experiences.

9. Finding and Relieving the Hotspot

The per-link analysis and its remedies.

• Measure per-link utilisation. Instrument each link (or router port) and record its utilisation over the workload — not just the fabric-wide average.
• The max is the bottleneck. The link with the highest utilisation is the hotspot; its saturation caps throughput. Everything else is secondary until it is relieved.
• Identify the cause. Hotspots usually trace to a popular home node (poor address interleaving) or a memory controller on one edge of the mesh drawing all miss traffic.
• Balance the traffic. Relieve it by interleaving addresses to spread the hot home (Chapter 15.3), placing or replicating the hot home/memory controller, or routing some traffic around the link. Then re-measure — the hotspot may move.

The point to carry:

The average-versus-maximum trap is a specific instance of a general fallacy: summarizing a bottleneck with a central-tendency statistic. Throughput is governed by the extreme (the busiest link), and averages, by construction, pull toward the middle and away from extremes — so the more idle links there are, the more the average understates the hotspot. This is why a large, well-provisioned fabric can be more misleading than a small one: it has more idle links to dilute the hotspot's contribution to the average, so the same 100% link reads as an even lower average. The correct statistic for a bottleneck is always the max (or the tail of the distribution), never the mean. The corollary for capacity planning is that fabric health cannot be certified by an aggregate or average number — it requires the per-link distribution, and specifically its peak. A design signed off on average utilisation has certified the wrong statistic, and the hotspot it missed is exactly the thing that will cap the product in the field.

10. Diagnosing a Fabric — average vs max

The same 16-link fabric, read two ways.

1. Per-link measurement. 15 links average 20% utilisation; one link — to the memory controller — is at 100%. The workload is memory-miss-heavy, so all miss traffic converges on that link.
2. Average reading. (15 × 20% + 1 × 100%) ÷ 16 = 25% average. "The fabric is 25% utilised — plenty of headroom." Wrong — throughput is capped by the 100% link.
3. Maximum reading. The max link is at 100% — the fabric is saturated at the memory-controller link. That is the real state, and the bottleneck to attack.
4. The fix. Add memory-controller ports / spread miss traffic across multiple controllers (balance), relieving the hot link. Widening any of the 15 idle links would have done nothing.

The average said 25% (healthy); the max said 100% (saturated). Only the max matched the workload's real experience. The DebugLab trusts the 25% average and misses the saturated link.

11. RTL / Hardware View — a fabric-utilisation model

Report the maximum (hotspot) and the average per-link utilisation; the max is the bottleneck. Representative analysis model.

// Representative fabric-utilisation model (educational).
// Given per-link utilisation, the HOTSPOT (max) is the real throughput bottleneck; the
// AVERAGE is diluted by idle links and misleads. Report both, but plan on the max.
module chi_fabric_util #(parameter NLINK = 16, parameter W = 8) (
  input  logic [W-1:0]  util [NLINK],   // per-link utilisation (0..100)
  output logic [W-1:0]  max_util,        // hotspot link utilisation (the bottleneck)
  output logic [W-1:0]  avg_util,        // mean (misleading when hotspots exist)
  output logic          saturated        // hotspot at/above 100 (or a threshold)
);
  logic [W+8:0] sum;
  always_comb begin
    max_util = '0;
    sum      = '0;
    for (int i = 0; i < NLINK; i++) begin
      if (util[i] > max_util) max_util = util[i];   // hotspot = max
      sum += util[i];
    end
    avg_util  = sum / NLINK;                         // mean (diluted)
    saturated = (max_util >= 8'd100);                // the hotspot caps throughput
  end
endmodule

The same behavior in Verilog-2001:

// Representative fabric-utilisation model (Verilog-2001, flattened links).
module chi_fabric_util #(parameter NLINK = 16, parameter W = 8) (
  input  [NLINK*W-1:0] util_flat,
  output reg [W-1:0]   max_util,
  output reg [W-1:0]   avg_util,
  output               saturated
);
  integer i; reg [W-1:0] u; reg [W+8:0] sum;
  always @* begin
    max_util = {W{1'b0}}; sum = {(W+9){1'b0}};
    for (i = 0; i < NLINK; i = i + 1) begin
      u = util_flat[i*W +: W];
      if (u > max_util) max_util = u;      // hotspot = max
      sum = sum + u;
    end
    avg_util = sum / NLINK;                 // mean (diluted)
  end
  assign saturated = (max_util >= 8'd100);
endmodule

And in VHDL:

-- Representative fabric-utilisation model (VHDL).
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
 
entity chi_fabric_util is
  generic ( NLINK : integer := 16; W : integer := 8 );
  port (
    util_flat : in  std_logic_vector(NLINK*W-1 downto 0);
    max_util  : out unsigned(W-1 downto 0);
    avg_util  : out unsigned(W-1 downto 0);
    saturated : out std_logic
  );
end entity;
 
architecture rtl of chi_fabric_util is
begin
  process (util_flat)
    variable mx  : unsigned(W-1 downto 0);
    variable sum : unsigned(W+8 downto 0);
    variable u   : unsigned(W-1 downto 0);
  begin
    mx := (others => '0'); sum := (others => '0');
    for i in 0 to NLINK-1 loop
      u := unsigned(util_flat((i+1)*W-1 downto i*W));
      if u > mx then mx := u; end if;       -- hotspot = max
      sum := sum + resize(u, W+9);
    end loop;
    max_util  <= mx;
    avg_util  <= resize(sum / NLINK, W);     -- mean (diluted)
    if mx >= 100 then saturated <= '1'; else saturated <= '0'; end if;
  end process;
end architecture;

All three compute both max_util (the hotspot, the real bottleneck) and avg_util (the diluted mean), and flag saturated from the max. The DebugLab plans on avg_util and ignores max_util.

12. Verification View — the hotspot, not the average, is the bottleneck

The properties tie the bottleneck to the max: max ≥ average, and saturation is a max property.

// Bind to chi_fabric_util.
// 1. The reported max is >= every link (it is the true hotspot).
property p_max_is_hotspot;
  logic [W-1:0] m; int i;
  @(*) (1) |-> (max_util >= util[0]);  // extended to all i in the sweep
endproperty
 
// 2. The max is always >= the average (the average never exceeds the hotspot).
property p_max_ge_avg;
  @(*) max_util >= avg_util;
endproperty
 
// 3. Saturation is a property of the MAX, not the average.
property p_saturation_from_max;
  @(*) saturated == (max_util >= 8'd100);
endproperty

The system point, beyond the checks:

p_max_ge_avg looks trivial — of course the maximum is at least the mean — but it encodes the entire lesson of the chapter and, arguably, of the whole module: the constraint lives at the extreme, and the summary statistic sits below it. The gap between max_util and avg_util is precisely the danger zone: the larger that gap, the more the average understates the real bottleneck, and a fabric with one hot link among many idle ones has the largest gap of all. So a useful health check is not "is the average low?" but "how far apart are the max and the average?" — a wide spread is the signature of a hotspot that aggregate metrics are hiding. This reframes performance certification as a distribution question rather than a single-number one: report the peak, report the spread, and treat a large max-minus-average gap as a red flag demanding a balancing pass. Closing the performance module, it is the same truth every chapter arrived at from a different direction — latency's dominant stage, throughput's bottleneck channel, scaling's serial point, the directory's coverage threshold, and now the fabric's hottest link. Performance is always set by the worst part, and the engineering discipline is to measure the worst part, not the average.

• What it proves: the max is the hotspot and ≥ the average; saturation follows the max.
• What it does not prove: the real per-link traffic distribution — that requires workload profiling.
• Bug signature: a decision made on avg_util while max_util is saturated.

13. Testbench — the average must not hide a saturated link

Builds a hotspot profile and confirms the max flags saturation while the average looks low.

module tb_chi_fabric_util;
  localparam NLINK = 16, W = 8;
  logic [W-1:0] util [NLINK];
  logic [W-1:0] max_util, avg_util;
  logic saturated;
  int errors = 0, i;
 
  chi_fabric_util #(.NLINK(NLINK), .W(W)) dut (.*);
 
  initial begin
    // 15 links at 20%, one link (the memory controller) at 100% -> hotspot.
    for (i = 0; i < NLINK; i++) util[i] = 20;
    util[7] = 100;   // the hotspot link
    #1;
    $display("max=%0d avg=%0d saturated=%0b", max_util, avg_util, saturated);
    // Average is low (~25%) but the fabric is saturated at the hotspot.
    if (avg_util > 40) begin errors++; $display("FAIL avg unexpectedly high"); end
    else $display("PASS average looks low (%0d%%) -- would mislead", avg_util);
    if (!saturated) begin errors++; $display("FAIL hotspot not flagged as saturated"); end
    else $display("PASS hotspot detected: max=%0d%% (real bottleneck)", max_util);
    if (max_util !== 100) begin errors++; $display("FAIL max wrong"); end
 
    // Balanced fabric: all links 60% -> not saturated, average = max.
    for (i = 0; i < NLINK; i++) util[i] = 60;
    #1;
    if (saturated) begin errors++; $display("FAIL balanced fabric flagged saturated"); end
    else $display("PASS balanced: max=%0d avg=%0d (agree)", max_util, avg_util);
 
    if (errors == 0) $display("ALL TESTS PASSED");
    else             $display("%0d FAILURE(S)", errors);
    $finish;
  end
endmodule

Expected output:

max=100 avg=25 saturated=1
PASS average looks low (25%) -- would mislead
PASS hotspot detected: max=100% (real bottleneck)
PASS balanced: max=60 avg=60 (agree)
ALL TESTS PASSED

14. DebugLab — planning on the average, missing the hotspot

16 links: 15 at ~20%, 1 at 100% (link to the memory controller)
average = (15*20 + 100)/16 = 25%  -> "plenty of headroom" (WRONG)
maximum = 100%  -> the fabric is SATURATED at the hot link
throughput capped by the hot link; the 15 idle links cannot help
"fix": widen an idle 20% link -> no change (it was never the bottleneck)
correct: read max=100% -> balance miss traffic across multiple MCs -> relieve hotspot

15. Common Mistakes

• Judging health on the average. Assumption: low average = headroom. Bug: hotspot missed (the DebugLab). Prevention: read the max.
• Trusting aggregate bandwidth. Assumption: summed capacity is deliverable. Bug: counts unusable links. Prevention: per-link analysis.
• Widening idle links. Assumption: more bandwidth helps. Bug: the hotspot is untouched. Prevention: relieve the max link.
• Ignoring spatial non-uniformity. Assumption: traffic is even. Bug: converged hotspots. Prevention: measure per link.
• Not re-measuring after balancing. Assumption: one fix suffices. Bug: the hotspot moves. Prevention: iterate.
• Confusing with channel bottleneck. Assumption: 15.2 covers it. Bug: channel vs link conflated. Prevention: 15.2 is channels; 15.7 is topology links.

16. Engineering Checklist

• Measure per-link utilisation, not just the fabric average.
• Find the maximum (hotspot) link — the real bottleneck.
• Treat a large max-minus-average gap as a hotspot red flag.
• Trace the hotspot to its cause (popular home, memory controller).
• Balance the traffic — interleave, place/replicate, or route around.
• Re-measure after balancing — the hotspot may move.

17. Key Takeaways

• Real-workload fabric utilisation is non-uniform — hotspots vs idle links.
• Throughput is capped by the maximum-utilisation link, not the average.
• A low average can hide a link pinned at 100%.
• Aggregate-bandwidth figures mislead the same way.
• Find the hotspot by the max, then balance its traffic.
• Certify on the peak, not the mean; the model here is representative.

18. Quick Revision

Fabric utilisation. Real workloads load a mesh or ring non-uniformly: traffic converges on links toward a popular home node or a memory controller, so one link becomes a hotspot pinned near 100% while most links sit nearly idle. Because throughput is capped by the busiest link, not the average, a fabric can report a comfortably low average utilisation — say 30% — and yet be fully saturated at the one hot link. The average is a central-tendency statistic diluted by the idle links, and it sits below the extreme that actually sets throughput; aggregate-bandwidth figures mislead the same way by summing capacity that never all runs at once. The failure to avoid: judging fabric health (or provisioning) on the average, concluding there is headroom, and adding bandwidth to idle links — while the hotspot keeps capping the workload, so nothing improves. The discipline: analyse the maximum per-link utilisation (a per-link histogram) to find the hotspot, treat a large max-minus-average gap as a red flag, and balance the hotspot's traffic — interleave addresses (Chapter 15.3), place or replicate the hot home/memory controller, or route around it — then re-measure. Certify on the peak, never the mean. This is the module's recurring truth: performance is set by the worst part. Representative model; this closes Module 15.

Coming Next

Chapter 16.1 — Request Processing. Module 15 analyzed CHI's performance; Module 16 turns to the RTL that implements it. Chapter 16.1 opens CHI RTL Design Thinking with the request-processing pipeline — how a home node receives an incoming request, looks it up in the directory, allocates a tracker entry, and launches the resulting snoops or memory access, and the pipeline structure that keeps it throughput-efficient.