Skip to content
VLSI Mentor

DDR · Module 32

GPUs

One requester population clears the latency-hiding condition by a factor of tens and another cannot clear it at all, and the same controller serves both. They are coupled — the small one gates the large one's work launch — so a throughput-maximising tuning is net negative below a computable launch interval.

Chapter 31.4 §4 established when latency stops being observable: a requester must sustain at least ceil(L / S) transactions outstanding, and a faster channel raises that bar rather than lowering it.

Every platform in this module so far has had one answer to that condition. This one has two, and they are at opposite extremes.

The dominating constraint is a bimodal requester mix. One population clears the latency-hiding condition by a factor of tens; the other cannot clear it at all. They are served by the same controller, under the same thirteen obligations, and no single tuning serves both.

And the two populations are not independent, which is the finding that makes the chapter worth writing.

The small latency-sensitive population sits on the critical path of launching the large one's work. So starving it to maximise throughput reduces the throughput population's utilisation — and a throughput-only metric cannot see the loss it caused.

Axis A1 is identical for the third chapter in this module. The same destructive read, the same thirteen obligations of 31.1 §5. What differs is who is asking and how patient they are, and §5 shows the two answers cannot be averaged.

1. The Shared Baseline, and This Chapter's Question

CURRICULUM-DERIVED from 32.1 §1: all five platform classes carry the same thirteen obligations of 31.1 §5, none removes any, and the module's question is always the same three steps.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
   1  WHICH CONSTRAINT
        a BIMODAL requester mix -- two populations whose answers to
        31.4 §4's latency condition are at opposite extremes (§3)

   2  WHAT DECISION
        SEPARATE the populations rather than averaging them, because
        no single tuning serves both and they are COUPLED (§5, §6)

   3  WHAT GRADE
        almost entirely D. The mechanisms are generic and the
        platform figures are not published (§8)

And one contrast with 32.1 worth drawing immediately, because the two are easily conflated.

Chapter 32.1 §3's classes and this chapter's populations are orthogonal. A class is what kind of request this is — demand, prefetch, writeback, translation — and it is a property of the mechanism that generated it. A population is how patient the requester is, which is a property of the requester's own structure. Both populations here contain all four classes, so the class taxonomy still applies and is not what this platform is about.

2. One Controller, Two Populations

The two populations, described by their structure rather than by their names.

Population T — throughput-shapedPopulation L — latency-sensitive
Structuremany independent lanes of work, each able to issue and waita short dependency chain — each step needs the previous result
Outstanding capabilityvery large — thousands of requests in flightvery small — bounded by the chain, often single digits
Answer to 31.4 §4's conditionclears it by tens of times overcannot clear it at all
What it wants from the controllerthroughput — and it is indifferent to Llow L — and it is indifferent to throughput
Share of requeststhe large majoritya small minority
Share of wall-clock consequenceits throughput is the visible outputit gates when T can start — §6

Rows five and six together are the chapter. The population with the small share has the large consequence, which is structurally the same finding as 32.1 §7's — share and importance are anti-correlated again, for a different reason.

And the mechanism behind row six is worth naming concretely. Before a throughput population can run, something must fetch its work description — a command, a descriptor, a pointer table, a state block. CURRICULUM-DERIVED from 29.3 §11 for the general shape: a prerequisite fetch is traffic nobody wrote, and the work that needed it cannot proceed until it returns. That is a dependency chain by construction, so population L exists on every platform of this class and is not an implementation choice.

What makes this platform class distinctive is the ratio. Both populations exist on a CPU platform too — a pointer chase is population L and a streaming traversal is population T, and 29.3 §5 and 29.3 §6 own both scenarios. On this platform class the throughput population is larger by orders of magnitude, so the temptation to tune exclusively for it is much stronger — and §6 is why yielding to it can be self-defeating.

3. The Condition, Applied Twice

CURRICULUM-DERIVED from 31.4 §4, which owns the condition and its direction:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
   latency is INVISIBLE to a requester iff   N >= ceil( L / S )

     L  the memory's completion latency, in cycles
     S  the spacing between successive returns the channel sustains
     N  the transactions the requester keeps outstanding

   and 31.4 §4's result: HALVING S DOUBLES the required N. A faster
   channel raises the bar.

Evaluate it once for each population. Grade D throughout — the figures are invented and the ratio is the transferable part.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
   GRADE D.  L = 200 cycles,  S = 4 cycles.
   DERIVED:  N* = ceil(200 / 4) = 50 outstanding transactions.

     population T :  N_T = 4096
         DERIVED  N_T / N*  =  81.9x
         -> latency is invisible. Raising L changes NOTHING
            observable for T until L exceeds 4096 * 4 = 16,384.

     population L :  N_L = 2   (a two-deep dependency chain)
         DERIVED  N_L / N*  =  0.04
         -> every cycle of L is SERIALISED. L pays 200 cycles per
            access, and doubling L doubles its cost exactly.

Two results, and the second is the one that forbids averaging.

The same L is free for one population and fully charged to the other. DERIVED: T's cost from L is zero within an enormous margin; L's cost is linear in L with slope one. So L is not a shared quantity with a shared cost — it is a quantity with two completely different price tags, and a single figure of merit cannot represent both.

And the margins are wildly asymmetric, which makes the tuning direction asymmetric too. DERIVED: T has 81.9× headroom, so a policy that doubles L still leaves it 41× clear. L has none, so the same policy doubles its cost. There is no symmetric trade here: one side's margin is effectively infinite and the other's is zero.

4. Why You Cannot Tune for Either

Take the two obvious tunings and evaluate both populations under each. All figures grade D; the signs are structural.

CURRICULUM-DERIVED from 23.4, which owns what reordering gains in throughput and costs in latency, and from 23.1 §9, which owns why the average is the wrong statistic.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
   TUNING 1 -- for THROUGHPUT.
     deep reorder window, long direction batches, coarse access
     granularity, aggressive row-hit preference (23.4)

     effect on S : improves. say S: 4 -> 3.4  (+18% throughput)
     effect on L : worsens.  say L: 200 -> 400 (deeper queues,
                   longer batches ahead of any given request)

     population T :  N* becomes ceil(400/3.4) = 118
                     N_T/N* = 4096/118 = 34.7x   -- still clear
                     gain: +18% throughput.  COST: none.
     population L :  cost 200 -> 400 cycles.  DOUBLED.

   TUNING 2 -- for LATENCY.
     shallow queues, no batching, near-FCFS service

     effect on L : improves. say L: 200 -> 120
     effect on S : worsens.  say S: 4 -> 5.2  (-23% throughput)

     population T :  loses 23% of its throughput -- its VISIBLE
                     output.
     population L :  cost 200 -> 120.  -40%.

So the trade is real in both directions and the two populations disagree completely. CURRICULUM-DERIVED from 30.8 §7: every optimisation has a cost that must be named, and here the cost lands entirely on the other population.

And the averaging failure is arithmetic rather than rhetorical. DERIVED, taking population shares of 97% T and 3% L:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
   mean latency under tuning 1:
       0.97 * (T's latency, which it does not feel) + 0.03 * 400

   the first term is the problem: T's requests DO have a latency
   and T DOES NOT CARE. Including them in a mean produces a number
   dominated by requests nobody is waiting for.

   23.1 §9 owns this exactly -- the average is the wrong statistic --
   and here it is worse than wrong: the 97% that dominates the mean
   is precisely the part with no deadline.

So a mean latency on this platform is a metric that improves when the wrong thing gets better. The correct instrument is a latency distribution per population, and §15's monitor keeps them separate for that reason.

5. The Coupling — Why Tuning for Throughput Can Lose Throughput

§3 and §4 treated the populations as independent. They are not, and this section is the chapter's sharpest result.

Population L's accesses are on the critical path of starting population T's work. A work launch requires the description to arrive; the description is a dependency-chained fetch; so every launch pays L's latency, serialised.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
   DERIVED from a stated model. GRADE D.

     launch interval        I = 2,000 cycles  (a new batch of T work
                                is dispatched this often)
     L's latency per launch = the chain's cost, serialised

   under TUNING 1 (§4): L's cost rises 200 -> 400 cycles.

     extra stall per launch      = 200 cycles
     as a fraction of the period = 200 / 2000 = 10.0%

   so T's UTILISATION falls by 10% -- it spends 10% more of each
   period waiting to be told what to do.

   and the throughput gain from tuning 1 was +18% (§4).

     net effect = 1.18 x (1 - 0.10) = 1.062
       -> +6.2%, not +18%. Two thirds of the gain is consumed by
          the coupling.

   recompute at I = 800 cycles (a finer-grained workload):
     stall fraction = 200 / 800 = 25.0%
     net = 1.18 x 0.75 = 0.885   ->  -11.5%. NET NEGATIVE.

DERIVED, and the crossover is worth stating as a condition rather than a number:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
   tuning for throughput is net positive iff

       (1 + g)  x  (1 - dL / I)  >  1

     g   the throughput gain
     dL  the increase in L's serialised cost
     I   the launch interval

   so the break-even launch interval is

       I*  =  dL x (1 + g) / g

   DERIVED with g = 0.18, dL = 200:
       I* = 200 x 1.18 / 0.18 = 1,311 cycles

   workloads with a launch interval SHORTER than ~1,300 cycles lose
   from this tuning. Longer ones gain.

Three consequences, and the first is the one a throughput-only metric hides.

A throughput-maximising policy can reduce delivered throughput, and the instrument that chose it cannot see the loss. CURRICULUM-DERIVED from 30.8 §2's ladder: the tuning improved rung 4 — controller-achieved bandwidth — while rung 5, application-observed, fell. That chapter's whole argument is that rung 1 or 4 moving does not move rung 5, and this is a mechanism by which rung 4 moving lowers rung 5.

The deciding quantity is the launch interval, which is a property of the workload. So the correct tuning is workload-dependent and the crossover is computable — which makes it a measurement rather than a preference. CURRICULUM-DERIVED from 23.5: neither fixed policy wins across workloads, and an adaptive one needs hysteresis as a requirement. The same conclusion applies here with a different quantity underneath it.

And the coupling is one-directional, which is why it is easy to miss. T's latency does not affect L at all — L's chain does not wait for T. So the interaction shows up only when you measure T's utilisation rather than its bandwidth, and bandwidth is what everybody measures. §19's row four is that measurement.

6. Separate, Do Not Average

The decision the constraint forces, and it has three parts.

Part 1 — the populations must be distinguishable at the controller. Like 32.1 §6's class labelling, this is an integration property: if the upstream protocol does not carry a population identifier, the controller cannot separate what it was not told. And unlike a class, a population cannot be inferred from the request — a demand read from a dependency chain and a demand read from a streaming lane are identical in every field except who sent them.

Part 2 — the resources must be partitioned, not shared with a policy. This is the part §13's defect gets wrong, and the argument is 17.4 §9's: a preference layer with no bounded override starves. With outstanding counts of 4096 against 2, any shared resource served by preference gives L an unmeasurable share — so the partition must be structural rather than policy-based.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
   what must be partitioned, and why sharing fails for each

     ADMISSION CREDITS   T can hold every credit. L then waits at
                         admission, and its wait is a function of
                         T's occupancy -- unbounded in L's terms.
                         §13's defect.

     QUEUE ENTRIES       L's entries sit behind T's thousands.
                         Head-of-line blocking across populations.

     REORDER WINDOW      a window deep enough to serve T's
                         throughput is deeper than L's whole chain.

   what must NOT be partitioned

     LEGALITY            the thirteen obligations are properties of
                         the DRAM and know nothing about who asked
                         (31.1 §5, 32.1 §8). Partitioning legality
                         would be 17.4 §3's "inverse error, which is
                         worse".

Part 3 — the policy must be per-population and the bound must be per-population too. CURRICULUM-DERIVED from 30.9 §5 Q2: a progress bound must be justified, and here the justification differs by population — L needs a tight bound because its bound is its latency, and T needs a loose one because its bound is irrelevant. One bound serving both is either unprovable or useless, which is the same structure 32.2 §6 found for throttle levels.

7. The Attach Choice Interacts With the Mix

This platform class is the one where 31.3's and 31.4's conclusions both apply, and they interact with §3 in a way neither chapter considered.

CURRICULUM-DERIVED from 31.4 §4: halving S doubles the required N. So a faster attach raises the bar N*.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
   DERIVED, grade D. Same L = 200, two attaches.

     attach 1 (perimeter, higher per-pin rate):  S = 4
         N* = 50
         T : 4096/50  = 81.9x clear
         L : 2/50     = cannot clear

     attach 2 (on-package, many narrow channels): S = 1.5
         N* = ceil(200/1.5) = 134
         T : 4096/134 = 30.6x clear
         L : 2/134    = cannot clear, and the RATIO is worse

   DERIVED: the faster attach helps T (more throughput) and moves L
   FURTHER from the condition it already could not meet.

So the attach decision is not neutral with respect to the mix, and the direction is counter-intuitive: the better attach makes the bimodality worse. Population T gains; population L's relative position deteriorates, because N* rose and N_L did not.

And the qualification that keeps this honest. L is not held constant across attaches in reality — CURRICULUM-DERIVED from 31.3 §3, the on-package attach has shorter transit and the same access, so its L is somewhat lower. DERIVED: if L falls to 160 while S falls to 1.5, N* becomes 107 rather than 134 — still a rise. The S term moves faster than the L term, so the conclusion survives the correction, and saying so is better than asserting it.

The design consequence: a platform choosing the faster attach must spend some of the gain on population L, either by shortening its chain upstream or by partitioning resources more aggressively in its favour. CURRICULUM-DERIVED from 26.4 §7: one requester cannot fill them — and population L is precisely a requester that cannot, so the many-channel attach gives it nothing.

8. The Evidence Situation

§Scope's grades, and for this platform class the shape is the poorest in the module.

What you might wantBest grade availableWhy
That a bimodal mix exists on this class of platformSTRUCTURALit follows from a work-launch dependency chain, not from a product — §2
HBM's interface width and per-stack bandwidthA, cited26.4 §1, attributing 4.8 §2 and 26.2 §8
That height changes capacity and leaves bandwidth aloneA26.4 §1 proved it with a property
A particular GPU's outstanding-transaction capabilitynonenot published, and it is per-engine and per-configuration
A particular GPU's memory scheduler policynonenot published
A particular GPU's launch intervalnone, and it is not a hardware propertyit is a property of the workload — §5
N_L, the dependency chain's depthC or Dinferable by measurement on specific hardware and software

Two observations, and the second is why this chapter is structured as it is.

Almost nothing about this platform class is grade A except what Module 26 already verified. So a chapter that promised platform specifics would be delivering grade C and D dressed as A18.4 §1's category drift, manufactured deliberately. The honest output is the structural argument plus a computable crossover, which is what §5 delivers.

And the deciding quantity is not a hardware property at all. The launch interval I belongs to the workload, so even a fully documented GPU would not tell you which tuning is right — §5's I* must be measured against the software that will run. CURRICULUM-DERIVED from 18.4 §5: asking what is this GPU's memory tuning is a category error of the same kind as asking for the address map, and for a stronger reason — the answer depends on a program that has not been written yet.

9. The Platform, as Blocks

A block diagram of a GPU-class memory platform organised around two requester populations. On the left, population T, the throughput-shaped population, consists of many independent lanes able to keep thousands of requests outstanding, and clears the latency-hiding condition by tens of times over. Below it, population L, the latency-sensitive population, is a short dependency chain that keeps only a handful of requests outstanding and cannot clear the condition at all. A crucial edge runs from population L back to population T, labelled launch, because L's accesses fetch the work descriptions that T needs before it can start; this is the coupling that makes a throughput-maximising policy potentially self-defeating. Both populations feed separate admission credit pools, because with outstanding counts of four thousand against two any shared pool lets the throughput population exhaust the other's admission entirely; chapter twelve's defect is a single shared pool. The two credit pools feed separate queues with separate policies, because a reorder window deep enough for the throughput population is deeper than the latency population's whole chain. Both queues converge on the legality filter, which must not be partitioned because the thirteen shared obligations are properties of the DRAM and know nothing about who asked. Legality feeds the commit point and then the memory device. The per-population monitor observes both admission paths and reports outstanding counts and latency distributions separately, because a mean over the combined mix is dominated by requests nobody is waiting for.Population TthousandsoutstandingPopulation La dependency chainT credit poolpartitionedL credit poolpartitionedT queue + policydeep reorderL queue + policyshallow, near-FCFSPer-pop monitornever averagedLegality filterNOT partitionedCommit pointstate may changeMemory devicethirteen obligationslaunchreqreqadmadmcandcandlegalcmd12

The edge from L back to T is the only feedback edge, and it is the chapter. Population L gates when population T can start, so the diagram's smallest block sits upstream of its largest one. CURRICULUM-DERIVED from 30.8 §4: that edge is how a rung-4 improvement produces a rung-5 regression.

And the legality block's caption is the one thing deliberately not duplicated. Two credit pools, two queues, two policies — one legality filter. CURRICULUM-DERIVED from 17.4 §3, partitioning legality would be the inverse error, which is worse: a per-population legality filter has two sources of truth about one device, and the device has no populations.

10. RTL — Partitioned Admission

The platform-profile block for this chapter. NUM_POP is the platform property, and the generate boundary shows that a single-population platform needs none of the partitioning — which is why 32.1's and 32.2's controllers did not have it.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// ---------------------------------------------------------------------
// population_admission -- the platform-profile block of §10.
//
// CLASSIFICATION: synthesisable, grade-D parameter values, CORRECT as
// written. The intentionally defective block is §13.
//
// WHAT IT IS: §6's part 2 as hardware -- a STRUCTURAL partition of
// admission credits, not a preference over a shared pool. With
// outstanding counts of 4096 against 2 (§3), any shared resource
// served by preference gives population L an unmeasurable share, and
// 17.4 §9 owns why: a preference layer with no bounded override
// starves.
//
// WHY IT EXISTS HERE: §5 derives that starving L can reduce T's
// delivered throughput, and §4 that no single tuning serves both. So
// the response is separation, and separation of a shared credit pool
// is the part that must be structural rather than policy.
//
// HOW TO RUN IT: backlog population T to its full credit allocation
// and then present a single population-L request.
// EXPECTED RESULT: L is admitted within its own allocation,
// independent of T's occupancy.
// EXPECTED TRACE: l_admit must rise while t_credits_used is at its
// maximum. §14's block cannot do this.
//
// SYNTHESIS: one credit counter per population plus a small compare.
//
// LIMITATIONS: partitions ADMISSION. It does not partition queues or
// reorder windows -- §6 lists those as also requiring partition, and
// 17.2 owns the entry pool. Nor does it partition LEGALITY, which
// §6 and 17.4 §3 both say must never be partitioned. Those
// boundaries are STATED and are not the bug.
// ---------------------------------------------------------------------
module population_admission #(
  // The platform property. A single-population platform needs no
  // partition, so the mechanism is ABSENT rather than disabled.
  parameter int NUM_POP    = 2,

  // Grade D allocations. On a real platform these are CONFIGURATION
  // inputs, for 32.1 §13's reason: an allocation is a platform
  // property and a platform property in a parameter is grade D
  // consumed as grade A. They are parameters here only to keep the
  // block readable, and the elaboration guard below is what a
  // runtime version would check instead.
  parameter int CREDITS_T  = 512,
  parameter int CREDITS_L  = 8,

  // COUNT, not INDEX: each counter must REPRESENT its allocation
  // inclusive, so it needs $clog2(allocation + 1) bits. Sized
  // $clog2(allocation) a full pool wraps to empty and admission
  // becomes unbounded -- the counter failing inside the exhaustion
  // case it exists to detect.
  parameter int CT_W       = $clog2(CREDITS_T + 1),
  parameter int CL_W       = $clog2(CREDITS_L + 1)
)(
  input  logic              clk,
  input  logic              rst_n,

  // Population T.
  input  logic              t_req,
  input  logic              t_done,
  output logic              t_admit,

  // Population L.
  input  logic              l_req,
  input  logic              l_done,
  output logic              l_admit,

  output logic [CT_W-1:0]   t_credits_used,
  output logic [CL_W-1:0]   l_credits_used,
  output logic              t_exhausted,
  output logic              l_exhausted,
  output logic              credit_underflow
);
  initial begin
    if (NUM_POP < 1) $fatal(1, "population_admission: NUM_POP >= 1");
    if (CREDITS_T < 1 || CREDITS_L < 1)
      $fatal(1, "population_admission: each allocation must be >= 1");
    // §6 part 2: L's allocation must at least cover its chain depth,
    // or the partition is present and useless. A chain of depth N_L
    // that cannot hold N_L credits serialises on admission as well as
    // on latency -- 31.2 §7's "present in the source, absent in the
    // silicon", applied to an allocation.
    if (NUM_POP > 1 && CREDITS_L < 2)
      $fatal(1, "population_admission: CREDITS_L < 2 cannot cover a dependency chain (§6)");
  end

  generate
  if (NUM_POP > 1) begin : g_partitioned

    logic [CT_W-1:0] ct;
    logic [CL_W-1:0] cl;

    // Two INDEPENDENT counters. This is the whole mechanism, and its
    // simplicity is the argument: a partition needs no arbitration,
    // no weights and no fairness bound, because neither population
    // can reach the other's resource at all.
    assign t_exhausted = (ct >= CREDITS_T[CT_W-1:0]);
    assign l_exhausted = (cl >= CREDITS_L[CL_W-1:0]);

    assign t_admit = t_req && !t_exhausted;
    assign l_admit = l_req && !l_exhausted;

    always_ff @(posedge clk) begin
      if (!rst_n) begin
        ct <= '0;  cl <= '0;  credit_underflow <= 1'b0;
      end else begin
        // Each population's counter moves only on its own events.
        // §6's part 1 is what makes this possible: without a
        // population identifier on the request, the controller
        // cannot separate what it was not told, and a population
        // CANNOT be inferred from the request's fields.
        unique case ({t_admit, t_done})
          2'b10 : ct <= ct + 1'b1;
          2'b01 : begin
                    if (ct == '0) credit_underflow <= 1'b1;
                    else          ct <= ct - 1'b1;
                  end
          default : ;   // both or neither: net zero
        endcase
        unique case ({l_admit, l_done})
          2'b10 : cl <= cl + 1'b1;
          2'b01 : begin
                    // A completion with no outstanding request is an
                    // ERROR, not a saturating subtract. Absorbing it
                    // lets the counter drift and every figure below
                    // it becomes meaningless -- 30.8 §11's rule.
                    if (cl == '0) credit_underflow <= 1'b1;
                    else          cl <= cl - 1'b1;
                  end
          default : ;
        endcase
      end
    end

    assign t_credits_used = ct;
    assign l_credits_used = cl;

  end else begin : g_single

    // One population: no partition exists, and the single pool is
    // correct rather than a compromise.
    logic [CT_W-1:0] ct;
    assign t_exhausted    = (ct >= CREDITS_T[CT_W-1:0]);
    assign l_exhausted    = 1'b0;
    assign t_admit        = t_req && !t_exhausted;
    assign l_admit        = 1'b0;
    assign l_credits_used = '0;
    always_ff @(posedge clk) begin
      if (!rst_n) begin ct <= '0; credit_underflow <= 1'b0; end
      else begin
        unique case ({t_admit, t_done})
          2'b10 : ct <= ct + 1'b1;
          2'b01 : begin
                    if (ct == '0) credit_underflow <= 1'b1;
                    else          ct <= ct - 1'b1;
                  end
          default : ;
        endcase
      end
    end
    assign t_credits_used = ct;

  end
  endgenerate
endmodule

The block's most important property is what it does not contain. No arbiter, no weights, no fairness bound, no age threshold. CURRICULUM-DERIVED from 17.4 §9, all of that machinery exists to make a shared resource fair — and a partition removes the need for it by removing the sharing. With a 2048:1 ratio in outstanding capability, a partition is cheaper and more provable than any fairness mechanism that could achieve the same effect.

11. RTL — Partitioning the Queue

§6 part 2 listed three resources that must be partitioned and §10 partitioned only the first, saying so in its limitations. This block is the second, and it matters independently: credits govern whether a request is admitted, and queue entries govern how long it waits once it is. A design with partitioned credits and a shared queue still exhibits head-of-line blocking across populations.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// ---------------------------------------------------------------------
// population_queues -- CORRECT as written. Grade-D depths.
//
// CLASSIFICATION: synthesisable. Consumes 17.2's entry discipline and
// does not reimplement it.
//
// WHAT IT IS: §6 part 2's second partition -- separate entry pools
// with separate reorder windows, so a population-L entry never sits
// behind a population-T backlog.
//
// WHY IT EXISTS HERE: §10 partitions ADMISSION. A request admitted
// into a shared queue has merely moved its wait from the credit stage
// to the queue stage, and §5's coupling does not care which stage the
// stall happened in. So the partition must cover both, and §6 says so.
//
// HOW TO RUN IT: fill population T's pool and present a
// population-L request.
// EXPECTED RESULT: L's entry is allocated and becomes a candidate
// without waiting for any T entry to retire.
// EXPECTED TRACE: l_cand must rise while t_pool_full is high.
//
// SYNTHESIS: two free-lists and two window counters. The entries
// themselves are 17.2's pool and are not duplicated here.
//
// LIMITATIONS: partitions ENTRIES and the REORDER WINDOW. It does not
// partition legality -- §6 and 17.4 §3 both forbid that -- and it
// does not choose among candidates, which is 17.4's layered mask.
// One entry per population per cycle, so 17.2 §13's simultaneous
// allocation-and-free case is out of scope and STATED.
// ---------------------------------------------------------------------
module population_queues #(
  parameter int DEPTH_T   = 64,     // grade D
  parameter int DEPTH_L   = 4,      // grade D: covers the chain (§6)
  // The reorder window per population. §4's tuning argument in
  // hardware: T wants a window deep enough to find row hits, and a
  // window deeper than L's whole chain can only delay L.
  parameter int WIN_T     = 32,
  parameter int WIN_L     = 1,      // near-FCFS, per §4's tuning 2
  // COUNT, not INDEX: each occupancy must represent its depth
  // inclusive, so $clog2(depth + 1). Sized $clog2(depth) a full pool
  // reads as empty and allocation becomes unbounded.
  parameter int OT_W      = $clog2(DEPTH_T + 1),
  parameter int OL_W      = $clog2(DEPTH_L + 1)
)(
  input  logic             clk,
  input  logic             rst_n,

  input  logic             t_alloc, t_retire,
  input  logic             l_alloc, l_retire,

  output logic             t_pool_full, l_pool_full,
  output logic             t_cand, l_cand,
  output logic [OT_W-1:0]  t_occ,
  output logic [OL_W-1:0]  l_occ,
  output logic             hol_blocked,
  output logic             pool_underflow
);
  initial begin
    if (DEPTH_T < 1 || DEPTH_L < 1)
      $fatal(1, "population_queues: each depth must be >= 1");
    // §6 part 2 and §16's two-sided bound: a window deeper than the
    // population's own pool can never fill, so it is a parameter that
    // does nothing -- 31.2 §7's "present in the source, absent in
    // the silicon" applied to a reorder window.
    if (WIN_T > DEPTH_T) $fatal(1, "population_queues: WIN_T exceeds DEPTH_T");
    if (WIN_L > DEPTH_L) $fatal(1, "population_queues: WIN_L exceeds DEPTH_L");
    // §4's tuning asymmetry, enforced: L's window must not be tuned
    // like T's, or the partition reproduces the shared behaviour it
    // exists to prevent.
    if (WIN_L > 4)
      $warning("population_queues: WIN_L = %0d reorders L's dependency chain (§4)", WIN_L);
  end

  logic [OT_W-1:0] ot;
  logic [OL_W-1:0] ol;

  assign t_pool_full = (ot >= DEPTH_T[OT_W-1:0]);
  assign l_pool_full = (ol >= DEPTH_L[OL_W-1:0]);

  // Candidacy is per population and independent BY CONSTRUCTION.
  // This is the whole mechanism: neither expression mentions the
  // other population's state, so §16's isolation property is
  // trivially true here and false in any shared design.
  assign t_cand = (ot != '0);
  assign l_cand = (ol != '0);

  // The observable §6 needs and §18 measures: an L entry exists and
  // cannot proceed while T's pool is full. In THIS block that is
  // impossible, so the signal is identically low -- and 30.5 §11's
  // lesson applies: a guard another structure always shadows is
  // UNTESTED, so the cover below watches it rather than assuming it.
  assign hol_blocked = l_alloc && !l_cand && t_pool_full;

  always_ff @(posedge clk) begin
    if (!rst_n) begin
      ot <= '0;  ol <= '0;  pool_underflow <= 1'b0;
    end else begin
      unique case ({t_alloc && !t_pool_full, t_retire})
        2'b10 : ot <= ot + 1'b1;
        2'b01 : begin
                  if (ot == '0) pool_underflow <= 1'b1;
                  else          ot <= ot - 1'b1;
                end
        default : ;
      endcase
      unique case ({l_alloc && !l_pool_full, l_retire})
        2'b10 : ol <= ol + 1'b1;
        2'b01 : begin
                  // A retire with nothing allocated is an ERROR, not
                  // a saturating subtract -- 30.8 §11: an instrument
                  // that can report an impossible value has never
                  // been checked.
                  if (ol == '0) pool_underflow <= 1'b1;
                  else          ol <= ol - 1'b1;
                end
        default : ;
      endcase
    end
  end

  assign t_occ = ot;
  assign l_occ = ol;
endmodule

The elaboration guard on WIN_L is the block's quietest lesson. A reorder window is exactly the mechanism 23.4 says trades latency for throughput — so applying T's window to L reproduces the behaviour the partition was built to prevent. A partition that copies the wrong policy into both halves has separated the resources and not the tuning, which is §6 part 3's requirement and the failure it warns about.

And hol_blocked is identically low in this block on purpose. It exists so the shared alternative can be instrumented with the same signal and compared — CURRICULUM-DERIVED from 30.10 §12's dead-or-live distinction: this cover must go dead in the partitioned design and be live in the shared one, and knowing which you are writing is the point.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
  // ---- The queue partition's own contract.
  // Isolation, the same class 31.3 §15 named for replicated channels
  // and §16 names for credits. Vacuously true in a shared pool, so a
  // design grown from one has never had it checked.
  property p_t_alloc_does_not_move_l_occ;
    @(posedge clk) disable iff (!rst_n)
      (t_alloc && !l_alloc && !l_retire) |=> $stable(l_occ);
  endproperty
  assert property (p_t_alloc_does_not_move_l_occ)
    else $error("a T allocation changed L's queue occupancy");

  // L's candidacy must not depend on T's pool at all -- clause 2 of
  // §13's contract, restated for the queue stage.
  property p_l_candidacy_independent;
    @(posedge clk) disable iff (!rst_n)
      (l_occ != '0) |-> l_cand;
  endproperty
  assert property (p_l_candidacy_independent)
    else $error("L had entries and was not a candidate");

  // Head-of-line blocking across populations must be IMPOSSIBLE in a
  // partitioned design. An invariant, because there is no cycle on
  // which it is acceptable.
  property p_no_cross_population_hol;
    @(posedge clk) disable iff (!rst_n) !hol_blocked;
  endproperty
  assert property (p_no_cross_population_hol)
    else $error("cross-population head-of-line blocking occurred");

  // ---- Covers.
  // The case the partition exists for: T's pool full while L runs.
  cover property (@(posedge clk) disable iff (!rst_n)
                  t_pool_full && l_cand);
  // L's own pool full -- the legitimate limit, distinguishable from
  // the starvation case above.
  cover property (@(posedge clk) disable iff (!rst_n) l_pool_full);
  // Both pools occupied, so the bimodality was exercised rather than
  // assumed -- the per-dimension rule of 31.3 §15.
  cover property (@(posedge clk) disable iff (!rst_n)
                  (t_occ > 0) && (l_occ > 0));
  // T's reorder window actually full, so §4's tuning-1 behaviour is
  // known to have been reached.
  cover property (@(posedge clk) disable iff (!rst_n) t_occ >= WIN_T);

12. RTL — Measuring Both Populations, and the Coupling

§4 showed that a mean latency over the combined mix improves when the wrong thing gets better. This block never averages, and it also measures §5's crossover, which is the chapter's deciding quantity.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// ---------------------------------------------------------------------
// population_monitor -- verification/telemetry, CORRECT as written.
//
// CLASSIFICATION: synthesisable telemetry. Drives nothing. Grade D.
//
// WHAT IT DOES: keeps per-population latency extremes and outstanding
// extremes SEPARATE, and measures the launch interval and L's
// serialised cost so §5's net-effect condition can be evaluated.
//
// WHY IT EXISTS HERE: §4 proves a combined mean is dominated by the
// 97% with no deadline, and §5 proves the deciding quantity is a
// workload property (the launch interval) that no datasheet supplies.
// Both consequences are measurements, and §8 establishes that they
// are the ONLY grade above D available on this platform class.
//
// HOW TO RUN IT: run the real workload and evaluate §5's inequality
// from launch_interval and l_lat_max.
// EXPECTED RESULT: net_positive high when (1+g)(1 - dL/I) > 1.
//
// SYNTHESIS: four extreme trackers, two counters. NO divider -- §5's
// condition is evaluated by cross-multiplication.
//
// LIMITATIONS: reports extremes and a launch interval, NOT
// distributions. 23.1 §12 owns the distribution tracker and this
// block does not duplicate it; extremes are what §5's inequality
// needs. And it cannot measure the throughput gain g -- that is a
// comparison against another configuration, which 23.5's
// counterfactual owns.
// ---------------------------------------------------------------------
module population_monitor #(
  parameter int WIN       = 1048576,
  parameter int LAT_W     = 20,
  parameter int OCC_W     = 13,
  // Grade D. The throughput gain of the tuning under evaluation,
  // expressed in percent. It is an INPUT on a real platform because
  // 23.5's counterfactual measurement is the only way to obtain it.
  parameter int GAIN_PCT  = 18,
  parameter int CNT_W     = $clog2(WIN + 1)
)(
  input  logic              clk,
  input  logic              rst_n,

  input  logic              t_start, t_end,
  input  logic              l_start, l_end,
  input  logic [LAT_W-1:0]  t_lat_sample,
  input  logic [LAT_W-1:0]  l_lat_sample,
  input  logic [OCC_W-1:0]  t_outstanding,
  input  logic [OCC_W-1:0]  l_outstanding,
  input  logic              work_launch,
  input  logic              win_tick,

  output logic [LAT_W-1:0]  t_lat_max, t_lat_min,
  output logic [LAT_W-1:0]  l_lat_max, l_lat_min,
  output logic [OCC_W-1:0]  t_occ_max, l_occ_max,
  output logic [CNT_W-1:0]  launch_interval,
  output logic              net_positive,
  output logic              interval_unmeasured,
  output logic              result_valid
);
  logic [LAT_W-1:0] tmx, tmn, lmx, lmn;
  logic [OCC_W-1:0] tox, lox;
  logic [CNT_W-1:0] since_launch, sum_int, n_int;

  always_ff @(posedge clk) begin
    if (!rst_n) begin
      // AND-accumulator discipline: minimum trackers start at
      // all-ones or the first sample never wins and every reported
      // minimum stays zero, which would make every span look
      // enormous and hide the bimodality entirely.
      tmx <= '0;  tmn <= '1;  lmx <= '0;  lmn <= '1;
      tox <= '0;  lox <= '0;
      since_launch <= '0;  sum_int <= '0;  n_int <= '0;
      t_lat_max <= '0;  t_lat_min <= '0;
      l_lat_max <= '0;  l_lat_min <= '0;
      t_occ_max <= '0;  l_occ_max <= '0;
      launch_interval <= '0;
      net_positive <= 1'b0;  interval_unmeasured <= 1'b1;
      result_valid <= 1'b0;
    end else if (win_tick) begin
      // Published SEPARATELY. There is deliberately no combined
      // figure anywhere in this block's interface -- §4's whole
      // result is that a combined mean improves when the wrong
      // thing gets better, so the block does not offer one.
      t_lat_max <= tmx;  t_lat_min <= tmn;
      l_lat_max <= lmx;  l_lat_min <= lmn;
      t_occ_max <= tox;  l_occ_max <= lox;

      // The launch interval, as a mean over observed launches.
      // Reported as UNMEASURED rather than as zero when too few
      // launches occurred -- an instrument that can report a
      // meaningless value has not been checked (30.8 §11).
      interval_unmeasured <= (n_int < 2);
      launch_interval     <= (n_int >= 2) ? (sum_int / n_int) : '0;

      // §5's condition, cross-multiplied so no divider is needed:
      //     (1 + g)(1 - dL/I) > 1
      //  => (100 + GAIN_PCT) * (I - dL) > 100 * I
      // with dL taken as L's observed worst serialised cost.
      net_positive <= (n_int >= 2) &&
        (((100 + GAIN_PCT) * ((sum_int / n_int) - lmx)) > (100 * (sum_int / n_int)));

      result_valid <= 1'b1;
      tmx <= '0;  tmn <= '1;  lmx <= '0;  lmn <= '1;
      tox <= '0;  lox <= '0;  sum_int <= '0;  n_int <= '0;
    end else begin
      result_valid <= 1'b0;
      if (t_end) begin
        if (t_lat_sample > tmx) tmx <= t_lat_sample;
        if (t_lat_sample < tmn) tmn <= t_lat_sample;
      end
      if (l_end) begin
        if (l_lat_sample > lmx) lmx <= l_lat_sample;
        if (l_lat_sample < lmn) lmn <= l_lat_sample;
      end
      if (t_outstanding > tox) tox <= t_outstanding;
      if (l_outstanding > lox) lox <= l_outstanding;

      // The launch interval. §8's finding: this is a WORKLOAD
      // property and no datasheet supplies it, so it must be
      // observed on the software that will actually run.
      if (work_launch) begin
        if (n_int != {CNT_W{1'b1}}) begin
          sum_int <= sum_int + since_launch;
          n_int   <= n_int + 1'b1;
        end
        since_launch <= '0;
      end else if (since_launch != {CNT_W{1'b1}}) begin
        since_launch <= since_launch + 1'b1;
      end
    end
  end
endmodule

The block deliberately offers no combined latency figure, and that absence is a design decision rather than an omission. CURRICULUM-DERIVED from 23.1 §9 — the average is the wrong statistic — and §4 shows it is worse than wrong here, because the 97% that dominates any mean is exactly the part with no deadline. An interface that cannot be misread is better than a documented caution.

And net_positive is the chapter's deciding quantity as a single bit. DERIVED from §5's condition, evaluated on measured quantities: launch_interval from the workload, l_lat_max as the serialised cost, and GAIN_PCT from a counterfactual comparison that 23.5 owns. Two of the three are measurable here and the third is not, which is stated in the limitations rather than hidden behind a number.

13. RTL Review — A Credit Pool

The intended contract:

  1. Each population may consume only its own credit allocation.
  2. A population's admission must not be blocked by another population's occupancy.
  3. Credits return on completion, to the population that consumed them.
  4. The allocation is a platform property and must be configurable.
  5. Per-population exhaustion must be reported, so starvation is observable.
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// ---------------------------------------------------------------------
// shared_credit_pool -- INTENTIONALLY DEFECTIVE, for review (§13).
//
// CLASSIFICATION: synthesisable, grade-D values, and CONTAINS A BUG.
//
// WHAT IT IS MEANT TO DO: the five-clause contract above -- admit
// requests against a per-population credit allocation.
//
// WHY IT EXISTS HERE: §6 part 2 argues the partition must be
// STRUCTURAL rather than policy, because with outstanding counts of
// 4096 against 2 (§3) any shared resource gives population L an
// unmeasurable share. This block is the shared version, and it is
// the natural design: one pool is simpler, uses the credits more
// efficiently, and measures better (§14).
//
// HOW TO RUN IT: backlog population T so it presents a request every
// cycle, then present a single population-L request.
// EXPECTED RESULT under clause 2: L is admitted promptly,
// independent of T's occupancy.
// EXPECTED TRACE: l_admit must rise within L's own allocation. It
// instead waits for T's backlog to drain.
//
// SYNTHESIS: one counter, one comparator.
//
// LIMITATIONS: admission only, as §10's block. That is STATED and is
// not the bug.
// ---------------------------------------------------------------------
module shared_credit_pool #(
  parameter int CREDITS_TOTAL = 520,     // grade D: 512 + 8
  parameter int C_W           = $clog2(CREDITS_TOTAL + 1)
)(
  input  logic             clk,
  input  logic             rst_n,

  // The population identifier IS on the interface. Clause 1 and 3
  // both depend on it.
  input  logic             req,
  input  logic             pop_id,        // 0 = T, 1 = L
  input  logic             done,
  input  logic             done_pop_id,

  output logic             admit,
  output logic [C_W-1:0]   credits_used,
  output logic             exhausted,
  output logic             credit_underflow
);
  logic [C_W-1:0] c;

  initial begin
    if (CREDITS_TOTAL < 2) $fatal(1, "shared_credit_pool: CREDITS_TOTAL >= 2");
  end

  // Clauses 3's accounting and 5's exhaustion report are both
  // present and both CORRECT for the pool as a whole. A reviewer
  // checking "are credits conserved?" and "is exhaustion reported?"
  // finds that they are, and that is what conceals the defect.
  assign exhausted = (c >= CREDITS_TOTAL[C_W-1:0]);
  assign admit     = req && !exhausted;   // <-- THE DEFECT

  always_ff @(posedge clk) begin
    if (!rst_n) begin
      c <= '0;  credit_underflow <= 1'b0;
    end else begin
      unique case ({admit, done})
        2'b10 : c <= c + 1'b1;
        2'b01 : begin
                  if (c == '0) credit_underflow <= 1'b1;
                  else         c <= c - 1'b1;
                end
        default : ;
      endcase
    end
  end

  assign credits_used = c;
endmodule

Before reading on: which clause, and why does the shared pool score better on the benchmark?

14. The Defect — The Pool That Measures Better

The violated clauses are 1 and 2, and the defect is that pop_id and done_pop_id are declared on the port list and never read. One pool, granted first-come — so population T, which presents a request every cycle, consumes every credit and holds them.

That is the fifth declared-and-never-read input in this curriculum, after 30.7 §9, 30.8 §10, 31.1 §12 and 31.4 §12. CURRICULUM-DERIVED from 30.9 §3: lint finds it before simulation starts, and by now that is not a tip — it is a pattern with five instances.

The trace, and the magnitude is the point.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
   GRADE D.  CREDITS_TOTAL = 520.  S = 4 cycles per completion.
   Population T is backlogged: 4096 requests pending, presenting
   one every cycle. Population L presents ONE request.

   steady state with the shared pool:
       T holds all 520 credits.
       a credit returns every S = 4 cycles.
       T presents a request EVERY cycle, so it takes each
       returned credit first.

   L's wait = until T's backlog stops presenting.
       T's backlog drains at 1 request per S cycles:
       DERIVED  4096 x 4 = 16,384 cycles.

   so L's observed latency:
       contract (partitioned, §10) :   200 cycles
       this block                  : 16,384 + 200 = 16,584 cycles
       DERIVED ratio               : 82.9x

And now §5's coupling multiplies it into a throughput loss.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
   DERIVED, using §5's model. Launch interval I = 2,000 cycles.

     dL = 16,584 - 200 = 16,384 cycles of extra serialised stall
     dL / I = 16,384 / 2,000 = 8.19

   the stall EXCEEDS the launch interval by a factor of eight, so
   population T cannot be re-launched on schedule at all. Its
   utilisation is set by L's admission wait, not by its own
   throughput.

   §5's condition: (1 + g)(1 - dL/I) > 1 requires dL/I < g/(1+g).
     with g = 0.18 :  dL/I must be < 0.153
     observed      :  8.19
   DERIVED: net effect is catastrophically negative.

Now the review question, and it is the best one in the module: why did this ship?

Because the shared pool makes the headline metric better.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
   a throughput benchmark runs population T almost exclusively.
   under the SHARED pool, T can hold all 520 credits.
   under the PARTITIONED pool (§10), T can hold only 512.

   DERIVED: T's available concurrency is 520 vs 512, +1.6%.
   And §3 showed T clears its condition 81.9x over, so the extra
   credits translate directly into a marginally deeper pipeline
   and a marginally better sustained rate.

   so the defective block WINS the benchmark, by a small honest
   margin, and the partitioned block LOSES it.

Three properties make this the module's most instructive defect.

The defect is not a bug in the metric's terms — it is an improvement. A reviewer comparing the two designs on throughput will correctly prefer the defective one. CURRICULUM-DERIVED from 30.8 §2: the benchmark measures rung 4, the shared pool genuinely improves rung 4, and rung 5 collapses. This is the sharpest instance in the curriculum of that chapter's central result.

The cost lands on a population the benchmark does not exercise. A throughput benchmark has almost no population L in it by construction — its whole purpose is to keep T saturated — so the harm is invisible to the measurement that chose the design.

And the failure looks like a software problem. DERIVED: T's utilisation is low while its bandwidth-when-running is excellent, so the symptom is the GPU is idle waiting for work — which points at the driver, the dispatcher or the application. CURRICULUM-DERIVED from 30.8 §4, the symptom presents in gap 4→5, which that chapter attributes to the SoC and the requester — and the investigation goes there, correctly by the ladder and wrongly in fact.

The correction, and it is a partition rather than a policy:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
  // CORRECTED. Clauses 1 and 2: per-population counters, so neither
  // population can reach the other's allocation at all.
  //
  // Note what is NOT added: no arbiter, no weight, no fairness
  // bound, no age threshold. 17.4 §9's machinery exists to make a
  // SHARED resource fair, and with a 2048:1 capability ratio (§3) no
  // weighting can achieve what removing the sharing achieves. A
  // partition is both cheaper and provable.
  assign t_exhausted = (ct >= CREDITS_T);
  assign l_exhausted = (cl >= CREDITS_L);
  assign admit = req && (pop_id ? !l_exhausted : !t_exhausted);
  // ... and ct / cl each move only on their own population's
  //     admit and done, keyed by pop_id and done_pop_id.

Two interface consequences, stated rather than assumed. The single exhausted output must become two, because clause 5 requires per-population reporting and one flag cannot express which population is starved. And CREDITS_TOTAL must become two allocations, which per 32.1 §13's lesson should be configuration inputs rather than parameters — an allocation is a platform property.

The general finding, and it is the module's most transferable:

When two populations differ in capability by orders of magnitude, sharing a resource with a policy cannot produce fairness that a partition produces for free — and the shared version will win any benchmark that exercises only the larger population. A design chosen on a metric that does not contain the harmed party will be chosen wrongly, every time, by a correct comparison.

15. SVA Review — Proving Conservation Instead of Access

The properties written for §13's block, and all three are correct and useful:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
  // Offered as "proves the credit pool is correct". All PASS on the
  // defective block.
  property p_credits_never_exceed_pool;
    @(posedge clk) disable iff (!rst_n)
      credits_used <= CREDITS_TOTAL;
  endproperty
  assert property (p_credits_never_exceed_pool)
    else $error("credits used exceeded the pool");

  property p_no_admit_when_exhausted;
    @(posedge clk) disable iff (!rst_n)
      exhausted |-> !admit;
  endproperty
  assert property (p_no_admit_when_exhausted)
    else $error("a request was admitted with no credit available");

  property p_credits_conserved;
    @(posedge clk) disable iff (!rst_n)
      (admit && !done) |=> (credits_used == $past(credits_used) + 1);
  endproperty
  assert property (p_credits_conserved)
    else $error("credit accounting drifted");

Q. All three pass. What have they proved?

That credits are conserved, bounded and never over-issued — which are safety properties about the pool as an object. None of them mentions pop_id, so none can observe that one population has taken everything: 30.9 §6's variety 2, for the fourth consecutive chapter.

And there is a second variety here, which is the more instructive one. The defect is a starvation, and starvation violates no safety property at all30.9 §6's variety 8: safety cannot detect conservatism. A pool that admits nothing satisfies every property above perfectly. So the omission is not just a missing signal — it is a missing kind of property, and adding a stronger safety property could never have found it.

What actually covers clauses 1 and 2:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
  // Clause 1. Names pop_id, so it cannot pass while pop_id is
  // ignored. Variety 2's repair.
  property p_population_uses_own_allocation;
    @(posedge clk) disable iff (!rst_n)
      (admit && pop_id == 1'b1) |-> (l_credits_used < CREDITS_L);
  endproperty
  assert property (p_population_uses_own_allocation)
    else $error("population L was admitted outside its own allocation");

  // Clause 2, and it must be a BOUNDED LIVENESS property because the
  // defect is a starvation -- variety 8's repair. The bound is
  // JUSTIFIED from L's own allocation and completion rate, never
  // chosen: with CREDITS_L credits and a completion every S cycles,
  // a request waits at most CREDITS_L * S cycles for one of its
  // OWN credits. 30.9 §5 Q2 owns that a bound must be derived.
  property p_l_admission_is_bounded;
    @(posedge clk) disable iff (!rst_n)
      (l_req && !l_admit) |-> ##[1:(CREDITS_L*S_CYCLES)] l_admit;
  endproperty
  assert property (p_l_admission_is_bounded)
    else $error("population L's admission was not bounded by its own allocation");

  // And the INDEPENDENCE property, which is clause 2 stated
  // structurally rather than temporally: L's admission must not be a
  // function of T's occupancy at all. This is the property a
  // partition makes trivially true and a shared pool cannot satisfy.
  property p_l_admit_independent_of_t;
    @(posedge clk) disable iff (!rst_n)
      (l_req && !l_exhausted) |-> l_admit;
  endproperty
  assert property (p_l_admit_independent_of_t)
    else $error("L's admission depended on T's occupancy");

  // Clause 5: per-population exhaustion. One flag cannot express
  // which population is starved, so the property names both.
  property p_exhaustion_is_per_population;
    @(posedge clk) disable iff (!rst_n)
      (l_req && !l_admit) |-> l_exhausted;
  endproperty
  assert property (p_exhaustion_is_per_population)
    else $error("L was refused without its own exhaustion being reported");

  // ---- Covers. The dimension this defect scales with is the
  //      POPULATION RATIO, and 31.3 §15's rule says the cover must be
  //      on that dimension.
  // The case that matters: T saturated AND L requesting. A throughput
  // benchmark never produces it, because its purpose is to keep T
  // saturated with no L traffic at all (§14).
  cover property (@(posedge clk) disable iff (!rst_n)
                  t_exhausted && l_req);
  // L admitted while T is saturated -- the positive outcome, and the
  // one §13's block cannot produce.
  cover property (@(posedge clk) disable iff (!rst_n)
                  t_exhausted && l_admit);
  // Both populations active at all, so the bimodality is known to
  // have been exercised rather than assumed.
  cover property (@(posedge clk) disable iff (!rst_n)
                  (t_outstanding > 0) && (l_outstanding > 0));
  // L's own allocation exhausted -- the legitimate refusal, which
  // must be distinguishable from the starvation case above.
  cover property (@(posedge clk) disable iff (!rst_n) l_exhausted);
  // And the single-population arm, per 31.1 §14's configuration rule.
  cover property (@(posedge clk) disable iff (!rst_n)
                  (NUM_POP == 1) && t_admit);

The first cover is the one that finds the defect and the one a throughput benchmark can never reach. CURRICULUM-DERIVED from 31.4 §14's rule — when the missing dimension belongs to the environment's model rather than its stimulus, running longer never reaches it — and here the environment's model is the benchmark's traffic composition. A benchmark with no population L in it cannot produce t_exhausted && l_req however long it runs, so the cover stays at zero, every assertion passes, and the design that wins the comparison is the broken one.

Follow-up an interviewer should ask: is that a benchmark problem or a verification problem? Both, and separating them is the answer. The benchmark is measuring what it was designed to measure and is not wrong. The verification plan is wrong, because 30.9 §2's obligation list should have contained population L's admission is bounded as a progress obligation — and a progress obligation cannot be discharged by a throughput measurement. The coverage item is the bridge, and it belongs in the plan rather than in the benchmark.

16. What the Admission Block's Assertions Prove

§15 reviewed the shared pool. §10's partitioned block needs its own set, and the interesting properties are the ones a partition makes trivially true — because a property that is trivially true in the good design and false in the bad one is exactly what a review needs.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
  // The structural independence of the two counters. In §10's block
  // this is true by construction; in §13's it is false. A property
  // that distinguishes two designs at a glance is worth more than a
  // property that constrains one of them tightly.
  property p_t_events_do_not_move_l;
    @(posedge clk) disable iff (!rst_n)
      (t_admit && !l_admit && !l_done) |=> $stable(l_credits_used);
  endproperty
  assert property (p_t_events_do_not_move_l)
    else $error("a population-T event changed population L's credit count");

  property p_l_events_do_not_move_t;
    @(posedge clk) disable iff (!rst_n)
      (l_admit && !t_admit && !t_done) |=> $stable(t_credits_used);
  endproperty
  assert property (p_l_events_do_not_move_t)
    else $error("a population-L event changed population T's credit count");

  // Each allocation is respected in RANGE -- a derived count, so
  // 30.8 §11's rule applies and the direction matters: an
  // understated count licenses over-admission.
  property p_allocations_in_range;
    @(posedge clk) disable iff (!rst_n)
      (t_credits_used <= CREDITS_T) && (l_credits_used <= CREDITS_L);
  endproperty
  assert property (p_allocations_in_range)
    else $error("a population exceeded its allocation");

  // Underflow must be REPORTED, not absorbed. A drifting credit
  // counter invalidates every figure that depends on it, and
  // 30.8 §11 owns that an instrument which can report an impossible
  // value has never been checked.
  property p_underflow_is_reported;
    @(posedge clk) disable iff (!rst_n)
      ((t_done && t_credits_used == '0) || (l_done && l_credits_used == '0))
        |=> credit_underflow;
  endproperty
  assert property (p_underflow_is_reported)
    else $error("a completion with no outstanding credit was absorbed silently");

  // The PLATFORM invariant, with the parameter in the ANTECEDENT and
  // not inside the claim -- 31.1 §14's test. It may be shared across
  // both configurations.
  property p_single_population_never_admits_l;
    @(posedge clk) disable iff (!rst_n)
      (NUM_POP == 1) -> !l_admit;
  endproperty
  assert property (p_single_population_never_admits_l)
    else $error("a single-population platform admitted a second population");

  // And the two-sided bound: a partition must not be so small that L
  // serialises on admission as well as on latency. §6 part 2's
  // "present in the source, absent in the silicon" -- a partition
  // that cannot hold the chain depth is a partition in name only.
  property p_l_allocation_covers_chain;
    @(posedge clk) disable iff (!rst_n)
      (l_outstanding > 0) |-> (CREDITS_L >= l_outstanding);
  endproperty
  assert property (p_l_allocation_covers_chain)
    else $error("L's allocation is smaller than its observed chain depth");

  // ---- Covers for the partitioned admission block. Every other
  //      assertion set in this module has them, and without them
  //      these properties are statements about whichever arm ran.
  // T saturated while L is admitted: the outcome §13's block cannot
  // produce, and the single most valuable cover in the chapter.
  cover property (@(posedge clk) disable iff (!rst_n)
                  t_exhausted && l_admit);
  // The single-population arm, per 31.1 §14's configuration rule.
  cover property (@(posedge clk) disable iff (!rst_n)
                  (NUM_POP == 1) && t_admit);
  // Both allocations simultaneously at their limits -- the only
  // state in which the partition's sizing is fully exercised.
  cover property (@(posedge clk) disable iff (!rst_n)
                  t_exhausted && l_exhausted);

The two isolation properties are the section's deliverable, and they generalise beyond this block. CURRICULUM-DERIVED from 31.3 §15, which named the same class for replicated channels: a design that separates resources needs a property saying the separation holds — and for a single shared pool that property is vacuously true, so a design that grew from a shared pool has never had it checked.

And p_l_allocation_covers_chain is the two-sided bound. A partition satisfying every property above can still be too small, at which point population L serialises on admission as well as on latency and the partition has bought nothing. CURRICULUM-DERIVED from 30.3 §9: a safety property cannot detect over-restriction, so the useful bound has to be stated in both directions.

17. The Larger Lever Is Not in the Memory

Everything so far has been what the controller can do. It is worth computing the bound on that, because the bound is low and the alternative is large.

§3 established the problem as N_L being far below N*. There are exactly two ways to fix an inequality, and the memory team owns only one side of it.

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
   the condition (31.4 §4):     N_L  >=  ceil( L / S )

   the MEMORY team's lever:  reduce L
       bounded below by the DRAM itself -- the thirteen obligations
       of 31.1 §5 do not go away, and 23.1 §7 owns which latency
       components are IRREDUCIBLE.

   the UPSTREAM team's lever:  raise N_L
       bounded by the dependency chain's structure, which is a
       software and dispatch-architecture property.

Now bound the memory lever, using §4's own figures. DERIVED, grade D:

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
   tuning 2 (§4) took L from 200 to 120 cycles -- a 40% reduction,
   and it cost population T 23% of its throughput.

   the effect on the condition:
       N* before = ceil(200/4) = 50
       N* after  = ceil(120/4) = 30

   and L's position:
       N_L = 2.   2/50 = 0.04   ->   2/30 = 0.067

   DERIVED: the memory team's most aggressive available tuning moves
   L from 4% of the requirement to 6.7% of it. L still cannot hide
   any latency at all, and T paid 23% of its throughput for that.

So the memory lever cannot solve this problem — it can only reduce the damage, and it pays for the reduction out of the other population. That is the same shape as 30.8 §6's ceiling computation: compute what your own lever can deliver before committing to it, because a ceiling below the requirement is a finding rather than an excuse.

The upstream lever is arithmetically larger, and it is worth naming its forms.

Upstream changeEffect on N_LWho owns it
Batch work descriptions so one fetch serves many launchesraises the work served per chain traversal, cutting launches per unit timethe dispatch architecture
Prefetch the next launch's description during the current oneconverts a serial chain into a pipelined one — N_L rises to the pipeline depththe dispatcher
Flatten the chain — fewer dependent indirections per launchreduces the chain's depth directly, so fewer serialised L paymentsthe data structure's designer
Overlap independent launchesseveral chains in flight, so N_L becomes the number of concurrent launchesthe scheduler above the memory

Row two is the largest and the most often available. DERIVED: prefetching one launch ahead takes N_L from 2 to 4, which doubles L's effective concurrency — more than the memory team's entire 40% latency reduction achieved, and it costs population T nothing at all.

And this is the honest conclusion of the chapter for a memory engineer. CURRICULUM-DERIVED from 30.8 §4, whose result is that three of the four gaps in the bandwidth ladder lie outside the memory — so the prior probability that a memory-performance problem is a memory problem is low. Here it is not even a performance problem in the memory's terms: the memory is behaving correctly and the requester cannot use it.

Saying it well matters, and 30.8 §4 owns the wording discipline. Our tuning can move L from 4% to 6.7% of the concurrency it needs, and it costs T 23%; prefetching one launch ahead doubles L's concurrency and costs T nothing is a collaborative finding with numbers attached. Not a memory problem is neither.

18. What Would You Measure?

Q. You own the memory subsystem on a platform of this class. What do you measure, in what order?

MeasurementWhat it settlesCostOwner
Does the upstream protocol carry a population identifier?§6 part 1 — without it nothing below is available to the controllerhours§6
Are admission credits and queues partitioned or shared?§14's defect, answerable by reading a port listhours§14
l_occ_max and t_occ_max separately§3's two answers to the condition, for this platformdays§12
T's utilisation, not its bandwidth§5's coupling — the loss a bandwidth metric cannot seedays§5, §12
The launch interval on the real software§5's I, which is a workload property no datasheet suppliesdays§12, §8
net_positive from §12whether the current tuning is net positive for this workloaddays§12
L's latency maximum, never a combined mean§4 — a mean is dominated by the 97% with no deadlinedays23.1 §9
The t_exhausted && l_req cover countwhether the starvation case was ever exercisedfree§15

Rows one and two cost hours and either can close the question. Row two in particular is answerable without running anythingdoes the credit accounting read a population identifier? If not, §14's defect is present regardless of how well the platform benchmarks.

Row four is the one that inverts a conclusion, and it is the chapter's practical payload. DERIVED from §5: a platform can show excellent memory bandwidth and poor delivered throughput simultaneously, and the two measurements are not in conflict — T is fast when it runs and it is not running. CURRICULUM-DERIVED from 30.8 §4, the symptom appears in gap 4→5 and that chapter attributes 4→5 to the SoC and the requester — which is correct by the ladder and wrong in this case, because the cause is inside the memory controller's admission.

And row eight is free and decisive. If the starvation cover is zero, the comparison that selected the design never contained the harmed population — §14's whole mechanism, available before any new measurement.

19. Common Wrong Answers

“A GPU-class memory system is throughput-optimised, full stop.” §2, §5. There are two populations and the small one gates the large one's ability to start.

“Latency does not matter here.” §3. It does not matter to population T, which clears the condition 81.9× over. Population L pays every cycle of it, serialised.

“Both populations want more bandwidth.” §2, §4. T wants throughput and is indifferent to L; L wants low L and is indifferent to throughput. No single tuning serves both.

“Average the two and optimise the mean.” §4. A mean latency is dominated by the 97% with no deadline, so it improves when the wrong thing gets better — 23.1 §9 owns why the average is the wrong statistic.

“Tune for throughput; it is the visible output.” §5. Two thirds of an 18% gain was consumed by the coupling at a launch interval of 2,000 cycles, and below about 1,300 cycles the tuning is net negative.

“The two populations are independent.” §5. The coupling is one-directional: L gates T's launch, and that is why it shows up only when you measure T's utilisation rather than its bandwidth.

“A faster attach helps everyone.” §7. Halving S doubles N*31.4 §4 — so the better attach moves population L further from a condition it already could not meet.

“Give population L a higher priority.” §6, §14. With a 2048:1 capability ratio, a preference over a shared resource gives L an unmeasurable share — 17.4 §9 owns that a preference layer with no bounded override starves. The partition must be structural.

“Partition the legality filter too, for symmetry.” §6, §9. The thirteen obligations are properties of the DRAM and the DRAM has no populations. Chapter 17.4 §3 calls that the inverse error, which is worse.

“A shared pool uses the credits more efficiently.” §14. It does, by 1.6% for T — and it costs L a factor of 82.9 in admission latency.

“The benchmark chose the better design.” §14. The benchmark measured rung 4 correctly and the design collapsed rung 5. A metric that does not contain the harmed party selects wrongly every time.

pop_id is on the interface, so the pool is population-aware.” §14. Declared and never read — the fifth instance in this curriculum, and lint finds it before simulation.

“The GPU is idle waiting for work, so it is a driver problem.” §14, §18. That is the symptom the defect produces, and it sends the investigation to gap 4→5 — correct by 30.8 §4's ladder and wrong in fact.

“The credit assertions pass, so the pool is correct.” §15. They prove conservation and boundedness. The defect is a starvation, and starvation violates no safety property — variety 8, so a stronger safety property could never have found it.

“We will find it by running the benchmark longer.” §15. A benchmark with no population L in it cannot produce the starvation case however long it runs — the missing dimension belongs to the environment's model.

“Partitioning is wasteful, so make L's share as small as possible.” §16. Below its chain depth the partition exists in name only and L serialises on admission as well as on latency — the two-sided bound.

“What is this GPU's memory tuning?” §8. A category error of 18.4 §5's kind, and for a stronger reason: the deciding quantity is the launch interval, which belongs to software that may not be written yet.

“This chapter tells me whether my model fits in device memory.” §Scope. It does not — that is a capacity question and /cxl/gpu-memory-bottlenecks owns it. A design can satisfy every result here and still not fit.

“We will fix L by tuning the memory.” §17. The most aggressive available tuning moves L from 4% to 6.7% of the concurrency it needs and costs T 23% of its throughput. Prefetching one launch ahead doubles L's concurrency and costs T nothing.

“HBM's extra bandwidth solves it.” §7, and 26.4 §7 owns the correction: one requester cannot fill them — and population L is precisely a requester that cannot, so a many-channel attach gives it nothing.

20. Self-Check

  1. State how a population differs from an obligation class, and give one request that belongs to a named class in each population.

  2. Write 31.4 §4's condition. Evaluate it for both populations using §3's grade-D figures and state both margins.

  3. Explain in two sentences why L has two different price tags on this platform and what that forbids.

  4. Take §4's tuning 1 and give the effect on each population. Then say why a combined mean latency improves under it.

  5. Derive §5's net-effect condition and compute the break-even launch interval for g = 0.10, dL = 150.

  6. Explain the coupling's direction and say which single measurement reveals it — and which widely-used one cannot.

  7. Explain why a faster attach makes the bimodality worse, and give the qualification that keeps the claim honest.

  8. List what §6 says must be partitioned and what must not, with the reason for the exclusion.

  9. Find the defect in §13 without reading §14. Then answer the harder question: why does it improve the headline metric?

  10. Explain why the credit assertions cannot detect the defect, naming both varieties involved and why a stronger safety property would not have helped.

  11. Compute the bound on the memory team's lever using §17's figures, then give two upstream changes that exceed it and say who owns each.

  12. Explain why the starvation cover is unreachable under a throughput benchmark, and say whether that is a benchmark problem or a verification-plan problem.

21. Where This Goes

This platform class runs one controller for two populations that want opposite things. One clears the latency-hiding condition by a factor of tens and the other cannot clear it at all; the same L is therefore free for one and fully charged to the other, so it has two price tags and cannot be averaged; the populations are coupled one-directionally, because the small one gates the large one's work launch, which makes a throughput-maximising tuning net negative below a computable launch interval; and the response is structural partition rather than policy, because with a 2048:1 capability ratio no preference can achieve what removing the sharing achieves for free.

Three results carry forward. When two populations differ by orders of magnitude in capability, the shared design wins any benchmark that exercises only the larger one — so a metric that does not contain the harmed party selects wrongly, every time, by a correct comparison. A design that separates resources needs an isolation property, and that property is vacuously true in the shared predecessor it grew from. And a partition can be too small as well as absent, so its bound is two-sided.

Chapter 32.4 takes the population that clears the condition and asks how it came to exist. Chapter 26.4 §7 owns the result that one requester cannot fill a many-channel memory, and this chapter has assumed throughout that population T simply has thousands of transactions in flight. On the next platform class that assumption is the design problem — the concurrency is not a property the memory can measure and hope for, it must be manufactured, and the number of independent request generators becomes an architectural specification rather than an implementation detail.

Continue learning

Standards & specifications

Governing standard
JEDEC JESD79 (DDR SDRAM)(opens JEDEC Solid State Technology Association in a new tab)

Defines the DDR SDRAM device itself — signals, command encoding, mode registers, timing parameters and the initialisation sequence — one document per generation. Memory-controller microarchitecture, address-mapping policy, PHY training algorithms and board-level design are not specified by it.

This page also covers RTL structure, verification approach and debugging technique. Those are engineering practice built on the standard, not requirements the standard itself imposes.

Where this fits

Part of the DDR curriculum.