Future Memory Systems

17.1 built a memory device. 17.2 built one whose media does not forget.

This chapter puts several kinds of memory in one system at once and asks the only question that then arises: which data lives where, and who decides.

1. The Engineering Problem — Capacity Stops Being Uniform

Six things change the moment a system has more than one kind of memory.

A page has a home, and the home is what the access costs. A system that models memory as one latency is not approximately right about a tiered machine — it reports the same number whatever the mix is. Section 5.

Promotion is a bet. Moving a page to faster memory costs something now against a saving that only materialises if the page is used again. Section 6.

The move has a price, and the price has to be repaid. A promotion that is never followed by enough accesses is strictly worse than leaving the page alone. Section 7.

The fast tier is finite. Promotion into a full tier means demotion out of it, so every promotion is really an exchange, and a policy that does not model the exchange models nothing. Section 9.

A policy that changes its mind is worse than no policy. Without a minimum dwell, a page whose heat oscillates pays the move cost every time and collects the saving never. Section 10.

And the number the system reports is not the number the workload feels. Capacity share reads well and traffic share is what hurts. Section 14.

This chapter against 17.2, stated precisely. That chapter owns one device whose media is unusual. This one owns several devices whose costs differ, and the placement decision between them. Nothing here depends on the memory being persistent; it depends only on it being slower.

2. The One-Sentence Model

Tiering is a bet that a page will be used enough to repay the cost of moving it, placed by a policy that cannot see the future — and every defect below is the bet being placed without one of its terms.

3. What This Chapter Owns

Deferred:

4. Teaching-Model Boundary

Three tiers, four resident pages, a three-page upper tier and a three-cycle dwell minimum are all far smaller than any real system. They are sized so every boundary is reachable in a short simulation and every result can be recomputed on paper.

What is not simplified is the structure: a per-tier cost charged at access, a promotion gated on heat and capacity, a migration debt repaid out of a per-access saving, an exchange rather than an insertion when the tier is full, and a dwell that must elapse before a page may move again. Each is shaped the way a real tiering system shapes it.

Two things are deliberately absent. There is no page-size model — migration cost is a single number here, and in a real system it scales with the page and interacts with the TLB. And there is no concurrency model: the migration happens atomically, where a real one has to handle accesses to a page that is mid-move. Both are worth chapters of their own and neither changes which gate exists.

5. RTL 1 — A Page Has A Home, And The Home Is The Cost

// A page has a home tier, and the tier is what the access costs.
module tier_map #(parameter int FLAT_COST = 0) (
  input  logic clk, rst_n,
  input  logic        access,
  input  logic [1:0]  page_tier,      // 0 near DRAM, 1 CXL DRAM, 2 CXL SCM
  input  logic [15:0] t0_ns, t1_ns, t2_ns,
  output logic [15:0] this_ns,
  output logic [31:0] total_ns,
  output logic [15:0] n_acc, n_t0, n_t1, n_t2,
  output logic        unknown_tier_err
);
  logic [15:0] tier_ns;
  always_comb begin
    case (page_tier)
      2'd0:    tier_ns = t0_ns;
      2'd1:    tier_ns = t1_ns;
      2'd2:    tier_ns = t2_ns;
      default: tier_ns = t2_ns;   // an unknown tier is charged the slowest
    endcase
  end
  // The flat model charges the nearest tier for everything, which is the
  // assumption a system makes when it has not been told the memory is tiered.
  assign this_ns = (FLAT_COST != 0) ? t0_ns : tier_ns;
  // A tier the map does not know about must not be charged as if it were near.
  assign unknown_tier_err = access && (page_tier > 2'd2);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      total_ns <= 32'd0; n_acc <= 16'd0; n_t0 <= 16'd0; n_t1 <= 16'd0; n_t2 <= 16'd0;
    end else if (access) begin
      total_ns <= total_ns + {16'd0, this_ns};
      n_acc <= n_acc + 16'd1;
      case (page_tier)
        2'd0: n_t0 <= n_t0 + 16'd1;
        2'd1: n_t1 <= n_t1 + 16'd1;
        2'd2: n_t2 <= n_t2 + 16'd1;
        default: ;
      endcase
    end
  end
endmodule

Ten accesses across three tiers at 80, 300 and 900 ns:

  tiers: acc=10 total=3920ns (near=4 mid=3 far=3) | flat total=800ns

Both builds counted the same three far accesses. n_t2 is 3 in each — the flat model is not blind to which tier a page is on, it simply does not charge for it. It reports 800 ns of access time against a real 3920, a factor of 4.9, and the per-tier counters it publishes look completely correct.

The unknown-tier default matters more than it looks. Charging an unrecognised tier at the slowest rate is the conservative direction: a system that charges it as near memory quietly assumes the best about the thing it does not understand. The check is a strict > and the bench drives tier 2 — the highest known tier — to prove the boundary is where it says it is.

6. RTL 2 — Promotion Is A Bet

// Promotion is a bet that a page will be accessed enough to repay the move.
module promote_policy #(parameter int ALWAYS_PROMOTE = 0) (
  input  logic clk, rst_n,
  input  logic       touch,
  input  logic [7:0] heat,          // accesses seen in the current window
  input  logic [7:0] threshold,
  input  logic       upper_has_room,
  output logic       promote,
  output logic [7:0] n_promoted, n_declined,
  output logic       cold_promote_err
);
  logic hot_enough;
  assign hot_enough = (heat >= threshold);
  // The correct policy promotes a page that is both hot enough and has somewhere
  // to go. The always-promote build moves every page it sees touched.
  assign promote = (ALWAYS_PROMOTE != 0) ? (touch && upper_has_room)
                                         : (touch && hot_enough && upper_has_room);
  // Moving a page that has not earned the move.
  assign cold_promote_err = promote && !hot_enough;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_promoted <= 8'd0; n_declined <= 8'd0;
    end else if (touch) begin
      if (promote) n_promoted <= n_promoted + 8'd1;
      else         n_declined <= n_declined + 8'd1;
    end
  end
endmodule

A threshold of four:

  promote: promoted=1 declined=3 cold=0 | always-promote promoted=3 cold=2

The threshold is inclusive and the bench drives it exactly: heat 3 does not promote, heat 4 does. > instead of >= is invisible to any test that only offers clearly-hot and clearly-cold pages, and it shifts every promotion decision in the system by one access.

Both builds decline the hot page when there is no room. That is the case that separates "the policy said yes" from "the move happened", and it is why upper_has_room is a term in both builds rather than a wrapper around the policy. A promotion decision that does not know whether the destination exists is not a decision.

7. RTL 3 — The Move Costs, And The Cost Must Be Repaid

// A move costs, and the cost must be repaid out of the saving.
module migration_cost #(parameter int FREE_MIGRATION = 0) (
  input  logic clk, rst_n,
  input  logic        moved, access_after,
  input  logic [15:0] move_ns, saving_ns,   // cost of the move, saving per access
  output logic [31:0] debt_ns, repaid_ns,
  output logic [15:0] breakeven_acc,
  output logic        in_profit, never_repaid_err
);
  logic [31:0] be_q;
  // The free-migration model charges nothing to move, so every promotion is
  // profitable from the first access and the policy has no reason to be selective.
  assign be_q = (saving_ns == 16'd0) ? 32'hFFFFFFFF
              : ((FREE_MIGRATION != 0) ? 32'd0
                                       : (({16'd0, move_ns} + {16'd0, saving_ns} - 32'd1)
                                          / {16'd0, saving_ns}));
  assign breakeven_acc = (be_q > 32'd65535) ? 16'hFFFF : be_q[15:0];
  assign in_profit = (repaid_ns >= debt_ns);
  // A page moved and then never accessed enough to repay the move.
  assign never_repaid_err = (debt_ns != 32'd0) && !in_profit;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      debt_ns <= 32'd0; repaid_ns <= 32'd0;
    end else begin
      if (moved && (FREE_MIGRATION == 0)) debt_ns <= debt_ns + {16'd0, move_ns};
      if (access_after)                   repaid_ns <= repaid_ns + {16'd0, saving_ns};
    end
  end
endmodule

A 2000 ns move saving 300 ns per access:

  migration: breakeven=7 debt=3900ns repaid=3900ns profit=1 | free breakeven=0

Seven, not six. 2000 ÷ 300 is 6.67, and six accesses repay 1800 of a 2000 ns debt. The + saving - 1 in the numerator is a ceiling division, and the mutation that removes it — "breakeven rounds down" — produces a policy that declares profit one access early on every migration it ever makes.

The free-migration build reports a break-even of zero: every promotion is profitable from the moment it is made, which is exactly why a system modelled that way has no reason to be selective about promotions and will move everything it touches.

The bench drives the profit boundary from both sides and exactly on it. Six accesses repay 1800 against 2000 — not in profit. The seventh reaches 2100 — in profit. Then a second, cheaper move takes the debt to exactly 3900 and six further accesses repay exactly 3900, which is the case where repaid == debt. Without that third case, > and >= are indistinguishable.

8. Waveform — A Page Promoted, And A Page That Should Not Have Been

Transcribed from the printed trace. One stimulus stream, both builds.

The two debt rows are the argument. They start at the same value and only one of them is falling.

9. RTL 4 — The Fast Tier Is Finite

// Promotion requires demotion: the upper tier is finite, and a tier whose
// residents are all pinned cannot accept a promotion at all.
module tier_capacity #(parameter int UNBOUNDED_UPPER = 0) (
  input  logic clk, rst_n,
  input  logic       promote_req, can_demote,
  input  logic [7:0] upper_cap,
  output logic       upper_has_room, demote_needed, promote_ok,
  output logic [7:0] upper_used, n_demoted, n_promoted, n_blocked
);
  // The unbounded build never runs out of the fast tier, which is the modelling
  // error that makes every promotion policy look free.
  assign upper_has_room = (UNBOUNDED_UPPER != 0) ? 1'b1 : (upper_used < upper_cap);
  // A promotion into a full tier must evict something first, and can only do so
  // if something in the tier is evictable.
  assign demote_needed  = promote_req && !upper_has_room && can_demote;
  assign promote_ok     = promote_req && (upper_has_room || demote_needed);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      upper_used <= 8'd0; n_demoted <= 8'd0; n_promoted <= 8'd0; n_blocked <= 8'd0;
    end else if (promote_req) begin
      if (upper_has_room) begin
        upper_used <= upper_used + 8'd1;
        n_promoted <= n_promoted + 8'd1;
      end else if (demote_needed) begin
        // One out, one in: occupancy is unchanged.
        n_demoted  <= n_demoted + 8'd1;
        n_promoted <= n_promoted + 8'd1;
      end else begin
        // A full tier with nothing evictable refuses the promotion.
        n_blocked <= n_blocked + 8'd1;
      end
    end
  end
endmodule

A three-page upper tier:

  capacity: used=3/3 promoted=4 demoted=1 | unbounded used=5 demoted=0

Four promotions into a three-page tier. The correct build performed one demotion and its occupancy never exceeded three. The unbounded build promoted the same pages, demoted nothing, and its occupancy is five — a tier holding more pages than it has.

10. RTL 5 — Hysteresis, Or A Policy That Changes Its Mind

// A page that oscillates between tiers pays the move cost every time and never
// collects the saving.
module thrash_detect #(parameter int NO_HYSTERESIS = 0) (
  input  logic clk, rst_n,
  input  logic       evaluate, wants_up, wants_down,
  input  logic [7:0] dwell_min,
  output logic       move_up, move_down, in_upper,
  output logic [7:0] dwell, n_moves, n_held,
  output logic       thrash_err
);
  logic settled;
  // A page must sit in its tier for a minimum dwell before it may move again.
  // The build without hysteresis moves the instant the policy changes its mind.
  assign settled   = (dwell >= dwell_min);
  assign move_up   = evaluate && wants_up   && !in_upper
                     && ((NO_HYSTERESIS != 0) || settled);
  assign move_down = evaluate && wants_down &&  in_upper
                     && ((NO_HYSTERESIS != 0) || settled);
  // Moving a page that has not sat still long enough to have been worth moving.
  assign thrash_err = (move_up || move_down) && !settled;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      in_upper <= 1'b0; dwell <= 8'd0; n_moves <= 8'd0; n_held <= 8'd0;
    end else if (evaluate) begin
      if (move_up || move_down) begin
        in_upper <= move_up;
        dwell    <= 8'd0;
        n_moves  <= n_moves + 8'd1;
      end else begin
        dwell   <= dwell + 8'd1;
        if (wants_up || wants_down) n_held <= n_held + 8'd1;
      end
    end
  end
endmodule

The bench drives twelve cycles of a policy that alternates its mind every cycle — the exact pattern hysteresis exists for:

  thrash: moves=3 held=14 thrash=0 | no-hysteresis moves=14 thrash=13

Three moves against fourteen. The hysteretic build held the page still on fourteen evaluations where the policy wanted it moved; the build without hysteresis moved it almost every cycle and thirteen of those moves were made before the page had settled.

At 2000 ns per move from section 7, fourteen moves is 28 µs of migration work for a page the system never held still long enough to benefit from. The thrashing system is doing more work than the one that does nothing at all, and its promotion counters look busy and healthy.

The n_held counter is worth reading alongside n_moves. Fourteen held evaluations means the policy asked for a move fourteen times and was refused, and each refusal is a migration that did not happen. A tiering system with a high held count is not a system that is failing to act; it is a system declining to pay for a move it would not have recovered.

The dwell boundary is where the two builds are closest. dwell >= dwell_min against dwell > dwell_min differs on exactly one evaluation per residency, which across a long run is a few percent more moves — invisible in any aggregate, and it shifts every hysteresis decision in the system by one interval.

The dwell boundary is inclusive and driven exactly, and in_upper <= move_up is asserted rather than assumed — the mutation that hardwires it to 1 makes every move a promotion, which a bench that only ever moves upward cannot see.

11. RTL 6 — The Mean Across The Mix

// The number a capacity planner reads: mean latency across the tier mix.
module effective_latency (
  input  logic clk, rst_n,
  input  logic        sample,
  input  logic [15:0] n0, n1, n2,
  input  logic [15:0] t0_ns, t1_ns, t2_ns,
  output logic [31:0] weighted_ns, total_acc,
  output logic [15:0] mean_ns,
  output logic [15:0] worst_ns,
  output logic        mean_hides_tail_err
);
  logic [31:0] mean_q;
  assign weighted_ns = {16'd0, n0} * {16'd0, t0_ns}
                     + {16'd0, n1} * {16'd0, t1_ns}
                     + {16'd0, n2} * {16'd0, t2_ns};
  assign total_acc = {16'd0, n0} + {16'd0, n1} + {16'd0, n2};
  assign mean_q  = (total_acc == 32'd0) ? 32'd0 : (weighted_ns / total_acc);
  assign mean_ns = mean_q[15:0];
  // The slowest tier that actually carries traffic, not the slowest tier present.
  assign worst_ns = (n2 != 16'd0) ? t2_ns : ((n1 != 16'd0) ? t1_ns : t0_ns);
  // A mean that is under half the worst tier in use is describing a distribution
  // whose tail it does not represent.
  assign mean_hides_tail_err = sample && (total_acc != 32'd0)
                               && ({16'd0, worst_ns} > ({16'd0, mean_ns} * 32'd2));
endmodule

A thousand accesses, 900 near, 90 middle, 10 far:

The first row is the number a tiered system reports and the reason it is misleading: a mean of 108 ns on a machine where one access in a hundred takes 900. Nothing about 108 is wrong, and nothing about it tells you that a percentile plot has a tail eight times the mean.

The third row is why worst_ns is the slowest tier in use rather than the slowest tier present. With no far traffic at all, the worst a request can experience is 300 ns, and a device that reports 900 because an SCM tier exists somewhere is reporting a latency nothing can experience. The mutation worst_ns = t2_ns is exactly that, and only the third row kills it.

12. RTL 7 — Which Page Leaves

// Which page leaves the fast tier. The victim choice is the policy.
module demote_victim #(parameter int EVICT_NEWEST = 0) (
  input  logic clk, rst_n,
  input  logic       evict,
  input  logic [7:0] age0, age1, age2, age3,   // cycles since last touch
  output logic [1:0] victim,
  output logic [7:0] victim_age, oldest_age, newest_age,
  output logic [7:0] n_evicted,
  output logic       wrong_victim_err
);
  logic [1:0] oldest, newest;
  assign oldest = (age0 >= age1)
                ? ((age0 >= age2) ? ((age0 >= age3) ? 2'd0 : 2'd3)
                                  : ((age2 >= age3) ? 2'd2 : 2'd3))
                : ((age1 >= age2) ? ((age1 >= age3) ? 2'd1 : 2'd3)
                                  : ((age2 >= age3) ? 2'd2 : 2'd3));
  assign newest = (age0 <= age1)
                ? ((age0 <= age2) ? ((age0 <= age3) ? 2'd0 : 2'd3)
                                  : ((age2 <= age3) ? 2'd2 : 2'd3))
                : ((age1 <= age2) ? ((age1 <= age3) ? 2'd1 : 2'd3)
                                  : ((age2 <= age3) ? 2'd2 : 2'd3));
  // Evicting the newest page is evicting the one most likely to be touched next,
  // and it is what a policy does when it treats the fast tier as a stack.
  assign victim = (EVICT_NEWEST != 0) ? newest : oldest;
  // ... victim_age / oldest_age / newest_age selected by case, omitted for length
  // Evicting a page that is not the coldest resident.
  assign wrong_victim_err = evict && (victim_age != oldest_age);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) n_evicted <= 8'd0;
    else if (evict) n_evicted <= n_evicted + 8'd1;
  end
endmodule

  victim: evictions=3 wrong=0 | newest-eviction wrong=2

Three evictions with three different age orderings, so the answer is never a fixed index. The third is the interesting one: all four ages equal. Every resident is equally cold, any victim is a coldest victim, and wrong_victim_err correctly fires in neither build.

That case is what makes the check victim_age != oldest_age rather than victim != oldest. Comparing indices would report a wrong victim whenever two pages tie and the two builds broke the tie differently, which is not an error — it is two correct answers.

wrong_victim_err is gated on evict, and the bench proves the gate by presenting a wildly unequal set of ages with no eviction in progress and asserting the checker stays quiet. A checker that fires while nothing is happening reports errors on an idle system.

13. RTL 8 — Did The Move Repay Itself

// Amortisation: a promotion is only correct if the page is accessed enough times
// afterwards to repay the move.
module amortisation #(parameter int ASSUME_REUSE = 0) (
  input  logic clk, rst_n,
  input  logic        settle,
  input  logic [15:0] acc_after, breakeven,
  output logic        was_worth_it,
  output logic [15:0] n_good, n_wasted, surplus,
  output logic        waste_err
);
  logic met;
  assign met = (acc_after >= breakeven);
  // The assuming build declares every promotion worthwhile, which is what a
  // policy does when it has no way to observe what happened after the move.
  assign was_worth_it = (ASSUME_REUSE != 0) ? 1'b1 : met;
  assign surplus = met ? (acc_after - breakeven) : 16'd0;
  // Calling a promotion worthwhile when the page was not accessed enough.
  assign waste_err = settle && was_worth_it && !met;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_good <= 16'd0; n_wasted <= 16'd0;
    end else if (settle) begin
      if (met) n_good   <= n_good + 16'd1;
      else     n_wasted <= n_wasted + 16'd1;
    end
  end
endmodule

  amortise: good=2 wasted=1 | assuming build waste-errs=1

Both builds count two good and one wasted. n_good and n_wasted are driven by met, which is the same expression in both — the counters are honest in the assuming build too. What differs is was_worth_it, the answer the policy acts on.

This is the third time in this batch that a broken build's counters are indistinguishable from a correct build's, and it is the same structural reason each time: the counter measures reality and the signal carries the claim.

surplus is gated on met for a reason the mutation makes clear. Removing the gate computes acc_after - breakeven unconditionally, and on an unmet promotion that is an unsigned subtraction going negative — three accesses against a break-even of seven reports a surplus of 65,532.

14. RTL 9 — Capacity Share Is Not Traffic Share

// What the system reports against what the workload experiences.
module tier_reporting #(parameter int REPORT_CAPACITY = 0) (
  input  logic clk, rst_n,
  input  logic        report,
  input  logic [15:0] cap_near_gb, cap_far_gb,
  input  logic [15:0] acc_near, acc_far,
  output logic [15:0] total_gb, far_share_pct, far_traffic_pct,
  output logic [15:0] headline_pct,
  output logic        misleading_report_err
);
  logic [31:0] cap_tot, acc_tot, cs_q, at_q;
  assign cap_tot = {16'd0, cap_near_gb} + {16'd0, cap_far_gb};
  assign acc_tot = {16'd0, acc_near}    + {16'd0, acc_far};
  assign total_gb = cap_tot[15:0];
  // Share of CAPACITY that is far, and share of TRAFFIC that is far. They are
  // different questions and a tiered system is sold on the first one.
  assign cs_q = (cap_tot == 32'd0) ? 32'd0 : (({16'd0, cap_far_gb} * 32'd100) / cap_tot);
  assign at_q = (acc_tot == 32'd0) ? 32'd0 : (({16'd0, acc_far}    * 32'd100) / acc_tot);
  assign far_share_pct   = cs_q[15:0];
  assign far_traffic_pct = at_q[15:0];
  // The headline number: capacity share reads well, traffic share is what the
  // workload feels.
  assign headline_pct = (REPORT_CAPACITY != 0) ? far_share_pct : far_traffic_pct;
  // Reporting a capacity share as though it described the access pattern.
  assign misleading_report_err = report && (REPORT_CAPACITY != 0)
                                 && (far_share_pct != far_traffic_pct);
endmodule

192 GB of 256 is far capacity; 100 of 1000 accesses are far traffic:

  report: total=256GB capacity_far=75% traffic_far=10% headline=10% | capacity build headline=75%

Seventy-five percent against ten. Both numbers are true and they answer different questions. "Three quarters of this machine's memory is CXL-attached" is a procurement statement. "One access in ten goes to the far tier" is the performance statement, and the two diverge by a factor of seven and a half here because tiering is working — the whole point is to put most of the capacity far away and most of the traffic near.

The failure is not reporting the capacity share. It is reporting the capacity share as though it described the access pattern, which is why misleading_report_err requires the two to actually differ. The bench drives the case where they coincide — 75% capacity and 75% traffic — and the check correctly goes quiet: a capacity report is not misleading when it happens to match.

15. RTL 10 — The Tiering Decision Assembled

// The tiering decision assembled: every gate a promotion must pass.
module tiering_system #(parameter int SKIP_AMORT = 0) (
  input  logic clk, rst_n,
  input  logic       evaluate,
  input  logic       hot_enough,      // the page has earned attention
  input  logic       upper_room,      // there is somewhere to put it
  input  logic       settled,         // it has not just moved
  input  logic       repays_move,     // it will be accessed enough to pay
  output logic       promote,
  output logic [3:0] fail_mask,
  output logic [7:0] n_eval, n_promoted,
  output logic       unpaid_promote_err
);
  // One bit per gate, so a declined promotion says which gate declined it.
  assign fail_mask[0] = ~hot_enough;
  assign fail_mask[1] = ~upper_room;
  assign fail_mask[2] = ~settled;
  assign fail_mask[3] = ~repays_move;
  // The skipping build drops the amortisation gate, which is the only one that
  // depends on what happens AFTER the decision.
  assign promote = (SKIP_AMORT != 0) ? (hot_enough && upper_room && settled)
                                     : (fail_mask == 4'd0);
  // Promoting a page that will not repay the move.
  assign unpaid_promote_err = evaluate && promote && (fail_mask != 4'd0);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_eval <= 8'd0; n_promoted <= 8'd0;
    end else if (evaluate) begin
      n_eval <= n_eval + 8'd1;
      if (promote) n_promoted <= n_promoted + 8'd1;
    end
  end
endmodule

Five evaluations — all gates passing, then each falsified alone:

  system: evaluated=5 promoted=1 | skip-amortisation promoted=2

The gate that is dropped is chosen deliberately. Heat, room and dwell are all knowable at the moment of the decision; amortisation is a statement about the future, and a system that cannot observe what happened after a move has no way to compute it. That is precisely why it is the gate most real policies are built without — and the one whose absence produces a system that migrates constantly and improves nothing.

Each of the four gates is falsified alone and produces its own single-bit mask, for the same reason as 17.2 section 15: 4'b1000 sends an engineer to the amortisation model and "not promoted" sends them to all four.

16. Quantitative Reasoning

Every number is from a printed line above.

Tier cost. 4 × 80 + 3 × 300 + 3 × 900 = 3920 ns. The flat model: 10 × 80 = 800. A factor of 4.9, with identical per-tier counters.

Break-even. ⌈2000 ÷ 300⌉ = 7 accesses. Six repay 1800 and are not enough. The rounding matters: floor gives 6, and a policy using it declares profit one access early on every migration.

Thrashing. Twelve oscillations produce 3 moves with hysteresis and 14 without. At 2000 ns each that is 6 µs against 28 µs, and the thrashing system's benefit is zero because no page ever stayed.

Capacity exchange. Four promotions into a three-page tier: 1 demotion, occupancy 3. The unbounded model: 0 demotions, occupancy 5 — two pages more than the tier holds.

The mean. 900/90/10 across 80/300/900 ns gives 108 ns mean against a 900 ns worst tier in use — a tail 8.3× the mean, on a system reporting a two-digit latency.

Capacity against traffic. 75% of capacity far, 10% of traffic far. A ratio of 7.5, and both numbers correct.

Surplus. Eleven accesses against a break-even of seven is a surplus of 4. Three accesses against seven is a surplus of zero, not 65,532, and only the gate makes that true.

17. Assertions

Every property is an immediate check written as cond !== 1'b1, sampled after a settle. 139 assertion sites across two testbenches.

18. Mutation Testing

83 mutations, one at a time, each required to make the baseline print RESULT: FAIL.

83 of 83 were killed.

The first run killed 75 and left 5 survivors, of which two were provably equivalent and were replaced rather than counted:

The two equivalent survivors are the interesting result of this chapter.

never_repaid_err = !in_profit survived because debt_ns == 0 implies repaid_ns >= debt_ns implies in_profit, so the debt != 0 term can never change the outcome. The term is defensive and documents intent, so it stays; the mutation was replaced with a reversed profit comparison.

The room test in the capacity fill path survived because reaching that path already required upper_has_room. That one was not harmless. Chasing it showed the model's third branch — a promotion refused outright — could not be reached from any input, which made n_blocked a counter that could never increment. The fix was to make the branch real: a full tier whose residents are all pinned genuinely cannot accept a promotion, can_demote became an input, and three new mutations now target the reshaped logic.

A representative sample:

19. Verification Strategy

Two builds, one stimulus. The parameter is the only difference, so every divergence in a printed line is attributable to one line of RTL.

Drive the boundary from both sides and on it. Heat exactly at the threshold. Repayment exactly equal to debt. Dwell exactly at the minimum. The exactly-equal case needed a second migration constructed so the numbers would land on it, because the first scenario's arithmetic stepped straight over the boundary.

Vary the answer so it cannot be a fixed index. The victim model is exercised with three different age orderings, and a fourth where all ages tie.

Drive the idle case for every gated checker. wrong_victim_err with no eviction, unknown_tier_err on the highest known tier, mean_hides_tail_err with no traffic at all.

Exercise the pathological input, not just the representative one. Twelve cycles of an oscillating policy is not a realistic access pattern; it is the pattern hysteresis exists to survive, and without it the difference between three moves and fourteen never appears.

Treat a surviving mutation as a question about the design. Two survivors here were equivalent, and following one of them found an unreachable branch and a dead counter that no amount of additional stimulus would have exposed.

20. Synthesis and Implementation Reality

None of this is hardware in the usual sense. The tiering policy is firmware or operating-system code, and the models here are executable specifications for it. What is hardware is the observation: the access counters per tier, the heat counters per page, and the timestamps the victim policy reads.

Heat is the expensive part. A per-page access counter across a terabyte of far memory is a very large number of counters. Real systems sample — a fraction of accesses, or a fraction of pages, or hardware that tracks only recently-accessed pages — which means heat is an estimate, and the threshold comparison in section 6 is being made against a noisy input. That does not change the structure; it changes how far above the threshold a page should be before the bet is worth placing.

The four-way comparator tree in section 12 does not scale. It is written flat for readability at four residents. At any realistic occupancy the victim search is a data-structure problem — an approximate-LRU clock, a sampled subset — and the approximation reintroduces the tie case that section 12's victim_age != oldest_age check exists to tolerate.

Migration is not atomic. The models move a page in one cycle. A real migration must handle accesses to a page that is mid-flight, which is the single largest source of complexity in a real implementation and is deliberately outside this chapter.

The dwell counter must survive the policy's own restart. A dwell that resets when the tiering daemon restarts allows a burst of thrashing at every restart — the same class of bug as 17.2 section 20's non-persistent wear-levelling pointer.

21. Silicon Observability

The third row is the one systems usually lack. Heat is measured before a promotion because that is what the decision needs; almost nothing measures accesses after one, which is why the amortisation gate is the one most policies are built without. A system that cannot answer "did that move pay for itself" cannot tell a working tiering policy from a thrashing one — both are busy.

22. Debug Lab

Symptom: a tiered system performs worse than the same machine with tiering disabled.

Compare migrations per interval against page count. If they are the same order of magnitude, pages are moving repeatedly rather than settling. Check the dwell minimum first — section 10's oscillation produced fourteen moves for one page.

Compute the mean migration benefit. Accesses after promotion, divided across promotions, against the break-even from section 7. If the mean is below break-even the policy is net-negative and turning it off is the correct immediate action.

Check whether the upper tier is at capacity. If it is, every promotion is an exchange and the demotion is evicting something. Compare demotion count against promotion count: they should be nearly equal on a full tier, and a demotion count near zero on a full tier means promotions are being blocked rather than exchanged.

Compare traffic share against capacity share. If far traffic is high, tiering is not doing its job — the hot data is not being found. If far traffic is low and performance is still bad, the problem is not placement and this is the wrong investigation.

Look at the tail, not the mean. A mean of 108 ns with a 900 ns tier in use is section 11's first row, and the applications complaining are the ones landing in the tail.

23. Design Review

What does a promotion cost, in nanoseconds, on this platform? If nobody can answer, the amortisation gate cannot exist and the policy is placing bets without knowing the stake.

How many accesses after a move repay it? Break-even must be a computed number, rounded up.

What is the minimum dwell, and what happens at policy restart? A dwell that resets allows a thrash burst on every restart.

Is a promotion into a full tier an exchange or a refusal? Both are defensible; not knowing which one the system does is not.

What does the victim policy do when residents tie? Any coldest page is a correct answer, and a policy that reports an error on a tie will report errors constantly.

Which number is on the dashboard: capacity share or traffic share? And does anybody reading it know which one it is?

24. How This Appears In Real Engineering

Tiering problems arrive as "the new memory made it slower", which is a claim that is usually true and almost never about the memory.

The characteristic case is a workload whose working set does not fit the fast tier. Every promotion evicts something that is about to be needed, the evicted page is promoted back, and the system spends its bandwidth moving pages rather than serving them. Migration counters are high, tier occupancy looks healthy, and every individual decision was locally correct.

The second characteristic case is a workload that never reuses anything — a streaming scan. Every page it touches looks hot exactly once, gets promoted, is never touched again, and section 13's amortisation gate is the only thing in the system that would have declined it.

The third is a reporting problem rather than a performance one: a machine advertised on its capacity share performing according to its traffic share, and the gap between the two numbers being discovered after purchase.

25. Common Misconceptions

"More fast memory is always better." Only if the fast tier is large enough to hold the working set. Below that threshold, promotions evict pages that are about to be needed and the system thrashes.

"Promotion is free — the page had to be read anyway." The page had to be read; it did not have to be copied, and the copy is bandwidth on the same link the workload is using. Section 7's break-even of seven accesses is the cost expressed in the only currency that matters.

"Mean latency tells you how the machine feels." A mean of 108 ns with a 900 ns tier in use describes a distribution whose tail is 8.3× the number reported.

"A tiering policy that moves a lot of pages is working hard." Or thrashing. Section 10's build made fourteen moves and produced no benefit at all, and its counters looked busier than the correct one's.

"The fast tier is where hot pages go." It is where hot pages go if there is room, which means every promotion into a full tier is also a demotion decision, and the demotion is at least as consequential as the promotion.

"Evict the newest — it was just brought in, so it is cheap to drop." It was just brought in because something wanted it. Section 12's build evicts the page most likely to be needed next, and it does so on every eviction.

"Seventy-five percent of our memory is CXL." True, and it says nothing about performance. Ten percent of the traffic goes there, which is the number that would.

26. Interview Reasoning

Q1. Why is a flat memory model not just an approximation on a tiered machine? Because it produces the same answer regardless of the tier mix. It cannot be wrong about a particular workload — it has no input that would let it be right about any.

Q2. What is a promotion, economically? A cost paid now against a saving that only exists if the page is accessed again. It is a bet, and the odds depend on information the policy does not have.

Q3. 2000ns to move a page, 300ns saved per access. How many accesses to break even? Seven. Six repays 1800 of 2000. Rounding down gives six and declares profit one access early, every time.

Q4. What is wrong with a promotion that is never followed by an access? It cost the move, it evicted something, and it returned nothing. It is strictly worse than doing nothing, which is not true of merely unhelpful work.

Q5. Why does a finite upper tier change the policy rather than just constrain it? Because every promotion becomes an exchange. The question stops being "is this page worth promoting" and becomes "is this page worth more than the one it displaces".

Q6. What does hysteresis buy you? The difference between three moves and fourteen on an oscillating workload. Without it a page pays the migration cost every time the policy changes its mind and collects the saving never.

Q7. A tiering system reports high migration counts. Good or bad? Unknowable from that number alone. Compare accesses-after-promotion against break-even. High migration with low reuse is thrashing; high migration with high reuse is the system working.

Q8. Why is "accesses after promotion" rarely measured? Because the decision only needs heat before the move. Measuring after requires tracking a page you have already acted on, which nothing else in the policy needs — and it is the only input to the amortisation gate.

Q9. Why is the worst tier "in use" rather than the worst tier present? Because a tier carrying no traffic cannot produce a latency anybody experiences. Reporting 900ns because an SCM tier exists somewhere describes a machine nobody is running.

Q10. A mean of 108ns with a 900ns tier. What is the problem? The mean is correct and the tail is 8.3× it. The applications complaining are in the tail, and the dashboard is showing the number that hides them.

Q11. Why evict the oldest rather than the newest? Because the newest was brought in because something wanted it. Evicting it discards the page most likely to be needed next, and does so on every eviction.

Q12. All four residents have the same age. Which do you evict? Any of them — they are all coldest. A policy that treats a tie as an error will report errors on every uniform workload, which is why the check compares ages and not indices.

Q13. Seventy-five percent of capacity is far and ten percent of traffic is. Is that good or bad? Good — it is what tiering is for. The problem is only ever reporting the first number as though it were the second.

Q14. Which of the four promotion gates is hardest to build? Amortisation, because it is the only one about the future. The other three are observable at the moment of the decision.

Q15. A machine gets slower when tiering is switched on. First check? Migrations per interval against page count. If they are comparable, pages are not settling and the dwell minimum is where to look.

Q16. Why does a streaming scan defeat a heat-threshold policy? Every page it touches is hot exactly once. Heat, room and dwell all pass; only amortisation would decline it, and that gate needs information collected after the move.

27. Exercises

1. Extend RTL 1 so each tier has separate read and write costs, as 17.2 section 12 required. Which of the existing assertions survive unchanged?

2. In RTL 2, make heat decay over time rather than accumulating. What new boundary appears, and which existing mutation would it newly kill?

3. RTL 3 charges a fixed move cost. Make it proportional to page size, and determine the page size at which a 300ns saving never repays within a thousand accesses.

4. Add a pin input to RTL 4 that marks individual residents unevictable. At what pinned fraction does the tier stop accepting promotions entirely?

5. In RTL 5, make dwell_min adaptive — growing each time a page returns to a tier it recently left. Re-run the twelve-cycle oscillation and count the moves.

6. RTL 6 reports a mean. Add a p99 estimate from the per-tier counts and determine the traffic mix at which the mean and the p99 differ by less than 20 percent.

7. Replace RTL 7's exact-LRU tree with a two-bit clock approximation. How often does it pick a victim that is not the coldest, and does wrong_victim_err still mean what it meant?

8. Add a fifth gate to RTL 10 for available migration bandwidth. Where does it belong in the mask, and is it knowable at the moment of the decision?

28. Summary

Tiering is the decision that appears the moment memory stops being uniform, and it has six parts.

A page has a home, and the home is the cost. 3920 ns of real access time reported as 800 by a flat model whose per-tier counters are perfectly correct.

Promotion is a bet on reuse that has not happened yet, gated on heat and on there being somewhere to put the page.

The move has a price. Seven accesses to repay 2000 ns at 300 ns a time — and a policy that rounds the division down declares profit early on every migration it makes.

The fast tier is finite, so promotion is an exchange. A model with an unbounded upper tier finished holding five pages in a three-page tier and demoted nothing.

Hysteresis is what separates a policy from a thrash. Three moves against fourteen on the same oscillating input, and the thrashing build's counters look busier.

And the reported number is not the experienced one. Seventy-five percent of capacity far, ten percent of traffic far, a mean of 108 ns hiding a 900 ns tail.

17.4 — Real Industry Memory Devices takes these three chapters' models and asks what to do with an actual device somebody is offering to sell you.