Type 1 Devices

Module 9 ended by pricing what CXL.mem guarantees cost. Module 10 changes the question — from what must a device guarantee to what kind of device is it, and the answer turns out to be determined entirely by which protocols it speaks.

This chapter takes the first of the three: a device that caches host memory and owns none of its own.

1. The Engineering Problem — Compute That Lives on Someone Else's Data

An accelerator has a job to do on data the host owns. It has three options, and two of them are bad.

It can read the data over and over. Every access is a full round trip, and the accelerator spends its life waiting rather than computing.

It can copy the data into private memory. Now it is fast, and the copy is a snapshot: the moment the host writes, the accelerator is working on data that is quietly wrong. Every DMA-offload programming model in history has spent its complexity budget on exactly this problem — explicit copies, explicit invalidations, and a class of bug where the answer is stale rather than incorrect.

Or it can hold coherent copies of host lines, close enough to compute against, with the system responsible for telling it when a line it holds has changed.

The third option is a Type 1 device, and the whole of its architecture follows from that one decision.

2. The One-Sentence Model

A Type 1 device borrows. Every line it holds belongs to the host, it must give each one back when asked, it exposes no memory of its own, and its cache is finite — so it must also give lines back when it simply runs out of room.

Call it the borrower. That single word decides the hardware: something to hold the borrowed lines, something to track what has been borrowed, something to return them, and nothing at all on the other side — because a borrower lends nothing.

3. What This Chapter Owns

Three published chapters already touch device types, and the boundaries are worth stating before anything else.

3.3 says it deliberately "introduces the device types precisely enough to reason about hardware, and stops there", and defers the full treatment here. Module 8 taught CXL.cache as a protocol; this chapter is about the device that speaks it — the structures it must build, the obligations those structures create, and the ones it is forbidden to have.

4. The Type Is Derived, Not Declared

  // .io is present in every CXL device; the coherent engines decide the type.
  assign proto_mask = {HAS_MEM, HAS_CACHE, 1'b1};
 
  wire [1:0] derived = ( HAS_CACHE &&  HAS_MEM) ? 2'd2 :
                       ( HAS_CACHE && !HAS_MEM) ? 2'd1 :
                       (!HAS_CACHE &&  HAS_MEM) ? 2'd3 : 2'd0;

=== EXP1: the device type is derived from the engines ===
  cache only        : mask=011 type=1
  cache + memory    : mask=111 type=2
  memory only       : mask=101 type=3
  neither           : mask=001 type=0 (io-only, not a CXL device type)
  cache without memory derives Type 1                  : ok
  cache with memory derives Type 2                     : ok
  memory without cache derives Type 3                  : ok
  neither engine is io-only, not a type                : ok
  .io is present in every device                       : ok
  declared Type 2, no engines : type=2 mismatch=1
  a declared type with no engines was caught           : ok

3.3 put this well: you do not choose a type and then implement it; you choose which engines to build, and the type is what you get. The RTL makes that literal — dev_type is a function of the parameters, not a stored field.

The last two lines are why that matters in hardware. The abuse instance declares Type 2 with no cache engine and no memory engine anywhere in it, and the mismatch checker catches it. A design that stores its type in a separate register can disagree with itself, and the disagreement is discovered by software that enables a protocol the silicon cannot speak.

Note the fourth row. Neither engine gives .io alone — a PCIe-style device on a CXL link. It has no CXL device type at all, because the types enumerate the coherent combinations. That is not a gap in the taxonomy; it is the taxonomy being about coherence.

5. Teaching-model boundary

6. RTL 2 — The Cache Is Finite, and Eviction Is an Obligation

This is the structure that makes the device a Type 1, and its most important property is the one people design around rather than for: it runs out.

      if (need_evict) begin
        // Evicting a modified line without telling the host loses the only
        // up-to-date copy in the system.
        if (!SILENT_EVICT) begin
          evict_notify <= 1'b1;
          evict_addr   <= tag_q[victim_q];
        end else if (mod_q[victim_q]) begin
          n_silent_q    <= n_silent_q + 16'd1;
          lost_line_err <= 1'b1;
        end

=== EXP2: a Type 1 cache is finite, and eviction is an obligation ===
  8 fills into 8 lines : occupancy=8 peak=8 evictions=0
  the array filled to exactly 8 lines                  : ok
  no eviction was needed while lines were free         : ok
  9th fill : evict_notify=1 evict_addr=0 evictions=1
  the 9th fill raised an eviction notification         : ok
  silent-evict variant : evictions=1 silent losses=1 lost-line flag=1
  the silent variant notified nobody                   : ok
  it dropped exactly one modified line                 : ok
  and the lost-line checker caught the only copy going away : ok
  an evicting fill is one in and one out, so occupancy held : ok
  three more evictions : victims were 1, 2 and 3 in order : ok

In a private cache, an eviction is housekeeping. You drop a line, and if it was dirty you write it back to memory you own. Nothing outside the device has an opinion.

In a coherent device cache, an eviction is a message. The line belongs to the host, the host is tracking that this device holds it, and dropping it without saying so leaves the system believing a copy exists where none does — and if the line was modified, the device has just destroyed the only up-to-date copy in the system.

That is the difference this chapter exists to make, and it is why evict_notify is an output rather than an internal signal.

Occupancy held at 8 through four evictions

The assertion c_occ == 8 after the evicting fills is not bookkeeping trivia. An evicting fill is one in and one out, so a design that counts it as a net gain over-reports its own occupancy and will size the next generation's array from a number that was never real. The mutation that made an evicting fill increment occupancy was killed by exactly that assertion.

Why the victims had to be 1, 2, 3 — and not merely different

The rotation check was the last mutation to fall, and it taught something worth keeping.

The first attempt asserted the three evicted addresses were distinct. That passes on a design whose victim pointer never rotates — because a stuck pointer keeps evicting the line it just filled, and those addresses are all different too. Distinctness was true and useless.

    // Distinctness is NOT enough: a stuck victim pointer also produces three
    // different addresses, because it keeps re-evicting the line it just
    // filled. Only the exact victims prove the pointer rotated.
    chk((ev1==8'd1) && (ev2==8'd2) && (ev3==8'd3), "the victims were lines 1, 2 and 3 in order");

A stuck victim turns an 8-line cache into a 1-line cache with seven lines of dead storage, and every occupancy counter reports it as full and healthy.

The last two rows are the same hardware event with opposite consequences. The notification is not an optimisation or a courtesy — without it the host still believes this device holds the line, and the modified data has ceased to exist anywhere in the system.

7. RTL 3 — Every Line It Holds Was Borrowed

      if (rsp_valid) begin
        // A response whose tag is not live belongs to nothing.
        if (!live_q[rsp_tag] && !(do_req && (req_tag == rsp_tag)))
          orphan_rsp_err <= 1'b1;
        live_q[rsp_tag] <= 1'b0;

=== EXP3: every line a Type 1 holds was borrowed ===
  4 borrows outstanding : live=00001111 requests=4
  four tags are live                                   : ok
  all returned          : live=00000000 responses=4 orphan=0 dup=0
  every borrow was returned                            : ok
  abuse: response for an unissued tag : orphan=1
  a response matching nothing was caught               : ok
  abuse: same tag issued twice while live : duplicate=1
  reusing a live tag was caught                        : ok

Same transaction-identity machinery as 9.2's host-side table, on the other end of the link — and the symmetry is the point. 9.2's host tracked its accesses into device memory; here a device tracks its borrows from host memory. Same obligation, opposite direction, and a Type 1 device is the side that only ever does the second.

Note the same-cycle exemption in the orphan check: a response arriving in the cycle its request is issued is legal, and a checker that ignores that possibility fires on correct behaviour — the defect 9.5 hit from the other direction.

Both illegal-stimulus cases needed an abuse instance. Driving a response for an unissued tag, or reusing a live tag, into the instance under test would trip that instance's own checker and score as a failure. This is now standing practice across the track: illegal stimulus gets its own DUT.

8. RTL 4 — What a Type 1 Must Not Have

The defining negative. A Type 1 device offers the host no memory, so an inbound host-memory access has no correct answer — and the tempting wrong behaviour is to produce one anyway.

  // A Type 1 device has no host-visible memory, so only configuration space
  // is answerable. Serving a memory request implies storage it does not have.
  assign serve  = cfg_req || (SERVE_ANYWAY && host_mem_req);
  assign refuse = host_mem_req && !SERVE_ANYWAY;

=== EXP4: a Type 1 exposes no memory to the host ===
  config access : serve=0 refuse=0 cfg count=1
  configuration space is answerable                    : ok
  host memory read : correct serve=0 refuse=1 | serve-anyway serve=1
  a host memory access was refused                     : ok
  the serve-anyway variant answered it                 : ok
  refusals=1 | serve-anyway flagged serving absent memory=1
  the variant flagged answering for memory it does not have : ok

A device that answers for memory it does not have is worse than one that fails. It returns something — whatever its buffers held — and the requester has no way to know. The refusal is the correct behaviour precisely because it is visible.

This is also the cleanest way to state the Type 1/Type 2 boundary in hardware terms. The difference is not that Type 2 is bigger. It is that Type 2 has an answer to this request and Type 1 does not, and that single asymmetry is what 10.2 will show costs so much.

9. RTL 5 — The Cache Is a Performance Structure, Not Only a Correctness One

  assign compute      = work_valid && (NO_STALL || line_present);
  assign request_line = work_valid && !line_present;

=== EXP5: compute stalls on a line the device does not hold ===
  5 resident + 3 missing : computes=5 misses=3 stalls=3
  compute happened only on resident lines              : ok
  three misses each stalled the pipeline               : ok
  no-stall variant       : computes=8 absent-data flag=1
  the no-stall variant computed on all eight           : ok
  ...and flagged producing a result from data it never had : ok

Everything above is about correctness. This module is about why anyone builds the thing: the accelerator's throughput is gated by residency. Work arrives every cycle; compute happens only on cycles where the line is present.

The no-stall variant is the accelerator version of 9.6's no-merge write. It computes on all eight — a 60% throughput improvement on every counter — by producing results from data it never had. The performance gain and the wrong answer are the same event, and this is the third time in this track that the fastest variant has been the incorrect one.

10. Waveform — Residency Gates Compute

Teaching-model timing derived from the simplified RTL in this chapter. Not CXL wire timing.

work_valid is high for all nine cycles and computes advances on six of them. The cache is why that number is six rather than two, and it is also why it is six rather than nine.

Read cache_occupancy alongside stalls: the array is filling exactly while the pipeline is stalled. That is the accelerator's entire performance story in one picture — the device is not computing, it is borrowing, and the ratio of those two activities is what a Type 1 design is optimising.

11. RTL 6 — Caching Devices Are the Scarce Resource

      // Exceeding the DECLARED limit is always reportable -- guarding this with
      // the check-enable would switch it off on the only case that can trip it.
      if (n_caching_q > LIMIT[7:0]) over_limit_err <= 1'b1;

=== EXP6: caching devices are the scarce resource ===
  6 attach attempts, limit 4 : caching=4 admitted=4 refused=2 peak=4
  exactly four were admitted                           : ok
  the other two were refused, not silently accepted    : ok
  peak attachment was exactly the limit                : ok
  no-check variant : caching=6 over-limit flag=1
  the unchecked variant went past its declared limit   : ok
  and the over-limit checker reported it               : ok
  after one detach : caching=3
  3 cycles of attach+detach together : caching=3
  simultaneous attach and detach left the count unchanged : ok

This is a topology-level property that constrains a device-level design, and it is specific to caching devices — memory devices do not carry it.

The published figure is that CXL 3.0 fanout supports up to 16 CXL.cache devices. The reason a limit exists at all is asymmetric coherence: the host resolves coherence and tracks which devices hold which lines, so every caching device adds tracking state on the host side. A memory expander does not, which is why the two classes scale so differently and why 10.3 will look so much easier to deploy in quantity.

The simultaneous attach-and-detach check exists because the attachment count is the tenth appearance of the two-assignment counter defect in this track. One device leaving as another arrives must leave the count unchanged; two separate non-blocking assignments make it fall by one, and the topology slowly under-reports how many caching devices it is tracking.

12. RTL 7 — Is the Cache Earning Its Area?

=== EXP7: is the cache earning its area? ===
  accesses=40 hits=26 fills=14 evictions=3 writebacks=2
  oracle: accesses=40 hits=26 fills=14 evictions=3
  accesses and hits matched an independent oracle      : ok
  hits are strictly fewer than accesses                : ok
  both conservation laws held                          : ok
  abuse: hits with no accesses, evictions with no fills : flags=1 1

A 65% hit rate on 40 accesses. The two conservation laws — you cannot hit more often than you looked, and you cannot evict more lines than you filled — are what make the hit rate trustworthy rather than merely reported.

The mutation that counted fills as hits is the one to remember. It reports a 100% hit rate on the same workload. Every fill is a miss being serviced, so counting it as a hit means the counter reports its best number precisely when the cache is performing worst. That is not a rounding error; it is a metric that inverts under load, and a team tuning against it will optimise in the wrong direction.

13. Quantitative Reasoning

Illustrative, with assumptions stated.

What residency is worth

From §9: 8 work items, 5 computed immediately, 3 stalled. Suppose a miss costs a full round trip of 250 ns and a hit costs 1 cycle at 1 GHz.

hit rate 0.65 : 0.65 x 1 ns + 0.35 x 250 ns = 88 ns per access
hit rate 0.90 : 0.90 x 1 ns + 0.10 x 250 ns = 25.9 ns per access
hit rate 0.99 : 0.99 x 1 ns + 0.01 x 250 ns =  3.5 ns per access

Going from 65% to 90% is a 3.4× improvement; from 90% to 99% is another 7.4×. The curve is dominated entirely by the miss term, which is why accelerator cache sizing is so sensitive near the working-set boundary — and why the honest question is never "how big is the cache" but "does the working set fit".

What the cache costs

storage = lines x (line bytes + tag bits + state bits)

For 1024 lines of 64 bytes with a 40-bit tag and 3 state bits:

data  = 1024 x 64 B      = 64 KB
meta  = 1024 x 43 bits   = 5.4 KB
total ~ 69.4 KB, of which 8% is metadata

The metadata fraction is the number worth carrying: 8% overhead to make 64 KB coherent. That is the price of the third option in §1, against a private copy that costs nothing extra in hardware and everything in software correctness.

What the fanout limit means for a rack

With a published limit of 16 CXL.cache devices in a CXL 3.0 fanout, a design that needs 24 caching accelerators does not need a bigger switch — it needs a different device type for some of them, or more root complexes. That is an architecture decision forced by a protocol limit, and it is the kind of constraint that is much cheaper to discover at architecture time than at bring-up.

14. Assertions

Icarus Verilog 13.0 does not support concurrent SVA here, so every property is synthesisable checker logic verified in simulation. 50 assertions.

Safety

Liveness

Performance goals

15. Mutation Testing

Twenty-four mutations. Twenty-four killed.

The first run scored 21 of 24, and the three escapes are worth separating because only one was a missing check.

Two needed stimulus that did not exist: a duplicate live tag, and a simultaneous attach and detach. Both went to abuse instances.

One was an assertion that was true and useless — the victim-rotation check described in §6. Asserting the evicted addresses were distinct passes on a design whose victim pointer is frozen, because a stuck pointer re-evicts the line it just filled and those addresses differ too. Only the exact victims — 1, 2, 3 — distinguish the two designs. This is the "assert the exact value, not a bound" rule from earlier batches appearing in a new disguise: here the useless assertion was a property rather than a bound, and it was equally hollow.

Note also over_limit_err, written deliberately without a guard on the check-enable parameter. That is the self-disabling-checker defect from 9.6 and 9.1, and the mutation that adds the guard back is killed by an abuse instance that exceeds its own declared limit.

16. Verification Plan

The coverage cross is cache state × line state × request type: free / full / evicting, crossed with clean / modified, crossed with fill / probe / config. The full × modified × fill point is where the only-copy-in-the-system risk lives, and it must be directed — a random workload reaches it rarely and never labels it.

17. Silicon Observability

The most valuable pair is eviction rate against fill rate. When they converge, the working set does not fit and the device is thrashing — every fill immediately evicts something that will be needed again. Hit rate alone shows this only indirectly and late; the ratio shows it immediately and says what to do about it.

18. Debug Lab

victim_q <= victim_q;   // rotation dropped in a late edit

  three more evictions : victims were 1, 2, 3
  the victims were lines 1, 2 and 3 in order           : ok

victim_q <= (victim_q == (LINES-1)) ? 4'd0 : victim_q + 4'd1;

// line is clean enough -- just drop it
vld_q[victim_q] <= 1'b0;

  silent-evict variant : evictions=1 silent losses=1 lost-line flag=1
  it dropped exactly one modified line                 : ok

evict_notify <= 1'b1;
evict_addr   <= tag_q[victim_q];

assign serve = cfg_req || host_mem_req;   // "just respond to everything"

  host memory read : correct serve=0 refuse=1 | serve-anyway serve=1
  the variant flagged answering for memory it does not have : ok

assign serve  = cfg_req;
assign refuse = host_mem_req;
if (host_mem_req && serve) served_absent_err <= 1'b1;

assign compute = work_valid;   // don't gate on residency, it's slow

  5 resident + 3 missing : computes=5 misses=3 stalls=3
  no-stall variant       : computes=8 absent-data flag=1
  ...and flagged producing a result from data it never had : ok

assign compute = work_valid && line_present;
if (compute && !line_present) computed_absent_err <= 1'b1;

wire counted_hit = hit || fill;   // "a fill satisfies the access too"

  accesses=40 hits=26 fills=14 evictions=3
  accesses and hits matched an independent oracle      : ok
  hits are strictly fewer than accesses                : ok

wire counted_hit = hit;
if (n_hit_q > n_access_q) hit_exceeds_access_err <= 1'b1;

if (admit)     n_caching_q <= n_caching_q + 8'd1;
if (do_detach) n_caching_q <= n_caching_q - 8'd1;

  3 cycles of attach+detach together : caching=3
  simultaneous attach and detach left the count unchanged : ok

case ({admit, do_detach})
  2'b10: n_caching_q <= n_caching_q + 8'd1;
  2'b01: n_caching_q <= n_caching_q - 8'd1;
  default: ;                                  // both or neither: hold
endcase

localparam [1:0] DEVICE_TYPE = 2'd2;   // a stored field, set by hand

  declared Type 2, no engines : type=2 mismatch=1
  a declared type with no engines was caught           : ok

wire [1:0] derived = (HAS_CACHE && HAS_MEM) ? 2'd2 : ...;
if (dev_type != derived) type_mismatch_err <= 1'b1;

if (do_req) live_q[req_tag] <= 1'b1;   // reused without checking

  abuse: response for an unissued tag : orphan=1
  abuse: same tag issued twice while live : duplicate=1
  reusing a live tag was caught                        : ok

if (live_q[req_tag]) dup_tag_err <= 1'b1;
if (!live_q[rsp_tag] && !(do_req && (req_tag == rsp_tag)))
  orphan_rsp_err <= 1'b1;

19. Design Review

1. Is the device type derived from the engines, or stored somewhere it can drift?
2. What happens on the fill that finds every line valid?
3. Is every eviction notified, including of clean lines?
4. What happens when a modified line is evicted, and what proves it?
5. Does the victim pointer rotate, and what assertion proves it rather than merely suggesting it?
6. What does the device do with an inbound host-memory access?
7. Can compute proceed on a line the device does not hold?
8. Can a tag be reused while its borrow is outstanding?
9. What is the hit rate, and what conservation law makes it trustworthy?
10. How many caching devices does the topology admit, and what happens to the one after that?
11. What state survives reset while lines are still borrowed?

20. How This Appears in Real Engineering

Architecture. The Type 1/Type 2 decision is settled by one question — does anything outside the device need to see its memory? If not, the memory engine is pure cost, and 6.5 prices it.

RTL. The cache array is small and carries four of this chapter's alarms. The eviction path is where private-cache instincts do the most damage.

DV. Fill-with-modified-lines-then-evict is the directed test that matters most, and a clean-line test cannot substitute for it.

Performance. Hit rate drives everything, and the miss term dominates the average so completely that working-set fit is the only sizing question worth asking.

Firmware. The device type determines which enable paths are legal; a stored type that disagrees with the silicon fails here first and looks like someone else's bug.

Post-silicon. Eviction rate against fill rate identifies thrashing immediately, where hit rate identifies it slowly and ambiguously.

21. Common Misconceptions

22. Interview Reasoning

23. Exercises

1. Calculation. A Type 1 accelerator has a 2048-line, 64-byte cache with 40-bit tags and 3 state bits. Compute total storage and the metadata fraction. Then compute average access time at hit rates of 0.7, 0.9 and 0.98 given a 1 ns hit and a 250 ns miss, and state which improvement is worth more.

2. Analysis. A device reports a 99% hit rate, a fill count equal to its eviction count, and poor accelerator throughput. Explain why those three facts are consistent, name the defect, and give the counter that distinguishes it from a genuinely well-performing cache.

3. RTL task. Extend dev_cache_array to track, per line, whether the host has requested it back but the eviction has not yet completed. State the new invariant that creates and the failure mode if the state is omitted.

4. Assertion task. Write the property that proves the victim pointer rotates. Explain why a property over the distinctness of evicted addresses does not prove it, and construct the design that satisfies distinctness while being wrong.

5. Waveform analysis. Using §10's trace, compute the achieved compute rate over the nine cycles and what it would have been with a hit on every cycle. Then state what the trace would look like if the fill latency were four cycles rather than one, and which counter would reveal it.

6. Debug task. An accelerator produces intermittently wrong results that correlate with input size. Give your investigation order across the four correctness alarms in this chapter, and the single directed test that separates a silent eviction from a compute-on-absent defect.

7. Design review. A colleague proposes adding a small .mem engine to a Type 1 device to expose its scratchpad, arguing it is "only a few registers of extra state". Give the strongest version of that argument, then state precisely what it changes about the device's obligations and what it makes the device.

24. Summary

A Type 1 device borrows.

• The type is derived from the engines present, never declared — a stored type can disagree with the silicon and enumerate as something the hardware cannot serve.
• .io is in every device; the coherent engines decide the type, and neither engine means no CXL device type at all.
• The cache is finite, so it must evict — and every eviction is a message, because the host is tracking that copy. Dropping a modified line silently destroys the only up-to-date copy in the system.
• An evicting fill is one in and one out. Counting it as a net gain over-reports occupancy and mis-sizes the next generation.
• Every line it holds was borrowed, tracked by identity — the mirror of 9.2's host-side table, on the other end of the link.
• A Type 1 has no host-visible memory, so an inbound memory access must be refused, not answered. That single asymmetry is the whole Type 1 / Type 2 boundary.
• Residency gates compute: work valid on nine cycles, compute on six. The cache is a performance structure, and the miss term dominates so heavily that working-set fit is the only sizing question.
• Caching devices are the scarce resource — every one adds host-side tracking state, and published fanout is up to 16 CXL.cache devices.
• Verification: 24 of 24 mutations killed, 50 assertions. The last to fall taught that an assertion can be true and useless — distinct evicted addresses do not prove a victim pointer rotates, because a frozen pointer produces distinct addresses too.
• The two-assignment counter defect appeared for the tenth time, here under-counting attached caching devices.

Next: 10.2 — Type 2 Devices, where the device both caches host memory and offers its own, and the two relationships interact in a way that makes it the hardest class in the taxonomy.