Switch Resource Sharing

16.1 built the switch's structure. 16.2 decided where a flit goes.

Both assumed there was somewhere to put it. This chapter is about that: the buffer, the credit that says it has room, and every way sharing it goes wrong.

1. The Engineering Problem — A Credit Is A Promise, And Promises Compose Badly

Six things make switch resource sharing harder than a queue.

A credit is a promise that a slot exists. The invariant is that the sender never holds more credits than the receiver has slots, and the only way to break it is to return a credit that was never spent. Section 5.

One buffer, several protocols. Sharing it makes the best case the whole buffer; reserving makes the worst case a fixed share. They fail in opposite directions and only measurement decides. Section 6.

Round-robin is not fairness by itself. It is fairness only if the pointer advances on a grant. A pointer that moves every cycle skips requesters that were not asking; one that never moves is fixed priority wearing a rotation. Section 8.

Virtual channels exist so one class cannot block another — and sharing the credit pool between them undoes the mechanism entirely while leaving the channel structure visibly intact. Section 9.

Strict priority is correct and starves. Ageing turns the guarantee from "eventually" into a bound on waiting. Section 10.

And two ports drawing from one pool, each holding what it has and waiting for more, is a deadlock the arbiter cannot break. Section 11.

This chapter against 16.1 and 16.2, stated precisely. 16.1 owns the stages a flit moves through and the head-of-line blocking one queue causes. 16.2 owns where it goes. This one owns who gets the slot — the credits, the reservations, the arbitration, and the guarantees. If a section here could be moved into either without loss, it is in the wrong chapter.

2. The One-Sentence Model

Every flit served needs a credit that exists, on a channel nothing else can exhaust, granted by an arbiter that bounds its waiting, from a pool no single port can take entirely — and every defect below is one of exists, nothing else, bounds or entirely missing.

3. What This Chapter Owns

Deferred:

4. Teaching-Model Boundary

An eight-slot buffer, two protocols, four requesters, a four-cycle ageing limit. A real switch has deeper buffers, more virtual channels and a far larger ageing constant.

What is faithful: the credit invariant, the reservation-versus-sharing trade, round-robin pointer discipline, virtual-channel independence, priority ageing, the pool deadlock and its floor, weighted bandwidth, and auditing a guarantee in both of its halves.

What is not: every depth, every weight, and every limit.

Every model is parameterised so the correct behaviour and a specific plausible failure are the same source under a different parameter, driven by one stimulus stream.

5. RTL 1 — The Credit That Says A Buffer Has Room

module credit_flow #(parameter int DOUBLE_RETURN = 0) (
  input  logic clk, rst_n,
  input  logic       send, slot_freed,
  output logic [3:0] credits, occupancy,
  output logic       may_send,
  output logic       overflow_err, credit_leak_err,
  output logic [7:0] n_sent, n_blocked, peak_occ, max_credits
);
  localparam logic [3:0] DEPTH = 4'd8;
  assign may_send = (cr_q != 4'd0);
  assign overflow_err = (occ_q > DEPTH);
  // More credits outstanding than the receiver ever had slots. Five bits:
  // a four-bit sum of 8 and 8 wraps to zero and the leak disappears.
  assign credit_leak_err = (({1'b0, cr_q} + {1'b0, occ_q}) > {1'b0, DEPTH});
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      cr_q <= DEPTH; occ_q <= 4'd0;
      n_sent <= 8'd0; n_blocked <= 8'd0;
      peak_occ <= 8'd0; max_credits <= 8'd0;
    end else begin
      if (send && may_send && slot_freed) begin
        occ_q <= occ_q;
        if (DOUBLE_RETURN != 0) cr_q <= cr_q + 4'd1;
      end else if (send && may_send) begin
        cr_q <= cr_q - 4'd1 + ((DOUBLE_RETURN != 0) ? 4'd1 : 4'd0);
        occ_q <= occ_q + 4'd1;
      // Guarded: freeing an empty buffer must not create a credit.
      end else if (slot_freed && occ_q != 4'd0) begin
        cr_q <= cr_q + 4'd1;
        occ_q <= occ_q - 4'd1;
      end
      if (send && may_send) begin
        n_sent <= n_sent + 8'd1;
        if ({4'd0, occ_q} + 8'd1 > peak_occ) peak_occ <= {4'd0, occ_q} + 8'd1;
      end else if (send) n_blocked <= n_blocked + 8'd1;
      if ({4'd0, cr_q} > max_credits) max_credits <= {4'd0, cr_q};
    end
  end
endmodule

  credits : depth 8, sent=8 blocked=1 peak=8 | double-return build leaked=1, held 8 credits

The invariant is credits + occupancy = depth, checked against a plain-integer oracle at three points: empty, full, and drained again. Every credit spent is a slot filled; every slot freed is a credit returned.

DOUBLE_RETURN returns a credit on the send as well as on the free, so the pool grows every time anything moves. The transcript catches it at one credit over the depth — the testbench samples after the first send, where the sum is exactly 9, rather than after the buffer is full.

Two guards are driven to their boundaries: a send with no credits is blocked and counted, and freeing an empty buffer does not create credits or underflow the occupancy.

6. RTL 2 — One Buffer, Two Protocols

module buffer_share #(parameter int NO_RESERVE = 0) (
  input  logic clk, rst_n,
  input  logic       req_a, req_b, free_a, free_b,
  output logic       grant_a, grant_b,
  output logic [3:0] used_a, used_b, free_slots,
  output logic       starved_err,
  output logic [7:0] n_a, n_b, n_denied_b, max_starve_b
);
  localparam logic [3:0] DEPTH = 4'd8;
  localparam logic [3:0] RESERVED_B = 4'd2;
  logic [3:0] ua_q, ub_q, used_total;
  logic [7:0] st_q;
  logic       a_headroom;
  assign used_total = ua_q + ub_q;
  assign free_slots = DEPTH - used_total;
  localparam logic [3:0] DEPTH = 4'd8;
  localparam logic [3:0] RESERVED_B = 4'd2;
  // A keeps out of B's reservation. NO_RESERVE lets A take the whole buffer.
  assign a_headroom = (NO_RESERVE != 0)
                    ? (used_total < DEPTH)
                    : (used_total < (DEPTH - RESERVED_B));
  assign grant_a = req_a && a_headroom;
  assign grant_b = req_b && (used_total < DEPTH);
  assign starved_err = req_b && !grant_b;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      ua_q <= 4'd0; ub_q <= 4'd0; st_q <= 8'd0;
      n_a <= 8'd0; n_b <= 8'd0; n_denied_b <= 8'd0; max_starve_b <= 8'd0;
    end else begin
      if (grant_a) begin ua_q <= ua_q + 4'd1; n_a <= n_a + 8'd1; end
      else if (free_a && ua_q != 4'd0) ua_q <= ua_q - 4'd1;
      if (grant_b) begin ub_q <= ub_q + 4'd1; n_b <= n_b + 8'd1; end
      else if (free_b && ub_q != 4'd0) ub_q <= ub_q - 4'd1;
      if (starved_err) begin
        n_denied_b <= n_denied_b + 8'd1;
        st_q <= st_q + 8'd1;
        if (st_q + 8'd1 > max_starve_b) max_starve_b <= st_q + 8'd1;
      end else st_q <= 8'd0;
    end
  end
endmodule

  buffer  : A used=6 of 8, B served=3 starved=1 | no-reserve build B served=0 starved run=3

A fills to six and stops, leaving B's two slots free. The no-reserve build lets A take all eight, and B is served zero times on the identical stimulus.

The reservation is a floor, not a share. It does not promise B any fraction; it promises B two slots exist whenever it asks. The transcript shows the honest consequence: the reserved build starves B once, after B has taken both of its slots and the buffer is genuinely full. That is a full buffer, not a starved protocol, and the two are distinguished by the run length — one against three.

The testbench drives B's usage into the same total as A's, so a design that counts only one protocol's occupancy is visible.

7. Waveform — Eight Cycles Of Ageing

Transcribed from the printed trace. Both builds see one stimulus stream.

Read grant_lo against noage_lo. The correct build grants the low-priority request once in eight cycles — which is not generous, and it is the entire difference between a bounded wait and an unbounded one.

8. RTL 3 — Who Gets The Grant

module arb_fair #(parameter int FIXED_PRIORITY = 0) (
  input  logic clk, rst_n,
  input  logic [3:0] req,
  output logic [3:0] grant,
  output logic [1:0] ptr,
  output logic       starvation_err,
  output logic [7:0] n_g0, n_g1, n_g2, n_g3, max_wait3
);
  logic [1:0] ptr_q;
  logic [7:0] w3_q;
  logic [3:0] rot, grot, g;
  assign ptr   = ptr_q;
  assign grant = g;
  // Rotate the request vector so the pointer's requester is bit 0, take the
  // lowest set bit, and rotate the grant back.
  assign rot  = (FIXED_PRIORITY != 0) ? req
              : ((req >> ptr_q) | (req << (3'd4 - {1'b0, ptr_q})));
  assign grot = rot & (~rot + 4'd1);
  assign g    = (FIXED_PRIORITY != 0) ? grot
              : ((grot << ptr_q) | (grot >> (3'd4 - {1'b0, ptr_q})));
  // Losing one arbitration is not starvation. Waiting longer than a full
  // round of four requesters is.
  assign starvation_err = req[3] && !g[3] && (w3_q >= 8'd4);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      ptr_q <= 2'd0; w3_q <= 8'd0;
      n_g0 <= 8'd0; n_g1 <= 8'd0; n_g2 <= 8'd0; n_g3 <= 8'd0; max_wait3 <= 8'd0;
    end else begin
      if (g[0]) n_g0 <= n_g0 + 8'd1;
      if (g[1]) n_g1 <= n_g1 + 8'd1;
      if (g[2]) n_g2 <= n_g2 + 8'd1;
      if (g[3]) n_g3 <= n_g3 + 8'd1;
      // The pointer advances past whoever was just served, and only then.
      if (g != 4'd0) begin
        if      (g[0]) ptr_q <= 2'd1;
        else if (g[1]) ptr_q <= 2'd2;
        else if (g[2]) ptr_q <= 2'd3;
        else           ptr_q <= 2'd0;
      end
      // The wait counts every cycle requester 3 asks and is refused.
      if (req[3] && !g[3]) begin
        w3_q <= w3_q + 8'd1;
        if (w3_q + 8'd1 > max_wait3) max_wait3 <= w3_q + 8'd1;
      end else w3_q <= 8'd0;
    end
  end
endmodule

  arbiter : grants r0=2 r3=2 starvation=3 | fixed priority r0=11 r3=0 starve run=11

With all four requesters asking continuously, round-robin serves each of them and no requester is ever more than one grant behind another. Fixed priority serves requester 0 eleven times and requester 3 not once.

Three properties the testbench establishes:

• Exactly one grant per cycle. The one-hot extraction rot & (~rot + 1) is asserted directly by summing the grant bits.
• The pointer advances on a grant, and only on a grant. With nobody asking, the pointer is asserted to be unchanged across three cycles.
• A lone requester is granted every cycle, and starves nobody — because there is nobody else.

9. RTL 4 — One Channel Must Not Block Another

module vc_independence #(parameter int SHARED_POOL = 0) (
  input  logic clk, rst_n,
  input  logic       send_lo, send_hi,
  input  logic [3:0] cred_lo, cred_hi, cred_shared,
  output logic       go_lo, go_hi,
  output logic       hi_blocked_by_lo_err,
  output logic [7:0] n_lo, n_hi, n_hi_blocked, max_hi_block
);
  logic [7:0] hb_q;
  // Each channel spends its own credits. SHARED_POOL makes both spend one,
  // so a low-priority burst exhausts the pool the high-priority class needs.
  assign go_lo = send_lo && ((SHARED_POOL != 0) ? (cred_shared != 4'd0)
                                                : (cred_lo != 4'd0));
  assign go_hi = send_hi && ((SHARED_POOL != 0) ? (cred_shared != 4'd0)
                                                : (cred_hi != 4'd0));
  // The high channel had credits of its own and was blocked anyway.
  assign hi_blocked_by_lo_err = send_hi && !go_hi && (cred_hi != 4'd0);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      hb_q <= 8'd0; n_lo <= 8'd0; n_hi <= 8'd0;
      n_hi_blocked <= 8'd0; max_hi_block <= 8'd0;
    end else begin
      if (go_lo) n_lo <= n_lo + 8'd1;
      if (go_hi) n_hi <= n_hi + 8'd1;
      if (hi_blocked_by_lo_err) begin
        n_hi_blocked <= n_hi_blocked + 8'd1;
        hb_q <= hb_q + 8'd1;
        if (hb_q + 8'd1 > max_hi_block) max_hi_block <= hb_q + 8'd1;
      end else hb_q <= 8'd0;
    end
  end
endmodule

  vc      : hi sent=9 blocked=0 | shared-pool build hi sent=5 blocked=4 for 3 cycles

This is the failure that is hardest to see in a design review, because the channel structure is completely intact. There are two virtual channels, they have separate identifiers, separate queues and separate arbitration. They share one credit pool, and that single line undoes the entire mechanism.

The monitor's third term is what makes it a defect rather than an observation: cred_hi != 0. The high channel being unable to send because it has no credits of its own is ordinary back pressure, and the testbench drives that case explicitly and asserts the error stays low. The error means: it had credits and was blocked anyway.

10. RTL 5 — Strict Priority, And The Fix For It

module prio_age #(parameter int NO_AGEING = 0) (
  input  logic clk, rst_n,
  input  logic       req_hi, req_lo,
  output logic       grant_hi, grant_lo,
  output logic [7:0] lo_age,
  output logic       promoted, starved_err,
  output logic [7:0] n_hi, n_lo, max_lo_wait
);
  logic [7:0] age_q;
  assign lo_age = age_q;
  localparam logic [7:0] AGE_LIMIT = 8'd4;
  // A low-priority request that has waited past the limit is promoted above
  // the high-priority one. NO_AGEING never promotes.
  assign promoted = (NO_AGEING == 0) && req_lo && (age_q >= AGE_LIMIT);
  assign grant_lo = req_lo && (!req_hi || promoted);
  assign grant_hi = req_hi && !promoted;
  assign starved_err = req_lo && !grant_lo && (age_q >= AGE_LIMIT);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      age_q <= 8'd0; n_hi <= 8'd0; n_lo <= 8'd0; max_lo_wait <= 8'd0;
    end else begin
      if (grant_hi) n_hi <= n_hi + 8'd1;
      if (grant_lo) n_lo <= n_lo + 8'd1;
      // The age resets on a grant, or a promoted request stays promoted.
      if (req_lo && !grant_lo) begin
        age_q <= age_q + 8'd1;
        if (age_q + 8'd1 > max_lo_wait) max_lo_wait <= age_q + 8'd1;
      end else age_q <= 8'd0;
    end
  end
endmodule

  priority: lo served=4 worst wait=4 promoted=1 | no-ageing build lo served=3 worst wait=8 starved=1

A worst wait of 4 against 8 and rising. Ageing does not make the low-priority class fast; it makes its waiting bounded, which is a completely different guarantee and the only one that can be written down.

promoted displaces the high-priority request for exactly one cycle, and the testbench asserts grant_hi goes low in that cycle. A promotion that does not displace anything is not a promotion.

The age resets on a grant, and the testbench checks it stays reset while the request keeps being served — the run it twice class, because an age that never resets promotes forever after the first time.

11. RTL 6 — Two Ports, One Pool

module pool_deadlock #(parameter int NO_FLOOR = 0) (
  input  logic clk, rst_n,
  input  logic       take_a, take_b, release_a, release_b,
  input  logic       need_more_a, need_more_b,
  output logic [3:0] held_a, held_b, pool_free,
  output logic       grant_a, grant_b,
  output logic       deadlock_err,
  output logic [7:0] n_grants, stuck_cycles, max_stuck
);
  logic [3:0] ha_q, hb_q;
  logic [7:0] sk_q;
  assign held_a    = ha_q;
  assign held_b    = hb_q;
  assign pool_free = POOL - ha_q - hb_q;
  localparam logic [3:0] POOL = 4'd8;
  localparam logic [3:0] MAX_PER_PORT = 4'd5;   // leaves 3 for the other
  // A port may not hold more than its share. NO_FLOOR lets either take
  // everything, which is what makes the cycle reachable.
  assign grant_a = take_a && (pool_free != 4'd0)
                   && ((NO_FLOOR != 0) || (ha_q < MAX_PER_PORT));
  // Both ports holding something and both waiting for more, with none left.
  assign deadlock_err = (pool_free == 4'd0)
                        && need_more_a && need_more_b
                        && (ha_q != 4'd0) && (hb_q != 4'd0);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      ha_q <= 4'd0; hb_q <= 4'd0; sk_q <= 8'd0;
      n_grants <= 8'd0; max_stuck <= 8'd0;
    end else begin
      if (grant_a)                        ha_q <= ha_q + 4'd1;
      else if (release_a && ha_q != 4'd0) ha_q <= ha_q - 4'd1;
      if (grant_b)                        hb_q <= hb_q + 4'd1;
      else if (release_b && hb_q != 4'd0) hb_q <= hb_q - 4'd1;
      // Both may be granted in the same cycle.
      n_grants <= n_grants + {7'd0, grant_a} + {7'd0, grant_b};
      if (deadlock_err) begin
        sk_q <= sk_q + 8'd1;
        if (sk_q + 8'd1 > max_stuck) max_stuck <= sk_q + 8'd1;
      end else sk_q <= 8'd0;
    end
  end
endmodule

  pool    : A held=5 B held=3 deadlock=1 stuck=3 cycles | no-floor build A held=7 B held=0

The deadlock condition has four terms and dropping any one changes what it means:

The last two are the interesting ones, and the testbench asserts them against the no-floor build directly: A holds everything, B holds nothing, both want more — and that is not a deadlock. It is a starvation, and the fix is a floor, not a release. Calling it a deadlock sends an engineer looking for a cycle that does not exist.

The floor is also driven at its boundary: a port below its floor still cannot take from an empty pool, because the floor caps what a port may hold and the pool caps what exists.

12. RTL 7 — Bandwidth As A Share, Not A Rate

module bw_share #(parameter int IGNORE_WEIGHT = 0) (
  input  logic clk, rst_n,
  input  logic        tick,
  input  logic [3:0]  weight_a, weight_b,
  input  logic        want_a, want_b,
  output logic        serve_a, serve_b,
  output logic [15:0] served_a, served_b, n_ticks,
  output logic [7:0]  share_a_pct, target_a_pct, deviation_pct
);
  logic [3:0]  acc_q, total_w;
  logic [31:0] w_share, w_target;
  assign total_w  = weight_a + weight_b;
  assign w_share  = {16'd0, served_a} * 32'd100;
  assign w_target = {28'd0, weight_a} * 32'd100;
  assign serve_a = want_a && ((IGNORE_WEIGHT != 0) ? !acc_q[0]
                                                   : (acc_q < weight_a));
  assign serve_b = want_b && !serve_a;
  assign share_a_pct  = (n_ticks == 16'd0) ? 8'd0 : (w_share / {16'd0, n_ticks});
  assign target_a_pct = (total_w == 4'd0) ? 8'd0 : (w_target / {28'd0, total_w});
  // How far the achieved share is from the promised one, either way.
  assign deviation_pct = (share_a_pct > target_a_pct)
                       ? (share_a_pct - target_a_pct)
                       : (target_a_pct - share_a_pct);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      acc_q <= 4'd0; served_a <= 16'd0; served_b <= 16'd0; n_ticks <= 16'd0;
    end else if (tick) begin
      n_ticks <= n_ticks + 16'd1;
      if (serve_a) served_a <= served_a + 16'd1;
      if (serve_b) served_b <= served_b + 16'd1;
      // The accumulator walks the weighted cycle.
      if (acc_q + 4'd1 >= total_w) acc_q <= 4'd0;
      else                         acc_q <= acc_q + 4'd1;
    end
  end
endmodule

  bandwidth: A promised 75% achieved 75% (deviation 0) | ignore-weight build achieved 50%, deviation 25

A weight is a promise and deviation_pct is the only thing that checks it. A weight of 3 against 1 promises 75 percent; the correct arbiter delivers exactly that over forty ticks, and the build that ignores the weight delivers 50 — 25 points away from a number nobody was measuring.

The deviation is computed as an absolute difference in both directions. A signed subtraction on unsigned operands wraps, and an arbiter delivering less than promised would report a deviation near 255 or near zero depending on the wrap — which is why the comparison comes first.

The achieved share is measured against ticks, not against A's own service, which would be 100 percent always.

13. RTL 8 — What Sharing Costs

module share_cost (...);
  assign saved     = (t_shared >= t_ded) ? 16'd0 : (t_ded - t_shared);
  // The blocked and waiting cycles come out of the saving.
  assign net_saved = (saved <= t_lost) ? 16'd0 : (saved - t_lost);
  assign saving_pct = (t_ded == 16'd0) ? 8'd0 : (w_save / {16'd0, t_ded});
  assign loss_pct   = (t_ded == 16'd0) ? 8'd0 : (w_loss / {16'd0, t_ded});
  assign net_pct    = (t_ded == 16'd0) ? 8'd0 : (w_net  / {16'd0, t_ded});
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      t_ded <= 16'd0; t_shared <= 16'd0; t_lost <= 16'd0; t_served <= 16'd0;
    end else if (tick) begin
      t_ded    <= t_ded + {8'd0, dedicated_slots};
      t_shared <= t_shared + {8'd0, shared_slots};
      // Both halves of the loss, or the saving is a headline.
      t_lost   <= t_lost + {8'd0, blocked_cycles} + {8'd0, wait_cycles};
      if (served_cycle) t_served <= t_served + 16'd1;
    end
  end
endmodule

  cost    : gross=37% loss=12% net=25% of the dedicated baseline

37 percent gross, 25 percent net. The twelve points between them are the blocking from section 9 and the waiting from section 10 — the interference that sharing causes, charged against the buffer slots that sharing saves.

Both subtractions saturate, and both are driven to saturation: a shared design that uses more slots than the dedicated one reports a saving of zero, not a large positive number from an underflow.

The loss includes both halves. Charging the blocking and not the waiting, or the reverse, understates the cost by half — and each is a separate mutation.

14. RTL 9 — Auditing The Guarantees

module share_audit (
  input  logic clk, rst_n,
  input  logic       check,
  input  logic [3:0] promised_slots, actual_slots,
  input  logic [7:0] promised_max_wait, actual_max_wait,
  output logic       slots_ok, wait_ok, honoured,
  output logic       broken_guarantee_err,
  output logic [7:0] n_checked, n_broken, worst_shortfall, worst_overrun
);
  logic [3:0] shortfall;
  logic [7:0] overrun;
  assign shortfall = (actual_slots >= promised_slots) ? 4'd0
                   : (promised_slots - actual_slots);
  assign overrun   = (actual_max_wait <= promised_max_wait) ? 8'd0
                   : (actual_max_wait - promised_max_wait);
  // A guarantee is both halves: the slots promised AND the wait bounded.
  assign honoured = slots_ok && wait_ok;
  assign slots_ok = (shortfall == 4'd0);
  assign wait_ok  = (overrun == 8'd0);
  assign broken_guarantee_err = check && !honoured;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_checked <= 8'd0; n_broken <= 8'd0;
      worst_shortfall <= 8'd0; worst_overrun <= 8'd0;
    end else if (check) begin
      n_checked <= n_checked + 8'd1;
      if (!honoured) n_broken <= n_broken + 8'd1;
      if ({4'd0, shortfall} > worst_shortfall)
        worst_shortfall <= {4'd0, shortfall};
      if (overrun > worst_overrun) worst_overrun <= overrun;
    end
  end
endmodule

  audit   : 3 of 5 guarantees broken, worst shortfall=2 slots, worst overrun=5 cycles

A guarantee has two halves and breaking either breaks it. The testbench drives each alone: the slots short with the wait inside its bound, and the wait overrunning with the slots all present. A design that checks only one reports the other as honoured.

Both subtractions saturate, and exceeding the promise is driven explicitly — four slots where two were promised is not a negative shortfall, it is no shortfall.

A reservation nobody audits is a comment. Sections 6 and 10 make promises; this is where they are checked against what was actually delivered.

15. RTL 10 — Resource Sharing Assembled

module share_top #(parameter int SERVE_ANYWAY = 0) (
  input  logic clk, rst_n,
  input  logic       request,
  input  logic       has_credit, channel_free, arb_grant, guarantee_ok,
  output logic       serve,
  output logic [3:0] refused_by,
  output logic       overcommit_err,
  output logic [7:0] n_requests, n_served, n_refused, n_overcommit
);
  assign refused_by = {~guarantee_ok, ~arb_grant, ~channel_free, ~has_credit};
  assign serve = request && ((refused_by == 4'd0) || (SERVE_ANYWAY != 0));
  // Served without a credit: a slot that does not exist has been promised.
  assign overcommit_err = serve && !has_credit;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      n_requests <= 8'd0; n_served <= 8'd0;
      n_refused <= 8'd0; n_overcommit <= 8'd0;
    end else if (request) begin
      n_requests <= n_requests + 8'd1;
      if (serve) n_served  <= n_served + 8'd1;
      else       n_refused <= n_refused + 8'd1;
      if (overcommit_err) n_overcommit <= n_overcommit + 8'd1;
    end
  end
endmodule

  assembled: 1 of 5 served, mask for guarantee=8 | serve-anyway build served 5, overcommitted 1

Five requests: one meeting every resource constraint and four each failing exactly one, with the mask naming which. Each bit position is asserted individually.

overcommit_err is the invariant — nothing is served against a credit that does not exist. It cannot fire in the correct build, and SERVE_ANYWAY is what makes it observable: five served, one of them against a slot the buffer does not have.

16. Quantitative Reasoning

Three worth a sentence.

11 grants against 0. Same four requesters, same continuous demand, one parameter. Fixed priority is not "less fair" — requester 3 is served exactly never.

Worst wait 4 against 8 and rising. Ageing does not make the low class fast. It makes the wait a number you can write down, which fixed priority cannot.

37 percent gross, 25 percent net. The twelve points are the interference in sections 9 and 10, and they are the reason a shared buffer is a decision rather than an optimisation.

17. Assertions

Every property is an immediate check written as cond !== 1'b1, sampled after a settle.

18. Mutation Testing

101 mutations, one at a time, each required to make the baseline print RESULT: FAIL.

101 of 101 were killed.

The first run killed 85 and left 16 survivors:

Two findings worth carrying forward.

A width bug can hide the very thing it is meant to detect. credit_leak_err compared a four-bit sum against the depth, and at a full buffer 8 + 8 wrapped to 0 — so the leak vanished exactly when it was largest. Both the design and the monitor overflowed the same way. Fixed by widening the comparison to five bits, and by sampling the leak at one credit over rather than at saturation.

A monitor can describe normal operation. The first starvation_err was "requester 3 asked and somebody else was served", which fires three cycles in four on a perfectly fair arbiter. Starvation is waiting longer than a full round; a monitor that fires on losing a single arbitration is a description of arbitration, and it would be disabled the first week it shipped.

A representative sample:

19. Verification Strategy

Parameterised twin builds. DOUBLE_RETURN, NO_RESERVE, FIXED_PRIORITY, SHARED_POOL, NO_AGEING, NO_FLOOR, IGNORE_WEIGHT, SERVE_ANYWAY. Every comparison in section 16 is one source under one parameter, driven by one stimulus stream.

Independent oracles. The credit invariant and the promised bandwidth share are each checked against a plain-integer function that shares no structure with the design.

Sample a leak where it is smallest. One credit over the depth, not eight — the boundary a one-off comparison gets wrong, and the place the arithmetic cannot overflow.

Every gating term falsified alone. A high channel with no credits of its own. A port below its floor with the pool empty. A shortfall with the wait inside its bound, and the reverse. One port waiting while the other still holds nothing. Each reads as obviously necessary and none had been driven false.

Break it, then do it again. Every latched interval — starvation, blocking, deadlock, ageing — is broken and re-entered, because a counter that never resets is invisible on a single occurrence.

Underflow guards driven past empty. Free more slots than were taken, release more than was held, evict from nothing.

Assert the value, not the range. Every ratio in sections 12, 13 and 14 is asserted to an exact number, following 16.1's five bound-satisfied survivors.

Delta discipline. Every combinational sample follows a settle, and every latched interval is sampled a cycle after the condition that clears it.

20. Synthesis and Implementation Reality

This is the most hardware-dense chapter in the module.

Credit counters are small and everywhere. One per channel per direction per port, a handful of bits each. The area is negligible; the correctness is not, and the invariant in section 5 is the thing to assert in RTL rather than compute in a testbench.

The buffer is the area. Section 6's eight slots are the dominant term in a switch port, and the reservation-versus-sharing decision is made once at the top of the design. A per-protocol floor costs nothing in gates — it is a comparison — and it is what turns a shared buffer from a gamble into a bounded one.

The round-robin rotation is two barrel shifters. req >> ptr | req << (N - ptr) and the reverse, plus a priority encoder. It is a well-known and cheap structure, and the expensive mistake is the pointer update policy rather than the datapath.

Ageing is a counter per requester and a comparator. It scales with requester count, which bounds how many classes can have a bounded wait. That is the actual constraint on how many priority levels a switch offers.

The pool floor is one comparator per port and it is the difference between a deadlock that is impossible and one that is unlikely. Section 11's four-term condition is diagnostic; the floor is what makes it unreachable.

The weighted arbiter's accumulator is the part that is subtly wrong in real designs. Section 12's acc_q walks the weighted cycle, and the failure mode is not that it is unfair — it is that nobody measures deviation_pct, so a weight that is never achieved is never noticed.

The divisions are firmware. Achieved share, saving, loss and net are computed from raw counters. What matters in hardware is that served_a, n_ticks, t_lost and the guarantee counters exist and mean what sections 12 to 14 say.

21. Silicon Observability

Three to alarm on.

credit_leak_err or overcommit_err non-zero at all means the switch has promised a slot that does not exist. Everything downstream is operating on an accounting error, and the symptom will appear as an overflow somewhere with no obvious cause.

max_wait3 or max_lo_wait exceeding its bound means a guarantee has been broken. The whole point of ageing and round-robin is that these are bounded; an unbounded value means the mechanism is not doing what it was put there for.

deviation_pct growing means an arbiter is drifting from its configured weights. It is the only signal here that checks a promise nobody else measures, and a weight that is never achieved is never noticed without it.

22. Debug Lab

22.1 A buffer overflowed and every counter says it could not have

Symptom. A receive buffer overran. The sender's credit counter never went negative and never reached zero unexpectedly.

The reading. credit_leak_err, and the sum of credits and occupancy against the depth.

The diagnosis. A credit was returned for something that was never spent. The sender's view is perfectly consistent — it had a credit and used it — and the pool has been larger than the buffer for some time.

Where to look. Every path that returns a credit. Section 5's failure returns one on the send as well as the free; in a real design the equivalent is a return path that fires on both a completion and a retry, or on both a drain and a flush.

22.2 One protocol is starved and the buffer is not full

Symptom. CXL.mem transactions are being denied buffer space. Occupancy reports well under the depth.

The reading. n_denied_b and max_starve_b, against peak_occ.

The middle row is section 6's NO_RESERVE, and the tell is that the average occupancy looks comfortable while one protocol is denied continuously.

22.3 A high-priority class is slow and its own credits are unused

Symptom. High-priority traffic is being delayed. Its credit counter shows credits available.

The reading. hi_blocked_by_lo_err.

The diagnosis. The virtual channels share a credit pool. The high channel has credits of its own and cannot use them, because the resource it actually needs was exhausted by the low channel.

Why a design review misses it. The channel structure is intact — separate identifiers, separate queues, separate arbitration. One shared pool undoes all of it, and nothing about the block diagram shows that.

22.4 Two ports stopped and neither will release

Symptom. Two switch ports have stopped making progress. Both hold buffer space. Neither is making a request that can be satisfied.

The reading. deadlock_err, and how much each port holds.

The diagnosis, in two sub-cases that need opposite responses.

If both ports hold something and both want more with the pool empty, it is a genuine deadlock. Nothing in the switch will break it; something has to release, and the structural fix is a per-port floor so the cycle cannot form.

If one port holds everything and the other holds nothing, it is not a deadlock — it is starvation, and deadlock_err correctly does not fire. Looking for a cycle here wastes the investigation; the fix is the same floor, for a different reason.

The four-term condition in section 11 is what separates them, and the difference matters because "deadlock" sends an engineer looking for a dependency cycle that does not exist.

23. Design Review

1. Does credits plus occupancy equal the depth, asserted in RTL? And is the comparison wide enough not to wrap at a full buffer?

2. Is there exactly one path that returns a credit? Every extra one is a potential double-return.

3. Does each protocol have a floor in the shared buffer? Without one, the best case is the whole buffer and the worst case is nothing.

4. Does the round-robin pointer advance on a grant, and only on a grant? Every cycle skips idle requesters; never is fixed priority.

5. Do the virtual channels share a credit pool? If yes, they are one channel with two names.

6. Is there an ageing mechanism, and what bound does it give? Strict priority gives no bound at all, and "eventually" is not a number.

7. Can one port take the entire shared pool? If yes, the deadlock in section 11 is reachable rather than impossible.

8. Is the achieved bandwidth share measured against the configured weight? A weight nobody checks is a comment.

9. Are the guarantees audited in both halves — slots and waiting? Checking one reports the other as honoured.

10. What does sharing cost, net of the blocking and waiting it causes? The gross saving is buffer slots; the net is the number that decides.

24. How This Appears In Real Engineering

The credit invariant is assumed rather than asserted. It is one comparison and it catches the entire class of accounting bug, and it is usually absent because credits "obviously" balance.

Reservations are added after the first starvation incident. Sharing the whole buffer is the default because it maximises the best case, and the worst case is not measured until a protocol stops entirely.

Round-robin is implemented and the pointer policy is not reviewed. The barrel shifters get attention because they are the interesting logic; the two-line pointer update is where fairness actually lives.

Virtual channels are given separate queues and a shared credit pool, because the queues are the visible structure and the pool is an implementation detail. It is the single most effective way to build channels that are not independent while passing every structural review.

Ageing is deferred as a second-order concern until a low-priority class misses a deadline, at which point the fix is a counter per requester and the question is why it was not there.

The weight deviation is never measured. Weights are configured, believed, and never checked against what was delivered — which is why an arbiter drifting from its configuration is normally discovered by a workload rather than by a counter.

25. Common Misconceptions

"Credits obviously balance." Only if exactly one path returns them. Section 5's double-return build is internally consistent from the sender's side and has more credits than the buffer has slots.

"A wider counter is safer." The comparison matters more than the counter. (cr + occ) > DEPTH on four-bit operands is zero at a full buffer, so the leak detector fails exactly where the leak is largest.

"Sharing the buffer is strictly better — the best case is bigger." The worst case is zero. Section 6's no-reserve build served one protocol not once over the whole run.

"Round-robin is fair." Round-robin with a pointer that advances every cycle is not, and one that never advances is fixed priority. The fairness is in the update policy, not the rotation.

"Requester 3 lost that arbitration, so it is being starved." Losing one arbitration among four requesters is arbitration working. Starvation is waiting longer than a full round, and a monitor that cannot tell the difference fires three cycles in four on a correct design.

"They are separate virtual channels, so they are independent." Only if they have separate credits. Sharing the pool makes them one channel with two identifiers, and every structural review passes.

"Strict priority is fine — the low class gets served eventually." "Eventually" is not a bound. Section 10's worst wait was 8 cycles and rising when the run ended.

"Both ports are stuck holding buffer space, so it is a deadlock." Not if one of them holds nothing. That is starvation, it needs a floor rather than a release, and looking for a dependency cycle wastes the investigation.

26. Interview Reasoning

Q1. What is a credit, and what is the invariant? A promise that a buffer slot exists. Credits plus occupancy equals the depth, always — every credit outstanding corresponds to a slot the receiver has and has not filled.

Q2. How is that invariant broken? By returning a credit that was never spent. The sender's own accounting stays consistent, so nothing on that side reports a problem, and the pool grows past the slots that exist.

Q3. Your leak detector failed to fire on a leaking design. Why? It compared a four-bit sum against the depth, and at a full buffer 8 plus 8 wraps to zero. The detector overflowed exactly where the leak was largest. Verilog sizes from the operands, not the destination.

Q4. Where should a resource leak be sampled? Where it is smallest — one unit over the limit. That is the same property, it is the boundary an off-by-one comparison gets wrong anyway, and it has none of the width risk of sampling at saturation.

Q5. Shared buffer or per-protocol reservation? Sharing maximises the best case; reserving bounds the worst. The measurement that decides is the starvation run length, not the average occupancy.

Q6. A protocol is starved and the buffer is not full. Explain. Another protocol holds the slots. Average occupancy looks comfortable because the holder releases and re-takes; the starved protocol is denied continuously, and only the run length shows it.

Q7. Is round-robin fair? Only if the pointer advances on a grant. Advancing every cycle skips requesters that were not asking; never advancing is fixed priority with extra logic.

Q8. What is starvation, precisely? Waiting longer than a full round of requesters. Losing a single arbitration among four is arbitration working, and a monitor that flags it fires three cycles in four on a correct design.

Q9. Fixed priority served requester 0 eleven times and requester 3 zero. Is that a bug? It is the specification working. Whether it is a bug depends on whether requester 3 had a guarantee — and if it did, fixed priority cannot keep it.

Q10. What do virtual channels actually guarantee? That one class blocking does not block another. Separate queues do not give that; separate credits do.

Q11. How would you detect virtual channels that are not independent? A monitor that fires when a channel has credits of its own and is blocked anyway. The third term is what distinguishes it from ordinary back pressure.

Q12. Why does that failure survive a design review? Because the structure is intact. Separate identifiers, separate queues, separate arbitration — and one shared credit pool that undoes all of it, which the block diagram does not show.

Q13. What does ageing buy? A bound. It does not make the low-priority class fast; it makes its waiting a number that can be written into a specification, which strict priority cannot provide at all.

Q14. What must a promotion actually do? Displace the higher-priority request for that cycle. A promotion that does not displace anything has not promoted anything.

Q15. Why must the age reset on a grant? Because an age that never resets promotes forever after the first time, and the class it was protecting becomes the highest priority permanently.

Q16. Two ports hold buffer space and both want more. Deadlock? Only if both hold something. If one holds everything and the other holds nothing there is no cycle — it is starvation, and looking for a dependency wastes the investigation.

Q17. How do you make the deadlock impossible rather than unlikely? A per-port floor: no port may hold more than a share that leaves enough for the others. It is one comparator, and it removes the state in which the cycle can form.

Q18. A port is below its floor and the pool is empty. May it take? No. The floor caps what a port may hold; the pool caps what exists. Both must be satisfied, and the testbench drives exactly that state.

Q19. What checks a bandwidth weight? The deviation between the achieved share and the promised one. Without it, a weight is configured, believed, and never verified against delivery.

Q20. Why compute the deviation as an absolute difference? Because a signed subtraction on unsigned operands wraps. An arbiter delivering less than promised would report a deviation near 255 or near zero depending on the wrap, and neither is the number.

Q21. Achieved share measured against what? Ticks observed. Against the requester's own service it reads 100 percent always, which is a metric that can never say anything.

Q22. What does sharing cost? Buffer slots saved, minus the blocking and waiting it causes. 37 percent gross, 12 percent lost, 25 percent net — and the net is the number that decides.

Q23. Why must the loss include both blocking and waiting? They are separate costs from separate mechanisms — section 9's channel interference and section 10's arbitration wait. Charging one understates the total by roughly half.

Q24. What are the two halves of a guarantee? The resource promised and the waiting bounded. Breaking either breaks the guarantee, and a design that checks one reports the other as honoured.

Q25. Four slots delivered where two were promised. What is the shortfall? Zero. Exceeding the promise is not a negative shortfall, and an unsaturated subtraction would report a very large one.

Q26. What is the assembled switch's sharing invariant? Nothing is served against a credit that does not exist. It cannot fire in a correct build, which is what makes it an invariant.

Q27. Why does every latched interval in this chapter get broken and re-entered in the testbench? Because a counter that never resets is invisible on a single occurrence. Three of the sixteen first-run survivors lived entirely in that gap.

Q28. Which is the more dangerous failure here — the credit leak or the shared virtual-channel pool? The shared pool, because it passes every structural review and produces a performance symptom rather than an error. The credit leak eventually overflows something and gets investigated.

Q29. If you could assert one property in RTL, which? Credits plus occupancy equals the depth. One comparison, and it catches the entire class of accounting bug at the point it happens rather than at the overflow it eventually causes.

Q30. And if you could expose one counter? deviation_pct. Every other counter here reports something going wrong; that one checks a promise that nothing else in the system verifies at all.

27. Exercises

1. Add a second credit return path to credit_flow — one on delivery and one on drain — and show that the invariant catches the double return regardless of which path fires twice.

2. Make buffer_share reserve for three protocols rather than two. Derive the maximum any one may hold, and show what happens to the best case as the reservation count grows.

3. Change arb_fair's pointer to advance to the requester after the one served, rather than to the one served plus one. Show whether fairness changes and explain why.

4. Give vc_independence a shared pool plus per-channel minimums. Find the smallest minimum that makes hi_blocked_by_lo_err unreachable.

5. Make prio_age's limit configurable per requester. Show the trade between the bound each class gets and the throughput the high-priority class loses.

6. Extend pool_deadlock to three ports. Derive the per-port floor that makes a three-way cycle impossible, and confirm it against the model.

7. Add a second weighted requester to bw_share and show that deviation_pct for one of them can be zero while the other's is large. Say what that means for how the metric should be exposed.

8. Take the 101-mutation suite and remove every "break it and re-enter it" sequence from the testbench. Confirm the three interval-reset mutations return, and find every other latched counter in Module 16 tested only once.

28. Summary

Every flit served needs a credit that exists, on a channel nothing else can exhaust, from an arbiter that bounds its waiting, out of a pool no single port can take entirely.

• A credit that exists: credits plus occupancy equals the depth, and the double-return build broke it at the first send — caught at one credit over, because at eight over the four-bit comparison wrapped to zero and the leak vanished.
• A channel nothing else can exhaust: 9 high-priority sends against 5, with the shared-pool build blocking a channel that had credits of its own for three consecutive cycles.
• An arbiter that bounds waiting: 2 grants each against 11 and 0, and a low-priority worst wait of 4 against 8 and rising.
• A pool no port can take entirely: A held 5 of 8 against 8 of 8, and the no-floor build's "deadlock" turned out to be a starvation — one port holding everything, the other holding nothing, and no cycle at all.

And the cost, measured: 37 percent gross saving, 12 percent lost to blocking and waiting, 25 percent net.

101 mutations, 101 killed. The one worth remembering: a leak detector that overflowed the same way the leak did, so it reported nothing exactly where the leak was largest. Sample a resource leak where it is smallest, not where it is worst.

16.4 asks what stops a switch growing, and what the second switch costs.