Skip to content
VLSI Mentor

Ethernet · Module 8

Latency Decomposition

Five terms with five owners: four are constants computable before a frame is sent, and queueing is the one that depends on load rather than speed — the only term that diverges, and the reason a mean and a tail are different questions.

Chapter 8.1 gave serialization and showed its share of a budget collapsing across four orders of magnitude. Chapter 8.2 gave propagation and showed it never moving at all. Chapter 8.3 gave the capacity side.

This chapter puts them in one budget, adds the term none of them covered, and argues that the sum is the least useful thing about it.

A latency measurement of 200 microseconds is a number. It is not actionable, because it says nothing about which of five very different mechanisms produced it — and the five have different owners, different remedies, and wildly different behaviour under load.

Four of them are boring in the useful sense. Serialization is bits ÷ rate. Propagation is length × 5 ns/m. Store-and-forward buffering is one serialization time per hop. Per-frame processing is a pipeline depth. All four are constants for a given frame on a given path, computable in advance, and unaffected by how busy anything is.

The fifth is queueing, and it is different in kind. It depends on load, not on speed. It is the only term that grows without bound as a link approaches saturation — and it is why a mean latency and a 99.9th-percentile latency are two different questions with two different answers.

1. Scope — What This Chapter Owns

This chapter owns the decomposition: the five terms, which stage owns each, how they are measured separately, why queueing behaves differently from the other four, and why a tail statistic is a different question from a mean.

Chapter 8.1 owns serialization and Chapter 8.2 owns propagation; both are used here without re-derivation. Chapter 8.3 owns capacity, and supplies the utilisation figure that drives the queueing term.

Chapter 7.1 and Chapter 7.2 own the datapaths whose depths become the processing term, and Chapter 7.2 §4's store-and-forward policy becomes a term of its own.

It does not own queue management. How a queue is scheduled, when it drops, and how congestion is signalled are a switching and flow-control subject. This chapter measures the delay a queue produces; it does not decide the queue's behaviour.

The question this chapter answers that its neighbours do not: given a latency number, which of five mechanisms produced it — and which statistic should have been measured in the first place?

2. Five Terms, Five Owners

Five latency terms with their owners. Serialization is set by the frame size and the line rate and is owned by the frame format. Propagation is set by the cable length and is owned by the building. Store and forward buffering is one serialization time per hop and is owned by the switch's forwarding policy. Per frame processing is set by pipeline depth and is owned by this design. Queueing is set by utilisation and is owned by whoever offered the load, and it is the only one of the five that is unbounded.Serializationframe size ÷ ratePropagationlength × 5 ns/mStore-and-forwardone frame time per hopPer-frame processingpipeline depthQueueingdepends on utilisationFour are constantscomputable in advanceOne is unboundedand owned by nobody local12
Figure 1 — the five terms and the stage that owns each; only the last depends on how busy the link is.

Take the four constants first, because their being constant is the point.

Serialization is frame bits ÷ line rate. For a maximum-size frame at 1 Gb/s it is 12.14 µs; at 100 Gb/s it is 121.4 ns. It does not depend on load at all — a frame takes the same time to clock out on an idle link and a saturated one.

Propagation is length × 5 ns/m, about 505 ns for a 100-metre link, and Chapter 8.2 showed it does not depend on the rate either.

Store-and-forward buffering is one serialization time per hop — Chapter 8.1 §8 — so five hops at 1 Gb/s cost 60.7 µs for maximum-size frames. It depends on the frame size and the hop count, and on neither the load nor the rate beyond serialization.

Per-frame processing is the design's own pipeline: address lookup, filter decision, descriptor handling. Tens to hundreds of nanoseconds, fixed by the RTL.

All four can be computed before a single frame is sent. Given a frame size, a path and a set of devices, the answer is arithmetic.

Queueing cannot. It is the time a frame spends waiting for the ones in front of it, and that depends on how many are in front of it, which depends on the offered load. It is the only term whose value is a property of the traffic rather than of the path.

3. Queueing Is the Only Term That Diverges

Latency terms plotted against link utilisation. Serialization, propagation, store and forward buffering and per frame processing are all flat: they do not change as the link gets busier. Queueing rises slowly at low utilisation, noticeably above eighty percent, and without bound as utilisation approaches one hundred percent. So a budget that is dominated by the constant terms at fifty percent utilisation is dominated entirely by queueing at ninety nine percent.50% utilisationqueueing = 1 service time80%4 service times95%19 service times99%99 service timesThe other fourunchanged at every pointBudget inverts againnot by rate this time, byload12
Figure 2 — four terms flat against utilisation and one that rises without bound as the link approaches saturation.

The shape comes from one relation. For a simple queue, the mean waiting time is the service time multiplied by ρ / (1 − ρ), where ρ is the utilisation.

That expression is well behaved until it is not. At half utilisation it is 1; at 90% it is 9; at 99% it is 99. The denominator is what does it — as ρ approaches 1, the delay approaches infinity while the link's rate has not changed at all.

Illustrative, at 1 Gb/s with maximum-size frames — a 12.30 µs wire slot:

UtilisationMultiplierMean queueingSerializationPropagation (100 m)
50%1.012.3 µs12.14 µs0.505 µs
80%4.049.2 µs12.14 µs0.505 µs
90%9.0110.7 µs12.14 µs0.505 µs
95%19.0233.8 µs12.14 µs0.505 µs
99%99.01218 µs12.14 µs0.505 µs

Read across the rows. The last two columns never move. The first column moves by a factor of a hundred, and by the last row it is a hundred times the serialization term and two thousand times the propagation.

Which is Chapter 8.1 §3's inversion happening for a second time and for a completely different reason. There, the budget's composition inverted because the rate changed. Here it inverts because the load changed — and unlike the rate, the load changes minute to minute.

4. RTL 1 — Measuring Each Stage Separately

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SYNTHESIZABLE INSTRUMENTATION.
//
// Timestamps a frame at each stage boundary and reports the intervals
// between them, so that each latency term is measured rather than
// inferred.
//
// The boundaries are chosen so that each interval contains EXACTLY ONE
// mechanism:
//
//   arrival -> dequeue    : queueing. Nothing else happens here.
//   dequeue -> proc_done  : per-frame processing. Lookup, filter,
//                           descriptor -- the pipeline.
//   proc_done -> tx_start : store-and-forward wait, if any.
//   tx_start -> tx_last   : serialization. Chapter 8.1's term, measured
//                           rather than computed, so the two can be
//                           cross-checked.
//
// Propagation is NOT measurable here: it happens outside the device.
// Chapter 8.2's estimator supplies it, and this module leaves a port for
// it rather than pretending to observe it.
package latency_pkg;
 
  typedef enum logic [2:0] {
    LT_QUEUEING,
    LT_PROCESSING,
    LT_STORE_FORWARD,
    LT_SERIALIZATION,
    LT_PROPAGATION,     // supplied, not measured
    LT_NUM
  } lat_term_e;
 
  localparam int unsigned TERMS = 5;
 
endpackage
 
module stage_latency_accumulator
  import latency_pkg::*;
#(
  parameter int unsigned TS_W  = 40,     // picoseconds
  parameter int unsigned CNT_W = 48
) (
  input  logic clk,
  input  logic rst_n,
  input  logic clear,
 
  // Stage-boundary events for one frame, with their timestamps.
  input  logic          ev_arrival,
  input  logic          ev_dequeue,
  input  logic          ev_proc_done,
  input  logic          ev_tx_start,
  input  logic          ev_tx_last,
  input  logic [TS_W-1:0] timestamp_ps,
 
  // Supplied from outside the device (Chapter 8.2).
  input  logic [TS_W-1:0] propagation_ps,
 
  output logic            frame_valid,
  output logic [TS_W-1:0] term_ps [TERMS],
  output logic [TS_W-1:0] total_ps,
 
  output logic [CNT_W-1:0] sum_ps  [TERMS],
  output logic [CNT_W-1:0] frames,
  output logic [TS_W-1:0]  worst_ps [TERMS]
);
 
  logic [TS_W-1:0] t_arr_q, t_deq_q, t_prc_q, t_txs_q;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      t_arr_q <= '0; t_deq_q <= '0; t_prc_q <= '0; t_txs_q <= '0;
      frame_valid <= 1'b0; total_ps <= '0; frames <= '0;
      for (int i = 0; i < TERMS; i++) begin
        term_ps[i] <= '0; sum_ps[i] <= '0; worst_ps[i] <= '0;
      end
    end else begin
      frame_valid <= 1'b0;
 
      if (clear) begin
        frames <= '0;
        for (int i = 0; i < TERMS; i++) sum_ps[i] <= '0;
        // worst_ps deliberately survives: the worst each term has ever
        // been is a design input, not a measurement window.
      end
 
      if (ev_arrival)   t_arr_q <= timestamp_ps;
      if (ev_dequeue)   t_deq_q <= timestamp_ps;
      if (ev_proc_done) t_prc_q <= timestamp_ps;
      if (ev_tx_start)  t_txs_q <= timestamp_ps;
 
      if (ev_tx_last) begin
        automatic logic [TS_W-1:0] q  = t_deq_q - t_arr_q;
        automatic logic [TS_W-1:0] pr = t_prc_q - t_deq_q;
        automatic logic [TS_W-1:0] sf = t_txs_q - t_prc_q;
        automatic logic [TS_W-1:0] sz = timestamp_ps - t_txs_q;
 
        term_ps[LT_QUEUEING]      <= q;
        term_ps[LT_PROCESSING]    <= pr;
        term_ps[LT_STORE_FORWARD] <= sf;
        term_ps[LT_SERIALIZATION] <= sz;
        term_ps[LT_PROPAGATION]   <= propagation_ps;
 
        total_ps    <= q + pr + sf + sz + propagation_ps;
        frame_valid <= 1'b1;
        frames      <= frames + 1'b1;
 
        sum_ps[LT_QUEUEING]      <= sum_ps[LT_QUEUEING]      + CNT_W'(q);
        sum_ps[LT_PROCESSING]    <= sum_ps[LT_PROCESSING]    + CNT_W'(pr);
        sum_ps[LT_STORE_FORWARD] <= sum_ps[LT_STORE_FORWARD] + CNT_W'(sf);
        sum_ps[LT_SERIALIZATION] <= sum_ps[LT_SERIALIZATION] + CNT_W'(sz);
        sum_ps[LT_PROPAGATION]   <= sum_ps[LT_PROPAGATION]   + CNT_W'(propagation_ps);
 
        if (q  > worst_ps[LT_QUEUEING])      worst_ps[LT_QUEUEING]      <= q;
        if (pr > worst_ps[LT_PROCESSING])    worst_ps[LT_PROCESSING]    <= pr;
        if (sf > worst_ps[LT_STORE_FORWARD]) worst_ps[LT_STORE_FORWARD] <= sf;
        if (sz > worst_ps[LT_SERIALIZATION]) worst_ps[LT_SERIALIZATION] <= sz;
      end
    end
  end
 
endmodule

Classification: synthesizable instrumentation.

What it teaches: that propagation gets a port and not a measurement, and the honesty of that is the module's most important property. It happens outside the device; nothing here can observe it. A design that folded it into one of the measured intervals would be attributing an external delay to an internal stage — and the stage would then appear slow in a way no amount of RTL work could fix.

Deliberately simplified: one frame in flight. A pipelined design needs a tag travelling with each frame so its four timestamps can be matched, which is more bookkeeping and exactly the same decomposition.

Production implication: the serialization interval is measured even though Chapter 8.1 can compute it exactly. The cross-check is the point: a measured serialization that disagrees with frame_octets × 8 × bit_period means the timestamps are not where the module thinks they are — a boundary event firing a cycle early, a clock-domain crossing, a tx_last that is really tx_done. The one term whose correct value is known independently is the one that validates the whole instrument.

5. The Mean and the Tail Are Different Questions

A latency requirement is almost never about the average, and a latency measurement almost always is.

The mean is the natural thing to compute — a sum and a division, one accumulator, no storage. It is also the statistic queueing is best at hiding behind.

Illustrative, with a deliberately bimodal distribution that a busy link produces routinely — most frames sail through an empty queue and a few arrive during a burst:

Fraction of framesLatency
queue empty99%5 µs
queue busy1%5 ms

The mean is 0.99 × 5 µs + 0.01 × 5000 µs = 54.95 µs. A requirement of "mean latency under 100 µs" passes comfortably.

The 99th percentile is 5 ms. A requirement of "99th-percentile latency under 100 µs" fails by a factor of fifty.

Same traffic, same measurement window, same device — and the two statistics disagree by two orders of magnitude.

Which is not a pathological case. It is what queueing does: the four constant terms produce a narrow distribution, and queueing adds a long right tail whose weight is small and whose extent is large. A mean averages the tail away by construction; that is what a mean is for.

So the statistic has to match the requirement, and the requirements that matter in practice are almost always tail statistics — because a system's behaviour is usually decided by its worst frames rather than its typical ones.

6. RTL 2 — Estimating the Tail Without Storing It

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SYNTHESIZABLE INSTRUMENTATION.
//
// Estimates latency percentiles from a fixed amount of state.
//
// The method is a histogram with LOGARITHMIC bucket edges, and the edges
// are the whole design:
//
//   uniform edges over 0..1 ms with 16 buckets gives 62.5 us resolution,
//     which puts fifteen buckets in a range the distribution never
//     visits and lumps everything interesting into the first one.
//
//   logarithmic edges give constant RELATIVE resolution, which is what a
//     percentile question actually wants: "is the tail at 90 us or at
//     900 us" matters, "is it at 90 or 95" usually does not.
//
// The estimate is accurate to within one bucket width, at any sample
// count, from sixteen counters -- and no exact method beats that trade
// for the decision the number feeds.
module latency_percentile_estimator
  import latency_pkg::*;
#(
  parameter int unsigned NBUCKET = 16,
  parameter int unsigned CNT_W   = 40,
  // Lowest bucket's upper edge, in picoseconds. Each subsequent bucket
  // doubles, so 16 buckets span 2^15 = 32768x -- from 1 us to 32 ms.
  parameter int unsigned BASE_PS = 1_000_000
) (
  input  logic clk,
  input  logic rst_n,
  input  logic clear,
 
  input  logic            sample_valid,
  input  logic [39:0]     latency_ps,
 
  // Which percentile to report, in tenths of a percent: 999 is the
  // 99.9th. A parameter would fix it at build time; an input lets one
  // instrument answer several questions.
  input  logic [9:0]      percentile_tenths,
 
  output logic [CNT_W-1:0] bucket [NBUCKET],
  output logic [CNT_W-1:0] samples,
 
  output logic             estimate_valid,
  // The bucket's upper edge: the estimate is "at most this".
  output logic [39:0]      percentile_upper_ps,
  output logic [3:0]       percentile_bucket,
  // True when the requested percentile falls in the top bucket, which
  // means the range was too small and the answer is a lower bound only.
  output logic             saturated
);
 
  // Logarithmic bucket selection: which power of two the sample exceeds.
  function automatic int unsigned bucket_of(input logic [39:0] v);
    bucket_of = 0;
    for (int i = 0; i < NBUCKET; i++)
      if (v >= (40'(BASE_PS) << i)) bucket_of = i + 1;
    if (bucket_of > NBUCKET-1) bucket_of = NBUCKET-1;
  endfunction
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      for (int i = 0; i < NBUCKET; i++) bucket[i] <= '0;
      samples <= '0; estimate_valid <= 1'b0;
      percentile_upper_ps <= '0; percentile_bucket <= '0; saturated <= 1'b0;
    end else begin
      estimate_valid <= 1'b0;
 
      if (clear) begin
        for (int i = 0; i < NBUCKET; i++) bucket[i] <= '0;
        samples <= '0;
      end else if (sample_valid) begin
        automatic int unsigned b = bucket_of(latency_ps);
        bucket[b] <= bucket[b] + 1'b1;
        samples   <= samples + 1'b1;
 
        // Walk the buckets accumulating counts until the requested
        // fraction is passed. Sixteen iterations, once per sample --
        // affordable, and it keeps the answer always current rather than
        // available only at a window boundary.
        begin
          automatic logic [CNT_W-1:0] acc = '0;
          automatic logic [CNT_W-1:0] target =
              ((samples + 1'b1) * CNT_W'(percentile_tenths)) / CNT_W'(1000);
          automatic int unsigned found = NBUCKET-1;
          for (int i = 0; i < NBUCKET; i++) begin
            acc = acc + ((i == b) ? (bucket[i] + 1'b1) : bucket[i]);
            if ((acc >= target) && (found == NBUCKET-1) && (i < NBUCKET-1))
              found = i;
          end
          percentile_bucket   <= 4'(found);
          percentile_upper_ps <= (40'(BASE_PS) << found);
          saturated           <= (found == NBUCKET-1);
          estimate_valid      <= 1'b1;
        end
      end
    end
  end
 
endmodule

Classification: synthesizable instrumentation.

What it teaches: that saturated is what keeps the estimate honest. A percentile that falls in the top bucket has no upper edge — the bucket is open-ended — so the reported figure is a lower bound rather than an estimate. A design that reports the top bucket's nominal edge as an answer is reporting a number it does not have, and the failure is silent because the number looks like every other answer.

Deliberately simplified: the percentile walk runs on every sample. A design at high frame rates would run it at a window boundary instead, which trades currency for cycles and does not change the estimate.

Production implication: the bucket edges double, which gives constant relative resolution — about 100% — and that is deliberately coarse. A percentile question is a decision question: "is the tail at 90 µs or 900 µs" changes what somebody does, and "is it 87 or 93" does not. Spending state on resolution the decision cannot use is the classic instrumentation mistake, and the sixteen counters here span 1 µs to 32 ms, which covers every latency this module will ever see.

7. RTL 3 — Separating Queueing From Everything Else

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SYNTHESIZABLE INSTRUMENTATION.
//
// Isolates the queueing term and relates it to the utilisation that
// produced it, so the relationship of Section 3 becomes observable
// rather than assumed.
//
// Two things are reported and they answer different questions:
//
//   the queueing delay itself     -- what this frame experienced
//   the delay against utilisation -- whether the queue is behaving as a
//        queue should, or whether something else is wrong
//
// The second matters because a queueing delay far above what the
// utilisation predicts is NOT a load problem. It is a scheduling
// problem, a head-of-line block, or a drain that has stalled -- and the
// prediction is the only way to tell those from ordinary busyness.
module queueing_delay_isolator
  import latency_pkg::*;
#(
  parameter int unsigned CNT_W = 40,
  // Service time in picoseconds at this link's rate for a mean-size
  // frame. Supplied rather than computed: Chapter 8.1 owns it.
  parameter int unsigned SERVICE_PS = 12_304_000
) (
  input  logic clk,
  input  logic rst_n,
  input  logic clear,
 
  input  logic             sample_valid,
  input  logic [39:0]      queueing_ps,
  // Utilisation in per mille, from Chapter 8.3's accounting.
  input  logic [CNT_W-1:0] utilisation_permille,
  input  logic [CNT_W-1:0] occupancy_frames,
 
  output logic             result_valid,
  // rho/(1-rho) multiplied by the service time. The delay a simple queue
  // at this utilisation would produce.
  output logic [CNT_W-1:0] predicted_ps,
  output logic [CNT_W-1:0] observed_ps,
 
  // Observed far above predicted: not a load problem.
  output logic             excess_beyond_model,
  output logic [CNT_W-1:0] c_excess,
  output logic [CNT_W-1:0] worst_ratio_permille,
 
  // The utilisation at which the queueing term first exceeded the sum of
  // the four constant terms. The single most useful operating figure in
  // this chapter: above it, the link's own load dominates its latency.
  input  logic [CNT_W-1:0] constant_terms_ps,
  output logic [CNT_W-1:0] crossover_utilisation_permille,
  output logic             crossover_seen
);
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      result_valid <= 1'b0; predicted_ps <= '0; observed_ps <= '0;
      excess_beyond_model <= 1'b0; c_excess <= '0;
      worst_ratio_permille <= '0;
      crossover_utilisation_permille <= '0; crossover_seen <= 1'b0;
    end else begin
      result_valid        <= 1'b0;
      excess_beyond_model <= 1'b0;
 
      if (clear) begin
        c_excess <= '0;
        // The worst ratio and the crossover deliberately survive: both
        // characterise the link rather than the measurement window.
      end else if (sample_valid) begin
        // rho/(1-rho) in per mille, guarded against rho = 1 which the
        // model sends to infinity and the arithmetic sends to a divide
        // by zero. A saturated link reports the cap, honestly.
        automatic logic [CNT_W-1:0] denom =
            (utilisation_permille >= CNT_W'(1000)) ? CNT_W'(1)
                                                   : (CNT_W'(1000) - utilisation_permille);
        automatic logic [CNT_W-1:0] mult = (utilisation_permille * 1000) / denom;
        automatic logic [CNT_W-1:0] pred = (CNT_W'(SERVICE_PS) / 1000) * mult;
 
        predicted_ps <= pred;
        observed_ps  <= CNT_W'(queueing_ps);
        result_valid <= 1'b1;
 
        // Four times the prediction is well outside anything a simple
        // queue produces, so it points at a mechanism rather than at
        // load.
        if (CNT_W'(queueing_ps) > (pred * 4)) begin
          excess_beyond_model <= 1'b1;
          c_excess            <= c_excess + 1'b1;
        end
 
        if (pred != '0) begin
          automatic logic [CNT_W-1:0] r = (CNT_W'(queueing_ps) * 1000) / pred;
          if (r > worst_ratio_permille) worst_ratio_permille <= r;
        end
 
        // The operating point at which load starts to dominate.
        if (!crossover_seen && (CNT_W'(queueing_ps) > constant_terms_ps)) begin
          crossover_utilisation_permille <= utilisation_permille;
          crossover_seen                 <= 1'b1;
        end
      end
    end
  end
 
endmodule

Classification: synthesizable instrumentation.

What it teaches: that the prediction is what turns a delay into a diagnosis. A queueing delay of 200 µs is ordinary at 95% utilisation and alarming at 20% — the same number, two entirely different findings — and only a comparison against what the utilisation predicts separates them. excess_beyond_model fires when the queue is behaving unlike a queue, which points at scheduling, head-of-line blocking or a stalled drain rather than at load.

Deliberately simplified: a single-server model with a fixed service time. Real queues have several service classes and variable frame sizes, and the prediction is correspondingly rough — which is fine, because it is used as a factor-of-four threshold rather than as a value.

Production implication: crossover_utilisation_permille is the most useful operating figure this chapter produces. It is the utilisation at which queueing overtakes the four constant terms combined — below it, latency is a property of the path; above it, a property of the load. A link operated below its crossover has predictable latency and one above it does not, and knowing the number turns a capacity decision into arithmetic instead of a rule of thumb.

8. RTL 4 — Reporting Which Term Owns the Latency

Three frames each with two hundred microseconds of total latency but completely different compositions. In the first, queueing dominates, so the finding belongs to whoever offered the load and the fix is capacity or scheduling. In the second, store and forward buffering dominates, so the finding belongs to the switching policy and the fix is cut through. In the third, per frame processing dominates, so the finding belongs to this design and the fix is in the pipeline. The total is identical in all three and says nothing about which.200 µs totalthe same number threetimesQueueing dominantthe load's ownerStore-and-forwarddominantthe switching policyProcessing dominantthis design's pipelineFix: capacityor schedulingFix: cut-througha policy changeFix: the RTLpipeline depth12
Figure 3 — the same total latency, three different compositions, three different owners and three different fixes.
Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SYNTHESIZABLE.
//
// Reports which term dominates and, separately, whether the dominance is
// large enough to be a lever.
//
// The output is deliberately a pair, because "which term is largest" is
// not by itself actionable:
//
//   dominant at 700 per mille -> a lever. Halving it halves the total.
//   dominant at 260 per mille -> a report. Four terms of roughly equal
//        size have a largest one and no fix worth making.
//
// And the OWNER is reported alongside, because that is the field a
// finding gets routed on. A latency problem owned by "the load" goes to
// a different person from one owned by "the pipeline".
module latency_attribution_reporter
  import latency_pkg::*;
#(
  parameter int unsigned CNT_W = 40
) (
  input  logic clk,
  input  logic rst_n,
  input  logic clear,
 
  input  logic             frame_valid,
  input  logic [39:0]      term_ps [TERMS],
  input  logic [39:0]      total_ps,
 
  output logic             report_valid,
  output lat_term_e        dominant_term,
  output logic [CNT_W-1:0] dominant_permille,
  output logic             dominant_is_lever,
 
  // How often each term was the dominant one. A distribution over
  // owners, which is more useful than any single frame's answer.
  output logic [CNT_W-1:0] dominance_count [TERMS],
  output lat_term_e        most_often_dominant,
  output logic [CNT_W-1:0] frames_reported
);
 
  localparam int unsigned LEVER_PERMILLE = 500;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      report_valid <= 1'b0; dominant_term <= LT_QUEUEING;
      dominant_permille <= '0; dominant_is_lever <= 1'b0;
      most_often_dominant <= LT_QUEUEING; frames_reported <= '0;
      for (int i = 0; i < TERMS; i++) dominance_count[i] <= '0;
    end else begin
      report_valid <= 1'b0;
 
      if (clear) begin
        frames_reported <= '0;
        for (int i = 0; i < TERMS; i++) dominance_count[i] <= '0;
      end else if (frame_valid) begin
        automatic logic [39:0] mx = '0;
        automatic int unsigned ix = 0;
        for (int i = 0; i < TERMS; i++)
          if (term_ps[i] > mx) begin mx = term_ps[i]; ix = i; end
 
        dominant_term     <= lat_term_e'(ix);
        dominant_permille <= (total_ps == '0) ? '0
                             : ((CNT_W'(mx) * 1000) / CNT_W'(total_ps));
        // A term is a lever only if halving it would visibly move the
        // total. Below half the budget it is a report.
        dominant_is_lever <= (total_ps != '0) &&
                             (((CNT_W'(mx) * 1000) / CNT_W'(total_ps))
                              >= CNT_W'(LEVER_PERMILLE));
        dominance_count[ix] <= dominance_count[ix] + 1'b1;
        frames_reported     <= frames_reported + 1'b1;
        report_valid        <= 1'b1;
 
        begin
          automatic logic [CNT_W-1:0] best = '0;
          automatic int unsigned bi = 0;
          for (int i = 0; i < TERMS; i++) begin
            automatic logic [CNT_W-1:0] cnt =
                (i == ix) ? (dominance_count[i] + 1'b1) : dominance_count[i];
            if (cnt > best) begin best = cnt; bi = i; end
          end
          most_often_dominant <= lat_term_e'(bi);
        end
      end
    end
  end
 
endmodule

Classification: synthesizable.

What it teaches: that most_often_dominant is a better answer than any single frame's dominant_term. Queueing is bursty by nature, so on a busy link individual frames alternate between "queueing dominated" and "everything else dominated" — and a snapshot picks up whichever the last frame happened to be. The distribution over frames is stable where the instantaneous answer is not.

Deliberately simplified: a fixed 500-per-mille lever threshold. A design with tighter budgets would set it lower, because a 30% term can be worth attacking when the margin is small.

Production implication: dominant_is_lever prevents the most common misuse of an attribution report. A dominant term at 260 per mille is not a finding — four terms of roughly equal size always have a largest one, and optimising it can improve the total by at most 26% even if it is eliminated entirely. Reporting the identity without the fraction produces engineers chasing terms that cannot pay, which is Chapter 8.1 §6's argument arriving with more terms.

9. RTL 5 — Conformance Against a Budget That Has Terms

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// SYNTHESIZABLE MONITOR.
//
// Checks a latency budget stated as SEVERAL constraints rather than one,
// because a real requirement usually is:
//
//   a per-term ceiling  -- "processing must not exceed 2 us"
//   a total ceiling     -- "end to end under 100 us"
//   a TAIL ceiling      -- "99.9th percentile under 250 us"
//
// The third is the one a mean cannot verify (Section 5) and the one this
// chapter's rejected property tries to replace with a mean.
//
// And the module reports which constraint was violated, because the
// three point at different owners: a per-term violation is a design
// fault, a total violation may be a path problem, and a tail violation
// is almost always load.
module latency_budget_conformance
  import latency_pkg::*;
#(
  parameter int unsigned CNT_W = 40
) (
  input  logic clk,
  input  logic rst_n,
  input  logic clear,
 
  input  logic        frame_valid,
  input  logic [39:0] term_ps [TERMS],
  input  logic [39:0] total_ps,
 
  input  logic        percentile_valid,
  input  logic [39:0] percentile_ps,
 
  // The three kinds of ceiling, independently configured.
  input  logic [39:0] term_limit_ps [TERMS],
  input  logic [39:0] total_limit_ps,
  input  logic [39:0] tail_limit_ps,
 
  output logic             term_violation,
  output lat_term_e        violating_term,
  output logic             total_violation,
  output logic             tail_violation,
 
  output logic [CNT_W-1:0] c_term_violations,
  output logic [CNT_W-1:0] c_total_violations,
  output logic [CNT_W-1:0] c_tail_violations,
 
  // Which constraint is TIGHTEST relative to what is observed. Reported
  // whether or not anything is violated, because it says where the
  // margin is thinnest before it runs out.
  output logic [1:0]       tightest_constraint,
  output logic [CNT_W-1:0] tightest_margin_permille
);
 
  localparam logic [1:0] TC_TERM  = 2'd0;
  localparam logic [1:0] TC_TOTAL = 2'd1;
  localparam logic [1:0] TC_TAIL  = 2'd2;
 
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      term_violation <= 1'b0; violating_term <= LT_QUEUEING;
      total_violation <= 1'b0; tail_violation <= 1'b0;
      c_term_violations <= '0; c_total_violations <= '0; c_tail_violations <= '0;
      tightest_constraint <= TC_TOTAL; tightest_margin_permille <= '0;
    end else begin
      term_violation  <= 1'b0;
      total_violation <= 1'b0;
      tail_violation  <= 1'b0;
 
      if (clear) begin
        c_term_violations <= '0; c_total_violations <= '0; c_tail_violations <= '0;
      end
 
      if (frame_valid) begin
        automatic logic        tv = 1'b0;
        automatic int unsigned ti = 0;
        automatic logic [CNT_W-1:0] worst_use = '0;
        automatic logic [1:0]       worst_which = TC_TOTAL;
 
        for (int i = 0; i < TERMS; i++) begin
          if ((term_limit_ps[i] != '0) && (term_ps[i] > term_limit_ps[i])) begin
            tv = 1'b1; ti = i;
          end
          if (term_limit_ps[i] != '0) begin
            automatic logic [CNT_W-1:0] use =
                (CNT_W'(term_ps[i]) * 1000) / CNT_W'(term_limit_ps[i]);
            if (use > worst_use) begin worst_use = use; worst_which = TC_TERM; end
          end
        end
 
        if (tv) begin
          term_violation    <= 1'b1;
          violating_term    <= lat_term_e'(ti);
          c_term_violations <= c_term_violations + 1'b1;
        end
 
        if ((total_limit_ps != '0) && (total_ps > total_limit_ps)) begin
          total_violation    <= 1'b1;
          c_total_violations <= c_total_violations + 1'b1;
        end
        if (total_limit_ps != '0) begin
          automatic logic [CNT_W-1:0] use =
              (CNT_W'(total_ps) * 1000) / CNT_W'(total_limit_ps);
          if (use > worst_use) begin worst_use = use; worst_which = TC_TOTAL; end
        end
 
        tightest_constraint      <= worst_which;
        tightest_margin_permille <= (worst_use >= CNT_W'(1000)) ? '0
                                     : (CNT_W'(1000) - worst_use);
      end
 
      // The tail is checked on the ESTIMATE, not on individual frames --
      // a percentile is a property of a distribution and no single
      // sample can violate it.
      if (percentile_valid && (tail_limit_ps != '0) &&
          (percentile_ps > tail_limit_ps)) begin
        tail_violation    <= 1'b1;
        c_tail_violations <= c_tail_violations + 1'b1;
      end
    end
  end
 
endmodule

Classification: synthesizable monitor.

What it teaches: that the tail constraint is checked against an estimate and never against a frame. A percentile is a property of a distribution; no individual measurement can violate it, and a design that flags every frame above the tail limit is flagging the frames the limit expects to exist. A 99.9th-percentile limit permits one frame in a thousand to exceed it — that is what the statistic means — so checking it per frame produces an alarm on exactly the behaviour the requirement allows.

Deliberately simplified: three constraint kinds. Real requirements add jitter bounds and per-class limits, which are more constraints of the same three shapes.

Production implication: tightest_constraint is reported whether or not anything is violated, and that is what makes it a planning output rather than an alarm. A budget with 4% margin on processing and 60% on the total is a design one revision away from failing — and no violation counter says so, because nothing has failed yet. It is Chapter 8.2 §9's binding-limit argument applied to time instead of distance.

10. What a Decomposition Buys That a Number Does Not

Collecting the argument, because it is the chapter's thesis and it is easy to lose among the modules.

A total is not actionable because it does not name an owner. Figure 3's three frames all measure 200 µs; one is a capacity problem, one is a switching-policy problem, and one is an RTL problem. The number is identical and the three findings go to three different people.

A decomposition separates what can be promised from what can only be observed. The four constant terms are computable before deployment and are a guarantee; queueing is decided by traffic somebody else offers and can only be measured. Two conversations that a single number forces into one.

It gives a floor that no amount of load reduction can beat. A path whose constant terms sum to 80 µs will never deliver 50 µs on an empty link — a design finding available with no traffic at all.

And it tells you when the promise stops holding. crossover_utilisation_permille is the point at which queueing overtakes the four constants, and above it the latency a device can promise is not the latency a user will see.

11. Assertions — Terms, Tails and the Statistic That Matches

Azvya Education Pvt. Ltd.VLSI Mentor
Snippet
// ---------------------------------------------------------------------
// P1 -- CONSERVATION. The terms sum to the total. A decomposition whose
// parts do not add up is measuring overlapping intervals.
// ---------------------------------------------------------------------
property p_terms_sum_to_total;
  @(posedge clk) disable iff (!rst_n)
    frame_valid |-> (total_ps == (term_ps[LT_QUEUEING] + term_ps[LT_PROCESSING] +
                                  term_ps[LT_STORE_FORWARD] + term_ps[LT_SERIALIZATION] +
                                  term_ps[LT_PROPAGATION]));
endproperty
a_terms_sum_to_total: assert property (p_terms_sum_to_total)
  else $error("latency terms do not sum to the total -- intervals overlap or a gap exists");
 
// ---------------------------------------------------------------------
// P2 -- Stage boundaries are ordered. A timestamp out of order means an
// event fired in the wrong place.
// ---------------------------------------------------------------------
property p_boundaries_ordered;
  @(posedge clk) disable iff (!rst_n)
    frame_valid |-> ((t_deq_q >= t_arr_q) && (t_prc_q >= t_deq_q) && (t_txs_q >= t_prc_q));
endproperty
a_boundaries_ordered: assert property (p_boundaries_ordered);
 
// ---------------------------------------------------------------------
// P3 -- CROSS-CHECK. Measured serialization equals the value Chapter 8.1
// computes. The one term whose correct answer is known independently,
// and therefore the one that validates the instrument.
// ---------------------------------------------------------------------
property p_serialization_matches_computed;
  @(posedge clk) disable iff (!rst_n)
    frame_valid |-> (term_ps[LT_SERIALIZATION] inside
                     {[computed_serialization_ps - 40'd2000 :
                       computed_serialization_ps + 40'd2000]});
endproperty
a_serialization_matches_computed: assert property (p_serialization_matches_computed)
  else $error("measured serialization disagrees with arithmetic -- a boundary event is misplaced");
 
// ---------------------------------------------------------------------
// P4 -- Propagation is supplied, never measured. Written as an interface
// property: no measured interval may be attributed to it.
// ---------------------------------------------------------------------
property p_propagation_is_supplied;
  @(posedge clk) disable iff (!rst_n)
    frame_valid |-> (term_ps[LT_PROPAGATION] == $past(propagation_ps));
endproperty
a_propagation_is_supplied: assert property (p_propagation_is_supplied);
 
// ---------------------------------------------------------------------
// P5 -- The four constant terms do not vary with utilisation. The
// chapter's central claim, checkable directly.
// ---------------------------------------------------------------------
property p_constants_are_constant;
  @(posedge clk) disable iff (!rst_n)
    (frame_valid && $stable(frame_octets) && !$stable(utilisation_permille))
      |-> ($stable(term_ps[LT_SERIALIZATION]) && $stable(term_ps[LT_PROPAGATION]));
endproperty
a_constants_are_constant: assert property (p_constants_are_constant);
 
// ---------------------------------------------------------------------
// P6 -- Extremes survive a counter clear: the worst each term has been
// is a design input, not a measurement window.
// ---------------------------------------------------------------------
property p_worst_survives_clear;
  @(posedge clk) disable iff (!rst_n)
    clear |=> ($stable(worst_ps[LT_QUEUEING]) && $stable(worst_ps[LT_PROCESSING]));
endproperty
a_worst_survives_clear: assert property (p_worst_survives_clear);
 
// ---------------------------------------------------------------------
// P7 -- Every sample lands in exactly one bucket.
// ---------------------------------------------------------------------
property p_one_bucket_per_sample;
  @(posedge clk) disable iff (!rst_n)
    sample_valid |=> (bucket_sum == $past(bucket_sum) + 1);
endproperty
a_one_bucket_per_sample: assert property (p_one_bucket_per_sample);
 
// ---------------------------------------------------------------------
// P8 -- Bucket edges are monotonically increasing. A histogram whose
// edges are not ordered produces percentiles that are not percentiles.
// ---------------------------------------------------------------------
// synopsys translate_off
a_bucket_edges_monotonic: assert final
  ((40'(BASE_PS) << 1) > 40'(BASE_PS) &&
   (40'(BASE_PS) << (NBUCKET-1)) > (40'(BASE_PS) << (NBUCKET-2)))
  else $fatal(1, "histogram bucket edges are not monotonic");
// synopsys translate_on
 
// ---------------------------------------------------------------------
// P9 -- THE HONESTY PROPERTY. A percentile in the top bucket is a lower
// bound, and the design says so rather than reporting the edge as an
// answer it does not have.
// ---------------------------------------------------------------------
property p_saturation_reported;
  @(posedge clk) disable iff (!rst_n)
    (estimate_valid && (percentile_bucket == 4'(NBUCKET-1))) |-> saturated;
endproperty
a_saturation_reported: assert property (p_saturation_reported);
 
// ---------------------------------------------------------------------
// P10 -- The percentile estimate is monotonic in the requested
// percentile: a higher percentile is never a smaller value.
// ---------------------------------------------------------------------
property p_percentile_monotonic;
  @(posedge clk) disable iff (!rst_n)
    (estimate_valid && (percentile_tenths > $past(percentile_tenths)) &&
     $stable(samples))
      |-> (percentile_upper_ps >= $past(percentile_upper_ps));
endproperty
a_percentile_monotonic: assert property (p_percentile_monotonic);
 
// ---------------------------------------------------------------------
// P11 -- The queueing prediction is guarded against a saturated link.
// rho = 1 sends the model to infinity and the arithmetic to a divide by
// zero; the design must report the cap rather than fault.
// ---------------------------------------------------------------------
property p_prediction_guarded;
  @(posedge clk) disable iff (!rst_n)
    (result_valid && ($past(utilisation_permille) >= CNT_W'(1000)))
      |-> !$isunknown(predicted_ps);
endproperty
a_prediction_guarded: assert property (p_prediction_guarded);
 
// ---------------------------------------------------------------------
// P12 -- excess_beyond_model requires a prediction to compare against.
// ---------------------------------------------------------------------
property p_excess_needs_prediction;
  @(posedge clk) disable iff (!rst_n)
    excess_beyond_model |-> (predicted_ps != '0);
endproperty
a_excess_needs_prediction: assert property (p_excess_needs_prediction);
 
// ---------------------------------------------------------------------
// P13 -- The crossover is captured once and survives a clear: it is an
// operating characteristic of the link.
// ---------------------------------------------------------------------
property p_crossover_stable;
  @(posedge clk) disable iff (!rst_n)
    crossover_seen |=> $stable(crossover_utilisation_permille);
endproperty
a_crossover_stable: assert property (p_crossover_stable);
 
// ---------------------------------------------------------------------
// P14 -- The dominant term really is the largest.
// ---------------------------------------------------------------------
property p_dominant_is_largest;
  @(posedge clk) disable iff (!rst_n)
    report_valid |-> (term_ps[dominant_term] == max_term_ps);
endproperty
a_dominant_is_largest: assert property (p_dominant_is_largest);
 
// ---------------------------------------------------------------------
// P15 -- A term is only a lever when it is at least half the budget.
// Reporting an identity without a fraction produces engineers chasing
// terms that cannot pay.
// ---------------------------------------------------------------------
property p_lever_requires_majority;
  @(posedge clk) disable iff (!rst_n)
    dominant_is_lever |-> (dominant_permille >= CNT_W'(LEVER_PERMILLE));
endproperty
a_lever_requires_majority: assert property (p_lever_requires_majority);
 
// ---------------------------------------------------------------------
// P16 -- THE TAIL CONSTRAINT IS CHECKED ON THE ESTIMATE. A percentile is
// a property of a distribution, so no individual frame can violate it.
// ---------------------------------------------------------------------
property p_tail_checked_on_estimate;
  @(posedge clk) disable iff (!rst_n)
    tail_violation |-> $past(percentile_valid);
endproperty
a_tail_checked_on_estimate: assert property (p_tail_checked_on_estimate)
  else $error("a tail limit was violated by a single frame -- a percentile is not a per-frame bound");
 
// ---------------------------------------------------------------------
// P17 -- The tightest constraint is reported whether or not anything is
// violated, which is what makes it a planning output.
// ---------------------------------------------------------------------
property p_tightest_always_reported;
  @(posedge clk) disable iff (!rst_n)
    frame_valid |=> (tightest_margin_permille <= CNT_W'(1000));
endproperty
a_tightest_always_reported: assert property (p_tightest_always_reported);
 
// ---------------------------------------------------------------------
// P18 -- COVERAGE. Each term dominant at least once, and a saturated
// percentile estimate.
// ---------------------------------------------------------------------
c_dom_queueing:   cover property (@(posedge clk) disable iff (!rst_n) (report_valid && dominant_term == LT_QUEUEING));
c_dom_processing: cover property (@(posedge clk) disable iff (!rst_n) (report_valid && dominant_term == LT_PROCESSING));
c_dom_sf:         cover property (@(posedge clk) disable iff (!rst_n) (report_valid && dominant_term == LT_STORE_FORWARD));
c_percentile_sat: cover property (@(posedge clk) disable iff (!rst_n) saturated);
c_excess_seen:    cover property (@(posedge clk) disable iff (!rst_n) excess_beyond_model);

12. Verification — Twenty-Four Scenarios and a Distribution the Mean Hides

#ScenarioStimulusWhat must be observed
1Idle link, maximum frameone frame, empty queuequeueing ≈ 0; serialization ≈ 12.14 µs at 1 Gb/s
2Idle link, minimum frameone frameserialization ≈ 512 ns; the other terms unchanged
3Conservationany framethe five terms sum to the total (P1)
4Boundary orderingany frametimestamps strictly ordered (P2)
5Serialization cross-checkframes of several sizesmeasured matches computed within tolerance (P3)
6Boundary event misplacedfire tx_start a cycle earlyP3 fires — the instrument validates itself
7Propagation suppliedchange the supplied figurethe term follows it; nothing measured changes (P4)
8Constants under loadsweep utilisation, same frame sizeserialization and propagation unchanged (P5)
9Queueing at 50%offered load half the capacityqueueing ≈ one service time, ~12.3 µs
10Queueing at 90%offered load 90%≈ 9 service times, ~110 µs
11Queueing at 99%offered load 99%≈ 99 service times, ~1218 µs
12Saturated linkoffered load at capacityprediction guarded; no divide by zero (P11)
13Excess beyond the modelinject head-of-line blocking at 30% loadexcess_beyond_model — not a load finding
14Crossover capturedsweep utilisation upwardcrossover_utilisation_permille recorded once (P13)
15Histogram, one bucket eachany samplesexactly one bucket per sample (P7)
16Bucket edges monotonicelaborationascending (P8)
17Percentile saturatedlatencies above the top edgesaturated; the figure is a lower bound (P9)
18Percentile monotonicrequest 500 then 999 tenthsthe estimate does not decrease (P10)
19Dominant term: queueing99% utilisationLT_QUEUEING; high per mille
20Dominant term: processingidle link, deep pipelineLT_PROCESSING
21Dominant but not a leverfour roughly equal termsdominance reported; dominant_is_lever low (P15)
22Tail checked on the estimateone frame above the tail limitno tail_violation (P16)
23Tail violatedthe estimate exceeds the limittail_violation; c_tail_violations increments
24Tightest constraint with no violationall limits met, one at 96%tightest_constraint reported; margin 40 per mille

13. Debugging — Which Term, Which Owner

Symptom — end-to-end latency is above budget and no stage looks slow.

Read the decomposition rather than the total. Four terms are computable in advance, so compare each against its arithmetic: serialization against bits ÷ rate, propagation against length × 5 ns/m, store-and-forward against one serialization time per hop. Whatever is left is queueing, and queueing is the only term that could have changed without anything being reconfigured.

Symptom — latency is fine on average and users complain.

Almost certainly the tail. Read percentile_upper_ps at the 99th and 99.9th, and compare against the mean: a bimodal distribution has a mean that looks nothing like its tail, and queueing is the mechanism that produces one. If the two are far apart, the mean was the wrong statistic and the requirement was probably always about the tail.

Symptom — queueing delay far above what the utilisation suggests.

excess_beyond_model, and it is not a load problem. A simple queue at 30% utilisation produces about half a service time of delay; four times the prediction points at a mechanism — head-of-line blocking, a scheduler starving a class, a drain that stalled. The prediction exists precisely to separate "busy" from "broken", and the two are indistinguishable from the delay alone.

Symptom — the terms do not sum to the total.

The intervals overlap or there is a gap between them. P1 catches it; the usual cause is a stage boundary event that fires at the wrong point, so one interval includes time that belongs to its neighbour. Check the serialization term first, because its correct value is known independently — if measured serialization disagrees with frame_octets × 8 × bit_period, the timestamps are not where the module thinks they are.

Symptom — a percentile that never moves however bad things get.

saturated. The requested percentile is landing in the top bucket, which has no upper edge, so the reported figure is a lower bound rather than an estimate. The histogram's range is too small for the traffic — extend BASE_PS or the bucket count.

Symptom — an engineer optimised the dominant term and the total barely moved.

Check dominant_permille. A term at 260 per mille cannot improve the total by more than 26% even if it is eliminated entirely, and four roughly equal terms always have a largest one. dominant_is_lever exists to stop exactly this, and its absence in a report is why the work was done.

Symptom — latency was predictable and became erratic with no configuration change.

Compare the current utilisation against crossover_utilisation_permille. Below the crossover, latency is a property of the path and is predictable; above it, queueing dominates and it is a property of the load — which changes minute to minute. Nothing was reconfigured; the link got busier, and the crossover is the number that says when that started to matter.

14. Common Misconceptions

"Latency is one number."

The wrong model: a single measurement describes a path.

What it costs: the number is not actionable. Figure 3's three frames all measure 200 µs and are a capacity problem, a switching-policy problem and an RTL problem — three findings, three owners, one indistinguishable measurement.

The corrected model: five terms with five owners. Four are constants computable before deployment; the fifth is queueing and is decided by traffic somebody else offered. A total loses exactly the information a decision needs.

"A faster link gives lower latency."

The wrong model: rate improvements scale the whole budget.

What it costs: an upgrade that does not deliver, and an investigation looking for a fault. Chapter 8.1 showed propagation does not scale at all — and this chapter adds that queueing does not scale with the rate either: it scales with utilisation, which an upgrade may not change if the offered load rises to match.

The corrected model: only serialization and store-and-forward scale inversely with the rate. Propagation is fixed by the cable and queueing by the load, and at high utilisation the last one dominates everything.

"Queueing is just another delay term."

The wrong model: one more constant to add in.

What it costs: a budget that is correct at 50% utilisation and wrong by a factor of a hundred at 99%, with nothing having been reconfigured.

The corrected model: it is the only unbounded term. The mean waiting time is the service time times ρ / (1 − ρ) — 1× at 50%, 9× at 90%, 99× at 99% — while every other term sits exactly where it was. A denominator approaching zero is not a delay like the others.

"The mean latency is under budget, so we are fine."

The wrong model: the mean represents the distribution.

What it costs: Section 11's rejected property. A distribution with 99% of frames at 5 µs and 1% at 5 ms has a mean of 55 µs and a 99th percentile of 5 ms — and the frames that break a system are the ones a mean is designed to average away.

The corrected model: match the statistic to the requirement. Mean, median, percentile and maximum are four different requirements, and queueing is precisely the mechanism that makes them diverge.

"Compute the percentile exactly."

The wrong model: more accuracy is better.

What it costs: storage proportional to the sample count, or sorting, for accuracy no decision uses.

The corrected model: a bucketed histogram with logarithmic edges answers the question from sixteen counters at any sample count. "Is the tail at 90 µs or 900 µs" changes what somebody does; "is it 87 or 93" does not — and the estimator must report saturation when the answer falls in the open-ended top bucket, because that is a lower bound rather than an estimate.

15. Interview Reasoning

"Decompose the latency of a frame crossing a switched Ethernet path."

The weak answer lists stages vaguely. The answer that ends the topic gives five terms with their owners: serialization (bits ÷ rate), propagation (length × 5 ns/m), store-and-forward buffering (one serialization time per hop), per-frame processing (pipeline depth), and queueing. The payoff is the classification: the first four are constants computable before any traffic exists, and the fifth is the only one that depends on load — so a decomposition splits a figure into a part that can be promised and a part that can only be measured.

"Why does latency get so much worse near saturation when the link's speed has not changed?"

Because queueing goes as ρ / (1 − ρ), and the denominator is what does it. At 90% utilisation the mean wait is nine service times; at 99% it is ninety-nine — a hundredfold move while serialization, propagation and every other term sit exactly where they were. The strong close is that this is a second budget inversion: Chapter 8.1 showed one driven by the rate, and this one is driven by the load, which changes minute to minute rather than once per upgrade.

"Your latency requirement is a 99.9th percentile and your measurement is a mean. What is wrong?"

They are different questions, and queueing is why. A distribution with 99% of frames at 5 µs and 1% at 5 ms has a mean of 55 µs and a 99th percentile of 5 ms — the mean passes a 250 µs budget and the tail misses it twentyfold. The complete answer names the direction of the error: a mean-denominated check is too weak, so it does not fire, which is far more dangerous than a check that is too strict and fires on correct behaviour.

"How would you measure a latency percentile in hardware?"

Not exactly. A bucketed histogram with logarithmic edges gives constant relative resolution from a fixed, tiny amount of state — sixteen counters spanning 1 µs to 32 ms — and the accuracy it gives up is accuracy no decision uses. The finishing detail is saturated: when the requested percentile lands in the open-ended top bucket, the figure is a lower bound, and reporting the nominal edge as an answer is reporting a number you do not have.

16. Understanding Check

Because it does not name an owner, and the five terms have five different ones.

Figure 3's three frames all measure 200 µs. In the first, queueing dominates — a capacity or scheduling finding belonging to whoever offered the load. In the second, store-and-forward buffering dominates — a switching-policy finding. In the third, per-frame processing dominates — an RTL finding.

Same number, three different people.

And the decomposition also separates what can be promised from what can only be observed. Serialization, propagation, store-and-forward and processing are computable before deployment and are a guarantee. Queueing is decided by traffic somebody else offers, so a device can measure it and cannot promise it.

Which turns one impossible conversation into two possible ones: is this device fast enough and is this link too busy — questions with different evidence and different remedies, that a single number forces together.

17. What's Next

The claim this chapter defended: a latency figure that cannot be decomposed is a number nobody can act on, and four of its five terms are promises while the fifth is only ever a measurement.

Serialization, propagation, store-and-forward buffering and per-frame processing are computable before a frame is sent, owned by the frame format, the building, the switching policy and this design respectively. Queueing is owned by whoever offered the load, goes as ρ / (1 − ρ), and moves by a factor of a hundred between half and full utilisation while nothing else in the budget moves at all. Which makes the crossover — the utilisation at which queueing overtakes the four constants — the most useful operating number in the module.

And queueing is why the statistic matters. It produces long right tails from short means, so a mean can pass a budget by fivefold while the tail misses it by twentyfold, on the same window. Match the statistic to the requirement, estimate the tail with buckets rather than exactly, and say so when the estimate saturates.

Module 8 ends here. Four chapters turned "how fast is it" into arithmetic with named terms, and each of them arrived at the same structural point: an aggregate over terms with different sensitivities loses the information a decision needs.

Chapter 9.1 — 10 Mbps: The Original Shared-Medium MAC opens the history module, and it inverts the track's usual direction. Modules 1 through 8 repeatedly found constants that were inherited — the 64-octet floor, the 96-bit gap, the length/type disjointness, the preamble's length.

9.1 goes back to where they came from and shows that every one of them was rational. Manchester coding cost 100% overhead and bought a self-clocking signal that a 1980 receiver could actually recover. The shared coax made CSMA/CD necessary and slot time sized the frame. The chapter is the implementation rather than the derivation — Chapter 1.2 owns why, and 9.1 owns how it was built.

The full path is on the Ethernet curriculum index.

Continue learning

Standards & specifications

Governing standard
IEEE Std 802.3 (Ethernet)(opens IEEE in a new tab)

Defines the Ethernet MAC, the media-independent interfaces and the physical-layer sublayers, including framing, access control, auto-negotiation and per-rate PHY specifications. VLAN tagging, priority and time-sensitive shaping are defined by IEEE 802.1, not by 802.3.

This page also covers RTL structure, verification approach and debugging technique. Those are engineering practice built on the standard, not requirements the standard itself imposes.

Where this fits

Part of the Ethernet curriculum.