Ethernet · Module 14
PAUSE Frames
A PAUSE takes effect 13.3 µs after it is decided at 1 Gb/s, and the headroom that gap needs grows with line rate — 1.63 KiB at 1 Gb/s, 7.67 KiB at 100 — because propagation is fixed and the rate is not.
Chapter 14.1 §14 ended with one avenue unexplored. A buffer converts a burst into a delay and a sustained excess into a delayed discard, and Chapter 12.1 §6 established that no buffer size changes the arithmetic.
The remaining option is to tell the sender to stop.
802.3x specifies exactly that: a MAC control frame carrying a duration, sent to the device at the other end of a link, asking it to transmit nothing for that long. It is 64 octets — exactly the minimum frame — and its semantics are as simple as a mechanism can be: stop everything, for this many quanta.
And the number that organises this chapter is the gap between deciding to send one and traffic actually stopping.
At 1 Gb/s that gap is 13.3 µs, and the sender keeps transmitting throughout it. So the queue must have 1.63 KiB of headroom above the watermark that triggered the pause, or the frames arriving during the gap are lost anyway and the mechanism has achieved nothing.
At 100 Gb/s the same gap is 0.63 µs and the headroom is 7.67 KiB. The gap shrank and the headroom grew 4.7× — because propagation delay is fixed and the line rate is not.
1. Scope — What This Chapter Owns
This chapter owns the mechanism: the frame's format, the quantum, the trigger and its hysteresis, the dead time and the headroom it demands, the receiving side's timer, and what "stop transmitting" means to a device that is mid-frame.
It does not own where the congestion is — Chapter 14.1 found the buffers and priced what they buy. This chapter reacts to that chapter's high watermark.
It does not own what happens next. A paused sender's own queues begin to build, and Chapter 14.3 owns the propagation of that backlog upstream and the head-of-line blocking it produces — which is the pathology Chapter 12.1 §6 predicted.
And it does not own per-priority pause. 802.3x PAUSE is all-or-nothing on a link; Chapter 14.4 owns the per-class version and the lossless fabrics built on it. This chapter establishes exactly what the all-or-nothing costs, which is the argument that chapter needs.
2. The Frame
Sixty-four octets, of which forty-two are padding, carrying one 16-bit number.
| Offset | Field | Value | Why |
|---|---|---|---|
| 0–5 | destination | 01-80-C2-00-00-01 | a reserved multicast — never forwarded |
| 6–11 | source | the sender's address | conventional |
| 12–13 | EtherType | 0x8808 | MAC Control — Chapter 5.5's type range |
| 14–15 | opcode | 0x0001 | PAUSE |
| 16–17 | pause_time | 0 to 65 535 | in quanta of 512 bit times |
| 18–59 | padding | zeros | to reach Chapter 5.6's minimum |
| 60–63 | FCS | — | Chapter 5.8 |
Two of those rows carry the whole design.
The destination is a reserved multicast address that Chapter 12.4's flooding must never touch. A PAUSE is link-local: it means stop sending to me, and forwarding it would mean stop sending to somebody else, which is not a statement the frame can make. Chapter 12.2 §4's reserved-source rule and Chapter 12.3's gates both have to exclude this address explicitly, because nothing about its form marks it as special — it is an ordinary group address in every respect except that everybody agrees not to forward it.
And pause_time is in quanta of 512 bit times, which makes its meaning rate-dependent:
| Line rate | One quantum | Maximum pause — 65 535 quanta |
|---|---|---|
| 100 Mb/s | 5.12 µs | 335.5 ms |
| 1 Gb/s | 512 ns | 33.55 ms |
| 10 Gb/s | 51.2 ns | 3.355 ms |
| 25 Gb/s | 20.48 ns | 1.342 ms |
| 100 Gb/s | 5.12 ns | 0.336 ms |
The maximum pause duration shrinks by 100× from 1 Gb/s to 100 Gb/s, because a quantum is defined in bit times rather than in seconds. A design that assumed 33 ms of pause authority has 336 µs of it on a 100 Gb/s link — and a congestion event lasting longer requires a stream of PAUSE frames rather than one.
3. RTL 1 — Building a PAUSE Frame
Sixty-four octets with one variable field, and the two constants in it are the whole of the protocol.
// -----------------------------------------------------------------------
// pause_pkg -- shared types for 802.3x flow control.
// -----------------------------------------------------------------------
package pause_pkg;
// The reserved multicast that must never be forwarded. Chapter 12.4's
// flooding, Chapter 12.2's learning and Chapter 12.3's gates all have
// to exclude it explicitly -- nothing about its FORM marks it special.
localparam logic [47:0] PAUSE_DA = 48'h0180_C200_0001;
localparam logic [15:0] MAC_CTRL = 16'h8808;
localparam logic [15:0] OP_PAUSE = 16'h0001;
// A quantum is 512 BIT TIMES, so its duration depends on the line rate.
localparam int QUANTUM_BITS = 512;
localparam int MAX_QUANTA = 65535;
typedef enum logic [2:0] {
PR_OK = 3'd0,
PR_NOT_CONTROL = 3'd1, // wrong EtherType
PR_WRONG_OP = 3'd2, // MAC control, but not PAUSE
PR_WRONG_DA = 3'd3, // a PAUSE addressed to somebody else
PR_BAD_LEN = 3'd4,
PR_NOT_ENABLED = 3'd5 // this port does not honour PAUSE
} pause_reject_e;
// What the transmitter is doing about a pause it has been given.
typedef enum logic [1:0] {
TX_RUNNING = 2'd0,
TX_FINISHING = 2'd1, // mid-frame -- must complete it
TX_PAUSED = 2'd2
} tx_state_e;
endpackage// -----------------------------------------------------------------------
// pause_frame_builder -- constructs a 64-octet MAC control frame.
//
// The frame is exactly the minimum length, which is not a coincidence:
// 6 + 6 + 2 + 2 + 2 = 18 octets of content, 42 of padding and 4 of FCS.
// Everything after the pause_time field is zeros that exist only to reach
// Chapter 5.6's floor.
// -----------------------------------------------------------------------
module pause_frame_builder
import pause_pkg::*;
#(
parameter int CNT_W = 32
)(
input logic clk,
input logic rst_n,
input logic send_req,
input logic [15:0] quanta,
input logic [47:0] my_address,
output logic frame_valid,
output logic [511:0] frame_head, // first 64 octets, MSB first
output logic [13:0] frame_len,
output logic is_release, // quanta == 0
output logic [CNT_W-1:0] c_pause_sent,
output logic [CNT_W-1:0] c_release_sent,
output logic [CNT_W-1:0] quanta_sent_total
);
// A pause_time of ZERO is a RELEASE, not a zero-length pause. It is the
// only way to end a pause early, and a design that treats it as "pause
// for no time" has a mechanism that cannot be cancelled.
assign is_release = (quanta == 16'd0);
assign frame_len = 14'd64;
always_comb begin
frame_head = '0;
frame_head[511 -: 48] = PAUSE_DA; // octets 0-5
frame_head[463 -: 48] = my_address; // octets 6-11
frame_head[415 -: 16] = MAC_CTRL; // octets 12-13
frame_head[399 -: 16] = OP_PAUSE; // octets 14-15
frame_head[383 -: 16] = quanta; // octets 16-17
// octets 18-59 remain zero: Chapter 5.6's padding to the 64-octet
// floor. Chapter 13.3 Section 12 established that padding beyond a
// payload's declared length is invisible to the layer above, and here
// there is no payload at all.
end
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
frame_valid <= 1'b0;
c_pause_sent <= '0;
c_release_sent <= '0;
quanta_sent_total <= '0;
end else begin
frame_valid <= send_req;
if (send_req) begin
if (is_release) begin
if (!(&c_release_sent)) c_release_sent <= c_release_sent + 1'b1;
end else begin
if (!(&c_pause_sent)) c_pause_sent <= c_pause_sent + 1'b1;
quanta_sent_total <= quanta_sent_total + CNT_W'(quanta);
end
end
end
end
endmoduleClassification: synthesizable.
What it teaches: that pause_time = 0 is a release, not a zero-length pause, and that the distinction is the only way a pause ends early. A device that asserted a 1000-quanta pause and then drained its queue in 200 must be able to say so — otherwise it holds the link idle for 800 quanta it no longer needs. A design that treats zero as "pause for no time" has a mechanism it cannot cancel, and Section 9's hysteresis becomes the only control over the pause's duration.
And it teaches that the frame is exactly the minimum length by construction rather than by choice. Eighteen octets of content plus four of FCS is 22; Chapter 5.6's floor forces 42 octets of padding onto a frame whose entire information content is a 16-bit number. 65% of every PAUSE frame is zeros, and at 1 Gb/s each one costs 672 ns of the link it is trying to relieve.
Deliberately simplified: the whole frame is produced in one wide bus. A real transmitter emits it through the same datapath as any other frame, which means it must arbitrate for the transmitter — and Section 11 shows that a PAUSE queued behind ordinary traffic arrives late by exactly the amount that matters.
Production implication: quanta_sent_total divided by the elapsed time is the fraction of the link this port has asked to be held idle, and it is the number that says whether PAUSE is being used or abused. A port pausing its neighbour for 30% of every second is not managing congestion; it is operating at 70% of the link rate — and doing so in a way that, per Section 6, stops traffic bound for destinations that are not congested at all.
4. RTL 2 — Recognising One
Four fields must match, and the one everybody omits is the destination address.
// -----------------------------------------------------------------------
// pause_frame_parser -- is this a PAUSE for us, and is it valid?
//
// The destination check is the one that gets left out, and omitting it
// means honouring a PAUSE that was addressed to a different device --
// which on a shared segment is a station stopping traffic it has no
// authority over.
// -----------------------------------------------------------------------
module pause_frame_parser
import pause_pkg::*;
#(
parameter int CNT_W = 32
)(
input logic clk,
input logic rst_n,
input logic rx_valid,
input logic [47:0] rx_da,
input logic [15:0] rx_ethertype,
input logic [15:0] rx_opcode,
input logic [15:0] rx_quanta,
input logic [13:0] rx_len,
input logic pause_enabled, // negotiated -- Chapter 11.2
output logic pause_valid,
output logic [15:0] quanta,
output logic is_release,
output pause_reject_e reject,
output logic consume, // never forward this frame
output logic [CNT_W-1:0] c_pause_rx,
output logic [CNT_W-1:0] c_rejected [6],
output logic [CNT_W-1:0] quanta_rx_total
);
always_comb begin
pause_valid = 1'b0;
reject = PR_OK;
if (rx_valid) begin
// THE DESTINATION CHECK. A PAUSE is link-local and addressed to the
// reserved multicast; one addressed anywhere else is not a PAUSE
// for this device, whatever its opcode says.
if (rx_da != PAUSE_DA) reject = PR_WRONG_DA;
else if (rx_ethertype != MAC_CTRL) reject = PR_NOT_CONTROL;
else if (rx_opcode != OP_PAUSE) reject = PR_WRONG_OP;
else if (rx_len != 14'd64) reject = PR_BAD_LEN;
else if (!pause_enabled) reject = PR_NOT_ENABLED;
else pause_valid = 1'b1;
end
end
// CONSUMED, NOT FORWARDED -- and consumed even when this port does not
// honour PAUSE. Chapter 12.4's flooding must never see this address:
// forwarding a PAUSE means telling somebody else to stop, which is not
// a statement the frame can make.
assign consume = rx_valid && (rx_da == PAUSE_DA) &&
(rx_ethertype == MAC_CTRL);
assign quanta = pause_valid ? rx_quanta : 16'd0;
assign is_release = pause_valid && (rx_quanta == 16'd0);
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
c_pause_rx <= '0;
quanta_rx_total <= '0;
for (int i = 0; i < 6; i++) c_rejected[i] <= '0;
end else if (rx_valid) begin
if (pause_valid) begin
c_pause_rx <= c_pause_rx + 1'b1;
quanta_rx_total <= quanta_rx_total + CNT_W'(rx_quanta);
end else begin
c_rejected[reject[2:0]] <= c_rejected[reject[2:0]] + 1'b1;
end
end
end
endmoduleClassification: synthesizable.
What it teaches: that consume is asserted independently of pause_valid, and the separation matters. A port that does not honour PAUSE must still absorb the frame rather than forward it — because Chapter 12.4's flooding would otherwise carry it to every other port, and a PAUSE relayed onward tells a device to stop sending to somebody it was never sending to.
And it teaches why the destination check cannot be skipped. On a link with a shared segment behind it — Chapter 12.1 §16's hub case — a station could emit a MAC control frame addressed to itself and a receiver checking only the EtherType and opcode would honour it. The reserved multicast is what makes a PAUSE addressed to this link rather than merely present on it.
Deliberately simplified: pause_enabled as a single bit. In practice it is the outcome of Chapter 11.2's ability resolution — PAUSE support and its direction are negotiated, and a link may honour PAUSE in one direction and not the other, which makes pause_enabled two bits and the failure modes asymmetric.
Production implication: c_rejected[PR_WRONG_DA] rising is the signature of a neighbour emitting malformed control frames, and it is worth separating from the others because it is the only rejection reason that indicates a sender problem rather than a configuration one. The other four mean this port declined; that one means somebody sent something wrong.
5. The Dead Time
Between deciding to pause and traffic stopping, four things happen in sequence and the sender keeps transmitting through all of them.
| Stage | At 1 Gb/s | Controlled by |
|---|---|---|
| build and serialise the PAUSE frame | 672 ns | us — 84 octets on the wire |
| propagate 100 m of copper | 500 ns | physics |
| the sender finishes its current frame | up to 12 144 ns | the sender |
| the sender stops | — | — |
| total dead time | 13.32 µs | — |
The third row dominates and is the one the pausing device cannot influence at all. A sender that began a maximum-length frame one bit before the PAUSE arrived will finish it — Ethernet has no mechanism for aborting a frame in progress, and Chapter 12.6 §8 established that a truncated frame is discarded by every receiver.
So the headroom a queue needs above its pause watermark is dead time × line rate:
| Line rate | Frame TX | Propagation | Finish current | Dead time | Headroom |
|---|---|---|---|---|---|
| 1 Gb/s | 672 ns | 500 ns | 12 144 ns | 13.32 µs | 1.63 KiB |
| 10 Gb/s | 67 ns | 500 ns | 1214 ns | 1.78 µs | 2.17 KiB |
| 25 Gb/s | 27 ns | 500 ns | 486 ns | 1.01 µs | 3.09 KiB |
| 100 Gb/s | 7 ns | 500 ns | 121 ns | 0.63 µs | 7.67 KiB |
The dead time falls 21× from 1 Gb/s to 100 and the headroom rises 4.7×, and the reason is in the propagation column: it does not move. At 1 Gb/s, 500 ns of propagation costs 62 octets of headroom. At 100 Gb/s the same 500 ns costs 6250 — and it has gone from 4% of the requirement to 80% of it.
Which is the chapter's central arithmetic and the one thing about PAUSE that gets worse as links get faster.
6. All or Nothing
A PAUSE stops the link. Not a queue, not a class, not a conversation — everything the neighbour would have sent.
Which is the property Chapter 12.1 §6 predicted would be a problem, stated as a mechanism:
| Congested classes, of 8 | Classes stopped | Collateral |
|---|---|---|
| 1 | 8 | 88% |
| 2 | 8 | 75% |
| 4 | 8 | 50% |
One congested traffic class stops seven that are not. And the seven include, on any real link, traffic bound for completely idle egress ports — because a PAUSE is applied at the link, and the link carries frames for every destination the neighbour reaches through it.
That is head-of-line blocking, arriving one chapter early, and Chapter 14.3 owns its propagation. What matters here is that it is not an implementation weakness — it is the specification. 802.3x has no field for which traffic to stop, because the frame has no way to name a class: Chapter 13.2's PCP field is in the frames being paused, not in the PAUSE itself.
And the asymmetry that makes it worse: the pausing device knows exactly which queue is congested and cannot say so.
| What the pausing device knows | What it can express |
|---|---|
| which egress queue is above its watermark | nothing |
| which priority class is filling it | nothing |
| which ingress port's traffic is responsible | nothing |
| how long to stop | this — and only this |
Sixteen bits of duration, and no bits of anything else. Chapter 14.4 adds the missing field, and the whole of that chapter's justification is the table above.
7. RTL 3 — Deciding to Pause
Two thresholds, hysteresis between them, and a rule about re-issuing that a single comparator cannot express.
// -----------------------------------------------------------------------
// pause_trigger -- watermark crossings to PAUSE and RELEASE decisions.
//
// The asymmetry between the thresholds is the whole design. Asserting at
// HIGH leaves Section 5's headroom for the dead time; releasing at LOW
// rather than at HIGH prevents the mechanism oscillating once per frame.
// -----------------------------------------------------------------------
module pause_trigger
import pause_pkg::*;
#(
parameter int HIGH_CELLS = 3072,
parameter int LOW_CELLS = 1024,
parameter int PAUSE_QUANTA = 1000, // ~512 us at 1 Gb/s
parameter int REISSUE_AT = 700, // re-issue before it expires
parameter int CNT_W = 32
)(
input logic clk,
input logic rst_n,
input logic quantum_tick, // one per 512 bit times
input logic [15:0] occupancy_cells,
input logic tx_ready, // the transmitter took it
output logic send_pause,
output logic [15:0] send_quanta,
output logic paused_state,
output logic [CNT_W-1:0] c_asserted,
output logic [CNT_W-1:0] c_reissued,
output logic [CNT_W-1:0] c_released,
output logic [CNT_W-1:0] quanta_outstanding,
output logic [CNT_W-1:0] c_late_reissue // expired before re-issuing
);
logic [15:0] remaining;
always_comb begin
send_pause = 1'b0;
send_quanta = 16'd0;
if (!paused_state && (occupancy_cells >= 16'(HIGH_CELLS))) begin
// ASSERT at HIGH, not at full. Section 5's headroom is the space
// between HIGH and the queue's limit, and it must cover the dead
// time's worth of arrivals.
send_pause = 1'b1;
send_quanta = 16'(PAUSE_QUANTA);
end else if (paused_state && (occupancy_cells <= 16'(LOW_CELLS))) begin
// RELEASE with pause_time = 0. Section 3 established this is the
// only way to end a pause early, and without it the link stays idle
// for however long was asked for.
send_pause = 1'b1;
send_quanta = 16'd0;
end else if (paused_state && (remaining <= 16'(PAUSE_QUANTA - REISSUE_AT))) begin
// RE-ISSUE BEFORE EXPIRY. A pause that lapses while the queue is
// still above LOW lets the sender resume, and the queue -- which
// has not drained -- overflows during the next dead time. The
// re-issue must arrive before the current one expires.
send_pause = 1'b1;
send_quanta = 16'(PAUSE_QUANTA);
end
end
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
paused_state <= 1'b0;
remaining <= '0;
c_asserted <= '0;
c_reissued <= '0;
c_released <= '0;
quanta_outstanding <= '0;
c_late_reissue <= '0;
end else begin
if (send_pause && tx_ready) begin
if (send_quanta == 16'd0) begin
paused_state <= 1'b0;
remaining <= '0;
if (!(&c_released)) c_released <= c_released + 1'b1;
end else begin
if (!paused_state) begin
if (!(&c_asserted)) c_asserted <= c_asserted + 1'b1;
end else begin
if (!(&c_reissued)) c_reissued <= c_reissued + 1'b1;
end
paused_state <= 1'b1;
remaining <= send_quanta;
quanta_outstanding <= quanta_outstanding + CNT_W'(send_quanta);
end
end
if (quantum_tick && (remaining != 16'd0)) begin
remaining <= remaining - 16'd1;
// The pause expired while the queue was still above LOW. The
// sender is now free to resume into a queue that has not drained.
if ((remaining == 16'd1) && (occupancy_cells > 16'(LOW_CELLS)))
if (!(&c_late_reissue)) c_late_reissue <= c_late_reissue + 1'b1;
end
end
end
endmoduleClassification: synthesizable.
What it teaches: that the mechanism needs three decisions, not one, and a design with a single comparator has two of them missing. Assert at HIGH — leaving Section 5's headroom. Release at LOW — the hysteresis, without which the trigger oscillates once per frame as occupancy crosses a single threshold repeatedly. And re-issue before expiry, because Section 2's ceiling means a pause is finite and a congestion event may not be.
And c_late_reissue catches the failure the re-issue exists to prevent. A pause that lapses while the queue is still above LOW lets the sender resume — and the queue, which has not drained, overflows during the next dead time. The mechanism has then achieved a pause followed by exactly the loss it was preventing, and the counters show a successful pause.
Deliberately simplified: a fixed PAUSE_QUANTA and a fixed re-issue point. Production designs scale the requested duration with the queue's fill rate — a queue filling fast is asked to stop for longer — which converges faster and makes the re-issue logic considerably harder to verify.
Production implication: c_reissued against c_asserted describes what kind of congestion this port sees. A ratio near zero means bursts: one pause per event, and the event ends within the pause. A large ratio means sustained congestion being held off by a continuous stream of PAUSE frames — which is Chapter 12.1 §6's arithmetic being denied rather than managed, and Section 12 shows what it costs the link.
8. RTL 4 — Computing the Headroom
The watermark's position is not a tuning parameter. It is determined by the link's rate, its length and the neighbour's maximum frame, and it should be computed rather than chosen.
// -----------------------------------------------------------------------
// headroom_calculator -- where the HIGH watermark must sit.
//
// Section 5's dead time, converted into cells. A design that picks a
// watermark by intuition is picking a link length and a neighbour's frame
// size without knowing it.
// -----------------------------------------------------------------------
module headroom_calculator
import pause_pkg::*;
#(
parameter int LINK_MBPS = 1000,
parameter int CELL_OCTETS = 128,
parameter int CABLE_M = 100,
parameter int NS_PER_M = 5, // ~5 ns/m in copper, ~5 in fibre
parameter int MAX_FRAME = 1518,
parameter int QUEUE_CELLS = 4096
)(
input logic clk,
input logic rst_n,
input logic [15:0] tx_backlog_octets, // ahead of the PAUSE in the TX path
output logic [19:0] dead_time_ns,
output logic [19:0] headroom_octets,
output logic [15:0] headroom_cells,
output logic [15:0] high_watermark_cells,
output logic [15:0] prop_share_pct,
output logic watermark_infeasible // headroom exceeds the queue
);
// Three components, Section 5's table. The PAUSE frame is 84 octets on
// the wire including Chapter 5.2's preamble and the interframe gap.
localparam int PAUSE_TX_NS = (84 * 8 * 1000) / LINK_MBPS;
localparam int PROP_NS = CABLE_M * NS_PER_M;
localparam int FINISH_NS = (MAX_FRAME * 8 * 1000) / LINK_MBPS;
// tx_backlog is what sits AHEAD of the PAUSE in our own transmit path.
// Section 11: a PAUSE queued behind ordinary traffic arrives late by
// exactly the amount that matters.
logic [19:0] backlog_ns;
assign backlog_ns = (20'(tx_backlog_octets) * 20'd8 * 20'd1000) / 20'(LINK_MBPS);
assign dead_time_ns = 20'(PAUSE_TX_NS) + 20'(PROP_NS) +
20'(FINISH_NS) + backlog_ns;
// Octets that arrive during the dead time, at the neighbour's line rate.
assign headroom_octets = (dead_time_ns * 20'(LINK_MBPS)) / 20'd8000;
assign headroom_cells = 16'((headroom_octets + 20'(CELL_OCTETS) - 20'd1) /
20'(CELL_OCTETS));
// The watermark is the queue's limit MINUS the headroom, not a fraction
// of the queue. A design that sets it at 75% has assumed a headroom.
assign high_watermark_cells = (16'(QUEUE_CELLS) > headroom_cells)
? (16'(QUEUE_CELLS) - headroom_cells) : 16'd0;
// Propagation's share, which is what grows with line rate -- Section 5.
assign prop_share_pct = 16'((20'(PROP_NS) * 20'd100) / dead_time_ns);
// A QUEUE TOO SMALL TO PAUSE FROM. If the headroom exceeds the queue,
// there is no watermark at which a pause can be issued in time, and the
// mechanism cannot work on this link at all.
assign watermark_infeasible = (headroom_cells >= 16'(QUEUE_CELLS));
endmoduleClassification: synthesizable, and almost entirely elaboration-time constants.
What it teaches: that the watermark is queue limit − headroom, not a fraction of the queue, and the difference is a design decision most implementations make by accident. A watermark at 75% has assumed a headroom of 25% of whatever the queue happens to be — which on a 4096-cell queue is 1024 cells, fifteen times more than the 66 cells 1 Gb/s over 100 m actually requires, and on a small queue may be far too little.
And watermark_infeasible is the condition that says PAUSE cannot work here at all. If the dead time's worth of arrivals exceeds the whole queue, there is no occupancy at which a pause can be issued in time — every pause is too late by construction. At 100 Gb/s over a 2 km fibre the headroom is (10 000 + 500 + 121) ns × 100 Gb/s ÷ 8 = 132 KiB, and a queue smaller than that cannot use the mechanism.
Deliberately simplified: a fixed cable length and a single maximum frame size. Both are properties of the neighbour's link that this device may not know — and Chapter 11.2's negotiation does not carry either — so a production design uses a configured worst case, which is another value that must be conservative rather than measured.
Production implication: prop_share_pct is the number that says whether this link's headroom is dominated by physics or by the neighbour. Below 10% the dead time is mostly the neighbour finishing a frame, and reducing the maximum frame size would help. Above 50% it is mostly cable, and nothing but a shorter link changes it — which at 100 Gb/s over 100 m is 80%, and over 2 km is 98%.
9. Where the Watermark Must Sit
Put Sections 5 and 8 together and the watermark's position falls out of four numbers, none of which is a preference.
| Link | Dead time | Headroom | Cells at 128 B | Watermark, 4096-cell queue |
|---|---|---|---|---|
| 1 Gb/s, 100 m | 13.32 µs | 1.63 KiB | 13 | 4083 — 99.7% |
| 10 Gb/s, 100 m | 1.78 µs | 2.17 KiB | 18 | 4078 — 99.6% |
| 100 Gb/s, 100 m | 0.63 µs | 7.67 KiB | 62 | 4034 — 98.5% |
| 10 Gb/s, 2 km fibre | 11.2 µs | 13.7 KiB | 110 | 3986 — 97.3% |
| 100 Gb/s, 2 km fibre | 10.6 µs | 132 KiB | 1058 | 3038 — 74.2% |
Four of the five rows put the watermark above 97%, which is a long way from the 75% a design would set by intuition — and the fifth is the one that shows why intuition is not a method.
A 100 Gb/s link over 2 km needs 1058 cells of headroom on a 4096-cell queue, which is 25.8% of it. On a 1024-cell queue it would be infeasible entirely — watermark_infeasible — and the mechanism simply cannot be used.
And the row-to-row movement is the finding. The same 4096-cell queue supports a watermark at 99.7% on one link and 74.2% on another, and the difference is entirely the link's length and rate rather than anything about the queue or the traffic.
10. RTL 5 — The Receiving Side's Timer
A device that has been paused counts down in quanta, and the counter's units are the one thing about PAUSE that is genuinely simple.
// -----------------------------------------------------------------------
// pause_timer -- counts down a received pause, in quanta.
//
// A quantum is 512 bit times, so the tick generator is a divider off the
// transmit clock rather than a wall-clock timer. That is what makes the
// duration rate-relative and Section 2's ceiling rate-dependent.
// -----------------------------------------------------------------------
module pause_timer
import pause_pkg::*;
#(
parameter int CNT_W = 32
)(
input logic clk,
input logic rst_n,
input logic quantum_tick, // one per 512 bit times
input logic pause_valid,
input logic [15:0] quanta,
output logic paused,
output logic [15:0] remaining,
output logic [CNT_W-1:0] c_paused_quanta,
output logic [CNT_W-1:0] c_overwritten, // a pause arrived while paused
output logic [CNT_W-1:0] c_released_early,
output logic [19:0] longest_pause_q,
output logic [15:0] idle_pct_x100 // share of time paused
);
logic [CNT_W-1:0] ticks_total, ticks_paused;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
remaining <= '0;
c_paused_quanta <= '0;
c_overwritten <= '0;
c_released_early <= '0;
longest_pause_q <= '0;
ticks_total <= '0;
ticks_paused <= '0;
idle_pct_x100 <= '0;
end else begin
if (pause_valid) begin
// A NEW PAUSE REPLACES the outstanding one -- it does not add to
// it. A device re-issuing every 700 quanta is REFRESHING a 1000
// quantum pause, not accumulating 1700.
if ((remaining != 16'd0) && (quanta != 16'd0))
if (!(&c_overwritten)) c_overwritten <= c_overwritten + 1'b1;
if ((remaining != 16'd0) && (quanta == 16'd0))
if (!(&c_released_early)) c_released_early <= c_released_early + 1'b1;
remaining <= quanta;
if (20'(quanta) > longest_pause_q) longest_pause_q <= 20'(quanta);
end else if (quantum_tick && (remaining != 16'd0)) begin
remaining <= remaining - 16'd1;
c_paused_quanta <= c_paused_quanta + 1'b1;
end
if (quantum_tick) begin
ticks_total <= ticks_total + 1'b1;
if (remaining != 16'd0) ticks_paused <= ticks_paused + 1'b1;
if (ticks_total >= CNT_W'(100_000)) begin
idle_pct_x100 <= 16'((ticks_paused * CNT_W'(10_000)) / ticks_total);
ticks_total <= '0;
ticks_paused <= '0;
end
end
end
end
assign paused = (remaining != 16'd0);
endmoduleClassification: synthesizable.
What it teaches: that a new pause replaces the outstanding one rather than adding to it, and that this is what makes Section 7's re-issue strategy work. A device re-issuing 1000 quanta every 700 is refreshing a pause, not accumulating 1700 — and a design that accumulated would build an unbounded pause from a bounded stream of requests, holding a link idle long after the congestion cleared.
And idle_pct_x100 is the measurement that says what PAUSE is actually costing this link. A receiver paused for 30% of every second is operating at 70% of its line rate — and Section 6 established that the 30% stops traffic for every destination, including idle ones. The number is available only at the paused end, which is the end least likely to be investigated.
Deliberately simplified: a quantum tick derived from a divider. On a link whose rate can change — Chapter 11.3's bring-up, or a renegotiation — the divider must be reprogrammed, and a stale divider makes every received pause the wrong duration by exactly the rate ratio.
Production implication: c_overwritten is expected and c_released_early is informative. A steady c_overwritten means the neighbour is re-issuing, which Section 7's c_reissued describes from the other side — and the two counters should agree. A disagreement means PAUSE frames are being lost on the link, which is a possibility nobody plans for: the frame that manages congestion is itself subject to it.
11. RTL 6 — Stopping the Transmitter
"Stop transmitting" is unambiguous only when the transmitter is idle. Everything interesting is what it means when it is not.
// -----------------------------------------------------------------------
// pause_transmit_gate -- what a paused transmitter actually does.
//
// Three rules that are easy to get wrong:
// 1. A frame in progress is FINISHED. Ethernet cannot abort one.
// 2. A pause does not stop MAC control frames -- otherwise a paused
// device could never send its own PAUSE or RELEASE.
// 3. The PAUSE we send must not queue behind ordinary traffic, or
// Section 8's dead time grows by the backlog ahead of it.
// -----------------------------------------------------------------------
module pause_transmit_gate
import pause_pkg::*;
#(
parameter int CNT_W = 32
)(
input logic clk,
input logic rst_n,
input logic paused,
input logic frame_pending,
input logic frame_in_progress,
input logic frame_is_control, // our own PAUSE or RELEASE
input logic frame_end,
output logic tx_permit,
output tx_state_e state,
output logic control_bypass,
output logic [CNT_W-1:0] c_frames_after_pause, // finished mid-frame
output logic [CNT_W-1:0] c_control_through,
output logic [CNT_W-1:0] c_blocked,
output logic [15:0] worst_finish_octets
);
tx_state_e st_q;
logic [15:0] finish_octets;
always_comb begin
tx_permit = 1'b0;
control_bypass = 1'b0;
// RULE 2, first, because it is the one that makes the mechanism
// bidirectional. A paused device must still be able to pause ITS
// neighbour -- otherwise congestion cannot propagate upstream at all,
// which is Chapter 14.3's subject.
if (frame_is_control) begin
tx_permit = 1'b1;
control_bypass = 1'b1;
end else if (frame_in_progress) begin
// RULE 1. Ethernet has no abort. Chapter 12.6 Section 8 established
// that a truncated frame is discarded by every receiver, so
// stopping mid-frame would destroy the frame AND waste the wire.
tx_permit = 1'b1;
end else if (!paused) begin
tx_permit = frame_pending;
end
end
always_comb begin
if (!paused) state = TX_RUNNING;
else if (frame_in_progress) state = TX_FINISHING;
else state = TX_PAUSED;
end
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
st_q <= TX_RUNNING;
finish_octets <= '0;
c_frames_after_pause <= '0;
c_control_through <= '0;
c_blocked <= '0;
worst_finish_octets <= '0;
end else begin
st_q <= state;
// MEASURE THE OVERRUN. How many octets did we transmit after being
// told to stop? Section 5 predicts up to a maximum frame, and a
// measurement far from that means the neighbour's assumption about
// our frame size is wrong.
if (paused && frame_in_progress) begin
finish_octets <= finish_octets + 16'd1;
end else if (frame_end) begin
if (paused && (finish_octets > worst_finish_octets))
worst_finish_octets <= finish_octets;
if (paused)
if (!(&c_frames_after_pause))
c_frames_after_pause <= c_frames_after_pause + 1'b1;
finish_octets <= '0;
end
if (control_bypass)
if (!(&c_control_through)) c_control_through <= c_control_through + 1'b1;
if (paused && frame_pending && !tx_permit)
if (!(&c_blocked)) c_blocked <= c_blocked + 1'b1;
end
end
endmoduleClassification: synthesizable.
What it teaches: that rule 2 comes first and is the one that makes the mechanism usable at all. A paused device must still be able to send its own PAUSE frames — otherwise a switch paused by its downstream neighbour could never pause its upstream one, congestion could not propagate, and the backlog would simply overflow at the first device that was told to stop. That propagation is Chapter 12.1 §6's global congestion arriving, and Chapter 14.3 is about what it does — but it cannot happen at all unless control frames bypass the pause.
And worst_finish_octets is the measurement that validates the neighbour's headroom. Section 8 sizes the headroom against a maximum-length frame, and this counter says what the neighbour actually had to finish. A value far below 1518 means the neighbour reserved more headroom than it needed; a value at 1518 means the assumption was correct and necessary.
Deliberately simplified: an octet-granular finish counter and a single control-frame class. Production designs distinguish PAUSE from other MAC control opcodes and give only PAUSE the bypass, because a general control bypass is a path around every flow-control mechanism in the device.
Production implication: c_control_through under a sustained pause should be non-zero on any switch in the middle of a topology, because that switch is passing the congestion upstream. A value of zero means this device absorbs backpressure and never propagates it — which sounds polite and means its own queues are the ones overflowing, silently, while the actual congestion is two hops away.
12. What a Pause Costs the Link
A PAUSE frame is 64 octets and holding the link idle costs far more than the frame does.
| Pause duration | At 1 Gb/s | Bits not sent | As a share of a second |
|---|---|---|---|
| 1 quantum | 512 ns | 512 | 0.00005% |
| 100 quanta | 51.2 µs | 51 200 | 0.005% |
| 1000 quanta | 512 µs | 512 000 | 0.05% |
| 65 535 quanta | 33.55 ms | 33.5 M | 3.36% |
And the re-issue stream Section 7 requires for sustained congestion:
| Re-issue interval | PAUSE frames per second | Their own bandwidth |
|---|---|---|
| every 700 quanta | 1 ÷ (700 × 512 ns) = 2790 /s | 2790 × 672 = 1.87 Mb/s |
| every 100 quanta | 19 531 /s | 13.1 Mb/s |
| every 10 quanta | 195 313 /s | 131 Mb/s — 13% of the link |
The bottom row is a mechanism spending 13% of a link telling the neighbour not to use it, and it is what a design with an aggressive re-issue interval produces under sustained congestion.
Which sets the honest bound on what PAUSE achieves. It does not create bandwidth. A link paused 30% of the time delivers 70% of its rate — and Chapter 12.1 §6's arithmetic is unchanged: the excess offered load still has nowhere to go. All PAUSE does is move the place it is refused from this switch's buffer to the neighbour's.
And that relocation is the entire subject of Chapter 14.3.
What a lost control frame costs
Section 19's callout named the asymmetry; here it is in numbers, because the two losses differ by four orders of magnitude.
| Lost frame | What the sender believes | What actually happens | Cost |
|---|---|---|---|
| a PAUSE | the neighbour stopped | it keeps sending | the queue overflows — the loss the mechanism was for |
| a RELEASE | the neighbour resumed | it stays paused | the link is idle for the remaining duration |
| a re-issue | the pause was refreshed | it expires | Section 7's c_late_reissue — a gap, then overflow |
And the durations:
| Recovered by | Time to recovery at 1 Gb/s | |
|---|---|---|
| a lost PAUSE | the next watermark crossing | one frame's worth — microseconds |
| a lost re-issue | the next re-issue interval | ≈ 154 µs at 300 quanta |
| a lost RELEASE | nothing | up to 33.55 ms — the full pause |
A lost PAUSE recovers in microseconds because the trigger fires again on the next arrival. A lost RELEASE recovers when the pause expires, which is whatever duration was requested — and Section 7's design requests 1000 quanta, so a single corrupted release costs 512 µs of idle link.
A design requesting the maximum costs 33.55 ms, which at 1 Gb/s is 4.2 MB of link capacity spent on a congestion that ended.
Which makes the redundant release cheap in a way worth spelling out. Three RELEASE frames are 3 × 672 = 2016 ns at 1 Gb/s — two microseconds to remove a millisecond-scale stall from the failure list, and the frames are idempotent so there is no correctness cost to sending them.
13. RTL 7 — Pause Telemetry
Six mechanisms, and one place to read whether PAUSE is working, over-used, or failing silently.
// -----------------------------------------------------------------------
// pause_telemetry -- is flow control helping, and at what cost?
//
// The two conditions that matter both look like success from a drop
// counter: a link paused so often it is running at a fraction of its
// rate, and a pause issued too late to prevent the drop it was for.
// -----------------------------------------------------------------------
module pause_telemetry
import pause_pkg::*;
#(
parameter int CNT_W = 32,
parameter int WIN = 1_000_000
)(
input logic clk,
input logic rst_n,
input logic quantum_tick,
input logic pause_sent,
input logic pause_rx,
input logic paused_now,
input logic drop_while_paused,
input logic drop_any,
input logic [15:0] occupancy_cells,
input logic [15:0] high_watermark,
input logic [CNT_W-1:0] c_late_reissue,
output logic window_valid,
output logic [15:0] paused_pct_x100,
output logic [15:0] effective_rate_pct,
output logic [15:0] pause_overhead_ppm,
output logic pause_ineffective, // paused AND still dropping
output logic pause_excessive, // link idle a large fraction
output logic pause_too_late, // Section 7's failure
output logic [CNT_W-1:0] c_sent,
output logic [CNT_W-1:0] c_rx
);
logic [CNT_W-1:0] win_ticks, win_paused, win_sent, win_drop_paused, win_drop;
logic [CNT_W-1:0] prev_late;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
win_ticks <= '0; win_paused <= '0; win_sent <= '0;
win_drop_paused <= '0; win_drop <= '0; prev_late <= '0;
c_sent <= '0; c_rx <= '0;
window_valid <= 1'b0;
paused_pct_x100 <= '0;
effective_rate_pct <= 16'd100;
pause_overhead_ppm <= '0;
pause_ineffective <= 1'b0;
pause_excessive <= 1'b0;
pause_too_late <= 1'b0;
end else begin
window_valid <= 1'b0;
if (pause_sent) begin win_sent <= win_sent + 1'b1; c_sent <= c_sent + 1'b1; end
if (pause_rx) c_rx <= c_rx + 1'b1;
if (drop_while_paused) win_drop_paused <= win_drop_paused + 1'b1;
if (drop_any) win_drop <= win_drop + 1'b1;
if (quantum_tick) begin
win_ticks <= win_ticks + 1'b1;
if (paused_now) win_paused <= win_paused + 1'b1;
if (win_ticks >= CNT_W'(WIN)) begin
paused_pct_x100 <= 16'((win_paused * CNT_W'(10_000)) / win_ticks);
// THE NUMBER NOBODY COMPUTES. A link paused 30% of the time is
// a 70% link, and no rate counter says so.
effective_rate_pct <= 16'(100 - ((win_paused * CNT_W'(100)) / win_ticks));
// The PAUSE frames' own bandwidth: 672 bits each, against
// 512 bits per quantum of link time.
pause_overhead_ppm <= 16'((win_sent * CNT_W'(672) * CNT_W'(1_000_000)) /
(win_ticks * CNT_W'(512)));
// PAUSED AND STILL DROPPING. Either the headroom is wrong --
// Section 8 -- or the congestion exceeds what pausing one
// neighbour can fix.
pause_ineffective <= (win_drop_paused > (win_drop >> 1)) &&
(win_drop != '0);
// The link is idle a large fraction of the time by request.
pause_excessive <= (win_paused > (win_ticks >> 2));
// Section 7's lapsed pause.
pause_too_late <= (c_late_reissue != prev_late);
prev_late <= c_late_reissue;
win_ticks <= '0; win_paused <= '0; win_sent <= '0;
win_drop_paused <= '0; win_drop <= '0;
window_valid <= 1'b1;
end
end
end
end
wire _unused = |occupancy_cells | |high_watermark;
endmoduleClassification: synthesizable.
What it teaches: that effective_rate_pct is the number nobody computes and everybody needs. A link paused 30% of the time is a 70% link — and no rate counter reports it, because from the transmitter's point of view it simply had nothing to send. The utilisation graph shows 70% of a gigabit and the operator concludes there is headroom, when in fact the link is at its ceiling and the ceiling is 700 Mb/s.
And pause_ineffective is the condition that says the mechanism is not working while every counter says it is. Paused and still dropping means either Section 8's headroom is wrong — the pause arrives too late — or the congestion exceeds what pausing one neighbour can fix, which on a switch with 23 other ingress ports is the normal case.
Deliberately simplified: a single window and a single neighbour. A real port pauses one neighbour and may be paused by another; the two are independent and a port can be simultaneously the pauser and the paused, which is Chapter 14.3's propagation seen at one device.
Production implication: pause_excessive above a quarter is the threshold at which flow control has stopped being a safety mechanism and become the link's operating mode. A link paused 25% of the time is delivering 750 Mb/s of a gigabit, permanently, and the correct response is capacity rather than tuning — Chapter 12.1 §6's arithmetic has not gone away, it has been converted into idle time.
14. RTL 8 — Conformance for a Mechanism With a Dead Time
The monitor's difficulty is that the property everybody wants — that a pause stops traffic — is false for a known interval, and the interval is the design's most important parameter.
// -----------------------------------------------------------------------
// pause_conformance_monitor -- checks a control loop whose response is
// late by construction.
//
// The invariant is NOT that no frames arrive after a PAUSE. It is that
// none arrive after the DEAD TIME has elapsed, that the headroom covers
// the dead time, and that the frame itself was well formed and consumed.
// Section 17 is about the difference.
// -----------------------------------------------------------------------
module pause_conformance_monitor
import pause_pkg::*;
#(
parameter int CNT_W = 32
)(
input logic clk,
input logic rst_n,
input logic quantum_tick,
input logic pause_tx, // we transmitted one
input logic [15:0] tx_quanta,
input logic [19:0] dead_time_ns,
input logic [19:0] ns_since_pause,
input logic rx_frame, // a frame arrived from them
input logic rx_is_control,
input logic pause_rx, // we received one
input logic we_transmitted,
input logic we_were_paused,
input logic tx_was_control,
input logic tx_in_progress,
input logic [15:0] headroom_cells,
input logic [15:0] queue_cells,
input logic [47:0] rx_pause_da,
input logic pause_forwarded,
output logic [CNT_W-1:0] v_late_arrival, // after the dead time
output logic [CNT_W-1:0] v_transmitted_paused,
output logic [CNT_W-1:0] v_pause_forwarded, // must never happen
output logic [CNT_W-1:0] v_headroom_short,
output logic [CNT_W-1:0] v_control_blocked,
output logic headroom_feasible,
output logic conformant
);
// A STANDING PROPERTY. If the dead time's arrivals exceed the queue,
// no watermark works and the mechanism cannot function on this link.
assign headroom_feasible = (headroom_cells < queue_cells);
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
v_late_arrival <= '0;
v_transmitted_paused <= '0;
v_pause_forwarded <= '0;
v_headroom_short <= '0;
v_control_blocked <= '0;
end else begin
// THE REAL CHECK. Frames arriving DURING the dead time are
// expected; frames arriving AFTER it are the neighbour ignoring us.
if (rx_frame && !rx_is_control &&
(ns_since_pause > dead_time_ns) && (tx_quanta != 16'd0))
if (!(&v_late_arrival)) v_late_arrival <= v_late_arrival + 1'b1;
// We transmitted ordinary traffic while paused, and not because we
// were finishing a frame.
if (we_transmitted && we_were_paused && !tx_was_control && !tx_in_progress)
if (!(&v_transmitted_paused))
v_transmitted_paused <= v_transmitted_paused + 1'b1;
// A PAUSE THAT LEFT THIS DEVICE. Chapter 12.4's flooding must never
// touch the reserved address -- forwarding one tells a third party
// to stop sending to somebody it was not sending to.
if (pause_forwarded && (rx_pause_da == PAUSE_DA))
if (!(&v_pause_forwarded))
v_pause_forwarded <= v_pause_forwarded + 1'b1;
// The watermark was set without enough room for the dead time.
if (pause_tx && !headroom_feasible)
if (!(&v_headroom_short)) v_headroom_short <= v_headroom_short + 1'b1;
// Our own control frames were blocked by a pause. Section 11's
// rule 2 -- without it, congestion cannot propagate at all.
if (we_were_paused && tx_was_control && !we_transmitted)
if (!(&v_control_blocked))
v_control_blocked <= v_control_blocked + 1'b1;
end
end
assign conformant = (v_late_arrival == '0) &&
(v_transmitted_paused == '0) &&
(v_pause_forwarded == '0) &&
(v_headroom_short == '0) &&
(v_control_blocked == '0) &&
headroom_feasible;
endmoduleClassification: synthesizable, and intended to stay in silicon.
What it teaches: that the check is after the dead time, not after the PAUSE, and the difference is the whole chapter. Frames arriving during the dead time are expected and correct — the neighbour was mid-frame, the signal was in flight, physics happened. Frames arriving after it are the neighbour ignoring the pause, which is a real and diagnosable condition and the one the monitor exists to find.
And v_pause_forwarded catches a failure with severe reach. A PAUSE relayed by a switch tells a third device to stop sending — to a link it may not even be congested on. Chapter 12.4's flood mask must exclude the reserved address, and this counter is what proves it does.
Deliberately simplified: ns_since_pause supplied as an input. A real monitor derives it from the same quantum tick the timer uses, and getting that wrong makes every arrival look late or none of them do — which is why Section 17's properties assert the dead-time computation separately.
Production implication: conformant here means the PAUSE was well formed, was consumed rather than forwarded, the headroom covered the dead time, our control frames were not blocked, and the neighbour stopped once the dead time elapsed. It does not mean no frames were lost — Chapter 14.1 §18 established that the loss is arithmetic — and it does not mean the pause was a good idea, which Section 12's effective_rate_pct is the number for.
15. Two Ends, Two Views, One Link
A port can be pausing its neighbour and being paused by another at the same time, and the two directions share no state.
| this port sends PAUSE | this port receives PAUSE | |
|---|---|---|
| triggered by | our egress queue filling | their egress queue filling |
| the mechanism | Section 7's trigger | Section 10's timer |
| what stops | their transmitter | our transmitter |
| the counter | c_asserted, c_reissued | c_paused_quanta, idle_pct_x100 |
| the cost lands on | their throughput | our throughput |
| negotiated by | Chapter 11.2 | independently |
The last row is worth dwelling on. PAUSE capability is negotiated per direction — a link may honour PAUSE from A to B and not from B to A, and the two ends' configurations need not agree about which. Chapter 11.4's asymmetric-failure argument applies exactly: a device that believes it can pause its neighbour and cannot will fill its queue and drop, while the neighbour reports nothing at all.
And Section 11's rule 2 is what connects the two directions inside one device. A switch that is being paused must still be able to send PAUSE frames, because its own queues are now filling — the backlog it cannot transmit has to go somewhere, and the only thing it can do is push it upstream.
Which is the propagation Chapter 14.3 owns, and it starts here:
| Hop | State |
|---|---|
| the congested egress | queue above HIGH — pauses its neighbour |
| the neighbour | transmitter stopped — its own queues fill |
| the neighbour | queues above HIGH — pauses its neighbour |
| two hops upstream | transmitter stopped — queues fill |
| … | the congestion has left the building |
Each hop's pause is correct, local and well-founded. The aggregate is Chapter 12.1 §6's global congestion, assembled from purely local decisions — and no device in the chain has done anything wrong.
The negotiation, and what it does not carry
Chapter 11.2's ability resolution decides whether PAUSE is honoured and in which direction. It does not decide anything else, and the gaps matter.
| Negotiated | Not negotiated |
|---|---|
| whether PAUSE is honoured | the neighbour's maximum frame size |
| in which direction | the link's length |
| — | the neighbour's transmit backlog |
| — | the quantum's meaning if the rate changes |
Every item in the right-hand column is an input to Section 8's headroom calculation, and none of them is available from the link.
So the headroom is computed from a configured worst case rather than a negotiated one — a maximum frame of 1518 octets even against a neighbour that never sends jumbo, a cable length that must be the longest the deployment permits, and a transmit backlog assumed to be whatever this port's own queue can hold.
And the fourth row is the one that produces a real bug. A quantum is 512 bit times, so its wall-clock duration changes when the link rate does. A renegotiation from 1 Gb/s to 100 Mb/s makes every quantum 10× longer — and a divider that was not reprogrammed makes every received pause ten times shorter than intended, silently.
Chapter 11.3 §7's precondition matrix is the machinery for this: the quantum divider consumes the negotiated speed, and a divider carrying the previous link's value is exactly that chapter's stale measurement used — valid, present, and describing a link that no longer exists.
16. When PAUSE Is the Right Answer
Everything above is a catalogue of costs, and PAUSE is nonetheless the correct mechanism in a specific and identifiable case. Naming it is fairer than the catalogue alone.
| PAUSE helps | PAUSE hurts | |
|---|---|---|
| the congestion is a transient | yes — it buys the buffer time | — |
| the congestion is sustained | — | it converts loss into a slow link |
| one class on the link | — | 88% collateral — Section 6 |
| all traffic on the link matters equally | yes | — |
| the link is short | yes — small headroom | — |
| the link is long and fast | — | 132 KiB of headroom at 100 Gb/s over 2 km |
| the receiver is a single endpoint | yes | — |
| the receiver is a switch with 23 other ports | — | the backlog propagates — Section 15 |
| the traffic cannot tolerate loss | yes — this is the case it was built for | — |
| the traffic tolerates loss and not delay | — | Chapter 14.1 §12 — buffering is delay |
The pattern in the left-hand column is a short link to a single endpoint carrying loss-intolerant traffic in bursts — which is precisely a storage array, a directly-attached compute node, or the deliberately underloaded fabric Chapter 12.6 §10 identified as cut-through's home.
And the right-hand column is a general-purpose switched network, where the collateral, the propagation and the headroom all work against it.
Which is why 802.3x PAUSE is enabled by default on almost nothing and is essential in a small number of deployments — and why Chapter 14.4's per-priority version exists to move some of the right-hand column into the left.
What PAUSE cannot do
The chapter's costs are stated; the limits are worth stating too, because two of them are structural rather than a matter of tuning.
| Limitation | Structural or tunable | Why |
|---|---|---|
| stops all classes | structural | no class field exists — Section 6 |
| dead time before it takes effect | structural | propagation and the neighbour's frame |
| headroom grows with line rate | structural | propagation is fixed in seconds |
| maximum duration shrinks with rate | structural | a quantum is 512 bit times |
| watermark set by intuition | tunable | Section 8 derives it |
| hysteresis too wide or narrow | tunable | Section 10's callout |
| PAUSE queued behind traffic | tunable | a strict-priority control class |
| the backlog propagates upstream | structural | Chapter 14.3 |
Five of the eight are structural, which is an unusually high proportion for a mechanism this simple — and it is why Chapter 14.4 replaces the frame rather than tuning it.
The one that is not fixed even there is the dead time. Per-priority pause has exactly the same propagation delay and the same neighbour finishing exactly the same frame; it changes what is stopped, not when.
17. Properties Worth Asserting, and One Worth Refusing
Every property here is stated relative to the dead time. The rejected one is stated relative to the PAUSE frame, and that single substitution makes it false.
The frame
// P1. A PAUSE is addressed to the reserved multicast, always.
property p_pause_da_is_reserved;
@(posedge clk) disable iff (!rst_n)
frame_valid |-> (frame_head[511 -: 48] == PAUSE_DA);
endproperty
a_pause_da: assert property (p_pause_da_is_reserved);
// P2. It carries the MAC Control EtherType and the PAUSE opcode.
property p_pause_type_and_opcode;
@(posedge clk) disable iff (!rst_n)
frame_valid |-> ((frame_head[415 -: 16] == MAC_CTRL) &&
(frame_head[399 -: 16] == OP_PAUSE));
endproperty
a_pause_fields: assert property (p_pause_type_and_opcode);
// P3. It is exactly 64 octets -- Chapter 5.6's minimum, reached with 42
// octets of padding around an 18-octet payload.
property p_pause_is_minimum_length;
@(posedge clk) disable iff (!rst_n)
frame_valid |-> (frame_len == 14'd64);
endproperty
a_pause_length: assert property (p_pause_is_minimum_length);
// P4. pause_time = 0 is a RELEASE, not a zero-length pause. Without this
// a pause cannot be cancelled.
property p_zero_is_release;
@(posedge clk) disable iff (!rst_n)
(frame_valid && (frame_head[383 -: 16] == 16'd0)) |-> is_release;
endproperty
a_zero_releases: assert property (p_zero_is_release);
// P5. A PAUSE is CONSUMED, never forwarded -- even by a port that does
// not honour it. Chapter 12.4's flooding must not touch this address.
property p_pause_never_forwarded;
@(posedge clk) disable iff (!rst_n)
(rx_valid && (rx_da == PAUSE_DA)) |-> consume;
endproperty
a_never_forwarded: assert property (p_pause_never_forwarded);
// P6. The DESTINATION is checked, not only the EtherType and opcode. A
// PAUSE addressed elsewhere is not a PAUSE for this device.
property p_destination_checked;
@(posedge clk) disable iff (!rst_n)
(rx_valid && (rx_da != PAUSE_DA)) |-> !pause_valid;
endproperty
a_da_checked: assert property (p_destination_checked);The trigger
// P7. Assert at HIGH, not at full. The space above HIGH is Section 5's
// headroom and it must cover the dead time.
property p_assert_at_high;
@(posedge clk) disable iff (!rst_n)
(send_pause && (send_quanta != 16'd0) && !paused_state)
|-> (occupancy_cells >= 16'(HIGH_CELLS));
endproperty
a_assert_high: assert property (p_assert_at_high);
// P8. Release at LOW, not at HIGH. Without hysteresis the trigger
// oscillates once per frame.
property p_release_at_low;
@(posedge clk) disable iff (!rst_n)
(send_pause && (send_quanta == 16'd0))
|-> (occupancy_cells <= 16'(LOW_CELLS));
endproperty
a_release_low: assert property (p_release_at_low);
// P9. HIGH is above LOW. The hysteresis exists.
property p_hysteresis_exists;
@(posedge clk) disable iff (!rst_n)
(HIGH_CELLS > LOW_CELLS);
endproperty
a_hysteresis: assert property (p_hysteresis_exists);
// P10. A pause is RE-ISSUED before it expires while the queue is still
// above LOW -- Section 2's ceiling means a pause is finite and a
// congestion event may not be.
property p_reissue_before_expiry;
@(posedge clk) disable iff (!rst_n)
(paused_state && (remaining <= 16'(PAUSE_QUANTA - REISSUE_AT)) &&
(occupancy_cells > 16'(LOW_CELLS)))
|-> send_pause;
endproperty
a_reissue: assert property (p_reissue_before_expiry);
// P11. A lapsed pause with the queue still above LOW is COUNTED -- it is
// the failure the re-issue exists to prevent.
property p_lapse_counted;
@(posedge clk) disable iff (!rst_n)
(quantum_tick && (remaining == 16'd1) && (occupancy_cells > 16'(LOW_CELLS)))
|=> (c_late_reissue > $past(c_late_reissue));
endproperty
a_lapse_counted: assert property (p_lapse_counted);The dead time and the headroom
// P12. THE DEAD TIME is the sum of its three parts plus our own transmit
// backlog -- and a design that omits the backlog under-reserves by
// exactly the amount its own transmitter is busy.
property p_dead_time_complete;
@(posedge clk) disable iff (!rst_n)
(dead_time_ns == (20'(PAUSE_TX_NS) + 20'(PROP_NS) + 20'(FINISH_NS) +
backlog_ns));
endproperty
a_dead_time: assert property (p_dead_time_complete);
// P13. The headroom is the dead time's arrivals at line rate.
property p_headroom_from_dead_time;
@(posedge clk) disable iff (!rst_n)
(headroom_octets == ((dead_time_ns * 20'(LINK_MBPS)) / 20'd8000));
endproperty
a_headroom: assert property (p_headroom_from_dead_time);
// P14. The watermark is queue MINUS headroom, not a fraction of the
// queue. A fraction is a guess about a duration.
property p_watermark_is_derived;
@(posedge clk) disable iff (!rst_n)
(high_watermark_cells == (16'(QUEUE_CELLS) - headroom_cells));
endproperty
a_watermark_derived: assert property (p_watermark_is_derived);
// P15. A link whose headroom exceeds its queue is flagged INFEASIBLE --
// there is no watermark at which a pause arrives in time.
property p_infeasible_flagged;
@(posedge clk) disable iff (!rst_n)
(headroom_cells >= 16'(QUEUE_CELLS)) |-> watermark_infeasible;
endproperty
a_infeasible: assert property (p_infeasible_flagged);
// P16. Propagation's share of the dead time is reported, because it is
// the component that grows with line rate and cannot be reduced.
property p_prop_share_computed;
@(posedge clk) disable iff (!rst_n)
(prop_share_pct <= 16'd100);
endproperty
a_prop_share: assert property (p_prop_share_computed);The receiving side
// P17. A new pause REPLACES the outstanding one; it does not accumulate.
// Otherwise a bounded stream of requests builds an unbounded pause.
property p_pause_replaces_not_adds;
@(posedge clk) disable iff (!rst_n)
(pause_valid && (quanta != 16'd0)) |=> (remaining == $past(quanta));
endproperty
a_replaces: assert property (p_pause_replaces_not_adds);
// P18. A release ends the pause immediately.
property p_release_ends_pause;
@(posedge clk) disable iff (!rst_n)
(pause_valid && (quanta == 16'd0)) |=> (remaining == 16'd0);
endproperty
a_release_now: assert property (p_release_ends_pause);
// P19. The counter decrements once per quantum and never below zero.
property p_timer_decrements;
@(posedge clk) disable iff (!rst_n)
(quantum_tick && (remaining != 16'd0) && !pause_valid)
|=> (remaining == $past(remaining) - 16'd1);
endproperty
a_timer: assert property (p_timer_decrements);
// P20. A frame in progress is FINISHED. Ethernet has no abort, and
// Chapter 12.6 Section 8 established a truncated frame is discarded.
property p_frame_in_progress_completes;
@(posedge clk) disable iff (!rst_n)
(paused && frame_in_progress) |-> tx_permit;
endproperty
a_finish_frame: assert property (p_frame_in_progress_completes);
// P21. CONTROL FRAMES BYPASS THE PAUSE. Without this a paused switch
// cannot pause ITS neighbour and congestion cannot propagate at all.
property p_control_bypasses;
@(posedge clk) disable iff (!rst_n)
(paused && frame_is_control) |-> (tx_permit && control_bypass);
endproperty
a_control_bypass: assert property (p_control_bypasses);
// P22. Ordinary traffic does NOT bypass.
property p_ordinary_blocked;
@(posedge clk) disable iff (!rst_n)
(paused && frame_pending && !frame_is_control && !frame_in_progress)
|-> !tx_permit;
endproperty
a_ordinary_stops: assert property (p_ordinary_blocked);Cost and conformance
// P23. THE NUMBER NOBODY COMPUTES. A link paused a fraction of the time
// is delivering that much less than its rate.
property p_effective_rate_reported;
@(posedge clk) disable iff (!rst_n)
window_valid |-> (effective_rate_pct ==
16'(100 - (paused_pct_x100 / 16'd100)));
endproperty
a_effective_rate: assert property (p_effective_rate_reported);
// P24. Paused and still dropping is reported -- either the headroom is
// wrong or the congestion exceeds what pausing one neighbour can fix.
property p_ineffective_reported;
@(posedge clk) disable iff (!rst_n)
(window_valid && pause_ineffective) |-> (c_sent != '0);
endproperty
a_ineffective: assert property (p_ineffective_reported);
// P25. The PAUSE frames' own bandwidth is accounted -- 672 bits each,
// on the link they are relieving.
property p_overhead_accounted;
@(posedge clk) disable iff (!rst_n)
(window_valid && (c_sent != '0)) |-> (pause_overhead_ppm != 16'd0);
endproperty
a_overhead: assert property (p_overhead_accounted);
// P26. THE CORRECT ARRIVAL CHECK. Frames after the DEAD TIME are the
// neighbour ignoring us; frames during it are physics.
property p_no_arrivals_after_dead_time;
@(posedge clk) disable iff (!rst_n)
(rx_frame && !rx_is_control && (tx_quanta != 16'd0) &&
(ns_since_pause > dead_time_ns))
|-> v_late_arrival_will_increment;
endproperty
a_late_arrival: assert property (p_no_arrivals_after_dead_time);
// P27. A PAUSE never leaves this device.
property p_pause_not_relayed;
@(posedge clk) disable iff (!rst_n)
(rx_pause_da == PAUSE_DA) |-> !pause_forwarded;
endproperty
a_not_relayed: assert property (p_pause_not_relayed);
// P28. Conformance INCLUDES headroom_feasible -- a standing property of
// the link, not a history of events.
property p_conformant_includes_feasible;
@(posedge clk) disable iff (!rst_n)
conformant |-> headroom_feasible;
endproperty
a_conformant_feasible: assert property (p_conformant_includes_feasible);
// P29. Conformance means the mechanism behaved as specified -- never
// that no frame was lost.
property p_conformant_definition;
@(posedge clk) disable iff (!rst_n)
conformant |-> ((v_late_arrival == '0) && (v_transmitted_paused == '0) &&
(v_pause_forwarded == '0) && (v_headroom_short == '0) &&
(v_control_blocked == '0));
endproperty
a_conformant_def: assert property (p_conformant_definition);
// P30. PAUSE capability is per DIRECTION. A device may pause and not be
// pausable, or the reverse, and the two are negotiated independently.
property p_direction_independent;
@(posedge clk) disable iff (!rst_n)
(pause_tx_enabled != pause_rx_enabled) |-> asymmetric_pause_configured;
endproperty
a_per_direction: assert property (p_direction_independent);18. Verification Scenarios
Seventy-four scenarios. Several have expected outcomes in which frames arrive after a PAUSE has been sent and the design is correct.
The frame
| # | Scenario | Expected |
|---|---|---|
| 1 | A well-formed PAUSE | pause_valid, quanta extracted |
| 2 | Correct opcode, wrong destination | PR_WRONG_DA — not for us |
| 3 | Correct destination, wrong EtherType | PR_NOT_CONTROL |
| 4 | MAC Control, opcode 0x0002 | PR_WRONG_OP — not a PAUSE |
| 5 | PAUSE on a port with pause_enabled low | PR_NOT_ENABLED — and still consumed |
| 6 | Any frame to 01-80-C2-00-00-01 | consumed, never forwarded |
| 7 | Frame length ≠ 64 | PR_BAD_LEN |
| 8 | A built PAUSE | exactly 64 octets, 42 of them padding |
| 9 | pause_time = 0 | a RELEASE, not a zero-length pause |
| 10 | A PAUSE reaching Chapter 12.4's flood mask | v_pause_forwarded — must never happen |
Quanta
| # | Scenario | Expected |
|---|---|---|
| 11 | One quantum at 1 Gb/s | 512 ns |
| 12 | One quantum at 100 Gb/s | 5.12 ns |
| 13 | Maximum pause at 1 Gb/s | 33.55 ms |
| 14 | Maximum pause at 100 Gb/s | 0.336 ms — 100× less |
| 15 | A congestion event longer than the maximum | requires a re-issue stream |
| 16 | Stale quantum divider after a rate change | every pause the wrong duration by the rate ratio |
Dead time and headroom
| # | Scenario | Expected |
|---|---|---|
| 17 | Dead time at 1 Gb/s, 100 m | 13.32 µs |
| 18 | Its components | 672 ns + 500 ns + 12 144 ns |
| 19 | Headroom at 1 Gb/s | 1.63 KiB — 13 cells |
| 20 | Headroom at 100 Gb/s, 100 m | 7.67 KiB — 62 cells |
| 21 | Propagation's share at 1 Gb/s | 4% |
| 22 | Propagation's share at 100 Gb/s | 80% |
| 23 | 10 Gb/s over 2 km fibre | headroom 13.7 KiB |
| 24 | 100 Gb/s over 2 km fibre | headroom 132 KiB |
| 25 | Same, on a 1024-cell queue | watermark_infeasible |
| 26 | Watermark set at 75%, 1 Gb/s, 100 m | 79× more headroom than needed |
| 27 | Watermark set at 75%, 100 Gb/s, 2 km | 34 cells short — every pause too late |
| 28 | Our own transmit backlog ahead of the PAUSE | adds to the dead time |
The trigger
| # | Scenario | Expected |
|---|---|---|
| 29 | Occupancy crosses HIGH | PAUSE asserted |
| 30 | Occupancy falls to LOW | RELEASE sent |
| 31 | Occupancy oscillating around one threshold | hysteresis prevents per-frame toggling |
| 32 | Pause approaching expiry, queue above LOW | re-issued |
| 33 | Pause expiring with the queue above LOW | c_late_reissue |
| 34 | Re-issue at 700 of 1000 quanta | the outstanding pause is refreshed, not extended |
| 35 | c_reissued near zero | bursty congestion — one pause per event |
| 36 | c_reissued large | sustained congestion held off by a stream |
The receiving side
| # | Scenario | Expected |
|---|---|---|
| 37 | PAUSE received while already paused | replaces, c_overwritten |
| 38 | RELEASE received while paused | ends immediately, c_released_early |
| 39 | Quantum tick while paused | remaining decrements by 1 |
| 40 | Paused, transmitter idle | stops |
| 41 | Paused, transmitter mid-frame | finishes the frame — no abort exists |
| 42 | Paused, our own PAUSE to send | transmitted — control bypasses |
| 43 | Without the control bypass | congestion cannot propagate at all |
| 44 | worst_finish_octets on a busy neighbour | near 1518 — the headroom assumption was necessary |
| 45 | idle_pct_x100 = 3000 | the link is delivering 70% of its rate |
Cost, propagation and conformance
| # | Scenario | Expected |
|---|---|---|
| 46 | Re-issue every 700 quanta | 2790 PAUSE frames/s — 1.87 Mb/s |
| 47 | Re-issue every 10 quanta | 195 313 /s — 131 Mb/s, 13% of the link |
| 48 | One class of eight congested | 8 stopped — 88% collateral |
| 49 | A paused switch's own queues | fill, and it pauses upstream |
| 50 | Three hops of propagation | the congestion has left the building |
| 51 | Every hop's decision | correct, local and well-founded |
| 52 | Frames arriving during the dead time | expected — not a violation |
| 53 | Frames arriving after the dead time | v_late_arrival — the neighbour is ignoring us |
| 54 | Ordinary traffic transmitted while paused | v_transmitted_paused |
| 55 | Control frames blocked by a pause | v_control_blocked — rule 2 broken |
| 56 | Paused and still dropping | pause_ineffective |
| 57 | Paused above 25% of the time | pause_excessive — capacity, not tuning |
| 58 | PAUSE negotiated in one direction only | asymmetric — Chapter 11.4's shape |
| 59 | headroom_feasible low | inside conformant — a standing property |
| 60 | Healthy run, one million frames | conformant high throughout |
| 61 | PAUSE queued behind a full transmit queue | +4.19 ms of dead time — the mechanism has stopped working |
| 62 | PAUSE in a strict-priority control class | 0 added dead time |
| 63 | HIGH − LOW of 8 cells at 1 Gb/s | 8.2 µs — under one maximum frame; oscillates |
| 64 | HIGH − LOW of 2048 cells | 2.10 ms of idle time after the congestion clears |
| 65 | Asymmetric negotiation, we pause and they did not agree | v_late_arrival fires — and they are entitled |
| 66 | Same, checked at link-up | the negotiation's outcome says so three weeks earlier |
| 67 | A lost RELEASE | the link is idle up to 33.55 ms; nothing retries it |
| 68 | A release sent three times | 2 µs at 1 Gb/s, and the 33 ms stall is gone |
| 69 | Rate renegotiated, quantum divider not reprogrammed | every pause the wrong duration by the rate ratio |
| 70 | A lost re-issue at 300 quanta | recovers in ≈ 154 µs |
| 71 | Configured worst-case frame size against a neighbour that never sends jumbo | over-reserved, and correctly so |
| 72 | Five of eight limitations | structural — not addressable by tuning |
| 73 | Per-priority pause against the dead time | unchanged — it changes what is stopped, not when |
| 74 | A PAUSE relayed by a switch | never legal — v_pause_forwarded |
19. Debugging PAUSE
Every row produces a link that is passing traffic. Several produce a design that pauses correctly and drops anyway.
| Symptom | Likely cause | The observable that decides it |
|---|---|---|
| Pausing and still dropping | the watermark is too high for the dead time | headroom_cells against the queue; pause_ineffective |
| Same, and the neighbour looks fine | expected — frames during the dead time are physics | v_late_arrival = 0 confirms the neighbour obeys |
Same, v_late_arrival non-zero | the neighbour genuinely ignores PAUSE | a real and diagnosable condition |
| Throughput at 70% with no drops | the link is paused 30% of the time | idle_pct_x100, effective_rate_pct |
| Loss returned after a link was lengthened | propagation grew; the headroom did not | prop_share_pct, dead_time_ns |
| Loss after a 1 Gb/s link became 100 Gb/s | headroom needs 4.7× more, not less | Section 9's table |
| PAUSE deployed and nothing improved | the congestion is sustained, not bursty | c_reissued against c_asserted |
| Traffic for an idle destination stopped | all-or-nothing — 88% collateral | Section 6; the remedy is Chapter 14.4 |
| Congestion appeared two hops away | backpressure propagated | c_control_through on the intermediate switches |
| A switch absorbs backpressure and never propagates | its own queues are overflowing silently | c_control_through = 0 |
| PAUSE frames on the wire between switches | a switch relayed one | v_pause_forwarded — never legal |
| The link stalls for 33 ms after a burst | a maximum pause with no release | c_released = 0 while c_asserted rises |
| Pause and release alternating every frame | the hysteresis is under one maximum frame | HIGH − LOW in µs, not cells |
| The link idles for milliseconds after a burst clears | the hysteresis is too wide | HIGH − LOW at 2048 cells is 2.10 ms |
| A pause that is 10× shorter than requested | the quantum divider was not reprogrammed after a rate change | Chapter 11.3 §7's stale measurement |
| Every headroom figure over-reserved | a configured worst-case frame size and cable length | correct, and worth stating as a deliberate choice |
| The mechanism helps and the collateral is unacceptable | structural — no class field exists | Chapter 14.4, not tuning |
| Loss reappeared on a link that was working | the neighbour's frame size or the cable changed | dead_time_ns against the configured worst case |
| Two ends disagree about how many pauses were sent | PAUSE frames lost on the link | c_asserted against the far end's c_pause_rx |
| A control frame blocked by our own pause gate | rule 2 broken | v_control_blocked — congestion cannot propagate |
20. Common Misconceptions
1 — "A PAUSE stops traffic immediately."
The wrong model: the frame is sent, the neighbour stops.
What it costs: the watermark. A design believing this sets its trigger at or near full, and the 1.63 KiB that arrives during the 13.3 µs dead time overflows the queue — the exact loss the mechanism was deployed to prevent, with every counter reporting a successful pause.
The corrected model: the dead time is serialisation + propagation + the neighbour finishing its current frame, and the third term is 91% of it at 1 Gb/s and entirely outside our control. The watermark must be queue limit − headroom, and the headroom is the dead time's arrivals at line rate.
2 — "Faster links need less headroom."
The wrong model: everything is faster, so the dead time shrinks and the reserve can shrink with it.
What it costs: the failure that appears on exactly the links most likely to need flow control. The dead time does shrink — 13.32 µs to 0.63 µs from 1 Gb/s to 100 — and the headroom grows 4.7×, from 1.63 KiB to 7.67, because propagation is fixed in seconds and the rate is not.
The corrected model: propagation goes from 4% of the dead time at 1 Gb/s to 80% at 100 Gb/s, and on a 2 km link at 100 Gb/s the headroom is 132 KiB — larger than many entire queues, at which point watermark_infeasible says the mechanism cannot be used at all.
3 — "A PAUSE stops the congested traffic."
The wrong model: flow control targets the flow that is causing the problem.
What it costs: 88% collateral when one of eight classes is congested — and the stopped traffic includes frames bound for completely idle egress ports. That is head-of-line blocking produced by the specification.
The corrected model: 802.3x has sixteen bits of duration and no bits of anything else. The pausing device knows which queue, which class and which ingress port is responsible and can express none of it — Chapter 13.2's PCP is in the frames being paused, not in the PAUSE. Chapter 14.4 adds the missing field.
4 — "Flow control prevents loss."
The wrong model: enable PAUSE and the drops stop.
What it costs: a link running at a fraction of its rate with nobody noticing. A link paused 30% of the time delivers 700 Mb/s of a gigabit, and no rate counter says so — the transmitter simply had nothing to send. And Chapter 12.1 §6's arithmetic is unchanged: the excess offered load still has nowhere to go.
The corrected model: PAUSE relocates the refusal from this switch's buffer to the neighbour's, and Chapter 14.3 owns where it ends up. effective_rate_pct is the number that says what it cost, and pause_excessive above a quarter means the correct response is capacity rather than tuning.
5 — "pause_time = 0 pauses for no time."
The wrong model: zero is a degenerate duration.
What it costs: a mechanism that cannot be cancelled. A device that asserted 1000 quanta and drained in 200 holds the link idle for 800 quanta it no longer needs — and at the maximum, 33.55 ms at 1 Gb/s on a link whose congestion cleared long ago.
The corrected model: zero is a RELEASE, and it is the only way a pause ends early. A lost release is worse than a lost pause — the first is retried by the next trigger and the second is retried by nothing — which argues for sending it more than once.
6 — "A paused device should not transmit anything."
The wrong model: stop means stop.
What it costs: the ability to propagate congestion at all. A switch that is being paused has its own queues filling, and the only thing it can do about them is pause its neighbour — which requires transmitting a control frame while paused. A design that blocks everything has a switch that absorbs backpressure and overflows silently, while the actual congestion is two hops away.
The corrected model: control frames bypass the pause and ordinary traffic does not, and a frame already in progress is finished because Ethernet has no abort. Three rules, and the first is what makes the mechanism composable across a topology.
21. Interview Reasoning
Q1 — "You send a PAUSE. When does traffic stop?"
Reason through it. After the dead time, which at 1 Gb/s over 100 m is 13.32 µs. Three components: the PAUSE frame's own serialisation — 672 ns for 84 on-wire octets — the propagation — 500 ns — and the neighbour finishing the frame it had already begun, up to 12 144 ns. The strong answer names which of the three the design controls: only the first, which is 5% of the total. Propagation is physics, and the neighbour's frame is 91% and entirely outside our control — Ethernet has no abort, Chapter 12.6 §8 established a truncated frame is discarded, and the pausing device cannot know whether the neighbour just began a maximum frame. So it must assume the worst, and the queue needs 1.63 KiB of headroom above the watermark.
Q2 — "Does a 100 Gb/s link need more or less pause headroom than a 1 Gb/s one?"
Reason through it. More — 4.7× more, and this is the counter-intuitive one. The dead time falls 21× because two of its three components scale with the rate. Propagation does not. At 1 Gb/s, 500 ns of propagation costs 62 octets of headroom; at 100 Gb/s the same 500 ns costs 6250. So the headroom goes from 1.63 KiB to 7.67 KiB while the dead time falls from 13.32 µs to 0.63. The strong answer takes it further: on a 2 km fibre at 100 Gb/s the headroom is 132 KiB, larger than many entire queues — at which point watermark_infeasible says there is no watermark at which a pause arrives in time and the mechanism cannot be used on that link.
Q3 — "Where should the pause watermark sit?"
Reason through it. At queue limit − headroom, derived, not at a fraction of the queue. A watermark at 75% has assumed a headroom of 25% of whatever the queue happens to be — on a 4096-cell queue that is 1024 cells against a requirement of 13 at 1 Gb/s over 100 m, 79× too conservative, wasting queue that on a shared-buffer switch is Chapter 14.1 §6's pool. And on a 100 Gb/s 2 km link the requirement is 1058 cells and 75% reserves 1024 — 34 cells short. The strong answer names why that matters: the same constant is wrong in both directions and nothing about the symptom distinguishes them — a design that pauses and still drops looks like a neighbour ignoring PAUSE, which is a more interesting hypothesis and entirely wrong.
Q4 — "Why does a PAUSE stop traffic that is not congested?"
Reason through it. Because 802.3x has sixteen bits of duration and no bits of anything else. The pausing device knows which egress queue is above its watermark, which priority class is filling it and which ingress port is responsible — and can express none of them, because Chapter 13.2's PCP field lives in the frames being paused rather than in the PAUSE itself. The strong answer quantifies it: one congested class of eight stops all eight — 88% collateral — and the stopped traffic includes frames bound for completely idle egress ports. That is head-of-line blocking produced by the specification rather than by an implementation weakness, it is exactly what Chapter 12.1 §6 predicted when it rejected backpressure, and Chapter 14.4 exists to add the missing field.
Q5 — "A link shows 70% utilisation and no drops. Users complain. What do you check?"
Reason through it. Whether the link is paused for the other 30%. A paused transmitter simply has nothing to send, so a rate counter reports 70% of a gigabit and an operator concludes there is headroom — when the link is at its ceiling and the ceiling is 700 Mb/s. The strong answer names the counter: idle_pct_x100 at the paused end, which is the end least likely to be investigated, and effective_rate_pct which nobody computes. It then names the threshold: paused above 25% of the time means flow control has stopped being a safety mechanism and become the link's operating mode, and Chapter 12.1 §6's arithmetic has been converted into idle time rather than removed.
Q6 — "Should a paused device be allowed to transmit?"
Reason through it. Yes, in two cases, and the first is what makes the mechanism work across a topology. A frame already in progress must be finished — Ethernet has no abort and a truncated frame is discarded by every receiver, so stopping mid-frame destroys the frame and wastes the wire. And control frames must bypass the pause, because a paused switch's own queues are now filling and the only thing it can do is pause its neighbour. The strong answer draws the consequence: without that bypass, congestion cannot propagate, the first device told to stop simply overflows, and the actual congestion two hops away is invisible. With it, each hop's decision is correct, local and well-founded — and the aggregate is Chapter 12.1 §6's global congestion, which is Chapter 14.3's subject.
22. Understanding Check
23. What's Next
This chapter built the mechanism and measured what it costs the link it is applied to. It stopped at the point where the backlog leaves.
Section 15 traced the first two hops. A congested egress pauses its neighbour; the neighbour's own queues fill; the neighbour pauses its neighbour; and the congestion moves upstream one link at a time — with every device behaving correctly and no device doing anything wrong.
Chapter 14.3 — Backpressure and Head-of-Line Blocking follows it the rest of the way. The frames stopped at each hop are not only the ones bound for the congested destination — Section 6's 88% collateral, compounding hop by hop — and the result is a throughput bound that has been known since 1987 and is considerably worse than most people guess.
And that bound is the argument for Chapter 14.4. Per-priority flow control uses Chapter 13.2's PCP field to add the one thing 802.3x lacks: a way to say which traffic to stop — which turns Section 6's 88% collateral into something bounded, and makes the lossless fabrics that need it possible at the cost of a deadlock risk this chapter has not had to consider.
Continue learning
Related tutorials
- Related topic
Full Duplex and What It Removed from the MAC
Full duplex makes five of the half-duplex MAC's six blocks unreachable and collapses the transmit state machine from five states to two. It also removes a constraint that had been throttling senders by accident, which is why link-level flow control had to be invented to replace it.
- Related topic
Priority Flow Control and Lossless Ethernet
PFC subdivides a link into eight independently stoppable classes, costs eight headrooms instead of one, and admits a failure Ethernet has never had: a cycle of classes each waiting for the next.
- Related topic
PCIe vs Ethernet — Where the Cost of Overload Lands
The same overload into two fabrics: one stalled the sender 59,405 times and lost nothing, the other discarded 59,405 frames. That single choice explains why one needs TCP and the other does not.
- Related topic
The Shared-Medium Problem
Why several independent transmitters on one medium is a distributed timing problem, not a formatting problem. Propagation delay makes every station's view of the medium stale, so two locally correct decisions can still collide — and that is the constraint the Ethernet MAC was built around.
Standards & specifications
- Governing standard
- IEEE Std 802.3 (Ethernet)(opens IEEE in a new tab)
Defines the Ethernet MAC, the media-independent interfaces and the physical-layer sublayers, including framing, access control, auto-negotiation and per-rate PHY specifications. VLAN tagging, priority and time-sensitive shaping are defined by IEEE 802.1, not by 802.3.
This page also covers RTL structure, verification approach and debugging technique. Those are engineering practice built on the standard, not requirements the standard itself imposes.
Where this fits
Part of the Ethernet curriculum.
