Video over Isochronous

Chapter 16.1 streamed something uniform. Every audio sample is the same size, every frame carries roughly the same number of them, and a lost sample is exactly one sample — an audible click and nothing more.

Video is not uniform and a video frame is not one packet. A single frame is tens or hundreds of isochronous packets that must be reassembled, and the receiver's problem is no longer how many samples but where one frame ends and the next begins.

That sounds like a solved problem — mark the last packet — until you remember the property that defines this transfer type: nothing retries. The packet carrying that mark can be lost, and a receiver that relies on it alone will merge two frames, then three, then every frame after that.

This chapter is the single bit that prevents it.

1. The Bandwidth That Forces the Design

Start with why video is on isochronous at all, because the numbers explain the rest of the chapter.

A high-speed isochronous endpoint can request more than one transaction per microframe. The multiplier lives in the same wMaxPacketSize field as the packet size, and the Linux header extracts it explicitly:

#define USB_ENDPOINT_MAXP_MASK	0x07ff
#define USB_EP_MAXP_MULT_SHIFT	11
#define USB_EP_MAXP_MULT_MASK	(3 << USB_EP_MAXP_MULT_SHIFT)
#define USB_EP_MAXP_MULT(m) \
	(((m) & USB_EP_MAXP_MULT_MASK) >> USB_EP_MAXP_MULT_SHIFT)

Bits 10:0 are the packet size; bits 12:11 are the multiplier. A high-bandwidth isochronous endpoint can therefore move up to 3 × 1024 bytes per microframe, and at 8000 microframes per second:

3 × 1024 × 8000  =  24 576 000 bytes/s  ≈  24.6 MB/s

That is the ceiling, and uncompressed video is nowhere near fitting under it. A 1920×1080 frame in YUY2 is 1920 × 1080 × 2 = 4 147 200 bytes; at 30 frames per second that is 124.4 MB/s — five times the entire high-bandwidth isochronous budget.

Two consequences follow, and both shape the hardware.

Frames are compressed, so they are not a fixed size. An MJPEG frame of a blank wall is small; one of foliage is large. The receiver cannot know in advance how many packets a frame will take, which rules out the simplest possible design — count packets — before it is even proposed.

A frame spans many packets no matter what. At 1024 bytes a packet, a 414 kB MJPEG frame is roughly 404 packets. The boundary between frames is therefore an event that occurs once every few hundred packets, and it must be detected reliably from a stream in which any individual packet may simply not arrive.

2. Why the Audio Answer Does Not Transfer

Chapter 16.1's device solved its problem by measuring a rate and reporting it. That works because audio's failure mode is cumulative drift, which is slow, continuous, and correctable by a small adjustment.

Video's failure mode is structural and instantaneous. There is no rate to nudge. A packet is either placed in the right frame or it is not, and if the receiver's idea of "current frame" is wrong, every subsequent packet is placed wrongly too until something resynchronises it.

The last row is the chapter. Audio degrades; video desynchronises. And a desynchronised receiver does not recover on its own — it needs evidence in the stream that a boundary occurred, evidence that does not depend on the packet that was lost.

3. The Payload Header

Every isochronous payload in a USB Video Class stream begins with a small header the device writes and the host parses. The two bits this chapter needs are:

The header carries more — presentation timestamps, a source-clock reference, a still-image marker, and an error bit by which the device flags a payload it knows to be suspect — but FID and EOF are the two that determine frame boundaries.

At first reading, FID looks redundant. EOF already says where a frame ends. Why does a stream need a second, weaker signal that only says something changed?

4. The Bit That Survives a Lost Packet

Because EOF can be lost, and FID is what remains when it is.

Consider a 404-packet frame whose last packet — the one carrying EOF — is dropped. Follow a receiver that watches only EOF:

The EOF-only receiver does not merely corrupt one frame. It never recovers, because after the merge its notion of "current frame" is permanently one frame behind, and every subsequent EOF closes a frame made of two halves that do not belong together.

FID converts an unbounded failure into a bounded one. One lost packet costs exactly one frame, because the next packet to arrive carries independent evidence that a boundary was crossed.

5. Completed and Abandoned Are Different Events

Now the design decision that the rest of the chapter turns on.

When FID toggles and the open frame never saw its EOF, that frame is over — but it is not finished. Its last packets are missing, its byte count is short, and its compressed payload is very likely undecodable. Handing it downstream as a frame would be worse than emitting nothing, because a decoder would attempt it and produce garbage rather than concealment.

So the hardware distinguishes two outcomes:

And both can fire in the same cycle. If the packet that toggles FID also carries EOF — a legitimate single-packet frame, which happens when a compressed frame is tiny — then that one packet abandons the previous frame and completes a new one simultaneously. Two distinct events, one service interval.

A single frame_valid pulse cannot express this, and a design that tries will either drop the abort or delay the completion. Two outputs is not redundancy; it is the honest shape of the event space.

6. The Hardware, Before Any Language

State retained: cur_fid (the FID of the frame being collected), synced (has any packet been seen), open (is a frame being collected), dirty (has anything tainted it), and a byte accumulator.

On reset or bus reset: everything clears, and the partial frame is discarded rather than emitted. A frame whose first half belongs to a session that has ended is not a frame.

On the first packet ever seen: adopt its FID and begin collecting. The receiver cannot know whether it joined mid-frame, so the first frame it reports may be short — which is why have_sync is an output.

On a packet whose FID differs from cur_fid: a new frame starts here. If a frame was open, abort it. Adopt the new FID, and begin the new frame with this packet's bytes only.

On a packet whose FID matches: accumulate. A zero-length packet is legal — it means no data this service interval, which is what a camera sends during blanking — and it must neither end the frame nor taint it.

On EOF, in any of those cases: the frame completes, its byte count is published, and dirty decides whether it is reported clean.

On the error bit, or on a host-reported lost interval: mark the frame dirty. The frame still completes — it is not abandoned, because its boundaries are known; it is simply flagged as suspect so the consumer can decide.

Byte accumulation saturates. A wrapped byte count reports a plausible small frame for one that is enormous, and the direction matters: a downstream buffer sized from that count would be overrun rather than simply rejecting the frame.

7. Verilog

The RTL contract

• What it models: frame reassembly from the isochronous packet stream of a UVC device.
• Why it exists: because a frame spans hundreds of packets (§1), nothing retries, and a receiver that watches only EOF never recovers from losing one packet (§4).
• Inputs: pkt_valid, pkt_len, pkt_fid, pkt_eof, pkt_err (from the payload header), pkt_missing (the host reported this interval lost), bus_reset.
• State retained: cur_fid, synced, open, dirty, bytes_r.
• Outputs: frame_done + frame_ok + frame_bytes, frame_abort, have_sync.
• Hardware implied: one comparator on FID, one saturating BYTES_W accumulator, four flags, one holding register.
• Reset: asynchronous active-low rst_n; bus_reset synchronous and equivalent. Both discard the partial frame rather than emitting it.
• Priority: FID mismatch outranks continuation; within either, EOF closes the frame in the same interval.
• Latency: every output is registered, so a completion is visible the interval after the packet that caused it.
• Boundaries: bytes_r saturates at BYTES_MAX rather than wrapping.
• Collision semantics: a packet that toggles FID and carries EOF asserts frame_abort and frame_done in the same cycle, for two different frames.
• Assumptions: the payload header has already been parsed; pkt_len is payload bytes with the header excluded; exactly one packet is presented per service interval.
• Omissions: no header parsing, no frame memory, no timestamps, no format negotiation.
• What DV should verify: that a lost EOF produces exactly one abort and does not corrupt the following frame; that bytes never leak across a boundary; that a zero-length packet is inert; that abort and done can coincide; that the byte count saturates.

// uvc_frame_assembler -- reassembles video frames from isochronous packets.
//
// A video frame is MANY isochronous packets. Each carries a payload header
// whose Frame ID (FID) bit is constant within a frame and TOGGLES between
// frames, plus an End-of-Frame (EOF) bit on the last packet.
//
// The FID bit exists because isochronous has no retry. If the packet
// carrying EOF is lost, a receiver that only watched EOF would run two
// frames together -- and then the next, and the next, because nothing would
// ever resynchronise it. The FID toggle gives an INDEPENDENT frame boundary
// that survives the loss of any single packet: the very next packet that
// arrives has the other FID value and the receiver knows a frame ended.
//
// Hence two distinct completion outputs, which may both fire in one cycle:
//   frame_done  -- a frame ended at its EOF and its byte count is usable
//   frame_abort -- a frame ended because FID toggled with no EOF seen; its
//                  data is incomplete and must not be presented as a frame
module uvc_frame_assembler #(
  parameter integer LEN_W   = 12,   // payload bytes in one packet
  parameter integer BYTES_W = 24    // bytes in one video frame
) (
  input  wire               clk,
  input  wire               rst_n,
  input  wire               bus_reset,
  input  wire               pkt_valid,    // a packet arrived this interval
  input  wire [LEN_W-1:0]   pkt_len,      // payload bytes (0 = no data)
  input  wire               pkt_fid,      // Frame ID bit from the header
  input  wire               pkt_eof,      // End of Frame bit
  input  wire               pkt_err,      // Error bit from the header
  input  wire               pkt_missing,  // the host lost this interval's data
  output wire               frame_done,   // a frame ended at EOF
  output wire               frame_ok,     // ...and it was clean
  output wire [BYTES_W-1:0] frame_bytes,  // ...and this many bytes
  output wire               frame_abort,  // a frame ended WITHOUT its EOF
  output wire               have_sync     // a FID has been adopted
);
  localparam [BYTES_W-1:0] BYTES_MAX = {BYTES_W{1'b1}};

  reg               cur_fid;
  reg               synced;     // a FID has been adopted since reset
  reg               open;       // a frame is currently being collected
  reg               dirty;      // an error or a loss touched this frame
  reg [BYTES_W-1:0] bytes_r;

  reg               done_r, ok_r, abort_r;
  reg [BYTES_W-1:0] done_bytes_r;

  assign frame_done  = done_r;
  assign frame_ok    = ok_r;
  assign frame_bytes = done_bytes_r;
  assign frame_abort = abort_r;
  assign have_sync   = synced;

  // Is this packet the first of a NEW frame? Only meaningful once synced.
  wire toggled = synced && (pkt_fid != cur_fid);

  // This payload is tainted if the device flagged it or the host lost it.
  // Named ONCE: repeating the condition inline at each of the three uses
  // invites the copies to drift apart, and section 12 shows the mutation
  // evidence that they had already begun to.
  wire taint = pkt_err | pkt_missing;

  // Saturating byte accumulation. A wrapped byte count reports a PLAUSIBLE
  // small frame for one that is actually enormous, which is the direction
  // that makes a downstream buffer overflow rather than reject the frame.
  wire [BYTES_W:0] sum = {1'b0, bytes_r} + {{(BYTES_W-LEN_W+1){1'b0}}, pkt_len};
  wire [BYTES_W-1:0] bytes_next = sum[BYTES_W] ? BYTES_MAX : sum[BYTES_W-1:0];

  always @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
      cur_fid <= 1'b0; synced <= 1'b0; open <= 1'b0; dirty <= 1'b0;
      bytes_r <= {BYTES_W{1'b0}};
      done_r <= 1'b0; ok_r <= 1'b0; abort_r <= 1'b0;
      done_bytes_r <= {BYTES_W{1'b0}};
    end else if (bus_reset) begin
      // A bus reset discards the partial frame. Presenting it would emit a
      // frame whose top half belongs to a session that has ended.
      cur_fid <= 1'b0; synced <= 1'b0; open <= 1'b0; dirty <= 1'b0;
      bytes_r <= {BYTES_W{1'b0}};
      done_r <= 1'b0; ok_r <= 1'b0; abort_r <= 1'b0;
      done_bytes_r <= {BYTES_W{1'b0}};
    end else begin
      done_r  <= 1'b0;
      abort_r <= 1'b0;

      if (pkt_valid) begin
        if (!synced) begin
          // First packet ever: adopt its FID and start collecting.
          synced  <= 1'b1;
          cur_fid <= pkt_fid;
          open    <= 1'b1;
          dirty   <= taint;
          bytes_r <= bytes_next;
          if (pkt_eof) begin
            done_r       <= 1'b1;
            ok_r         <= ~taint;
            done_bytes_r <= bytes_next;
            open         <= 1'b0;
            bytes_r      <= {BYTES_W{1'b0}};
            dirty        <= 1'b0;
          end
        end else if (toggled) begin
          // A NEW frame begins here. If the old one never saw its EOF, it
          // is abandoned -- NOT completed. The two are different events and
          // they can occur in the same cycle as this packet's own EOF.
          if (open) abort_r <= 1'b1;
          cur_fid <= pkt_fid;
          open    <= 1'b1;
          dirty   <= taint;
          bytes_r <= {{(BYTES_W-LEN_W){1'b0}}, pkt_len};
          if (pkt_eof) begin
            // A single-packet frame: it opens and closes in one interval.
            done_r       <= 1'b1;
            ok_r         <= ~taint;
            done_bytes_r <= {{(BYTES_W-LEN_W){1'b0}}, pkt_len};
            open         <= 1'b0;
            bytes_r      <= {BYTES_W{1'b0}};
            dirty        <= 1'b0;
          end
        end else begin
          // A continuation packet of the frame already open. A zero-length
          // packet is LEGAL and means "no data this interval" -- it must not
          // end the frame and must not mark it dirty.
          bytes_r <= bytes_next;
          if (taint) dirty <= 1'b1;
          if (pkt_eof) begin
            done_r       <= 1'b1;
            ok_r         <= ~(dirty | taint);
            done_bytes_r <= bytes_next;
            open         <= 1'b0;
            bytes_r      <= {BYTES_W{1'b0}};
            dirty        <= 1'b0;
          end
        end
      end
    end
  end
endmodule

Two details are worth naming, and the second was found the hard way.

toggled is qualified by synced. Before any packet has been seen, cur_fid is meaningless, so comparing against it would manufacture a spurious frame boundary on the very first packet. The synced flag is not a convenience — without it the first frame of every session is aborted.

taint is named once. It is the condition this payload is suspect, and it is used in four places. An earlier version of this module wrote pkt_err | pkt_missing inline at each use — six occurrences of the same expression — and §13 is the account of how mutation testing found that and why it mattered.

8. SystemVerilog

Same hardware. Two things become types.

package uvc_pkg;
  // The payload header bits this design acts on, as a TYPE. The UVC class
  // specification packs them into one byte of every isochronous payload;
  // naming them here means a reader cannot confuse EOF with the error bit,
  // and the testbench can drive a header rather than four loose wires.
  typedef struct packed {
    logic fid;   // Frame ID -- constant within a frame, toggles between
    logic eof;   // End of Frame -- set on the last packet of a frame
    logic err;   // Error -- the device marks this payload as suspect
  } uvc_hdr_t;

  // What this packet IS relative to the frame currently open. Naming the
  // three cases makes their exclusivity checkable; in the Verilog they are
  // three arms of an if/else chain and the exclusivity is only positional.
  typedef enum logic [1:0] {
    P_NONE,      // no packet this service interval
    P_FIRST,     // the very first packet since reset -- adopt its FID
    P_NEWFRAME,  // FID toggled: a new frame starts here
    P_CONTINUE   // same FID: a continuation of the open frame
  } prole_e;
endpackage

module uvc_frame_assembler_sv
  import uvc_pkg::*;
#(
  parameter int unsigned LEN_W   = 12,
  parameter int unsigned BYTES_W = 24
) (
  input  logic               clk,
  input  logic               rst_n,
  input  logic               bus_reset,
  input  logic               pkt_valid,
  input  logic [LEN_W-1:0]   pkt_len,
  input  uvc_hdr_t           pkt_hdr,
  input  logic               pkt_missing,
  output logic               frame_done,
  output logic               frame_ok,
  output logic [BYTES_W-1:0] frame_bytes,
  output logic               frame_abort,
  output logic               have_sync
);
  initial begin
    if (BYTES_W <= LEN_W)
      $fatal(1, "BYTES_W=%0d must exceed LEN_W=%0d: a frame holds many packets",
             BYTES_W, LEN_W);
  end

  localparam logic [BYTES_W-1:0] BYTES_MAX = '1;

  logic               cur_fid, synced, open, dirty;
  logic [BYTES_W-1:0] bytes_r;

  assign have_sync = synced;

  prole_e role;
  always_comb begin
    if      (!pkt_valid)                 role = P_NONE;
    else if (!synced)                    role = P_FIRST;
    else if (pkt_hdr.fid != cur_fid)     role = P_NEWFRAME;
    else                                 role = P_CONTINUE;
  end

  // This payload is tainted if the device said so or the host lost it.
  wire taint = pkt_hdr.err | pkt_missing;

  // Saturating accumulation, widened BEFORE the add so the carry is visible.
  logic [BYTES_W:0] sum;
  logic [BYTES_W-1:0] bytes_next, len_ext;
  always_comb begin
    len_ext    = BYTES_W'(pkt_len);
    sum        = {1'b0, bytes_r} + {1'b0, len_ext};
    bytes_next = sum[BYTES_W] ? BYTES_MAX : sum[BYTES_W-1:0];
  end

  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n || bus_reset) begin
      // A bus reset discards the partial frame: presenting it would emit a
      // frame whose first half belongs to a session that has ended.
      cur_fid <= 1'b0; synced <= 1'b0; open <= 1'b0; dirty <= 1'b0;
      bytes_r <= '0;
      frame_done <= 1'b0; frame_ok <= 1'b0; frame_abort <= 1'b0;
      frame_bytes <= '0;
    end else begin
      frame_done  <= 1'b0;
      frame_abort <= 1'b0;

      unique case (role)
        P_NONE: ;   // nothing arrived; hold everything

        P_FIRST: begin
          synced  <= 1'b1;
          cur_fid <= pkt_hdr.fid;
          if (pkt_hdr.eof) begin
            frame_done  <= 1'b1;
            frame_ok    <= ~taint;
            frame_bytes <= bytes_next;
            open <= 1'b0; bytes_r <= '0; dirty <= 1'b0;
          end else begin
            open <= 1'b1; bytes_r <= bytes_next; dirty <= taint;
          end
        end

        P_NEWFRAME: begin
          // The open frame never saw its EOF, so it is ABANDONED, not
          // completed. Abort and done are different events and both may
          // fire this cycle when the new frame is a single packet.
          if (open) frame_abort <= 1'b1;
          cur_fid <= pkt_hdr.fid;
          if (pkt_hdr.eof) begin
            frame_done  <= 1'b1;
            frame_ok    <= ~taint;
            frame_bytes <= len_ext;
            open <= 1'b0; bytes_r <= '0; dirty <= 1'b0;
          end else begin
            open <= 1'b1; bytes_r <= len_ext; dirty <= taint;
          end
        end

        P_CONTINUE: begin
          // A zero-length packet is LEGAL and means "no data this interval".
          // It must not end the frame and must not taint it.
          if (taint) dirty <= 1'b1;
          if (pkt_hdr.eof) begin
            frame_done  <= 1'b1;
            frame_ok    <= ~(dirty | taint);
            frame_bytes <= bytes_next;
            open <= 1'b0; bytes_r <= '0; dirty <= 1'b0;
          end else begin
            bytes_r <= bytes_next;
          end
        end
      endcase
    end
  end
endmodule

uvc_hdr_t makes the header an object rather than four loose wires. The testbench drives pkt_hdr.eof rather than the third of six positional arguments, and a reader cannot transpose EOF with the error bit — a transposition that in the Verilog is a one-character edit that still compiles.

prole_e names the three cases and makes their exclusivity checkable. In the Verilog, first packet, new frame and continuation exist only as the positions of an if/else if chain; the exclusivity is real but is a property of the ordering rather than of anything declared. Naming them also puts §6's rule — a FID match is a continuation — in the definition of P_CONTINUE rather than in the middle of a chain.

The $fatal guard catches BYTES_W <= LEN_W, a parameterisation in which the frame accumulator cannot hold even two packets. It produces a design that runs, saturates constantly, and reports every frame as maximum length.

9. VHDL

library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;

package uvc_pkg is
  -- The payload header bits this design acts on, as a RECORD. The UVC class
  -- specification packs them into one byte of every isochronous payload.
  type uvc_hdr_t is record
    fid : std_logic;   -- Frame ID: constant within a frame, toggles between
    eof : std_logic;   -- End of Frame: set on the last packet of a frame
    err : std_logic;   -- Error: the device marks this payload as suspect
  end record;

  -- What this packet is relative to the frame currently open. VHDL's
  -- enumeration is a genuine distinct type -- unlike the Verilog, where the
  -- three cases exist only as the positions of an if/else chain.
  type prole_t is (P_NONE, P_FIRST, P_NEWFRAME, P_CONTINUE);
end package;

library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
use work.uvc_pkg.all;

entity uvc_frame_assembler_vhdl is
  generic (
    LEN_W   : positive := 12;
    BYTES_W : positive := 24
  );
  port (
    clk         : in  std_logic;
    rst_n       : in  std_logic;
    bus_reset   : in  std_logic;
    pkt_valid   : in  std_logic;
    pkt_len     : in  unsigned(LEN_W-1 downto 0);
    pkt_hdr     : in  uvc_hdr_t;
    pkt_missing : in  std_logic;
    frame_done  : out std_logic;
    frame_ok    : out std_logic;
    frame_bytes : out unsigned(BYTES_W-1 downto 0);
    frame_abort : out std_logic;
    have_sync   : out std_logic
  );
end entity;

architecture rtl of uvc_frame_assembler_vhdl is
  constant BYTES_MAX : unsigned(BYTES_W-1 downto 0) := (others => '1');

  signal cur_fid, synced, frame_open, dirty : std_logic := '0';
  signal bytes_r : unsigned(BYTES_W-1 downto 0) := (others => '0');

  signal done_r, ok_r, abort_r : std_logic := '0';
  signal done_bytes_r : unsigned(BYTES_W-1 downto 0) := (others => '0');

  signal role  : prole_t;
  signal taint : std_logic;
  signal sum        : unsigned(BYTES_W downto 0) := (others => '0');
  signal bytes_next : unsigned(BYTES_W-1 downto 0) := (others => '0');
  signal len_ext    : unsigned(BYTES_W-1 downto 0) := (others => '0');
begin
  assert BYTES_W > LEN_W
    report "BYTES_W must exceed LEN_W: a video frame holds many packets"
    severity failure;

  frame_done  <= done_r;
  frame_ok    <= ok_r;
  frame_bytes <= done_bytes_r;
  frame_abort <= abort_r;
  have_sync   <= synced;

  role <= P_NONE     when pkt_valid = '0' else
          P_FIRST    when synced = '0' else
          P_NEWFRAME when pkt_hdr.fid /= cur_fid else
          P_CONTINUE;

  -- Tainted if the device said so, or the host reported the interval lost.
  taint <= pkt_hdr.err or pkt_missing;

  -- Saturating accumulation. resize() widens BEFORE the add so the carry
  -- out of BYTES_W bits is a real bit rather than a discarded one.
  len_ext    <= resize(pkt_len, BYTES_W);
  sum        <= resize(bytes_r, BYTES_W+1) + resize(len_ext, BYTES_W+1);
  bytes_next <= BYTES_MAX when sum(BYTES_W) = '1'
                else sum(BYTES_W-1 downto 0);

  process (clk, rst_n)
  begin
    if rst_n = '0' then
      cur_fid <= '0'; synced <= '0'; frame_open <= '0'; dirty <= '0';
      bytes_r <= (others => '0');
      done_r <= '0'; ok_r <= '0'; abort_r <= '0';
      done_bytes_r <= (others => '0');
    elsif rising_edge(clk) then
      if bus_reset = '1' then
        -- Discard the partial frame: presenting it would emit a frame whose
        -- first half belongs to a session that has ended.
        cur_fid <= '0'; synced <= '0'; frame_open <= '0'; dirty <= '0';
        bytes_r <= (others => '0');
        done_r <= '0'; ok_r <= '0'; abort_r <= '0';
        done_bytes_r <= (others => '0');
      else
        done_r  <= '0';
        abort_r <= '0';

        case role is
          when P_NONE =>
            null;                     -- nothing arrived; hold everything

          when P_FIRST =>
            synced  <= '1';
            cur_fid <= pkt_hdr.fid;
            if pkt_hdr.eof = '1' then
              done_r       <= '1';
              ok_r         <= not taint;
              done_bytes_r <= bytes_next;
              frame_open <= '0'; bytes_r <= (others => '0'); dirty <= '0';
            else
              frame_open <= '1'; bytes_r <= bytes_next; dirty <= taint;
            end if;

          when P_NEWFRAME =>
            -- The open frame never saw its EOF, so it is ABANDONED, not
            -- completed. Abort and done are different events, and both fire
            -- in this cycle when the new frame is a single packet.
            if frame_open = '1' then
              abort_r <= '1';
            end if;
            cur_fid <= pkt_hdr.fid;
            if pkt_hdr.eof = '1' then
              done_r       <= '1';
              ok_r         <= not taint;
              done_bytes_r <= len_ext;
              frame_open <= '0'; bytes_r <= (others => '0'); dirty <= '0';
            else
              frame_open <= '1'; bytes_r <= len_ext; dirty <= taint;
            end if;

          when P_CONTINUE =>
            -- A zero-length packet is LEGAL and means "no data this
            -- interval": it must not end the frame and must not taint it.
            if taint = '1' then
              dirty <= '1';
            end if;
            if pkt_hdr.eof = '1' then
              done_r       <= '1';
              ok_r         <= not (dirty or taint);
              done_bytes_r <= bytes_next;
              frame_open <= '0'; bytes_r <= (others => '0'); dirty <= '0';
            else
              bytes_r <= bytes_next;
            end if;
        end case;
      end if;
    end if;
  end process;
end architecture;

The record and the enumeration do what the struct and the typed enum do, with one difference worth noting: VHDL's prole_t is a genuine distinct type with no numeric encoding at all. There is no logic [1:0] underneath it, so there is no way to write a comparison against a raw value, and the case must be exhaustive over the four named values. The SystemVerilog enum has an encoding that can be inspected and, if one is careless, compared against integers.

resize before the add, again. resize(bytes_r, BYTES_W+1) + resize(len_ext, BYTES_W+1) widens both operands to BYTES_W+1 so the carry is a real bit. Adding at BYTES_W and inspecting a carry that was never computed is the standard form of this defect.

null in P_NONE is deliberate and not filler. VHDL requires every case choice to have a statement, so the do nothing branch must be written out. The Verilog and SystemVerilog express the same case by the absence of a branch — which is harder to review, because an omission looks identical to an oversight.

10. Comparing the Three

All three describe the same hardware, and §12's mutation results confirm it: the same six defects die in all three, and after the correction described in §13 the Verilog and SystemVerilog counts agree exactly.

11. The Testbenches

All three benches use the same independence principle that found Chapter 15.3's real bug: the model does not use the design's method.

The design accumulates in a saturating vector and decides new frame inside a three-way role selection. Every bench instead accumulates in an unbounded integer and clamps only at completion, and decides new frame purely from the FID sequence it has observed — never from anything the design computed:

    if (v) begin
      if (!m_synced) begin
        m_synced=1; m_cur_fid=fid; m_open=1; m_dirty=(err|miss); m_bytes=len;
      end else if (fid != m_cur_fid) begin        // the SEQUENCE, not the DUT
        if (m_open) begin e_abort=1; n_abort++; end
        m_cur_fid=fid; m_open=1; m_dirty=(err|miss); m_bytes=len;
      end else begin
        m_bytes += len;                           // UNBOUNDED accumulation
        if (err|miss) m_dirty=1;
      end
      if (eof) begin
        e_done=1;
        e_ok    = ~m_dirty;
        e_bytes = (m_bytes > BYTES_MAX) ? BYTES_MAX : m_bytes;  // clamp LAST
        m_open=0; m_bytes=0; m_dirty=0;
      end
    end

Clamping last rather than continuously is the load-bearing difference. The design saturates on every packet; the model saturates once, at the end. A design that saturated at the wrong threshold, or that saturated and then continued accumulating, cannot be reproduced by a model that only ever clamps the final total.

The directed sequence covers the boundaries before any randomisation:

The randomised phase draws six independent values per interval — valid, length, FID toggle, EOF, error, missing. Chapter 15.4 found a bench that derived several supposedly independent bits from one draw, which made the very coincidence it was meant to exercise unreachable. Here the coincidence that matters — abort and done together — occurred 67 times.

12. Mutation Testing — Across All Three Languages

V1 is the chapter's thesis measured. Removing the FID toggle — leaving a receiver that watches only EOF — costs 1212 failures, more than any other mutation here, and for the reason §4 predicted: the defect does not corrupt one frame, it corrupts every frame after the first loss.

The VHDL counts differ by a few percent because uniform and $random generate different sequences after the identical directed phase. That the three agree on every verdict while disagreeing on every count is the right outcome — it demonstrates equivalent designs driven by genuinely different random stimulus, rather than three transcriptions of one bench.

V6 is worth a second look. A zero-length packet ending the frame costs 1461 failures — the largest count in the table — entirely because §11's realism fix made ZLPs common. Under the original stimulus, with six ZLPs in 6000 intervals, this mutation would have been nearly invisible. The same defect, the same checks, a 200× difference in detection, decided purely by whether the stimulus resembled a real camera.

13. The Mutation That Found a Defect in the RTL

The first run of this matrix did not look like the table above. It looked like this:

V3   the header error bit is ignored      20     34     36
V5   a host-lost interval is ignored      12     27     30

The Verilog was detecting roughly half of what the other two detected, on two mutations that were supposed to be identical. Three explanations were possible: a semantic difference between the languages, a weaker Verilog testbench, or a difference in the designs themselves.

It was the third, and it was my own defect. Counting how many times each implementation wrote the taint condition:

  Verilog       : 6
  SystemVerilog : 1
  VHDL          : 1

The SystemVerilog and VHDL name the condition once as a signal. The Verilog wrote pkt_err | pkt_missing inline at every use. So the same textual mutation changed all uses in two languages and one of several in the third — the Verilog mutant was simply a weaker mutant, describing a design in which only continuation packets ignored the error bit while first and new-frame packets still honoured it.

Factoring the Verilog to name taint once, exactly as the other two do, and re-measuring:

  V3   Verilog now: 34   (was 20, SV=34, VHDL=36)
  V5   Verilog now: 27   (was 12, SV=27, VHDL=30)

Exact convergence with SystemVerilog.

The general lesson: an unexplained difference between two measurements of the same thing is data. The temptation with V3 at 20 against 34 is to record that both were killed and move on. The number was trying to say something.

14. The Waveform

This waveform is in the controller clock domain, one column per service interval. It is not bus signalling: pkt_valid is the controller's indication that a payload arrived and was header-parsed, not a shape on D+/D−. Reading it as wire timing would suggest a microframe is one clock.

Interval 6 is the entire chapter. frame_abort asserts and frame_done does not. A receiver without FID would show nothing at interval 6, would keep open high, and would eventually emit a single frame of 2000 + 800 + 200 = 3000 bytes containing the tail of one image and the head of another — and would then be permanently one frame out of step.

Interval 9 is the control case. FID toggles again, but the previous frame completed normally at interval 7, so open was already low and no abort is asserted. Mutation V4 removes exactly that if (open) qualification and costs 815 failures.

15. Assertions

  // A1. SAFETY: done and abort describe DIFFERENT frames, so a frame that
  //     ended at EOF is never also reported abandoned. They may coincide in
  //     a cycle, but never for the same frame -- expressed here as: an abort
  //     implies a frame was open and this packet did NOT continue it.
  property p_abort_means_open_and_toggled;
    @(posedge clk) disable iff (!rst_n || bus_reset)
      frame_abort |-> $past(open_q) && $past(fid_toggled);
  endproperty
  a_abort_means_open_and_toggled: assert property (p_abort_means_open_and_toggled);

  // A2. SAFETY, and the property that makes FID worth having: after an
  //     abort, the new frame's byte count contains NONE of the abandoned
  //     frame's bytes. Checked against the bench's own packet observation.
  property p_no_byte_leak;
    @(posedge clk) disable iff (!rst_n || bus_reset)
      frame_done |-> (frame_bytes == bytes_observed_since_frame_start);
  endproperty
  a_no_byte_leak: assert property (p_no_byte_leak);

  // A3. SAFETY: a zero-length packet is inert -- it ends nothing.
  property p_zlp_inert;
    @(posedge clk) disable iff (!rst_n || bus_reset)
      (pkt_valid && pkt_len == 0 && !pkt_eof && !fid_toggled)
        |=> !frame_done && !frame_abort;
  endproperty
  a_zlp_inert: assert property (p_zlp_inert);

  // A4. PROGRESS: a stream that keeps delivering EOFs keeps producing
  //     frames. A design that never completes anything satisfies every
  //     safety property above and is useless.
  property p_eof_produces_frame;
    @(posedge clk) disable iff (!rst_n || bus_reset)
      (pkt_valid && pkt_eof && have_sync) |=> frame_done;
  endproperty
  a_eof_produces_frame: assert property (p_eof_produces_frame);

  // A5. BOUNDEDNESS: the byte count never wraps. Stated as monotonicity
  //     within a frame, which a wrapping counter violates immediately.
  property p_bytes_monotonic;
    @(posedge clk) disable iff (!rst_n || bus_reset)
      (pkt_valid && !fid_toggled && !pkt_eof && open_q)
        |=> (bytes_r >= $past(bytes_r));
  endproperty
  a_bytes_monotonic: assert property (p_bytes_monotonic);

  // A6. RESET: a bus reset emits nothing, ever. The partial frame is
  //     discarded rather than completed.
  property p_reset_emits_nothing;
    @(posedge clk)
      bus_reset |=> !frame_done && !frame_abort && !have_sync;
  endproperty
  a_reset_emits_nothing: assert property (p_reset_emits_nothing);

Assertion contracts

A3's vacuity row is the one to study, because it is §11's finding restated as an assertion contract. The property was correct and non-trivial the whole time, and it was very nearly vacuous — six activations in 6000 intervals — for a reason no check could report: the stimulus did not resemble the device. A vacuity review that only asks "did the antecedent ever hold?" would have answered yes and moved on. The useful question is how often, and is that representative of the real input.

A2 is deliberately phrased against the bench's own observation, bytes_observed_since_frame_start, rather than against the design's bytes_r. §29 of this standard warns against properties phrased in terms of the decision under test — and bytes_r is exactly what mutation V2 corrupts.

16. Verification: Where UVM Starts to Earn Its Place

Chapter 16.1 declined UVM and said so explicitly: a single scalar published every 2^K frames does not justify an agent, a driver and a factory. This chapter is where the justification arrives, and it is worth being precise about what changed.

What changed is that the correctness property spans a structure, not a value. The question is no longer is this number right but did this stream of hundreds of packets partition correctly into frames — a property about grouping, ordering, and recovery after loss.

The coverage model is where this environment earns the most, because §11's two findings were both coverage questions that no check could raise:

  cross: packet_length_class   × frame_position
         (zero, small, max)      (first, interior, last)
  cross: frame_outcome         × taint_source
         (done, abort)           (clean, header_err, host_lost)
  cross: toggle_coincides_eof  × frame_was_open
         (yes, no)               (yes, no)        -- the abort+done case
  bin:   frames_ending_in_abort, consecutive aborts
  bin:   byte_count_saturated

The third cross is the one a hand-written bench keeps missing. Abort-and-done in the same cycle occurred 67 times here and only because six independent draws made it likely; in an environment where frames are generated as units, it happens whenever a single-packet frame follows a lossy one — which a constrained-random frame-size distribution produces on its own.

The monitor's independence is not stylistic. If the monitor is handed frame boundaries by the sequence, the scoreboard compares the generator against itself and mutation V1 — the entire point of this chapter — becomes invisible, because both sides would agree that a frame ended where the generator said it did. The monitor must derive boundaries from FID and EOF exactly as the design does, and the reference model must derive them from the packet list. Two different derivations of the same fact; that is what makes the comparison meaningful.

And the reference model must not saturate. §11's benches accumulate in an unbounded integer and clamp once. A reference model that copied the design's per-packet saturation would agree with a design that saturated at the wrong threshold — the Module 14 and 15.3 lesson, restated for a structure rather than a scalar.

17. Debugging: the Camera That Tears After a Dropped Packet

A UVC webcam works correctly at 640×480. At 1920×1080 the image is usually fine, but occasionally the picture tears — the top of one frame sits above the bottom of a different one — and once it starts, every subsequent frame is torn until the stream is restarted. Bus analysers show occasional missing packets, which is expected on a busy bus.

Read the persistence before anything else. Once it starts, it never recovers is the signature §4 describes exactly: a receiver whose frame boundary has shifted and has no independent evidence to shift it back. A transient error that produces a permanent symptom means the recovery mechanism is missing, not that the errors are unusual.

And the resolution dependence is a clue, not a red herring. At 1080p a frame is ~400 packets rather than ~40, so the probability that any given frame loses a packet is ten times higher — and the probability that the lost one is the EOF packet rises with it. The bug exists at 640×480 too; it just fires ten times less often.

The chain, from the outside in:

Protocol analyser — are packets actually missing? Confirm the premise. If the host reports lost intervals, note whether the losses correlate with bus load (expected) or with a fixed period (a scheduling problem, Chapter 17 territory).

Protocol analyser — does the device set FID at all? Decode the payload header of consecutive frames. A device that leaves FID constant has made the mechanism unavailable to every host, and no amount of host-side work can recover boundaries that were never marked. This is a firmware bug, and it is common.

Controller state — does frame_abort ever assert? If FID is toggling on the wire and the controller never aborts, the design is mutation V1 or V4: either the toggle is ignored, or the abort is computed and not signalled. Both look identical from outside and are distinguished immediately by whether cur_fid tracks the wire.

RTL — the first divergence. Compare the bench's independent frame partition against the design's, packet by packet. The first interval where they disagree is the bug. In practice it is one of four: the toggle not qualified by synced (the first frame aborts spuriously), the abort not qualified by open (every toggle aborts), bytes not cleared at the boundary (V2 — frames grow monotonically), or the toggle ignored entirely (V1).

18. Common Misconceptions

"EOF marks the end of a frame, so FID is redundant." EOF is carried inside the frame it terminates, so losing that packet loses the boundary (§4). FID is carried by every packet.

"A lost packet corrupts one frame." With FID, yes. Without it, it corrupts every frame afterwards (§4's table).

"FID tells you how many frames were lost." It is one bit. A whole missing frame produces a toggle indistinguishable from a normal boundary (§4). Chapter 16.3 needs that count and must obtain it elsewhere.

"A zero-length packet means something went wrong." It means no data this service interval and is completely normal during blanking (§6). Mutation V6 treats it as a frame terminator and costs 1461 failures.

"An incomplete frame should be passed on so the decoder can salvage it." A compressed frame missing its tail is generally undecodable, and presenting it invites a decoder to produce garbage rather than conceal (§5). frame_abort exists to say this is not a frame.

"frame_done and frame_abort are mutually exclusive." They describe different frames and can fire in the same cycle (§5) — 67 times in the measured run.

"Uncompressed 1080p video works over USB 2.0 isochronous." It needs 124.4 MB/s against a 24.6 MB/s ceiling (§1).

19. Exercises

1. A 1080p MJPEG frame averages 400 kB at 30 fps. Compute the packets per frame at 1024 bytes each, the required data rate, and the fraction of the high-bandwidth isochronous ceiling it consumes. Then compute what frame rate would saturate it.

2. Extend the design to count consecutive aborts and assert a resync_lost output after N in a row. Decide what N should be and justify it from §4's observation that FID cannot distinguish one lost frame from three.

3. §13 found a condition duplicated six times in the Verilog. Search the other two implementations in this module for any condition written more than once, and say whether the mutation matrix would have revealed it.

4. Implement mutation V1 in the VHDL by hand, then predict — before running — whether its failure count will be closer to 1212 or to 1171, and explain which property of the benches decides that.

5. Write an SVA property that catches V2 (bytes leaking across a boundary) without referring to bytes_r. Then explain why A2 as written in §15 catches it and A5 does not.

6. The design assumes exactly one packet per service interval. High-bandwidth endpoints deliver up to three. Determine what must change, and whether frame_abort can now fire more than once in an interval.

7. §11's ZLP stimulus was wrong in a way no check could detect. Propose a standing mechanism — not a one-off measurement — that would have flagged it in a regression, and say which section of the UVM environment in §16 provides it.

20. Summary

High-bandwidth isochronous tops out near 24.6 MB/s (§1), so video is always compressed and therefore variable in size — which rules out counting packets and forces the receiver to detect boundaries from the stream itself.

A frame spans hundreds of packets, and any of them may be lost (§2). Video's failure mode is not audio's slow drift but loss of synchronisation, which does not self-correct.

EOF alone is a fragile boundary because it is carried by the packet it delimits (§4). Losing that one packet merges two frames — and then every frame after, permanently. FID, one bit in every packet, makes the boundary recoverable from any surviving packet of the next frame, converting an unbounded failure into the loss of exactly one frame.

Completed and abandoned are different events (§5), and a packet that toggles FID while carrying EOF causes both in one interval, for two different frames. A single completion output cannot express that.

All three HDL implementations model the same hardware and all three were simulated, including the VHDL (§21). Six mutations died in all three languages (§12), and V1 — removing the FID toggle — is the largest count in the table at 1212, exactly as §4 predicts for a defect that corrupts the stream rather than a frame.

Two findings came from measuring what the stimulus reached rather than from any check (§11). Zero-length packets occurred 6 times in 6000 intervals because the length distribution described a camera that does not exist; fixing it to a realistic quarter raised mutation V6's detectability by two orders of magnitude. And a 16 MB frame is unreachable by any random stream, so saturation is a directed test or it is untested.

And the mutation matrix found a defect in my own RTL (§13). An unexplained asymmetry — the Verilog detecting 20 where the others detected 34 — turned out to be a taint condition written inline six times in the Verilog against once in the other two. Factoring it made the counts converge exactly with SystemVerilog. Every test had passed; nothing else in the flow would have surfaced it.

21. Tooling, Honestly

Icarus also emitted a sensitivity warning on the SystemVerilog — sorry: constant selects in always_* processes are not fully supported (the process will be sensitive to all bits in 'pkt_hdr[2:0]'). That is Icarus being conservative: it makes the block sensitive to more than the code selects, which cannot cause a missed event. It is noted here because a warning that is safe in one tool is not automatically safe in another, and a synthesis tool's inferred sensitivity is a different question entirely.

22. What Comes Next

This chapter's receiver could always tell that a frame boundary occurred. FID toggles, and the boundary is unambiguous.

What it could never tell is how many boundaries it missed. §4 said so directly: one bit cannot distinguish one lost frame from three, because the toggle looks identical either way. For video that is tolerable — the display shows the frames that arrive and the viewer sees a stutter.

Chapter 16.3 cannot tolerate it. A sensor stream, a scientific instrument, a data acquisition front end — these produce samples whose position in time is part of the measurement, and a gap whose size is unknown corrupts everything after it. The receiver must know not just that data is missing but exactly how much, and it must know it without a retry, without a handshake, and without the sender ever learning that anything was lost.

Browse the full path on the USB tutorials index.