The Response Channel (RSP)

Module 6 · Chapter 6.3 · CHI Channels

Project thread — 6.2 opened REQ, where transactions begin. This chapter opens RSP, where they are steered and finished. 6.4 takes the data channel, DAT.

1. Learning Outcomes

By the end of this chapter you should be able to:

• List the key fields of an RSP packet — response opcode, SrcID, TgtID, TxnID, Resp, RespErr.
• Name the response types RSP carries — Comp, CompAck, DBIDResp, RetryAck, PCrdGrant, SnpResp.
• Explain completion semantics: a read completes on data + completion.
• Distinguish combined completion (CompData on DAT) from separate completion (Comp on RSP).
• Describe the Comp → CompAck handshake and how it orders a transaction.
• Implement a representative completion tracker handling either completion form in SystemVerilog, Verilog-2001, and VHDL.

2. Why Should I Learn This?

RSP is the channel that says "done", "retry", "here is your buffer", and "here is my snoop result" — the control backbone of every transaction. Reading it is how you know a transaction's outcome and when it is truly complete.

Its subtlety is completion. A read is not finished when its data arrives — it is finished when both the data and the completion have arrived, and CHI lets completion arrive two ways: bundled with the data, or as a separate Comp on RSP. Handling only one form is a classic hang bug. Getting the completion contract right is the core skill this chapter builds.

3. Key Terms

4. Previous Chapter Connection

Chapter 6.2 detailed REQ — the packet that starts a transaction, its routing to the Home Node, and the RetryAck/PCrdGrant retry. Those two retry messages actually travel on RSP; you met them there as REQ's partners.

This chapter opens RSP fully. It details every data-less response — the completions, acknowledgements, buffer grants, retries, and snoop responses — and the completion semantics that tell a requester when a transaction is genuinely done. Where 6.2 was the request side, 6.3 is the response side, and together they frame the control flow of every CHI transaction.

5. Core Concept — data-less responses and completion

RSP is the channel for every response that carries no payload.

• The packet. An RSP flit carries a response opcode (Comp, CompAck, DBIDResp…), SrcID / TgtID (routing), TxnID (which transaction it answers), a Resp field (a coherence result, e.g. a snoop's resulting state), and RespErr (OK / error). No data — data is DAT's job.
• The response types. RSP carries Comp (completion), CompAck (the requester's ack), DBIDResp / CompDBIDResp (grant a write buffer, alone or with completion), RetryAck and PCrdGrant (the retry pair, Chapter 6.2), SnpResp (a snoop response with the resulting state), and ReadReceipt (a request-accepted acknowledgement).
• Completion semantics. A read is complete only when both its data (DAT) and its completion have arrived. Completion may come combined with the data — CompData on DAT carries data and completion — or separate — a standalone Comp on RSP alongside data-only beats. Completion logic must accept either form.
• Ordering via CompAck. After completion, the requester sends CompAck on RSP; the Home Node uses it to finalize ordering and release the address (Chapters 4.3, 5.3). Comp → CompAck is the handshake that orders a transaction.

The synthesis:

RSP carries the outcomes: completion, acknowledgement, buffer grants, retries, snoop results — everything without a payload. Its defining rule is that a transaction completes on data plus completion, and completion may be combined (CompData) or separate (Comp) — so treat the two as interchangeable arrivals. Read RSP to know a transaction's result and when it is truly finished.

6. Engineering Mental Model — the receipts and confirmations

If REQ was the work order dropped in the mail (Chapter 6.2), RSP is the stream of receipts and confirmations that come back.

• A "done" slip (Comp) says the job reached its commit point. A "got it" reply (CompAck) from you closes the loop so the office can move on (order the next job).
• A "here is your shelf number" note (DBIDResp) tells you where to deliver your goods (write data on DAT).
• A "we are full, hold" / "go ahead now" pair (RetryAck / PCrdGrant) paces you.
• A "checked — here is its status" reply (SnpResp) reports what a snooped party held.

Crucially, the "done" confirmation can ride along with the delivered goods (CompData) or arrive as its own slip (separate Comp). You are finished when you have both the goods and a "done" — never assume which envelope the "done" rides in.

7. Engineering Diagram — the RSP packet

Two rows of fields, no payload. The opcode says what response; TxnID says for which transaction; Resp and RespErr say the outcome.

8. Response Types on RSP

The response opcode selects one of several outcomes. A representative set:

Two facts to carry: only DAT carries data — every RSP type is control — and RSP is bidirectional (Comp/DBIDResp flow home-to-requester; CompAck/SnpResp flow requester/snooped-to-home). The opcode names the outcome; the TxnID ties it to its transaction.

9. Completion Semantics

The heart of RSP is when a transaction is done. For a read:

• Data and completion are separate concerns. The requester needs the data (from DAT) and a completion signal. A read is finished only when it has both.
• Completion can be combined with data. A CompData packet on DAT carries the data and the completion together — one arrival satisfies both.
• Or completion can be separate. The Home Node may send DataSepResp (data only) on DAT and a standalone Comp on RSP — two arrivals, in any order.
• Then CompAck orders it. Once complete, the requester sends CompAck on RSP; the Home Node uses it to finalize ordering and release the line (Chapters 4.3, 5.3).

The point to carry:

Completion is data + a completion signal, and the completion signal may be combined (CompData) or separate (Comp). Completion logic must treat both forms as equivalent arrivals — latch data and completion independently, and finish when both are in. Assuming completion always arrives one way is the classic RSP bug (the DebugLab). Then close ordering with CompAck.

10. The RSP Handshake

A write shows RSP's response flow clearly: buffer grant, then completion, then acknowledgement.

Three RSP messages carried the write's control: grant, complete, acknowledge. The data itself was one DAT beat. RSP is the outcome channel; DAT is the payload channel.

11. RTL / Hardware View — a completion tracker for either form

Completion semantics in RTL: a read finishes on data plus completion, where completion may be combined (CompData) or separate (Comp). Here is that tracker. Representative and simplified.

// Representative read-completion tracker (educational).
// A read completes when it has DATA and a COMPLETION. Completion arrives either
// combined with data (CompData: sets both) or separate (Comp on RSP + data-only
// beats on DAT). Latch data and completion independently; done when both are in.
module read_completion (
  input  logic clk,
  input  logic rst_n,
  input  logic start,          // read issued (clears the tracker)
  input  logic comp_data,      // DAT: CompData — data AND completion
  input  logic data_sep,       // DAT: DataSepResp — data only
  input  logic comp_sep,       // RSP: Comp — completion only
  output logic done            // data + completion received -> complete
);
  logic got_data, got_comp;
  always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n || start) begin
      got_data <= 1'b0; got_comp <= 1'b0;
    end else begin
      if (comp_data) begin got_data <= 1'b1; got_comp <= 1'b1; end // combined
      if (data_sep)  got_data <= 1'b1;                              // data only
      if (comp_sep)  got_comp <= 1'b1;                              // completion only
    end
  end
  assign done = got_data && got_comp;
endmodule

The same behavior in Verilog-2001:

// Representative read-completion tracker (Verilog-2001).
module read_completion (
  input  clk, rst_n, start, comp_data, data_sep, comp_sep,
  output done
);
  reg got_data, got_comp;
  always @(posedge clk or negedge rst_n)
    if (!rst_n || start) begin got_data <= 1'b0; got_comp <= 1'b0; end
    else begin
      if (comp_data) begin got_data <= 1'b1; got_comp <= 1'b1; end
      if (data_sep)  got_data <= 1'b1;
      if (comp_sep)  got_comp <= 1'b1;
    end
  assign done = got_data && got_comp;
endmodule

And in VHDL:

-- Representative read-completion tracker (VHDL).
library ieee;
use ieee.std_logic_1164.all;
 
entity read_completion is
  port (
    clk, rst_n, start, comp_data, data_sep, comp_sep : in  std_logic;
    done                                             : out std_logic
  );
end entity;
 
architecture rtl of read_completion is
  signal got_data, got_comp : std_logic := '0';
begin
  process(clk, rst_n)
  begin
    if rst_n = '0' then
      got_data <= '0'; got_comp <= '0';
    elsif rising_edge(clk) then
      if start = '1' then
        got_data <= '0'; got_comp <= '0';
      else
        if comp_data = '1' then got_data <= '1'; got_comp <= '1'; end if;  -- combined
        if data_sep  = '1' then got_data <= '1'; end if;                    -- data only
        if comp_sep  = '1' then got_comp <= '1'; end if;                    -- completion only
      end if;
    end if;
  end process;
 
  done <= got_data and got_comp;
end architecture;

All three accept completion from either a combined CompData or a separate Comp, and finish on data-plus-completion. That both forms feed the same got_comp is the completion contract; omitting one is the DebugLab.

12. Verification View — completion needs data and a completion, either form

Two properties: completion requires both, and a separate Comp completes just as a combined CompData does.

// Bind to read_completion.
// 1. Done only when both data and a completion have been received.
property p_both_required;
  @(posedge clk) disable iff (!rst_n) done |-> (got_data && got_comp);
endproperty
assert property (p_both_required);
 
// 2. Either completion form satisfies completion: CompData OR a separate Comp
//    sets got_comp — the two are interchangeable arrivals.
property p_either_completion_form;
  @(posedge clk) disable iff (!rst_n)
    (comp_data || comp_sep) |=> got_comp;
endproperty
assert property (p_either_completion_form);

The system point, beyond the two checks:

CHI lets the Home Node deliver completion whichever way is efficient — bundled with data when it has both ready (CompData), or separately when data and ordering resolve at different times (DataSepResp + Comp). A correct requester is agnostic to that choice: it latches data and completion independently and finishes on the pair. Hard-coding one form couples the requester to a delivery decision that is not its to make, and the first time the fabric picks the other form, the read hangs waiting for a completion that already arrived — in the shape it ignored.

• What it proves: completion needs both signals; either completion form counts.
• What it does not prove: the full transaction (this is the completion condition only).
• Bug signature: a read that hangs when completion arrived in the unhandled form (the DebugLab).

13. Testbench — complete via combined and via separate

Runs a read completed by CompData, and another completed by DataSepResp + a separate Comp.

module tb_read_completion;
  logic clk = 0, rst_n, start, comp_data, data_sep, comp_sep, done;
  int errors = 0;
 
  read_completion dut (.*);
  always #5 clk = ~clk;
 
  task automatic pulse(output logic sig); sig = 1; @(posedge clk); #1; sig = 0; endtask
 
  initial begin
    rst_n = 0; start = 0; comp_data = 0; data_sep = 0; comp_sep = 0;
    @(posedge clk); rst_n = 1;
 
    // Combined: one CompData satisfies data + completion.
    start = 1; @(posedge clk); #1; start = 0;
    pulse(comp_data);
    if (!done) begin errors++; $display("FAIL combined CompData did not complete"); end
    else $display("PASS combined CompData completed");
 
    // Separate: data-only then a separate Comp (either order).
    start = 1; @(posedge clk); #1; start = 0;
    pulse(comp_sep);                 // completion arrives first
    if (done) begin errors++; $display("FAIL completed with no data"); end
    else $display("PASS not done on Comp alone");
    pulse(data_sep);                 // then the data
    if (!done) begin errors++; $display("FAIL separate Comp+data did not complete"); end
    else $display("PASS separate Comp + data completed");
 
    if (errors == 0) $display("ALL TESTS PASSED");
    else             $display("%0d FAILURE(S)", errors);
    $finish;
  end
endmodule

Expected output:

PASS combined CompData completed
PASS not done on Comp alone
PASS separate Comp + data completed

14. DebugLab — completing only on combined CompData

cyc  channel  message         got_data  got_comp  note
 10  DAT      DataSepResp        1          0      data recorded
 12  RSP      Comp               1          0      SEPARATE Comp ignored -> not latched
 ..  (wait)                      1          0      done never asserts -> hang

15. Common Mistakes

• Completing only on combined CompData. Assumption: completion always rides the data. Bug: missed separate Comp, hang (the DebugLab). Prevention: accept either completion form.
• Looking for data on RSP. Assumption: responses carry payload. Bug: reading nothing. Prevention: RSP is data-less; data is on DAT.
• Skipping CompAck. Assumption: the read ends at completion. Bug: the Home Node never orders/releases the line. Prevention: send CompAck where required.
• Ignoring RespErr. Assumption: every response is OK. Bug: acting on errored data. Prevention: check RespErr.
• Mishandling DBIDResp. Assumption: send write data whenever ready. Bug: data with no buffer (Chapter 5.5). Prevention: send WriteData only after DBIDResp.
• Assuming a fixed RSP order. Assumption: responses arrive in a set sequence. Bug: mis-correlated outcomes. Prevention: correlate by TxnID; order is not guaranteed.

16. Engineering Checklist

• Read the RSP opcode (Comp / CompAck / DBIDResp / RetryAck / PCrdGrant / SnpResp).
• Complete a read on data + completion, accepting combined or separate completion.
• Send CompAck to close ordering where the transaction requires it.
• Send WriteData only after a DBIDResp buffer grant.
• Check RespErr on every response.
• Correlate every RSP by TxnID — never assume arrival order.

17. Key Takeaways

• RSP carries data-less responses: Comp, CompAck, DBIDResp, RetryAck, PCrdGrant, SnpResp.
• Its fields: response opcode, SrcID/TgtID, TxnID, Resp (state), RespErr.
• A read completes on data + completion; completion may be combined (CompData on DAT) or separate (Comp on RSP).
• Completion logic must accept either form — latch data and completion independently.
• Comp → CompAck orders a transaction; the Home Node releases the line on CompAck.
• RSP is the outcome channel, DAT the payload channel; the model here is representative.

18. Quick Revision

The RSP channel. Carries data-less responses. Fields: response opcode, SrcID/TgtID, TxnID (which transaction), Resp (a coherence-state result), RespErr (OK/error). Types: Comp (completion), CompAck (requester's ack, closes ordering), DBIDResp / CompDBIDResp (write-buffer grant), RetryAck / PCrdGrant (retry pair), SnpResp (snoop result). Completion semantics: a read completes on data + a completion, and completion may be combined with data (CompData on DAT) or separate (Comp on RSP) — completion logic must accept either, latching data and completion independently. Then CompAck finalizes ordering (the Home Node releases the line). RSP is the outcome channel; only DAT carries payload. Representative model; 6.4 covers DAT.

Coming Next

Chapter 6.4 — The Data Channel (DAT). RSP carries outcomes without payload; DAT carries the payload itself. The next chapter details the data channel: its packet structure, how a cache line moves as multiple beats, the byte-mask handling that lets a partial write touch only some bytes, and the data opcodes (CompData, WriteData, snoop data) that connect back to the completions you just met on RSP. It is the one channel with bulk data, and the last of the four.