Barrier Transactions

Module 12 · Chapter 12.3 · Ordering and Consistency

Project thread — 12.1 called for barriers; this chapter introduces the barrier transaction. 12.4 and 12.5 detail DMB and DSB.

1. Learning Outcomes

By the end of this chapter you should be able to:

• Describe a barrier as a first-class CHI operation the interconnect carries.
• State what a barrier enforces — before-accesses ordered ahead of after-accesses.
• Relate the DMB / DSB instructions to CHI-level barrier behaviour.
• Explain why a barrier must be honoured at every ordering point the accesses traverse.
• Diagnose the partial ordering when a barrier is dropped somewhere in the fabric.
• Implement a representative barrier model in SystemVerilog, Verilog-2001, and VHDL.

2. Why Should I Learn This?

Chapter 12.1 established that cross-address ordering is the programmer's job, expressed with a barrier — but a barrier is only useful if the hardware actually enforces it, and enforcing it in a distributed interconnect is not trivial. A barrier is not a local hint that one core reorders around; it is an ordering constraint that must hold across the fabric, between accesses that may route to different homes and through different paths. CHI makes the barrier a transaction so that ordering can be carried where it is needed.

The lesson is about completeness of enforcement. A barrier's guarantee is global — every observer must see before-accesses ahead of after-accesses — so it must be respected at every point that orders those accesses. A single component that ignores or drops the barrier creates a hole through which reordering escapes, and the software's carefully placed barrier silently fails for the paths through that component. Understanding barrier transactions is understanding how ordering is propagated, and why partial enforcement is no enforcement.

3. Key Terms

4. Previous Chapter Connection

Chapter 12.1 showed that cross-address ordering is weak and must be requested with a barrier; Chapter 12.2 showed how the home serializes conflicting transactions per line. This chapter is the barrier those chapters pointed to — the mechanism that imposes an order across addresses.

Serialization (12.2) ordered transactions to one line; a barrier orders accesses across different lines and homes, which serialization alone cannot. To do that in a distributed fabric, CHI carries the barrier as a transaction so the ordering constraint reaches every relevant point. So the DMB and DSB a program issues (Chapters 12.4, 12.5) become barrier transactions the interconnect propagates. This chapter frames the barrier as that propagated operation, and the next two detail the two flavours.

5. Core Concept — a barrier the interconnect carries and every point honours

A barrier enforces an order across addresses, and CHI carries it as a transaction so that order holds across the whole fabric.

• What it enforces. A barrier splits an agent's accesses into before and after: every before-access is ordered ahead of every after-access, as observed by all other agents. It ties otherwise-unordered cross-address accesses into a sequence.
• A first-class transaction. CHI does not treat a barrier as a local reordering hint; it carries it as an operation across the interconnect, so the constraint reaches the ordering points the accesses use.
• From DMB/DSB. The DMB and DSB instructions a program executes map to CHI-level barrier behaviour — the hardware realizes the instruction's ordering by propagating a barrier (Chapters 12.4, 12.5).
• Honoured everywhere. The guarantee is global, so the barrier must be respected at every ordering point the before- and after-accesses traverse — every home, bridge, or serialization stage. Miss one and ordering leaks there.

The synthesis:

A barrier orders before-accesses ahead of after-accesses, as observed by all agents — the cross-address order Chapter 12.1 said the weak model omits. CHI carries it as a first-class transaction so the order holds across the fabric, realizing the DMB/DSB instructions above. Because the guarantee is global, the barrier must be honoured at every ordering point the accesses traverse; a single point that drops it leaves reordering on the paths through it.

6. Engineering Mental Model — a checkpoint on every road

A barrier is a checkpoint that must stand on every road out of a city.

• The rule: all "before" trucks must clear the city before any "after" truck may leave. To enforce it, you place a checkpoint that holds "after" trucks until the "before" trucks have passed.
• But the city has many roads. A checkpoint on the main highway is not enough — an "after" truck could slip out on a side road that has no checkpoint, overtaking a "before" truck still en route. The order is broken on that road.
• So the checkpoint rule must be enforced on every road the trucks might take. Only then is the guarantee — all before-trucks ahead of all after-trucks — true for every observer, no matter which road they watch.

One un-manned road defeats the whole scheme. A barrier is the same: it must be honoured at every ordering point the accesses can pass through, or reordering escapes by the unguarded path.

7. Engineering Diagram — a barrier ordering before ahead of after

Read the last message: the observer sees @A before @B because the barrier, carried by the interconnect, ordered them. The barrier sits between the two accesses and is honoured across the fabric — so no observer can see @B before @A.

8. What a Barrier Orders

The barrier's guarantee, itemized.

The rule to carry: a barrier ties before-accesses ahead of after-accesses, as every observer sees it — the one thing per-address coherence cannot do. CHI makes it a carried transaction so the ordering reaches every point the accesses use, and it realizes the program's DMB/DSB. The guarantee is only as strong as its weakest ordering point: honoured everywhere, it holds; dropped anywhere, it leaks there.

9. A Barrier Must Be Honoured Everywhere

The completeness requirement deserves its own statement.

• The guarantee is global. "Before ahead of after, for all observers" is a statement about the whole system, not one agent — so it must hold on every path.
• Every ordering point counts. Homes, bridges, and serialization stages each order the accesses that pass through them; each must honour the barrier for the accesses it handles.
• A dropped barrier is a hole. A component that ignores the barrier does not order the accesses routing through it — reordering survives on those paths, invisible until an observer watches them.
• Propagation is the mechanism. CHI carries the barrier so it reaches every relevant point; the point must then honour it. Both carrying and honouring are required.

The point to carry:

A barrier is a conjunction: it holds only if every ordering point in the accesses' paths respects it. That makes partial enforcement worthless — a barrier honoured at nine points out of ten still permits reordering through the tenth, and an observer watching that path sees the un-ordered result. So a barrier is not "mostly" enforced or "usually" enforced; it is enforced iff it is enforced everywhere. This is why CHI carries it as a transaction that propagates: the design must guarantee the barrier reaches and is honoured at each point the before- and after-accesses can traverse, because the software's correctness rests on the weakest link in that chain.

10. Reading the Barrier — a handoff across the fabric

Trace a barrier enforcing a cross-home handoff.

1. Before-access. The agent writes data at A, homed at HN1.
2. Barrier. The agent issues a barrier, carried into the interconnect. It marks the boundary: A's write must be ordered before anything after.
3. After-access. The agent writes a flag at B, homed at HN2 — a different home.
4. Propagation. The barrier is honoured at both HN1 and HN2 (and any bridge between), so the write to A is ordered ahead of the write to B wherever an observer looks.
5. Observer. Any agent that sees the flag at B set also sees the data at A written — the cross-address order holds across two homes.

The handoff worked because the barrier was honoured at both homes the accesses used. Had HN2 ignored the barrier, an observer reading through HN2 could see the flag before the data — the DebugLab.

11. RTL / Hardware View — a barrier model

A barrier gates the after-accesses on the before-accesses being ordered, and only counts as enforced when honoured at every point. Representative.

// Representative barrier model (educational).
// A barrier orders before-accesses ahead of after-accesses. An after-access is
// ordered only once the before-accesses are ordered AND the barrier is honoured at
// every ordering point it traverses. Missing any point -> not fully enforced.
module chi_barrier (
  input  logic before_ordered,      // the before-accesses are ordered/observed
  input  logic barrier_issued,      // a barrier was placed between before and after
  input  logic honoured_all_points, // the barrier is honoured at EVERY ordering point
  output logic after_may_pass,      // an after-access may be ordered after the barrier
  output logic fully_enforced,      // the cross-address order is guaranteed everywhere
  output logic partial               // barrier issued but not honoured everywhere
);
  // Enforced only if issued, the before is ordered, and every point honours it.
  assign fully_enforced = barrier_issued && before_ordered && honoured_all_points;
  // After-accesses proceed in order once the barrier is fully enforced.
  assign after_may_pass = fully_enforced;
  // A barrier that is issued but not honoured everywhere enforces only partially.
  assign partial = barrier_issued && !honoured_all_points;
endmodule

The same behavior in Verilog-2001:

// Representative barrier model (Verilog-2001).
module chi_barrier (
  input  before_ordered, barrier_issued, honoured_all_points,
  output after_may_pass, fully_enforced, partial
);
  assign fully_enforced = barrier_issued && before_ordered && honoured_all_points;
  assign after_may_pass = fully_enforced;
  assign partial        = barrier_issued && !honoured_all_points;
endmodule

And in VHDL:

-- Representative barrier model (VHDL).
library ieee;
use ieee.std_logic_1164.all;
 
entity chi_barrier is
  port (
    before_ordered      : in  std_logic;
    barrier_issued      : in  std_logic;
    honoured_all_points : in  std_logic;
    after_may_pass      : out std_logic;
    fully_enforced      : out std_logic;
    partial             : out std_logic
  );
end entity;
 
architecture rtl of chi_barrier is
  signal fe : std_logic;
begin
  fe <= barrier_issued and before_ordered and honoured_all_points;
  fully_enforced <= fe;
  after_may_pass <= fe;
  partial        <= barrier_issued and (not honoured_all_points);
end architecture;

All three treat a barrier as enforced only when issued, the before-accesses are ordered, and every point honours it — and flag partial when it is honoured at some points but not all. The DebugLab is the partial case.

12. Verification View — enforcement is all-or-nothing

The properties that capture a barrier's completeness: full enforcement requires honouring at every point.

// Bind to chi_barrier.
// 1. Full enforcement requires the barrier issued, before ordered, and all points honouring.
property p_full_requires_all;
  @(*) fully_enforced == (barrier_issued && before_ordered && honoured_all_points);
endproperty
 
// 2. An after-access may pass only when the barrier is fully enforced.
property p_after_gated;
  @(*) after_may_pass |-> fully_enforced;
endproperty
 
// 3. A barrier issued but not honoured everywhere is only partial (not enforced).
property p_partial_not_enforced;
  @(*) (barrier_issued && !honoured_all_points) |-> (partial && !fully_enforced);
endproperty

The system point, beyond the checks:

A barrier's semantics are global, which makes its implementation a distributed-systems problem: a global ordering constraint must be realized by many local components each doing their part, and the constraint holds only if all of them do. This is why CHI elevates the barrier to a transaction — a local hint could never reach the remote ordering points that also handle the accesses. The correctness condition is therefore a universal quantifier over ordering points: for the barrier to mean anything, every point the accesses traverse must honour it. Verifying a barrier is verifying that universal — not that it works on the common path, but that no path exists where it is dropped. Ordering guarantees are only as trustworthy as their least-cooperative component.

• What it proves: full enforcement requires all points honouring; after-accesses are gated on it.
• What it does not prove: the specific barrier flavour's semantics — DMB (12.4) versus DSB (12.5).
• Bug signature: a barrier issued and partial set — honoured somewhere but not everywhere.

13. Testbench — full versus partial enforcement

Drives the enforcement combinations and checks the barrier is only effective when honoured everywhere.

module tb_chi_barrier;
  logic before_ordered, barrier_issued, honoured_all_points;
  logic after_may_pass, fully_enforced, partial;
  int errors = 0;
 
  chi_barrier dut (.*);
 
  task automatic check(input logic bo, bi, ha,
                       input logic exp_fe, exp_partial, input string name);
    before_ordered = bo; barrier_issued = bi; honoured_all_points = ha; #1;
    if (fully_enforced !== exp_fe || partial !== exp_partial) begin
      errors++; $display("FAIL %s: fe=%0b partial=%0b", name, fully_enforced, partial);
    end else $display("PASS %s: fe=%0b partial=%0b", name, fully_enforced, partial);
  endtask
 
  initial begin
    check(1'b1, 1'b1, 1'b1, 1'b1, 1'b0, "issued, before ordered, all honour -> enforced");
    check(1'b1, 1'b1, 1'b0, 1'b0, 1'b1, "issued but NOT honoured everywhere -> PARTIAL");
    check(1'b1, 1'b0, 1'b0, 1'b0, 1'b0, "no barrier -> not enforced, not partial");
    check(1'b0, 1'b1, 1'b1, 1'b0, 1'b0, "before not ordered yet -> not enforced");
 
    if (errors == 0) $display("ALL TESTS PASSED");
    else             $display("%0d FAILURE(S)", errors);
    $finish;
  end
endmodule

Expected output:

PASS issued, before ordered, all honour -> enforced: fe=1 partial=0
PASS issued but NOT honoured everywhere -> PARTIAL: fe=0 partial=1
PASS no barrier -> not enforced, not partial: fe=0 partial=0
PASS before not ordered yet -> not enforced: fe=0 partial=0
ALL TESTS PASSED

14. DebugLab — a barrier dropped somewhere in the fabric

agent: write @A (before);  BARRIER;  write @B (after)
interconnect: HN1 honours barrier, bridge X does NOT propagate it
observer reading @A via HN1 and @B via bridge X:
  @A ordered at HN1, @B NOT ordered by the dropped barrier at X
  -> observer sees @B (after) before @A (before)  -> reordering leaks
paths not through X are correctly ordered

15. Common Mistakes

• A barrier dropped at one point. Assumption: honouring it mostly is enough. Bug: reordering leaks (the DebugLab). Prevention: honour at every ordering point.
• Treating a barrier as local. Assumption: it only affects the issuing core. Bug: remote reordering. Prevention: carry it across the fabric.
• Omitting the barrier. Assumption: cross-address order is automatic. Bug: reordering (Chapter 12.1). Prevention: issue a barrier where order matters.
• Barrier before only, not after ordering. Assumption: one side suffices. Bug: partial order. Prevention: order both before and after around the barrier.
• Confusing the flavours. Assumption: all barriers are the same. Bug: wrong semantics. Prevention: DMB versus DSB (Chapters 12.4, 12.5).
• Assuming carrying implies honouring. Assumption: propagation is enough. Bug: a point that receives but ignores it. Prevention: every point must honour it.

16. Engineering Checklist

• Treat a barrier as a first-class transaction the interconnect carries.
• Enforce before-accesses ahead of after-accesses, as all observers see them.
• Honour the barrier at every ordering point the accesses traverse.
• Carry the barrier to remote homes and bridges, not just locally.
• Treat enforcement as all-or-nothing — a dropped point is a hole.
• Map the program's DMB/DSB to the correct barrier behaviour (Chapters 12.4, 12.5).

17. Key Takeaways

• A barrier orders before-accesses ahead of after-accesses, as observed by all agents.
• CHI carries the barrier as a first-class transaction so ordering holds across the fabric.
• The program's DMB/DSB instructions map to CHI-level barrier behaviour.
• A barrier must be honoured at every ordering point the accesses traverse.
• A barrier dropped at one point leaves reordering on the paths through it — partial is not enforced.
• Carry it everywhere, honour it everywhere; the model here is representative.

18. Quick Revision

Barrier transactions. A barrier enforces the cross-address order the weak memory model omits (Chapter 12.1): every access issued before it is ordered ahead of every access after it, as observed by all agents. CHI makes the barrier a first-class transaction the interconnect carries, so the ordering reaches the points the accesses use — remote homes, bridges, serialization stages — not just the issuing core. The program's DMB and DSB instructions map to this barrier behaviour (detailed in 12.4, 12.5). The guarantee is global, so it must be honoured at every ordering point the before- and after-accesses traverse: a barrier honoured at most points but dropped at one leaves reordering on the paths through that point, and an observer there sees the after-access before the before-access — the barrier silently fails where it leaked. Partial enforcement is no enforcement. Carry the barrier across the whole fabric and ensure every ordering point honours it; a global order is only as strong as its weakest point. Representative model; 12.4 details DMB semantics.

Coming Next

Chapter 12.4 — DMB Semantics. The barrier transaction comes in two strengths; this chapter takes the lighter one. Chapter 12.4 covers the Data Memory Barrier — an ordering barrier that guarantees before-accesses are observed before after-accesses, but does not wait for them to complete — and the classic mistake of using a DMB where completion is required, which the next chapter's DSB provides.