DMB Semantics

Module 12 · Chapter 12.4 · Ordering and Consistency

Project thread — 12.3 introduced the barrier. This chapter is the DMB — the ordering barrier. 12.5 is the DSB — the completion barrier.

1. Learning Outcomes

By the end of this chapter you should be able to:

• Define the DMB as an ordering barrier — before-accesses observed before after-accesses.
• State that a DMB does not wait for the before-accesses to complete.
• Distinguish ordering (observation) from completion.
• Identify when ordering (a DMB) is sufficient — message passing, locks.
• Diagnose the failure of using a DMB where completion is required.
• Implement a representative DMB model in SystemVerilog, Verilog-2001, and VHDL.

2. Why Should I Learn This?

The Data Memory Barrier is the workhorse of concurrent code — the barrier you reach for to make message passing and locks correct on a weakly-ordered system. It does exactly one thing, precisely: it orders memory accesses so that everything before it is observed ahead of everything after it. Most synchronization needs nothing more, and the DMB is the efficient way to get it, because it orders without stalling.

But that efficiency comes from a specific limitation: a DMB does not wait. It guarantees the order in which accesses are observed, not that the earlier ones have finished. That gap — ordering versus completion — is subtle and catches even experienced engineers, because "observed before" sounds like "done before" but is not. Using a DMB where you actually need completion — before poking a device that will act on data that may still be in flight — is a real, hard-to-find bug. Knowing precisely what a DMB does and does not promise is knowing when it is enough and when you need its stronger sibling.

3. Key Terms

4. Previous Chapter Connection

Chapter 12.3 introduced the barrier as a first-class transaction that orders before-accesses ahead of after-accesses across the fabric. It described the barrier's ordering role in general. This chapter specializes it to the DMB — and pins down what "orders" means: observation order, not completion.

The general barrier of 12.3 guaranteed a relative order as observed by all agents. The DMB is exactly that guarantee, and no more: it says nothing about whether the ordered accesses have finished. That is the line this chapter draws — a DMB is an ordering barrier — and it sets up the contrast with the DSB (Chapter 12.5), which adds the completion guarantee the DMB withholds. Message passing (Chapter 12.1) needed ordering, so a DMB suffices there; cases needing completion do not.

5. Core Concept — orders observation, does not wait for completion

A DMB guarantees an order of observation and nothing about completion.

• Orders observation. After a DMB, every other agent observes the before-accesses ahead of the after-accesses. The relative order is guaranteed for all observers.
• Does not wait. A DMB does not stall until the before-accesses complete. Execution proceeds past the DMB while the before-accesses may still be in flight — they are merely guaranteed to be observed first.
• Ordering is enough for synchronization. Message passing and locks need only the relative order — a reader that sees the flag has, by the ordering, seen the data. So a DMB is sufficient and efficient for these.
• Completion is a different guarantee. "Observed before" is not "finished." When something must actually be done — a write landed in a device — ordering is not enough; that needs a DSB (Chapter 12.5).

The synthesis:

A DMB is an ordering barrier: it makes before-accesses observed ahead of after-accesses by all agents, which is exactly what message passing and locks require. It does not wait — the before-accesses may still be in flight after the DMB, merely ordered first. Ordering is not completion: a DMB orders when accesses are observed, not whether they have finished. For completion, use a DSB.

6. Engineering Mental Model — mailing letters in order

A DMB is like guaranteeing your letters are postmarked in order — not that they have arrived.

• You post letter A (the data), then a DMB, then letter B (the flag). The DMB guarantees the post office stamps A before B — anyone tracking the mail sees A's postmark precede B's.
• So a recipient who receives B knows A was posted first, and A is on its way. For a handshake — "when you get the flag, the data is coming" — that ordering is all you need.
• But the DMB does not mean A has been delivered. A could still be in a mail truck when you post B. If you then phoned someone and said "act on letter A now — it's delivered," you would be wrong; it was only posted first, not delivered.
• To guarantee A has actually arrived before you proceed, you need a stronger step that waits for delivery — that is the DSB.

Postmarked-in-order (DMB) is enough to establish a sequence; delivered-before-you-continue (DSB) is what you need when arrival matters.

7. Engineering Diagram — ordered but still in flight

Read the last message: the observer sees @A before @B (ordering holds), but the note says A not yet complete — the DMB did not wait for it. The order is guaranteed; the finishing is not. That is the DMB's exact promise, and its exact limit.

8. Ordering versus Completion

The two guarantees, and which the DMB provides.

The rule to carry: a DMB provides ordering — a relative observation order — and not completion. It is the right, efficient tool when you need accesses ordered relative to each other (synchronization), and the wrong tool when you need them finished before proceeding (a device that will act on the data). Ordering says observed first; completion says done. The DMB gives only the first.

9. When Ordering Is Enough — and When It Is Not

The boundary between the DMB's sufficiency and its limit deserves its own statement.

• Synchronization needs ordering. Message passing, lock acquire/release, publish/subscribe — all rest on a relative order (see the flag, you have seen the data). A DMB provides exactly this.
• Ordering is cheaper. Because the DMB does not stall for completion, it is the efficient choice wherever ordering suffices — which is most synchronization.
• Completion needs a wait. When a subsequent action depends on the before-accesses having actually landed — a device register poked to process a buffer, a DMA kicked off — ordering is not enough; the writes must be complete.
• The mismatch is the bug. Using a DMB where completion is required is the classic error: the accesses are ordered but not finished, and the dependent action proceeds on incomplete state.

The point to carry:

"Observed before" and "completed" are different points on an access's life, and the DMB guarantees only the first. For inter-agent ordering — where the whole point is the relative sequence other agents see — that is precisely the right guarantee, and paying for completion would waste the stall. But for a dependency on the effect having happened — the data must be in the device before you signal it — ordering is silent on what you need. The skill is reading which one a situation demands: a relative order between accesses (DMB), or the certainty that earlier accesses are done (DSB). Reach for the DMB by default for synchronization; reach past it when completion is the actual requirement.

10. Reading the Semantics — a lock release

Trace a lock release, where a DMB is exactly right.

1. Write protected data. Inside the critical section, the core writes shared data at A.
2. DMB. A DMB orders the data write ahead of the lock release.
3. Release the lock. The core writes the lock variable at B to unlocked.
4. Another core acquires. A second core sees the lock free (B), and by the DMB's ordering, is guaranteed to observe the protected data (A) — it sees the updates.
5. Ordering sufficed. The handoff needed only that the data be observed before the release — a DMB. No completion wait was required; the data need not have finished in the abstract, only be observed ahead of the unlock.

The DMB was the correct, efficient barrier because the requirement was ordering between the data write and the release. Now change the scenario to "write a buffer, then tell a DMA engine it is ready" — that needs the buffer complete, and a DMB would be wrong: the DebugLab.

11. RTL / Hardware View — a DMB model

A DMB provides ordering but not completion. Its model exposes exactly that: it orders before ahead of after, and does not assert completion. Representative.

// Representative DMB (Data Memory Barrier) model (educational).
// A DMB ORDERS before-accesses ahead of after-accesses (observation order) but does
// NOT wait for the before-accesses to COMPLETE. So it provides ordering, not
// completion; after a DMB, before-accesses may still be in flight.
module chi_dmb (
  input  logic dmb_issued,        // a DMB was executed
  input  logic before_in_flight,  // the before-accesses are still outstanding
  output logic provides_ordering, // before observed ahead of after
  output logic provides_completion,// does the DMB guarantee before have completed?
  output logic before_may_be_in_flight // after the DMB, before can still be outstanding
);
  // A DMB provides ORDERING of observation.
  assign provides_ordering = dmb_issued;
  // A DMB does NOT provide completion -- that is the DSB.
  assign provides_completion = 1'b0;
  // Consistent with no completion guarantee: before-accesses may still be in flight.
  assign before_may_be_in_flight = dmb_issued && before_in_flight;
endmodule

The same behavior in Verilog-2001:

// Representative DMB model (Verilog-2001).
module chi_dmb (
  input  dmb_issued, before_in_flight,
  output provides_ordering, provides_completion, before_may_be_in_flight
);
  assign provides_ordering       = dmb_issued;
  assign provides_completion     = 1'b0;
  assign before_may_be_in_flight = dmb_issued && before_in_flight;
endmodule

And in VHDL:

-- Representative DMB model (VHDL).
library ieee;
use ieee.std_logic_1164.all;
 
entity chi_dmb is
  port (
    dmb_issued             : in  std_logic;
    before_in_flight       : in  std_logic;
    provides_ordering      : out std_logic;
    provides_completion    : out std_logic;
    before_may_be_in_flight: out std_logic
  );
end entity;
 
architecture rtl of chi_dmb is
begin
  provides_ordering       <= dmb_issued;
  provides_completion     <= '0';
  before_may_be_in_flight <= dmb_issued and before_in_flight;
end architecture;

All three assert provides_ordering for a DMB and hard-tie provides_completion to 0 — a DMB never promises completion, so before-accesses may still be in flight after it. The DebugLab is depending on the completion a DMB does not give.

12. Verification View — ordering yes, completion no

The properties that pin the DMB's guarantee: it orders, it does not complete.

// Bind to chi_dmb.
// 1. A DMB provides ordering of observation.
property p_dmb_orders;
  @(*) dmb_issued |-> provides_ordering;
endproperty
 
// 2. A DMB NEVER provides a completion guarantee.
property p_dmb_no_completion;
  @(*) provides_completion == 1'b0;
endproperty
 
// 3. After a DMB, before-accesses may still be in flight (no completion wait).
property p_before_may_be_in_flight;
  @(*) (dmb_issued && before_in_flight) |-> before_may_be_in_flight;
endproperty

The system point, beyond the checks:

The DMB embodies a precise engineering economy: buy only the guarantee you need. Synchronization between agents is fundamentally about relative order — the reader must see the writer's accesses in a fixed sequence — and completion is irrelevant to that, because an access observed in the right order serves the handshake whether or not it has otherwise "finished." So the DMB withholds the completion wait deliberately, and that withholding is a feature: it avoids a stall the common case does not need. The corresponding discipline is to recognize the rarer case — a dependency on the effect having materialized somewhere specific — and not to reach for the DMB there. The barrier's minimalism is its strength for ordering and its trap for completion.

• What it proves: a DMB orders and never completes; before may remain in flight.
• What it does not prove: the completion semantics — that is the DSB (Chapter 12.5).
• Bug signature: logic depending on provides_completion from a DMB — it is always 0.

13. Testbench — the DMB provides ordering, not completion

Drives the DMB model and checks it orders but never completes.

module tb_chi_dmb;
  logic dmb_issued, before_in_flight;
  logic provides_ordering, provides_completion, before_may_be_in_flight;
  int errors = 0;
 
  chi_dmb dut (.*);
 
  task automatic check(input logic di, bif,
                       input logic exp_ord, exp_comp, exp_inflight, input string name);
    dmb_issued = di; before_in_flight = bif; #1;
    if (provides_ordering !== exp_ord || provides_completion !== exp_comp ||
        before_may_be_in_flight !== exp_inflight) begin
      errors++; $display("FAIL %s: ord=%0b comp=%0b inflight=%0b",
                         name, provides_ordering, provides_completion, before_may_be_in_flight);
    end else $display("PASS %s: ord=%0b comp=%0b inflight=%0b",
                      name, provides_ordering, provides_completion, before_may_be_in_flight);
  endtask
 
  initial begin
    check(1'b1, 1'b1, 1'b1, 1'b0, 1'b1, "DMB, before in flight -> orders, no completion, in flight");
    check(1'b1, 1'b0, 1'b1, 1'b0, 1'b0, "DMB, before settled   -> orders, no completion");
    check(1'b0, 1'b1, 1'b0, 1'b0, 1'b0, "no DMB -> no ordering guarantee");
 
    if (errors == 0) $display("ALL TESTS PASSED");
    else             $display("%0d FAILURE(S)", errors);
    $finish;
  end
endmodule

Expected output:

PASS DMB, before in flight -> orders, no completion, in flight: ord=1 comp=0 inflight=1
PASS DMB, before settled   -> orders, no completion: ord=1 comp=0 inflight=0
PASS no DMB -> no ordering guarantee: ord=0 comp=0 inflight=0
ALL TESTS PASSED

14. DebugLab — using a DMB where completion is required

driver:  write buffer @A (large);  DMB;  write doorbell @DEV
DMB orders A observed-before DEV, but does NOT wait for A to COMPLETE
  -> buffer writes still IN FLIGHT when doorbell is written
DMA engine sees doorbell -> starts reading buffer @A
  buffer not fully written -> DMA reads partial / stale data

15. Common Mistakes

• DMB where completion is needed. Assumption: a DMB waits. Bug: device acts on in-flight data (the DebugLab). Prevention: use a DSB for completion.
• Thinking "observed before" means "done." Assumption: ordering implies completion. Bug: premature dependent action. Prevention: they are different guarantees.
• DSB everywhere for safety. Assumption: the strong barrier is always safe. Bug: needless stalls. Prevention: a DMB where ordering suffices.
• Omitting the DMB in synchronization. Assumption: order is automatic. Bug: reordering (Chapter 12.1). Prevention: a DMB where a relative order matters.
• Barrier on one side only. Assumption: producer suffices. Bug: consumer reorders. Prevention: order both sides.
• Assuming a DMB drains buffers. Assumption: it flushes write buffers. Bug: it only orders. Prevention: a DSB drains/waits.

16. Engineering Checklist

• Use a DMB to order before-accesses ahead of after-accesses.
• Rely on a DMB for synchronization — message passing, locks — where ordering suffices.
• Do not assume a DMB waits — before-accesses may still be in flight.
• Use a DSB when a subsequent action needs the before-accesses completed.
• Distinguish observed-before (DMB) from done (DSB).
• Prefer the DMB where it is enough — it avoids the DSB's stall.

17. Key Takeaways

• A DMB is an ordering barrier: before-accesses are observed ahead of after-accesses.
• A DMB does not wait for the before-accesses to complete — they may still be in flight.
• Ordering is enough for synchronization — message passing, locks — and cheaper than waiting.
• Completion is a different guarantee; a DMB does not provide it.
• Using a DMB where completion is required lets a device act on not-yet-landed data.
• Order with a DMB, complete with a DSB; the model here is representative.

18. Quick Revision

DMB semantics. The Data Memory Barrier is an ordering barrier: it guarantees the memory accesses issued before it are observed by other agents ahead of the accesses issued after it. That relative order is exactly what synchronization — message passing, lock acquire/release — needs, and the DMB is the efficient way to get it, because it does not wait: it orders the before-accesses ahead of the after-accesses but does not stall until they complete, so after a DMB the earlier writes may still be in flight, merely guaranteed to be observed first. Ordering is not completion — "observed before" is not "done." The classic mistake is using a DMB where completion is required: writing a data buffer, a DMB, then a doorbell to a DMA engine, assuming the DMB waited for the buffer to land — but it only ordered it, so the device reads a buffer still in flight. For completion, use a DSB. Order with a DMB; complete with a DSB. Representative model; 12.5 details DSB semantics.

Coming Next

Chapter 12.5 — DSB Semantics. The DMB ordered without waiting; the DSB waits. Chapter 12.5 covers the Data Synchronization Barrier — the stronger barrier that blocks until all prior accesses have completed before any subsequent instruction proceeds — the completion guarantee the DMB withholds, and the barrier the buffer-then-doorbell pattern actually needs.