Transferring Packet and Byte-Stream Data

Part II · Module 21 · Chapter 21.7 · Transferring Packet and Byte-Stream Data

1. The Case the Module Is For

Chapters 21.1 to 21.6 built the pieces: carry the size deliberately, know the array kind, ask for the bounds, do not assume a major order. This chapter puts them on the thing they exist for — a payload.

A packet body. A burst of bus data. A frame captured off an interface. Three properties together, and it is the combination that makes it the hard case:

• the length is decided at run time, so Chapter 21.3's fixed contract does not apply
• the model must keep it, because the next transaction is compared against this one
• somebody else owns the storage it arrived in

The first is Chapter 21.4's problem, now solved. The second and third are Chapter 20.8's problem — ownership and lifetime — arriving for an array instead of a string.

2. Validating the Instrument First

The payload operation here is a CRC, because a sum would not notice most of the things this chapter is about. Chapter 18.3 established the rule that comes with using a real algorithm: check the instrument before trusting what it measures.

CRC-8/SMBUS has a published check value — the CRC of the ASCII string 123456789. Measured:

  CRC-8/SMBUS over "123456789" = 0xF4 (published check value 0xF4): match

So the implementation below is the algorithm it claims to be, and every number after this is a real CRC of real bytes rather than the output of whatever the code happens to do.

static uint8_t crc8(const uint8_t *p, size_t n) {
    uint8_t c = 0x00u;
    for (size_t i = 0; i < n; i++) { c ^= p[i];
        for (int b = 0; b < 8; b++) c = (uint8_t)((c & 0x80u) ? ((c << 1) ^ 0x07u) : (c << 1)); }
    return c;
}

3. The Length

Measured over a 64-byte buffer at four run-time lengths:

   1 bytes -> CRC 0x69
   7 bytes -> CRC 0xE6
  16 bytes -> CRC 0xF6
  64 bytes -> CRC 0x06

Four different, entirely ordinary answers. Now the off-by-one, which is the whole hazard:

  16 bytes      -> 0xF6
  15 bytes      -> 0xE4   <- indistinguishable as a value
  17 bytes      -> 0xD5   <- reads one byte it was not given

Chapter 21.1 §4 made this point with a checksum and it is worth making again with a real CRC, because the property is the point of a CRC: a one-byte difference in the input produces a completely unrelated output. That is exactly what makes a CRC good at detecting corruption and exactly what makes a wrong length undetectable from the result — 0xE4 is not near 0xF6, so there is no sense in which the answer looks slightly off.

Of the three, only 17 touches a byte it was not given, and only that one is a memory error in principle. The 15-byte call reads memory it owns and is wrong anyway.

4. The Ownership

From the LRM, the rule is Chapter 20.8's rule: the storage an argument refers to is valid for the duration of the call. If the model needs the payload afterwards — and §1 says it does — the model must copy it into storage it owns.

typedef struct { uint8_t *bytes; size_t len; } Packet;

static int pkt_take(Packet *p, const uint8_t *src, size_t n) {
    p->bytes = malloc(n ? n : 1);          /* malloc(0) is implementation-defined */
    if (!p->bytes) { p->len = 0; return 0; }
    memcpy(p->bytes, src, n); p->len = n; return 1;
}
static void pkt_free(Packet *p) { free(p->bytes); p->bytes = NULL; p->len = 0; }

Measured, with the caller's buffer deliberately wiped afterwards to stand in for storage going away:

  copied 16 bytes into storage the model owns
  caller's buffer wiped; the copy still reads 0xA0 A1 A2 ...
  CRC of the copy: 0xF6 -- equal to the 16-byte CRC above: yes
  freed; len is now 0 and the pointer is NULL

Three things that structure matters for.

The pointer and the length are one object. Measured, sizeof(Packet) is 16 — eight bytes of pointer and eight of length. They are allocated together, assigned together, and cleared together.

pkt_free clears both. Chapter 20.10 §3 made the same move for a handle: a free that leaves the caller's copy intact leaves a dangling pointer with no way to say so. Here it would also leave a length claiming the data is still there.

The copy is the model's, so its lifetime is the model's decision — which is the whole benefit. Nothing the simulator does afterwards can invalidate it.

5. The Finding: The Two Halves Do Not Fail the Same Way

"Pass a pointer and a length" has been this module's refrain since Chapter 21.1 §5, and it is right. What it conceals is that the two halves decay asymmetrically, and the asymmetry decides where the undetectable bugs are.

A stale pointer is undefined behaviour. That is bad, and it is also a category a tool can have an opinion about — Chapter 20.10 §7 measured a heap use-after-free reported by AddressSanitizer with no options at all. A model that keeps a pointer past its lifetime is at least the kind of wrong that can be found.

A stale length is an ordinary integer. It is 16. It was 16 during the call, it is still 16 afterwards, and it will be 16 for as long as the variable lives. Nothing happened to it, because nothing can happen to it. It has simply stopped describing anything, and there is no tool, in principle, that can object to a correct integer.

So of the two things you were told to carry, the one with a detector is the one everyone worries about, and the one with no detector in principle is the one people cheerfully store in a separate variable, a class member, or a static.

Hence the structure in §4. The length lives in the same object as the pointer it describes, is assigned in the same statement, and is cleared in the same function. Not because that makes it checkable — nothing makes it checkable — but because it removes the gap in which the two can drift apart. Measured, sizeof(Packet) is 16: that is a pointer and a length with no way to have one without the other.

6. The Misconception: The Data Will Still Be There

"The buffer is in the simulator's memory and the simulator is not going anywhere. I will keep the pointer and copy it later if I turn out to need it."

From the LRM, the storage is valid for the duration of the call — Chapter 20.8's rule, and it is a rule about permission rather than about what happens to be in memory.

Measured, with the caller's buffer wiped after the call to stand in for the storage going away:

  caller's buffer wiped; the copy still reads 0xA0 A1 A2 ...
  CRC of the copy: 0xF6 -- equal to the 16-byte CRC above: yes

The copy answered correctly. A retained pointer would have been reading whatever is there now.

Three reasons the belief is tempting and still wrong:

The simulator is free to reuse that storage. It owns it. A buffer handed to one call can legitimately back the next transaction, be returned to a pool, or be part of a value that no longer exists.

"Later" is not a time you control. Your model decides to copy when it sees a second transaction — which is, by definition, during a later call, after the first buffer's lifetime ended.

It usually works for a while, which is this module's pattern for the fourth time: 21.2 §5's byte-identical models, 21.5 §6's zero-based range, 21.6 §6's symmetric matrix. Storage that has not yet been reused reads correctly, so the lazy version passes until the simulator's allocation pattern changes — a longer test, a different seed, a new release.

The corrected position:

Copy during the call or do not keep it at all. The decision of what to keep belongs at the point where you still have permission to read, not at the point where you discover you want it.

And the practical form: if a model might need a payload later, copy it on arrival. A copy you turn out not to need costs a memcpy; a pointer you turn out to need costs a bug with no symptom until it has one.

7. What You Should Be Able to Say Now

1. The hard case is the combination: run-time length, must be kept, someone else owns the storage.
2. Validate the instrument first (18.3) — measured, CRC-8/SMBUS over 123456789 is 0xF4, matching the published check value.
3. Measured lengths: 1 → 0x69, 7 → 0xE6, 16 → 0xF6, 64 → 0x06. All ordinary-looking.
4. The off-by-one is invisible from the value — measured, 15 → 0xE4 and 17 → 0xD5 against 16 → 0xF6. A CRC's whole purpose guarantees the wrong answer looks unrelated, not nearly right.
5. Only reading too far is a memory error; reading too little is not, and is wrong anyway.
6. The storage is valid for the duration of the call — copy what you need (20.8's rule, for an array).
7. Measured, the copy survived the caller's buffer being wiped and produced the same 0xF6.
8. The finding — the two halves decay asymmetrically: a stale pointer is undefined behaviour a sanitizer may report; a stale length is a correct integer that no tool can object to, in principle.
9. So keep them in one object — measured, sizeof(Packet) is 16 — assigned together, cleared together.
10. Zero length is legal and has a real answer — measured 0x00 — and malloc(0) is implementation-defined, so never ask for it.
11. "Copy it later" is not available: later is a different call, after the permission ended.

8. Exercises

1. Implement CRC-8/SMBUS and check it against 0xF4 before using it for anything. What would you have concluded from the rest of the chapter if this step were skipped and the implementation were wrong?
2. CRC the same buffer at 15, 16 and 17 bytes. Which of the three is a memory error, and which of the three is detectable from the result?
3. Build the Packet struct and copy a payload into it. Wipe the source buffer and confirm the copy still CRCs correctly.
4. Make pkt_free leave len unchanged. Describe a sequence of calls where that produces a wrong answer with no crash.
5. §5 says a stale length has no possible detector. Construct the argument in two sentences, and say what that implies about where the length should be stored.
6. CRC a zero-length payload. Is the result distinguishable from an error code? What does that say about using it as one?
7. Replace malloc(n ? n : 1) with malloc(n) and work out what a zero-length packet does on an implementation that returns null for malloc(0). How would you tell that from an allocation failure?
8. For a model you have written, find every place a length is stored. Is each one in the same object as the pointer it describes?

9. Summary

• The hard case: run-time length · must be kept · somebody else's storage.
• Validate the instrument — measured CRC-8/SMBUS check value 0xF4, matched.
• Measured lengths 1 / 7 / 16 / 64 → 0x69 / 0xE6 / 0xF6 / 0x06, all ordinary.
• The off-by-one is invisible — 15 → 0xE4, 17 → 0xD5; a CRC guarantees the wrong answer looks unrelated, not nearly right.
• Valid for the duration of the call — copy on arrival (20.8's rule for an array).
• Measured: the copy survived the source being wiped, same 0xF6.
• The finding: a stale pointer is undefined behaviour with a possible detector; a stale length is a correct integer with none, in principle.
• One object for both — measured sizeof(Packet) 16; assign together, clear together.
• Zero length is legal (0x00 is a real answer) and malloc(0) is implementation-defined — never ask for it.
• "Copy it later" is not on offer: later is a different call.

Everything in this module has had the same shape at its core — a read that stays inside the buffer, returns real values, and is wrong about what they are. It is worth collecting those cases in one place and asking of each one what, if anything, would ever have told you.