Packet Transformation Functions

Part II · Module 22 · Chapter 22.6 · Packet Transformation Functions

1. Writing, Not Just Reading

Module 21 was about a buffer arriving: how long it is, where its indices start, who owns it, how long it stays valid. Chapter 21.7 ended with the rule — copy on arrival, because the storage is valid for the call.

A transformation import is different. The testbench is not asking "what is in this?" but "change this for me", and that adds two questions a read never has:

• Is there room? A read can only be too long. A write can also be too long for somewhere else.
• May the source and the destination overlap? A read has no destination.

Both are the function's responsibility, both are invisible to every tool, and §5 measures what the second one costs when it is left unasked.

2. The Function, and the Length

/* Transform a buffer in place. Returns the number of bytes written,
   or (size_t)-1 if the contract was violated. */
size_t xform_inplace(uint8_t *buf, size_t n);

/* The same transform, source and destination separate, destination
   carrying its own capacity. */
size_t xform_copy(const uint8_t *src, size_t n, uint8_t *dst, size_t dstcap);

The length is an argument — Chapter 21.1's rule, unchanged and non-negotiable. The return value is the count actually written, which gives the caller something to check against.

Measured, in a plain C binary:

  before                  01 02 03 04 05 06 07 08
  after                   58 5E 5C 52 50 56 54 4A
  bytes written: 8

And measured, the same xform.c through the import:

  before : 01 02 03 04 05 06 07 08
  xform(8) reported 8 bytes written
  after  : 58 5e 5c 52 50 56 54 4a

Byte-identical — Chapter 22.4 §3's structure doing its job again: one file, no simulator types, two programs, the same answer.

3. Capacity Is the Function's Job

When the destination is separate, somebody has to check that the result fits, and it cannot be the caller — the caller is on the other side of a boundary where, as Chapter 21.3 §5 established, no compiler sees both sides.

if (n > dstcap) return (size_t)-1;        /* refuse, before writing anything */

Measured:

  8 bytes into a 4-byte destination: refused (contract violated)
  8 bytes into an 8-byte destination: wrote 8

And through the import, with a length larger than the buffer the C side owns:

  xform(999) on a 64-byte buffer reported -1  (refused)

Three properties of that check are worth naming, because each is a decision.

It refuses before writing anything, so a rejected call leaves the destination untouched rather than half-transformed.

It reports the refusal in the return value, so the caller can tell. A function that silently truncates to dstcap would also be "safe" and would produce Chapter 21.8's silent corruption.

And the capacity is a separate argument from the length. n is how much data there is; dstcap is how much room there is. Collapsing them into one number is Chapter 21.8's second catalogue entry — a length in the wrong unit — waiting to happen.

4. The Finding: Overlap Is Not a Memory Error

Here is the question that has no analogue in Module 21.

A transform that moves data within a buffer reads some bytes and writes others, and if the ranges overlap, the order of the loop decides whether the reads see the original data or the data this loop already wrote.

/* shift every byte one position later */
for (size_t i = 1; i < n; i++) buf[i] = buf[i-1];          /* wrong direction */
for (size_t i = n-1; i >= 1; i--) buf[i] = buf[i-1];       /* safe direction */

Measured, both on the same six bytes:

  shift, wrong direction   A0 A0 A0 A0 A0 A0
  shift, safe direction    A0 A0 A1 A2 A3 A4
  both are in-bounds; only one is the intended result

Six copies of the first byte. The forward loop writes buf[1] = buf[0], then reads the buf[1] it just wrote to produce buf[2], and so on — each iteration propagating the same value forward.

Now count what reported it: nothing. Every index is within the array. No read is uninitialised, no write is out of bounds, no pointer is dangling, and there is no undefined behaviour anywhere in that loop. It is a correct C program computing the wrong thing — Chapter 21.8's conclusion exactly, arriving for a write instead of a read.

And the write case is worse in one specific way. A bad read gives you wrong data; a bad overlapping write destroys the input. After the wrong-direction loop there is nothing left to recompute the right answer from, no way to retry, and nothing to compare against. The evidence is gone along with the data.

5. Zero, and the Rest of the Contract

Measured:

  xform_inplace(buf, 0) wrote 0 bytes
  xform_copy(NULL, 0, NULL, 0) returned 0 (0 = fine, SIZE_MAX = refused)
  xform(0) reported 0        [through the import]

Zero bytes is a legal request with a real answer, exactly as Chapter 21.7 §6 found for a zero-length payload. A transform of nothing writes nothing and succeeds; treating it as an error invents a failure that is not there.

So the contract a transformation import owes its caller is five sentences, and the function's declaration is the place to put them:

/* Transform n bytes in place.
   n is a BYTE count, not an element count.
   Writes in place: the input is destroyed.
   Source and destination are the same buffer; no other overlap is permitted.
   n == 0 is legal and writes nothing.
   Returns the number of bytes written, or (size_t)-1 if n exceeds the
   buffer the function was given. */
size_t xform_inplace(uint8_t *buf, size_t n);

Every line of that is a decision somebody made, and not one of them is checked by anything — the pattern Chapter 21.8 established and this chapter extends to writes.

6. The Misconception: In Place Is the Efficient Choice

"Transforming in place avoids a copy, so it is the better option when you can use it."

The premise is true and the conclusion skips the cost that actually matters for a reference model.

In place destroys the input. That is not a side effect; it is what "in place" means. Measured, after xform_inplace, the bytes 01 02 … are gone and 58 5E … is in their place.

Three things you no longer have:

Anything to compare against. When the model and the design disagree, the first question is "what was the input?" — and the answer is no longer in the buffer.

The ability to retry. Re-running the transform does not restore anything; the callout in §2 measured that this transform applied twice does not return the original.

A second opinion. Running an alternative implementation over the same input, which is Chapter 18.3's independence check, requires the input to still exist.

Against that, a copy of a packet-sized buffer is a memcpy in a model that is being called across a language boundary by an event-driven simulator — Chapter 22.1 §2's observation about where the time actually goes.

The accurate position:

Choose in place when the caller has its own copy or genuinely does not need the input afterwards, and say so in the function's name and its comment. Otherwise take a source and a destination. The copy is cheap; the thing in-place costs you is the evidence you need on the day the model and the design disagree.

7. What You Should Be Able to Say Now

1. Writing adds two questions reading never had: is there room, and may the ranges overlap.
2. The length is an argument (21.1) and the return value is the count written, so the caller has something to check.
3. Measured: 01 02 … 08 became 58 5E 5C 52 50 56 54 4A in a C binary and byte-identically through the import.
4. Capacity is the function's job — measured, 8 into a 4-byte destination refused, 8 into 8 wrote 8, and an oversized length through the import returned -1.
5. Refuse before writing, report the refusal, and keep capacity a separate argument from length.
6. The finding — overlap is not a memory error. Measured: a shift in the wrong direction gave six copies of the first byte, all in bounds, nothing reported.
7. A bad overlapping write destroys the input, so unlike a bad read there is nothing left to recompute from.
8. Zero is legal — measured, 0 written, returned 0, in both builds.
9. This transform is not its own inverse (measured) — 22.5's round trip needs a specified inverse to exist.
10. In place is not primarily an efficiency choice — it destroys the input, which costs you the comparison, the retry, and the second opinion.
11. All five answers are decisions; the compiler supplies none of them.

8. Exercises

1. Write a byte-wise transform taking a buffer and a length. Run it in a C main and through an import. Are the bytes identical?
2. Add a separate destination with its own capacity. Offer it more data than it can hold — does your function refuse before writing, or part-way through?
3. Why must the capacity be a separate argument from the length? Give the bug that results from collapsing them.
4. Implement a shift both directions over the same buffer. Print both results. Which tool reported the wrong one?
5. §4 says a bad overlapping write is worse than a bad read. State the difference in one sentence, from the point of view of someone debugging a mismatch the next morning.
6. Apply your transform twice. Does it restore the input? What does your answer mean for using Chapter 22.5's technique on it?
7. Call your transform with a length of zero. Is the result an error in your implementation? Should it be?
8. Write the five-sentence contract comment for a transform in your own code. Which of the five did you not previously have an answer for?

9. Summary

• Writing adds capacity and overlap to Module 21's questions.
• Length is an argument; the return value is the count written.
• Measured: 58 5E 5C 52 50 56 54 4A, byte-identical in a C binary and through the import.
• Capacity is checked by the function — measured, 8 into 4 refused, 8 into 8 wrote 8, oversized through the import -1. Refuse before writing; report it; keep capacity separate from length.
• The finding: a wrong-direction shift gave six copies of the first byte, in bounds, nothing reported.
• A bad overlapping write destroys the input — no retry, no comparison, no second opinion.
• Zero is legal — measured, 0 written, returned 0.
• Not an involution (measured) — a round trip needs a specified inverse.
• In place costs evidence, not cycles.
• Five answers, five decisions, none supplied by the compiler.

Every import so far has been a function of its arguments: give it the same input and it gives you the same answer, now or in an hour. That is what made each of them testable in a C main with nothing else around it — and it is exactly the property the next chapter gives up on purpose.