Array Layout Mistakes

Part II · Module 25 · Chapter 25.4 · Array Layout Mistakes

1. Working Backwards

Chapter 21.8 catalogued six array faults and measured what each one does to the data. This chapter has the data and needs the fault.

That is a different problem, and it is only tractable if the payload was built to be diagnostic. §2 is that decision, and it has to be made before the failure.

2. A Payload That Names Its Own Positions

The twelve bytes used throughout this chapter:

  11 12 13 14 21 22 23 24 31 32 33 34

Two properties, both deliberate.

Every byte names its own position — high nibble the row, low nibble the column. So a byte that turns up in the wrong place says where it came from, which is Chapter 21.6's asymmetric-stimulus rule applied to a buffer.

No byte is 0x00. Padding, uninitialised storage and the default value of an unwritten element are all zero, so a payload containing a legitimate zero cannot distinguish "this position was never written" from "this position holds a zero".

That second property is not fussiness. My first version of this measurement started the pattern at 0x00, and the off-by-one fault then reported its multiset as unchanged — it looked like a permutation — because the byte it dropped was 0x00 and the padding that replaced it was also 0x00. The pattern was changed and the result corrected.

The lesson is the chapter's own, arriving early: the diagnostic value of a payload is a property of the payload, and a value that collides with the padding destroys the signature you were relying on.

3. Four Faults, Four Signatures

Measured, each applied to those twelve bytes.

A stride that is too large — the element size assumed to be four bytes where it is one:

  11 21 31 00 00 00 00 00 00 00 00 00

Correct values, in the wrong places, then zeros. Three of twelve positions filled — and the ratio 12 ÷ 3 is the assumed stride.

An index base off by one:

  12 13 14 21 22 23 24 31 32 33 34 00

Everything shifted down one, the first byte gone, a zero arriving at the end. The direction of the shift says which way the base is wrong.

A length one too short:

  11 12 13 14 21 22 23 24 31 32 33 00

Identical to the original except the last byte. The quietest of the four, and the one most likely to survive a spot check.

Transposed dimensions — a 3 × 4 array read as 4 × 3:

  11 14 23 32 12 21 24 33 13 22 31 34

Every value is a real element, none is missing, and all of them have moved.

4. The Discriminating Question

Before matching signatures by eye, one question halves the search. Measured for all four:

  stride too large   multiset unchanged: no  -- values lost
  off by one         multiset unchanged: no  -- values lost
  length too short   multiset unchanged: no  -- values lost
  transposed         multiset unchanged: YES -- a PERMUTATION

Is the multiset of values unchanged?

No — values were lost or replaced. The fault involves a count, a stride or a bound: something read the wrong number of things, or the wrong distance apart. Three of the four.

Yes — every value is present, in the wrong place. The fault is a permutation: a major-order or index-mapping error. One of the four.

That split is Chapter 21.8 §3's, used here as the first triage step rather than as a warning, and it has a direct consequence: a checksum over the payload detects the three and can never detect the fourth. If your scoreboard uses a whole-payload CRC and it fired, you are in the top three rows. If it passed and the data is still wrong, you are in the bottom one.

5. The Procedure

One — check the multiset. §4. Halves the search, costs a sort.

Two — count the non-padding positions. Three of twelve says a stride of four. Eleven of twelve says an off-by-one on the length or the base.

Three — look at where the zeros are. Trailing zeros after a few values means a stride. A single trailing zero means a length or a base. No zeros at all means a permutation.

Four — read a misplaced byte. This is what the self-describing pattern is for: a 23 sitting where 12 should be says the element at row 2 column 3 was read for position row 1 column 2, and that mapping is the fault, written down.

Five — fix it where Module 21 says to, not at the point of failure: ask the array for its bounds and element size (21.5), let the accessor compute the address (21.6), carry the length deliberately (21.1).

6. The Misconception: Diff the Payloads

"Capture the failing payload, diff it against the expected one, and the problem will be obvious."

A diff tells you where the bytes differ. It does not tell you what moved them, and on realistic traffic those are very different amounts of information.

With realistic traffic, a diff of two payloads of plausible-looking data gives a list of positions. Every one of §3's four faults can produce a list of positions, and the lists overlap — a stride error and a length error both end in zeros; a base error and a transposition both leave real values in wrong places.

With a self-describing payload, each differing byte carries its origin, so the diff becomes a mapping from destination to source — which is exactly the fault, stated.

And the deeper point, which is Chapter 21.6's and Chapter 24.8's arriving together: a diff is a comparison, and what a comparison can tell you is fixed by what was compared. A diff of two byte arrays whose values carry no positional information cannot recover positional information, however carefully it is read.

The accurate position:

The payload is a diagnostic instrument and it has to be built as one, before the failure. One permanent test case with self-describing values, no byte equal to the padding, and a non-square shape turns "the payload is corrupted" into a readable statement about which element went where.

7. What You Should Be Able to Say Now

1. This chapter runs backwards from Module 21 — given the payload, recover the fault.
2. It only works if the payload was built for it: every byte names its position, and no byte equals the padding.
3. Measured, the pattern matters: a payload starting at 0x00 made an off-by-one look like a permutation, because the dropped byte and the padding were both 0x00.
4. Stride too large: 11 21 31 00 … — 3 of 12 positions filled, and 12 ÷ 3 is the assumed stride.
5. Index base off by one: everything shifted, first byte gone, a zero at the end.
6. Length one too short: identical except the last byte — the quietest of the four.
7. Transposed: 11 14 23 32 … — every value real, none missing, all moved.
8. The discriminating question: is the multiset unchanged? Measured — only for the transposition.
9. So a whole-payload checksum catches three of four and is structurally unable to catch the fourth (21.8 §3).
10. Five-step procedure: multiset · count the filled positions · locate the zeros · read a misplaced byte · fix it where Module 21 says.
11. A diff gives positions; a self-describing payload gives a mapping — and the mapping is the fault.
12. Keep one diagnostic payload in the suite permanently, non-square, padding-distinct.

8. Exercises

1. Build a self-describing payload for an interface you work on. What is its padding value, and does any element equal it?
2. Apply a stride error to it. How many positions are filled, and what does the ratio tell you?
3. Apply an off-by-one on the base and on the length. How do the two signatures differ?
4. Transpose a non-square payload. Is any value missing? What does that imply about a checksum?
5. Repeat exercise 3 with a payload that contains the padding value. Which signature becomes ambiguous, and why?
6. Your scoreboard's CRC fired on a corrupted payload. Which of the four faults is ruled out?
7. A byte reading 23 turns up where 12 belongs. Write down the index mapping that implies.
8. §6 says a diff gives positions and not causes. Construct two different faults that produce the same diff on realistic traffic.

9. Summary

• Backwards from Module 21: the payload is given, the fault is the answer.
• The payload must be built for it — self-describing, and no byte equal to the padding (measured: a 0x00 made two faults indistinguishable).
• Stride too large → 11 21 31 00 …, 3 of 12 filled; the ratio is the stride.
• Base off by one → shifted, first byte gone, zero at the end.
• Length one short → only the last byte wrong.
• Transposed → every value real, none missing, all moved.
• The question: is the multiset unchanged? Measured — only for the transposition.
• A checksum catches three of four and never the fourth.
• Procedure: multiset · filled count · zero positions · read a misplaced byte · fix at the source.
• Keep a permanent diagnostic payload: non-square, padding-distinct, position-encoding.

Every fault so far has been about a value: its type, its width, or which one you got. The next class is about storage that was valid and then was not — where the bytes are real, the arithmetic is right, and the only thing wrong is when you looked.