Array Mistakes That Silently Corrupt Data

Part II · Module 21 · Chapter 21.8 · Array Mistakes That Silently Corrupt Data

1. Six Mistakes, One File, No Diagnostics

Each of the preceding chapters ended with a defence. This one collects what the defences are against, because the purpose here is recognition: these are the errors that return an answer instead of an error.

All six are in one program, measured against a reference:

  reference: 16 bytes -> CRC 0xF6

One — a length one too small.

  1. length one too small        -> 0xE4  (differs: yes)

Two — a length in the wrong unit. Four elements counted as four bytes:

  2. length from the wrong unit  -> 0x8E  (4 uint32_t read as 4 bytes)

Three — an assumed element size. The same sixteen bytes:

        as words : 0xA3A2A1A0 0xA7A6A5A4 0xABAAA9A8 0xAFAEADAC
        as bytes : A0 A1 A2 A3 A4 A5 A6 A7 A8 A9 AA AB AC AD AE AF
        same storage, two different element COUNTS: 4 and 16

Four — an assumed index base. Chapter 21.5's measurement: elements 3 to 10 read as 0 to 7, three never read and five shifted.

Five — an assumed major order. Chapter 21.6's, with the detail that matters here:

        sum row-major 138, sum transposed 138, equal: YES
        -> a sum-based scoreboard cannot see it

Six — an assumed element signedness. Chapter 20.4's boundary, now applied per element:

  6. signedness of the element   : 0xAF read signed is -81, unsigned is 175
        sum of 16 elements: signed -1416, unsigned 2680

2. The Closing Measurement

The six above are the module's content. This is its thesis, stated as a number.

Measured, the entire catalogue compiled with AddressSanitizer and run:

  [ASan reports from the entire catalogue: 0]
  [exit code: 0]

Zero. Not one report, and a clean exit.

That is not a failure of the tool, and reading it as one would be the wrong lesson. Every read in that file stays inside an object the program owns. There is no undefined behaviour anywhere in it — no overflow, no dangling pointer, no use after free. The sanitizer had nothing to find because nothing in its domain happened.

Every one of the six is a correct program computing the wrong thing, and that is a category no memory tool is about.

It is worth setting beside Chapter 20.8 §7 and Chapter 20.10 §7, where the same tool gave two different answers by default — silent on a stack-use-after-return, loud on a heap-use-after-free. Those were facts about the tool's configuration. This is a fact about its subject matter, and no option changes it.

3. Two Kinds, and Why the Distinction Is Worth Having

The six are not all the same shape, and the split predicts which of your existing checks can possibly help.

Four of them change the values. A short length, a wrong unit, a wrong element size, a wrong signedness — in each case the multiset of values your model works with is different from the one it was sent. So an aggregate check can disagree: a CRC over everything, a sum, a count. Measured, the CRC did disagree — 0xE4 and 0x8E against 0xF6 — and the signed sum did too, -1416 against 2680.

Two of them only move the values. A wrong index base and a wrong major order are permutations. Every value is genuine, nothing is out of range, and measured, the sum is 138 either way.

For those two, no aggregate check can ever disagree — not because yours is weak, but because the quantity it computes is invariant under the error. A sum, a count, a minimum, a maximum, a histogram, a checksum over storage order: every one of them is structurally incapable of noticing, and will pass for as long as the bug exists.

So the actionable question for an existing scoreboard is one sentence:

Does anything I compare depend on where a value sits, or only on which values are present?

If only the latter, your checks cover four of these six and are blind to the other two by construction — and Chapter 18.2's point applies in its most literal form, because a check that cannot fail carries no information when it passes.

4. Where Each One Is Actually Fixed

The catalogue's third column is a set of pointers, not new material.

Every entry in the right-hand column is a decision somebody made, not a flag somebody enabled. That is the pattern the figure is arranged to show, and §2 is the reason it has to be that way.

5. The Misconception: We Run Under Sanitizers, So the Array Handling Is Fine

"Our C model is built with AddressSanitizer in CI. If there were an array problem we would know about it."

Measured false in the strongest form available: the whole catalogue, built with AddressSanitizer and run, produced 0 reports and exit code 0.

The reasoning behind the belief is sound and its scope is wrong. A sanitizer answers one question — did this program touch memory it had no right to touch? — and answers it well. Chapter 21.1 §6 measured it catching a read eight bytes past a buffer, and Chapter 20.10 §7 measured it catching a use-after-free with no options at all.

None of this chapter's six is that question. Reading fifteen bytes of a sixteen-byte buffer is permitted. Reading four 32-bit elements of a sixteen-byte buffer is permitted. Reading every element in a different order is permitted. Interpreting a byte as signed is permitted. The program has every right to do all of it, and that is precisely the problem.

Two things follow.

A clean sanitizer run is evidence about memory safety and nothing else. It is real evidence and worth having — keep it, and keep detect_stack_use_after_return=1 on, which Chapter 20.8 §7 measured as off by default. But do not read it as evidence about correctness of interpretation.

The checks that would catch these are checks you write. The third column of the figure has six entries and not one of them is a tool.

The corrected position:

Memory tools police permission, not meaning. Every mistake in this chapter is a correct program computing the wrong thing, which is outside what any sanitizer is about — so the defence is the interface discipline in Chapters 21.2 to 21.7, and a scoreboard that compares something position-sensitive.

6. What You Should Be Able to Say Now

1. Six silent array mistakes — short length, wrong unit, wrong element size, wrong index base, wrong major order, wrong signedness.
2. Measured against 0xF6: a short length gives 0xE4, a wrong unit 0x8E — both ordinary-looking.
3. The same sixteen bytes are four elements or sixteen, depending on an assumption.
4. The closing measurement: the entire catalogue under AddressSanitizer produced 0 reports, exit code 0.
5. That is not the tool failing — every read stays inside an object the program owns, so there is no undefined behaviour to find.
6. Four of the six change the values, so an aggregate check can disagree — measured, the CRC and the signed sum did.
7. Two of them only permute — measured, sum 138 either way — so no aggregate check can ever disagree.
8. The question to ask of a scoreboard: does anything it compares depend on where a value sits, or only on which values are present?
9. Three of the six have a necessarily-correct common case — byte-identical models (21.2 §5), low = 0 (21.5 §6), a symmetric matrix (21.6 §6) — so passing tests accumulate no evidence about them.
10. Every defence is a decision, not a flag — compare the length, name the unit, ask for the element size and bounds, use a position-sensitive check, pin the declared type.
11. Sanitizers police permission, not meaning.

7. Exercises

1. Build all six mistakes in one file and confirm it compiles with no warnings at -Wall -Wextra -Wpedantic. Which of the six did you expect a diagnostic for?
2. Run that file under AddressSanitizer with detect_stack_use_after_return=1. How many reports? Explain the result without describing the tool as inadequate.
3. Classify each of the six as value-changing or permuting. For each permuting one, name a check in your own project that cannot detect it.
4. Compute the sum of a 3 × 4 array read both ways. Then construct a stimulus for which the two sums differ, and state the property you added.
5. Sum sixteen bytes as signed and as unsigned. At what point in the data does the two interpretations' difference begin, and what does that imply about test values?
6. For each row of §4's table, find the chapter section that built the defence and write the one-line version of it from memory.
7. Take a scoreboard you have. For each comparison it makes, mark whether it is position-sensitive. What fraction of this chapter's catalogue does it cover?
8. §4's callout says an assumption with a necessarily-correct common case cannot be tested into visibility. Give one example from outside this module.

8. Summary

• Six silent mistakes, all measured against a reference CRC of 0xF6: short length 0xE4 · wrong unit 0x8E · wrong element size (4 elements or 16) · wrong index base (3 unread, 5 shifted) · wrong major order (sum 138 either way) · wrong signedness (-1416 against 2680).
• The closing measurement: the entire catalogue under AddressSanitizer — 0 reports, exit code 0.
• Not a tool failure: every read is inside an object the program owns, so there is no undefined behaviour in the file at all.
• Four change the values (an aggregate check can disagree); two only permute (no aggregate check ever can).
• Ask of your scoreboard: does it compare anything that depends on position?
• Three have a necessarily-correct common case — 21.2 §5, 21.5 §6, 21.6 §6 — so passing tests prove nothing about them.
• Every defence is a decision, not a flag.
• Sanitizers police permission, not meaning.

9. Module 21 in One Page

Eight chapters from one sentence: a C array is bare storage with no size, and a SystemVerilog array is not. Everything since has been a way of carrying what C does not.

• The asymmetry, measured — the count is 8 in the declaring scope and 2 inside a function whose parameter says [8]; a declared parameter bound binds nothing; and [static N] is the one keyword that is checked, at -O0 and -O2 (21.1, 21.3).
• The question that decides everything downstream — packed or unpacked, read off the declaration. Packed is one value and goes through Chapter 20.6's specified chunks; unpacked is separate objects. Measured, 8-bit elements make the two byte-identical (21.2).
• A fixed size is a contract with one enforcer — 1 of 3 wrong calls diagnosed, and the DPI call is not a C call site (21.3).
• An open array is an opaque handle — three objects, three sizes, 8 / 24 / 32, and the stride arrives with the handle (21.4).
• Ask for the bounds — measured low 3, high 10, size 8, where the reflex loop rejects three indices, never reads three elements, and shifts five (21.5).
• Ask per dimension — a wrong major order permutes, sum 138 either way, and a symmetric matrix reads identically transposed (21.6).
• A payload meets ownership — valid for the call, copy on arrival, and the pointer and the length decay differently: a stale pointer is undefined behaviour with a possible detector, a stale length is a correct integer with none (21.7).
• And the catalogue: six silent mistakes, zero sanitizer reports (21.8).

The thread is one sentence, and it is the same one Module 20 ended on, sharpened: every one of these bugs compiles, runs, and returns a number. What changed in this module is that the tools got less able to help, not more — because an array mistake that stays inside the buffer is not a memory error, and no sanitizer is about meaning. The defence has been identical in every chapter: ask the array instead of assuming, carry the size and the unit deliberately, and compare something that depends on where a value sits.

Modules 19 to 21 settled the mechanics: how a call is declared, how every type crosses, and how an array carries what C cannot. All of it has been about the boundary itself. What comes next is the import direction taken from a one-line example to code worth relying on — returning computed results, writing through output arguments, a CRC that earns its place, encode and decode pairs checked against each other, a payload transformed on the testbench's behalf, and state that has to survive from one call to the next.