What the Preprocessor Does Before the Compiler

Part I · Module 2 · Chapter 2.5 · What the Preprocessor Does Before the Compiler

1. A Separate Step With Its Own Language

Look at the two kinds of line you have been writing:

#include <iostream>        // no semicolon
int main()                 // semicolon rules apply below here

The first has no semicolon, and that is not an oversight. Lines beginning with # are preprocessor directives. They belong to a different language with different rules, processed by a different step, before compilation begins.

The preprocessor does three things and nothing else:

• Copies the contents of other files into yours — #include.
• Substitutes text you have named — #define.
• Deletes regions you asked it to skip — #ifndef and friends.

What it does not do is just as important. It does not know what a type is, what a function is, or what C++ means. It is working on text. If the text it produces happens to be nonsense, it hands over the nonsense without comment and the compiler reports the problem.

2. The Shape

The steps are drawn in order, but do not read too much into the separation — a real preprocessor works through the file once, handling each directive as it meets it. The order that matters is the one at the bottom: everything above happens before the compiler sees anything.

3. #include Is a Literal Paste

Chapter 2.2 said this line does more than it looks like. Here is the measurement.

The option is -E: preprocess only, and show me the result.

$ c++ -std=c++17 -E main.cpp -o main.i

main.cpp is the twelve-line program from Chapter 2.4. On the machine used for this chapter:

main.cpp     13 lines,       283 bytes
main.i    64,206 lines,  3,012,617 bytes

Thirteen lines became sixty-four thousand. Three hundred bytes became three megabytes.

That is what #include <iostream> did. The preprocessor found the file called iostream, pasted its entire contents into yours in place of the directive, and then did the same for every file that file included, and so on until there was nothing left to paste.

Now look at the end of the result:

64197: int main()
64198: {
64199:     const int transfers = 8;
64200:     const int cycles_per_transfer = 2;
64201:
64202:     std::cout << "transfers = " << transfers << '\n';
64203:     std::cout << "cycles    = " << cycles_for(transfers, cycles_per_transfer) << '\n';
64204:
64205:     return 0;
64206: }

Your code, character for character as you wrote it, sitting at the bottom of sixty-four thousand lines that arrived from elsewhere.

Isolating the cause

One number on its own proves nothing. So run the same command on the other file in the project — the one that includes only a small header of our own:

cycles.cpp    7 lines,   173 bytes
cycles.i     21 lines,   389 bytes

Seven lines became twenty-one. Same preprocessor, same project, same command.

The difference is entirely what was included. <iostream> describes a complete, general-purpose input and output system, and describing it takes a lot of text. cycles.hpp declares one function.

4. #define Is a Literal Substitution

The second thing the preprocessor does. The simple form gives a name to some text:

#define CYCLES_PER_TRANSFER 2

From that point on, wherever the preprocessor sees CYCLES_PER_TRANSFER, it writes 2 instead. Not "a constant equal to 2" — the character 2, substituted into the text.

There is also a form that takes arguments:

#define TOTAL(transfers)  3 + transfers * CYCLES_PER_TRANSFER

This looks like a function. It is not a function. Nothing is called, nothing is passed, nothing returns. The preprocessor finds TOTAL(8) in your text and replaces it with the replacement text, putting 8 wherever transfers appeared.

The difference between "looks like a function" and "is a text substitution" is where the trouble lives.

5. The Trap, Measured

Here is a complete program using that macro:

#include <iostream>

#define CYCLES_PER_TRANSFER 2
#define TOTAL(transfers)    3 + transfers * CYCLES_PER_TRANSFER

int main()
{
    std::cout << "TOTAL(8)     = " << TOTAL(8) << '\n';
    std::cout << "TOTAL(8) * 2 = " << TOTAL(8) * 2 << '\n';
    return 0;
}

It compiles with no warnings at -Wall -Wextra -Wpedantic. Predict the output before reading on. A burst of 8 transfers takes 19 cycles, so two of them should take 38.

Actual output:

TOTAL(8)     = 19
TOTAL(8) * 2 = 35

The first line is right. The second is wrong, by three.

Now use -E and look at what the compiler actually received:

    std::cout << "TOTAL(8)     = " << 3 + 8 * 2 << '\n';
    std::cout << "TOTAL(8) * 2 = " << 3 + 8 * 2 * 2 << '\n';

There it is. The macro's text was pasted in as-is, and then ordinary arithmetic precedence applied to the result:

• Line one: 3 + 8 * 2 — multiplication first, so 3 + 16, which is 19. Correct by luck.
• Line two: 3 + 8 * 2 * 2 — still multiplication first, so 3 + 32, which is 35. The * 2 multiplied the per-transfer cost instead of the total.

A function would not behave this way. A function computes a value and then the result is used. A macro pastes text and lets precedence sort out the pieces afterwards.

The fix

Parenthesise everything — the whole replacement, and every use of a parameter:

#define TOTAL(transfers)    (3 + (transfers) * CYCLES_PER_TRANSFER)

Output:

TOTAL(8)     = 19
TOTAL(8) * 2 = 38

The outer parentheses make the whole expansion one unit, so the * 2 applies to all of it. The inner ones protect the parameter, which matters the moment someone writes TOTAL(4 + 4).

6. What to Use Instead

Macros were the only tool for these jobs in C. C++ has better ones, and the rule in modern code is straightforward:

The reason is not style. A const int has a type, so the compiler checks it. It has a scope, so it stops existing where you stop wanting it. It has a name the debugger knows. A macro has none of those things: it is text, visible from its definition to the end of the file, invisible to the debugger, and ignorant of types.

Two jobs genuinely remain macro work:

• Include guards. You saw why in Chapter 2.4 — making text disappear is something only the preprocessor can do.
• Conditional compilation, for genuine build variation.

That second one is worth seeing once:

#include <iostream>

int main()
{
    const int total = 19;

#ifdef ENABLE_TRACE
    std::cout << "[trace] cycles = " << total << '\n';
#endif

    std::cout << "result = " << total << '\n';
    return 0;
}

Two builds of the same file:

$ c++ -std=c++17 -Wall -Wextra -Wpedantic cond.cpp -o cond && ./cond
result = 19

$ c++ -std=c++17 -Wall -Wextra -Wpedantic -DENABLE_TRACE cond.cpp -o cond && ./cond
[trace] cycles = 19
result = 19

-D defines a name from the command line, without editing the source. When it is absent, that trace line is not skipped at run time — it is not in the file the compiler saw at all. This is how one source file produces a traced and an untraced build.

Used in small amounts this is useful. Used heavily it becomes a source file with several different programs tangled inside it, and no reader can tell which one they are looking at.

7. Why This Makes Header Errors Legible

The payoff. Three things that look impossible make sense once you know the preprocessor is pasting text.

An error reported in a file you did not write. Chapter 2.4's guard failure reported the error inside cycles.hpp, with notes naming the two places it was included from. That is the only sensible place to report it: by the time the compiler ran, the header's text was physically inside your file, twice.

An error whose line number is enormous. Report an error in a file that preprocessed to 64,206 lines and the number will not match anything in your editor. The compiler tracks original file and line through the expansion, which is why you usually see sensible locations — but when you do see a strange one, this is why.

A header that works alone and breaks when another is included first. Both pasted text into the same file. Order matters because the result is one document, assembled in the order the directives appear.

The one-sentence version, which is worth memorising:

By the time the compiler starts, your file and everything it included are one piece of text. Every header question is a question about that text.

8. Mistakes Worth Making Once

Treating a function-like macro as a function. It is a text substitution. Precedence applies to the pasted result, and parameters used twice are evaluated twice.

Writing a macro without enough parentheses. 3 + transfers * 2 computed 35 instead of 38 and nothing warned. Parenthesise the whole replacement and every parameter use — or, better, write a function.

Expecting #ifdef to be a run-time decision. It is not an if. The excluded text never reaches the compiler, so it is not checked for errors either. A region that has been excluded for six months may no longer compile at all, and nobody has noticed.

Using #define for constants out of habit. A macro has no type and no scope. const int setup_cycles = 3; is better in every way. This is one of the clearer differences between C habits and C++ practice, which Chapter 1.4 §8 listed.

Putting a macro name in lower case. Macros ignore scope, so a lower-case macro can silently replace an ordinary name somewhere far away. Upper case is the near-universal convention precisely because it warns the reader.

9. What You Should Be Able to Say Now

1. Lines starting with # are preprocessor directives, not C++, and have no semicolons.
2. The preprocessor copies, substitutes and deletes text. It does not understand types, functions or arithmetic.
3. It runs before the compiler. The compiler is the first step that understands C++.
4. -E shows you exactly what the compiler received.
5. #include is a literal paste, recursively. Measured: 13 lines became 64,206; your own main was at line 64,197, unchanged.
6. The expansion comes from what you include — the same preprocessor turned a 7-line file into 21 when it included only a small local header.
7. That explains why main.o was 10,312 bytes and cycles.o was 560, and why headers that pull in a lot cost every file that includes them.
8. #define is a literal substitution. A function-like macro is not a function: nothing is called and nothing returns.
9. Measured: an unparenthesised macro gave 35 where the answer was 38, and -E showed 3 + 8 * 2 * 2 as the cause.
10. Parentheses fix precedence but not repeated evaluation — a parameter used twice is evaluated twice.
11. Prefer const/constexpr, real functions and type aliases. Macros remain correct for include guards and conditional compilation.
12. #ifdef removes text before compilation, so excluded code is not checked at all.
13. By the time the compiler starts, your file and its includes are one piece of text. Every header question is a question about that text.

10. Exercises

Exercise 1 — predict the expansion. Given #define HALF(x) x / 2, write out the text the preprocessor produces for HALF(4 + 4), then say what it evaluates to and what the author almost certainly wanted.

Exercise 2 — repair it. Fix the macro in Exercise 1 with parentheses. Then say which of the two macro traps your fix addresses and which it does not.

Exercise 3 — explain the measurement. main.cpp expanded to 64,206 lines and cycles.cpp to 21. Both went through the same preprocessor. Explain the difference in one sentence, and say what it predicts about the two compile times.

Exercise 4 — the excluded region. A block guarded by #ifdef ENABLE_TRACE has not been built for a year, and someone has since renamed a variable it uses. Does the project still compile? What happens the day someone adds -DENABLE_TRACE, and what does that tell you about where such blocks should be used?

Exercise 5 — macro or constant? A colleague defines #define SETUP_CYCLES 3. Give three specific advantages of const int setup_cycles = 3; instead. Each should name a property the macro does not have.

Exercise 6 — read the strange error. A build fails with an error inside a standard library header you have never opened, triggered by a line in your own file. Using §7, explain why this is normal rather than a broken toolchain, and describe how you would narrow it down.

Exercise 7 — the double call. A macro #define MAX(a, b) ((a) > (b) ? (a) : (b)) is fully parenthesised. Explain what goes wrong with MAX(next_sample(), limit) and why no amount of parenthesising fixes it.

11. Summary

• The preprocessor is a separate step with its own language. It copies, substitutes and deletes text, and it does not understand C++.
• -E shows the file the compiler actually received. Nothing in this chapter has to be taken on faith.
• #include is a literal, recursive paste. 13 lines became 64,206; the same tool turned a 7-line file into 21. The difference is what you included.
• That cost reappears later as slower compiles and bigger object files — main.o 10,312 bytes against cycles.o 560.
• #define is a literal substitution, not a function. An unparenthesised macro computed 35 where the answer was 38, and the expansion 3 + 8 * 2 * 2 showed exactly why.
• Parentheses fix precedence, not repeated evaluation.
• Prefer const/constexpr, functions and type aliases. Macros stay right for include guards and conditional compilation.
• Excluded text is never compiled, so it can rot unnoticed.
• Your file and its includes become one piece of text, and that is what makes header errors readable.

The preprocessor's output is where this chapter stops and the compiler's work begins. That text — one source file with everything it included pasted in — has a name, and Chapter 1.7 used it without opening it up: a translation unit. Chapter 2.6 follows one from there to the end, through the compiler and the assembler to an object file, and then watches the linker take several object files and resolve the names between them. All four stages, with the file each one produces open on screen.