The Cost of Dispatch: Vtables as a Mental Model

Part I · Module 17 · Chapter 17.7 · The Cost of Dispatch: Vtables as a Mental Model

1. The Model

Each polymorphic class has one table of function addresses, shared by every object of that class. Each object carries one pointer to its class's table. A virtual call loads the pointer, reads a fixed slot, and branches.

object                     Model's table
+---------------+          +------------------------+
| table pointer | -------> | slot 0: ~Model         |
| id            |          | slot 1: name()         |
| ...           |          | slot 2: step()         |
+---------------+          +------------------------+

Chapter 17.2 §4 measured the three instructions that walk that picture. This chapter is about what they cost.

2. What the Object Pays

From Chapter 17.1 §6:

sizeof(Empty)   = 1
sizeof(One)     = 8      (1 virtual + dtor)
sizeof(Five)    = 8      (5 virtuals + dtor)
sizeof(Twenty)  = 8      (20 virtuals + dtor)
sizeof(Both)    = 16     (TWO polymorphic bases)

One pointer per object, regardless of how many virtual functions. The table itself is per-class and exists once in the program, so twenty virtual functions cost twenty slots once — not twenty slots per object.

The 8 bytes is the cost worth actually thinking about, and the condition is population. A type you create a handful of pays nothing noticeable. A type you create ten million of pays 80 MB, and if the type was otherwise two ints you have tripled it.

3. What the Call Pays: The Instructions

viaVirtual(const VB& b) { return b.get(); }
    ldr  x8, [x0]          ; the table pointer
    ldr  x1, [x8, #16]     ; the slot
    br   x1                ; branch

viaPlain(const Plain& p) { return p.get(); }
    mov  w0, #2
    ret

Three instructions against two — but notice which two. The direct version did not call anything; it was inlined to a constant. That is the whole subject of this chapter, and §4 and §5 are the two ways of reading it.

4. The Comparison People Quote

100,000,000 calls each, with a compiler barrier, three runs:

non-virtual, static type :  30.4 / 30.6 / 30.5 ms   (0.304 ns per call)
virtual, through base ref:  91.4 / 91.4 / 91.8 ms   (0.914 ns per call)
ratio                    :  3.00 / 2.98 / 3.01 x
difference               :  ~0.609 ns per call

Three times slower, reproducibly. This is the number that circulates, and it is correct — for what it measures.

What it measures is the non-virtual call being inlined to a constant versus the virtual call being a real call. The 0.304 ns is not the cost of a function call; it is the cost of a loop iteration doing almost nothing, because Plain::get disappeared.

5. The Comparison That Isolates the Indirection

Make the non-virtual call non-inlinable, so both sides are real calls:

    struct Plain { __attribute__((noinline)) int get() const { return 2; } };

non-virtual, NOINLINE call :  91.8 / 91.7 ms   (0.918 / 0.917 ns per call)
virtual, through base ref  :  92.5 / 92.3 ms   (0.925 / 0.923 ns per call)
ratio                      :  1.01x
difference                 :  0.007 / 0.006 ns per call

One per cent. Six picoseconds.

So the two costs separate cleanly, and they are wildly different in size:

Virtual dispatch is not slow. Not being inlined is slow. A virtual call is slow only in comparison to a call that would have vanished.

That reframing matters because it tells you when to care. If the function is big enough that it would not have been inlined anyway — reading a file, stepping a model, formatting output — dispatch costs you one per cent of a call, which is nothing. If the function is a one-line accessor in a tight loop, the inlining is the thing you lost, and it is worth 3x.

6. Why Lost Inlining Costs More Than the Branch

The branch itself is cheap because a modern core predicts it well: the same call site usually reaches the same target, and the predictor learns that. Six picoseconds is roughly what a correctly-predicted indirect branch costs.

Inlining is worth more because it is not just the call overhead. Once the body is in place the compiler can constant-fold through it, keep values in registers across what was a call boundary, eliminate dead work, and vectorise a loop that previously contained a call. §3's mov w0, #2 is all four of those at once: the function became a constant, so nothing remained.

That is the honest shape of the cost. Not an indirection you pay, but an optimisation you forgo — and the size of what you forgo depends on how much the compiler could have done with the body, which is why a one-line getter is the worst case and a substantial function is the best.

7. What final Recovers

Chapter 17.3 §10 measured this and deferred the explanation:

viaBaseRef (const VB&)  3 instr  ldr / ldr / br      dispatched
viaDerived (const VD&)  3 instr  ldr / ldr / br      dispatched
viaFinalFn (const VDF&) 2 instr  mov w0,#3 / ret     devirtualised + inlined
viaFinalCls(const VDS&) 2 instr  mov w0,#4 / ret     devirtualised + inlined

Two things worth reading carefully.

An exact static type is not enough. viaDerived takes a const VD& and still dispatched, because a reference to VD may refer to an object of some class derived from VD in another translation unit. The compiler cannot prove otherwise.

final is that proof, and what it buys is not the removal of three instructions — it is inlining. mov w0,#3; ret means the body was substituted and folded, which is §6's list. So final does not save you the branch; it gives you back the optimisation the branch was preventing.

That makes final on concrete leaf classes a cheap performance measure as well as the design statement Chapter 17.3 §10 described — and it only helps where the compiler can see the exact type, which is the call sites that were not really polymorphic anyway.

8. The Honest Caveats

Four, because a benchmark in a chapter is a weak instrument and should be labelled as one.

This is one target. Apple clang on arm64. The numbers will differ on x86-64, on a different compiler, and on a different core.

This is the best case for prediction. The loop called the same target every iteration, so the branch predictor was always right. A call site that alternates unpredictably between implementations costs a mispredict, which is tens of cycles rather than six picoseconds — and that is a real effect this measurement does not capture.

This measures nothing about cache behaviour. A table lookup touches a second cache line; in a loop that already fits in cache this is free, and in a cache-hostile access pattern it is not. Module 25 is where performance is a subject rather than an aside.

And a microbenchmark answers a microbenchmark's question. The right instrument for "is dispatch costing me anything" is a profiler on the real program, which will usually say no.

9. The Model Is Not a Guarantee

The frozen description of this chapter insists on this and it is worth stating plainly: the table picture is how every mainstream implementation works, and the standard does not require it.

What the standard specifies is behaviour — a virtual call reaches the final overrider for the object's dynamic type. It says nothing about tables, pointers, slots, or offsets. An implementation could use a hash of a type identifier, or inline caches, or anything else that produces the right answer.

The practical consequences of that distinction:

Reason with the model, never encode it. It is excellent for predicting cost and explaining behaviour, and it is not something to rely on in code. Casting a polymorphic object to void** to read its table pointer is undefined behaviour that works until it does not.

sizeof is not specified to be one pointer — measured 8 here, but that is this implementation.

And one real linkage note the model explains. -Weverything reported, on these very files:

warning: 'Derived' has no out-of-line virtual method definitions; its vtable
         will be emitted in every translation unit [-Wweak-vtables]

A class whose virtual functions are all defined inline gives the linker no single place to put the table, so it is emitted everywhere and merged. It is a size and link-time concern rather than a correctness one, and the conventional fix is to define one virtual function — often the destructor — in a .cpp file. It is also a nice demonstration that the table is a real artefact with real linkage, even though the standard never mentions it.

10. When the Cost Is Worth Avoiding

Given §5, the list is short and specific.

A tiny function called in a hot loop, where the lost inlining is the 3x. This is the only common case, and the fix is usually not to abandon polymorphism but to move the loop inside one virtual call — dispatch once, iterate a million times, rather than dispatching a million times.

A type created in enormous numbers, where the 8 bytes per object is the cost rather than the call. Chapter 17.1 §6's measurement.

A genuinely closed type set, where you are paying for openness you do not use. Module 18's templates give you the abstraction with the inlining intact, and Chapter 18.1 compares the two properly.

And what is not on the list: a virtual call on anything substantial. One per cent of a call you were going to make is not a reason to complicate a design that Chapter 17.1 §5 measured as letting already-compiled code drive implementations added later.

11. What to Actually Do

Separate the two costs before quoting either: the indirection is ~1%, the lost inlining is 3x.

Expect one pointer per object, and think about it only when the population is large.

Mark concrete leaf classes final — it restores inlining where the type is knowable.

Dispatch once and loop inside, rather than dispatching per iteration.

Use a profiler on the real program, not a microbenchmark, to decide whether any of this matters.

Reason with the table model and never encode it — it is not a standard guarantee.

Define one virtual function out of line if -Wweak-vtables matters to your build.

And do not avoid polymorphism on cost grounds where the implementation set is genuinely open.

12. Mistakes Worth Making Once

Quoting 3x as the cost of a virtual call. Measured: that is the lost inlining; the indirection is 1.01x.

Comparing against an inlined call without saying so. The baseline was a constant.

Believing the branch is the expensive part. Measured: 0.006 ns.

Assuming an exact static type devirtualises. Measured: still 3 instructions.

Adding virtual to a type created in millions without counting the 8 bytes.

Dispatching per loop iteration when you could dispatch once.

Encoding the vtable layout in code. It is a model, not a guarantee.

Benchmarking with no barrier. Chapter 14.8 §4 — the fast case vanishes and the ratio becomes nonsense.

Generalising this measurement to another target — one compiler, one core, one predictable call pattern.

Treating -Wweak-vtables as a correctness problem. It is about linkage and size.

13. Interview Relevance

"What does a virtual call cost?" Roughly one per cent against a comparable non-inlinable direct call — measured at 0.006 ns here — plus a lost inlining opportunity that can be worth 3x. Separating those two is the answer that stands out.

"Why do people say virtual functions are slow?" Because the usual comparison is against a call the compiler inlined away, which measures inlining rather than dispatch.

"How big is the per-object cost?" One pointer, regardless of how many virtual functions — measured 8 bytes for one, five and twenty.

"Does final make calls faster?" It permits devirtualisation, and what that actually buys is inlining — measured, three instructions collapsing to a folded constant.

"Is the vtable standardised?" No. The standard specifies the behaviour; the table is how every mainstream implementation achieves it.

"When would you avoid dispatch for performance?" A tiny function in a hot loop, a type created in enormous numbers, or a closed type set — where a template is the better tool.

"What would you measure?" A profiler on the real program; a microbenchmark answers a microbenchmark's question.

14. What You Should Be Able to Say Now

1. One table per class, one pointer per object, and a call loads the pointer, reads a fixed slot, and branches.
2. Measured: the pointer is 8 bytes for one, five or twenty virtual functions; 16 for two polymorphic bases.
3. Measured, the quoted comparison: virtual 0.914 ns against non-virtual 0.304 ns — 3.00x, reproducible over three runs.
4. But the non-virtual side was inlined to mov w0,#2 — a constant, not a call.
5. Measured, the fair comparison: both non-inlinable gave 0.918 ns against 0.925 ns — 1.01x, a difference of 0.006 ns.
6. So the indirection is ~1% and the lost inlining is 3x — two different costs of very different sizes.
7. Dispatch is not slow; not being inlined is slow.
8. Inlining is worth more than the branch because it enables constant folding, register allocation across the call, dead-code elimination and vectorisation.
9. Measured: an exact static type still dispatched; final devirtualised and inlined — 3 instructions to a folded constant.
10. So final's benefit is restored inlining, not a saved branch.
11. Caveats: one target, a perfectly predicted branch, no cache effects measured, and a microbenchmark's scope.
12. The table model is not a standard guarantee — the standard specifies behaviour only.
13. -Wweak-vtables reflects a real linkage artefact; define one virtual function out of line.
14. Avoid dispatch for cost only in a hot loop over a tiny function, with an enormous object population, or a closed type set.
15. Dispatch once and loop inside is the usual fix.
16. A ratio without its denominator is not a measurement.

15. Exercises

Exercise 1 — the model. Draw the object, the pointer and the table for a two-implementation hierarchy, and mark which part is per-object and which per-class.

Exercise 2 — the per-object cost. Give sizeof for a class with one, five and twenty virtual functions, and for one with two polymorphic bases.

Exercise 3 — the two comparisons. State both ratios, say what each baseline was doing, and explain why they do not conflict.

Exercise 4 — design the fair test. Describe how you would isolate the indirection from the inlining, and say what the barrier is for.

Exercise 5 — what inlining buys. List four optimisations that become possible once a body is inlined, and point to the measurement showing all four at once.

Exercise 6 — final. Explain what final proves to the compiler and what it actually recovers, with the instruction counts.

Exercise 7 — the caveats. Name the four limits of this chapter's benchmark and say which one you think matters most in a verification environment.

Exercise 8 — the model's status. State one thing the standard guarantees and two things the table picture only describes.

Exercise 9 — decide. For each, say whether dispatch cost is worth considering: a step() called once per simulation cycle on a thousand models; a one-line id() in an inner loop over ten million transactions; a write() that formats a log line.

16. Summary

• One table per class, one pointer per object — measured 8 bytes for one, five or twenty virtual functions.
• Measured, the quoted number: 3.00x — but the baseline had been inlined to a constant, so it measures inlining, not dispatch.
• Measured, the fair number: 1.01x, a difference of 0.006 ns per call. The indirection is nothing.
• Dispatch is not slow; not being inlined is slow, and inlining is worth more because of what it enables beyond the call.
• Measured: an exact static type still dispatched; final devirtualised and inlined — so final restores the optimisation rather than saving the branch.
• Caveats stated: one target, a perfectly predicted branch, no cache measurement, microbenchmark scope.
• The table is an implementation model, not a standard guarantee — reason with it, never encode it.
• Avoid dispatch for cost only in a hot loop over a tiny function, at enormous object counts, or where the type set is closed.
• A ratio without its denominator is not a measurement.

The mechanism is understood and priced, and one capability remains unexamined. Everything in this module has gone one way: client code holds a base reference, calls through the interface, and never learns what it is actually holding. That is the design the module exists to teach. But the language also provides the reverse — a way to ask, at run time, whether the object in front of you really is a particular derived type, and to obtain a usable pointer to it if so. The next chapter is that facility, which is less often needed than reached for. It measures what the checked cast does on success and on failure, what the unchecked alternative does instead, which is nothing, and what both cost — the failing cast turning out to be six times more expensive than the succeeding one, for a reason that follows directly from what it has to prove. And it takes seriously the observation in its own frozen description: that needing to ask is usually a statement about the interface rather than about the object.