Static and Shared Libraries

Part I · Module 24 · Chapter 24.6 · Static and Shared Libraries

1. Building Both

The toolchain on this machine provides ar, ranlib, nm, otool and install_name_tool, and both library forms build. (cmake, ninja, ccache and lld are not installed here; Chapter 24.8 says what that means for that chapter.)

# compile to object files
clang++ -std=c++20 -O2 -c one.cpp -o one.o
clang++ -std=c++20 -O2 -c two.cpp -o two.o
clang++ -std=c++20 -O2 -c big.cpp -o big.o

# a static library: an archive of object files, plus an index
ar rcs libutil.a one.o two.o big.o
ranlib libutil.a

# a shared library
clang++ -std=c++20 -O2 -dynamiclib -o libutil.dylib one.cpp two.cpp big.cpp \
        -install_name @rpath/libutil.dylib

The archive really is a container, and you can list it:

__.SYMDEF SORTED
one.o
two.o
big.o

__.SYMDEF SORTED is the index — a map from symbol name to the member that defines it, which ar s and ranlib generate. The linker reads that index to decide which members it needs, which is §3's whole subject. Without it, linking against the archive would require scanning every member.

Naming is a convention the toolchain depends on: libNAME.a and libNAME.dylib (.so on Linux, .dll/.lib on Windows), so that -lNAME finds them. The platform suffix differences are why a build system rather than a hand-written command is the eventual answer.

2. Linking Against Each

# static: the archive is consumed at link time
clang++ -std=c++20 -O2 app.cpp libutil.a -o app_static

# shared: a dependency is recorded, resolved at load time
clang++ -std=c++20 -O2 app.cpp -L. -lutil -Wl,-rpath,$PWD -o app_shared

The difference is visible in the finished binary:

…/out/app_shared:
	@rpath/libutil.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 2100.43.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1356.0.0)

app_shared records @rpath/libutil.dylib as a runtime dependency. app_static records none — the code is inside it. Note that even the static binary depends on libc++ and libSystem dynamically, which is the normal arrangement: the standard library and the C runtime are shared, and your own code may be either.

@rpath is the mechanism by which the binary says "look for this in the run paths recorded in me", and -Wl,-rpath,… is what records one. Getting this wrong is the commonest shared-library deployment failure: the program builds, runs on the build machine, and fails to start elsewhere with a message about a library that cannot be loaded. otool -L on the binary and install_name_tool to fix a recorded path are the tools; on Linux the equivalents are ldd, patchelf and $ORIGIN.

3. An Archive Is Searched, Not Linked

This is the mechanism everything else in the chapter follows from.

When the linker reaches an archive on the command line, it does not add the archive's contents to the program. It looks at the set of symbols that are currently undefined, consults the index, and extracts only those members that define one of them — then repeats, because an extracted member may itself have undefined symbols.

Measured: an application that calls exactly one function, against an archive whose third member holds 80 KB of initialised data that nothing references.

  big2.o                       80728 bytes
  libutil2.a                   82144 bytes
  app2_static  (archive)       33480 bytes   bigUnused present: 0
  app2_allobjs (all .o)        99648 bytes   bigUnused present: 1

Linking the archive gives a 33,480-byte binary with no trace of bigUnused. Listing the same three object files on the command line gives 99,648 bytes and includes it — 66,168 bytes larger, a factor of 2.98 — because an object file named explicitly on the link line is always included, while an archive member is included only on demand.

Two practical consequences:

Link order matters for archives. The linker processes the command line left to right and only knows about undefined symbols it has already seen, so an archive placed before the object that needs it may be searched while nothing is pending, extract nothing, and leave you with an undefined-symbol error for a function that is demonstrably in the library. The rule is: objects first, then libraries, and libraries in dependency order. Modern linkers are more forgiving than they used to be, but the ordering rule is still the correct mental model and still the first thing to check.

Granularity is the translation unit, not the function. A member is extracted whole. Putting fifty unrelated functions in one .cpp means referencing any one of them pulls in all fifty; a library that wants to be small keeps its translation units small. The linker flags -dead_strip on this platform and --gc-sections with -ffunction-sections on Linux reduce this further by discarding unreferenced sections, which is a finer granularity than the member.

4. The Process, Step by Step

Step 6 is an optimisation and step 7 is its consequence. §5 is the measurement.

5. The Self-Registration Bug

A registry that modules add themselves to at startup — the standard plugin pattern, and the one that makes a simulator's model table extensible without a central list:

// reg.h
std::vector<std::string>& registry();
bool registerModel(const char* name);

// model_axi.cpp — nothing in this TU is referenced from anywhere else
#include "reg.h"
static const bool registered = registerModel("axi");

// model_lsu.cpp
static const bool registered = registerModel("lsu");

Three link commands, same source files:

  registered models: 0
  registered models: 2
    axi
    lsu

From the archive: zero. From the same object files: two. The first of those is a program that builds without a warning, starts, finds no models, and reports that nothing is configured.

The cause is exactly step 6. Nothing in the program references model_axi.o by name — the whole point of self-registration is that no central list mentions it — so the linker's index lookup never asks for it, so it is never extracted, so the dynamic initialisation of registered never happens. The archive did nothing wrong; it applied §3's rule.

And the escape hatch, measured on the same archive:

  registered models: 2
    axi
    lsu

That is -Wl,-all_load, which forces every member of every following archive into the link. On Linux the equivalent is -Wl,--whole-archive … -Wl,--no-whole-archive, and the more surgical -Wl,-force_load,libmodels.a here or -u symbol to name a single symbol you want pulled in.

Four ways to deal with this, in increasing order of how much I would trust them:

The honest conclusion is that self-registration and static archives do not combine well, and that the pattern's appeal — no central list — is exactly what makes it fragile. If you need it, a shared library is the natural home, because §6's rule is different.

6. Shared Libraries Are Different in Four Ways

Everything in them is loaded. A shared library is an image, not a searchable container, so there is no selective extraction and §5's bug does not occur — the global constructors of every translation unit in the library run when the library is loaded. That is the main reason to put a self-registering plugin in one.

Resolution happens at load time, and can fail then. A missing or wrong-version dylib is a startup failure rather than a build failure, which moves an error from your machine to the user's. otool -L and @rpath from §2 are the tools; the failure mode is the reason static linking is attractive for a tool you ship.

Symbol interposition and visibility matter. Every non-static symbol in a dylib is exported by default, which bloats the dynamic symbol table and lets callers bind to things you considered internal. -fvisibility=hidden with explicit __attribute__((visibility("default"))) on your API is the disciplined arrangement, and it also speeds up loading.

The ABI is now a promise. Chapter 24.5 §7's table is a list of library changes that break existing binaries. For a static library that table only matters at the next rebuild; for a shared library it matters to every program already deployed against it, which is why shared-library interfaces are conservative and why extern "C" plus opaque pointers is such a common boundary design.

One further C++-specific hazard is worth naming. A static or inline variable in a header included by both a shared library and its consumer can end up duplicated, with the library and the program each having their own copy — so a singleton is not one, and state set through one does not appear through the other. Chapter 24.1 §5's weak external mechanism is what usually prevents it, and it depends on visibility settings and on both sides agreeing. If a singleton misbehaves across a dylib boundary, this is the first thing to check.

7. Choosing

For the DPI case the frozen purpose names, the point is that the simulator loads your library at run time, so it must be shared, every translation unit's initialisation runs, and the boundary has to be extern "C" — which is Chapter 24.5 §5's lost type checking accepted deliberately, and Chapter 28's subject.

8. The Table

9. What to Actually Do

• Put object files before libraries on the link line, and libraries in dependency order.
• Keep translation units small in a library, because extraction granularity is the member — and consider -dead_strip or -ffunction-sections plus --gc-sections.
• Never rely on a global constructor in an archive member nothing references. Measured: registered models: 0, with no diagnostic.
• If you need self-registration, use a shared library or accept a central list that names each module. The list is the only version that cannot fail silently.
• Use -force_load on the specific archive rather than -all_load globally, when forcing is the answer.
• Record an @rpath and check it with otool -L before shipping anything dynamic.
• Compile a dylib with -fvisibility=hidden and export your API explicitly.
• Treat a shared library's interface as a promise — Chapter 24.5 §7 is the list of changes that break it.
• Suspect header-level static/inline state first when a singleton misbehaves across a dylib boundary.
• Prefer static for tools you distribute and shared for code a host process loads.

10. Mistakes Worth Making Once

A library before its consumers on the link line. An undefined symbol for a function that is unquestionably in the library.

Self-registration in a static archive. The registry is empty and the build was clean.

-all_load as the fix. It works and pulls in everything, including what you removed on purpose.

Fifty functions in one .cpp of a library. Referencing one pulls in all fifty.

Shipping a dylib without checking otool -L. It runs on your machine.

Assuming a static binary has no runtime dependencies. libc++ and libSystem are still dynamic.

Exporting everything from a dylib. A large dynamic symbol table and callers bound to your internals.

Adding a defaulted parameter to a shared library's function. Chapter 24.5 §7 — the symbol changed, and deployed callers cannot find it.

A header-level singleton across a dylib boundary. Two copies, and the one you set is not the one you read.

11. Interview Relevance

The frozen curriculum marks this chapter High, and the opening question is usually the easy one: "what is the difference between a static and a shared library?" Size, distribution and when resolution happens is the expected answer and is table stakes.

What distinguishes a strong answer is the mechanism: an archive is searched, not linked. The linker extracts only members that resolve a pending undefined symbol, which is why link order matters, why extraction granularity is the translation unit, and why the static binary measured 2.98× smaller than one built from the same object files listed explicitly.

The best thing to be able to tell is the self-registration bug, because it is a real debugging story with a measured outcome: the same object files gave registered models: 2 when listed and 0 when archived, with no warning at either compile or link time, and the fix is -force_load, a central list, or a shared library. Explaining why the linker cannot do better — the dependency exists only in the author's intent and nowhere in the artefacts — is the part that shows understanding rather than recall.

12. What You Should Be Able to Say Now

• That a .a is object files plus a symbol index (__.SYMDEF SORTED), and a dylib is a loadable image.
• That an archive is searched, not linked — only members resolving a pending symbol are extracted, repeatedly until nothing is pending.
• That this measured 33,480 against 99,648 bytes, a factor of 2.98, and that it is why link order matters and why extraction granularity is the translation unit.
• That a static binary still depends dynamically on libc++ and libSystem.
• That an archive member nothing references is dropped together with its global constructors — measured registered models: 0 against 2 from the same objects, with no diagnostic.
• The four responses — -all_load, -force_load, -u symbol, a central list — and that only the central list cannot fail silently.
• That a shared library loads everything, so the bug does not occur there, and that this is why self-registering plugins belong in one.
• That a dylib's resolution can fail at load time on someone else's machine, and that otool -L, @rpath and install_name_tool are the tools.
• That a dylib exports everything by default, and that -fvisibility=hidden plus explicit exports is the disciplined arrangement.
• That a header-level static/inline variable can be duplicated across a dylib boundary, breaking a singleton.

13. Exercises

Exercise 1 — selectivity. Build §3's archive with one large unreferenced member. Compare the binary against one built from the object files. Confirm with nm that the member's symbol is absent from the archive-linked one.

Exercise 2 — link order. Put the archive before the object that needs it and record the error. Then swap them.

Exercise 3 — the registry. Reproduce §5 all three ways. Then add -u with the mangled name of one registering variable and confirm exactly one model appears.

Exercise 4 — the shared version. Put the two registering translation units in a dylib, link against it, and confirm both models register with no special flags.

Exercise 5 — deployment. Build app_shared, run otool -L, then move the dylib and run the program. Read the failure. Fix it with install_name_tool or an @rpath.

Exercise 6 — granularity. Put two unrelated functions in one library translation unit, reference one, and check whether the other's code is in the binary. Then split them into two files and repeat.

14. Summary

• A static library is an archive of object files plus a symbol index; a shared library is a loadable image. Measured, ar rcs produced __.SYMDEF SORTED and three members.
• An archive is searched, not linked. The linker extracts only members that resolve a currently undefined symbol, repeating until nothing is pending — which makes link order significant and makes the translation unit the unit of extraction.
• That selectivity is worth a lot. Measured: 33,480 bytes from the archive against 99,648 from the same three object files listed explicitly — 2.98×, because an 80 KB unreferenced member was never pulled in.
• And it is also a bug. Two translation units whose only content is a self-registering global constructor gave registered models: 0 when linked from the archive and 2 when linked from the same object files — with no warning at compile or link time. -Wl,-all_load restored both.
• The linker is not at fault: nothing referenced those members, which is exactly what self-registration arranges. The dependency exists in the author's intent and in none of the artefacts, so no mechanism can see it. The reliable fix is a central list that names each module; a shared library is the next best, because everything in a dylib is loaded.
• Shared libraries differ in four ways: no selective extraction, resolution at load time on the user's machine, default-exported symbols (use -fvisibility=hidden), and an ABI that is a promise to every deployed caller.
• Watch for a header-level static/inline variable duplicated across a dylib boundary — a singleton that is not one.
• Choose static for tools you ship and shared for code a host process loads — which is why a simulator-loaded DPI model is a shared library with an extern "C" boundary.

Chapter 24.7 turns to the flags that decide what the compiler tells you and what it does with undefined behaviour — including the same expression producing two different answers depending on whether it was folded or executed.