What the Compiler Does Between Your Code and the Machine

Knowing roughly what happens between source and execution explains a set of behaviours that otherwise look arbitrary: why a benchmark reports impossible speed, why a debug build behaves differently, why the same code is fast in one place and slow in another, and why some bugs appear only with optimisation enabled.

For a separate people-operations perspective, the additional reading covers further reading on time management.

The stages

Lexing turns characters into tokens. x = 1 + 2 becomes identifier, equals, number, plus, number.

Parsing turns tokens into a tree reflecting structure. Precedence is resolved here: 1 + 2 * 3 becomes addition-of-1-and-(multiplication-of-2-and-3), not the other grouping. Syntax errors come from this stage, and their frequently poor quality is because the parser knows the grammar and nothing about intent.

Semantic analysis resolves names to declarations, checks types, and reports the errors that are about meaning rather than form. This is where type systems do their work — see what type systems catch.

Intermediate representation. The tree is lowered into a simpler form: control flow becomes explicit blocks and jumps, expressions are broken into elementary operations, and language-specific constructs are expanded. Almost all optimisation happens here, on a representation deliberately simpler than the source.

Optimisation. Passes rewrite the IR into an equivalent but cheaper form. Discussed below, because this is where the surprises live.

Code generation. IR becomes machine instructions for a target architecture, including register allocation — deciding which values live in the small number of CPU registers and which are spilled to memory. Register allocation is one of the largest determinants of generated code quality.

Ahead-of-time versus just-in-time changes when this happens, not what. A JIT observes the running program and can optimise using facts an ahead-of-time compiler cannot know — which branch is actually taken, what concrete type actually appears at a call site — and can revert those optimisations if the assumption stops holding.

The optimisations worth knowing about

Not an exhaustive list — the ones whose effects you will observe.

Constant folding and propagation. Expressions computable at compile time are computed then. int x = 60 * 60 * 24; compiles to a constant.

Dead code elimination. Code whose result is never used is removed. This is why benchmark loops disappear. If you compute a value and never observe it, the computation is not required to happen, and reporting "0.3 nanoseconds per iteration" means the loop was deleted. Consuming the result in a way the compiler cannot see through is the fix.

Inlining. A function call is replaced by the function's body. This removes call overhead and — more importantly — exposes the body to the caller's context, enabling further optimisation. Inlining is frequently the enabler for everything else, which is why it is one of the most impactful decisions a compiler makes.

Loop optimisations. Invariant computations hoisted out, loops unrolled to reduce branch overhead, sometimes vectorised into instructions operating on several values at once.

Common subexpression elimination. The same computation appearing twice is done once.

Escape analysis. If an object provably does not outlive the function that created it, it can be allocated on the stack rather than the heap, or eliminated entirely. In managed languages this is a large win and it explains why allocation in a hot loop is sometimes free and sometimes not.

Devirtualisation. A call through an interface, where the concrete type turns out to be known, becomes a direct call and can then be inlined.

Why the optimiser is allowed to do this

The rule is that observable behaviour must be preserved, and "observable" is defined by the language specification, not by intuition.

Anything the specification does not define is fair game. This is why undefined behaviour is more consequential than it first appears: the compiler is entitled to assume it does not occur, and to optimise on that basis.

The classic case in C and C++: signed integer overflow is undefined, so the compiler may assume x + 1 > x is always true, and delete a check that was written specifically to detect overflow. The check was written by someone who reasoned about the hardware; the compiler reasons about the specification.

Practical consequences:

Undefined behaviour bugs frequently appear only with optimisation on. The unoptimised build did the naive thing; the optimised build exercised the assumption.

Reproducing them requires the same flags. "Works in debug" is a symptom, not a defence.

Sanitisers are the right tool. Address, undefined-behaviour and thread sanitisers find these where reading the code does not.

Memory ordering in concurrent code is specified, and both compiler and CPU may reorder within what the specification permits. Code that appears correct by reading it can be reordered into incorrectness. This is what atomics and memory barriers exist for, and it is why "it worked on my machine" is especially weak for concurrency.

Where the time goes at compile time

Useful when builds are slow.

Parsing is fast. Rarely the problem in itself.

Header and module inclusion can be enormous in C and C++. A source file that includes a widely-used header may compile a large amount of code repeatedly. Precompiled headers, forward declarations and modules address this.

Template and generic instantiation. Each distinct instantiation is compiled separately. Heavy generic code multiplies work, and it is a common cause of surprising build times.

Optimisation passes are where the time is, and they are why release builds take much longer than debug builds.

Link-time optimisation moves work to the link stage, where the whole program is visible. Better output, slower and less parallel builds — and it can enable cross-module inlining that changes performance substantially.

Incremental compilation avoids redoing unchanged work, which is why a change to a widely-included header triggers a long rebuild while a change to one implementation file does not.

Reading what was generated

More accessible than people expect, and occasionally the only way to answer a question.

Compiler explorer tools show the assembly for a source snippet, side by side, across compilers and flags. Ten minutes with one teaches more about optimisation than any amount of reading.

Optimisation reports — most compilers can explain why a loop was not vectorised or a function was not inlined. This is more useful than guessing, and the reasons are usually specific and fixable.

Managed runtimes can print the JIT's output too. Both the JVM and .NET can dump generated code, and the JVM can log inlining decisions, which is frequently the interesting part.

What to look for, if the question is performance: whether the hot function was inlined, whether the loop was vectorised, whether values are being spilled to memory rather than kept in registers.

What this changes in practice

Do not micro-optimise things the compiler already handles. Manual loop unrolling, replacing multiplication with shifts, and caching a value in a local are usually either done for you or actively counterproductive. Write clearly and check the output.

Do optimise things it cannot do. Algorithms, data layout, memory access patterns, and the number of times you cross a network boundary. The compiler will not turn your quadratic algorithm into a linear one, and it will not fix an N+1 query.

Take undefined behaviour seriously, and run sanitisers.

Benchmark with the flags you ship, and consume your results.

Expect debug and release to differ, and treat a bug that appears only in release as a real bug rather than a build-system quirk.

The summary

Six stages, and the optimiser in the middle explains most of the surprises.

The compiler preserves behaviour the specification defines, and nothing else — which is what makes undefined behaviour dangerous rather than merely untidy.

Your benchmark loop was deleted because you did not use the result.

Read the generated code when it matters. It is one tool away and it answers questions that reading the source cannot.