What Actually Happens When You Run Out of Memory

"The process ran out of memory" describes several different events with different symptoms and different fixes. Distinguishing them is most of the diagnosis.

For a separate people-operations application of the same measurement discipline, see workforce optimization software.

Allocation usually succeeds even when there is no memory

On Linux, malloc returning a pointer does not mean the memory exists. The kernel overcommits: it hands out address space and only allocates physical pages when they are written to.

Consequences:

Allocation rarely fails. Code checking for a null return from malloc is checking for something that mostly does not happen.

The failure occurs at first touch, not at allocation. A program that allocates a large buffer succeeds immediately and dies later, when it writes to it — far from the code that requested it.

Reported virtual memory is not consumption. A process showing 40GB of virtual size may be using 2GB of physical memory. The number that matters is resident set size, and even that includes shared pages counted against every process mapping them.

Overcommit behaviour is configurable via vm.overcommit_memory, and the default heuristic mode is what most systems run.

The OOM killer

When physical memory and swap are genuinely exhausted, the kernel must free some, and it chooses a process to kill.

Selection is by score, roughly proportional to memory used, adjusted by oom_score_adj. The largest consumer is usually chosen, and it is not necessarily the culprit. A database using memory correctly gets killed because a leaking sidecar exhausted the machine.

Recognising it:

• The process is killed with SIGKILL, so no cleanup, no shutdown hook, no final log line from the application
• dmesg or the kernel log contains an "Out of memory: Killed process" line with the score and the memory state at the time
• Exit code 137 (128 + 9) in a container

The application logs will show nothing, because SIGKILL cannot be handled. This is the signature: an application that stops mid-sentence with no error is usually an OOM kill, not a crash.

Container memory limits behave differently

Inside a container, the limit is a cgroup limit, and the mechanism is not the same.

Hitting a cgroup memory limit kills the process in that cgroup, regardless of how much memory the host has free. The host can be at 20% utilisation while your container is killed.

Page cache counts toward the limit. A process reading large files accumulates page cache attributed to the cgroup. It is reclaimable, and under memory pressure the kernel should reclaim it rather than kill — but the interaction is version-dependent and a workload doing heavy file I/O can be killed for cache it does not need.

cgroup v2 adds pressure information. memory.pressure reports time spent stalled on memory reclaim, and it is the best early-warning signal available. It rises before anything is killed.

The common mistake is setting the limit from observed usage. A process using 500MB in testing does not need a 512MB limit — it needs headroom for peaks, for the allocator's own overhead, and for page cache. Limits set to observed usage produce kills under normal variation.

Swap: the slow failure

Before killing anything, the kernel may swap pages to disk. This is the failure mode that looks like something else entirely.

The symptom is not an error but latency. Response times rise by orders of magnitude. Everything still works, very slowly. Health checks time out, the orchestrator restarts the process, the new one warms up and starts thrashing too.

Diagnosis: high si/so in vmstat, high iowait, disk activity that does not correspond to your workload.

Many container platforms disable swap entirely, which converts this slow failure into a fast one. That is usually the better outcome — a killed process is easier to diagnose than a system that is technically alive and unusably slow.

The runtime's memory is not the process's memory

For managed runtimes, the heap is one component of resident memory, and the others are frequently what pushes a container over its limit.

In the JVM: heap, metaspace, thread stacks (one per thread, typically around 1MB), code cache, garbage collector structures, and direct byte buffers allocated outside the heap. A container limit set from the maximum heap size will be exceeded.

In Python: the interpreter, C extension allocations, and the fact that freed objects are frequently returned to the allocator's pool rather than the operating system, so resident memory does not shrink after a peak.

In Go: the runtime returns memory to the operating system on its own schedule, which historically meant resident size stayed high long after the memory was free.

The general point: watch resident set size against the limit, not the runtime's own heap metric. They diverge, and the divergence is what kills you.

Diagnosing which one you have

Killed instantly, no application logs, exit 137. OOM kill. Check the kernel log to see whether it was the host or the cgroup.

Gradual slowdown, high iowait, disk busy. Swap thrash.

Application-level out-of-memory error with a stack trace. The runtime hit its own limit — the JVM heap, for instance — and this is the easy case, because you get a trace and often a heap dump.

Killed under load but not at rest. Memory scales with concurrency: request buffers, per-connection state, thread stacks. Look at memory per in-flight request rather than at a total.

Resident memory climbing steadily over days. A leak, or a cache without a bound. In managed runtimes, an unbounded collection holding references is the usual cause — the objects are reachable, so they are not garbage.

Preventing it

Bound every cache. An unbounded cache is a leak with a justification. Size limits, entry limits, or expiry — something.

Bound concurrency. Memory use is proportional to in-flight requests. A thread pool or semaphore that limits concurrency limits memory as a side effect, and it also prevents the queue-everything failure mode.

Stream instead of buffering. Reading an entire response, file or result set into memory means memory use is determined by the largest input you ever receive. Streaming makes it constant.

Set container limits with headroom, and configure the runtime to respect them — modern JVMs and .NET are container-aware, but verify rather than assume.

Alert on memory pressure, not on usage. Usage at 80% may be perfectly healthy. memory.pressure rising means reclaim is costing you.

Load-test to failure at least once. Knowing what your system does when memory runs out — and that it is a fast kill rather than a slow thrash — is worth an afternoon and it is knowledge you otherwise acquire at 3am.

The summary

Allocation succeeds; the failure comes at first touch, far from the cause.

The OOM killer picks the largest process, not the guilty one. Exit 137 with no application logs is the signature.

Container limits are cgroup limits and fire regardless of host memory.

Swap converts a crash into a slowdown, which is harder to diagnose and often worse.

Watch resident set size against the limit, bound your caches and your concurrency, and stream rather than buffer.