Python Testing & Debugging Beyond unittestAdvanced Python testing playbooks
Debugging & Performance

Catching Per-Test Memory Growth in pytest

Memory leaks in test suites announce themselves in unhelpful ways. The CI job's memory climbs steadily until the runner kills it at the 80th percentile of the suite, or pytest-xdist workers start dying with no traceback, or the whole run just gets slower as the garbage collector works harder. The symptom appears at the end; the cause is some test, or the code it exercises, that keeps a little memory alive every time it runs. With several thousand tests, finding it by bisection takes an afternoon.

A measuring fixture finds it in one run. Record memory before and after every test, collect garbage first so only genuinely retained memory counts, and report the tests with the largest growth at the end of the session. Most results are harmless one-off caches; the few that matter show up clearly once you run them repeatedly and see the growth scale with the number of runs.

Prerequisites

Solution

Python
# conftest.py — opt-in with: pytest --memgrowth
import gc
import os
import tracemalloc

import psutil
import pytest

_growth: list[tuple[int, int, str]] = []
_proc = psutil.Process(os.getpid())

def pytest_addoption(parser):
    parser.addoption("--memgrowth", action="store_true", help="report per-test memory growth")

@pytest.fixture(autouse=True)
def _measure_memory(request):
    if not request.config.getoption("--memgrowth"):
        yield
        return
    if not tracemalloc.is_tracing():
        tracemalloc.start(10)
    gc.collect()
    traced_before, _ = tracemalloc.get_traced_memory()
    rss_before = _proc.memory_info().rss
    yield
    gc.collect()
    traced_after, _ = tracemalloc.get_traced_memory()
    rss_after = _proc.memory_info().rss
    _growth.append((traced_after - traced_before, rss_after - rss_before, request.node.nodeid))

def pytest_terminal_summary(terminalreporter):
    if not _growth:
        return
    terminalreporter.section("memory growth (retained after test)")
    for traced, rss, nodeid in sorted(_growth, reverse=True)[:15]:
        terminalreporter.write_line(f"{traced / 1024:>9.1f} KiB traced  {rss / 1024:>9.1f} KiB rss  {nodeid}")
Bash
pytest --memgrowth -p no:randomly -q
# then confirm a suspect:
pytest --memgrowth --count=50 "tests/test_export.py::test_csv_export"
Measuring retained memory around each test For each test, the fixture runs a garbage collection and records traced memory and resident set size, lets the test run, collects again and records both values a second time. The differences are stored per test and the largest are printed in the terminal summary at the end of the session. Collect, measure, run, collect, measure gc.collect() measure before test runs gc.collect() measure after record delta traced · rss · nodeid At session end: the fifteen largest deltas, printed in the terminal summary.
Collecting before both measurements means cyclic garbage waiting for the collector never counts as growth.

Why this works

The two measurements complement each other. tracemalloc.get_traced_memory() returns the bytes currently allocated through Python's allocator — exact, unaffected by allocator caching, but blind to memory allocated by C extensions. RSS from psutil counts every page the process holds, native or not, but it is coarse: Python's small-object allocator and glibc's malloc both keep freed memory for reuse, so RSS often stays flat when objects are freed and jumps in steps when new arenas are mapped. A test that shows traced growth has retained Python objects; a test that shows RSS growth with no traced growth points at native memory, which is a job for memray.

Starting tracemalloc once, lazily, rather than per test, avoids repeatedly paying its startup cost and keeps the traced baseline continuous across the session. The frame depth of 10 costs some overhead, which is why the fixture is behind an opt-in flag: a normal run pays nothing.

Separating leaks from caches

The first report is always noisy. The first test that imports a heavy module shows megabytes of growth; the first test to call a function decorated with functools.lru_cache fills the cache; the first request through a web client builds a connection pool. None of that is a leak. It happens once and then stays constant.

A leak grows every time. The confirmation step is to run a suspect repeatedly — pytest-repeat's --count=50 does this — and look at the per-run deltas. A cache shows one large delta and forty-nine near zero. A leak shows fifty similar deltas, and total growth proportional to the count. That linear signature is the one that matters, because it is the one that eventually kills a long-running process.

Cache versus leak under repetition Two lines plot cumulative memory over fifty repetitions of one test. The cache line jumps on the first run and then stays flat. The leak line rises by a similar amount on every run, forming a straight upward slope. Run it fifty times and look at the slope repetition retained memory cache: one step, then flat leak: grows every run
Only the straight slope needs fixing; the one-off step is a cache doing its job.

Explaining a confirmed leak

Once repetition confirms a leak, the question becomes what is being retained and who holds it. tracemalloc already has the first half of the answer, because it has been recording stacks. Take a snapshot after a few warm-up repetitions and another after many more, and compare them grouped by traceback:

Python
tracemalloc.start(25)
for _ in range(5):
    run_suspect()                   # warm caches
gc.collect(); first = tracemalloc.take_snapshot()
for _ in range(200):
    run_suspect()
gc.collect(); second = tracemalloc.take_snapshot()
for stat in second.compare_to(first, "traceback")[:3]:
    print(stat)
    print("\n".join(stat.traceback.format()))

The top entry is typically a single allocation site whose count grew by exactly 200, or a multiple of it — one leaked object per run. Its traceback shows where the object was created. The second half of the answer — what keeps it alive — comes from the garbage collector: gc.get_referrers(obj) on one of those objects, or objgraph.show_backrefs for a picture of the reference chain back to a module-level name. The chain usually ends at something global: a registry, a class attribute, a logging handler list, an lru_cache on a method that captures self.

From suspect test to reference chain Four steps lead from a suspect to an explanation. The growth report names a test. Repetition confirms linear growth. A snapshot comparison grouped by traceback shows the allocation site with one new object per run. A backreference graph traces what keeps those objects alive back to a global such as a registry or class attribute. Where, then who growth report names a test --count=50 linear growth compare_to allocation site backrefs the global holding it The chain almost always ends at module-level state.
tracemalloc answers where the object was created; the garbage collector answers why it is still alive.

Leaks that live in the tests themselves

A surprising share of per-test growth comes from test code rather than the application. The recurring causes are worth checking first, because they are quick to fix.

Mocks record every call, with arguments, in call_args_list; a module-level mock that is never reset accumulates every argument passed to it across the whole session, including large payloads. Signal handlers and event listeners registered in a test and never disconnected keep their closures — and everything those closures reference — alive. Logging handlers added to a logger in a test, without removal in teardown, keep their buffers and formatters. And objects stashed on request.config or in a module-level list "for debugging" live until the session ends.

The common fix is to put each of these behind a fixture with teardown. monkeypatch, mocker from pytest-mock, and caplog all clean up automatically; hand-rolled patching and registration usually do not. When the growth report points at a test and the application code looks innocent, look at the test's setup first. A useful habit is to grep the suspect module for patch( calls outside with blocks or decorators, .connect( and .addHandler( without matching teardown, and module-level lists or dicts that tests append to; in practice one of those four patterns explains most of the growth that turns out to belong to the tests themselves rather than the product.

Keeping it from coming back

A leak found and fixed tends to return in a different form unless something checks for it. Two lightweight guards work well. The first is a scheduled CI job that runs the suite with --memgrowth weekly and posts the top of the report somewhere visible; nobody needs to act on it every week, but a new entry near the top is easy to spot. The second is a targeted regression test for each fixed leak: run the operation a few hundred times inside the test, measure traced memory before and after with a warm-up first, and assert that the growth stays below a small bound. That test is cheap, deterministic enough to live in the normal suite, and fails exactly when the specific leak reappears, with the tracemalloc machinery already in place to explain it. Pair it with a comment linking the original investigation, so whoever sees it fail next knows what they are looking at.

Edge cases and failure modes

  • Test order effects. With random ordering, the first-use cost moves between tests and the report changes every run. Disable randomisation (-p no:randomly) for measurement runs.
  • Fixtures with wider scope. A module-scoped fixture's allocations appear in the first test that uses it. Growth attributed to a test may belong to its fixtures; check with --setup-show.
  • xdist. Each worker measures independently and the summary hook runs per worker. Run memory measurement without -n, or aggregate results through pytest_testnodedown.
  • Native growth only. RSS growth with flat traced memory means C-level retention — an extension cache or a leak in native code. Profile with memray --native.
  • Overhead distorting timing. tracemalloc slows allocation-heavy tests noticeably. Never combine memory measurement with timing assertions.

Frequently Asked Questions

How do I find which test is leaking memory? Measure memory before and after each test with an autouse fixture, record the difference, and report the tests with the largest retained growth. Running each suspicious test repeatedly then separates true leaks, which grow every run, from one-off caches that grow once.

Should I use tracemalloc or RSS for per-test measurement? tracemalloc gives precise Python allocation counts and the lines responsible, but misses native memory and adds overhead. RSS includes everything but is noisy because allocators keep freed memory. Use RSS to find suspects cheaply and tracemalloc to explain them.

Why does the first test in a module always show growth? Imports, lazily built caches, compiled regexes and fixture setup allocate on first use and are kept for the rest of the run. That is expected; a leak is growth that repeats every time the same test runs.

← Back to Memory Profiling with tracemalloc