[{"data":1,"prerenderedAt":435},["ShallowReactive",2],{"page-\u002Fsystematic-debugging-performance-profiling\u002Fmemory-profiling-with-tracemalloc\u002Ffinding-memory-leaks-with-tracemalloc-snapshots\u002F":3},{"id":4,"title":5,"body":6,"description":401,"extension":402,"meta":403,"navigation":85,"path":431,"seo":432,"stem":433,"__hash__":434},"content\u002Fsystematic-debugging-performance-profiling\u002Fmemory-profiling-with-tracemalloc\u002Ffinding-memory-leaks-with-tracemalloc-snapshots\u002Findex.md","Finding Memory Leaks with tracemalloc Snapshots",{"type":7,"value":8,"toc":394},"minimark",[9,18,23,54,58,63,218,226,242,257,268,272,290,294,352,356,362,368,384,390],[10,11,12,13,17],"p",{},"A service's resident memory climbs steadily and never plateaus; restarts are the only mitigation. The leak is not a crash, so there is no traceback to follow — just a number going up. ",[14,15,16],"code",{},"tracemalloc"," snapshots turn that into an exact line: bracket the suspect operation with two snapshots, diff them, and the line whose retained bytes grew with iteration count is your leak.",[19,20,22],"h2",{"id":21},"prerequisites","Prerequisites",[24,25,26,45],"ul",{},[27,28,29,33,34,36,37,40,41,44],"li",{},[30,31,32],"strong",{},"Python 3.4+"," for ",[14,35,16],{},"; ",[30,38,39],{},"3.6+"," for the ",[14,42,43],{},"compare_to"," ordering used here.",[27,46,47,48,53],{},"The snapshot and statistics basics from ",[49,50,52],"a",{"href":51},"\u002Fsystematic-debugging-performance-profiling\u002Fmemory-profiling-with-tracemalloc\u002F","memory profiling with tracemalloc",".",[19,55,57],{"id":56},"solution","Solution",[10,59,60,61,53],{},"The technique relies on the fact that a real leak grows linearly with iteration count while warm-up allocations (caches, interned strings, lazy imports) are one-time. Warm up first, snapshot a baseline, loop many times, snapshot again, then ",[14,62,43],{},[64,65,70],"pre",{"className":66,"code":67,"language":68,"meta":69,"style":69},"language-python shiki shiki-themes github-light github-dark","import tracemalloc\n\n# A classic leak: an unbounded module-level cache that nothing ever evicts.\n_CACHE = {}\n\ndef handle_request(request_id):\n    # Each call retains a 1 KiB payload keyed by id; keys are never removed.\n    _CACHE[request_id] = bytes(1024)\n    return _CACHE[request_id]\n\n\ntracemalloc.start(25)                 # 25 frames so we can see the call path\n\nhandle_request(-1)                    # warm-up pass: absorb one-time allocations\nbaseline = tracemalloc.take_snapshot()\n\nfor i in range(10_000):               # loop the suspect operation many times\n    handle_request(i)\n\nafter = tracemalloc.take_snapshot()\n\n# Diff the two snapshots; size_diff is byte growth between them.\ntop = after.compare_to(baseline, \"lineno\")\nfor stat in top[:3]:\n    print(f\"+{stat.size_diff\u002F1024:8.1f} KiB  count {stat.count_diff:>6}  {stat.traceback[0]}\")\n","python","",[14,71,72,80,87,93,99,104,110,116,122,128,133,138,144,149,155,161,166,172,178,183,189,194,200,206,212],{"__ignoreMap":69},[73,74,77],"span",{"class":75,"line":76},"line",1,[73,78,79],{},"import tracemalloc\n",[73,81,83],{"class":75,"line":82},2,[73,84,86],{"emptyLinePlaceholder":85},true,"\n",[73,88,90],{"class":75,"line":89},3,[73,91,92],{},"# A classic leak: an unbounded module-level cache that nothing ever evicts.\n",[73,94,96],{"class":75,"line":95},4,[73,97,98],{},"_CACHE = {}\n",[73,100,102],{"class":75,"line":101},5,[73,103,86],{"emptyLinePlaceholder":85},[73,105,107],{"class":75,"line":106},6,[73,108,109],{},"def handle_request(request_id):\n",[73,111,113],{"class":75,"line":112},7,[73,114,115],{},"    # Each call retains a 1 KiB payload keyed by id; keys are never removed.\n",[73,117,119],{"class":75,"line":118},8,[73,120,121],{},"    _CACHE[request_id] = bytes(1024)\n",[73,123,125],{"class":75,"line":124},9,[73,126,127],{},"    return _CACHE[request_id]\n",[73,129,131],{"class":75,"line":130},10,[73,132,86],{"emptyLinePlaceholder":85},[73,134,136],{"class":75,"line":135},11,[73,137,86],{"emptyLinePlaceholder":85},[73,139,141],{"class":75,"line":140},12,[73,142,143],{},"tracemalloc.start(25)                 # 25 frames so we can see the call path\n",[73,145,147],{"class":75,"line":146},13,[73,148,86],{"emptyLinePlaceholder":85},[73,150,152],{"class":75,"line":151},14,[73,153,154],{},"handle_request(-1)                    # warm-up pass: absorb one-time allocations\n",[73,156,158],{"class":75,"line":157},15,[73,159,160],{},"baseline = tracemalloc.take_snapshot()\n",[73,162,164],{"class":75,"line":163},16,[73,165,86],{"emptyLinePlaceholder":85},[73,167,169],{"class":75,"line":168},17,[73,170,171],{},"for i in range(10_000):               # loop the suspect operation many times\n",[73,173,175],{"class":75,"line":174},18,[73,176,177],{},"    handle_request(i)\n",[73,179,181],{"class":75,"line":180},19,[73,182,86],{"emptyLinePlaceholder":85},[73,184,186],{"class":75,"line":185},20,[73,187,188],{},"after = tracemalloc.take_snapshot()\n",[73,190,192],{"class":75,"line":191},21,[73,193,86],{"emptyLinePlaceholder":85},[73,195,197],{"class":75,"line":196},22,[73,198,199],{},"# Diff the two snapshots; size_diff is byte growth between them.\n",[73,201,203],{"class":75,"line":202},23,[73,204,205],{},"top = after.compare_to(baseline, \"lineno\")\n",[73,207,209],{"class":75,"line":208},24,[73,210,211],{},"for stat in top[:3]:\n",[73,213,215],{"class":75,"line":214},25,[73,216,217],{},"    print(f\"+{stat.size_diff\u002F1024:8.1f} KiB  count {stat.count_diff:>6}  {stat.traceback[0]}\")\n",[64,219,224],{"className":220,"code":222,"language":223,"meta":69},[221],"language-text","+10240.0 KiB  count  10000  leak.py:9\n+    1.2 KiB  count     31  leak.py:18\n","text",[14,225,222],{"__ignoreMap":69},[10,227,228,229,232,233,236,237,241],{},"The first entry — line 9, the ",[14,230,231],{},"_CACHE[request_id] = bytes(1024)"," assignment — grew by ~10 MiB across 10,000 iterations with a matching ",[14,234,235],{},"count_diff"," of 10,000 blocks. That one-to-one growth between bytes and block count is the signature of a leak. To see ",[238,239,240],"em",{},"who"," drove the allocation, switch the grouping to traceback and format the path:",[64,243,245],{"className":66,"code":244,"language":68,"meta":69,"style":69},"top_tb = after.compare_to(baseline, \"traceback\")\nprint(\"\\n\".join(top_tb[0].traceback.format()))   # full call stack to the leaking line\n",[14,246,247,252],{"__ignoreMap":69},[73,248,249],{"class":75,"line":76},[73,250,251],{},"top_tb = after.compare_to(baseline, \"traceback\")\n",[73,253,254],{"class":75,"line":82},[73,255,256],{},"print(\"\\n\".join(top_tb[0].traceback.format()))   # full call stack to the leaking line\n",[10,258,259,260,263,264,267],{},"If the same leaking line is reached from many callers, the ",[14,261,262],{},"'traceback'"," grouping separates them so you can tell which call site is unbounded — exactly the case where ",[14,265,266],{},"nframe=1"," would hide the answer.",[19,269,271],{"id":270},"why-this-works","Why this works",[10,273,274,275,278,279,281,282,285,286,289],{},"A snapshot records the ",[238,276,277],{},"currently live"," tracked allocations. Anything freed between the two snapshots does not appear in the diff, so transient buffers cancel out and only retained growth survives. Because a leak retains a new block every iteration, its ",[14,280,235],{}," scales with the loop count while bounded structures stay flat. Grouping by ",[14,283,284],{},"lineno"," collapses all blocks from the offending line into a single ranked entry, and sorting by ",[14,287,288],{},"size_diff"," puts the worst offender first.",[19,291,293],{"id":292},"edge-cases-and-failure-modes","Edge cases and failure modes",[24,295,296,302,316,322,342],{},[27,297,298,301],{},[30,299,300],{},"Warm-up not excluded",": skipping the baseline-after-warm-up step floods the diff with import and cache allocations that look like leaks but plateau — always warm up first.",[27,303,304,307,308,311,312,315],{},[30,305,306],{},"GC-deferred frees",": objects in reference cycles are not freed until ",[14,309,310],{},"gc"," runs; call ",[14,313,314],{},"gc.collect()"," before the second snapshot to avoid mistaking deferred frees for a leak.",[27,317,318,321],{},[30,319,320],{},"Too few iterations",": a small loop lets a one-time 5 MiB cache outrank a slow leak; loop enough that linear growth dominates.",[27,323,324,327,328,331,332,334,335,338,339,53],{},[30,325,326],{},"C-extension memory",": raw ",[14,329,330],{},"malloc"," in a native library is invisible; if ",[14,333,16],{}," shows nothing but RSS climbs, reach for ",[14,336,337],{},"memray"," or ",[14,340,341],{},"valgrind",[27,343,344,347,348,53],{},[30,345,346],{},"Per-test leaks vs per-process",": a leak that only appears across a pytest session usually means a session-scoped fixture retains state — confirm with the scoping guidance in ",[49,349,351],{"href":350},"\u002Fadvanced-pytest-architecture-configuration\u002Fmastering-pytest-fixtures\u002F","mastering pytest fixtures",[19,353,355],{"id":354},"frequently-asked-questions","Frequently Asked Questions",[10,357,358,361],{},[30,359,360],{},"How many times should I repeat the operation before the second snapshot?","\nRepeat enough times that a genuine leak dwarfs one-time warm-up allocations, typically hundreds to thousands of iterations. A leak grows roughly linearly with iterations, while caches and interned objects plateau, which makes the leaking line obvious in the diff.",[10,363,364,367],{},[30,365,366],{},"Why does the first run always show growth even with no leak?","\nThe first iterations allocate caches, compiled regexes, lazily imported modules, and interned strings that never free. Take the baseline snapshot after a warm-up pass so these one-time allocations are excluded from the comparison.",[10,369,370,373,374,376,377,380,381,383],{},[30,371,372],{},"Can tracemalloc find leaks in C extensions?","\nOnly partially. ",[14,375,16],{}," sees allocations routed through Python's allocators, so objects a C extension creates via ",[14,378,379],{},"PyObject_Malloc"," are visible, but raw ",[14,382,330],{}," outside the Python heap is not. Use memray or valgrind for native leaks.",[10,385,386,387],{},"← Back to ",[49,388,389],{"href":51},"Memory Profiling with tracemalloc",[391,392,393],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":69,"searchDepth":82,"depth":82,"links":395},[396,397,398,399,400],{"id":21,"depth":82,"text":22},{"id":56,"depth":82,"text":57},{"id":270,"depth":82,"text":271},{"id":292,"depth":82,"text":293},{"id":354,"depth":82,"text":355},"Pin down a Python memory leak with tracemalloc: take a baseline snapshot, loop the suspect operation, diff snapshots, and attribute retained bytes to the leaking line.","md",{"slug":404,"type":405,"breadcrumb":406,"datePublished":407,"dateModified":407,"faq":408,"howto":415},"finding-memory-leaks-with-tracemalloc-snapshots","long_tail","Finding Leaks","2026-06-18",[409,411,413],{"q":360,"a":410},"Repeat enough times that a genuine leak dwarfs one-time warm-up allocations, typically hundreds to thousands of iterations. A leak grows roughly linearly with iterations, while caches and interned objects plateau, which makes the leaking line obvious in the diff.",{"q":366,"a":412},"The first iterations allocate caches, compiled regexes, lazily imported modules, and interned strings that never free. Take the baseline snapshot after a warm-up pass so these one-time allocations are excluded from the comparison.",{"q":372,"a":414},"Only partially. tracemalloc sees allocations routed through Python's allocators, so objects a C extension creates via PyObject_Malloc are visible, but raw malloc outside the Python heap is not. Use memray or valgrind for native leaks.",{"name":416,"description":417,"steps":418},"How to find a memory leak with tracemalloc snapshots","Bracket a repeated operation with two snapshots and diff them to attribute retained bytes to the leaking source line.",[419,422,425,428],{"name":420,"text":421},"Start tracing with call-path depth","Call tracemalloc.start(25) so each allocation records enough frames to identify the caller.",{"name":423,"text":424},"Warm up, then take a baseline snapshot","Run the operation once to absorb one-time allocations, then call take_snapshot() to capture the baseline.",{"name":426,"text":427},"Loop the suspect operation and snapshot again","Repeat the operation many times, then take a second snapshot so a real leak grows clear of the noise.",{"name":429,"text":430},"Diff and read the top growing line","Call after.compare_to(before, 'lineno') and inspect the entries with the largest size_diff.","\u002Fsystematic-debugging-performance-profiling\u002Fmemory-profiling-with-tracemalloc\u002Ffinding-memory-leaks-with-tracemalloc-snapshots",{"title":5,"description":401},"systematic-debugging-performance-profiling\u002Fmemory-profiling-with-tracemalloc\u002Ffinding-memory-leaks-with-tracemalloc-snapshots\u002Findex","HT_orLzTdP4pT7ojYFNOl8Te_Dd-mDbNEpScMa_kXFc",1781793487691]