[{"data":1,"prerenderedAt":1096},["ShallowReactive",2],{"page-\u002Fsystematic-debugging-performance-profiling\u002Fmemory-profiling-with-tracemalloc\u002Ffinding-reference-cycles-with-gc-and-objgraph\u002F":3},{"id":4,"title":5,"body":6,"description":1062,"extension":1063,"meta":1064,"navigation":98,"path":1092,"seo":1093,"stem":1094,"__hash__":1095},"content\u002Fsystematic-debugging-performance-profiling\u002Fmemory-profiling-with-tracemalloc\u002Ffinding-reference-cycles-with-gc-and-objgraph\u002Findex.md","Finding Reference Cycles with gc and objgraph",{"type":7,"value":8,"toc":1051},"minimark",[9,26,31,63,67,70,147,165,168,268,277,409,413,427,530,534,545,598,609,616,620,623,633,639,661,671,733,737,801,805,808,820,855,868,879,885,888,956,960,966,977,998,1008,1012,1041,1047],[10,11,12,16,17,20,21,25],"p",{},[13,14,15],"code",{},"tracemalloc"," reports that 40 MB was allocated at ",[13,18,19],{},"parsers.py:88",", and the line is a perfectly ordinary object construction. That is the allocation site, not the bug. Something is still holding those objects, and finding it needs a different tool: one that walks ",[22,23,24],"em",{},"backwards"," from an object to whatever refers to it.",[27,28,30],"h2",{"id":29},"prerequisites","Prerequisites",[32,33,34,54],"ul",{},[35,36,37,41,42,45,46,49,50,53],"li",{},[38,39,40],"strong",{},"Python 3.9+","; ",[13,43,44],{},"gc"," is standard library, ",[13,47,48],{},"objgraph"," (3.6+) needs ",[13,51,52],{},"pip install objgraph"," and graphviz for images.",[35,55,56,57,62],{},"A snapshot diff that already named the growing type — see ",[58,59,61],"a",{"href":60},"\u002Fsystematic-debugging-performance-profiling\u002Fmemory-profiling-with-tracemalloc\u002Ffinding-memory-leaks-with-tracemalloc-snapshots\u002F","finding memory leaks with tracemalloc snapshots",".",[27,64,66],{"id":65},"solution","Solution",[10,68,69],{},"Work in three steps: collect, count, then walk backwards.",[71,72,77],"pre",{"className":73,"code":74,"language":75,"meta":76,"style":76},"language-python shiki shiki-themes github-light github-dark","import gc\nimport objgraph\n\ngc.collect()                                    # 1. remove merely-uncollected noise\nobjgraph.show_growth(limit=10)                  # 2. which types grew since last call\n# ParsedFrame     18422    +18422\n# dict            24019     +6104\n# Line            18422    +18422\n\nframe = objgraph.by_type(\"ParsedFrame\")[0]      # 3. take one and walk backwards\nobjgraph.show_backrefs([frame], max_depth=4, filename=\"backrefs.png\")\n","python","",[13,78,79,87,93,100,106,112,118,124,130,135,141],{"__ignoreMap":76},[80,81,84],"span",{"class":82,"line":83},"line",1,[80,85,86],{},"import gc\n",[80,88,90],{"class":82,"line":89},2,[80,91,92],{},"import objgraph\n",[80,94,96],{"class":82,"line":95},3,[80,97,99],{"emptyLinePlaceholder":98},true,"\n",[80,101,103],{"class":82,"line":102},4,[80,104,105],{},"gc.collect()                                    # 1. remove merely-uncollected noise\n",[80,107,109],{"class":82,"line":108},5,[80,110,111],{},"objgraph.show_growth(limit=10)                  # 2. which types grew since last call\n",[80,113,115],{"class":82,"line":114},6,[80,116,117],{},"# ParsedFrame     18422    +18422\n",[80,119,121],{"class":82,"line":120},7,[80,122,123],{},"# dict            24019     +6104\n",[80,125,127],{"class":82,"line":126},8,[80,128,129],{},"# Line            18422    +18422\n",[80,131,133],{"class":82,"line":132},9,[80,134,99],{"emptyLinePlaceholder":98},[80,136,138],{"class":82,"line":137},10,[80,139,140],{},"frame = objgraph.by_type(\"ParsedFrame\")[0]      # 3. take one and walk backwards\n",[80,142,144],{"class":82,"line":143},11,[80,145,146],{},"objgraph.show_backrefs([frame], max_depth=4, filename=\"backrefs.png\")\n",[10,148,149,152,153,156,157,160,161,164],{},[13,150,151],{},"show_growth"," is the fastest orientation: called twice around a workload, it prints only the types whose live-instance count increased. ",[13,154,155],{},"show_backrefs"," renders the chain from a sample object up to the roots, and the picture answers the question directly — a ",[13,158,159],{},"dict"," that belongs to a module, a ",[13,162,163],{},"list"," on a long-lived singleton, a closure cell in a registered callback.",[10,166,167],{},"When graphviz is unavailable, the same walk works in text with the standard library alone:",[71,169,171],{"className":73,"code":170,"language":75,"meta":76,"style":76},"import gc\n\ndef who_holds(obj, depth: int = 2) -> None:\n    \"\"\"Print the containers referring to obj, up to `depth` levels back.\"\"\"\n    seen = {id(obj)}\n    frontier = [(obj, 0)]\n    while frontier:\n        current, level = frontier.pop()\n        if level >= depth:\n            continue\n        for ref in gc.get_referrers(current):\n            if id(ref) in seen or isinstance(ref, type(who_holds.__code__)):\n                continue                        # skip our own frames and code objects\n            seen.add(id(ref))\n            kind = type(ref).__name__\n            size = len(ref) if hasattr(ref, \"__len__\") else \"-\"\n            print(f\"{'  ' * level}held by {kind} (len={size})\")\n            frontier.append((ref, level + 1))\n",[13,172,173,177,181,186,191,196,201,206,211,216,221,226,232,238,244,250,256,262],{"__ignoreMap":76},[80,174,175],{"class":82,"line":83},[80,176,86],{},[80,178,179],{"class":82,"line":89},[80,180,99],{"emptyLinePlaceholder":98},[80,182,183],{"class":82,"line":95},[80,184,185],{},"def who_holds(obj, depth: int = 2) -> None:\n",[80,187,188],{"class":82,"line":102},[80,189,190],{},"    \"\"\"Print the containers referring to obj, up to `depth` levels back.\"\"\"\n",[80,192,193],{"class":82,"line":108},[80,194,195],{},"    seen = {id(obj)}\n",[80,197,198],{"class":82,"line":114},[80,199,200],{},"    frontier = [(obj, 0)]\n",[80,202,203],{"class":82,"line":120},[80,204,205],{},"    while frontier:\n",[80,207,208],{"class":82,"line":126},[80,209,210],{},"        current, level = frontier.pop()\n",[80,212,213],{"class":82,"line":132},[80,214,215],{},"        if level >= depth:\n",[80,217,218],{"class":82,"line":137},[80,219,220],{},"            continue\n",[80,222,223],{"class":82,"line":143},[80,224,225],{},"        for ref in gc.get_referrers(current):\n",[80,227,229],{"class":82,"line":228},12,[80,230,231],{},"            if id(ref) in seen or isinstance(ref, type(who_holds.__code__)):\n",[80,233,235],{"class":82,"line":234},13,[80,236,237],{},"                continue                        # skip our own frames and code objects\n",[80,239,241],{"class":82,"line":240},14,[80,242,243],{},"            seen.add(id(ref))\n",[80,245,247],{"class":82,"line":246},15,[80,248,249],{},"            kind = type(ref).__name__\n",[80,251,253],{"class":82,"line":252},16,[80,254,255],{},"            size = len(ref) if hasattr(ref, \"__len__\") else \"-\"\n",[80,257,259],{"class":82,"line":258},17,[80,260,261],{},"            print(f\"{'  ' * level}held by {kind} (len={size})\")\n",[80,263,265],{"class":82,"line":264},18,[80,266,267],{},"            frontier.append((ref, level + 1))\n",[10,269,270,271,273,274,276],{},"The output usually identifies the container within two levels: a ",[13,272,159],{}," of length 18,000 is the cache nobody bounded, and a ",[13,275,163],{}," growing by one per request is the registry nobody unregisters from.",[278,279,282,405],"figure",{"className":280},[281],"diagram",[283,284,291,292,291,296,291,300,291,318,291,326,291,335,291,344,291,350,291,355,291,361,291,364,291,368,291,371,291,375,291,378,291,382,291,385,291,389,291,392,291,396,291,399],"svg",{"viewBox":285,"role":286,"ariaLabelledBy":287,"xmlns":290},"0 0 760 172","img",[288,289],"retentionwalk-t","retentionwalk-d","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","\n  ",[293,294,295],"title",{"id":288},"From allocation site to retaining reference",[297,298,299],"desc",{"id":289},"A left-to-right investigation: a tracemalloc diff names the allocation line, a garbage collection removes uncollected noise, an instance count identifies the growing type, and a back-reference walk names the container that holds it.",[301,302,303,304,291],"defs",{},"\n    ",[305,306,313],"marker",{"id":307,"viewBox":308,"refX":309,"refY":310,"markerWidth":311,"markerHeight":311,"orient":312},"retentionwalk-a","0 0 10 10","9","5","7","auto-start-reverse",[314,315],"path",{"d":316,"fill":317},"M0 0 L10 5 L0 10 z","#81b29a",[319,320],"rect",{"x":321,"y":321,"width":322,"height":323,"rx":324,"fill":325},"0","760","172","14","#fffdf8",[327,328,295],"text",{"x":329,"y":330,"textAnchor":331,"fontSize":332,"fontWeight":333,"fill":334},"380","30","middle","15","700","#3d405b",[319,336],{"x":337,"y":338,"width":339,"height":340,"rx":341,"fill":342,"stroke":317,"strokeWidth":343},"26","62","142","76","12","#f4f1de","1.8",[327,345,349],{"x":346,"y":347,"textAnchor":331,"fontSize":348,"fontWeight":333,"fill":334},"97","96","12.5","allocation site",[327,351,354],{"x":346,"y":352,"textAnchor":331,"fontSize":353,"fill":334},"113","11","tracemalloc diff",[82,356],{"x1":357,"y1":358,"x2":359,"y2":358,"stroke":317,"strokeWidth":343,"markerEnd":360},"176","100","208","url(#retentionwalk-a)",[319,362],{"x":363,"y":338,"width":339,"height":340,"rx":341,"fill":325,"stroke":317,"strokeWidth":343},"214",[327,365,367],{"x":366,"y":347,"textAnchor":331,"fontSize":348,"fontWeight":333,"fill":334},"286","collect",[327,369,370],{"x":366,"y":352,"textAnchor":331,"fontSize":353,"fill":334},"remove noise",[82,372],{"x1":373,"y1":358,"x2":374,"y2":358,"stroke":317,"strokeWidth":343,"markerEnd":360},"364","396",[319,376],{"x":377,"y":338,"width":339,"height":340,"rx":341,"fill":342,"stroke":317,"strokeWidth":343},"403",[327,379,381],{"x":380,"y":347,"textAnchor":331,"fontSize":348,"fontWeight":333,"fill":334},"474","count by type",[327,383,384],{"x":380,"y":352,"textAnchor":331,"fontSize":353,"fill":334},"what is still live",[82,386],{"x1":387,"y1":358,"x2":388,"y2":358,"stroke":317,"strokeWidth":343,"markerEnd":360},"552","584",[319,390],{"x":391,"y":338,"width":339,"height":340,"rx":341,"fill":325,"stroke":317,"strokeWidth":343},"592",[327,393,395],{"x":394,"y":347,"textAnchor":331,"fontSize":348,"fontWeight":333,"fill":334},"663","walk backwards",[327,397,398],{"x":394,"y":352,"textAnchor":331,"fontSize":353,"fill":334},"what holds it",[327,400,404],{"x":329,"y":401,"textAnchor":331,"fontSize":402,"fontStyle":403,"fill":334},"164","11.5","italic","Allocation and retention are separate facts, which is why one tool cannot answer both.",[406,407,408],"figcaption",{},"The first stage names the code that created the object; only the last stage names the code responsible for keeping it.",[27,410,412],{"id":411},"why-this-works","Why this works",[10,414,415,416,419,420,423,424,426],{},"CPython frees an object when its reference count reaches zero. A cycle keeps every member's count above zero, so the cyclic collector exists to find groups that are unreachable from any root and reclaim them — which means a cycle among ",[22,417,418],{},"unreachable"," objects is a delay, not a leak. A genuine leak is an object still reachable from a live root: a module global, a class attribute, a running frame, a registered callback. ",[13,421,422],{},"gc.get_referrers"," walks the reference graph in reverse and therefore answers exactly that question, and ",[13,425,48],{}," is a presentation layer over the same data with cycle handling and filtering.",[278,428,430,527],{"className":429},[281],[283,431,291,436,291,439,291,442,291,445,291,447,291,456,291,460,291,464,291,468,291,472,291,476,291,479,291,482,291,485,291,489,291,492,291,495,291,497,291,501,291,504,291,507,291,510,291,514,291,517,291,520,291,524],{"viewBox":432,"role":286,"ariaLabelledBy":433,"xmlns":290},"0 0 760 270",[434,435],"leakkind-t","leakkind-d",[293,437,438],{"id":434},"Three memory symptoms and what each means",[297,440,441],{"id":435},"A table of three memory symptoms - growth that disappears after a collect, growth that survives a collect, and RSS growth with a flat tracemalloc total - with the cause and the tool for each.",[319,443],{"x":321,"y":321,"width":322,"height":444,"rx":324,"fill":325},"270",[327,446,438],{"x":329,"y":330,"textAnchor":331,"fontSize":332,"fontWeight":333,"fill":334},[319,448],{"x":449,"y":450,"width":451,"height":452,"rx":453,"fill":454,"stroke":334,"strokeWidth":455},"24","52","712","40","10","#f2cc8f","1.5",[327,457,459],{"x":458,"y":340,"fontSize":341,"fontWeight":333,"fill":334},"38","Criterion",[327,461,463],{"x":462,"y":340,"textAnchor":331,"fontSize":341,"fontWeight":333,"fill":334},"390","Cause",[327,465,467],{"x":466,"y":340,"textAnchor":331,"fontSize":341,"fontWeight":333,"fill":334},"620","Next tool",[319,469],{"x":449,"y":470,"width":451,"height":452,"fill":342,"stroke":334,"strokeWidth":471},"92","1",[327,473,475],{"x":458,"y":474,"fontSize":402,"fill":334},"116","gone after gc.collect()",[327,477,478],{"x":462,"y":474,"textAnchor":331,"fontSize":402,"fill":334},"deferred cycle collection",[327,480,481],{"x":466,"y":474,"textAnchor":331,"fontSize":402,"fill":334},"usually not a bug",[319,483],{"x":449,"y":484,"width":451,"height":452,"fill":325,"stroke":334,"strokeWidth":471},"132",[327,486,488],{"x":458,"y":487,"fontSize":402,"fill":334},"156","survives gc.collect()",[327,490,491],{"x":462,"y":487,"textAnchor":331,"fontSize":402,"fill":334},"a live reference holds it",[327,493,494],{"x":466,"y":487,"textAnchor":331,"fontSize":402,"fill":334},"objgraph backrefs",[319,496],{"x":449,"y":323,"width":451,"height":452,"fill":342,"stroke":334,"strokeWidth":471},[327,498,500],{"x":458,"y":499,"fontSize":402,"fill":334},"196","RSS grows, tracemalloc flat",[327,502,503],{"x":462,"y":499,"textAnchor":331,"fontSize":402,"fill":334},"native allocation or fragmentation",[327,505,506],{"x":466,"y":499,"textAnchor":331,"fontSize":402,"fill":334},"memray or OS tools",[319,508],{"x":449,"y":509,"width":451,"height":452,"fill":325,"stroke":334,"strokeWidth":471},"212",[327,511,513],{"x":458,"y":512,"fontSize":402,"fill":334},"236","objects in gc.garbage",[327,515,516],{"x":462,"y":512,"textAnchor":331,"fontSize":402,"fill":334},"uncollectable cycle",[327,518,519],{"x":466,"y":512,"textAnchor":331,"fontSize":402,"fill":334},"inspect finalizers",[82,521],{"x1":522,"y1":450,"x2":522,"y2":523,"stroke":334,"strokeWidth":471},"274","252",[82,525],{"x1":526,"y1":450,"x2":526,"y2":523,"stroke":334,"strokeWidth":471},"505",[406,528,529],{},"Running a collection first is what separates the second row from the first, and the second row is the only one that is a Python-level leak.",[27,531,533],{"id":532},"uncollectable-cycles","Uncollectable cycles",[10,535,536,537,540,541,544],{},"Most cycles are collected. A few are not, and ",[13,538,539],{},"gc.DEBUG_SAVEALL"," makes them visible by moving everything the collector could not free into ",[13,542,543],{},"gc.garbage"," instead of discarding it:",[71,546,548],{"className":73,"code":547,"language":75,"meta":76,"style":76},"import gc\n\ngc.set_debug(gc.DEBUG_SAVEALL)\nrun_workload()\ngc.collect()\nprint(f\"{len(gc.garbage)} uncollectable object(s)\")\nfor obj in gc.garbage[:5]:\n    print(type(obj).__name__, repr(obj)[:80])\ngc.set_debug(0)\ngc.garbage.clear()                      # otherwise this list is itself a leak\n",[13,549,550,554,558,563,568,573,578,583,588,593],{"__ignoreMap":76},[80,551,552],{"class":82,"line":83},[80,553,86],{},[80,555,556],{"class":82,"line":89},[80,557,99],{"emptyLinePlaceholder":98},[80,559,560],{"class":82,"line":95},[80,561,562],{},"gc.set_debug(gc.DEBUG_SAVEALL)\n",[80,564,565],{"class":82,"line":102},[80,566,567],{},"run_workload()\n",[80,569,570],{"class":82,"line":108},[80,571,572],{},"gc.collect()\n",[80,574,575],{"class":82,"line":114},[80,576,577],{},"print(f\"{len(gc.garbage)} uncollectable object(s)\")\n",[80,579,580],{"class":82,"line":120},[80,581,582],{},"for obj in gc.garbage[:5]:\n",[80,584,585],{"class":82,"line":126},[80,586,587],{},"    print(type(obj).__name__, repr(obj)[:80])\n",[80,589,590],{"class":82,"line":132},[80,591,592],{},"gc.set_debug(0)\n",[80,594,595],{"class":82,"line":137},[80,596,597],{},"gc.garbage.clear()                      # otherwise this list is itself a leak\n",[10,599,600,601,604,605,608],{},"The historical cause — a ",[13,602,603],{},"__del__"," method on a cycle member — was fixed in Python 3.4, so objects appearing here today usually come from a C extension that does not implement traversal correctly, or from a frame captured in a traceback that is itself stored somewhere. That second case is common and easy to fix: storing ",[13,606,607],{},"exc.__traceback__"," on an object keeps every frame, and every frame's locals, alive indefinitely.",[10,610,611,612,615],{},"Note that ",[13,613,614],{},"DEBUG_SAVEALL"," is a diagnostic only. Leaving it enabled turns every collected cycle into retained memory, so it belongs in a scratch reproduction rather than in a service.",[27,617,619],{"id":618},"choosing-the-right-fix","Choosing the right fix",[10,621,622],{},"The reference chain determines the fix, and there are only four shapes.",[10,624,625,628,629,632],{},[38,626,627],{},"An unbounded cache"," becomes a bounded one: ",[13,630,631],{},"functools.lru_cache(maxsize=...)",", a TTL cache, or an explicit eviction policy. The cache key's cardinality is worth checking at the same time — a cache keyed on a request id never hits and is pure leak.",[10,634,635,638],{},[38,636,637],{},"A registry with no unregister"," gets one, and the object that registered becomes responsible for removing itself, ideally through a context manager so the removal cannot be skipped.",[10,640,641,644,645,648,649,652,653,656,657,660],{},[38,642,643],{},"A callback holding a strong reference"," becomes a ",[13,646,647],{},"weakref.ref"," or a ",[13,650,651],{},"WeakMethod",", so the callback does not keep its subject alive. ",[13,654,655],{},"weakref.WeakValueDictionary"," and ",[13,658,659],{},"WeakSet"," are the collection forms.",[10,662,663,666,667,670],{},[38,664,665],{},"A closure capturing more than it needs"," is rewritten to capture the small value rather than the large object — a common accident when a lambda references ",[13,668,669],{},"self"," only to read one attribute.",[71,672,674],{"className":73,"code":673,"language":75,"meta":76,"style":76},"import weakref\n\nclass EventBus:\n    def __init__(self):\n        self._subscribers: weakref.WeakSet = weakref.WeakSet()\n\n    def subscribe(self, handler) -> None:\n        self._subscribers.add(handler)          # does not keep handler alive\n\n    def publish(self, event) -> None:\n        for handler in list(self._subscribers):  # copy: the set may shrink mid-iteration\n            handler(event)\n",[13,675,676,681,685,690,695,700,704,709,714,718,723,728],{"__ignoreMap":76},[80,677,678],{"class":82,"line":83},[80,679,680],{},"import weakref\n",[80,682,683],{"class":82,"line":89},[80,684,99],{"emptyLinePlaceholder":98},[80,686,687],{"class":82,"line":95},[80,688,689],{},"class EventBus:\n",[80,691,692],{"class":82,"line":102},[80,693,694],{},"    def __init__(self):\n",[80,696,697],{"class":82,"line":108},[80,698,699],{},"        self._subscribers: weakref.WeakSet = weakref.WeakSet()\n",[80,701,702],{"class":82,"line":114},[80,703,99],{"emptyLinePlaceholder":98},[80,705,706],{"class":82,"line":120},[80,707,708],{},"    def subscribe(self, handler) -> None:\n",[80,710,711],{"class":82,"line":126},[80,712,713],{},"        self._subscribers.add(handler)          # does not keep handler alive\n",[80,715,716],{"class":82,"line":132},[80,717,99],{"emptyLinePlaceholder":98},[80,719,720],{"class":82,"line":137},[80,721,722],{},"    def publish(self, event) -> None:\n",[80,724,725],{"class":82,"line":143},[80,726,727],{},"        for handler in list(self._subscribers):  # copy: the set may shrink mid-iteration\n",[80,729,730],{"class":82,"line":228},[80,731,732],{},"            handler(event)\n",[27,734,736],{"id":735},"edge-cases-and-failure-modes","Edge cases and failure modes",[32,738,739,747,760,770,776,784],{},[35,740,741,746],{},[38,742,743,745],{},[13,744,422],{}," sees your own frame."," The list includes the frame that called it, so filtering frames and the function's own locals is required or every object appears to be held by the debugger.",[35,748,749,755,756,759],{},[38,750,751,754],{},[13,752,753],{},"objgraph.show_backrefs"," on a hot type is slow."," The graph explodes; sample one object and cap ",[13,757,758],{},"max_depth"," at three or four.",[35,761,762,765,766,769],{},[38,763,764],{},"Interned and cached objects look retained."," Small ints, short strings and empty tuples are shared process-wide; growth in ",[13,767,768],{},"int"," counts is almost never meaningful.",[35,771,772,775],{},[38,773,774],{},"A generator's frame keeps its locals alive"," until it is exhausted or closed, so an abandoned generator retains everything it referenced.",[35,777,778,783],{},[38,779,780,782],{},[13,781,659],{}," iteration during collection can shrink mid-loop."," Copy to a list before iterating, as in the example above.",[35,785,786,789,790,792,793,792,795,656,797,800],{},[38,787,788],{},"Weak references cannot point at every type."," ",[13,791,163],{},", ",[13,794,159],{},[13,796,768],{},[13,798,799],{},"str"," do not support them; wrap the value in a small object when you need a weak reference to plain data.",[27,802,804],{"id":803},"a-worked-retention-hunt","A worked retention hunt",[10,806,807],{},"Putting the tools in order on a real symptom shows how few steps are needed. The symptom: a worker's RSS grows by roughly 30 MB per hour and never falls.",[10,809,810,813,814,816,817,819],{},[38,811,812],{},"Step one — is it Python-level?"," Compare the process RSS with the ",[13,815,15],{}," total. If ",[13,818,15],{}," accounts for the growth, the object graph will explain it; if not, the growth is native or fragmentation and the tools on this page will find nothing.",[71,821,823],{"className":73,"code":822,"language":75,"meta":76,"style":76},"import tracemalloc, psutil, os\ntracemalloc.start(5)\nrun_workload(minutes=10)\ncurrent, peak = tracemalloc.get_traced_memory()\nrss = psutil.Process(os.getpid()).memory_info().rss\nprint(f\"traced {current\u002F1e6:.1f} MB, peak {peak\u002F1e6:.1f} MB, rss {rss\u002F1e6:.1f} MB\")\n",[13,824,825,830,835,840,845,850],{"__ignoreMap":76},[80,826,827],{"class":82,"line":83},[80,828,829],{},"import tracemalloc, psutil, os\n",[80,831,832],{"class":82,"line":89},[80,833,834],{},"tracemalloc.start(5)\n",[80,836,837],{"class":82,"line":95},[80,838,839],{},"run_workload(minutes=10)\n",[80,841,842],{"class":82,"line":102},[80,843,844],{},"current, peak = tracemalloc.get_traced_memory()\n",[80,846,847],{"class":82,"line":108},[80,848,849],{},"rss = psutil.Process(os.getpid()).memory_info().rss\n",[80,851,852],{"class":82,"line":114},[80,853,854],{},"print(f\"traced {current\u002F1e6:.1f} MB, peak {peak\u002F1e6:.1f} MB, rss {rss\u002F1e6:.1f} MB\")\n",[10,856,857,789,860,863,864,867],{},[38,858,859],{},"Step two — which type?",[13,861,862],{},"objgraph.show_growth()"," around one cycle of the workload names it: ",[13,865,866],{},"ParsedFrame +18422"," is unambiguous.",[10,869,870,789,873,875,876,878],{},[38,871,872],{},"Step three — what holds it?",[13,874,422],{}," on a sample, filtered to skip frames, reports a ",[13,877,159],{}," of 18,422 entries. One more level up names the module that owns the dict.",[10,880,881,884],{},[38,882,883],{},"Step four — why is it unbounded?"," Reading that dict's key expression is usually the end of the investigation: keyed on a request id, so no key is ever reused and no entry is ever evicted.",[10,886,887],{},"The fix is a bounded cache, and the confirmation is the same measurement repeated: run the workload again and check that the traced total is flat. Four steps, three tools, and the only genuinely difficult part is knowing that step one exists — teams that skip it spend hours looking for a Python reference to memory that a C extension allocated.\nThe investigation order matters more than the tools, so it is worth stating as a rule: measure whether the growth is Python-level, then name the type, then name the referrer, then fix ownership. Skipping straight to the referrer walk on a process whose growth is native produces hours of searching for a Python reference that does not exist.",[278,889,891,953],{"className":890},[281],[283,892,291,897,291,900,291,903,291,910,291,913,291,915,291,917,291,920,291,923,291,926,291,928,291,931,291,933,291,935,291,937,291,940,291,943,291,945,291,947,291,950],{"viewBox":893,"role":286,"ariaLabelledBy":894,"xmlns":290},"0 0 760 154",[895,896],"retorder-t","retorder-d",[293,898,899],{"id":895},"The four questions, in the order that answers them fastest",[297,901,902],{"id":896},"A left-to-right sequence of four diagnostic questions: is the growth Python-level, which type is growing, what refers to it, and who should own its lifetime.",[301,904,303,905,291],{},[305,906,908],{"id":907,"viewBox":308,"refX":309,"refY":310,"markerWidth":311,"markerHeight":311,"orient":312},"retorder-a",[314,909],{"d":316,"fill":454},[319,911],{"x":321,"y":321,"width":322,"height":912,"rx":324,"fill":325},"154",[327,914,899],{"x":329,"y":330,"textAnchor":331,"fontSize":332,"fontWeight":333,"fill":334},[319,916],{"x":337,"y":338,"width":339,"height":340,"rx":341,"fill":342,"stroke":454,"strokeWidth":343},[327,918,919],{"x":346,"y":347,"textAnchor":331,"fontSize":348,"fontWeight":333,"fill":334},"Python-level?",[327,921,922],{"x":346,"y":352,"textAnchor":331,"fontSize":353,"fill":334},"tracemalloc vs RSS",[82,924],{"x1":357,"y1":358,"x2":359,"y2":358,"stroke":454,"strokeWidth":343,"markerEnd":925},"url(#retorder-a)",[319,927],{"x":363,"y":338,"width":339,"height":340,"rx":341,"fill":325,"stroke":454,"strokeWidth":343},[327,929,930],{"x":366,"y":347,"textAnchor":331,"fontSize":348,"fontWeight":333,"fill":334},"which type?",[327,932,151],{"x":366,"y":352,"textAnchor":331,"fontSize":353,"fill":334},[82,934],{"x1":373,"y1":358,"x2":374,"y2":358,"stroke":454,"strokeWidth":343,"markerEnd":925},[319,936],{"x":377,"y":338,"width":339,"height":340,"rx":341,"fill":342,"stroke":454,"strokeWidth":343},[327,938,939],{"x":380,"y":347,"textAnchor":331,"fontSize":348,"fontWeight":333,"fill":334},"what refers?",[327,941,942],{"x":380,"y":352,"textAnchor":331,"fontSize":353,"fill":334},"get_referrers",[82,944],{"x1":387,"y1":358,"x2":388,"y2":358,"stroke":454,"strokeWidth":343,"markerEnd":925},[319,946],{"x":391,"y":338,"width":339,"height":340,"rx":341,"fill":325,"stroke":454,"strokeWidth":343},[327,948,949],{"x":394,"y":347,"textAnchor":331,"fontSize":348,"fontWeight":333,"fill":334},"who owns it?",[327,951,952],{"x":394,"y":352,"textAnchor":331,"fontSize":353,"fill":334},"the actual fix",[406,954,955],{},"Each question is cheap and each one eliminates a category, so the sequence usually terminates in minutes rather than hours.",[27,957,959],{"id":958},"frequently-asked-questions","Frequently Asked Questions",[10,961,962,965],{},[38,963,964],{},"Does Python leak memory when objects form a cycle?","\nNot permanently — the cyclic collector reclaims cycles whose members are unreachable. Cycles matter because collection is deferred and because an object reachable from a live root is never collected at all, cycle or not.",[10,967,968,971,973,974,976],{},[38,969,970],{},"What is the difference between tracemalloc and objgraph?",[13,972,15],{}," says where memory was allocated; ",[13,975,48],{}," says what still refers to it. Growth diagnosis needs both: the allocation site names the code that created the object, the reference chain names the code that will not let go.",[10,978,979,982,983,990,991,994,995,997],{},[38,980,981],{},"Is gc.collect() in production a reasonable fix?","\nRarely. It hides growth caused by deferred collection but does nothing for objects held by a live reference, and it pauses the process. Use it in diagnosis to distinguish the two cases, not as a remedy.\n",[38,984,985,986,989],{},"Is ",[13,987,988],{},"weakref"," always the right fix for a retaining reference?","\nNo. A weak reference is correct when the referrer genuinely does not own the object's lifetime — a cache, an observer list, a back-pointer. Where the referrer ",[22,992,993],{},"should"," keep the object alive, a weak reference converts a memory leak into a much worse bug: an object that disappears while still in use. Fix ownership first and reach for ",[13,996,988],{}," only when the answer is that nothing there owns it.",[10,999,1000,1007],{},[38,1001,1002,1003,1006],{},"Does ",[13,1004,1005],{},"gc.freeze()"," help with growth?","\nIt helps with copy-on-write memory in forked workers, not with leaks: it moves existing objects into a permanent generation so the collector stops touching them, which keeps shared pages shared after a fork. It has no effect on objects your code still references.",[27,1009,1011],{"id":1010},"related","Related",[32,1013,1014,1021,1027,1034],{},[35,1015,1016,1020],{},[58,1017,1019],{"href":1018},"\u002Fsystematic-debugging-performance-profiling\u002Fmemory-profiling-with-tracemalloc\u002F","Memory profiling with tracemalloc"," — the allocation half of the investigation.",[35,1022,1023,1026],{},[58,1024,1025],{"href":60},"Finding memory leaks with tracemalloc snapshots"," — producing the diff that names the type.",[35,1028,1029,1033],{},[58,1030,1032],{"href":1031},"\u002Fsystematic-debugging-performance-profiling\u002Fmemory-profiling-with-tracemalloc\u002Fcomparing-tracemalloc-snapshots-to-locate-growth\u002F","Comparing tracemalloc snapshots to locate growth"," — reading a diff without being misled.",[35,1035,1036,1040],{},[58,1037,1039],{"href":1038},"\u002Fsystematic-debugging-performance-profiling\u002Finteractive-debugging-with-pdb-and-ipdb\u002Fpost-mortem-debugging-with-pdb-pm\u002F","Post-mortem debugging with pdb.pm()"," — why a stored traceback retains an entire stack.",[10,1042,1043,1044],{},"← Back to ",[58,1045,1046],{"href":1018},"Memory Profiling with tracemalloc",[1048,1049,1050],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":76,"searchDepth":89,"depth":89,"links":1052},[1053,1054,1055,1056,1057,1058,1059,1060,1061],{"id":29,"depth":89,"text":30},{"id":65,"depth":89,"text":66},{"id":411,"depth":89,"text":412},{"id":532,"depth":89,"text":533},{"id":618,"depth":89,"text":619},{"id":735,"depth":89,"text":736},{"id":803,"depth":89,"text":804},{"id":958,"depth":89,"text":959},{"id":1010,"depth":89,"text":1011},"Locate the reference that keeps an object alive: gc.get_referrers, gc.DEBUG_SAVEALL, objgraph back-reference chains, and when a cycle is actually the problem.","md",{"slug":1065,"type":1066,"breadcrumb":1067,"datePublished":1068,"dateModified":1068,"faq":1069,"howto":1076},"finding-reference-cycles-with-gc-and-objgraph","article","Reference Cycles","2026-08-01",[1070,1072,1074],{"q":964,"a":1071},"Not permanently — the cyclic collector reclaims cycles whose members are unreachable. Cycles matter because collection is deferred and because an object reachable from a live root is never collected at all, cycle or not.",{"q":970,"a":1073},"tracemalloc says where memory was allocated; objgraph says what still refers to it. Growth diagnosis needs both: the allocation site names the code that created the object, the reference chain names the code that will not let go.",{"q":981,"a":1075},"Rarely. It hides growth caused by deferred collection but does nothing for objects held by a live reference, and it pauses the process. Use it in diagnosis to distinguish the two cases, not as a remedy.",{"name":1077,"description":1078,"steps":1079},"How to find what keeps an object alive","Move from an allocation site to the reference chain that retains the object.",[1080,1083,1086,1089],{"name":1081,"text":1082},"Collect first to remove noise","Run gc.collect so anything merely uncollected disappears from the investigation.",{"name":1084,"text":1085},"Count live instances by type","Use objgraph to see which types grew between two points in the run.",{"name":1087,"text":1088},"Walk the back-references","Follow referrers from a sample object to a root to find the retaining container.",{"name":1090,"text":1091},"Fix the retention, not the allocation","Bound the cache, unregister the callback, or hold a weak reference.","\u002Fsystematic-debugging-performance-profiling\u002Fmemory-profiling-with-tracemalloc\u002Ffinding-reference-cycles-with-gc-and-objgraph",{"title":5,"description":1062},"systematic-debugging-performance-profiling\u002Fmemory-profiling-with-tracemalloc\u002Ffinding-reference-cycles-with-gc-and-objgraph\u002Findex","35NY3bRhQ8WAEsj6o7SBuIAgo3B3HDBhRZaAOvp7Mwc",1785613403108]