[{"data":1,"prerenderedAt":987},["ShallowReactive",2],{"page-\u002Fsystematic-debugging-performance-profiling\u002Fdebugging-async-code-and-event-loops\u002Ftracking-down-a-hung-await-with-task-stacks\u002F":3},{"id":4,"title":5,"body":6,"description":953,"extension":954,"meta":955,"navigation":110,"path":983,"seo":984,"stem":985,"__hash__":986},"content\u002Fsystematic-debugging-performance-profiling\u002Fdebugging-async-code-and-event-loops\u002Ftracking-down-a-hung-await-with-task-stacks\u002Findex.md","Tracking Down a Hung await with Task Stacks",{"type":7,"value":8,"toc":942},"minimark",[9,18,24,29,60,64,166,272,297,324,480,484,506,528,538,542,545,552,582,674,678,681,704,711,764,768,779,782,786,849,853,873,886,900,904,933,938],[10,11,12,13,17],"p",{},"An async program that hangs gives you nothing to go on. There is no exception and no traceback, often no log line — the process just stops making progress. In tests, the symptom is a CI job that runs until its global timeout and is killed, leaving a log that ends in the middle of the test list. The usual culprits are an ",[14,15,16],"code",{},"await"," on something that will never complete (a future nobody resolves, a queue nobody fills, a lock never released) or a synchronous call that blocks the event loop thread so nothing else can run.",[10,19,20,21,23],{},"Those two cases need different tools, and telling them apart is the first step. If the event loop is still running, you can ask it where every task is waiting, and the answer points at the ",[14,22,16],{}," that never returns. If the loop itself is blocked, no coroutine can run — including a diagnostic one — and you need a thread-level stack dump, which shows the synchronous call holding the thread.",[25,26,28],"h2",{"id":27},"prerequisites","Prerequisites",[30,31,32,52],"ul",{},[33,34,35,36,39,40,43,44,47,48,51],"li",{},"Python 3.11 or later (3.14 for ",[14,37,38],{},"asyncio ps"," \u002F ",[14,41,42],{},"pstree","), ",[14,45,46],{},"pytest-timeout >= 2.3",", optionally ",[14,49,50],{},"py-spy",".",[33,53,54,55,51],{},"Background from ",[56,57,59],"a",{"href":58},"\u002Fsystematic-debugging-performance-profiling\u002Fdebugging-async-code-and-event-loops\u002F","Debugging async code and event loops",[25,61,63],{"id":62},"solution","Solution",[65,66,71],"pre",{"className":67,"code":68,"language":69,"meta":70,"style":70},"language-python shiki shiki-themes github-light github-dark","# app\u002Fdiagnostics.py — install once at startup (Unix).\nimport asyncio\nimport faulthandler\nimport signal\nimport sys\n\ndef install_task_dump(loop: asyncio.AbstractEventLoop) -> None:\n    def dump() -> None:\n        tasks = asyncio.all_tasks(loop)\n        print(f\"--- {len(tasks)} tasks ---\", file=sys.stderr)\n        for task in tasks:\n            print(f\"\\n{task.get_name()}  {task.get_coro().__qualname__}\", file=sys.stderr)\n            task.print_stack(file=sys.stderr)\n    loop.add_signal_handler(signal.SIGUSR1, dump)        # runs on the loop: needs it alive\n    faulthandler.register(signal.SIGUSR2, all_threads=True)  # runs anywhere: works when blocked\n","python","",[14,72,73,81,87,93,99,105,112,118,124,130,136,142,148,154,160],{"__ignoreMap":70},[74,75,78],"span",{"class":76,"line":77},"line",1,[74,79,80],{},"# app\u002Fdiagnostics.py — install once at startup (Unix).\n",[74,82,84],{"class":76,"line":83},2,[74,85,86],{},"import asyncio\n",[74,88,90],{"class":76,"line":89},3,[74,91,92],{},"import faulthandler\n",[74,94,96],{"class":76,"line":95},4,[74,97,98],{},"import signal\n",[74,100,102],{"class":76,"line":101},5,[74,103,104],{},"import sys\n",[74,106,108],{"class":76,"line":107},6,[74,109,111],{"emptyLinePlaceholder":110},true,"\n",[74,113,115],{"class":76,"line":114},7,[74,116,117],{},"def install_task_dump(loop: asyncio.AbstractEventLoop) -> None:\n",[74,119,121],{"class":76,"line":120},8,[74,122,123],{},"    def dump() -> None:\n",[74,125,127],{"class":76,"line":126},9,[74,128,129],{},"        tasks = asyncio.all_tasks(loop)\n",[74,131,133],{"class":76,"line":132},10,[74,134,135],{},"        print(f\"--- {len(tasks)} tasks ---\", file=sys.stderr)\n",[74,137,139],{"class":76,"line":138},11,[74,140,141],{},"        for task in tasks:\n",[74,143,145],{"class":76,"line":144},12,[74,146,147],{},"            print(f\"\\n{task.get_name()}  {task.get_coro().__qualname__}\", file=sys.stderr)\n",[74,149,151],{"class":76,"line":150},13,[74,152,153],{},"            task.print_stack(file=sys.stderr)\n",[74,155,157],{"class":76,"line":156},14,[74,158,159],{},"    loop.add_signal_handler(signal.SIGUSR1, dump)        # runs on the loop: needs it alive\n",[74,161,163],{"class":76,"line":162},15,[74,164,165],{},"    faulthandler.register(signal.SIGUSR2, all_threads=True)  # runs anywhere: works when blocked\n",[65,167,171],{"className":168,"code":169,"language":170,"meta":70,"style":70},"language-bash shiki shiki-themes github-light github-dark","kill -USR1 \u003Cpid>          # where is each task awaiting?\nkill -USR2 \u003Cpid>          # what is each thread executing right now?\npy-spy dump --pid \u003Cpid>   # the same, from outside, no code changes\n\n# Python 3.14+: the await graph of a running process, from outside.\npython -m asyncio pstree \u003Cpid>\n","bash",[14,172,173,201,219,241,245,250],{"__ignoreMap":70},[74,174,175,179,182,186,190,194,197],{"class":76,"line":77},[74,176,178],{"class":177},"sj4cs","kill",[74,180,181],{"class":177}," -USR1",[74,183,185],{"class":184},"szBVR"," \u003C",[74,187,189],{"class":188},"sZZnC","pi",[74,191,193],{"class":192},"sVt8B","d",[74,195,196],{"class":184},">",[74,198,200],{"class":199},"sJ8bj","          # where is each task awaiting?\n",[74,202,203,205,208,210,212,214,216],{"class":76,"line":83},[74,204,178],{"class":177},[74,206,207],{"class":177}," -USR2",[74,209,185],{"class":184},[74,211,189],{"class":188},[74,213,193],{"class":192},[74,215,196],{"class":184},[74,217,218],{"class":199},"          # what is each thread executing right now?\n",[74,220,221,224,227,230,232,234,236,238],{"class":76,"line":89},[74,222,50],{"class":223},"sScJk",[74,225,226],{"class":188}," dump",[74,228,229],{"class":177}," --pid",[74,231,185],{"class":184},[74,233,189],{"class":188},[74,235,193],{"class":192},[74,237,196],{"class":184},[74,239,240],{"class":199},"   # the same, from outside, no code changes\n",[74,242,243],{"class":76,"line":95},[74,244,111],{"emptyLinePlaceholder":110},[74,246,247],{"class":76,"line":101},[74,248,249],{"class":199},"# Python 3.14+: the await graph of a running process, from outside.\n",[74,251,252,254,257,260,263,265,267,269],{"class":76,"line":107},[74,253,69],{"class":223},[74,255,256],{"class":177}," -m",[74,258,259],{"class":188}," asyncio",[74,261,262],{"class":188}," pstree",[74,264,185],{"class":184},[74,266,189],{"class":188},[74,268,193],{"class":192},[74,270,271],{"class":184},">\n",[65,273,275],{"className":67,"code":274,"language":69,"meta":70,"style":70},"# Tests: bound every external wait, and dump stacks if something still hangs.\nasync def test_consumer_drains_queue(broker):\n    async with asyncio.timeout(5):\n        await consumer.run_until_empty()\n",[14,276,277,282,287,292],{"__ignoreMap":70},[74,278,279],{"class":76,"line":77},[74,280,281],{},"# Tests: bound every external wait, and dump stacks if something still hangs.\n",[74,283,284],{"class":76,"line":83},[74,285,286],{},"async def test_consumer_drains_queue(broker):\n",[74,288,289],{"class":76,"line":89},[74,290,291],{},"    async with asyncio.timeout(5):\n",[74,293,294],{"class":76,"line":95},[74,295,296],{},"        await consumer.run_until_empty()\n",[65,298,302],{"className":299,"code":300,"language":301,"meta":70,"style":70},"language-toml shiki shiki-themes github-light github-dark","# pyproject.toml\n[tool.pytest.ini_options]\ntimeout = 60\ntimeout_method = \"thread\"      # dumps every thread's stack on timeout\n","toml",[14,303,304,309,314,319],{"__ignoreMap":70},[74,305,306],{"class":76,"line":77},[74,307,308],{},"# pyproject.toml\n",[74,310,311],{"class":76,"line":83},[74,312,313],{},"[tool.pytest.ini_options]\n",[74,315,316],{"class":76,"line":89},[74,317,318],{},"timeout = 60\n",[74,320,321],{"class":76,"line":95},[74,322,323],{},"timeout_method = \"thread\"      # dumps every thread's stack on timeout\n",[325,326,329,476],"figure",{"className":327},[328],"diagram",[330,331,338,339,338,343,338,347,338,365,338,373,338,383,338,390,338,396,338,401,338,409,338,415,338,419,338,424,338,428,338,432,338,440,338,444,338,450,338,455,338,459,338,461,338,464,338,468,338,473],"svg",{"viewBox":332,"role":333,"ariaLabelledBy":334,"xmlns":337},"0 0 800 256","img",[335,336],"hw-t","hw-d","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","\n  ",[340,341,342],"title",{"id":335},"Is the loop blocked, or is a task waiting?",[344,345,346],"desc",{"id":336},"A decision flow starts with sending SIGUSR1 to trigger the task dump coroutine. If the dump prints, the loop is alive and the task stacks show which await is waiting on something that never completes. If nothing prints, the loop thread is blocked by synchronous code, and a thread stack dump from faulthandler or py-spy shows the blocking call.",[348,349,350,351,338],"defs",{},"\n    ",[352,353,360],"marker",{"id":354,"viewBox":355,"refX":356,"refY":357,"markerWidth":358,"markerHeight":358,"orient":359},"hw-a","0 0 10 10","9","5","7","auto-start-reverse",[361,362],"path",{"d":363,"fill":364},"M0 0 L10 5 L0 10 z","#81b29a",[366,367],"rect",{"x":368,"y":368,"width":369,"height":370,"rx":371,"fill":372},"0","800","256","14","#fffdf8",[374,375,382],"text",{"x":376,"y":377,"textAnchor":378,"fontSize":379,"fontWeight":380,"fill":381},"400","28","middle","15.5","700","#3d405b","First question: does the loop still run?",[366,384],{"x":385,"y":386,"width":387,"height":388,"rx":389,"fill":381},"26","104","190","56","10",[374,391,395],{"x":392,"y":393,"textAnchor":378,"fontSize":394,"fontWeight":380,"fill":372},"121","128","12","kill -USR1",[374,397,400],{"x":392,"y":398,"textAnchor":378,"fontSize":399,"fill":372},"146","10.5","task dump on the loop",[366,402],{"x":403,"y":404,"width":405,"height":406,"rx":389,"fill":407,"stroke":364,"strokeWidth":408},"290","52","200","60","#e6f0ea","2",[374,410,414],{"x":411,"y":412,"textAnchor":378,"fontSize":413,"fontWeight":380,"fill":381},"390","78","11.5","dump prints",[374,416,418],{"x":411,"y":417,"textAnchor":378,"fontSize":399,"fill":381},"98","loop alive, a task waits",[366,420],{"x":403,"y":421,"width":405,"height":406,"rx":389,"fill":422,"stroke":423,"strokeWidth":408},"152","#fbe9e3","#e07a5f",[374,425,427],{"x":411,"y":426,"textAnchor":378,"fontSize":413,"fontWeight":380,"fill":381},"178","nothing prints",[374,429,431],{"x":411,"y":430,"textAnchor":378,"fontSize":399,"fill":381},"198","loop thread blocked",[76,433],{"x1":434,"y1":435,"x2":436,"y2":437,"stroke":364,"strokeWidth":438,"markerEnd":439},"218","122","286","88","1.6","url(#hw-a)",[76,441],{"x1":434,"y1":442,"x2":436,"y2":443,"stroke":364,"strokeWidth":438,"markerEnd":439},"142","176",[366,445],{"x":446,"y":404,"width":447,"height":406,"rx":389,"fill":448,"stroke":381,"strokeWidth":449},"560","214","#f4f1de","1.5",[374,451,454],{"x":452,"y":412,"textAnchor":378,"fontSize":453,"fontWeight":380,"fill":381},"667","11","read task stacks",[374,456,458],{"x":452,"y":417,"textAnchor":378,"fontSize":399,"fill":457},"#2a5f49","find the await that never returns",[366,460],{"x":446,"y":421,"width":447,"height":406,"rx":389,"fill":448,"stroke":381,"strokeWidth":449},[374,462,463],{"x":452,"y":426,"textAnchor":378,"fontSize":453,"fontWeight":380,"fill":381},"faulthandler \u002F py-spy dump",[374,465,467],{"x":452,"y":430,"textAnchor":378,"fontSize":399,"fill":466},"#8f3d22","find the synchronous call",[76,469],{"x1":470,"y1":471,"x2":472,"y2":471,"stroke":364,"strokeWidth":438,"markerEnd":439},"492","82","556",[76,474],{"x1":470,"y1":475,"x2":472,"y2":475,"stroke":364,"strokeWidth":438,"markerEnd":439},"182",[477,478,479],"figcaption",{},"A coroutine-based dump that never prints is itself the diagnosis: nothing on the loop can run.",[25,481,483],{"id":482},"why-this-works","Why this works",[10,485,486,489,490,494,495,498,499,502,503,505],{},[14,487,488],{},"loop.add_signal_handler"," schedules the callback ",[491,492,493],"em",{},"on the event loop",", so it runs only when the loop gets a chance to process callbacks. That is a feature: if it runs, the loop is healthy and the hang is a task waiting on something. ",[14,496,497],{},"asyncio.all_tasks"," returns every unfinished task, and ",[14,500,501],{},"task.print_stack()"," prints the frames of the task's coroutine chain as of its current suspension point — the innermost frame is the ",[14,504,16],{}," it is stuck on.",[10,507,508,511,512,515,516,519,520,523,524,527],{},[14,509,510],{},"faulthandler.register"," installs a C-level signal handler that writes the Python stack of every thread directly to a file descriptor, without needing the interpreter to reach a safe point in Python code. It works even when the loop thread is stuck in a blocking ",[14,513,514],{},"socket.recv",", a ",[14,517,518],{},"time.sleep",", or a deadlock on a ",[14,521,522],{},"threading.Lock",". ",[14,525,526],{},"py-spy dump"," reads the same information from outside the process by inspecting its memory, which also works on processes that never installed a handler.",[10,529,530,531,534,535,537],{},"Python 3.14 adds ",[14,532,533],{},"python -m asyncio ps"," and ",[14,536,42],{},", which use the new remote debugging interface to read the task graph of another process — which tasks exist and which task is awaiting which — without any in-process setup. It shows exactly the \"who waits on whom\" structure that makes deadlocks between tasks visible.",[25,539,541],{"id":540},"reading-a-task-dump","Reading a task dump",[10,543,544],{},"A typical task dump from a hung worker:",[65,546,550],{"className":547,"code":549,"language":374,"meta":70},[548],"language-text","--- 3 tasks ---\n\nTask-1  main\n  File \"app\u002Fworker.py\", line 88, in main\n    await asyncio.gather(consumer(q), producer(q))\n\nconsumer  consumer\n  File \"app\u002Fworker.py\", line 52, in consumer\n    item = await q.get()\n\nproducer  producer\n  File \"app\u002Fworker.py\", line 70, in producer\n    await client.fetch_page(cursor)\n  File \"app\u002Fclient.py\", line 31, in fetch_page\n    async with self._lock:\n",[14,551,549],{"__ignoreMap":70},[10,553,554,557,558,561,562,565,566,569,570,573,574,577,578,581],{},[14,555,556],{},"main"," waits on ",[14,559,560],{},"gather",", which is normal. ",[14,563,564],{},"consumer"," waits on an empty queue, which is also normal — it is waiting for the producer. The producer is waiting to acquire ",[14,567,568],{},"self._lock",". The question becomes: who holds the lock? Search the dump for any other task inside a ",[14,571,572],{},"async with self._lock"," block; if there is none, the lock was acquired and never released — often by a code path that returned early or raised inside a manual ",[14,575,576],{},"acquire()"," without ",[14,579,580],{},"finally",". The dump turned \"it hangs\" into \"the client's lock is leaked on some error path\", which is directly fixable.",[325,583,585,671],{"className":584},[328],[330,586,338,591,338,594,338,597,338,604,338,607,338,610,338,615,338,620,338,625,338,629,338,634,338,637,338,641,338,644,338,647,338,651,338,654,338,659,338,663,338,667],{"viewBox":587,"role":333,"ariaLabelledBy":588,"xmlns":337},"0 0 800 236",[589,590],"hwg-t","hwg-d",[340,592,593],{"id":589},"The await chain in the hung worker",[344,595,596],{"id":590},"Main awaits gather of consumer and producer. The consumer awaits q.get on an empty queue, waiting for the producer. The producer awaits the client lock, which no task holds because it was leaked on an error path. The whole chain is stuck behind the leaked lock.",[348,598,350,599,338],{},[352,600,602],{"id":601,"viewBox":355,"refX":356,"refY":357,"markerWidth":358,"markerHeight":358,"orient":359},"hwg-a",[361,603],{"d":363,"fill":364},[366,605],{"x":368,"y":368,"width":369,"height":606,"rx":371,"fill":372},"236",[374,608,609],{"x":376,"y":377,"textAnchor":378,"fontSize":379,"fontWeight":380,"fill":381},"Follow the waits to the thing nobody will release",[366,611],{"x":385,"y":612,"width":613,"height":614,"rx":389,"fill":448,"stroke":381,"strokeWidth":449},"96","150","50",[374,616,619],{"x":617,"y":618,"textAnchor":378,"fontSize":413,"fill":381},"101","126","main · gather",[366,621],{"x":622,"y":612,"width":613,"height":614,"rx":389,"fill":623,"stroke":624,"strokeWidth":408},"226","#f7f0da","#f2cc8f",[374,626,564],{"x":627,"y":628,"textAnchor":378,"fontSize":413,"fill":381},"301","118",[374,630,633],{"x":627,"y":631,"textAnchor":378,"fontSize":389,"fill":632},"136","#8a5a00","await q.get()",[366,635],{"x":636,"y":612,"width":613,"height":614,"rx":389,"fill":623,"stroke":624,"strokeWidth":408},"426",[374,638,640],{"x":639,"y":628,"textAnchor":378,"fontSize":413,"fill":381},"501","producer",[374,642,643],{"x":639,"y":631,"textAnchor":378,"fontSize":389,"fill":632},"async with _lock",[366,645],{"x":646,"y":612,"width":613,"height":614,"rx":389,"fill":422,"stroke":423,"strokeWidth":408},"626",[374,648,650],{"x":649,"y":628,"textAnchor":378,"fontSize":413,"fontWeight":380,"fill":466},"701","_lock held",[374,652,653],{"x":649,"y":631,"textAnchor":378,"fontSize":389,"fill":381},"by no live task",[76,655],{"x1":426,"y1":392,"x2":656,"y2":392,"stroke":364,"strokeWidth":657,"markerEnd":658},"222","1.8","url(#hwg-a)",[76,660],{"x1":661,"y1":392,"x2":662,"y2":392,"stroke":364,"strokeWidth":657,"markerEnd":658},"378","422",[76,664],{"x1":665,"y1":392,"x2":666,"y2":392,"stroke":364,"strokeWidth":657,"markerEnd":658},"578","622",[374,668,670],{"x":376,"y":669,"textAnchor":378,"fontSize":453,"fill":381},"196","A lock with no holder in the dump was leaked on an error path.",[477,672,673],{},"The end of the chain is the bug: a resource something is waiting for that nothing alive will ever release.",[25,675,677],{"id":676},"preventing-hangs-with-timeouts-at-the-boundaries","Preventing hangs with timeouts at the boundaries",[10,679,680],{},"Dumps explain hangs after the fact. Timeouts turn future hangs into errors with tracebacks, which is far cheaper to debug. The principle is to bound every wait on something outside the program's control, at the point where the wait happens.",[10,682,683,684,687,688,691,692,695,696,699,700,703],{},"Network calls are the obvious case: HTTP clients, database drivers and message brokers should all have connect and read timeouts configured, not left at \"wait forever\". Internal coordination is the less obvious case. A ",[14,685,686],{},"queue.get()"," that waits for a producer, an ",[14,689,690],{},"event.wait()"," for a signal from another task, a ",[14,693,694],{},"lock"," acquisition — each of these hangs forever if the other side has a bug. Wrapping them in ",[14,697,698],{},"async with asyncio.timeout(...)"," with a generous limit costs nothing in the normal case and converts a silent hang into a ",[14,701,702],{},"TimeoutError"," whose traceback names the exact await.",[10,705,706,707,710],{},"Choose limits from what the operation should take, multiplied by a safety margin, and put them in configuration rather than scattered literals. In tests, set them much tighter than production — a queue that should fill within milliseconds in a test can have a one-second limit — so that a hang fails the test quickly and precisely instead of waiting for the global pytest-timeout. The global timeout remains as the last line of defence, with ",[14,708,709],{},"timeout_method = \"thread\""," so that even an unexpected hang produces every thread's stack in the CI log.",[325,712,714,761],{"className":713},[328],[330,715,338,720,338,723,338,726,338,728,338,731,338,737,338,741,338,745,338,750,338,755,338,758],{"viewBox":716,"role":333,"ariaLabelledBy":717,"xmlns":337},"0 0 800 226",[718,719],"hwt-t","hwt-d",[340,721,722],{"id":718},"Layers of timeouts",[344,724,725],{"id":719},"Three nested layers bound a test. Innermost, per-await timeouts on queues, events, locks and network calls raise TimeoutError at the exact await. Next, the pytest-timeout global limit dumps all thread stacks if something unbounded hangs. Outermost, the CI job timeout kills the job with no diagnostics and should never be the one that fires.",[366,727],{"x":368,"y":368,"width":369,"height":622,"rx":371,"fill":372},[374,729,730],{"x":376,"y":377,"textAnchor":378,"fontSize":379,"fontWeight":380,"fill":381},"The innermost timeout should always fire first",[366,732],{"x":733,"y":734,"width":735,"height":736,"rx":371,"fill":422,"stroke":423,"strokeWidth":657},"40","46","720","164",[374,738,740],{"x":406,"y":739,"fontSize":413,"fontWeight":380,"fill":466},"68","CI job timeout — kills, no diagnostics",[366,742],{"x":743,"y":743,"width":744,"height":628,"rx":394,"fill":623,"stroke":624,"strokeWidth":657},"80","640",[374,746,749],{"x":747,"y":748,"fontSize":413,"fontWeight":380,"fill":632},"100","102","pytest-timeout (thread) — dumps every stack",[366,751],{"x":752,"y":753,"width":446,"height":754,"rx":389,"fill":407,"stroke":364,"strokeWidth":408},"120","114","70",[374,756,757],{"x":376,"y":442,"textAnchor":378,"fontSize":394,"fontWeight":380,"fill":457},"asyncio.timeout at each await",[374,759,760],{"x":376,"y":736,"textAnchor":378,"fontSize":399,"fill":381},"TimeoutError with the exact await in the traceback",[477,762,763],{},"Each outer layer is a fallback with worse diagnostics; tight inner limits keep hangs in the innermost box.",[25,765,767],{"id":766},"hangs-that-only-happen-in-ci","Hangs that only happen in CI",[10,769,770,771,774,775,778],{},"A hang that never reproduces locally is usually a timing difference, and the dump is still the fastest route to it. Make sure CI produces one: ",[14,772,773],{},"pytest-timeout"," with the thread method prints every thread's stack when a test exceeds its limit, and adding ",[14,776,777],{},"faulthandler.dump_traceback_later(timeout - 10, exit=False)"," in a session fixture gives a second dump shortly before, in case the first is lost. Both land in the job log.",[10,780,781],{},"Common CI-only causes show up clearly in those stacks. A test waiting on a service container that has not finished starting shows an await inside the client's connect call. A test that relies on a background task being scheduled before an assertion shows the task still pending, because the slower runner has not reached it yet. And a deadlock between two tasks that only interleave badly under load shows each task waiting on a lock or event owned by the other. In each case, the fix is to replace an assumption about timing with explicit synchronisation — a readiness probe, an awaited event, a consistent lock order — rather than adding a sleep.",[25,783,785],{"id":784},"edge-cases-and-failure-modes","Edge cases and failure modes",[30,787,788,802,821,827,839],{},[33,789,790,794,795,798,799,51],{},[791,792,793],"strong",{},"Windows."," ",[14,796,797],{},"add_signal_handler"," is not available with the default Proactor loop. Trigger the dump from a debug HTTP endpoint or a watchdog thread using ",[14,800,801],{},"loop.call_soon_threadsafe",[33,803,804,794,807,534,809,812,813,816,817,820],{},[791,805,806],{},"Hidden inner tasks.",[14,808,560],{},[14,810,811],{},"wait_for"," create inner tasks; they appear in ",[14,814,815],{},"all_tasks"," with generated names. Name your own tasks with ",[14,818,819],{},"create_task(..., name=...)"," so dumps are readable.",[33,822,823,826],{},[791,824,825],{},"Awaiting a future from another loop."," A future created on one loop and awaited on another never completes. The dump shows the await; check where the future was created.",[33,828,829,794,832,835,836,838],{},[791,830,831],{},"Timeouts that swallow context.",[14,833,834],{},"asyncio.timeout"," raises ",[14,837,702],{}," at the hung await, with that await in the traceback — keep it rather than catching and replacing it.",[33,840,841,844,845,848],{},[791,842,843],{},"Deadlock across threads."," A coroutine awaiting ",[14,846,847],{},"run_in_executor"," whose worker is blocked on a lock the loop thread holds needs a thread dump, not a task dump.",[25,850,852],{"id":851},"frequently-asked-questions","Frequently Asked Questions",[10,854,855,858,859,862,863,865,866,868,869,872],{},[791,856,857],{},"How do I see where every asyncio task is waiting?","\nIterate ",[14,860,861],{},"asyncio.all_tasks()"," and call ",[14,864,501],{}," on each, from inside the loop — for example from a signal handler registered with ",[14,867,488],{},". On Python 3.14, ",[14,870,871],{},"python -m asyncio pstree \u003Cpid>"," shows the await tree of a running process from outside.",[10,874,875,878,879,882,883,885],{},[791,876,877],{},"What if the whole event loop is stuck, not just one task?","\nThen something is blocking the loop thread synchronously and no coroutine can run, including a debugging one. Use ",[14,880,881],{},"faulthandler.dump_traceback"," or ",[14,884,526],{}," to see the thread's stack; it will show the blocking call.",[10,887,888,891,892,895,896,899],{},[791,889,890],{},"How do I stop a hung async test from blocking CI forever?","\nUse pytest-timeout with ",[14,893,894],{},"timeout_method = thread",", which dumps all thread stacks when the limit is hit, and wrap awaited operations in ",[14,897,898],{},"asyncio.timeout()"," so they fail with a traceback at the await that hung.",[25,901,903],{"id":902},"related","Related",[30,905,906,912,919,926],{},[33,907,908,911],{},[56,909,910],{"href":58},"Debugging Async Code and Event Loops"," — asyncio debugging fundamentals.",[33,913,914,918],{},[56,915,917],{"href":916},"\u002Fsystematic-debugging-performance-profiling\u002Fdebugging-async-code-and-event-loops\u002Ffinding-blocking-calls-with-asyncio-debug-mode\u002F","Finding Blocking Calls with asyncio Debug Mode"," — when the loop is blocked.",[33,920,921,925],{},[56,922,924],{"href":923},"\u002Fsystematic-debugging-performance-profiling\u002Fdebugging-async-code-and-event-loops\u002Fdiagnosing-task-was-destroyed-warnings\u002F","Diagnosing \"Task Was Destroyed\" Warnings"," — tasks nobody owns.",[33,927,928,932],{},[56,929,931],{"href":930},"\u002Fsystematic-debugging-performance-profiling\u002Fcpu-profiling-with-cprofile-and-py-spy\u002Fprofiling-a-running-process-with-py-spy\u002F","Profiling a Running Process with py-spy"," — dumps from outside the process.",[10,934,935,936],{},"← Back to ",[56,937,910],{"href":58},[939,940,941],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .sj4cs, html code.shiki .sj4cs{--shiki-default:#005CC5;--shiki-dark:#79B8FF}html pre.shiki code .szBVR, html code.shiki .szBVR{--shiki-default:#D73A49;--shiki-dark:#F97583}html pre.shiki code .sZZnC, html code.shiki .sZZnC{--shiki-default:#032F62;--shiki-dark:#9ECBFF}html pre.shiki code .sVt8B, html code.shiki .sVt8B{--shiki-default:#24292E;--shiki-dark:#E1E4E8}html pre.shiki code .sJ8bj, html code.shiki .sJ8bj{--shiki-default:#6A737D;--shiki-dark:#6A737D}html pre.shiki code .sScJk, html code.shiki .sScJk{--shiki-default:#6F42C1;--shiki-dark:#B392F0}",{"title":70,"searchDepth":83,"depth":83,"links":943},[944,945,946,947,948,949,950,951,952],{"id":27,"depth":83,"text":28},{"id":62,"depth":83,"text":63},{"id":482,"depth":83,"text":483},{"id":540,"depth":83,"text":541},{"id":676,"depth":83,"text":677},{"id":766,"depth":83,"text":767},{"id":784,"depth":83,"text":785},{"id":851,"depth":83,"text":852},{"id":902,"depth":83,"text":903},"Find where an asyncio program is stuck: dumping every task's stack with all_tasks and print_stack, Python 3.14's asyncio ps and pstree, faulthandler for blocked loops, and timeouts in tests.","md",{"slug":956,"type":957,"breadcrumb":958,"datePublished":959,"dateModified":959,"faq":960,"howto":967},"tracking-down-a-hung-await-with-task-stacks","article","Hung await","2026-09-18",[961,963,965],{"q":857,"a":962},"Iterate asyncio.all_tasks() and call task.print_stack() on each, from inside the loop — for example from a signal handler registered with loop.add_signal_handler. On Python 3.14, python -m asyncio pstree \u003Cpid> shows the await tree of a running process from outside.",{"q":877,"a":964},"Then something is blocking the loop thread synchronously and no coroutine can run, including a debugging one. Use faulthandler.dump_traceback or py-spy dump to see the thread's stack; it will show the blocking call.",{"q":890,"a":966},"Use pytest-timeout with timeout_method = thread, which dumps all thread stacks when the limit is hit, and wrap awaited operations in asyncio.timeout() so they fail with a traceback at the await that hung.",{"name":968,"description":969,"steps":970},"How to find a hung await","Decide whether the loop is blocked or a task is waiting, then dump the right kind of stack.",[971,974,977,980],{"name":972,"text":973},"Check whether the loop is alive","If a signal-registered coroutine dump runs, the loop is alive and some task is waiting; if not, the loop thread is blocked.",{"name":975,"text":976},"Dump task stacks","Print the stack of every task from all_tasks, or use asyncio pstree on 3.14.",{"name":978,"text":979},"Dump thread stacks","For a blocked loop, use faulthandler or py-spy dump to see the synchronous call.",{"name":981,"text":982},"Add timeouts","Wrap awaits on external resources in asyncio.timeout so future hangs fail with a traceback.","\u002Fsystematic-debugging-performance-profiling\u002Fdebugging-async-code-and-event-loops\u002Ftracking-down-a-hung-await-with-task-stacks",{"title":5,"description":953},"systematic-debugging-performance-profiling\u002Fdebugging-async-code-and-event-loops\u002Ftracking-down-a-hung-await-with-task-stacks\u002Findex","BqRAHQeBoS-AGXH7qlfxUwUY_hQc3vGbQf1zBZ6__mY",1789718765723]