[{"data":1,"prerenderedAt":1279},["ShallowReactive",2],{"page-\u002Ftesting-async-and-concurrent-python\u002Ftimeouts-cancellation-and-deadlines\u002Ftesting-cancellation-and-cleanup-paths\u002F":3},{"id":4,"title":5,"body":6,"description":1242,"extension":1243,"meta":1244,"navigation":89,"path":1275,"seo":1276,"stem":1277,"__hash__":1278},"content\u002Ftesting-async-and-concurrent-python\u002Ftimeouts-cancellation-and-deadlines\u002Ftesting-cancellation-and-cleanup-paths\u002Findex.md","Testing Cancellation and Cleanup Paths",{"type":7,"value":8,"toc":1231},"minimark",[9,18,23,60,64,67,287,466,470,488,495,499,572,576,579,582,661,730,741,799,802,868,871,874,878,881,938,1049,1055,1059,1062,1080,1091,1151,1155,1167,1177,1189,1193,1222,1227],[10,11,12,13,17],"p",{},"Cancellation is the path production takes whenever a deadline fires, a client disconnects or a shutdown begins, and it is the path test suites cover least. The usual state of affairs is a ",[14,15,16],"code",{},"finally"," block that has never executed under cancellation, a lock that is released only on the happy path, and a connection that leaks once per timeout — discovered when the pool exhausts under load rather than in a test.",[19,20,22],"h2",{"id":21},"prerequisites","Prerequisites",[24,25,26,42,53],"ul",{},[27,28,29,30,33,34,37,38,41],"li",{},"Python 3.11+ for ",[14,31,32],{},"asyncio.timeout","; 3.8+ for ",[14,35,36],{},"CancelledError"," inheriting from ",[14,39,40],{},"BaseException",".",[27,43,44,47,48,41],{},[14,45,46],{},"pytest >= 8.0"," with an async runner configured per ",[49,50,52],"a",{"href":51},"\u002Ftesting-async-and-concurrent-python\u002Fpytest-asyncio-in-depth\u002F","pytest-asyncio in depth",[27,54,55,56,59],{},"An understanding that cancellation is delivered as an exception at the next ",[14,57,58],{},"await",", not as a thread kill.",[19,61,63],{"id":62},"solution","Solution",[10,65,66],{},"Start the work as a task, cancel it at a point the test controls, and assert on the state cleanup was supposed to restore.",[68,69,74],"pre",{"className":70,"code":71,"language":72,"meta":73,"style":73},"language-python shiki shiki-themes github-light github-dark","import asyncio\n\nimport pytest\n\n\nclass Worker:\n    def __init__(self, pool, lock):\n        self._pool, self._lock = pool, lock\n        self.started = asyncio.Event()\n\n    async def run(self):\n        connection = await self._pool.acquire()\n        await self._lock.acquire()\n        try:\n            self.started.set()            # the test's synchronisation point\n            while True:\n                await asyncio.sleep(3600) # a suspension point cancellation can reach\n        finally:\n            self._lock.release()          # must happen on the cancelled path too\n            await self._pool.release(connection)\n\n\nasync def test_cancellation_releases_every_resource(pool, lock):\n    worker = Worker(pool, lock)\n    task = asyncio.create_task(worker.run())\n\n    # Deterministic: wait for the worker's own signal, never a sleep.\n    await asyncio.wait_for(worker.started.wait(), timeout=1)\n\n    task.cancel()\n    with pytest.raises(asyncio.CancelledError):\n        await task                        # awaiting lets the unwinding finish\n\n    # The assertions that matter are about state, not about the exception.\n    assert not lock.locked(), \"the lock was not released on cancellation\"\n    assert pool.in_use == 0, \"the connection was not returned to the pool\"\n","python","",[14,75,76,84,91,97,102,107,113,119,125,131,136,142,148,154,160,166,172,178,184,190,196,201,206,212,218,224,229,235,241,246,252,258,264,269,275,281],{"__ignoreMap":73},[77,78,81],"span",{"class":79,"line":80},"line",1,[77,82,83],{},"import asyncio\n",[77,85,87],{"class":79,"line":86},2,[77,88,90],{"emptyLinePlaceholder":89},true,"\n",[77,92,94],{"class":79,"line":93},3,[77,95,96],{},"import pytest\n",[77,98,100],{"class":79,"line":99},4,[77,101,90],{"emptyLinePlaceholder":89},[77,103,105],{"class":79,"line":104},5,[77,106,90],{"emptyLinePlaceholder":89},[77,108,110],{"class":79,"line":109},6,[77,111,112],{},"class Worker:\n",[77,114,116],{"class":79,"line":115},7,[77,117,118],{},"    def __init__(self, pool, lock):\n",[77,120,122],{"class":79,"line":121},8,[77,123,124],{},"        self._pool, self._lock = pool, lock\n",[77,126,128],{"class":79,"line":127},9,[77,129,130],{},"        self.started = asyncio.Event()\n",[77,132,134],{"class":79,"line":133},10,[77,135,90],{"emptyLinePlaceholder":89},[77,137,139],{"class":79,"line":138},11,[77,140,141],{},"    async def run(self):\n",[77,143,145],{"class":79,"line":144},12,[77,146,147],{},"        connection = await self._pool.acquire()\n",[77,149,151],{"class":79,"line":150},13,[77,152,153],{},"        await self._lock.acquire()\n",[77,155,157],{"class":79,"line":156},14,[77,158,159],{},"        try:\n",[77,161,163],{"class":79,"line":162},15,[77,164,165],{},"            self.started.set()            # the test's synchronisation point\n",[77,167,169],{"class":79,"line":168},16,[77,170,171],{},"            while True:\n",[77,173,175],{"class":79,"line":174},17,[77,176,177],{},"                await asyncio.sleep(3600) # a suspension point cancellation can reach\n",[77,179,181],{"class":79,"line":180},18,[77,182,183],{},"        finally:\n",[77,185,187],{"class":79,"line":186},19,[77,188,189],{},"            self._lock.release()          # must happen on the cancelled path too\n",[77,191,193],{"class":79,"line":192},20,[77,194,195],{},"            await self._pool.release(connection)\n",[77,197,199],{"class":79,"line":198},21,[77,200,90],{"emptyLinePlaceholder":89},[77,202,204],{"class":79,"line":203},22,[77,205,90],{"emptyLinePlaceholder":89},[77,207,209],{"class":79,"line":208},23,[77,210,211],{},"async def test_cancellation_releases_every_resource(pool, lock):\n",[77,213,215],{"class":79,"line":214},24,[77,216,217],{},"    worker = Worker(pool, lock)\n",[77,219,221],{"class":79,"line":220},25,[77,222,223],{},"    task = asyncio.create_task(worker.run())\n",[77,225,227],{"class":79,"line":226},26,[77,228,90],{"emptyLinePlaceholder":89},[77,230,232],{"class":79,"line":231},27,[77,233,234],{},"    # Deterministic: wait for the worker's own signal, never a sleep.\n",[77,236,238],{"class":79,"line":237},28,[77,239,240],{},"    await asyncio.wait_for(worker.started.wait(), timeout=1)\n",[77,242,244],{"class":79,"line":243},29,[77,245,90],{"emptyLinePlaceholder":89},[77,247,249],{"class":79,"line":248},30,[77,250,251],{},"    task.cancel()\n",[77,253,255],{"class":79,"line":254},31,[77,256,257],{},"    with pytest.raises(asyncio.CancelledError):\n",[77,259,261],{"class":79,"line":260},32,[77,262,263],{},"        await task                        # awaiting lets the unwinding finish\n",[77,265,267],{"class":79,"line":266},33,[77,268,90],{"emptyLinePlaceholder":89},[77,270,272],{"class":79,"line":271},34,[77,273,274],{},"    # The assertions that matter are about state, not about the exception.\n",[77,276,278],{"class":79,"line":277},35,[77,279,280],{},"    assert not lock.locked(), \"the lock was not released on cancellation\"\n",[77,282,284],{"class":79,"line":283},36,[77,285,286],{},"    assert pool.in_use == 0, \"the connection was not returned to the pool\"\n",[288,289,292,458],"figure",{"className":290},[291],"diagram",[293,294,301,302,301,306,301,310,301,328,301,336,301,345,301,355,301,361,301,366,301,373,301,378,301,382,301,386,301,390,301,394,301,398,301,401,301,406,301,412,301,415,301,419,301,423,301,428,301,433,301,438,301,442,301,446,301,449,301,454],"svg",{"viewBox":295,"role":296,"ariaLabelledBy":297,"xmlns":300},"0 0 820 268","img",[298,299],"can-t","can-d","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","\n  ",[303,304,305],"title",{"id":298},"Where cancellation is delivered and what unwinds",[307,308,309],"desc",{"id":299},"A timeline for one task. It acquires a connection and a lock, signals that it has started, and suspends on an await. The test cancels it, a CancelledError is raised at that await, the finally block releases the lock and returns the connection, and awaiting the task re-raises the cancellation in the test.",[311,312,313,314,301],"defs",{},"\n    ",[315,316,323],"marker",{"id":317,"viewBox":318,"refX":319,"refY":320,"markerWidth":321,"markerHeight":321,"orient":322},"can-a","0 0 10 10","9","5","7","auto-start-reverse",[324,325],"path",{"d":326,"fill":327},"M0 0 L10 5 L0 10 z","#3d405b",[329,330],"rect",{"x":331,"y":331,"width":332,"height":333,"rx":334,"fill":335},"0","820","268","14","#fffdf8",[337,338,344],"text",{"x":339,"y":340,"textAnchor":341,"fontSize":342,"fontWeight":343,"fill":327},"410","28","middle","16","700","Cancellation unwinds; it does not kill",[329,346],{"x":347,"y":348,"width":349,"height":350,"rx":351,"fill":352,"stroke":353,"strokeWidth":354},"26","56","168","58","10","#e6f0ea","#81b29a","1.8",[337,356,360],{"x":357,"y":358,"textAnchor":341,"fontSize":359,"fontWeight":343,"fill":327},"110","80","11.5","acquire",[337,362,365],{"x":357,"y":363,"textAnchor":341,"fontSize":364,"fill":327},"100","11","connection + lock",[79,367],{"x1":368,"y1":369,"x2":370,"y2":369,"stroke":327,"strokeWidth":371,"markerEnd":372},"198","85","222","1.5","url(#can-a)",[329,374],{"x":375,"y":348,"width":349,"height":350,"rx":351,"fill":376,"stroke":377,"strokeWidth":354},"228","#f7f0da","#f2cc8f",[337,379,381],{"x":380,"y":358,"textAnchor":341,"fontSize":359,"fontWeight":343,"fill":327},"312","started.set()",[337,383,385],{"x":380,"y":363,"textAnchor":341,"fontSize":364,"fill":384},"#8a5a00","test proceeds",[79,387],{"x1":388,"y1":369,"x2":389,"y2":369,"stroke":327,"strokeWidth":371,"markerEnd":372},"400","424",[329,391],{"x":392,"y":348,"width":349,"height":350,"rx":351,"fill":393,"stroke":327,"strokeWidth":354},"430","#f4f1de",[337,395,397],{"x":396,"y":358,"textAnchor":341,"fontSize":359,"fontWeight":343,"fill":327},"514","await sleep",[337,399,400],{"x":396,"y":363,"textAnchor":341,"fontSize":364,"fill":327},"suspension point",[79,402],{"x1":403,"y1":369,"x2":404,"y2":369,"stroke":405,"strokeWidth":371,"markerEnd":372},"602","626","#e07a5f",[329,407],{"x":408,"y":348,"width":409,"height":350,"rx":351,"fill":410,"stroke":405,"strokeWidth":411},"632","162","#fbe9e3","2",[337,413,36],{"x":414,"y":358,"textAnchor":341,"fontSize":359,"fontWeight":343,"fill":327},"713",[337,416,418],{"x":414,"y":363,"textAnchor":341,"fontSize":364,"fill":417},"#8f3d22","raised here",[79,420],{"x1":414,"y1":421,"x2":414,"y2":422,"stroke":327,"strokeWidth":371,"markerEnd":372},"120","146",[329,424],{"x":392,"y":425,"width":426,"height":427,"rx":351,"fill":352,"stroke":353,"strokeWidth":411},"150","364","54",[337,429,432],{"x":430,"y":431,"textAnchor":341,"fontSize":359,"fontWeight":343,"fill":327},"612","174","finally: release lock, return connection",[337,434,437],{"x":430,"y":435,"textAnchor":341,"fontSize":364,"fill":436},"194","#2a5f49","this is what the test asserts on",[329,439],{"x":347,"y":425,"width":440,"height":427,"rx":351,"fill":335,"stroke":441,"strokeWidth":371},"380","rgba(61,64,91,0.35)",[337,443,445],{"x":444,"y":431,"textAnchor":341,"fontSize":359,"fill":327},"216","a task blocked in synchronous code",[337,447,448],{"x":444,"y":435,"textAnchor":341,"fontSize":364,"fill":417},"never reaches the await, never cancels",[329,450],{"x":347,"y":444,"width":451,"height":452,"rx":319,"fill":335,"stroke":441,"strokeWidth":453},"768","36","1.4",[337,455,457],{"x":339,"y":456,"textAnchor":341,"fontSize":359,"fill":327},"240","Awaiting the cancelled task is what guarantees the finally has finished before the assertions run.",[459,460,461,462,465],"figcaption",{},"Skipping the ",[14,463,464],{},"await task"," is the commonest mistake: the assertions then race the unwinding and pass or fail depending on timing.",[19,467,469],{"id":468},"why-this-works","Why this works",[10,471,472,475,476,478,479,481,482,484,485,487],{},[14,473,474],{},"task.cancel()"," does not stop anything immediately. It arranges for ",[14,477,36],{}," to be raised inside the coroutine at its next suspension point, which unwinds the stack and runs every ",[14,480,16],{}," on the way out. Because that unwinding is itself asynchronous, the test must ",[14,483,464],{}," before asserting — otherwise the assertions run while the ",[14,486,16],{}," may not have executed.",[10,489,490,491,494],{},"Waiting on the worker's own ",[14,492,493],{},"started"," event rather than sleeping is what makes the cancellation land at a known place. A sleep-based version cancels at whatever point the scheduler happens to be at, which means the test sometimes cancels before the resources are acquired and therefore proves nothing.",[19,496,498],{"id":497},"edge-cases-and-failure-modes","Edge cases and failure modes",[24,500,501,515,535,549,555],{},[27,502,503,507,508,511,512,41],{},[504,505,506],"strong",{},"A coroutine blocked in synchronous code."," ",[14,509,510],{},"time.sleep",", a blocking socket read or a CPU loop never reaches an await, so the cancellation is never delivered. Push the work to ",[14,513,514],{},"asyncio.to_thread",[27,516,517,523,524,526,527,530,531,534],{},[504,518,519,522],{},[14,520,521],{},"except Exception"," around the await."," It does not catch ",[14,525,36],{}," in 3.8+, which is correct — but ",[14,528,529],{},"except BaseException"," and bare ",[14,532,533],{},"except:"," do, and both silently absorb cancellation.",[27,536,537,545,546,548],{},[504,538,539,541,542,544],{},[14,540,58],{}," inside ",[14,543,16],{}," without shielding."," Once cancellation is in flight, a further ",[14,547,58],{}," in the cleanup is cancelled immediately, so the cleanup never completes.",[27,550,551,554],{},[504,552,553],{},"Asserting before awaiting the task."," The unwinding has not finished; the assertion is a race.",[27,556,557,507,560,563,564,567,568,571],{},[504,558,559],{},"Cancelling a task that already finished.",[14,561,562],{},"cancel()"," returns ",[14,565,566],{},"False"," and nothing happens, so a test that cancels too late passes vacuously. Assert on ",[14,569,570],{},"task.cancelled()"," when the distinction matters.",[19,573,575],{"id":574},"making-cancellation-reachable-in-the-first-place","Making cancellation reachable in the first place",[10,577,578],{},"Half the cancellation bugs in a codebase are not bugs in the cleanup at all — they are places where the cancellation can never be delivered, so the cleanup is irrelevant. Those are worth finding before writing any assertion about unwinding.",[10,580,581],{},"A coroutine is cancellable only at its suspension points. Between them it runs to completion regardless of what anyone requests. Three patterns remove every suspension point from a region and therefore make it uncancellable:",[68,583,585],{"className":70,"code":584,"language":72,"meta":73,"style":73},"import asyncio\nimport time\n\nimport requests\n\n\nasync def uncancellable_three_ways(rows):\n    # 1. Blocking I\u002FO: the loop itself stops, so nothing can be delivered.\n    response = requests.get(\"https:\u002F\u002Fslow.example\")     # no await anywhere\n\n    # 2. A CPU loop: no await, so no suspension point for seconds at a time.\n    digest = sum(hash(row) for row in rows)             # rows may be enormous\n\n    # 3. A blocking sleep, which is both of the above at once.\n    time.sleep(5)\n    return response, digest\n",[14,586,587,591,596,600,605,609,613,618,623,628,632,637,642,646,651,656],{"__ignoreMap":73},[77,588,589],{"class":79,"line":80},[77,590,83],{},[77,592,593],{"class":79,"line":86},[77,594,595],{},"import time\n",[77,597,598],{"class":79,"line":93},[77,599,90],{"emptyLinePlaceholder":89},[77,601,602],{"class":79,"line":99},[77,603,604],{},"import requests\n",[77,606,607],{"class":79,"line":104},[77,608,90],{"emptyLinePlaceholder":89},[77,610,611],{"class":79,"line":109},[77,612,90],{"emptyLinePlaceholder":89},[77,614,615],{"class":79,"line":115},[77,616,617],{},"async def uncancellable_three_ways(rows):\n",[77,619,620],{"class":79,"line":121},[77,621,622],{},"    # 1. Blocking I\u002FO: the loop itself stops, so nothing can be delivered.\n",[77,624,625],{"class":79,"line":127},[77,626,627],{},"    response = requests.get(\"https:\u002F\u002Fslow.example\")     # no await anywhere\n",[77,629,630],{"class":79,"line":133},[77,631,90],{"emptyLinePlaceholder":89},[77,633,634],{"class":79,"line":138},[77,635,636],{},"    # 2. A CPU loop: no await, so no suspension point for seconds at a time.\n",[77,638,639],{"class":79,"line":144},[77,640,641],{},"    digest = sum(hash(row) for row in rows)             # rows may be enormous\n",[77,643,644],{"class":79,"line":150},[77,645,90],{"emptyLinePlaceholder":89},[77,647,648],{"class":79,"line":156},[77,649,650],{},"    # 3. A blocking sleep, which is both of the above at once.\n",[77,652,653],{"class":79,"line":162},[77,654,655],{},"    time.sleep(5)\n",[77,657,658],{"class":79,"line":168},[77,659,660],{},"    return response, digest\n",[68,662,664],{"className":70,"code":663,"language":72,"meta":73,"style":73},"import asyncio\n\n\nasync def cancellable_version(rows, client):\n    # 1. A real async client yields at every network operation.\n    response = await client.get(\"https:\u002F\u002Fslow.example\")\n\n    # 2. CPU work moved to a thread; the await is a suspension point, and the\n    #    thread finishes on its own after cancellation rather than being killed.\n    digest = await asyncio.to_thread(lambda: sum(hash(row) for row in rows))\n\n    # 3. An async sleep is a suspension point by construction.\n    await asyncio.sleep(5)\n    return response, digest\n",[14,665,666,670,674,678,683,688,693,697,702,707,712,716,721,726],{"__ignoreMap":73},[77,667,668],{"class":79,"line":80},[77,669,83],{},[77,671,672],{"class":79,"line":86},[77,673,90],{"emptyLinePlaceholder":89},[77,675,676],{"class":79,"line":93},[77,677,90],{"emptyLinePlaceholder":89},[77,679,680],{"class":79,"line":99},[77,681,682],{},"async def cancellable_version(rows, client):\n",[77,684,685],{"class":79,"line":104},[77,686,687],{},"    # 1. A real async client yields at every network operation.\n",[77,689,690],{"class":79,"line":109},[77,691,692],{},"    response = await client.get(\"https:\u002F\u002Fslow.example\")\n",[77,694,695],{"class":79,"line":115},[77,696,90],{"emptyLinePlaceholder":89},[77,698,699],{"class":79,"line":121},[77,700,701],{},"    # 2. CPU work moved to a thread; the await is a suspension point, and the\n",[77,703,704],{"class":79,"line":127},[77,705,706],{},"    #    thread finishes on its own after cancellation rather than being killed.\n",[77,708,709],{"class":79,"line":133},[77,710,711],{},"    digest = await asyncio.to_thread(lambda: sum(hash(row) for row in rows))\n",[77,713,714],{"class":79,"line":138},[77,715,90],{"emptyLinePlaceholder":89},[77,717,718],{"class":79,"line":144},[77,719,720],{},"    # 3. An async sleep is a suspension point by construction.\n",[77,722,723],{"class":79,"line":150},[77,724,725],{},"    await asyncio.sleep(5)\n",[77,727,728],{"class":79,"line":156},[77,729,660],{},[10,731,732,733,735,736,740],{},"The second case deserves a caveat: ",[14,734,514],{}," makes the ",[737,738,739],"em",{},"caller"," cancellable, not the work. The thread runs to completion whatever happens, because Python cannot interrupt a thread. For a long CPU loop the honest fix is to chunk it and check a flag between chunks, or to move it to a process pool where it can be terminated.",[288,742,744,796],{"className":743},[291],[293,745,301,750,301,753,301,756,301,759,301,763,301,769,301,774,301,778,301,781,301,785,301,788,301,792],{"viewBox":746,"role":296,"ariaLabelledBy":747,"xmlns":300},"0 0 800 240",[748,749],"reach-t","reach-d",[303,751,752],{"id":748},"Cancellability across a coroutine's body",[307,754,755],{"id":749},"A coroutine body drawn as a bar. Regions containing awaits are cancellable; a blocking HTTP call, a long CPU loop and a blocking sleep are drawn as uncancellable stretches during which a cancellation request is queued but cannot be delivered. A note records that the request waits until the next suspension point.",[329,757],{"x":331,"y":331,"width":758,"height":456,"rx":334,"fill":335},"800",[337,760,762],{"x":388,"y":340,"textAnchor":341,"fontSize":761,"fontWeight":343,"fill":327},"15.5","A cancellation request waits for the next await",[329,764],{"x":765,"y":766,"width":767,"height":768,"rx":351,"fill":352,"stroke":353,"strokeWidth":411},"34","62","732","52",[337,770,773],{"x":427,"y":771,"fontSize":772,"fontWeight":343,"fill":327},"84","12","cancellable",[337,775,777],{"x":427,"y":776,"fontSize":364,"fill":436},"104","await client.get(…) · await asyncio.sleep(…) · await queue.get()",[329,779],{"x":765,"y":780,"width":767,"height":768,"rx":351,"fill":410,"stroke":405,"strokeWidth":411},"126",[337,782,784],{"x":427,"y":783,"fontSize":772,"fontWeight":343,"fill":327},"148","uncancellable",[337,786,787],{"x":427,"y":349,"fontSize":364,"fill":417},"requests.get(…) · a long CPU loop · time.sleep(…) · a C call holding the GIL",[329,789],{"x":765,"y":790,"width":767,"height":791,"rx":319,"fill":335,"stroke":441,"strokeWidth":371},"190","38",[337,793,795],{"x":388,"y":794,"textAnchor":341,"fontSize":359,"fill":327},"214","cancel() during the lower band is queued, not lost — it fires at the next await, however long that takes.",[459,797,798],{},"A test whose cancellation appears to be ignored is usually landing in the lower band. The fix is in the code's structure, not in the test.",[10,800,801],{},"A simple test catches the whole category: cancel the task and assert it finishes within a short bound.",[68,803,805],{"className":70,"code":804,"language":72,"meta":73,"style":73},"import asyncio\n\nimport pytest\n\n\nasync def test_cancellation_is_prompt(worker):\n    task = asyncio.create_task(worker.run())\n    await worker.started.wait()\n\n    task.cancel()\n    # If the body has an uncancellable stretch, this wait_for times out and\n    # the test names the problem precisely.\n    with pytest.raises(asyncio.CancelledError):\n        await asyncio.wait_for(task, timeout=0.5)\n",[14,806,807,811,815,819,823,827,832,836,841,845,849,854,859,863],{"__ignoreMap":73},[77,808,809],{"class":79,"line":80},[77,810,83],{},[77,812,813],{"class":79,"line":86},[77,814,90],{"emptyLinePlaceholder":89},[77,816,817],{"class":79,"line":93},[77,818,96],{},[77,820,821],{"class":79,"line":99},[77,822,90],{"emptyLinePlaceholder":89},[77,824,825],{"class":79,"line":104},[77,826,90],{"emptyLinePlaceholder":89},[77,828,829],{"class":79,"line":109},[77,830,831],{},"async def test_cancellation_is_prompt(worker):\n",[77,833,834],{"class":79,"line":115},[77,835,223],{},[77,837,838],{"class":79,"line":121},[77,839,840],{},"    await worker.started.wait()\n",[77,842,843],{"class":79,"line":127},[77,844,90],{"emptyLinePlaceholder":89},[77,846,847],{"class":79,"line":133},[77,848,251],{},[77,850,851],{"class":79,"line":138},[77,852,853],{},"    # If the body has an uncancellable stretch, this wait_for times out and\n",[77,855,856],{"class":79,"line":144},[77,857,858],{},"    # the test names the problem precisely.\n",[77,860,861],{"class":79,"line":150},[77,862,257],{},[77,864,865],{"class":79,"line":156},[77,866,867],{},"        await asyncio.wait_for(task, timeout=0.5)\n",[10,869,870],{},"Half a second is a deliberate choice: long enough that a healthy unwinding always completes, short enough that a blocking call of any realistic length fails it. Adding this one test per long-running coroutine costs very little and converts \"shutdown sometimes takes thirty seconds\" from an operational mystery into a named test failure.",[10,872,873],{},"The same bound is worth applying to the application's own shutdown routine, which is usually a task group or a gather over every background worker. A shutdown test that cancels the group and asserts it completes within a second exercises every worker's cancellation path at once, and it fails the moment any of them grows a blocking call — which is far earlier and far cheaper than discovering it during a deploy.",[19,875,877],{"id":876},"shielding-the-part-that-must-complete","Shielding the part that must complete",[10,879,880],{},"Some cleanup genuinely must finish: flushing a buffer, releasing a distributed lock, writing an audit record. Shielding it protects it from the in-flight cancellation, and bounding the shield protects the shield from becoming a new hang.",[68,882,884],{"className":70,"code":883,"language":72,"meta":73,"style":73},"import asyncio\n\n\nasync def stream_and_flush(buffer, connection):\n    try:\n        await stream_forever(buffer, connection)\n    finally:\n        # Shielded so the flush completes even though we are being cancelled,\n        # and bounded so a stuck flush cannot outlive the cancellation forever.\n        async with asyncio.timeout(2.0):\n            await asyncio.shield(buffer.flush(connection))\n",[14,885,886,890,894,898,903,908,913,918,923,928,933],{"__ignoreMap":73},[77,887,888],{"class":79,"line":80},[77,889,83],{},[77,891,892],{"class":79,"line":86},[77,893,90],{"emptyLinePlaceholder":89},[77,895,896],{"class":79,"line":93},[77,897,90],{"emptyLinePlaceholder":89},[77,899,900],{"class":79,"line":99},[77,901,902],{},"async def stream_and_flush(buffer, connection):\n",[77,904,905],{"class":79,"line":104},[77,906,907],{},"    try:\n",[77,909,910],{"class":79,"line":109},[77,911,912],{},"        await stream_forever(buffer, connection)\n",[77,914,915],{"class":79,"line":115},[77,916,917],{},"    finally:\n",[77,919,920],{"class":79,"line":121},[77,921,922],{},"        # Shielded so the flush completes even though we are being cancelled,\n",[77,924,925],{"class":79,"line":127},[77,926,927],{},"        # and bounded so a stuck flush cannot outlive the cancellation forever.\n",[77,929,930],{"class":79,"line":133},[77,931,932],{},"        async with asyncio.timeout(2.0):\n",[77,934,935],{"class":79,"line":138},[77,936,937],{},"            await asyncio.shield(buffer.flush(connection))\n",[68,939,941],{"className":70,"code":940,"language":72,"meta":73,"style":73},"import asyncio\n\nimport pytest\n\n\nasync def test_flush_completes_despite_cancellation(buffer, connection):\n    task = asyncio.create_task(stream_and_flush(buffer, connection))\n    await buffer.first_write.wait()\n\n    task.cancel()\n    with pytest.raises(asyncio.CancelledError):\n        await task\n\n    assert buffer.flushed_bytes > 0, \"the shielded flush did not run\"\n    assert connection.closed, \"the connection was not closed after the flush\"\n\n\nasync def test_stuck_flush_does_not_outlive_the_cancellation(buffer, stuck_connection):\n    task = asyncio.create_task(stream_and_flush(buffer, stuck_connection))\n    await buffer.first_write.wait()\n\n    task.cancel()\n    with pytest.raises((asyncio.CancelledError, TimeoutError)):\n        await asyncio.wait_for(task, timeout=5)   # must not hang here\n",[14,942,943,947,951,955,959,963,968,973,978,982,986,990,995,999,1004,1009,1013,1017,1022,1027,1031,1035,1039,1044],{"__ignoreMap":73},[77,944,945],{"class":79,"line":80},[77,946,83],{},[77,948,949],{"class":79,"line":86},[77,950,90],{"emptyLinePlaceholder":89},[77,952,953],{"class":79,"line":93},[77,954,96],{},[77,956,957],{"class":79,"line":99},[77,958,90],{"emptyLinePlaceholder":89},[77,960,961],{"class":79,"line":104},[77,962,90],{"emptyLinePlaceholder":89},[77,964,965],{"class":79,"line":109},[77,966,967],{},"async def test_flush_completes_despite_cancellation(buffer, connection):\n",[77,969,970],{"class":79,"line":115},[77,971,972],{},"    task = asyncio.create_task(stream_and_flush(buffer, connection))\n",[77,974,975],{"class":79,"line":121},[77,976,977],{},"    await buffer.first_write.wait()\n",[77,979,980],{"class":79,"line":127},[77,981,90],{"emptyLinePlaceholder":89},[77,983,984],{"class":79,"line":133},[77,985,251],{},[77,987,988],{"class":79,"line":138},[77,989,257],{},[77,991,992],{"class":79,"line":144},[77,993,994],{},"        await task\n",[77,996,997],{"class":79,"line":150},[77,998,90],{"emptyLinePlaceholder":89},[77,1000,1001],{"class":79,"line":156},[77,1002,1003],{},"    assert buffer.flushed_bytes > 0, \"the shielded flush did not run\"\n",[77,1005,1006],{"class":79,"line":162},[77,1007,1008],{},"    assert connection.closed, \"the connection was not closed after the flush\"\n",[77,1010,1011],{"class":79,"line":168},[77,1012,90],{"emptyLinePlaceholder":89},[77,1014,1015],{"class":79,"line":174},[77,1016,90],{"emptyLinePlaceholder":89},[77,1018,1019],{"class":79,"line":180},[77,1020,1021],{},"async def test_stuck_flush_does_not_outlive_the_cancellation(buffer, stuck_connection):\n",[77,1023,1024],{"class":79,"line":186},[77,1025,1026],{},"    task = asyncio.create_task(stream_and_flush(buffer, stuck_connection))\n",[77,1028,1029],{"class":79,"line":192},[77,1030,977],{},[77,1032,1033],{"class":79,"line":198},[77,1034,90],{"emptyLinePlaceholder":89},[77,1036,1037],{"class":79,"line":203},[77,1038,251],{},[77,1040,1041],{"class":79,"line":208},[77,1042,1043],{},"    with pytest.raises((asyncio.CancelledError, TimeoutError)):\n",[77,1045,1046],{"class":79,"line":214},[77,1047,1048],{},"        await asyncio.wait_for(task, timeout=5)   # must not hang here\n",[10,1050,1051,1052,1054],{},"The second test is the one that justifies the bound. Without the ",[14,1053,32],{}," around the shield, a flush against an unresponsive connection would keep the task alive indefinitely, and the only symptom would be a shutdown that never completes — a failure that is extremely hard to attribute in production and trivial to catch here.",[19,1056,1058],{"id":1057},"cancellation-in-groups-and-gathers","Cancellation in groups and gathers",[10,1060,1061],{},"The unit of cancellation differs between the two composition primitives, and tests need to match.",[10,1063,1064,1065,1068,1069,1072,1073,1076,1077,1079],{},"With ",[14,1066,1067],{},"asyncio.gather(..., return_exceptions=False)",", cancelling the gathering ",[737,1070,1071],{},"task"," propagates cancellation to the children, but a child raising does not cancel its siblings — they keep running unsupervised. A test that asserts \"when one fails the others stop\" will fail against ",[14,1074,1075],{},"gather",", correctly, because ",[14,1078,1075],{}," makes no such promise.",[10,1081,1082,1083,1086,1087,41],{},"With a task group, the first failure cancels every sibling and the block raises an ",[14,1084,1085],{},"ExceptionGroup",". The assertion shape changes accordingly, and the details are in ",[49,1088,1090],{"href":1089},"\u002Ftesting-async-and-concurrent-python\u002Ftesting-with-anyio-and-trio\u002Ftesting-code-that-uses-task-groups\u002F","testing code that uses task groups",[288,1092,1094,1145],{"className":1093},[291],[293,1095,301,1100,301,1103,301,1106,301,1109,301,1112,301,1117,301,1122,301,1126,301,1130,301,1133,301,1137,301,1141],{"viewBox":1096,"role":296,"ariaLabelledBy":1097,"xmlns":300},"0 0 800 234",[1098,1099],"grp-t","grp-d",[303,1101,1102],{"id":1098},"Cancellation reach under gather and under a task group",[307,1104,1105],{"id":1099},"Two rows. Under gather, cancelling the outer task cancels the children, but a child failing on its own leaves the siblings running. Under a task group, a child failing cancels every sibling and the block waits for all of them before raising an exception group.",[329,1107],{"x":331,"y":331,"width":758,"height":1108,"rx":334,"fill":335},"234",[337,1110,1111],{"x":388,"y":340,"textAnchor":341,"fontSize":761,"fontWeight":343,"fill":327},"Who gets cancelled when",[329,1113],{"x":347,"y":1114,"width":1115,"height":1116,"rx":364,"fill":376,"stroke":377,"strokeWidth":411},"50","748","76",[337,1118,1121],{"x":1119,"y":1120,"fontSize":772,"fontWeight":343,"fill":327},"46","74","asyncio.gather",[337,1123,1125],{"x":1119,"y":1124,"fontSize":364,"fill":327},"96","cancel the outer task → children cancelled · child raises → siblings keep running",[337,1127,1129],{"x":1119,"y":1128,"fontSize":364,"fill":384},"116","assert siblings still running; do not assert they stopped",[329,1131],{"x":347,"y":1132,"width":1115,"height":1116,"rx":364,"fill":352,"stroke":353,"strokeWidth":411},"140",[337,1134,1136],{"x":1119,"y":1135,"fontSize":772,"fontWeight":343,"fill":327},"164","TaskGroup \u002F create_task_group",[337,1138,1140],{"x":1119,"y":1139,"fontSize":364,"fill":327},"186","child raises → every sibling cancelled → block waits → ExceptionGroup raised",[337,1142,1144],{"x":1119,"y":1143,"fontSize":364,"fill":436},"206","assert the siblings' cleanup ran, and count the group's leaves",[459,1146,1147,1148,1150],{},"Migrating a test from ",[14,1149,1075],{}," to a task group without changing its assertions usually leaves it asserting something that is no longer the contract.",[19,1152,1154],{"id":1153},"frequently-asked-questions","Frequently Asked Questions",[10,1156,1157,1160,1161,1163,1164,1166],{},[504,1158,1159],{},"Why does my coroutine ignore task.cancel()?","\nEither it never reaches an ",[14,1162,58],{},", because it is blocked in synchronous code, or it catches the ",[14,1165,36],{}," and does not re-raise. Cancellation is delivered as an exception at the next suspension point, so a CPU loop or a blocking socket read is immune to it until it yields.",[10,1168,1169,1172,1173,1176],{},[504,1170,1171],{},"Should cleanup be shielded?","\nOnly the part that must complete, and only with its own deadline. ",[14,1174,1175],{},"asyncio.shield"," around a short release or rollback is legitimate; shielding a whole cleanup coroutine means a timeout can no longer stop it, which reintroduces the hang the timeout existed to prevent.",[10,1178,1179,1182,1183,1185,1186,1188],{},[504,1180,1181],{},"Is catching CancelledError ever correct?","\nYes, to run cleanup — and then it must be re-raised. Swallowing it converts a cancellation into a silent delay and breaks every caller that expects the task to stop. From Python 3.8 it inherits from ",[14,1184,40],{}," specifically so a bare ",[14,1187,521],{}," does not catch it by accident.",[19,1190,1192],{"id":1191},"related","Related",[24,1194,1195,1202,1209,1215],{},[27,1196,1197,1201],{},[49,1198,1200],{"href":1199},"\u002Ftesting-async-and-concurrent-python\u002Ftimeouts-cancellation-and-deadlines\u002F","Timeouts, Cancellation & Deadlines"," — the layered deadlines that trigger cancellation in production.",[27,1203,1204,1208],{},[49,1205,1207],{"href":1206},"\u002Ftesting-async-and-concurrent-python\u002Fpytest-asyncio-in-depth\u002Ftesting-async-generators-and-context-managers\u002F","Testing Async Generators and Context Managers"," — the same cleanup questions for early exit rather than cancellation.",[27,1210,1211,1214],{},[49,1212,1213],{"href":1089},"Testing Code That Uses Task Groups"," — asserting sibling cancellation.",[27,1216,1217,1221],{},[49,1218,1220],{"href":1219},"\u002Fsystematic-debugging-performance-profiling\u002Fdebugging-async-code-and-event-loops\u002Fdiagnosing-task-was-destroyed-warnings\u002F","Diagnosing \"Task was destroyed\" Warnings"," — what an uncancelled, unreferenced task looks like at shutdown.",[10,1223,1224,1225],{},"← Back to ",[49,1226,1200],{"href":1199},[1228,1229,1230],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":73,"searchDepth":86,"depth":86,"links":1232},[1233,1234,1235,1236,1237,1238,1239,1240,1241],{"id":21,"depth":86,"text":22},{"id":62,"depth":86,"text":63},{"id":468,"depth":86,"text":469},{"id":497,"depth":86,"text":498},{"id":574,"depth":86,"text":575},{"id":876,"depth":86,"text":877},{"id":1057,"depth":86,"text":1058},{"id":1153,"depth":86,"text":1154},{"id":1191,"depth":86,"text":1192},"Assert that cancelled coroutines clean up: delivering CancelledError, shielding the essential part, re-raising after cleanup, and checking released locks and connections.","md",{"slug":1245,"type":1246,"breadcrumb":1247,"datePublished":1248,"dateModified":1248,"faq":1249,"howto":1256},"testing-cancellation-and-cleanup-paths","article","Cancellation","2026-09-18",[1250,1252,1254],{"q":1159,"a":1251},"Either it never reaches an await, because it is blocked in synchronous code, or it catches the CancelledError and does not re-raise. Cancellation is delivered as an exception at the next suspension point, so a CPU loop or a blocking socket read is immune to it until it yields.",{"q":1171,"a":1253},"Only the part that must complete, and only with its own deadline. asyncio.shield around a short release or rollback is legitimate; shielding a whole cleanup coroutine means a timeout can no longer stop it, which reintroduces the hang the timeout existed to prevent.",{"q":1181,"a":1255},"Yes, to run cleanup — and then it must be re-raised. Swallowing it converts a cancellation into a silent delay and breaks every caller that expects the task to stop. From Python 3.8 it inherits from BaseException specifically so a bare except Exception does not catch it by accident.",{"name":1257,"description":1258,"steps":1259},"How to test that cancellation cleans up correctly","Start the work as a task, cancel it at a known point, and assert on the state its cleanup was supposed to restore.",[1260,1263,1266,1269,1272],{"name":1261,"text":1262},"Start the work as a task","Create a task so the test holds a handle it can cancel, rather than awaiting the coroutine directly.",{"name":1264,"text":1265},"Wait for a known point","Use an event the code sets, not a sleep, so the cancellation lands at a predictable place.",{"name":1267,"text":1268},"Cancel and await the task","Call cancel and then await inside pytest.raises(asyncio.CancelledError) so the unwinding completes before assertions run.",{"name":1270,"text":1271},"Assert on restored state","Check the released lock, the returned connection and the rolled-back transaction rather than that cleanup was called.",{"name":1273,"text":1274},"Cover the shielded region","Add a test that the essential cleanup completes even when the cancellation arrives during it.","\u002Ftesting-async-and-concurrent-python\u002Ftimeouts-cancellation-and-deadlines\u002Ftesting-cancellation-and-cleanup-paths",{"title":5,"description":1242},"testing-async-and-concurrent-python\u002Ftimeouts-cancellation-and-deadlines\u002Ftesting-cancellation-and-cleanup-paths\u002Findex","XFb_TCYULlJNE74mSyAxd4KAuYjsaxhIqPhye56BLnM",1789718767408]