[{"data":1,"prerenderedAt":1772},["ShallowReactive",2],{"page-\u002Ftesting-async-and-concurrent-python\u002Ftesting-threads-and-race-conditions\u002F":3},{"id":4,"title":5,"body":6,"description":1731,"extension":1732,"meta":1733,"navigation":324,"path":1768,"seo":1769,"stem":1770,"__hash__":1771},"content\u002Ftesting-async-and-concurrent-python\u002Ftesting-threads-and-race-conditions\u002Findex.md","Testing Threads & Race Conditions",{"type":7,"value":8,"toc":1713},"minimark",[9,13,18,65,69,72,84,284,288,303,359,366,380,391,526,530,535,732,736,788,792,799,941,952,956,970,1022,1032,1036,1039,1067,1073,1076,1080,1203,1207,1210,1217,1246,1252,1255,1366,1376,1380,1383,1547,1558,1570,1582,1586,1589,1596,1617,1621,1628,1642,1652,1658,1664,1668,1703,1709],[10,11,12],"p",{},"Concurrency bugs have a characteristic lifecycle: they are found in production, cannot be reproduced locally, get a defensive lock added on the suspicion that it might help, and reappear eighteen months later. The step that breaks the cycle is a test that fails every single run while the bug is present. That is achievable far more often than teams assume, because most Python races have a window an interleaving can be forced into rather than waited for.",[14,15,17],"h2",{"id":16},"prerequisites","Prerequisites",[19,20,21,34,50,57],"ul",{},[22,23,24,25,29,30,33],"li",{},"Python 3.9+; ",[26,27,28],"code",{},"threading.Barrier"," and ",[26,31,32],{},"concurrent.futures"," are standard library, and nothing here needs a third-party plugin.",[22,35,36,39,40,43,44,49],{},[26,37,38],{},"pytest >= 8.0",", plus ",[26,41,42],{},"pytest-timeout"," so a deadlocked test fails rather than hanging — see ",[45,46,48],"a",{"href":47},"\u002Ftesting-async-and-concurrent-python\u002Ftimeouts-cancellation-and-deadlines\u002Ffailing-fast-with-pytest-timeout\u002F","failing fast with pytest-timeout",".",[22,51,52,53,56],{},"Familiarity with the difference between a lock, an event and a barrier, and with why ",[26,54,55],{},"time.sleep"," is not a synchronisation primitive.",[22,58,59,60,64],{},"For async code, the cooperative equivalent is in ",[45,61,63],{"href":62},"\u002Ftesting-async-and-concurrent-python\u002F","testing async and concurrent Python"," — the techniques differ because the scheduler differs.",[14,66,68],{"id":67},"core-concept-races-have-windows-and-windows-can-be-held-open","Core concept: races have windows, and windows can be held open",[10,70,71],{},"A race condition is a window between two operations that the code assumes are atomic. The bug is not that threads run concurrently; it is that the invariant is false for a few nanoseconds and something observed it during that time. Testing for it means holding the window open deliberately.",[10,73,74,75,79,80,83],{},"There are two ways to hold it open, and they suit different code. The first is a ",[76,77,78],"em",{},"barrier",": every thread blocks until all of them have arrived, so they enter the contended region simultaneously rather than by luck. The second is a ",[76,81,82],{},"seam",": a hook inside the window that the test can block on, which lets one thread be frozen mid-sequence while another runs to completion. Barriers need no production change and catch races between identical operations; seams need a small injectable callback and catch races between different operations.",[85,86,89,280],"figure",{"className":87},[88],"diagram",[90,91,98,99,98,103,98,107,98,125,98,133,98,142,98,148,98,152,98,156,98,164,98,167,98,170,98,179,98,185,98,189,98,194,98,198,98,202,98,209,98,216,98,221,98,225,98,230,98,233,98,237,98,243,98,247,98,249,98,251,98,253,98,255,98,260,98,265,98,270,98,274],"svg",{"viewBox":92,"role":93,"ariaLabelledBy":94,"xmlns":97},"0 0 840 290","img",[95,96],"window-t","window-d","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","\n  ",[100,101,102],"title",{"id":95},"Holding a race window open with a barrier",[104,105,106],"desc",{"id":96},"Three threads arrive at a barrier at different times and all block. When the third arrives the barrier releases and all three enter the read-modify-write region together, so the lost-update window is exercised on every run rather than occasionally.",[108,109,110,111,98],"defs",{},"\n    ",[112,113,120],"marker",{"id":114,"viewBox":115,"refX":116,"refY":117,"markerWidth":118,"markerHeight":118,"orient":119},"window-a","0 0 10 10","9","5","7","auto-start-reverse",[121,122],"path",{"d":123,"fill":124},"M0 0 L10 5 L0 10 z","#3d405b",[126,127],"rect",{"x":128,"y":128,"width":129,"height":130,"rx":131,"fill":132},"0","840","290","14","#fffdf8",[134,135,141],"text",{"x":136,"y":137,"textAnchor":138,"fontSize":139,"fontWeight":140,"fill":124},"420","28","middle","16","700","Arrival is random; release is simultaneous",[134,143,147],{"x":144,"y":145,"fontSize":146,"fontWeight":140,"fill":124},"46","80","12","thread 1",[134,149,151],{"x":144,"y":150,"fontSize":146,"fontWeight":140,"fill":124},"140","thread 2",[134,153,155],{"x":144,"y":154,"fontSize":146,"fontWeight":140,"fill":124},"200","thread 3",[157,158],"line",{"x1":159,"y1":160,"x2":161,"y2":160,"stroke":162,"strokeWidth":163},"116","76","790","rgba(61,64,91,0.35)","1.3",[157,165],{"x1":159,"y1":166,"x2":161,"y2":166,"stroke":162,"strokeWidth":163},"136",[157,168],{"x1":159,"y1":169,"x2":161,"y2":169,"stroke":162,"strokeWidth":163},"196",[126,171],{"x":172,"y":173,"width":174,"height":175,"rx":118,"fill":176,"stroke":177,"strokeWidth":178},"126","60","118","32","#e6f0ea","#81b29a","1.6",[134,180,184],{"x":181,"y":182,"textAnchor":138,"fontSize":183,"fill":124},"185","81","10.5","setup",[126,186],{"x":172,"y":187,"width":188,"height":175,"rx":118,"fill":176,"stroke":177,"strokeWidth":178},"120","182",[134,190,193],{"x":191,"y":192,"textAnchor":138,"fontSize":183,"fill":124},"217","141","setup (slower)",[126,195],{"x":172,"y":196,"width":197,"height":175,"rx":118,"fill":176,"stroke":177,"strokeWidth":178},"180","146",[134,199,184],{"x":200,"y":201,"textAnchor":138,"fontSize":183,"fill":124},"199","201",[126,203],{"x":204,"y":205,"width":131,"height":206,"rx":207,"fill":208},"330","48","176","4","#f2cc8f",[134,210,215],{"x":211,"y":212,"textAnchor":138,"fontSize":213,"fontWeight":140,"fill":214},"337","240","11","#8a5a00","barrier.wait()",[126,217],{"x":218,"y":173,"width":219,"height":175,"rx":118,"fill":220,"stroke":208,"strokeWidth":178},"252","74","#f7f0da",[134,222,224],{"x":223,"y":182,"textAnchor":138,"fontSize":183,"fill":124},"289","blocked",[126,226],{"x":227,"y":187,"width":228,"height":175,"rx":229,"fill":220,"stroke":208,"strokeWidth":178},"316","10","3",[126,231],{"x":232,"y":196,"width":144,"height":175,"rx":118,"fill":220,"stroke":208,"strokeWidth":178},"280",[134,234,236],{"x":235,"y":201,"textAnchor":138,"fontSize":183,"fill":124},"303","wait",[126,238],{"x":239,"y":173,"width":154,"height":175,"rx":118,"fill":240,"stroke":241,"strokeWidth":242},"356","#fbe9e3","#e07a5f","1.8",[134,244,246],{"x":245,"y":182,"textAnchor":138,"fontSize":183,"fill":124},"456","read → add → write",[126,248],{"x":239,"y":187,"width":154,"height":175,"rx":118,"fill":240,"stroke":241,"strokeWidth":242},[134,250,246],{"x":245,"y":192,"textAnchor":138,"fontSize":183,"fill":124},[126,252],{"x":239,"y":196,"width":154,"height":175,"rx":118,"fill":240,"stroke":241,"strokeWidth":242},[134,254,246],{"x":245,"y":201,"textAnchor":138,"fontSize":183,"fill":124},[126,256],{"x":257,"y":258,"width":259,"height":160,"rx":228,"fill":132,"stroke":124,"strokeWidth":178},"580","108","210",[134,261,264],{"x":262,"y":263,"textAnchor":138,"fontSize":146,"fontWeight":140,"fill":124},"685","134","assert counter == 3",[134,266,269],{"x":262,"y":267,"textAnchor":138,"fontSize":213,"fill":268},"156","#8f3d22","fails deterministically",[134,271,273],{"x":262,"y":272,"textAnchor":138,"fontSize":213,"fill":124},"174","when the lock is missing",[157,275],{"x1":276,"y1":166,"x2":277,"y2":166,"stroke":124,"strokeWidth":278,"markerEnd":279},"560","576","1.5","url(#window-a)",[281,282,283],"figcaption",{},"The barrier converts \"all three threads happened to be in the critical section at once\" from an accident into a precondition of the test.",[14,285,287],{"id":286},"the-gil-does-not-save-you","The GIL does not save you",[10,289,290,291,294,295,298,299,302],{},"The most persistent misconception about Python concurrency is that the global interpreter lock makes shared state safe. It makes ",[76,292,293],{},"bytecode"," atomic, which is a much weaker guarantee than it sounds. ",[26,296,297],{},"counter += 1"," compiles to a load, a binary add and a store, and since Python 3.10 the interpreter may switch threads between any two bytecodes at a granularity controlled by ",[26,300,301],{},"sys.setswitchinterval",". Three bytecodes is three opportunities for another thread to interleave.",[304,305,310],"pre",{"className":306,"code":307,"language":308,"meta":309,"style":309},"language-python shiki shiki-themes github-light github-dark","import dis\n\n\ndef increment(counter):\n    counter.value += 1\n\n\ndis.dis(increment)\n","python","",[26,311,312,319,326,331,337,343,348,353],{"__ignoreMap":309},[313,314,316],"span",{"class":157,"line":315},1,[313,317,318],{},"import dis\n",[313,320,322],{"class":157,"line":321},2,[313,323,325],{"emptyLinePlaceholder":324},true,"\n",[313,327,329],{"class":157,"line":328},3,[313,330,325],{"emptyLinePlaceholder":324},[313,332,334],{"class":157,"line":333},4,[313,335,336],{},"def increment(counter):\n",[313,338,340],{"class":157,"line":339},5,[313,341,342],{},"    counter.value += 1\n",[313,344,346],{"class":157,"line":345},6,[313,347,325],{"emptyLinePlaceholder":324},[313,349,351],{"class":157,"line":350},7,[313,352,325],{"emptyLinePlaceholder":324},[313,354,356],{"class":157,"line":355},8,[313,357,358],{},"dis.dis(increment)\n",[304,360,364],{"className":361,"code":363,"language":134,"meta":309},[362],"language-text","  2           0 LOAD_FAST                0 (counter)\n              2 DUP_TOP\n              4 LOAD_ATTR                0 (value)     ← read\n              6 LOAD_CONST               1 (1)\n              8 INPLACE_ADD                             ← modify\n             10 ROT_TWO\n             12 STORE_ATTR               0 (value)     ← write\n",[26,365,363],{"__ignoreMap":309},[10,367,368,369,372,373,372,376,379],{},"The read at offset 4 and the write at offset 12 are separated by four bytecodes. Every check-then-act pattern has the same shape: ",[26,370,371],{},"if key not in cache: cache[key] = compute(key)",", ",[26,374,375],{},"if self._conn is None: self._conn = connect()",[26,377,378],{},"if os.path.exists(p): os.remove(p)",". Under free-threaded builds (PEP 703, available as an official variant from 3.13) the window widens further because there is no interpreter lock serialising the bytecodes at all — code that was accidentally safe becomes reliably unsafe, which makes these tests worth writing now rather than at the migration.",[10,381,382,383,386,387,390],{},"Two practical consequences follow. First, ",[26,384,385],{},"sys.setswitchinterval(0.000001)"," in a ",[26,388,389],{},"conftest.py"," makes the interpreter switch threads far more aggressively during the test session, which widens every window in the code under test at no cost to correctness. It is a blunt instrument and it slows the suite slightly, but for a module full of concurrency tests it turns several probabilistic failures into reliable ones. Second, any invariant that spans more than one attribute access needs a lock regardless of how short the code looks — brevity is not atomicity.",[85,392,394,519],{"className":393},[88],[90,395,98,400,98,403,98,406,98,410,98,414,98,421,98,426,98,430,98,434,98,438,98,441,98,445,98,449,98,452,98,456,98,459,98,462,98,466,98,469,98,473,98,477,98,480,98,483,98,487,98,490,98,494,98,497,98,500,98,503,98,506,98,509,98,515],{"viewBox":396,"role":93,"ariaLabelledBy":397,"xmlns":97},"0 0 800 232",[398,399],"gil-t","gil-d",[100,401,402],{"id":398},"Where a thread switch can land inside one statement",[104,404,405],{"id":399},"The statement counter.value plus equals one expands into six bytecodes. Arrows mark the four points between bytecodes at which the interpreter may release the lock and switch threads, with the read at offset four and the store at offset twelve separated by the whole window.",[126,407],{"x":128,"y":128,"width":408,"height":409,"rx":131,"fill":132},"800","232",[134,411,413],{"x":412,"y":137,"textAnchor":138,"fontSize":139,"fontWeight":140,"fill":124},"400","One statement, six bytecodes, four switch points",[126,415],{"x":416,"y":417,"width":418,"height":419,"rx":116,"fill":420,"stroke":124,"strokeWidth":278},"34","52","732","40","#f4f1de",[134,422,425],{"x":412,"y":423,"textAnchor":138,"fontSize":424,"fontWeight":140,"fill":124},"78","12.5","counter.value += 1",[126,427],{"x":416,"y":428,"width":159,"height":417,"rx":429,"fill":132,"stroke":162,"strokeWidth":278},"112","8",[134,431,433],{"x":432,"y":263,"textAnchor":138,"fontSize":183,"fill":124},"92","LOAD_FAST",[134,435,437],{"x":432,"y":436,"textAnchor":138,"fontSize":228,"fill":124},"152","offset 0",[126,439],{"x":440,"y":428,"width":159,"height":417,"rx":429,"fill":176,"stroke":177,"strokeWidth":242},"158",[134,442,444],{"x":443,"y":263,"textAnchor":138,"fontSize":183,"fill":124},"216","LOAD_ATTR",[134,446,448],{"x":443,"y":436,"textAnchor":138,"fontSize":228,"fill":447},"#2a5f49","read · offset 4",[126,450],{"x":451,"y":428,"width":159,"height":417,"rx":429,"fill":132,"stroke":162,"strokeWidth":278},"282",[134,453,455],{"x":454,"y":263,"textAnchor":138,"fontSize":183,"fill":124},"340","LOAD_CONST",[134,457,458],{"x":454,"y":436,"textAnchor":138,"fontSize":228,"fill":124},"offset 6",[126,460],{"x":461,"y":428,"width":159,"height":417,"rx":429,"fill":132,"stroke":162,"strokeWidth":278},"406",[134,463,465],{"x":464,"y":263,"textAnchor":138,"fontSize":183,"fill":124},"464","INPLACE_ADD",[134,467,468],{"x":464,"y":436,"textAnchor":138,"fontSize":228,"fill":124},"offset 8",[126,470],{"x":471,"y":428,"width":472,"height":417,"rx":429,"fill":132,"stroke":162,"strokeWidth":278},"530","110",[134,474,476],{"x":475,"y":263,"textAnchor":138,"fontSize":183,"fill":124},"585","ROT_TWO",[134,478,479],{"x":475,"y":436,"textAnchor":138,"fontSize":228,"fill":124},"offset 10",[126,481],{"x":482,"y":428,"width":174,"height":417,"rx":429,"fill":240,"stroke":241,"strokeWidth":242},"648",[134,484,486],{"x":485,"y":263,"textAnchor":138,"fontSize":183,"fill":124},"707","STORE_ATTR",[134,488,489],{"x":485,"y":436,"textAnchor":138,"fontSize":228,"fill":268},"write · offset 12",[491,492],"polygon",{"points":493,"fill":208},"154,104 148,94 160,94",[491,495],{"points":496,"fill":208},"278,104 272,94 284,94",[491,498],{"points":499,"fill":208},"402,104 396,94 408,94",[491,501],{"points":502,"fill":208},"526,104 520,94 532,94",[491,504],{"points":505,"fill":208},"644,104 638,94 650,94",[157,507],{"x1":443,"y1":206,"x2":485,"y2":206,"stroke":241,"strokeWidth":178,"strokeDashArray":508},[117,207],[134,510,514],{"x":511,"y":512,"textAnchor":138,"fontSize":513,"fontWeight":140,"fill":268},"461","198","11.5","the window another thread can enter",[134,516,518],{"x":412,"y":517,"textAnchor":138,"fontSize":213,"fill":124},"218","Gold markers show where the interpreter may hand the lock to another thread.",[281,520,521,522,525],{},"The lock is held per bytecode, not per statement. Everything between the read and the store is exposed, which is why ",[26,523,524],{},"+="," on shared state needs a lock of its own.",[14,527,529],{"id":528},"step-by-step-implementation","Step-by-step implementation",[531,532,534],"h3",{"id":533},"_1-write-the-failing-test-with-a-barrier","1. Write the failing test with a barrier",[304,536,538],{"className":306,"code":537,"language":308,"meta":309,"style":309},"import threading\n\nimport pytest\n\n\nclass Counter:\n    def __init__(self):\n        self.value = 0\n\n    def increment(self):\n        current = self.value          # read\n        current += 1                  # modify\n        self.value = current          # write — the window is between read and here\n\n\n@pytest.mark.timeout(5)               # a deadlock fails; it does not hang\ndef test_increment_is_atomic():\n    counter = Counter()\n    workers = 8\n    # Every thread blocks here until all 8 have arrived, then all proceed.\n    gate = threading.Barrier(workers, timeout=5)\n\n    def worker():\n        gate.wait()\n        for _ in range(1000):\n            counter.increment()\n\n    threads = [threading.Thread(target=worker) for _ in range(workers)]\n    for thread in threads:\n        thread.start()\n    for thread in threads:\n        thread.join(timeout=5)\n        assert not thread.is_alive(), \"worker did not finish\"\n\n    assert counter.value == workers * 1000\n",[26,539,540,545,549,554,558,562,567,572,577,582,588,594,600,606,611,616,622,628,634,640,646,652,657,663,669,675,681,686,692,698,704,709,715,721,726],{"__ignoreMap":309},[313,541,542],{"class":157,"line":315},[313,543,544],{},"import threading\n",[313,546,547],{"class":157,"line":321},[313,548,325],{"emptyLinePlaceholder":324},[313,550,551],{"class":157,"line":328},[313,552,553],{},"import pytest\n",[313,555,556],{"class":157,"line":333},[313,557,325],{"emptyLinePlaceholder":324},[313,559,560],{"class":157,"line":339},[313,561,325],{"emptyLinePlaceholder":324},[313,563,564],{"class":157,"line":345},[313,565,566],{},"class Counter:\n",[313,568,569],{"class":157,"line":350},[313,570,571],{},"    def __init__(self):\n",[313,573,574],{"class":157,"line":355},[313,575,576],{},"        self.value = 0\n",[313,578,580],{"class":157,"line":579},9,[313,581,325],{"emptyLinePlaceholder":324},[313,583,585],{"class":157,"line":584},10,[313,586,587],{},"    def increment(self):\n",[313,589,591],{"class":157,"line":590},11,[313,592,593],{},"        current = self.value          # read\n",[313,595,597],{"class":157,"line":596},12,[313,598,599],{},"        current += 1                  # modify\n",[313,601,603],{"class":157,"line":602},13,[313,604,605],{},"        self.value = current          # write — the window is between read and here\n",[313,607,609],{"class":157,"line":608},14,[313,610,325],{"emptyLinePlaceholder":324},[313,612,614],{"class":157,"line":613},15,[313,615,325],{"emptyLinePlaceholder":324},[313,617,619],{"class":157,"line":618},16,[313,620,621],{},"@pytest.mark.timeout(5)               # a deadlock fails; it does not hang\n",[313,623,625],{"class":157,"line":624},17,[313,626,627],{},"def test_increment_is_atomic():\n",[313,629,631],{"class":157,"line":630},18,[313,632,633],{},"    counter = Counter()\n",[313,635,637],{"class":157,"line":636},19,[313,638,639],{},"    workers = 8\n",[313,641,643],{"class":157,"line":642},20,[313,644,645],{},"    # Every thread blocks here until all 8 have arrived, then all proceed.\n",[313,647,649],{"class":157,"line":648},21,[313,650,651],{},"    gate = threading.Barrier(workers, timeout=5)\n",[313,653,655],{"class":157,"line":654},22,[313,656,325],{"emptyLinePlaceholder":324},[313,658,660],{"class":157,"line":659},23,[313,661,662],{},"    def worker():\n",[313,664,666],{"class":157,"line":665},24,[313,667,668],{},"        gate.wait()\n",[313,670,672],{"class":157,"line":671},25,[313,673,674],{},"        for _ in range(1000):\n",[313,676,678],{"class":157,"line":677},26,[313,679,680],{},"            counter.increment()\n",[313,682,684],{"class":157,"line":683},27,[313,685,325],{"emptyLinePlaceholder":324},[313,687,689],{"class":157,"line":688},28,[313,690,691],{},"    threads = [threading.Thread(target=worker) for _ in range(workers)]\n",[313,693,695],{"class":157,"line":694},29,[313,696,697],{},"    for thread in threads:\n",[313,699,701],{"class":157,"line":700},30,[313,702,703],{},"        thread.start()\n",[313,705,707],{"class":157,"line":706},31,[313,708,697],{},[313,710,712],{"class":157,"line":711},32,[313,713,714],{},"        thread.join(timeout=5)\n",[313,716,718],{"class":157,"line":717},33,[313,719,720],{},"        assert not thread.is_alive(), \"worker did not finish\"\n",[313,722,724],{"class":157,"line":723},34,[313,725,325],{"emptyLinePlaceholder":324},[313,727,729],{"class":157,"line":728},35,[313,730,731],{},"    assert counter.value == workers * 1000\n",[531,733,735],{"id":734},"_2-make-the-fix-and-watch-the-same-test-pass","2. Make the fix, and watch the same test pass",[304,737,739],{"className":306,"code":738,"language":308,"meta":309,"style":309},"import threading\n\n\nclass Counter:\n    def __init__(self):\n        self.value = 0\n        self._lock = threading.Lock()\n\n    def increment(self):\n        with self._lock:              # the whole read-modify-write is now atomic\n            self.value += 1\n",[26,740,741,745,749,753,757,761,765,770,774,778,783],{"__ignoreMap":309},[313,742,743],{"class":157,"line":315},[313,744,544],{},[313,746,747],{"class":157,"line":321},[313,748,325],{"emptyLinePlaceholder":324},[313,750,751],{"class":157,"line":328},[313,752,325],{"emptyLinePlaceholder":324},[313,754,755],{"class":157,"line":333},[313,756,566],{},[313,758,759],{"class":157,"line":339},[313,760,571],{},[313,762,763],{"class":157,"line":345},[313,764,576],{},[313,766,767],{"class":157,"line":350},[313,768,769],{},"        self._lock = threading.Lock()\n",[313,771,772],{"class":157,"line":355},[313,773,325],{"emptyLinePlaceholder":324},[313,775,776],{"class":157,"line":579},[313,777,587],{},[313,779,780],{"class":157,"line":584},[313,781,782],{},"        with self._lock:              # the whole read-modify-write is now atomic\n",[313,784,785],{"class":157,"line":590},[313,786,787],{},"            self.value += 1\n",[531,789,791],{"id":790},"_3-use-a-seam-when-the-racing-operations-differ","3. Use a seam when the racing operations differ",[10,793,794,795,798],{},"A barrier works when every thread runs the same code. When the race is between two ",[76,796,797],{},"different"," operations — a reader and a writer, an initialiser and a consumer — the test needs to freeze one of them at a chosen point:",[304,800,802],{"className":306,"code":801,"language":308,"meta":309,"style":309},"import threading\nfrom unittest.mock import patch\n\n\ndef test_double_initialisation_is_prevented(service):\n    entered = threading.Event()\n    release = threading.Event()\n    original = service._build_connection\n\n    def slow_build():\n        entered.set()                 # tell the test we are inside the window\n        release.wait(timeout=5)       # hold the window open\n        return original()\n\n    with patch.object(service, \"_build_connection\", slow_build):\n        first = threading.Thread(target=service.get_connection)\n        first.start()\n        assert entered.wait(timeout=5), \"first thread never entered the builder\"\n\n        # Second thread arrives while the first is mid-initialisation.\n        second_result = []\n        second = threading.Thread(target=lambda: second_result.append(service.get_connection()))\n        second.start()\n\n        release.set()\n        first.join(timeout=5)\n        second.join(timeout=5)\n\n    assert service.build_count == 1, \"the connection was built twice\"\n",[26,803,804,808,813,817,821,826,831,836,841,845,850,855,860,865,869,874,879,884,889,893,898,903,908,913,917,922,927,932,936],{"__ignoreMap":309},[313,805,806],{"class":157,"line":315},[313,807,544],{},[313,809,810],{"class":157,"line":321},[313,811,812],{},"from unittest.mock import patch\n",[313,814,815],{"class":157,"line":328},[313,816,325],{"emptyLinePlaceholder":324},[313,818,819],{"class":157,"line":333},[313,820,325],{"emptyLinePlaceholder":324},[313,822,823],{"class":157,"line":339},[313,824,825],{},"def test_double_initialisation_is_prevented(service):\n",[313,827,828],{"class":157,"line":345},[313,829,830],{},"    entered = threading.Event()\n",[313,832,833],{"class":157,"line":350},[313,834,835],{},"    release = threading.Event()\n",[313,837,838],{"class":157,"line":355},[313,839,840],{},"    original = service._build_connection\n",[313,842,843],{"class":157,"line":579},[313,844,325],{"emptyLinePlaceholder":324},[313,846,847],{"class":157,"line":584},[313,848,849],{},"    def slow_build():\n",[313,851,852],{"class":157,"line":590},[313,853,854],{},"        entered.set()                 # tell the test we are inside the window\n",[313,856,857],{"class":157,"line":596},[313,858,859],{},"        release.wait(timeout=5)       # hold the window open\n",[313,861,862],{"class":157,"line":602},[313,863,864],{},"        return original()\n",[313,866,867],{"class":157,"line":608},[313,868,325],{"emptyLinePlaceholder":324},[313,870,871],{"class":157,"line":613},[313,872,873],{},"    with patch.object(service, \"_build_connection\", slow_build):\n",[313,875,876],{"class":157,"line":618},[313,877,878],{},"        first = threading.Thread(target=service.get_connection)\n",[313,880,881],{"class":157,"line":624},[313,882,883],{},"        first.start()\n",[313,885,886],{"class":157,"line":630},[313,887,888],{},"        assert entered.wait(timeout=5), \"first thread never entered the builder\"\n",[313,890,891],{"class":157,"line":636},[313,892,325],{"emptyLinePlaceholder":324},[313,894,895],{"class":157,"line":642},[313,896,897],{},"        # Second thread arrives while the first is mid-initialisation.\n",[313,899,900],{"class":157,"line":648},[313,901,902],{},"        second_result = []\n",[313,904,905],{"class":157,"line":654},[313,906,907],{},"        second = threading.Thread(target=lambda: second_result.append(service.get_connection()))\n",[313,909,910],{"class":157,"line":659},[313,911,912],{},"        second.start()\n",[313,914,915],{"class":157,"line":665},[313,916,325],{"emptyLinePlaceholder":324},[313,918,919],{"class":157,"line":671},[313,920,921],{},"        release.set()\n",[313,923,924],{"class":157,"line":677},[313,925,926],{},"        first.join(timeout=5)\n",[313,928,929],{"class":157,"line":683},[313,930,931],{},"        second.join(timeout=5)\n",[313,933,934],{"class":157,"line":688},[313,935,325],{"emptyLinePlaceholder":324},[313,937,938],{"class":157,"line":694},[313,939,940],{},"    assert service.build_count == 1, \"the connection was built twice\"\n",[10,942,943,944,947,948,49],{},"This is the pattern that catches double-initialisation, cache stampedes and duplicated side effects, and it is deterministic: the second thread is ",[76,945,946],{},"guaranteed"," to arrive during the first thread's window because the test controls when the window closes. The patching technique it relies on is covered in ",[45,949,951],{"href":950},"\u002Fadvanced-mocking-test-doubles-in-python\u002Fpatching-strategies-for-complex-codebases\u002Fpatching-class-attributes-with-patch-object\u002F","patching class attributes with patch.object",[531,953,955],{"id":954},"_4-surface-exceptions-raised-in-worker-threads","4. Surface exceptions raised in worker threads",[10,957,958,959,962,963,965,966,969],{},"An exception inside a ",[26,960,961],{},"threading.Thread"," target is printed to stderr and then discarded; the test does not fail. ",[26,964,32],{}," fixes this because ",[26,967,968],{},"future.result()"," re-raises:",[304,971,973],{"className":306,"code":972,"language":308,"meta":309,"style":309},"from concurrent.futures import ThreadPoolExecutor\n\n\ndef test_workers_do_not_raise():\n    with ThreadPoolExecutor(max_workers=4) as pool:\n        futures = [pool.submit(do_work, i) for i in range(4)]\n\n    # result() re-raises in the calling thread, so worker failures fail the test.\n    results = [future.result(timeout=5) for future in futures]\n    assert sorted(results) == [0, 1, 2, 3]\n",[26,974,975,980,984,988,993,998,1003,1007,1012,1017],{"__ignoreMap":309},[313,976,977],{"class":157,"line":315},[313,978,979],{},"from concurrent.futures import ThreadPoolExecutor\n",[313,981,982],{"class":157,"line":321},[313,983,325],{"emptyLinePlaceholder":324},[313,985,986],{"class":157,"line":328},[313,987,325],{"emptyLinePlaceholder":324},[313,989,990],{"class":157,"line":333},[313,991,992],{},"def test_workers_do_not_raise():\n",[313,994,995],{"class":157,"line":339},[313,996,997],{},"    with ThreadPoolExecutor(max_workers=4) as pool:\n",[313,999,1000],{"class":157,"line":345},[313,1001,1002],{},"        futures = [pool.submit(do_work, i) for i in range(4)]\n",[313,1004,1005],{"class":157,"line":350},[313,1006,325],{"emptyLinePlaceholder":324},[313,1008,1009],{"class":157,"line":355},[313,1010,1011],{},"    # result() re-raises in the calling thread, so worker failures fail the test.\n",[313,1013,1014],{"class":157,"line":579},[313,1015,1016],{},"    results = [future.result(timeout=5) for future in futures]\n",[313,1018,1019],{"class":157,"line":584},[313,1020,1021],{},"    assert sorted(results) == [0, 1, 2, 3]\n",[10,1023,1024,1025,1027,1028,1031],{},"Where ",[26,1026,961],{}," is unavoidable, ",[26,1029,1030],{},"threading.excepthook"," (3.8+) can collect worker exceptions into a list the test asserts is empty. Silent worker failures are the single most common reason a concurrency test passes while the code is broken.",[14,1033,1035],{"id":1034},"verification","Verification",[10,1037,1038],{},"A race test has a property ordinary tests do not: it must fail before it can be trusted. Verify it by reverting the fix — comment out the lock, run the test, confirm it fails on every one of ten consecutive runs.",[304,1040,1044],{"className":1041,"code":1042,"language":1043,"meta":309,"style":309},"language-bash shiki shiki-themes github-light github-dark","pytest tests\u002Ftest_counter.py::test_increment_is_atomic --count=10 -q   # pytest-repeat\n","bash",[26,1045,1046],{"__ignoreMap":309},[313,1047,1048,1052,1056,1060,1063],{"class":157,"line":315},[313,1049,1051],{"class":1050},"sScJk","pytest",[313,1053,1055],{"class":1054},"sZZnC"," tests\u002Ftest_counter.py::test_increment_is_atomic",[313,1057,1059],{"class":1058},"sj4cs"," --count=10",[313,1061,1062],{"class":1058}," -q",[313,1064,1066],{"class":1065},"sJ8bj","   # pytest-repeat\n",[304,1068,1071],{"className":1069,"code":1070,"language":134,"meta":309},[362],".........F                                                    [100%]\nE       assert 7994 == 8000\n",[26,1072,1070],{"__ignoreMap":309},[10,1074,1075],{},"Ten passes with the lock and ten failures without it is the evidence that the test is measuring the thing it claims to measure. A test that fails only occasionally without the fix is still probabilistic — the barrier is not tight enough, and usually the reason is that the contended region is too short relative to thread start-up. Increasing the iteration count inside each worker, rather than the number of workers, widens the window without adding scheduling noise.",[14,1077,1079],{"id":1078},"troubleshooting","Troubleshooting",[1081,1082,1083,1099],"table",{},[1084,1085,1086],"thead",{},[1087,1088,1089,1093,1096],"tr",{},[1090,1091,1092],"th",{},"Symptom",[1090,1094,1095],{},"Root cause",[1090,1097,1098],{},"Fix",[1100,1101,1102,1128,1144,1155,1166,1185],"tbody",{},[1087,1103,1104,1108,1111],{},[1105,1106,1107],"td",{},"Test hangs forever",[1105,1109,1110],{},"A deadlock, or a barrier waiting for a thread that died",[1105,1112,1113,1114,1117,1118,29,1121,1124,1125],{},"Pass ",[26,1115,1116],{},"timeout"," to ",[26,1119,1120],{},"Barrier",[26,1122,1123],{},"join","; add ",[26,1126,1127],{},"@pytest.mark.timeout",[1087,1129,1130,1135,1141],{},[1105,1131,1132],{},[26,1133,1134],{},"BrokenBarrierError",[1105,1136,1137,1138],{},"One participant raised before reaching ",[26,1139,1140],{},"wait()",[1105,1142,1143],{},"Wrap worker bodies so failures are recorded, then re-raise in the main thread",[1087,1145,1146,1149,1152],{},[1105,1147,1148],{},"Test passes without the fix",[1105,1150,1151],{},"Window too narrow, or threads not actually concurrent",[1105,1153,1154],{},"Raise per-worker iterations; confirm workers start before any joins",[1087,1156,1157,1160,1163],{},[1105,1158,1159],{},"Passes locally, fails in CI",[1105,1161,1162],{},"CI has fewer cores, changing the scheduler's behaviour",[1105,1164,1165],{},"Make the test deterministic with a seam rather than tuning counts",[1087,1167,1168,1171,1176],{},[1105,1169,1170],{},"Worker exception invisible",[1105,1172,1173,1175],{},[26,1174,961],{}," swallows exceptions",[1105,1177,1178,1179,1182,1183],{},"Use ",[26,1180,1181],{},"ThreadPoolExecutor"," and call ",[26,1184,968],{},[1087,1186,1187,1193,1196],{},[1105,1188,1189,1190],{},"Flaky only under ",[26,1191,1192],{},"-n auto",[1105,1194,1195],{},"Shared module state across xdist workers",[1105,1197,1198,1199],{},"Isolate with per-worker fixtures; see ",[45,1200,1202],{"href":1201},"\u002Fsystematic-debugging-performance-profiling\u002Fdebugging-tests-in-ci-and-containers\u002Fdebugging-a-test-that-only-fails-under-xdist\u002F","debugging a test that only fails under xdist",[14,1204,1206],{"id":1205},"deadlocks-the-failure-with-no-traceback","Deadlocks: the failure with no traceback",[10,1208,1209],{},"A race produces a wrong answer; a deadlock produces no answer at all. Two threads each holding a lock the other wants will wait forever, and pytest's default behaviour is to wait with them until the CI platform kills the job.",[10,1211,1212,1213,1216],{},"The first line of defence is ordering discipline in the code: every code path acquires locks in the same global order, which makes a cycle impossible by construction. The second is detection in the tests. ",[26,1214,1215],{},"faulthandler",", in the standard library, can be armed to dump every thread's stack after a timeout:",[304,1218,1220],{"className":306,"code":1219,"language":308,"meta":309,"style":309},"import faulthandler\n\n# In conftest.py: after 30 seconds of no progress, dump all thread stacks\n# and abort. Every stuck thread's frame is printed, which names the locks.\nfaulthandler.dump_traceback_later(30, exit=True)\n",[26,1221,1222,1227,1231,1236,1241],{"__ignoreMap":309},[313,1223,1224],{"class":157,"line":315},[313,1225,1226],{},"import faulthandler\n",[313,1228,1229],{"class":157,"line":321},[313,1230,325],{"emptyLinePlaceholder":324},[313,1232,1233],{"class":157,"line":328},[313,1234,1235],{},"# In conftest.py: after 30 seconds of no progress, dump all thread stacks\n",[313,1237,1238],{"class":157,"line":333},[313,1239,1240],{},"# and abort. Every stuck thread's frame is printed, which names the locks.\n",[313,1242,1243],{"class":157,"line":339},[313,1244,1245],{},"faulthandler.dump_traceback_later(30, exit=True)\n",[304,1247,1250],{"className":1248,"code":1249,"language":134,"meta":309},[362],"Thread 0x00007f2a (most recent call first):\n  File \"app\u002Fcache.py\", line 41 in refresh      ← waiting on _write_lock\n  File \"app\u002Fcache.py\", line 88 in get\nThread 0x00007f2b (most recent call first):\n  File \"app\u002Fcache.py\", line 62 in evict        ← waiting on _index_lock\n  File \"app\u002Fcache.py\", line 91 in put\n",[26,1251,1249],{"__ignoreMap":309},[10,1253,1254],{},"Two threads, two locks, opposite order: the dump contains the complete diagnosis. Without it the same deadlock is a job that timed out with no output at all.",[85,1256,1258,1357],{"className":1257},[88],[90,1259,98,1264,98,1267,98,1270,98,1282,98,1285,98,1289,98,1292,98,1296,98,1299,98,1302,98,1305,98,1308,98,1313,98,1317,98,1320,98,1322,98,1325,98,1328,98,1332,98,1334,98,1343,98,1350,98,1353],{"viewBox":1260,"role":93,"ariaLabelledBy":1261,"xmlns":97},"0 0 780 240",[1262,1263],"dead-t","dead-d",[100,1265,1266],{"id":1262},"A lock-ordering cycle between two threads",[104,1268,1269],{"id":1263},"Thread one holds the write lock and waits for the index lock. Thread two holds the index lock and waits for the write lock. The two wait edges form a cycle, which is the deadlock. Acquiring both locks in one global order removes the cycle.",[108,1271,110,1272,110,1277,98],{},[112,1273,1275],{"id":1274,"viewBox":115,"refX":116,"refY":117,"markerWidth":118,"markerHeight":118,"orient":119},"dead-a",[121,1276],{"d":123,"fill":241},[112,1278,1280],{"id":1279,"viewBox":115,"refX":116,"refY":117,"markerWidth":118,"markerHeight":118,"orient":119},"dead-b",[121,1281],{"d":123,"fill":177},[126,1283],{"x":128,"y":128,"width":1284,"height":212,"rx":131,"fill":132},"780",[134,1286,1288],{"x":1287,"y":137,"textAnchor":138,"fontSize":139,"fontWeight":140,"fill":124},"390","Holds one, waits for the other, in both directions",[126,1290],{"x":173,"y":1291,"width":196,"height":173,"rx":213,"fill":420,"stroke":124,"strokeWidth":242},"64",[134,1293,147],{"x":1294,"y":1295,"textAnchor":138,"fontSize":424,"fontWeight":140,"fill":124},"150","90",[134,1297,1298],{"x":1294,"y":472,"textAnchor":138,"fontSize":213,"fill":124},"in refresh()",[126,1300],{"x":1301,"y":1291,"width":196,"height":173,"rx":213,"fill":420,"stroke":124,"strokeWidth":242},"540",[134,1303,151],{"x":1304,"y":1295,"textAnchor":138,"fontSize":424,"fontWeight":140,"fill":124},"630",[134,1306,1307],{"x":1304,"y":472,"textAnchor":138,"fontSize":213,"fill":124},"in evict()",[126,1309],{"x":173,"y":1310,"width":196,"height":1311,"rx":213,"fill":176,"stroke":177,"strokeWidth":1312},"154","56","2",[134,1314,1316],{"x":1294,"y":1315,"textAnchor":138,"fontSize":146,"fontWeight":140,"fill":124},"178","_write_lock",[134,1318,1319],{"x":1294,"y":169,"textAnchor":138,"fontSize":213,"fill":447},"held by thread 1",[126,1321],{"x":1301,"y":1310,"width":196,"height":1311,"rx":213,"fill":176,"stroke":177,"strokeWidth":1312},[134,1323,1324],{"x":1304,"y":1315,"textAnchor":138,"fontSize":146,"fontWeight":140,"fill":124},"_index_lock",[134,1326,1327],{"x":1304,"y":169,"textAnchor":138,"fontSize":213,"fill":447},"held by thread 2",[157,1329],{"x1":1294,"y1":1330,"x2":1294,"y2":1294,"stroke":177,"strokeWidth":242,"markerEnd":1331},"128","url(#dead-b)",[157,1333],{"x1":1304,"y1":1330,"x2":1304,"y2":1294,"stroke":177,"strokeWidth":242,"markerEnd":1331},[157,1335],{"x1":1336,"y1":1337,"x2":1338,"y2":1339,"stroke":241,"strokeWidth":242,"strokeDashArray":1340,"markerEnd":1342},"244","88","534","170",[1341,207],"6","url(#dead-a)",[157,1344],{"x1":1345,"y1":1346,"x2":1347,"y2":1348,"stroke":241,"strokeWidth":242,"strokeDashArray":1349,"markerEnd":1342},"536","104","246","186",[1341,207],[134,1351,1352],{"x":1287,"y":172,"textAnchor":138,"fontSize":513,"fontWeight":140,"fill":268},"waits for →",[134,1354,1356],{"x":1287,"y":1355,"textAnchor":138,"fontSize":513,"fontWeight":140,"fill":268},"162","← waits for",[281,1358,1359,1360,1362,1363,1365],{},"The dashed edges are the deadlock. Any global ordering rule — always take ",[26,1361,1324],{}," before ",[26,1364,1316],{}," — makes one of the two edges impossible.",[10,1367,1368,1369,1371,1372,49],{},"The full setup, including how to combine it with ",[26,1370,42],{},"'s thread method so both the dump and a clean failure are produced, is in ",[45,1373,1375],{"href":1374},"\u002Ftesting-async-and-concurrent-python\u002Ftesting-threads-and-race-conditions\u002Fdumping-stacks-on-deadlock-with-faulthandler\u002F","dumping stacks on deadlock with faulthandler",[14,1377,1379],{"id":1378},"when-a-stress-loop-is-still-the-right-tool","When a stress loop is still the right tool",[10,1381,1382],{},"Deterministic tests catch the race you already understand. They cannot find the one nobody has thought of, and for that a stress loop remains the only option — run the operation from many threads, many times, and assert the invariant at the end.",[304,1384,1386],{"className":306,"code":1385,"language":308,"meta":309,"style":309},"import random\nimport threading\n\nimport pytest\n\n\n@pytest.mark.slow                      # nightly, not on every push\n@pytest.mark.timeout(120)\ndef test_cache_invariant_under_random_load(cache):\n    stop = threading.Event()\n    errors: list[BaseException] = []\n\n    def churn(seed: int) -> None:\n        rng = random.Random(seed)      # per-thread seed: reproducible from the report\n        try:\n            while not stop.is_set():\n                key = rng.randrange(64)\n                if rng.random() \u003C 0.3:\n                    cache.evict(key)\n                else:\n                    cache.get_or_set(key, lambda: key * 2)\n        except BaseException as exc:   # noqa: BLE001 - re-raised in the main thread\n            errors.append(exc)\n\n    threads = [threading.Thread(target=churn, args=(seed,)) for seed in range(8)]\n    for thread in threads:\n        thread.start()\n    stop.wait(10)                      # ten seconds of contention\n    stop.set()\n    for thread in threads:\n        thread.join(timeout=10)\n\n    assert not errors, errors\n    assert cache.size() \u003C= cache.capacity, \"eviction lost track of the bound\"\n",[26,1387,1388,1393,1397,1401,1405,1409,1413,1418,1423,1428,1433,1438,1442,1447,1452,1457,1462,1467,1472,1477,1482,1487,1492,1497,1501,1506,1510,1514,1519,1524,1528,1533,1537,1542],{"__ignoreMap":309},[313,1389,1390],{"class":157,"line":315},[313,1391,1392],{},"import random\n",[313,1394,1395],{"class":157,"line":321},[313,1396,544],{},[313,1398,1399],{"class":157,"line":328},[313,1400,325],{"emptyLinePlaceholder":324},[313,1402,1403],{"class":157,"line":333},[313,1404,553],{},[313,1406,1407],{"class":157,"line":339},[313,1408,325],{"emptyLinePlaceholder":324},[313,1410,1411],{"class":157,"line":345},[313,1412,325],{"emptyLinePlaceholder":324},[313,1414,1415],{"class":157,"line":350},[313,1416,1417],{},"@pytest.mark.slow                      # nightly, not on every push\n",[313,1419,1420],{"class":157,"line":355},[313,1421,1422],{},"@pytest.mark.timeout(120)\n",[313,1424,1425],{"class":157,"line":579},[313,1426,1427],{},"def test_cache_invariant_under_random_load(cache):\n",[313,1429,1430],{"class":157,"line":584},[313,1431,1432],{},"    stop = threading.Event()\n",[313,1434,1435],{"class":157,"line":590},[313,1436,1437],{},"    errors: list[BaseException] = []\n",[313,1439,1440],{"class":157,"line":596},[313,1441,325],{"emptyLinePlaceholder":324},[313,1443,1444],{"class":157,"line":602},[313,1445,1446],{},"    def churn(seed: int) -> None:\n",[313,1448,1449],{"class":157,"line":608},[313,1450,1451],{},"        rng = random.Random(seed)      # per-thread seed: reproducible from the report\n",[313,1453,1454],{"class":157,"line":613},[313,1455,1456],{},"        try:\n",[313,1458,1459],{"class":157,"line":618},[313,1460,1461],{},"            while not stop.is_set():\n",[313,1463,1464],{"class":157,"line":624},[313,1465,1466],{},"                key = rng.randrange(64)\n",[313,1468,1469],{"class":157,"line":630},[313,1470,1471],{},"                if rng.random() \u003C 0.3:\n",[313,1473,1474],{"class":157,"line":636},[313,1475,1476],{},"                    cache.evict(key)\n",[313,1478,1479],{"class":157,"line":642},[313,1480,1481],{},"                else:\n",[313,1483,1484],{"class":157,"line":648},[313,1485,1486],{},"                    cache.get_or_set(key, lambda: key * 2)\n",[313,1488,1489],{"class":157,"line":654},[313,1490,1491],{},"        except BaseException as exc:   # noqa: BLE001 - re-raised in the main thread\n",[313,1493,1494],{"class":157,"line":659},[313,1495,1496],{},"            errors.append(exc)\n",[313,1498,1499],{"class":157,"line":665},[313,1500,325],{"emptyLinePlaceholder":324},[313,1502,1503],{"class":157,"line":671},[313,1504,1505],{},"    threads = [threading.Thread(target=churn, args=(seed,)) for seed in range(8)]\n",[313,1507,1508],{"class":157,"line":677},[313,1509,697],{},[313,1511,1512],{"class":157,"line":683},[313,1513,703],{},[313,1515,1516],{"class":157,"line":688},[313,1517,1518],{},"    stop.wait(10)                      # ten seconds of contention\n",[313,1520,1521],{"class":157,"line":694},[313,1522,1523],{},"    stop.set()\n",[313,1525,1526],{"class":157,"line":700},[313,1527,697],{},[313,1529,1530],{"class":157,"line":706},[313,1531,1532],{},"        thread.join(timeout=10)\n",[313,1534,1535],{"class":157,"line":711},[313,1536,325],{"emptyLinePlaceholder":324},[313,1538,1539],{"class":157,"line":717},[313,1540,1541],{},"    assert not errors, errors\n",[313,1543,1544],{"class":157,"line":723},[313,1545,1546],{},"    assert cache.size() \u003C= cache.capacity, \"eviction lost track of the bound\"\n",[10,1548,1549,1550,1553,1554,1557],{},"Three details make the difference between a stress test that is useful and one that is merely slow. Each thread gets its own seeded ",[26,1551,1552],{},"Random",", so a failure can be replayed exactly by re-running with the reported seeds. Exceptions are collected rather than printed, so a worker crash fails the test. And the assertion is an invariant that must hold at ",[76,1555,1556],{},"any"," point — a size bound, a conservation law, a monotonic counter — rather than an exact expected value, because under random load there is no single expected value.",[10,1559,1560,1561,1564,1565,1569],{},"Mark these ",[26,1562,1563],{},"slow"," and keep them out of the pull-request suite. A ten-second test that fails once a month is valuable in a nightly job and corrosive in a feedback loop, where its occasional failure teaches everyone to re-run the build rather than to read it. The marker-based split is the same one used for ",[45,1566,1568],{"href":1567},"\u002Fintegration-database-and-service-testing\u002F","integration tests",", and the reasoning is identical: a suite people trust is one where red means broken.",[10,1571,1572,1573,1576,1577,1581],{},"For invariant-driven exploration with better shrinking than a random loop can offer, a ",[26,1574,1575],{},"RuleBasedStateMachine"," generates operation sequences and then reduces a failure to its minimal reproduction — see ",[45,1578,1580],{"href":1579},"\u002Fproperty-based-fuzz-testing-strategies\u002Fstateful-and-model-based-testing\u002Fmodeling-a-cache-with-invariants-and-bundles\u002F","modeling a cache with invariants and bundles",", which applies exactly this technique to the same problem.",[14,1583,1585],{"id":1584},"fixtures-that-are-safe-to-share-across-threads","Fixtures that are safe to share across threads",[10,1587,1588],{},"A thread-safety test puts unusual demands on its fixtures, because the fixture's object is now touched by several threads at once. Three rules keep this from becoming a second source of flakiness.",[10,1590,1591,1592,1595],{},"Build the shared object ",[76,1593,1594],{},"inside"," the test, not in a session-scoped fixture: the point of the test is that this object is contended, and sharing it with other tests reintroduces order dependence. Where a genuinely shared resource is unavoidable — a database connection pool, a temporary directory — give each thread its own handle from the pool rather than sharing one handle, since most client libraries document per-connection thread safety and nothing stronger. And never let a fixture's teardown run while a worker thread is still alive: join every thread inside the test body, with a timeout, before any assertion that could fail and skip the joins.",[10,1597,1598,1599,1602,1603,1606,1607,1609,1610,1613,1614,1616],{},"That last rule is the one most often broken. An assertion failing mid-test propagates immediately, the fixture tears down the object the workers are still using, and the resulting error — a closed file, a released lock, a dead connection — is reported instead of the assertion that actually failed. Wrapping the thread lifecycle in a ",[26,1600,1601],{},"try","\u002F",[26,1604,1605],{},"finally",", or using ",[26,1608,1181],{}," as a context manager so its ",[26,1611,1612],{},"__exit__"," joins the workers, keeps the real failure visible. The habit worth forming is to treat every worker thread as a resource with an owner: whoever started it joins it, in a ",[26,1615,1605],{},", with a timeout, before anything else in the test is allowed to fail.",[14,1618,1620],{"id":1619},"frequently-asked-questions","Frequently Asked Questions",[10,1622,1623,1627],{},[1624,1625,1626],"strong",{},"Why can't I just run the test 1000 times to catch a race?","\nBecause the scheduler is not adversarial. A race whose window is a few hundred nanoseconds may need millions of iterations on an idle machine and zero on a loaded one, so a loop that passes proves nothing and a loop that fails wastes minutes. Forcing the interleaving with a barrier turns a probabilistic test into a deterministic one that runs in milliseconds.",[10,1629,1630,1633,1634,1637,1638,1641],{},[1624,1631,1632],{},"Does the GIL mean Python code can't have data races?","\nNo. The GIL makes individual bytecodes atomic, not statements. ",[26,1635,1636],{},"x += 1"," compiles to a load, an add and a store, and the interpreter can switch threads between any two of them. Anything involving a check followed by an action — a cache miss followed by a fill, a ",[26,1639,1640],{},"None"," check followed by an assignment — is a race regardless of the GIL.",[10,1643,1644,1647,1648,1651],{},[1624,1645,1646],{},"How do I test code that uses a ThreadPoolExecutor?","\nSubmit work through the executor as production does, but control the workers with a barrier so they arrive at the contended section together. Collect futures and call ",[26,1649,1650],{},"result()"," on each so exceptions in workers surface as test failures rather than being swallowed by the pool.",[10,1653,1654,1657],{},[1624,1655,1656],{},"What is the right way to assert that a lock is actually held?","\nDo not inspect the lock. Assert on the invariant the lock exists to protect: run N threads doing M increments and assert the total is exactly N times M. A lock check passes even when the lock protects the wrong region; an invariant check does not.",[10,1659,1660,1663],{},[1624,1661,1662],{},"Should race tests run in the normal suite or separately?","\nIn the normal suite, if they are deterministic. A barrier-driven test takes milliseconds and fails every time the bug is present, which is exactly what a regression test should do. Only stress loops and long-running fuzzers belong in a separate nightly job.",[14,1665,1667],{"id":1666},"related-guides","Related guides",[19,1669,1670,1677,1684,1689,1696],{},[22,1671,1672,1673,49],{},"Work through a complete reproduction in ",[45,1674,1676],{"href":1675},"\u002Ftesting-async-and-concurrent-python\u002Ftesting-threads-and-race-conditions\u002Freproducing-a-race-condition-deterministically\u002F","reproducing a race condition deterministically",[22,1678,1679,1680,49],{},"Learn the primitive-by-primitive patterns in ",[45,1681,1683],{"href":1682},"\u002Ftesting-async-and-concurrent-python\u002Ftesting-threads-and-race-conditions\u002Ftesting-thread-safety-with-barriers-and-events\u002F","testing thread safety with barriers and events",[22,1685,1686,1687,49],{},"Turn a hang into a diagnosis with ",[45,1688,1375],{"href":1374},[22,1690,1691,1692,49],{},"Explore interleavings you did not think of using ",[45,1693,1695],{"href":1694},"\u002Fproperty-based-fuzz-testing-strategies\u002Fstateful-and-model-based-testing\u002F","stateful and model-based testing",[22,1697,1698,1699,49],{},"For cooperative rather than preemptive concurrency, the equivalent techniques are in ",[45,1700,1702],{"href":1701},"\u002Ftesting-async-and-concurrent-python\u002Fpytest-asyncio-in-depth\u002F","pytest-asyncio in depth",[10,1704,1705,1706],{},"← Back to ",[45,1707,1708],{"href":62},"Testing Async & Concurrent Python",[1710,1711,1712],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .sScJk, html code.shiki .sScJk{--shiki-default:#6F42C1;--shiki-dark:#B392F0}html pre.shiki code .sZZnC, html code.shiki .sZZnC{--shiki-default:#032F62;--shiki-dark:#9ECBFF}html pre.shiki code .sj4cs, html code.shiki .sj4cs{--shiki-default:#005CC5;--shiki-dark:#79B8FF}html pre.shiki code .sJ8bj, html code.shiki .sJ8bj{--shiki-default:#6A737D;--shiki-dark:#6A737D}",{"title":309,"searchDepth":321,"depth":321,"links":1714},[1715,1716,1717,1718,1724,1725,1726,1727,1728,1729,1730],{"id":16,"depth":321,"text":17},{"id":67,"depth":321,"text":68},{"id":286,"depth":321,"text":287},{"id":528,"depth":321,"text":529,"children":1719},[1720,1721,1722,1723],{"id":533,"depth":328,"text":534},{"id":734,"depth":328,"text":735},{"id":790,"depth":328,"text":791},{"id":954,"depth":328,"text":955},{"id":1034,"depth":321,"text":1035},{"id":1078,"depth":321,"text":1079},{"id":1205,"depth":321,"text":1206},{"id":1378,"depth":321,"text":1379},{"id":1584,"depth":321,"text":1585},{"id":1619,"depth":321,"text":1620},{"id":1666,"depth":321,"text":1667},"Make concurrency bugs reproducible: barriers and events instead of sleeps, forced interleavings, GIL-aware assumptions, deadlock detection, and thread-safe fixtures.","md",{"slug":1734,"type":1735,"breadcrumb":1736,"datePublished":1737,"dateModified":1737,"faq":1738,"howto":1749},"testing-threads-and-race-conditions","topic","Threads & Races","2026-09-18",[1739,1741,1743,1745,1747],{"q":1626,"a":1740},"Because the scheduler is not adversarial. A race whose window is a few hundred nanoseconds may need millions of iterations on an idle machine and zero on a loaded one, so a loop that passes proves nothing and a loop that fails wastes minutes. Forcing the interleaving with a barrier turns a probabilistic test into a deterministic one that runs in milliseconds.",{"q":1632,"a":1742},"No. The GIL makes individual bytecodes atomic, not statements. x += 1 compiles to a load, an add and a store, and the interpreter can switch threads between any two of them. Anything involving a check followed by an action — a cache miss followed by a fill, a None check followed by an assignment — is a race regardless of the GIL.",{"q":1646,"a":1744},"Submit work through the executor as production does, but control the workers with a barrier so they arrive at the contended section together. Collect futures and call result() on each so exceptions in workers surface as test failures rather than being swallowed by the pool.",{"q":1656,"a":1746},"Do not inspect the lock. Assert on the invariant the lock exists to protect: run N threads doing M increments and assert the total is exactly N times M. A lock check passes even when the lock protects the wrong region; an invariant check does not.",{"q":1662,"a":1748},"In the normal suite, if they are deterministic. A barrier-driven test takes milliseconds and fails every time the bug is present, which is exactly what a regression test should do. Only stress loops and long-running fuzzers belong in a separate nightly job.",{"name":1750,"description":1751,"steps":1752},"How to make a race condition reproduce on demand","Identify the contended window, force every thread to arrive at it simultaneously, and assert on the invariant rather than on timing.",[1753,1756,1759,1762,1765],{"name":1754,"text":1755},"Locate the check-then-act window","Find the read-modify-write or check-then-act sequence that is not atomic, and note the exact line where a thread switch would break the invariant.",{"name":1757,"text":1758},"Insert a controllable yield point","Give the code under test a seam — an injected callback or a patched function — that the test can use to pause a thread mid-sequence.",{"name":1760,"text":1761},"Synchronise the threads with a barrier","Use threading.Barrier(n) so every worker blocks until all n have arrived, guaranteeing they enter the contended region together.",{"name":1763,"text":1764},"Assert on the invariant","Check the aggregate result, such as a final counter value or the number of rows created, rather than checking whether a lock was taken.",{"name":1766,"text":1767},"Add a timeout to every join","Pass a timeout to Barrier and to thread.join so a deadlock fails the test with a stack dump instead of hanging the suite.","\u002Ftesting-async-and-concurrent-python\u002Ftesting-threads-and-race-conditions",{"title":5,"description":1731},"testing-async-and-concurrent-python\u002Ftesting-threads-and-race-conditions\u002Findex","KU5o6WJIGKznIuS-JnpPv-ZP_wLGP9-GcRK3ST6x9C0",1789718765719]