[{"data":1,"prerenderedAt":1023},["ShallowReactive",2],{"page-\u002Fproperty-based-fuzz-testing-strategies\u002Fstateful-and-model-based-testing\u002Fdebugging-rulebasedstatemachine-failures\u002F":3},{"id":4,"title":5,"body":6,"description":989,"extension":990,"meta":991,"navigation":114,"path":1019,"seo":1020,"stem":1021,"__hash__":1022},"content\u002Fproperty-based-fuzz-testing-strategies\u002Fstateful-and-model-based-testing\u002Fdebugging-rulebasedstatemachine-failures\u002Findex.md","Debugging RuleBasedStateMachine Failures",{"type":7,"value":8,"toc":978},"minimark",[9,18,23,49,53,56,122,142,145,187,190,356,360,363,464,468,475,570,577,580,584,587,597,615,650,654,657,663,741,752,762,765,769,828,903,907,913,919,929,935,939,968,974],[10,11,12,13,17],"p",{},"A ",[14,15,16],"code",{},"RuleBasedStateMachine"," finds a bug that no example-based test would have: a sequence of eleven operations after which the cache reports a size the model disagrees with. The report is a wall of generated calls, and the instinct is to stare at it. There is a faster method, and it starts from the bottom of the output rather than the top.",[19,20,22],"h2",{"id":21},"prerequisites","Prerequisites",[24,25,26,41],"ul",{},[27,28,29,33,34,36,37,40],"li",{},[30,31,32],"strong",{},"hypothesis 6.90+"," with ",[14,35,16],{}," from ",[14,38,39],{},"hypothesis.stateful",".",[27,42,43,44,40],{},"The machine design in ",[45,46,48],"a",{"href":47},"\u002Fproperty-based-fuzz-testing-strategies\u002Fstateful-and-model-based-testing\u002F","stateful and model-based testing",[19,50,52],{"id":51},"solution","Solution",[10,54,55],{},"Hypothesis prints the failing run as executable Python. That output is the artefact — read it, then run it.",[57,58,63],"pre",{"className":59,"code":60,"language":61,"meta":62,"style":62},"language-console shiki shiki-themes github-light github-dark","Falsifying example:\nstate = CacheMachine()\nstate.put(key='0', value=0)\nstate.put(key='1', value=0)\nstate.get(key='0')\nstate.put(key='2', value=0)\nstate.teardown()\n\nExplanation: assertion failed in invariant size_never_exceeds_capacity\n","console","",[14,64,65,73,79,85,91,97,103,109,116],{"__ignoreMap":62},[66,67,70],"span",{"class":68,"line":69},"line",1,[66,71,72],{},"Falsifying example:\n",[66,74,76],{"class":68,"line":75},2,[66,77,78],{},"state = CacheMachine()\n",[66,80,82],{"class":68,"line":81},3,[66,83,84],{},"state.put(key='0', value=0)\n",[66,86,88],{"class":68,"line":87},4,[66,89,90],{},"state.put(key='1', value=0)\n",[66,92,94],{"class":68,"line":93},5,[66,95,96],{},"state.get(key='0')\n",[66,98,100],{"class":68,"line":99},6,[66,101,102],{},"state.put(key='2', value=0)\n",[66,104,106],{"class":68,"line":105},7,[66,107,108],{},"state.teardown()\n",[66,110,112],{"class":68,"line":111},8,[66,113,115],{"emptyLinePlaceholder":114},true,"\n",[66,117,119],{"class":68,"line":118},9,[66,120,121],{},"Explanation: assertion failed in invariant size_never_exceeds_capacity\n",[10,123,124,125,129,130,133,134,137,138,141],{},"Read it from the bottom. The invariant names ",[126,127,128],"em",{},"what"," broke; the last rule before it names ",[126,131,132],{},"when",". Here the third ",[14,135,136],{},"put"," evicted an entry, but the earlier ",[14,139,140],{},"get"," refreshed recency in the real cache and not in the model — so the model evicted the wrong key.",[10,143,144],{},"Convert it into a plain regression test immediately, before fixing anything:",[57,146,150],{"className":147,"code":148,"language":149,"meta":62,"style":62},"language-python shiki shiki-themes github-light github-dark","def test_lru_refresh_on_get_regression():\n    cache = LruCache(capacity=3)\n    cache.put(\"0\", 0)\n    cache.put(\"1\", 0)\n    cache.get(\"0\")              # refreshes recency: the step the model missed\n    cache.put(\"2\", 0)\n    assert cache.keys() == [\"1\", \"0\", \"2\"]\n","python",[14,151,152,157,162,167,172,177,182],{"__ignoreMap":62},[66,153,154],{"class":68,"line":69},[66,155,156],{},"def test_lru_refresh_on_get_regression():\n",[66,158,159],{"class":68,"line":75},[66,160,161],{},"    cache = LruCache(capacity=3)\n",[66,163,164],{"class":68,"line":81},[66,165,166],{},"    cache.put(\"0\", 0)\n",[66,168,169],{"class":68,"line":87},[66,170,171],{},"    cache.put(\"1\", 0)\n",[66,173,174],{"class":68,"line":93},[66,175,176],{},"    cache.get(\"0\")              # refreshes recency: the step the model missed\n",[66,178,179],{"class":68,"line":99},[66,180,181],{},"    cache.put(\"2\", 0)\n",[66,183,184],{"class":68,"line":105},[66,185,186],{},"    assert cache.keys() == [\"1\", \"0\", \"2\"]\n",[10,188,189],{},"That test is deterministic, fast, and survives every future change to the machine. The state machine's job was to find the sequence; keeping the sequence is your job.",[191,192,195,352],"figure",{"className":193},[194],"diagram",[196,197,204,205,204,209,204,213,204,231,204,239,204,248,204,256,204,262,204,267,204,272,204,282,204,287,204,291,204,295,204,299,204,303,204,307,204,311,204,314,204,318,204,322,204,326,204,330,204,334,204,337,204,340,204,344,204,348],"svg",{"viewBox":198,"role":199,"ariaLabelledBy":200,"xmlns":203},"0 0 760 434","img",[201,202],"smdebug-t","smdebug-d","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","\n  ",[206,207,208],"title",{"id":201},"Reading a state machine failure",[210,211,212],"desc",{"id":202},"A vertical four-step reading order: identify the invariant or assertion that failed, find the last rule executed before it, reconstruct the state both model and system held at that point, and copy the sequence into a plain regression test.",[214,215,216,217,204],"defs",{},"\n    ",[218,219,226],"marker",{"id":220,"viewBox":221,"refX":222,"refY":223,"markerWidth":224,"markerHeight":224,"orient":225},"smdebug-a","0 0 10 10","9","5","7","auto-start-reverse",[227,228],"path",{"d":229,"fill":230},"M0 0 L10 5 L0 10 z","#e07a5f",[232,233],"rect",{"x":234,"y":234,"width":235,"height":236,"rx":237,"fill":238},"0","760","434","14","#fffdf8",[240,241,208],"text",{"x":242,"y":243,"textAnchor":244,"fontSize":245,"fontWeight":246,"fill":247},"380","30","middle","15","700","#3d405b",[232,249],{"x":250,"y":251,"width":252,"height":253,"rx":254,"fill":238,"stroke":230,"strokeWidth":255},"34","58","320","64","12","1.8",[240,257,261],{"x":258,"y":259,"textAnchor":244,"fontSize":260,"fontWeight":246,"fill":247},"194","86","12.5","the failing check",[240,263,266],{"x":258,"y":264,"textAnchor":244,"fontSize":265,"fill":247},"103","11","invariant or rule assertion",[68,268],{"x1":258,"y1":269,"x2":258,"y2":270,"stroke":230,"strokeWidth":255,"markerEnd":271},"128","149","url(#smdebug-a)",[232,273],{"x":274,"y":253,"width":275,"height":276,"rx":277,"fill":238,"stroke":278,"strokeWidth":279,"strokeDashArray":280},"384","346","52","10","#81b29a","1.5",[223,281],"4",[240,283,286],{"x":284,"y":285,"textAnchor":244,"fontSize":265,"fill":247},"557","93","Names what broke",[232,288],{"x":250,"y":289,"width":252,"height":253,"rx":254,"fill":290,"stroke":230,"strokeWidth":255},"156","#f4f1de",[240,292,294],{"x":258,"y":293,"textAnchor":244,"fontSize":260,"fontWeight":246,"fill":247},"184","the last rule",[240,296,298],{"x":258,"y":297,"textAnchor":244,"fontSize":265,"fill":247},"201","immediately before the failure",[68,300],{"x1":258,"y1":301,"x2":258,"y2":302,"stroke":230,"strokeWidth":255,"markerEnd":271},"226","247",[232,304],{"x":274,"y":305,"width":275,"height":276,"rx":277,"fill":290,"stroke":278,"strokeWidth":279,"strokeDashArray":306},"162",[223,281],[240,308,310],{"x":284,"y":309,"textAnchor":244,"fontSize":265,"fill":247},"191","Names when it broke",[232,312],{"x":250,"y":313,"width":252,"height":253,"rx":254,"fill":238,"stroke":230,"strokeWidth":255},"254",[240,315,317],{"x":258,"y":316,"textAnchor":244,"fontSize":260,"fontWeight":246,"fill":247},"282","the two states",[240,319,321],{"x":258,"y":320,"textAnchor":244,"fontSize":265,"fill":247},"299","model versus system",[68,323],{"x1":258,"y1":324,"x2":258,"y2":325,"stroke":230,"strokeWidth":255,"markerEnd":271},"324","345",[232,327],{"x":274,"y":328,"width":275,"height":276,"rx":277,"fill":238,"stroke":278,"strokeWidth":279,"strokeDashArray":329},"260",[223,281],[240,331,333],{"x":284,"y":332,"textAnchor":244,"fontSize":265,"fill":247},"289","Names why they disagree",[232,335],{"x":250,"y":336,"width":252,"height":253,"rx":254,"fill":290,"stroke":230,"strokeWidth":255},"352",[240,338,339],{"x":258,"y":242,"textAnchor":244,"fontSize":260,"fontWeight":246,"fill":247},"a plain test",[240,341,343],{"x":258,"y":342,"textAnchor":244,"fontSize":265,"fill":247},"397","paste the printed sequence",[232,345],{"x":274,"y":346,"width":275,"height":276,"rx":277,"fill":290,"stroke":278,"strokeWidth":279,"strokeDashArray":347},"358",[223,281],[240,349,351],{"x":284,"y":350,"textAnchor":244,"fontSize":265,"fill":247},"387","Keeps the finding forever",[353,354,355],"figcaption",{},"The printed sequence is executable, so the fastest path from report to diagnosis is to run it as an ordinary test.",[19,357,359],{"id":358},"why-this-works","Why this works",[10,361,362],{},"A state machine run is a sequence of rule invocations chosen by Hypothesis, with invariants evaluated after each one. When an invariant fails, everything before it succeeded — so the state was consistent until the last rule ran. That gives an exact attribution that a single large assertion at the end of a run cannot: the failure is localised to one operation applied to one known state. The printed sequence is generated from the same recorded choices, which is why it reproduces deterministically outside Hypothesis.",[191,364,366,461],{"className":365},[194],[196,367,204,372,204,375,204,378,204,381,204,383,204,389,204,394,204,398,204,402,204,406,204,411,204,414,204,417,204,420,204,422,204,425,204,428,204,431,204,435,204,438,204,441,204,444,204,448,204,451,204,454,204,458],{"viewBox":368,"role":199,"ariaLabelledBy":369,"xmlns":203},"0 0 760 270",[370,371],"smsignals-t","smsignals-d",[206,373,374],{"id":370},"What each part of the report tells you",[210,376,377],{"id":371},"A table of four parts of a state machine failure report - the falsifying sequence, the failing check, the bundle references and the explanation line - describing what each identifies.",[232,379],{"x":234,"y":234,"width":235,"height":380,"rx":237,"fill":238},"270",[240,382,374],{"x":242,"y":243,"textAnchor":244,"fontSize":245,"fontWeight":246,"fill":247},[232,384],{"x":385,"y":276,"width":386,"height":387,"rx":277,"fill":388,"stroke":247,"strokeWidth":279},"24","712","40","#f2cc8f",[240,390,393],{"x":391,"y":392,"fontSize":254,"fontWeight":246,"fill":247},"38","76","Criterion",[240,395,397],{"x":396,"y":392,"textAnchor":244,"fontSize":254,"fontWeight":246,"fill":247},"390","Identifies",[240,399,401],{"x":400,"y":392,"textAnchor":244,"fontSize":254,"fontWeight":246,"fill":247},"620","Use it to",[232,403],{"x":385,"y":404,"width":386,"height":387,"fill":290,"stroke":247,"strokeWidth":405},"92","1",[240,407,410],{"x":391,"y":408,"fontSize":409,"fill":247},"116","11.5","the call sequence",[240,412,413],{"x":396,"y":408,"textAnchor":244,"fontSize":409,"fill":247},"the minimal path",[240,415,416],{"x":400,"y":408,"textAnchor":244,"fontSize":409,"fill":247},"build a regression test",[232,418],{"x":385,"y":419,"width":386,"height":387,"fill":238,"stroke":247,"strokeWidth":405},"132",[240,421,261],{"x":391,"y":289,"fontSize":409,"fill":247},[240,423,424],{"x":396,"y":289,"textAnchor":244,"fontSize":409,"fill":247},"the broken property",[240,426,427],{"x":400,"y":289,"textAnchor":244,"fontSize":409,"fill":247},"name the bug",[232,429],{"x":385,"y":430,"width":386,"height":387,"fill":290,"stroke":247,"strokeWidth":405},"172",[240,432,434],{"x":391,"y":433,"fontSize":409,"fill":247},"196","bundle references",[240,436,437],{"x":396,"y":433,"textAnchor":244,"fontSize":409,"fill":247},"which objects were reused",[240,439,440],{"x":400,"y":433,"textAnchor":244,"fontSize":409,"fill":247},"spot aliasing bugs",[232,442],{"x":385,"y":443,"width":386,"height":387,"fill":238,"stroke":247,"strokeWidth":405},"212",[240,445,447],{"x":391,"y":446,"fontSize":409,"fill":247},"236","the explanation line",[240,449,450],{"x":396,"y":446,"textAnchor":244,"fontSize":409,"fill":247},"rule or invariant",[240,452,453],{"x":400,"y":446,"textAnchor":244,"fontSize":409,"fill":247},"attribute the failure",[68,455],{"x1":456,"y1":276,"x2":456,"y2":457,"stroke":247,"strokeWidth":405},"274","252",[68,459],{"x1":460,"y1":276,"x2":460,"y2":457,"stroke":247,"strokeWidth":405},"505",[353,462,463],{},"Together these answer what broke, when, and against which objects — which is usually the entire diagnosis.",[19,465,467],{"id":466},"making-the-model-talk","Making the model talk",[10,469,470,471,474],{},"When model and system disagree, the report shows the sequence but not the two states. Adding a ",[14,472,473],{},"__repr__"," to the model, and printing it from the failing invariant, turns a fifteen-call reproduction into an immediate answer:",[57,476,478],{"className":147,"code":477,"language":149,"meta":62,"style":62},"from hypothesis.stateful import RuleBasedStateMachine, invariant, rule\n\nclass CacheMachine(RuleBasedStateMachine):\n    def __init__(self):\n        super().__init__()\n        self.real = LruCache(capacity=3)\n        self.model = {}\n        self.order = []\n\n    def __repr__(self):\n        return f\"\u003Cmodel keys={self.order} real={self.real.keys()}>\"\n\n    @invariant()\n    def keys_agree(self):\n        assert self.real.keys() == self.order, (\n            f\"divergence: real={self.real.keys()} model={self.order}\"\n        )\n",[14,479,480,485,489,494,499,504,509,514,519,523,529,535,540,546,552,558,564],{"__ignoreMap":62},[66,481,482],{"class":68,"line":69},[66,483,484],{},"from hypothesis.stateful import RuleBasedStateMachine, invariant, rule\n",[66,486,487],{"class":68,"line":75},[66,488,115],{"emptyLinePlaceholder":114},[66,490,491],{"class":68,"line":81},[66,492,493],{},"class CacheMachine(RuleBasedStateMachine):\n",[66,495,496],{"class":68,"line":87},[66,497,498],{},"    def __init__(self):\n",[66,500,501],{"class":68,"line":93},[66,502,503],{},"        super().__init__()\n",[66,505,506],{"class":68,"line":99},[66,507,508],{},"        self.real = LruCache(capacity=3)\n",[66,510,511],{"class":68,"line":105},[66,512,513],{},"        self.model = {}\n",[66,515,516],{"class":68,"line":111},[66,517,518],{},"        self.order = []\n",[66,520,521],{"class":68,"line":118},[66,522,115],{"emptyLinePlaceholder":114},[66,524,526],{"class":68,"line":525},10,[66,527,528],{},"    def __repr__(self):\n",[66,530,532],{"class":68,"line":531},11,[66,533,534],{},"        return f\"\u003Cmodel keys={self.order} real={self.real.keys()}>\"\n",[66,536,538],{"class":68,"line":537},12,[66,539,115],{"emptyLinePlaceholder":114},[66,541,543],{"class":68,"line":542},13,[66,544,545],{},"    @invariant()\n",[66,547,549],{"class":68,"line":548},14,[66,550,551],{},"    def keys_agree(self):\n",[66,553,555],{"class":68,"line":554},15,[66,556,557],{},"        assert self.real.keys() == self.order, (\n",[66,559,561],{"class":68,"line":560},16,[66,562,563],{},"            f\"divergence: real={self.real.keys()} model={self.order}\"\n",[66,565,567],{"class":68,"line":566},17,[66,568,569],{},"        )\n",[10,571,572,573,576],{},"Putting the two sequences in the assertion message is the single highest-value line in a state machine. The failure then reads ",[14,574,575],{},"real=['1','0','2'] model=['1','2','0']",", which names the operation that ordered them differently without any further investigation.",[10,578,579],{},"Two related habits help. Keep invariants small and numerous rather than one large one, so the name of the failing invariant is itself informative. And assert the cheap structural properties — sizes, key sets, monotonic counters — rather than deep equality, because a diff of two large structures is harder to read than a single mismatched count.",[19,581,583],{"id":582},"when-the-sequence-will-not-shrink","When the sequence will not shrink",[10,585,586],{},"A twenty-call counterexample means the shrinker could not remove steps, and there are two usual reasons.",[10,588,589,592,593,596],{},[30,590,591],{},"Preconditions that reject shortened sequences."," ",[14,594,595],{},"@precondition(lambda self: self.model)"," prevents a rule from running on an empty model, which is correct — but if most rules are gated this way, a shortened sequence becomes unrunnable and the shrinker discards it. Keep preconditions as permissive as correctness allows, and prefer rules that handle the empty case gracefully over rules that refuse to run.",[10,598,599,602,603,606,607,610,611,614],{},[30,600,601],{},"State that leaks between runs."," The shrinker replays candidate sequences in the same process. If the machine's ",[14,604,605],{},"__init__"," does not fully reset the system under test — a module-level registry, a class attribute, a database that ",[14,608,609],{},"teardown"," did not clean — a shorter sequence behaves differently and the reduction fails. Implement ",[14,612,613],{},"teardown()"," on the machine, not in a fixture, so it runs after every sequence rather than once per test.",[57,616,618],{"className":147,"code":617,"language":149,"meta":62,"style":62},"class ApiMachine(RuleBasedStateMachine):\n    def teardown(self):\n        # Runs after EVERY sequence, including every shrink candidate.\n        for order_id in self.created:\n            self.client.delete(f\"\u002Forders\u002F{order_id}\")\n        self.created.clear()\n",[14,619,620,625,630,635,640,645],{"__ignoreMap":62},[66,621,622],{"class":68,"line":69},[66,623,624],{},"class ApiMachine(RuleBasedStateMachine):\n",[66,626,627],{"class":68,"line":75},[66,628,629],{},"    def teardown(self):\n",[66,631,632],{"class":68,"line":81},[66,633,634],{},"        # Runs after EVERY sequence, including every shrink candidate.\n",[66,636,637],{"class":68,"line":87},[66,638,639],{},"        for order_id in self.created:\n",[66,641,642],{"class":68,"line":93},[66,643,644],{},"            self.client.delete(f\"\u002Forders\u002F{order_id}\")\n",[66,646,647],{"class":68,"line":99},[66,648,649],{},"        self.created.clear()\n",[19,651,653],{"id":652},"instrumenting-the-machine-for-faster-diagnosis","Instrumenting the machine for faster diagnosis",[10,655,656],{},"A state machine that prints nothing until it fails makes every failure a reconstruction exercise. Three cheap instruments turn the report into a narrative.",[10,658,659,662],{},[30,660,661],{},"Log each rule as it runs."," Hypothesis prints the final sequence, but seeing the run unfold — with the state after each step — shows where the divergence began rather than where it was noticed:",[57,664,666],{"className":147,"code":665,"language":149,"meta":62,"style":62},"from hypothesis.stateful import RuleBasedStateMachine, rule\n\nclass CacheMachine(RuleBasedStateMachine):\n    def __init__(self):\n        super().__init__()\n        self.log: list[str] = []\n\n    @rule(key=st.text(min_size=1, max_size=2), value=st.integers())\n    def put(self, key, value):\n        self.real.put(key, value)\n        self.model[key] = value\n        self.log.append(f\"put({key!r}, {value}) -> {sorted(self.model)}\")\n\n    def teardown(self):\n        if self.log and len(self.log) \u003C 40:\n            print(\"\\n\".join(self.log))       # printed by pytest only on failure\n",[14,667,668,673,677,681,685,689,694,698,703,708,713,718,723,727,731,736],{"__ignoreMap":62},[66,669,670],{"class":68,"line":69},[66,671,672],{},"from hypothesis.stateful import RuleBasedStateMachine, rule\n",[66,674,675],{"class":68,"line":75},[66,676,115],{"emptyLinePlaceholder":114},[66,678,679],{"class":68,"line":81},[66,680,493],{},[66,682,683],{"class":68,"line":87},[66,684,498],{},[66,686,687],{"class":68,"line":93},[66,688,503],{},[66,690,691],{"class":68,"line":99},[66,692,693],{},"        self.log: list[str] = []\n",[66,695,696],{"class":68,"line":105},[66,697,115],{"emptyLinePlaceholder":114},[66,699,700],{"class":68,"line":111},[66,701,702],{},"    @rule(key=st.text(min_size=1, max_size=2), value=st.integers())\n",[66,704,705],{"class":68,"line":118},[66,706,707],{},"    def put(self, key, value):\n",[66,709,710],{"class":68,"line":525},[66,711,712],{},"        self.real.put(key, value)\n",[66,714,715],{"class":68,"line":531},[66,716,717],{},"        self.model[key] = value\n",[66,719,720],{"class":68,"line":537},[66,721,722],{},"        self.log.append(f\"put({key!r}, {value}) -> {sorted(self.model)}\")\n",[66,724,725],{"class":68,"line":542},[66,726,115],{"emptyLinePlaceholder":114},[66,728,729],{"class":68,"line":548},[66,730,629],{},[66,732,733],{"class":68,"line":554},[66,734,735],{},"        if self.log and len(self.log) \u003C 40:\n",[66,737,738],{"class":68,"line":560},[66,739,740],{},"            print(\"\\n\".join(self.log))       # printed by pytest only on failure\n",[10,742,743,592,746,749,750,40],{},[30,744,745],{},"Assert the cheap invariant often and the expensive one rarely.",[14,747,748],{},"@invariant()"," runs after every rule, so a full deep-equality check between model and system on every step is slow enough to reduce the number of sequences explored. Check sizes and key sets every step, and full contents in ",[14,751,609],{},[10,753,754,757,758,761],{},[30,755,756],{},"Name the bundle contents."," When rules consume values produced by other rules, printing the bundle at each step turns ",[14,759,760],{},"v3"," in the report into something meaningful — usually the exact aliasing bug the machine found.",[10,763,764],{},"The combination costs a few lines and converts the typical debugging session from \"reconstruct fifteen calls by hand\" into \"read the log until the two states diverge\".",[19,766,768],{"id":767},"edge-cases-and-failure-modes","Edge cases and failure modes",[24,770,771,785,791,804,810,816],{},[27,772,773,784],{},[30,774,775,776,779,780,783],{},"Bundles print as ",[14,777,778],{},"v1",", ",[14,781,782],{},"v2"," placeholders."," Those are references to values produced by earlier rules; the report is still executable because the variables are assigned in the printed code.",[27,786,787,790],{},[30,788,789],{},"An exception inside a rule stops the sequence there."," The report shows the rules up to and including the raising one, which is exactly the attribution you want — no invariant needed.",[27,792,793,799,800,803],{},[30,794,795,798],{},[14,796,797],{},"stateful_step_count"," bounds the sequence length."," A bug that needs more operations than the limit will never be found; raise it rather than raising ",[14,801,802],{},"max_examples"," when the bug is depth-related.",[27,805,806,809],{},[30,807,808],{},"Invariants run on the initial state too",", before any rule, so an invariant that assumes setup has happened fails immediately and confusingly.",[27,811,812,815],{},[30,813,814],{},"Machines are not thread-safe by design."," A rule that spawns threads makes the sequence non-deterministic and the shrink useless; model concurrency as explicit rules instead.",[27,817,818,824,825,827],{},[30,819,12,820,823],{},[14,821,822],{},"Flaky"," report from a state machine"," means the same sequence produced different outcomes — almost always leaked state, and almost always fixed by a real ",[14,826,609],{},".\nOne structural note about where machines belong in a suite. A state machine is slower than an ordinary test by one or two orders of magnitude, because each example is a sequence of operations and there are many examples. Mark them and exclude them from the pre-commit run, then give them a generous budget in a scheduled job — the same split recommended for expensive properties. A machine running twenty-five examples on every pull request is both too slow to be welcome and too shallow to find anything.",[191,829,831,900],{"className":830},[194],[196,832,204,836,204,839,204,842,204,844,204,846,204,848,204,850,204,853,204,856,204,858,204,860,204,862,204,865,204,867,204,869,204,872,204,875,204,877,204,880,204,882,204,885,204,887,204,890,204,893,204,896,204,898],{"viewBox":368,"role":199,"ariaLabelledBy":833,"xmlns":203},[834,835],"smbudget-t","smbudget-d",[206,837,838],{"id":834},"Two budgets for a state machine",[210,840,841],{"id":835},"A table comparing a pull-request budget and a nightly budget for a state machine, across max_examples, step count, total operations and what each is expected to catch.",[232,843],{"x":234,"y":234,"width":235,"height":380,"rx":237,"fill":238},[240,845,838],{"x":242,"y":243,"textAnchor":244,"fontSize":245,"fontWeight":246,"fill":247},[232,847],{"x":385,"y":276,"width":386,"height":387,"rx":277,"fill":388,"stroke":247,"strokeWidth":279},[240,849,393],{"x":391,"y":392,"fontSize":254,"fontWeight":246,"fill":247},[240,851,852],{"x":396,"y":392,"textAnchor":244,"fontSize":254,"fontWeight":246,"fill":247},"Pull request",[240,854,855],{"x":400,"y":392,"textAnchor":244,"fontSize":254,"fontWeight":246,"fill":247},"Nightly",[232,857],{"x":385,"y":404,"width":386,"height":387,"fill":290,"stroke":247,"strokeWidth":405},[240,859,802],{"x":391,"y":408,"fontSize":409,"fill":247},[240,861,277],{"x":396,"y":408,"textAnchor":244,"fontSize":409,"fill":247},[240,863,864],{"x":400,"y":408,"textAnchor":244,"fontSize":409,"fill":247},"200",[232,866],{"x":385,"y":419,"width":386,"height":387,"fill":238,"stroke":247,"strokeWidth":405},[240,868,797],{"x":391,"y":289,"fontSize":409,"fill":247},[240,870,871],{"x":396,"y":289,"textAnchor":244,"fontSize":409,"fill":247},"20",[240,873,874],{"x":400,"y":289,"textAnchor":244,"fontSize":409,"fill":247},"60",[232,876],{"x":385,"y":430,"width":386,"height":387,"fill":290,"stroke":247,"strokeWidth":405},[240,878,879],{"x":391,"y":433,"fontSize":409,"fill":247},"operations per run",[240,881,864],{"x":396,"y":433,"textAnchor":244,"fontSize":409,"fill":247},[240,883,884],{"x":400,"y":433,"textAnchor":244,"fontSize":409,"fill":247},"12,000",[232,886],{"x":385,"y":443,"width":386,"height":387,"fill":238,"stroke":247,"strokeWidth":405},[240,888,889],{"x":391,"y":446,"fontSize":409,"fill":247},"catches",[240,891,892],{"x":396,"y":446,"textAnchor":244,"fontSize":409,"fill":247},"obvious regressions",[240,894,895],{"x":400,"y":446,"textAnchor":244,"fontSize":409,"fill":247},"deep interleavings",[68,897],{"x1":456,"y1":276,"x2":456,"y2":457,"stroke":247,"strokeWidth":405},[68,899],{"x1":460,"y1":276,"x2":460,"y2":457,"stroke":247,"strokeWidth":405},[353,901,902],{},"Raising the step count matters more than raising the example count: most state bugs need a long sequence, not many short ones.",[19,904,906],{"id":905},"frequently-asked-questions","Frequently Asked Questions",[10,908,909,912],{},[30,910,911],{},"How do I turn a state machine failure into a normal test?","\nCopy the printed call sequence into a plain test function. Hypothesis prints it as executable Python, so pasting it produces a deterministic regression test that runs in milliseconds.",[10,914,915,918],{},[30,916,917],{},"Why does the failure name an invariant rather than a rule?","\nBecause invariants run after every rule, so the first one to notice a broken state reports the failure. The rule that caused it is the last one printed before the invariant fired.",[10,920,921,924,925,928],{},[30,922,923],{},"Why will my state machine failure not shrink?","\nUsually a precondition that rejects most shortened sequences, or state that leaks between runs so a shorter sequence no longer reproduces. Both make the shrinker's candidates look healthy.\n",[30,926,927],{},"Can I run a state machine under pytest-xdist?","\nYes — each machine test is an ordinary test item and distributes normally. The caveat is the same as for any test touching shared resources: the machine's system under test must be partitioned per worker, or two workers will drive the same database and produce failures that are artefacts of the parallelism rather than of the model.",[10,930,931,934],{},[30,932,933],{},"What if the model is the thing that is wrong?","\nThat is a normal outcome and a useful one: a divergence proves the two implementations disagree, not which one is correct. Read the specification before changing either. A model corrected to match a buggy system is worse than no model, because every future run now agrees with the bug.",[19,936,938],{"id":937},"related","Related",[24,940,941,947,954,961],{},[27,942,943,946],{},[45,944,945],{"href":47},"Stateful and model-based testing"," — designing the model these failures compare against.",[27,948,949,953],{},[45,950,952],{"href":951},"\u002Fproperty-based-fuzz-testing-strategies\u002Fstateful-and-model-based-testing\u002Fmodeling-rest-apis-as-state-machines\u002F","Modeling REST APIs as state machines"," — teardown and determinism for machines that talk to a service.",[27,955,956,960],{},[45,957,959],{"href":958},"\u002Fproperty-based-fuzz-testing-strategies\u002Fadvanced-property-based-testing\u002Fwhy-hypothesis-shrinking-stalls-and-how-to-fix-it\u002F","Why Hypothesis shrinking stalls and how to fix it"," — the general form of an unshrinkable counterexample.",[27,962,963,967],{},[45,964,966],{"href":965},"\u002Fproperty-based-fuzz-testing-strategies\u002Fhypothesis-framework-fundamentals\u002Freproducing-failures-with-example-and-the-database\u002F","Reproducing Hypothesis failures with @example"," — keeping the finding after the fix.",[10,969,970,971],{},"← Back to ",[45,972,973],{"href":47},"Stateful and Model-Based Testing",[975,976,977],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":62,"searchDepth":75,"depth":75,"links":979},[980,981,982,983,984,985,986,987,988],{"id":21,"depth":75,"text":22},{"id":51,"depth":75,"text":52},{"id":358,"depth":75,"text":359},{"id":466,"depth":75,"text":467},{"id":582,"depth":75,"text":583},{"id":652,"depth":75,"text":653},{"id":767,"depth":75,"text":768},{"id":905,"depth":75,"text":906},{"id":937,"depth":75,"text":938},"Read and reduce a Hypothesis state machine failure: the printed call sequence, bundle references, invariant attribution, preconditions that block shrinking, and model repr.","md",{"slug":992,"type":993,"breadcrumb":994,"datePublished":995,"dateModified":995,"faq":996,"howto":1003},"debugging-rulebasedstatemachine-failures","article","State Machine Failures","2026-08-01",[997,999,1001],{"q":911,"a":998},"Copy the printed call sequence into a plain test function. Hypothesis prints it as executable Python, so pasting it produces a deterministic regression test that runs in milliseconds.",{"q":917,"a":1000},"Because invariants run after every rule, so the first one to notice a broken state reports the failure. The rule that caused it is the last one printed before the invariant fired.",{"q":923,"a":1002},"Usually a precondition that rejects most shortened sequences, or state that leaks between runs so a shorter sequence no longer reproduces. Both make the shrinker's candidates look healthy.",{"name":1004,"description":1005,"steps":1006},"How to debug a RuleBasedStateMachine failure","Read the reported sequence, attribute the failure to a rule, and turn it into a regression test.",[1007,1010,1013,1016],{"name":1008,"text":1009},"Read the printed sequence from the bottom","The last rule before the failing invariant is the one that broke the state.",{"name":1011,"text":1012},"Copy the sequence into a plain test","The output is executable Python and makes the failure deterministic.",{"name":1014,"text":1015},"Print the model state at the point of failure","Compare model and system so the divergence names the lying operation.",{"name":1017,"text":1018},"Remove preconditions that block shrinking","Check that shortened sequences are still runnable so the shrinker can reduce them.","\u002Fproperty-based-fuzz-testing-strategies\u002Fstateful-and-model-based-testing\u002Fdebugging-rulebasedstatemachine-failures",{"title":5,"description":989},"property-based-fuzz-testing-strategies\u002Fstateful-and-model-based-testing\u002Fdebugging-rulebasedstatemachine-failures\u002Findex","aCjKXQGH8ug61J4hY0EFcRhnyKap7ikMvc8DGm_udXo",1785613404504]