[{"data":1,"prerenderedAt":928},["ShallowReactive",2],{"page-\u002Fadvanced-pytest-architecture-configuration\u002Fadvanced-parametrization-techniques\u002Fgenerating-cases-with-pytest-generate-tests\u002F":3},{"id":4,"title":5,"body":6,"description":892,"extension":893,"meta":894,"navigation":97,"path":924,"seo":925,"stem":926,"__hash__":927},"content\u002Fadvanced-pytest-architecture-configuration\u002Fadvanced-parametrization-techniques\u002Fgenerating-cases-with-pytest-generate-tests\u002Findex.md","Generating Cases with pytest_generate_tests",{"type":7,"value":8,"toc":880},"minimark",[9,21,24,29,50,54,57,209,246,254,257,409,413,427,442,446,489,493,496,552,599,602,676,680,690,704,707,766,770,777,796,800,803,810,813,817,823,832,838,842,871,876],[10,11,12,16,17,20],"p",{},[13,14,15],"code",{},"@pytest.mark.parametrize"," covers the common case: a fixed list of inputs, written next to the test that uses them. It cannot express cases that are not known when the test is written — one per JSON file in a fixtures directory, a set selected by a command-line option, a matrix that depends on which optional dependencies are installed. ",[13,18,19],{},"pytest_generate_tests"," is the hook for that. It runs during collection, sees each test function's requested arguments, and can parametrize them with anything computable at that moment.",[10,22,23],{},"The hook is powerful enough to become a maintenance hazard, because it runs for every test function in scope and silently shapes what gets collected. Used with a guard on the argument name, cheap loading and stable ids, it is the cleanest way to turn a directory of data files into a readable set of test cases.",[25,26,28],"h2",{"id":27},"prerequisites","Prerequisites",[30,31,32,39,47],"ul",{},[33,34,35,38],"li",{},[13,36,37],{},"pytest >= 8.0",".",[33,40,41,42,38],{},"The static parametrization techniques in ",[43,44,46],"a",{"href":45},"\u002Fadvanced-pytest-architecture-configuration\u002Fadvanced-parametrization-techniques\u002F","advanced parametrization techniques",[33,48,49],{},"Case data in a form cheap to enumerate — a directory of files, a small manifest, a configuration value.",[25,51,53],{"id":52},"solution","Solution",[10,55,56],{},"Generate one test case per file in a fixtures directory, guarded so only tests that ask for it are affected.",[58,59,64],"pre",{"className":60,"code":61,"language":62,"meta":63,"style":63},"language-python shiki shiki-themes github-light github-dark","# tests\u002Fparsers\u002Fconftest.py\nimport json\nfrom functools import lru_cache\nfrom pathlib import Path\n\nCASES_DIR = Path(__file__).parent \u002F \"cases\"\n\n\n@lru_cache(maxsize=None)\ndef _load_cases():\n    # Loaded once per session, not once per test function.\n    return sorted(CASES_DIR.glob(\"*.json\"))\n\n\ndef pytest_generate_tests(metafunc):\n    # Guard: only tests that request `case_file` are parametrized.\n    if \"case_file\" not in metafunc.fixturenames:\n        return\n    files = _load_cases()\n    metafunc.parametrize(\n        \"case_file\",\n        files,\n        ids=[f.stem for f in files],     # stable, readable: the file name\n    )\n","python","",[13,65,66,74,80,86,92,99,105,110,115,121,127,133,139,144,149,155,161,167,173,179,185,191,197,203],{"__ignoreMap":63},[67,68,71],"span",{"class":69,"line":70},"line",1,[67,72,73],{},"# tests\u002Fparsers\u002Fconftest.py\n",[67,75,77],{"class":69,"line":76},2,[67,78,79],{},"import json\n",[67,81,83],{"class":69,"line":82},3,[67,84,85],{},"from functools import lru_cache\n",[67,87,89],{"class":69,"line":88},4,[67,90,91],{},"from pathlib import Path\n",[67,93,95],{"class":69,"line":94},5,[67,96,98],{"emptyLinePlaceholder":97},true,"\n",[67,100,102],{"class":69,"line":101},6,[67,103,104],{},"CASES_DIR = Path(__file__).parent \u002F \"cases\"\n",[67,106,108],{"class":69,"line":107},7,[67,109,98],{"emptyLinePlaceholder":97},[67,111,113],{"class":69,"line":112},8,[67,114,98],{"emptyLinePlaceholder":97},[67,116,118],{"class":69,"line":117},9,[67,119,120],{},"@lru_cache(maxsize=None)\n",[67,122,124],{"class":69,"line":123},10,[67,125,126],{},"def _load_cases():\n",[67,128,130],{"class":69,"line":129},11,[67,131,132],{},"    # Loaded once per session, not once per test function.\n",[67,134,136],{"class":69,"line":135},12,[67,137,138],{},"    return sorted(CASES_DIR.glob(\"*.json\"))\n",[67,140,142],{"class":69,"line":141},13,[67,143,98],{"emptyLinePlaceholder":97},[67,145,147],{"class":69,"line":146},14,[67,148,98],{"emptyLinePlaceholder":97},[67,150,152],{"class":69,"line":151},15,[67,153,154],{},"def pytest_generate_tests(metafunc):\n",[67,156,158],{"class":69,"line":157},16,[67,159,160],{},"    # Guard: only tests that request `case_file` are parametrized.\n",[67,162,164],{"class":69,"line":163},17,[67,165,166],{},"    if \"case_file\" not in metafunc.fixturenames:\n",[67,168,170],{"class":69,"line":169},18,[67,171,172],{},"        return\n",[67,174,176],{"class":69,"line":175},19,[67,177,178],{},"    files = _load_cases()\n",[67,180,182],{"class":69,"line":181},20,[67,183,184],{},"    metafunc.parametrize(\n",[67,186,188],{"class":69,"line":187},21,[67,189,190],{},"        \"case_file\",\n",[67,192,194],{"class":69,"line":193},22,[67,195,196],{},"        files,\n",[67,198,200],{"class":69,"line":199},23,[67,201,202],{},"        ids=[f.stem for f in files],     # stable, readable: the file name\n",[67,204,206],{"class":69,"line":205},24,[67,207,208],{},"    )\n",[58,210,212],{"className":60,"code":211,"language":62,"meta":63,"style":63},"# tests\u002Fparsers\u002Ftest_parser.py\nimport json\n\n\ndef test_parser_matches_expected_output(case_file, parser):\n    case = json.loads(case_file.read_text())\n    assert parser.parse(case[\"input\"]) == case[\"expected\"]\n",[13,213,214,219,223,227,231,236,241],{"__ignoreMap":63},[67,215,216],{"class":69,"line":70},[67,217,218],{},"# tests\u002Fparsers\u002Ftest_parser.py\n",[67,220,221],{"class":69,"line":76},[67,222,79],{},[67,224,225],{"class":69,"line":82},[67,226,98],{"emptyLinePlaceholder":97},[67,228,229],{"class":69,"line":88},[67,230,98],{"emptyLinePlaceholder":97},[67,232,233],{"class":69,"line":94},[67,234,235],{},"def test_parser_matches_expected_output(case_file, parser):\n",[67,237,238],{"class":69,"line":101},[67,239,240],{},"    case = json.loads(case_file.read_text())\n",[67,242,243],{"class":69,"line":107},[67,244,245],{},"    assert parser.parse(case[\"input\"]) == case[\"expected\"]\n",[58,247,252],{"className":248,"code":250,"language":251,"meta":63},[249],"language-text","tests\u002Fparsers\u002Ftest_parser.py::test_parser_matches_expected_output[empty-object] PASSED\ntests\u002Fparsers\u002Ftest_parser.py::test_parser_matches_expected_output[nested-arrays] PASSED\ntests\u002Fparsers\u002Ftest_parser.py::test_parser_matches_expected_output[unicode-keys] FAILED\n","text",[13,253,250],{"__ignoreMap":63},[10,255,256],{},"Adding a case is now adding a file. No test code changes, and the new case appears in the report under its file name.",[258,259,262,405],"figure",{"className":260},[261],"diagram",[263,264,271,272,271,276,271,280,271,298,271,306,271,314,271,323,271,329,271,334,271,338,271,342,271,348,271,355,271,358,271,362,271,365,271,369,271,373,271,379,271,383,271,387,271,390,271,393,271,400],"svg",{"viewBox":265,"role":266,"ariaLabelledBy":267,"xmlns":270},"0 0 820 262","img",[268,269],"pgt-t","pgt-d","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","\n  ",[273,274,275],"title",{"id":268},"From a directory of files to named test items",[277,278,279],"desc",{"id":269},"During collection, pytest calls the hook for each test function. The hook checks whether the function requests case_file, loads the cached list of JSON files, and parametrizes the argument with one value per file and the file stem as the id. Collection then produces one test item per file, each named after its file.",[281,282,283,284,271],"defs",{},"\n    ",[285,286,293],"marker",{"id":287,"viewBox":288,"refX":289,"refY":290,"markerWidth":291,"markerHeight":291,"orient":292},"pgt-a","0 0 10 10","9","5","7","auto-start-reverse",[294,295],"path",{"d":296,"fill":297},"M0 0 L10 5 L0 10 z","#3d405b",[299,300],"rect",{"x":301,"y":301,"width":302,"height":303,"rx":304,"fill":305},"0","820","262","14","#fffdf8",[251,307,313],{"x":308,"y":309,"textAnchor":310,"fontSize":311,"fontWeight":312,"fill":297},"410","28","middle","16","700","Cases are data; the hook turns data into items",[299,315],{"x":316,"y":317,"width":318,"height":319,"rx":320,"fill":321,"stroke":297,"strokeWidth":322},"26","70","180","120","11","#f4f1de","1.6",[251,324,328],{"x":325,"y":326,"textAnchor":310,"fontSize":327,"fontWeight":312,"fill":297},"116","96","12","cases\u002F",[251,330,333],{"x":331,"y":332,"fontSize":320,"fill":297},"44","122","empty-object.json",[251,335,337],{"x":331,"y":336,"fontSize":320,"fill":297},"142","nested-arrays.json",[251,339,341],{"x":331,"y":340,"fontSize":320,"fill":297},"162","unicode-keys.json",[69,343],{"x1":344,"y1":345,"x2":346,"y2":345,"stroke":297,"strokeWidth":322,"markerEnd":347},"210","130","246","url(#pgt-a)",[299,349],{"x":350,"y":317,"width":351,"height":319,"rx":320,"fill":352,"stroke":353,"strokeWidth":354},"252","240","#f7f0da","#f2cc8f","2",[251,356,19],{"x":357,"y":326,"textAnchor":310,"fontSize":327,"fontWeight":312,"fill":297},"372",[251,359,361],{"x":360,"y":332,"fontSize":320,"fill":297},"270","requests case_file? → yes",[251,363,364],{"x":360,"y":336,"fontSize":320,"fill":297},"load cached file list",[251,366,368],{"x":360,"y":340,"fontSize":320,"fill":367},"#8a5a00","parametrize with ids=stem",[69,370],{"x1":371,"y1":345,"x2":372,"y2":345,"stroke":297,"strokeWidth":322,"markerEnd":347},"496","532",[299,374],{"x":375,"y":317,"width":376,"height":319,"rx":320,"fill":377,"stroke":378,"strokeWidth":354},"538","256","#e6f0ea","#81b29a",[251,380,382],{"x":381,"y":326,"textAnchor":310,"fontSize":327,"fontWeight":312,"fill":297},"666","collected items",[251,384,386],{"x":385,"y":332,"fontSize":320,"fill":297},"556","test_parser[empty-object]",[251,388,389],{"x":385,"y":336,"fontSize":320,"fill":297},"test_parser[nested-arrays]",[251,391,392],{"x":385,"y":340,"fontSize":320,"fill":297},"test_parser[unicode-keys]",[299,394],{"x":316,"y":395,"width":396,"height":397,"rx":289,"fill":305,"stroke":398,"strokeWidth":399},"208","768","36","rgba(61,64,91,0.35)","1.4",[251,401,404],{"x":308,"y":402,"textAnchor":310,"fontSize":403,"fill":297},"231","11.5","Adding a case means adding a file; the test body never changes.",[406,407,408],"figcaption",{},"The ids come from the file names, so the report says which case failed in the same words the fixtures directory uses.",[25,410,412],{"id":411},"why-this-works","Why this works",[10,414,415,416,419,420,422,423,426],{},"During collection pytest builds a ",[13,417,418],{},"Metafunc"," object for every test function — describing its name, module, requested arguments and markers — and calls every ",[13,421,19],{}," implementation in scope with it. A call to ",[13,424,425],{},"metafunc.parametrize"," inside the hook has exactly the same effect as the decorator would have had, generating one item per value and appending the id to the name.",[10,428,429,430,433,434,437,438,441],{},"Because the hook sees ",[13,431,432],{},"metafunc.fixturenames",", it can decide per test whether to act. That is what makes the guard work: tests that do not mention ",[13,435,436],{},"case_file"," pass through untouched, so the hook can live in a ",[13,439,440],{},"conftest.py"," without affecting unrelated tests beneath it.",[25,443,445],{"id":444},"edge-cases-and-failure-modes","Edge cases and failure modes",[30,447,448,458,468,474,480],{},[33,449,450,454,455,457],{},[451,452,453],"strong",{},"No guard."," A hook that parametrizes unconditionally breaks every test in scope that does not accept the argument. Always check ",[13,456,432],{}," first.",[33,459,460,463,464,467],{},[451,461,462],{},"Expensive loading."," The hook runs once per test function, including during ",[13,465,466],{},"pytest -k one_unrelated_test",". Cache the loaded data at module level.",[33,469,470,473],{},[451,471,472],{},"Unstable ids."," Ids built from dictionary ordering or object reprs change between runs. Derive them from file names or explicit keys.",[33,475,476,479],{},[451,477,478],{},"An empty case list."," Parametrizing with an empty list produces a single skipped item with a reason, which is easy to miss. Assert the directory is non-empty in the hook, or accept the skip deliberately.",[33,481,482,485,486,488],{},[451,483,484],{},"Combining with the decorator."," The hook and ",[13,487,15],{}," can target different arguments of the same test and produce a product. Targeting the same argument twice raises.",[25,490,492],{"id":491},"letting-the-command-line-choose-the-cases","Letting the command line choose the cases",[10,494,495],{},"The second common use of the hook is selecting cases from configuration rather than from files. A suite that tests against several external services, several data sets or several optional backends often wants to run all of them nightly and only one on a developer's machine. A command-line option read inside the hook expresses that without duplicating any test.",[58,497,499],{"className":60,"code":498,"language":62,"meta":63,"style":63},"# conftest.py\ndef pytest_addoption(parser):\n    parser.addoption(\"--backend\", action=\"append\", default=[],\n                     help=\"backend to test against (repeatable); default: memory\")\n\n\ndef pytest_generate_tests(metafunc):\n    if \"backend\" not in metafunc.fixturenames:\n        return\n    chosen = metafunc.config.getoption(\"backend\") or [\"memory\"]\n    metafunc.parametrize(\"backend\", chosen, ids=chosen)\n",[13,500,501,506,511,516,521,525,529,533,538,542,547],{"__ignoreMap":63},[67,502,503],{"class":69,"line":70},[67,504,505],{},"# conftest.py\n",[67,507,508],{"class":69,"line":76},[67,509,510],{},"def pytest_addoption(parser):\n",[67,512,513],{"class":69,"line":82},[67,514,515],{},"    parser.addoption(\"--backend\", action=\"append\", default=[],\n",[67,517,518],{"class":69,"line":88},[67,519,520],{},"                     help=\"backend to test against (repeatable); default: memory\")\n",[67,522,523],{"class":69,"line":94},[67,524,98],{"emptyLinePlaceholder":97},[67,526,527],{"class":69,"line":101},[67,528,98],{"emptyLinePlaceholder":97},[67,530,531],{"class":69,"line":107},[67,532,154],{},[67,534,535],{"class":69,"line":112},[67,536,537],{},"    if \"backend\" not in metafunc.fixturenames:\n",[67,539,540],{"class":69,"line":117},[67,541,172],{},[67,543,544],{"class":69,"line":123},[67,545,546],{},"    chosen = metafunc.config.getoption(\"backend\") or [\"memory\"]\n",[67,548,549],{"class":69,"line":129},[67,550,551],{},"    metafunc.parametrize(\"backend\", chosen, ids=chosen)\n",[58,553,557],{"className":554,"code":555,"language":556,"meta":63,"style":63},"language-bash shiki shiki-themes github-light github-dark","pytest -q                                          # memory only, fast\npytest -q --backend memory --backend sql --backend redis   # the full matrix\n","bash",[13,558,559,573],{"__ignoreMap":63},[67,560,561,565,569],{"class":69,"line":70},[67,562,564],{"class":563},"sScJk","pytest",[67,566,568],{"class":567},"sj4cs"," -q",[67,570,572],{"class":571},"sJ8bj","                                          # memory only, fast\n",[67,574,575,577,579,582,586,588,591,593,596],{"class":69,"line":76},[67,576,564],{"class":563},[67,578,568],{"class":567},[67,580,581],{"class":567}," --backend",[67,583,585],{"class":584},"sZZnC"," memory",[67,587,581],{"class":567},[67,589,590],{"class":584}," sql",[67,592,581],{"class":567},[67,594,595],{"class":584}," redis",[67,597,598],{"class":571},"   # the full matrix\n",[10,600,601],{},"This is the one situation where dynamic parametrization is clearly better than fixture params. A parametrized fixture always produces every variant; the hook can produce whichever variants the invocation asked for, so the same test file serves the fast local loop and the thorough nightly run without a single conditional in the tests.",[258,603,605,673],{"className":604},[261],[263,606,271,611,271,614,271,617,271,621,271,626,271,631,271,636,271,640,271,644,271,649,271,653,271,656,271,660,271,664,271,667,271,670],{"viewBox":607,"role":266,"ariaLabelledBy":608,"xmlns":270},"0 0 800 236",[609,610],"opt-t","opt-d",[273,612,613],{"id":609},"One test file, different matrices per invocation",[277,615,616],{"id":610},"The same test file collected under two invocations. With no options, the hook parametrizes with the memory backend only, producing a fast local run. With three backend options, the hook produces three items per test for the nightly job. No test code differs between the two.",[299,618],{"x":301,"y":301,"width":619,"height":620,"rx":304,"fill":305},"800","236",[251,622,625],{"x":623,"y":309,"textAnchor":310,"fontSize":624,"fontWeight":312,"fill":297},"400","15.5","The invocation decides the breadth",[299,627],{"x":316,"y":628,"width":629,"height":630,"rx":327,"fill":377,"stroke":378,"strokeWidth":354},"50","360","164",[251,632,564],{"x":633,"y":634,"textAnchor":310,"fontSize":635,"fontWeight":312,"fill":297},"206","76","12.5",[251,637,639],{"x":331,"y":638,"fontSize":320,"fill":297},"106","test_save[memory]",[251,641,643],{"x":331,"y":642,"fontSize":320,"fill":297},"128","test_load[memory]",[251,645,648],{"x":331,"y":646,"fontSize":320,"fontWeight":312,"fill":647},"166","#2a5f49","2 items · seconds",[251,650,652],{"x":331,"y":651,"fontSize":320,"fill":297},"188","the developer's loop",[299,654],{"x":655,"y":628,"width":629,"height":630,"rx":327,"fill":352,"stroke":353,"strokeWidth":354},"414",[251,657,659],{"x":658,"y":634,"textAnchor":310,"fontSize":635,"fontWeight":312,"fill":297},"594","pytest --backend × 3",[251,661,663],{"x":662,"y":638,"fontSize":320,"fill":297},"432","test_save[memory|sql|redis]",[251,665,666],{"x":662,"y":642,"fontSize":320,"fill":297},"test_load[memory|sql|redis]",[251,668,669],{"x":662,"y":646,"fontSize":320,"fontWeight":312,"fill":367},"6 items · minutes",[251,671,672],{"x":662,"y":651,"fontSize":320,"fill":297},"the nightly job",[406,674,675],{},"A parametrized fixture cannot do this: it always produces every variant. The hook produces exactly what was asked for.",[25,677,679],{"id":678},"collection-cost-and-how-to-keep-it-flat","Collection cost, and how to keep it flat",[10,681,682,683,686,687,689],{},"Because the hook runs during collection, its cost is paid on every invocation — including the ones that select a single unrelated test with ",[13,684,685],{},"-k",". A hook that parses a hundred JSON files on each call, in a ",[13,688,440],{}," covering three hundred test functions, parses thirty thousand files before any test runs. The symptom is a suite that takes twenty seconds to start and a developer who stops running tests individually because \"collection is slow\".",[10,691,692,693,695,696,699,700,703],{},"Three habits keep it flat. The guard on ",[13,694,432],{}," means the hook does nothing for the functions that do not ask for the argument, which is usually most of them. Caching the loaded data at module level — ",[13,697,698],{},"functools.lru_cache"," on the loader, as above — means the files are read once per session rather than once per function. And enumerating cheaply before loading expensively means the ids can come from file names while the file contents are read only inside the test, where they are needed; ",[13,701,702],{},"sorted(CASES_DIR.glob(\"*.json\"))"," touches only directory entries.",[10,705,706],{},"With all three in place, the hook's collection cost is a directory listing, and the test's own cost includes reading exactly one file. That split is worth preserving as the case count grows: a thousand cases should cost a thousand small reads spread across the run, not a thousand reads concentrated at the start of every invocation.",[258,708,710,763],{"className":709},[261],[263,711,271,715,271,718,271,721,271,723,271,726,271,730,271,733,271,736,271,739,271,743,271,746,271,748,271,751,271,754,271,757,271,760],{"viewBox":607,"role":266,"ariaLabelledBy":712,"xmlns":270},[713,714],"ccost-t","ccost-d",[273,716,717],{"id":713},"Where the hook's work happens",[277,719,720],{"id":714},"Two arrangements. In the naive hook, every call parses every case file during collection, so selecting one unrelated test still pays for all of them. In the disciplined hook, collection only lists file names from a cached directory scan, and each test reads its own file when it runs.",[299,722],{"x":301,"y":301,"width":619,"height":620,"rx":304,"fill":305},[251,724,725],{"x":623,"y":309,"textAnchor":310,"fontSize":624,"fontWeight":312,"fill":297},"List during collection; read during the test",[299,727],{"x":316,"y":628,"width":629,"height":630,"rx":327,"fill":728,"stroke":729,"strokeWidth":354},"#fbe9e3","#e07a5f",[251,731,732],{"x":633,"y":634,"textAnchor":310,"fontSize":635,"fontWeight":312,"fill":297},"naive hook",[251,734,735],{"x":331,"y":638,"fontSize":320,"fill":297},"every call parses every file",[251,737,738],{"x":331,"y":642,"fontSize":320,"fill":297},"no guard, no cache",[251,740,742],{"x":331,"y":646,"fontSize":320,"fontWeight":312,"fill":741},"#8f3d22","-k one_test still pays",[251,744,745],{"x":331,"y":651,"fontSize":320,"fill":297},"for thousands of parses",[299,747],{"x":655,"y":628,"width":629,"height":630,"rx":327,"fill":377,"stroke":378,"strokeWidth":354},[251,749,750],{"x":658,"y":634,"textAnchor":310,"fontSize":635,"fontWeight":312,"fill":297},"disciplined hook",[251,752,753],{"x":662,"y":638,"fontSize":320,"fill":297},"guard on the argument name",[251,755,756],{"x":662,"y":642,"fontSize":320,"fill":297},"cached directory listing",[251,758,759],{"x":662,"y":646,"fontSize":320,"fontWeight":312,"fill":647},"collection: one listing",[251,761,762],{"x":662,"y":651,"fontSize":320,"fill":297},"each test reads its own file",[406,764,765],{},"The ids come from names, the data comes from contents, and only the second needs to be read by the test that uses it.",[25,767,769],{"id":768},"keeping-the-hook-discoverable","Keeping the hook discoverable",[10,771,772,773,776],{},"The hook's weakness is that it is invisible from the test: a reader of ",[13,774,775],{},"test_parser_matches_expected_output(case_file, parser)"," sees an argument that is neither a fixture defined anywhere obvious nor a decorator on the function. Two conventions remove the mystery.",[10,778,779,780,782,783,785,786,788,789,788,792,795],{},"Put the hook in the ",[13,781,440],{}," of the directory that uses it, never at the root, so the search space for \"where does ",[13,784,436],{}," come from\" is one file. And give the argument a name that is obviously data rather than infrastructure — ",[13,787,436],{},", ",[13,790,791],{},"sample",[13,793,794],{},"fixture_document"," — so readers do not go looking for a fixture of that name. A one-line comment on the test pointing at the hook costs nothing and saves the next reader the search entirely, which is the right trade for a mechanism that is powerful precisely because it is indirect.",[25,797,799],{"id":798},"golden-files-and-updating-expectations","Golden files and updating expectations",[10,801,802],{},"Directory-driven cases pair naturally with golden files: each case stores an input and the expected output, and the test compares the two. The awkward moment comes when the expected output legitimately changes — a formatter improves, a serializer adds a field — and dozens of golden files need updating.",[10,804,805,806,809],{},"A small option makes that deliberate rather than tedious. An ",[13,807,808],{},"--update-golden"," flag, read in the test or a fixture, rewrites the expected output from the actual output instead of asserting, and the resulting diff is reviewed like any other change. The discipline that keeps this safe is that updating is never the default: a normal run always asserts, and regenerating expectations is an explicit act whose result a human reads in the pull request.",[10,811,812],{},"Without that flag teams tend to regenerate golden files with ad-hoc scripts, and the regenerated files are reviewed less carefully than they should be, because the script feels like tooling rather than a change in behaviour. Putting the update path inside the test suite, next to the comparison it replaces, keeps the two in step and makes the review of an expectation change look exactly like the review of the code change that caused it.",[25,814,816],{"id":815},"frequently-asked-questions","Frequently Asked Questions",[10,818,819,822],{},[451,820,821],{},"When should I use pytest_generate_tests instead of @pytest.mark.parametrize?","\nWhen the cases are not known when the test is written — they come from files on disk, a command-line option, an environment, or a computation that depends on configuration. For a fixed list written in the source, the decorator is clearer and should be preferred.",[10,824,825,828,829,831],{},[451,826,827],{},"Where can pytest_generate_tests be defined?","\nIn a test module, where it applies to tests in that module, or in a ",[13,830,440],{},", where it applies to every test beneath it. In either place it is called once per test function during collection.",[10,833,834,837],{},[451,835,836],{},"Does loading cases in the hook slow down collection?","\nYes, because it runs during collection for every test function in scope, including runs that select only one unrelated test. Keep the loading cheap, cache it at module level, and guard it so it only does work for tests that request the relevant argument.",[25,839,841],{"id":840},"related","Related",[30,843,844,850,857,864],{},[33,845,846,849],{},[43,847,848],{"href":45},"Advanced Parametrization Techniques"," — the static techniques this hook generalises.",[33,851,852,856],{},[43,853,855],{"href":854},"\u002Fadvanced-pytest-architecture-configuration\u002Fadvanced-parametrization-techniques\u002Fgenerating-readable-test-ids\u002F","Generating Readable Test IDs"," — id strategies that stay stable across runs.",[33,858,859,863],{},[43,860,862],{"href":861},"\u002Fadvanced-pytest-architecture-configuration\u002Fbuilding-custom-pytest-plugins\u002Fadding-command-line-options-with-pytest-addoption\u002F","Adding Command-Line Options with pytest_addoption"," — letting the command line choose which cases run.",[33,865,866,870],{},[43,867,869],{"href":868},"\u002Fproperty-based-fuzz-testing-strategies\u002Fadvanced-property-based-testing\u002Fround-trip-properties-for-serializers-and-parsers\u002F","Round-Trip Properties for Serializers and Parsers"," — generating cases rather than enumerating them.",[10,872,873,874],{},"← Back to ",[43,875,848],{"href":45},[877,878,879],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .sScJk, html code.shiki .sScJk{--shiki-default:#6F42C1;--shiki-dark:#B392F0}html pre.shiki code .sj4cs, html code.shiki .sj4cs{--shiki-default:#005CC5;--shiki-dark:#79B8FF}html pre.shiki code .sJ8bj, html code.shiki .sJ8bj{--shiki-default:#6A737D;--shiki-dark:#6A737D}html pre.shiki code .sZZnC, html code.shiki .sZZnC{--shiki-default:#032F62;--shiki-dark:#9ECBFF}",{"title":63,"searchDepth":76,"depth":76,"links":881},[882,883,884,885,886,887,888,889,890,891],{"id":27,"depth":76,"text":28},{"id":52,"depth":76,"text":53},{"id":411,"depth":76,"text":412},{"id":444,"depth":76,"text":445},{"id":491,"depth":76,"text":492},{"id":678,"depth":76,"text":679},{"id":768,"depth":76,"text":769},{"id":798,"depth":76,"text":799},{"id":815,"depth":76,"text":816},{"id":840,"depth":76,"text":841},"Parametrize tests dynamically with the pytest_generate_tests hook: cases from files or command-line options, conditional parametrization, ids, and collection-time costs.","md",{"slug":895,"type":896,"breadcrumb":19,"datePublished":897,"dateModified":897,"faq":898,"howto":905},"generating-cases-with-pytest-generate-tests","article","2026-09-18",[899,901,903],{"q":821,"a":900},"When the cases are not known when the test is written — they come from files on disk, a command-line option, an environment, or a computation that depends on configuration. For a fixed list written in the source, the decorator is clearer and should be preferred.",{"q":827,"a":902},"In a test module, where it applies to tests in that module, or in a conftest.py, where it applies to every test beneath it. In either place it is called once per test function during collection.",{"q":836,"a":904},"Yes, because it runs during collection for every test function in scope, including runs that select only one unrelated test. Keep the loading cheap, cache it at module level, and guard it so it only does work for tests that request the relevant argument.",{"name":906,"description":907,"steps":908},"How to parametrize tests dynamically","Implement the hook, guard it by argument name, load cases cheaply, and give every generated case a stable id.",[909,912,915,918,921],{"name":910,"text":911},"Implement the hook where it should apply","Define pytest_generate_tests(metafunc) in the test module or a conftest.py covering the relevant tests.",{"name":913,"text":914},"Guard on the argument name","Return early unless the requested argument is in metafunc.fixturenames so unrelated tests are untouched.",{"name":916,"text":917},"Load cases once","Read files or configuration at module level or with a cache so collection stays fast.",{"name":919,"text":920},"Call metafunc.parametrize with ids","Pass the cases and an ids list derived from something stable such as the file name.",{"name":922,"text":923},"Verify with collect-only","Confirm the expected number of items and readable ids before running the tests.","\u002Fadvanced-pytest-architecture-configuration\u002Fadvanced-parametrization-techniques\u002Fgenerating-cases-with-pytest-generate-tests",{"title":5,"description":892},"advanced-pytest-architecture-configuration\u002Fadvanced-parametrization-techniques\u002Fgenerating-cases-with-pytest-generate-tests\u002Findex","WzEYOmtsnjN4mzwazS3g_qN4vez9S4qUprH87I7P1qM",1789718767536]