[{"data":1,"prerenderedAt":1815},["ShallowReactive",2],{"page-\u002Fadvanced-pytest-architecture-configuration\u002Fassertion-introspection-and-reporting\u002F":3},{"id":4,"title":5,"body":6,"description":1774,"extension":1775,"meta":1776,"navigation":320,"path":1811,"seo":1812,"stem":1813,"__hash__":1814},"content\u002Fadvanced-pytest-architecture-configuration\u002Fassertion-introspection-and-reporting\u002Findex.md","Assertion Introspection & Test Reporting",{"type":7,"value":8,"toc":1753},"minimark",[9,22,27,59,63,73,91,284,288,293,334,370,376,380,483,490,497,501,602,605,609,641,671,685,689,732,736,739,783,789,821,835,839,962,966,969,984,987,1031,1042,1064,1068,1071,1077,1087,1093,1218,1222,1237,1290,1309,1416,1433,1437,1440,1459,1469,1527,1530,1534,1537,1546,1557,1563,1569,1572,1576,1579,1597,1631,1634,1641,1645,1659,1675,1688,1694,1703,1707,1742,1749],[10,11,12,13,17,18,21],"p",{},"pytest's most valuable feature is not fixtures or parametrization — it is that a bare ",[14,15,16],"code",{},"assert"," produces a failure message describing exactly which parts of the expression differed. That behaviour is not magic in the interpreter; it is a bytecode rewrite applied to specific modules under specific conditions. Knowing where those conditions stop applying, and how to extend the output for your own types, is the difference between a failure that explains itself and one that says ",[14,19,20],{},"assert False",".",[23,24,26],"h2",{"id":25},"prerequisites","Prerequisites",[28,29,30,41,48,56],"ul",{},[31,32,33,36,37,40],"li",{},[14,34,35],{},"pytest >= 8.0",". The hook signatures here are stable across 7 and 8, but ",[14,38,39],{},"junit_family"," defaults changed in 6.",[31,42,43,44,47],{},"Write access to ",[14,45,46],{},"conftest.py"," at the root of the test tree, since hook implementations must be collected before the modules they affect.",[31,49,50,51,21],{},"Familiarity with hookwrappers, covered in ",[52,53,55],"a",{"href":54},"\u002Fadvanced-pytest-architecture-configuration\u002Fbuilding-custom-pytest-plugins\u002Fwriting-a-hookwrapper-for-test-reports\u002F","writing a hookwrapper for test reports",[31,57,58],{},"A CI system that consumes JUnit XML, if the reporting half is relevant.",[23,60,62],{"id":61},"core-concept-rewriting-happens-at-import-to-selected-modules","Core concept: rewriting happens at import, to selected modules",[10,64,65,66,68,69,72],{},"When pytest imports a test module it does not execute the source directly. It parses the source to an AST, walks it, and replaces every ",[14,67,16],{}," statement with a block that evaluates the expression's sub-parts into temporary variables, checks the result, and — on failure — builds an explanation from those temporaries. The rewritten module is compiled and cached as a ",[14,70,71],{},".pyc",", so the cost is paid once per source change.",[10,74,75,76,80,81,83,84,86,87,90],{},"The crucial detail is ",[77,78,79],"em",{},"which"," modules get this treatment: test modules matched by the collection patterns, ",[14,82,46],{}," files, and plugins registered through entry points. Everything else — your application code, and any helper module you import from a test — is loaded normally, so an ",[14,85,16],{}," inside it produces a bare ",[14,88,89],{},"AssertionError"," with no explanation at all.",[92,93,96,280],"figure",{"className":94},[95],"diagram",[97,98,105,106,105,110,105,114,105,132,105,140,105,149,105,159,105,165,105,171,105,174,105,177,105,181,105,186,105,190,105,195,105,204,105,207,105,210,105,215,105,219,105,222,105,226,105,229,105,232,105,236,105,238,105,244,105,249,105,253,105,257,105,261,105,264,105,268,105,275],"svg",{"viewBox":99,"role":100,"ariaLabelledBy":101,"xmlns":104},"0 0 840 296","img",[102,103],"rw-t","rw-d","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","\n  ",[107,108,109],"title",{"id":102},"Which modules get assertion rewriting",[111,112,113],"desc",{"id":103},"Three import paths. A test module matched by the collection pattern is parsed to an AST, rewritten and cached, producing a detailed failure message. A conftest or registered plugin follows the same path. A helper module imported from a test is loaded unrewritten unless it is registered, so its assertion produces only a bare AssertionError.",[115,116,117,118,105],"defs",{},"\n    ",[119,120,127],"marker",{"id":121,"viewBox":122,"refX":123,"refY":124,"markerWidth":125,"markerHeight":125,"orient":126},"rw-a","0 0 10 10","9","5","7","auto-start-reverse",[128,129],"path",{"d":130,"fill":131},"M0 0 L10 5 L0 10 z","#3d405b",[133,134],"rect",{"x":135,"y":135,"width":136,"height":137,"rx":138,"fill":139},"0","840","296","14","#fffdf8",[141,142,148],"text",{"x":143,"y":144,"textAnchor":145,"fontSize":146,"fontWeight":147,"fill":131},"420","28","middle","16","700","Introspection is a property of the importer, not of assert",[133,150],{"x":151,"y":152,"width":153,"height":154,"rx":155,"fill":156,"stroke":157,"strokeWidth":158},"26","52","196","56","10","#e6f0ea","#81b29a","2",[141,160,164],{"x":161,"y":162,"textAnchor":145,"fontSize":163,"fontWeight":147,"fill":131},"124","76","12","test_orders.py",[141,166,170],{"x":161,"y":167,"textAnchor":145,"fontSize":168,"fill":169},"95","11","#2a5f49","matches python_files",[133,172],{"x":151,"y":173,"width":153,"height":154,"rx":155,"fill":156,"stroke":157,"strokeWidth":158},"118",[141,175,46],{"x":161,"y":176,"textAnchor":145,"fontSize":163,"fontWeight":147,"fill":131},"142",[141,178,180],{"x":161,"y":179,"textAnchor":145,"fontSize":168,"fill":169},"161","always rewritten",[133,182],{"x":151,"y":183,"width":153,"height":154,"rx":155,"fill":184,"stroke":185,"strokeWidth":158},"184","#fbe9e3","#e07a5f",[141,187,189],{"x":161,"y":188,"textAnchor":145,"fontSize":163,"fontWeight":147,"fill":131},"208","tests\u002Fhelpers.py",[141,191,194],{"x":161,"y":192,"textAnchor":145,"fontSize":168,"fill":193},"227","#8f3d22","plain import",[196,197],"line",{"x1":198,"y1":199,"x2":200,"y2":201,"stroke":157,"strokeWidth":202,"markerEnd":203},"226","80","286","120","1.6","url(#rw-a)",[196,205],{"x1":198,"y1":206,"x2":200,"y2":206,"stroke":157,"strokeWidth":202,"markerEnd":203},"146",[196,208],{"x1":198,"y1":209,"x2":200,"y2":209,"stroke":185,"strokeWidth":202,"markerEnd":203},"212",[133,211],{"x":212,"y":173,"width":153,"height":154,"rx":155,"fill":213,"stroke":214,"strokeWidth":158},"292","#f7f0da","#f2cc8f",[141,216,218],{"x":217,"y":176,"textAnchor":145,"fontSize":163,"fontWeight":147,"fill":131},"390","AST rewrite",[141,220,221],{"x":217,"y":179,"textAnchor":145,"fontSize":168,"fill":131},"cached as .pyc",[133,223],{"x":212,"y":183,"width":153,"height":154,"rx":155,"fill":224,"stroke":225,"strokeWidth":202},"#f4f1de","rgba(61,64,91,0.35)",[141,227,228],{"x":217,"y":188,"textAnchor":145,"fontSize":163,"fontWeight":147,"fill":131},"normal import",[141,230,231],{"x":217,"y":192,"textAnchor":145,"fontSize":168,"fill":131},"bytecode unchanged",[196,233],{"x1":234,"y1":206,"x2":235,"y2":206,"stroke":131,"strokeWidth":202,"markerEnd":203},"492","552",[196,237],{"x1":234,"y1":209,"x2":235,"y2":209,"stroke":131,"strokeWidth":202,"markerEnd":203},[133,239],{"x":240,"y":241,"width":242,"height":243,"rx":155,"fill":156,"stroke":157,"strokeWidth":158},"558","112","256","68",[141,245,248],{"x":246,"y":247,"textAnchor":145,"fontSize":163,"fontWeight":147,"fill":131},"686","136","assert 3 == 4",[141,250,252],{"x":246,"y":251,"textAnchor":145,"fontSize":168,"fill":131},"156","E  assert 3 == 4",[141,254,256],{"x":246,"y":255,"textAnchor":145,"fontSize":168,"fill":169},"172","with both operands shown",[133,258],{"x":240,"y":259,"width":242,"height":260,"rx":155,"fill":184,"stroke":185,"strokeWidth":158},"190","62",[141,262,89],{"x":246,"y":263,"textAnchor":145,"fontSize":163,"fontWeight":147,"fill":131},"214",[141,265,267],{"x":246,"y":266,"textAnchor":145,"fontSize":168,"fill":193},"234","no operands, no diff",[133,269],{"x":151,"y":270,"width":271,"height":272,"rx":273,"fill":139,"stroke":225,"strokeWidth":274},"258","788","30","8","1.4",[141,276,279],{"x":143,"y":277,"textAnchor":145,"fontSize":278,"fill":131},"278","11.5","pytest.register_assert_rewrite(\"tests.helpers\") moves the bottom row onto the top path.",[281,282,283],"figcaption",{},"The commonest way to lose introspection is to move shared assertions into a helper module — the refactor that was supposed to improve the tests silently removes their diagnostics.",[23,285,287],{"id":286},"step-by-step-implementation","Step-by-step implementation",[289,290,292],"h3",{"id":291},"_1-keep-introspection-in-shared-helpers","1. Keep introspection in shared helpers",[294,295,300],"pre",{"className":296,"code":297,"language":298,"meta":299,"style":299},"language-python shiki shiki-themes github-light github-dark","# conftest.py — must run before anything imports the helper\nimport pytest\n\n# Rewrites tests\u002Fhelpers.py too, so asserts inside it explain themselves.\npytest.register_assert_rewrite(\"tests.helpers\")\n","python","",[14,301,302,309,315,322,328],{"__ignoreMap":299},[303,304,306],"span",{"class":196,"line":305},1,[303,307,308],{},"# conftest.py — must run before anything imports the helper\n",[303,310,312],{"class":196,"line":311},2,[303,313,314],{},"import pytest\n",[303,316,318],{"class":196,"line":317},3,[303,319,321],{"emptyLinePlaceholder":320},true,"\n",[303,323,325],{"class":196,"line":324},4,[303,326,327],{},"# Rewrites tests\u002Fhelpers.py too, so asserts inside it explain themselves.\n",[303,329,331],{"class":196,"line":330},5,[303,332,333],{},"pytest.register_assert_rewrite(\"tests.helpers\")\n",[294,335,337],{"className":296,"code":336,"language":298,"meta":299,"style":299},"# tests\u002Fhelpers.py\ndef assert_valid_invoice(invoice) -> None:\n    # With registration: \"assert Decimal('12.30') == Decimal('12.34')\".\n    # Without it: \"AssertionError\" and nothing else.\n    assert invoice.total == sum(line.amount for line in invoice.lines)\n    assert invoice.currency in {\"GBP\", \"USD\"}\n",[14,338,339,344,349,354,359,364],{"__ignoreMap":299},[303,340,341],{"class":196,"line":305},[303,342,343],{},"# tests\u002Fhelpers.py\n",[303,345,346],{"class":196,"line":311},[303,347,348],{},"def assert_valid_invoice(invoice) -> None:\n",[303,350,351],{"class":196,"line":317},[303,352,353],{},"    # With registration: \"assert Decimal('12.30') == Decimal('12.34')\".\n",[303,355,356],{"class":196,"line":324},[303,357,358],{},"    # Without it: \"AssertionError\" and nothing else.\n",[303,360,361],{"class":196,"line":330},[303,362,363],{},"    assert invoice.total == sum(line.amount for line in invoice.lines)\n",[303,365,367],{"class":196,"line":366},6,[303,368,369],{},"    assert invoice.currency in {\"GBP\", \"USD\"}\n",[10,371,372,373,375],{},"Registration must happen before the first import of the module, which in practice means the top of the root ",[14,374,46],{},". A plugin distributed through entry points gets the same treatment automatically.",[289,377,379],{"id":378},"_2-teach-pytest-to-compare-your-own-types","2. Teach pytest to compare your own types",[294,381,383],{"className":296,"code":382,"language":298,"meta":299,"style":299},"# conftest.py\nfrom decimal import Decimal\n\nfrom myapp.money import Money\n\n\ndef pytest_assertrepr_compare(config, op, left, right):\n    \"\"\"Custom failure output for Money == Money.\"\"\"\n    if op != \"==\" or not isinstance(left, Money) or not isinstance(right, Money):\n        return None          # let other implementations handle it\n\n    lines = [\"Money instances differ:\"]\n    if left.currency != right.currency:\n        lines.append(f\"   currency: {left.currency} != {right.currency}\")\n    if left.amount != right.amount:\n        delta = Decimal(right.amount - left.amount) \u002F 100\n        lines.append(f\"   amount:   {left.amount} != {right.amount}  (delta {delta:+})\")\n    return lines\n",[14,384,385,390,395,399,404,408,412,418,424,430,436,441,447,453,459,465,471,477],{"__ignoreMap":299},[303,386,387],{"class":196,"line":305},[303,388,389],{},"# conftest.py\n",[303,391,392],{"class":196,"line":311},[303,393,394],{},"from decimal import Decimal\n",[303,396,397],{"class":196,"line":317},[303,398,321],{"emptyLinePlaceholder":320},[303,400,401],{"class":196,"line":324},[303,402,403],{},"from myapp.money import Money\n",[303,405,406],{"class":196,"line":330},[303,407,321],{"emptyLinePlaceholder":320},[303,409,410],{"class":196,"line":366},[303,411,321],{"emptyLinePlaceholder":320},[303,413,415],{"class":196,"line":414},7,[303,416,417],{},"def pytest_assertrepr_compare(config, op, left, right):\n",[303,419,421],{"class":196,"line":420},8,[303,422,423],{},"    \"\"\"Custom failure output for Money == Money.\"\"\"\n",[303,425,427],{"class":196,"line":426},9,[303,428,429],{},"    if op != \"==\" or not isinstance(left, Money) or not isinstance(right, Money):\n",[303,431,433],{"class":196,"line":432},10,[303,434,435],{},"        return None          # let other implementations handle it\n",[303,437,439],{"class":196,"line":438},11,[303,440,321],{"emptyLinePlaceholder":320},[303,442,444],{"class":196,"line":443},12,[303,445,446],{},"    lines = [\"Money instances differ:\"]\n",[303,448,450],{"class":196,"line":449},13,[303,451,452],{},"    if left.currency != right.currency:\n",[303,454,456],{"class":196,"line":455},14,[303,457,458],{},"        lines.append(f\"   currency: {left.currency} != {right.currency}\")\n",[303,460,462],{"class":196,"line":461},15,[303,463,464],{},"    if left.amount != right.amount:\n",[303,466,468],{"class":196,"line":467},16,[303,469,470],{},"        delta = Decimal(right.amount - left.amount) \u002F 100\n",[303,472,474],{"class":196,"line":473},17,[303,475,476],{},"        lines.append(f\"   amount:   {left.amount} != {right.amount}  (delta {delta:+})\")\n",[303,478,480],{"class":196,"line":479},18,[303,481,482],{},"    return lines\n",[294,484,488],{"className":485,"code":487,"language":141,"meta":299},[486],"language-text","E   assert Money(1230, 'GBP') == Money(1234, 'GBP')\nE     Money instances differ:\nE        amount:   1230 != 1234  (delta +0.04)\n",[14,489,487],{"__ignoreMap":299},[10,491,492,493,496],{},"Returning ",[14,494,495],{},"None"," for unhandled cases is not optional courtesy — a hook that returns a list unconditionally suppresses every other implementation, including pytest's own excellent diff for dictionaries and sequences.",[289,498,500],{"id":499},"_3-attach-diagnostics-to-failures-automatically","3. Attach diagnostics to failures automatically",[294,502,504],{"className":296,"code":503,"language":298,"meta":299,"style":299},"# conftest.py\nimport pytest\n\n\n@pytest.hookimpl(hookwrapper=True)\ndef pytest_runtest_makereport(item, call):\n    report = (yield).get_result()\n\n    # Only decorate real failures in the call phase, not setup or teardown.\n    if report.when != \"call\" or not report.failed:\n        return\n\n    # Anything appended here appears in the terminal AND in the JUnit XML.\n    client = item.funcargs.get(\"http_client\")\n    if client is not None and client.last_response is not None:\n        report.sections.append((\n            \"captured HTTP exchange\",\n            f\"{client.last_request.method} {client.last_request.url}\\n\"\n            f\"→ {client.last_response.status_code}\\n{client.last_response.text[:2000]}\",\n        ))\n",[14,505,506,510,514,518,522,527,532,537,541,546,551,556,560,565,570,575,580,585,590,596],{"__ignoreMap":299},[303,507,508],{"class":196,"line":305},[303,509,389],{},[303,511,512],{"class":196,"line":311},[303,513,314],{},[303,515,516],{"class":196,"line":317},[303,517,321],{"emptyLinePlaceholder":320},[303,519,520],{"class":196,"line":324},[303,521,321],{"emptyLinePlaceholder":320},[303,523,524],{"class":196,"line":330},[303,525,526],{},"@pytest.hookimpl(hookwrapper=True)\n",[303,528,529],{"class":196,"line":366},[303,530,531],{},"def pytest_runtest_makereport(item, call):\n",[303,533,534],{"class":196,"line":414},[303,535,536],{},"    report = (yield).get_result()\n",[303,538,539],{"class":196,"line":420},[303,540,321],{"emptyLinePlaceholder":320},[303,542,543],{"class":196,"line":426},[303,544,545],{},"    # Only decorate real failures in the call phase, not setup or teardown.\n",[303,547,548],{"class":196,"line":432},[303,549,550],{},"    if report.when != \"call\" or not report.failed:\n",[303,552,553],{"class":196,"line":438},[303,554,555],{},"        return\n",[303,557,558],{"class":196,"line":443},[303,559,321],{"emptyLinePlaceholder":320},[303,561,562],{"class":196,"line":449},[303,563,564],{},"    # Anything appended here appears in the terminal AND in the JUnit XML.\n",[303,566,567],{"class":196,"line":455},[303,568,569],{},"    client = item.funcargs.get(\"http_client\")\n",[303,571,572],{"class":196,"line":461},[303,573,574],{},"    if client is not None and client.last_response is not None:\n",[303,576,577],{"class":196,"line":467},[303,578,579],{},"        report.sections.append((\n",[303,581,582],{"class":196,"line":473},[303,583,584],{},"            \"captured HTTP exchange\",\n",[303,586,587],{"class":196,"line":479},[303,588,589],{},"            f\"{client.last_request.method} {client.last_request.url}\\n\"\n",[303,591,593],{"class":196,"line":592},19,[303,594,595],{},"            f\"→ {client.last_response.status_code}\\n{client.last_response.text[:2000]}\",\n",[303,597,599],{"class":196,"line":598},20,[303,600,601],{},"        ))\n",[10,603,604],{},"This is the highest-value hook in the file. A failing API test that prints the request it made and the response it got needs no reproduction step; the same test without it starts a twenty-minute cycle of adding prints and re-running CI.",[289,606,608],{"id":607},"_4-emit-machine-readable-results","4. Emit machine-readable results",[294,610,614],{"className":611,"code":612,"language":613,"meta":299,"style":299},"language-toml shiki shiki-themes github-light github-dark","# pyproject.toml\n[tool.pytest.ini_options]\naddopts = \"--junitxml=reports\u002Fjunit.xml -o junit_family=xunit2\"\njunit_logging = \"system-out\"        # captured output lands in the XML\njunit_duration_report = \"call\"      # duration excludes fixture setup\n","toml",[14,615,616,621,626,631,636],{"__ignoreMap":299},[303,617,618],{"class":196,"line":305},[303,619,620],{},"# pyproject.toml\n",[303,622,623],{"class":196,"line":311},[303,624,625],{},"[tool.pytest.ini_options]\n",[303,627,628],{"class":196,"line":317},[303,629,630],{},"addopts = \"--junitxml=reports\u002Fjunit.xml -o junit_family=xunit2\"\n",[303,632,633],{"class":196,"line":324},[303,634,635],{},"junit_logging = \"system-out\"        # captured output lands in the XML\n",[303,637,638],{"class":196,"line":330},[303,639,640],{},"junit_duration_report = \"call\"      # duration excludes fixture setup\n",[294,642,644],{"className":296,"code":643,"language":298,"meta":299,"style":299},"def test_charge_is_idempotent(record_property, gateway):\n    # Properties appear as \u003Cproperty> elements the CI dashboard can display.\n    record_property(\"gateway_version\", gateway.version)\n    record_property(\"idempotency_key\", gateway.last_key)\n    assert gateway.charge_count == 1\n",[14,645,646,651,656,661,666],{"__ignoreMap":299},[303,647,648],{"class":196,"line":305},[303,649,650],{},"def test_charge_is_idempotent(record_property, gateway):\n",[303,652,653],{"class":196,"line":311},[303,654,655],{},"    # Properties appear as \u003Cproperty> elements the CI dashboard can display.\n",[303,657,658],{"class":196,"line":317},[303,659,660],{},"    record_property(\"gateway_version\", gateway.version)\n",[303,662,663],{"class":196,"line":324},[303,664,665],{},"    record_property(\"idempotency_key\", gateway.last_key)\n",[303,667,668],{"class":196,"line":330},[303,669,670],{},"    assert gateway.charge_count == 1\n",[10,672,673,676,677,680,681,684],{},[14,674,675],{},"xunit2"," is the family nearly every modern CI parser expects; the legacy ",[14,678,679],{},"xunit1"," output silently loses properties in several of them. ",[14,682,683],{},"junit_duration_report = \"call\""," matters for anyone using the report to find slow tests, since the default includes fixture setup and makes every test that touches a session fixture look slow.",[289,686,688],{"id":687},"_5-add-a-summary-line-the-reader-will-actually-see","5. Add a summary line the reader will actually see",[294,690,692],{"className":296,"code":691,"language":298,"meta":299,"style":299},"# conftest.py\ndef pytest_terminal_summary(terminalreporter, exitstatus, config):\n    failed = terminalreporter.stats.get(\"failed\", [])\n    if not failed:\n        return\n    terminalreporter.write_sep(\"=\", \"artefacts for failed tests\", red=True)\n    for report in failed:\n        terminalreporter.write_line(f\"  {report.nodeid} → reports\u002F{report.nodeid}.log\")\n",[14,693,694,698,703,708,713,717,722,727],{"__ignoreMap":299},[303,695,696],{"class":196,"line":305},[303,697,389],{},[303,699,700],{"class":196,"line":311},[303,701,702],{},"def pytest_terminal_summary(terminalreporter, exitstatus, config):\n",[303,704,705],{"class":196,"line":317},[303,706,707],{},"    failed = terminalreporter.stats.get(\"failed\", [])\n",[303,709,710],{"class":196,"line":324},[303,711,712],{},"    if not failed:\n",[303,714,715],{"class":196,"line":330},[303,716,555],{},[303,718,719],{"class":196,"line":366},[303,720,721],{},"    terminalreporter.write_sep(\"=\", \"artefacts for failed tests\", red=True)\n",[303,723,724],{"class":196,"line":414},[303,725,726],{},"    for report in failed:\n",[303,728,729],{"class":196,"line":420},[303,730,731],{},"        terminalreporter.write_line(f\"  {report.nodeid} → reports\u002F{report.nodeid}.log\")\n",[23,733,735],{"id":734},"verification","Verification",[10,737,738],{},"Write one deliberately failing test and read its whole output — terminal and XML — before trusting any of this.",[294,740,744],{"className":741,"code":742,"language":743,"meta":299,"style":299},"language-bash shiki shiki-themes github-light github-dark","pytest tests\u002Ftest_meta_failure.py -q ; xmllint --format reports\u002Fjunit.xml | head -30\n","bash",[14,745,746],{"__ignoreMap":299},[303,747,748,752,756,760,764,767,770,773,777,780],{"class":196,"line":305},[303,749,751],{"class":750},"sScJk","pytest",[303,753,755],{"class":754},"sZZnC"," tests\u002Ftest_meta_failure.py",[303,757,759],{"class":758},"sj4cs"," -q",[303,761,763],{"class":762},"sVt8B"," ; ",[303,765,766],{"class":750},"xmllint",[303,768,769],{"class":758}," --format",[303,771,772],{"class":754}," reports\u002Fjunit.xml",[303,774,776],{"class":775},"szBVR"," |",[303,778,779],{"class":750}," head",[303,781,782],{"class":758}," -30\n",[294,784,787],{"className":785,"code":786,"language":141,"meta":299},[486],"E   assert Money(1230, 'GBP') == Money(1234, 'GBP')\nE     Money instances differ:\nE        amount:   1230 != 1234  (delta +0.04)\n------------------------- captured HTTP exchange -------------------------\nPOST https:\u002F\u002Fbilling.test\u002Fcharges\n→ 402\n{\"error\": \"insufficient_funds\"}\n",[14,788,786],{"__ignoreMap":299},[294,790,794],{"className":791,"code":792,"language":793,"meta":299,"style":299},"language-xml shiki shiki-themes github-light github-dark","\u003Ctestcase classname=\"tests.test_meta_failure\" name=\"test_total\" time=\"0.031\">\n  \u003Cproperties>\u003Cproperty name=\"gateway_version\" value=\"2.4.1\"\u002F>\u003C\u002Fproperties>\n  \u003Cfailure message=\"assert Money(1230, 'GBP') == Money(1234, 'GBP')\">…\u003C\u002Ffailure>\n  \u003Csystem-out>POST https:\u002F\u002Fbilling.test\u002Fcharges …\u003C\u002Fsystem-out>\n\u003C\u002Ftestcase>\n","xml",[14,795,796,801,806,811,816],{"__ignoreMap":299},[303,797,798],{"class":196,"line":305},[303,799,800],{},"\u003Ctestcase classname=\"tests.test_meta_failure\" name=\"test_total\" time=\"0.031\">\n",[303,802,803],{"class":196,"line":311},[303,804,805],{},"  \u003Cproperties>\u003Cproperty name=\"gateway_version\" value=\"2.4.1\"\u002F>\u003C\u002Fproperties>\n",[303,807,808],{"class":196,"line":317},[303,809,810],{},"  \u003Cfailure message=\"assert Money(1230, 'GBP') == Money(1234, 'GBP')\">…\u003C\u002Ffailure>\n",[303,812,813],{"class":196,"line":324},[303,814,815],{},"  \u003Csystem-out>POST https:\u002F\u002Fbilling.test\u002Fcharges …\u003C\u002Fsystem-out>\n",[303,817,818],{"class":196,"line":330},[303,819,820],{},"\u003C\u002Ftestcase>\n",[10,822,823,824,826,827,830,831,834],{},"If the custom comparison text is missing, the hook returned ",[14,825,495],{}," — usually an ",[14,828,829],{},"isinstance"," check that does not match because the objects are subclasses. If the section is missing from the XML, ",[14,832,833],{},"junit_logging"," is unset.",[23,836,838],{"id":837},"troubleshooting","Troubleshooting",[840,841,842,858],"table",{},[843,844,845],"thead",{},[846,847,848,852,855],"tr",{},[849,850,851],"th",{},"Symptom",[849,853,854],{},"Root cause",[849,856,857],{},"Fix",[859,860,861,878,891,905,921,943],"tbody",{},[846,862,863,867,870],{},[864,865,866],"td",{},"Helper assertions show no detail",[864,868,869],{},"Module imported without rewriting",[864,871,872,875,876],{},[14,873,874],{},"pytest.register_assert_rewrite"," at the top of ",[14,877,46],{},[846,879,880,885,888],{},[864,881,882],{},[14,883,884],{},"PytestAssertRewriteWarning",[864,886,887],{},"Module already imported when registration ran",[864,889,890],{},"Move the registration above the import, or out of a plugin's body",[846,892,893,896,899],{},[864,894,895],{},"Custom comparison never appears",[864,897,898],{},"Hook returned a list for every case, or wrong operand types",[864,900,901,902,904],{},"Return ",[14,903,495],{}," when unhandled; check subclasses",[846,906,907,910,915],{},[864,908,909],{},"Properties missing from XML",[864,911,912,914],{},[14,913,39],{}," left at the legacy default",[864,916,917,918],{},"Set ",[14,919,920],{},"junit_family=xunit2",[846,922,923,926,937],{},[864,924,925],{},"Report hook runs but nothing shows",[864,927,928,929,932,933,936],{},"Added to ",[14,930,931],{},"report.sections"," outside the ",[14,934,935],{},"call"," phase",[864,938,939,940],{},"Guard on ",[14,941,942],{},"report.when == \"call\"",[846,944,945,948,959],{},[864,946,947],{},"Rewriting appears disabled entirely",[864,949,950,951,954,955,958],{},"Running with ",[14,952,953],{},"--assert=plain",", or ",[14,956,957],{},"PYTHONDONTWRITEBYTECODE"," plus a read-only tree",[864,960,961],{},"Remove the flag; the cache falls back to in-memory",[23,963,965],{"id":964},"reading-a-rewritten-assertion","Reading a rewritten assertion",[10,967,968],{},"It is worth seeing what the rewrite produces once, because it explains both the power and the limits.",[294,970,972],{"className":296,"code":971,"language":298,"meta":299,"style":299},"def test_totals():\n    assert order.total() == expected_total(order)\n",[14,973,974,979],{"__ignoreMap":299},[303,975,976],{"class":196,"line":305},[303,977,978],{},"def test_totals():\n",[303,980,981],{"class":196,"line":311},[303,982,983],{},"    assert order.total() == expected_total(order)\n",[10,985,986],{},"becomes, in essence:",[294,988,990],{"className":296,"code":989,"language":298,"meta":299,"style":299},"def test_totals():\n    tmp_left = order.total()\n    tmp_right = expected_total(order)\n    tmp_result = tmp_left == tmp_right\n    if not tmp_result:\n        raise AssertionError(\n            _format_explanation(tmp_left, \"==\", tmp_right)\n        )\n",[14,991,992,996,1001,1006,1011,1016,1021,1026],{"__ignoreMap":299},[303,993,994],{"class":196,"line":305},[303,995,978],{},[303,997,998],{"class":196,"line":311},[303,999,1000],{},"    tmp_left = order.total()\n",[303,1002,1003],{"class":196,"line":317},[303,1004,1005],{},"    tmp_right = expected_total(order)\n",[303,1007,1008],{"class":196,"line":324},[303,1009,1010],{},"    tmp_result = tmp_left == tmp_right\n",[303,1012,1013],{"class":196,"line":330},[303,1014,1015],{},"    if not tmp_result:\n",[303,1017,1018],{"class":196,"line":366},[303,1019,1020],{},"        raise AssertionError(\n",[303,1022,1023],{"class":196,"line":414},[303,1024,1025],{},"            _format_explanation(tmp_left, \"==\", tmp_right)\n",[303,1027,1028],{"class":196,"line":420},[303,1029,1030],{},"        )\n",[10,1032,1033,1034,1037,1038,1041],{},"Each sub-expression is evaluated exactly once into a temporary, which is what lets the message show both sides without re-running anything. Two consequences follow. Side effects are not duplicated, so ",[14,1035,1036],{},"assert queue.pop() == 3"," is safe. And expressions are evaluated eagerly left to right, so ",[14,1039,1040],{},"assert x is not None and x.value == 3"," still short-circuits correctly — the rewrite preserves boolean semantics rather than flattening them.",[10,1043,1044,1045,1048,1049,1052,1053,1056,1057,1060,1061,1063],{},"The limit is that the explanation is built from the operands' ",[14,1046,1047],{},"repr()",". An object whose ",[14,1050,1051],{},"repr"," is ",[14,1054,1055],{},"\u003COrder object at 0x7f…>"," produces a useless message no matter how good the rewriting is, which makes a decent ",[14,1058,1059],{},"__repr__"," on domain types one of the highest-leverage testability changes available. Ten minutes adding ",[14,1062,1059],{}," to five model classes improves every future failure in the suite.",[23,1065,1067],{"id":1066},"report-hooks-in-the-order-they-fire","Report hooks, in the order they fire",[10,1069,1070],{},"Reporting hooks are easy to use incorrectly because several of them look interchangeable. They are not; each sees a different stage.",[10,1072,1073,1076],{},[14,1074,1075],{},"pytest_runtest_makereport"," is called three times per test — once each for setup, call and teardown — and is the only place with access to both the item (fixtures, markers, the node) and the outcome. Anything that needs fixture state at failure time belongs here.",[10,1078,1079,1082,1083,1086],{},[14,1080,1081],{},"pytest_runtest_logreport"," receives each finished report and is the right place to ",[77,1084,1085],{},"forward"," results: to a dashboard, a message queue, a file. It has the report but not the item, so it cannot reach fixtures.",[10,1088,1089,1092],{},[14,1090,1091],{},"pytest_terminal_summary"," runs once at the end with the accumulated statistics, which makes it the place for aggregate output — a list of artefact paths, a count of quarantined tests, a link to the run.",[92,1094,1096,1210],{"className":1095},[95],[97,1097,105,1102,105,1105,105,1108,105,1115,105,1118,105,1121,105,1126,105,1129,105,1133,105,1135,105,1138,105,1141,105,1144,105,1148,105,1151,105,1157,105,1159,105,1161,105,1166,105,1170,105,1174,105,1179,105,1183,105,1187,105,1190,105,1193,105,1197,105,1201,105,1206],{"viewBox":1098,"role":100,"ariaLabelledBy":1099,"xmlns":104},"0 0 840 262",[1100,1101],"hooks-t","hooks-d",[107,1103,1104],{"id":1100},"Report hook order across one test's lifecycle",[111,1106,1107],{"id":1101},"A timeline for a single test. Setup, call and teardown each produce a report through pytest_runtest_makereport, and each finished report is then passed to pytest_runtest_logreport. After every test has run, pytest_terminal_summary fires once with the accumulated statistics.",[115,1109,117,1110,105],{},[119,1111,1113],{"id":1112,"viewBox":122,"refX":123,"refY":124,"markerWidth":125,"markerHeight":125,"orient":126},"hooks-a",[128,1114],{"d":130,"fill":131},[133,1116],{"x":135,"y":135,"width":136,"height":1117,"rx":138,"fill":139},"262",[141,1119,1120],{"x":143,"y":144,"textAnchor":145,"fontSize":146,"fontWeight":147,"fill":131},"Three reports per test, then one summary per run",[133,1122],{"x":1123,"y":1124,"width":1125,"height":152,"rx":155,"fill":224,"stroke":131,"strokeWidth":202},"34","54","180",[141,1127,1128],{"x":161,"y":162,"textAnchor":145,"fontSize":163,"fontWeight":147,"fill":131},"setup",[141,1130,1132],{"x":161,"y":1131,"textAnchor":145,"fontSize":168,"fill":131},"94","fixtures run",[133,1134],{"x":266,"y":1124,"width":1125,"height":152,"rx":155,"fill":156,"stroke":157,"strokeWidth":158},[141,1136,935],{"x":1137,"y":162,"textAnchor":145,"fontSize":163,"fontWeight":147,"fill":131},"324",[141,1139,1140],{"x":1137,"y":1131,"textAnchor":145,"fontSize":168,"fill":169},"the test body",[133,1142],{"x":1143,"y":1124,"width":1125,"height":152,"rx":155,"fill":224,"stroke":131,"strokeWidth":202},"434",[141,1145,1147],{"x":1146,"y":162,"textAnchor":145,"fontSize":163,"fontWeight":147,"fill":131},"524","teardown",[141,1149,1150],{"x":1146,"y":1131,"textAnchor":145,"fontSize":168,"fill":131},"finalizers run",[196,1152],{"x1":161,"y1":1153,"x2":161,"y2":1154,"stroke":131,"strokeWidth":1155,"markerEnd":1156},"110","134","1.5","url(#hooks-a)",[196,1158],{"x1":1137,"y1":1153,"x2":1137,"y2":1154,"stroke":131,"strokeWidth":1155,"markerEnd":1156},[196,1160],{"x1":1146,"y1":1153,"x2":1146,"y2":1154,"stroke":131,"strokeWidth":1155,"markerEnd":1156},[133,1162],{"x":1123,"y":1163,"width":1164,"height":1165,"rx":155,"fill":213,"stroke":214,"strokeWidth":158},"138","580","46",[141,1167,1169],{"x":1137,"y":1168,"textAnchor":145,"fontSize":163,"fontWeight":147,"fill":131},"160","pytest_runtest_makereport — has the item AND the outcome",[141,1171,1173],{"x":1137,"y":1172,"textAnchor":145,"fontSize":168,"fill":131},"177","attach logs, request bodies, screenshots here",[133,1175],{"x":1176,"y":1124,"width":1177,"height":1178,"rx":155,"fill":139,"stroke":225,"strokeWidth":202},"634","176","130",[141,1180,1182],{"x":1181,"y":199,"textAnchor":145,"fontSize":163,"fontWeight":147,"fill":131},"722","logreport",[141,1184,1186],{"x":1181,"y":1185,"textAnchor":145,"fontSize":168,"fill":131},"102","report only,",[141,1188,1189],{"x":1181,"y":201,"textAnchor":145,"fontSize":168,"fill":131},"no fixtures",[141,1191,1192],{"x":1181,"y":206,"textAnchor":145,"fontSize":168,"fill":131},"forward results",[141,1194,1196],{"x":1181,"y":1195,"textAnchor":145,"fontSize":168,"fill":131},"166","to a dashboard",[133,1198],{"x":1123,"y":1199,"width":1200,"height":1165,"rx":155,"fill":184,"stroke":185,"strokeWidth":158},"198","776",[141,1202,1205],{"x":1203,"y":1204,"textAnchor":145,"fontSize":163,"fontWeight":147,"fill":131},"422","220","pytest_terminal_summary — once per run, with every statistic",[141,1207,1209],{"x":1203,"y":1208,"textAnchor":145,"fontSize":168,"fill":131},"237","artefact index, quarantine counts, a link to the full report",[281,1211,1212,1213,1215,1216,21],{},"Choosing the wrong hook is the usual reason a diagnostic never appears: fixture state is gone by ",[14,1214,1182],{},", and outcomes do not exist yet in ",[14,1217,1128],{},[23,1219,1221],{"id":1220},"capture-and-where-the-output-actually-goes","Capture, and where the output actually goes",[10,1223,1224,1225,1228,1229,1232,1233,1236],{},"Half the confusion about pytest reporting comes from capture. By default pytest replaces ",[14,1226,1227],{},"sys.stdout",", ",[14,1230,1231],{},"sys.stderr"," and the file descriptors beneath them for the duration of each test, buffers everything, and prints it only if the test fails. That is the right default — a passing suite should be quiet — but it means the same ",[14,1234,1235],{},"print"," behaves differently depending on flags nobody remembers setting.",[294,1238,1240],{"className":741,"code":1239,"language":743,"meta":299,"style":299},"pytest                     # captured; shown only on failure\npytest -s                  # capture disabled entirely; everything streams live\npytest --capture=sys       # replaces sys.stdout only; C-level writes escape\npytest --capture=fd        # the default: file descriptors too, so C output is caught\npytest -rP                 # show captured output for PASSED tests as well\n",[14,1241,1242,1250,1260,1270,1280],{"__ignoreMap":299},[303,1243,1244,1246],{"class":196,"line":305},[303,1245,751],{"class":750},[303,1247,1249],{"class":1248},"sJ8bj","                     # captured; shown only on failure\n",[303,1251,1252,1254,1257],{"class":196,"line":311},[303,1253,751],{"class":750},[303,1255,1256],{"class":758}," -s",[303,1258,1259],{"class":1248},"                  # capture disabled entirely; everything streams live\n",[303,1261,1262,1264,1267],{"class":196,"line":317},[303,1263,751],{"class":750},[303,1265,1266],{"class":758}," --capture=sys",[303,1268,1269],{"class":1248},"       # replaces sys.stdout only; C-level writes escape\n",[303,1271,1272,1274,1277],{"class":196,"line":324},[303,1273,751],{"class":750},[303,1275,1276],{"class":758}," --capture=fd",[303,1278,1279],{"class":1248},"        # the default: file descriptors too, so C output is caught\n",[303,1281,1282,1284,1287],{"class":196,"line":330},[303,1283,751],{"class":750},[303,1285,1286],{"class":758}," -rP",[303,1288,1289],{"class":1248},"                 # show captured output for PASSED tests as well\n",[10,1291,1292,1295,1296,1299,1300,1302,1303,1305,1306,21],{},[14,1293,1294],{},"--capture=fd"," versus ",[14,1297,1298],{},"sys"," matters more than it looks. A C extension, a subprocess or anything writing to file descriptor 1 directly bypasses ",[14,1301,1227],{},"; only the file-descriptor mode catches it. A test that shells out and sees no output in its failure report is usually running under ",[14,1304,1298],{}," capture, set by a plugin or a stale flag in ",[14,1307,1308],{},"addopts",[92,1310,1312,1413],{"className":1311},[95],[97,1313,105,1318,105,1321,105,1324,105,1328,105,1332,105,1338,105,1343,105,1346,105,1349,105,1352,105,1356,105,1359,105,1363,105,1367,105,1370,105,1372,105,1375,105,1379,105,1381,105,1384,105,1386,105,1389,105,1392,105,1394,105,1397,105,1402,105,1405,105,1410],{"viewBox":1314,"role":100,"ariaLabelledBy":1315,"xmlns":104},"0 0 820 252",[1316,1317],"cap-t","cap-d",[107,1319,1320],{"id":1316},"Where captured output ends up under each mode",[111,1322,1323],{"id":1317},"Three columns show what happens to Python prints, C-level writes and logging records under file-descriptor capture, sys capture and no capture. File-descriptor capture catches all three and shows them on failure; sys capture lets C-level writes escape to the terminal; disabling capture streams everything live and attaches nothing to the report.",[133,1325],{"x":135,"y":135,"width":1326,"height":1327,"rx":138,"fill":139},"820","252",[141,1329,1331],{"x":1330,"y":144,"textAnchor":145,"fontSize":146,"fontWeight":147,"fill":131},"410","Three capture modes, three different failure reports",[133,1333],{"x":1334,"y":1335,"width":1336,"height":1337,"rx":123,"fill":224,"stroke":131,"strokeWidth":1155},"24","50","186","38",[141,1339,1342],{"x":1340,"y":1341,"textAnchor":145,"fontSize":163,"fontWeight":147,"fill":131},"117","74","writer",[133,1344],{"x":1345,"y":1335,"width":1336,"height":1337,"rx":123,"fill":156,"stroke":157,"strokeWidth":158},"218",[141,1347,1294],{"x":1348,"y":1341,"textAnchor":145,"fontSize":163,"fontWeight":147,"fill":131},"311",[133,1350],{"x":1351,"y":1335,"width":1336,"height":1337,"rx":123,"fill":213,"stroke":214,"strokeWidth":158},"412",[141,1353,1355],{"x":1354,"y":1341,"textAnchor":145,"fontSize":163,"fontWeight":147,"fill":131},"505","--capture=sys",[133,1357],{"x":1358,"y":1335,"width":259,"height":1337,"rx":123,"fill":184,"stroke":185,"strokeWidth":158},"606",[141,1360,1362],{"x":1361,"y":1341,"textAnchor":145,"fontSize":163,"fontWeight":147,"fill":131},"701","-s (none)",[141,1364,1366],{"x":1365,"y":173,"fontSize":278,"fill":131},"40","print() in Python",[141,1368,1369],{"x":1348,"y":173,"textAnchor":145,"fontSize":278,"fill":169},"in the report",[141,1371,1369],{"x":1354,"y":173,"textAnchor":145,"fontSize":278,"fill":169},[141,1373,1374],{"x":1361,"y":173,"textAnchor":145,"fontSize":278,"fill":193},"live, not attached",[141,1376,1378],{"x":1365,"y":1377,"fontSize":278,"fill":131},"152","C extension write()",[141,1380,1369],{"x":1348,"y":1377,"textAnchor":145,"fontSize":278,"fill":169},[141,1382,1383],{"x":1354,"y":1377,"textAnchor":145,"fontSize":278,"fill":193},"escapes to terminal",[141,1385,1374],{"x":1361,"y":1377,"textAnchor":145,"fontSize":278,"fill":193},[141,1387,1388],{"x":1365,"y":1336,"fontSize":278,"fill":131},"logging records",[141,1390,1391],{"x":1348,"y":1336,"textAnchor":145,"fontSize":278,"fill":169},"caplog + report",[141,1393,1391],{"x":1354,"y":1336,"textAnchor":145,"fontSize":278,"fill":169},[141,1395,1396],{"x":1361,"y":1336,"textAnchor":145,"fontSize":278,"fill":131},"caplog only",[196,1398],{"x1":1334,"y1":1399,"x2":1400,"y2":1399,"stroke":1401,"strokeWidth":274},"128","796","rgba(61,64,91,0.14)",[196,1403],{"x1":1334,"y1":1404,"x2":1400,"y2":1404,"stroke":1401,"strokeWidth":274},"162",[133,1406],{"x":1334,"y":1407,"width":1408,"height":1409,"rx":123,"fill":139,"stroke":225,"strokeWidth":1155},"202","772","36",[141,1411,1412],{"x":1330,"y":198,"textAnchor":145,"fontSize":278,"fill":131},"Only the default mode puts everything a failure needs into the report that CI keeps.",[281,1414,1415],{},"Disabling capture is the right move while debugging interactively and the wrong one in CI, where nothing is attached to the report a human will read later.",[10,1417,1418,1419,1422,1423,1425,1426,1428,1429,1432],{},"Logging is captured separately from streams, through a handler pytest installs, which is why ",[14,1420,1421],{},"caplog"," can assert on records that never appeared on stdout. Keeping the two straight — streams for ",[14,1424,1235],{}," and subprocess output, ",[14,1427,1421],{}," for ",[14,1430,1431],{},"logging"," — removes most of the surprise.",[23,1434,1436],{"id":1435},"reporting-for-humans-and-for-machines","Reporting for humans and for machines",[10,1438,1439],{},"The two audiences want different things and it is worth serving both deliberately rather than hoping one format suffices.",[10,1441,1442,1443,1446,1447,1450,1451,1454,1455,1458],{},"A human reading a terminal wants the failure at the end, in order of severity, with enough context to act. ",[14,1444,1445],{},"-q"," for the run, ",[14,1448,1449],{},"--tb=short"," for compact tracebacks, and ",[14,1452,1453],{},"-x"," when iterating locally give that. The full traceback style is rarely useful in CI, where the volume of ",[14,1456,1457],{},"--tb=long"," output from twenty failures buries the first one.",[10,1460,1461,1462,1465,1466,1468],{},"A machine wants stable identifiers and a parseable file. JUnit XML is the common denominator, but two additions make it far more useful. Per-test properties record the run's context — build number, dependency versions, feature flags — so a historical query can ask which versions correlate with a failure. And a stable ",[14,1463,1464],{},"nodeid"," matters more than anything else: a test whose id changes because its parametrization ids are generated from object ",[14,1467,1051],{}," cannot be tracked across runs, so flake-detection tooling sees a new test every time.",[294,1470,1472],{"className":296,"code":1471,"language":298,"meta":299,"style":299},"# conftest.py — stamp every run with its context, once.\ndef pytest_configure(config):\n    config.stash[\"run_context\"] = {\n        \"commit\": os.environ.get(\"GIT_COMMIT\", \"local\"),\n        \"python\": platform.python_version(),\n    }\n\n\ndef pytest_runtest_setup(item):\n    for key, value in item.config.stash[\"run_context\"].items():\n        item.user_properties.append((key, value))\n",[14,1473,1474,1479,1484,1489,1494,1499,1504,1508,1512,1517,1522],{"__ignoreMap":299},[303,1475,1476],{"class":196,"line":305},[303,1477,1478],{},"# conftest.py — stamp every run with its context, once.\n",[303,1480,1481],{"class":196,"line":311},[303,1482,1483],{},"def pytest_configure(config):\n",[303,1485,1486],{"class":196,"line":317},[303,1487,1488],{},"    config.stash[\"run_context\"] = {\n",[303,1490,1491],{"class":196,"line":324},[303,1492,1493],{},"        \"commit\": os.environ.get(\"GIT_COMMIT\", \"local\"),\n",[303,1495,1496],{"class":196,"line":330},[303,1497,1498],{},"        \"python\": platform.python_version(),\n",[303,1500,1501],{"class":196,"line":366},[303,1502,1503],{},"    }\n",[303,1505,1506],{"class":196,"line":414},[303,1507,321],{"emptyLinePlaceholder":320},[303,1509,1510],{"class":196,"line":420},[303,1511,321],{"emptyLinePlaceholder":320},[303,1513,1514],{"class":196,"line":426},[303,1515,1516],{},"def pytest_runtest_setup(item):\n",[303,1518,1519],{"class":196,"line":432},[303,1520,1521],{},"    for key, value in item.config.stash[\"run_context\"].items():\n",[303,1523,1524],{"class":196,"line":438},[303,1525,1526],{},"        item.user_properties.append((key, value))\n",[10,1528,1529],{},"Three lines of context in every record is what turns a pile of XML into something answerable. The question worth being able to answer is not \"did this test fail?\" but \"has this test failed before, on which commits, and under which interpreter\" — and none of that is recoverable after the fact if the reports did not carry it.",[23,1531,1533],{"id":1532},"what-good-failure-output-contains","What good failure output contains",[10,1535,1536],{},"The test of this whole area is simple: can a colleague diagnose a CI failure from the build log alone, without re-running anything? Four things make that possible, and none of them are expensive.",[10,1538,1539,1543,1544,21],{},[1540,1541,1542],"strong",{},"The values that differed",", which assertion rewriting gives for free as long as the assertion is in a rewritten module and the objects have a usable ",[14,1545,1051],{},[10,1547,1548,1551,1552,1556],{},[1540,1549,1550],{},"The inputs",", which parametrization supplies in the test id — one reason ",[52,1553,1555],{"href":1554},"\u002Fadvanced-pytest-architecture-configuration\u002Fadvanced-parametrization-techniques\u002Fgenerating-readable-test-ids\u002F","readable test ids"," are worth the effort — and which factories supply if the test constructed its data explicitly.",[10,1558,1559,1562],{},[1540,1560,1561],{},"The environment",", meaning the versions, flags and configuration this run used. A header printed once per run costs nothing and answers \"was this the run with the new dependency?\" immediately.",[10,1564,1565,1568],{},[1540,1566,1567],{},"The interaction",", for anything that crossed a boundary: the SQL, the HTTP exchange, the message published. This is what the report hookwrapper attaches, and it is the item most often missing.",[10,1570,1571],{},"A team that adds all four typically finds the median time to diagnose a CI failure falls from tens of minutes to a couple, because the second and third re-runs disappear entirely. It is one of the few changes to a test suite whose benefit is immediate and unambiguous, and it applies equally to a suite of fifty tests and one of fifty thousand.",[23,1573,1575],{"id":1574},"keeping-assertion-helpers-worth-having","Keeping assertion helpers worth having",[10,1577,1578],{},"Shared assertion helpers are a good idea that goes wrong in a predictable way, and registering them for rewriting only solves half of it.",[10,1580,1581,1582,1584,1585,1588,1589,1592,1593,1596],{},"The half that registration solves is diagnostics: a registered helper's ",[14,1583,16],{}," reports its operands. The half it does not solve is ",[77,1586,1587],{},"location",". A failure inside ",[14,1590,1591],{},"assert_valid_invoice"," reports a line in the helper, and if forty tests call it, the traceback tells you which test only through the frames above. pytest handles this with ",[14,1594,1595],{},"__tracebackhide__",":",[294,1598,1600],{"className":296,"code":1599,"language":298,"meta":299,"style":299},"def assert_valid_invoice(invoice) -> None:\n    # Hides this frame from the traceback, so the failure points at the CALLER.\n    __tracebackhide__ = True\n    assert invoice.total == sum(line.amount for line in invoice.lines), (\n        f\"invoice {invoice.id} total {invoice.total} != sum of lines\"\n    )\n",[14,1601,1602,1606,1611,1616,1621,1626],{"__ignoreMap":299},[303,1603,1604],{"class":196,"line":305},[303,1605,348],{},[303,1607,1608],{"class":196,"line":311},[303,1609,1610],{},"    # Hides this frame from the traceback, so the failure points at the CALLER.\n",[303,1612,1613],{"class":196,"line":317},[303,1614,1615],{},"    __tracebackhide__ = True\n",[303,1617,1618],{"class":196,"line":324},[303,1619,1620],{},"    assert invoice.total == sum(line.amount for line in invoice.lines), (\n",[303,1622,1623],{"class":196,"line":330},[303,1624,1625],{},"        f\"invoice {invoice.id} total {invoice.total} != sum of lines\"\n",[303,1627,1628],{"class":196,"line":366},[303,1629,1630],{},"    )\n",[10,1632,1633],{},"With the flag set, the reported failure line is the test's call site rather than the helper's internals, which is where the reader wants to start. Without it, every failure in the suite points at the same three lines of helper code.",[10,1635,1636,1637,1640],{},"The second discipline is to keep helpers thin. A helper containing branching logic — \"if the invoice is in a foreign currency, check differently\" — is a second implementation of the behaviour under test, and when it disagrees with the code the failure is genuinely ambiguous. Helpers should assert facts, not compute expectations; anything that needs an ",[14,1638,1639],{},"if"," probably belongs in the test, where the reader can see it.",[23,1642,1644],{"id":1643},"frequently-asked-questions","Frequently Asked Questions",[10,1646,1647,1650,1651,1653,1654,1656,1657,21],{},[1540,1648,1649],{},"Why do assertions in a helper module show no introspection?","\npytest rewrites the AST of test modules and of plugins it knows about, not of arbitrary imported modules. A helper that lives outside those is imported unrewritten, so its asserts produce a bare ",[14,1652,89],{},". Register the helper with ",[14,1655,874],{}," before it is first imported, usually at the top of ",[14,1658,46],{},[10,1660,1661,1664,1665,1668,1669,1671,1672,1674],{},[1540,1662,1663],{},"How do I customise the diff pytest prints for my own types?","\nImplement ",[14,1666,1667],{},"pytest_assertrepr_compare"," in ",[14,1670,46],{}," or a plugin. It receives the operator and both operands and returns a list of lines, which pytest prints in place of the default representation. Return ",[14,1673,495],{}," for comparisons you do not handle so other implementations still get a chance.",[10,1676,1677,1680,1681,1683,1684,1687],{},[1540,1678,1679],{},"Does assertion rewriting slow down collection?","\nOnly the first time. Rewritten modules are cached as ",[14,1682,71],{}," files under ",[14,1685,1686],{},"__pycache__"," keyed on the source's mtime and size, so subsequent runs load the rewritten bytecode directly. A cold CI runner pays the rewrite once; a runner with a warm cache pays nothing.",[10,1689,1690,1693],{},[1540,1691,1692],{},"What is the difference between JUnit XML and pytest's own report hooks?","\nJUnit XML is an output format most CI platforms parse to show a test list and failure messages. Report hooks are the mechanism that lets you observe or modify each test's result as it happens, which is how you attach artefacts, add properties to the XML, or send results elsewhere.",[10,1695,1696,1699,1700,1702],{},[1540,1697,1698],{},"Can I make a failure include application logs automatically?","\nYes, with a ",[14,1701,1075],{}," hookwrapper that inspects the report and, when it failed, attaches captured output or files to the report's sections. Everything added there appears in the terminal output and in the XML, so CI shows it without anyone re-running the test.",[23,1704,1706],{"id":1705},"related-guides","Related guides",[28,1708,1709,1716,1723,1728,1735],{},[31,1710,1711,1712,21],{},"Extend the diff for domain types in ",[52,1713,1715],{"href":1714},"\u002Fadvanced-pytest-architecture-configuration\u002Fassertion-introspection-and-reporting\u002Fcustomizing-failure-output-with-assertrepr-compare\u002F","customizing failure output with assertrepr_compare",[31,1717,1718,1719,21],{},"Wire the results into CI with ",[52,1720,1722],{"href":1721},"\u002Fadvanced-pytest-architecture-configuration\u002Fassertion-introspection-and-reporting\u002Fproducing-junit-xml-reports-for-ci-dashboards\u002F","producing JUnit XML reports for CI dashboards",[31,1724,1725,1726,21],{},"Learn the hookwrapper mechanics in ",[52,1727,55],{"href":54},[31,1729,1730,1731,21],{},"Make captured logs part of the failure using ",[52,1732,1734],{"href":1733},"\u002Fadvanced-pytest-architecture-configuration\u002Fpytest-configuration-best-practices\u002Fcapturing-logs-with-caplog-and-log-cli\u002F","capturing logs with caplog and log_cli",[31,1736,1737,1738,21],{},"Collect the artefacts a failure references with ",[52,1739,1741],{"href":1740},"\u002Fsystematic-debugging-performance-profiling\u002Fdebugging-tests-in-ci-and-containers\u002Fcapturing-artifacts-from-a-failed-ci-test-run\u002F","capturing artifacts from a failed CI test run",[10,1743,1744,1745],{},"← Back to ",[52,1746,1748],{"href":1747},"\u002Fadvanced-pytest-architecture-configuration\u002F","Advanced Pytest Architecture & Configuration",[1750,1751,1752],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .sScJk, html code.shiki .sScJk{--shiki-default:#6F42C1;--shiki-dark:#B392F0}html pre.shiki code .sZZnC, html code.shiki .sZZnC{--shiki-default:#032F62;--shiki-dark:#9ECBFF}html pre.shiki code .sj4cs, html code.shiki .sj4cs{--shiki-default:#005CC5;--shiki-dark:#79B8FF}html pre.shiki code .sVt8B, html code.shiki .sVt8B{--shiki-default:#24292E;--shiki-dark:#E1E4E8}html pre.shiki code .szBVR, html code.shiki .szBVR{--shiki-default:#D73A49;--shiki-dark:#F97583}html pre.shiki code .sJ8bj, html code.shiki .sJ8bj{--shiki-default:#6A737D;--shiki-dark:#6A737D}",{"title":299,"searchDepth":311,"depth":311,"links":1754},[1755,1756,1757,1764,1765,1766,1767,1768,1769,1770,1771,1772,1773],{"id":25,"depth":311,"text":26},{"id":61,"depth":311,"text":62},{"id":286,"depth":311,"text":287,"children":1758},[1759,1760,1761,1762,1763],{"id":291,"depth":317,"text":292},{"id":378,"depth":317,"text":379},{"id":499,"depth":317,"text":500},{"id":607,"depth":317,"text":608},{"id":687,"depth":317,"text":688},{"id":734,"depth":311,"text":735},{"id":837,"depth":311,"text":838},{"id":964,"depth":311,"text":965},{"id":1066,"depth":311,"text":1067},{"id":1220,"depth":311,"text":1221},{"id":1435,"depth":311,"text":1436},{"id":1532,"depth":311,"text":1533},{"id":1574,"depth":311,"text":1575},{"id":1643,"depth":311,"text":1644},{"id":1705,"depth":311,"text":1706},"Make pytest failures explain themselves: assertion rewriting internals, custom comparison output, report hooks, JUnit XML for CI, and diagnostics attached to every failure.","md",{"slug":1777,"type":1778,"breadcrumb":1779,"datePublished":1780,"dateModified":1780,"faq":1781,"howto":1792},"assertion-introspection-and-reporting","topic","Assertions & Reporting","2026-09-18",[1782,1784,1786,1788,1790],{"q":1649,"a":1783},"pytest rewrites the AST of test modules and of plugins it knows about, not of arbitrary imported modules. A helper that lives outside those is imported unrewritten, so its asserts produce a bare AssertionError. Register the helper with pytest.register_assert_rewrite before it is first imported, usually at the top of conftest.py.",{"q":1663,"a":1785},"Implement pytest_assertrepr_compare in conftest.py or a plugin. It receives the operator and both operands and returns a list of lines, which pytest prints in place of the default representation. Return None for comparisons you do not handle so other implementations still get a chance.",{"q":1679,"a":1787},"Only the first time. Rewritten modules are cached as .pyc files under __pycache__ keyed on the source's mtime and size, so subsequent runs load the rewritten bytecode directly. A cold CI runner pays the rewrite once; a runner with a warm cache pays nothing.",{"q":1692,"a":1789},"JUnit XML is an output format most CI platforms parse to show a test list and failure messages. Report hooks are the mechanism that lets you observe or modify each test's result as it happens, which is how you attach artefacts, add properties to the XML, or send results elsewhere.",{"q":1698,"a":1791},"Yes, with a pytest_runtest_makereport hookwrapper that inspects the report and, when it failed, attaches captured output or files to the report's sections. Everything added there appears in the terminal output and in the XML, so CI shows it without anyone re-running the test.",{"name":1793,"description":1794,"steps":1795},"How to make pytest failures self-explanatory","Keep introspection working in helpers, add custom comparison output, and attach diagnostics to every failing report.",[1796,1799,1802,1805,1808],{"name":1797,"text":1798},"Register helper modules for rewriting","Call pytest.register_assert_rewrite for any shared assertion helper before it is imported so its asserts keep introspection.",{"name":1800,"text":1801},"Add a comparison representation for your types","Implement pytest_assertrepr_compare to print a meaningful diff for domain objects instead of two repr strings.",{"name":1803,"text":1804},"Attach diagnostics in a report hookwrapper","Use pytest_runtest_makereport to add logs, request bodies or screenshots to the report when the outcome is a failure.",{"name":1806,"text":1807},"Emit machine-readable results","Generate JUnit XML with junit_family=xunit2 and add per-test properties so the CI dashboard shows more than a name.",{"name":1809,"text":1810},"Verify the output a failure actually produces","Write a deliberately failing test once and read its terminal and XML output end to end.","\u002Fadvanced-pytest-architecture-configuration\u002Fassertion-introspection-and-reporting",{"title":5,"description":1774},"advanced-pytest-architecture-configuration\u002Fassertion-introspection-and-reporting\u002Findex","YkWfNx4sbE8I_LYm5pD7_vhE99MbdSxHng8yjEJoE1Y",1789718765720]