[{"data":1,"prerenderedAt":948},["ShallowReactive",2],{"page-\u002Fsystematic-debugging-performance-profiling\u002Flogging-and-observability-for-debugging\u002Ftracing-a-request-through-a-test-with-opentelemetry\u002F":3},{"id":4,"title":5,"body":6,"description":914,"extension":915,"meta":916,"navigation":113,"path":944,"seo":945,"stem":946,"__hash__":947},"content\u002Fsystematic-debugging-performance-profiling\u002Flogging-and-observability-for-debugging\u002Ftracing-a-request-through-a-test-with-opentelemetry\u002Findex.md","Tracing a Request Through a Test with OpenTelemetry",{"type":7,"value":8,"toc":903},"minimark",[9,18,21,26,57,61,261,324,332,440,444,459,466,477,481,484,495,530,533,544,607,611,614,625,628,638,735,739,746,757,760,764,819,823,843,855,861,865,894,899],[10,11,12,13,17],"p",{},"Logs tell you what happened; traces tell you what happened ",[14,15,16],"em",{},"inside what",". When an integration test fails because a request took the wrong branch, a trace shows the whole path: the handler span, the database query spans under it, the outbound HTTP call to the pricing service that returned an error, and the retry that followed. Each span carries timing, attributes and status, and the parent-child structure makes the order of events and their nesting explicit in a way interleaved log lines never do.",[10,19,20],{},"OpenTelemetry makes that tree available inside a test with very little setup. An in-memory exporter collects finished spans instead of sending them anywhere; the test can then assert that required attributes are present, or — more often — print the tree when the test fails so the next person to debug it can see where the request went. The main pitfalls are global state (the tracer provider can be set only once per process) and asynchronous export, both of which have simple fixes.",[22,23,25],"h2",{"id":24},"prerequisites","Prerequisites",[27,28,29,49],"ul",{},[30,31,32,36,37,40,41,44,45,48],"li",{},[33,34,35],"code",{},"opentelemetry-sdk >= 1.25",", ",[33,38,39],{},"opentelemetry-instrumentation-httpx"," and ",[33,42,43],{},"-sqlalchemy"," (optional), ",[33,46,47],{},"pytest >= 8.0",".",[30,50,51,52,48],{},"Background from ",[53,54,56],"a",{"href":55},"\u002Fsystematic-debugging-performance-profiling\u002Flogging-and-observability-for-debugging\u002F","Logging and observability for debugging",[22,58,60],{"id":59},"solution","Solution",[62,63,68],"pre",{"className":64,"code":65,"language":66,"meta":67,"style":67},"language-python shiki shiki-themes github-light github-dark","# conftest.py\nimport pytest\nfrom opentelemetry import trace\nfrom opentelemetry.sdk.trace import TracerProvider\nfrom opentelemetry.sdk.trace.export import SimpleSpanProcessor\nfrom opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter\n\n_EXPORTER = InMemorySpanExporter()\n\ndef pytest_configure(config):\n    provider = TracerProvider()\n    provider.add_span_processor(SimpleSpanProcessor(_EXPORTER))   # synchronous export\n    trace.set_tracer_provider(provider)                           # once per process\n\n@pytest.fixture\ndef spans():\n    _EXPORTER.clear()\n    yield _EXPORTER\n    _EXPORTER.clear()\n\ndef render_tree(finished):\n    by_parent = {}\n    for s in finished:\n        by_parent.setdefault(s.parent.span_id if s.parent else None, []).append(s)\n    lines = []\n    def walk(parent, depth):\n        for s in sorted(by_parent.get(parent, []), key=lambda s: s.start_time):\n            ms = (s.end_time - s.start_time) \u002F 1e6\n            lines.append(f\"{'  ' * depth}{s.name}  {ms:.1f} ms  {s.status.status_code.name}\")\n            walk(s.context.span_id, depth + 1)\n    walk(None, 0)\n    return \"\\n\".join(lines)\n","python","",[33,69,70,78,84,90,96,102,108,115,121,126,132,138,144,150,155,161,167,173,179,184,189,195,201,207,213,219,225,231,237,243,249,255],{"__ignoreMap":67},[71,72,75],"span",{"class":73,"line":74},"line",1,[71,76,77],{},"# conftest.py\n",[71,79,81],{"class":73,"line":80},2,[71,82,83],{},"import pytest\n",[71,85,87],{"class":73,"line":86},3,[71,88,89],{},"from opentelemetry import trace\n",[71,91,93],{"class":73,"line":92},4,[71,94,95],{},"from opentelemetry.sdk.trace import TracerProvider\n",[71,97,99],{"class":73,"line":98},5,[71,100,101],{},"from opentelemetry.sdk.trace.export import SimpleSpanProcessor\n",[71,103,105],{"class":73,"line":104},6,[71,106,107],{},"from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter\n",[71,109,111],{"class":73,"line":110},7,[71,112,114],{"emptyLinePlaceholder":113},true,"\n",[71,116,118],{"class":73,"line":117},8,[71,119,120],{},"_EXPORTER = InMemorySpanExporter()\n",[71,122,124],{"class":73,"line":123},9,[71,125,114],{"emptyLinePlaceholder":113},[71,127,129],{"class":73,"line":128},10,[71,130,131],{},"def pytest_configure(config):\n",[71,133,135],{"class":73,"line":134},11,[71,136,137],{},"    provider = TracerProvider()\n",[71,139,141],{"class":73,"line":140},12,[71,142,143],{},"    provider.add_span_processor(SimpleSpanProcessor(_EXPORTER))   # synchronous export\n",[71,145,147],{"class":73,"line":146},13,[71,148,149],{},"    trace.set_tracer_provider(provider)                           # once per process\n",[71,151,153],{"class":73,"line":152},14,[71,154,114],{"emptyLinePlaceholder":113},[71,156,158],{"class":73,"line":157},15,[71,159,160],{},"@pytest.fixture\n",[71,162,164],{"class":73,"line":163},16,[71,165,166],{},"def spans():\n",[71,168,170],{"class":73,"line":169},17,[71,171,172],{},"    _EXPORTER.clear()\n",[71,174,176],{"class":73,"line":175},18,[71,177,178],{},"    yield _EXPORTER\n",[71,180,182],{"class":73,"line":181},19,[71,183,172],{},[71,185,187],{"class":73,"line":186},20,[71,188,114],{"emptyLinePlaceholder":113},[71,190,192],{"class":73,"line":191},21,[71,193,194],{},"def render_tree(finished):\n",[71,196,198],{"class":73,"line":197},22,[71,199,200],{},"    by_parent = {}\n",[71,202,204],{"class":73,"line":203},23,[71,205,206],{},"    for s in finished:\n",[71,208,210],{"class":73,"line":209},24,[71,211,212],{},"        by_parent.setdefault(s.parent.span_id if s.parent else None, []).append(s)\n",[71,214,216],{"class":73,"line":215},25,[71,217,218],{},"    lines = []\n",[71,220,222],{"class":73,"line":221},26,[71,223,224],{},"    def walk(parent, depth):\n",[71,226,228],{"class":73,"line":227},27,[71,229,230],{},"        for s in sorted(by_parent.get(parent, []), key=lambda s: s.start_time):\n",[71,232,234],{"class":73,"line":233},28,[71,235,236],{},"            ms = (s.end_time - s.start_time) \u002F 1e6\n",[71,238,240],{"class":73,"line":239},29,[71,241,242],{},"            lines.append(f\"{'  ' * depth}{s.name}  {ms:.1f} ms  {s.status.status_code.name}\")\n",[71,244,246],{"class":73,"line":245},30,[71,247,248],{},"            walk(s.context.span_id, depth + 1)\n",[71,250,252],{"class":73,"line":251},31,[71,253,254],{},"    walk(None, 0)\n",[71,256,258],{"class":73,"line":257},32,[71,259,260],{},"    return \"\\n\".join(lines)\n",[62,262,264],{"className":64,"code":263,"language":66,"meta":67,"style":67},"# test_checkout_trace.py\nfrom opentelemetry.trace import StatusCode\n\ndef test_checkout_marks_pricing_failure(client, spans, pricing_down):\n    resp = client.post(\"\u002Fcheckout\", json={\"cart\": \"c1\"})\n    assert resp.status_code == 503, render_tree(spans.get_finished_spans())\n\n    finished = {s.name: s for s in spans.get_finished_spans()}\n    call = finished[\"POST pricing\"]\n    assert call.status.status_code is StatusCode.ERROR\n    assert call.attributes[\"http.response.status_code\"] == 500\n    assert finished[\"checkout\"].attributes[\"cart.id\"] == \"c1\"\n",[33,265,266,271,276,280,285,290,295,299,304,309,314,319],{"__ignoreMap":67},[71,267,268],{"class":73,"line":74},[71,269,270],{},"# test_checkout_trace.py\n",[71,272,273],{"class":73,"line":80},[71,274,275],{},"from opentelemetry.trace import StatusCode\n",[71,277,278],{"class":73,"line":86},[71,279,114],{"emptyLinePlaceholder":113},[71,281,282],{"class":73,"line":92},[71,283,284],{},"def test_checkout_marks_pricing_failure(client, spans, pricing_down):\n",[71,286,287],{"class":73,"line":98},[71,288,289],{},"    resp = client.post(\"\u002Fcheckout\", json={\"cart\": \"c1\"})\n",[71,291,292],{"class":73,"line":104},[71,293,294],{},"    assert resp.status_code == 503, render_tree(spans.get_finished_spans())\n",[71,296,297],{"class":73,"line":110},[71,298,114],{"emptyLinePlaceholder":113},[71,300,301],{"class":73,"line":117},[71,302,303],{},"    finished = {s.name: s for s in spans.get_finished_spans()}\n",[71,305,306],{"class":73,"line":123},[71,307,308],{},"    call = finished[\"POST pricing\"]\n",[71,310,311],{"class":73,"line":128},[71,312,313],{},"    assert call.status.status_code is StatusCode.ERROR\n",[71,315,316],{"class":73,"line":134},[71,317,318],{},"    assert call.attributes[\"http.response.status_code\"] == 500\n",[71,320,321],{"class":73,"line":140},[71,322,323],{},"    assert finished[\"checkout\"].attributes[\"cart.id\"] == \"c1\"\n",[62,325,330],{"className":326,"code":328,"language":329,"meta":67},[327],"language-text","AssertionError:\ncheckout  48.2 ms  ERROR\n  SELECT carts  1.9 ms  UNSET\n  POST pricing  20.4 ms  ERROR\n  POST pricing  22.1 ms  ERROR\n","text",[33,331,328],{"__ignoreMap":67},[333,334,337,436],"figure",{"className":335},[336],"diagram",[338,339,346,347,346,351,346,355,346,363,346,372,346,382,346,387,346,394,346,403,346,408,346,412,346,417,346,421,346,426,346,432],"svg",{"viewBox":340,"role":341,"ariaLabelledBy":342,"xmlns":345},"0 0 800 256","img",[343,344],"ot-t","ot-d","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","\n  ",[348,349,350],"title",{"id":343},"A request's span tree captured in a test",[352,353,354],"desc",{"id":344},"A root span named checkout contains three child spans laid out on a timeline: a SELECT carts database span, then two POST pricing HTTP spans, both marked as errors, showing a retry. The in-memory exporter collects all four finished spans so the test can assert on them or print the tree.",[356,357],"rect",{"x":358,"y":358,"width":359,"height":360,"rx":361,"fill":362},"0","800","256","14","#fffdf8",[329,364,371],{"x":365,"y":366,"textAnchor":367,"fontSize":368,"fontWeight":369,"fill":370},"400","28","middle","15.5","700","#3d405b","One request, one tree",[356,373],{"x":374,"y":375,"width":376,"height":377,"rx":378,"fill":379,"stroke":380,"strokeWidth":381},"60","56","680","30","6","#fbe9e3","#e07a5f","1.8",[329,383,386],{"x":384,"y":384,"fontSize":385,"fontWeight":369,"fill":370},"76","11.5","checkout",[329,388,393],{"x":389,"y":384,"textAnchor":390,"fontSize":391,"fill":392},"724","end","10.5","#8f3d22","48.2 ms · ERROR",[356,395],{"x":396,"y":397,"width":374,"height":398,"rx":399,"fill":400,"stroke":401,"strokeWidth":402},"90","100","26","5","#e6f0ea","#81b29a","1.6",[329,404,407],{"x":405,"y":406,"fontSize":391,"fill":370},"160","118","SELECT carts · 1.9 ms",[356,409],{"x":405,"y":410,"width":411,"height":398,"rx":399,"fill":379,"stroke":380,"strokeWidth":402},"140","280",[329,413,416],{"x":414,"y":415,"fontSize":391,"fill":370},"176","158","POST pricing · 500",[356,418],{"x":419,"y":420,"width":411,"height":398,"rx":399,"fill":379,"stroke":380,"strokeWidth":402},"450","180",[329,422,425],{"x":423,"y":424,"fontSize":391,"fill":370},"466","198","POST pricing (retry) · 500",[73,427],{"x1":374,"y1":428,"x2":429,"y2":428,"stroke":430,"strokeWidth":431},"226","740","rgba(61,64,91,0.4)","1.2",[329,433,435],{"x":365,"y":434,"textAnchor":367,"fontSize":391,"fill":370},"244","time →",[437,438,439],"figcaption",{},"The tree shows at a glance that the failure came from pricing, that it was retried once, and how long each step took.",[22,441,443],{"id":442},"why-this-works","Why this works",[10,445,446,447,450,451,454,455,458],{},"The SDK's ",[33,448,449],{},"TracerProvider"," sends every finished span to its processors. ",[33,452,453],{},"SimpleSpanProcessor"," exports each span synchronously as it ends, so by the time the request returns, every span it created is already in the in-memory exporter. The production default, ",[33,456,457],{},"BatchSpanProcessor",", exports on a background thread in batches, which is efficient but means a test can finish before its spans arrive; in tests, synchronous export removes that race.",[10,460,461,462,465],{},"The provider is set in ",[33,463,464],{},"pytest_configure"," because OpenTelemetry's global provider can be set only once — later calls log a warning and are ignored. Setting it at session start guarantees every tracer created by application code or instrumentation libraries during the run uses the test provider. The function-scoped fixture then only needs to clear the exporter, so each test sees just its own spans.",[10,467,468,469,472,473,476],{},"Instrumentation libraries do the rest. ",[33,470,471],{},"HTTPXClientInstrumentor().instrument()"," wraps outgoing requests in client spans with standard attributes; ",[33,474,475],{},"SQLAlchemyInstrumentor().instrument(engine=engine)"," does the same for queries. Because they use the global provider, their spans land in the same exporter and nest under whatever span was current when the call was made.",[22,478,480],{"id":479},"traces-as-a-debugging-attachment","Traces as a debugging attachment",[10,482,483],{},"Most tests should not assert on spans at all. Span names and attributes change as instrumentation libraries evolve, and asserting on them couples tests to observability details. The more durable use is as a debugging aid: when a test fails, print the tree.",[10,485,486,487,490,491,494],{},"A hook in ",[33,488,489],{},"conftest.py"," can do that automatically for every test that uses the ",[33,492,493],{},"spans"," fixture:",[62,496,498],{"className":64,"code":497,"language":66,"meta":67,"style":67},"@pytest.hookimpl(hookwrapper=True)\ndef pytest_runtest_makereport(item, call):\n    outcome = yield\n    rep = outcome.get_result()\n    if rep.when == \"call\" and rep.failed and \"spans\" in item.fixturenames:\n        rep.sections.append((\"trace\", render_tree(_EXPORTER.get_finished_spans())))\n",[33,499,500,505,510,515,520,525],{"__ignoreMap":67},[71,501,502],{"class":73,"line":74},[71,503,504],{},"@pytest.hookimpl(hookwrapper=True)\n",[71,506,507],{"class":73,"line":80},[71,508,509],{},"def pytest_runtest_makereport(item, call):\n",[71,511,512],{"class":73,"line":86},[71,513,514],{},"    outcome = yield\n",[71,516,517],{"class":73,"line":92},[71,518,519],{},"    rep = outcome.get_result()\n",[71,521,522],{"class":73,"line":98},[71,523,524],{},"    if rep.when == \"call\" and rep.failed and \"spans\" in item.fixturenames:\n",[71,526,527],{"class":73,"line":104},[71,528,529],{},"        rep.sections.append((\"trace\", render_tree(_EXPORTER.get_finished_spans())))\n",[10,531,532],{},"The failure report then contains a \"trace\" section next to captured logs and stdout. For flaky integration tests in particular, where the failure is rare and the logs are noisy, a compact tree of what the request did — which calls, in which order, how long each took, which failed — is often the fastest route to the cause.",[10,534,535,536,539,540,543],{},"Assert on spans only where tracing itself is a requirement: a dashboard depends on ",[33,537,538],{},"cart.id"," being set, an alert depends on errors being marked with ",[33,541,542],{},"StatusCode.ERROR",", a service-level objective depends on a span existing for every checkout. Those assertions protect real consumers of the telemetry and are worth their maintenance cost.",[333,545,547,604],{"className":546},[336],[338,548,346,553,346,556,346,559,346,561,346,564,346,572,346,576,346,581,346,585,346,589,346,592,346,596,346,599],{"viewBox":549,"role":341,"ariaLabelledBy":550,"xmlns":345},"0 0 800 226",[551,552],"otd-t","otd-d",[348,554,555],{"id":551},"Where the trace appears in a failure report",[352,557,558],{"id":552},"A pytest failure report for a failing test contains stacked sections: the assertion error, captured log call, captured stdout, and a trace section added by a report hook, which shows the span tree of the request that the test made.",[356,560],{"x":358,"y":358,"width":359,"height":428,"rx":361,"fill":362},[329,562,563],{"x":365,"y":366,"textAnchor":367,"fontSize":368,"fontWeight":369,"fill":370},"A failure report with the trace attached",[356,565],{"x":410,"y":566,"width":567,"height":568,"rx":569,"fill":570,"stroke":370,"strokeWidth":571},"46","520","164","12","#f4f1de","1.5",[356,573],{"x":405,"y":374,"width":574,"height":377,"rx":378,"fill":379,"stroke":380,"strokeWidth":575},"480","1.4",[329,577,580],{"x":414,"y":578,"fontSize":579,"fill":392},"80","11","E  assert 200 == 503",[356,582],{"x":405,"y":583,"width":574,"height":377,"rx":378,"fill":362,"stroke":584},"96","rgba(61,64,91,0.35)",[329,586,588],{"x":414,"y":587,"fontSize":579,"fill":370},"116","Captured log call",[356,590],{"x":405,"y":591,"width":574,"height":377,"rx":378,"fill":362,"stroke":584},"132",[329,593,595],{"x":414,"y":594,"fontSize":579,"fill":370},"152","Captured stdout call",[356,597],{"x":405,"y":598,"width":574,"height":377,"rx":378,"fill":400,"stroke":401,"strokeWidth":381},"168",[329,600,603],{"x":414,"y":601,"fontSize":579,"fontWeight":369,"fill":602},"188","#2a5f49","trace — checkout ▸ SELECT ▸ POST pricing ×2",[437,605,606],{},"Attached automatically, the tree turns a bare assertion error into a readable account of the request.",[22,608,610],{"id":609},"following-a-request-across-services-in-an-integration-test","Following a request across services in an integration test",[10,612,613],{},"The in-memory exporter captures spans from one process. Integration tests often start the service under test in a subprocess or container, and the interesting spans are produced there. Two approaches keep the whole path visible.",[10,615,616,617,620,621,624],{},"The first is to run a collector for the test session. Start an OpenTelemetry Collector container (or the lightweight Jaeger all-in-one image) in a session fixture, point the service's ",[33,618,619],{},"OTEL_EXPORTER_OTLP_ENDPOINT"," at it, and query the collector's API at the end of a failing test for all spans with the test's trace id. The test process starts the trace — its HTTP client instrumentation injects a ",[33,622,623],{},"traceparent"," header into the request — and the service continues it, so everything shares one trace id and can be retrieved as a single tree.",[10,626,627],{},"The second is lighter: have the service write spans to a file with the console exporter, one JSON object per line, into a directory the test can read. After a failure, the test loads the file, filters by trace id and renders the same tree. There is no extra container, and the file doubles as a CI artefact.",[10,629,630,631,633,634,637],{},"In both cases, the key detail is propagation. Without the ",[33,632,623],{}," header, the service starts a new trace for every request and the test cannot find its spans. The HTTP client instrumentation adds the header automatically; hand-built requests with a bare socket or a mocked transport need ",[33,635,636],{},"opentelemetry.propagate.inject(headers)"," called explicitly.",[333,639,641,732],{"className":640},[336],[338,642,346,647,346,650,346,653,346,669,346,672,346,675,346,680,346,684,346,688,346,693,346,696,346,699,346,702,346,706,346,709,346,715,346,719,346,723,346,729],{"viewBox":643,"role":341,"ariaLabelledBy":644,"xmlns":345},"0 0 800 236",[645,646],"otx-t","otx-d",[348,648,649],{"id":645},"One trace across the test and the service",[352,651,652],{"id":646},"The test process starts a trace and its HTTP client injects a traceparent header into the request. The service process continues the same trace and exports its spans to a collector or a file. After a failure, the test retrieves all spans with its trace id and renders a single tree covering both processes.",[654,655,656,657,346],"defs",{},"\n    ",[658,659,665],"marker",{"id":660,"viewBox":661,"refX":662,"refY":399,"markerWidth":663,"markerHeight":663,"orient":664},"otx-a","0 0 10 10","9","7","auto-start-reverse",[666,667],"path",{"d":668,"fill":401},"M0 0 L10 5 L0 10 z",[356,670],{"x":358,"y":358,"width":359,"height":671,"rx":361,"fill":362},"236",[329,673,674],{"x":365,"y":366,"textAnchor":367,"fontSize":368,"fontWeight":369,"fill":370},"traceparent ties the two processes together",[356,676],{"x":398,"y":677,"width":678,"height":578,"rx":579,"fill":400,"stroke":401,"strokeWidth":679},"70","200","2",[329,681,683],{"x":682,"y":397,"textAnchor":367,"fontSize":569,"fontWeight":369,"fill":370},"126","test process",[329,685,687],{"x":682,"y":686,"textAnchor":367,"fontSize":391,"fill":370},"122","starts trace abc…",[356,689],{"x":690,"y":677,"width":678,"height":578,"rx":579,"fill":691,"stroke":692,"strokeWidth":679},"300","#f7f0da","#f2cc8f",[329,694,695],{"x":365,"y":397,"textAnchor":367,"fontSize":569,"fontWeight":369,"fill":370},"service process",[329,697,698],{"x":365,"y":686,"textAnchor":367,"fontSize":391,"fill":370},"continues trace abc…",[356,700],{"x":701,"y":677,"width":678,"height":578,"rx":579,"fill":370},"574",[329,703,705],{"x":704,"y":397,"textAnchor":367,"fontSize":569,"fontWeight":369,"fill":362},"674","collector or file",[329,707,708],{"x":704,"y":686,"textAnchor":367,"fontSize":391,"fill":362},"spans by trace id",[73,710],{"x1":711,"y1":712,"x2":713,"y2":712,"stroke":401,"strokeWidth":381,"markerEnd":714},"230","110","296","url(#otx-a)",[329,716,623],{"x":717,"y":397,"textAnchor":367,"fontSize":718,"fill":602},"263","9.5",[73,720],{"x1":721,"y1":712,"x2":722,"y2":712,"stroke":401,"strokeWidth":381,"markerEnd":714},"504","570",[666,724],{"d":725,"fill":726,"stroke":380,"strokeWidth":381,"strokeDashArray":727,"markerEnd":714},"M674 154 C 674 206, 126 206, 126 154","none",[378,728],"4",[329,730,731],{"x":365,"y":678,"textAnchor":367,"fontSize":391,"fill":392},"on failure: fetch and render the whole tree",[437,733,734],{},"With propagation in place, the failing test can show spans from every process the request touched.",[22,736,738],{"id":737},"using-span-timings-to-explain-slow-tests","Using span timings to explain slow tests",[10,740,741,742,745],{},"Spans also answer a question that is awkward with other tools: ",[14,743,744],{},"why is this particular test slow?"," A profiler shows where CPU time goes, but integration tests are usually slow because they wait — on the database, on HTTP calls, on retries with backoff. Span durations measure exactly that waiting, per operation.",[10,747,748,749,752,753,756],{},"Rendering the tree for a slow test often makes the problem obvious. A test that takes three seconds might show a single fast request followed by two pricing calls of a second each, revealing that the fake pricing service was configured with a realistic timeout the test never needed. Another might show forty sequential ",[33,750,751],{},"SELECT"," spans where one query with a join would do — an N+1 query pattern that is equally slow in production. Neither would stand out in a ",[33,754,755],{},"--durations"," report, which only gives the test's total time.",[10,758,759],{},"A small addition to the report hook — appending the tree when a test exceeds a duration threshold, not only when it fails — turns this into a routine check. Slow tests then carry their own explanation in the CI output.",[22,761,763],{"id":762},"edge-cases-and-failure-modes","Edge cases and failure modes",[27,765,766,780,786,803,809],{},[30,767,768,772,773,776,777,779],{},[769,770,771],"strong",{},"Provider already set."," If application import code calls ",[33,774,775],{},"set_tracer_provider"," before ",[33,778,464],{},", the test provider is ignored. Make application setup skip provider creation when one exists, or gate it behind an environment variable.",[30,781,782,785],{},[769,783,784],{},"Spans from other tests."," Background threads that finish after their test leak spans into the next one. Join threads in teardown, or filter by trace id.",[30,787,788,791,792,795,796,799,800,48],{},[769,789,790],{},"Async context propagation."," Spans started in ",[33,793,794],{},"asyncio"," tasks nest correctly only if context propagates; tasks created with ",[33,797,798],{},"create_task"," copy context automatically, but thread pools do not — use ",[33,801,802],{},"contextvars.copy_context().run",[30,804,805,808],{},[769,806,807],{},"xdist."," Each worker process has its own provider and exporter; nothing needs sharing, but do not expect spans from one worker in another.",[30,810,811,814,815,818],{},[769,812,813],{},"Sampling."," A sampler configured from the environment may drop spans. Use ",[33,816,817],{},"ALWAYS_ON"," in the test provider.",[22,820,822],{"id":821},"frequently-asked-questions","Frequently Asked Questions",[10,824,825,828,829,831,832,834,835,838,839,842],{},[769,826,827],{},"How do I capture OpenTelemetry spans in a pytest test?","\nConfigure a ",[33,830,449],{}," with a ",[33,833,453],{}," and an ",[33,836,837],{},"InMemorySpanExporter"," in a fixture, run the code under test, then read ",[33,840,841],{},"exporter.get_finished_spans()",". Clear the exporter between tests.",[10,844,845,848,849,851,852,854],{},[769,846,847],{},"Why are no spans exported in my test?","\nUsually because the global tracer provider was set before the test fixture ran, since OpenTelemetry allows setting it only once, or because a ",[33,850,457],{}," has not flushed. Use ",[33,853,453],{}," in tests and set the provider once per session.",[10,856,857,860],{},[769,858,859],{},"Should tests assert on spans?","\nOnly where tracing is part of the contract, such as required attributes for dashboards or error status on failures. Otherwise use spans as a debugging aid attached to failing tests rather than as assertions.",[22,862,864],{"id":863},"related","Related",[27,866,867,873,880,887],{},[30,868,869,872],{},[53,870,871],{"href":55},"Logging and Observability for Debugging"," — logs, metrics and traces for debugging.",[30,874,875,879],{},[53,876,878],{"href":877},"\u002Fsystematic-debugging-performance-profiling\u002Flogging-and-observability-for-debugging\u002Fstructured-logging-that-survives-pytest-capture\u002F","Structured Logging That Survives pytest Capture"," — logs to pair with spans.",[30,881,882,886],{},[53,883,885],{"href":884},"\u002Fsystematic-debugging-performance-profiling\u002Fdebugging-async-code-and-event-loops\u002Ftracking-down-a-hung-await-with-task-stacks\u002F","Tracking Down a Hung await with Task Stacks"," — when a span never ends.",[30,888,889,893],{},[53,890,892],{"href":891},"\u002Fsystematic-debugging-performance-profiling\u002Fdebugging-tests-in-ci-and-containers\u002Fcapturing-artifacts-from-a-failed-ci-test-run\u002F","Capturing Artifacts from a Failed CI Test Run"," — exporting traces as CI artefacts.",[10,895,896,897],{},"← Back to ",[53,898,871],{"href":55},[900,901,902],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":67,"searchDepth":80,"depth":80,"links":904},[905,906,907,908,909,910,911,912,913],{"id":24,"depth":80,"text":25},{"id":59,"depth":80,"text":60},{"id":442,"depth":80,"text":443},{"id":479,"depth":80,"text":480},{"id":609,"depth":80,"text":610},{"id":737,"depth":80,"text":738},{"id":762,"depth":80,"text":763},{"id":821,"depth":80,"text":822},{"id":863,"depth":80,"text":864},"Use OpenTelemetry spans inside pytest: an in-memory exporter fixture, asserting on span trees, attributes and errors, instrumenting HTTP and database clients, and reading traces from failing tests.","md",{"slug":917,"type":918,"breadcrumb":919,"datePublished":920,"dateModified":920,"faq":921,"howto":928},"tracing-a-request-through-a-test-with-opentelemetry","article","OpenTelemetry in tests","2026-09-18",[922,924,926],{"q":827,"a":923},"Configure a TracerProvider with a SimpleSpanProcessor and an InMemorySpanExporter in a fixture, run the code under test, then read exporter.get_finished_spans(). Clear the exporter between tests.",{"q":847,"a":925},"Usually because the global tracer provider was set before the test fixture ran, since OpenTelemetry allows setting it only once, or because a BatchSpanProcessor has not flushed. Use SimpleSpanProcessor in tests and set the provider once per session.",{"q":859,"a":927},"Only where tracing is part of the contract, such as required attributes for dashboards or error status on failures. Otherwise use spans as a debugging aid attached to failing tests rather than as assertions.",{"name":929,"description":930,"steps":931},"How to trace a request through a pytest test","Install an in-memory exporter, run the request, and inspect or assert on the finished span tree.",[932,935,938,941],{"name":933,"text":934},"Set a session tracer provider","Create a TracerProvider with an InMemorySpanExporter via SimpleSpanProcessor and set it globally once.",{"name":936,"text":937},"Clear spans per test","Yield the exporter from a function-scoped fixture and clear it before each test.",{"name":939,"text":940},"Instrument clients","Enable the HTTP and database instrumentations so outbound calls produce child spans.",{"name":942,"text":943},"Inspect the tree","Group spans by parent id to rebuild the request tree, and print it when a test fails.","\u002Fsystematic-debugging-performance-profiling\u002Flogging-and-observability-for-debugging\u002Ftracing-a-request-through-a-test-with-opentelemetry",{"title":5,"description":914},"systematic-debugging-performance-profiling\u002Flogging-and-observability-for-debugging\u002Ftracing-a-request-through-a-test-with-opentelemetry\u002Findex","xsfsK_4xcXtZ2LYJrZZsQmJhlD8Tt_eAM5Q2e6l1cYM",1789718769196]