Python Testing & Debugging Beyond unittestAdvanced Python testing playbooks
Pytest & CI

Why pytest-cov Reports 0% Under xdist

The suite passes, the coverage step prints TOTAL 0%, and the only change was adding -n 8. Nothing about the tests moved — the measurement did. Under pytest-xdist each worker is a separate operating-system process with its own interpreter and its own trace function, so coverage is no longer one measurement but N, and something has to put them back together.

Prerequisites

Solution

Three settings, applied together, make parallel coverage correct. Each fixes a distinct failure.

TOML
# pyproject.toml
[tool.coverage.run]
branch = true
source = ["src/myapp"]
parallel = true                      # 1. per-process data files, suffixed by pid
concurrency = ["multiprocessing"]    # 2. trace code that runs in spawned children
data_file = "build/.coverage"        # 3. one known location, outside the repo root

[tool.pytest.ini_options]
addopts = "--cov --cov-report=term-missing --cov-report=xml"
Bash
$ pytest -n 8 -q                     # pytest-cov starts inside each worker

The single most important change is the last line: pytest --cov, never coverage run -m pytest. pytest-cov implements the xdist hooks that start measurement inside each worker process and collect the data back to the controller at session end. coverage run wraps only the controller, which spawns workers that do no measuring at all — hence exactly zero.

parallel = true makes each process write build/.coverage.hostname.pid.random instead of all of them writing build/.coverage. Without it the processes race, the last writer wins, and the report reflects one worker's view — which looks like a plausible but wrong percentage rather than a zero.

concurrency = ["multiprocessing"] extends the same treatment to processes your tests spawn themselves. It is unrelated to xdist workers, but the two are almost always needed together in suites that shell out.

Where measurement starts and where it is combined A sequence diagram with three lanes: the pytest controller, a worker process, and the coverage data files. The controller starts workers, pytest-cov starts a coverage instance inside each worker, each worker writes its own suffixed data file at exit, and the controller combines the files into one report at session end. Where measurement starts and where it is combined controller worker gw0 data files start worker pytest-cov starts tracing write .coverage.pid combine all files one dataset to report coverage run -m pytest wraps only the leftmost lane, which is why the result is zero.
The plugin has to be inside the worker for anything to be recorded, and inside the controller for anything to be combined.

Why this works

coverage.py records data in the process where its trace function is installed, and writes it to a file when that process exits. parallel = true makes those filenames unique per process, and coverage combine merges the arc data from all of them into a single dataset. pytest-cov automates both halves: it registers an xdist hook that starts a Coverage instance in each worker before tests begin, and a session-finish hook in the controller that combines whatever the workers wrote. Removing either half produces a report — it is just a report of a dataset nobody populated.

Diagnosing which half is broken

The reported number tells you which stage failed, and one command confirms it.

Bash
$ coverage debug data
-- data ------------------------------------------------------
path: /srv/app/build/.coverage
has_arcs: True
8 files:
  gw0: 1122 lines
  gw1: 1098 lines
  ...

Eight files for eight workers means both halves worked. One file means the workers never measured — the run was wrapped by coverage run, or --cov was missing. No such file means the data file path differs between the workers and the report step, which happens when the working directory changes or data_file is relative and a worker runs from elsewhere. Eight files but a low percentage means the combine happened before some workers finished writing, which in practice means a custom CI step is running coverage report in parallel with the pytest process rather than after it.

Bash
# The three symptoms, and the check that distinguishes them
$ pytest -n 8 --cov -q && coverage debug data | head -3

Run that once when setting parallel coverage up, and again whenever the number moves without a corresponding change in the tests.

Reading the symptom back to the cause A table matching four coverage symptoms under xdist - exactly zero percent, a plausible but low number, a missing data file, and coverage that varies between runs - to the cause and the fix for each. Reading the symptom back to the cause Criterion Cause Fix exactly 0% workers never measured use pytest --cov plausible but low one worker reported set parallel = true no data file path differs per process absolute data_file varies between runs combine ran too early report after pytest exits
The four symptoms map one-to-one onto the four configuration mistakes, so the number itself is the diagnostic.

Coverage in CI with parallel jobs

A second layer of the same problem appears when CI shards the suite across machines rather than across processes. Each job produces its own combined data file, and the final report needs one more combine — this time across artefacts rather than across processes.

YAML
# .github/workflows/test.yml (excerpt)
jobs:
  test:
    strategy:
      matrix: { shard: [1, 2, 3, 4] }
    steps:
      - run: pytest -n 4 --cov --cov-report= --shard=${{ matrix.shard }}/4
      - uses: actions/upload-artifact@v4
        with: { name: cov-${{ matrix.shard }}, path: build/.coverage.* }
  report:
    needs: test
    steps:
      - uses: actions/download-artifact@v4
      - run: |
          coverage combine build/.coverage.*      # merge every shard's files
          coverage report --show-missing
          coverage xml

Two details matter. --cov-report= with an empty value suppresses per-shard reporting, which is meaningless in isolation and misleading in logs. And the artefacts must preserve the per-process suffixes: uploading a file renamed to .coverage from each shard makes the four artefacts collide on download, which reintroduces the original bug at a higher level.

The same shape applies to any CI system — the requirement is only that the raw data files, not the rendered reports, are what travel between jobs.

Keeping the parallel number trustworthy

Once the combine works, the remaining risk is that it quietly stops working again. Coverage configuration is exactly the kind of setting nobody reads until the number looks wrong, so three guards are worth adding while the fix is fresh.

Assert the data-file count in CI. The check is two lines and it fails loudly at the point of regression:

Bash
$ pytest -n 8 --cov -q
$ python - <<'EOF'
import sqlite3, glob, sys
files = glob.glob("build/.coverage.*")
print(f"{len(files)} data file(s)")
sys.exit(0 if len(files) >= 8 else 1)     # one per worker, at minimum
EOF

Pin the plugin versions. pytest-cov and pytest-xdist coordinate through hooks whose names have changed across majors, and an unpinned upgrade in a CI image is the classic cause of a coverage number that halves overnight with no code change. Pin both in the lockfile and upgrade them deliberately, together.

Compare serial and parallel once per release. Running the suite with and without -n should produce the same percentage within rounding; a persistent gap means some tests are not running in the parallel configuration at all, which is a bigger problem than the coverage report. That comparison takes one extra CI job on a schedule, and it is the only check that catches a worker crashing silently mid-run.

A last operational note: keep the raw data files as CI artefacts for a few days. When someone asks why coverage moved between two commits, having both datasets makes the answer a coverage combine and a diff rather than an archaeology exercise.

Edge cases and failure modes

  • --cov with no argument measures everything importable. Combined with xdist that produces a large, slow, meaningless report; always set source in the configuration.
  • A stale data file from a previous run inflates the result. coverage erase (or a clean build directory) before the run makes each measurement independent.
  • -p no:cacheprovider breaks nothing, but a read-only filesystem does. Workers must be able to write their data files; a container with a read-only working directory fails silently at exit.
  • --dist loadgroup does not change the coverage story. Distribution mode affects which worker runs which test, never how measurement is combined.
  • Subprocess coverage needs COVERAGE_PROCESS_START in addition to everything above, because a spawned interpreter starts with no trace function at all.
  • Reporting inside the pytest run and again in CI double-counts nothing but wastes time. Pick one place to render the report; the data file is the artefact that matters.

One related surprise: coverage under -n 0 is not the same code path as coverage with no -n at all. -n 0 still loads xdist and runs tests in the controller process, which exercises the plugin's serial path; omitting the flag skips xdist entirely. When debugging a combine problem, compare against a run with the flag removed rather than set to zero, so the comparison isolates the parallel machinery.

Narrowing a parallel coverage problem A left-to-right diagnostic sequence: run serially to establish the true number, run with one worker to test the plugin path, run with eight workers to test the combine, and compare the data file counts at each step. Narrowing a parallel coverage problem serial run the true baseline one worker plugin path only eight workers combine under load compare counts where it diverges
Each step adds one variable, so the step where the number changes names the broken stage.

Frequently Asked Questions

Why does coverage report 0% only when I add -n? Because each xdist worker is a separate process writing its own coverage data file. Unless pytest-cov owns the run and combines those files, the report is generated from a data file that no worker wrote to, which reads as zero.

Can I run coverage run -m pytest with xdist? Not reliably. That form starts coverage in the controller process only, and the workers inherit nothing. Use pytest --cov so the plugin can start measurement inside each worker and combine the results at the end.

Does parallel = true alone fix it? No. parallel = true makes each process write a uniquely suffixed data file, which prevents them overwriting each other, but something still has to combine them. pytest-cov does that automatically; a manual coverage run does not.

Why is coverage lower rather than zero under xdist? Usually a partial combine: some data files were written after the combine ran, or a subprocess wrote a file the combine never saw. Check the file count with coverage debug data before assuming the tests themselves changed.

← Back to Coverage Measurement and Enforcement