Running Hypothesis Under pytest-xdist

Property tests are some of the slowest in a suite, which makes them natural candidates for pytest-xdist. The combination mostly just works — Hypothesis tests are independent test items like any others, and xdist distributes them across workers — but three details change under parallelism. The on-disk example database is shared by default, deadlines measured in wall-clock time become noisy when workers compete for CPU, and reproducing a failure requires knowing that the worker it happened on does not matter.

Handling those three details is a few lines of configuration. The result is property tests that run at the speed of the parallel suite, with failures that are as easy to reproduce as in a serial run.

Prerequisites

hypothesis >= 6.100, pytest-xdist >= 3.5, pytest >= 8.0.
Settings profiles as in Hypothesis integration with pytest and frameworks.

Solution

# conftest.py
import os

from hypothesis import settings
from hypothesis.database import DirectoryBasedExampleDatabase

worker = os.environ.get("PYTEST_XDIST_WORKER", "master")

settings.register_profile(
    "ci",
    max_examples=300,
    deadline=None,                      # contention, not code, dominates timing here
    print_blob=True,                    # every failure prints @reproduce_failure(...)
    database=DirectoryBasedExampleDatabase(f".hypothesis/examples/{worker}"),
)
settings.register_profile("dev", max_examples=30)
settings.load_profile(os.environ.get("HYPOTHESIS_PROFILE", "dev"))

# CI: parallel, with the CI profile.
HYPOTHESIS_PROFILE=ci pytest -n auto -q

# Locally: reproduce a failure reported from worker gw5, serially.
pytest tests/test_parser.py::test_round_trip -q    # after pasting @reproduce_failure

The blob, not the worker's database, is what makes a CI failure reproducible locally; the per-worker directories simply stop workers interfering.

Why this works

Each xdist worker is a separate process that loads the configuration independently, so reading PYTEST_XDIST_WORKER when registering the profile gives every worker its own database directory. The workers then never write to the same place, which removes the rare but confusing races where one worker deletes an entry another is replaying.

Reproduction does not depend on the worker because Hypothesis's generation is a function of the test and its seed, not of the process that ran it. print_blob=True makes each failure print a @reproduce_failure decorator encoding the exact choices that produced the failing example; pasting it onto the test and running without -n replays that example deterministically on any machine.

Edge cases and failure modes

Deadlines under contention. Eight workers on four cores slow every example, and a deadline tuned on an idle laptop fires spuriously. Disable or widen it in the parallel profile.
Budgets multiplied unintentionally. max_examples is per test, not per run, so parallelism does not reduce the number of examples — it only spreads them. Keep the CI budget what CI can afford serially divided by the worker count.
Health checks tripping on slow generation. HealthCheck.too_slow measures generation time, which also suffers under contention. Suppress it only in the CI profile if it fires spuriously there.
Stale per-worker databases. Cached across CI runs, a worker's database can replay an example from a different code version. Clear them when caching, or disable the database in CI.
Blob versions. A @reproduce_failure blob is tied to the Hypothesis version that printed it. Reproduce with the same version, pinned in the lockfile.

Choosing whether to keep the database in CI

The per-worker database is one of two reasonable CI strategies, and the other — no database at all — is often simpler.

With database=None in the CI profile, every CI run is a fresh search. Nothing is replayed, so nothing stale can be replayed, and there are no directories to cache, isolate or clean. Failures are reproduced through the printed blob, and any failure worth keeping becomes an explicit @example in the test file, where it runs on every future execution and is visible in code review.

Keeping per-worker databases, and caching them between runs, adds replay: a failure found on Monday is retried first on Tuesday, even if the random search would not have found it again. That is valuable for long-running nightly searches, where a hard-won counterexample should not be lost. It is less valuable on pull requests, where the pinned @example approach gives the same durability with less machinery.

A common split is database=None for pull-request runs and per-worker databases, persisted as artefacts, for the nightly deep search. The pull-request suite stays simple and deterministic in its inputs; the nightly search accumulates counterexamples and promotes the important ones to explicit examples.

Either stage alone works; together they give fast deterministic pull-request runs and a search that keeps what it finds.

Sizing the budget for a parallel run

Parallelism changes how long the suite takes, not how much work it does. Every property test still runs max_examples examples; xdist only decides which worker runs them. That has two consequences worth planning for.

First, the wall-clock time of the property tests is bounded below by the slowest single test. If one stateful machine takes ninety seconds at the CI budget, no number of workers makes the suite finish faster than ninety seconds, because a single test is never split across workers. When the property tests dominate the run, look at the distribution of per-test durations with --durations=20 before adding workers: a long tail of one or two expensive tests is fixed by lowering their individual budgets or moving them to the nightly profile, not by more parallelism.

Second, the load balancing only works if the scheduler can see the work. The default --dist load sends tests to idle workers in small batches, which copes well with uneven test durations. --dist loadfile and --dist loadscope group tests by module or class, which is useful when module-scoped fixtures are expensive, but it means one module full of heavy property tests lands on a single worker while the others idle. If the property tests live together in one file, prefer load, or split the file.

A practical sequence is to set the CI budget first — what gives acceptable confidence per test — then measure the serial runtime of the property tests, then choose a worker count that brings the total inside the pipeline's time limit. Picking the worker count first and then raising max_examples "because there is spare capacity" tends to produce a suite that is fast on the CI runner and painfully slow on a developer laptop running the same profile.

Before adding workers, check whether one expensive property test is holding the run open; lowering its budget is the cheaper fix.

Walking through a worker failure

A typical report from a parallel run looks like this: worker gw5 fails test_round_trip with a falsifying example and, because print_blob=True is set, a line beginning @reproduce_failure('6.112.0', b'AXic...'). The rest of the suite passes.

The first move is to ignore the worker name. It records where the test happened to be scheduled, which depends on timing and on how many workers were available, and it has no influence on the generated data. Copy the decorator, add it above @given on the failing test, and run that single test node without -n. The failure reproduces immediately and deterministically, with the same shrunk example, in a debugger if needed.

If the failure does not reproduce serially, the cause is almost never Hypothesis. It is the usual xdist suspects: state shared through a module-level global that another test on the same worker mutated, a temporary file path that two workers both used, or a fixture whose scope leaked data between tests. In that case the blob still helps, because it removes the input as a variable — the example is fixed, so any remaining difference is in the environment. Run the failing test alongside the other tests that shared its worker, using the collection order xdist logged, to find the interfering neighbour.

Once the cause is fixed, remove the @reproduce_failure decorator — it is a debugging aid tied to one Hypothesis version — and keep the example permanently with @example(...), which survives upgrades and documents the case for readers of the test.

Frequently Asked Questions

Is the Hypothesis example database safe with parallel workers? The default directory-based database tolerates concurrent access in the common case, but several workers writing and deleting entries in one directory can race and produce confusing replays. Giving each worker its own directory, or disabling the database in CI, removes the problem entirely.

How do I reproduce a failure that happened on one xdist worker? Use the @reproduce_failure decorator Hypothesis prints, or the printed seed, and run just that test without -n. Hypothesis's generation does not depend on which worker ran the test, so the failure reproduces serially.

Why do Hypothesis deadlines fail more often under xdist? Because workers compete for CPU, so individual examples take longer than on an idle machine. A deadline tuned serially can be exceeded by a healthy example. Raise it, or set deadline=None for tests whose timing is not the point.

Hypothesis Integration with pytest & Frameworks — profiles and the example database.
pytest-xdist vs pytest-parallel Performance — how xdist distributes work.
Reproducing Failures with @example and the Database — pinning what a search finds.
Choosing max_examples and deadline Settings — budgets for serial and parallel runs.

← Back to Hypothesis Integration with pytest & Frameworks