Hypothesis Integration with pytest & Frameworks

Hypothesis is a pytest plugin, and most of the friction teams hit comes from the seam between the two: a fixture that runs once while the test body runs two hundred times, an example database that two parallel workers fight over, a failure in CI that nobody can reproduce because the seed was not recorded. None of these are subtle once the execution model is clear, and all of them are configuration rather than code.

Prerequisites

hypothesis >= 6.100 and pytest >= 8.0.
hypothesis[django] if the Django integration is in use; pytest-xdist if the suite runs in parallel.
The @given and settings basics from Hypothesis framework fundamentals.
A transactional database fixture for the integration half — see database fixtures and transactional tests.

Core concept: one test item, many executions

pytest's unit of work is the test item. Hypothesis's unit of work is the example. A single @given test is one item that pytest sets up once, tears down once, and reports once — while the function body runs once per generated example, potentially hundreds of times, inside that single setup.

Everything that goes wrong follows from that mismatch.

The health check is not pedantry. A test whose examples share mutable state can pass because of an ordering accident and fail on a different seed.

Step-by-step implementation

1. Register profiles instead of scattering settings

# conftest.py
import os

from hypothesis import HealthCheck, settings

settings.register_profile("dev", max_examples=25, deadline=None)
settings.register_profile(
    "ci",
    max_examples=300,
    deadline=1000,                       # ms; a slow example is a real signal
    derandomize=False,                   # keep searching in CI
    print_blob=True,                     # print @reproduce_failure on failure
)
settings.register_profile("nightly", max_examples=2000, deadline=None)

settings.load_profile(os.environ.get("HYPOTHESIS_PROFILE", "dev"))

print_blob=True in CI is the single most useful setting here: on failure, Hypothesis prints a @reproduce_failure decorator that reproduces the exact example locally, without needing the example database.

2. Fix the fixture scoping

import pytest
from hypothesis import given, strategies as st


# WRONG: cart is created once, mutated by every example.
@pytest.fixture
def cart():
    return Cart()


@given(st.lists(st.integers(min_value=1, max_value=100)))
def test_total_is_the_sum(cart, prices):      # health check fires
    for price in prices:
        cart.add(price)
    assert cart.total() == sum(prices)        # fails from example two onward

from hypothesis import given, strategies as st


# RIGHT: per-example state created inside the body.
@given(st.lists(st.integers(min_value=1, max_value=100)))
def test_total_is_the_sum(prices):
    cart = Cart()                             # fresh for every example
    for price in prices:
        cart.add(price)
    assert cart.total() == sum(prices)

The rule is simple: anything mutated by the test belongs inside the body. Fixtures may supply immutable configuration and expensive shared resources, never per-example state.

3. Give database-backed properties their own transaction

from hypothesis import given, settings, HealthCheck


@settings(suppress_health_check=[HealthCheck.function_scoped_fixture],
          max_examples=50, deadline=None)
@given(invoices())
def test_persisted_invoice_round_trips(db_session, invoice):
    # One savepoint per EXAMPLE, rolled back before the next one runs.
    savepoint = db_session.begin_nested()
    try:
        db_session.add(invoice)
        db_session.flush()
        assert db_session.get(Invoice, invoice.id).total_minor == invoice.total_minor
    finally:
        savepoint.rollback()

Suppressing the health check is legitimate here only because the body explicitly restores state per example. Suppressing it without that is how a property test starts passing for the wrong reason.

4. Use the Django integration rather than fighting it

from hypothesis import given
from hypothesis.extra.django import TestCase, from_model

from myapp.models import Customer


class CustomerProperties(TestCase):
    """Each example runs in its own nested atomic block, rolled back after."""

    @given(from_model(Customer, country=st.sampled_from(["GB", "DE"])))
    def test_display_name_is_never_empty(self, customer):
        assert customer.display_name().strip()

hypothesis.extra.django.TestCase handles the per-example transaction automatically, and from_model derives a strategy from the model's field definitions — including max_length, null and choices, so generated instances are valid by construction.

5. Make parallel runs safe

# conftest.py
import os

from hypothesis.database import DirectoryBasedExampleDatabase
from hypothesis import settings

worker = os.environ.get("PYTEST_XDIST_WORKER", "master")
settings.register_profile(
    "xdist",
    database=DirectoryBasedExampleDatabase(f".hypothesis/examples-{worker}"),
)

Two workers writing to one example database produce intermittent, confusing failures on a directory that is not designed for concurrent writers. One directory per worker removes it for the cost of a few kilobytes.

Verification

Check three things after wiring this up.

# 1. The right profile is active.
pytest tests/ -q --hypothesis-show-statistics 2>&1 | head -5

Using Hypothesis profile 'ci' (max_examples=300, deadline=1000)

# 2. Health checks are not being suppressed silently across the suite.
grep -rn "suppress_health_check" tests/ | wc -l

# 3. A deliberate failure prints a reproducible blob.
pytest tests/test_deliberate_failure.py -q 2>&1 | grep reproduce_failure

@reproduce_failure('6.108.4', b'AXicY2BgYGRgYGBkYAQAAA4AAg==')

A grep result of zero for the second check is suspicious in the other direction: a suite with database-backed properties and no suppressions probably has function-scoped fixtures producing warnings nobody reads.

Troubleshooting

Symptom	Root cause	Fix
`HealthCheck.function_scoped_fixture`	Mutable fixture shared across examples	Create state inside the test body
Property passes alone, fails in the suite	State leaking between examples or tests	Roll back per example; check autouse fixtures
`DeadlineExceeded` only in CI	Deadline tuned on a fast machine	Raise it, or set `deadline=None` for I/O tests
Flaky example-database errors under `-n`	Workers sharing one directory	One database directory per worker
Cannot reproduce a CI failure	Seed and blob not recorded	`print_blob=True`; upload `.hypothesis` as an artefact
Tests slow after adding properties	Default `max_examples` in the dev loop	Profile-based budgets, small locally

The example database, and what it is for

Hypothesis keeps a small on-disk database of the inputs that previously failed, and replays them first on the next run. That is what makes a fixed bug stay fixed locally: the failing example is tried again immediately rather than re-derived by chance.

In CI the value is different and often misunderstood. A fresh runner has an empty database, so nothing is replayed, and a run that finds a new failure writes it to a directory that is discarded when the job ends. Two responses are reasonable and they are not equivalent.

Persist it as an artefact. Uploading .hypothesis/ on failure lets a developer download it and reproduce the exact example locally. Caching it between runs additionally makes CI re-try previous failures, at the cost of a cache that can go stale and hide a regression behind a replayed example.

Disable it and rely on explicit examples. database=None in the CI profile makes every run a clean search, and any failure worth keeping is promoted to an @example decorator in the source. This is the more disciplined option: a regression pinned in the code is visible in review and survives cache eviction, where one living only in a database directory does not.

Both are defensible. What is not defensible is relying on an uncached CI database, which replays nothing and records nothing.

Reporting and statistics

Property tests generate their own diagnostics, and reading them changes how the tests are written. --hypothesis-show-statistics prints, per test, how the budget was spent.

pytest tests/test_parser.py --hypothesis-show-statistics -q

  test_parser.py::test_round_trip:
    - during generate phase (0.42 seconds):
      - Typical runtimes: < 1ms, ~ 54% in data generation
      - 300 passing examples, 0 failing examples, 41 invalid examples
    - Stopped because settings.max_examples=300

Three numbers matter in that block. Invalid examples are those rejected by assume() or a filter — forty-one out of three hundred is acceptable, three hundred out of six hundred is a strategy that needs restructuring. Time in data generation above about half says the strategy is doing more work than the property; the usual cause is building heavyweight objects per example when the property only needs a few fields. And the stopping reason tells you whether the budget was exhausted or the search gave up early, which distinguishes "we searched properly" from "generation kept failing".

The other reporting lever is event(), which labels examples so the statistics show the distribution. Labelling by branch — event("cache_hit" if hit else "cache_miss") — is the fastest way to discover that a property intended to exercise both paths has been taking one of them ninety-nine times in a hundred.

Marking, selecting and budgeting in CI

Property tests behave differently from example-based tests in a pipeline, and giving them their own marker makes that manageable.

[tool.pytest.ini_options]
markers = ["property: generative test; budget scales with HYPOTHESIS_PROFILE"]

pytest -m "not property" -q                       # fast feedback, examples only
HYPOTHESIS_PROFILE=ci pytest -m property -q       # the real search
HYPOTHESIS_PROFILE=nightly pytest -m property -q  # deep search, scheduled

Separating them buys two things. A pull-request run stays fast and deterministic, which matters because a property test that finds a new failure on an unrelated change is confusing in review — the change did not break it, the search just got lucky. And the nightly deep run has somewhere to report to, so a genuine discovery becomes a ticket rather than a red build on somebody's unrelated pull request.

That distinction is worth making explicit to a team adopting properties. An example-based test failing means "your change broke this". A property test failing on a fresh seed means "a latent bug was found", which may predate the change entirely. Treating the second as a merge blocker makes people distrust the suite; routing it to a nightly job with an owner makes it a source of real defects.

Frameworks beyond Django

The integration pattern generalises. Hypothesis needs two things from a framework: a way to build valid domain objects, and a way to reset state between examples.

For SQLAlchemy, build strategies from the mapped classes as shown in designing strategies for domain data, and use a nested transaction per example. For FastAPI or Flask, generate request payloads and assert on responses through the test client, with the application's dependency overrides supplying fakes so no network is involved. For Pydantic, st.builds(Model) works directly once field types are registered, and validation errors become a legitimate outcome to assert on rather than an obstacle.

from hypothesis import given, strategies as st
from pydantic import ValidationError


@given(st.builds(dict, name=st.text(), age=st.integers()))
def test_model_either_validates_or_reports_precisely(payload):
    try:
        model = CustomerModel(**payload)
    except ValidationError as exc:
        # A rejection is fine; an unhelpful rejection is not.
        assert exc.errors(), "validation failed with no field information"
    else:
        assert model.age >= 0

That shape — "either it succeeds and the invariant holds, or it fails with a usable error" — is one of the most productive property templates for anything with a validation layer, because it covers the whole input space without requiring the test to know which inputs are valid.

Budgets, deadlines and the feedback loop

Two numbers decide whether property tests are a pleasure or a tax, and both should differ by environment.

max_examples is the search budget. Twenty-five is plenty in an edit-run loop, where the point is to notice a break quickly; three hundred in CI does the real searching; a few thousand nightly explores the space properly. Running three hundred examples locally on every save is how teams conclude that property testing is too slow, when what is too slow is the profile.

deadline is a per-example time limit, and it exists to catch the pathological input that takes a hundred times longer than the typical one — a genuinely valuable signal for parsers and algorithms. It is also the setting most likely to produce a spurious CI failure, because a shared runner can pause any process for a second. The compromise that works: keep a deadline for pure computation, set deadline=None for anything touching I/O, and never tune it so tightly that a busy machine trips it.

One more interaction matters for large suites. Property tests multiply their cost by the number of examples, so a suite where every test is a property runs orders of magnitude longer than one where properties are used deliberately for the code that benefits — parsers, serializers, invariant-bearing data structures, state machines. Example-based tests remain the right tool for specific business rules with specific expected values, and the two are complementary rather than competing, as set out in property-based and fuzz testing strategies.

Where a property test belongs in the suite

Adding properties to an existing suite works best when it is targeted rather than universal, and the targets are recognisable.

Anything with a round trip. Serialize/deserialize, encode/decode, parse/render, save/load. The property writes itself and the bugs are real — see round-trip properties for serializers and parsers.

Anything with an algebraic law. Sorting is idempotent, a merge is associative, a discount is never negative, a total is the sum of its parts. These are the properties a reviewer can check by reading.

Anything with a reference implementation. A fast path and a slow path that must agree, an optimised query and the naive one, a cache and the source of truth. Equivalence properties are the highest-value use of the tool and need no invention.

Anything stateful. A cache, a connection pool, a state machine, a queue. These want RuleBasedStateMachine rather than @given, and they belong in stateful and model-based testing.

Most suites end up with a handful of properties covering the universal claims and a large number of examples covering the specific ones. Converting everything in either direction makes the suite worse.

Introducing properties to a team

The adoption failures are social rather than technical, and two habits avoid most of them.

Write the first properties against code that already has good example tests, so the properties find genuine latent bugs rather than merely restating what is already covered. A property that finds a real defect in week one makes the case for itself; one that only duplicates existing coverage looks like ceremony.

Then treat every discovered counterexample as two changes: the fix, and an @example decorator pinning the case. The pinned example is the artefact that persuades reviewers, because it turns a probabilistic search into a concrete regression test they can read. It also survives a future decision to disable the example database, which is the point at which teams otherwise lose their accumulated findings without noticing. A short convention written next to the profiles — every counterexample gets an @example, every suppression gets a comment explaining what restores isolation — is enough governance for this, and it fits in a paragraph of the contributing guide.

Frequently Asked Questions

Why does Hypothesis warn about function-scoped fixtures? Because a function-scoped fixture is set up once per test function, not once per generated example, so every example after the first sees state left by the one before. Hypothesis raises HealthCheck.function_scoped_fixture to say the test is not isolated. Either widen the fixture and reset state inside the test, or move the setup into the test body where it runs per example.

How do I share a database between Hypothesis examples safely? Wrap each example in its own transaction inside the test body and roll it back, rather than relying on a per-test fixture. With Django, hypothesis.extra.django.TestCase does this for you by running each example in a nested atomic block.

Do Hypothesis and pytest-xdist work together? Yes, with one caveat: the example database is a directory on disk, and parallel workers writing to it can conflict. Point each worker at its own database directory, or set the database to None in CI and rely on explicit @example entries for regressions.

How do I reproduce a CI failure locally? Copy the @reproduce_failure decorator Hypothesis prints, or re-run with the printed seed via --hypothesis-seed. Better still, persist the .hypothesis directory as a CI artefact so the failing example is available without re-deriving it.

Should max_examples differ between local runs and CI? Yes. A dev profile with a small budget keeps the edit-run loop fast; a CI profile with a larger one does the real searching. Register both with settings.register_profile and select with HYPOTHESIS_PROFILE so nobody has to remember flags.

Resolve the fixture question in combining @given with pytest fixtures safely.
Apply it to an ORM with property-testing Django models with Hypothesis.
Make parallel runs safe using running Hypothesis under pytest-xdist.
Keep the search from dominating the clock with reducing Hypothesis test execution time.
Generate valid inputs in the first place via designing strategies for domain data.

← Back to Property-Based & Fuzz Testing Strategies