Python Testing & Debugging Beyond unittestAdvanced Python testing playbooks

Hypothesis & Fuzzing

Generating Custom Strategies with hypothesis.strategies

You have a domain object — a date range, a financial record, a config with cross-field rules — and the obvious approach, st.builds(Model).filter(is_valid), has turned your suite into a CI bottleneck: --hypothesis-show-statistics shows most generated examples being discarded, shrinking stalls, and runs occasionally Unsatisfiable. The fix is to stop generating arbitrary data and filtering it, and instead build valid-by-construction strategies whose every output already satisfies the invariants. This guide shows the @st.composite, st.builds, and type-registration patterns that get rejection rates below the 15% threshold where Hypothesis stays fast and shrinking stays deterministic.

Prerequisites

Solution

The core technique is conditional routing: pick the categorical fields first, then draw dependent fields from bounds derived from those choices, so no invalid object is ever produced.

Python
from datetime import date, timedelta
import hypothesis.strategies as st
from hypothesis import given, settings, assume, Phase, Verbosity

@st.composite
def valid_time_ranges(draw: st.DrawFn) -> dict[str, date | int]:
    # Draw start from a bounded domain so CI runs stay deterministic
    start = draw(st.dates(min_value=date(2020, 1, 1), max_value=date(2024, 12, 31)))
    # Route end_date generation: a valid future date OR the same day — never < start
    end = draw(st.one_of(
        st.dates(min_value=start, max_value=start + timedelta(days=365)),
        st.just(start),
    ))
    assume(end >= start)  # cheap edge guard only; routing already guarantees validity
    return {"start_date": start, "end_date": end, "duration_days": (end - start).days}

@given(time_range=valid_time_ranges())
@settings(max_examples=200, phases=[Phase.generate, Phase.shrink],
          verbosity=Verbosity.normal, database=None)
def test_time_range_invariants(time_range: dict[str, date | int]) -> None:
    assert time_range["start_date"] <= time_range["end_date"]  # holds by construction
    assert time_range["duration_days"] >= 0

For pure constructors whose fields are independent, st.builds is more declarative and resolves type hints automatically. Register the strategy so st.from_type() finds it everywhere:

Python
from dataclasses import dataclass, field
from typing import Optional, Literal
import hypothesis.strategies as st
from hypothesis import given, settings, Phase

@dataclass
class UserConfig:
    username: str
    tier: Literal["free", "pro", "enterprise"]
    max_requests: Optional[int] = None
    metadata: dict[str, str] = field(default_factory=dict)

def user_config_strategy() -> st.SearchStrategy[UserConfig]:
    return st.builds(
        UserConfig,
        username=st.text(min_size=3, max_size=20).filter(str.isalnum),  # cheap, rarely rejects
        tier=st.sampled_from(["free", "pro", "enterprise"]),
        max_requests=st.one_of(st.none(), st.integers(min_value=100, max_value=10_000)),
        metadata=st.dictionaries(st.text(min_size=1, max_size=15), st.text(max_size=50)),
    )

st.register_type_strategy(UserConfig, user_config_strategy())  # st.from_type() now resolves it

@given(config=st.from_type(UserConfig))
@settings(max_examples=100, phases=[Phase.generate, Phase.shrink])
def test_user_config(config: UserConfig) -> None:
    assert config.username.isalnum()
    if config.max_requests is not None:
        assert config.max_requests >= 100
Composite strategy composition tree A composite strategy draws categorical fields first, then routes dependent fields from bounds derived from those choices, producing a valid object. Valid-by-construction composition @st.composite draw() sampled_from(tier) categorical, drawn first dates(min=start) bounds depend on start one_of(branch) route, do not filter valid object — no rejection retry
The composite draws categorical fields first, then derives dependent-field bounds from those choices, so the returned object is valid without any post-generation filtering.

To extract a minimal failing input without the @given runner, use hypothesis.find(strategy, predicate) — it applies the same shrinking machinery and returns the smallest input satisfying the predicate, ideal for isolating a known business-rule violation.

Why this works

Hypothesis's rejection sampler discards invalid examples and retries; once rejection exceeds ~15%, throughput collapses and shrinking becomes non-deterministic because the search tree is fragmented by discarded branches. Routing generation with st.one_of and st.sampled_from moves validation from after generation to during it, so the shrinking engine only ever explores valid inputs and can converge on a minimal counterexample in milliseconds. st.builds adds declarative type resolution for the independent-field case, while st.register_type_strategy makes that resolution automatic across the suite.

Edge cases and failure modes

  • .filter() still present on a hot path — even one filter with a low pass rate dominates runtime; replace it with a pre-computed st.sampled_from(valid_values) when the valid set is finite.
  • Circular type resolution — A references B references A causes infinite generation; break it with st.recursive(..., max_leaves=...) or st.just() placeholders, and bound depth explicitly in recursive composites.
  • Unhashable types in st.dictionaries/st.sets — generated mutable values raise TypeError; convert to tuple/frozenset before they are used as keys.
  • Mutable state leakage — if the test mutates a generated object, copy.deepcopy() it first, or a cached example can carry mutations into the next run.
  • Global registration in a monorepo — register_type_strategy is process-global; prefer local registration in the test module (or a fixture that registers and unregisters) to avoid polluting other suites.

Frequently Asked Questions

Why are my custom strategies so slow? Almost always because .filter() is discarding most generated examples and retrying. Replace it with @st.composite conditional routing or a pre-filtered st.sampled_from so every draw is valid by construction, and verify the rejection rate stays under 15% with --hypothesis-show-statistics.

When should I use st.builds versus @st.composite? Use st.builds for pure constructors whose fields are independent — it maps keyword strategies to arguments and resolves type hints automatically. Use @st.composite when fields are correlated, such as end_date >= start_date, because builds cannot enforce cross-field constraints.

How do I register a strategy so st.from_type() finds it automatically? Call st.register_type_strategy(YourType, your_strategy()). Hypothesis walks the MRO and checks registered strategies before built-in inference. Prefer local registration in test modules to avoid polluting the global cache in large repositories.

This page builds on advanced property-based testing; for the underlying generation model see the Hypothesis framework fundamentals, and when these strategies slow CI consult reducing Hypothesis test execution time.

← Back to Advanced Property-Based Testing