Designing Strategies for Domain Data

A property test is only as good as the values it generates. st.text() will find that a function breaks on a lone surrogate; it will never find that an invoice breaks when its due date precedes its issue date, because it cannot generate an invoice. Designing strategies for domain types is where property-based testing stops being a novelty and starts finding the bugs an example-based suite cannot enumerate.

Prerequisites

• hypothesis >= 6.100, with hypothesis[numpy] or hypothesis[pandas] if array and dataframe strategies are needed.
• pytest >= 8.0, and the fundamentals of @given and settings from Hypothesis framework fundamentals.
• Domain types with explicit validation, since a strategy that generates "valid" objects needs a definition of valid to aim at.
• Familiarity with @composite, introduced in composing strategies with flatmap and composite.

Core concept: constrain at the source, not after the fact

Every strategy design decision reduces to one question: is the constraint expressed in how the value is built, or checked after it is built?

st.integers().filter(lambda n: n % 3 == 0) throws away two thirds of what it generates. st.integers().map(lambda n: n * 3) generates only multiples of three and discards nothing. Both produce the same set of values; the second is three times faster, shrinks better, and never trips the filter_too_much health check.

Step-by-step implementation

1. Constrain the primitives

from decimal import Decimal

from hypothesis import strategies as st

# Money in minor units: an integer, never negative, bounded by a plausible max.
amounts = st.integers(min_value=0, max_value=10_000_000)

# Currency codes from the real set, not arbitrary three-letter strings.
currencies = st.sampled_from(["GBP", "USD", "EUR", "JPY"])

# Names: printable, non-empty, no control characters — the constraint is in
# the alphabet, so nothing is generated and then thrown away.
names = st.text(
    alphabet=st.characters(min_codepoint=32, blacklist_categories=("Cs", "Cc")),
    min_size=1,
    max_size=80,
)

Bounding amounts at ten million rather than leaving it unbounded is a deliberate trade. An unbounded integer strategy will eventually generate values that overflow a database column, and the resulting failure is about the column rather than the code. Bound to what the domain permits, and test the boundary explicitly with an example-based test.

2. Compose dependent fields

import datetime as dt

from hypothesis import strategies as st

from myapp.models import Invoice


@st.composite
def invoices(draw, *, status=None):
    issued = draw(st.datetimes(
        min_value=dt.datetime(2020, 1, 1),
        max_value=dt.datetime(2030, 1, 1),
        timezones=st.timezones(),          # real zones, so DST edges appear
    ))
    # Due date derived from issued: the ordering invariant holds by construction.
    days = draw(st.integers(min_value=0, max_value=365))
    due = issued + dt.timedelta(days=days)

    return Invoice(
        id=draw(st.uuids()).hex,
        issued_at=issued,
        due_at=due,
        total_minor=draw(amounts),
        currency=draw(currencies),
        status=status if status is not None else draw(st.sampled_from(["open", "paid"])),
    )

The status keyword is what makes this strategy reusable. A test about payment needs invoices(status="open") and one about archiving needs invoices(status="paid"); without the parameter each test filters, and filtering for a value that occurs half the time doubles the work.

3. Register the strategy against the type

from hypothesis import strategies as st

from myapp.models import Invoice

# Now st.from_type(Invoice) and st.builds(Order) both produce valid invoices.
st.register_type_strategy(Invoice, invoices())

Registration is what makes st.builds useful on aggregates. Without it, st.builds(Order) inspects Order.__init__, finds an Invoice annotation and fails or produces something invalid; with it, the whole object graph is generated correctly from one registration per type.

4. Check what is actually being generated

from hypothesis import event, given


@given(invoices())
def test_settlement_is_idempotent(invoice):
    # event() labels each example so the statistics show the distribution.
    event(f"currency={invoice.currency}")
    event(f"zero_amount={invoice.total_minor == 0}")

    ledger = settle(invoice)
    assert settle(invoice) == ledger

pytest tests/test_settlement.py --hypothesis-show-statistics -q

  - Events:
    * 51.20%, currency=GBP
    * 23.40%, zero_amount=False
    *  1.00%, zero_amount=True      ← the interesting case is rare

One per cent coverage of the zero case means the property is effectively untested there. The fix is a strategy that produces it deliberately — st.one_of(st.just(0), amounts) — rather than hoping for it.

5. Confirm the shrunk example is meaningful

Introduce a bug on purpose and read the counterexample. A good strategy reports something like Invoice(total_minor=0, currency='GBP', …) — minimal, valid, obviously a boundary. A poor one reports an invoice with an empty id and a due date before its issue date, which means the strategy is generating objects the system would never see and the failure may not be real.

Verification

Two checks tell you whether the strategy is fit for use.

from hypothesis import find

# 1. Can it produce the edge cases you care about at all?
zero = find(invoices(), lambda i: i.total_minor == 0)
ambiguous = find(invoices(), lambda i: i.issued_at.dst() != i.due_at.dst())

# 2. Does everything it produces satisfy the domain's own validator?
from hypothesis import given


@given(invoices())
def test_every_generated_invoice_is_valid(invoice):
    invoice.validate()     # raises if any invariant is violated

The second test is worth keeping permanently. It costs a second per run and it fails the day someone adds a model constraint without updating the strategy, which is otherwise discovered as a confusing failure in an unrelated property test.

Troubleshooting

Dates, times and zones

Temporal data deserves its own treatment because almost every generated-date bug is really a timezone bug.

import datetime as dt

from hypothesis import given, strategies as st


@given(st.datetimes(
    min_value=dt.datetime(2020, 1, 1),
    max_value=dt.datetime(2030, 1, 1),
    timezones=st.timezones(),         # IANA zones, including the awkward ones
))
def test_round_trip_through_utc(moment):
    # Converting to UTC and back must preserve the instant, though not
    # necessarily the wall-clock representation.
    assert moment.astimezone(dt.timezone.utc).astimezone(moment.tzinfo) == moment

st.timezones() draws from the system's IANA database, which includes zones with half-hour offsets, zones that have changed offset within the generated range, and zones where a given local time either does not exist or occurs twice. Those are precisely the cases that break naive date arithmetic, and no fixed timezone(timedelta(hours=1)) will ever produce them.

Two further constraints are worth applying deliberately. min_value and max_value should bracket the range the system actually handles, because a generated date in the year 1 will break a database column and teach nothing. And allow_imaginary=False is available when the code genuinely cannot handle a local time that does not exist — but reach for it only after confirming the code is supposed to reject those rather than handle them.

Arrays, frames and scientific data

Numeric code has its own extras package, and the strategies there encode constraints that are tedious to express by hand.

import numpy as np
from hypothesis import given
from hypothesis.extra import numpy as npst


@given(npst.arrays(
    dtype=np.float64,
    shape=npst.array_shapes(min_dims=1, max_dims=2, max_side=16),
    # Excluding NaN and infinity here, and testing them separately, keeps the
    # property statement honest: this one is about finite arithmetic.
    elements=st.floats(min_value=-1e6, max_value=1e6, allow_nan=False,
                       allow_infinity=False),
))
def test_normalisation_preserves_shape(arr):
    result = normalise(arr)
    assert result.shape == arr.shape
    assert np.isfinite(result).all()

The decision to exclude NaN is not a shortcut. A property such as "normalisation preserves shape" is true for arrays containing NaN; a property such as "the result sums to one" is not. Splitting them into two tests with two strategies states each property precisely, where a single test with a blanket strategy would force a weaker assertion — the argument developed in testing numeric code with floats and NaN edge cases.

For dataframes, hypothesis.extra.pandas provides column, data_frames and indexes, with the same principle: declare the dtype and the element strategy per column so generated frames are schema-valid, and let the index be generated rather than assumed unique.

Shaping the distribution deliberately

Hypothesis biases generation toward values that have historically found bugs — zero, empty collections, boundary integers, surrogate characters — but it knows nothing about which values matter in your domain. Left alone, a strategy over a five-year date range produces mid-range dates almost every time, and the month-end and year-end cases that break billing code appear in a fraction of a per cent of examples.

Three tools shape this, in increasing order of force.

st.one_of mixes an explicitly interesting generator into the general one:

from hypothesis import strategies as st

month_ends = st.sampled_from([28, 29, 30, 31])
# Roughly half the generated days are month-end candidates.
days = st.one_of(month_ends, st.integers(min_value=1, max_value=31))

target() tells the engine which direction is "more interesting", so it steers generation rather than sampling blindly:

from hypothesis import given, target


@given(orders())
def test_discount_never_exceeds_subtotal(order):
    discount = compute_discount(order)
    # Steer toward large discounts: that is where the bug will be.
    target(float(discount), label="discount")
    assert discount <= order.subtotal

And @example pins a case permanently, which is the right response to a bug found in production:

from hypothesis import example, given


@given(invoices())
@example(Invoice(issued_at=LEAP_DAY, due_at=LEAP_DAY, total_minor=0, currency="JPY"))
def test_settlement_handles_leap_day(invoice):
    assert settle(invoice).total == invoice.total_minor

The three serve different purposes and are often confused. one_of changes what is generated; target changes the search; @example guarantees a specific case runs every time, regardless of seed or budget. A regression found in production belongs in @example — relying on the generator to rediscover it is relying on chance.

Strategies that mirror a schema

When the data has an external schema — a JSON contract, an Avro record, a protobuf message — generating from the schema rather than from a hand-written strategy keeps the two in step automatically.

from hypothesis import given, strategies as st

# A recursive strategy for arbitrary JSON, bounded so it terminates.
json_values = st.recursive(
    st.none() | st.booleans() | st.floats(allow_nan=False, allow_infinity=False)
    | st.text(max_size=20),
    lambda children: st.lists(children, max_size=4)
    | st.dictionaries(st.text(min_size=1, max_size=10), children, max_size=4),
    max_leaves=12,          # without this, generation can blow the stack
)


@given(json_values)
def test_serializer_round_trips(value):
    assert deserialize(serialize(value)) == value

max_leaves is not optional. st.recursive without a bound generates structures whose size is unbounded in expectation, which shows up as a test that occasionally takes minutes and occasionally exhausts memory. Bounding it produces the same coverage of shapes at a fraction of the cost, because the bugs in serializers are about type handling and nesting, not about depth.

For a published JSON Schema, hypothesis-jsonschema generates conforming instances directly, which is the schema-driven equivalent of registering a type strategy: one declaration, and every test that needs a valid payload gets one. It pairs naturally with the response validation described in contract testing for HTTP APIs — generate requests from the schema, validate responses against it, and the two halves of the contract are both exercised by construction.

Keeping strategies maintainable

A strategy module is production code for the test suite, and it decays the same way factories do without a few conventions.

Keep it next to the models. myapp/testing/strategies.py, shipped with the package, means every consumer's tests generate valid objects and a model change breaks one file.

Parameterise rather than duplicate. Two strategies differing only in a fixed status should be one strategy with a keyword argument. Duplicates drift, and the drift is invisible until a property fails against only one of them.

Never let a strategy call the application. A strategy that builds an object by invoking a service is running the code under test during generation, which makes failures ambiguous and shrinking unreliable.

Assert the strategy's own invariant. The test_every_generated_invoice_is_valid test above is the guard that keeps all of this honest, and it costs nothing.

Where a strategy grows a parameter for every test that uses it, that is the same signal as the god factory: the tests are asking for narrowing that would be clearer as separate, named strategies. open_invoices() and overdue_invoices() read better at the call site than invoices(status="open", due_before=…), and they shrink toward the boundary each test actually cares about.

Designing for shrinking

Shrinking is what turns a random failure into a readable bug report, and a strategy's structure determines how well it works. Hypothesis shrinks by simplifying the choices the generator made, not the final value, so a strategy whose output depends on those choices in a simple, monotone way shrinks to something meaningful, while one that hashes or shuffles them does not.

Three design rules follow.

Prefer map over filter. A filtered strategy shrinks by retrying candidates until one passes the predicate again, which is slow and often gets stuck. A mapped one shrinks the input and transforms it, which always terminates.

Draw in the order the reader would. In a @composite, drawing the start date before the duration means shrinking reduces the duration first and the start date second, producing a counterexample like "a zero-day range on 1 January" rather than an arbitrary pair.

Avoid deriving values from hashes or randomness. draw(st.integers()).__hash__() produces a value with no ordering relationship to the choice behind it, so simplifying the choice does not simplify the output and the shrinker wanders.

A practical test of all three rules: introduce a deliberate off-by-one into the code and read the counterexample. If it is small, valid and obviously at a boundary, the strategy is well designed. If it is a large arbitrary object, the shrinker could not simplify it, and the cause is nearly always a filter or a derived value somewhere in the chain — the diagnosis process set out in why Hypothesis shrinking stalls and how to fix it.

Frequently Asked Questions

Should I generate invalid data and filter, or generate only valid data? Generate only valid data wherever the constraint can be expressed constructively. filter() discards inputs after generating them, so a tight constraint wastes most of the work and eventually trips the filter-too-much health check. Build the value from parts that cannot be wrong — a date range generated as a start plus a positive delta rather than two independent dates filtered for ordering.

How do I generate objects whose fields depend on each other? With @composite, which gives you a draw function so later fields can be generated from earlier ones. A subscription's end date drawn relative to its start, or a line item's tax drawn from the customer's country, are both natural in a composite and awkward with builds() alone.

Why does my strategy shrink to something implausible? Shrinking moves toward each strategy's defined minimum, which for text is the empty string and for integers is zero. If the minimal counterexample is valid input, the bug is real and the small example is a feature. If it is not valid input, the strategy is too loose and should exclude it constructively.

How do I keep generated timestamps from breaking on daylight saving? Generate an aware datetime with an explicit timezone strategy rather than a naive one plus a fixed offset. Hypothesis's timezones() strategy draws real zones, which is what surfaces the ambiguous and non-existent local times that a fixed offset can never produce.

Can strategies be reused across a codebase? Yes, and they should be. A module of domain strategies next to the models gives every test the same generators, so a constraint added to the model is added to one strategy rather than to a dozen tests. Treat them as part of the domain layer's test API.

Related guides

• Generate scientific data with generating DataFrames and arrays with Hypothesis extras.
• Handle temporal edge cases in constraining dates and timezones in strategies.
• Learn the composition primitives from composing strategies with flatmap and composite.
• Keep generation from dominating the run with reducing Hypothesis test execution time.
• Combine generated data with fixtures safely per combining @given with pytest fixtures safely.

← Back to Property-Based & Fuzz Testing Strategies