Python Testing & Debugging Beyond unittestAdvanced Python testing playbooks
Hypothesis & Fuzzing

Using assume() Without Tripping the Filter Health Check

hypothesis.errors.FailedHealthCheck: It looks like your strategy is filtering out a lot of data is Hypothesis telling you that most of what it generates never reaches your assertion. Every call to assume(...) that returns false and every .filter(...) that rejects a value throws away an example and asks for another. When too many are thrown away in a row, Hypothesis gives up and raises the filter_too_much health check rather than silently running a test that barely tests anything.

The check is worth respecting. A strategy that rejects ninety percent of its output is slow, and worse, the ten percent it keeps are skewed: they cluster around whatever values happen to satisfy the condition most easily. The fix is almost never to suppress the check. It is to generate valid inputs directly, keeping assume() for the rare conditions that are genuinely easier to reject than to construct.

Prerequisites

Solution

Python
# Before — rejects most generated data.
from hypothesis import assume, given, strategies as st

@given(st.integers(), st.integers())
def test_range_slice(lo, hi):
    assume(0 <= lo < hi <= 1000)          # rejects the vast majority of pairs
    assert len(list(range(lo, hi))) == hi - lo
Python
# After — constructs valid pairs directly; nothing is rejected.
@st.composite
def ordered_pair(draw, upper=1000):
    lo = draw(st.integers(0, upper - 1))
    hi = draw(st.integers(lo + 1, upper))
    return lo, hi

@given(ordered_pair())
def test_range_slice(pair):
    lo, hi = pair
    assert len(list(range(lo, hi))) == hi - lo
Python
# Still fine — assume() rejecting a small fraction.
@given(st.lists(st.integers(), min_size=1))
def test_mean_within_bounds(xs):
    assume(len(set(xs)) > 1)              # rejects only all-equal lists
    assert min(xs) <= sum(xs) / len(xs) <= max(xs)
Rejection versus construction On the left, a wide strategy generates many integer pairs and assume discards most of them, so only a few reach the test. On the right, a composite strategy draws the lower bound and then draws the upper bound above it, so every generated pair is valid and reaches the test. Throw away, or build right the first time integers() x integers() + assume 1000 pairs generated ~990 rejected by assume ~10 reach the assertion health check: filter_too_much ordered_pair() composite draw lo in [0, 999] draw hi in [lo+1, 1000] every pair valid full budget spent on real tests
Constructing the relationship between values replaces a rejection rate of almost everything with none at all.

Why this works

Hypothesis counts valid examples towards max_examples and tracks invalid ones separately. Rejection by assume() or .filter() marks the current example invalid, and generation starts again from scratch. The health check fires when the ratio of invalid to valid examples is high early in the run — Hypothesis would otherwise burn through a large generation budget and still test only a handful of inputs.

Construction avoids the problem because every draw is conditioned on the earlier ones. Drawing hi from integers(lo + 1, upper) makes lo < hi true by construction, not by luck. The same principle covers most filters: a lower bound becomes min_value, a non-empty requirement becomes min_size=1, uniqueness becomes st.lists(..., unique=True) or st.sets(...), and "one of these values" becomes st.sampled_from(...).

Construction also shrinks better. When a test fails, Hypothesis shrinks by simplifying the underlying choices it made. A composite strategy with explicit dependencies shrinks towards the simplest valid pair — (0, 1) — while a heavily filtered strategy often shrinks poorly, because many simplifications produce inputs the filter rejects, and shrinking stalls on a messy example.

Recognising which conditions to rebuild

Not every assume() needs replacing. The question is what fraction of inputs it rejects, and --hypothesis-show-statistics answers it directly:

Plain text
- during generate phase (0.41 seconds):
    - Typical runtimes: < 1ms, of which < 1ms in data generation
    - 100 passing examples, 0 failing examples, 912 invalid examples

Nine invalid examples for every valid one is well into rebuild territory. As a rule of thumb, a rejection rate below about a third is harmless and not worth restructuring; above half, look for a construction; near the health-check threshold, rebuilding is required.

Conditions fall into a few recognisable families. Range conditions (x > 0, len(s) < 50) map to strategy bounds. Ordering conditions (a < b, sorted lists) map to derived draws or sorted() inside a composite. Distinctness conditions map to unique=True, st.sets, or unique_by. Membership conditions — a key must be in a dict — map to drawing the dict first, then st.sampled_from(sorted(d)). What remains after those families are the genuinely irregular conditions: "the matrix is invertible", "the graph is connected". Those are the legitimate home of assume(), and for most generators they reject a small enough fraction to be fine.

Mapping filter conditions to strategy constructions A table-like diagram maps four families of conditions to constructions: range conditions to min and max bounds, ordering conditions to derived draws, distinctness conditions to unique lists or sets, and membership conditions to sampling from a previously drawn collection. Irregular conditions such as invertibility remain a good use of assume. Most filters have a constructive twin condition construct it with assume(x > 0), len(s) < 50 min_value / max_size bounds assume(a < b), sorted input draw a, then draw b from (a, upper] assume(len(set(xs)) == len(xs)) st.lists(..., unique=True), st.sets assume(k in d) draw d, then sampled_from(sorted(d)) matrix invertible, graph connected keep assume() — if it rarely rejects
Work down the table before reaching for suppression: the last row is the only one where assume() is the natural tool.

How rejection distorts the inputs you do test

Speed is the obvious cost of heavy filtering; the subtler cost is bias. When a filter keeps only a small slice of the generated space, the values that survive are not a fair sample of the valid inputs. They are whichever valid inputs the underlying strategy happens to produce most often.

Take assume(0 <= lo < hi <= 1000) over two unbounded integers. Hypothesis favours small integers and boundary values, so the pairs that survive are dominated by tiny ranges near zero — (0, 1), (0, 2), (1, 3). Ranges near the top of the interval, or spanning most of it, almost never appear. A bug that only shows up when hi equals the upper bound could go unfound for thousands of runs, not because the test is wrong but because the filtered strategy rarely reaches that region.

The composite version draws lo across the whole interval and hi across whatever remains, so both ends of the range get explored, and Hypothesis's own boundary heuristics apply to the bounds you passed — it tries upper itself on purpose. Constructed strategies inherit that boundary-seeking behaviour; filtered ones mostly lose it.

This is also why a test that passes reliably with a heavy filter is weaker evidence than it looks. The statistics line saying "100 passing examples" is true, but those hundred examples may cover only a narrow corner of the valid space. After rebuilding a strategy, it is common for a test that had passed for months to fail on its first run — not because the code changed, but because the inputs finally reached the corner where the bug was.

Filtered versus constructed coverage of the valid range Two horizontal bars represent the valid range from 0 to 1000. In the filtered bar, surviving examples cluster near zero, leaving the upper end unexplored. In the constructed bar, examples spread across the whole range and include the upper bound itself. Where the surviving examples land filtered upper range almost never reached constructed upper bound tried on purpose 0 1000
Rebuilding the strategy fixes coverage as well as speed: boundary values come back into play.

Edge cases and failure modes

  • Filters hidden in shared strategies. A .filter() deep inside a reusable domain strategy makes every test using it slow. Check statistics for the tests that use shared strategies, not only the one you are writing.
  • Chained filters multiply. Two filters that each keep half the values keep a quarter together. Rebuild the one that rejects most first.
  • assume() after expensive work. Calling assume late in the test body wastes all the work done before it. Put the check as early as possible, or better, in the strategy.
  • filter_too_much only on CI. Different profiles and database contents can change early rejection rates. Treat it as a strategy problem even if it only fires in one environment.
  • Suppression spreading. A suppress_health_check=[HealthCheck.filter_too_much] in a shared profile hides the problem for every test. Keep any suppression on the individual test, with a comment.

A worked rebuild: generating valid date ranges

Consider a booking system with the rule that a stay starts on or after today, lasts one to thirty nights, and must not cross a blackout period. The first attempt generates two dates and filters:

Python
@given(st.dates(), st.dates())
def test_booking(start, end):
    assume(TODAY <= start < end)
    assume((end - start).days <= 30)
    assume(not overlaps_blackout(start, end))
    ...

The first two assumptions together reject almost everything: two arbitrary dates spanning several millennia are rarely ordered, in the future, and within a month of each other. Rebuilding with construction handles them completely — draw a start within a sensible horizon, then draw a length:

Python
stays = st.builds(
    lambda start, nights: (start, start + timedelta(days=nights)),
    st.dates(min_value=TODAY, max_value=TODAY + timedelta(days=730)),
    st.integers(1, 30),
)

@given(stays)
def test_booking(stay):
    start, end = stay
    assume(not overlaps_blackout(start, end))   # rejects a small fraction
    ...

The blackout condition stays as an assume() because blackouts are sparse and constructing around them would need the strategy to know the calendar. With the other conditions built in, the remaining rejection rate is small, the health check stops firing, and — because the start date is now bounded to a two-year horizon — every generated stay is the kind of input the production system actually receives. The rebuild improved both speed and the realism of what the test exercises.

Frequently Asked Questions

What does filter_too_much mean? Hypothesis generated many inputs that were rejected by assume() or .filter() before it found enough valid ones. It stops because the test is spending most of its effort on inputs it throws away, which also means the valid inputs it does test are poorly distributed.

Is assume() bad practice? No. assume() is fine for rejecting a small fraction of inputs, especially conditions that are awkward to express in a strategy. It becomes a problem when it rejects most inputs, which is when the strategy should be rebuilt to generate valid values directly.

Should I suppress the filter_too_much health check? Rarely. Suppressing it hides the fact that most examples are wasted. Suppress it only when the rejection rate is inherently high, the valid inputs cannot be constructed directly, and you have checked with statistics that enough valid examples still run.

← Back to Hypothesis Framework Fundamentals