Contract Testing for HTTP APIs

The failure this section is about has a distinctive signature. A provider team renames a field, ships it on Tuesday, and every consumer's error rate rises on Wednesday — each consumer's suite having been green throughout, because none of them tested the provider. Contract testing moves that detection to the moment of the change and the repository that made it, which is the only point at which the fix is cheap.

Prerequisites

• An HTTP integration with a schema, or one you are willing to write: an OpenAPI 3.x document is the usual form.
• openapi-core, schemathesis or jsonschema for response validation; pact-python >= 2.2 for consumer-driven contracts.
• pytest >= 8.0, and a test client for the consumer side — httpx, requests or the framework's own.
• Familiarity with faking at the transport layer, since the consumer tests still need a stand-in: see mocking network and HTTP calls.

Core concept: two directions, two guarantees

"Contract testing" covers two distinct techniques that answer different questions, and teams often adopt one while believing they have the benefits of both.

Schema validation asks: does this response conform to the document the provider published? It runs on the consumer's side, costs one assertion per response, and catches type changes, removed fields and undocumented status codes. It requires the provider to publish a schema and to keep it accurate.

Consumer-driven contracts ask: does the provider still satisfy what its consumers actually depend on? The consumer publishes a pact — a list of request/response pairs it relies on — and the provider's pipeline replays them against its real implementation. It catches everything schema validation does, plus semantic changes, and it fails in the provider's build rather than the consumer's.

Step-by-step implementation

1. Validate every response against the schema

import pytest
from openapi_core import OpenAPI
from openapi_core.contrib.requests import RequestsOpenAPIRequest, RequestsOpenAPIResponse


@pytest.fixture(scope="session")
def contract():
    # The provider's published document, vendored at a known version so a
    # remote edit cannot silently change what your tests assert.
    return OpenAPI.from_file_path("contracts/billing-openapi-2.4.yaml")


@pytest.fixture
def checked_client(http_client, contract):
    """A client that validates every response before returning it."""
    def get(path, **kwargs):
        response = http_client.get(path, **kwargs)
        contract.validate_response(
            RequestsOpenAPIRequest(response.request),
            RequestsOpenAPIResponse(response),
        )   # raises on an undocumented status, a missing field, a wrong type
        return response

    http_client.checked_get = get
    return http_client

Wrapping the client rather than asserting in each test is what makes this sustainable: one place to change, and no test can forget to validate. Vendoring the document rather than fetching it at runtime matters too — a schema pulled from the provider on every run means their edit changes your test results with no commit in your repository.

2. Describe what the consumer actually needs

import atexit

from pact import Consumer, Provider

pact = Consumer("checkout").has_pact_with(Provider("billing"), pact_dir="pacts")
pact.start_service()
atexit.register(pact.stop_service)


def test_checkout_reads_the_invoice_total(billing_client):
    expected = {
        "id": "inv_123",
        # Matchers, not literals: the consumer depends on the TYPE and the
        # presence of the field, not on this particular value.
        "total_minor": pact.Format().integer,
        "currency": "GBP",
    }

    (pact
     .given("invoice inv_123 exists and is open")   # provider state, by name
     .upon_receiving("a request for an invoice")
     .with_request("get", "/invoices/inv_123")
     .will_respond_with(200, body=expected))

    with pact:
        invoice = billing_client.fetch_invoice("inv_123")

    assert invoice.total_minor == 0 or isinstance(invoice.total_minor, int)

Two decisions in that pact are the ones that determine whether it is useful in a year. Matchers instead of literal values mean the provider can change the value without breaking the contract, which is correct — the consumer does not depend on the invoice being £12.34. And given(...) names a provider state rather than describing how to create it, so the provider's verification step decides how to arrange it.

3. Verify the pact in the provider's pipeline

# In the PROVIDER's repository
import pytest
from pact import Verifier


@pytest.fixture
def provider_states(app):
    """Maps the state names consumers used onto real setup in this service."""
    def setup(state: str) -> None:
        if state == "invoice inv_123 exists and is open":
            app.repository.insert(Invoice(id="inv_123", status="open", total_minor=1234))
        else:
            raise AssertionError(f"unknown provider state: {state}")
    return setup


def test_billing_satisfies_its_consumers(live_server, provider_states):
    verifier = Verifier(provider="billing", provider_base_url=live_server.url)
    success, _ = verifier.verify_pacts(
        "pacts/checkout-billing.json",
        provider_states_setup_url=f"{live_server.url}/_pact/state",
    )
    assert success == 0

Raising on an unknown state rather than ignoring it is deliberate. A consumer that adds an interaction with a state the provider has not implemented should fail loudly during verification, not silently skip the check.

4. Keep recordings honest

import pytest
import vcr

# Recordings make consumer tests fast and offline — and stale, silently.
billing_vcr = vcr.VCR(
    cassette_library_dir="tests/cassettes",
    record_mode="none",            # never record accidentally in CI
    match_on=["method", "scheme", "host", "port", "path", "query"],
)


@pytest.mark.contract
def test_cassettes_still_match_the_schema(contract):
    """Runs nightly against the real sandbox with record_mode='all'."""
    with billing_vcr.use_cassette("invoice_open.yaml", record_mode="all"):
        response = requests.get("https://sandbox.billing.test/invoices/inv_123")
    contract.validate_response(...)

record_mode="none" in the default configuration is the safeguard that matters. Without it, a cassette that fails to match is silently re-recorded against whatever is reachable, which turns a detected drift into an updated file and a green build.

Verification

A contract setup is working when a deliberate break fails in the right place. Test it once, on purpose:

# In the provider, rename a field the consumer depends on, then:
pytest tests/contract/test_pact_verification.py -q

Verifying a pact between checkout and billing
  Given invoice inv_123 exists and is open
    a request for an invoice
      returns a response which
        has a matching body
          $.total_minor: Expected 'total_minor' but was missing
1 interaction, 1 failure

The failure names the consumer, the state, the path and the missing field — a complete description of who will break and why. Seeing this output once gives the team confidence to trust a green verification afterwards, which is the entire value of the setup.

Troubleshooting

Provider states are the hard part

Everything about consumer-driven contracts is straightforward except provider states, and they are where implementations bog down.

A state is a named precondition — "invoice inv_123 exists and is open" — that the provider must arrange before replaying an interaction. The consumer chose the name; the provider decides what it means. The failure mode is states that are too specific, because each one becomes a bespoke setup routine in the provider's codebase, and a hundred consumers' interactions produce a hundred setup functions nobody maintains.

The correction is to standardise the vocabulary. A dozen states — "an open invoice exists", "a paid invoice exists", "no invoice exists" — cover nearly every interaction if the consumer uses matchers rather than literal identifiers. When a consumer asks for state "invoice inv_9f3c exists with total 4211 in EUR", that is a signal the consumer is asserting on data it should not care about.

The second difficulty is state setup speed. Each interaction resets the provider's state, so a hundred interactions is a hundred setups; if each one truncates and reseeds a database, verification takes minutes. The transactional pattern applies here too — wrap each interaction's setup in a transaction and roll it back, exactly as in database fixtures and transactional tests.

Third-party providers, where verification is impossible

Most of the above assumes both sides are yours. When the provider is a payment processor or a mapping service, there is no pipeline to add a verification step to, and the technique degrades to something still worth doing.

Vendor their OpenAPI document — or write one from the parts you use — and validate every response against it, including the ones your recorded cassettes replay. Then add a small scheduled job that makes real requests against the provider's sandbox and validates those responses too. That job is not a test of your code; it is a monitor, and it should notify rather than fail a build. What it buys is notice: a field that changed type in the sandbox gives days or weeks of warning before it reaches production, which is enough time to adapt calmly.

The second half of the strategy is defensive parsing. Code that reads payload["data"]["attributes"]["amount"] breaks on any structural change; code that parses the payload into a typed model with explicit field mapping fails at one place with a clear message. Combining a schema check in tests with a strict parser in production means an unexpected change produces a specific, logged, attributable error rather than a KeyError deep in a handler — and the difference in diagnosis time is measured in hours.

Versioning a contract that has to change

Contracts exist to make change visible, not to prevent it. A provider that can never add a field or deprecate an endpoint has replaced one problem with a worse one, so the practice needs an explicit story for evolution.

The rule that does most of the work is the expand–migrate–contract sequence. Adding a field is always safe, because consumers verified against matchers ignore fields they did not ask for. Removing one is not, so a removal becomes three changes over time: add the replacement, wait until no verified consumer depends on the old field, then remove it. The middle step is exactly what a broker records — which consumer versions, deployed where, verified against which provider version — and it is the reason brokers earn their place once more than one consumer exists.

Two smaller conventions keep evolution manageable. Consumers should match on the minimum they need: requesting fewer fields means fewer reasons to fail verification, and a pact that mirrors the entire response is a contract on the provider's whole payload. And additive changes should not require a new pact at all — if adding a field breaks verification, the consumer is matching too strictly, usually by comparing whole bodies rather than the fields it reads.

Where a genuinely breaking change is unavoidable and coordination is impossible, versioning the endpoint is the escape hatch. Two paths, two schemas, two sets of verified pacts, and a deprecation date on the old one. It is more work than it looks — the provider now maintains both — which is precisely why the expand–migrate–contract path is worth exhausting first.

Choosing what to adopt

Not every integration deserves the same investment, and the useful ordering is by blast radius.

An internal service with several consumers and an active team is the case consumer-driven contracts were designed for; adopt them fully, broker included, because the coordination cost they remove is real. An internal service with one consumer can usually get by with schema validation plus a shared integration test, since both sides are deployed together often enough that the detection gap is short. A third-party provider gets vendored schemas, validated recordings and a scheduled sandbox monitor. And an integration you call once a month from a batch job probably needs none of it — a clear error and an alert is proportionate.

What is never proportionate is adopting the full apparatus for one integration and nothing for the other twelve. The failure mode of contract testing as a practice is that it becomes a ceremony performed on whichever integration someone was burned by last, while the next breakage arrives through one of the untouched ones. Deciding the policy per integration, writing it down, and revisiting it when the topology changes is more valuable than any particular tool choice.

What contract testing does not replace

Two things get quietly dropped when a team adopts contracts, and both are missed later.

The first is a small number of genuine end-to-end tests. A contract proves that each pair of services agrees on the messages between them; it proves nothing about whether the sequence of calls accomplishes anything. A checkout flow can have four perfectly verified contracts and still fail because the second service returns before the third has committed. Keep a handful of tests that drive the whole path against real deployments, accept that they are slow and occasionally flaky, and run them on a schedule rather than on every merge.

The second is monitoring. Contract verification runs against a provider's build, which is a different artefact from what is currently deployed, and deployment can lag or roll back. A synthetic request against production, checked against the same schema the tests use, closes that gap for the cost of one scheduled job. It also catches the failure no contract can — a provider that is up, schema-correct, and returning an empty list because its database replica is behind.

Both are cheap to keep and expensive to reintroduce after they have been deleted as redundant, which is the usual sequence. The framing that avoids it: contracts are about agreement, end-to-end tests are about composition, and monitoring is about reality. Dropping any one of the three leaves a class of failure with nothing watching for it.

Frequently Asked Questions

What does a contract test catch that an integration test does not? A breaking change made by the provider, before it is deployed. An integration test runs your code against whatever the provider is running now, so it goes red after the breakage ships. A contract verified in the provider's own pipeline fails their build, which is the only place the change can still be cheaply reverted.

Do I need a Pact broker to do consumer-driven contract testing? Not to start. Pact files are JSON and can be committed to the provider's repository or published as a build artifact. A broker becomes worthwhile when several consumers exist, because it tracks which versions of which consumer have verified against which provider version, which a file in a repository cannot.

Is validating responses against OpenAPI enough on its own? It catches schema drift, which is most of what breaks, and it costs almost nothing once wired into the test client. It does not catch semantic changes — a field that keeps its type but changes meaning, or a status code that starts being returned in a new situation. Schema validation is the cheap first layer, not the whole story.

How do recorded HTTP exchanges fit with contract testing? Recordings make consumer tests fast and offline, but they freeze the provider's behaviour at the moment of recording and go stale silently. Pair them with a scheduled job that re-records against the real service, or with schema validation of the recorded payloads, so a drifted cassette fails rather than lying.

Who owns the contract when the provider is a third party? You do, unilaterally. There is no verification step you can run in their pipeline, so the contract becomes a schema you assert their responses against, plus a scheduled test against their sandbox. The value is early warning rather than prevention, which is still far better than discovering the change from a production error rate.

Related guides

• Add the cheap layer first with validating responses against an OpenAPI schema.
• Set up the full loop in consumer-driven contract tests with Pact Python.
• Keep recorded exchanges from rotting using recording and replaying HTTP with VCR.py.
• Make the consumer's own tests fast and offline with mocking httpx clients with respx.
• Give the provider's verification step fast state setup via database fixtures and transactional tests.

← Back to Integration, Database & Service Testing