Python Testing & Debugging Beyond unittestAdvanced Python testing playbooks
Isolation & Contracts

Catching Signature Drift with spec_set

Autospec checks how a mock is called. It does not check how a test configures it. A test that sets client.retrys = 3 on an autospecced client — misspelling retries — succeeds silently: the mock gains a new attribute, the code under test reads client.retries, gets an auto-created child mock instead of 3, and the test either passes for the wrong reason or fails somewhere far from the typo. The same thing happens, more insidiously, when the real class renames an attribute and the test keeps setting the old name.

spec_set closes that gap. A mock created with it rejects assignment to any attribute the real object does not have, so the misspelling or the stale name raises AttributeError on the line that set it. It is a one-word change with an outsized effect on how quickly configuration drift is noticed.

The underlying issue is that unittest.mock is permissive by design. A plain Mock or MagicMock accepts any attribute read, any attribute write and any call, because it was built for exploratory use where anything goes. Each of spec, autospec and spec_set removes one of those freedoms. Most suites adopt the first two and stop, leaving writes unchecked — and writes are precisely how tests configure the state the code under test reads. Closing that last gap costs one keyword argument per mock factory and changes no test that was already correct.

Prerequisites

Solution

Python
from dataclasses import dataclass
from unittest.mock import create_autospec

import pytest


class HttpClient:
    retries: int = 3                          # class-level: visible to the spec
    timeout: float = 5.0

    def get(self, path: str) -> dict: ...


def test_misspelt_configuration_fails_immediately():
    client = create_autospec(HttpClient, instance=True, spec_set=True)

    with pytest.raises(AttributeError, match="retrys"):
        client.retrys = 0                     # the typo is caught at assignment


def test_real_configuration_is_accepted():
    client = create_autospec(HttpClient, instance=True, spec_set=True)
    client.retries = 0                        # a real attribute: fine
    client.get.return_value = {"ok": True}

    assert fetch_without_retry(client) == {"ok": True}
What spec, autospec and spec_set each check Three levels of strictness. spec rejects reading attributes the real object lacks. autospec additionally enforces call signatures. spec_set additionally rejects assigning attributes the real object lacks. Only spec_set catches a test that configures a misspelt or renamed attribute. Reads, calls, and writes — three different checks check spec autospec + spec_set mock.unknown (read) rejected rejected rejected mock.get(wrong_arg=1) accepted rejected rejected mock.retrys = 0 (write) accepted accepted rejected The bottom row is where configuration typos and stale attribute names hide.
Autospec plus spec_set is the only combination that checks every way a test interacts with a mock.

Why this works

A mock with a spec keeps a list of the attribute names the specification object exposes. Reading an attribute not on the list raises. spec_set makes the mock consult the same list on assignment, through its __setattr__, and raise for any name not present. Because the list comes from the real class, any attribute the real class renames or removes becomes unassignable on the mock the moment the class changes — and every test that still configures the old name fails, on the configuring line, with the old name in the message.

The check happens at assignment time, which is the earliest possible moment: the failing line is the one with the wrong name on it, not some later line where the code reads a value it did not expect. That locality is what makes spec_set failures quick to fix compared with the confusing downstream failures a silently-accepted typo produces.

create_autospec(..., spec_set=True) combines this with signature enforcement, and applies it recursively to child attributes, so a nested configuration such as client.session.timeout = 1 is checked at every level.

Edge cases and failure modes

  • Attributes set only in __init__. self.retries = 3 inside __init__ is invisible to a class-based spec, so spec_set rejects a legitimate assignment. Declare the attribute as a class-level annotation, or spec from an instance.
  • Dataclasses. Fields are class-level annotations, so spec_set sees them — dataclasses work well with it out of the box.
  • Properties. Assigning to a mocked property with spec_set fails because properties are read-only on the class. Configure them with PropertyMock on the type instead.
  • __slots__ classes. The slot names are visible to the spec and work normally.
  • Dynamic attributes. Objects that accept arbitrary attributes by design — configuration bags, SimpleNamespace — defeat the point of spec_set. Declare a dataclass for the fields the code reads and spec against that.

Where drift actually comes from

The failure spec_set prevents is rarely a typo on the day a test is written; the author usually runs the test and notices. It is a rename months later. A class's timeout_seconds becomes timeout, the production code is updated, the type checker is satisfied, and forty tests continue to set timeout_seconds on their mocks. Without spec_set, those tests still pass — the attribute they set is simply never read, and the code reads an auto-created child mock for timeout instead. The tests have stopped testing what they claim to test, and nothing indicates it.

With spec_set, the rename breaks all forty on the line that sets the old name. That feels like a cost at the moment of the rename and is actually the whole point: the forty tests were about to become silently meaningless, and the failures are a precise list of the places that need updating. A search-and-replace fixes them in a minute, and every one of them is then testing the real attribute again.

Configuration and settings objects are where this pays off most, because they are read in many places and renamed during refactors more often than behavioural classes. A Settings dataclass mocked with spec_set across a suite turns every settings rename into a compile-time-like check across every test that configures it. Often the better move for settings is not to mock them at all but to construct a real instance with overridden values — a dataclass with defaults is its own best test double — which gets the same protection from the constructor's own argument checking. A misspelt keyword to a dataclass constructor fails immediately, with no mocking library involved.

A rename with and without spec_set A class renames timeout_seconds to timeout. Without spec_set, forty tests keep setting the old name, the code reads an auto-created mock for the new name, and the tests pass while testing nothing. With spec_set, all forty fail on the line that sets the old name, giving a precise list of places to update. timeout_seconds → timeout without spec_set tests set mock.timeout_seconds = 1 code reads mock.timeout → a Mock 40 tests green, testing nothing no signal anything changed with spec_set mock.timeout_seconds = 1 → AttributeError on that line 40 precise failures fixed by one search-and-replace
Forty red tests is a better outcome than forty green tests that stopped meaning anything — and it takes a minute to fix.

Making instance attributes visible to the spec

The most common reason teams abandon spec_set is the first legitimate assignment it rejects. A class that sets its attributes in __init__ — self.retries = retries — exposes nothing at class level, so a class-based spec does not know retries exists, and client.retries = 0 on the mock raises. It looks like spec_set being unreasonable; it is actually the spec being incomplete.

There are three fixes, in order of preference. The first is to declare the attributes as class-level annotations, which is good practice anyway for type checkers and costs one line per attribute:

Python
class HttpClient:
    retries: int
    timeout: float

    def __init__(self, retries: int = 3, timeout: float = 5.0) -> None:
        self.retries = retries
        self.timeout = timeout

Annotations without values do not create class attributes at runtime, but create_autospec reads them, so the spec now includes retries and timeout. The second fix is to spec from a real instance — create_autospec(HttpClient(), spec_set=True) — which sees the instance attributes directly, at the cost of constructing the real object. The third, for classes you do not own, is a Protocol or dataclass declaring the attributes the code reads, as with dynamic clients.

Three ways to make instance attributes visible Three options for attributes assigned only in init. Class-level annotations make them visible to a class-based spec at no runtime cost. Speccing from a real instance sees them directly but constructs the real object. A Protocol or dataclass declaring the attributes works for classes you do not own. Give the spec the attributes it cannot otherwise see class annotations retries: int no runtime cost preferred spec an instance create_autospec(Cls()) sees instance attributes needs a cheap constructor Protocol / dataclass declare what you read for classes you don't own verify against the real class
The first option also improves type checking of the production code, so it is rarely extra work in a codebase that already uses annotations.

Rolling it out across a suite

Switching a large suite to spec_set is best done by collaborator rather than all at once. Start with the objects most often configured by attribute assignment — settings, clients with timeouts and retry counts, domain models whose status tests set — and change the fixtures that build their mocks to pass spec_set=True. Run the suite; every failure is either a real stale attribute name, which gets fixed, or an instance attribute the spec cannot see, which gets a class annotation.

Because the change lives in the fixtures, individual tests do not need editing unless they were genuinely wrong. After a few collaborators the pattern is established and the remaining ones follow quickly, and from then on every rename in those classes is caught by the suite rather than discovered when a test that should have failed turns out to have been passing for months.

Frequently Asked Questions

What is the difference between spec and spec_set?spec restricts which attributes can be read from the mock; reading one the real object lacks raises AttributeError. spec_set additionally restricts which attributes can be assigned, so configuring mock.retrys = 3 when the real attribute is retries also raises. spec_set catches drift in the test's own configuration.

Does create_autospec use spec_set? Only if asked. create_autospec(Target, spec_set=True) enforces both signatures and assignment restrictions. Without spec_set=True, autospecced mocks still allow arbitrary attribute assignment.

Why would a test assign to a mock attribute at all? To configure state the code reads: a client's timeout, a config object's feature flags, a model's status. Those assignments are exactly where a rename in the real class goes unnoticed, because the test sets the old name and the code reads the new one.

← Back to Autospec & Strict Mocking