Patching Builtins and sys.modules Safely

A test does sys.modules["boto3"] = MagicMock() to dodge a slow import, passes locally, then the next test that imports boto3 for real gets the mock — and the suite fails only under random ordering or pytest-xdist. The same trap appears when patching open or print: assign in the wrong namespace and the override silently does nothing, or assign without restoration and it bleeds across the whole run. sys.modules and builtins are process-global, so the fix is always the same shape: snapshot-and-restore via patch.dict/patch.object (or monkeypatch), and target the exact namespace where the name is resolved.

Prerequisites

• Python 3.8+ (unittest.mock.mock_open, patch.dict, and patch.object are all standard since 3.3).
• pytest 7.0+ for monkeypatch examples; the unittest.mock examples need no third-party packages.
• Optional: pytest-xdist if you want to validate isolation under parallel workers.

Solution

sys.modules is the interpreter's import cache: an import x first checks sys.modules["x"] and returns the cached object if present. A raw assignment mutates that global and is never undone. patch.dict snapshots the dictionary, applies your overrides, and restores it on exit — even if the test raises:

import sys
from unittest.mock import patch, MagicMock

def test_isolated_sys_modules():
    fake = MagicMock()
    fake.region.return_value = "us-east-1"
    # patch.dict snapshots sys.modules and restores it on exit.
    with patch.dict(sys.modules, {"slow_sdk": fake}):
        import slow_sdk                       # resolves to `fake` from the cache
        assert slow_sdk.region() == "us-east-1"
    # Outside the block, sys.modules no longer contains the fake.
    assert "slow_sdk" not in sys.modules or sys.modules["slow_sdk"] is not fake

Under pytest, monkeypatch.setitem is the idiomatic equivalent and is reverted automatically in fixture teardown:

import sys
from unittest.mock import MagicMock

def test_with_monkeypatch(monkeypatch):
    fake = MagicMock()
    monkeypatch.setitem(sys.modules, "slow_sdk", fake)   # auto-restored on teardown
    import slow_sdk
    assert slow_sdk is fake

Patching a builtin has a second pitfall on top of restoration: targeting. CPython resolves a name like open by walking the consuming module's globals and then builtins, so you must patch where the code looks it up, not where it is defined. The simplest correct target is builtins.open (or mymod.open if the module did from builtins import open-style rebinding). mock_open builds a file double that already supports the context-manager and iteration protocols:

from unittest.mock import patch, mock_open

def load_first_line(path):
    with open(path) as fh:                    # this `open` resolves to builtins.open
        return fh.readline()

def test_open_with_mock_open():
    m = mock_open(read_data="line-1\nline-2\n")
    # Patch the builtin where the function resolves it.
    with patch("builtins.open", m):
        assert load_first_line("ignored.txt") == "line-1\n"
    m.assert_called_once_with("ignored.txt")  # call is recorded

For builtins whose call signature matters (so a wrong-arity call fails the test the way it would in production), add autospec=True via patch.object:

import builtins
from unittest.mock import patch

def test_print_signature_enforced(capsys):
    with patch.object(builtins, "print", autospec=True) as mock_print:
        print("hello", "world", sep="-")     # valid call against the real signature
        mock_print.assert_called_once_with("hello", "world", sep="-")

When you must coordinate several global patches, contextlib.ExitStack (or stacked with) guarantees reverse-order teardown even if one patch raises mid-setup:

import sys, builtins
from contextlib import ExitStack
from unittest.mock import patch, MagicMock, mock_open

def test_multiple_global_patches():
    with ExitStack() as stack:
        stack.enter_context(patch.dict(sys.modules, {"slow_sdk": MagicMock()}))
        stack.enter_context(patch("builtins.open", mock_open(read_data="x")))
        # ...exercise code that touches both globals...
        import slow_sdk
        assert slow_sdk is sys.modules["slow_sdk"]
    # All patches reverted in reverse registration order here.

Why this works

patch.dict and monkeypatch.setitem treat the global as a transaction: they record the prior value (or absence) of each key and write it back on teardown, so leakage is impossible regardless of test order or exceptions. Targeting builtins.open rather than a stale per-module reference works because the function's bytecode performs the lookup at call time through its module globals into builtins, which is exactly the object you replaced. Choosing mock_open over a hand-wired double matters because file usage almost always goes through the with context-manager protocol, which mock_open implements for you — the same auto-configuring behaviour documented in the deep dive into unittest.mock.

Edge cases and failure modes

• Module already imported before the patch. If slow_sdk was imported during pytest collection, patch.dict replaces the cached object but any module that already bound from slow_sdk import thing keeps its old reference. Patch before first import, or patch the bound name in the consuming module.
• Wrong builtin namespace. Patching builtins.open has no effect on a module that did something unusual to rebind open locally; confirm the target with inspect.getmodule or by patching the consuming module's open directly (patch("mymod.open")).
• patch().start() without stop(). Calling start() outside a context manager and forgetting stop() (or addCleanup(p.stop)) is the classic leak. Prefer the with form or monkeypatch, which cannot forget.
• autospec on C-level builtins. A few builtins have signatures that introspection cannot fully model; if autospec=True raises during setup, fall back to a plain mock or mock_open and assert call args manually. For the cases where introspection does succeed, autospec and strict mocking keeps the double honest against the real signature.
• Async teardown ordering. Patching globals around await boundaries can outlive the coroutine if the patch context exits before a scheduled callback runs; scope patches to the awaited region. This overlaps with the "Event loop is closed" failures discussed under systematic debugging and performance profiling.

Restoring global state deterministically

Every patch of a builtin or of sys.modules mutates process-wide state, so the question that matters is not how to set it but how to guarantee it is put back — including when the test raises part-way through.

monkeypatch and mock.patch both restore on failure because they unwind in a finally. Hand-rolled assignment does not, and a single sys.modules["boto3"] = fake left behind by a failing test poisons every later test in the session with a fake AWS client. If you must assign directly, wrap it in a fixture with a yield so teardown runs regardless of outcome.

import sys
import types
import pytest

@pytest.fixture
def fake_boto3():
    """Install a stand-in module and guarantee removal, even if the test raises."""
    original = sys.modules.get("boto3")          # may legitimately be None
    module = types.ModuleType("boto3")
    module.client = lambda *a, **kw: object()
    sys.modules["boto3"] = module
    try:
        yield module
    finally:
        if original is None:
            sys.modules.pop("boto3", None)       # it was never imported: remove it
        else:
            sys.modules["boto3"] = original      # restore the real module object

The original is None branch is the part hand-written cleanups get wrong: popping a key that was never there is not the same as restoring a module that was, and setting sys.modules["boto3"] = None makes every later import raise ImportError in a way that looks like a packaging problem.

Two further rules keep global patches from leaking across a parallel run. First, never patch a builtin at session scope — with pytest-xdist each worker is a separate process, so a session-scoped patch multiplies rather than shares, and a test that reads the patched builtin in one worker sees a different object than the assertion checking it in another. Second, assert the restoration in a guard test that runs last: a single assert "boto3" not in sys.modules at the end of the module catches leaks the moment they are introduced, rather than three weeks later when an unrelated test starts failing.

Frequently Asked Questions

Why does my mock leak into other tests after patching sys.modules? Because a bare assignment like sys.modules['pkg'] = MagicMock() is never reverted. sys.modules is a process-global cache, so the next import of pkg returns your mock. Use patch.dict(sys.modules, ...) or monkeypatch.setitem, both of which restore the original on teardown.

Where do I patch a builtin like open so the override actually takes effect? Patch it in the namespace where the code under test looks it up. For most code that is builtins.open or the consuming module's namespace (mymod.open). mock_open() from unittest.mock gives a ready-made file handle that supports read, readline, and iteration.

Do I still need to clean up sys.modules under pytest-xdist? Yes. xdist isolates workers in separate processes but each worker runs many tests in one interpreter, so within-worker leakage still happens. Use patch.dict or monkeypatch so teardown reverts the cache regardless of worker.

Is patching builtins.open safe when the code also reads files through pathlib? Only partly. Path.read_text and Path.open route through io.open, which is the same object as builtins.open, so a builtins.open patch does intercept them. But os.scandir, os.stat and any C extension that opens files directly do not, so a patch that looks total is not. Prefer tmp_path with real files when the code touches the filesystem through more than one API.

Related

• Where to patch: understanding mock.patch targets — the namespace-resolution rule that decides whether your builtins.open patch takes effect at all.
• Deep dive into unittest.mock — how mock_open, MagicMock, and patch.dict are wired internally.
• Autospec and strict mocking — enforce a builtin's real signature so wrong-arity calls fail like production.
• Controlling time and randomness in tests — the same snapshot-and-restore discipline applied to time, datetime, and random.
• Systematic debugging and performance profiling — diagnosing the order-dependent and async-teardown failures that global-state leaks produce.

← Back to Patching Strategies for Complex Codebases