Pytest & CI

Creating conftest.py hierarchies for monorepos

Creating conftest.py hierarchies for monorepos

Scaling pytest across a multi-package monorepo introduces architectural complexity that flat test configurations cannot sustain. As test suites grow, implicit fixture resolution, unpredictable plugin loading, and collection bloat frequently degrade CI reliability and developer velocity. Establishing a deterministic conftest.py hierarchy is not merely an organizational preference; it is a foundational requirement for maintaining strict scope boundaries, preventing cross-package leakage, and enabling parallelized execution. This guide details production-grade strategies for architecting, debugging, and validating conftest.py hierarchies in modern Python monorepos.

Monorepo Test Architecture: Why conftest.py Hierarchies Fail at Scale

When scaling pytest across a monorepo, developers frequently encounter unpredictable fixture resolution and silent collection failures. Understanding how pytest traverses directories and loads implicit plugins is foundational to avoiding architectural debt. The Advanced Pytest Architecture & Configuration framework provides the necessary mental model for mapping collection boundaries to package structures.

At its core, pytest does not rely on Python's standard import system for test discovery. Instead, it executes an upward directory traversal starting from each collected test file. During this traversal, pytest identifies conftest.py files and registers them as implicit plugins. These plugins are loaded into the FixtureManager in Last-In-First-Out (LIFO) order, meaning the nearest conftest.py to the test file takes precedence over parent directories. While this mechanism simplifies local test execution, it becomes a liability in monorepos where multiple services share overlapping fixture names or where sys.path manipulation interferes with collection boundaries.

Collection bloat typically originates from flat conftest.py structures. When a single root configuration file declares dozens of session-scoped fixtures, pytest eagerly evaluates them for every collected test, regardless of whether the test actually requests them. This eager evaluation compounds with autouse=True fixtures, triggering expensive database migrations, network stubs, or environment variable injections across unrelated packages. Furthermore, namespace package discovery (PEP 420) introduces subtle pitfalls: directories lacking __init__.py may be silently skipped during traversal, breaking expected fixture inheritance chains and causing FixtureLookupError exceptions that only manifest in CI environments.

The interaction between testpaths in pyproject.toml and sys.path resolution further complicates matters. If testpaths is misconfigured, pytest may inadvertently collect production modules, triggering import-time side effects or registering fixtures in unintended scopes. Without explicit boundary enforcement, monorepo test suites devolve into tightly coupled dependency graphs where modifying a single fixture in one package triggers cascading failures across unrelated services.

Designing the Hierarchy: Root, Domain, and Module Scopes

A resilient monorepo testing strategy requires explicit boundary enforcement. By delegating responsibilities across root, domain, and module tiers, teams can prevent cross-package fixture collisions. Detailed implementation patterns for this tiered approach are documented in Managing Conftest Hierarchies, which outlines scope delegation and safe override mechanics.

The three-tier delegation model maps directly to pytest's fixture scoping system:

  1. Root Tier (/tests/conftest.py): Handles global infrastructure, CI environment detection, and cross-cutting concerns. Fixtures here should be strictly session-scoped and lazily initialized. This tier is the appropriate location for pytest_addoption hooks, global logging configuration, and baseline pytest_plugins declarations.
  2. Domain/Service Tier (/tests/<package>/conftest.py): Encapsulates service-specific mocks, database schemas, and API client factories. Fixtures here typically operate at module or class scope. This tier enforces package isolation by overriding root fixtures with domain-specific implementations and applying marker-based filtering to prevent accidental cross-domain execution.
  3. Module/Edge-Case Tier (/tests/<package>/test_<module>/conftest.py): Reserved for highly localized parametrization, temporary directory overrides, and test-specific state resets. These fixtures should remain function-scoped and avoid autouse=True unless strictly necessary for cleanup.

Strict scope isolation relies on explicit pytest_plugins registration rather than implicit Python imports. Using import statements to share fixtures across packages bypasses pytest's internal dependency graph, causing fixtures to be registered in the wrong collection phase and breaking request.node introspection. Instead, declare shared fixture modules as strings in pytest_plugins = ["tests.shared.fixtures.db"]. This ensures pytest loads them as proper plugins with correct hook execution order.

Additionally, configure testpaths in pyproject.toml to restrict collection to dedicated test directories. Never place __init__.py files inside test directories. Their presence converts test folders into Python packages, altering sys.path resolution and causing pytest to treat conftest.py files as regular modules rather than implicit plugins. This distinction is critical for maintaining predictable pytest fixture scope isolation across large codebases.

Implementation: Minimal Reproducible Hierarchy

The following directory structure demonstrates a production-ready hierarchy. Each conftest.py file is scoped to its logical boundary, with explicit plugin registration replacing implicit import leakage.

Plain text
monorepo/
├── pyproject.toml
├── src/
│ ├── auth/
│ └── billing/
└── tests/
 ├── conftest.py # Root: Global CI & Plugin Registration
 ├── shared/
 │ └── fixtures/
 │ ├── db.py
 │ └── network.py
 ├── auth/
 │ ├── conftest.py # Domain: Auth-specific overrides
 │ └── test_sessions.py
 └── billing/
 ├── conftest.py # Domain: Billing-specific overrides
 └── test_invoices.py

Root conftest.py: Global Infrastructure & Plugin Registration

The root configuration establishes the baseline environment and registers shared fixture modules. It avoids heavy initialization, deferring expensive operations until explicitly requested.

Python
# tests/conftest.py
import pytest
from pathlib import Path

# Explicit plugin registration replaces cross-package imports
pytest_plugins = ["tests.shared.fixtures.db", "tests.shared.fixtures.network"]

def pytest_addoption(parser):
 """Register CLI options for environment-specific toggles."""
 parser.addoption(
 "--env", 
 default="local", 
 choices=["local", "ci", "staging"],
 help="Target environment for test execution"
 )

@pytest.fixture(scope="session", autouse=True)
def global_config(request):
 """Session-scoped configuration resolver.
 Uses request.config.getoption() to inject environment context.
 """
 env = request.config.getoption("--env")
 return {
 "env": env,
 "db_url": f"sqlite:///test_{env}.db",
 "mock_network": env == "ci"
 }

Domain conftest.py: Service-Specific Overrides & Isolation

Domain-level configurations override root fixtures when necessary and enforce package boundaries using markers and autouse guards.

Python
# tests/auth/conftest.py
import pytest

@pytest.fixture(scope="module")
def service_client(global_config):
 """Domain-scoped client factory.
 Inherits global_config but applies auth-specific initialization.
 """
 return create_auth_client(global_config["db_url"])

@pytest.fixture(autouse=True)
def enforce_domain_boundary(request):
 """Prevents cross-domain test execution without explicit markers.
 Uses request.node.keywords for marker introspection.
 """
 if "cross_domain" not in request.node.keywords:
 yield
 else:
 pytest.skip("Cross-domain tests require explicit @pytest.mark.cross_domain")

Key implementation principles:

  • Placement outside src/: All conftest.py files reside in the tests/ directory to prevent production import pollution.
  • pytest_plugins syntax: Always use list format. String concatenation or dynamic assignment breaks plugin discovery.
  • Fixture override mechanics: Child conftest.py files automatically override parent fixtures with identical names. Use @pytest.fixture(autouse=True) cautiously to avoid unintended teardown side effects.
  • Environment toggles: request.config.getoption() enables deterministic behavior across local development and CI pipelines without modifying test code.

Edge-Case Resolution: Cross-Package Fixture Collisions & Namespace Packages

When sibling packages define identically named fixtures, pytest resolves them based on collection order, leading to non-deterministic test behavior. Use pytest --fixtures --verbose to trace definition paths and enforce explicit naming conventions.

Fixture Name Resolution Order

Pytest's FixtureManager builds a dependency graph during collection. If two conftest.py files in the same directory tree define @pytest.fixture(name="db_session"), the nearest file to the test file wins. However, if tests are collected in parallel or via glob patterns, the resolution order becomes unpredictable. This manifests as intermittent AssertionError failures where tests expect a mocked database but receive a production connection.

Resolution Strategy:

  1. Prefix fixtures with domain identifiers (e.g., auth_db_session, billing_db_session).
  2. Use @pytest.fixture(name="explicit_name") to enforce uniqueness.
  3. Implement a CI linter to scan for duplicate fixture definitions before merge.

Diagnostic Script: Fixture Collision Detector

Automated collision detection prevents hierarchy drift. The following AST-based scanner identifies duplicate fixture names across the monorepo without executing tests.

Python
# scripts/scan_fixture_collisions.py
import ast
import sys
from pathlib import Path

def scan_conftest_fixtures(root: Path) -> dict[str, list[str]]:
 """Scan all conftest.py files and report duplicate fixture names."""
 fixtures: dict[str, list[str]] = {}
 
 for conf in root.rglob("conftest.py"):
 try:
 tree = ast.parse(conf.read_text())
 except SyntaxError:
 continue
 
 for node in ast.walk(tree):
 if isinstance(node, ast.FunctionDef):
 for dec in node.decorator_list:
 # Handle @pytest.fixture and @pytest.fixture(name="...")
 if isinstance(dec, ast.Call) and hasattr(dec.func, "attr") and dec.func.attr == "fixture":
 # Check for explicit name kwarg
 name_kwarg = next((kw.value.value for kw in dec.keywords if kw.arg == "name"), None)
 fixture_name = name_kwarg or node.name
 fixtures.setdefault(fixture_name, []).append(str(conf.relative_to(root)))
 elif isinstance(dec, ast.Name) and dec.id == "fixture":
 fixtures.setdefault(node.name, []).append(str(conf.relative_to(root)))
 
 return {k: v for k, v in fixtures.items() if len(v) > 1}

if __name__ == "__main__":
 root = Path(sys.argv[1] if len(sys.argv) > 1 else ".")
 collisions = scan_conftest_fixtures(root)
 if collisions:
 print("️ Fixture collisions detected:")
 for name, paths in collisions.items():
 print(f" {name}: {', '.join(paths)}")
 sys.exit(1)
 print("✅ No fixture collisions found.")

PEP 420 Namespace Package Pitfalls

Directories without __init__.py are treated as implicit namespace packages. While beneficial for source code, they disrupt pytest's upward traversal. If a parent directory lacks __init__.py, pytest may skip conftest.py files entirely, breaking fixture inheritance.

Resolution: Add empty __init__.py to test directories containing conftest.py, or explicitly configure pythonpath = ["."] in pyproject.toml. Alternatively, use pytest --ignore=src to force explicit test path resolution and bypass namespace package ambiguity.

Advanced Control: Custom Hooks & Collection Modification

For large-scale monorepos, static conftest.py files often require runtime augmentation. Hooking into pytest_collection_modifyitems allows teams to filter, reorder, or inject markers without modifying test files.

Dynamic Test Filtering & Marker Injection

The pytest_collection_modifyitems hook receives the complete list of collected Item objects before execution. This enables package-aware filtering, which is essential for CI pipelines running targeted test suites.

Python
# tests/conftest.py (advanced hook integration)
import pytest

def pytest_collection_modifyitems(config, items):
 """Filter tests based on --package CLI option or inject markers."""
 target_pkg = config.getoption("--package", default=None)
 if not target_pkg:
 return
 
 skip_marker = pytest.mark.skip(reason=f"Excluded: --package={target_pkg}")
 for item in items:
 # Check if test belongs to target package path
 if target_pkg not in str(item.fspath):
 item.add_marker(skip_marker)

Early Plugin Bootstrapping via pytest_configure

The pytest_configure hook executes before collection begins, making it ideal for dynamic pytest_plugins assignment or environment validation. However, modifying pytest_plugins at this stage requires caution to avoid hook recursion.

Python
def pytest_configure(config):
 """Validate environment and conditionally register plugins."""
 env = config.getoption("--env")
 if env == "staging":
 # Dynamically register staging-specific fixtures
 config.pluginmanager.register(StagingNetworkMock(), "staging_network")
 
 # Prevent recursive hook execution
 if not hasattr(config, "_hierarchy_validated"):
 validate_hierarchy_integrity(config)
 config._hierarchy_validated = True

Avoiding Hook Recursion & Stack Overflow

Pytest's hook execution follows a strict chain. Modifying plugin state or calling config.pluginmanager.register() during collection can trigger infinite recursion if not guarded. Always use configuration flags (config._hierarchy_validated) to ensure hooks execute exactly once. Additionally, avoid calling pytest.main() or re-invoking collection inside hooks, as this resets the internal state machine and corrupts fixture caches.

Validation & Profiling: Ensuring Hierarchy Integrity

Continuous validation prevents hierarchy drift. Integrate pytest --trace-config into CI pipelines to audit plugin loading sequences and enforce deterministic collection across environments.

Plugin Loading Audit

pytest --trace-config outputs the exact order in which conftest.py files and plugins are registered. Scan this output for unexpected duplicates or out-of-order loading. In CI, pipe the output to a grep filter to assert that only expected plugins are active:

Bash
pytest --trace-config 2>&1 | grep -E "conftest|plugin" > /tmp/plugin_audit.log
# CI assertion: Verify no duplicate conftest paths
if grep -q "duplicate" /tmp/plugin_audit.log; then
 echo "❌ Plugin loading collision detected"
 exit 1
fi

Fixture Overhead Profiling

Use pytest --durations=10 to identify the slowest fixtures. Combine with pytest --trace-config to verify conftest loading order. For granular measurement, wrap session-scoped fixtures with timing decorators:

Python
import time
import pytest
from functools import wraps

def profiled_fixture(func):
 @wraps(func)
 def wrapper(*args, **kwargs):
 start = time.perf_counter()
 result = yield from func(*args, **kwargs)
 duration = time.perf_counter() - start
 if duration > 2.0:
 pytest._log.warning(f"Fixture {func.__name__} took {duration:.2f}s")
 return result
 return wrapper

CI Pipeline Validation Scripts

Automate hierarchy checks using pre-commit hooks and CI gates:

  1. Pre-commit: Run the AST collision detector and enforce testpaths configuration.
  2. CI Gate: Execute pytest --collect-only --strict-markers to verify marker consistency.
  3. Profiling Gate: Fail builds if --durations output exceeds baseline thresholds by >15%.

These automated checks ensure that conftest.py best practices python developers rely on remain enforced as the monorepo scales.


Common Pitfalls & Resolution Matrix

PitfallSymptomDiagnosisResolution
Fixture Name ShadowingTests in package B unexpectedly use package A's fixture implementation.pytest --fixtures --verbose shows identical names in sibling conftest.py files.Prefix fixtures with domain identifiers or use @pytest.fixture(name="explicit_name"). Implement CI linter.
conftest.py Inside src/Pytest collects production code; fixtures leak into runtime imports.Check testpaths in pyproject.toml. Run pytest --collect-only to verify tree.Move all conftest.py to tests/. Use norecursedirs = src. Remove __init__.py from test dirs.
pytest_plugins MisconfigurationPlugins fail silently or raise ModuleNotFoundError.Enable pytest --trace-config. Check for string concatenation instead of list assignment.Always use pytest_plugins = ["pkg.module"]. Validate paths with importlib.util.find_spec().
Namespace Package Discovery FailurePytest skips conftest.py in directories lacking __init__.py.Run python -c "import sys; print(sys.path)". Verify pytest's collection root.Add __init__.py to test directories or configure pythonpath in pyproject.toml.

Frequently Asked Questions

Can I safely share fixtures across completely unrelated monorepo packages? Yes, but only through a centralized tests/shared/ directory referenced via pytest_plugins in each package's conftest.py. Avoid placing shared fixtures in the root conftest.py unless they are truly global. Use explicit imports or plugin registration to maintain traceability and prevent implicit scope leakage.

Why does pytest ignore my root conftest.py when running tests from a subdirectory? Pytest only loads conftest.py files that are ancestors of the collected test files. If you run `pytest services/auth/tests