Python Testing & Debugging Beyond unittestAdvanced Python testing playbooks
Pytest & CI

Packaging a pytest Plugin with Entry Points

A plugin that lives in conftest.py works only in the directory tree that contains it. The moment a second repository needs the same markers, fixtures or reporting hooks, the copy-paste starts — and the two copies diverge within a quarter. Packaging turns the plugin into a dependency: one version, installed where it is needed, discovered automatically by every pytest run in that environment.

Prerequisites

  • pytest 7.0+, pip 23+ and a PEP 621 pyproject.toml; the entry-point group name has been pytest11 since pytest 2.
  • A plugin module whose hooks already work from a conftest.py, as built in building custom pytest plugins.

Solution

The whole mechanism is one metadata table. Move the hooks into an importable module, then declare it under project.entry-points.pytest11:

Plain text
myplugin/
├── pyproject.toml
└── src/
    └── myplugin/
        ├── __init__.py
        └── hooks.py          # the hook implementations
TOML
# pyproject.toml
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "pytest-slowguard"
version = "0.1.0"
description = "Fails the run when a test exceeds its declared budget"
requires-python = ">=3.9"
dependencies = ["pytest>=7.0"]

# The entry point: <plugin name> = "<importable module>"
[project.entry-points.pytest11]
slowguard = "myplugin.hooks"

The left-hand name is what pytest calls the plugin — it is the name in -p no:slowguard and in --trace-config output. The right-hand value is an importable module, not a file path; pytest imports it and registers every hook implementation it finds at module level.

Python
# src/myplugin/hooks.py
import pytest

def pytest_addoption(parser):
    parser.addini("slow_budget", "seconds a test may take", default="2.0")

@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    report = outcome.get_result()
    budget = float(item.config.getini("slow_budget"))
    if report.when == "call" and report.duration > budget:
        report.outcome = "failed"
        report.longrepr = f"took {report.duration:.2f}s, budget {budget:.2f}s"

Install it into the target environment and it is live for every run in that environment, with no configuration in the consuming project:

Bash
$ pip install -e .                 # while developing the plugin
$ pip install pytest-slowguard     # in the consuming project
$ pytest -q                        # the plugin is already registered
How pytest finds an installed plugin A left-to-right startup sequence: pytest scans installed distributions for the pytest11 entry point group, imports each named module, registers the hook implementations it finds, and only then loads conftest files and begins collection. How pytest finds an installed plugin scan entry points group pytest11 import the module by dotted path register hooks module-level impls load conftests after plugins -p no:<name> removes a plugin at the first stage, before its module is imported.
Installed plugins register before any conftest is read, which is why a plugin fixture can be overridden by a local conftest but not the other way round.

Why this works

Python packaging writes entry-point metadata into the installed distribution, and importlib.metadata can enumerate every entry in a given group across the whole environment. At startup pytest queries the pytest11 group, imports each module, and hands it to pluggy's registry — the same registry conftest.py files are added to later. From pluggy's point of view there is no difference between a hook that arrived from an installed package and one that arrived from a conftest; only the registration order differs, and that order is what determines which fixture definition wins when names collide.

Verifying registration in a clean environment

The failure mode that wastes the most time is a plugin that works in the development checkout and silently does nothing after installation, usually because the module moved or the wheel excluded it. Three checks catch it before release.

Bash
$ python -c "from importlib.metadata import entry_points; \
             print([e for e in entry_points(group='pytest11')])"
[EntryPoint(name='slowguard', value='myplugin.hooks', group='pytest11')]

$ pytest --trace-config -q 2>&1 | grep slowguard
  plugin name: slowguard, class: <module 'myplugin.hooks' from '...'>

$ pytest -p no:slowguard -q        # confirms the name is what you think it is

The first proves the metadata exists, the second proves pytest imported the module, and the third proves the plugin name is the one documented for users disabling it. Run all three against a wheel installed into a fresh virtualenv, not against the editable install — an editable install resolves modules from the source tree and will happily import a file the wheel does not contain.

Add the same three checks to the plugin's own CI as a job that builds the wheel, installs it into a clean environment, and runs a smoke test. Packaging errors are exactly the class of bug that unit tests cannot see, because the tests import the module directly rather than through the entry point.

Three ways a plugin can be loaded A table comparing an installed entry-point plugin, a conftest.py, and the -p command line flag, across when each registers, how widely it applies, and the usual reason to choose it. Three ways a plugin can be loaded Criterion Registers Scope pytest11 entry point at startup, first the environment conftest.py at collection a directory tree -p module / addopts at startup one invocation pytest_plugins in rootdir conftest at collection the project
Registration order decides precedence: installed plugins are outermost, and a local conftest can override the fixtures they provide.

Choosing a name and a support window

Two decisions outlive the code. The distribution name should start with pytest-, which is the ecosystem convention and how people find plugins on PyPI; the entry-point name should be short and namespaced enough that -p no:<name> is unambiguous in a project with thirty plugins installed.

The support window is a requires-python and a pytest version range, and the range matters more than most authors expect. Hook signatures do change across major pytest versions — hookwrapper=True gained a modern wrapper=True alternative in pytest 8, and several hooks changed arguments in 7 — so pinning pytest>=7.0,<9 states what you have actually tested rather than implying support you have not. Run the plugin's test suite against every supported pytest major in CI; a matrix of two or three versions costs minutes and prevents the most common bug report a plugin receives.

For internal plugins that will never see PyPI, the same mechanics apply with a private index or a direct VCS dependency. The value is unchanged: one implementation, one version to upgrade, and a dependency graph that tells you which repositories are affected when the hook behaviour changes.

Declaring options and defaults the packaged way

A plugin that only registers hooks is easy to package. One that needs configuration has to decide where that configuration lives, and the answer determines how the plugin behaves in a repository it does not control.

pytest_addoption is the single entry point for both command-line flags and ini options, and the difference between them matters. A flag suits a per-run decision — --slowguard-off for a debugging session. An ini option suits a project-level policy — the budget every test in this repository must meet. Declaring both, with the flag overriding the ini value, is the shape users expect:

Python
# src/myplugin/hooks.py
def pytest_addoption(parser):
    group = parser.getgroup("slowguard")
    group.addoption("--slow-budget", type=float, default=None,
                    help="override the per-test budget for this run")
    parser.addini("slow_budget", "seconds a test may take", default="2.0")

def resolve_budget(config) -> float:
    # Command line wins; otherwise the project's ini value; otherwise the default.
    return config.getoption("--slow-budget") or float(config.getini("slow_budget"))

Grouping the options with parser.getgroup keeps pytest --help readable once several plugins are installed — ungrouped options land in a flat list where nobody can tell which plugin owns what.

Two packaging details follow from having options. Document the ini keys in the README with a copy-pasteable pyproject.toml block, because --help shows flags prominently and ini options only in a separate section that users rarely read. And never read configuration at import time: pytest_addoption runs before the ini file is parsed, so any module-level getini call fails. Resolve configuration inside a hook or a fixture, where the fully parsed config object is available.

Finally, decide what happens with no configuration at all. A plugin that fails the run because a required ini key is missing will be uninstalled within a day; one that applies a sensible default and reports what it used on the first run is the one that survives. State the effective configuration once per session in pytest_report_header, which puts it at the top of every CI log.

Where plugin configuration should live A table comparing a command-line flag, an ini option, an environment variable and a hard-coded default across who sets each, how long the setting lasts, and what it suits. Where plugin configuration should live Criterion Set by Lasts --flag the person running one invocation ini option the project every run in the repo environment variable CI configuration the environment code default the plugin author until overridden
Offer all four in that precedence order, and report the effective value in the run header so nobody has to guess which one applied.

Edge cases and failure modes

  • The entry point value must be a module, not a package with the hooks in __init__ only by accident. Point it at the module that defines the hooks; pointing at the package works only if the hooks are re-exported there.
  • Editable installs mask missing files. Always smoke-test a built wheel in a fresh virtualenv, or a src-layout mistake ships to users.
  • Plugins load before conftest files, so a fixture defined in both places resolves to the conftest one. That is usually desirable, but it means a plugin cannot force a fixture on a project that overrides it.
  • -p no:<name> uses the entry-point name. Users who try the module path get "plugin not found", which is worth documenting in the README.
  • Two plugins implementing the same hook both run. Ordering follows tryfirst/trylast and registration order, exactly as described in writing a hookwrapper for test reports.

Frequently Asked Questions

What is the pytest11 entry point group? It is the group name pytest scans at startup to discover installed plugins. Every entry in the pytest11 group maps a plugin name to an importable module, and pytest imports and registers each one automatically.

Why is my plugin not loaded after pip install? Usually because the entry point points at a module path that does not exist in the installed wheel, or because the package was installed without the metadata. Check with pytest --trace-config, which lists every registered plugin and its module.

How do I stop my plugin from loading in one project? Run pytest with -p no:pluginname, or set the same value in addopts. The name is the entry point name, not the module path.

← Back to Building Custom Pytest Plugins