Pytest & CI

Pytest Markers for Conditional Test Execution

A test guarded with @pytest.mark.skipif(os.environ["CI"] == "true", ...) can crash the entire collection with KeyError when the variable is absent, and an undeclared marker silently emits PytestUnknownMarkWarning that masks typos until a test runs everywhere it should have been skipped. Both stem from the same fact: pytest evaluates marker conditions during the collection phase, at module import time, long before any fixture or test body runs. This guide shows how to write import-safe skipif/xfail conditions, register markers to enforce hygiene, and inject markers from CI without editing test files.

Prerequisites

  • pytest >= 8.0, Python 3.9+.
  • Markers declared and strict mode enabled in pyproject.toml:
TOML
# pyproject.toml
[tool.pytest.ini_options]
addopts = "--strict-markers"
markers = [
  "skip_platform: skip tests based on OS constraints",
  "requires_db: skip tests when the database is unavailable",
  "xfail_flaky: known intermittent failure",
]

Marker registration is part of the broader configuration discipline in Pytest Configuration Best Practices; how conditions are parsed during collection ties back to Advanced Pytest Architecture & Configuration.

Solution

Build conditions from deterministic, import-safe values and read environment variables with .get() so a missing key degrades gracefully instead of raising during collection:

Python
import os
import sys
import platform
import pytest

# Evaluated once at import time from constants — never a network/DB call.
WINDOWS_ONLY = pytest.mark.skipif(
    sys.platform != "win32",
    reason="Requires Windows-specific registry APIs",
)

LINUX_PY310 = pytest.mark.skipif(
    sys.platform != "linux" or sys.version_info < (3, 10),
    reason="Needs Linux kernel features and Python 3.10+ match statement",
)

# os.environ["TEST_DB_URL"] would KeyError at collection if unset;
# .get() returns None, so the condition is simply truthy/falsy.
SKIP_IF_NO_DB = pytest.mark.skipif(
    not os.environ.get("TEST_DB_URL"),
    reason="TEST_DB_URL not configured; skipping integration tests",
)

@WINDOWS_ONLY
def test_windows_registry_access():
    assert platform.system() == "Windows"

@SKIP_IF_NO_DB
def test_db_round_trip():
    ...

To gate a single parametrized case rather than the whole function, attach the marker to that case — function-level markers evaluate before parameters expand:

Python
@pytest.mark.parametrize("payload", [
    {"v": 1},
    pytest.param({"v": 2}, marks=pytest.mark.xfail(reason="schema v2 not shipped")),
])
def test_payload(payload):
    assert validate(payload)

For environment-aware filtering without touching test files, inject markers in pytest_collection_modifyitems, which runs after collection but before fixture setup:

Python
# conftest.py
import os
import pytest

def pytest_collection_modifyitems(config, items):
    """Inject skip markers from CI env vars — keeps tests clean."""
    fast_mode = os.environ.get("CI_FAST_MODE", "false").lower() == "true"
    skip_slow = pytest.mark.skip(reason="slow tests skipped in fast CI stage")
    for item in items:
        if fast_mode and "slow" in item.keywords:
            item.add_marker(skip_slow)

Confirm the resulting marker stack with pytest --collect-only -v before trusting it.

Why this works

Marker conditions are plain Python expressions evaluated when the module is imported during collection, so anything they reference must already be resolvable — constants like sys.platform and sys.version_info always are, while os.environ["..."] is not. Using .get() converts a fatal KeyError into a benign falsy value. Deferring environment-driven decisions to pytest_collection_modifyitems moves them to a point where os.environ is safely readable and the decision is centralized, which keeps the same condition deterministic across every pytest-xdist worker.

Edge cases and failure modes

  • CI string casing. CI systems inject TRUE/FALSE as strings, both truthy in Python. Always normalize with .lower() == "true".
  • xfail(strict=True) on flaky tests. A strict xfail reports FAILED when the test unexpectedly passes — correct for tracking known failures, but a trap for timing-flaky tests. Pair it with deterministic failure conditions, or use pytest-rerunfailures for genuine flakiness instead.
  • Referencing fixtures in conditions. Fixtures instantiate during setup, after collection, so skipif/xfail cannot read fixture values — that raises NameError. Call pytest.skip() or pytest.xfail() inside the test body when the decision needs fixture state.
  • Expensive condition functions. A condition calling subprocess.run, socket.gethostbyname, or an ORM probe runs once per module at collection and inflates pytest --collect-only --durations=0. Precompute booleans at import (_IS_CI = os.environ.get("CI", "").lower() == "true").
  • Worker-divergent conditions under parallelism. Conditions using os.getpid() or mutable module globals evaluate differently per worker, causing inconsistent skips. Use only CI-provided environment variables or pre-filtered -k lists.

Frequently Asked Questions

Why does pytest.mark.skipif not evaluate correctly with parametrized tests? Marker conditions evaluate during collection, before parameter expansion, so a marker on the function sees the unexpanded test. To gate a specific parameter set, attach the marker to that case with pytest.param(value, marks=pytest.mark.skipif(condition, reason="...")).

How can I dynamically skip tests by CI environment variable without editing test files? Implement pytest_collection_modifyitems in the root conftest.py, read os.environ inside the hook, build pytest.mark.skipif objects, and attach them with item.add_marker(). This keeps test sources clean and centralizes CI logic.

What causes PytestUnknownMarkWarning and how do I suppress it safely? Pytest warns when a marker is not declared in configuration. Register markers under [tool.pytest.ini_options] markers in pyproject.toml and add --strict-markers so typos become errors, enabling IDE autocompletion and consistent resolution.

← Back to Pytest Configuration Best Practices