LLM Best Practices

Python: Testing

4 min read

View markdown source

Overview

pytest is the only test runner to use for Python. This page covers fixture design, parametrization, property-based testing with Hypothesis, coverage reporting, and mocking strategy. For project setup (where to put tests, how to invoke pytest), see python. For language-agnostic testing principles, see testing.

Scope fixtures to the narrowest lifetime

A pytest fixture has one of four scopes: function, class, module, or session. Use the narrowest scope that keeps tests fast and isolated. A database connection fixture that opens and closes per test is slow; a session-scoped connection that all tests share risks state leakage.

import pytest
from myapp.db import Database
 
@pytest.fixture(scope="session")
def db() -> Database:
    """One connection for the full test run."""
    conn = Database.connect(":memory:")
    conn.migrate()
    yield conn
    conn.close()
 
@pytest.fixture
def clean_db(db: Database):
    """Per-test rollback so tests do not bleed state."""
    with db.transaction() as tx:
        yield db
        tx.rollback()

Session-scoped fixtures provide the connection; function-scoped fixtures wrap it in a transaction and roll back.

Use parametrize instead of loops

@pytest.mark.parametrize runs one test ID per case, gives each case an independent pass/fail, and names cases in the output. A for loop inside a test produces one test ID that passes or fails entirely.

import pytest
from myapp.math import factorial
 
@pytest.mark.parametrize("n, expected", [
    (0, 1),
    (1, 1),
    (5, 120),
    (10, 3628800),
])
def test_factorial(n: int, expected: int) -> None:
    assert factorial(n) == expected

Add ids=["zero", "one", "five", "ten"] when the default representation is hard to read in CI output.

Write property tests with Hypothesis

Hypothesis generates hundreds of inputs for a property assertion. It shrinks failures to the minimal reproducing case and stores them in the hypothesis database.

from hypothesis import given, strategies as st
from myapp.math import factorial
 
@given(st.integers(min_value=0, max_value=20))
def test_factorial_non_negative(n: int) -> None:
    assert factorial(n) >= 1
 
@given(st.lists(st.integers()))
def test_sort_is_idempotent(xs: list[int]) -> None:
    assert sorted(sorted(xs)) == sorted(xs)

Use @settings(max_examples=500) to increase coverage on critical paths. Hypothesis is most valuable for encoding, parsing, and mathematical invariants; it finds edge cases that parametrize misses.

Prefer monkeypatch over MagicMock for simple replacements

monkeypatch replaces attributes, environment variables, and imports cleanly, and reverts them automatically after the test. MagicMock is more capable but also more verbose and easier to misconfigure.

def test_env_based_config(monkeypatch: pytest.MonkeyPatch) -> None:
    monkeypatch.setenv("DB_URL", "sqlite:///:memory:")
    cfg = load_config()
    assert cfg.db_url == "sqlite:///:memory:"
 
def test_http_call(monkeypatch: pytest.MonkeyPatch) -> None:
    monkeypatch.setattr("myapp.client.httpx.get", lambda url, **kw: FakeResponse(200))
    result = fetch_status("https://example.com")
    assert result == 200

Use unittest.mock.MagicMock for complex call-assertion scenarios (verify argument order, call count across multiple calls). For everything else, monkeypatch is enough.

Measure coverage with pytest-cov; enforce a floor

Add pytest-cov and set a minimum coverage threshold in CI. Below-floor builds fail, preventing regressions.

[tool.pytest.ini_options]
addopts = "--cov=src --cov-report=term-missing --cov-fail-under=85"
testpaths = ["tests"]

Coverage is a floor, not a goal. 100% line coverage on code that never fails does not mean the code is tested. Pair coverage with property tests and integration tests to cover the paths that line coverage misses.

Use autouse fixtures for global test preconditions

autouse=True applies a fixture to every test in its scope without explicit declaration. Use it for environment setup that all tests require.

@pytest.fixture(autouse=True)
def reset_env(monkeypatch: pytest.MonkeyPatch) -> None:
    monkeypatch.setenv("ENV", "test")
    monkeypatch.delenv("PROD_API_KEY", raising=False)

Limit autouse to genuinely universal preconditions. Overusing it hides test dependencies and makes failures hard to trace.

Follow the test pyramid

Unit tests cover individual functions with no I/O. Integration tests cover module boundaries (database, HTTP). End-to-end tests cover the deployed system. Keep the pyramid wide at the bottom.

  • Unit: fast, no network, no filesystem, monkeypatch for dependencies.
  • Integration: real database (in-memory or test container), real HTTP calls to a test server.
  • End-to-end: minimal count; run against a staging environment in CI.

See testing for the full pyramid and the cases each layer must cover (golden path, empty, boundary, error).

Graph View

Backlinks