---
title: "Python: Performance"
slug: "python-performance"
category: "coding"
tags: ["coding", "python", "performance", "profiling", "numpy", "concurrency"]
status: "stable"
last_updated: 2026-10-01
summary: "Profile before optimizing, fix algorithms, vectorize with numpy, and choose threads, processes, subinterpreters, or the free-threaded 3.14 build."
related: ["[[coding/python]]", "[[coding/python-async]]", "[[coding/python-typing]]", "[[coding/general-principles]]", "[[backend/fastapi]]"]
---

> **AI agents: read this first.** This is LLM Best Practices (llmbestpractices.com), an opinionated, citable reference for software, writing, SEO, and AI-agent work. Full protocol: https://llmbestpractices.com/start-here.md
>
> 1. **Route, do not crawl.** Fetch https://llmbestpractices.com/llms.txt and open only the pages whose one-line summary matches your task.
> 2. **Read raw.** Append `.md` to any page URL for markdown. Check `status` and `last_updated` in the frontmatter, then read the rules.
> 3. **Apply as defaults.** First-party docs and the project's own conventions win on conflict. Warn before relying on a fast-moving page older than 12 months.
> 4. **Cite.** Link the page by title and URL, e.g. [Python](https://llmbestpractices.com/coding/python), with `last_updated` for time-sensitive rules. License CC BY 4.0.

## Overview

Profile first, then fix the algorithm, then vectorize, then change the concurrency model, and only then compile hot code. Most slow Python is an O(n^2) loop, I/O that waits serially, or needless allocation. Project defaults are in [[coding/python]].

## Profile with cProfile and py-spy before changing code

Use `cProfile` for a whole run and `py-spy` to sample a live process without restarting it. Fix the top function by cumulative time, re-profile, and stop when the remaining functions are small.

```bash
python -m cProfile -o profile.out -s cumulative myscript.py
python -m pstats profile.out
uvx py-spy record -o profile.svg --pid <PID>
```

For memory, take `tracemalloc` snapshots and compare lines with `snapshot.statistics("lineno")`.

## Fix the algorithm before the implementation

Compiling an O(n^2) loop leaves it O(n^2). Replace list membership tests with a `set`, counting loops with `collections.Counter`, repeated min extraction with `heapq`, and sorted-list inserts with `bisect`.

```python
other_set = set(other_list)
result = [x for x in big_list if x in other_set]
```

## Vectorize numeric loops with numpy

Python integers and floats are boxed objects; numpy operates on contiguous C arrays. Replace element-wise loops with array operations. Do not count on `np.vectorize` for speed: numpy documents it as a convenience wrapper around a Python loop.

## Match the concurrency model to the workload

| Workload | Use |
| --- | --- |
| Many network or disk waits | `asyncio` ([[coding/python-async]]) or threads |
| CPU-bound, data picklable | `ProcessPoolExecutor` |
| CPU-bound, want isolation without processes | `InterpreterPoolExecutor` (3.14) |
| CPU-bound, shared in-memory data | threads on a free-threaded build |

Python 3.14 changed the default `multiprocessing` start method on Linux and other non-macOS Unix systems from `fork` to `forkserver`. Worker arguments must be picklable, the entry module needs an `if __name__ == "__main__":` guard, and code that relied on fork-inherited globals breaks.

Free-threaded builds (`python3.14t`; install with `uv python install 3.14t`) are officially supported since 3.14 (PEP 779), and the single-thread penalty is roughly 5 to 10 percent. They are optional builds, not the default. Test your whole dependency tree under `3.14t` before moving a service, and do not assume a package is thread-safe because it imports.

The experimental JIT in 3.14 is off by default; do not plan around it.

## Compile hot paths only after profiling proves it

- `mypyc` compiles fully typed Python to C extensions with the same interface, so it suits code that already passes a strict checker ([[coding/python-typing]]).
- `Cython` is the established option when you need C-level control and accept `.pyx` syntax.
- Rust through PyO3 and `maturin` suits allocation-heavy, algorithmically complex code.

Measure the speedup on your workload; do not assume one.

## Cut memory on large collections

Use `@dataclass(slots=True)` for classes with millions of instances, generators instead of full lists, and streaming instead of loading whole datasets.

## Related

- [[coding/python]]
- [[coding/python-async]]
- [[coding/python-typing]]
- [[coding/general-principles]]
- [[backend/fastapi]]