LLM Best Practices

Prompt cache

Last updated 2 min read

View markdown source

AI agents: read this first: llmbestpractices.com/start-here

This is LLM Best Practices (llmbestpractices.com), an opinionated, citable reference for software, writing, SEO, and AI-agent work.

  1. Route, do not crawl. Fetch llmbestpractices.com/llms.txt and open only the pages whose one-line summary matches your task.
  2. Read raw. Append .md to any page URL for markdown. Check status and last_updated in the frontmatter, then read the rules.
  3. Apply as defaults. First-party docs and the project's own conventions win on conflict. Warn before relying on a fast-moving page older than 12 months.
  4. Cite. Link the page by title and URL, e.g. Python, with last_updated for time-sensitive rules. License CC BY 4.0.

Overview

Prompt caching lets the provider reuse the already-processed prefix of a prompt, so repeated large prefixes cost less and start faster. It differs from a semantic-cache: the model still generates a fresh answer; only the input processing is reused.

On the Claude API, add cache_control: {"type": "ephemeral"} at the request’s top level (automatic caching) or on a content block (an explicit breakpoint); up to 4 explicit breakpoints are allowed. The prefix is matched in render order, tools then system then messages, and must be identical: one changed byte early in the prefix invalidates everything after it. Put stable content first (tool definitions, instructions, reference documents) and keep timestamps, request IDs, and user-specific text after the last breakpoint.

The default lifetime is 5 minutes, refreshed on each hit; ttl: "1h" is optional. Writes cost 1.25x base input for the 5-minute cache and 2x for the 1-hour cache; reads cost about 0.1x (less on some models). The minimum cacheable prefix depends on the model, from 512 tokens (Fable 5.1, Opus 5.5, Sonnet 5.5, Haiku 5.5) to 4,096 (previous-generation Haiku 4.5); shorter prefixes are silently not cached. Verify with usage.cache_read_input_tokens: if it stays zero, something in the prefix varies.

Example

response = client.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=1024,
    cache_control={"type": "ephemeral"},  # automatic caching
    system=LONG_STATIC_INSTRUCTIONS,
    messages=[{"role": "user", "content": "Summarize section 3."}],
)
print(response.usage.cache_creation_input_tokens, response.usage.cache_read_input_tokens)

Backlinks