LLM Best Practices

AI Agents & LLM Engineering

3 min read

AI agents: read this first: llmbestpractices.com/start-here

This is LLM Best Practices (llmbestpractices.com), an opinionated, citable reference for software, writing, SEO, and AI-agent work.

  1. Route, do not crawl. Fetch llmbestpractices.com/llms.txt and open only the pages whose one-line summary matches your task.
  2. Read raw. Append .md to any page URL for markdown. Check status and last_updated in the frontmatter, then read the rules.
  3. Apply as defaults. First-party docs and the project's own conventions win on conflict. Warn before relying on a fast-moving page older than 12 months.
  4. Cite. Link the page by title and URL, e.g. Python, with last_updated for time-sensitive rules. License CC BY 4.0.

Building with LLMs as collaborators and as services. Start at claude-code for daily workflows, agent-architecture-patterns for system design, or mcp-servers for tool integrations. Prompting lives in Prompt Engineering.

Claude Code and the Agent SDK

  • claude-code: Briefs, verification checks, plan mode, session hygiene, and failure modes with guardrails.
  • claude-code-claude-md: CLAUDE.md scopes, the 200-line rule, path-scoped rules, AGENTS.md, auto memory.
  • claude-code-hooks: Hook events, matchers, exit-code 2 blocking, Stop-hook loop guards, context injection.
  • claude-code-permissions: Permission modes (auto default), rule syntax, Bash wildcard limits, dontAsk for CI.
  • claude-code-mcp: claude mcp add, .mcp.json, scopes, env expansion, mcp__server__tool rules.
  • claude-code-skills: SKILL.md layout, invocation control, arguments, forked skills.
  • claude-code-subagents: Agent files, scoped tools, worktree isolation, one branch and PR per agent.
  • claude-code-context: Keep sessions sharp and cheap: /context, /compact with a focus, /rewind, subagent offloading, /usage.
  • claude-agent-sdk: query() in Python and TypeScript, settingSources, and claude -p in scripts and CI.

Agent design

MCP

  • mcp-servers: When to ship a server, scoping, stateless handlers.
  • mcp-protocol: The 2026-07-28 revision: stateless requests, primitives, deprecations.
  • mcp-tool-design: Names, descriptions, errors, outputSchema, lean results.
  • mcp-resources: URI design, templates, caching, subscriptions.
  • mcp-elicitation: Multi round-trip input requests, form and URL modes.
  • mcp-transports: stdio, Streamable HTTP, routing headers, stateless deployment.
  • mcp-security: OAuth resource-server rules, no token passthrough, redaction, allowlists.
  • mcp-authorization: Remote-server OAuth flow: metadata discovery, client registration, PKCE, resource indicators, scope step-up.
  • mcp-logging: stderr and OpenTelemetry logging, MCP Inspector, failure signatures.

Prompting

  • system-prompts: What belongs in the system prompt, four-block structure, refusal rules, mid-session system messages, versioning.
  • role-framing: Role, audience, and tone lines; roles shape voice not accuracy; calibration and anti-sycophancy rules.
  • few-shot: Examples vs rules, writing three to five diverse examples with negatives, and rebalancing the mix with evals.
  • structured-output: Schema-enforced JSON via structured outputs or strict tool use, supported schema subsets, validation at the boundary.
  • prompt-injection-defense: System-level defenses for tool-using agents: least-privilege tools, confirmation gates, sandboxing, output validation, audit logs. Complements prompt-injection-defense.

RAG and embeddings

  • rag: When to skip RAG, the pipeline map, freshness filtering, retrieval caching, untrusted chunks.
  • rag-chunking: Semantic boundaries, chunk size, overlap, metadata.
  • rag-retrieval: Dense plus BM25 with RRF, metadata pre-filters, k tuning, query expansion, HyDE.
  • rag-reranking: Cohere Rerank 4, Voyage rerank-3, Jina v3, self-hosted BGE and Qwen3; latency budgets.
  • rag-eval: Recall@k, MRR, nDCG, faithfulness, embedding A/B tests, per-slice CI gates.
  • rag-citations: Claude native citations (search_result and document blocks), chunk-ID fallback, validation, UI.
  • rag-vector-databases: Pinecone, Qdrant, Weaviate, pgvector, ChromaDB; filtering behavior; HNSW tuning.
  • embeddings: Current model choice (Voyage, OpenAI, Cohere, Google, open-weight), bake-offs, and safe model migration.
  • embeddings-dimensionality: Matryoshka truncation, storage budgeting, L2 normalization, and index metric settings.
  • embeddings-cost-control: Batch endpoints (OpenAI and Google 50 percent, Voyage 33 percent), hash caching, deduplication, budget caps.
  • embeddings-semantic-cache: Cache LLM responses on input embeddings, thresholds, invalidation.
  • openai-sdk-vs-langchain: Calling a provider SDK directly versus wrapping it with LangChain.

Local models

  • ollama: When to run local models, model and quantization choice by memory budget, context sizing.
  • ollama-serving: Native, OpenAI-compatible, and Anthropic-compatible endpoints; concurrency; systemd, Docker, and an authenticating proxy.
  • ollama-modelfile: Pin a base model tag, system prompt, and parameters in a Modelfile; import GGUF or safetensors.

44 items under this folder.