Overview
Reliable agents come from narrowing scope, constraining actions, and instrumenting every step, not from a smarter model: a demo that works once on a happy path fails on the long tail of real traffic. This page is the production checklist behind agent-architecture-patterns; operations live in llmops-best-practices and llm-observability.
Scope the task as narrowly as it goes
Reliability falls as autonomy rises. Give the agent the smallest job with clear success criteria, and use a fixed workflow when you can name the steps.
Constrain tools and treat their output as untrusted
Expose the minimum tool set, validate every argument against a schema and then business rules, require confirmation for destructive actions, and sandbox filesystem and network access. Tool results and retrieved text can carry instructions; see prompt-injection-defense and tool-use-and-function-calling.
Bound every loop
Set a hard ceiling on steps, tool calls, wall-clock time, and tokens. A stuck agent should halt and escalate. Detect repeats by comparing the last few tool calls and arguments, and force state through a schema so the controller can see where the agent is.
Make retries and resumption safe
- Retry transient failures (rate limits, overload, timeouts) with exponential backoff and jitter; do not retry invalid requests.
- Give every side-effecting tool an idempotency key so a retry cannot create a second ticket or payment.
- Checkpoint state after each step so a crash resumes from the last good point instead of restarting. Anthropic’s research system resumes from where an error occurred for this reason.
- For long-running agents, shift traffic to a new version gradually while the old one keeps running (rainbow deployments), so a deploy does not kill in-flight runs.
Evaluate before every release
Build a golden set of representative tasks, run it on every prompt or model change, and gate the release on per-slice thresholds. Judge open-ended outputs with an LLM grader calibrated against human labels. See evaluation.
Observe every step
Log each step with a trace ID: prompt, tool calls and results, token counts, latency, and stop reason. Alert on loop-cap hits, tool error rates, and cost spikes. Without per-step traces a production failure cannot be reproduced.
Degrade gracefully and cap cost
Plan for the model being slow, wrong, or down: timeouts, retries, and a fallback ladder from a strong model to a cheaper one to a deterministic default (see model-routing). Cap spend per request and per user; see cost-control.
Verify before launch
- The golden set clears the release threshold.
- Adversarial and malformed inputs are refused or escalated.
- A forced tool failure recovers or halts cleanly.
- Traces, cost caps, and alerts fire in a staging run.