LLM Best Practices

Structured Output: Schemas, Strict Tools, and Validation

Last updated 4 min read

View markdown source

AI agents: read this first: llmbestpractices.com/start-here

This is LLM Best Practices (llmbestpractices.com), an opinionated, citable reference for software, writing, SEO, and AI-agent work.

  1. Route, do not crawl. Fetch llmbestpractices.com/llms.txt and open only the pages whose one-line summary matches your task.
  2. Read raw. Append .md to any page URL for markdown. Check status and last_updated in the frontmatter, then read the rules.
  3. Apply as defaults. First-party docs and the project's own conventions win on conflict. Warn before relying on a fast-moving page older than 12 months.
  4. Cite. Link the page by title and URL, e.g. Python, with last_updated for time-sensitive rules. License CC BY 4.0.

Overview

Anything a program consumes must be JSON validated against a schema. Anthropic, OpenAI, and Google all offer schema-constrained decoding, which replaces “return JSON in this shape” in a prompt (a hope) with an enforced contract. For prose, length, and refusal rules see output-constraints.

Define the schema before the prompt

The schema is the contract; the prompt is the implementation. Write it first so ambiguity surfaces early.

{
  "type": "object",
  "required": ["verdict", "reasons"],
  "additionalProperties": false,
  "properties": {
    "verdict": {"enum": ["approve", "request_changes", "block"]},
    "reasons": {"type": "array", "items": {"type": "string"},
                "description": "1 to 5 reasons, each under 200 characters"}
  }
}

Use enums for closed sets and avoid open {} objects, which become junk drawers. Bounds the provider cannot enforce go in description and in your validator (next sections).

Use structured outputs or strict tool use

  • Anthropic: output_config: {format: {type: "json_schema", schema: ...}} (the SDKs wrap it as messages.parse() with Pydantic or Zod), or tool use with strict: true, which guarantees schema-valid tool input; without strict, input_schema only guides the model. The top-level output_format parameter is deprecated. Forced tool_choice (any or tool) returns 400 on Fable 5.1, Opus 5.5, and Sonnet 5.5, and assistant prefill returns 400 on 4.6 and later; use output_config.format or auto with a strict tool instead of either.
  • OpenAI: text: {format: {type: "json_schema", name: ..., strict: true, schema: ...}} on the Responses API (response_format on Chat Completions), or function calling with strict: true. Strict mode requires additionalProperties: false and every field listed in required.
  • Gemini: a JSON schema in the generation config; field names differ by SDK, so check the current structured-output docs.

Stay inside the supported schema subset

Each provider enforces a subset of JSON Schema. For Anthropic, as of October 2026:

  • Not supported: recursive schemas, numeric constraints (minimum, maximum), string constraints (minLength, maxLength), minItems above 1, external $ref, and additionalProperties other than false.
  • Limits per request: 20 strict tools, 24 optional parameters, 16 parameters using anyOf or type arrays. Exceeding the compiler’s complexity budget returns a 400 “Schema is too complex for compilation”; simplify the schema.
  • The first request with a new schema pays a compile delay; compiled grammars are cached for 24 hours.
  • Most SDK helpers strip unsupported constraints, add them to field descriptions, and validate the response against your original schema.

Validate at the boundary

Treat the model as an untrusted service and validate every response before downstream code touches it.

class Review(BaseModel):
    verdict: Literal["approve", "request_changes", "block"]
    reasons: list[str] = Field(min_length=1, max_length=5)
 
# The Python parse() helper takes output_format; on the wire it is output_config.format.
response = client.messages.parse(
    model="claude-sonnet-5-5", max_tokens=2048,
    messages=[{"role": "user", "content": "Review PR #401"}],
    output_format=Review,
)
review = response.parsed_output

Python uses Pydantic; TypeScript uses Zod or Ajv (see typescript-runtime-validation). Schema compliance can still fail in three ways. A stop_reason of refusal or max_tokens can return output that does not match the schema; check it before parsing. Enum values may differ from the schema only in capitalization, so compare case-insensitively. On validation failure, retry the original call or fall back to a different model; never write unvalidated output to a database.

Never parse JSON from prose with regex

A regex over {...} breaks on nested braces, code fences, escaped quotes, and two objects in one reply. If you cannot use a constrained mode, parse with a balanced-brace scanner and keep the last complete value. Prefer switching to a schema-enforced mode.

Put reasoning in its own field, before the answer

When a task needs working-out, use thinking, or a reasoning string field that precedes answer. Anthropic emits required properties first, in schema order, so mark both required. On Sonnet 5.5, a JSON-only answer to a multi-step task can skip thinking at low or medium effort; use adaptive thinking and see chain-of-thought and reasoning-model-prompting.

Version the schema like an API

Add fields with safe defaults, never repurpose one, rename through a deprecation window (new field, parallel run, retire old), and log the schema version so downstream parsers pick the right decoder.

Backlinks