LLM Best Practices

Reasoning model parameters: effort, token caps, and sampling

3 min read

View markdown source

Overview

Set request parameters for reasoning models deliberately rather than copying them from chat-model code. The hidden reasoning shares the output cap with the visible answer, depth is controlled by effort or a thinking budget depending on the model, and sampling parameters are rejected or ignored. This page covers those settings; the prompt-side rules are in reasoning-model-prompting.

Use higher max-tokens, and control depth with effort

Set max_tokens well above a chat-model ceiling and control thinking depth with the provider’s effort setting. Reasoning models emit a hidden reasoning budget before the visible answer. A max_tokens of 1024 may be entirely consumed by reasoning, leaving zero tokens for the visible response.

Practical ceilings as of 2026:

  • Current Claude models (Opus 5.5, Sonnet 5, Fable 5.1, and Opus 4.7 onward) run thinking adaptively: the model decides how much to think per request rather than following a fixed token budget. budget_tokens is rejected outright on these models (deprecated on Opus 4.6 and Sonnet 4.6); control depth with output_config.effort (low through max) instead, and set max_tokens generously (64k+ at xhigh/max effort) since thinking and the visible response draw from the same cap. The default effort differs by model (medium on Opus 5.5, high on most others), so set it explicitly.
  • On Opus 5.5 and Fable 5.1, thinking cannot be disabled at all; lower effort is the only way to spend less on it.
  • Haiku 4.5 and older Claude models (pre-4.6) still take thinking: {type: "enabled", budget_tokens: N}, a fixed budget that must be smaller than max_tokens. Budget 16k to 64k tokens for hard problems, 4k for easy ones, if you are on one of these.
  • OpenAI reasoning models: reasoning tokens count against the output cap (max_output_tokens on the Responses API, max_completion_tokens on Chat Completions). Set it to 16k or higher for non-trivial problems, and tune reasoning.effort rather than the cap.
  • DeepSeek and Gemini reasoning models: similar dynamics; consult the current docs.

Setting too small a budget is the most common reasoning-model failure mode in production. On a fixed-budget model the symptom is a silent mid-thought truncation; on an adaptive model with max_tokens set too low, the symptom is the same, so leave headroom either way.

Skip temperature tuning for most cases

Leave sampling parameters at their defaults on reasoning models. Reasoning models are less sensitive to temperature than chat models because the heavy lifting happens inside the reasoning budget at a fixed sampling regime. Current Claude models (Opus 5.5, Sonnet 5, Fable 5.1, and everything from Opus 4.7 onward) reject temperature, top_p, and top_k outright; steer behavior with prompting and effort instead of sampling parameters.

OpenAI’s reasoning models similarly reject or ignore sampling parameters. Do not waste cycles tuning a parameter the model ignores or rejects.

Pitfalls

  • Setting max_tokens to a chat-model ceiling (1024-4096). The reasoning budget exhausts the cap.
  • Tuning temperature on a model that ignores it.

Backlinks