---
title: "Model Routing and Fallback Ladders"
slug: "model-routing"
category: "ai-agents"
tags: ["ai-agents", "cost", "routing", "model-selection", "fallback", "evaluation"]
status: "stable"
last_updated: 2026-09-27
summary: "Route each LLM request to the smallest model tier that passes the eval, escalate on low confidence, and order providers into a fallback ladder with a quality gate at every rung."
related: ["[[ai-agents/cost-control]]", "[[ai-agents/evaluation]]", "[[ai-agents/ollama]]", "[[ai-agents/multi-agent]]", "[[comparisons/claude-vs-gpt]]", "[[prompt-engineering/prompt-caching-strategies]]"]
---

## Overview

Most production traffic does not need a frontier model, and sending all of it to one is the most common source of avoidable LLM spend. Routing assigns each request to a model tier based on difficulty; a fallback ladder retries a request on a stronger tier when the cheaper one fails a confidence check. Both depend on an eval suite that scores each tier per slice, so read [[ai-agents/evaluation]] first. This page is the routing half of [[ai-agents/cost-control]].

## Route by difficulty, not by default

Pick the smallest model that passes the eval for each request type. Run a router as the first hop.

- Triage with a small fast model (Haiku, GPT-5.6 Luna, Gemini Flash-Lite). It classifies the request.
- Routine work goes to a mid-tier model (Sonnet, GPT-5.6 Terra).
- Hard cases escalate to the frontier (Opus or Fable, GPT-5.6 Sol or GPT-6 Astra, Gemini Pro) on multi-step reasoning or low-confidence triage output.

For the Claude vs GPT vendor decision behind these tiers, see [[comparisons/claude-vs-gpt]].

Score each tier on the golden set. Promote a request only when the cheaper tier's accuracy on that slice is below the bar.

Before building a multi-model cascade, measure the simpler alternative: one capable model at a lower `effort` setting. Prompt caches are scoped per model, so every extra tier forfeits cache reuse, and a cheaper request that needs more retries is not cheaper per completed task.

## Build a fallback ladder

Order providers cheapest to most expensive and try them in series with a quality gate at each step.

```text
1. Local model via Ollama for known-easy patterns (see [[ai-agents/ollama]]).
2. Cheap hosted API (Haiku, Gemini Flash-Lite, GPT-5.6 Luna) for routine tasks.
3. Frontier API (Opus, Gemini Pro, GPT-5.6 Sol) when steps 1 and 2 fail confidence checks.
```

The ladder works only with a confidence signal at each rung. Without one, you pay for both calls.

## Log every routing decision

Record the tier the router picked, whether the request escalated, and why (low triage confidence, a failed quality gate, a provider error) on every call. Without that log, a router that quietly sends most traffic to the frontier tier looks identical to one that works. Aggregate escalation rate per request type next to cost per task in the dashboards described in [[ai-agents/cost-control]]; a rising escalation rate is the first sign that a cheaper tier has stopped passing its slice.

## Related

- [[ai-agents/cost-control]]
- [[ai-agents/evaluation]]
- [[ai-agents/ollama]]
- [[ai-agents/multi-agent]]
- [[comparisons/claude-vs-gpt]]
- [[prompt-engineering/prompt-caching-strategies]]