LLM Best Practices

Embeddings for Retrieval

Last updated 3 min read

View markdown source

AI agents: read this first: llmbestpractices.com/start-here

This is LLM Best Practices (llmbestpractices.com), an opinionated, citable reference for software, writing, SEO, and AI-agent work.

  1. Route, do not crawl. Fetch llmbestpractices.com/llms.txt and open only the pages whose one-line summary matches your task.
  2. Read raw. Append .md to any page URL for markdown. Check status and last_updated in the frontmatter, then read the rules.
  3. Apply as defaults. First-party docs and the project's own conventions win on conflict. Warn before relying on a fast-moving page older than 12 months.
  4. Cite. Link the page by title and URL, e.g. Python, with last_updated for time-sensitive rules. License CC BY 4.0.

Overview

An embedding model maps text, code, or images to fixed-length vectors, and an index is valid only for the model that produced it. Anthropic does not ship an embedding model; its documentation points to Voyage AI. Model, dimension, and metric set the quality ceiling for rag; the rules below cover picking and replacing the model.

Pick from the current model list, then bake off

Prices are per million input tokens, from each vendor’s docs on 2026-10-01.

ModelDefault dimShorter dimsContextPriceNotes
voyage-4-large1024256, 512, 204832K$0.12Best Voyage quality
voyage-41024256, 512, 204832K$0.06Balanced default
voyage-4-lite1024256, 512, 204832K$0.02Latency and cost
voyage-4-nano1024256, 512, 204832Kself-hostOpen weights (Apache 2.0)
text-embedding-3-small1536any lower via dimensions8K$0.02
text-embedding-3-large3072any lower via dimensions8K$0.13
Cohere embed-v5.0-pro / -fast2048256 to 1536128Ksee CohereText and images
gemini-embedding-23072128 to 30728K$0.20 textText, image, video, audio, PDF
  • Voyage 4 models share one embedding space, so you can index documents with voyage-4-large and embed queries with voyage-4-lite. Do not mix with other families.
  • For code use voyage-code-4. For chunks that lose meaning outside their document, use voyage-context-4 through contextualized_embed().
  • gemini-embedding-001 is text-only with a 2K-token input limit.
  • Multilingual corpora: include voyage-4, text-embedding-3-large, and bge-m3 in the bake-off, and test on real non-English queries; names and brand terms in other scripts show up even in English products.
  • Self-hosted: qwen3-embedding (0.6B, 4B, 8B), embeddinggemma (768 dim), nomic-embed-text, or bge-m3, served by ollama or a dedicated server.
  • Set the query-versus-document hint when the API has one. Voyage says never to omit input_type.
vo = voyageai.Client()
docs = vo.embed(chunks, model="voyage-4", input_type="document", output_dimension=1024).embeddings
q = vo.embed([query], model="voyage-4", input_type="query", output_dimension=1024).embeddings[0]

Public leaderboards average over many domains. Run two or three candidates at the same dimension against your golden set (see rag-eval), then weigh quality against price and any multilingual or multimodal need.

Tag every vector with its model

Never mix vectors from different models in one index, even when the dimensions match; they live in unrelated spaces. Store model ID, model version, and dimension as metadata on every record, and reject queries embedded with a different model.

Replace models with a shadow index

A new model means re-embedding the full corpus.

  1. Create a second index; write every new document to both.
  2. Backfill the old corpus into the new index in batches.
  3. Query the old index while you score the new one on the golden set.
  4. Flip query traffic when recall matches or beats the old index; keep the old one for a rollback window.

Skipping step 3 is the usual way to ship a retrieval regression. Switching vendors is the same procedure, so budget for it before committing.

Cache and batch the embedding calls

Key the cache on model ID, model version, and a hash of the exact input text; use batch endpoints for ingestion. Details and prices are in embeddings-cost-control. Dimension choice, normalization, and the index metric are in embeddings-dimensionality.

Backlinks