Overview
Embedding cost is input tokens times the model price, and it does not depend on the output dimension. The levers are fewer tokens (cache, deduplicate), cheaper tokens (batch endpoints, smaller tier), and a hard budget per run. For caching LLM answers rather than vectors, see embeddings-semantic-cache.
Use batch endpoints for every ingest that can wait
| Provider | Discount | Window | Limits |
|---|---|---|---|
OpenAI Batch API (/v1/embeddings) | 50% | 24h | 50,000 embedding inputs per batch, 200 MB file |
| Google Gemini Batch API | 50% | async | see Gemini docs |
| Voyage Batch API | 33% | 12h | 100K inputs per job, 1B tokens in flight per organization |
batch_file = client.files.create(file=open("batch_input.jsonl", "rb"), purpose="batch")
client.batches.create(input_file_id=batch_file.id, endpoint="/v1/embeddings", completion_window="24h")Each JSONL line is {"custom_id": "doc-1", "method": "POST", "url": "/v1/embeddings", "body": {"model": "text-embedding-3-large", "input": "...", "dimensions": 1024}}. Match results by custom_id; output order is not guaranteed. Keep synchronous calls for the query path only.
For synchronous ingest, batch inputs per request. Voyage allows up to 1,000 inputs and a model-dependent token cap (120K for voyage-4-large, 320K for voyage-4, 1M for voyage-4-lite). OpenAI allows 2,048 inputs and 300K tokens per request.
Cache vectors by content hash
Embeddings for a fixed model and input are reproducible enough to cache indefinitely. Include everything that changes the output in the key: model ID, output dimension, input_type, and the exact text.
import hashlib
def cache_key(model: str, dim: int, input_type: str, text: str) -> str:
return hashlib.sha256(f"{model}|{dim}|{input_type}|{text}".encode()).hexdigest()Voyage prepends different prompts for query and document, so the same text embeds differently for each type. Store the normalized vector (see embeddings-dimensionality) in Redis, DynamoDB, or SQLite. Drop all entries for a model when you migrate off it.
Deduplicate before the API call
Overlapping chunks, repeated boilerplate, and the same document arriving from two sources all embed twice. Hash each chunk, embed unique hashes, then map vectors back to original positions. Count the duplicates in your own corpus to size the saving; see rag-chunking for keeping overlap small.
Count tokens before sending
Over-length input either fails (OpenAI limits each input to 8,192 tokens) or is cut silently (Voyage’s truncation and Ollama’s truncate both default to true). Count with the provider’s tokenizer, and chunk or truncate on purpose. Character counts mislead on code and non-English text.
Cap spend per run
Compute tokens x price before a job starts and fail when it exceeds the budget; log tokens per run and alert on a run well above the historical average, since spikes usually mean a retry loop or an unexpected corpus change.
est_cost = total_tokens / 1_000_000 * PRICE_PER_MILLION
if est_cost > MAX_COST_PER_RUN:
raise RuntimeError(f"Estimated ${est_cost:.2f} exceeds budget ${MAX_COST_PER_RUN:.2f}")Split query and corpus models only within one family
A cheaper query-side model works only when the vectors share a space. Voyage documents that all Voyage 4 embeddings are compatible, so you can embed the corpus with voyage-4-large and queries with voyage-4-lite. Across other families, queries and documents must use the same model. Compare against the all-large baseline on your golden set first.