LLM Best Practices

ChromaDB: Collections and Embeddings

Last updated 4 min read

View markdown source

AI agents: read this first: llmbestpractices.com/start-here

This is LLM Best Practices (llmbestpractices.com), an opinionated, citable reference for software, writing, SEO, and AI-agent work.

  1. Route, do not crawl. Fetch llmbestpractices.com/llms.txt and open only the pages whose one-line summary matches your task.
  2. Read raw. Append .md to any page URL for markdown. Check status and last_updated in the frontmatter, then read the rules.
  3. Apply as defaults. First-party docs and the project's own conventions win on conflict. Warn before relying on a fast-moving page older than 12 months.
  4. Cite. Link the page by title and URL, e.g. Python, with last_updated for time-sensitive rules. License CC BY 4.0.

Overview

A ChromaDB collection owns one embedding space, one distance metric, an HNSW index configuration, and one metadata schema. Most retrieval bugs trace back to a collection that mixes embedding models or has a metadata schema that drifted. These rules cover when to split collections, what to fix at creation, and who computes the embeddings.

Create one collection per retrieval task

Split collections by workload and embedding model: docs_support, code_snippets, product_catalog. For many small tenants, keep one collection and filter on a tenant_id metadata key; use separate collections only when tenants need hard isolation or different embedding models. Always pass the tenant filter on every query, because a missing filter leaks rows across tenants.

Names are 3 to 512 characters from [a-zA-Z0-9._-], starting and ending with an alphanumeric. Prefix by service or environment (prod_support_docs) when instances are shared. Use get_or_create_collection for idempotent startup; create_collection raises when the collection exists.

Set the distance metric at creation

The default distance is l2. Choose cosine for text embeddings, or ip for inner product, through configuration. The space cannot be changed later, so a wrong choice means a new collection. ef_search is among the settings you can change afterwards with collection.modify.

collection = client.get_or_create_collection(
    name="docs_support",
    configuration={"hnsw": {"space": "cosine"}},
    metadata={"embedding_model": "voyage-3", "dimension": 1024},
    embedding_function=None,
)

The older metadata={"hnsw:space": "cosine"} form still works in 1.5.9, but configuration is the documented path. Custom metadata keys, such as the model name above, are yours to read back with collection.metadata.

Keep metadata flat and typed before ingest

  • Decide the keys first: source, tenant_id, created_at, lang, doc_id, chunk_index.
  • Values are str, int, float, bool, or lists of those. A nested dict raises ValueError; store complex payloads as a JSON string and parse them client-side.
  • Use one type per key forever. A range filter such as {"created_at": {"$gte": 5}} silently skips rows whose value has a different type, so never write an ISO string to an integer timestamp key.
  • Metadata-only changes do not need re-embedding: call collection.update(ids=..., metadatas=...).
collection.add(
    ids=["chunk-001"],
    documents=["Refunds are processed within 5 business days."],
    embeddings=[embedding],
    metadatas=[{"tenant_id": "acme", "source": "zendesk", "created_at": 1736294400, "lang": "en"}],
)

Give embeddings exactly one owner

Chroma enforces the dimension, not the model. A collection created with 1024-dimension vectors rejects a 384-dimension query with an error, but two models that share a dimension mix silently and give meaningless scores. Choose one owner:

  • Collection-owned: pass an embedding_function and call add(documents=...) and query(query_texts=...). Chroma persists the function and its parameters in the collection configuration and rebuilds it on get_collection. The default is all-MiniLM-L6-v2 (384 dimensions), which runs locally; hosted providers such as OpenAI call their API on every add and query.
  • Pipeline-owned: create the collection with embedding_function=None, compute vectors yourself, and always pass embeddings= and query_embeddings=. A query_texts call on such a collection raises an error. This fits batching, caching, and rate limiting outside Chroma, and keeps tests free of network calls.

Record embedding_model and dimension in collection metadata and assert them in every ingest worker before writing.

Change the model by creating a new collection

Provider models can change behind a name, and switching models invalidates every stored vector. Pin the exact model identifier. When the provider offers no version pinning, embed a fixed reference sentence at ingest and store a hash of its vector in collection metadata; a changed hash signals a silent model update. To upgrade, create a new collection, re-embed the corpus, compare recall on a golden set, move query traffic, and keep the old collection for rollback. See embeddings for the upgrade protocol and embeddings-dimensionality for Matryoshka truncation, the usual source of dimension mismatches.

Reset only in development

Gate client.delete_collection(name) followed by create_collection behind an explicit environment variable (RESET_CHROMA=true), never in a production path. In production, upsert by id and store a version metadata key so partial re-ingests converge. See chromadb-persistence for backups.

Backlinks