LLM Best Practices

ChromaDB Best Practices

Last updated 2 min read

View markdown source

AI agents: read this first: llmbestpractices.com/start-here

This is LLM Best Practices (llmbestpractices.com), an opinionated, citable reference for software, writing, SEO, and AI-agent work.

  1. Route, do not crawl. Fetch llmbestpractices.com/llms.txt and open only the pages whose one-line summary matches your task.
  2. Read raw. Append .md to any page URL for markdown. Check status and last_updated in the frontmatter, then read the rules.
  3. Apply as defaults. First-party docs and the project's own conventions win on conflict. Warn before relying on a fast-moving page older than 12 months.
  4. Cite. Link the page by title and URL, e.g. Python, with last_updated for time-sensitive rules. License CC BY 4.0.

Overview

Use ChromaDB for prototypes, internal tools, and small production rag workloads where one node holds the vectors and one application queries them. These notes target the Chroma 1.x line (Python chromadb 1.5.9, JavaScript chromadb 3.5.0 as of 2026-10-01). Each topic has its own page; this one holds the decisions.

Pick the client by deployment, not preference

import chromadb
 
client = chromadb.EphemeralClient()                         # in memory; tests and notebooks
client = chromadb.PersistentClient(path="./.chroma")        # embedded; one process owns the directory
client = chromadb.HttpClient(host="localhost", port=8000)   # server started with `chroma run --path ./.chroma`
client = chromadb.CloudClient(tenant="...", database="...", api_key="...")  # Chroma Cloud

AsyncHttpClient is the async variant of HttpClient. Use PersistentClient for scripts, single-worker services, and notebooks. Use the server once several processes or services need the data. Deployment, auth, and backup are in chromadb-persistence.

Model collections around retrieval tasks

One collection holds one embedding space and one distance metric. Split collections by retrieval task or embedding model (docs_support, code_snippets), and separate tenants with a tenant_id metadata filter unless tenants need hard isolation or different embedding configurations. Design rules, naming, and embedding ownership are in chromadb-collections.

Query with a filter and a bounded n_results

results = collection.query(
    query_embeddings=[query_embedding],
    n_results=10,
    where={"$and": [{"tenant_id": "t_123"}, {"lang": "en"}]},
    where_document={"$contains": "refund"},
)
  • n_results defaults to 10. A selective filter returns fewer rows than n_results, never padding.
  • Multiple conditions need an explicit $and. A where dict with two keys raises an error.
  • Chroma has no local BM25 index; the search hybrid API works on Chroma Cloud only. Merge keyword and vector results in your application; see chromadb-filters and rag-retrieval.

Outgrow ChromaDB on measured signals

Chroma’s single-node guidance sizes RAM at about 0.245 million 1024-dimension vectors per GB, and its tests held up to roughly 7 million embeddings. Move to pgvector when you already run postgres and want SQL joins and one backup story, to Qdrant when filtered search at scale is the bottleneck, or to Chroma Cloud when you want to stay on the same API. Signals, sizing, and the migration path are in chromadb-scale-limits; for the managed comparison see pinecone-vs-pgvector.

Backlinks