LLM Best Practices

ChromaDB: Persistence, Server Mode, and Backup

4 min read

View markdown source

Overview

ChromaDB offers three client modes: ephemeral (in-memory), persistent (file on disk, single process), and server (HTTP, multi-process). Choosing the wrong mode for your deployment pattern is the most common operational mistake. Two PersistentClient instances pointed at the same path will corrupt the SQLite index; a server mode with no authentication will expose your collection to any process on the network.

Use PersistentClient only when a single process owns the data directory

PersistentClient holds an exclusive lock on the SQLite backing store. It is correct for single-process workloads: CLIs, background workers, single-container services.

import chromadb
 
client = chromadb.PersistentClient(path="/data/chroma")
  • The path must be writable by the process. On Docker, mount a named volume so data survives container restarts.
  • Do not share the path between two Python processes. The second process will silently read stale data or corrupt the index.
  • For multi-worker web frameworks (Gunicorn, uWSGI), use server mode instead; each worker gets its own HTTP connection.

Run server mode for multi-process access

The chromadb/chroma Docker image exposes port 8000 and manages the SQLite backing store in a single process. All other processes talk to it via HTTP.

docker run -d \
  --name chroma \
  -p 8000:8000 \
  -v chroma_data:/chroma/chroma \
  chromadb/chroma:latest
import chromadb
client = chromadb.HttpClient(host="chroma", port=8000)

In Docker Compose, place the chroma service on the same network as your app container so the hostname resolves. Do not expose port 8000 to the public internet without authentication.

Add authentication and TLS before exposing the server

ChromaDB server mode has optional token-based authentication. Enable it in any environment reachable from outside localhost.

docker run -d \
  --name chroma \
  -p 8000:8000 \
  -e CHROMA_SERVER_AUTH_CREDENTIALS="my-secret-token" \
  -e CHROMA_SERVER_AUTH_CREDENTIALS_PROVIDER=chromadb.auth.token.TokenConfigServerAuthCredentialsProvider \
  -e CHROMA_SERVER_AUTH_PROVIDER=chromadb.auth.token.TokenAuthServerProvider \
  -v chroma_data:/chroma/chroma \
  chromadb/chroma:latest
from chromadb.config import Settings
client = chromadb.HttpClient(
    host="chroma", port=8000,
    settings=Settings(
        chroma_client_auth_provider="chromadb.auth.token.TokenAuthClientProvider",
        chroma_client_auth_credentials="my-secret-token",
    ),
)

For production, terminate TLS at a reverse proxy (nginx, Caddy) in front of ChromaDB and restrict port 8000 to the internal network.

Back up by copying the data directory while the server is idle

ChromaDB’s on-disk format is a SQLite database plus binary segment files. The correct backup strategy is a filesystem snapshot or a directory copy taken while no writes are in flight.

# Stop the container, snapshot the volume, restart
docker stop chroma
docker run --rm -v chroma_data:/source -v /backup:/dest alpine \
  tar czf /dest/chroma-$(date +%Y%m%d).tar.gz /source
docker start chroma

Back up before every major schema or model change. Backups are the migration rollback plan. See chromadb-scale-limits for the signals that indicate it is time to migrate to a different database.

Migrate collections by exporting to Parquet and reimporting

When moving to a new ChromaDB instance, a different server version, or a different vector database, export the collection data first.

import pandas as pd
 
def export_collection(collection, batch_size=1000):
    all_ids, all_docs, all_metas, all_embeddings = [], [], [], []
    offset = 0
    while True:
        batch = collection.get(
            limit=batch_size,
            offset=offset,
            include=["documents", "metadatas", "embeddings"],
        )
        if not batch["ids"]:
            break
        all_ids += batch["ids"]
        all_docs += batch["documents"]
        all_metas += batch["metadatas"]
        all_embeddings += batch["embeddings"]
        offset += batch_size
    df = pd.DataFrame({
        "id": all_ids,
        "document": all_docs,
        "metadata": all_metas,
        "embedding": all_embeddings,
    })
    df.to_parquet(f"{collection.name}.parquet", index=False)

Import to the new instance with collection.add() in batches of 500 to 1000. Plan one migration, not three.

Version the data directory path alongside code changes

Treat the ChromaDB data path as part of the deployment manifest. When you re-embed the corpus with a new model, use a new path or a new collection name, not an overwrite.

  • Pattern: /data/chroma/v1/ for model version 1, /data/chroma/v2/ for model version 2.
  • Run both versions in parallel during the transition. Switch query traffic when eval metrics confirm parity.
  • The old version is the rollback; keep it until recall on the new version matches for at least one week in production.

Graph View

Backlinks