ChromaDB Production Deployment: The Stuff That Actually Matters

ChromaDB got a lot better after they rewrote it in Rust. Version 0.5.0 broke a bunch of stuff, but the recent releases are way more stable. Still not fast though - tried it on a 5M vector dataset and queries took forever.

Three Ways People Deploy This Thing

Docker: Works fine until you fuck up the volume mount. You need to mount /data - not /chroma like the old docs said. Spent a whole Saturday figuring out why my data kept disappearing. The error messages don't help either.

Kubernetes: Uses a lot of memory and setting up persistent storage is a pain. There's a community Helm chart that I've used. The default settings are garbage though - you'll need to bump up the memory limits.

Chroma Cloud: Free tier plus usage costs. No monthly minimum anymore. Worth it if you don't want to deal with infrastructure bullshit. Their pricing is confusing though - you pay per collection operation and vector storage.

Memory Usage Reality

ChromaDB's docs recommend 2GB minimum. From running this in production:

• Under 1M vectors: 4GB works most of the time
• 1M-10M vectors: 8-16GB, really depends on your queries
• Over 10M vectors: 32GB+ or just use something else

Memory usage just keeps growing and never comes back down. I restart mine every few days.

Getting Persistence Right

ChromaDB uses SQLite for persistence. SQLite is reliable but needs proper volume mounting.

Persistence was broken in early versions, works fine now if you mount correctly:

docker run -d \
  --name chromadb \
  -p 8000:8000 \
  -v /local/path:/data \
  -e IS_PERSISTENT=true \
  chromadb/chroma:latest

Mount /data inside the container. The docs used to say /chroma/chroma but that never worked for me. Wasted way too much time on this.

Storage that works:

• Local SSD volumes (fastest, see SSD performance benchmarks)
• AWS EBS (reliable, decent performance with gp3)
• GCP Persistent Disks (expensive but solid)

Storage that doesn't:

• Network storage like EFS or Azure Files (SQLite hates network latency)
• Containers without persistent volumes (data disappears, obviously)

Security Setup

ChromaDB has basic auth but it's off by default. Most people put nginx or Traefik in front:

server {
    listen 443 ssl;
    server_name your-chroma.company.com;
    
    location / {
        proxy_pass http://chromadb:8000;
        proxy_set_header Host $host;
    }
}

Don't trust ChromaDB's built-in security for anything important. Treat it like an internal database and use proper network security if you're on K8s. Check out OWASP guidelines for API security best practices.

Docker works fine for small deployments. Kubernetes is overkill until you need multiple replicas, then it's essential. Check the Kubernetes documentation for deployment patterns.

Docker: Keep It Simple

The ChromaDB Docker image is straightforward. Here's a setup that actually works:

docker run -d \
  --name chromadb \
  -p 8000:8000 \
  -v chromadb-data:/data \
  -e IS_PERSISTENT=true \
  --restart unless-stopped \
  --memory=4g \
  chromadb/chroma:latest

Don't forget these or you'll be debugging all night:

• Mount /data correctly
• Set memory limits or it'll eat everything
• Use named volumes for easier backup
• The health check lies - it'll say everything's fine while your app returns 500s

Kubernetes: Complex but Scalable

Kubernetes makes sense for larger deployments. The community Helm chart works better than writing your own manifests.

Basic K8s deployment:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: chromadb-pvc
spec:
  accessModes: [ReadWriteOnce]
  resources:
    requests:
      storage: 50Gi
  storageClassName: gp3
---
apiVersion: apps/v1  
kind: Deployment
metadata:
  name: chromadb
spec:
  replicas: 1  # Don't try multiple replicas
  selector:
    matchLabels:
      app: chromadb
  template:
    spec:
      containers:
      - name: chromadb
        image: chromadb/chroma:latest
        resources:
          requests:
            memory: 4Gi
            cpu: 1
          limits:
            memory: 8Gi
            cpu: 2
        volumeMounts:
        - name: data
          mountPath: /data
        env:
        - name: IS_PERSISTENT
          value: \"true\"
      volumes:
      - name: data
        persistentVolumeClaim:
          claimName: chromadb-pvc

Storage Requirements

Important: Use block storage, not network storage.

Works:

• AWS EBS volumes (gp3 recommended)
• GCP Persistent Disks
• Local SSDs for high performance (NVMe drives)

Doesn't work:

• NFS, EFS, or Azure File shares (SQLite needs low latency)
• Shared storage between multiple instances (data corruption guaranteed)

Monitoring Setup

Focus on the basics with Prometheus or whatever monitoring you have:

Memory usage - Alert at 80% usage, restart at 90% (OOMKilled is coming)
Disk space - ChromaDB can fill disk quickly during bulk operations (monitor with node exporter)
Query latency - P95 should stay under 100ms (Grafana dashboards help visualize this)
Container restarts - Track restart frequency with kubectl

Simple monitoring commands:

## Check memory usage
docker stats chromadb --no-stream

## Check disk usage  
docker exec chromadb df -h /data

## Kubernetes pod restarts
kubectl get pod -l app=chromadb

Real Production Costs

AWS t3.large example (monthly):

• Instance: ~$65
• EBS storage: $8-12 depending on size
• Data transfer: varies, maybe $15-40
• Total: around $100/month, give or take

Kubernetes cluster:

• EKS control plane: $73/month
• Worker nodes: probably $250-500
• Storage/networking: $60-150
• Total: $400-700/month maybe

I spend way too much time babysitting this thing - probably 5-10 hours a month restarting containers and debugging weird query timeouts

Chroma Cloud starts looking reasonable when you factor in operational overhead.