Deploy for production

Sift runs as one Docker Compose stack of nine containers on a single host. This page describes what the stack is made of, what the host needs, and what to change before a deployment holds real data. For a first look on a laptop, Install Sift is shorter.

What the host needs

Requirement	Detail
Docker	Docker Engine 24 or later, with Compose v2 (2.24 or later: the compose file uses the env_file form with path and required).
Memory and disk	About 8 GB of RAM and 20 GB of disk.
CPU	CPU only. Scoring and training run on the CPU; no GPU is used.
Architecture	Images are built for one CPU architecture (linux/amd64 or linux/arm64). Use images built for your host’s.
Host tools	make and python3 for make init and the backup and restore scripts, which also use shasum.
Network	Outbound HTTPS to your Kindo API. On first start, a one-time download of the model weights (about 134 MB) unless you stage them yourself (Install without internet access). Outbound HTTPS to any webhook receiver an admin adds.

The services

Every backend container runs the same image, kindo-sift-backend:<version> (or its -setfit variant), in a different role. The containers run as uid 1000.

Service	What it does	What it keeps
postgres	Postgres 16. The system of record: filters, versions, models, events, scores, filtered events, approvals, users and the audit log.	postgres-data volume
redis	Redis 7. Work queues, the Kindo rate limit, assistant conversations, sign-in throttle counters and short-lived caches. Nothing in it needs to survive: every decision and label is in Postgres.	redis-data volume
seaweedfs	S3-compatible object store for trained model files, in bucket sift-artifacts. Its telemetry, admin UI and WebDAV are switched off.	seaweedfs-data volume
sift-api	The HTTP API. On every start it applies database migrations, creates the bucket, seeds the first admin account and stages the model weights, in that order, before serving.	Writes model-weights
sift-worker-io	Polling sources and scoring what they return, triggering agent runs, collecting verdicts, drift checks and the periodic sweeps. Eight at a time.	Reads model-weights
sift-worker-ml	Training and evaluation, two at a time. Kept apart so a long training run cannot hold up polling.	Reads model-weights
sift-worker-notify	Raising and clearing alerts, and sending them to webhooks.	Nothing
sift-beat	The scheduler. It queues the periodic work and does none itself. Run exactly one: two would mean every poll twice.	Nothing
sift-web	nginx serving the console, these docs at /docs/, and the API behind /api.	Nothing

The workers and the scheduler wait until sift-api is healthy, so migrations always land first. Every service has a health check; docker compose ps should show each one healthy, not only running.

One published port

The stack publishes one port, SIFT_WEB_PORT (8080 by default), on sift-web. The console is at /, the API at /api/v1/, and the health checks at /api/healthz and /api/readyz. The API’s own port (8000) and the object store’s S3 port (8333) are not published: nginx is where the security headers, and TLS when you turn it on, are applied, and the login throttle relies on every request arriving through it. With the TLS overlay, HTTPS is published as well and the plain port only redirects.

Volumes

Volume	Holds	Notes
postgres-data	All durable state	In every backup.
seaweedfs-data	Trained model files	In every backup.
redis-data	Queues and caches	Not backed up; nothing in it needs to survive.
model-weights	The pinned model weights, about 134 MB (267 MB with the setfit variant)	Not backed up; reproducible from their pinned checksums. Do not delete it as part of an upgrade.

Docker prefixes each name with the Compose project, which defaults to the name of the directory you run from: kindo-sift_postgres-data for a checkout in kindo-sift/. make down keeps the volumes. make clean deletes all of them, and with them every filter, label and model.

Sizing

Sift has no measured sizing guide beyond the minimum above. What is measured is the scorer, on Sift’s own load check (one scoring slot, the Text analysis recipe, an Apple M3 Max running arm64 containers): about 171,800 one-line alerts an hour with a p99 of about 30 ms per event. Long events cost far more: an event of a few hundred words costs about 17 times a one-line alert. sift-worker-io scores eight at a time. Treat these as indications from one machine, not as capacity for yours; an amd64 host measures differently.

The pool sizes are set in docker-compose.yml (--concurrency=8 for io, 2 for ml). If you change the ml pool’s, set SIFT_ML_TRAIN_THREADS to match, so training does not ask for more CPU threads than the host has.

There is no high-availability mode. Sift runs on one host, with one scheduler, and ships no Kubernetes manifests or Helm chart.

Install from a release

A release gives you its files (the repository at the release’s tag: the compose files, Makefile, scripts/, .env.example) and an image bundle, kindo-sift-<version>-images.tar.gz, with its .sha256. The bundle holds every image the stack runs, including both backend variants. It does not hold the model weights.

1. Unpack the release’s files on the host and change into that directory.

2. Check the bundle against its checksum:

   sha256sum -c kindo-sift-<version>-images.tar.gz.sha256

3. Load the images:

   gunzip -c kindo-sift-<version>-images.tar.gz | docker load

4. Write .env, with a generated secret key and datastore passwords:

   make init

   make init creates .env from .env.example with mode 600, generating SIFT_SECRET_KEY, POSTGRES_PASSWORD and SEAWEEDFS_SECRET_KEY. It never overwrites an existing .env.

5. Edit .env for production, as in the checklist below.

6. Put your certificate in place and turn on the TLS overlay (Serve over HTTPS).

7. Start the stack:

   make up

8. Watch the API’s first start:

   docker compose logs -f sift-api

   The start steps log JSON lines with "logger":"entrypoint". You want migrations_done, buckets_done, seed_admin_done and model_weights_ready. The weight download happens before the API serves anything, so the first start can take several minutes; the health check allows fifteen.

9. Check readiness through the web port:

   curl -fsS http://localhost:8080/api/readyz

   Every check in the answer should read ok. With the TLS overlay, use your https:// address.

10. Sign in to the console as the admin account and connect Kindo.

make smoke runs the stack’s acceptance checks: every container healthy, every SIFT_* setting in .env reaching the API, the io and ml worker queues answering, sign-in and access control, a filter’s lifecycle, and the security headers. Know three things before you run it on a real deployment:

• It signs in with SIFT_DEFAULT_ADMIN_USERNAME and SIFT_DEFAULT_ADMIN_PASSWORD from your shell environment (admin/admin if unset), not from .env. Export the account’s current credentials first.
• It leaves two things behind on every run: a retired filter named lifecycle-check-<number> (which also ticks the dashboard’s first-filter step), and a reviewer account named smoke-reviewer-<number>, which it creates to check that a reviewer cannot reach the admin pages.
• It reaches the stack at http://localhost:<SIFT_WEB_PORT>. Under the TLS overlay that port only redirects, so its checks through the console fail there.

Production checklist

Item	Why
SIFT_ENVIRONMENT=production	Refuses to start on any published or weak credential, marks the session cookie Secure, and turns off the API’s interactive docs (/api/docs and /api/openapi.json).
A real SIFT_DEFAULT_ADMIN_PASSWORD, set before the first start	The seeded account is admin/admin otherwise, and while any account still has the password admin the console shows a banner that cannot be dismissed. Production refuses the shipped value or anything under 12 characters. The value seeds the account only when it does not exist; it never resets one.
SIFT_SECRET_KEY copied somewhere safe, apart from the backups	It encrypts the stored Kindo key and webhook secrets and signs sessions. It is not in any backup, and a restore without it loses those secrets. See Back up and restore.
TLS	The Secure cookie is not sent over plain HTTP, so without TLS nobody can stay signed in. Use the TLS overlay or your own TLS front end.
SIFT_TRUSTED_PROXY_HOPS	1 for the shipped nginx alone, 2 behind a load balancer that appends to X-Forwarded-For.
SIFT_CONSOLE_URL	The console’s address as people reach it. Alerts link to it; unset, their links are empty.
SIFT_METRICS_TOKEN	Turns on Prometheus metrics at /api/metrics. See Monitor Sift.
An outside check on /api/readyz	Two failures, Redis not answering and a silent worker pool, can never be sent as alerts by Sift itself. See Monitor Sift.
Scheduled backups	Nothing in Sift schedules them. See Back up and restore.

What production mode refuses, each named in one start-up error that gives the setting and never its value:

• SIFT_SECRET_KEY empty, shorter than 32 characters, or one of the keys printed in Sift’s own repository;
• SIFT_DEFAULT_ADMIN_PASSWORD equal to admin, or shorter than 12 characters;
• a Postgres password that is empty or the example sift;
• an object-store key equal to the example siftsiftsift.

make init generates the first three secrets, so a fresh .env only needs the admin password. See Configure Sift for how settings are applied.

Caution

The repository ships docker-compose.override.yml.example and several other compose files for development and testing. Never use them in a real deployment. The development override forces development mode, replaces your Kindo tenant with generated events and verdicts, publishes the API on loopback and runs the console’s development server. Compose reads a file named docker-compose.override.yml automatically, so make sure none exists in the directory you deploy from.

Related

• Configure Sift
• Serve over HTTPS
• Security and data handling
• Upgrade Sift