Install Sift

This page gets Sift running on one host with Docker Compose, and gets you signed in to the console with a password of your own. It takes a few minutes of typing and, on the first start, a download of about 134 MB. For a deployment other people will rely on, read Deploy for production as well.

Before you start

You need a host with:

Docker	Docker Engine 24 or later, with Docker Compose 2.24 or later
Memory	About 8 GB
Disk	About 20 GB
CPU	amd64 or arm64. No GPU: Sift scores on CPU.
Tools	make and python3 on the host, for make init
Network	Outbound HTTPS to Hugging Face on first start, for the model weights. Later, to your Kindo API.

You also need the Sift release, in one of two forms:

• A release bundle: an images file (kindo-sift-<version>-images.tar.gz) with its .sha256 checksum, and a source archive of the release (kindo-sift-<version>-source.tar.gz) holding the compose file, Makefile and scripts. The images are loaded rather than built; the bundle includes the larger SetFit image too.
• The source: a checkout of the Sift repository. make up builds the images the first time.

If the host has no internet access at all, follow Install without internet access instead.

Install

1. If you have a release bundle, check it, load the images, and unpack the source:

   sha256sum -c kindo-sift-<version>-images.tar.gz.sha256
   gunzip -c kindo-sift-<version>-images.tar.gz | docker load
   tar xzf kindo-sift-<version>-source.tar.gz

2. In the directory that holds docker-compose.yml, create the configuration file:

   make init

   This writes .env from .env.example, readable only by you, and generates three secrets in it: SIFT_SECRET_KEY (which signs sessions and encrypts the Kindo key), POSTGRES_PASSWORD and SEAWEEDFS_SECRET_KEY. It never overwrites an existing .env. The console’s admin account is left at admin / admin.

3. Optional: to use the Text analysis (small sample) recipe, which needs a larger image, add this line to .env before starting:

   SIFT_EXTRAS=setfit

   The image is about 1.1 GB larger and the first start downloads another 133 MB of weights. Without it, the console shows that recipe as unavailable; every other recipe works.

4. Start the stack:

   make up

   It prints the console’s address, http://localhost:8080. To use another port, set SIFT_WEB_PORT in .env. make up and make smoke read the port from your shell, not from .env, so export the same value there too, or the printed address and the smoke checks use 8080.

5. Wait for the first start to finish, and check that Sift is ready:

   make ps
   curl http://localhost:8080/api/readyz

   Every service in make ps should read healthy, and readyz should answer "status":"ok".

What the first start does

The API container prepares everything before the rest of the stack starts. It runs the same steps on every start, but each one does nothing when its work is already done:

1. waits for Postgres;
2. applies the database migrations;
3. creates the object-storage bucket for trained models;
4. creates the admin account, if no account by that name exists;
5. downloads the pinned model weights Sift scores events with, about 134 MB, into the model-weights volume, and checks them against their recorded digest.

The download is the slow part. The API is given up to fifteen minutes for it, and the workers start once it is done. To follow along:

make logs SERVICE=sift-api

Look for model_weights_ready. If the weights on disk ever fail their digest check, the container stops with model_weights_digest_mismatch rather than score with the wrong weights. If the download fails, Sift starts anyway and logs model_weights_unavailable; run make models to try again.

Check the install (optional)

make smoke runs Sift’s acceptance checks against the running stack: every service healthy, sign-in and permissions working, the worker queues answering.

make smoke

It signs in as admin with the password in your shell’s SIFT_DEFAULT_ADMIN_PASSWORD (not the one in .env), or admin if that is unset, so run it before you change the password, or export that variable first. Every run leaves two things behind: a reviewer account named smoke-reviewer-<number> and a retired filter named lifecycle-check-<number>. You can disable the account under the gear icon (Administration) › Users. make smoke does not work once the stack is behind the TLS overlay.

Sign in and change the password

1. Open http://localhost:8080 in a browser.

   You see Sign in to the operations console.

2. Enter admin in Username and admin in Password, and press Sign in.

   A banner reads Default administrator password still in use. It stays on every page until the password changes; it cannot be dismissed.

3. Press Change password on the banner. (Later, the same page is in the user menu, the person icon at the top right.)

4. Enter Current password, New password (at least 12 characters) and Confirm new password.

5. Press Change password.

   You see “Password changed. Every other session of this account has been signed out.” The banner is gone.

Caution

Sift stores a Kindo API key. Anyone who can reach the console with the default password can sign in as an admin and use that key. Change it before you connect Kindo.

The dashboard now shows Get Sift running, a checklist of the steps that remain. See Getting started.

Everyday commands

Command	What it does
make ps	Shows each service and its health.
make logs SERVICE=sift-api	Follows one service’s log.
make down	Stops the stack. Keeps all data.
make up	Starts it again.
make clean	Stops the stack and deletes every data volume.

Danger

make clean deletes your filters, labels, trained models and the stored Kindo key, with no confirmation. Use make down to stop Sift.

These docs

The documentation is served by Sift itself, at /docs/ on the console’s address, with no sign-in. The console’s user menu has a Documentation link, and the sign-in page has Read the Sift documentation.

What can go wrong

You see	What to do
make up stops with “set SIFT_SECRET_KEY in .env”	Run make init first.
A service stays starting for a long time on first start	The weights are still downloading. Follow make logs SERVICE=sift-api. On a slow link, stage them by hand: see Install without internet access.
readyz answers "status":"degraded"	One of Postgres, Redis or the object store is not answering. Its entry under checks says which. See Troubleshooting.
“Incorrect username or password.” after a redeploy	The admin account is created only when absent; a redeploy never resets its password. Use the password you set.

Next

• Connect Kindo, as an admin.
• Deploy for production before real use: HTTPS, the production settings, and backups.