Your first filter

This walkthrough takes one filter from a description to the day it starts filtering events. The first part, up to starting shadow, takes well under an hour. Then Sift collects verdicts for a few days, and you come back to approve a threshold. Each step links to the guide that covers it in full.

Before you start

• You are signed in as an operator or admin.
• An admin has connected Kindo and chosen Sift’s model (Connect Kindo).
• The source you want to filter is connected as an integration in your Kindo tenant, and you know roughly which events from it your agent should act on.

1. Describe the filter

1. Press New filter on the dashboard or the Filters page.

   You see What should this filter catch?, with four steps across the top: Source, Build the filter, Review and create, Connect the agent. If Kindo is not connected or not answering, the page says so and stops here.

2. Under Start from, choose A source.

   The other choices start from an agent you already run (Sift reads its past runs first, which costs money) or from a plain description. See Create a filter.

3. Pick the integration. Type in Find an integration to narrow the list.

   Sift checks that it can read the integration, and shows “Sift can read” and its name. It then drafts a few starting points under Start from one of these.

4. Press a starting point, or write your own in Description: one or two sentences about which events matter, such as “Escalate incidents that paged a human and were not auto-resolved.”

5. Press Continue.

2. Build it with the assistant

The page Building the filter opens a conversation with Sift’s assistant. It reads your request, finds the source, fetches a sample of events, analyzes them, and proposes a filter. Each stage shows as it runs.

1. Answer its questions in the reply box and press Send.

   When it has a proposal, a card shows what the filter will read, how it will decide, and any Open questions.

2. Press Review and create →.

The assistant cannot create or change anything. It only proposes; nothing exists until you create it in the next step. It may also answer that no filter is worth building on this source, which is an answer, not a failure.

3. Review and create

The page Review and create shows the proposal, editable: Name, Intent (the sentence the gold set is judged against), What it will do, the recipes to Compare against, the Posture, and Advanced settings, all at their defaults.

1. Read the Intent closely. The gold set is judged against it. Changing it later, while the filter is still in authoring, throws away the gold set; once the filter is in shadow it cannot change at all.

2. Press Create filter.

   The page reads Created: “In authoring. It filters nothing yet.” Below it is a short list: Connect the agent, Review the gold set (Sift has started building it), and Start shadow. If the filter feeds an agent you already have, there is no Connect the agent item: go on to section 5.

4. Connect the agent

Sift has generated a paired agent for this filter: a Kindo agent built to take one event per run and record a verdict Sift can read. It has to exist in Kindo before anything reaches it.

1. Press Get the agent →.

2. Under What makes an event worth acting on?, write Your criteria. Without them the agent answers insufficient data for every event.

3. Optional: under What should the agent do about it?, write Your actions, such as opening a ticket.

4. If you changed either answer, press Apply your answers.

5. Press Create in Kindo and connect.

   Sift creates the agent in Kindo with its own key, reads it back, and pairs the filter to it. You see “Created and paired” with the agent’s name, and a short list of what is left to do in Kindo, such as sharing the agent with the people who will edit it: it starts private to the user the key belongs to.

If your Kindo cannot create agents through the API, use Other ways to connect it on the same page to download the definition, import it in Kindo, and paste the new agent’s id back. See Connect a filter to its agent.

5. Review the gold set

The gold set is about fifty events from your source, each with a proposed answer: should this event reach the agent? Your judgement on each is the filter’s first approval.

1. Open the filter and its Gold set tab. (From the filter’s Actions tab, press Review the gold set.)

   While Sift is still building the set, the tab says so. It reads events from the source and proposes a label for each.

2. For each example, read the event against the intent shown at the top, and press Escalate or Filter. The keys E and W do the same, and U jumps to the next unreviewed example.

3. When every example is judged, press Approve the gold set, add a note if you want one, and confirm with Approve all 50 examples (the number is the set’s size).

   The set is locked. Approving it needs an operator or admin; a reviewer can judge the examples but not approve. Approving does not start the filter: it still reads nothing until the next step. See Review the gold set.

6. Start in shadow

1. Open the filter’s Actions tab.

   Under Needs you, a card reads Start watching in shadow. It appears once the gold set is approved and the agent is connected; until then, the Setup list says which is missing.

2. Optionally, write a Note (optional) for the audit log.

3. Press Start in shadow, and confirm.

   Where the stream’s size is known, the confirmation says what shadow will cost the agent, such as “About 4,000–6,000 agent runs an hour until a threshold is approved”. If that is more than the agent can take and no budget is set, it links to Set a budget, and the button reads Start sending everything.

The filter now reads its source, scores every event, and sends every one to the agent. It records what it would filter, and each verdict becomes a label. From now on, agent runs cost what they would with no filter, unless you set an agent budget or sampled shadow (see Put Sift in front of a big stream). See Shadow and filtering.

7. Wait for labels

There is nothing to do here but watch. The Actions tab shows the setup steps Collect 400 labels (the number is the filter’s label floor), then Collect 20 actionable events, each with a progress count and, when Sift can make one, an estimate of how long is left. When both are met, Sift trains a model on its own, measures it on the newest labels, and proposes an operating point. The filter’s state changes to proposed. It still sends everything.

If no labels arrive, the Actions tab says so. The usual cause is an agent that records no verdict; see Work through Actions.

8. Approve the threshold

1. On the Actions tab, press Approve the threshold.

   The page Threshold promotion shows the proposed threshold and what it was measured to do on the newest labels: recall, the actionable events it would have missed, and the share of events it would filter.

2. Read the evidence. If the proposed point filters more than you accept, choose a different measured point from the table.

3. Write why in Rationale. It goes in the audit log with the evidence attached.

4. Press Approve this operating point, and confirm with Approve — begin filtering below the threshold.

The filter’s state is now active, with a Filtering badge. Events that score below the threshold no longer start agent runs, apart from the small leak-through sample Sift keeps sending to measure what it misses. See Approve a threshold for what each figure means and when to reject.

What happens next

Sift keeps measuring the filter and watching for drift. If its traffic changes from what you approved, it moves the filter back to shadow on its own and tells you. To stop it filtering yourself at any time, use Stop or hold › Stop filtering in the filter’s header. See Watch a running filter and Hold, stop, resume or retire a filter.