Filters and their lifecycle
A filter decides which events from one source are worth one agent’s run. This page explains what a filter is made of, the seven states it can be in, and what moves it from one to the next. The short version: a filter starts out sending everything, and it filters events only after a person has approved it twice.
What a filter is
Section titled “What a filter is”A filter is two things: a spec, which says what the filter does, and a lifecycle state, which says what it is allowed to do right now. Each filter reads one source and sends to one agent.
The spec has these parts:
| Part | What it holds |
|---|---|
| Source | The Kindo integration tool Sift calls, its parameters, and how often: the poll interval, 30 to 900 seconds, 60 by default. |
| Intent | One or two sentences, up to 2,000 characters, saying which events matter. The gold set is judged against it. |
| Stages | The recipe in each slot of the pipeline, and whether each runs for real or is only measured. |
| Target | The agent the filter sends to, and how the event is delivered: as the whole event to the paired agent, or mapped onto an existing agent’s inputs. |
| Operating point | The threshold (or hourly budget, or fixed rule), the leak-through rate, the trigger ceiling and what to do on an alarm. See Operating point and approval. |
| Acceptance | The posture, the label floor and the holdout. |
| Drift | The alarm levels for the drift signals, and the retrain schedule. |
You see most of these on the filter’s Settings tab. The filter settings reference lists every field with its default and range.
Versions
Section titled “Versions”Every save writes a new version of the spec. A saved version is never edited afterwards, so an approval always refers to exactly what was approved. Settings › History lists the versions.
Whether a save keeps the filter’s threshold approval depends on what changed:
- A change that cannot make it filter more (the poll interval, the drift alarm levels, a higher leak-through rate, the name) carries the approval forward to the new version.
- A change that could make it filter more, or read something other than what was approved (the operating point, the deciding recipe, the source query, the target, a lower leak-through rate), needs a new approval. On an active filter such a save moves it to shadow in the same step, so it stops filtering until the new version is approved.
- A change to a pipeline stage alone, on an active filter, waits as a waiting version (the console marks it Waits for approval). The filter keeps filtering on the version it runs, and the waiting one replaces it only when its own operating point is approved. A filter has at most one waiting version; a second such save replaces it.
The intent, and the source fields Sift uses to tell events apart, can change only
while the filter is in draft or authoring. After that, a new intent is a new filter.
The states
Section titled “The states”| State | Badge | Scores events | Filters | What it means |
|---|---|---|---|---|
| draft | draft | No | No | Created, with no spec to run yet. |
| authoring | authoring | No | No | A spec exists and is being reviewed. The gold set is built and reviewed here. |
| shadow | shadow | Yes | No | Scores every event and sends every one to the agent, recording what it would filter. An agent budget or sampled shadow, if you set one, sends less without filtering anything. |
| proposed | proposed | Yes | No | A trained model has an operating point waiting for approval. Still sends everything. |
| active | Filtering | Yes | Yes | Filters events that fall below the approved operating point. |
| paused | paused | No | No | Holding events: Sift is not reading the source, so nothing is scored, filtered or sent. |
| retired | retired | No | No | End of life. Nothing is scored, and there is no way back. |
Only an active filter filters events. Shadow and proposed do the same work as active, scoring every event, but send everything; that is how Sift measures a filter before anyone lets it filter. Shadow and filtering explains why.
The status badges beside a filter’s name show at most two things: first whether it is filtering, then whether anything is wrong. An active filter shows Filtering; any other state shows its own name, which also tells a screen reader that the filter is not filtering. A second badge, Attention or Fault, appears when the filter’s health needs someone, and links to its Actions tab. An active filter whose operating point filters nothing shows no first badge and always carries an Attention badge saying so.
How a filter moves
Section titled “How a filter moves”These are the only moves a filter can make, and Sift checks the gate on every one. The console offers only the moves that are possible from the current state.
| From | To | Who or what | Gate |
|---|---|---|---|
| draft | authoring | Sift, when the first spec version is written | A spec |
| authoring | draft | A person abandons the draft spec | None |
| authoring | shadow | A person: Start in shadow on the Actions tab | Gold-set approval, and the paired agent connected if the filter sends whole events |
| shadow | proposed | Sift, when training produces a model with an operating point; or a person, from the Actions tab | A trained model for the deciding recipe |
| proposed | active | A person approves the threshold, or resumes it on the model already running (Resume) | A threshold approval, new or standing |
| proposed | shadow | A person rejects the proposed threshold | None |
| active, shadow, proposed | paused | A person: Hold events | None |
| active | shadow | A person: Stop filtering; or Sift, on an alarm or a missing approval | None |
| paused | shadow | A person: Move it back to shadow | None |
| shadow, paused | active | A person: Resume | A standing threshold approval for the current version |
| any except retired | retired | A person, with a written reason | None |
Two moves change what an active filter filters without changing its state: a retrained model replacing the running one, and a waiting version replacing the running version. Both need a threshold approval, exactly as the move into active does.
The table has a deliberate shape. Every move that could mean more events are filtered needs a recorded approval. Every move that means fewer, into shadow or paused, needs none, so nothing stands between you and turning filtering off.
What Sift moves on its own
Section titled “What Sift moves on its own”Sift takes five moves without a person:
- draft to authoring, when the filter’s first spec version is written.
- shadow to proposed, when a training run finishes with an operating point to propose.
- active to shadow on drift, when the hourly drift check finds a signal past its alarm level.
- active to shadow on the trigger ceiling, when the filter’s escalations in one hour reach its ceiling.
- active to shadow when no approval stands, when the version being scored has no standing threshold approval.
The last three are fall-backs. Each withdraws the approval the filter was running on, and is recorded in the audit log as the system’s act. For drift and the ceiling you can set the filter to keep filtering and raise an alert instead; a missing approval always falls back. See Shadow and filtering.
Retiring
Section titled “Retiring”Retiring ends a filter for good: it stops reading, scoring and filtering, and there is no move out of retired. Nothing is deleted. Its history, approvals and filtered events stay readable. To start again, create a new filter. You retire a filter from Settings › Retire, with a written reason.
Who can move a filter
Section titled “Who can move a filter”Every state change, hold, resume and approval needs the operator or admin role. A reviewer sees the same screens without the controls. See Users and roles.