Operating point and approval

A filter’s operating point is where it draws the line between sending an event and filtering it, plus the guards around that line. Sift measures what each possible line would do, proposes one from a policy you state, and lets the filter start filtering only when a person approves it with a written reason. This page explains each of those steps.

What an operating point holds

Part	What it is	Default
Threshold	The score at or above which Decide sends an event. Events scoring below it are filtered.	0, which sends everything
Leak-through rate	The share of the events it would filter that are sent anyway, to measure misses.	3% (1% to 5%)
Trigger ceiling	The most escalations a filter that is filtering may send in an hour.	200
On alarm	Stop filtering (move to shadow), Keep filtering (and raise an alert), or Filter at a safer point (on drift, move to a looser point approved with the threshold).	Stop filtering

A threshold suits recipes that learn a score from labels. Two kinds of recipe are read differently:

Anomaly detection works to an hourly budget instead: it sends the most unusual events of each hour, up to the budget, and filters the rest. Its evidence is what the ranking did to live traffic in shadow, so its recall may not be measured.
Fixed rules and Collapse repeats carry their decision in their own settings. There is no line to choose: you approve what they would have filtered in shadow, or for Collapse its own evidence. See Recipes.

The leak-through rate, ceiling and alarm response are explained in Shadow and filtering.

The frontier: every line, measured

When Sift trains a model, it scores the holdout: the newest 20% of the filter’s labels (or 30%), kept out of training. See Verdicts, labels and training.

From the holdout Sift builds the frontier: a range of possible thresholds, each measured on the holdout’s real verdicts, plus the point that escalates everything. For each point:

Measure	Meaning
Reach the agent	Share of events at or above the threshold.
Filtered	Share below it, less the leak-through sample, which is sent and so is not a saving.
Recall	Share of the holdout’s actionable events that still reach the agent.
Precision	Of what reaches the agent, the share that was actionable.
Missed	Actionable events in the holdout this point would filter.

The threshold screen shows a selection of these points in a table, What you would be choosing between, per 100 events: how many reach the agent, how many are filtered, and of 10 that mattered, how many are missed.

Each point’s recall also carries a 90% range, which is wider the fewer actionable events the holdout held: a holdout that catches all of 34 actionable events reads about 92–100%, not 100%. On a filter trained on sampled shadow, each judged event counts for the events it stands for, so the rates describe the stream, not the sample.

The figures are measured against the agent’s verdicts. Where the agent calls benign events actionable, the missed counts read high. If the holdout holds fewer than 20 actionable events, recall moves in steps too large to mean anything, and Sift proposes no threshold at all.

Posture: the policy that picks a point

You do not type a threshold. You choose a posture: how many extra agent runs you will pay to catch one more event that matters. Sift then picks the point on the frontier that is worth the most at that price: it filters as much as it can while each event it still catches is worth the agent runs it costs.

The price is stated relative to what a catch costs on this stream with no filter at all, so the same posture means the same policy on a busy stream and a quiet one.

Posture	Multiple	In practice
Maximise savings	0.5×	Filters more and accepts missing more.
Balanced	2×	Pays twice the base cost of a catch.
Catch everything	Pays anything	Filters little; on a weak model it may filter nothing, and Sift then says so.
Custom	Your own multiple	Set under Catch-cost multiple.

You set the filter’s posture under Settings › General › Policy. A spec written without one uses a recall floor instead (acceptance.min_value, 0.99 by default).

Sift refuses a point that catches no more than chance: one whose recall is no higher than the share of events it sends. Such a point is not a filter, whatever the posture.

The approvals

A filter needs two separate approvals before it filters, both by a person with the operator or admin role. The console shows the forms to a reviewer, but the server refuses a reviewer’s decision.

Gold-set approval comes first. It confirms Sift reads the filter’s intent the way you meant it, and it allows the filter into shadow; it does not start shadow. An operator or admin then presses Start in shadow on the Actions tab (once the paired agent is connected, if the filter has one). See The gold set.
Threshold approval comes after training and is the only way a filter starts filtering. It approves one operating point for one spec version.

A third kind, grouping approval, approves a change that only groups escalations under Shape delivery. It never starts a filter filtering and never moves it into active.

When a threshold is proposed

When a training run on a filter in shadow produces an operating point, Sift moves the filter to proposed and the Actions tab asks for a decision. The threshold screen, Threshold promotion, shows the filter’s intent, the measured figures at the proposed point, the frontier, and what approving would do.

As the approver you can:

Approve the posture’s point. The button says what happens, for example Approve — begin filtering below 0.41. The threshold is shown to two decimals.
Change the posture with the Posture control, which picks that posture’s point from the same frontier with no retrain. Tick Also make this the filter’s posture to keep it for the next retrain.
Choose a different point by stepping along the frontier or picking a row in the table. Every point offered was measured; nothing is typed.
Reject it (Reject this operating point). A filter in shadow or proposed returns to shadow and keeps collecting labels; an active filter stays as it is. The rejected proposal is not offered again; the next training proposes another.

Every decision needs a written Rationale, up to 2,000 characters. The audit log records the decision and the rationale; the evidence you were shown is kept with the approval.

Acknowledgements

In some cases the form asks you to tick a statement before approving, and the server refuses the approval without it:

Case	When
Almost no saving	The point filters less than 5% of events. The screen heads it This filter would save almost nothing.
Not better than what runs	A retrained model, on a filter already filtering, did no better than the running model on the same events, or was not compared.
Recall not measured	An anomaly budget whose recall is not measured, the running model at a point outside its recorded frontier, or a waiting version whose figures could not be taken.

Between 5% and 10% the screen marks A thin saving but asks for no acknowledgement. An operator can change both levels for the deployment; see the configuration reference.

What an approval records

Approving a trained proposal writes the chosen operating point into a new spec version and records the approval against that version. (Approving a waiting version swaps in the version already saved; approving the running model writes no new version.) The approval records who, when, the rationale, the posture, and the figures shown. A later change that could make it filter more events needs a new approval; one that cannot carries this approval forward. See Filters and their lifecycle.

The latest threshold decision for the filter’s current version is its standing decision. It is what every move into active checks, what a resume re-affirms, and what a fall-back withdraws.

Approvals on a filter already filtering

Three kinds of proposal arrive while a filter is active, and the filter keeps filtering on what it runs now while they wait:

a retrained model, which the approval swaps in for the running one (Replace the running model with this one);
a stage’s first model joining the stages that filter now, for example a Decide stage added behind an active Collapse repeats. The choice reads “stage” joins, filtering, and the stage filters below its own threshold behind what already runs;
a waiting version that changes a pipeline stage, which the approval swaps in for the running version.

Rejecting any of them leaves the running model, version and approval as they were.