Recipes

A recipe is one of Sift’s fixed techniques for judging events. Every stage in a filter’s pipeline is a recipe plus its settings. This page lists the eight recipes, the slot each fills, how many labels each needs, and the settings you are most likely to choose, so you can pick one for a stage.

Why recipes are fixed

Sift never writes code for a filter. A recipe is a built-in technique whose behaviour is set entirely by parameters stored in the filter’s spec: which fields it reads, how alike two events must be, how many events to escalate an hour. That keeps every filter auditable. A reviewer can read exactly what a filter does from its spec, a rule is data that can be shown and stored in the audit log, and the same spec always means the same behaviour. The assistant that proposes a filter chooses among these recipes and sets their parameters; it cannot add a new one.

The eight recipes

Listed in the console’s order, from the ones that need nothing to the ones that need the most history.

Console title	Recipe id	Slot	Minimum labels	Operating point	What it does
Collapse repeats	dedup_cluster	Collapse repeats	0	Fixed	Groups near-identical events; one from each burst reaches the agent and the rest are absorbed.
Fixed rule	rule_prefilter	Decide or Always escalate	0	Fixed	Conditions on event fields. No model: for when the rule is already known.
Plain-language rules	decision_rules	Always escalate	0	Fixed	Yes/no questions a decision model asks of each filtered event, plus field rules. Never filters.
Text analysis (small sample)	text_clf_setfit	Decide	50	Threshold	Learns from a small set of labels what in an event’s text matters, for a cold start on text.
Text analysis	text_clf_embed_lr	Decide	300	Threshold	Learns from the labels what in an event’s text marks it actionable.
Anomaly detection	anomaly_novelty	Decide	0	Hourly budget	Escalates the events least like normal traffic, a fixed number per hour.
Structured field analysis	tabular_gbdt	Decide	500	Threshold	Learns from the labels which field values mark an event actionable: auth, netflow or EDR telemetry.
Group escalations	digest	Shape delivery	0	Fixed	Sends escalations that arrive together in one agent run. Never filters.

The operating point column says how the stage’s decision is set:

• Threshold. The recipe scores each event between 0 and 1, and Decide escalates events at or above an approved threshold. Sift measures the trade-off on held-out labels and proposes a point.
• Hourly budget. The recipe ranks events, and the most unusual N each hour escalate. How unusual an event is has no fixed scale, so a budget holds steadier than a threshold would.
• Fixed. The recipe’s own rule is the decision: a condition matches or it does not, an event repeats or it does not. There is no threshold to tune, but a stage still needs an approval before it filters.

See Operating point and approval.

Minimum labels and the label floor

Recipes that learn from the agent’s verdicts need labels before Sift can train them. A filter’s label floor is the higher of two numbers: the filter’s own Min labels to train (Settings › General, 400 by default) and the recipe’s minimum from the table. So a filter on Text analysis (small sample) still waits for 400 labels unless you lower Min labels to train; you can lower it as far as 50, the recipe’s minimum, and no further. See Verdicts, labels and training.

Each recipe

Collapse repeats (dedup_cluster)

Use it when the source sends bursts of near-identical events: an alert storm, a log line repeated a thousand times. Sift compares each event’s text with recent events and folds it into an earlier one when they are alike enough. The event that escalates tells the agent how many it stands for, so a storm is triaged as a storm. It needs no labels, and a filter can filter with Collapse alone, approved on Collapse’s own evidence.

Setting	Default	What it controls
text_fields	none; required	The fields compared.
similarity_threshold	0.92 (0.5 to 1.0)	How alike two events must be. Higher folds only near-identical events.
window_minutes	60 (1 to 1,440)	How far back an event looks for one it repeats.
max_cluster_size	500 (2 to 10,000)	The most events one representative may stand for.

Fixed rule (rule_prefilter)

Use it when you already know the rule, and a model would only approximate it. It takes up to 50 conditions on event fields, each comparing a field with a value (eq, ne, gt, gte, lt, lte, in, not_in, contains, matches, exists), combined with all (the default) or any. In Decide, an event that matches escalates and the rest are filtered. In Always escalate, an event that matches escalates whatever the later stages decide.

Plain-language rules (decision_rules)

Always escalate only. Use it for things that must reach the agent and are easier to say in a sentence than in field conditions (“Does this mention a production database?”). Each rule is a question, up to 300 characters, that a decision model asks of each event an earlier stage filtered. The same stage can also hold field rules, which need no model.

Setting	Default	What it controls
Rules	at least one rule or field rule	Up to 10 plain-language rules and up to 10 field rules.
reach, per rule	normal	The most filtered events a rule may fire on: narrow 0.1%, normal 0.5%, wide 2%. It fires only on a yes, so often less.
max_rescues_per_hour	20 (1 to 200)	The most filtered events the rules may send on in an hour. Beyond it they are recorded, not sent.

It needs a decision model for new work. A filter that already has it keeps it if the model goes away; its plain-language rules pause and its field rules keep working.

Text analysis (small sample) (text_clf_setfit)

Use it for a text source when you want a model before there is enough history for Text analysis: it is meant for between 50 and 300 labels. It reads text_fields (required); its training settings are in the filter settings reference.

Training it needs an optional larger install, which an operator sets up (see Install without internet access). Any install can score a model that was already trained.

Text analysis (text_clf_embed_lr)

The usual choice for events whose meaning is in their text: tickets, alert descriptions, emails. It reads text_fields (required) and needs 300 labels. By default it is set for the usual case, where actionable events are rare.

Anomaly detection (anomaly_novelty)

Use it where labels cannot exist or are too rare to learn from, and “unlike normal” is what matters. It measures how unusual each event is compared with past traffic, over text_fields, numeric_fields or both.

Its operating point is top_n_per_hour, 50 by default: the most unusual events escalated each hour. The budget cannot exceed the filter’s trigger ceiling. In a filter’s first hour, or after a gap in traffic long enough that there is no recent ranking, it escalates everything rather than filter without a measurement. With no labels to hold out, its recall is measured live: in shadow every event still reaches the agent, so the agent’s verdicts on each ranked hour show what the budget would have missed. Until those verdicts include 20 actionable events, the recall is reported as not measured, and an approval rests on the budget’s cost alone.

Structured field analysis (tabular_gbdt)

Use it for structured telemetry where the signal is in field values, not prose. It reads numeric_fields and categorical_fields (at least one field is required) and needs 500 labels.

Group escalations (digest)

Shape delivery only. It groups escalations that arrive close together into one agent run, holds a group at most a stated wait, and never filters anything. You choose a preset:

Preset (console)	API value	Wait	Groups when	Events per run
No added delay	no_delay (default)	none	always, within one poll	25
Storms only	storms_only	300 s (5 minutes)	30 or more runs went out in the past hour	25
Short wait	short_wait	300 s (5 minutes)	always	25
Not urgent	not_urgent	900 s (15 minutes)	always	10
Custom	custom	you set all four

Outside the preset: group_by (up to 3 fields whose values decide which events share a run), max_chars 32,000 per run, and split_rate 0.02 (0.005 to 0.05), the share of events sent singly so Sift can measure whether grouping changes the agent’s verdicts. Grouping needs a paired agent that takes groups.

Which recipes you can use

Three things decide whether a recipe is offered for new work:

1. The build. Every recipe in the table is built and offered.
2. The install. Text analysis (small sample) needs an optional larger install to train. Plain-language rules need a decision model configured. Where one is missing, the console says why and what would fix it.
3. The administrator. An admin can turn any recipe off under the gear icon (Administration) › Filter recipes. A recipe that is off is not offered for new filters: Sift’s assistant does not propose it and the slate editor leaves it out.

Turning a recipe off refuses new work only. A filter already using it keeps scoring, retraining and saving versions.

Related pages

• The pipeline: slots and stages
• Compare recipes in a bake-off
• Filter recipes: turning recipes on and off.
• Filter settings reference: every parameter and range.
• Install without internet access: installing the setfit variant and its weights.