Finding the unusual.
WHAT IT IS
Identifying observations differing significantly from normal.
WHERE IT APPLIES
Fraud Security monitoring Equipment failure prediction Quality control Infrastructure monitoring
WHAT MAKES IT DIFFICULT
Anomalies are rare, so there is little to learn from Normal changes over time What counts as anomalous is context-dependent
WHAT APPROACHES EXIST
- Statistical: deviation from expected distribution
- Distance-based: far from other observations
- Model-based: poorly reconstructed by a model of normal
- Rule-based: known bad patterns
WHAT SEASONALITY MEANS
Regular variation by time of day, day of week, or season.
WHY IT MATTERS
Ignoring it flags every Monday morning as anomalous.
WHAT TO MODEL
Expected behaviour including that variation.
WHAT ALERT FATIGUE IS
So many alerts that they are ignored.
WHY IT IS THE PRINCIPAL FAILURE MODE
A detection system nobody acts on provides nothing.
WHAT TO TUNE FOR
The rate humans can actually investigate.
WHAT TO PROVIDE WITH EVERY ALERT
Context explaining why it fired.
WHAT TO REVIEW
Alerts that fired and were dismissed.
