Catching problems automatically.
WHAT TO TEST AT INGESTION
Schema matches expectation Row counts within bounds Required fields present Types valid
WHAT TO TEST AFTER TRANSFORMATION
Key uniqueness Referential integrity Accepted values Numerical ranges Aggregate reconciliation
WHAT TO DO ON FAILURE
Decide per test whether it warns or stops the pipeline.
WHAT SHOULD STOP IT
Anything that would publish wrong figures.
WHAT SHOULD WARN
Anomalies worth investigating that do not invalidate results.
WHAT ANOMALY DETECTION ADDS
Flagging values outside historical norms, without predefined rules.
WHAT IT CATCHES
Problems nobody anticipated.
WHAT IT COSTS
False alerts, requiring tuning.
WHAT TO BASELINE FROM
Enough history to capture normal variation, including seasonality.
WHY SEASONALITY
Otherwise every month end and every holiday is an anomaly.
WHAT TO RECORD
Test results over time.
WHY
So degradation is visible before it becomes a failure.
WHAT TO REVIEW
Tests that never fail, which may be testing nothing.
WHAT TO ADD AFTER EVERY INCIDENT
A test that would have caught it.
