The summary.
FIT A LINEAR OR LOGISTIC MODEL FIRST AND RECORD ITS PERFORMANCE
It is fast, interpretable and frequently competitive. Everything after must justify itself against it.
USE GRADIENT BOOSTING FOR TABULAR PROBLEMS
Deep learning rarely beats it there and costs far more.
TARGET ENCODING LEAKS UNLESS COMPUTED WITHIN FOLDS
And add an indicator for missingness — the fact that a value is absent is frequently informative in itself.
TIME SERIES VALIDATION MUST BE FORWARD-LOOKING
And always compare against the naive forecast: the last value, or the value a season ago. Many models fail to beat it.
EVALUATE RECOMMENDATIONS BY RANKING, NOT ERROR
And measure coverage and diversity — a system recommending only popular items scores well and serves nobody.
CHECK PERFORMANCE BY SEGMENT, NOT ONLY IN AGGREGATE
Aggregate figures conceal failure on important subgroups.
