The whole category in one page.
GENERALISATION IS THE ONLY THING THAT MATTERS
Training performance is trivially achievable and worthless. Everything — splitting, regularisation, honest evaluation — exists to protect the estimate of how a model behaves on data it has not seen.
LEAKAGE IS THE MOST DAMAGING ERROR IN THE DISCIPLINE
The model tests excellently and fails completely. Establish when every feature's value becomes available, and treat performance that seems too good as a symptom rather than a success.
BUILD A BASELINE, AND MAKE EVERYTHING JUSTIFY ITSELF AGAINST IT
Many models do not beat the simple rule or the previous value. Knowing that early saves months.
ERROR ANALYSIS BEATS METRIC CHASING
Metrics say how much is wrong. Only examining the failures says what to fix.
ASK WHAT DECISION CHANGES BECAUSE OF THE PREDICTION
If none does, accuracy is irrelevant. And most projects end for want of a production path, not model quality — so build that path first, with a trivial model.
MODELS DECAY BECAUSE THE WORLD MOVES
Monitor input distributions, since outcomes arrive late or never. Investigate drift before retraining, or you entrench the cause.
DEPLOYED MODELS AFFECT PEOPLE
Document intended use and limits, measure performance by group, keep a genuine human path for consequential decisions, and be willing to say a model should not ship.
