The summary.
GENERALISATION IS THE ONLY THING THAT MATTERS
Performance on training data is trivially achievable and worthless. Training performance far exceeding validation performance is overfitting; both being poor is underfitting.
More data reduces overfitting more reliably than any regularisation technique.
THE LOSS FUNCTION DEFINES WHAT THE MODEL BECOMES GOOD AT
Choose it wrongly and you get a model optimising something other than what you care about.
BUILD A BASELINE BEFORE TRAINING ANYTHING
The simplest thing that could work. It establishes whether a model adds anything — and many do not.
ERROR ANALYSIS IS THE MOST VALUABLE ACTIVITY AVAILABLE
Sample the wrong predictions, categorise them, and count. Metrics tell you how much is wrong; only this tells you what to fix.
Mislabelled training data caps achievable performance, so fix it rather than modelling around it.
ASK WHAT DECISION CHANGES BECAUSE OF THIS PREDICTION
If none does, the model has no value regardless of accuracy.
