Construction and fitting.
WHAT TO START WITH
The simplest model that could work.
WHY
It establishes a baseline, and frequently suffices.
WHAT A BASELINE IS
A simple approach the model must beat to be worth anything.
WHAT THE TRAINING PROCESS INVOLVES
Defining the model Choosing a loss function matching the task Choosing an optimiser Training over epochs Evaluating on held-out data
WHAT TO WATCH DURING TRAINING
Training and validation performance together.
WHAT DIVERGENCE INDICATES
Overfitting: the model memorising rather than generalising.
WHAT REDUCES IT
More data Simpler models Regularisation Stopping early Augmenting the data
WHAT UNDERFITTING LOOKS LIKE
Poor performance on both training and validation data.
WHAT THAT INDICATES
A model too simple, or insufficient training.
WHAT TO TUNE
Learning rate, first. It affects results more than almost anything.
WHAT TO RECORD FOR EVERY RUN
The configuration, the data version, and the results.
WHY
Otherwise you cannot reproduce or compare anything.
