The summary.
ESTABLISH WHETHER MACHINE LEARNING IS THE RIGHT APPROACH AT ALL
Rules, a lookup or a simpler statistical method are frequently better — and a ready-made API may remove the need for any model work.
Start with an existing model before training your own; transfer learning makes useful models possible from thousands of examples rather than millions.
THE DATA IS THE PROBLEM, NOT THE MODEL
Preparation consumes the majority of the effort, and split before fitting any transformation.
Leakage — anything from the test set influencing training — produces models that look excellent and fail in production.
ACCURACY MISLEADS WITH IMBALANCED CLASSES
Choose metrics by which error is more costly, examine the confusion matrix, and evaluate across segments — a model can perform well overall and badly for a group.
Set the deployment threshold before training, or it moves to whatever the model achieved.
MODELS DEGRADE SILENTLY
Log inputs, predictions and outcomes, monitor input distribution against training data, and plan retraining.
ON-DEVICE INFERENCE COSTS NO DATA AND WORKS OFFLINE
Measure accuracy after conversion and inference time on the oldest device you support.
UNDEPLOY ENDPOINTS NOT IN USE — THEY BILL WHETHER OR NOT THEY RECEIVE REQUESTS.
