The summary.
PREFER BATCH SCORING WHERE IT FITS
No latency requirement, no serving infrastructure, trivially reprocessed. Do not build serving infrastructure for a batch requirement.
PREPROCESSING MUST TRAVEL WITH THE MODEL
Otherwise the model receives values unlike what it learned from. Package them as one unit.
Never deploy a model whose provenance is unknown — and loading an untrusted model file can execute code.
VALIDATE INPUT STRICTLY
Invalid input produces a confident wrong answer, not an error. And never return a prediction based on silently substituted values.
Always return the model version; it is essential for investigating a disputed prediction later.
PROFILE THE WHOLE PATH — PREPROCESSING IS FREQUENTLY SLOWER THAN THE MODEL
And establish whether accelerated hardware is justified at your volume. It frequently is not.
BEHAVIOURAL TESTS CATCH WHAT AGGREGATE METRICS NEVER REVEAL
Invariance to irrelevant changes, correct direction on relevant ones.
SHADOW DEPLOYMENT GIVES REAL-WORLD BEHAVIOUR WITH NO RISK
And never deploy without the ability to revert immediately.
