Getting models used.
WHAT DEPLOYMENT OPTIONS EXIST
A serving component behind an API Batch prediction on a schedule Embedded in an application, on device In the browser
WHAT TO CHOOSE BY
Latency requirements Whether predictions are needed individually or in bulk Whether data can leave the device Cost
WHAT SERVING PROVIDES
Models loaded and versioned, with a consistent interface.
WHAT TO VERSION
The model, alongside the code.
WHY
You must be able to establish which model produced a prediction.
WHAT TO LOG
Inputs, predictions and outcomes.
WHY OUTCOMES
Without them you cannot measure whether the model is still working.
WHAT TO MONITOR
Prediction latency Input distribution, compared with training data Performance, where outcomes become known
WHAT DRIFT IS
The data changing over time, so the model no longer fits.
WHY IT MATTERS
Models degrade silently, and nobody notices without monitoring.
WHAT TO PLAN
Retraining, on a schedule or triggered by drift.
WHAT TO PROVIDE
A way to revert to a previous model.
