How predictions reach users.
WHAT THE PATTERNS ARE
Batch scoring, written to a table An interface serving predictions on request Embedded in an application On the device itself Streaming, scoring events as they arrive
WHAT BATCH SCORING SUITS
Predictions that need not reflect the last moment.
WHY IT IS PREFERABLE WHERE IT FITS
No latency requirement, no serving infrastructure, trivially reprocessed.
WHAT REQUEST-BASED SERVING REQUIRES
An interface, and infrastructure to run it Latency within budget Availability guarantees
WHAT EMBEDDED DEPLOYMENT SUITS
Small models, and applications avoiding a network call.
WHAT ON-DEVICE DEPLOYMENT PROVIDES
No data leaving the device Working offline No per-request cost
WHY THAT MATTERS HERE
No connectivity requirement, and no metered data consumed.
WHAT IT COSTS
Model size constraints Update distribution
WHAT TO ESTABLISH BEFORE CHOOSING
How fresh the prediction must be, and what latency the caller tolerates.
WHAT TO PREFER BY DEFAULT
The simplest pattern meeting the requirement.
WHAT TO AVOID
Building serving infrastructure for a batch requirement.
