Serving requests.
WHAT THE INTERFACE MUST DO
Accept input Validate it Preprocess it Predict Postprocess Return a response
WHAT TO VALIDATE
Presence of required fields Types and ranges Category values
WHY STRICTLY
Invalid input produces a confident wrong answer, not an error.
WHAT TO RETURN
The prediction A confidence or probability, where meaningful The model version used
WHY THE VERSION
It is essential for investigating a disputed prediction later.
WHAT TO LOG
Input features, output, model version, latency, and a request identifier.
WHAT TO BE CAREFUL WITH IN LOGGING
Personal data, which must be handled accordingly.
WHAT TO CONFIGURE
Timeouts A maximum request size Rate limiting
WHAT BATCHING PROVIDES
Grouping concurrent requests for efficient inference.
WHAT IT COSTS
Slight latency, traded for throughput.
WHAT TO HANDLE
Model loading at startup, not per request Failure to load, clearly Graceful degradation when a feature source is unavailable
WHAT TO NEVER DO
Return a prediction based on silently substituted inputs.
