The summary.
LEAKAGE IS THE MOST DAMAGING ERROR AVAILABLE
The model tests excellently and fails entirely in production. Every feature must reflect only what was known at the prediction moment.
Most warehouses store current state, which cannot answer that — build history with validity periods for anything used as a feature.
TRAINING AND SERVING MUST USE ONE FEATURE DEFINITION
Reimplementing feature logic for serving is how skew happens, and the model then receives different values than it learned from.
Monitor feature distributions in production against training.
PREFER BATCH PREDICTION WHERE IT FITS
No latency requirement, no serving infrastructure, easy to reprocess. And never substitute a missing feature silently — record the default or refuse.
Log every prediction with its features; it is the only way to investigate a wrong one later.
NEVER SHIP A NOTEBOOK AS A PRODUCTION PIPELINE
Cells run out of order and state persists invisibly.
CHUNKING THAT SPLITS A CONCEPT IN HALF IS THE COMMONEST CAUSE OF BAD RETRIEVAL
Split on structure, and carry permissions with every chunk so retrieval cannot return what the asker may not see.
Changing the embedding model requires reprocessing the entire corpus — plan for it before it is needed.
ALWAYS RECORD THE COORDINATE REFERENCE SYSTEM, AND NEVER MEASURE DISTANCE IN DEGREES
