The summary.
MOST DATA INCIDENTS ORIGINATE UPSTREAM, IN CHANGES NOBODY COMMUNICATED
Data contracts move that cost back to the producer. Where none exist, document your expectations and test against them — it turns a silent break into a detected one.
ADD A TEST AFTER EVERY INCIDENT THAT WOULD HAVE CAUGHT IT
And review tests that never fail — they may be testing nothing.
DATA NOBODY CAN FIND IS REBUILT REPEATEDLY, DIFFERENTLY EACH TIME
Which is what a catalogue prevents. Governance fails as a committee producing documents and works when built into the tooling.
NOT INGESTING A FIELD IS THE STRONGEST PRIVACY CONTROL AVAILABLE
Mask or tokenise at ingestion where analysis does not need the actual values, and build the ability to locate an individual's data before you are asked to delete it.
GIVE ANALYSTS MODELLED TABLES, NOT RAW DATA
Raw data requires knowledge they should not need. And warehouses are not built for the many small low-latency queries applications make.
DELIVERING A CORRECT ANSWER TO A BADLY FRAMED QUESTION IS A FAILURE
Establish what decision the data informs, and push back on real-time requests with no real-time decision.
BUILD ONE PIPELINE THAT SOMEONE ACTUALLY USES BEFORE CHOOSING A STACK
A technically excellent platform nobody trusts has failed.
