The discipline and its purpose.
WHAT IT IS
Building and operating the systems that collect, move, store and prepare data so others can use it.
WHAT THAT INCLUDES
Extracting data from source systems Moving it reliably Transforming it into usable shape Storing it appropriately Making it available to analysts, applications and models Keeping all of that working
WHAT DISTINGUISHES IT FROM DATA ANALYSIS
Analysis answers questions from data. Engineering makes the data available to be asked.
WHAT DISTINGUISHES IT FROM SOFTWARE ENGINEERING
The same practices apply, but the failure modes differ: data is wrong rather than code crashing.
WHY THAT MATTERS
A pipeline that runs successfully and produces wrong numbers is worse than one that fails, because nobody notices.
WHAT THE CENTRAL RESPONSIBILITY IS
That the numbers people act on are correct.
WHAT MOST OF THE WORK ACTUALLY IS
Moving data between systems, reconciling differences, and handling the cases nobody documented.
WHAT THE UNGLAMOROUS MAJORITY IS
Dealing with source systems that change without warning.
WHAT TO LEARN FIRST
SQL, thoroughly How data is modelled How to build something that runs reliably every day
