Knowledgebase

Data Engineering Skills and Tools Print

  • dataengineering, data, database, guide, howto, solution, zillionkinghost, hosting
  • 0

What the work requires.

WHAT IS ESSENTIAL

SQL, thoroughly One programming language, usually Python Understanding of how databases work Version control Command line competence

WHY SQL FIRST

It is the common language of data, and most work is expressible in it.

WHAT ELSE MATTERS

Data modelling Understanding of distributed systems Scheduling and orchestration Cloud platform basics Testing practice

WHAT TOOLS EXIST, BY CATEGORY

Ingestion and connectors Transformation frameworks Orchestrators Warehouses and query engines Streaming platforms Quality and observability tools

WHAT TO BE CAREFUL WITH

Adopting many tools before establishing the need Tools requiring substantial operation for modest benefit

WHAT TO PREFER

Fewer tools, understood well.

WHAT MATTERS MORE THAN TOOLING

Whether the data is correct and arrives when expected.

WHAT EMPLOYERS ACTUALLY ASK ABOUT

SQL, modelling, and how you handled something that went wrong.

WHAT TO BUILD FOR A PORTFOLIO

A working pipeline, end to end, with tests and documentation.


Was this answer helpful?
Back

Are you happy with your experience? Leave us a review on Trustpilot.


Trustpilot