checklists.oneAndroid appSign in
‹ Study plans

Data engineer study plan

From SQL and Python to pipelines, a warehouse and a project you can show.

23 items · 5 sections · study, career, tech, programming

Start this checklist

Core skills (5)

  • Learn SQL: joins, group by, window functionsOpen
  • Work through the PostgreSQL tutorialOpen
  • Read the Python tutorialOpen
  • Do the pandas getting-started guidesOpen
  • Learn Git with the free Pro Git bookOpen

Tools (4)

  • Run services in containers: do the Docker get-startedOpen
  • Query files fast with DuckDBOpen
  • Learn the Linux command line: grep, cut, sort, cron
  • Build a script that downloads a CSV, cleans it and loads it into Postgres

Pipelines (5)

  • Learn Apache Airflow concepts: DAGs, tasks, schedulesOpen
  • Model data with dbt: staging and martsOpen
  • Add checks for data qualityOpen
  • Make your pipeline safe to run twice without duplicates
  • Add alerts for a failed run

Bigger data (4)

  • Try Spark SQL on a few million rowsOpen
  • Read the Kafka docs on streams and topicsOpen
  • Learn star schema, fact and dimension tables
  • Learn partitions and file formats such as Parquet

Project and job (5)

  • Follow the free Data Engineering ZoomcampOpen
  • Build one pipeline from a public API to a dashboard and put it on GitHub
  • Write a README with a diagram of how data moves
  • Practise 20 SQL interview questions
  • Apply for junior data roles and analytics engineer roles

Written from github.com

Similar