week_2_workflow_orchestration

Week 2: Workflow Orchestration

If you're looking for Airflow videos from the 2022 edition, check the 2022 cohort folder.

If you're looking for Prefect videos from the 2023 edition, check the 2023 cohort folder.

Python code from videos is linked below.

Data Lake (GCS)

• What is a Data Lake
• ELT vs. ETL
• Alternatives to components (S3/HDFS, Redshift, Snowflake etc.)
• Video
• Slides

Workflow Orchestration

Mage videos coming soon

Code repository

To be posted after the mage videos

Homework

To be posted after the mage videos

Community notes

Did you take notes? You can share them here.

• Add your notes here (above this line)

2023 notes

• Blog by Marcos Torregrosa (Prefect)
• Notes from Victor Padilha
• Notes by Alain Boisvert
• Notes by Candace Williams
• Notes from Xia He-Bleinagel
• Notes from froukje
• Notes from Balaji

2022 notes

Most of these notes are about Airflow, but you might find them useful.

• Notes from Alvaro Navas
• Notes from Aaron Wright
• Notes from Abd
• Blog post by Isaac Kargar
• Blog, notes, walkthroughs by Sandy Behrens
• Notes from Vincenzo Galante
• More on Pandas vs SQL, Prefect capabilities, and testing your data, by Vera