All Courses Automate Workflows with Airflow
Automate Workflows with Airflow
Understand the core concepts of Airflow, including DAGs, Operators, and Hooks. Learn to design, schedule, and monitor data pipelines efficiently using Airflow. Gain proficiency in defining workflows using Python code and Airflow's TaskFlow API. Master the configuration of connections, sensors, and triggers to orchestrate complex data workflows. Acquire skills in integrating Airflow with cloud services like AWS S3, Google Cloud Storage, and databases like PostgreSQL and MySQL. Develop the ability to troubleshoot, debug, and optimize Airflow workflows for performance and reliability.
Instructors:
SJ Sushmitha J
Start Learning
You have opted to be notified for this course. You will receive an email when the course becomes available.
2 Enrolled
19 Lessons

This comprehensive course on Airflow offers a deep dive into the world of workflow orchestration and automation. Starting with fundamental concepts, participants will progress through hands-on exercises and real-world scenarios to become proficient Airflow users. The course begins with an exploration of Airflow's architecture and components, including Directed Acyclic Graphs (DAGs), Operators, and Hooks. Participants will learn to design and schedule workflows using Python code and the TaskFlow API, gaining insights into best practices for organizing and structuring workflows efficiently. By the end of the course, you will have the skills and knowledge to confidently design, deploy, and manage robust data pipelines using Airflow, paving the way for enhanced productivity and efficiency in their data engineering workflows.



Course Content
Airflow introduction and installation
run Airflow 2.0 in docker container
Airflow Core Concepts in 5 mins
Airflow Task Lifecycle and Architecture
DAG_ create dag with bash operator!
Airflow Python Operator and XCom
Airflow TaskFlow API
Airflow DAG Catchup and Backfill Video
Airflow Scheduler with Cron Expression
Airflow Connection connect to Postgres
Airflow docker install python packages
Airflow AWS S3 Sensor Operator
Airflow Hooks S3 PostgreSQL
5 MUST KNOW Airflow debug tips and tricks
How to REMOVE airflow example DAGs locally and in docker
How to create S3 connection for AWS and MinIO
Airflow Email Notification on Failure
End-to-End Machine Learning Pipeline with Docker Operator
Airflow Skills Review
Reviews
Review the course
Help us improve our course material.
Write a review