From the course: Learning Apache Airflow
Unlock this course with a free trial
Join today to access over 23,400 courses taught by industry experts.
Python operators and dependencies - Apache Airflow Tutorial
From the course: Learning Apache Airflow
Python operators and dependencies
- [Narrator] Now in this movie, we'll see that dependencies with Python operators work exactly like dependencies with pash operators. In fact, dependencies work the same way with all operators that you use to specify tasks. In this particular DAG we'll be creating multiple Python operators. Once again, on line eight, I have an import for the Python operator and I have four different Python functions defined on lines 14 through 25. The functions are called task A, B, C, and D. And within each function we have a simple Python print statement indicating that the task has been executed. These are the four callables that our Python operators will invoke. Let's instantiate the DAG in exactly the same way as before. Nothing interesting there. I've instantiated four different tasks using the Python operator tasks, A, B, C, and D. Each task has its own unique task ID and each task takes in a Python callable as an input argument.…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Contents
-
-
-
-
-
-
(Locked)
Introducing the Python operator3m 14s
-
(Locked)
Python operators and dependencies2m 47s
-
(Locked)
Passing parameters to Python callables3m 17s
-
(Locked)
Introducing XCom4m 15s
-
(Locked)
Using XCom to pass values5m 53s
-
(Locked)
Implementing a data transformation DAG pipeline, part 16m 30s
-
(Locked)
Implementing a data transformation DAG pipeline, part 24m 13s
-
(Locked)
-
-
-
-