Create multiple task in Airflow

How to create and execute multiple tasks in airflow?

 If you are not familiar with airflow then first download airflow by clicking here and later on you can start by hello world program in airflow click here to see how we can write our first airflow program. So today we gone see how to execute multiple tasks in airflow let’s get started.

 First, we can see How we can sequentially execute multiple python task in airflow?

from airflow import DAG
from airflow.operators.python import PythonOperator
from datetime import datetime


def task1():
    print("Hello task1")

def task2():
    print("Hello task2")

def task3():
    print("Hello task3")

def task4():
    print("Hello task4")


with DAG(dag_id = 'hello_world_dag1', 
    start_date=datetime(2021,8,19),
    schedule_interval='@hourly',
    catchup=False) as dag:

    python_task1 = PythonOperator(
        task_id="python_task",
        python_callable=task1
    )

    python_task2 = PythonOperator(
        task_id='python_task2',
        python_callable=task2
    )

    python_task3 = PythonOperator(
        task_id='python_task3',
        python_callable=task3
    )

    python_task4 = PythonOperator(
        task_id='python_task4',
        python_callable=task4
    )

python_task1 >> python_task2 >> python_task3 >> python_task4

Here task1 will run first then task2 then after task3 and so on…

python_task1 >> [python_task2, python_task3] >> python_task4

Here task 1 will run first, then task 2 and task 3 will start simultaneously after that task 4 will start.

If you find any issue. Please let us know.

For more airflow blog click here

👍👍👍👍👍👍👍