How to create SUBDAG in airflow ?

We can put many tasks(operators) in one subdag. We can keep our dag clean by using subdag.

By clicking on the subdag which is section-1 here, we can see the tasks under it.

Which is look like as below,

We will create our own subdag. Before that you look at airflow subdag example once by clicking here. This will give you a basic idea about the subdag. let’s start creating subdag.

First create subdags directory inside dags folder

inside subdag_example.py

from airflow import DAG
from datetime import datetime
from airflow.operators.subdag import SubDagOperator
from airflow.operators.bash import BashOperator



def subdag_example(parent_dag_id, child_dag_id, default_args):

    with DAG(dag_id=f'{parent_dag_id}.{child_dag_id}', default_args=default_args) as dag:
        python_task2 = BashOperator(
            task_id='python_task2',
            bash_command='sleep 2'
            )

        python_task3 = BashOperator(
            task_id='python_task3',
            bash_command='sleep 2'
            )
        return dag

inside Hello_world_ex.py write below code

from airflow import DAG
from airflow.operators.python import PythonOperator
from datetime import datetime
from airflow.operators.subdag import SubDagOperator
from subdags.subdag_example import subdag_example

default_args={
    'start_date':datetime(2021,8,19)
}

def task1():
    print("Hello task1")


def task4():
    print("Hello task4")


with DAG(dag_id = 'hello_world_dag1', 
    default_args = default_args,
    schedule_interval='@hourly', catchup=False) as dag:

    python_task1 = PythonOperator(
        task_id="python_task",
        python_callable=task1
    )

    example_subdag = SubDagOperator(
        task_id = 'example_tasks',
        subdag = subdag_example('hello_world_dag1', 'example_tasks', default_args)
    )

    
    python_task4 = PythonOperator(
        task_id='python_task4',
        python_callable=task4
    )

    python_task1 >> example_subdag >> python_task4

in subDagOperator we pass subdag id and subdag function.

You can see more airflow blog by clicking here.