airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sajan Ravindran <>
Subject Subdags getting created partially
Date Wed, 06 Jun 2018 17:17:13 GMT
Hi All,

We are trying to create a set of subdags, given a parent dag, which are to
be created and executed in batches.

The following piece of code creates the subdags:
def create_subdag(instr_test, check_batch_size, batch_start, batch_end):
    instr_subdag_func = SUB_DAG_PRE + "_" + str(instr_test)
    instr_subdag_id = SUB_DAG_PRE + "-" + str(instr_test)

    instr_subdag_func = SubDagOperator(
        subdag=instr_subdag_csv(MAIN_DAG_ID, instr_subdag_id, args,

    check_batch_size = check_batch_size + 1

    return check_batch_size

However, it seems that given a total no. of subdags to be created, not all
are gettng created. In our case, out of 54, only 38 are getting created.
Can someone please advise as to what might be going wrong ?

Also, do subdags also get listed out when we run the command : airflow
list_dags ? Just wanted to confirm if that is expected.


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message