airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Goutam Kumar Sahoo <goutamkumar.sa...@infosys.com>
Subject RE: TriggerDagRunOperator sub tasks are scheduled to run after few hours
Date Mon, 10 Sep 2018 05:59:19 GMT
HI All,

Could anyone please address this issue ? We are really stuck here and not able to proceed
further

Thanks & Regards
Goutam Sahoo

From: Goutam Kumar Sahoo
Sent: Friday, September 07, 2018 4:51 PM
To: 'dev@airflow.incubator.apache.org' <dev@airflow.incubator.apache.org>; 'dev@airflow.apache.org'
<dev@airflow.apache.org>
Subject: TriggerDagRunOperator sub tasks are scheduled to run after few hours

HI Experts

In our project, we are trying to replicate the existing  job scheduling implemented in Microsoft
Orchestrater to new scheduler called "Apache Airflow" .

During this replication process we have an requirement to create Master DAG which in turn
will call Other child DAGs based on different condition. We referred to the existing example
(example_trigger_controller_dag, example_trigger_target_dag) available in Airflow and tried
running them manually.

When we triggered the controller DAG manually , it is got triggered immediately with execution
time created in local timezone (PDT) whereas the child DAG is scheduled to run few hours later.
Even though child DAGRUN is showing as running, it is doing nothing and waiting for the time
to satisfy.  Our requirement is to trigger the Target/Child DAG run as soon it is triggered
by controller DAG

[cid:image001.png@01D448F9.7FEECCC0]

[cid:image002.png@01D448F9.7FEECCC0]
If you see here , the trigger time and execution time is having a 7 hours GAP.

We have tried to solve this many different ways but couldn't succeed. We have asked for help
in many forums but didn't get any satisfactory answer form any of them


Please help us ASAP as this issue is one of the show stopper for us .




Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message