airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Justin Palmer <jus...@github.com>
Subject Re: TriggerDagRunOperator sub tasks are scheduled to run after few hours later
Date Mon, 02 Oct 2017 22:38:44 GMT
Hi Sai,

At GitHub we had a similar issue with TriggerDagRunOperator.  In addition
to scheduling tasks in the future, we wanted a stable execution date that
was based on the triggering DAG.  We created a plugin that is basically a
copy of TriggerDagRunOperator, except it passes in the `execution_date` to
the triggered DAG.   I've linked the plugin below.  Feel free to use it
under the MIT license.

https://gist.github.com/Caged/f356430518247d1bbc2439a153e3c79e

-Justin

On Mon, Oct 2, 2017 at 2:02 PM jspsai@gmail.com <jspsai@gmail.com> wrote:

> Hi experts,
>
> I am running the TriggerDagRunOperator example that comes with Airflow.
>
> example_trigger_controller_dag
> example_trigger_target_dag
>
> I've created 2 new dags when are based on the above controller and trigger
> dags.
>
> when I run the example_trigger_controller_dag, i see that the
> examle_trigger_target_dag is scheduled to run after a few hours later, how
> to schedule it to run immediately?
> here is the logs where the Sub task that is scheduled to run at 17:53:31
> and the Base task has ran at 13:53:31.
>
> [2017-10-02 13:53:31,624] {base_task_runner.py:95} INFO - Subtask:
> [2017-10-02 13:53:31,623] {dagrun_operator.py:74} INFO - Creating DagRun
> <DagRun Trigger_Target_Dag @ 2017-10-02 17:53:31:
> trig__2017-10-02T13:53:31.248671, externally triggered: True>
>
> I might me doing something really stupid here.
> please help.
>
> Thanks
> -Sai.
>
>
>
>
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message