airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guillermo Rodríguez Cano <wsch...@gmail.com>
Subject Re: Regarding airflow scheduler starting PID periodically.
Date Thu, 10 May 2018 06:39:40 GMT
Hello,

I am not sure if I understood... you want to trigger DAGs based on the PIDs
of the tasks?

As you say (I am not an expert on OSes here) you don't control this.

I think that the TriggerDagRunOperator (https://airflow.apache.org/co
de.html#airflow.operators.TriggerDagRunOperator) would work for you. Right?

/Guillermo


On Tue, May 8, 2018 at 5:06 PM nitish sharma <nitishcse412@gmail.com> wrote:

> Hi Team,
>
> I have a situation where i want to create dag with unique dag id and run
> that dag once as soon as it is created.
>
> For eg :  1.py , 2.py are two different dags. 1.py (has 3 tasks) and
> 2.py(has 2 tasks).
>
> Once these are run i would have next set of dags with me for eg 3.py, 4.py,
> 5.py and so on.
>
> I see a challenger here as scheduler keeps generating process ID to
> generate tasks
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message