airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Gong <go...@hotmail.com>
Subject Re: airflow dags share the same database?
Date Thu, 05 Jan 2017 15:23:19 GMT
Hi, Bolke,


Thanks for the reply.


Can you clarify some points ?



>* SQLSensor pointing to the other database

Do you mean after task2(dag2) is done, it run a SQLSensor to update the db for dag1 ?

> * Put a file somewhere, where it is being polled by the other dag/task

> * Trigger the dag1 from task2 (easier with the Rest API in master)

Do you mean task2 can trigger the entire dag1 or can trigger the task1 in dag1 only ?

Thanks.
Michael


________________________________
From: Bolke de Bruin <bdbruin@gmail.com>
Sent: Wednesday, January 4, 2017 8:32 PM
To: dev@airflow.incubator.apache.org
Subject: Re: airflow dags share the same database?

Some suggestions:

* SQLSensor pointing to the other database
* Put a file somewhere, where it is being polled by the other dag/task
* Trigger the dag1 from task2 (easier with the Rest API in master)

Bolke

> On 4 Jan 2017, at 21:08, Michael Gong <gonwg@hotmail.com> wrote:
>
> Hi,
>
>
> Say due to some policy, I have to run 2 dag instances on 2 different servers.
>
>
> If I want task 1 from dag1 depends on task 2 from dag2, how can I do it ?
>
>
> One idea I can  think of is that the 2 dag instances share the same MySQL db, will it
work ?
>
>
>
> Any ideas are welcomed.
>
>
> thanks.
>
> Michael
>
>
>
>
>
>
>
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message