Mailing-List: contact dev-help@airflow.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@airflow.incubator.apache.org
MIME-Version: 1.0
From: David Batista <dba@hellofresh.com>
Date: Thu, 11 May 2017 14:54:05 +0200
Message-ID: <CAOAhhjH1+-oiH3YuHCqFvmpBPn+7m5ZEmHcnooAe-s68fViOYg@mail.gmail.com>
Subject: Halt other DAGs until a certain DAG has completed
To: dev@airflow.incubator.apache.org
Content-Type: multipart/alternative; boundary="001a1148f920e882ab054f3f1743"
archived-at: Thu, 11 May 2017 12:54:53 -0000

--001a1148f920e882ab054f3f1743
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi everyone,

I asked a similar question before, but I thought about rephrasing/detailing
it with an example, maybe someone else has this issue:

Let's say I have DAG_1 running and for some reason (e.g.: due to failing
and re-tries, took some more time due to huge amounts of data, etc.)
Airflow triggers DAG_2 because the time that matches the crontab for DAG_2
was reached.

What happens now, is that Airflow will start running tasks from bot DAG_1
and DAG_2. In a scenario where there are limited resources, I would like to
put all tasks on DAG_2 on hold until DAG_1 finishes completely.

I know that one way to do this is to have an External Sensor on DAG_2
poking for the success of DAG_1. But, is there a way to tell Airflow, that
even if the time reached to run other DAGs, if a specific DAG still has not
complete put everything else on hold?

Cheers,
David

--=20

[image: logo]  =20
  <http://www.facebook.com/hellofreshde>   <http://twitter.com/HelloFreshde=
>
   <http://instagram.com/hellofreshde/>   <http://blog.hellofresh.de/>  =20
<https://app.adjust.com/ayje08?campaign=3DHellofresh&deep_link=3Dhellofresh=
%3A%2F%2F&post_deep_link=3Dhttps%3A%2F%2Fwww.hellofresh.com%2Fapp%2F%3Futm_=
medium%3Demail%26utm_source%3Demail_signature&fallback=3Dhttps%3A%2F%2Fwww.=
hellofresh.com%2Fapp%2F%3Futm_medium%3Demail%26utm_source%3Demail_signature=
>
=20
*HelloFresh App =E2=80=93Download Now!*=20
<https://app.adjust.com/ayje08?campaign=3DHellofresh&deep_link=3Dhellofresh=
%3A%2F%2F&post_deep_link=3Dhttps%3A%2F%2Fwww.hellofresh.com%2Fapp%2F%3Futm_=
medium%3Demail%26utm_source%3Demail_signature&fallback=3Dhttps%3A%2F%2Fwww.=
hellofresh.com%2Fapp%2F%3Futm_medium%3Demail%26utm_source%3Demail_signature=
>
*We're active in:*=20
US <https://www.hellofresh.com/?utm_medium=3Demail&utm_source=3Demail_signa=
ture>
 |  DE=20
<https://www.hellofresh.de/?utm_medium=3Demail&utm_source=3Demail_signature=
> | =20
UK=20
<https://www.hellofresh.co.uk/?utm_medium=3Demail&utm_source=3Demail_signat=
ure>=20
|  NL=20
<https://www.hellofresh.nl/?utm_medium=3Demail&utm_source=3Demail_signature=
> | =20
AU=20
<https://www.hellofresh.com.au/?utm_medium=3Demail&utm_source=3Demail_signa=
ture>
 |  BE=20
<https://www.hellofresh.be/?utm_medium=3Demail&utm_source=3Demail_signature=
> | =20
AT <https://www.hellofresh.at/?utm_medium=3Demail&utm_source=3Demail_signat=
ure>=20
|  CH=20
<https://www.hellofresh.ch/?utm_medium=3Demail&utm_source=3Demail_signature=
> |=20
CA <https://www.hellofresh.ca/?utm_medium=3Demail&utm_source=3Demail_signat=
ure>=20

www.HelloFreshGroup.com=20
<http://www.hellofreshgroup.com/?utm_medium=3Demail&utm_source=3Demail_sign=
ature>
=20
We are hiring around the world =E2=80=93 Click here to join us=20
<https://www.hellofresh.com/jobs/?utm_medium=3Demail&utm_source=3Demail_sig=
nature>

--=20

<https://www.hellofresh.com/jobs/?utm_medium=3Demail&utm_source=3Demail_sig=
nature>
HelloFresh SE, Berlin (Sitz der Gesellschaft) | Vorst=C3=A4nde: Dominik S.=
=20
Richter (Vorsitzender), Thomas W. Griesel, Christian G=C3=A4rtner | Vorsitz=
ender=20
des Aufsichtsrats: Jeffrey Lieberman | Eingetragen beim Amtsgericht=20
Charlottenburg, HRB 182382 B | USt-Id Nr.: DE 302210417

*CONFIDENTIALITY NOTICE:* This message (including any attachments) is=20
confidential and may be privileged. It may be read, copied and used only by=
=20
the intended recipient. If you have received it in error please contact the=
=20
sender (by return e-mail) immediately and delete this message. Any=20
unauthorized use or dissemination of this message in whole or in parts is=
=20
strictly prohibited.

--001a1148f920e882ab054f3f1743--