airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From srinivas.ramabhadran@gmail.com <srinivas.ramabhad...@gmail.com>
Subject Simple DAG Structure
Date Mon, 23 Jul 2018 00:17:52 GMT
Hi - 

   I have recently started using Airflow version 1.9.0 and am having some difficulty setting
up a very simple DAG. I have three tasks A, B and C. I'd like A to run every day at 10am and
B at 11am. C depends on BOTH A and B running successfully.

   Initially, I decided to create one DAG, add all three tasks to it and set C as downstream
to A and B. I then set the schedule_interval of the DAG to @daily. But this meant I couldn't
run A and B at 10am and 11am respectively since the they are PythonOperators and tasks dont
support schedule_interval (or, at least, it's deprecated syntax and gets ignored).

   I scratched that idea and then created A and B as DAGs, specified the schedule interval
as per the cron syntax: '00 10 * * *' for A and '00 11 * * *' for B.  But now when I set C
as a downstream of A and B, it complains that C can't belong to two different dags.

   How do I accomplish such a simple dependency structure? 

Ram.

Mime
View raw message