airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Boris Tyukin <bo...@boristyukin.com>
Subject Re: Airflow dependency won't change
Date Thu, 17 Aug 2017 23:00:24 GMT
hopefully someone chimes in and explain. I just remember I've read to
change the dag name if schedule needs to be changed that's why I suggested
it. I've tried to swap order of tasks but I often add/delete tasks from
dags as I develop and that seems to be working fine without dag renaming.

Conceptually while it is okay to generate DAGs dynamically, it is not okay
to change DAG from time to time - tasks should be pretty much static as Max
explained in another thread.

On Thu, Aug 17, 2017 at 6:11 PM, Weiwei Zhang <vvianzhang@gmail.com> wrote:

> - Boris, You are right. After I change the dag id to something else, the
> dependency holds. I am very curious but why i cannot just switch the order
> and don't need to change the dag id. Thanks a lot!
>
>
>
> On Wed, Aug 9, 2017 at 10:21 AM, Boris Tyukin <boris@boristyukin.com>
> wrote:
>
> > Hit refresh button from UI to make sure it shows the proper order before
> > you run. you might also try to restart scheduler.
> >
> > if it does not help, try to rename your dag_id to something like
> mydag_v2.
> >
> > On Wed, Aug 9, 2017 at 12:48 PM, Weiwei Zhang <vvianzhang@gmail.com>
> > wrote:
> >
> > > Hi guys,
> > >
> > > I have two tasks in a DAG, t1 and t2. It used to be t2.set_upstream(t1)
> > and
> > > now I want to refactor the logic by setting t1.set_upstream(t2).
> However,
> > > when I try to run this DAG, it either will try to run two tasks
> > > simultaneously or it will try to run t2 first and towards the end, it
> > will
> > > also run t1 before t2 finishes. I am very confused about this behavior.
> > Am
> > > I missing something here? I am using Airflow 1.8.1.
> > >
> > > Thanks,
> > > -Weiwei
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message