airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Meickle <>
Subject Re: [2.0 spring cleaning] Deprecate subdags
Date Fri, 12 Apr 2019 15:15:46 GMT
Given their bad reputation, would it be appropriate to deprecate subDAGs
now to advertise that they're no longer considered a suitable
implementation? If a new and better implementation is created, would it
even be similar enough to subDAGs that we'd want to continue to call the
feature that?

They feel like a "new Airflow user trap" right now - I have had to tell my
team never to use them, because they seem appealing and are in the official

On Fri, Apr 12, 2019 at 10:51 AM Ash Berlin-Taylor <> wrote:

> I'd like to find time to fix subdags as they do provide a useful
> abstraction - but I agree right now they aren't great (I avoid them because
> of this)
> I have half thoughts of how to it should work, I just need to look at the
> code in depth to see if that makes sense. Now 1.10.3 is out I might have a
> bit more time to do this.
> -ash
> > On 12 Apr 2019, at 15:48, James Meickle <>
> wrote:
> >
> > I think we should deprecate SubDAGs given the complexity they add and the
> > limited usage and use cases. Or, we should invest effort in redesigning
> > their API and implementation. I think that having to account for
> > subdag-introduced complexity makes Airflow's code much harder to maintain
> > and buggier, looking at how many open issues there are that reference
> > subdags (and how unrelated in topic they are):
> >

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message