airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chen Tong <cix...@gmail.com>
Subject Re: [2.0 spring cleaning] Deprecate subdags
Date Fri, 12 Apr 2019 15:00:17 GMT
SubDAG is really useful in my use case, otherwise the graph will be hard to
be managed. I guess it's better come out a better design rather than
deprecate this feature.



On Fri, Apr 12, 2019 at 10:51 AM Ash Berlin-Taylor <ash@apache.org> wrote:

> I'd like to find time to fix subdags as they do provide a useful
> abstraction - but I agree right now they aren't great (I avoid them because
> of this)
>
> I have half thoughts of how to it should work, I just need to look at the
> code in depth to see if that makes sense. Now 1.10.3 is out I might have a
> bit more time to do this.
>
> -ash
>
> > On 12 Apr 2019, at 15:48, James Meickle <jmeickle@quantopian.com.INVALID>
> wrote:
> >
> > I think we should deprecate SubDAGs given the complexity they add and the
> > limited usage and use cases. Or, we should invest effort in redesigning
> > their API and implementation. I think that having to account for
> > subdag-introduced complexity makes Airflow's code much harder to maintain
> > and buggier, looking at how many open issues there are that reference
> > subdags (and how unrelated in topic they are):
> >
> https://issues.apache.org/jira/browse/AIRFLOW-3292?jql=project%20%3D%20AIRFLOW%20AND%20status%20%3D%20Open%20AND%20text%20~%20%22subdag%22
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message