airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Riccomini <criccom...@apache.org>
Subject Re: Airflow 2.0
Date Thu, 14 Dec 2017 22:19:08 GMT
Agree with Bolke that it's a good idea, but major work. I will push back on
this for 2.0 mostly due to time concerns. I don't want 2.0 to take months
and months to get out.

> Btw: what about dropping sqlite support?

I'm fine with this, but isn't this really useful for demos? Seems to me
that the out of the box experience from airflow is one of the things that
make sit so compelling.

On Thu, Dec 14, 2017 at 2:12 PM, Bolke de Bruin <bdbruin@gmail.com> wrote:

> That will take quite some work. It is a a good idea but also a major
> change. Not sure if we should target that.
>
> Btw: what about dropping sqlite support?
>
> Verstuurd vanaf mijn iPad
>
> > Op 14 dec. 2017 om 21:19 heeft Gael Magnan <gaelmagnan@gmail.com> het
> volgende geschreven:
> >
> > Hi,
> >
> > haven't been following much lately but on the import side of things,
> isn't
> > Airflow 2 the best moment to change to a pip plugin system for imports of
> > third party stuff?
> > I.E being able to add a new type of credentials, operator etc.. without
> > touching to the airflow code itself or having them in a special folder.
> >
> > Regards
> > Gael
> >
> >
> >
> > Le jeu. 14 déc. 2017 à 14:17, Chris Riccomini <criccomini@apache.org> a
> > écrit :
> >
> >> I'm fine with sensor refactor. Added to Wiki.
> >>
> >> On Thu, Dec 14, 2017 at 11:16 AM, Chris Riccomini <
> criccomini@apache.org>
> >> wrote:
> >>
> >>> @Bolke,
> >>>
> >>>> Should we, before 2.0, start the graduation from the incubator?
> >>>
> >>> No, I'd rather keep them separate. We can certainly start graduation,
> but
> >>> I don't want to block 2.0. Can pursue them in parallel.
> >>>
> >>> On Thu, Dec 14, 2017 at 11:15 AM, Andy Hadjigeorgiou <
> >> andyxhadji@gmail.com
> >>>> wrote:
> >>>
> >>>> Does it make sense to include sensors.py refactor in 2.0, so we can
> >>>> retire the old import structure easily and support the new sensors
> >> package
> >>>> import structure?
> >>>>
> >>>> - Andy
> >>>>
> >>>> On Thu, Dec 14, 2017 at 2:12 PM, Driesprong, Fokko
> <fokko@driesprong.frl
> >>>
> >>>> wrote:
> >>>>
> >>>>> Hi all,
> >>>>>
> >>>>> Good initiative. I would be happy to refactor the sensors package.
I
> >>>>> started on it but it changes a lot, all the imports will break.
> >>>>>
> >>>>> https://github.com/apache/incubator-airflow/pull/2875
> >>>>>
> >>>>> What do you guys think?
> >>>>>
> >>>>> Cheers, Fokko
> >>>>>
> >>>>> 2017-12-14 20:09 GMT+01:00 Chris Riccomini <criccomini@apache.org>:
> >>>>>
> >>>>>> I have created a wiki here:
> >>>>>>
> >>>>>> https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+2.0
> >>>>>>
> >>>>>> To track features and progress.
> >>>>>>
> >>>>>> On Thu, Dec 14, 2017 at 11:08 AM, Chris Riccomini <
> >>>>> criccomini@apache.org>
> >>>>>> wrote:
> >>>>>>
> >>>>>>>> Re: #2: Is there a current ticket out for removing the
legacy
> >>>>> import
> >>>>>>> style?
> >>>>>>>
> >>>>>>> No, I don't think so, but you can create one! :)
> >>>>>>>
> >>>>>>> On Thu, Dec 14, 2017 at 11:06 AM, Andy Hadjigeorgiou <
> >>>>>> andyxhadji@gmail.com
> >>>>>>>> wrote:
> >>>>>>>
> >>>>>>>> This sounds great, something I'd like to see updated
for 2.0
> >>>>> release (or
> >>>>>>>> before) is the Airflow documentation
> >>>>>>>> <http://airflow.readthedocs.io/en/latest/installation.html>
(
> >>>>>>>> http://airflow.readthedocs.io/en/latest/installation.html).
It
> >>>>> seems
> >>>>>> that
> >>>>>>>> updating the repo does not update this site - and given
that we
> >>>>> will be
> >>>>>>>> removing certain deprecated features I imagine the docs
will
> >> change
> >>>>>>>> substantially.
> >>>>>>>>
> >>>>>>>> Re: #2: Is there a current ticket out for removing the
legacy
> >> import
> >>>>>>>> style?
> >>>>>>>> I'm happy to help drive that forward.
> >>>>>>>>
> >>>>>>>> - Andy
> >>>>>>>>
> >>>>>>>> On Thu, Dec 14, 2017 at 1:55 PM, Chris Riccomini <
> >>>>> criccomini@apache.org
> >>>>>>>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hey all,
> >>>>>>>>>
> >>>>>>>>> With 1.9.0 wrapping up soon (hopefully), there's
been some
> >>>>> discussion
> >>>>>> on
> >>>>>>>>> the having the next release be Airflow 2.0 (rather
than 1.10).
> >>>>> This
> >>>>>>>> would
> >>>>>>>>> allow us to break compatibility, and clean up some
stuff.
> >> Proposed
> >>>>>>>> things
> >>>>>>>>> to include in 2.0 are:
> >>>>>>>>>
> >>>>>>>>> 1) New webserver that Joy Gao has been working on.
> >>>>>>>>> 2) Remove the legacy import style that's been deprecated
since
> >> at
> >>>>>> least
> >>>>>>>> 1.8
> >>>>>>>>> 3) New timzone feature
> >>>>>>>>> 4) Move API out of experimental
> >>>>>>>>>
> >>>>>>>>> I want to keep the list fairly tight, preferably
to things that
> >>>>> have
> >>>>>>>>> already been done, so that we can ship it fairly
quickly (in the
> >>>>> next
> >>>>>>>>> couple of months).
> >>>>>>>>>
> >>>>>>>>> Does this sound like a good list?
> >>>>>>>>>
> >>>>>>>>> Cheers,
> >>>>>>>>> Chris
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message