airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bolke de Bruin <bdbr...@gmail.com>
Subject Re: Airflow 2.0
Date Thu, 14 Dec 2017 22:12:39 GMT
That will take quite some work. It is a a good idea but also a major change. Not sure if we
should target that.

Btw: what about dropping sqlite support?

Verstuurd vanaf mijn iPad

> Op 14 dec. 2017 om 21:19 heeft Gael Magnan <gaelmagnan@gmail.com> het volgende
geschreven:
> 
> Hi,
> 
> haven't been following much lately but on the import side of things, isn't
> Airflow 2 the best moment to change to a pip plugin system for imports of
> third party stuff?
> I.E being able to add a new type of credentials, operator etc.. without
> touching to the airflow code itself or having them in a special folder.
> 
> Regards
> Gael
> 
> 
> 
> Le jeu. 14 déc. 2017 à 14:17, Chris Riccomini <criccomini@apache.org> a
> écrit :
> 
>> I'm fine with sensor refactor. Added to Wiki.
>> 
>> On Thu, Dec 14, 2017 at 11:16 AM, Chris Riccomini <criccomini@apache.org>
>> wrote:
>> 
>>> @Bolke,
>>> 
>>>> Should we, before 2.0, start the graduation from the incubator?
>>> 
>>> No, I'd rather keep them separate. We can certainly start graduation, but
>>> I don't want to block 2.0. Can pursue them in parallel.
>>> 
>>> On Thu, Dec 14, 2017 at 11:15 AM, Andy Hadjigeorgiou <
>> andyxhadji@gmail.com
>>>> wrote:
>>> 
>>>> Does it make sense to include sensors.py refactor in 2.0, so we can
>>>> retire the old import structure easily and support the new sensors
>> package
>>>> import structure?
>>>> 
>>>> - Andy
>>>> 
>>>> On Thu, Dec 14, 2017 at 2:12 PM, Driesprong, Fokko <fokko@driesprong.frl
>>> 
>>>> wrote:
>>>> 
>>>>> Hi all,
>>>>> 
>>>>> Good initiative. I would be happy to refactor the sensors package. I
>>>>> started on it but it changes a lot, all the imports will break.
>>>>> 
>>>>> https://github.com/apache/incubator-airflow/pull/2875
>>>>> 
>>>>> What do you guys think?
>>>>> 
>>>>> Cheers, Fokko
>>>>> 
>>>>> 2017-12-14 20:09 GMT+01:00 Chris Riccomini <criccomini@apache.org>:
>>>>> 
>>>>>> I have created a wiki here:
>>>>>> 
>>>>>> https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+2.0
>>>>>> 
>>>>>> To track features and progress.
>>>>>> 
>>>>>> On Thu, Dec 14, 2017 at 11:08 AM, Chris Riccomini <
>>>>> criccomini@apache.org>
>>>>>> wrote:
>>>>>> 
>>>>>>>> Re: #2: Is there a current ticket out for removing the legacy
>>>>> import
>>>>>>> style?
>>>>>>> 
>>>>>>> No, I don't think so, but you can create one! :)
>>>>>>> 
>>>>>>> On Thu, Dec 14, 2017 at 11:06 AM, Andy Hadjigeorgiou <
>>>>>> andyxhadji@gmail.com
>>>>>>>> wrote:
>>>>>>> 
>>>>>>>> This sounds great, something I'd like to see updated for
2.0
>>>>> release (or
>>>>>>>> before) is the Airflow documentation
>>>>>>>> <http://airflow.readthedocs.io/en/latest/installation.html>
(
>>>>>>>> http://airflow.readthedocs.io/en/latest/installation.html).
It
>>>>> seems
>>>>>> that
>>>>>>>> updating the repo does not update this site - and given that
we
>>>>> will be
>>>>>>>> removing certain deprecated features I imagine the docs will
>> change
>>>>>>>> substantially.
>>>>>>>> 
>>>>>>>> Re: #2: Is there a current ticket out for removing the legacy
>> import
>>>>>>>> style?
>>>>>>>> I'm happy to help drive that forward.
>>>>>>>> 
>>>>>>>> - Andy
>>>>>>>> 
>>>>>>>> On Thu, Dec 14, 2017 at 1:55 PM, Chris Riccomini <
>>>>> criccomini@apache.org
>>>>>>> 
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Hey all,
>>>>>>>>> 
>>>>>>>>> With 1.9.0 wrapping up soon (hopefully), there's been
some
>>>>> discussion
>>>>>> on
>>>>>>>>> the having the next release be Airflow 2.0 (rather than
1.10).
>>>>> This
>>>>>>>> would
>>>>>>>>> allow us to break compatibility, and clean up some stuff.
>> Proposed
>>>>>>>> things
>>>>>>>>> to include in 2.0 are:
>>>>>>>>> 
>>>>>>>>> 1) New webserver that Joy Gao has been working on.
>>>>>>>>> 2) Remove the legacy import style that's been deprecated
since
>> at
>>>>>> least
>>>>>>>> 1.8
>>>>>>>>> 3) New timzone feature
>>>>>>>>> 4) Move API out of experimental
>>>>>>>>> 
>>>>>>>>> I want to keep the list fairly tight, preferably to things
that
>>>>> have
>>>>>>>>> already been done, so that we can ship it fairly quickly
(in the
>>>>> next
>>>>>>>>> couple of months).
>>>>>>>>> 
>>>>>>>>> Does this sound like a good list?
>>>>>>>>> 
>>>>>>>>> Cheers,
>>>>>>>>> Chris
>> 

Mime
View raw message