airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bolke de Bruin <bdbr...@gmail.com>
Subject Re: Airflow 1.10
Date Mon, 26 Mar 2018 19:24:40 GMT
If we can make sure we take this as an opportunity to make this our graduation release that
would be perfect. This would mean making sure no licensing issue anymore, security vulnerabilities
properly documented and published then I am all for it!

Verstuurd vanaf mijn iPad

> Op 26 mrt. 2018 om 20:52 heeft Chris Riccomini <criccomini@apache.org> het volgende
geschreven:
> 
> Hey all,
> 
> Now that the RBAC UI has been merged in, I wanted to revive this thread
> again. Timezone stuff has been in for a while. I'm not sure on the status
> of the K8s stuff, but I do feel it's time to start getting the ball rolling
> on a 1.10 release. What do others think? Any volunteers for a release
> manager? :)
> 
> Cheers,
> Chris
> 
> On Tue, Jan 16, 2018 at 12:42 PM, George Leslie-Waksman <
> george@cloverhealth.com.invalid> wrote:
> 
>> +1
>> 
>>> On Tue, Jan 16, 2018 at 11:08 AM Joy Gao <joyg@wepay.com> wrote:
>>> 
>>> The new FAB UI does not modify the existing API (i.e. www2/api/ will be a
>>> copy of www/api/), and the endpoints are registered as blueprints to the
>>> flask app the same way as before, so it is fully backward compatible.
>>> 
>>> Although FAB offers REST APIs on the models out-of-the-box, it currently
>> is
>>> still in BETA and does not support any HTTP authentication scheme,
>> meaning
>>> it would require a cookie to mimic a normal user session in order to be
>>> used. In the long run (for Airflow 2.X+), I believe the best approach is
>> to
>>> patch FAB with auth backends, so we can apply @has_access annotation on
>> all
>>> rest endpoints to streamline authentication.  For 1.10, the current plan
>> is
>>> to leave the API as is. I wouldn't recommend folks to use the FAB-based
>>> REST APIs yet. I would also like to release the new UI as an alpha
>> version,
>>> and wait for 2.0 before promoting it to the default version. This will
>> give
>>> us some time to address any new UI bugs which I overlooked.
>>> 
>>> +1 on polishing! (With the exception of "*Rest Api should standardise and
>>> have proper swagger definitions" and any other bugs that require major
>>> overhaul*, which I think can wait until 2.0)
>>> 
>>> 
>>> On Sun, Jan 14, 2018 at 12:18 PM, Bolke de Bruin <bdbruin@gmail.com>
>>> wrote:
>>> 
>>>> Chris, Joy,
>>>> 
>>>> Can you shed some light on the backward compatibility of the new UI,
>>>> particularly with regards to the API? The API for example cannot use
>> the
>>>> login from FAB afaik.
>>>> 
>>>> As much of the work is already in for 1.10 I think focus should be on
>>>> polishing. There are some minor quirks and slightly annoying bugs.
>>>> 
>>>> - It seems a dag with schedule “none” can still run when turned on from
>>>> the UI (unconfirmed)
>>>> - Exceptions are swallowed when importing a custom logging conf
>>>> - UI only displays UTC
>>>> - Logging ends up duplicated (fixed in master)
>>>> - Tasks instantiated outside airflow do not set default time zone
>>> (@msumit)
>>>> - Log file retrieval feels archaic (non local?)
>>>> - Rest Api should standardise and have proper swagger definitions
>>>> 
>>>> And probably some others.
>>>> 
>>>> Bolke
>>>> 
>>>>> On 14 Jan 2018, at 15:41, Driesprong, Fokko <fokko@driesprong.frl>
>>>> wrote:
>>>>> 
>>>>> I think 1.10 is a good idea. I'm working on this refactoring of the
>>>> sensor
>>>>> structure: https://github.com/apache/incubator-airflow/pull/2875
>>>>> 
>>>>> Would be awesome to get this in. At my current project we use sensors
>>> in
>>>> a
>>>>> few places, but still there is some work to be done. For example,
>> don't
>>>>> allocate an executor slot to the sensors, but have a more
>> sophisticated
>>>> way
>>>>> of poking.
>>>>> 
>>>>> Cheers, Fokko
>>>>> 
>>>>> 
>>>>> 
>>>>> 2018-01-12 21:19 GMT+01:00 Chris Riccomini <criccomini@apache.org>:
>>>>> 
>>>>>> Just the operator (AIRFLOW-1517)
>>>>>> 
>>>>>> On Fri, Jan 12, 2018 at 11:21 AM, Anirudh Ramanathan <
>>>>>> ramanathana@google.com.invalid> wrote:
>>>>>> 
>>>>>>> Sounds awesome. Is k8s support here referring to both the executor
>>> and
>>>>>> the
>>>>>>> operator?
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> 
>>>>>>> 
>>>>>>>> On Jan 12, 2018 11:18 AM, "Sid Anand" <sanand@apache.org>
wrote:
>>>>>>>> 
>>>>>>>> +1
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Fri, Jan 12, 2018 at 10:56 AM, Chris Riccomini <
>>>>>> criccomini@apache.org
>>>>>>>> 
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Hey all,
>>>>>>>>> 
>>>>>>>>> After some past discussion on Airflow 1.10 vs 2.0, I
think we've
>>>>>>>> converged
>>>>>>>>> on a 1.10 as the next step. 1.10 will include:
>>>>>>>>> 
>>>>>>>>> * Timezone changes
>>>>>>>>> * Kubernetes support
>>>>>>>>> * New UI
>>>>>>>>> 
>>>>>>>>> The first two have been merged in, as I saw Bolke just
merged K8s
>>> (I
>>>>>>> saw
>>>>>>>> a
>>>>>>>>> few follow-on patches coming, though), and I think the
new UI is
>>>>>>>> probably a
>>>>>>>>> couple of weeks out from a PR on master.
>>>>>>>>> 
>>>>>>>>> What do people think of starting the release process
on master in
>>>>>> Feb?
>>>>>>>>> Given that it took a month last time, I expect 1.10 to
be
>> released
>>> in
>>>>>>>>> March. Thoughts?
>>>>>>>>> 
>>>>>>>>> Cheers,
>>>>>>>>> Chris
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>> 
>>>> 
>>> 
>> 

Mime
View raw message