airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Naik Kaxil <k.n...@reply.com>
Subject Re: Airflow 1.10
Date Wed, 28 Mar 2018 10:01:09 GMT
+1 for 1.10

On 27/03/2018, 21:56, "Feng Lu" <fenglu@google.com.INVALID> wrote:

    +1 for 1.10, thank you Chris!
    
    On Mon, Mar 26, 2018 at 12:44 PM Milan van der Meer <
    milan.vandermeer@riaktr.com> wrote:
    
    > The fix for the on_kill method for all the operators should really be
    > included in the 1.10 release.
    > https://github.com/apache/incubator-airflow/pull/2975
    >
    > On Mon, Mar 26, 2018 at 8:52 PM, Chris Riccomini <criccomini@apache.org>
    > wrote:
    >
    > > Hey all,
    > >
    > > Now that the RBAC UI has been merged in, I wanted to revive this thread
    > > again. Timezone stuff has been in for a while. I'm not sure on the status
    > > of the K8s stuff, but I do feel it's time to start getting the ball
    > rolling
    > > on a 1.10 release. What do others think? Any volunteers for a release
    > > manager? :)
    > >
    > > Cheers,
    > > Chris
    > >
    > > On Tue, Jan 16, 2018 at 12:42 PM, George Leslie-Waksman <
    > > george@cloverhealth.com.invalid> wrote:
    > >
    > > > +1
    > > >
    > > > On Tue, Jan 16, 2018 at 11:08 AM Joy Gao <joyg@wepay.com> wrote:
    > > >
    > > > > The new FAB UI does not modify the existing API (i.e. www2/api/ will
    > > be a
    > > > > copy of www/api/), and the endpoints are registered as blueprints
to
    > > the
    > > > > flask app the same way as before, so it is fully backward compatible.
    > > > >
    > > > > Although FAB offers REST APIs on the models out-of-the-box, it
    > > currently
    > > > is
    > > > > still in BETA and does not support any HTTP authentication scheme,
    > > > meaning
    > > > > it would require a cookie to mimic a normal user session in order
to
    > be
    > > > > used. In the long run (for Airflow 2.X+), I believe the best approach
    > > is
    > > > to
    > > > > patch FAB with auth backends, so we can apply @has_access annotation
    > on
    > > > all
    > > > > rest endpoints to streamline authentication.  For 1.10, the current
    > > plan
    > > > is
    > > > > to leave the API as is. I wouldn't recommend folks to use the
    > FAB-based
    > > > > REST APIs yet. I would also like to release the new UI as an alpha
    > > > version,
    > > > > and wait for 2.0 before promoting it to the default version. This
    > will
    > > > give
    > > > > us some time to address any new UI bugs which I overlooked.
    > > > >
    > > > > +1 on polishing! (With the exception of "*Rest Api should standardise
    > > and
    > > > > have proper swagger definitions" and any other bugs that require
    > major
    > > > > overhaul*, which I think can wait until 2.0)
    > > > >
    > > > >
    > > > > On Sun, Jan 14, 2018 at 12:18 PM, Bolke de Bruin <bdbruin@gmail.com>
    > > > > wrote:
    > > > >
    > > > > > Chris, Joy,
    > > > > >
    > > > > > Can you shed some light on the backward compatibility of the
new
    > UI,
    > > > > > particularly with regards to the API? The API for example cannot
    > use
    > > > the
    > > > > > login from FAB afaik.
    > > > > >
    > > > > > As much of the work is already in for 1.10 I think focus should
be
    > on
    > > > > > polishing. There are some minor quirks and slightly annoying
bugs.
    > > > > >
    > > > > > - It seems a dag with schedule “none” can still run when
turned on
    > > from
    > > > > > the UI (unconfirmed)
    > > > > > - Exceptions are swallowed when importing a custom logging conf
    > > > > > - UI only displays UTC
    > > > > > - Logging ends up duplicated (fixed in master)
    > > > > > - Tasks instantiated outside airflow do not set default time
zone
    > > > > (@msumit)
    > > > > > - Log file retrieval feels archaic (non local?)
    > > > > > - Rest Api should standardise and have proper swagger definitions
    > > > > >
    > > > > > And probably some others.
    > > > > >
    > > > > > Bolke
    > > > > >
    > > > > > > On 14 Jan 2018, at 15:41, Driesprong, Fokko <fokko@driesprong.frl
    > >
    > > > > > wrote:
    > > > > > >
    > > > > > > I think 1.10 is a good idea. I'm working on this refactoring
of
    > the
    > > > > > sensor
    > > > > > > structure: https://github.com/apache/incubator-airflow/pull/2875
    > > > > > >
    > > > > > > Would be awesome to get this in. At my current project we
use
    > > sensors
    > > > > in
    > > > > > a
    > > > > > > few places, but still there is some work to be done. For
example,
    > > > don't
    > > > > > > allocate an executor slot to the sensors, but have a more
    > > > sophisticated
    > > > > > way
    > > > > > > of poking.
    > > > > > >
    > > > > > > Cheers, Fokko
    > > > > > >
    > > > > > >
    > > > > > >
    > > > > > > 2018-01-12 21:19 GMT+01:00 Chris Riccomini <
    > criccomini@apache.org
    > > >:
    > > > > > >
    > > > > > >> Just the operator (AIRFLOW-1517)
    > > > > > >>
    > > > > > >> On Fri, Jan 12, 2018 at 11:21 AM, Anirudh Ramanathan
<
    > > > > > >> ramanathana@google.com.invalid> wrote:
    > > > > > >>
    > > > > > >>> Sounds awesome. Is k8s support here referring to
both the
    > > executor
    > > > > and
    > > > > > >> the
    > > > > > >>> operator?
    > > > > > >>>
    > > > > > >>> Thanks,
    > > > > > >>>
    > > > > > >>>
    > > > > > >>> On Jan 12, 2018 11:18 AM, "Sid Anand" <sanand@apache.org>
    > wrote:
    > > > > > >>>
    > > > > > >>>> +1
    > > > > > >>>>
    > > > > > >>>>
    > > > > > >>>> On Fri, Jan 12, 2018 at 10:56 AM, Chris Riccomini
<
    > > > > > >> criccomini@apache.org
    > > > > > >>>>
    > > > > > >>>> wrote:
    > > > > > >>>>
    > > > > > >>>>> Hey all,
    > > > > > >>>>>
    > > > > > >>>>> After some past discussion on Airflow 1.10
vs 2.0, I think
    > > we've
    > > > > > >>>> converged
    > > > > > >>>>> on a 1.10 as the next step. 1.10 will include:
    > > > > > >>>>>
    > > > > > >>>>> * Timezone changes
    > > > > > >>>>> * Kubernetes support
    > > > > > >>>>> * New UI
    > > > > > >>>>>
    > > > > > >>>>> The first two have been merged in, as I
saw Bolke just merged
    > > K8s
    > > > > (I
    > > > > > >>> saw
    > > > > > >>>> a
    > > > > > >>>>> few follow-on patches coming, though), and
I think the new UI
    > > is
    > > > > > >>>> probably a
    > > > > > >>>>> couple of weeks out from a PR on master.
    > > > > > >>>>>
    > > > > > >>>>> What do people think of starting the release
process on
    > master
    > > in
    > > > > > >> Feb?
    > > > > > >>>>> Given that it took a month last time, I
expect 1.10 to be
    > > > released
    > > > > in
    > > > > > >>>>> March. Thoughts?
    > > > > > >>>>>
    > > > > > >>>>> Cheers,
    > > > > > >>>>> Chris
    > > > > > >>>>>
    > > > > > >>>>
    > > > > > >>>
    > > > > > >>
    > > > > >
    > > > > >
    > > > >
    > > >
    > >
    >
    >
    >
    > --
    >
    >
    >
    >
    > *Milan van der Meer*
    >
    >
    > *Software Engineer*www.riaktr.com
    > *| Brussels Office**BE *+32 498 45 96 22 <0032498459622>* | Skype *
    > *milan.vandermeer.ria*
    >
    
    
    



Kaxil Naik 

Data Reply
38 Grosvenor Gardens
London SW1W 0EB - UK 
phone: +44 (0)20 7730 6000
k.naik@reply.com
www.reply.com
Mime
View raw message