airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Feng Lu <fen...@google.com.INVALID>
Subject Re: Airflow 1.10
Date Tue, 27 Mar 2018 20:56:09 GMT
+1 for 1.10, thank you Chris!

On Mon, Mar 26, 2018 at 12:44 PM Milan van der Meer <
milan.vandermeer@riaktr.com> wrote:

> The fix for the on_kill method for all the operators should really be
> included in the 1.10 release.
> https://github.com/apache/incubator-airflow/pull/2975
>
> On Mon, Mar 26, 2018 at 8:52 PM, Chris Riccomini <criccomini@apache.org>
> wrote:
>
> > Hey all,
> >
> > Now that the RBAC UI has been merged in, I wanted to revive this thread
> > again. Timezone stuff has been in for a while. I'm not sure on the status
> > of the K8s stuff, but I do feel it's time to start getting the ball
> rolling
> > on a 1.10 release. What do others think? Any volunteers for a release
> > manager? :)
> >
> > Cheers,
> > Chris
> >
> > On Tue, Jan 16, 2018 at 12:42 PM, George Leslie-Waksman <
> > george@cloverhealth.com.invalid> wrote:
> >
> > > +1
> > >
> > > On Tue, Jan 16, 2018 at 11:08 AM Joy Gao <joyg@wepay.com> wrote:
> > >
> > > > The new FAB UI does not modify the existing API (i.e. www2/api/ will
> > be a
> > > > copy of www/api/), and the endpoints are registered as blueprints to
> > the
> > > > flask app the same way as before, so it is fully backward compatible.
> > > >
> > > > Although FAB offers REST APIs on the models out-of-the-box, it
> > currently
> > > is
> > > > still in BETA and does not support any HTTP authentication scheme,
> > > meaning
> > > > it would require a cookie to mimic a normal user session in order to
> be
> > > > used. In the long run (for Airflow 2.X+), I believe the best approach
> > is
> > > to
> > > > patch FAB with auth backends, so we can apply @has_access annotation
> on
> > > all
> > > > rest endpoints to streamline authentication.  For 1.10, the current
> > plan
> > > is
> > > > to leave the API as is. I wouldn't recommend folks to use the
> FAB-based
> > > > REST APIs yet. I would also like to release the new UI as an alpha
> > > version,
> > > > and wait for 2.0 before promoting it to the default version. This
> will
> > > give
> > > > us some time to address any new UI bugs which I overlooked.
> > > >
> > > > +1 on polishing! (With the exception of "*Rest Api should standardise
> > and
> > > > have proper swagger definitions" and any other bugs that require
> major
> > > > overhaul*, which I think can wait until 2.0)
> > > >
> > > >
> > > > On Sun, Jan 14, 2018 at 12:18 PM, Bolke de Bruin <bdbruin@gmail.com>
> > > > wrote:
> > > >
> > > > > Chris, Joy,
> > > > >
> > > > > Can you shed some light on the backward compatibility of the new
> UI,
> > > > > particularly with regards to the API? The API for example cannot
> use
> > > the
> > > > > login from FAB afaik.
> > > > >
> > > > > As much of the work is already in for 1.10 I think focus should be
> on
> > > > > polishing. There are some minor quirks and slightly annoying bugs.
> > > > >
> > > > > - It seems a dag with schedule “none” can still run when turned
on
> > from
> > > > > the UI (unconfirmed)
> > > > > - Exceptions are swallowed when importing a custom logging conf
> > > > > - UI only displays UTC
> > > > > - Logging ends up duplicated (fixed in master)
> > > > > - Tasks instantiated outside airflow do not set default time zone
> > > > (@msumit)
> > > > > - Log file retrieval feels archaic (non local?)
> > > > > - Rest Api should standardise and have proper swagger definitions
> > > > >
> > > > > And probably some others.
> > > > >
> > > > > Bolke
> > > > >
> > > > > > On 14 Jan 2018, at 15:41, Driesprong, Fokko <fokko@driesprong.frl
> >
> > > > > wrote:
> > > > > >
> > > > > > I think 1.10 is a good idea. I'm working on this refactoring
of
> the
> > > > > sensor
> > > > > > structure: https://github.com/apache/incubator-airflow/pull/2875
> > > > > >
> > > > > > Would be awesome to get this in. At my current project we use
> > sensors
> > > > in
> > > > > a
> > > > > > few places, but still there is some work to be done. For example,
> > > don't
> > > > > > allocate an executor slot to the sensors, but have a more
> > > sophisticated
> > > > > way
> > > > > > of poking.
> > > > > >
> > > > > > Cheers, Fokko
> > > > > >
> > > > > >
> > > > > >
> > > > > > 2018-01-12 21:19 GMT+01:00 Chris Riccomini <
> criccomini@apache.org
> > >:
> > > > > >
> > > > > >> Just the operator (AIRFLOW-1517)
> > > > > >>
> > > > > >> On Fri, Jan 12, 2018 at 11:21 AM, Anirudh Ramanathan <
> > > > > >> ramanathana@google.com.invalid> wrote:
> > > > > >>
> > > > > >>> Sounds awesome. Is k8s support here referring to both
the
> > executor
> > > > and
> > > > > >> the
> > > > > >>> operator?
> > > > > >>>
> > > > > >>> Thanks,
> > > > > >>>
> > > > > >>>
> > > > > >>> On Jan 12, 2018 11:18 AM, "Sid Anand" <sanand@apache.org>
> wrote:
> > > > > >>>
> > > > > >>>> +1
> > > > > >>>>
> > > > > >>>>
> > > > > >>>> On Fri, Jan 12, 2018 at 10:56 AM, Chris Riccomini
<
> > > > > >> criccomini@apache.org
> > > > > >>>>
> > > > > >>>> wrote:
> > > > > >>>>
> > > > > >>>>> Hey all,
> > > > > >>>>>
> > > > > >>>>> After some past discussion on Airflow 1.10 vs
2.0, I think
> > we've
> > > > > >>>> converged
> > > > > >>>>> on a 1.10 as the next step. 1.10 will include:
> > > > > >>>>>
> > > > > >>>>> * Timezone changes
> > > > > >>>>> * Kubernetes support
> > > > > >>>>> * New UI
> > > > > >>>>>
> > > > > >>>>> The first two have been merged in, as I saw
Bolke just merged
> > K8s
> > > > (I
> > > > > >>> saw
> > > > > >>>> a
> > > > > >>>>> few follow-on patches coming, though), and I
think the new UI
> > is
> > > > > >>>> probably a
> > > > > >>>>> couple of weeks out from a PR on master.
> > > > > >>>>>
> > > > > >>>>> What do people think of starting the release
process on
> master
> > in
> > > > > >> Feb?
> > > > > >>>>> Given that it took a month last time, I expect
1.10 to be
> > > released
> > > > in
> > > > > >>>>> March. Thoughts?
> > > > > >>>>>
> > > > > >>>>> Cheers,
> > > > > >>>>> Chris
> > > > > >>>>>
> > > > > >>>>
> > > > > >>>
> > > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>
>
>
> --
>
>
>
>
> *Milan van der Meer*
>
>
> *Software Engineer*www.riaktr.com
> *| Brussels Office**BE *+32 498 45 96 22 <0032498459622>* | Skype *
> *milan.vandermeer.ria*
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message