airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Milan van der Meer <milan.vanderm...@riaktr.com>
Subject Re: Airflow 1.10
Date Mon, 26 Mar 2018 19:44:46 GMT
The fix for the on_kill method for all the operators should really be
included in the 1.10 release.
https://github.com/apache/incubator-airflow/pull/2975

On Mon, Mar 26, 2018 at 8:52 PM, Chris Riccomini <criccomini@apache.org>
wrote:

> Hey all,
>
> Now that the RBAC UI has been merged in, I wanted to revive this thread
> again. Timezone stuff has been in for a while. I'm not sure on the status
> of the K8s stuff, but I do feel it's time to start getting the ball rolling
> on a 1.10 release. What do others think? Any volunteers for a release
> manager? :)
>
> Cheers,
> Chris
>
> On Tue, Jan 16, 2018 at 12:42 PM, George Leslie-Waksman <
> george@cloverhealth.com.invalid> wrote:
>
> > +1
> >
> > On Tue, Jan 16, 2018 at 11:08 AM Joy Gao <joyg@wepay.com> wrote:
> >
> > > The new FAB UI does not modify the existing API (i.e. www2/api/ will
> be a
> > > copy of www/api/), and the endpoints are registered as blueprints to
> the
> > > flask app the same way as before, so it is fully backward compatible.
> > >
> > > Although FAB offers REST APIs on the models out-of-the-box, it
> currently
> > is
> > > still in BETA and does not support any HTTP authentication scheme,
> > meaning
> > > it would require a cookie to mimic a normal user session in order to be
> > > used. In the long run (for Airflow 2.X+), I believe the best approach
> is
> > to
> > > patch FAB with auth backends, so we can apply @has_access annotation on
> > all
> > > rest endpoints to streamline authentication.  For 1.10, the current
> plan
> > is
> > > to leave the API as is. I wouldn't recommend folks to use the FAB-based
> > > REST APIs yet. I would also like to release the new UI as an alpha
> > version,
> > > and wait for 2.0 before promoting it to the default version. This will
> > give
> > > us some time to address any new UI bugs which I overlooked.
> > >
> > > +1 on polishing! (With the exception of "*Rest Api should standardise
> and
> > > have proper swagger definitions" and any other bugs that require major
> > > overhaul*, which I think can wait until 2.0)
> > >
> > >
> > > On Sun, Jan 14, 2018 at 12:18 PM, Bolke de Bruin <bdbruin@gmail.com>
> > > wrote:
> > >
> > > > Chris, Joy,
> > > >
> > > > Can you shed some light on the backward compatibility of the new UI,
> > > > particularly with regards to the API? The API for example cannot use
> > the
> > > > login from FAB afaik.
> > > >
> > > > As much of the work is already in for 1.10 I think focus should be on
> > > > polishing. There are some minor quirks and slightly annoying bugs.
> > > >
> > > > - It seems a dag with schedule “none” can still run when turned on
> from
> > > > the UI (unconfirmed)
> > > > - Exceptions are swallowed when importing a custom logging conf
> > > > - UI only displays UTC
> > > > - Logging ends up duplicated (fixed in master)
> > > > - Tasks instantiated outside airflow do not set default time zone
> > > (@msumit)
> > > > - Log file retrieval feels archaic (non local?)
> > > > - Rest Api should standardise and have proper swagger definitions
> > > >
> > > > And probably some others.
> > > >
> > > > Bolke
> > > >
> > > > > On 14 Jan 2018, at 15:41, Driesprong, Fokko <fokko@driesprong.frl>
> > > > wrote:
> > > > >
> > > > > I think 1.10 is a good idea. I'm working on this refactoring of the
> > > > sensor
> > > > > structure: https://github.com/apache/incubator-airflow/pull/2875
> > > > >
> > > > > Would be awesome to get this in. At my current project we use
> sensors
> > > in
> > > > a
> > > > > few places, but still there is some work to be done. For example,
> > don't
> > > > > allocate an executor slot to the sensors, but have a more
> > sophisticated
> > > > way
> > > > > of poking.
> > > > >
> > > > > Cheers, Fokko
> > > > >
> > > > >
> > > > >
> > > > > 2018-01-12 21:19 GMT+01:00 Chris Riccomini <criccomini@apache.org
> >:
> > > > >
> > > > >> Just the operator (AIRFLOW-1517)
> > > > >>
> > > > >> On Fri, Jan 12, 2018 at 11:21 AM, Anirudh Ramanathan <
> > > > >> ramanathana@google.com.invalid> wrote:
> > > > >>
> > > > >>> Sounds awesome. Is k8s support here referring to both the
> executor
> > > and
> > > > >> the
> > > > >>> operator?
> > > > >>>
> > > > >>> Thanks,
> > > > >>>
> > > > >>>
> > > > >>> On Jan 12, 2018 11:18 AM, "Sid Anand" <sanand@apache.org>
wrote:
> > > > >>>
> > > > >>>> +1
> > > > >>>>
> > > > >>>>
> > > > >>>> On Fri, Jan 12, 2018 at 10:56 AM, Chris Riccomini <
> > > > >> criccomini@apache.org
> > > > >>>>
> > > > >>>> wrote:
> > > > >>>>
> > > > >>>>> Hey all,
> > > > >>>>>
> > > > >>>>> After some past discussion on Airflow 1.10 vs 2.0,
I think
> we've
> > > > >>>> converged
> > > > >>>>> on a 1.10 as the next step. 1.10 will include:
> > > > >>>>>
> > > > >>>>> * Timezone changes
> > > > >>>>> * Kubernetes support
> > > > >>>>> * New UI
> > > > >>>>>
> > > > >>>>> The first two have been merged in, as I saw Bolke
just merged
> K8s
> > > (I
> > > > >>> saw
> > > > >>>> a
> > > > >>>>> few follow-on patches coming, though), and I think
the new UI
> is
> > > > >>>> probably a
> > > > >>>>> couple of weeks out from a PR on master.
> > > > >>>>>
> > > > >>>>> What do people think of starting the release process
on master
> in
> > > > >> Feb?
> > > > >>>>> Given that it took a month last time, I expect 1.10
to be
> > released
> > > in
> > > > >>>>> March. Thoughts?
> > > > >>>>>
> > > > >>>>> Cheers,
> > > > >>>>> Chris
> > > > >>>>>
> > > > >>>>
> > > > >>>
> > > > >>
> > > >
> > > >
> > >
> >
>



-- 




*Milan van der Meer*


*Software Engineer*www.riaktr.com
*| Brussels Office**BE *+32 498 45 96 22 <0032498459622>* | Skype *
*milan.vandermeer.ria*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message