airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Van Boxel <a...@vanboxel.be>
Subject Re: Airflow Developers Meeting - 08/03 Notes
Date Sun, 28 Aug 2016 15:37:10 GMT
Hey Maxime, I started to list them all and seems to be a lot of them (one
depending on the other). I think they had some breaking changes. Maybe we
should keep them for Airflow 1.8. I'll keep running on master for now.



On Wed, Aug 24, 2016 at 1:59 AM Jakob Homan <jghoman@gmail.com> wrote:

> ASF provides per-projects blog infrastructure:
> http://www.apache.org/dev/project-blogs
>
> -Jakob
>
>
> On 23 August 2016 at 15:07, Maxime Beauchemin
> <maximebeauchemin@gmail.com> wrote:
> > I'm planning on working on a minimal apache release this week. If someone
> > can point to the commit or PRs I can add them to `branch-1.7.2-apache`
> and
> > start moving towards the release.
> >
> > Max
> >
> > On Tue, Aug 23, 2016 at 11:13 AM, Alex Van Boxel <alex@vanboxel.be>
> wrote:
> >
> >> All stuff is in master (I'm running pre-production on master), if it
> needs
> >> to be cherry-picked to a branch I can always test it. I would be grate
> if
> >> it's in the next release so I can start blogging about it.
> >>
> >> On Tue, Aug 23, 2016 at 8:05 PM Bolke de Bruin <bdbruin@gmail.com>
> wrote:
> >>
> >> > (Sorry for the late response, I was on holiday)
> >> >
> >> > I think the G* operators just need to be cherry picked. This will
> make us
> >> > deviate slightly from the
> >> > previous release, but makes sure we don’t have to ‘fix’ history
> >> afterwards.
> >> >
> >> > Anyone against this?
> >> >
> >> > - B.
> >> >
> >> > > Op 8 aug. 2016, om 14:21 heeft Alex Van Boxel <alex@vanboxel.be>
> het
> >> > volgende geschreven:
> >> > >
> >> > > Sorry, Bolke. What needs to be done for Google Cloud
> operators/hooks?
> >> > >
> >> > > If you bring me up-to-speed I can do this. I'm currently working an
> a
> >> CI
> >> > > setup for the Google stuff).
> >> > >
> >> > > On Sat, Aug 6, 2016 at 3:56 PM Bolke de Bruin <bdbruin@gmail.com>
> >> wrote:
> >> > >
> >> > >> I have indeed a branch with cherry picked commits. This branch
is
> >> > >> available in the apache repo. The branch is available as
> >> > >> “branch-1.7.2-apache”. What is left to do is to decide weather
we
> do
> >> > >> upgrade the google cloud operators/hooks as part of this release
as
> >> the
> >> > >> licenses were only added after these changes came in. I prefer
this
> >> > >> approach as we stay away from custom commits as much as possible
> then.
> >> > >>
> >> > >> - Bolke
> >> > >>
> >> > >>> Op 5 aug. 2016, om 00:21 heeft Gurer Kiratli <
> >> gurer.kiratli@airbnb.com
> >> > .INVALID>
> >> > >> het volgende geschreven:
> >> > >>>
> >> > >>> Agenda
> >> > >>>
> >> > >>>
> >> > >>>  -
> >> > >>>
> >> > >>>  Committers sync-up: progress and plans
> >> > >>>  -
> >> > >>>
> >> > >>>     Max to do a recap since last release
> >> > >>>     -
> >> > >>>
> >> > >>>     Airbnb
> >> > >>>     -
> >> > >>>
> >> > >>>     Anyone else?
> >> > >>>     -
> >> > >>>
> >> > >>>  Cooperation Best Practices
> >> > >>>  -
> >> > >>>
> >> > >>>  Solicit for feedback for Impersonation Design Review
> >> > >>>  -
> >> > >>>
> >> > >>>  Release Schedule, Management
> >> > >>>  -
> >> > >>>
> >> > >>>  Roadmap discussion
> >> > >>>
> >> > >>>
> >> > >>> Attendees: Jeremiah Lowin, Chris Riccomini, Andrew Phillips(?),
> >> Maxime
> >> > >>> Beauchemin, Paul Yang, Dan Davydov, Xuanji Li, George Ke,
Arthur
> >> > Wiedmer,
> >> > >>> Gurer Kiratli
> >> > >>>
> >> > >>> Notes
> >> > >>>
> >> > >>>
> >> > >>>  -
> >> > >>>
> >> > >>>  Recap for the overall project
> >> > >>>  -
> >> > >>>
> >> > >>>     [Max]
> >> > >>>
> >> > >>
> https://gist.github.com/mistercrunch/5460483ec764e2a1cb816c6b1d6ad5a3
> >> > >>>     -
> >> > >>>
> >> > >>>  Airbnb Recap
> >> > >>>  -
> >> > >>>
> >> > >>>     [Paul] Airbnb is continuing work on features related to
> migrating
> >> > >>>     more jobs onto Airflow - DB connection scalability,
> >> impersonation,
> >> > >>>     refreshing web UI, DAG git versioning, task resource isolation
> >> > >>>     -
> >> > >>>
> >> > >>>  Other Recaps
> >> > >>>  -
> >> > >>>
> >> > >>>     [Chris] Improvements to Import functionality. Joy did
some
> work
> >> > >>>     around CLIs and mark success in collaboration with Bolke.
> >> > >>>     -
> >> > >>>
> >> > >>>     [Jeremiah] Came up with the Merge tool. Focusing on the
work
> >> around
> >> > >>>     configuration.
> >> > >>>     -
> >> > >>>
> >> > >>>  Meeting cadence
> >> > >>>  -
> >> > >>>
> >> > >>>     [Max] We can have monthly meetings. Let’s start with
monthly
> we
> >> can
> >> > >>>     change the cadence. We should invite all contributors.
We
> should
> >> > >>> specify in
> >> > >>>     the invite that everyone is welcome. Currently posting
this on
> >> > >>> the mailing
> >> > >>>     list and wiki.
> >> > >>>     -
> >> > >>>
> >> > >>>     *Action Item*  Gurer to schedule monthly meetings next
one at
> >> > >> Airbnb.
> >> > >>>     Airbnb is welcoming folks to come onsite
> >> > >>>     -
> >> > >>>
> >> > >>>  Blog/Documentation
> >> > >>>  -
> >> > >>>
> >> > >>>     [Max, Chris, Jeremiah] Having a blog would be useful.
We can
> >> > >>>     structure our thoughts.
> >> > >>>     -
> >> > >>>
> >> > >>>     [Andrew] Blog should have better language with screenshots
and
> >> all.
> >> > >>>     This raises the level of effort bar and this might mean
that
> >> > >>> nobody really
> >> > >>>     does it.
> >> > >>>     -
> >> > >>>
> >> > >>>     [Max] Maybe use Medium? We can try it out.
> >> > >>>     -
> >> > >>>
> >> > >>>     [Max] Haven’t looked at the documentation for a long
time. I
> >> don’t
> >> > >>>     really know about its quality.
> >> > >>>     -
> >> > >>>
> >> > >>>     [Arthur] Looking at the email list responses, seems like
the
> >> > >>>     documentation is missing certain elements.
> >> > >>>     -
> >> > >>>
> >> > >>>     [Max] Maybe we can interview people who just started and
> identify
> >> > >>>     gaps.
> >> > >>>     -
> >> > >>>
> >> > >>>     *Action Item* [Jeremiah] We can also ask the mailing list
> about
> >> > >> this.
> >> > >>>     He will send this out.
> >> > >>>     -
> >> > >>>
> >> > >>>  Cooperation guidelines
> >> > >>>  -
> >> > >>>
> >> > >>>     [Max, Arthur]
> >> > >>>
> >> > >>
> >> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/Community+Guidelines
> >> > >>>     -
> >> > >>>
> >> > >>>     [Chris] Let’s link the guidelines to contributor md
. This
> should
> >> > be
> >> > >>>     discoverable
> >> > >>>     -
> >> > >>>
> >> > >>>  Release
> >> > >>>  -
> >> > >>>
> >> > >>>     [Chris, Max] Bolke might have branch with High charts
and
> >> Licensing
> >> > >>>     cherry picked. We can use this as a dry run. We can test
the
> >> > >> process.
> >> > >>>     -
> >> > >>>
> >> > >>>     [Max] Big release is probably around September.
> >> > >>>     -
> >> > >>>
> >> > >>>     [Andrew] If we want to have a true practice, we need to
be
> >> > >> diligently
> >> > >>>     following Apache guidelines.
> >> > >>>     -
> >> > >>>
> >> > >>>     [Max] We might want to revive the email thread about how
we
> >> should
> >> > >> do
> >> > >>>     this.
> >> > >>>     -
> >> > >>>
> >> > >>>     [Chris] It would be great for someone who hasn’t done
this
> before
> >> > to
> >> > >>>     do this.
> >> > >>>     -
> >> > >>>
> >> > >>>     [Andrew] Building a script for this is smart otherwise
the
> manual
> >> > >>>     work is painful from experience.
> >> > >>>
> >> > >>
> https://cwiki.apache.org/confluence/display/JCLOUDS/Releasing+jclouds
> >> > >>>
> >> > >>>
> >> > >>> https://cwiki.apache.org/confluence/display/JCLOUDS/
> >> Validate+a+Release
> >> > >>>
> >> > >>>  -
> >> > >>>
> >> > >>>     *Action Item* [Max] Max will own this. This will happen
by
> 08/19.
> >> > >>>     -
> >> > >>>
> >> > >>>     [Arthur] For the main release around mid September. We
need to
> >> > >>>     confirm this date.
> >> > >>>     -
> >> > >>>
> >> > >>>  Roadmap
> >> > >>>  -
> >> > >>>
> >> > >>>     [Max] Maybe each company puts their own section? Or should
we
> >> have
> >> > a
> >> > >>>     more cohesive roadmap?
> >> > >>>     -
> >> > >>>
> >> > >>>     *Action Item* Gurer to schedule a roadmap discussion
> >> > >>>     -
> >> > >>>
> >> > >>>  Misc
> >> > >>>  -
> >> > >>>
> >> > >>>     [Max] For executor thinking about sticking with Celery
and
> maybe
> >> > >>>     having Docker implemented on top of it.
> >> > >>>     -
> >> > >>>
> >> > >>>     [Max] We might want to explore Sphinx for generating
> >> documentation
> >> > >>>     from code.
> >> > >>>     - [Max] Please review the impersonation design review.
This is
> >> > >> pretty
> >> > >>>     simple and reversible.
> >> > >>
> >> > >>
> >> >
> >> >
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message