apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chinmay Kolhatkar <chin...@datatorrent.com>
Subject Re: Bleeding edge branch ?
Date Tue, 12 Jul 2016 06:36:35 GMT
I'm -0 on this idea.

Here is the reason:
Unless we see a real case where users want to see everything on latest,
this branch might quickly become low hanging fruit and eventually get
obsolete because its anyway a "no gaurantee" branch.

We have a bunch of dependencies which we'll have to take care of to really
make it bleeding edge. Specially about malhar, its a long list. That looks
like quite significant work.
Moreover, if this branch is going to be in "may or may not work" state; I,
as a user or developer, would bank on what certainly works.

I also think that, if its going to be "no gaurantee" then its worth
spending time contributions towards master rather than bleeding-edge branch.

If a question of "should we upgrade?" comes, the community is mature to
take that call then and work accordingly.

-Chinmay.



On Tue, Jul 12, 2016 at 11:42 AM, Priyanka Gugale <priyag@apache.org> wrote:

> +1 for creating such branch.
> One of us will have to rebase it with master branch at intervals. I don't
> think everyone will cherry-pick their commits here. We can make it once in
> a month activity. Are we considering updating all dependency library
> version as well?
>
> -Priyanka
>
> On Tue, Jul 12, 2016 at 2:34 AM, Munagala Ramanath <ram@datatorrent.com>
> wrote:
>
> > Following up on some comments, wanted to clarify what I have in mind for
> > this branch:
> >
> > 1. The main goal is to stay up-to-date with new releases, so if a
> question
> > of the form
> >     "A new release of X is available, should we upgrade ?" comes up, the
> > answer is
> >     *always* an *emphatic* yes; otherwise it doesn't bleed enough (:-) as
> > Sanjay points out.
> > 2. Pull requests are submitted as always; there is no requirement to
> > generate an additional
> >     pull requests against this branch. It may get merged/cherry-picked
> > depending on who has the
> >    time and inclination to do it.
> > 3. There is no expectation of dedication of any additional resources, so
> > people work on
> >     it as and when time is available. ("No guarantee" means exactly
> that).
> > So there is no
> >     question of "maintaining" this branch.
> > 4. This branch is not to be encumbered with legacy and/or backward
> > compatibility issues.
> > 5. This branch is not an experimental sandbox to try out new algorithms,
> > architectural changes
> >     and other such changes.
> >
> > As always, I'm open to other ideas, but that is what I had in mind when I
> > made the suggestion.
> >
> > Ram
> >
> >
> >
> > On Mon, Jul 11, 2016 at 1:45 PM, Sanjay Pujare <sanjay@datatorrent.com>
> > wrote:
> >
> > > As the name suggests the "bleeding-edge" branch ideally should use
> > bleeding
> > > edge versions so I would like to see Java 8 used there (and Hadoop 3
> when
> > > it does eventually come out) to make the maintenance effort
> worthwhile...
> > >
> > > On Mon, Jul 11, 2016 at 12:05 PM, David Yan <david@datatorrent.com>
> > wrote:
> > >
> > > > I'm -0 on Java 8, but I'm +1 on the rest, and I'm especially strong
> +1
> > > for
> > > > upgrading the Hadoop dependency version.
> > > >
> > > > Here are my reasons:
> > > >
> > > > - Hadoop 3 will require Java 8, but Hadoop 2.7.2 still supports Java
> 7
> > > and
> > > > there will probably be some time (I'm guessing more than one year)
> for
> > > > Hadoop 3 to become GA and for major distros to support Hadoop 3. The
> > > > maintenance effort for having two branches, one for Java 7 and one
> for
> > > Java
> > > > 8 is not worth it at this time.
> > > >
> > > > - Apex currently uses Hadoop 2.2 dependencies, marked "provided". And
> > > > Hadoop 2.4 has been released more than two years ago, and it added a
> > lot
> > > of
> > > > features in the API that Apex can make use of. Most distros already
> > > bundle
> > > > Hadoop 2.6 or later. Although some old versions of Cloudera that
> > include
> > > > hadoop version earlier than 2.4 still have not reached end-of-life
> yet,
> > > the
> > > > number of users using those old versions is probably very small.
> > > >
> > > > David
> > > >
> > > >
> > > > On Mon, Jul 11, 2016 at 8:59 AM, Munagala Ramanath <
> > ram@datatorrent.com>
> > > > wrote:
> > > >
> > > > > We've had a number of issues recently related to dependencies on
> old
> > > > > versions
> > > > > of various packages/libraries such as Hadoop itself, Google guava,
> > > > > HTTPClient,
> > > > > mbassador, etc.
> > > > >
> > > > > How about we create a "bleeding-edge" branch in both Core and
> Malhar
> > > > which
> > > > > will use the latest versions of these various dependencies, upgrade
> > to
> > > > Java
> > > > > 8 so
> > > > > we can use the new Java features, etc. ?
> > > > >
> > > > > This will give us an opportunity to discover these sorts of
> problems
> > > > early
> > > > > and,
> > > > > when we are ready to pull the trigger for a major version, we have
> a
> > > > branch
> > > > > ready
> > > > > for merge with, hopefully, minimal additional effort.
> > > > >
> > > > > There will be no guarantees w.r.t. this branch so people using it
> use
> > > it
> > > > at
> > > > > their own
> > > > > risk.
> > > > >
> > > > > Ram
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message