hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yongjun Zhang <yzh...@cloudera.com>
Subject Re: Looking to a Hadoop 3 release
Date Fri, 19 Feb 2016 20:05:48 GMT
Thanks Andrew for initiating the effort!

+1 on pushing 3.x with extended alpha cycle, and continuing the more stable
2.x releases.

--Yongjun

On Thu, Feb 18, 2016 at 5:58 PM, Andrew Wang <andrew.wang@cloudera.com>
wrote:

> Hi Kai,
>
> Sure, I'm open to it. It's a new major release, so we're allowed to make
> these kinds of big changes. The idea behind the extended alpha cycle is
> that downstreams can give us feedback. This way if we do anything too
> radical, we can address it in the next alpha and have downstreams re-test.
>
> Best,
> Andrew
>
> On Thu, Feb 18, 2016 at 5:23 PM, Zheng, Kai <kai.zheng@intel.com> wrote:
>
> > Thanks Andrew for driving this. Wonder if it's a good chance for
> > HADOOP-12579 (Deprecate and remove WriteableRPCEngine) to be in. Note
> it's
> > not an incompatible change, but feel better to be done in the major
> release.
> >
> > Regards,
> > Kai
> >
> > -----Original Message-----
> > From: Andrew Wang [mailto:andrew.wang@cloudera.com]
> > Sent: Friday, February 19, 2016 7:04 AM
> > To: hdfs-dev@hadoop.apache.org; Kihwal Lee <kihwal@yahoo-inc.com>
> > Cc: mapreduce-dev@hadoop.apache.org; common-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org
> > Subject: Re: Looking to a Hadoop 3 release
> >
> > Hi Kihwal,
> >
> > I think there's still value in continuing the 2.x releases. 3.x comes
> with
> > the incompatible bump to a JDK8 runtime, and also the fact that 3.x won't
> > be beta or GA for some number of months. In the meanwhile, it'd be good
> to
> > keep putting out regular, stable 2.x releases.
> >
> > Best,
> > Andrew
> >
> >
> > On Thu, Feb 18, 2016 at 2:50 PM, Kihwal Lee <kihwal@yahoo-inc.com.invalid
> >
> > wrote:
> >
> > > Moving Hadoop 3 forward sounds fine. If EC is one of the main
> > > motivations, are we getting rid of branch-2.8?
> > >
> > > Kihwal
> > >
> > >       From: Andrew Wang <andrew.wang@cloudera.com>
> > >  To: "common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>
> > > Cc: "yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>; "
> > > mapreduce-dev@hadoop.apache.org" <mapreduce-dev@hadoop.apache.org>;
> > > hdfs-dev <hdfs-dev@hadoop.apache.org>
> > >  Sent: Thursday, February 18, 2016 4:35 PM
> > >  Subject: Re: Looking to a Hadoop 3 release
> > >
> > > Hi all,
> > >
> > > Reviving this thread. I've seen renewed interest in a trunk release
> > > since HDFS erasure coding has not yet made it to branch-2. Along with
> > > JDK8, the shell script rewrite, and many other improvements, I think
> > > it's time to revisit Hadoop 3.0 release plans.
> > >
> > > My overall plan is still the same as in my original email: a series of
> > > regular alpha releases leading up to beta and GA. Alpha releases make
> > > it easier for downstreams to integrate with our code, and making them
> > > regular means features can be included when they are ready.
> > >
> > > I know there are some incompatible changes waiting in the wings (i.e.
> > > HDFS-6984 making FileStatus a PB rather than Writable, some of
> > > HADOOP-9991 bumping dependency versions) that would be good to get in.
> > > If you have changes like this, please set the target version to 3.0.0
> > > and mark them "Incompatible". We can use this JIRA query to track:
> > >
> > >
> > > https://issues.apache.org/jira/issues/?jql=project%20in%20(HADOOP%2C%2
> > > 0HDFS%2C%20YARN%2C%20MAPREDUCE)%20and%20%22Target%20Version%2Fs%22%20%
> > > 3D%20%223.0.0%22%20and%20resolution%3D%22unresolved%22%20and%20%22Hado
> > > op%20Flags%22%3D%22Incompatible%20change%22%20order%20by%20priority
> > >
> > > There's some release-related stuff that needs to be sorted out
> > > (namely, the new CHANGES.txt and release note generation from Yetus),
> > > but I'd tentatively like to roll the first alpha a month out, so third
> > > week of March.
> > >
> > > Best,
> > > Andrew
> > >
> > > On Mon, Mar 9, 2015 at 7:23 PM, Raymie Stata <rstata@altiscale.com>
> > wrote:
> > >
> > > > Avoiding the use of JDK8 language features (and, presumably, APIs)
> > > > means you've abandoned #1, i.e., you haven't (really) bumped the JDK
> > > > source version to JDK8.
> > > >
> > > > Also, note that releasing from trunk is a way of achieving #3, it's
> > > > not a way of abandoning it.
> > > >
> > > >
> > > >
> > > > On Mon, Mar 9, 2015 at 7:10 PM, Andrew Wang
> > > > <andrew.wang@cloudera.com>
> > > > wrote:
> > > > > Hi Raymie,
> > > > >
> > > > > Konst proposed just releasing off of trunk rather than cutting a
> > > > branch-2,
> > > > > and there was general agreement there. So, consider #3 abandoned.
> > > > > 1&2
> > > can
> > > > > be achieved at the same time, we just need to avoid using JDK8
> > > > > language features in trunk so things can be backported.
> > > > >
> > > > > Best,
> > > > > Andrew
> > > > >
> > > > > On Mon, Mar 9, 2015 at 7:01 PM, Raymie Stata
> > > > > <rstata@altiscale.com>
> > > > wrote:
> > > > >
> > > > >> In this (and the related threads), I see the following three
> > > > requirements:
> > > > >>
> > > > >> 1. "Bump the source JDK version to JDK8" (ie, drop JDK7 support).
> > > > >>
> > > > >> 2. "We'll still be releasing 2.x releases for a while, with
> > > > >> similar feature sets as 3.x."
> > > > >>
> > > > >> 3. Avoid the "risk of split-brain behavior" by "minimize
> > > > >> backporting headaches. Pulling trunk > branch-2 > branch-2.x
is
> > already tedious.
> > > > >> Adding a branch-3, branch-3.x would be obnoxious."
> > > > >>
> > > > >> These three cannot be achieved at the same time.  Which do we
> > abandon?
> > > > >>
> > > > >>
> > > > >> On Mon, Mar 9, 2015 at 12:45 PM, sanjay Radia
> > > > >> <sanjayosrc@gmail.com>
> > > > >> wrote:
> > > > >> >
> > > > >> >> On Mar 5, 2015, at 3:21 PM, Siddharth Seth <sseth@apache.org>
> > > wrote:
> > > > >> >>
> > > > >> >> 2) Simplification of configs - potentially separating
client
> > > > >> >> side
> > > > >> configs
> > > > >> >> and those used by daemons. This is another source of
perpetual
> > > > confusion
> > > > >> >> for users.
> > > > >> > + 1 on this.
> > > > >> >
> > > > >> > sanjay
> > > > >>
> > > >
> > >
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message