Mailing-List: contact yarn-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-dev@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of jdu@hortonworks.com designates
 64.78.52.184 as permitted sender)
Subject: Re: Looking to a Hadoop 3 release
MIME-Version: 1.0
From: Junping Du <jdu@hortonworks.com>
To: "common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>,
	"mapreduce-dev@hadoop.apache.org" <mapreduce-dev@hadoop.apache.org>,
	"hdfs-dev@hadoop.apache.org" <hdfs-dev@hadoop.apache.org>,
	"yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>
Thread-Topic: Looking to a Hadoop 3 release
Thread-Index: AQHQVT+BUK3LWHckUECNVasdp1KEop0LL+EA//+Y97Q=
Date: Tue, 3 Mar 2015 14:12:55 +0000
Message-ID: <1425392868218.91834@hortonworks.com>
References: 
 <CAGB5D2Za9UHLjw1_+xNcJ0nSyWkJT2fXogCq_SUi6baFEv6+gQ@mail.gmail.com>,<54F5A330.7090001@oss.nttdata.co.jp>
In-Reply-To: <54F5A330.7090001@oss.nttdata.co.jp>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Thanks all for good discussions here.
+1 on supporting Java 8 ASAP. In addition, I agree that we should separatin=
g this effort with cutting down Hadoop 3.=20
IMO, Hadoop is still very cool today, and we should only consider Hadoop 3 =
until we have revolutionary feature (like YARN for 2.0) which deserve to br=
eak fundamental compatibilities. Or it may just cause more distractions for=
 community effort.
Just 2 cents.

Thanks,

Junping
________________________________________
From: Akira AJISAKA <ajisakaa@oss.nttdata.co.jp>
Sent: Tuesday, March 03, 2015 12:04 PM
To: common-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; hdfs-dev=
@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: Looking to a Hadoop 3 release

Thanks Andrew for bringing this up.
+1 mostly looks fine but I'm thinking it's not now to cut branch-3.

 > classpath isolation

IMHO, classpath isolation is a good thing to do.
We should pay down the technical dept ASAP. I'm willing to help.

I'm thinking we can cut branch-3 and release 3.0 alpha
after HADOOP-11656 is fixed. That is, I'd like to mark
this issue as a blocker for 3.0.
I wonder that even if we cut branch-3 now, trunk and
branch-3 would be the same for a while. That seems useless.

 > JDK8

As Steve suggested, JDK8 can be in both trunk and branch-2.
+1 for moving to JDK8 ASAP.

 > maintaining 2.x

For user side, now there is little merit to upgrade to 3.x.
More important thing is how long 2.x will be maintained.
Therefore we should consider when to stop backporting
new features to 2.x, and when to stop maintaining 2.x.
I'd like to maintain 2.x as long as possible, at least
one year after 3.x GA release.

* Other issue

What's the current status of HDFS symlink?
If HADOOP-10019 requires some incompatible changes,
I'd like to include in 3.x.

Regards,
Akira

On 3/2/15 15:19, Andrew Wang wrote:
> Hi devs,
>
> It's been a year and a half since 2.x went GA, and I think we're about du=
e
> for a 3.x release.
> Notably, there are two incompatible changes I'd like to call out, that wi=
ll
> have a tremendous positive impact for our users.
>
> First, classpath isolation being done at HADOOP-11656, which has been a
> long-standing request from many downstreams and Hadoop users.
>
> Second, bumping the source and target JDK version to JDK8 (related to
> HADOOP-11090), which is important since JDK7 is EOL in April 2015 (two
> months from now). In the past, we've had issues with our dependencies
> discontinuing support for old JDKs, so this will future-proof us.
>
> Between the two, we'll also have quite an opportunity to clean up and
> upgrade our dependencies, another common user and developer request.
>
> I'd like to propose that we start rolling a series of monthly-ish series =
of
> 3.0 alpha releases ASAP, with myself volunteering to take on the RM and
> other cat herding responsibilities. There are already quite a few changes
> slated for 3.0 besides the above (for instance the shell script rewrite) =
so
> there's already value in a 3.0 alpha, and the more time we give downstrea=
ms
> to integrate, the better.
>
> This opens up discussion about inclusion of other changes, but I'm hoping
> to freeze incompatible changes after maybe two alphas, do a beta (with no
> further incompat changes allowed), and then finally a 3.x GA. For those
> keeping track, that means a 3.x GA in about four months.
>
> I would also like to stress though that this is not intended to be a big
> bang release. For instance, it would be great if we could maintain wire
> compatibility between 2.x and 3.x, so rolling upgrades work. Keeping
> branch-2 and branch-3 similar also makes backports easier, since we're
> likely maintaining 2.x for a while yet.
>
> Please let me know any comments / concerns related to the above. If peopl=
e
> are friendly to the idea, I'd like to cut a branch-3 and start working on
> the first alpha.
>
> Best,
> Andrew
>