Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-dev@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of andrew.wang@cloudera.com
 designates 209.85.220.179 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <1425349807827.88706@hortonworks.com>
References: 
 <CAGB5D2Za9UHLjw1_+xNcJ0nSyWkJT2fXogCq_SUi6baFEv6+gQ@mail.gmail.com>
 <1425349807827.88706@hortonworks.com>
From: Andrew Wang <andrew.wang@cloudera.com>
Date: Mon, 2 Mar 2015 20:08:43 -0800
Message-ID: 
 <CAGB5D2a=5_46UcyKc+EoG08gYo=R-VDj+rnr7q9pjxj2kjuVNA@mail.gmail.com>
Subject: Re: Looking to a Hadoop 3 release
To: "hdfs-dev@hadoop.apache.org" <hdfs-dev@hadoop.apache.org>
Cc: "common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>,
	"mapreduce-dev@hadoop.apache.org" <mapreduce-dev@hadoop.apache.org>,
	"yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>
Content-Type: multipart/alternative; boundary=bcaec5015ed1c411f805105a7ebd

--bcaec5015ed1c411f805105a7ebd
Content-Type: text/plain; charset=UTF-8

 Thanks as always for the feedback everyone. Some inline comments to Arun's
email, as his were the most extensive:


>  Given that we already agreed to put in JDK7 in 2.7, and that the
> classpath is a fairly minor irritant given some existing solutions (e.g. a
> new default classloader), how do you quantify the benefit for users?
>
> I looked at our thread on this topic from last time, and we (meaning at
least myself and Tucu) agreed to a one-time exception to the JDK7 bump in
2.x for practical reasons. We waited for so long that we had some assurance
JDK6 was on the outs. Multiple distros also already had bumped their min
version to JDK7. This is not true this time around. Bumping the JDK version
is hugely impactful on the end user, and my email on the earlier thread
still reflects my thoughts on JDK compatibility:

http://mail-archives.apache.org/mod_mbox/hadoop-common-dev/201406.mbox/%3CCAGB5D2a5fEDfBApQyER_zyhc8a4Xd_ea1wJSsxxkiAiDZO9%2BNg%40mail.gmail.com%3E

Regarding classpath isolation, based on what I hear from our customers,
it's still a big problem (even after the MR classloader work). The latest
Jackson version bump was quite painful for our downstream projects, and the
HDFS client still leaks a lot of dependencies. Would welcome more
discussion of this on HADOOP-11656, Steve, Colin, and Haohui have already
chimed in.

Having the freedom to upgrade our dependencies at will would also be a big
win for us as developers.

 We could just do JDK8 in hadoop-2.10 or some such, you are definitely
> welcome to run the RM role for that release.
>
>  Furthermore, I'm really concerned that this will be used as an
> opportunity to further break compat in more egregious ways.
>
>  Also, are you foreseeing more compat breaks? OTOH, if we all agree that
> we should absolutely prevent compat breakages such as the client-server
> wire protocol, I feel the point of a major release is kinda lost.
>
>
Right now, the incompatible changes would be JDK8, classpath isolation, and
whatever is already in trunk. I can audit these existing trunk changes when
branch-3 is cut.

I would like to keep this list as short as possible, to preserve wire
compat and rolling upgrade. As far as major releases go, this is not one to
be scared of. However, since it's incompatible, it still needs that major
version bump.

Best,
Andrew

P.S. Vinod, the shell script rewrite is incompatible. Allen intentionally
excluded it from branch-2 for this reason.

--bcaec5015ed1c411f805105a7ebd--