hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kumar Vavilapalli <vino...@hortonworks.com>
Subject Re: Looking to a Hadoop 3 release
Date Thu, 05 Mar 2015 19:40:51 GMT
The 'resistance' is not so much about  a new major release, more so about the content and the
roadmap of the release. Other than the two specific features raised (the need for breaking
compat for them is something that I am debating), I haven't seen a roadmap of branch-3 about
any more features that this community needs to discuss about. If all the difference between
branch-2 and branch-3 is going to be JDK + a couple of incompat changes, it is a big problem
in two dimensions (1) it's a burden keeping the branches in sync and avoiding the split-brain
we experienced with 1.x, 2.x or worse branch-0.23, branch-2 and (2) very hard to ask people
to not break more things in branch-3.

We seem to have agreed upon a course of action for JDK7. And now we are taking a different
direction for JDK8. Going by this new proposal, come 2016, we will have to deal with JDK9
and 3 mainline incompatible hadoop releases.

Regarding, individual improvements like classpath isolation, shell script stuff, Jason Lowe
captured it perfectly on HADOOP-11656 - it should be possible for every major feature that
we develop to be a opt in, unless the change is so great and users can balance out the incompatibilities
for the new stuff they are getting. Even with an ground breaking change like with YARN, we
spent a bit of time to ensure compatibility (MAPREDUCE-5108) that has paid so many times over
in return. Breaking compatibility shouldn't come across as too cheap a thing.


On Mar 4, 2015, at 10:15 AM, Andrew Wang <andrew.wang@cloudera.com<mailto:andrew.wang@cloudera.com>>

Where does this resistance to a new major release stem from? As I've
described from the beginning, this will look basically like a 2.x release,
except for the inclusion of classpath isolation by default and target
version JDK8. I've expressed my desire to maintain API and wire
compatibility, and we can audit the set of incompatible changes in trunk to
ensure this. My proposal for doing alpha and beta releases leading up to GA
also gives downstreams a nice amount of time for testing and validation.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message