hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Allen Wittenauer ...@altiscale.com>
Subject In hindsight... Re: Thinking ahead to hadoop-2.6
Date Mon, 15 Sep 2014 17:48:18 GMT

	It’s now September.  With the passage of time, I have a lot of doubts about this plan and
where that trajectory takes us.

* The list of changes that are already in branch-2 scare the crap out of any risk adverse
person (Hello to my fellow operations people!). Not only are the number of changes extremely
high, but in addition there are a lot of major, blockbuster features in what is supposed to
be a minor release.  Combined with the fact that we’ve had to do some micro releases, it
seems to hint that branch-2 is getting less stable over time.

*  One of the plans talked about was rolling a 2.7 release that drops JDK6 and makes JDK7
the standard.  If 2.7 comes after 2.6 in October, date wise  makes it somewhere around January
2015.  JDK7 EOL’s in April 2015.  So we’ll have a viable JDK7 release for exactly 3 months.
 Frankly, it is too late for us to talk about JDK7 and we need to start thinking about JDK8.
 

* trunk is currently sitting at 3 years old.  There is a lot of stuff that has been hanging
around that really needs to get into people hands so that we can start stabilizing it for
a “real” release.  


To me this all says one thing:

	Drop the 2.6.0 release, branch trunk, and start rolling a 3.0.0-alpha with JDK8 as the minimum.
 2.5.1 becomes the base for all sustaining work.  This gives the rest of the community time
to move to JDK8 if they haven’t already.  For downstream vendors, it gives a roadmap for
their customers who will be asking about JDK8 sooner rather than later.  By the time 3.0 stabilizes,
we’re probably looking at April, which is perfect timing.

	One of the issues I’ve heard mention is that 3.0 doesn’t have anything “compelling”
in it.  Well, dropping 2.6 makes the feature list the carrot, JDK8 support is obviously the
stick.

	Thoughts?

	


On Aug 15, 2014, at 6:07 PM, Subramaniam Krishnan <subru@apache.org> wrote:

> Thanks for initiating the thread Arun.
> 
> Can we add YARN-1051 <https://issues.apache.org/jira/browse/YARN-1051> to
> the list? We have most of the patches for the sub-JIRAs under review and
> have committed a couple.
> 
> -Subru
> 
> ---------- Forwarded message ----------
> 
> From: Arun C Murthy <acm@hortonworks.com>
> 
> Date: Tue, Aug 12, 2014 at 1:34 PM
> 
> Subject: Thinking ahead to hadoop-2.6
> 
> To: "common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>, "
> hdfs-dev@hadoop.apache.org" <hdfs-dev@hadoop.apache.org>, "
> mapreduce-dev@hadoop.apache.org" <mapreduce-dev@hadoop.apache.org>,
> 
> "yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>
> 
> 
> 
> 
> 
> Folks,
> 
> 
> 
> With hadoop-2.5 nearly done, it's time to start thinking ahead to
> hadoop-2.6.
> 
> 
> 
> Currently, here is the Roadmap per the wiki:
> 
> 
> 
>        • HADOOP
> 
>                • Credential provider HADOOP-10607
> 
>        • HDFS
> 
>                • Heterogeneous storage (Phase 2) - Support APIs for using
> storage tiers by the applications HDFS-5682
> 
>                • Memory as storage tier HDFS-5851
> 
>        • YARN
> 
>                • Dynamic Resource Configuration YARN-291
> 
>                • NodeManager Restart YARN-1336
> 
>                • ResourceManager HA Phase 2 YARN-556
> 
>                • Support for admin-specified labels in YARN YARN-796
> 
>                • Support for automatic, shared cache for YARN application
> artifacts YARN-1492
> 
>                • Support NodeGroup layer topology on YARN YARN-18
> 
>                • Support for Docker containers in YARN YARN-1964
> 
>                • YARN service registry YARN-913
> 
> 
> 
> My suspicion is, as is normal, some will make the cut and some won't.
> 
> Please do add/subtract from the list as appropriate. Ideally, it would be
> good to ship hadoop-2.6 in a 6-8 weeks (say, October) to keep up a cadence.
> 
> 
> 
> More importantly, as we discussed previously, we'd like hadoop-2.6 to be
> the *last* Apache Hadoop 2.x release which support JDK6. I'll start a
> discussion with other communities (HBase, Pig, Hive, Oozie etc.) and see
> how they feel about this.
> 
> 
> 
> thanks,
> 
> Arun
> 
> 
> 
> 
> 
> --
> 
> CONFIDENTIALITY NOTICE
> 
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.


Mime
View raw message