hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Evans <ev...@yahoo-inc.com>
Subject Re: Instructions on how to run Apache Hadoop 2.0.2-alpha?
Date Fri, 04 Jan 2013 23:28:38 GMT
It is very long and confusing. Here is my understanding of what happened even though I was
not around for all of it.

0.1 - 0.20 was mostly main line development.  At that point there was a split and 0.20 was
forked to add security, 0.20-security, and also to add in append support for H-BASE 0.20-append.
 For a long time these branches were maintained and has bug fixes added to it while main line
development went on with 0.21 and 0.22.  0.20-security was merged with 0.20-append and was
renamed 1.0.  It has continued with bug fixes and some new feature work back ported from trunk.

But development did not stop at 0.22.  0.23 was created to support YARN/MR-V2.  At about the
same time 0.24 was made for trunk.  When 0.20-security+append became 1.0, 0.23 became 2.0
and 0.24 became 3.0.  We at Yahoo! Have been in the process of stabilizing YARN for about
a year or so.  Because 2.0 looked like new features were going to continue to go in we decided
to keep 0.23 alive for stabilization purposes while 2.0 continued on.  It has since gotten
several new features like HDFS-HA and protocol buffer based RPC for HDFS, and a few new features
for YARN/MR.  So you can almost think of 0.23 as 2.0--.  We hope to move to 2.0 in the near
future once it stabilizes and loses the alpha tag, at which point 0.23 would go away.

--Bobby Evans

From: Glen Mazza <gmazza@talend.com<mailto:gmazza@talend.com>>
Reply-To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Date: Friday, January 4, 2013 10:30 AM
To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Subject: Re: Instructions on how to run Apache Hadoop 2.0.2-alpha?

OK, looking at the Hadoop branches: http://svn.apache.org/viewvc/hadoop/common/branches/ and
tags: http://svn.apache.org/viewvc/hadoop/common/tags/, it's thankfully not that bad.
There is no 0.24 in Hadoop, and the Maven pom files for the 2.0.2-alpha branch indeed say
2.0.2-alpha.  The Cloudera article linked to below mentions 0.24, but it's from 2011 and may
have been at a time before the Hadoop team decided to relabel to 2.0 instead.  Looking at
this again: http://hadoop.apache.org/docs/r2.0.2-alpha/hadoop-yarn/hadoop-yarn-site/SingleCluster.html,
it appears to provide workable instructions, I'll try again with it.  Thanks for your assistance.


On 01/04/2013 11:17 AM, Chen He wrote:
Hi Glen

I agree with you. There are many versions and confusing. If you really want to know, you can
check the developing and publishing documents. In 0.23's document, they must announced how
many patches are included, as well as the 2.0.x.

Then you will exactly understand which one is which one.


On Fri, Jan 4, 2013 at 6:58 AM, Glen Mazza <gmazza@talend.com<mailto:gmazza@talend.com>>
Thanks for your response.  That's pretty vital information--I'm not used to separate development/publication
versions.  Is the 0.23-->2.0.2 renumbering stated anywhere on the Hadoop website or wiki?
 It's very confusing as people otherwise think there are three separate branches -- 0.2x.y,
1.x, and 2.x when there are really just two.

Also the releases page: http://hadoop.apache.org/releases.html mentions *both* 0.23.x and
2.0.2 releases -- wouldn't the former be numbered as the latter, if as you're saying, 0.23
is published as 2.0.x?


On 01/04/2013 07:02 AM, Chen He wrote:
0.24 is the developing version, 2.0.2 is the publication version.

0.23 or above is published as 2.0.x

On Fri, Jan 4, 2013 at 5:45 AM, Glen Mazza <gmazza@talend.com<mailto:gmazza@talend.com>>
Actually, those instructions are for Hadoop 0.24, not 2.0.2-alpha.


On 11/30/2012 03:40 PM, Cristian Cira wrote:
Dear Glen,

try http://blog.cloudera.com/blog/2011/11/building-and-deploying-mr2/

Cristian Cira
Graduate Research Assistant
Parallel Architecture and System Laboratory(PASL)
Shelby Center 2105
Auburn University, AL 36849

From: Glen Mazza [gmazza@talend.com<mailto:gmazza@talend.com>]
Sent: Friday, November 30, 2012 2:37 PM
To: user@hadoop.apache.org<mailto:user@hadoop.apache.org>
Subject: Instructions on how to run Apache Hadoop 2.0.2-alpha?

Hi, I was able to run Hadoop 1.0.4 single-node with no difficulty following these instructions:
 http://hadoop.apache.org/docs/stable/single_node_setup.html .

The corresponding page that we're referred to at the bottom of here: http://hadoop.apache.org/docs/current/
for Hadoop 2.0.2 is radically different, however, it doesn't seem to have anything to do with
starting Hadoop:  http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/SingleCluster.html
.  Is it because I'm looking at the wrong page or is it that Hadoop 2.0.2 is really so different
that the instructions have radically changed?  Is there anything similar to http://hadoop.apache.org/docs/stable/single_node_setup.html
that will show me how to run Hadoop 2.0.2?


Glen Mazza
Talend Community Coders - coders.talend.com<http://coders.talend.com>
blog: www.jroller.com/gmazza<http://www.jroller.com/gmazza><http://www.jroller.com/gmazza>

Glen Mazza
Talend Community Coders - coders.talend.com<http://coders.talend.com>
blog: www.jroller.com/gmazza<http://www.jroller.com/gmazza>

Glen Mazza
Talend Community Coders - coders.talend.com<http://coders.talend.com>
blog: www.jroller.com/gmazza<http://www.jroller.com/gmazza>

Glen Mazza
Talend Community Coders - coders.talend.com
blog: www.jroller.com/gmazza<http://www.jroller.com/gmazza>

View raw message