hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom White <...@cloudera.com>
Subject Re: Hadoop 0.22/HBase 0.92 package repos are now available
Date Thu, 29 Sep 2011 18:57:31 GMT
On Wed, Sep 28, 2011 at 6:28 PM, Roman Shaposhnik <rvs@apache.org> wrote:
> On Wed, Sep 28, 2011 at 11:54 AM,  <Milind.Bhandarkar@emc.com> wrote:
>> This is great progress folks !!!
>
> Thanks. Actually, for all those who care about .23 I've created a hadoop-0.23
> branch in Bigtop and hooked up 3 additional Jenkins jobs:
>     http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop23-centos5/
>     http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop23-ubuntu-lucid/
>     http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop23/
>
> The good news is that HDFS and common are ok, can be packaged and
> uses. The bad news for .23 are way more plentiful though:
>     1. HBase now breaks with the profile that is supposed to be compatible
>     with .23. Perhaps this is one of the issues that Todd has mentioned
>     earlier, but I'm not sure.
>
>     2. MR2 (or more precise YARN) seem to be in a pretty weird shape.
>     First of all, I can't really build it the same way that the rest of the
>     Hadoop 0.23 builds (which is what specified in top-level BUILDING.txt).
>     Somehow it requires assembly:single goal to be executed explicitly.
>     Worse yet, it seems to be following a different binary layout compared to
>     HDFS and common (conf vs. etc/hadoop, etc.). Also, I'm not sure how
>     to make natives bits of pipes/utils compile short of manual execution
>     of ant.
>
>     3. fuse-dfs is now an orphan. Its not hooked to the build system and seems
>     to be in limbo: https://issues.apache.org/jira/browse/HDFS-1757
>
> I would really, really, really appreciate if somebody can help triaging these
> issues or at least would be nice enough to point me to existing
> JIRAs/discussions.

Regarding 2. At the moment you can build MR following the instructions
in the INSTALL file. In fact the nightly build set up by Giri and
Mahadev builds a MR tarball
(https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-MapReduce-trunk).
There are some JIRAs to remove the inconsistencies with the rest of
the build, such as MAPREDUCE-3024, MAPREDUCE-3014, HADOOP-7590,
HADOOP-7642. I hope to work on some of these soon, but any help is
always appreciated.

Cheers,
Tom

>
> My goal here is to get us to a point where we can start executing
> automated integration testing against a fully distributed cluster and
> start publishing these results on the same Jenkins to monitor the
> health and progress of .22/.23 and corresponding Bigtop stacks.
>
> Thanks,
> Roman.
>

Mime
View raw message