hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anupam Seth <anup...@yahoo-inc.com>
Subject RE: trouble building MR-279 branch
Date Fri, 01 Jul 2011 22:02:56 GMT
Hi Jim,

Not sure if this helps at all, but not sure if you could get an idea by looking at the included
examples: 

http://svn.apache.org/viewvc/hadoop/common/branches/MR-279/mapreduce/src/examples/org/apache/hadoop/examples

that can be run as:

hadoop jar $HADOOP_MAPRED_HOME/hadoop-mapred-examples-0.22.0-SNAPSHOT.jar wordcount input
output

Regards,
Anupam

-----Original Message-----
From: Jim Falgout [mailto:jim.falgout@pervasive.com] 
Sent: Friday, July 01, 2011 4:23 PM
To: mapreduce-dev@hadoop.apache.org
Subject: RE: trouble building MR-279 branch

Thanks Anupam! I had skipped the step to get protobuf into the maven repository. Also, I had
Apache Forrest version 0.9 installed. That caused the builds to fail on generating some docs.
Installed version 0.8 and it worked smoothly.

On another note: do you know of any design docs for building an application executor using
the new resource manager? Just wondering how to get started?

-----Original Message-----
From: Anupam Seth [mailto:anupams@yahoo-inc.com] 
Sent: Friday, July 01, 2011 2:10 PM
To: mapreduce-dev@hadoop.apache.org
Subject: RE: trouble building MR-279 branch

Hi Jim,

Following are some notes (thanks to Jeff Naisbitt & Bobby Evans).

Build YARN
==========
These are in addition to what you would normally do.

Install protobuf
use Protobuf 2.4.0a (Download from http://code.google.com/p/protobuf/downloads/list) I actually
used 2.4.1 and it worked

    install the protoc executable (configure, make, make install)
    install the maven artifact (cd java; mvn install) 

Installing protoc requires gcc 4.1.x or higher (RHEL5).

If the make step fails with (Valid until a fix is released for protobuf 2.4.0a) ./google/protobuf/descriptor.h:1152:
error: `google::protobuf::internal::Mutex*google::protobuf::DescriptorPool::mutex_' is private
Replace descriptor.cc with http://protobuf.googlecode.com/svn-history/r380/trunk/src/google/protobuf/descriptor.cc

Setup environment

export MAVEN_OPTS=-Xmx512m
export LD_LIBRARY_PATH=/usr/local/lib

Checkout the MR-279 branch
svn checkout http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279

Build common

cd MR-279/common
ant veryclean mvn-install tar

Build hdfs

cd ../hdfs
ant veryclean mvn-install -Dresolvers=internal tar

Build yarn

cd ../mapreduce
mvn clean install assembly:assembly
# If tests are failing, add this option to skip them: -Dmaven.test.skip.exec=true # NOTE:
in order to build on a Mac, you must also include the <code>-P-cbuild</code> option
# (e.g. <code>mvn clean install assembly:assembly -P-cbuild -Dmaven.test.skip.exec=true</code>)
ant veryclean jar jar-test tar -Dresolvers=internal

-----Original Message-----
From: Jim Falgout [mailto:jim.falgout@pervasive.com]
Sent: Friday, July 01, 2011 2:00 PM
To: mapreduce-dev@hadoop.apache.org
Subject: trouble building MR-279 branch

Hi,

I've checked out the MR-279 branch and am having trouble getting it to build. Any docs/blogs/notes
around that may give some hints?

The build is unable to resolve the yarn-server even though I ran a "mvn-install" in the yarn
source area to install it in the local repository.

Thanks,
Jim.



Mime
View raw message