hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravi Teja <ravit...@huawei.com>
Subject Re: trouble building MR-279 branch
Date Thu, 28 Jul 2011 11:38:40 GMT

Hi,

I have followed the procedure mentioned below, for building MRv2, but was
not able to succeed with the task. I have got some compilation errors. 


Please help me resolving this issue.



[INFO]
------------------------------------------------------------------------ 
[INFO] Reactor Build Order: 
[INFO] 
[INFO] yarn-api 
[INFO] yarn-common 
[INFO] yarn-server-common 
[INFO] yarn-server-nodemanager 
[INFO] yarn-server-resourcemanager 
[INFO] yarn-server-tests 
[INFO] yarn-server 
[INFO] yarn 
[INFO] hadoop-mapreduce-client-core 
[INFO] hadoop-mapreduce-client-common 
[INFO] hadoop-mapreduce-client-shuffle 
[INFO] hadoop-mapreduce-client-app 
[INFO] hadoop-mapreduce-client-hs 
[INFO] hadoop-mapreduce-client-jobclient 
[INFO] hadoop-mapreduce-client 
[INFO] hadoop-mapreduce 
[INFO]

[INFO]
------------------------------------------------------------------------ 
[INFO] Building yarn-api 1.0-SNAPSHOT 
[INFO]
------------------------------------------------------------------------ 
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run
(create-protobuf-generated-sources-directory) @ yarn-api --- 
[INFO] Executing tasks 

main: 
[INFO] Executed tasks 
[INFO] 
[INFO] --- exec-maven-plugin:1.2:exec (generate-sources) @ yarn-api --- 
[INFO] 
[INFO] --- build-helper-maven-plugin:1.5:add-source (add-source) @ yarn-api
--- 
[INFO] Source directory:
D:\Code\MR-279\mapreduce\yarn\yarn-api\target\generated-sources\proto added.

[INFO] 
[INFO] --- maven-resources-plugin:2.4.3:resources (default-resources) @
yarn-api --- 
[INFO] Using 'UTF-8' encoding to copy filtered resources. 
[INFO] skip non existing resourceDirectory
D:\Code\MR-279\mapreduce\yarn\yarn-api\src\main\resources 
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ yarn-api
--- 
[INFO] Compiling 121 source files to
D:\Code\MR-279\mapreduce\yarn\yarn-api\target\classes 
[INFO] ------------------------------------------------------------- 
[ERROR] COMPILATION ERROR : 
[INFO] ------------------------------------------------------------- 
[ERROR]
\Code\MR-279\mapreduce\yarn\yarn-api\src\main\java\org\apache\hadoop\yarn\ap
i\records\impl\pb\ContainerPBImpl.java:[22,33] package
org.apache.commons.logging does not exist 
[ERROR]
\Code\MR-279\mapreduce\yarn\yarn-api\src\main\java\org\apache\hadoop\yarn\ap
i\records\impl\pb\ContainerPBImpl.java:[23,33] package
org.apache.commons.logging does not exist 
[INFO] 2 errors 
[INFO] ------------------------------------------------------------- 
[INFO]
------------------------------------------------------------------------ 
[INFO] Reactor Summary: 
[INFO] 
[INFO] yarn-api .......................................... FAILURE [5.610s] 
[INFO] yarn-common ....................................... SKIPPED 
[INFO] yarn-server-common ................................ SKIPPED 
[INFO] yarn-server-nodemanager ........................... SKIPPED 
[INFO] yarn-server-resourcemanager ....................... SKIPPED 
[INFO] yarn-server-tests ................................. SKIPPED 
[INFO] yarn-server ....................................... SKIPPED 
[INFO] yarn .............................................. SKIPPED 
[INFO] hadoop-mapreduce-client-core ...................... SKIPPED 
[INFO] hadoop-mapreduce-client-common .................... SKIPPED 
[INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED 
[INFO] hadoop-mapreduce-client-app ....................... SKIPPED 
[INFO] hadoop-mapreduce-client-hs ........................ SKIPPED 
[INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED 
[INFO] hadoop-mapreduce-client ........................... SKIPPED 
[INFO] hadoop-mapreduce .................................. SKIPPED 
[INFO]
------------------------------------------------------------------------ 
[INFO] BUILD FAILURE 
[INFO]
------------------------------------------------------------------------ 
[INFO] Total time: 5.844s 
[INFO] Finished at: Thu Jul 28 16:03:07 IST 2011 
[INFO] Final Memory: 9M/35M 
[INFO]
------------------------------------------------------------------------ 
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.3.2:compile
(default-compile) on project yarn-api: Compilation failure: Compilation
failure: 
[ERROR]
\Code\MR-279\mapreduce\yarn\yarn-api\src\main\java\org\apache\hadoop\yarn\ap
i\records\impl\pb\ContainerPBImpl.java:[22,33] package
org.apache.commons.logging does not exist 
[ERROR]
\Code\MR-279\mapreduce\yarn\yarn-api\src\main\java\org\apache\hadoop\yarn\ap
i\records\impl\pb\ContainerPBImpl.java:[23,33] package
org.apache.commons.logging does not exist 
[ERROR] -> [Help 1]

Regards,
Ravi Teja






-----Original Message-----
From: Josh Wills [mailto:jwills@cloudera.com] 
Sent: Saturday, July 02, 2011 6:34 AM
To: mapreduce-dev@hadoop.apache.org
Subject: Re: trouble building MR-279 branch

Hey Anupam,

I think Jim is asking about building non-MR application masters using the
RM/NM. I think Arun has promised to post a design doc soon. In the meantime,
I'm just reading through the code and fixing little issues I come across
(e.g., https://issues.apache.org/jira/browse/MAPREDUCE-2630 and
https://issues.apache.org/jira/browse/MAPREDUCE-2633).

Jim, what sort of app are you looking to build? I'd be happy to have some
company wading through the code.

Best,
Josh

On Fri, Jul 1, 2011 at 3:02 PM, Anupam Seth <anupams@yahoo-inc.com> wrote:

> Hi Jim,
>
> Not sure if this helps at all, but not sure if you could get an idea by
> looking at the included examples:
>
>
>
http://svn.apache.org/viewvc/hadoop/common/branches/MR-279/mapreduce/src/exa
mples/org/apache/hadoop/examples
>
> that can be run as:
>
> hadoop jar $HADOOP_MAPRED_HOME/hadoop-mapred-examples-0.22.0-SNAPSHOT.jar
> wordcount input output
>
> Regards,
> Anupam
>
> -----Original Message-----
> From: Jim Falgout [mailto:jim.falgout@pervasive.com]
> Sent: Friday, July 01, 2011 4:23 PM
> To: mapreduce-dev@hadoop.apache.org
> Subject: RE: trouble building MR-279 branch
>
> Thanks Anupam! I had skipped the step to get protobuf into the maven
> repository. Also, I had Apache Forrest version 0.9 installed. That caused
> the builds to fail on generating some docs. Installed version 0.8 and it
> worked smoothly.
>
> On another note: do you know of any design docs for building an
application
> executor using the new resource manager? Just wondering how to get
started?
>
> -----Original Message-----
> From: Anupam Seth [mailto:anupams@yahoo-inc.com]
> Sent: Friday, July 01, 2011 2:10 PM
> To: mapreduce-dev@hadoop.apache.org
> Subject: RE: trouble building MR-279 branch
>
> Hi Jim,
>
> Following are some notes (thanks to Jeff Naisbitt & Bobby Evans).
>
> Build YARN
> ==========
> These are in addition to what you would normally do.
>
> Install protobuf
> use Protobuf 2.4.0a (Download from
> http://code.google.com/p/protobuf/downloads/list) I actually used 2.4.1
> and it worked
>
>    install the protoc executable (configure, make, make install)
>    install the maven artifact (cd java; mvn install)
>
> Installing protoc requires gcc 4.1.x or higher (RHEL5).
>
> If the make step fails with (Valid until a fix is released for protobuf
> 2.4.0a) ./google/protobuf/descriptor.h:1152: error:
>
`google::protobuf::internal::Mutex*google::protobuf::DescriptorPool::mutex_'
> is private Replace descriptor.cc with
>
http://protobuf.googlecode.com/svn-history/r380/trunk/src/google/protobuf/de
scriptor.cc
>
> Setup environment
>
> export MAVEN_OPTS=-Xmx512m
> export LD_LIBRARY_PATH=/usr/local/lib
>
> Checkout the MR-279 branch
> svn checkout http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279
>
> Build common
>
> cd MR-279/common
> ant veryclean mvn-install tar
>
> Build hdfs
>
> cd ../hdfs
> ant veryclean mvn-install -Dresolvers=internal tar
>
> Build yarn
>
> cd ../mapreduce
> mvn clean install assembly:assembly
> # If tests are failing, add this option to skip them:
> -Dmaven.test.skip.exec=true # NOTE: in order to build on a Mac, you must
> also include the <code>-P-cbuild</code> option # (e.g. <code>mvn clean
> install assembly:assembly -P-cbuild -Dmaven.test.skip.exec=true</code>)
> ant veryclean jar jar-test tar -Dresolvers=internal
>
> -----Original Message-----
> From: Jim Falgout [mailto:jim.falgout@pervasive.com]
> Sent: Friday, July 01, 2011 2:00 PM
> To: mapreduce-dev@hadoop.apache.org
> Subject: trouble building MR-279 branch
>
> Hi,
>
> I've checked out the MR-279 branch and am having trouble getting it to
> build. Any docs/blogs/notes around that may give some hints?
>
> The build is unable to resolve the yarn-server even though I ran a
> "mvn-install" in the yarn source area to install it in the local
repository.
>
> Thanks,
> Jim.
>
>
>


Mime
View raw message