hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravi Teja <ravit...@huawei.com>
Subject RE: trouble building MR-279 branch
Date Fri, 05 Aug 2011 10:59:26 GMT
Hi,

The Hudson build is failing now with the following error.
I am also facing following error. How is this resolved, as this is the first
step of the build by which, even the yarn jars don't compile, neither the
jars are present in the repositories.

	Even if we install the jars manually from the previous Hudson
build,it may fail the compilation if any code changes are done after it.

>From where are the jars taken, to resolve this dependency?

Please help me resolving this issue.


[ivy:resolve]
::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve]                                   ::          UNRESOLVED
DEPENDENCIES         ::
[ivy:resolve]
::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve]                                   ::
org.apache.hadoop#yarn-server-common;0.279.0: not found
[ivy:resolve]                                   ::
org.apache.hadoop#hadoop-mapreduce-client-core;0.279.0: not found
[ivy:resolve]                                   ::
org.apache.hadoop#yarn-common;0.279.0: not found
[ivy:resolve]
::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve] 
[ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS


Thanks and Regards,
Ravi Teja


-----Original Message-----
From: Ravi Teja [mailto:raviteja@huawei.com] 
Sent: Thursday, July 28, 2011 5:09 PM
To: 'mapreduce-dev@hadoop.apache.org'
Subject: Re: trouble building MR-279 branch


Hi,

I have followed the procedure mentioned below, for building MRv2, but was
not able to succeed with the task. I have got some compilation errors. 


Please help me resolving this issue.



[INFO]
------------------------------------------------------------------------ 
[INFO] Reactor Build Order: 
[INFO] 
[INFO] yarn-api 
[INFO] yarn-common 
[INFO] yarn-server-common 
[INFO] yarn-server-nodemanager 
[INFO] yarn-server-resourcemanager 
[INFO] yarn-server-tests 
[INFO] yarn-server 
[INFO] yarn 
[INFO] hadoop-mapreduce-client-core 
[INFO] hadoop-mapreduce-client-common 
[INFO] hadoop-mapreduce-client-shuffle 
[INFO] hadoop-mapreduce-client-app 
[INFO] hadoop-mapreduce-client-hs 
[INFO] hadoop-mapreduce-client-jobclient 
[INFO] hadoop-mapreduce-client 
[INFO] hadoop-mapreduce 
[INFO]

[INFO]
------------------------------------------------------------------------ 
[INFO] Building yarn-api 1.0-SNAPSHOT 
[INFO]
------------------------------------------------------------------------ 
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run
(create-protobuf-generated-sources-directory) @ yarn-api --- 
[INFO] Executing tasks 

main: 
[INFO] Executed tasks 
[INFO] 
[INFO] --- exec-maven-plugin:1.2:exec (generate-sources) @ yarn-api --- 
[INFO] 
[INFO] --- build-helper-maven-plugin:1.5:add-source (add-source) @ yarn-api
--- 
[INFO] Source directory:
D:\Code\MR-279\mapreduce\yarn\yarn-api\target\generated-sources\proto added.

[INFO] 
[INFO] --- maven-resources-plugin:2.4.3:resources (default-resources) @
yarn-api --- 
[INFO] Using 'UTF-8' encoding to copy filtered resources. 
[INFO] skip non existing resourceDirectory
D:\Code\MR-279\mapreduce\yarn\yarn-api\src\main\resources 
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ yarn-api
--- 
[INFO] Compiling 121 source files to
D:\Code\MR-279\mapreduce\yarn\yarn-api\target\classes 
[INFO] ------------------------------------------------------------- 
[ERROR] COMPILATION ERROR : 
[INFO] ------------------------------------------------------------- 
[ERROR]
\Code\MR-279\mapreduce\yarn\yarn-api\src\main\java\org\apache\hadoop\yarn\ap
i\records\impl\pb\ContainerPBImpl.java:[22,33] package
org.apache.commons.logging does not exist 
[ERROR]
\Code\MR-279\mapreduce\yarn\yarn-api\src\main\java\org\apache\hadoop\yarn\ap
i\records\impl\pb\ContainerPBImpl.java:[23,33] package
org.apache.commons.logging does not exist 
[INFO] 2 errors 
[INFO] ------------------------------------------------------------- 
[INFO]
------------------------------------------------------------------------ 
[INFO] Reactor Summary: 
[INFO] 
[INFO] yarn-api .......................................... FAILURE [5.610s] 
[INFO] yarn-common ....................................... SKIPPED 
[INFO] yarn-server-common ................................ SKIPPED 
[INFO] yarn-server-nodemanager ........................... SKIPPED 
[INFO] yarn-server-resourcemanager ....................... SKIPPED 
[INFO] yarn-server-tests ................................. SKIPPED 
[INFO] yarn-server ....................................... SKIPPED 
[INFO] yarn .............................................. SKIPPED 
[INFO] hadoop-mapreduce-client-core ...................... SKIPPED 
[INFO] hadoop-mapreduce-client-common .................... SKIPPED 
[INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED 
[INFO] hadoop-mapreduce-client-app ....................... SKIPPED 
[INFO] hadoop-mapreduce-client-hs ........................ SKIPPED 
[INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED 
[INFO] hadoop-mapreduce-client ........................... SKIPPED 
[INFO] hadoop-mapreduce .................................. SKIPPED 
[INFO]
------------------------------------------------------------------------ 
[INFO] BUILD FAILURE 
[INFO]
------------------------------------------------------------------------ 
[INFO] Total time: 5.844s 
[INFO] Finished at: Thu Jul 28 16:03:07 IST 2011 
[INFO] Final Memory: 9M/35M 
[INFO]
------------------------------------------------------------------------ 
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.3.2:compile
(default-compile) on project yarn-api: Compilation failure: Compilation
failure: 
[ERROR]
\Code\MR-279\mapreduce\yarn\yarn-api\src\main\java\org\apache\hadoop\yarn\ap
i\records\impl\pb\ContainerPBImpl.java:[22,33] package
org.apache.commons.logging does not exist 
[ERROR]
\Code\MR-279\mapreduce\yarn\yarn-api\src\main\java\org\apache\hadoop\yarn\ap
i\records\impl\pb\ContainerPBImpl.java:[23,33] package
org.apache.commons.logging does not exist 
[ERROR] -> [Help 1]

Regards,
Ravi Teja






-----Original Message-----
From: Josh Wills [mailto:jwills@cloudera.com] 
Sent: Saturday, July 02, 2011 6:34 AM
To: mapreduce-dev@hadoop.apache.org
Subject: Re: trouble building MR-279 branch

Hey Anupam,

I think Jim is asking about building non-MR application masters using the
RM/NM. I think Arun has promised to post a design doc soon. In the meantime,
I'm just reading through the code and fixing little issues I come across
(e.g., https://issues.apache.org/jira/browse/MAPREDUCE-2630 and
https://issues.apache.org/jira/browse/MAPREDUCE-2633).

Jim, what sort of app are you looking to build? I'd be happy to have some
company wading through the code.

Best,
Josh

On Fri, Jul 1, 2011 at 3:02 PM, Anupam Seth <anupams@yahoo-inc.com> wrote:

> Hi Jim,
>
> Not sure if this helps at all, but not sure if you could get an idea by
> looking at the included examples:
>
>
>
http://svn.apache.org/viewvc/hadoop/common/branches/MR-279/mapreduce/src/exa
mples/org/apache/hadoop/examples
>
> that can be run as:
>
> hadoop jar $HADOOP_MAPRED_HOME/hadoop-mapred-examples-0.22.0-SNAPSHOT.jar
> wordcount input output
>
> Regards,
> Anupam
>
> -----Original Message-----
> From: Jim Falgout [mailto:jim.falgout@pervasive.com]
> Sent: Friday, July 01, 2011 4:23 PM
> To: mapreduce-dev@hadoop.apache.org
> Subject: RE: trouble building MR-279 branch
>
> Thanks Anupam! I had skipped the step to get protobuf into the maven
> repository. Also, I had Apache Forrest version 0.9 installed. That caused
> the builds to fail on generating some docs. Installed version 0.8 and it
> worked smoothly.
>
> On another note: do you know of any design docs for building an
application
> executor using the new resource manager? Just wondering how to get
started?
>
> -----Original Message-----
> From: Anupam Seth [mailto:anupams@yahoo-inc.com]
> Sent: Friday, July 01, 2011 2:10 PM
> To: mapreduce-dev@hadoop.apache.org
> Subject: RE: trouble building MR-279 branch
>
> Hi Jim,
>
> Following are some notes (thanks to Jeff Naisbitt & Bobby Evans).
>
> Build YARN
> ==========
> These are in addition to what you would normally do.
>
> Install protobuf
> use Protobuf 2.4.0a (Download from
> http://code.google.com/p/protobuf/downloads/list) I actually used 2.4.1
> and it worked
>
>    install the protoc executable (configure, make, make install)
>    install the maven artifact (cd java; mvn install)
>
> Installing protoc requires gcc 4.1.x or higher (RHEL5).
>
> If the make step fails with (Valid until a fix is released for protobuf
> 2.4.0a) ./google/protobuf/descriptor.h:1152: error:
>
`google::protobuf::internal::Mutex*google::protobuf::DescriptorPool::mutex_'
> is private Replace descriptor.cc with
>
http://protobuf.googlecode.com/svn-history/r380/trunk/src/google/protobuf/de
scriptor.cc
>
> Setup environment
>
> export MAVEN_OPTS=-Xmx512m
> export LD_LIBRARY_PATH=/usr/local/lib
>
> Checkout the MR-279 branch
> svn checkout http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279
>
> Build common
>
> cd MR-279/common
> ant veryclean mvn-install tar
>
> Build hdfs
>
> cd ../hdfs
> ant veryclean mvn-install -Dresolvers=internal tar
>
> Build yarn
>
> cd ../mapreduce
> mvn clean install assembly:assembly
> # If tests are failing, add this option to skip them:
> -Dmaven.test.skip.exec=true # NOTE: in order to build on a Mac, you must
> also include the <code>-P-cbuild</code> option # (e.g. <code>mvn clean
> install assembly:assembly -P-cbuild -Dmaven.test.skip.exec=true</code>)
> ant veryclean jar jar-test tar -Dresolvers=internal
>
> -----Original Message-----
> From: Jim Falgout [mailto:jim.falgout@pervasive.com]
> Sent: Friday, July 01, 2011 2:00 PM
> To: mapreduce-dev@hadoop.apache.org
> Subject: trouble building MR-279 branch
>
> Hi,
>
> I've checked out the MR-279 branch and am having trouble getting it to
> build. Any docs/blogs/notes around that may give some hints?
>
> The build is unable to resolve the yarn-server even though I ran a
> "mvn-install" in the yarn source area to install it in the local
repository.
>
> Thanks,
> Jim.
>
>
>


Mime
View raw message