hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mahadev Konar <maha...@hortonworks.com>
Subject Re: trouble building MR-279 branch
Date Fri, 05 Aug 2011 16:44:21 GMT
Thanks for pointing out Ravi. We are working on fixing it. The jars that you
mention are created as part of MR-279 build.

For building MR-279, all you have to do in mvn clean install -DskipTests and
that should work (given you have already installed common and hdfs from
trunk).

thanks
mahadev


On Fri, Aug 5, 2011 at 3:59 AM, Ravi Teja <raviteja@huawei.com> wrote:

> Hi,
>
> The Hudson build is failing now with the following error.
> I am also facing following error. How is this resolved, as this is the
> first
> step of the build by which, even the yarn jars don't compile, neither the
> jars are present in the repositories.
>
>        Even if we install the jars manually from the previous Hudson
> build,it may fail the compilation if any code changes are done after it.
>
> From where are the jars taken, to resolve this dependency?
>
> Please help me resolving this issue.
>
>
> [ivy:resolve]
> ::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:resolve]                                   ::          UNRESOLVED
> DEPENDENCIES         ::
> [ivy:resolve]
> ::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:resolve]                                   ::
> org.apache.hadoop#yarn-server-common;0.279.0: not found
> [ivy:resolve]                                   ::
> org.apache.hadoop#hadoop-mapreduce-client-core;0.279.0: not found
> [ivy:resolve]                                   ::
> org.apache.hadoop#yarn-common;0.279.0: not found
> [ivy:resolve]
> ::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:resolve]
> [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
>
>
> Thanks and Regards,
> Ravi Teja
>
>
> -----Original Message-----
> From: Ravi Teja [mailto:raviteja@huawei.com]
> Sent: Thursday, July 28, 2011 5:09 PM
> To: 'mapreduce-dev@hadoop.apache.org'
> Subject: Re: trouble building MR-279 branch
>
>
> Hi,
>
> I have followed the procedure mentioned below, for building MRv2, but was
> not able to succeed with the task. I have got some compilation errors.
>
>
> Please help me resolving this issue.
>
>
>
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Reactor Build Order:
> [INFO]
> [INFO] yarn-api
> [INFO] yarn-common
> [INFO] yarn-server-common
> [INFO] yarn-server-nodemanager
> [INFO] yarn-server-resourcemanager
> [INFO] yarn-server-tests
> [INFO] yarn-server
> [INFO] yarn
> [INFO] hadoop-mapreduce-client-core
> [INFO] hadoop-mapreduce-client-common
> [INFO] hadoop-mapreduce-client-shuffle
> [INFO] hadoop-mapreduce-client-app
> [INFO] hadoop-mapreduce-client-hs
> [INFO] hadoop-mapreduce-client-jobclient
> [INFO] hadoop-mapreduce-client
> [INFO] hadoop-mapreduce
> [INFO]
>
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Building yarn-api 1.0-SNAPSHOT
> [INFO]
> ------------------------------------------------------------------------
> [INFO]
> [INFO] --- maven-antrun-plugin:1.6:run
> (create-protobuf-generated-sources-directory) @ yarn-api ---
> [INFO] Executing tasks
>
> main:
> [INFO] Executed tasks
> [INFO]
> [INFO] --- exec-maven-plugin:1.2:exec (generate-sources) @ yarn-api ---
> [INFO]
> [INFO] --- build-helper-maven-plugin:1.5:add-source (add-source) @ yarn-api
> ---
> [INFO] Source directory:
> D:\Code\MR-279\mapreduce\yarn\yarn-api\target\generated-sources\proto
> added.
>
> [INFO]
> [INFO] --- maven-resources-plugin:2.4.3:resources (default-resources) @
> yarn-api ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory
> D:\Code\MR-279\mapreduce\yarn\yarn-api\src\main\resources
> [INFO]
> [INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ yarn-api
> ---
> [INFO] Compiling 121 source files to
> D:\Code\MR-279\mapreduce\yarn\yarn-api\target\classes
> [INFO] -------------------------------------------------------------
> [ERROR] COMPILATION ERROR :
> [INFO] -------------------------------------------------------------
> [ERROR]
>
> \Code\MR-279\mapreduce\yarn\yarn-api\src\main\java\org\apache\hadoop\yarn\ap
> i\records\impl\pb\ContainerPBImpl.java:[22,33] package
> org.apache.commons.logging does not exist
> [ERROR]
>
> \Code\MR-279\mapreduce\yarn\yarn-api\src\main\java\org\apache\hadoop\yarn\ap
> i\records\impl\pb\ContainerPBImpl.java:[23,33] package
> org.apache.commons.logging does not exist
> [INFO] 2 errors
> [INFO] -------------------------------------------------------------
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] yarn-api .......................................... FAILURE [5.610s]
> [INFO] yarn-common ....................................... SKIPPED
> [INFO] yarn-server-common ................................ SKIPPED
> [INFO] yarn-server-nodemanager ........................... SKIPPED
> [INFO] yarn-server-resourcemanager ....................... SKIPPED
> [INFO] yarn-server-tests ................................. SKIPPED
> [INFO] yarn-server ....................................... SKIPPED
> [INFO] yarn .............................................. SKIPPED
> [INFO] hadoop-mapreduce-client-core ...................... SKIPPED
> [INFO] hadoop-mapreduce-client-common .................... SKIPPED
> [INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
> [INFO] hadoop-mapreduce-client-app ....................... SKIPPED
> [INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
> [INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
> [INFO] hadoop-mapreduce-client ........................... SKIPPED
> [INFO] hadoop-mapreduce .................................. SKIPPED
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 5.844s
> [INFO] Finished at: Thu Jul 28 16:03:07 IST 2011
> [INFO] Final Memory: 9M/35M
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:2.3.2:compile
> (default-compile) on project yarn-api: Compilation failure: Compilation
> failure:
> [ERROR]
>
> \Code\MR-279\mapreduce\yarn\yarn-api\src\main\java\org\apache\hadoop\yarn\ap
> i\records\impl\pb\ContainerPBImpl.java:[22,33] package
> org.apache.commons.logging does not exist
> [ERROR]
>
> \Code\MR-279\mapreduce\yarn\yarn-api\src\main\java\org\apache\hadoop\yarn\ap
> i\records\impl\pb\ContainerPBImpl.java:[23,33] package
> org.apache.commons.logging does not exist
> [ERROR] -> [Help 1]
>
> Regards,
> Ravi Teja
>
>
>
>
>
>
> -----Original Message-----
> From: Josh Wills [mailto:jwills@cloudera.com]
> Sent: Saturday, July 02, 2011 6:34 AM
> To: mapreduce-dev@hadoop.apache.org
> Subject: Re: trouble building MR-279 branch
>
> Hey Anupam,
>
> I think Jim is asking about building non-MR application masters using the
> RM/NM. I think Arun has promised to post a design doc soon. In the
> meantime,
> I'm just reading through the code and fixing little issues I come across
> (e.g., https://issues.apache.org/jira/browse/MAPREDUCE-2630 and
> https://issues.apache.org/jira/browse/MAPREDUCE-2633).
>
> Jim, what sort of app are you looking to build? I'd be happy to have some
> company wading through the code.
>
> Best,
> Josh
>
> On Fri, Jul 1, 2011 at 3:02 PM, Anupam Seth <anupams@yahoo-inc.com> wrote:
>
> > Hi Jim,
> >
> > Not sure if this helps at all, but not sure if you could get an idea by
> > looking at the included examples:
> >
> >
> >
>
> http://svn.apache.org/viewvc/hadoop/common/branches/MR-279/mapreduce/src/exa
> mples/org/apache/hadoop/examples
> >
> > that can be run as:
> >
> > hadoop jar $HADOOP_MAPRED_HOME/hadoop-mapred-examples-0.22.0-SNAPSHOT.jar
> > wordcount input output
> >
> > Regards,
> > Anupam
> >
> > -----Original Message-----
> > From: Jim Falgout [mailto:jim.falgout@pervasive.com]
> > Sent: Friday, July 01, 2011 4:23 PM
> > To: mapreduce-dev@hadoop.apache.org
> > Subject: RE: trouble building MR-279 branch
> >
> > Thanks Anupam! I had skipped the step to get protobuf into the maven
> > repository. Also, I had Apache Forrest version 0.9 installed. That caused
> > the builds to fail on generating some docs. Installed version 0.8 and it
> > worked smoothly.
> >
> > On another note: do you know of any design docs for building an
> application
> > executor using the new resource manager? Just wondering how to get
> started?
> >
> > -----Original Message-----
> > From: Anupam Seth [mailto:anupams@yahoo-inc.com]
> > Sent: Friday, July 01, 2011 2:10 PM
> > To: mapreduce-dev@hadoop.apache.org
> > Subject: RE: trouble building MR-279 branch
> >
> > Hi Jim,
> >
> > Following are some notes (thanks to Jeff Naisbitt & Bobby Evans).
> >
> > Build YARN
> > ==========
> > These are in addition to what you would normally do.
> >
> > Install protobuf
> > use Protobuf 2.4.0a (Download from
> > http://code.google.com/p/protobuf/downloads/list) I actually used 2.4.1
> > and it worked
> >
> >    install the protoc executable (configure, make, make install)
> >    install the maven artifact (cd java; mvn install)
> >
> > Installing protoc requires gcc 4.1.x or higher (RHEL5).
> >
> > If the make step fails with (Valid until a fix is released for protobuf
> > 2.4.0a) ./google/protobuf/descriptor.h:1152: error:
> >
>
> `google::protobuf::internal::Mutex*google::protobuf::DescriptorPool::mutex_'
> > is private Replace descriptor.cc with
> >
>
> http://protobuf.googlecode.com/svn-history/r380/trunk/src/google/protobuf/de
> scriptor.cc
> >
> > Setup environment
> >
> > export MAVEN_OPTS=-Xmx512m
> > export LD_LIBRARY_PATH=/usr/local/lib
> >
> > Checkout the MR-279 branch
> > svn checkout
> http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279
> >
> > Build common
> >
> > cd MR-279/common
> > ant veryclean mvn-install tar
> >
> > Build hdfs
> >
> > cd ../hdfs
> > ant veryclean mvn-install -Dresolvers=internal tar
> >
> > Build yarn
> >
> > cd ../mapreduce
> > mvn clean install assembly:assembly
> > # If tests are failing, add this option to skip them:
> > -Dmaven.test.skip.exec=true # NOTE: in order to build on a Mac, you must
> > also include the <code>-P-cbuild</code> option # (e.g. <code>mvn
clean
> > install assembly:assembly -P-cbuild -Dmaven.test.skip.exec=true</code>)
> > ant veryclean jar jar-test tar -Dresolvers=internal
> >
> > -----Original Message-----
> > From: Jim Falgout [mailto:jim.falgout@pervasive.com]
> > Sent: Friday, July 01, 2011 2:00 PM
> > To: mapreduce-dev@hadoop.apache.org
> > Subject: trouble building MR-279 branch
> >
> > Hi,
> >
> > I've checked out the MR-279 branch and am having trouble getting it to
> > build. Any docs/blogs/notes around that may give some hints?
> >
> > The build is unable to resolve the yarn-server even though I ran a
> > "mvn-install" in the yarn source area to install it in the local
> repository.
> >
> > Thanks,
> > Jim.
> >
> >
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message