hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@hortonworks.com>
Subject Re: Problem with MRv2 Build
Date Wed, 10 Aug 2011 05:28:05 GMT
Looks like you don't have autotool chain required to build the native code for launching containers.

It isn't critical - you can run with the java based DefaultContainerExecutor.

Just pass -P-cbuild to maven to skip building the native code.

Arun

On Aug 9, 2011, at 10:24 PM, Haoyuan Li wrote:

> Hi Praveen,
> 
> Thank you for the info. It did help. Everything before "mvn clean install
> assembly:assembly -DskipTests" worked well, and I got the following errors.
> Do you have any clue?
> 
> Best,
> 
> Haoyuan
> 
> main:
>     [echo] project.build.directory:
> /home/haoyuan/hadoop/trunk/mapreduce/yarn/yarn-server/yarn-server-nodemanager/target
>     [copy] Copying 1 file to
> /home/haoyuan/hadoop/trunk/mapreduce/yarn/yarn-server/yarn-server-nodemanager/target
>     [copy] Copying
> /home/haoyuan/hadoop/trunk/mapreduce/yarn/yarn-server/yarn-server-nodemanager/pom.xml
> to
> /home/haoyuan/hadoop/trunk/mapreduce/yarn/yarn-server/yarn-server-nodemanager/target/saner-pom.xml
> [INFO] Executed tasks
> [INFO]
> [INFO] --- make-maven-plugin:1.0-beta-1:autoreconf (autoreconf) @
> yarn-server-nodemanager ---
> [DEBUG] org.codehaus.mojo:make-maven-plugin:jar:1.0-beta-1:
> [DEBUG]    org.codehaus.plexus:plexus-utils:jar:1.5.6:compile
> [DEBUG]    org.codehaus.mojo:cbuild-utils:jar:1.0-beta-1:compile
> [DEBUG]
> org.codehaus.plexus:plexus-container-default:jar:1.0-alpha-9-stable-1:compile
> [DEBUG]          junit:junit:jar:4.4:test (scope managed from compile)
> (version managed from 3.8.1)
> [DEBUG]          classworlds:classworlds:jar:1.1-alpha-2:compile
> [DEBUG]       org.apache.maven:maven-core:jar:2.1.0:compile
> [DEBUG]
> org.apache.maven:maven-plugin-parameter-documenter:jar:2.1.0:compile
> [DEBUG]
> org.apache.maven.reporting:maven-reporting-api:jar:2.1.0:compile
> [DEBUG]             org.apache.maven.doxia:doxia-sink-api:jar:1.1:compile
> [DEBUG]             org.apache.maven.doxia:doxia-logging-api:jar:1.1:compile
> [DEBUG]
> org.apache.maven:maven-repository-metadata:jar:2.1.0:compile
> [DEBUG]          org.apache.maven:maven-error-diagnostics:jar:2.1.0:compile
> [DEBUG]          commons-cli:commons-cli:jar:1.0:compile
> [DEBUG]          org.apache.maven:maven-plugin-descriptor:jar:2.1.0:compile
> [DEBUG]
> org.codehaus.plexus:plexus-interactivity-api:jar:1.0-alpha-4:compile
> [DEBUG]          org.apache.maven:maven-monitor:jar:2.1.0:compile
> [DEBUG]          org.sonatype.plexus:plexus-sec-dispatcher:jar:1.1:compile
> [DEBUG]             org.sonatype.plexus:plexus-cipher:jar:1.1:compile
> [DEBUG]       org.apache.maven:maven-model:jar:2.1.0:compile
> [DEBUG]       org.apache.maven:maven-artifact-manager:jar:2.1.0:compile
> [DEBUG]
> backport-util-concurrent:backport-util-concurrent:jar:3.1:compile
> [DEBUG]       org.codehaus.plexus:plexus-interpolation:jar:1.8.1:compile
> [DEBUG]    org.apache.maven.shared:file-management:jar:1.2.1:compile
> [DEBUG]       org.apache.maven.shared:maven-shared-io:jar:1.1:compile
> [DEBUG]
> org.apache.maven.wagon:wagon-provider-api:jar:1.0-beta-5:compile (version
> managed from 1.0-alpha-6)
> [DEBUG]    org.apache.maven:maven-plugin-api:jar:2.1.0:compile
> [DEBUG]    org.apache.maven:maven-artifact:jar:2.1.0:compile
> [DEBUG]    org.apache.maven:maven-project:jar:2.1.0:compile
> [DEBUG]       org.apache.maven:maven-settings:jar:2.1.0:compile
> [DEBUG]       org.apache.maven:maven-profile:jar:2.1.0:compile
> [DEBUG]       org.apache.maven:maven-plugin-registry:jar:2.1.0:compile
> [DEBUG] Created new class realm
> plugin>org.codehaus.mojo:make-maven-plugin:1.0-beta-1
> [DEBUG] Importing foreign packages into class realm
> plugin>org.codehaus.mojo:make-maven-plugin:1.0-beta-1
> [DEBUG]   Imported:  < maven.api
> [DEBUG] Populating class realm
> plugin>org.codehaus.mojo:make-maven-plugin:1.0-beta-1
> [DEBUG]   Included: org.codehaus.mojo:make-maven-plugin:jar:1.0-beta-1
> [DEBUG]   Included: org.codehaus.plexus:plexus-utils:jar:1.5.6
> [DEBUG]   Included: org.codehaus.mojo:cbuild-utils:jar:1.0-beta-1
> [DEBUG]   Included: org.apache.maven.reporting:maven-reporting-api:jar:2.1.0
> [DEBUG]   Included: org.apache.maven.doxia:doxia-sink-api:jar:1.1
> [DEBUG]   Included: org.apache.maven.doxia:doxia-logging-api:jar:1.1
> [DEBUG]   Included: commons-cli:commons-cli:jar:1.0
> [DEBUG]   Included:
> org.codehaus.plexus:plexus-interactivity-api:jar:1.0-alpha-4
> [DEBUG]   Included: org.sonatype.plexus:plexus-sec-dispatcher:jar:1.1
> [DEBUG]   Included: org.sonatype.plexus:plexus-cipher:jar:1.1
> [DEBUG]   Included:
> backport-util-concurrent:backport-util-concurrent:jar:3.1
> [DEBUG]   Included: org.codehaus.plexus:plexus-interpolation:jar:1.8.1
> [DEBUG]   Included: org.apache.maven.shared:file-management:jar:1.2.1
> [DEBUG]   Included: org.apache.maven.shared:maven-shared-io:jar:1.1
> [DEBUG]   Excluded:
> org.codehaus.plexus:plexus-container-default:jar:1.0-alpha-9-stable-1
> [DEBUG]   Excluded: junit:junit:jar:4.4
> [DEBUG]   Excluded: classworlds:classworlds:jar:1.1-alpha-2
> [DEBUG]   Excluded: org.apache.maven:maven-core:jar:2.1.0
> [DEBUG]   Excluded:
> org.apache.maven:maven-plugin-parameter-documenter:jar:2.1.0
> [DEBUG]   Excluded: org.apache.maven:maven-repository-metadata:jar:2.1.0
> [DEBUG]   Excluded: org.apache.maven:maven-error-diagnostics:jar:2.1.0
> [DEBUG]   Excluded: org.apache.maven:maven-plugin-descriptor:jar:2.1.0
> [DEBUG]   Excluded: org.apache.maven:maven-monitor:jar:2.1.0
> [DEBUG]   Excluded: org.apache.maven:maven-model:jar:2.1.0
> [DEBUG]   Excluded: org.apache.maven:maven-artifact-manager:jar:2.1.0
> [DEBUG]   Excluded: org.apache.maven.wagon:wagon-provider-api:jar:1.0-beta-5
> [DEBUG]   Excluded: org.apache.maven:maven-plugin-api:jar:2.1.0
> [DEBUG]   Excluded: org.apache.maven:maven-artifact:jar:2.1.0
> [DEBUG]   Excluded: org.apache.maven:maven-project:jar:2.1.0
> [DEBUG]   Excluded: org.apache.maven:maven-settings:jar:2.1.0
> [DEBUG]   Excluded: org.apache.maven:maven-profile:jar:2.1.0
> [DEBUG]   Excluded: org.apache.maven:maven-plugin-registry:jar:2.1.0
> [DEBUG] Configuring mojo
> org.codehaus.mojo:make-maven-plugin:1.0-beta-1:autoreconf from plugin realm
> ClassRealm[plugin>org.codehaus.mojo:make-maven-plugin:1.0-beta-1, parent:
> sun.misc.Launcher$AppClassLoader@1a45a877]
> [DEBUG] Configuring mojo
> 'org.codehaus.mojo:make-maven-plugin:1.0-beta-1:autoreconf' with basic
> configurator -->
> [DEBUG]   (f) arguments = [-i]
> [DEBUG]   (f) workDir =
> /home/haoyuan/hadoop/trunk/mapreduce/yarn/yarn-server/yarn-server-nodemanager/src/main/c/container-executor
> [DEBUG] -- end configuration --
> [DEBUG] Executing: /bin/sh -l -c cd
> /home/haoyuan/hadoop/trunk/mapreduce/yarn/yarn-server/yarn-server-nodemanager/src/main/c/container-executor
> && autoreconf -i
> [DEBUG] /bin/sh: autoreconf: not found
> [INFO]
> 
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Skipping hadoop-mapreduce
> [INFO] This project has been banned from the build due to previous failures.
> [INFO]
> ------------------------------------------------------------------------
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] yarn-api .......................................... SUCCESS [8.707s]
> [INFO] yarn-common ....................................... SUCCESS [8.763s]
> [INFO] yarn-server-common ................................ SUCCESS [3.396s]
> [INFO] yarn-server-nodemanager ........................... FAILURE [4.423s]
> [INFO] yarn-server-resourcemanager ....................... SKIPPED
> [INFO] yarn-server-tests ................................. SKIPPED
> [INFO] yarn-server ....................................... SKIPPED
> [INFO] yarn .............................................. SKIPPED
> [INFO] hadoop-mapreduce-client-core ...................... SKIPPED
> [INFO] hadoop-mapreduce-client-common .................... SKIPPED
> [INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
> [INFO] hadoop-mapreduce-client-app ....................... SKIPPED
> [INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
> [INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
> [INFO] hadoop-mapreduce-client ........................... SKIPPED
> [INFO] hadoop-mapreduce .................................. SKIPPED
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 25.797s
> [INFO] Finished at: Tue Aug 09 22:21:20 PDT 2011
> [INFO] Final Memory: 61M/308M
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal
> org.codehaus.mojo:make-maven-plugin:1.0-beta-1:autoreconf (autoreconf) on
> project yarn-server-nodemanager: autoreconf command returned an exit value
> != 0. Aborting build; see debug output for more information. -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.codehaus.mojo:make-maven-plugin:1.0-beta-1:autoreconf (autoreconf)
> on project yarn-server-nodemanager: autoreconf command returned an exit
> value != 0. Aborting build; see debug output for more information.
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:217)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
> at
> org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
> at
> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:319)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
> at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
> Caused by: org.apache.maven.plugin.MojoExecutionException: autoreconf
> command returned an exit value != 0. Aborting build; see debug output for
> more information.
> at
> org.codehaus.mojo.autotools.AutoreconfExecMojo.execute(AutoreconfExecMojo.java:135)
> at
> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
> ... 19 more
> [ERROR]
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR]   mvn <goals> -rf :yarn-server-nodemanager
> 
> 
> 
> On Tue, Aug 9, 2011 at 8:58 PM, Praveen Sripati <praveensripati@gmail.com>wrote:
> 
>> Haoyuan,
>> 
>> The instructions at the mentioned URL are not updated.
>> 
>> 1. Get the code from trunk.
>> 2. Remove the mapreduce folder.
>> 3. Get the mapreduce code from MR-279 branch.
>> 4. Run the build scripts.
>> 
>> echo "Common build ....."
>> mvn clean install package -Pbintar -DskipTests
>> 
>> echo "HDFS build ....."
>> cd hdfs
>> ant veryclean mvn-install -Dresolvers=internal
>> 
>> echo "MapReduce build ....."
>> cd ../mapreduce
>> mvn clean install assembly:assembly -DskipTests
>> ant veryclean jar jar-test  -Dresolvers=internal
>> 
>> The hadoop-mapreduce-1.0-SNAPSHOT-all.tar.gz file will be generated in the
>> mapreduce/target folder.
>> 
>> Rest of the instructions or more or less the same for running the sample
>> program.
>> 
>> Thanks,
>> Praveen
>> 
>> 
>> On Wed, Aug 10, 2011 at 9:06 AM, Haoyuan Li <haoyuan.li@gmail.com> wrote:
>> 
>>> Hi Tom,
>>> 
>>> I also have trouble in building the current MR-279. How to compile common
>>> by using mvn?
>>> 
>>> And do you mean that
>>> 
>> http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALLdoesn't
>>> work anymore?
>>> 
>>> Thank you.
>>> 
>>> Best,
>>> 
>>> Haoyuan
>>> 
>>> 
>>> On Thu, Aug 4, 2011 at 8:19 AM, Thomas Graves <tgraves@yahoo-inc.com
>>> wrote:
>>> 
>>>> You have to use common and hdfs from trunk branch now and only mapreduce
>>>> comes from MR-279 branch.
>>>> 
>>>> Something like this:
>>>> svn co http://svn.apache.org/repos/asf/hadoop/common/trunk/
>>>> rm -rf mapreduce
>>>> svn checkout
>>>> http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce
>>>> 
>>>> Note that common is now using mvn to build.
>>>> 
>>>> Tom
>>>> 
>>>> 
>>>> On 8/4/11 10:13 AM, "Praveen Sripati" <praveensripati@gmail.com> wrote:
>>>> 
>>>>> Hi,
>>>>> 
>>>>> I did get the latest code from
>>>>> 
>>>>> http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/
>>>>> 
>>>>> and ran the below command for mapreduce build
>>>>> 
>>>>> mvn clean install assembly:assembly -DskipTests
>>>>> 
>>>>> to get the following error
>>>>> 
>>>>> [INFO] yarn-api .......................................... FAILURE
>>>> [2.565s]
>>>>> [INFO] yarn-common ....................................... SKIPPED
>>>>> [INFO] yarn-server-common ................................ SKIPPED
>>>>> 
>>>>> [ERROR] Failed to execute goal on project yarn-api: Could not resolve
>>>>> dependencies for project org.apache.hadoop:yarn-api:jar:1.0-SNAPSHOT:
>>>> The
>>>>> following artifacts could not be resolved:
>>>>> org.apache.hadoop:hadoop-common:jar:0.23.0-SNAPSHOT,
>>>>> org.apache.hadoop:hadoop-annotations:jar:0.23.0-SNAPSHOT,
>>>>> org.apache.hadoop:hadoop-common:jar:tests:0.23.0-SNAPSHOT,
>>>>> org.apache.hadoop:hadoop-hdfs:jar:0.23.0-SNAPSHOT: Could not find
>>>> artifact
>>>>> org.apache.hadoop:hadoop-common:jar:0.23.0-SNAPSHOT -> [Help 1]
>>>>> 
>>>>> Can someone please help me out with the build problem? The last time
I
>>>> got
>>>>> the code from SVN the build was successful, looks like some changes in
>>>> the
>>>>> code introduced this problem.
>>>>> 
>>>>> Thanks,
>>>>> Praveen
>>>> 
>>>> 
>>> 
>> 


Mime
View raw message