hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 此案彼岸 <sweetsmile...@126.com>
Subject Re: hadoop source code build failed
Date Sun, 10 Jun 2012 15:36:12 GMT
于 2012/6/8 10:15, Nishchay P 写道:
> Hi Wenjing,
>
>    It looks like the website
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> oop/hadoop-common/ doesn't have 0.22.0 may be if you can checkout
> 0.22.1-SNAPSHOT the build might work.
>
> Thanks,
> Nishchay
>
>
>
>
> On 08/06/12 6:23 AM, "HU Wenjing A" <Wenjing.a.Hu@alcatel-sbell.com.cn>
> wrote:
>
>> Hi,
>>
>>  I am sorry if this question was asked a lot of times before but I am
>> unable to solve my problem. I check out the hadoop 0.22.0 source code and
>> want to build it using ant&ivy,but I encountered some problems.As far as
>> my understanding goes, I need to build common, hdfs, and mapred. It seems
>> that common could be build successfully, but hdfs and mapred failed the
>> build. Here is the error I receive for those two part:
>>
>> hdfs:
>>
>> Buildfile: /root/hadoop-0.22.0/hadoop_compile/hdfs/build.xml
>>
>> clover.setup:
>>
>> clover.info:
>>     [echo]
>>     [echo]      Clover not found. Code coverage reports disabled.
>>     [echo]
>>
>> clover:
>>
>> ivy-download:
>>      [get] Getting:
>> http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar
>>      [get] To: /root/hadoop-0.22.0/hadoop_compile/hdfs/ivy/ivy-2.1.0.jar
>>      [get] Not modified - so not downloaded
>>
>> ivy-init-dirs:
>>
>> ivy-probe-antlib:
>>
>> ivy-init-antlib:
>>
>> ivy-init:
>> [ivy:configure] :: Apache Ivy 2.3.0-rc1 - 20120416000235 ::
>> http://ant.apache.org/ivy/ ::
>> [ivy:configure] :: loading settings :: file =
>> /root/hadoop-0.22.0/hadoop_compile/hdfs/ivy/ivysettings.xml
>>
>> ivy-resolve-common:
>> [ivy:resolve]
>> [ivy:resolve] :: problems summary ::
>> [ivy:resolve] :::: WARNINGS
>> [ivy:resolve]           module not found:
>> org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT
>> [ivy:resolve]   ==== apache-snapshot: tried
>> [ivy:resolve]
>> https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>> doop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.pom
>> [ivy:resolve]     -- artifact
>> org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT!hadoop-common.jar:
>> [ivy:resolve]
>> https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>> doop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.jar
>> [ivy:resolve]   ==== maven2: tried
>> [ivy:resolve]
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.22.0-SNAPS
>> HOT/hadoop-common-0.22.0-SNAPSHOT.pom
>> [ivy:resolve]     -- artifact
>> org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT!hadoop-common.jar:
>> [ivy:resolve]
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.22.0-SNAPS
>> HOT/hadoop-common-0.22.0-SNAPSHOT.jar
>> [ivy:resolve]           ::::::::::::::::::::::::::::::::::::::::::::::
>> [ivy:resolve]           ::          UNRESOLVED DEPENDENCIES         ::
>> [ivy:resolve]           ::::::::::::::::::::::::::::::::::::::::::::::
>> [ivy:resolve]           ::
>> org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT: not found
>> [ivy:resolve]           ::::::::::::::::::::::::::::::::::::::::::::::
>> [ivy:resolve]
>> [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
>>
>> BUILD FAILED
>> /root/hadoop-0.22.0/hadoop_compile/hdfs/build.xml:1814: impossible to
>> resolve dependencies:
>>        resolve failed - see output for details
>>
>> Total time: 52 seconds
>>
>>
>> mapreduce:
>>
>> Buildfile: /root/hadoop-0.22.0/hadoop_compile/mapreduce/build.xml
>>
>> clover.setup:
>>
>> clover.info:
>>     [echo]
>>     [echo]      Clover not found. Code coverage reports disabled.
>>     [echo]
>>
>> clover:
>>
>> ivy-download:
>>      [get] Getting:
>> http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar
>>      [get] To:
>> /root/hadoop-0.22.0/hadoop_compile/mapreduce/ivy/ivy-2.1.0.jar
>>      [get] Not modified - so not downloaded
>>
>> ivy-init-dirs:
>>
>> ivy-probe-antlib:
>>
>> ivy-init-antlib:
>>
>> ivy-init:
>> [ivy:configure] :: Apache Ivy 2.3.0-rc1 - 20120416000235 ::
>> http://ant.apache.org/ivy/ ::
>> [ivy:configure] :: loading settings :: file =
>> /root/hadoop-0.22.0/hadoop_compile/mapreduce/ivy/ivysettings.xml
>>
>> ivy-resolve-common:
>> [ivy:resolve]
>> [ivy:resolve] :: problems summary ::
>> [ivy:resolve] :::: WARNINGS
>> [ivy:resolve]           module not found:
>> org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT
>> [ivy:resolve]   ==== apache-snapshot: tried
>> [ivy:resolve]
>> https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>> doop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.pom
>> [ivy:resolve]     -- artifact
>> org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT!hadoop-common.jar:
>> [ivy:resolve]
>> https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>> doop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.jar
>> [ivy:resolve]   ==== maven2: tried
>> [ivy:resolve]
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.22.0-SNAPS
>> HOT/hadoop-common-0.22.0-SNAPSHOT.pom
>> [ivy:resolve]     -- artifact
>> org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT!hadoop-common.jar:
>> [ivy:resolve]
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.22.0-SNAPS
>> HOT/hadoop-common-0.22.0-SNAPSHOT.jar
>> [ivy:resolve]           module not found:
>> org.apache.hadoop#hadoop-common-test;0.22.0-SNAPSHOT
>> [ivy:resolve]   ==== apache-snapshot: tried
>> [ivy:resolve]
>> https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>> doop/hadoop-common-test/0.22.0-SNAPSHOT/hadoop-common-test-0.22.0-SNAPSHOT
>> .pom
>> [ivy:resolve]     -- artifact
>> org.apache.hadoop#hadoop-common-test;0.22.0-SNAPSHOT!hadoop-common-test.ja
>> r:
>> [ivy:resolve]
>> https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>> doop/hadoop-common-test/0.22.0-SNAPSHOT/hadoop-common-test-0.22.0-SNAPSHOT
>> .jar
>> [ivy:resolve]   ==== maven2: tried
>> [ivy:resolve]
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common-test/0.22.0-
>> SNAPSHOT/hadoop-common-test-0.22.0-SNAPSHOT.pom
>> [ivy:resolve]     -- artifact
>> org.apache.hadoop#hadoop-common-test;0.22.0-SNAPSHOT!hadoop-common-test.ja
>> r:
>> [ivy:resolve]
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common-test/0.22.0-
>> SNAPSHOT/hadoop-common-test-0.22.0-SNAPSHOT.jar
>> [ivy:resolve]           module not found:
>> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT
>> [ivy:resolve]   ==== apache-snapshot: tried
>> [ivy:resolve]
>> https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>> doop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom
>> [ivy:resolve]     -- artifact
>> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
>> [ivy:resolve]
>> https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>> doop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar
>> [ivy:resolve]   ==== maven2: tried
>> [ivy:resolve]
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHO
>> T/hadoop-hdfs-0.22.0-SNAPSHOT.pom
>> [ivy:resolve]     -- artifact
>> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
>> [ivy:resolve]
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHO
>> T/hadoop-hdfs-0.22.0-SNAPSHOT.jar
>> [ivy:resolve]           ::::::::::::::::::::::::::::::::::::::::::::::
>> [ivy:resolve]           ::          UNRESOLVED DEPENDENCIES         ::
>> [ivy:resolve]           ::::::::::::::::::::::::::::::::::::::::::::::
>> [ivy:resolve]           ::
>> org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT: not found
>> [ivy:resolve]           ::
>> org.apache.hadoop#hadoop-common-test;0.22.0-SNAPSHOT: not found
>> [ivy:resolve]           :: org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT:
>> not found
>> [ivy:resolve]           ::::::::::::::::::::::::::::::::::::::::::::::
>> [ivy:resolve]
>> [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
>>
>> BUILD FAILED
>> /root/hadoop-0.22.0/hadoop_compile/mapreduce/build.xml:2117: impossible
>> to resolve dependencies:
>>        resolve failed - see output for details
>>
>> Total time: 39 seconds
>>
>> I searched the internet and see someone says this is because of the ivy
>> configuration and give solution like below:
>>
>> for hdfs part ,modify the ivy.xml as following:(---- means delete,+++
>> means add)
>>
>> hdfs/ivy.xml
>>
>> <dependency org="org.apache.hadoop" name="hadoop-common"
>>
>> -----    rev="${hadoop-common.version}"
>>
>> +++ rev="${hadoop-common.version}-SNAPSHOT"
>>
>>    conf="common->default"/>
>>
>> hdfs/src/contrib/hdfsproxy/ivy.xml
>>
>>       <dependency org="org.apache.hadoop"
>>
>>         name="hadoop-common"
>>
>> ----       rev="${hadoop-common.version}"
>>
>> +++      rev="${hadoop-common.version}-SNAPSHOT"
>>
>>         conf="common->default"/>
>>
>>       <dependency org="org.apache.hadoop"
>>
>>         name="hadoop-common-test"
>>
>> ---        rev="${hadoop-common.version}"
>>
>> +++     rev="${hadoop-common.version}-SNAPSHOT"
>>
>>         conf="common->default"/>
>>
>> hdfs/src/contrib/thriftfs/ivy.xml
>>
>>       <dependency org="org.apache.hadoop"
>>
>>             name="hadoop-common"
>>
>> ---        rev="${hadoop-common.version}"
>>
>> +++     rev="${hadoop-common.version}-SNAPSHOT"
>>
>>         conf="common->default"/>
>>
>>
>> for mapreduce part:
>> first modify all the ivy.xml in project mapreduce:
>>
>>      rev="${hadoop-common.version}"  replaced by
>> rev="${hadoop-common.version}-SNAPSHOT"
>>
>>      rev="${hadoop-hdfs.version}"  replaced by
>> rev="${hadoop-hdfs.version}-SNAPSHOT"
>> then modify the file buil.xml
>>
>>        <unzip
>>
>>        ----
>> src="${common.ivy.lib.dir}/hadoop-hdfs-${hadoop-hdfs.version}.jar"
>>
>>        +++
>> src="${common.ivy.lib.dir}/hadoop-hdfs-${hadoop-hdfs.version}-SNAPSHOT.jar
>> " dest="${build.dir}">
>>
>>
>> Bur after the modification ,the build still failed .So could you give me
>> some suggestions about the source code build especially for the 0.22.0
>> version
>>
>> Thanks & Best regards
>> wenjing
>>
>>
>>
>
> ________________________________
>
> The information contained in this communication is intended solely for the use of the
individual or entity to whom it is addressed and others authorized to receive it. It may contain
confidential or legally privileged information. If you are not the intended recipient you
are hereby notified that any disclosure, copying, distribution or taking any action in reliance
on the contents of this information is strictly prohibited and may be unlawful. If you have
received this communication in error, please notify us immediately by responding to this email
and then delete it from your system. The firm is neither liable for the proper and complete
transmission of the information contained in this communication nor for any delay in its receipt.
>
Hi Nishchay,

I am Wenjing just using my personal mailbox. First thank you so much for
response. I just saw this mail and according to your response I checked
the url and found it's true there are some files missing for the version
0.22.0. Actually I have changed back to version 0.21.0 to do the souce
code build and all of the three main parts of the code can be build
successfully. :)
Thanks again. O(∩_∩)O~

Thanks & best regards,
Wenjing


Mime
View raw message