hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From HU Wenjing A <Wenjing.a...@alcatel-sbell.com.cn>
Subject hadoop source code build failed
Date Fri, 08 Jun 2012 00:53:24 GMT

Hi,

  I am sorry if this question was asked a lot of times before but I am unable to solve my
problem. I check out the hadoop 0.22.0 source code and want to build it using ant&ivy,but
I encountered some problems.As far as my understanding goes, I need to build common, hdfs,
and mapred. It seems that common could be build successfully, but hdfs and mapred failed the
build. Here is the error I receive for those two part:

hdfs:

Buildfile: /root/hadoop-0.22.0/hadoop_compile/hdfs/build.xml

clover.setup:

clover.info:
     [echo]
     [echo]      Clover not found. Code coverage reports disabled.
     [echo]

clover:

ivy-download:
      [get] Getting: http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar
      [get] To: /root/hadoop-0.22.0/hadoop_compile/hdfs/ivy/ivy-2.1.0.jar
      [get] Not modified - so not downloaded

ivy-init-dirs:

ivy-probe-antlib:

ivy-init-antlib:

ivy-init:
[ivy:configure] :: Apache Ivy 2.3.0-rc1 - 20120416000235 :: http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file = /root/hadoop-0.22.0/hadoop_compile/hdfs/ivy/ivysettings.xml

ivy-resolve-common:
[ivy:resolve]
[ivy:resolve] :: problems summary ::
[ivy:resolve] :::: WARNINGS
[ivy:resolve]           module not found: org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT
[ivy:resolve]   ==== apache-snapshot: tried
[ivy:resolve]     https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.pom
[ivy:resolve]     -- artifact org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT!hadoop-common.jar:
[ivy:resolve]     https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.jar
[ivy:resolve]   ==== maven2: tried
[ivy:resolve]     http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.pom
[ivy:resolve]     -- artifact org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT!hadoop-common.jar:
[ivy:resolve]     http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.jar
[ivy:resolve]           ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve]           ::          UNRESOLVED DEPENDENCIES         ::
[ivy:resolve]           ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve]           :: org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT: not found
[ivy:resolve]           ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve]
[ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS

BUILD FAILED
/root/hadoop-0.22.0/hadoop_compile/hdfs/build.xml:1814: impossible to resolve dependencies:
        resolve failed - see output for details

Total time: 52 seconds


mapreduce:

Buildfile: /root/hadoop-0.22.0/hadoop_compile/mapreduce/build.xml

clover.setup:

clover.info:
     [echo]
     [echo]      Clover not found. Code coverage reports disabled.
     [echo]

clover:

ivy-download:
      [get] Getting: http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar
      [get] To: /root/hadoop-0.22.0/hadoop_compile/mapreduce/ivy/ivy-2.1.0.jar
      [get] Not modified - so not downloaded

ivy-init-dirs:

ivy-probe-antlib:

ivy-init-antlib:

ivy-init:
[ivy:configure] :: Apache Ivy 2.3.0-rc1 - 20120416000235 :: http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file = /root/hadoop-0.22.0/hadoop_compile/mapreduce/ivy/ivysettings.xml

ivy-resolve-common:
[ivy:resolve]
[ivy:resolve] :: problems summary ::
[ivy:resolve] :::: WARNINGS
[ivy:resolve]           module not found: org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT
[ivy:resolve]   ==== apache-snapshot: tried
[ivy:resolve]     https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.pom
[ivy:resolve]     -- artifact org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT!hadoop-common.jar:
[ivy:resolve]     https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.jar
[ivy:resolve]   ==== maven2: tried
[ivy:resolve]     http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.pom
[ivy:resolve]     -- artifact org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT!hadoop-common.jar:
[ivy:resolve]     http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.jar
[ivy:resolve]           module not found: org.apache.hadoop#hadoop-common-test;0.22.0-SNAPSHOT
[ivy:resolve]   ==== apache-snapshot: tried
[ivy:resolve]     https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common-test/0.22.0-SNAPSHOT/hadoop-common-test-0.22.0-SNAPSHOT.pom
[ivy:resolve]     -- artifact org.apache.hadoop#hadoop-common-test;0.22.0-SNAPSHOT!hadoop-common-test.jar:
[ivy:resolve]     https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common-test/0.22.0-SNAPSHOT/hadoop-common-test-0.22.0-SNAPSHOT.jar
[ivy:resolve]   ==== maven2: tried
[ivy:resolve]     http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common-test/0.22.0-SNAPSHOT/hadoop-common-test-0.22.0-SNAPSHOT.pom
[ivy:resolve]     -- artifact org.apache.hadoop#hadoop-common-test;0.22.0-SNAPSHOT!hadoop-common-test.jar:
[ivy:resolve]     http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common-test/0.22.0-SNAPSHOT/hadoop-common-test-0.22.0-SNAPSHOT.jar
[ivy:resolve]           module not found: org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT
[ivy:resolve]   ==== apache-snapshot: tried
[ivy:resolve]     https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom
[ivy:resolve]     -- artifact org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
[ivy:resolve]     https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar
[ivy:resolve]   ==== maven2: tried
[ivy:resolve]     http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom
[ivy:resolve]     -- artifact org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
[ivy:resolve]     http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar
[ivy:resolve]           ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve]           ::          UNRESOLVED DEPENDENCIES         ::
[ivy:resolve]           ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve]           :: org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT: not found
[ivy:resolve]           :: org.apache.hadoop#hadoop-common-test;0.22.0-SNAPSHOT: not found
[ivy:resolve]           :: org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT: not found
[ivy:resolve]           ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve]
[ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS

BUILD FAILED
/root/hadoop-0.22.0/hadoop_compile/mapreduce/build.xml:2117: impossible to resolve dependencies:
        resolve failed - see output for details

Total time: 39 seconds

I searched the internet and see someone says this is because of the ivy configuration and
give solution like below:

for hdfs part ,modify the ivy.xml as following:(---- means delete,+++ means add)

hdfs/ivy.xml

<dependency org="org.apache.hadoop" name="hadoop-common"

-----    rev="${hadoop-common.version}"

+++ rev="${hadoop-common.version}-SNAPSHOT"

    conf="common->default"/>

hdfs/src/contrib/hdfsproxy/ivy.xml

       <dependency org="org.apache.hadoop"

         name="hadoop-common"

----       rev="${hadoop-common.version}"

+++      rev="${hadoop-common.version}-SNAPSHOT"

         conf="common->default"/>

       <dependency org="org.apache.hadoop"

         name="hadoop-common-test"

---        rev="${hadoop-common.version}"

+++     rev="${hadoop-common.version}-SNAPSHOT"

         conf="common->default"/>

hdfs/src/contrib/thriftfs/ivy.xml

       <dependency org="org.apache.hadoop"

             name="hadoop-common"

---        rev="${hadoop-common.version}"

+++     rev="${hadoop-common.version}-SNAPSHOT"

         conf="common->default"/>


for mapreduce part:
first modify all the ivy.xml in project mapreduce:

      rev="${hadoop-common.version}"  replaced by  rev="${hadoop-common.version}-SNAPSHOT"

      rev="${hadoop-hdfs.version}"  replaced by  rev="${hadoop-hdfs.version}-SNAPSHOT"
then modify the file buil.xml

        <unzip

        ----       src="${common.ivy.lib.dir}/hadoop-hdfs-${hadoop-hdfs.version}.jar"

        +++        src="${common.ivy.lib.dir}/hadoop-hdfs-${hadoop-hdfs.version}-SNAPSHOT.jar"
dest="${build.dir}">


Bur after the modification ,the build still failed .So could you give me some suggestions
about the source code build especially for the 0.22.0 version

Thanks & Best regards
wenjing



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message