hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pankaj Misra <pankaj.mi...@impetus.co.in>
Subject Apache Hadoop 0.23.1 Source Build Failing
Date Thu, 09 Aug 2012 10:05:39 GMT
Dear All,

I am building hadoop 0.23.1 release from source with native support. I have already built/installed
the following pre-requisites for native support
1. gcc-c++ 4.7.1
2. protoc 2.4.1
3. autotools chain
4. JDK 1.6.0_33
5. zlib 1.2.5-6
6. lzo 2.06-2

I have also set the following variables and exported them
export LD_LIBRARY_PATH=/usr/local/lib

Other variables are also set as given below.
export LD_LIBRARY_PATH=/usr/local/lib
export JAVA_HOME=/usr/java/jdk1.6.0_33
export ANT_HOME=/home/fedora/apache-ant-1.8.4
export MAVEN_HOME=/home/fedora/apache-maven-3.0.4
export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$MAVEN_HOME/bin:$PATH
export HADOOP_COMMON_HOME=/home/fedora/hadoop/bin/release-0.23.1/hadoop-0.23.1
export HADOOP_HDFS_HOME=$HADOOP_COMMON_HOME
export YARN_HOME=$HADOOP_COMMON_HOME
export HADOOP_MAPRED_HOME=$HADOOP_COMMON_HOME
export HADOOP_CONF_DIR=$HADOOP_COMMON_HOME/etc/hadoop


I am using the following to build the source with native support(using root user access).
mvn clean install -Pdist -Pnative -DskipTests=true

However, after building a number of sub-projects, the build fails at the nodemanager with
the following error
[INFO] Compiling 129 source files to /home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/classes
[INFO]
[INFO] --- make-maven-plugin:1.0-beta-1:autoreconf (compile) @ hadoop-yarn-server-nodemanager
---
[INFO]
[INFO] --- make-maven-plugin:1.0-beta-1:configure (compile) @ hadoop-yarn-server-nodemanager
---
[INFO] checking for gcc... gcc
[INFO] checking whether the C compiler works... no
[INFO] configure: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
[INFO] configure: error: C compiler cannot create executables
[INFO] See `config.log' for more details


The config.log mentions the following issues.

configure:2562: checking whether the C compiler works
configure:2584: gcc -DHADOOP_CONF_DIR=/etc/hadoop -m32   conftest.c  >&5
/bin/ld: cannot find crt1.o: No such file or directory
/bin/ld: cannot find crti.o: No such file or directory
/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching
for -lgcc_s
/bin/ld: cannot find -lgcc_s
/bin/ld: skipping incompatible /usr/lib64/libc.so when searching for -lc
/bin/ld: cannot find -lc
/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching
for -lgcc_s
/bin/ld: cannot find -lgcc_s
/bin/ld: cannot find crtn.o: No such file or directory
collect2: error: ld returned 1 exit status
configure:2588: $? = 1
configure:2626: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "linux-container-executor"
| #define PACKAGE_TARNAME "linux-container-executor"
| #define PACKAGE_VERSION "1.0.0"
| #define PACKAGE_STRING "linux-container-executor 1.0.0"
| #define PACKAGE_BUGREPORT "mapreduce-dev@hadoop.apache.org"
| #define PACKAGE_URL ""
| /* end confdefs.h.  */
|
| int
| main ()
| {
|
|   ;
|   return 0;
| }
configure:2631: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
configure:2633: error: C compiler cannot create executables

Overall build summary is as given below
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ................................ SUCCESS [7.920s]
[INFO] Apache Hadoop Project POM ......................... SUCCESS [1.405s]
[INFO] Apache Hadoop Annotations ......................... SUCCESS [6.452s]
[INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.637s]
[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [3.236s]
[INFO] Apache Hadoop Auth ................................ SUCCESS [8.805s]
[INFO] Apache Hadoop Auth Examples ....................... SUCCESS [5.105s]
[INFO] Apache Hadoop Common .............................. SUCCESS [1:55.833s]
[INFO] Apache Hadoop Common Project ...................... SUCCESS [0.210s]
[INFO] Apache Hadoop HDFS ................................ SUCCESS [1:24.218s]
[INFO] Apache Hadoop HttpFS .............................. SUCCESS [20.992s]
[INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.127s]
[INFO] hadoop-yarn ....................................... SUCCESS [0.598s]
[INFO] hadoop-yarn-api ................................... SUCCESS [57.281s]
[INFO] hadoop-yarn-common ................................ SUCCESS [1:01.264s]
[INFO] hadoop-yarn-server ................................ SUCCESS [0.190s]
[INFO] hadoop-yarn-server-common ......................... SUCCESS [19.280s]
[INFO] hadoop-yarn-server-nodemanager .................... FAILURE [17.016s]
[INFO] hadoop-yarn-server-web-proxy ...................... SKIPPED
[INFO] hadoop-yarn-server-resourcemanager ................ SKIPPED
[INFO] hadoop-yarn-server-tests .......................... SKIPPED
[INFO] hadoop-mapreduce-client ........................... SKIPPED
[INFO] hadoop-mapreduce-client-core ...................... SKIPPED
[INFO] hadoop-yarn-applications .......................... SKIPPED
[INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED
[INFO] hadoop-yarn-site .................................. SKIPPED
[INFO] hadoop-mapreduce-client-common .................... SKIPPED
[INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
[INFO] hadoop-mapreduce-client-app ....................... SKIPPED
[INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
[INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
[INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
[INFO] hadoop-mapreduce .................................. SKIPPED
[INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED
[INFO] Apache Hadoop Distributed Copy .................... SKIPPED
[INFO] Apache Hadoop Archives ............................ SKIPPED
[INFO] Apache Hadoop Rumen ............................... SKIPPED
[INFO] Apache Hadoop Extras .............................. SKIPPED
[INFO] Apache Hadoop Tools Dist .......................... SKIPPED
[INFO] Apache Hadoop Tools ............................... SKIPPED
[INFO] Apache Hadoop Distribution ........................ SKIPPED
[INFO] Apache Hadoop Client .............................. SKIPPED
[INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED
[INFO] ----------------------------------------------------------------------

My doubts here are that is the if C compiler was not able to create the executables, the other
projects which were build before node manager should have also failed, since they are also
using the auto tools chain. I am able to build the entire source code if I build without native
support, but I do not want to skip the native support build and I am trying to build the entire
source code with native libraries support.

My current OS is Fedora 17 64-bit and I have also got the same issue on CentOS 6.2 64-bit.
I dont know if I am missing something for successful build with native libraries, and would
request for the community's help for the same.



Thanks and Regards
Pankaj Misra

________________________________

Impetus webinar: Designing a Test Automation Framework for Interoperable Systems; July 25
(10:00am PT). http://lf1.me/0E/

Follow us on www.twitter.com/impetustech


NOTE: This message may contain information that is confidential, proprietary, privileged or
otherwise protected by law. The message is intended solely for the named addressee. If received
in error, please destroy and notify the sender. Any use of this email is prohibited when received
in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this
communication has been maintained nor that the communication is free of errors, virus, interception
or interference.

Mime
View raw message