hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11345) On Suse/PPC64, building Hadoop Pipes (with -Pnative) requires to add -lcrypto at link stage
Date Mon, 08 Dec 2014 19:34:13 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238323#comment-14238323
] 

Colin Patrick McCabe commented on HADOOP-11345:
-----------------------------------------------

The idea here is that {{find_package(OpenSSL REQUIRED)}} should be setting {{OPENSSL_LIBRARIES}}
to the correct value for your platform, and using {{target_link_libraries}} should be adding
the correct {{\-l}} flag to the linker line.  As-is, it looks like it's adding {{\-lssl}},
but that is not working for you.

I am using openSUSE 12.3 with no problems, what version are you using?  I am on x86, though.

It's hard to think of fixes here that doesn't involve rewriting that {{find_package}} script,
which I'd really like to avoid.  There may be some options we can pass to the {{find_package}}
script?

It's also a little concerning that we are linking openSSL statically into {{libhadooppipes.a}}.
 It would be nice to get rid of the static library build altogether... can someone who is
still using libhadooppipes speak up?

> On Suse/PPC64, building Hadoop Pipes (with -Pnative) requires to add -lcrypto at link
stage
> -------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-11345
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11345
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: tools
>    Affects Versions: 2.4.1
>         Environment: Suse on PPC64
> # uname -a
> Linux hybridf 3.0.101-0.40-ppc64 #1 SMP Thu Sep 18 13:09:38 UTC 2014 (44b8c95) ppc64
ppc64 ppc64 GNU/Linux
>            Reporter: Tony Reix
>            Priority: Minor
>
> Compiling Hadoop Pipes fails on Suse/PPC64.
> [INFO] Apache Hadoop Extras ............................... SUCCESS [  5.855 s]
> [INFO] Apache Hadoop Pipes ................................ FAILURE [  8.134 s]
> Traces of building Hadoop Pipes:
> main:
>     [mkdir] Created dir: /home/hadoop/hadoop-common/hadoop-tools/hadoop-pipes/target/native
>      [exec] Current OS is Linux
>      [exec] Executing 'cmake' with arguments:
>      [exec] '/home/hadoop/hadoop-common/hadoop-tools/hadoop-pipes/src/'
>      [exec] '-DJVM_ARCH_DATA_MODEL=64'
>      [exec]
>      [exec] The ' characters around the executable and arguments are
>      [exec] not part of the command.
> Execute:Java13CommandLauncher: Executing 'cmake' with arguments:
> '/home/hadoop/hadoop-common/hadoop-tools/hadoop-pipes/src/'
> '-DJVM_ARCH_DATA_MODEL=64'
> The ' characters around the executable and arguments are
> not part of the command.
>      [exec] -- The C compiler identification is GNU
>      [exec] -- The CXX compiler identification is GNU
>      [exec] -- Check for working C compiler: /opt/at7.0/bin/gcc
>      [exec] -- Check for working C compiler: /opt/at7.0/bin/gcc -- works
>      [exec] -- Detecting C compiler ABI info
>      [exec] -- Detecting C compiler ABI info - done
>      [exec] -- Check for working CXX compiler: /opt/at7.0/bin/c++
>      [exec] -- Check for working CXX compiler: /opt/at7.0/bin/c++ -- works
>      [exec] -- Detecting CXX compiler ABI info
>      [exec] JAVA_HOME=, JAVA_JVM_LIBRARY=/opt/ibm/java-ppc64-71/jre/lib/ppc64/compressedrefs/libjvm.so
>      [exec] JAVA_INCLUDE_PATH=/opt/ibm/java-ppc64-71/include, JAVA_INCLUDE_PATH2=/opt/ibm/java-ppc64-71/include/linux
>      [exec] Located all JNI components successfully.
>      [exec] -- Detecting CXX compiler ABI info - done
>      [exec] -- Found OpenSSL: /usr/lib64/libssl.so
>      [exec] -- Configuring done
>      [exec] -- Generating done
>      [exec] -- Build files have been written to: /home/hadoop/hadoop-common/hadoop-tools/hadoop-pipes/target/native
> ............
>     [exec] /usr/bin/cmake -E cmake_progress_report /home/hadoop/hadoop-common/hadoop-tools/hadoop-pipes/target/native/CMakeFiles
4
>      [exec] [ 57%] Building CXX object CMakeFiles/pipes-sort.dir/main/native/examples/impl/sort.cc.o
>      [exec] /opt/at7.0/bin/c++    -g -Wall -O2 -D_REENTRANT -D_GNU_SOURCE -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64 -I/home/hadoop/hadoop-common/hadoop-tools/hadoop-pipes/src/main/native/utils/api
-I/home/hadoop/hadoop-common/hadoop-tools/hadoop-pipes/src/main/native/pipes/api -I/home/hadoop/hadoop-common/hadoop-tools/hadoop-pipes/src
  -o CMakeFiles/pipes-sort.dir/main/native/examples/impl/sort.cc.o -c /home/hadoop/hadoop-common/hadoop-tools/hadoop-pipes/src/main/native/examples/impl/sort.cc
>      [exec] Linking CXX executable examples/pipes-sort
>      [exec] /usr/bin/cmake -E cmake_link_script CMakeFiles/pipes-sort.dir/link.txt --verbose=1
>      [exec] /opt/at7.0/bin/c++    -g -Wall -O2 -D_REENTRANT -D_GNU_SOURCE -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64  -fPIC CMakeFiles/pipes-sort.dir/main/native/examples/impl/sort.cc.o
 -o examples/pipes-sort -rdynamic libhadooppipes.a libhadooputils.a -lssl -lpthread
>      [exec] /opt/at7.0/lib/gcc/powerpc64-linux/4.8.4/../../../../powerpc64-linux/bin/ld:
libhadooppipes.a(HadoopPipes.cc.o): undefined reference to symbol 'HMAC_Init'
>      [exec] /opt/at7.0/lib64/libcrypto.so.1.0.0: error adding symbols: DSO missing from
command line
>      [exec] collect2: error: ld returned 1 exit status
> However, adding -lcrypto to the failing command did fix the issue:
> # cd hadoop-common/hadoop-tools/hadoop-pipes/target/native
> # /opt/at7.0/bin/c++    -g -Wall -O2 -D_REENTRANT -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
 -fPIC CMakeFiles/pipes-sort.dir/main/native/examples/impl/sort.cc.o  -o examples/pipes-sort
-rdynamic libhadooppipes.a libhadooputils.a -lssl -lpthread -lcrypto
> #
> I do not have any Suse / x86_64 machine available. So, I do not know if the issue is
general for Suse or specific for Suse/PPC64 .
> Anyway, I don't know how Hadoop should be changed for fixing this issue.
> Maybe, in hadoop-tools/hadoop-pipes/src/CMakeLists.txt , something should be added to:
> target_link_libraries(hadooppipes
>     ${OPENSSL_LIBRARIES}
>     pthread



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message