impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Henry Robinson <he...@cloudera.com>
Subject Re: Error linking impalad with -build_shared_libs option
Date Wed, 24 Aug 2016 17:59:40 GMT
Seems great to me - but do we know why this fixes the issue? Not that that
should prevent us from merging the patch.

On 24 August 2016 at 08:54, Tim Armstrong <tarmstrong@cloudera.com> wrote:

> The fix works for me too, thanks Martin!
>
> I gave the patch a +1 but will upgrade to +2 if it seems everyone is happy
> with it.
>
> Cheers,
> Tim
>
> On Wed, Aug 24, 2016 at 8:15 AM, Yonghyun Hwang <yonghyun@cloudera.com>
> wrote:
>
> > Hello Martin,
> >
> > Your fix works great! :) As for the build failure, what I've found is
> that
> > cmake 3.2.3 causes a build failure w/o your fix while cmake 2.8.12 works
> > good.
> >
> > After the commit for "IMPALA-3924: Ubuntu16 support", cmake 3.2.3 got
> > deployed under "toolchain", which reveals this build issue.
> >
> > So, any one using the newer cmake would require your fix to properly
> build
> > impala w/ -so.
> >
> > Thank you,
> > Yonghyun
> >
> > On Tue, Aug 23, 2016 at 11:40 PM, Daniel Hecht <dhecht@cloudera.com>
> > wrote:
> >
> > > Hi Martin,
> > >
> > > Nice to hear from you. https://issues.cloudera.org/browse/IMPALA-3979
> > >
> > > Thanks,
> > > Dan
> > >
> > > On Tuesday, August 23, 2016, Martin Grund (Das Grundprinzip.de) <
> > > grundprinzip@gmail.com> wrote:
> > >
> > > > I've seen this in the past and propose a patch to fix it:
> > > > https://gerrit.cloudera.org/#/c/4108/
> > > >
> > > > Let me know if you have JIRA to track this to update the commit
> > message.
> > > >
> > > > Martin
> > > >
> > > > On Mon, Aug 22, 2016 at 10:15 PM Laszlo Gaal <
> laszlo.gaal@cloudera.com
> > > > <javascript:;>>
> > > > wrote:
> > > >
> > > > > link.txt is generated by cmake based on the contents of
> > CMakeLists.txt
> > > > and
> > > > > the project/environment settings. Most of the link-time
> dependencies
> > > seem
> > > > > to be set up in $(IMPALA_HOME)/be/CMakeLists.txt, based on
> settings
> > > > > calculated one level up in $(IMPALA_HOME)/CMakeLists.txt.
> > > > > $(IMPALA_HOME)/be/CMakeLists.txt looks complex; I'll look more
> into
> > it
> > > > > tomorrow.
> > > > >
> > > > > Thanks,
> > > > >
> > > > >     - LaszloG
> > > > >
> > > > > On Mon, Aug 22, 2016 at 6:07 PM, Yonghyun Hwang <
> > yonghyun@cloudera.com
> > > > <javascript:;>>
> > > > > wrote:
> > > > >
> > > > > > I observed the same linking error and digged this into further.
> The
> > > > below
> > > > > > is what I've found.
> > > > > >
> > > > > > *1. how to reproduce*
> > > > > >
> > > > > > # assuming that impala-config.sh is sourced
> > > > > > $ cd ${IMPALA_HOME}
> > > > > > *$ make -j8 impalad*
> > > > > > ...
> > > > > > [100%] Built target CodeGen
> > > > > > Linking CXX executable ../../build/debug/service/impalad
> > > > > > ...
> > > > > > ../../build/debug/util/libUtil.so: undefined reference to
> > > > > > `google_breakpad::ExceptionHandler::WriteMinidump(std::string
> > > const&,
> > > > > bool
> > > > > > (*)(google_breakpad::MinidumpDescriptor const&, void*, bool),
> > > void*)'
> > > > > > ../../build/debug/util/libUtil.so: undefined reference to
> > > > > > `snappy::MaxCompressedLength(unsigned long)'
> > > > > > ../../build/debug/util/libUtil.so: undefined reference to
> > > `my_strlen'
> > > > > > ../../build/debug/util/libUtil.so: undefined reference to
> > > > `LZ4_compress'
> > > > > > ../../build/debug/util/libUtil.so: undefined reference to
> > > > > > `BZ2_bzDecompress'
> > > > > > ...
> > > > > >
> > > > > >
> > > > > > *$ cd ${IMPALA_HOME}/be/src/service*
> > > > > > *$ ${IMPALA_HOME}/toolchain/cmake-3.2.3-p1/bin/cmake -E  *
> > > > > > *cmake_link_script CMakeFiles/impalad.dir/link.txt --verbose=*
> > > > > > ...
> > > > > > ../../build/debug/util/libUtil.so: undefined reference to
> > > > > > ...
> > > > > > *../../build/debug/util/libUtil.so: undefined reference to
> > > > > `LZ4_compress'*
> > > > > > *../../build/debug/util/libUtil.so: undefined reference to
> > > > > > `BZ2_bzDecompress'*
> > > > > > ../../build/debug/util/libUtil.so: undefined reference to
> > > > > > *`snappy::RawCompress*(char const*,
> > > > > > unsigned long, char*, unsigned long*)'
> > > > > > ...
> > > > > > boost::regex_traits<char, boost::cpp_regex_traits<char>
>
> > > > > >::do_assign(char
> > > > > > const*, char
> > > > > > const*, unsigned int)'
> > > > > > ...
> > > > > > ../../build/debug/util/libUtil.so: undefined reference to
> > > *`snappy::*
> > > > > > RawUncompress(char
> > > > > > const*, unsigned long, char*)'
> > > > > >
> > > > > > *2. cause of the failure*
> > > > > >
> > > > > > $ cd ${IMPALA_HOME}/be/src/service
> > > > > > *$ cat CMakeFiles/impalad.dir/link.txt*
> > > > > > ${IMPALA_HOME}/toolchain/gcc-4.9.2/bin/g++    -Wall
> > > -Wno-sign-compare
> > > > > > -Wno-unknown-pragmas -pthread -fno-strict-aliasing -std=c++14
> > > > > > -Wno-deprecated ...
> > > > > > *../../../toolchain/snappy-1.1.3/lib/libsnappy.a
> > > > > > ../../../toolchain/lz4-svn/lib/liblz4.a*
> > > > > > ...
> > > > > > *../../../toolchain/boost-1.57.0/lib/libboost_regex.a*
> > > > > > ...
> > > > > > *../../build/debug/util/libUtil.so*
> > > > > > -lpthread -ldl -lssl -lcrypto
> > > > > >
> > > > > > Apparently, libUtil.so __should__ show before libsnappy.a and
> > > *regex.a.
> > > > > >
> > > > > >
> > > > > > Next step is to find out who creates CMakeFiles/impalad.dir/link.
> > txt
> > > > and
> > > > > > why libUtil.so is placed after libsnappy.a, liblz4.a, and
> > > > > libboost_regex.a
> > > > > >
> > > > > > Thank you,
> > > > > > Yonghyun
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, Aug 17, 2016 at 10:13 AM, Henry Robinson <
> > henry@cloudera.com
> > > > <javascript:;>>
> > > > > > wrote:
> > > > > >
> > > > > > > Only that it's IMPALA-3979. Does anyone have any theories
yet?
> > > > > > >
> > > > > > > On 17 August 2016 at 10:10, Dimitris Tsirogiannis <
> > > > > > > dtsirogiannis@cloudera.com <javascript:;>> wrote:
> > > > > > >
> > > > > > > > Hey guys,
> > > > > > > >
> > > > > > > > When I build from the latest master using the
> > -build_shared_libs
> > > > > > option I
> > > > > > > > get linking errors (see [1]). Building with static
works
> fine.
> > > Any
> > > > > > ideas?
> > > > > > > >
> > > > > > > > Thanks
> > > > > > > > Dimitris
> > > > > > > >
> > > > > > > >
> > > > > > > > [1]
> > > > > > > > Linking CXX executable ../../build/debug/service/impalad
> > > > > > > > ../../build/debug/util/libUtil.so: error: undefined
> reference
> > to
> > > > > > > > 'BZ2_bzBuffToBuffCompress'
> > > > > > > > ../../build/debug/util/libUtil.so: error: undefined
> reference
> > to
> > > > > > > > 'snappy::MaxCompressedLength(unsigned long)'
> > > > > > > > ../../build/debug/util/libUtil.so: error: undefined
> reference
> > to
> > > > > > > > 'snappy::RawCompress(char const*, unsigned long, char*,
> > unsigned
> > > > > > long*)'
> > > > > > > > ../../build/debug/util/libUtil.so: error: undefined
> reference
> > to
> > > > > > > > 'LZ4_compress'
> > > > > > > > ../../build/debug/util/libUtil.so: error: undefined
> reference
> > to
> > > > > > > > 'BZ2_bzDecompressEnd'
> > > > > > > > ../../build/debug/util/libUtil.so: error: undefined
> reference
> > to
> > > > > > > > 'BZ2_bzDecompressInit'
> > > > > > > > ../../build/debug/util/libUtil.so: error: undefined
> reference
> > to
> > > > > > > > 'BZ2_bzBuffToBuffDecompress'
> > > > > > > > ../../build/debug/util/libUtil.so: error: undefined
> reference
> > to
> > > > > > > > 'BZ2_bzDecompress'
> > > > > > > > ../../build/debug/util/libUtil.so: error: undefined
> reference
> > to
> > > > > > > > 'snappy::GetUncompressedLength(char const*, unsigned
long,
> > > > unsigned
> > > > > > > > long*)'
> > > > > > > > ../../build/debug/util/libUtil.so: error: undefined
> reference
> > to
> > > > > > > > 'snappy::RawUncompress(char const*, unsigned long,
char*)'
> > > > > > > > ../../build/debug/util/libUtil.so: error: undefined
> reference
> > to
> > > > > > > > 'LZ4_uncompress'
> > > > > > > > ../../build/debug/util/libUtil.so: error: undefined
> reference
> > to
> > > > > > > > 'my_strlen'
> > > > > > > > ../../build/debug/util/libUtil.so: error: undefined
> reference
> > to
> > > > > > > > 'google_breakpad::ExceptionHandler::
> WriteMinidump(std::string
> > > > const&,
> > > > > > > bool
> > > > > > > > (*)(google_breakpad::MinidumpDescriptor const&,
void*,
> bool),
> > > > void*)'
> > > > > > > > ../../build/debug/util/libUtil.so: error: undefined
> reference
> > to
> > > > > > > > 'google_breakpad::ExceptionHandler::ExceptionHandler(google_
> > > > > > > > breakpad::MinidumpDescriptor
> > > > > > > > const&, bool (*)(void*), bool (*)(google_breakpad::
> > > > MinidumpDescriptor
> > > > > > > > const&, void*, bool), void*, bool, int)'
> > > > > > > > ../../build/debug/util/libUtil.so: error: undefined
> reference
> > to
> > > > > > > > 'ProfilerStart'
> > > > > > > > ../../build/debug/util/libUtil.so: error: undefined
> reference
> > to
> > > > > > > > 'ProfilerStop'
> > > > > > > > ../../build/debug/util/libUtil.so: error: undefined
> reference
> > to
> > > > > > > > 'boost::basic_regex<char, boost::regex_traits<char,
> > > > > > > > boost::cpp_regex_traits<char> > >::do_assign(char
const*,
> char
> > > > > const*,
> > > > > > > > unsigned int)'
> > > > > > > > ../../build/debug/util/libUtil.so: error: undefined
> reference
> > to
> > > > > > > > 'boost::re_detail::perl_matcher<__gnu_cxx::__normal_
> > > iterator<char
> > > > > > > const*,
> > > > > > > > std::string>,
> > > > > > > > std::allocator<boost::sub_match<__gnu_cxx::__normal_
> > > iterator<char
> > > > > > > const*,
> > > > > > > > std::string> > >, boost::regex_traits<char,
> > > > > > > boost::cpp_regex_traits<char> >
> > > > > > > > >::find()'
> > > > > > > > ../../build/debug/util/libUtil.so: error: undefined
> reference
> > to
> > > > > > > > 'boost::re_detail::perl_matcher<__gnu_cxx::__normal_
> > > iterator<char
> > > > > > > const*,
> > > > > > > > std::string>,
> > > > > > > > std::allocator<boost::sub_match<__gnu_cxx::__normal_
> > > iterator<char
> > > > > > > const*,
> > > > > > > > std::string> > >, boost::regex_traits<char,
> > > > > > > boost::cpp_regex_traits<char> >
> > > > > > > > >::construct_init(boost::basic_regex<char,
> > > > boost::regex_traits<char,
> > > > > > > > boost::cpp_regex_traits<char> > > const&,
> > > > > > > > boost::regex_constants::_match_flags)'
> > > > > > > > /opt/Impala-Toolchain/openldap-2.4.25/lib/libldap.a(
> > > > > > > > os-ip.o):os-ip.c:function
> > > > > > > > ldap_int_poll: warning: `sys_nerr' is deprecated;
use
> > `strerror'
> > > or
> > > > > > > > `strerror_r' instead
> > > > > > > > /opt/Impala-Toolchain/openldap-2.4.25/lib/libldap.a(
> > > > > > > > os-ip.o):os-ip.c:function
> > > > > > > > ldap_int_poll: warning: `sys_errlist' is deprecated;
use
> > > `strerror'
> > > > > or
> > > > > > > > `strerror_r' instead
> > > > > > > > collect2: error: ld returned 1 exit status
> > > > > > > > be/src/service/CMakeFiles/impalad.dir/build.make:166:
recipe
> > for
> > > > > > target
> > > > > > > > 'be/build/debug/service/impalad' failed
> > > > > > > > make[3]: *** [be/build/debug/service/impalad] Error
1
> > > > > > > > CMakeFiles/Makefile2:3682: recipe for target
> > > > > > > > 'be/src/service/CMakeFiles/impalad.dir/all' failed
> > > > > > > > make[2]: *** [be/src/service/CMakeFiles/impalad.dir/all]
> > Error 2
> > > > > > > > CMakeFiles/Makefile2:3694: recipe for target
> > > > > > > > 'be/src/service/CMakeFiles/impalad.dir/rule' failed
> > > > > > > > make[1]: *** [be/src/service/CMakeFiles/impalad.dir/rule]
> > Error
> > > 2
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Henry Robinson
> > > > > > > Software Engineer
> > > > > > > Cloudera
> > > > > > > 415-994-6679
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>



-- 
Henry Robinson
Software Engineer
Cloudera
415-994-6679

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message