kudu-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jdcry...@apache.org
Subject [4/5] incubator-kudu git commit: Set -rpath in all thirdparty libs for compat with gold
Date Fri, 12 Feb 2016 00:16:56 GMT
Set -rpath in all thirdparty libs for compat with gold

Make sure that thirdparty shared libs are built with a -rpath that
includes thirdparty/installed-deps* because the GNU gold linker
specifies RUNPATH by default, instead of RPATH. This is due to the fact
that at the time of this writing, gold defaults to --enable-new-dtags
while binutils ld defaults to --disable-new-dtags [1].

Unfortunately, glibc does not search an executable's RUNPATH for
transitive dependencies [2], like it does for RPATH. This results in
symbol not found errors for full builds done using gold as the linker
unless we apply this change.

This patch adds the installed-deps directory to the -rpath of thirdparty
shared objects so that the correct paths are set and the symbols are
resolved.

This issue was originally identified by Todd, and here is the text of
his email to the dev list:

a) I usually use gold rather than binutils ld as my linker. gold generates
RUNPATH instead of RPATH by default, which has the difference that RUNPATH
doesn't propagate transitively. In other words, even though
protoc-gen-insertions has installed-deps/lib in its RUNPATH, it doesn't use
_that_ RPATH when looking for dependencies of libglog.so, because
libglog.so has its own RUNPATH.

b) my libglog.so's RUNPATH doesn't have installed-deps/. It turns out that,
after I rebuilt my thirdparty from git completely clean, that one didn't
end up with it either. It only got installed-deps/ in its RUNPATH before
because I had an old 'libgflags.la' file lying around.

c) we no longer generate 'libgflags.la' from the gflags build
after 7a68e1f8baee04eaf259f7567ddc39e09a0f0938 which switched from using
autotools to cmake to build and install gflag. This means that we don't
generate the libtool control file 'libgflags.la' anymore, though my old one
got orphaned in the build directory.

[1] http://blog.tremily.us/posts/rpath/
[2] https://sourceware.org/bugzilla/show_bug.cgi?id=13945

Change-Id: Ic50b3667f96cec73497248751e9afc785e3f7979
Reviewed-on: http://gerrit.cloudera.org:8080/2123
Reviewed-by: Adar Dembo <adar@cloudera.com>
Tested-by: Kudu Jenkins


Project: http://git-wip-us.apache.org/repos/asf/incubator-kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kudu/commit/2567ed00
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kudu/tree/2567ed00
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kudu/diff/2567ed00

Branch: refs/heads/master
Commit: 2567ed0087ed6664a93ce1400e5f4f89c730cb14
Parents: 241e650
Author: Mike Percy <mpercy@apache.org>
Authored: Thu Feb 11 19:35:03 2016 +0200
Committer: Jean-Daniel Cryans <jdcryans@gerrit.cloudera.org>
Committed: Thu Feb 11 23:29:25 2016 +0000

----------------------------------------------------------------------
 thirdparty/build-thirdparty.sh | 24 ++++++++++++++++++++++--
 1 file changed, 22 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-kudu/blob/2567ed00/thirdparty/build-thirdparty.sh
----------------------------------------------------------------------
diff --git a/thirdparty/build-thirdparty.sh b/thirdparty/build-thirdparty.sh
index 8e4acbe..9febc2d 100755
--- a/thirdparty/build-thirdparty.sh
+++ b/thirdparty/build-thirdparty.sh
@@ -154,6 +154,19 @@ fi
 EXTRA_CFLAGS="-g $EXTRA_CFLAGS"
 EXTRA_CXXFLAGS="-g $EXTRA_CXXFLAGS"
 
+save_env
+
+# Specifying -Wl,-rpath has different default behavior on GNU binutils ld vs.
+# the GNU gold linker. ld sets RPATH (due to defaulting to --disable-new-dtags)
+# and gold sets RUNPATH (due to defaulting to --enable-new-dtags). At the time
+# of this writing, contrary to the way RPATH is treated, when RUNPATH is
+# specified on a binary, glibc doesn't respect it for transitive (non-direct)
+# library dependencies (see https://sourceware.org/bugzilla/show_bug.cgi?id=13945).
+# So we must set RUNPATH for all deps-of-deps on the dep libraries themselves.
+# This applies both here and the locations elsewhere in this script where we
+# add $PREFIX/lib to -Wl,-rpath.
+EXTRA_LDFLAGS="-Wl,-rpath,$PREFIX/lib $EXTRA_LDFLAGS"
+
 if [ -n "$OS_LINUX" ] && [ -n "$F_ALL" -o -n "$F_LIBUNWIND" ]; then
   build_libunwind
 fi
@@ -204,10 +217,15 @@ if [ -n "$OS_LINUX" ] && [ -n "$F_ALL" -o -n "$F_NVML" ]; then
   build_nvml
 fi
 
+restore_env
+
 ### Build C++ dependencies
 
 PREFIX=$PREFIX_DEPS
 
+save_env
+EXTRA_LDFLAGS="-Wl,-rpath,$PREFIX/lib $EXTRA_LDFLAGS"
+
 if [ -n "$F_ALL" -o -n "$F_GFLAGS" ]; then
   build_gflags
 fi
@@ -236,6 +254,8 @@ if [ -n "$F_ALL" -o -n "$F_CRCUTIL" ]; then
   build_crcutil
 fi
 
+restore_env
+
 ## Build C++ dependencies with TSAN instrumentation
 
 if [ -n "$F_TSAN" ]; then
@@ -298,7 +318,7 @@ if [ -n "$F_TSAN" ]; then
   EXTRA_CXXFLAGS="-isystem $PREFIX_LIBSTDCXX_TSAN/include/c++/$GCC_VERSION/backward $EXTRA_CXXFLAGS"
   EXTRA_CXXFLAGS="-isystem $PREFIX_LIBSTDCXX_TSAN/include/c++/$GCC_VERSION $EXTRA_CXXFLAGS"
   EXTRA_CXXFLAGS="-L$PREFIX_LIBSTDCXX_TSAN/lib $EXTRA_CXXFLAGS"
-  EXTRA_LDFLAGS="-Wl,-rpath,$PREFIX_LIBSTDCXX_TSAN/lib $EXTRA_LDFLAGS"
+  EXTRA_LDFLAGS="-Wl,-rpath,$PREFIX_LIBSTDCXX_TSAN/lib,-rpath,$PREFIX/lib $EXTRA_LDFLAGS"
 
   if [ -n "$F_ALL" -o -n "$F_PROTOBUF" ]; then
     build_protobuf
@@ -311,7 +331,7 @@ if [ -n "$F_TSAN" ]; then
   EXTRA_CXXFLAGS="-isystem $PREFIX_LIBSTDCXX/include/c++/$GCC_VERSION/backward $EXTRA_CXXFLAGS"
   EXTRA_CXXFLAGS="-isystem $PREFIX_LIBSTDCXX/include/c++/$GCC_VERSION $EXTRA_CXXFLAGS"
   EXTRA_CXXFLAGS="-L$PREFIX_LIBSTDCXX/lib $EXTRA_CXXFLAGS"
-  EXTRA_LDFLAGS="-Wl,-rpath,$PREFIX_LIBSTDCXX/lib $EXTRA_LDFLAGS"
+  EXTRA_LDFLAGS="-Wl,-rpath,$PREFIX_LIBSTDCXX/lib,-rpath,$PREFIX/lib $EXTRA_LDFLAGS"
 
   if [ -n "$F_ALL" -o -n "$F_GFLAGS" ]; then
     build_gflags


Mime
View raw message