hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8806) libhadoop.so: dlopen should be better at locating libsnappy.so, etc.
Date Sun, 16 Sep 2012 03:12:08 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13456529#comment-13456529
] 

Colin Patrick McCabe commented on HADOOP-8806:
----------------------------------------------

bq. On the downside, setting DT_RPATH=$ORIGIN on libhadoop.so pollutes the search path for
later dlopens in the main executable. In my test:
{code}
  main.c
    -> dlopens /path/to/libbar.so with DT_RPATH=$ORIGIN
          -> dlopens libfoo.so
    -> dlopens libfoo.so
{code}

I'm afraid you have been misled.

At the point when you are dlopening libfoo.so a second time, it is already linked to the executable.
 Essentially, the dynamic linker realizes that you already have libfoo loaded, and doesn't
try to reload it a second time.

This behavior is sort of hinted at in the man page for dlopen, although it's not spelled out
explicitly.  See the documentation for {{RTLD_NOLOAD}}, for example.  According to the man
page, "This can be used to test if the library is already resident."  To me, this implies
that:
* libraries can be either resident or not
* libraries can't be loaded twice in the same executable

Although those stipulations aren't spelled out explicitly.  (Time to submit a patch?)

Here's a modification of your test program which illustrates what I mean.  Basically it does:
{code}
  main.c
    -> dlopens /path/to/libbar.so with DT_RPATH=$ORIGIN
          -> dlopens libfoo.so
    -> dlopens libbaz.so
{code}

The last dlopen fails because {{libbaz.so}} is not in the library search path (although it
would succeed for {{libfoo.so}} opening it, due to {{RPATH}}.)
                
> libhadoop.so: dlopen should be better at locating libsnappy.so, etc.
> --------------------------------------------------------------------
>
>                 Key: HADOOP-8806
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8806
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>            Priority: Minor
>         Attachments: HADOOP-8806.003.patch, rpathtest2.tar.gz, rpathtest.tar.gz
>
>
> libhadoop calls {{dlopen}} to load {{libsnappy.so}} and {{libz.so}}.  These libraries
can be bundled in the {{$HADOOP_ROOT/lib/native}} directory.  For example, the {{-Dbundle.snappy}}
build option copies {{libsnappy.so}} to this directory.  However, snappy can't be loaded from
this directory unless {{LD_LIBRARY_PATH}} is set to include this directory.
> Can we make this configuration "just work" without needing to rely on {{LD_LIBRARY_PATH}}?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message