hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nigel Daley (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-3298) Add support for transitive native library dependencies in DistributedCache
Date Tue, 22 Apr 2008 14:32:21 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-3298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Nigel Daley updated HADOOP-3298:
--------------------------------

      Description: 
[Hadoop 1660] add the tasks's working directory to it's JAVA_LIBRARY_PATH, this allows to
distribute JNI based native libs with the DistributedCache by symbolically linking them to
the tasks's working directory.

There are many instances when the JNI components have transitive dependencies to other native
libraries. Even if we have symlinks for these in  tasks's working directory, they'll not resolved
as transitive dependencies of native libraries are looked up in the shell's LD_LIBRARY_PATH
and not in the JAVA_LIBRARY_PATH. This can be tackled by adding CWD to JAVA_LIBRARY_PATH in
the *hadoop-env.sh*.
A better way to do this would be to add the CWD only in environment of the Task Tracker as
only it needs it.


  was:

[Hadoop 1660] add the tasks's working directory to it's JAVA_LIBRARY_PATH, this allows to
distribute JNI based native libs with the DistributedCache by symbolically linking them to
the tasks's working directory.

There are many instances when the JNI components have transitive dependencies to other native
libraries. Even if we have symlinks for these in  tasks's working directory, they'll not resolved
as transitive dependencies of native libraries are looked up in the shell's LD_LIBRARY_PATH
and not in the JAVA_LIBRARY_PATH. This can be tackled by adding CWD to JAVA_LIBRARY_PATH in
the *hadoop-env.sh*.
A better way to do this would be to add the CWD only in environment of the Task Tracker as
only it needs it.


    Fix Version/s:     (was: 0.18.0)

> Add support for transitive native library dependencies in DistributedCache 
> ---------------------------------------------------------------------------
>
>                 Key: HADOOP-3298
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3298
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>         Environment: unix (different handling would be required for windows)
>            Reporter: Subramaniam Krishnan
>
> [Hadoop 1660] add the tasks's working directory to it's JAVA_LIBRARY_PATH, this allows
to distribute JNI based native libs with the DistributedCache by symbolically linking them
to the tasks's working directory.
> There are many instances when the JNI components have transitive dependencies to other
native libraries. Even if we have symlinks for these in  tasks's working directory, they'll
not resolved as transitive dependencies of native libraries are looked up in the shell's LD_LIBRARY_PATH
and not in the JAVA_LIBRARY_PATH. This can be tackled by adding CWD to JAVA_LIBRARY_PATH in
the *hadoop-env.sh*.
> A better way to do this would be to add the CWD only in environment of the Task Tracker
as only it needs it.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message