hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-4577) Add Jar "lib" directory to TaskRunner's library.path setting to allow JNI libraries to be deployed via JAR file
Date Tue, 09 Dec 2008 18:39:44 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-4577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Arun C Murthy updated HADOOP-4577:

    Status: Open  (was: Patch Available)

Fair enough. Like I said, the patch looks good - do you mind fixing the documentation in http://hadoop.apache.org/core/docs/current/mapred_tutorial.html
and the comment in TaskRunner.java too?

      // Add java.library.path; necessary for loading native libraries.
      // 1. To support native-hadoop library i.e. libhadoop.so, we add the 
      //    parent processes' java.library.path to the child. 
      // 2. We also add the 'cwd' of the task to it's java.library.path to help 
      //    users distribute native libraries via the DistributedCache.
      // 3. The user can also specify extra paths to be added to the 
      //    java.library.path via mapred.child.java.opts.

If you are tied up currently to fix the mapred_tutorial.html, please go ahead and file a new
jira to fix the documentation alone... thanks!

> Add Jar "lib" directory to TaskRunner's library.path setting to allow JNI libraries to
be deployed via JAR file  
> -----------------------------------------------------------------------------------------------------------------
>                 Key: HADOOP-4577
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4577
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.18.1
>         Environment: Hadoop 18.1 Cluster with custom JNI shared libraries deployed in
lib directory of deployment JAR.
>            Reporter: Ahad Rana
>            Assignee: Ahad Rana
>             Fix For: 0.20.0
>         Attachments: HADOOP-4577-v1.patch
> It is extremely convenient to be able to deploy JNI libraries utilized in a custom map-reduce
job via the job's JAR file. The TaskRunner already establishes a precedent by automatically
adding any jar files contained in the "lib" directory of the job jar to the child map/reduce
process's classpath. Following this convention, it should also be possible to deploy custom
JNI libraries in the same lib directory. This involves adding the path to the job jar's lib
directory to the VM's library.path setting (after the jar has been expanded in the job cache
directory). This does not elimintate the need add dependent shared libraries that may be referenced
by the JNI libraries to the system's LD_LIBRARY_PATH variable. In our deployment configuration,
we usually pre-install third party shared libraries across the cluster and only deploy our
custom JNI libraries via the job jar.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message