hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2801) Include the native libs in java.library.path
Date Thu, 15 Sep 2011 00:04:09 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105023#comment-13105023

Arun C Murthy commented on MAPREDUCE-2801:

Bobby, the more I think about this approach the less I like it.

Essentially relying on NM to supply critical env for the task is a broken concept - this ensures
that you can't run the same task with different env. Here 'env' can be jars, shared-objects

I'm thinking we shud just bite the bullet, go the whole 9 yards and fix the classpath/ld_library_path
etc. in the *correct* way i.e. throw out our reliance on bin/mapred and do everything via
the DistributedCache i.e. MAPREDUCE-2880. 


Apologies for dragging this. IAC, this won't fix the original issue - the NM doesn't have
the right java.library.path for setting up compression libs correctly.

Arguably we can fix this, but I'd rather do it the *right* way i.e. MAPREDUCE-2880.

> Include the native libs in java.library.path 
> ---------------------------------------------
>                 Key: MAPREDUCE-2801
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2801
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ramya Sunil
>            Assignee: Robert Joseph Evans
>             Fix For: 0.23.0
>         Attachments: MR-2801.patch, MR-2801.patch
> For the child tasks in mrv2, java.library.path is set to just $PWD and the native libs
are not included. Whereas in 0.20.x, java.library.path for child tasks was set to <path
to native libs>:$PWD

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message