hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kihwal Lee (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-3259) ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
Date Tue, 25 Oct 2011 12:18:33 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Kihwal Lee updated MAPREDUCE-3259:
----------------------------------

    Attachment: mapreduce-3259.patch

The attached patch is for trunk, but should also apply to branch-0.23.

Although the client-side environment may be set by users or by the admin, the framework including
what may be needed by the hadoop code base in java.library.path seems more reasonable. After
all libhadoop.so may be required to run things even if users don't explicitly use any features
that directly depend on it. 

Since meaningful tests need specific environment setup and the optional native build, no automated
test was added. The risk of this change should be minimal. 

Manual testing was carried out as follows.
- Enable JniBasedUnixUserGroupsMapping.
- Build without patch.
- Test failures observed in node manager and job client.
- Apply patch and rebuild.
- Without the library, many tests in node manager and job client failed.
- Installed the library.
- All tests pass.


                
> ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor
> --------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-3259
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3259
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 0.23.0, 0.24.0
>            Reporter: Kihwal Lee
>            Assignee: Kihwal Lee
>            Priority: Critical
>         Attachments: mapreduce-3259.patch
>
>
> As seen in MAPREDUCE-2915, java.library.path is not being passed when the LCE spawns
a JVM for ContainerLocalizer. 
> However, unlike branch-0.20-security, the task runtime in 0.23 is unaffected by this.
This is because tasks' run-time environment is specified in the launch script by client. Setting
LD_LIBRARY_PATH is the primary way of specifying the locations of required native library
in this case. The config property, mapreduce.admin.user.env is always set in the job environment
and the default value is to add the path to the hadoop native library to LD_LABRARY_PATH.
> For JVM's being launched by the hadoop system scripts, java.library.path is set.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message