hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Joseph Evans (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4458) Warn if java.library.path is used for AM or Task
Date Tue, 23 Oct 2012 16:11:13 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13482441#comment-13482441

Robert Joseph Evans commented on MAPREDUCE-4458:

A few things.

# the log message for the AM does not have a space in between LD_LIBRARY_PATH and mrAppOptions.
 I am not even really sure why we are outputting mrAppOptions.
# The admin and the normal opts for map and reduce tasks are smashed together without any
space in between.  Even though this should not really be a problem in practice it would be
nice to add it in for correctness sake.
# It seems like a lot of the warning could be combined into a single static method {code}private
static void warnForJavaLibPath(String opts, String component, String envConf) {
  if (opts != null && opts.contains("-Djava.library.path")) {
    LOG.warn("Overriding java.library.path for " + component + ...);
# It might be good to not just have LD_LIBRARY_PATH, but also point them to the config that
can be used to set LD_LIBRARY_PATH.
> Warn if java.library.path is used for AM or Task
> ------------------------------------------------
>                 Key: MAPREDUCE-4458
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4458
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: mrv2
>    Affects Versions: 0.23.3, 3.0.0, 2.0.2-alpha
>            Reporter: Robert Joseph Evans
>            Assignee: Robert Parker
>         Attachments: MAPREDUCE-4458.patch
> If java.library.path is used on the command line for launching an MRAppMaster or an MR
Task, it could conflict with how standard Hadoop/HDFS JNI libraries and dependencies are found.
 At a minimum the client should output a warning and ask the user to switch to LD_LIBRARY_PATH.
 It would be nice to automatically do this for them but parsing the command line is scary
so just a warning is probably good enough for now.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message