hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Joseph Evans (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4458) Warn if java.library.path is used for AM or Task
Date Tue, 23 Oct 2012 16:11:13 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13482441#comment-13482441
] 

Robert Joseph Evans commented on MAPREDUCE-4458:
------------------------------------------------

A few things.

# the log message for the AM does not have a space in between LD_LIBRARY_PATH and mrAppOptions.
 I am not even really sure why we are outputting mrAppOptions.
# The admin and the normal opts for map and reduce tasks are smashed together without any
space in between.  Even though this should not really be a problem in practice it would be
nice to add it in for correctness sake.
# It seems like a lot of the warning could be combined into a single static method {code}private
static void warnForJavaLibPath(String opts, String component, String envConf) {
  if (opts != null && opts.contains("-Djava.library.path")) {
    LOG.warn("Overriding java.library.path for " + component + ...);
  }
{code}
# It might be good to not just have LD_LIBRARY_PATH, but also point them to the config that
can be used to set LD_LIBRARY_PATH.
                
> Warn if java.library.path is used for AM or Task
> ------------------------------------------------
>
>                 Key: MAPREDUCE-4458
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4458
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: mrv2
>    Affects Versions: 0.23.3, 3.0.0, 2.0.2-alpha
>            Reporter: Robert Joseph Evans
>            Assignee: Robert Parker
>         Attachments: MAPREDUCE-4458.patch
>
>
> If java.library.path is used on the command line for launching an MRAppMaster or an MR
Task, it could conflict with how standard Hadoop/HDFS JNI libraries and dependencies are found.
 At a minimum the client should output a warning and ask the user to switch to LD_LIBRARY_PATH.
 It would be nice to automatically do this for them but parsing the command line is scary
so just a warning is probably good enough for now.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message