hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Terence Yim (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-4727) Unable to override the $HADOOP_CONF_DIR env variable for container
Date Wed, 24 Feb 2016 01:24:18 GMT
Terence Yim created YARN-4727:

             Summary: Unable to override the $HADOOP_CONF_DIR env variable for container
                 Key: YARN-4727
                 URL: https://issues.apache.org/jira/browse/YARN-4727
             Project: Hadoop YARN
          Issue Type: Bug
          Components: nodemanager
    Affects Versions: 2.6.4, 2.7.2, 2.5.2, 2.4.1
            Reporter: Terence Yim

Given the default config of "yarn.nodemanager.env-whitelist", application should be able to
set the env variable $HADOOP_CONF_DIR to value other than the one in the NodeManager system
environment. However, I believe due to a bug in the {{org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch}}
class, it is not possible so.

>From the {{sanitizeEnv()}} method in the ContainerLaunch class (https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java#L991)

String[] whitelist = conf.get(YarnConfiguration.NM_ENV_WHITELIST, YarnConfiguration.DEFAULT_NM_ENV_WHITELIST).split(",");
for(String whitelistEnvVariable : whitelist) {
  putEnvIfAbsent(environment, whitelistEnvVariable.trim());


private static void putEnvIfAbsent(
    Map<String, String> environment, String variable) {
  if (environment.get(variable) == null) {
    putEnvIfNotNull(environment, variable, System.getenv(variable));
The {{putEnvIfAbsent}} call simply use the {{whitelistEnvVariable}} to call {{System.getenv(whitelistEnvVariable)}}
and set it to the environment map. It should be using the one from the {{launchContext}} instead.

This message was sent by Atlassian JIRA

View raw message