hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jim Brennan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7677) HADOOP_CONF_DIR should not be automatically put in task environment
Date Mon, 29 Jan 2018 22:43:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16344137#comment-16344137

Jim Brennan commented on YARN-7677:

Based on the discussion here and in YARN-7226, and after discussing with [~jlowe] and [~ebadger],
I have put up a patch for this.

The change is as follows:
 # Remove the line in ContainerLaunch.sh that explicitly adds HADOOP_CONF_DIR to the environment
(as noted above).
 # Instead of ignoring the whitelist in the case of docker, always add the whitelist environment
variables that are not already defined in the containers context using theĀ {{var:-default}}
variable expansion syntax.

In the docker case where a whitelist environment variable is defined in the image, this will
prevent the launch script from overwriting it with the one from the Nodemanager's environment.

The non-docker case behaves the same as before, except that the whitelisted environment variables

 that are not defined by the container context are set using theĀ {{var:-default}} syntax,
but in this case the default value is always used.

> HADOOP_CONF_DIR should not be automatically put in task environment
> -------------------------------------------------------------------
>                 Key: YARN-7677
>                 URL: https://issues.apache.org/jira/browse/YARN-7677
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Eric Badger
>            Assignee: Jim Brennan
>            Priority: Major
>         Attachments: YARN-7677.001.patch
> Currently, {{HADOOP_CONF_DIR}} is being put into the task environment whether it's set
by the user or not. It completely bypasses the whitelist and so there is no way for a task
to not have {{HADOOP_CONF_DIR}} set. This causes problems in the Docker use case where Docker
containers will set up their own environment and have their own {{HADOOP_CONF_DIR}} preset
in the image itself. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message