hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7677) Docker image cannot set HADOOP_CONF_DIR
Date Fri, 02 Feb 2018 13:51:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350353#comment-16350353

Jason Lowe commented on YARN-7677:

I realize now that the theoretical example cannot work in practice.  In order for there to
be a "hook" variable for the user to leverage, the variable would need to have escaped variable
expansion by the shell when it was originally set.  The variable would need to be set in the
NM's environment like, JAVA_HOME="/some/node/path/\$JDKVER".  While that could be a valid
path for the user when it is expanded in the container launch script, it is not a valid setting
for JAVA_HOME in the nodemanager itself.  NM whitelist variables are going to be variables
coming from a shell environment and not from XML property settings, so it's highly unlikely
they will retain unexpanded variable references.

In short, I'm cool with simply placing the NM whitelist variables first and simplifying YARN-5714
to list the variables in the launch script in the order they appear in their corresponding
configuration properties.  My apologies for the detour.

> Docker image cannot set HADOOP_CONF_DIR
> ---------------------------------------
>                 Key: YARN-7677
>                 URL: https://issues.apache.org/jira/browse/YARN-7677
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 3.0.0
>            Reporter: Eric Badger
>            Assignee: Jim Brennan
>            Priority: Major
>         Attachments: YARN-7677.001.patch, YARN-7677.002.patch
> Currently, {{HADOOP_CONF_DIR}} is being put into the task environment whether it's set
by the user or not. It completely bypasses the whitelist and so there is no way for a task
to not have {{HADOOP_CONF_DIR}} set. This causes problems in the Docker use case where Docker
containers will set up their own environment and have their own {{HADOOP_CONF_DIR}} preset
in the image itself. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message