hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7677) Docker image cannot set HADOOP_CONF_DIR
Date Thu, 01 Feb 2018 22:39:06 GMT

    [ https://issues.apache.org/jira/browse/YARN-7677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349420#comment-16349420

Jason Lowe commented on YARN-7677:

bq. the primary use case is AM-defined vars being able to reference NM-defined vars, which
we could accomplish by writing NM vars to the launch script first.

That assumes NM vars don't ever reference variables that could be overridden by the user.
 For example, admin exports JAVA_HOME="/some/node/path/$JDKVER" and also sets JDKVER="1.8.0_u152",
making sure both JAVA_HOME and JDKVER are both in the whitelist.  The idea here is users can
select java versions by simply setting JDKVER to the version they need and don't need to know
the specific path where it exists.  That's a case where NM whitelist variables could reference
user variables, and listing NM vars first won't work.  Granted it might not always be working
right now, but if it is, forcing NM variables to be first would guarantee it stopped working.

> Docker image cannot set HADOOP_CONF_DIR
> ---------------------------------------
>                 Key: YARN-7677
>                 URL: https://issues.apache.org/jira/browse/YARN-7677
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 3.0.0
>            Reporter: Eric Badger
>            Assignee: Jim Brennan
>            Priority: Major
>         Attachments: YARN-7677.001.patch, YARN-7677.002.patch
> Currently, {{HADOOP_CONF_DIR}} is being put into the task environment whether it's set
by the user or not. It completely bypasses the whitelist and so there is no way for a task
to not have {{HADOOP_CONF_DIR}} set. This causes problems in the Docker use case where Docker
containers will set up their own environment and have their own {{HADOOP_CONF_DIR}} preset
in the image itself. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message