hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jim Brennan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7677) Docker image cannot set HADOOP_CONF_DIR
Date Wed, 14 Feb 2018 23:23:01 GMT

    [ https://issues.apache.org/jira/browse/YARN-7677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16364945#comment-16364945
] 

Jim Brennan commented on YARN-7677:
-----------------------------------

{quote}I noticed that sanitizeWindowsEnv no longer adds CLASSPATH to the set of NM vars as
it did in the previous patch. Intentional?
{quote}
Yes.  That was intentional.  I don't think it should be adding it.   sanitizeWindowsEnv
only adds to an existing CLASSPATH to make it work in the Windows Env.  It seems to me like
the CLASSPATH belongs with the user variables, after the NM variables.

I think this becomes more important when we address YARN-5714 - if we preserve the order
of the ContainerLaunchContext (user) variables, I don't think we'd want to pull the CLASSPATH
out and define it with the NM variables, potentially reordering it with respect to other user
variables.  That said, there is currently no guarantee about the relative ordering of user-defined
variables.

> Docker image cannot set HADOOP_CONF_DIR
> ---------------------------------------
>
>                 Key: YARN-7677
>                 URL: https://issues.apache.org/jira/browse/YARN-7677
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 3.0.0
>            Reporter: Eric Badger
>            Assignee: Jim Brennan
>            Priority: Major
>         Attachments: YARN-7677.001.patch, YARN-7677.002.patch, YARN-7677.003.patch, YARN-7677.004.patch,
YARN-7677.005.patch
>
>
> Currently, {{HADOOP_CONF_DIR}} is being put into the task environment whether it's set
by the user or not. It completely bypasses the whitelist and so there is no way for a task
to not have {{HADOOP_CONF_DIR}} set. This causes problems in the Docker use case where Docker
containers will set up their own environment and have their own {{HADOOP_CONF_DIR}} preset
in the image itself. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message