hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Badger (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7226) Whitelisted variables do not support delayed variable expansion
Date Fri, 22 Sep 2017 19:16:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16176961#comment-16176961
] 

Eric Badger commented on YARN-7226:
-----------------------------------

I can see the need for Docker containers to supply their own env vars in the image, but I
agree that it should not change previous behavior of the whitelist. The client gets to decide
what image to use, so they should know the layout of the image anyway. And if they don't,
then they can fall back to the environment of the docker image. If we want for there to be
a separate list of env vars that cannot be changed, then we can do that, but I think it should
be empty and/or disabled by default to preserve current behavior. Otherwise, as [~jlowe] said,
users won't be able to supply their own values for the env vars in the whitelist. So my vote
would be for putting this patch in and then filing a followup jira to add a NM configurable
list of variables that cannot be changed by the user (empty/disabled by default)

> Whitelisted variables do not support delayed variable expansion
> ---------------------------------------------------------------
>
>                 Key: YARN-7226
>                 URL: https://issues.apache.org/jira/browse/YARN-7226
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 2.9.0, 2.8.1, 3.0.0-alpha4
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>         Attachments: YARN-7226.001.patch, YARN-7226.002.patch
>
>
> The nodemanager supports a configurable list of environment variables, via yarn.nodemanager.env-whitelist,
that will be propagated to the container's environment unless those variables were specified
in the container launch context.  Unfortunately the handling of these whitelisted variables
prevents using delayed variable expansion.  For example, if a user shipped their own version
of hadoop with their job via the distributed cache and specified:
> {noformat}
> HADOOP_COMMON_HOME={{PWD}}/my-private-hadoop/
> {noformat}
>  as part of their job, the variable will be set as the *literal* string:
> {noformat}
> $PWD/my-private-hadoop/
> {noformat}
> rather than having $PWD expand to the container's current directory as it does for any
other, non-whitelisted variable being set to the same value.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message