hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-7226) Whitelisted variables do not support delayed variable expansion
Date Tue, 03 Oct 2017 16:01:00 GMT

     [ https://issues.apache.org/jira/browse/YARN-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jason Lowe updated YARN-7226:
-----------------------------
    Attachment: YARN-7226-branch-2.8.006.patch

The unit test failures are unrelated and are all caused by a bind exception due to an address
being in use.  I'm guessing some process leaked from a previous Jenkins run on the box and
is sitting on the desired port.  Those unit tests pass for me locally with the patch applied.

Attaching the branch-2.8 version of the patch.

> Whitelisted variables do not support delayed variable expansion
> ---------------------------------------------------------------
>
>                 Key: YARN-7226
>                 URL: https://issues.apache.org/jira/browse/YARN-7226
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 2.9.0, 2.8.1, 3.0.0-alpha4
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>         Attachments: YARN-7226.001.patch, YARN-7226.002.patch, YARN-7226.003.patch, YARN-7226.004.patch,
YARN-7226.005.patch, YARN-7226.006.patch, YARN-7226-branch-2.006.patch, YARN-7226-branch-2.8.006.patch
>
>
> The nodemanager supports a configurable list of environment variables, via yarn.nodemanager.env-whitelist,
that will be propagated to the container's environment unless those variables were specified
in the container launch context.  Unfortunately the handling of these whitelisted variables
prevents using delayed variable expansion.  For example, if a user shipped their own version
of hadoop with their job via the distributed cache and specified:
> {noformat}
> HADOOP_COMMON_HOME={{PWD}}/my-private-hadoop/
> {noformat}
>  as part of their job, the variable will be set as the *literal* string:
> {noformat}
> $PWD/my-private-hadoop/
> {noformat}
> rather than having $PWD expand to the container's current directory as it does for any
other, non-whitelisted variable being set to the same value.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message