hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Badger (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7226) Whitelisted variables do not support delayed variable expansion
Date Tue, 26 Sep 2017 16:10:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16181019#comment-16181019
] 

Eric Badger commented on YARN-7226:
-----------------------------------

Hey [~jlowe], thanks for the patch. I have one question on it. 

{noformat}
+  @Override
+  public boolean useWhitelistEnv(Map<String, String> env) {
+    try {
+      LinuxContainerRuntime runtime = pickContainerRuntime(env);
+      return runtime.useWhitelistEnv(env);
+    } catch (ContainerExecutionException e) {
+      LOG.debug("Unable to determine runtime");
+      return false;
+    }
+  }
+
{noformat}
Is there a reason for us to catch the exception here?  I would think that if we can't figure
out which runtime to use that that would be fatal to the launch of the job.

Only other thing would be to fix the whitespace checkstyle issue

> Whitelisted variables do not support delayed variable expansion
> ---------------------------------------------------------------
>
>                 Key: YARN-7226
>                 URL: https://issues.apache.org/jira/browse/YARN-7226
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 2.9.0, 2.8.1, 3.0.0-alpha4
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>         Attachments: YARN-7226.001.patch, YARN-7226.002.patch, YARN-7226.003.patch
>
>
> The nodemanager supports a configurable list of environment variables, via yarn.nodemanager.env-whitelist,
that will be propagated to the container's environment unless those variables were specified
in the container launch context.  Unfortunately the handling of these whitelisted variables
prevents using delayed variable expansion.  For example, if a user shipped their own version
of hadoop with their job via the distributed cache and specified:
> {noformat}
> HADOOP_COMMON_HOME={{PWD}}/my-private-hadoop/
> {noformat}
>  as part of their job, the variable will be set as the *literal* string:
> {noformat}
> $PWD/my-private-hadoop/
> {noformat}
> rather than having $PWD expand to the container's current directory as it does for any
other, non-whitelisted variable being set to the same value.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message