hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5714) ContainerExecutor does not order environment map
Date Thu, 01 Feb 2018 21:37:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-5714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349312#comment-16349312

Jason Lowe commented on YARN-5714:

Sorry to come to this late. We ran across an instance of this debugging some environment variable
ordering issues in YARN-7677.

While the LinkedHashMap solution sounds nice and clean, I don't think it can work in practice.
The problem is we have more than one list of environment variables: the user-provided list
and the inherited variables via the NM whitelist. To make things worse, we don't know how
they could be interconnected. The problem in YARN-7677 occurred because variables in the user's
settings referenced variables in the NM whitelist, and the whitelist variables were listed
after the user variables. Theoretically an admin could setup NM variables that have "plugin"
variables that could come from user-provided settings, and thus we can't always assume NM
whitelist variables come before user variables.

In short, I think we'll have to do some sort of dependency analysis. I'm not a fan of parsing
the shell syntax to figure out the dependencies in the value, but I don't see a viable alternative
to get all the variables, user-specified and otherwise, listed in the proper order.

> ContainerExecutor does not order environment map
> ------------------------------------------------
>                 Key: YARN-5714
>                 URL: https://issues.apache.org/jira/browse/YARN-5714
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 2.4.1, 2.5.2, 2.7.3, 2.6.4, 3.0.0-alpha1
>         Environment: all (linux and windows alike)
>            Reporter: Remi Catherinot
>            Assignee: Remi Catherinot
>            Priority: Trivial
>              Labels: oct16-medium
>         Attachments: YARN-5714.001.patch, YARN-5714.002.patch, YARN-5714.003.patch, YARN-5714.004.patch,
YARN-5714.005.patch, YARN-5714.006.patch
>   Original Estimate: 120h
>  Remaining Estimate: 120h
> when dumping the launch container script, environment variables are dumped based on the
order internally used by the map implementation (hash based). It does not take into consideration
that some env varibales may refer each other, and so that some env variables must be declared
before those referencing them.
> In my case, i ended up having LD_LIBRARY_PATH which was depending on HADOOP_COMMON_HOME
being dumped before HADOOP_COMMON_HOME. Thus it had a wrong value and so native libraries
weren't loaded. jobs were running but not at their best efficiency. This is just a use case
falling into that bug, but i'm sure others may happen as well.
> I already have a patch running in my production environment, i just estimate to 5 days
for packaging the patch in the right fashion for JIRA + try my best to add tests.
> Note : the patch is not OS aware with a default empty implementation. I will only implement
the unix version on a 1st release. I'm not used to windows env variables syntax so it will
take me more time/research for it.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message