hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7677) HADOOP_CONF_DIR should not be automatically put in task environment
Date Wed, 03 Jan 2018 21:00:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-7677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16310273#comment-16310273

Eric Yang commented on YARN-7677:

[~ebadger] My initial reaction would be to make docker container to follow the host layout
for single cluster setup.  Some Hadoop features will not work, i.e. short circuits read, if
host and docker containers are not matching.  For docker container to regenerate fine tuned
Hadoop configuration to match host, it could take some effort from docker container developer.
 There is a high probability that developers end up using bind-mount to transport Hadoop configuration.

If we handle security properly with white list mount (YARN-5534), container-executor validation
(YARN-7590), and check sudo privileges before launching privileged container (YARN-7221).
 Any particular reason that we shouldn't allow read-only bind-mount {{HADOOP_CONF_DIR}}? 

> HADOOP_CONF_DIR should not be automatically put in task environment
> -------------------------------------------------------------------
>                 Key: YARN-7677
>                 URL: https://issues.apache.org/jira/browse/YARN-7677
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Eric Badger
>            Assignee: Eric Badger
> Currently, {{HADOOP_CONF_DIR}} is being put into the task environment whether it's set
by the user or not. It completely bypasses the whitelist and so there is no way for a task
to not have {{HADOOP_CONF_DIR}} set. This causes problems in the Docker use case where Docker
containers will set up their own environment and have their own {{HADOOP_CONF_DIR}} preset
in the image itself. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message