hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhankun Tang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5360) Decouple host user and Docker container user
Date Mon, 01 Aug 2016 04:52:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401522#comment-15401522

Zhankun Tang commented on YARN-5360:

[~sidharta-s], Thanks for the explaination. And I appreciate that if you can share the spark
docker image. I would like to have a try.

This JIRA is originally not only just dropping --user but also searching for a flexible interface
different with YARN-4266 to decouple host user and Docker container user. The *main difference*
is that this JIRA would like to expose this --user to application while YARN-4266 utilize
whitelisted user as admin configuration to drop --user. But both changes would have several
same implications like log aggregation, etc.

Since it is not recommended to change existing use case or expose --user to application, I
think we can move to YARN-4266 to discuss more details. 

> Decouple host user and Docker container user
> --------------------------------------------
>                 Key: YARN-5360
>                 URL: https://issues.apache.org/jira/browse/YARN-5360
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: yarn
>            Reporter: Zhankun Tang
>            Assignee: Zhankun Tang
> There is *a dependency between job submitting user and the user in the Docker image*
in LCE currently. For instance, in order to run the Docker container as yarn user, we can
choose set the "yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user" to yarn
and leave "yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users" default (true).
Then LCE will choose yarn ( UID maybe 1001) as the user running jobs.
> LCE will mount the generated launch_container.sh (owned by the running job user) and
/etc/passwd (*current the code is mounting to container's /etc/password, I think it's a mistake*)
into the Docker container and utilizes "docker run --user=<run_as_user>" option to get
it done internally.
> Mounting /etc/passwd to the container is a not good choice due to override original users
defined in Docker image. As far as I know, since Docker v1.8 (or maybe earlier), the Docker
run command "--user=" option accepts UID and *when passing UID, the user does not have to
exist in the container*. So we could use UID instead of user name to construct the Docker
run command to eliminate the dependency that create the same user in the Docker image. This
enables LCE the ability to launch any Docker container safely regardless what users in it.
> But this is not enough to decouple host user and Docker container user. The final solution
we are searching for are focused on allowing users to run their Docker images flexibly without
involving dependencies of YARN and make sure the container won't bring in security risk.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message