hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhankun Tang (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-4266) Allow whitelisted users to disable user re-mapping/squashing when launching docker containers
Date Tue, 15 Nov 2016 07:22:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-4266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15666332#comment-15666332
] 

Zhankun Tang edited comment on YARN-4266 at 11/15/16 7:22 AM:
--------------------------------------------------------------

[~sidharta-s], Great thanks for reviewing this!
{quote}
Like I mentioned in a earlier comment, the usermod operation only makes changes to the home
directory and not elsewhere in the image. Modifying the rest of the image is not scalable
and could significantly slow down the launch of every container where we go down this path.
{quote}
Yes. Agree with this. This is a drawback that we cannot avoid at present.
{quote}
Running this usermod operation also requires that launch_container.sh be launched as a privileged
user. Also, note that running docker run --pid=host … bash ../launch_container.sh $newUID
$containerUsername does not guarantee that launch_container.sh as described here will work
correctly - if the image has a ‘USER’ directive, launch_container.sh will be run as that
user and that user might not have privileges to run a usermod operation.
{quote}
You might missed the part in the patch that we'll use "\-\-user=root" to guarantee successful
"usermod". We first inspect the Docker image, if it setup a non-root user and wants to run
with it, we'll use "--user=root". If the setup user in image is root, we'll just let it go.

{quote}
In addition, I don’t believe we should be using ­­—pid=host. This exposes other containers’s
processes into this container - which will break isolation and possibly behavior for certain
applications (applications that assume a single instance is running on a ’node’, for example).
{quote}
thanks for pointing this. I forget to delete this when I'm trying different implementation(sudo
issue if I remember correctly). I have a double-check and --pid=host is not needed.
{quote}
Lastly, adding more commands that run inside the container (usermod in this case) adds even
more requirements for the docker image being launched : we already require that bash, find,
ls etc be present in the image.
{quote}
This usermod is installed by default in most distributions I guess. Since we already require
several commands in the image, why cannot we document this too?

IMO, this is the light-weight way that can work securely and won't break down the log staff.
The drawbacks are:
1. usermod is a requirement in Docker image
2. usermod only changes the UID of files in home directory.
I indeed got some complaint about current user remapping from customer. So I think this JIRA
is an important feature to make YARN a good supporter for container(Docker and others maybe)
and possibly not only big data Docker images. We should invite more people on this. [~templedf],
[~vvasudev], [~shanekumpf@gmail.com], [~zyluo]?


was (Author: tangzhankun):
[~sidharta-s], Great thanks for reviewing this!
{quote}
Like I mentioned in a earlier comment, the usermod operation only makes changes to the home
directory and not elsewhere in the image. Modifying the rest of the image is not scalable
and could significantly slow down the launch of every container where we go down this path.
{quote}
Yes. Agree with this. This is a drawback that we cannot avoid at present.
{quote}
Running this usermod operation also requires that launch_container.sh be launched as a privileged
user. Also, note that running docker run --pid=host … bash ../launch_container.sh $newUID
$containerUsername does not guarantee that launch_container.sh as described here will work
correctly - if the image has a ‘USER’ directive, launch_container.sh will be run as that
user and that user might not have privileges to run a usermod operation.
{quote}
You might missed the part in the patch that we'll use "\-\-user=root" to guarantee successful
"usermod". We first inspect the Docker image, if it setup a non-root user and wants to run
with it, we'll use "--user=root". If the setup user in image is root, we'll just let it go.

{quote}
In addition, I don’t believe we should be using ­­—pid=host. This exposes other containers’s
processes into this container - which will break isolation and possibly behavior for certain
applications (applications that assume a single instance is running on a ’node’, for example).
{quote}
thanks for pointing this. I forget to delete this when I'm trying different implementation(sudo
issue if I remember correctly). I have a double-check and --pid=host is not needed.
{quote}
Lastly, adding more commands that run inside the container (usermod in this case) adds even
more requirements for the docker image being launched : we already require that bash, find,
ls etc be present in the image.
{quote}
This usermod is installed by default in most distributions I guess. Since we already require
several commands in the image, why cannot we document this too?

IMO, this is the light-weight way that can work securely and won't break down the log staff.
I got some complaint about current user remapping from customer.
Anyway, this JIRA is an important feature to make YARN a good supporter for container(Docker
and others maybe) and possibly not only big data Docker images. We should involve more people
on this. [~templedf], [~vvasudev], [~shanekumpf@gmail.com], [~zyluo]?

> Allow whitelisted users to disable user re-mapping/squashing when launching docker containers
> ---------------------------------------------------------------------------------------------
>
>                 Key: YARN-4266
>                 URL: https://issues.apache.org/jira/browse/YARN-4266
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: yarn
>            Reporter: Sidharta Seethana
>            Assignee: Zhankun Tang
>         Attachments: YARN-4266-branch-2.8.001.patch, YARN-4266_Allow_whitelisted_users_to_disable_user_re-mapping.pdf,
YARN-4266_Allow_whitelisted_users_to_disable_user_re-mapping_v2.pdf, YARN-4266_Allow_whitelisted_users_to_disable_user_re-mapping_v3.pdf
>
>
> Docker provides a mechanism (the --user switch) that enables us to specify the user the
container processes should run as. We use this mechanism today when launching docker containers
. In non-secure mode, we run the docker container based on `yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user`
and in secure mode, as the submitting user. However, this mechanism breaks down with a large
number of 'pre-created' images which don't necessarily have the users available within the
image. Examples of such images include shared images that need to be used by multiple users.
We need a way in which we can allow a pre-defined set of users to run containers based on
existing images, without using the --user switch. There are some implications of disabling
this user squashing that we'll need to work through : log aggregation, artifact deletion etc.,



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message