hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7516) Security check for trusted docker image
Date Wed, 31 Jan 2018 18:57:01 GMT

    [ https://issues.apache.org/jira/browse/YARN-7516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347388#comment-16347388

Eric Yang commented on YARN-7516:

[~billie.rinaldi] [~shanekumpf@gmail.com] How do we resolve the conflict of having the same
image exist in both list?  For example, a spark image that can run YARN cluster mode as well
as a standalone spark may require the image to present in both lists.  If we need to build
two spark images for each programming paradigm, then we inconveniently double the work for
our developers.  IMHO, security enforcement should not interfere with programming paradigm.
 We might be digress from the original goal to sandbox untrusted images and provide minimum
amount of capabilities to test software from Internet.  It might be preferred to build a user
controllable flag for switching between yarn-mode vs default-mode for better control to decide
whether localizer directories and launcher script should be mounted.  I think that problem
needs to be discussed in a separate JIRA (YARN-7654) to prevent morphing of the current one,
if you don't mind.

My concern over this current patch, drop all capabilities for untrusted image might limit
too much.  We might want to allow CHOWN,SETGID,SETUID,NET_BIND_SERVICE,KILL capabilities for
multi-users docker image to function.

> Security check for trusted docker image
> ---------------------------------------
>                 Key: YARN-7516
>                 URL: https://issues.apache.org/jira/browse/YARN-7516
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Eric Yang
>            Assignee: Eric Yang
>            Priority: Major
>         Attachments: YARN-7516.001.patch, YARN-7516.002.patch, YARN-7516.003.patch, YARN-7516.004.patch,
YARN-7516.005.patch, YARN-7516.006.patch, YARN-7516.007.patch, YARN-7516.008.patch, YARN-7516.009.patch,
YARN-7516.010.patch, YARN-7516.011.patch, YARN-7516.012.patch, YARN-7516.013.patch, YARN-7516.014.patch,
> Hadoop YARN Services can support using private docker registry image or docker image
from docker hub.  In current implementation, Hadoop security is enforced through username
and group membership, and enforce uid:gid consistency in docker container and distributed
file system.  There is cloud use case for having ability to run untrusted docker image on
the same cluster for testing.  
> The basic requirement for untrusted container is to ensure all kernel and root privileges
are dropped, and there is no interaction with distributed file system to avoid contamination.
 We can probably enforce detection of untrusted docker image by checking the following:
> # If docker image is from public docker hub repository, the container is automatically
flagged as insecure, and disk volume mount are disabled automatically, and drop all kernel
> # If docker image is from private repository in docker hub, and there is a white list
to allow the private repository, disk volume mount is allowed, kernel capabilities follows
the allowed list.
> # If docker image is from private trusted registry with image name like "private.registry.local:5000/centos",
and white list allows this private trusted repository.  Disk volume mount is allowed, kernel
capabilities follows the allowed list.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message