hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Yang (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-7516) Security check for untrusted docker image
Date Thu, 11 Jan 2018 20:16:01 GMT

    [ https://issues.apache.org/jira/browse/YARN-7516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322876#comment-16322876
] 

Eric Yang edited comment on YARN-7516 at 1/11/18 8:15 PM:
----------------------------------------------------------

[~ebadger] {{--privileged --cap-drop='ALL'}} will not have disk devices other than container
root file system are hidden unless explicitly specified.  Since we removed the device specification
from docker run command.  It will essentially create enough read only device for systemd or
supervisor like program to run without harming the host OS.

Here are the output of /dev for comparison:

Host:
{code}
autofs           crash  hpet       loop-control        nvram   rtc0      tty1   tty18  tty26
 tty34  tty42  tty50  tty59  ttyS0    vcs    vga_arbiter
block            disk   hugepages  mapper              oldmem  shm       tty10  tty19  tty27
 tty35  tty43  tty51  tty6   ttyS1    vcs1   vhost-net
btrfs-control    dm-0   initctl    mcelog              port    snapshot  tty11  tty2   tty28
 tty36  tty44  tty52  tty60  ttyS2    vcs6   virtio-ports
bus              dm-2   input      mem                 ppp     snd       tty12  tty20  tty29
 tty37  tty45  tty53  tty61  ttyS3    vcsa   vport1p1
char             dri    kmsg       mqueue              ptmx    stderr    tty13  tty21  tty3
  tty38  tty46  tty54  tty62  uhid     vcsa1  zero
console          fb0    kvm        net                 pts     stdin     tty14  tty22  tty30
 tty39  tty47  tty55  tty63  uinput   vcsa6
core             fd     log        network_latency     random  stdout    tty15  tty23  tty31
 tty4   tty48  tty56  tty7   urandom  vda
cpu              full   loop0      network_throughput  raw     tty       tty16  tty24  tty32
 tty40  tty49  tty57  tty8   usbmon0  vda1
cpu_dma_latency  fuse   loop1      null                rtc     tty0      tty17  tty25  tty33
 tty41  tty5   tty58  tty9   usbmon1  vfio
{code}

{{--privileged}}
{code}
autofs           dm-0  hpet          mcelog              oldmem  shm       tty1   tty17  tty24
 tty31  tty39  tty46  tty53  tty60  ttyS1    vcs    vfio
btrfs-control    dm-1  input         mem                 port    snapshot  tty10  tty18  tty25
 tty32  tty4   tty47  tty54  tty61  ttyS2    vcs1   vga_arbiter
bus              dm-2  kmsg          mqueue              ppp     snd       tty11  tty19  tty26
 tty33  tty40  tty48  tty55  tty62  ttyS3    vcs6   vhost-net
console          dri   kvm           net                 ptmx    stderr    tty12  tty2   tty27
 tty34  tty41  tty49  tty56  tty63  uhid     vcsa   vport1p1
core             fb0   loop-control  network_latency     pts     stdin     tty13  tty20  tty28
 tty35  tty42  tty5   tty57  tty7   uinput   vcsa1  zero
cpu              fd    loop0         network_throughput  random  stdout    tty14  tty21  tty29
 tty36  tty43  tty50  tty58  tty8   urandom  vcsa6
cpu_dma_latency  full  loop1         null                raw     tty       tty15  tty22  tty3
  tty37  tty44  tty51  tty59  tty9   usbmon0  vda
crash            fuse  mapper        nvram               rtc0    tty0      tty16  tty23  tty30
 tty38  tty45  tty52  tty6   ttyS0  usbmon1  vda1
{code}

{{--privileged --cap-drop='ALL'}}
{code}
autofs           dm-0  input         mem                 port    snapshot  tty10  tty18  tty25
 tty32  tty4   tty47  tty54  tty61  ttyS2    vcs1   vga_arbiter
btrfs-control    dm-1  kmsg          mqueue              ppp     snd       tty11  tty19  tty26
 tty33  tty40  tty48  tty55  tty62  ttyS3    vcs6   vhost-net
bus              dri   kvm           net                 ptmx    stderr    tty12  tty2   tty27
 tty34  tty41  tty49  tty56  tty63  uhid     vcsa   vport1p1
console          fb0   loop-control  network_latency     pts     stdin     tty13  tty20  tty28
 tty35  tty42  tty5   tty57  tty7   uinput   vcsa1  zero
core             fd    loop0         network_throughput  random  stdout    tty14  tty21  tty29
 tty36  tty43  tty50  tty58  tty8   urandom  vcsa6
cpu              full  loop1         null                raw     tty       tty15  tty22  tty3
  tty37  tty44  tty51  tty59  tty9   usbmon0  vda
cpu_dma_latency  fuse  mapper        nvram               rtc0    tty0      tty16  tty23  tty30
 tty38  tty45  tty52  tty6   ttyS0  usbmon1  vda1
crash            hpet  mcelog        oldmem              shm     tty1      tty17  tty24  tty31
 tty39  tty46  tty53  tty60  ttyS1  vcs      vfio
{code}

Notice that I have a dm-2 disk device, but it is not showing with all capabilities are dropped.

The goal is have ability to try multi-process containers which uses systemd or supervisor
to start without endangering host system.


was (Author: eyang):
[~ebadger] {{--privileged --cap-drop='ALL'}} will not have /dev/shm, and disk devices other
than container root file system are hidden unless explicitly specified.  Since we removed
the device specification from docker run command.  It will essentially create enough read
only device for systemd or supervisor like program to run without harming the host OS.

Here are the output of /dev for comparison:

Host:
{code}
autofs           crash  hpet       loop-control        nvram   rtc0      tty1   tty18  tty26
 tty34  tty42  tty50  tty59  ttyS0    vcs    vga_arbiter
block            disk   hugepages  mapper              oldmem  shm       tty10  tty19  tty27
 tty35  tty43  tty51  tty6   ttyS1    vcs1   vhost-net
btrfs-control    dm-0   initctl    mcelog              port    snapshot  tty11  tty2   tty28
 tty36  tty44  tty52  tty60  ttyS2    vcs6   virtio-ports
bus              dm-2   input      mem                 ppp     snd       tty12  tty20  tty29
 tty37  tty45  tty53  tty61  ttyS3    vcsa   vport1p1
char             dri    kmsg       mqueue              ptmx    stderr    tty13  tty21  tty3
  tty38  tty46  tty54  tty62  uhid     vcsa1  zero
console          fb0    kvm        net                 pts     stdin     tty14  tty22  tty30
 tty39  tty47  tty55  tty63  uinput   vcsa6
core             fd     log        network_latency     random  stdout    tty15  tty23  tty31
 tty4   tty48  tty56  tty7   urandom  vda
cpu              full   loop0      network_throughput  raw     tty       tty16  tty24  tty32
 tty40  tty49  tty57  tty8   usbmon0  vda1
cpu_dma_latency  fuse   loop1      null                rtc     tty0      tty17  tty25  tty33
 tty41  tty5   tty58  tty9   usbmon1  vfio
{code}

{{--privileged}}
{code}
autofs           dm-0  hpet          mcelog              oldmem  shm       tty1   tty17  tty24
 tty31  tty39  tty46  tty53  tty60  ttyS1    vcs    vfio
btrfs-control    dm-1  input         mem                 port    snapshot  tty10  tty18  tty25
 tty32  tty4   tty47  tty54  tty61  ttyS2    vcs1   vga_arbiter
bus              dm-2  kmsg          mqueue              ppp     snd       tty11  tty19  tty26
 tty33  tty40  tty48  tty55  tty62  ttyS3    vcs6   vhost-net
console          dri   kvm           net                 ptmx    stderr    tty12  tty2   tty27
 tty34  tty41  tty49  tty56  tty63  uhid     vcsa   vport1p1
core             fb0   loop-control  network_latency     pts     stdin     tty13  tty20  tty28
 tty35  tty42  tty5   tty57  tty7   uinput   vcsa1  zero
cpu              fd    loop0         network_throughput  random  stdout    tty14  tty21  tty29
 tty36  tty43  tty50  tty58  tty8   urandom  vcsa6
cpu_dma_latency  full  loop1         null                raw     tty       tty15  tty22  tty3
  tty37  tty44  tty51  tty59  tty9   usbmon0  vda
crash            fuse  mapper        nvram               rtc0    tty0      tty16  tty23  tty30
 tty38  tty45  tty52  tty6   ttyS0  usbmon1  vda1
{code}

{{--privileged --cap-drop='ALL'}}
{code}
autofs           dm-0  input         mem                 port    snapshot  tty10  tty18  tty25
 tty32  tty4   tty47  tty54  tty61  ttyS2    vcs1   vga_arbiter
btrfs-control    dm-1  kmsg          mqueue              ppp     snd       tty11  tty19  tty26
 tty33  tty40  tty48  tty55  tty62  ttyS3    vcs6   vhost-net
bus              dri   kvm           net                 ptmx    stderr    tty12  tty2   tty27
 tty34  tty41  tty49  tty56  tty63  uhid     vcsa   vport1p1
console          fb0   loop-control  network_latency     pts     stdin     tty13  tty20  tty28
 tty35  tty42  tty5   tty57  tty7   uinput   vcsa1  zero
core             fd    loop0         network_throughput  random  stdout    tty14  tty21  tty29
 tty36  tty43  tty50  tty58  tty8   urandom  vcsa6
cpu              full  loop1         null                raw     tty       tty15  tty22  tty3
  tty37  tty44  tty51  tty59  tty9   usbmon0  vda
cpu_dma_latency  fuse  mapper        nvram               rtc0    tty0      tty16  tty23  tty30
 tty38  tty45  tty52  tty6   ttyS0  usbmon1  vda1
crash            hpet  mcelog        oldmem              shm     tty1      tty17  tty24  tty31
 tty39  tty46  tty53  tty60  ttyS1  vcs      vfio
{code}

Notice that I have a dm-2 disk device, but it is not showing with all capabilities are dropped.

The goal is have ability to try multi-process containers which uses systemd or supervisor
to start without endangering host system.

> Security check for untrusted docker image
> -----------------------------------------
>
>                 Key: YARN-7516
>                 URL: https://issues.apache.org/jira/browse/YARN-7516
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Eric Yang
>            Assignee: Eric Yang
>         Attachments: YARN-7516.001.patch, YARN-7516.002.patch, YARN-7516.003.patch, YARN-7516.004.patch,
YARN-7516.005.patch, YARN-7516.006.patch, YARN-7516.007.patch
>
>
> Hadoop YARN Services can support using private docker registry image or docker image
from docker hub.  In current implementation, Hadoop security is enforced through username
and group membership, and enforce uid:gid consistency in docker container and distributed
file system.  There is cloud use case for having ability to run untrusted docker image on
the same cluster for testing.  
> The basic requirement for untrusted container is to ensure all kernel and root privileges
are dropped, and there is no interaction with distributed file system to avoid contamination.
 We can probably enforce detection of untrusted docker image by checking the following:
> # If docker image is from public docker hub repository, the container is automatically
flagged as insecure, and disk volume mount are disabled automatically, and drop all kernel
capabilities.
> # If docker image is from private repository in docker hub, and there is a white list
to allow the private repository, disk volume mount is allowed, kernel capabilities follows
the allowed list.
> # If docker image is from private trusted registry with image name like "private.registry.local:5000/centos",
and white list allows this private trusted repository.  Disk volume mount is allowed, kernel
capabilities follows the allowed list.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message