hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Yang (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-7197) Add support for a volume blacklist for docker containers
Date Mon, 30 Oct 2017 18:14:01 GMT

    [ https://issues.apache.org/jira/browse/YARN-7197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16225452#comment-16225452
] 

Eric Yang edited comment on YARN-7197 at 10/30/17 6:13 PM:
-----------------------------------------------------------

[~jlowe] 

{quote}Either /run isn't in the whitelist in the first place rendering the blacklist entry
moot or /run is in the whitelist and the user can simply mount {{/run}} and access the blacklist
path.{quote}

Let's expand on the real world example.  A hacker tries to take control of {{/run/docker.socket}}
to acquire root privileges and spawn root containers to access vital system area to become
root on the host system.  The system admin placed {{/var}} in read-write white list for ability
to write to database and log directories, without black list capability.  Hacker explicitly
specify {{/var/run/docker.socket}} to be included, put the socket in {{/tmp/docker.socket}}.
 Hacker generates a docker image with {{/etc/group}} modified to include his own name or setuid
bit binary in container.  Hack can successfully gain control to host level docker without
much effort.

{{/run}} contains a growing list of software that put their pid file or socket in this location.
 System admin can't say no to not allow other software to place their socket in {{/run}} location
and share between containers due to company requirement.  However, he still doesn't want to
let hacker gain root access.

h3. Solution 1:
System admin placed {{/var/*}} and {{/run/\*}} (except /run/docker.socket), carefully in read-write
white list.  None of the symlink is exposed.  Hacker can not get in.

h3. Solution 2 (All symlinks, and hardcoded locations are banned):
(Current proposed patch)
System admin specifies:
  white-list-read-write: {{/var}}, {{/run/\*}} (except /run/docker.socket), {{/mnt/hdfs/user/\*}}
(exception yarn)
  black-list: {{/var/run}},{{/run/docker.socket}},{{/mnt/hdfs/user/yarn}}
Hacker attempt to mount a symlink location resulting in access denied from container startup,
or explicit hard coded location also result in ban.

h3. Solution 3: (Replace black list location with empty directories):
(Jason proposed implementation)
System admin specifies:
  white-list-read-write: {{/var}},{{/run}},{{/mnt/hdfs/user}}
  black-list: {{/run/docker.socket}},{{/mnt/hdfs/user/yarn}}
Hacker attempt to mount a symlink location resulting in access denied from container startup,
or mount /run/docker.socket manually, but result in empty file.

All solutions requires system administrator to enforce ability to upload secure image to private
registry to prevent torjan horse in docker image.
 
I can see the appeal that without having to do a high upkeep of white-list-read-write directories
by the new proposal.  The third solution can throw people off, if they do not know about black-list
is hijacked to empty location.  However, the depth of directories might defeat second solution.
 If community favors the third solution, I can revise patch accordingly.


was (Author: eyang):
[~jlowe] 

{quote}Either /run isn't in the whitelist in the first place rendering the blacklist entry
moot or /run is in the whitelist and the user can simply mount {{/run}} and access the blacklist
path.{quote}

Let's expand on the real world example.  A hacker tries to take control of {{/run/docker.socket}}
to acquire root privileges and spawn root containers to access vital system area to become
root on the host system.  The system admin placed {{/var}} in read-write white list for ability
to write to database and log directories, without black list capability.  Hacker explicitly
specify {{/var/run/docker.socket}} to be included, put the socket in {{/tmp/docker.socket}}.
 Hacker generates a docker image with {{/etc/group}} modified to include his own name or setuid
bit binary in container.  Hack can successfully gain control to host level docker without
much effort.

{{/run}} contains a growing list of software that put their pid file or socket in this location.
 System admin can't say no to not allow other software to place their socket in {{/run}} location
and share between containers due to company requirement.  However, he still doesn't want to
let hacker gain root access.

Solution 1:
System admin placed {{/var/*}} and {{/run/\*}} (except /run/docker.socket), carefully in read-write
white list.  None of the symlink is exposed.  Hacker can not get in.

Solution 2 (All symlinks are banned and explicit hardcoded locations):
(Current proposed patch)
System admin specifies:
  white-list-read-write: {{/var}}, {{/run/\*}} (except /run/docker.socket), {{/mnt/hdfs/user/\*}}
(exception yarn)
  black-list: {{/var/run}},{{/run/docker.socket}},{{/mnt/hdfs/user/yarn}}
Hacker attempt to mount a symlink location resulting in access denied from container startup,
or explicit hard coded location also result in ban.

Solution 3: (Ban symlink and replace black list location with empty directory):
(Jason proposed implementation)
System admin specifies:
  white-list-read-write: {{/var}},{{/run}},{{/mnt/hdfs/user}}
  black-list: {{/run/docker.socket}},{{/mnt/hdfs/user/yarn}}
Hacker attempt to mount a symlink location resulting in access denied from container startup,
or mount /run/docker.socket manually, but result in empty file.

All solutions requires system administrator to enforce ability to upload secure image to private
registry to prevent torjan horse in docker image.
 
I can see the appeal that without having to do a high upkeep of white-list-read-write directories
by the new proposal.  The third solution can throw people off, if they do not know about black-list
is hijacked to empty location.  However, the depth of directories will defeat second solution.
 If community favors the third solution, I can revise patch accordingly.

> Add support for a volume blacklist for docker containers
> --------------------------------------------------------
>
>                 Key: YARN-7197
>                 URL: https://issues.apache.org/jira/browse/YARN-7197
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: yarn
>            Reporter: Shane Kumpf
>            Assignee: Eric Yang
>         Attachments: YARN-7197.001.patch, YARN-7197.002.patch
>
>
> Docker supports bind mounting host directories into containers. Work is underway to allow
admins to configure a whilelist of volume mounts. While this is a much needed and useful feature,
it opens the door for misconfiguration that may lead to users being able to compromise or
crash the system. 
> One example would be allowing users to mount /run from a host running systemd, and then
running systemd in that container, rendering the host mostly unusable.
> This issue is to add support for a default blacklist. The default blacklist would be
where we put files and directories that if mounted into a container, are likely to have negative
consequences. Users are encouraged not to remove items from the default blacklist, but may
do so if necessary.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message