hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Yang (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-7197) Add support for a volume blacklist for docker containers
Date Mon, 30 Oct 2017 18:20:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16225452#comment-16225452
] 

Eric Yang edited comment on YARN-7197 at 10/30/17 6:19 PM:
-----------------------------------------------------------

[~jlowe] 

{quote}Either /run isn't in the whitelist in the first place rendering the blacklist entry
moot or /run is in the whitelist and the user can simply mount {{/run}} and access the blacklist
path.{quote}

Let's expand on the real world example.  A hacker tries to take control of {{/run/docker.socket}}
to acquire root privileges and spawn root containers to access vital system area to become
root on the host system.  The system admin placed {{/var}} in read-write white list for ability
to write to database and log directories, without black list capability.  Hacker explicitly
specify {{/var/run/docker.socket}} to be included, put the socket in {{/tmp/docker.socket}}.
 Hacker generates a docker image with {{/etc/group}} modified to include his own name or setuid
bit binary in container.  Hack can successfully gain control to host level docker without
much effort.

{{/run}} contains a growing list of software that put their pid file or socket in this location.
 System admin can't say no to not allow other software (i.e. hdfs short circuit read) to place
their socket in {{/run}} location and share between containers due to company requirement.
 However, he still doesn't want to let hacker gain root access.

h3. Solution 1:
System admin placed {{/var/*}}, {{/run/\*}} (except /run/docker.socket), and {{/mnt/hdfs/user/\*}}
(except yarn), carefully in read-write white list.  None of the symlink is exposed.  Hacker
can not get in.

h3. Solution 2 (All symlinks, and hardcoded locations are banned):
(Current proposed patch)
System admin specifies:
  white-list-read-write: {{/var}}, {{/run/\*}} (except /run/docker.socket), {{/mnt/hdfs/user/\*}}
(exception yarn)
  black-list: {{/var/run}},{{/run/docker.socket}},{{/mnt/hdfs/user/yarn}}
Hacker attempt to mount a symlink location resulting in access denied from container startup,
or explicit hard coded location also result in ban.

h3. Solution 3: (Replace black list location with empty directories):
(Jason proposed implementation)
System admin specifies:
  white-list-read-write: {{/var}},{{/run}},{{/mnt/hdfs/user}}
  black-list: {{/run/docker.socket}},{{/mnt/hdfs/user/yarn}}
Hacker attempt to mount a symlink location resulting in access denied from container startup,
or mount /run/docker.socket manually, but result in empty file.

All solutions requires system administrator to enforce ability to upload secure image to private
registry to prevent torjan horse in docker image.
 
I can see the appeal that without having to do a high upkeep of white-list-read-write directories
by the new proposal.  The third solution can throw people off, if they do not know about black-list
is hijacked to empty location.  However, the depth of directories might defeat second solution.
 If community favors the third solution, I can revise patch accordingly.


was (Author: eyang):
[~jlowe] 

{quote}Either /run isn't in the whitelist in the first place rendering the blacklist entry
moot or /run is in the whitelist and the user can simply mount {{/run}} and access the blacklist
path.{quote}

Let's expand on the real world example.  A hacker tries to take control of {{/run/docker.socket}}
to acquire root privileges and spawn root containers to access vital system area to become
root on the host system.  The system admin placed {{/var}} in read-write white list for ability
to write to database and log directories, without black list capability.  Hacker explicitly
specify {{/var/run/docker.socket}} to be included, put the socket in {{/tmp/docker.socket}}.
 Hacker generates a docker image with {{/etc/group}} modified to include his own name or setuid
bit binary in container.  Hack can successfully gain control to host level docker without
much effort.

{{/run}} contains a growing list of software that put their pid file or socket in this location.
 System admin can't say no to not allow other software (i.e. hdfs short circuit read) to place
their socket in {{/run}} location and share between containers due to company requirement.
 However, he still doesn't want to let hacker gain root access.

h3. Solution 1:
System admin placed {{/var/*}}, {{/run/\*}} (except /run/docker.socket), and {{/mnt/hdfs/user/*}}
(except yarn), carefully in read-write white list.  None of the symlink is exposed.  Hacker
can not get in.

h3. Solution 2 (All symlinks, and hardcoded locations are banned):
(Current proposed patch)
System admin specifies:
  white-list-read-write: {{/var}}, {{/run/\*}} (except /run/docker.socket), {{/mnt/hdfs/user/\*}}
(exception yarn)
  black-list: {{/var/run}},{{/run/docker.socket}},{{/mnt/hdfs/user/yarn}}
Hacker attempt to mount a symlink location resulting in access denied from container startup,
or explicit hard coded location also result in ban.

h3. Solution 3: (Replace black list location with empty directories):
(Jason proposed implementation)
System admin specifies:
  white-list-read-write: {{/var}},{{/run}},{{/mnt/hdfs/user}}
  black-list: {{/run/docker.socket}},{{/mnt/hdfs/user/yarn}}
Hacker attempt to mount a symlink location resulting in access denied from container startup,
or mount /run/docker.socket manually, but result in empty file.

All solutions requires system administrator to enforce ability to upload secure image to private
registry to prevent torjan horse in docker image.
 
I can see the appeal that without having to do a high upkeep of white-list-read-write directories
by the new proposal.  The third solution can throw people off, if they do not know about black-list
is hijacked to empty location.  However, the depth of directories might defeat second solution.
 If community favors the third solution, I can revise patch accordingly.

> Add support for a volume blacklist for docker containers
> --------------------------------------------------------
>
>                 Key: YARN-7197
>                 URL: https://issues.apache.org/jira/browse/YARN-7197
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: yarn
>            Reporter: Shane Kumpf
>            Assignee: Eric Yang
>         Attachments: YARN-7197.001.patch, YARN-7197.002.patch
>
>
> Docker supports bind mounting host directories into containers. Work is underway to allow
admins to configure a whilelist of volume mounts. While this is a much needed and useful feature,
it opens the door for misconfiguration that may lead to users being able to compromise or
crash the system. 
> One example would be allowing users to mount /run from a host running systemd, and then
running systemd in that container, rendering the host mostly unusable.
> This issue is to add support for a default blacklist. The default blacklist would be
where we put files and directories that if mounted into a container, are likely to have negative
consequences. Users are encouraged not to remove items from the default blacklist, but may
do so if necessary.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message