hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Badger (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7506) Overhaul the design of the Linux container-executor regarding Docker and future runtimes
Date Thu, 16 Nov 2017 19:18:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16255814#comment-16255814
] 

Eric Badger commented on YARN-7506:
-----------------------------------

I suppose we could move the docker portion of container-executor into a standalone root java
process, but I'm not convinced that gives us a whole lot in terms of security. Moving the
docker portion out of container-executor doesn't get rid of the container-executor. So if
someone compromises yarn, they will still be able to wield the container-executor. I guess
what moving docker out of the container-executor would give us is that we would have a smaller
surface area to introduce bugs that would misuse executing system calls. But we weren't making
those system calls in the docker portion of the container-executor anyway. We would also have
a smaller C code surface area, which may be more secure since most Hadoop programmers are
more comfortable in Java as opposed to C. 

The idea of moving the docker portion of container-executor into a java process is attractive
because of the development aspect. However, that would be a pretty giant effort and would
require a complete rewrite of most of the docker implementation. So I'm not sure that the
effort would be worth it. 

> Overhaul the design of the Linux container-executor regarding Docker and future runtimes
> ----------------------------------------------------------------------------------------
>
>                 Key: YARN-7506
>                 URL: https://issues.apache.org/jira/browse/YARN-7506
>             Project: Hadoop YARN
>          Issue Type: Wish
>          Components: nodemanager
>            Reporter: Miklos Szegedi
>              Labels: Docker, container-executor
>
> I raise this topic to discuss a potential improvement of the container executor tool
in node manager.
> container-executor has two main purposes. It executes Linux *system calls not available
from Java*, and it executes tasks *available to root that are not available to the yarn user*.
Historically container-executor did both by doing impersonation. The yarn user is separated
from root because it runs network services, so *the yarn user should be restricted* by design.
Because of this it has it's own config file container-executor.cfg writable by root only that
specifies what actions are allowed for the yarn user. However, the requirements have changed
with Docker and that raises the following questions:
> 1. The Docker feature of YARN requires root permissions to *access the Docker socket*
but it does not run any system calls, so could the Docker related code in container-executor
be *refactored into a separate Java process ran as root*? Java would make the development
much faster and more secure. 
> 2. The Docker feature only needs the Docker unix socket. It is not a good idea to let
the yarn user directly access the socket, since that would elevate its privileges to root.
However, the Java tool running as root mentioned in the previous question could act as a *proxy
on the Docker socket* operating directly on the Docker REST API *eliminating the need to use
the Docker CLI*. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message