hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hitesh Sharma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3611) Support Docker Containers In LinuxContainerExecutor
Date Thu, 01 Dec 2016 03:28:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-3611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15710732#comment-15710732
] 

Hitesh Sharma commented on YARN-3611:
-------------------------------------

Hi folks,

Docker is now available on Windows and is fully supported by Docker INC (I'm talking about
launching Windows containers via Docker).

https://www.docker.com/microsoft

Unfortunately in the current design Docker is being limited to Linux only. I think we need
to revisit this and have a way to share the same code across Docker support for Windows and
Linux. Another goal to keep in mind is to have DockerContainerExecutor be completely OS agnostic.
As in certain cases Docker client might actually be talking to a daemon on a remote machine
or a VM (which maybe Linux or Windows). Would love to hear some thoughts on how to achieve
Docker support for Windows by reusing all the good work being done here.

Thanks!









> Support Docker Containers In LinuxContainerExecutor
> ---------------------------------------------------
>
>                 Key: YARN-3611
>                 URL: https://issues.apache.org/jira/browse/YARN-3611
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: yarn
>            Reporter: Sidharta Seethana
>            Assignee: Sidharta Seethana
>
> Support Docker Containers In LinuxContainerExecutor
> LinuxContainerExecutor provides useful functionality today with respect to localization,
cgroups based resource management and isolation for CPU, network, disk etc. as well as security
with a well-defined mechanism to execute privileged operations using the container-executor
utility.  Bringing docker support to LinuxContainerExecutor lets us use all of this functionality
when running docker containers under YARN, while not requiring users and admins to configure
and use a different ContainerExecutor. 
> There are several aspects here that need to be worked through :
> * Mechanism(s) to let clients request docker-specific functionality - we could initially
implement this via environment variables without impacting the client API.
> * Security - both docker daemon as well as application
> * Docker image localization
> * Running a docker container via container-executor as a specified user
> * “Isolate” the docker container in terms of CPU/network/disk/etc
> * Communicating with and/or signaling the running container (ensure correct pid handling)
> * Figure out workarounds for certain performance-sensitive scenarios like HDFS short-circuit
reads 
> * All of these need to be achieved without changing the current behavior of LinuxContainerExecutor



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message