hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sidharta Seethana (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-3611) Support Docker Containers In LinuxContainerExecutor
Date Sat, 09 May 2015 00:26:59 GMT
Sidharta Seethana created YARN-3611:

             Summary: Support Docker Containers In LinuxContainerExecutor
                 Key: YARN-3611
                 URL: https://issues.apache.org/jira/browse/YARN-3611
             Project: Hadoop YARN
          Issue Type: Bug
          Components: yarn
            Reporter: Sidharta Seethana
            Assignee: Sidharta Seethana

Support Docker Containers In LinuxContainerExecutor

LinuxContainerExecutor provides useful functionality today with respect to localization, cgroups
based resource management and isolation for CPU, network, disk etc. as well as security with
a well-defined mechanism to execute privileged operations using the container-executor utility.
 Bringing docker support to LinuxContainerExecutor lets us use all of this functionality when
running docker containers under YARN, while not requiring users and admins to configure and
use a different ContainerExecutor. 

There are several aspects here that need to be worked through :

* Mechanism(s) to let clients request docker-specific functionality - we could initially implement
this via environment variables without impacting the client API.
* Security - both docker daemon as well as application
* Docker image localization
* Running a docker container via container-executor as a specified user
* “Isolate” the docker container in terms of CPU/network/disk/etc
* Communicating with and/or signaling the running container (ensure correct pid handling)
* Figure out workarounds for certain performance-sensitive scenarios like HDFS short-circuit
* All of these need to be achieved without changing the current behavior of LinuxContainerExecutor

This message was sent by Atlassian JIRA

View raw message