hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sidharta Seethana (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3853) Add docker container runtime support to LinuxContainterExecutor
Date Mon, 20 Jul 2015 21:26:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634107#comment-14634107
] 

Sidharta Seethana commented on YARN-3853:
-----------------------------------------

[~vvasudev] , thanks for the review. Regarding the docker image env var being added to the
whitelist, that doesn't belong here (it is DockerContainerExecutor specific which overrides
the writeLaunchEnv function anyway).  About missing debug statements - these are no longer
necessary since PrivilegedOperationExecutor logs this information already (in debug mode).
I'll fix the rest of the things you mention and upload a new patch.

> Add docker container runtime support to LinuxContainterExecutor
> ---------------------------------------------------------------
>
>                 Key: YARN-3853
>                 URL: https://issues.apache.org/jira/browse/YARN-3853
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: yarn
>            Reporter: Sidharta Seethana
>            Assignee: Sidharta Seethana
>         Attachments: YARN-3853.001.patch
>
>
> Create a new DockerContainerRuntime that implements support for docker containers via
container-executor. LinuxContainerExecutor should default to current behavior when launching
containers but switch to docker when requested. 
> Overview
> ===
> The current mechanism of launching/signaling containers is moved to its own (default)
container runtime. In order to use docker container runtime a couple of environment variables
have to be set. This will have to be revisited when we have a first class client side API
to specify different container types and associated parameters. Using ‘pi’ as an example
and using a custom docker image, this is how you could use the docker container runtime (LinuxContainerExecutor
must be in use and the docker daemon needs to be running) :
> {code}
> export YARN_EXAMPLES_JAR=./share/hadoop/mapreduce/hadoop-mapreduce-examples-*.jar
> bin/yarn jar $YARN_EXAMPLES_JAR pi -Dmapreduce.map.env="YARN_CONTAINER_RUNTIME_TYPE=docker,YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=ashahab/hadoop-trunk"
-Dyarn.app.mapreduce.am.env="YARN_CONTAINER_RUNTIME_TYPE=docker,YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=ashahab/hadoop-trunk"
 -Dmapreduce.reduce.env="YARN_CONTAINER_RUNTIME_TYPE=docker,YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=ashahab/hadoop-trunk"
4 1000
> {code}
>  
> LinuxContainerExecutor can delegate to either runtime on a per container basis. If the
docker container type is selected, LinuxContainerExecutor delegates to the DockerContainerRuntime
which in turn uses docker support in the container-executor binary to launch/manage docker
containers ( see YARN-3852 ) . 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message