hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chun Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1964) Create Docker analog of the LinuxContainerExecutor in YARN
Date Fri, 07 Nov 2014 03:34:41 GMT

    [ https://issues.apache.org/jira/browse/YARN-1964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201527#comment-14201527
] 

Chun Chen commented on YARN-1964:
---------------------------------

Hi, [~ashahab], thanks for the patch. I’m also working on this and we are running dockers
on Yarn now. Based on our experience, here are my comments:
1. Since yarn.nodemanager.docker-container-executor.image-name is a application specified
container launch environment argument which would be exported in launch_container.sh and bash
doesn't allow dot separated environment variables, so you might need to change it to ApplicationConstants.Environment.DOCKER_IMAGE_NAME.
2. Remove “--net=host” from docker run command cause cluster administrator might not want
docker containers to use host network directly. 
3. Define a new environment maybe ApplicationConstants.Environment.DOCKER_RUN_ARGS to allow
application to specify customer options of docker run command such as -P, -e, etc. And in
this way, we can also specify "—net=host” from application side.
4. Remove localDirMount from docker run command, there is no need to mount local dir.
5. Use ApplicationConstants.Environment.HADOOP_YARN_HOME  instead of “HADOOP_YARN_HOME"
{code}
    exclusionSet.add("HADOOP_YARN_HOME");
    exclusionSet.add("HADOOP_COMMON_HOME");
    exclusionSet.add("HADOOP_HDFS_HOME");
    exclusionSet.add("HADOOP_COMMON_HOME");
    exclusionSet.add("JAVA_HOME");
{code}

> Create Docker analog of the LinuxContainerExecutor in YARN
> ----------------------------------------------------------
>
>                 Key: YARN-1964
>                 URL: https://issues.apache.org/jira/browse/YARN-1964
>             Project: Hadoop YARN
>          Issue Type: New Feature
>    Affects Versions: 2.2.0
>            Reporter: Arun C Murthy
>            Assignee: Abin Shahab
>         Attachments: YARN-1964.patch, YARN-1964.patch, YARN-1964.patch, YARN-1964.patch,
YARN-1964.patch, YARN-1964.patch, YARN-1964.patch, yarn-1964-branch-2.2.0-docker.patch, yarn-1964-branch-2.2.0-docker.patch,
yarn-1964-docker.patch, yarn-1964-docker.patch, yarn-1964-docker.patch, yarn-1964-docker.patch,
yarn-1964-docker.patch
>
>
> Docker (https://www.docker.io/) is, increasingly, a very popular container technology.
> In context of YARN, the support for Docker will provide a very elegant solution to allow
applications to *package* their software into a Docker container (entire Linux file system
incl. custom versions of perl, python etc.) and use it as a blueprint to launch all their
YARN containers with requisite software environment. This provides both consistency (all YARN
containers will have the same software environment) and isolation (no interference with whatever
is installed on the physical machine).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message