hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Attila Kanto (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1964) Create Docker analog of the LinuxContainerExecutor in YARN
Date Sat, 13 Dec 2014 15:20:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-1964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14245368#comment-14245368
] 

Attila Kanto commented on YARN-1964:
------------------------------------

Hi [~ashahab],

I have tested it with the configuration described in: http://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/DockerContainerExecutor.html
and it works with sequenceiq/hadoop-docker:2.6.0 as well.

Same teragen test was executed what is described in documentation: 
~ $ hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar teragen -Dmapreduce.map.env="yarn.nodemanager.docker-container-executor.image-name=sequenceiq/hadoop-docker:2.6.0"
-Dyarn.app.mapreduce.am.env="yarn.nodemanager.docker-container-executor.image-name=sequenceiq/hadoop-docker:2.6.0"
1 /tmp/out

The execution was successful and also the containers are launched correctly by DCE:
~ $ docker ps -a
CONTAINER ID        IMAGE                            COMMAND                CREATED      
      STATUS              PORTS               NAMES
20ce5a4b1654        sequenceiq/hadoop-docker:2.6.0   "bash /tmp/hadoop-ro   3 seconds ago
      Up 2 seconds                            container_1418483101027_0003_01_000003
82ec1e3eec73        sequenceiq/hadoop-docker:2.6.0   "bash /tmp/hadoop-ro   3 seconds ago
      Up 3 seconds                            container_1418483101027_0003_01_000002
f7c436fd5086        sequenceiq/hadoop-docker:2.6.0   "bash /tmp/hadoop-ro   10 seconds ago
     Up 9 seconds                            container_1418483101027_0003_01_000001
b8516f47cb86        my_dce:latest                    "/etc/bootstrap.sh -   3 minutes ago
      Up 3 minutes                            jolly_bell

Attila

> Create Docker analog of the LinuxContainerExecutor in YARN
> ----------------------------------------------------------
>
>                 Key: YARN-1964
>                 URL: https://issues.apache.org/jira/browse/YARN-1964
>             Project: Hadoop YARN
>          Issue Type: New Feature
>    Affects Versions: 2.2.0
>            Reporter: Arun C Murthy
>            Assignee: Abin Shahab
>             Fix For: 2.6.0
>
>         Attachments: YARN-1964.patch, YARN-1964.patch, YARN-1964.patch, YARN-1964.patch,
YARN-1964.patch, YARN-1964.patch, YARN-1964.patch, YARN-1964.patch, YARN-1964.patch, YARN-1964.patch,
YARN-1964.patch, yarn-1964-branch-2.2.0-docker.patch, yarn-1964-branch-2.2.0-docker.patch,
yarn-1964-docker.patch, yarn-1964-docker.patch, yarn-1964-docker.patch, yarn-1964-docker.patch,
yarn-1964-docker.patch
>
>
> Docker (https://www.docker.io/) is, increasingly, a very popular container technology.
> In context of YARN, the support for Docker will provide a very elegant solution to allow
applications to *package* their software into a Docker container (entire Linux file system
incl. custom versions of perl, python etc.) and use it as a blueprint to launch all their
YARN containers with requisite software environment. This provides both consistency (all YARN
containers will have the same software environment) and isolation (no interference with whatever
is installed on the physical machine).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message