hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Leitao Guo (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-3282) DockerContainerExecutor should support environment variables setting
Date Sat, 28 Feb 2015 10:19:04 GMT

     [ https://issues.apache.org/jira/browse/YARN-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Leitao Guo updated YARN-3282:
-----------------------------
    Attachment: YARN-3282.01.patch

When using DockerContainerExecutor, mapreduce jobs can set docker environment variables via
"yarn.nodemanager.docker-container-executor.env" after the patch.

e.g. 
{code}
bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples.jar wordcount -Dyarn.app.mapreduce.am.env="yarn.nodemanager.docker-container-executor.image-name=sequenceiq/hadoop-docker:2.6.0"
-Dmapreduce.map.env="yarn.nodemanager.docker-container-executor.image-name=sequenceiq/hadoop-docker:2.6.0"
 -Dmapreduce.reduce.env="yarn.nodemanager.docker-container-executor.image-name=sequenceiq/hadoop-docker:2.6.0"
  -Dyarn.nodemanager.docker-container-executor.env="-v /data/transcode:/data/tmp -v /etc/qcs:/etc/qcs
-v /mnt:/mnt -e VTC_MQTYPE=rabbitmq -e VTC_APP=ugc -e VTC_LOCATION=sh -e VTC_RUNTIME=vtc"
/wordcount_input /wordcount_output
{code}

> DockerContainerExecutor should support environment variables setting
> --------------------------------------------------------------------
>
>                 Key: YARN-3282
>                 URL: https://issues.apache.org/jira/browse/YARN-3282
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: applications, nodemanager
>    Affects Versions: 2.6.0
>            Reporter: Leitao Guo
>         Attachments: YARN-3282.01.patch
>
>
> Currently, DockerContainerExecutor will mount "yarn.nodemanager.local-dirs" and "yarn.nodemanager.log-dirs"
to containers automatically. However applications maybe need set more environment variables
before launching containers. 
> In our applications, just as the following command, we need to attach several directories
and set some environment variables to docker containers. 
> {code}
> docker run -i -t -v /data/transcode:/data/tmp -v /etc/qcs:/etc/qcs -v /mnt:/mnt -e VTC_MQTYPE=rabbitmq
-e VTC_APP=ugc -e VTC_LOCATION=sh -e VTC_RUNTIME=vtc sequenceiq/hadoop-docker:2.6.0 /bin/bash
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message