hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jasson Chenwei <ynjassionc...@gmail.com>
Subject Re: Errors reported by contianer-executor on Hadoop-3.0.0-alpha2
Date Tue, 23 May 2017 23:44:47 GMT
hi,

Thanks, Daniel. This is helpful.  I still have some problems in setting up
Docker environment.

e.g.,

I found  NodeManager will write the launch script to:
yarn-temp/nm-local-dir/application_1495578395673_0001/app-id/container-id/aunch_container.sh

The path is got by this function in ContainerLaunch.java
protected String getContainerPrivateDir(String appIdStr,
      String containerIdStr) {
    return getAppPrivateDir(appIdStr) + Path.SEPARATOR + containerIdStr
        + Path.SEPARATOR;
  }


However, the DockerContaienrRunTime will have the workdir:
yarn-temp/nm-local-dir/usercache/cwei/appcache/app-id/container-id which
presents a miss-match.


As a result, when docker container is launched, it would not find launch
script.

I understand the docker support in DockerLinuxContainerRuntime is still
under development. Could anybody provide a stable version that I can use?



Thanks

Wei


On Tue, May 23, 2017 at 10:43 AM, Daniel Templeton <daniel@cloudera.com>
wrote:

> The docs are available here:
>
> https://aajisaka.github.io/hadoop-project/hadoop-yarn/hadoop
> -yarn-site/DockerContainers.html
>
> Be aware that Docker support has a major security vulnerability unless
> you're building from source, and you've update since May 18:
>
> https://twitter.com/_a__w_/status/857406483910107137
>
> Daniel
>
>
> On 5/23/17 9:35 AM, Jasson Chenwei wrote:
>
>> hi ,all
>>
>> I am planning to launch MapReduce applications with docker support on
>> Hadoop-3.0.0-alpha2.
>>
>> But, when I execute ./bin/container-executor which is used by Hadoop to
>> launch secured containers. I get:
>>
>> *Usage: container-executor --checksetup*
>> *       container-executor --mount-cgroups <hierarchy>
>> <controller=path>...*
>> *[DISABLED] container-executor --tc-modify-state <command-file>*
>> *[DISABLED] container-executor --tc-read-state <command-file>*
>> *[DISABLED] container-executor --tc-read-stats <command-file>*
>> *[DISABLED] container-executor --run-docker <command-file>*
>> *       container-executor <user> <yarn-user> <command> <command-args>*
>> *       where command and command-args: *
>> *            initialize container:   0 appid tokens nm-local-dirs
>> nm-log-dirs cmd app...*
>> *            launch container:       1 appid containerid workdir
>> container-script tokens pidfile nm-local-dirs nm-log-dirs resources *
>> *[DISABLED]  launch docker container:       4 appid containerid workdir
>> container-script tokens pidfile nm-local-dirs nm-log-dirs
>> docker-command-file resources *
>> *            signal container:       2 container-pid signal*
>> *            delete as user:         3 relative-path*
>> *            list as user:           5 relative-path*
>>
>>
>>
>> Clearly, the container-executor --run-docker is disabled by default. My
>> question is how to enable this option via building option/configuration
>> option ?
>>
>>
>> Appreciate anyone's help!
>>
>> Wei
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message