hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 陈浩骏 <chen.hao...@immomo.com>
Subject Unable to localize container, when using LinuxContainerExecutor
Date Mon, 06 Feb 2017 10:33:38 GMT
Hi,

   I tried to run secure containers in cluster, and followed the instructions from http://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-common/SecureMode.html#LinuxContainerExecutor
   After that, I ran the mapreduce example pi and got the following exception:

2017-02-06 17:46:24,018 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
main : command provided 0
2017-02-06 17:46:24,018 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
main : run as user is hadoop
2017-02-06 17:46:24,018 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
main : requested yarn user is hadoop
2017-02-06 17:46:24,019 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
Failed to create directory /data1/yarn/nm,/data2/yarn/nm,/data3/yarn/nm/usercache/hadoop -
No such file or directory
2017-02-06 17:46:24,019 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
Localizer failed
java.io.IOException: Application application_1486374303689_0001 initialization failed (exitCode=20)
with output: main : command provided 0
main : run as user is hadoop
main : requested yarn user is hadoop
Failed to create directory /data1/yarn/nm,/data2/yarn/nm,/data3/yarn/nm/usercache/hadoop -
No such file or directory

        at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.startLocalizer(LinuxContainerExecutor.java:241)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1088)
Caused by: ExitCodeException exitCode=20:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
        at org.apache.hadoop.util.Shell.run(Shell.java:455)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
        at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.startLocalizer(LinuxContainerExecutor.java:232)
        ... 1 more


  Here is the container-executor.cfg under /etc/hadoop owned by root (configurations are same
to the yarn-site.xml):
yarn.nodemanager.local-dirs=/data1/yarn/nm,/data2/yarn/nm,/data3/yarn/nm
yarn.nodemanager.log-dirs=/data1/hadoop/logs/container
yarn.nodemanager.linux-container-executor.group=hadoop
banned.users=yarn
min.user.id=500
allowed.system.users=hadoop,dm

    Hope to get some helps.

    Regards,
    Reid Chan.
Mime
View raw message