hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "萝卜丝炒饭" <1427357...@qq.com>
Subject Re: Unable to localize container, when using LinuxContainerExecutor
Date Tue, 07 Feb 2017 06:31:25 GMT
have you created the parent folder, such as  /data1/yarn

---Original---
From: "陈浩骏"<chen.haojun@immomo.com>
Date: 2017/2/7 00:43:47
To: "user@hadoop.apache.org"<user@hadoop.apache.org>;
Subject: Unable to localize container, when using LinuxContainerExecutor


 Hi, 
 
    I tried to run secure containers in cluster, and followed the instructions from http://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-common/SecureMode.html#LinuxContainerExecutor

    After that, I ran the mapreduce example pi and got the following exception:
    
 2017-02-06 17:46:24,018 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
main : command provided 0
 2017-02-06 17:46:24,018 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
main : run as user is hadoop
 2017-02-06 17:46:24,018 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
main : requested yarn user is hadoop
 2017-02-06 17:46:24,019 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
Failed to create directory /data1/yarn/nm,/data2/yarn/nm,/data3/yarn/nm/usercache/hadoop -
No such file or directory
 2017-02-06 17:46:24,019 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
Localizer failed
 java.io.IOException: Application application_1486374303689_0001 initialization failed (exitCode=20)
with output: main : command provided 0
 main : run as user is hadoop
 main : requested yarn user is hadoop
 Failed to create directory /data1/yarn/nm,/data2/yarn/nm,/data3/yarn/nm/usercache/hadoop
- No such file or directory
 
 
         at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.startLocalizer(LinuxContainerExecutor.java:241)
         at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1088)
 Caused by: ExitCodeException exitCode=20:
         at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
         at org.apache.hadoop.util.Shell.run(Shell.java:455)
         at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
         at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.startLocalizer(LinuxContainerExecutor.java:232)
         ... 1 more
 
 
 
 
   Here is the container-executor.cfg under /etc/hadoop owned by root (configurations are
same to the yarn-site.xml):
  yarn.nodemanager.local-dirs=/data1/yarn/nm,/data2/yarn/nm,/data3/yarn/nm
 yarn.nodemanager.log-dirs=/data1/hadoop/logs/container
 yarn.nodemanager.linux-container-executor.group=hadoop
 banned.users=yarn
 min.user.id=500
 allowed.system.users=hadoop,dm
 
 
 
     Hope to get some helps.
 
 
     Regards,
     Reid Chan.
Mime
View raw message