ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmitro Lisnichenko" <dlysniche...@hortonworks.com>
Subject Re: Review Request 31907: Nodemanager fails to start with error java.io.IOException: Linux container executor not configured properly (error=24)
Date Tue, 10 Mar 2015 18:52:04 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31907/#review75932
-----------------------------------------------------------

Ship it!


Ship It!

- Dmitro Lisnichenko


On March 10, 2015, 6:50 p.m., Andrew Onischuk wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31907/
> -----------------------------------------------------------
> 
> (Updated March 10, 2015, 6:50 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-10013
>     https://issues.apache.org/jira/browse/AMBARI-10013
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> In Ambari deployed clusters, Nodemanager fails to start with the following
> exception for the test case where we write container_executor.cfg
> 
> **nodemanager log**
>     
>     
>     
>     2015-03-05 20:16:19,431 INFO  nodemanager.NodeManager (SignalLogger.java:register(91))
- registered UNIX signal handlers for [TERM, HUP, INT]
>     2015-03-05 20:16:21,363 WARN  nodemanager.LinuxContainerExecutor (LinuxContainerExecutor.java:init(184))
- Exit code from container executor initialization is : 24
>     ExitCodeException exitCode=24: File /etc/hadoop/conf.empty must be owned by root,
but is owned by 1005
>     
>             at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
>             at org.apache.hadoop.util.Shell.run(Shell.java:455)
>             at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
>             at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.init(LinuxContainerExecutor.java:181)
>             at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:209)
>             at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>             at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:462)
>             at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:509)
>     2015-03-05 20:16:21,366 INFO  nodemanager.ContainerExecutor (ContainerExecutor.java:logOutput(283))
-
>     2015-03-05 20:16:21,367 INFO  service.AbstractService (AbstractService.java:noteFailure(272))
- Service NodeManager failed in state INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
Failed to initialize container executor
>     org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to initialize container
executor
>             at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:211)
>             at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>             at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:462)
>             at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:509)
>     Caused by: java.io.IOException: Linux container executor not configured properly
(error=24)
>             at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.init(LinuxContainerExecutor.java:187)
>             at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:209)
>             ... 3 more
>     Caused by: ExitCodeException exitCode=24: File /etc/hadoop/conf.empty must be owned
by root, but is owned by 1005
>     
>             at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
>             at org.apache.hadoop.util.Shell.run(Shell.java:455)
>             at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
>             at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.init(LinuxContainerExecutor.java:181)
>             ... 4 more
>     2015-03-05 20:16:21,374 WARN  service.AbstractService (ServiceOperations.java:stopQuietly(82))
- When stopping the service NodeManager : java.lang.NullPointerException
>     java.lang.NullPointerException
>             at org.apache.hadoop.yarn.server.nodemanager.NodeManager.stopRecoveryStore(NodeManager.java:161)
>             at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStop(NodeManager.java:273)
>             at org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
>             at org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
>             at org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
>             at org.apache.hadoop.service.AbstractService.init(AbstractService.java:171)
>             at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:462)
>             at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:509)
>     2015-03-05 20:16:21,375 FATAL nodemanager.NodeManager (NodeManager.java:initAndStartNodeManager(465))
- Error starting NodeManager
>     org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to initialize container
executor
>             at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:211)
>             at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>             at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:462)
>             at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:509)
>     Caused by: java.io.IOException: Linux container executor not configured properly
(error=24)
>             at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.init(LinuxContainerExecutor.java:187)
>             at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:209)
>             ... 3 more
>     Caused by: ExitCodeException exitCode=24: File /etc/hadoop/conf.empty must be owned
by root, but is owned by 1005
>     
>             at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
>             at org.apache.hadoop.util.Shell.run(Shell.java:455)
>             at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
>             at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.init(LinuxContainerExecutor.java:181)
>             ... 4 more
>     
> 
> **container_executor.cfg**
>     
>     
>     
>     cat /etc/hadoop/conf/container-executor.cfg
>     yarn.nodemanager.local-dirs=/grid/0/hadoop/yarn/local
>     yarn.nodemanager.log-dirs=/grid/0/hadoop/yarn/log
>     yarn.nodemanager.linux-container-executor.group=hadoop
>     banned.users=yarn
>     min.user.id=1000
>     allowed.system.users=nobody
>     
> 
> Please note that test code does not touch conf.empty folder
> 
> **conf.empty permissions**
>     
>     
>     
>     ls -l /etc/hadoop/
>     total 4
>     lrwxrwxrwx 1 root root     22 Mar  4 21:01 conf -> /etc/hadoop/conf.empty
>     drwxr-xr-x 2 hdfs hadoop 4096 Mar  5 19:10 conf.empty
>     
> 
> Attached are the conf directory, nodemanager logs, RM logs and console logs
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/shared_initialization.py
8d67320 
>   ambari-server/src/test/python/stacks/2.0.6/hooks/before-ANY/test_before_any.py 5a750d2

> 
> Diff: https://reviews.apache.org/r/31907/diff/
> 
> 
> Testing
> -------
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message