hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Masatake Iwasaki (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3386) Cgroups feature should work with default hierarchy settings of CentOS 7
Date Mon, 23 Mar 2015 04:30:10 GMT

    [ https://issues.apache.org/jira/browse/YARN-3386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375382#comment-14375382
] 

Masatake Iwasaki commented on YARN-3386:
----------------------------------------

The list below is the default settings in CentOS 7::
{noformat}
$ cat /proc/mounts | grep cgroup
tmpfs /sys/fs/cgroup tmpfs rw,nosuid,nodev,noexec,mode=755 0 0
cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd
0 0
cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0
cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpuacct,cpu 0 0
cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0
cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0
cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0
cgroup /sys/fs/cgroup/net_cls cgroup rw,nosuid,nodev,noexec,relatime,net_cls 0 0
cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0
cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0
cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0
{noformat}

{{CgroupsLCEResourcesHandler#parseMtab}} parses this and set the value of {{controllerPath}}
for cpu to {{/sys/fs/cgroup/cpu,cpuacct/hadoop-yarn}}.

As a result, container-executor tries to write the pid to {{/sys/fs/cgroup/cpu}} (which is
the part before commna in the path) and fails.
{noformat}
2015-03-23 21:32:01,186 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
Exit code: 27
2015-03-23 21:32:01,186 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
Stack trace: ExitCodeException exitCode=27:
2015-03-23 21:32:01,186 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
      at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
2015-03-23 21:32:01,186 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
      at org.apache.hadoop.util.Shell.run(Shell.java:455)
2015-03-23 21:32:01,186 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
      at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
2015-03-23 21:32:01,186 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
      at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:293)
2015-03-23 21:32:01,186 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
      at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
2015-03-23 21:32:01,186 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
      at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
2015-03-23 21:32:01,186 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
      at java.util.concurrent.FutureTask.run(FutureTask.java:262)
2015-03-23 21:32:01,186 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
2015-03-23 21:32:01,186 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
2015-03-23 21:32:01,186 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
      at java.lang.Thread.run(Thread.java:744)
2015-03-23 21:32:01,186 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
2015-03-23 21:32:01,187 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
Shell output: main : command provided 1
2015-03-23 21:32:01,187 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
main : user is nobody
2015-03-23 21:32:01,187 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
main : requested yarn user is iwasakims
2015-03-23 21:32:01,187 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
Can't open file /sys/fs/cgroup/cpu as node manager - Is a directory
2015-03-23 21:32:01,187 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch:
Container exited with a non-zero exit code 27
{noformat}


> Cgroups feature should work with default hierarchy settings of CentOS 7
> -----------------------------------------------------------------------
>
>                 Key: YARN-3386
>                 URL: https://issues.apache.org/jira/browse/YARN-3386
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Masatake Iwasaki
>            Assignee: Masatake Iwasaki
>
> The path found by CgroupsLCEResourcesHandler#parseMtab contains comma and results in
failure of container-executor.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message