hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4048) Linux kernel panic under strict CPU limits
Date Wed, 13 Jan 2016 17:50:40 GMT

    [ https://issues.apache.org/jira/browse/YARN-4048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15096669#comment-15096669

Naganarasimha G R commented on YARN-4048:

Hi [~geynard],
Sorry to hear you too faced the same problem, 
what we did is we have a configuration to optionally opt for cpuset based approach if we face
issues like this. And suppose its 16 core machine and we configured 75% of cpu can be used
for YARN, then we try to configure 12 (we try to round it off to lower value if the configurations
doesnt match) cores for YARN in the CPU  cgroup subsystem. So yarn's containers will be ensured
to run only on the first 12 cores of the system and remaining 4 will be at the system's disposal
for other processes. *This approach ensures CPU is isolated with other processes but not among
the yarn's containers.*

> Linux kernel panic under strict CPU limits
> ------------------------------------------
>                 Key: YARN-4048
>                 URL: https://issues.apache.org/jira/browse/YARN-4048
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 2.7.1
>            Reporter: Chengbing Liu
>            Priority: Critical
>         Attachments: panic.png
> With YARN-2440 and YARN-2531, we have seen some kernel panics happening under heavy pressure.
Even with YARN-2809, it still panics.
> We are using CentOS 6.5, hadoop 2.5.0-cdh5.2.0 with the above patches. I guess the latest
version also has the same issue.

This message was sent by Atlassian JIRA

View raw message