hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sandflee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4599) Set OOM control for memory cgroups
Date Wed, 25 May 2016 07:27:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-4599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15299606#comment-15299606

sandflee commented on YARN-4599:

Thanks [~kasha],
bq. Any metrics on how often the NM has to intervene?
it depends on the business of NM and how user set container memory.  In our cluster with about
5000 NM, average of the NM resource allocation rate is 90%, the actually mem usage rate is
60%. most user request memory more than container really use.  without this improvement, several
thousands container oom kill event happens,  with the fix, not more than ten oom event happens.

bq. how long does it take for the NM to unblock containers once everything is paused ?
we payed not much attention on this,  usually nm  handle oom event very quickly, the most
time maybe used by kernel to free pagecache/container mem, but we didn't stat this.

bq. Oh, and any interest in contributing to do this natively in YARN?
yes, I'd like to update our patch in one or two weeks.

> Set OOM control for memory cgroups
> ----------------------------------
>                 Key: YARN-4599
>                 URL: https://issues.apache.org/jira/browse/YARN-4599
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 2.9.0
>            Reporter: Karthik Kambatla
>            Assignee: Karthik Kambatla
>         Attachments: yarn-4599-not-so-useful.patch
> YARN-1856 adds memory cgroups enforcing support. We should also explicitly set OOM control
so that containers are not killed as soon as they go over their usage. Today, one could set
the swappiness to control this, but clusters with swap turned off exist.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message