hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Miklos Szegedi (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-4599) Set OOM control for memory cgroups
Date Tue, 15 May 2018 22:10:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-4599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16476365#comment-16476365
] 

Miklos Szegedi edited comment on YARN-4599 at 5/15/18 10:09 PM:
----------------------------------------------------------------

Thanks for the review [~snemeth].
{quote} [Question] In constructor: If both {{controlPhysicalMemory}} and {{controlVirtualMemory}} is
on, you only warn log a line.
{quote}
{{this.controlPhysicalMemory = controlPhysicalMemory && !controlVirtualMemory;}} virtual
will override
{quote}G.) In run(): I'm curious whether it can happen that in the while loop's statement, {{events.read}} would
read more data than 8 bytes or is it perfectly safe to rely on that on every read, at most
8 bytes will be read?
{quote}
It cannot return more than the buffer size that is 8.

H) I like to have big logic together.

 
{quote}I.) In run(): {{throw new YarnRuntimeException("OOM was not resolved in time.");}}} -->
I would include how much time was spent in the log message for better troubleshooting.
{quote}
The watchdog function handles this.
{quote}B.) In run(): Call to...
{quote}
It does not matter, it wraps into two lines anyways.
{quote}C.) In {{killContainerIfOOM()}}: 
{quote}
Not sure what you mean here. Yes there is a conversion, since the request is in MB and the
limit is in bytes.
{quote}A.) {{testConstructorHandler():(}}
{quote}
Good catch.


was (Author: miklos.szegedi@cloudera.com):
Thanks for the review [~snemeth].
{quote}{quote} [Question] In constructor: If both {{controlPhysicalMemory}} and {{controlVirtualMemory}} is
on, you only warn log a line.
{quote}{quote}
{{this.controlPhysicalMemory = controlPhysicalMemory && !controlVirtualMemory;}} virtual
will override
{quote}G.) In run(): I'm curious whether it can happen that in the while loop's statement, {{events.read}} would
read more data than 8 bytes or is it perfectly safe to rely on that on every read, at most
8 bytes will be read?
{quote}
It cannot return more than the buffer size that is 8.

H) I like to have big logic together.

 
{quote}I.) In run(): {{throw new YarnRuntimeException("OOM was not resolved in time.");}}} -->
I would include how much time was spent in the log message for better troubleshooting.
{quote}
The watchdog function handles this.
{quote}B.) In run(): Call to...
{quote}
It does not matter, it wraps into two lines anyways.
{quote}C.) In {{killContainerIfOOM()}}: 
{quote}
Not sure what you mean here. Yes there is a conversion, since the request is in MB and the
limit is in bytes.
{quote}A.) {{testConstructorHandler():(}}
{quote}
Good catch.

> Set OOM control for memory cgroups
> ----------------------------------
>
>                 Key: YARN-4599
>                 URL: https://issues.apache.org/jira/browse/YARN-4599
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager
>    Affects Versions: 2.9.0
>            Reporter: Karthik Kambatla
>            Assignee: Miklos Szegedi
>            Priority: Major
>              Labels: oct16-medium
>         Attachments: Elastic Memory Control in YARN.pdf, YARN-4599.000.patch, YARN-4599.001.patch,
YARN-4599.002.patch, YARN-4599.003.patch, YARN-4599.004.patch, YARN-4599.005.patch, YARN-4599.006.patch,
YARN-4599.007.patch, YARN-4599.sandflee.patch, yarn-4599-not-so-useful.patch
>
>
> YARN-1856 adds memory cgroups enforcing support. We should also explicitly set OOM control
so that containers are not killed as soon as they go over their usage. Today, one could set
the swappiness to control this, but clusters with swap turned off exist.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message