hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sidharta Seethana (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1856) cgroups based memory monitoring for containers
Date Mon, 07 Dec 2015 23:14:11 GMT

    [ https://issues.apache.org/jira/browse/YARN-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15045966#comment-15045966
] 

Sidharta Seethana commented on YARN-1856:
-----------------------------------------

Hi [~vvasudev], some comments below on the latest version of the patch :

{code}
String CGROUP_PARAM_HARD_LIMIT = "limit_in_bytes";
String CGROUP_PARAM_SOFT_LIMIT = "soft_limit_in_bytes";
String CGROUP_PARAM_SWAPPINESS = "swappiness";
{code}

The constants should have ‘MEMORY’ in their names. For example, CGROUP_PARAM_HARD_LIMIT
is better named as CGROUP_PARAM_MEMORY_HARD_LIMIT in order to avoid future collisions. This
is similar to how BLKIO is used in the previous line (classid should be fixed at some point
too)

{code}
@Private
public static final String NM_MEMORY_RESOURCE_CGROUPS_SOFT_LIMIT_PERC =
    NM_MEMORY_RESOURCE_PREFIX + "cgroups.soft-limit-percentage";
@Private
public static final float DEFAULT_NM_MEMORY_RESOURCE_CGROUPS_SOFT_LIMIT_PERC =
    0.9f;
{code}

{code}
softLimitPerc = conf.getFloat(
    YarnConfiguration.NM_MEMORY_RESOURCE_CGROUPS_SOFT_LIMIT_PERC,
    YarnConfiguration.DEFAULT_NM_MEMORY_RESOURCE_CGROUPS_SOFT_LIMIT_PERC);
if (softLimitPerc < 0.0f || softLimitPerc > 100.0f) {
  throw new ResourceHandlerException(
      "Illegal value '" + softLimitPerc + "' "
          + YarnConfiguration.NM_MEMORY_RESOURCE_CGROUPS_SOFT_LIMIT_PERC
          + ". Value must be between 0 and 100.");
}
{code}

Is the soft limit conf setting meant to represent a percentage or is it a fraction between
0 and 1? From the default value of 0.9f and the application of the soft limit it appears to
be a fraction, but the name of the setting and its validation check seem to indicate that
it is meant to be a percentage. This needs to be fixed.






> cgroups based memory monitoring for containers
> ----------------------------------------------
>
>                 Key: YARN-1856
>                 URL: https://issues.apache.org/jira/browse/YARN-1856
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager
>    Affects Versions: 2.3.0
>            Reporter: Karthik Kambatla
>            Assignee: Varun Vasudev
>         Attachments: YARN-1856.001.patch, YARN-1856.002.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message