hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2440) Cgroups should allow YARN containers to be limited to allocated cores
Date Mon, 08 Sep 2014 21:30:29 GMT

    [ https://issues.apache.org/jira/browse/YARN-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126140#comment-14126140
] 

Vinod Kumar Vavilapalli commented on YARN-2440:
-----------------------------------------------

Just caught up with the discussion. I can get behind an absolute limit too. Specifically in
the context of heterogeneous clusters where uniform % configurations can go really bad where
the only resort will then be to do per-node configuration - not ideal. Would that be a valid
use-case for putting in the absolute limit? [~jlowe]? Even if it were, I am okay punting that
off to a separate JIRA.

Comments on the patch:
 - containers-limit-cpu-percentage -> {{yarn.nodemanager.resource.percentage-cpu-limit}}
to be consistent? Similarly NM_CONTAINERS_CPU_PERC? I don't like the tag  'resource', it should
have been 'resources' but it is what it is.
 - You still have refs to YarnConfiguration.NM_CONTAINERS_CPU_ABSOLUTE in the patch. Similarly
the javadoc in NodeManagerHardwareUtils needs to be updated if we are not adding the absolute
cpu config. It should no longer refer to "number of cores that should be used for YARN containers"
 - TestCgroupsLCEResourcesHandler: You can use mockito if you only want to override num-processors
in TestResourceCalculatorPlugin. Similarly in TestNodeManagerHardwareUtils.
 - The tests may fail on a machine with > 4 cores? :)

> Cgroups should allow YARN containers to be limited to allocated cores
> ---------------------------------------------------------------------
>
>                 Key: YARN-2440
>                 URL: https://issues.apache.org/jira/browse/YARN-2440
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Varun Vasudev
>            Assignee: Varun Vasudev
>         Attachments: apache-yarn-2440.0.patch, apache-yarn-2440.1.patch, apache-yarn-2440.2.patch,
apache-yarn-2440.3.patch, apache-yarn-2440.4.patch, screenshot-current-implementation.jpg
>
>
> The current cgroups implementation does not limit YARN containers to the cores allocated
in yarn-site.xml.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message