hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Vasudev (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-2440) Cgroups should allow YARN containers to be limited to allocated cores
Date Tue, 09 Sep 2014 19:18:29 GMT

     [ https://issues.apache.org/jira/browse/YARN-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Varun Vasudev updated YARN-2440:
    Attachment: apache-yarn-2440.5.patch

Uploaded new patch to address Vinod's concerns.

bq.    containers-limit-cpu-percentage -> yarn.nodemanager.resource.percentage-cpu-limit
to be consistent? Similarly NM_CONTAINERS_CPU_PERC? I don't like the tag 'resource', it should
have been 'resources' but it is what it is.

I'm worried that calling it that will lead users to think it's a percentage of the vcores
that they specify. In the patch I've changed it to yarn.nodemanager.resource.percentage-physical-cpu-limit
but if you or Jason feel strongly about it, I can change it to yarn.nodemanager.resource.percentage-cpu-limit.

bq.    You still have refs to YarnConfiguration.NM_CONTAINERS_CPU_ABSOLUTE in the patch. Similarly
the javadoc in NodeManagerHardwareUtils needs to be updated if we are not adding the absolute
cpu config. It should no longer refer to "number of cores that should be used for YARN containers"


bq.    TestCgroupsLCEResourcesHandler: You can use mockito if you only want to override num-processors
in TestResourceCalculatorPlugin. Similarly in TestNodeManagerHardwareUtils.

Switched to mockito.

bq.    The tests may fail on a machine with > 4 cores?
Don't think so. The tests mock the getNumProcessors() function so we should be fine.

> Cgroups should allow YARN containers to be limited to allocated cores
> ---------------------------------------------------------------------
>                 Key: YARN-2440
>                 URL: https://issues.apache.org/jira/browse/YARN-2440
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Varun Vasudev
>            Assignee: Varun Vasudev
>         Attachments: apache-yarn-2440.0.patch, apache-yarn-2440.1.patch, apache-yarn-2440.2.patch,
apache-yarn-2440.3.patch, apache-yarn-2440.4.patch, apache-yarn-2440.5.patch, screenshot-current-implementation.jpg
> The current cgroups implementation does not limit YARN containers to the cores allocated
in yarn-site.xml.

This message was sent by Atlassian JIRA

View raw message