Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Tue, 9 Sep 2014 19:18:29 +0000 (UTC)
From: "Varun Vasudev (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12736026.1408699954000.30217.1410290309644@Atlassian.JIRA>
In-Reply-To: <JIRA.12736026.1408699954000@Atlassian.JIRA>
References: <JIRA.12736026.1408699954000@Atlassian.JIRA>
 <JIRA.12736026.1408699954334@arcas>
Subject: [jira] [Updated] (YARN-2440) Cgroups should allow YARN containers
 to be limited to allocated cores
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/YARN-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Varun Vasudev updated YARN-2440:
--------------------------------
    Attachment: apache-yarn-2440.5.patch

Uploaded new patch to address Vinod's concerns.

bq.    containers-limit-cpu-percentage -> yarn.nodemanager.resource.percentage-cpu-limit to be consistent? Similarly NM_CONTAINERS_CPU_PERC? I don't like the tag 'resource', it should have been 'resources' but it is what it is.

I'm worried that calling it that will lead users to think it's a percentage of the vcores that they specify. In the patch I've changed it to yarn.nodemanager.resource.percentage-physical-cpu-limit but if you or Jason feel strongly about it, I can change it to yarn.nodemanager.resource.percentage-cpu-limit.

bq.    You still have refs to YarnConfiguration.NM_CONTAINERS_CPU_ABSOLUTE in the patch. Similarly the javadoc in NodeManagerHardwareUtils needs to be updated if we are not adding the absolute cpu config. It should no longer refer to "number of cores that should be used for YARN containers"

Fixed.

bq.    TestCgroupsLCEResourcesHandler: You can use mockito if you only want to override num-processors in TestResourceCalculatorPlugin. Similarly in TestNodeManagerHardwareUtils.

Switched to mockito.

bq.    The tests may fail on a machine with > 4 cores?
Don't think so. The tests mock the getNumProcessors() function so we should be fine.


> Cgroups should allow YARN containers to be limited to allocated cores
> ---------------------------------------------------------------------
>
>                 Key: YARN-2440
>                 URL: https://issues.apache.org/jira/browse/YARN-2440
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Varun Vasudev
>            Assignee: Varun Vasudev
>         Attachments: apache-yarn-2440.0.patch, apache-yarn-2440.1.patch, apache-yarn-2440.2.patch, apache-yarn-2440.3.patch, apache-yarn-2440.4.patch, apache-yarn-2440.5.patch, screenshot-current-implementation.jpg
>
>
> The current cgroups implementation does not limit YARN containers to the cores allocated in yarn-site.xml.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)