hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2413) capacity scheduler will overallocate vcores
Date Tue, 12 Aug 2014 23:20:11 GMT

    [ https://issues.apache.org/jira/browse/YARN-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14094873#comment-14094873
] 

Allen Wittenauer commented on YARN-2413:
----------------------------------------

I might have missed something, but there appears to be a very bad bug here.  Given the following
settings:

{code}
          "mapreduce.map.cpu.vcores": 10
          "yarn.scheduler.maximum-allocation-vcores": 220
          "mapreduce.reduce.cpu.vcores": 10
          "yarn.app.mapreduce.am.resource.cpu-vcores": 10
          "yarn.scheduler.minimum-allocation-vcores": 10
          "yarn.nodemanager.resource.cpu-vcores": 221
{code}

The resource manager is only allocating 1 vcore per container:

{code}
2014-08-12 15:49:31,809 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode:
Assigned container container_1407883573269_0001_01_000001 of capacity <memory:2048, vCores:1>
on host 10.248.3.50:8041, which has 1 containers, <memory:2048, vCores:1> used and <memory:2050,
vCores:220> available after allocation
{code}

... which is clearly wrong, as the number of vcores requested should have been 10 and the
available remaining should have been 211.

> capacity scheduler will overallocate vcores
> -------------------------------------------
>
>                 Key: YARN-2413
>                 URL: https://issues.apache.org/jira/browse/YARN-2413
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 3.0.0, 2.2.0
>            Reporter: Allen Wittenauer
>            Priority: Critical
>
> It doesn't appear that the capacity scheduler is properly allocation vcores when making
scheduling decisions, which results in overallocation of CPU resources.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message