hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3730) scheduler reserve more resource than required
Date Sat, 30 May 2015 18:18:18 GMT

    [ https://issues.apache.org/jira/browse/YARN-3730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14566130#comment-14566130

Naganarasimha G R commented on YARN-3730:

hi [~gu chi], 
Which version did you find this problem ? If its below 2.6.0, please test with the latest
as there have been some improvements wrt to reservation in YARN-1769. If its with 2.6.0 and
above version, share some RM logs with debug enabled, so that we can do further analysis.

> scheduler reserve more resource than required
> ---------------------------------------------
>                 Key: YARN-3730
>                 URL: https://issues.apache.org/jira/browse/YARN-3730
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: scheduler
>            Reporter: gu-chi
> Using capacity scheduler, environment is 3 NM and each has 9 vcores, I ran a spark task
with 4 executors and each executor 5 cores, as suspected, only 1 executor not able to start
and will be reserved, but actually more containers are reserved. This way, I can not run some
other smaller tasks. As I checked the capacity scheduler, the 'needContainers' method in LeafQueue.java
has a computation of 'starvation', this cause the scenario of more container reserved than
required, any idea or suggestion on this?

This message was sent by Atlassian JIRA

View raw message