hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karam Singh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4565) When sizeBasedWeight enabled for FairOrderingPolicy in CapacityScheduler, Sometimes lead to situation where all queue resources consumed by AMs only
Date Fri, 08 Jan 2016 16:07:39 GMT

    [ https://issues.apache.org/jira/browse/YARN-4565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15089424#comment-15089424
] 

Karam Singh commented on YARN-4565:
-----------------------------------

>From offline chat with [~wangda]
I think I found what happened:
When application added to scheduler, CapacityScheduler#allocate will be called:
    if (updateDemandForQueue != null) {
      updateDemandForQueue.getOrderingPolicy().demandUpdated(application);
    }
And in FairOrderingPolicy#demandUpdated, when sizeBasedWeight is enabled:
    if (sizeBasedWeight) {
      entityRequiresReordering(schedulableEntity);
    }
It will call reordering the schedulableEntity. It will essentially insert the entity to the
TreeSet.
This could happen before the application (schedulableEntity) is not in active application
list. So we can get application's container allocated before application activated.


> When sizeBasedWeight enabled for FairOrderingPolicy in CapacityScheduler, Sometimes lead
to situation where all queue resources consumed by AMs only
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-4565
>                 URL: https://issues.apache.org/jira/browse/YARN-4565
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacity scheduler, capacityscheduler
>    Affects Versions: 2.8.0, 2.7.1
>            Reporter: Karam Singh
>
> When sizeBasedWeight enabled for FairOrderingPolicy in CapacityScheduler, Sometimes lead
to situation where all queue resources consumed by AMs only,
> So from users perpective it appears that all application in queue are stuck, whole queue
capacity is comsumed by AMs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message