hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wei Yan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2154) FairScheduler: Improve preemption to preempt only those containers that would satisfy the incoming request
Date Fri, 05 Sep 2014 20:44:28 GMT

    [ https://issues.apache.org/jira/browse/YARN-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123541#comment-14123541
] 

Wei Yan commented on YARN-2154:
-------------------------------

bq. 1. For each queue, the child {{Schedulable}}s should be sorted in the ascending order
of (desiredShare - fairShare). In case of FSLeafQueue, this corresponds to running applications.
Do we still need to do the sorting? As we traverse from the root queue, and take applications
from the queues which are starvated for min/fair share. I think we collect all applications
from starvated queues.

bq. 4. Once the application list is computed, we traverse through the ResourceRequests (ordered
first by application, second by priority) and preempt any matching containers currently assigned
to an application over its share.
To sort the applications, do we still need to follow the policy? For example, for two applications
belong to the same FSLeafQueue, they should be sorted according to the leafQueue's policy;
for two applications belong to two different leaf queues, may need to refer to parent queue's
policy. 
And, after preempting one container and assigning the resource to one application in the list,
we also need to check whether that application and its parent queue is still starvated.

> FairScheduler: Improve preemption to preempt only those containers that would satisfy
the incoming request
> ----------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-2154
>                 URL: https://issues.apache.org/jira/browse/YARN-2154
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: fairscheduler
>    Affects Versions: 2.4.0
>            Reporter: Karthik Kambatla
>            Assignee: Karthik Kambatla
>            Priority: Critical
>
> Today, FairScheduler uses a spray-gun approach to preemption. Instead, it should only
preempt resources that would satisfy the incoming request. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message