hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashwin Shankar (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-2214) preemptContainerPreCheck() in FSParentQueue delays convergence towards fairness
Date Wed, 25 Jun 2014 21:29:25 GMT
Ashwin Shankar created YARN-2214:
------------------------------------

             Summary: preemptContainerPreCheck() in FSParentQueue delays convergence towards
fairness
                 Key: YARN-2214
                 URL: https://issues.apache.org/jira/browse/YARN-2214
             Project: Hadoop YARN
          Issue Type: Bug
          Components: scheduler
    Affects Versions: 2.5.0
            Reporter: Ashwin Shankar


preemptContainerPreCheck() in FSParentQueue rejects preemption requests if the parent queue
is below fair share. This can cause a delay in converging towards fairness when the starved
leaf queue and the queue above fairshare belong under a non-root parent queue(ie their least
common ancestor is a parent queue which is not root).
Here is an example :
root.parent has fair share = 80% and usage = 80%
root.parent.child1 has fair share =40% usage = 80%
root.parent.child2 has fair share=40% usage=0%

Now a job is submitted to child2 and the demand is 40%.
Preemption will kick in and try to reclaim all the 40% from child1.
When it preempts the first container from child1,the usage of root.parent will become <80%,
which is less than root.parent's fair share,causing preemption to stop.So only one container
gets preempted in this round although the need is a lot more. child2 would eventually get
to half its fair share but only after multiple rounds of preemption.

Solution is to remove preemptContainerPreCheck() in FSParentQueue and keep it only in FSLeafQueue(which
is already there).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message