hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2214) preemptContainerPreCheck() in FSParentQueue delays convergence towards fairness
Date Fri, 25 Jul 2014 05:06:38 GMT

    [ https://issues.apache.org/jira/browse/YARN-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14074073#comment-14074073

Hadoop QA commented on YARN-2214:

{color:green}+1 overall{color}.  Here are the results of testing the latest attachment 
  against trunk revision .

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 1 new or modified
test files.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

    {color:green}+1 eclipse:eclipse{color}.  The patch built with eclipse:eclipse.

    {color:green}+1 findbugs{color}.  The patch does not introduce any new Findbugs (version
2.0.3) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:green}+1 core tests{color}.  The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

    {color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-YARN-Build/4426//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4426//console

This message is automatically generated.

> preemptContainerPreCheck() in FSParentQueue delays convergence towards fairness
> -------------------------------------------------------------------------------
>                 Key: YARN-2214
>                 URL: https://issues.apache.org/jira/browse/YARN-2214
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 2.5.0
>            Reporter: Ashwin Shankar
>            Assignee: Ashwin Shankar
>         Attachments: YARN-2214-v1.txt, YARN-2214-v2.txt
> preemptContainerPreCheck() in FSParentQueue rejects preemption requests if the parent
queue is below fair share. This can cause a delay in converging towards fairness when the
starved leaf queue and the queue above fairshare belong under a non-root parent queue(ie their
least common ancestor is a parent queue which is not root).
> Here is an example :
> root.parent has fair share = 80% and usage = 80%
> root.parent.child1 has fair share =40% usage = 80%
> root.parent.child2 has fair share=40% usage=0%
> Now a job is submitted to child2 and the demand is 40%.
> Preemption will kick in and try to reclaim all the 40% from child1.
> When it preempts the first container from child1,the usage of root.parent will become
<80%, which is less than root.parent's fair share,causing preemption to stop.So only one
container gets preempted in this round although the need is a lot more. child2 would eventually
get to half its fair share but only after multiple rounds of preemption.
> Solution is to remove preemptContainerPreCheck() in FSParentQueue and keep it only in
FSLeafQueue(which is already there).

This message was sent by Atlassian JIRA

View raw message