hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Djellel Eddine Difallah (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-897) CapacityScheduler wrongly sorted queues
Date Fri, 19 Jul 2013 01:42:49 GMT

    [ https://issues.apache.org/jira/browse/YARN-897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13713241#comment-13713241
] 

Djellel Eddine Difallah commented on YARN-897:
----------------------------------------------

That is what I tried before, but because the parentQueue doesn't know which queue made the
call, we don't know what to reinsert. Basically all we can do is to go down to tree to figure
this out. Alternatively, we can modify the signature of completedContainer to include the
Queue...
That's why we figured that it's more efficient to have the leaf queue explicitly trigger the
call.
                
> CapacityScheduler wrongly sorted queues
> ---------------------------------------
>
>                 Key: YARN-897
>                 URL: https://issues.apache.org/jira/browse/YARN-897
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacityscheduler
>    Affects Versions: 2.0.4-alpha
>            Reporter: Djellel Eddine Difallah
>            Priority: Blocker
>         Attachments: TestBugParentQueue.java, YARN-897-1.patch, YARN-897-2.patch
>
>
> The childQueues of a ParentQueue are stored in a TreeSet where UsedCapacity defines the
sort order. This ensures the queue with least UsedCapacity to receive resources next. On containerAssignment
we correctly update the order, but we miss to do so on container completions. This corrupts
the TreeSet structure, and under-capacity queues might starve for resources.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message