hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nathan Roberts (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3683) Capacity scheduler LeafQueues maximum capacity calculation issues
Date Thu, 19 Jan 2012 21:36:43 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13189395#comment-13189395
] 

Nathan Roberts commented on MAPREDUCE-3683:
-------------------------------------------

My feeling is that an undefined max-capacity should be the same as a max-capacity of 100%.
Then it would follow that a child's max-capacity would be the product of its ancestors' max-capacity
values. This would be very similar to how absolutecapacity is calculated and I think that
consistency would help reduce confusion. 
                
> Capacity scheduler LeafQueues maximum capacity calculation issues
> -----------------------------------------------------------------
>
>                 Key: MAPREDUCE-3683
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3683
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Thomas Graves
>            Assignee: Arun C Murthy
>            Priority: Blocker
>         Attachments: MAPREDUCE-3683.patch
>
>
> In the Capacity scheduler if you configure the queues to be hierarchical where you have
root -> parent queue -> leaf queue, the leaf queue doesn't take into account its parents
maximum capacity when calculate its own maximum capacity, instead it seems to use the parents
capacity.  Looking at the code its using the parents absoluteCapacity and I think it should
be using the parents absoluteMaximumCapacity.
> It also seems to only use the parents capacity in the leaf queues max capacity calculation
when the leaf queue has a max capacity configured. If the leaf queues maximum-capacity is
not configured, then it can use 100% of the cluster.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message