hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nathan Roberts (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3683) Capacity scheduler LeafQueues maximum capacity calculation issues
Date Mon, 23 Jan 2012 22:27:41 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191556#comment-13191556

Nathan Roberts commented on MAPREDUCE-3683:

Let me clarify that last comment. Is there a specific reason we need to distinguish between
a setting of 100% and UNDEFINED? I'm just wondering because if we're not benefiting from the
distinction, it's adding confusion:
* Log messages of the form: "absoluteMaxCapacity = 3.4028235E38 [= Float.MAX_VALUE if maximumCapacity
undefined, (parentAbsoluteCapacity * maximumCapacity) / 100 otherwise ]"
* webservices and webui both report 100% in the case where it's UNDEFINED
* Code like the following in a few places:

    float absoluteMaxCapacity = 
      (Math.round(maximumCapacity * 100) == 
        CapacitySchedulerConfiguration.UNDEFINED) ? 
       Float.MAX_VALUE : 
       (parent.getAbsoluteMaximumCapacity() * maximumCapacity);
> Capacity scheduler LeafQueues maximum capacity calculation issues
> -----------------------------------------------------------------
>                 Key: MAPREDUCE-3683
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3683
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Thomas Graves
>            Assignee: Arun C Murthy
>            Priority: Blocker
>         Attachments: MAPREDUCE-3683.patch
> In the Capacity scheduler if you configure the queues to be hierarchical where you have
root -> parent queue -> leaf queue, the leaf queue doesn't take into account its parents
maximum capacity when calculate its own maximum capacity, instead it seems to use the parents
capacity.  Looking at the code its using the parents absoluteCapacity and I think it should
be using the parents absoluteMaximumCapacity.
> It also seems to only use the parents capacity in the leaf queues max capacity calculation
when the leaf queue has a max capacity configured. If the leaf queues maximum-capacity is
not configured, then it can use 100% of the cluster.  

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message