hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nathan Roberts (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3945) maxApplicationsPerUser is wrongly calculated
Date Tue, 21 Jul 2015 15:24:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-3945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14635246#comment-14635246

Nathan Roberts commented on YARN-3945:

My feeling is the documentation on minimum-user-limit-percent needs a rewrite. It makes it
sound like minimum-user-limit-percent caps the amount of resource to say 50% if there are
2 applications submitted to the queue. This isn't the case (afaik). My understanding is that
it tries to guarantee all active  applications this percentage of a queue's capacity (configured
or current, whichever is larger). Note: an active application is one that is currently requesting
resources, a running application that has all the resources it needs, is NOT active.  If one
application stops asking for additional resources, the other applications can certainly go
higher than the 50%. user-limit-factor is what determines the absolute maximum capacity a
user can consume within a queue. 

Basically, minimum-user-limit percent defines how fair the queue is. The lower the value,
the sooner the queue will try to spread resources evenly across all users in the queue. The
higher the value, the more fifo it behaves. 

> maxApplicationsPerUser is wrongly calculated
> --------------------------------------------
>                 Key: YARN-3945
>                 URL: https://issues.apache.org/jira/browse/YARN-3945
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacityscheduler
>    Affects Versions: 2.7.1
>            Reporter: Naganarasimha G R
>            Assignee: Naganarasimha G R
> maxApplicationsPerUser is currently calculated based on the formula
> {{maxApplicationsPerUser = (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor)}}
but description of userlimit is 
> {quote}
> Each queue enforces a limit on the percentage of resources allocated to a user at any
given time, if there is demand for resources. The user limit can vary between a minimum and
maximum value.{color:red} The the former (the minimum value) is set to this property value
{color} and the latter (the maximum value) depends on the number of users who have submitted
applications. For e.g., suppose the value of this property is 25. If two users have submitted
applications to a queue, no single user can use more than 50% of the queue resources. If a
third user submits an application, no single user can use more than 33% of the queue resources.
With 4 or more users, no user can use more than 25% of the queues resources. A value of 100
implies no user limits are imposed. The default is 100. Value is specified as a integer.
> {quote}
> configuration related to minimum limit should not be made used in a formula to calculate
max applications for a user

This message was sent by Atlassian JIRA

View raw message