hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3945) maxApplicationsPerUser is wrongly calculated
Date Tue, 16 Feb 2016 21:44:18 GMT

    [ https://issues.apache.org/jira/browse/YARN-3945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149365#comment-15149365

Wangda Tan commented on YARN-3945:

bq. so based on this there will be no elasticity even though the resources are free in some
other queue, is this expected ?
I think so

bq. Are we trying to avoid elasticity because we try to avoid preempting AM's even when preemption
is enabled?
That's the one purpose, the other purpose is, when preemption is disabled, we will not suffer
with too many AMs launched with queue's available resource increases and then come back.

To make move this task forward, I would suggest:
# Resolve bug of maxApplicationsPerUser should be capped by maxApplicationsPerQueue
# Computation of user AM limit should be symmetric to computation of user-limit, and user
AM limit should be capped by queue's AM limit
# Avoid flexibility of computing queue and user's AM-limit (do not consider queue max cap).
This needs more discussion.

My understanding is, #1 and #2 are scope of this JIRA, #3 could be done separately.


> maxApplicationsPerUser is wrongly calculated
> --------------------------------------------
>                 Key: YARN-3945
>                 URL: https://issues.apache.org/jira/browse/YARN-3945
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacityscheduler
>    Affects Versions: 2.7.1
>            Reporter: Naganarasimha G R
>            Assignee: Naganarasimha G R
>         Attachments: YARN-3945.20150728-1.patch, YARN-3945.20150729-1.patch, YARN-3945.V1.003.patch
> maxApplicationsPerUser is currently calculated based on the formula
> {{maxApplicationsPerUser = (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor)}}
but description of userlimit is 
> {quote}
> Each queue enforces a limit on the percentage of resources allocated to a user at any
given time, if there is demand for resources. The user limit can vary between a minimum and
maximum value.{color:red} The the former (the minimum value) is set to this property value
{color} and the latter (the maximum value) depends on the number of users who have submitted
applications. For e.g., suppose the value of this property is 25. If two users have submitted
applications to a queue, no single user can use more than 50% of the queue resources. If a
third user submits an application, no single user can use more than 33% of the queue resources.
With 4 or more users, no user can use more than 25% of the queues resources. A value of 100
implies no user limits are imposed. The default is 100. Value is specified as a integer.
> {quote}
> configuration related to minimum limit should not be made used in a formula to calculate
max applications for a user

This message was sent by Atlassian JIRA

View raw message