hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5889) Improve user-limit calculation in capacity scheduler
Date Wed, 14 Dec 2016 20:54:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749453#comment-15749453
] 

Jason Lowe commented on YARN-5889:
----------------------------------

I think I can get behind a proposal that preserves the FIFO/priority behavior at least up
through usage < MULP and then becomes fair once a user is beyond MULP.  We need to be careful
that active users are still allowed to grow to fill the queue (e.g.: a single user in the
queue doesn't get capped at MULP).  Also note that a cached active user limit calculation
will become out-of-date more often than before, since resource-used-by-active-users changes
a lot more often than queue-capacity does.  Essentially any resource change for an active
user invalidates the user limit cache for all active users.  I'm not too worried about that,
just pointing out that it will get computed more often with the new proposal.


> Improve user-limit calculation in capacity scheduler
> ----------------------------------------------------
>
>                 Key: YARN-5889
>                 URL: https://issues.apache.org/jira/browse/YARN-5889
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacity scheduler
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: YARN-5889.v0.patch, YARN-5889.v1.patch, YARN-5889.v2.patch
>
>
> Currently user-limit is computed during every heartbeat allocation cycle with a write
lock. To improve performance, this tickets is focussing on moving user-limit calculation out
of heartbeat allocation flow.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message