hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Graves (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-3434) Interaction between reservations and userlimit can result in significant ULF violation
Date Wed, 08 Apr 2015 19:00:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14485798#comment-14485798
] 

Thomas Graves edited comment on YARN-3434 at 4/8/15 6:59 PM:
-------------------------------------------------------------

[~wangda] YARN-3243 fixes part of the problem with the max capacities, but it doesn't solve
the user limit side of it.   The user limit check is never done again in assignContainer()
if it skipped the checks in assignContainers() based on reservations but then is allowed to
shouldAllocOrReserveNewContainer.  I'll have a patch up for this shortly I would appreciate
it if you could take a look and give me feedback.


was (Author: tgraves):
[~wangda] YARN-3243 fixes part of the problem with the max capacities, but it doesn't solve
the user limit side of it.   The user limit check is never done again.  I'll have a patch
up for this shortly I would appreciate it if you could take a look and give me feedback.

> Interaction between reservations and userlimit can result in significant ULF violation
> --------------------------------------------------------------------------------------
>
>                 Key: YARN-3434
>                 URL: https://issues.apache.org/jira/browse/YARN-3434
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacityscheduler
>    Affects Versions: 2.6.0
>            Reporter: Thomas Graves
>            Assignee: Thomas Graves
>
> ULF was set to 1.0
> User was able to consume 1.4X queue capacity.
> It looks like when this application launched, it reserved about 1000 containers, each
8G each, within about 5 seconds. I think this allowed the logic in assignToUser() to allow
the userlimit to be surpassed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message