hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Carlo Curino (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4003) ReservationQueue inherit getAMResourceLimit() from LeafQueue, but behavior is not consistent
Date Wed, 05 Aug 2015 18:42:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-4003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14658665#comment-14658665
] 

Carlo Curino commented on YARN-4003:
------------------------------------

I think you are correct, but given that the size of the allocation for a ReservationQueue
can fluctuate substantially, I think we need to rely on either hard-limits on the number of
applications running per queue, or careful user submissions will need to protect from the
bad scenario you describe. 
Picking the opposite strategy (using actual reservation size to limit number of AMs) prevents
us to scavange resources early in the reservation, and I am not sure even works properly if
the reservation "shrinks" (does ProportionalCPP kill AMs running in a queue, if the capacity
is shrinked?).

I agree with your concerns, but I don't see a better way out.



> ReservationQueue inherit getAMResourceLimit() from LeafQueue, but behavior is not consistent
> --------------------------------------------------------------------------------------------
>
>                 Key: YARN-4003
>                 URL: https://issues.apache.org/jira/browse/YARN-4003
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>            Reporter: Carlo Curino
>         Attachments: YARN-4003.patch
>
>
> The inherited behavior from LeafQueue (limit AM % based on capacity) is not a good fit
for ReservationQueue (that have highly dynamic capacity). 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message