hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Payne (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4945) [Umbrella] Capacity Scheduler Preemption Within a queue
Date Mon, 22 Aug 2016 20:05:21 GMT

    [ https://issues.apache.org/jira/browse/YARN-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15431513#comment-15431513

Eric Payne commented on YARN-4945:

[~sunilg], Thanks a lot for your reply!

- user-limit: may be partial or full pending resource request will become resource to obtain
for this app. This is depending on user-limit_headroom - current_used. This much can be considered
as demand from this app.

I want to make sure we are talking about the same thing, so I would like to expressly clarify
what I mean by {{user-limit}} because I feel that it is ambiguous and may be causing confusion.

In the statement above, I think you are referring to {{yarn.scheduler.capacity.root.QUEUE1.user-limit-factor}},
which plays a role in determining each user's headroom in a queue. {{user-limit-factor}} is
important to consider when calculating how much of an app's pending resources should be preempted
from other apps. Failure to consider this caused us problems and resulted in YARN-3769.

However, in the context of intra-queue preemption, {{yarn.scheduler.capacity.root.QUEUE1.minimum-user-limit-percent}}
is the property I want to focus on.  My goal is to ensure that each queue is evenly divided
between the appropriate number of users, as defined by {{minimum-user-limit-percent}}.

bq. with this poc, i am coming with framework and priority preemption.
Thank you very much for doing that!

bq. However for doc, it will be good if we could have it common for priority and user-limit.

Also, I think it would be helpful to define use cases so that everyone is clear about what
problems we are trying to solve. I will make an attempt at that and post a doc here.

> [Umbrella] Capacity Scheduler Preemption Within a queue
> -------------------------------------------------------
>                 Key: YARN-4945
>                 URL: https://issues.apache.org/jira/browse/YARN-4945
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Wangda Tan
>         Attachments: IntraQueuepreemption-CapacityScheduler (Design).pdf, YARN-2009-wip.patch
> This is umbrella ticket to track efforts of preemption within a queue to support features
> YARN-2009. YARN-2113. YARN-4781.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message