hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2113) Add cross-user preemption within CapacityScheduler's leaf-queue
Date Fri, 31 Mar 2017 18:51:41 GMT

    [ https://issues.apache.org/jira/browse/YARN-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15951489#comment-15951489

Wangda Tan commented on YARN-2113:


I agree with some of your point, for example, app priority can implicitly create priority
arm race. We should properly handle it, for example, only allow the queue admin to be a moderator
to update app priority. 

However, some of your point only valid when there's only one app for each user. In the following

3 apps in the queue, from 2 users.

A1 A2 A3
U1 U2 U1

A1/A3/A4 have lower priority and admin update A2's priority in between.
Assume user limit is 100 for this queue. Usages of these apps are:
A1 = 50, no pending resource.
A2 = 30, pending 50.
A3 = 20

1) Should we allow A2 preempt A3? 
2) Should we allow A2 preempt A1?

>From your previous comment, I think your answer to question #2 is no, however, if we don't
allow #1, does that mean one user can "bully" other users by just keeping the first submitted
app running to get more user limit?

> Add cross-user preemption within CapacityScheduler's leaf-queue
> ---------------------------------------------------------------
>                 Key: YARN-2113
>                 URL: https://issues.apache.org/jira/browse/YARN-2113
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: scheduler
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Sunil G
>         Attachments: TestNoIntraQueuePreemptionIfBelowUserLimitAndDifferentPrioritiesWithExtraUsers.txt,
YARN-2113.0001.patch, YARN-2113.0002.patch, YARN-2113.0003.patch, YARN-2113.0004.patch, YARN-2113.v0.patch
> Preemption today only works across queues and moves around resources across queues per
demand and usage. We should also have user-level preemption within a queue, to balance capacity
across users in a predictable manner.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message