hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2009) Priority support for preemption in ProportionalCapacityPreemptionPolicy
Date Thu, 22 Sep 2016 17:12:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15513840#comment-15513840

Sunil G commented on YARN-2009:

Hi [~eepayne]

Couple of doubts on the comments.

bq.Shouldn't the following be tq.getUsed() - tq.getActuallyToBePreempted()? tq.getGuaranteed()
only returns the queue's guaranteed capacity but if apps in the queue are using extra resources,
then you want to subtract from the total usage.
If we pick on getUsed, we will be working on from a variable base value "used". So it will
be hard to predict or calculate the preemption logic for analysis. But i think i am agreeing
on the factor that the *used* can more that *guaranteed* and *unallocated* could even go negative
in specific cases. So we could take a *max(guaranteed, used)*. Will this be fine?

bq.Shouldn't this also take into consideration used capacity of all parent queues as well?
I think we might need to consider only LeafQueue as we are working on each LeafQueue's one
by one for intra-queue preemption.

bq.f user1 has used the entire queue with a low priority app, user1's headroom will be 0.
But, if that same user starts a higher priority app, that higher priority app needs to preempt
from the lower priority app, doesn't it?
I am doing this check in the first loop which runs from high priroity app to low priority
app to calculate its idealAssigned. If an app's user-limit is already met, then we need not
have to consider any more demand from that app. Hence such apps idealAssigned can be kept
 as 0.  On the same line, assume if we are doing preemption here. app1 of user1 used entire
queue. app2 of user2 asks more resources. if we preempt some container from app1, will scheduler
allocate to app2? (provided there are some demand from other users). If i am not wrong, it
will not got. pls correct me if I am wrong.

I think this same scenario is mentioned by you in above comments. May be I could consider
app2's demand provided there are no other user's waiting in queue with apps.

> Priority support for preemption in ProportionalCapacityPreemptionPolicy
> -----------------------------------------------------------------------
>                 Key: YARN-2009
>                 URL: https://issues.apache.org/jira/browse/YARN-2009
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler
>            Reporter: Devaraj K
>            Assignee: Sunil G
>         Attachments: YARN-2009.0001.patch, YARN-2009.0002.patch
> While preempting containers based on the queue ideal assignment, we may need to consider
preempting the low priority application containers first.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message