hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Payne (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2009) Priority support for preemption in ProportionalCapacityPreemptionPolicy
Date Wed, 26 Oct 2016 22:11:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15609848#comment-15609848
] 

Eric Payne commented on YARN-2009:
----------------------------------

bq. there are cases where unnecessary preemption still happens.
I have a multi-tenant use case where 1 or 2 intra-queue preemptions occur unnecessarily _every
cycle_.
bq. many of potential computation mistakes should be able to be corrected automatically by
several rounds
This one will not. It happens every round.

I can provide the details of my manual test if you would like. I haven't found a way yet to
change the configs to make this stop happening. If you have suggestions, please let me know.

Many of our queues are used by multiple users, and if a high priority app can cause apps from
other users to be unnecessarily preempted every preemption cycle, that will not be acceptable
by our users.

bq. Let's focus all the user-limit related issues in the next patch.
The patch works well if users don't share queues. However, my concern is that if users share
queues AND they want to use this feature to manage priorities between apps of the same user,
this will cause problems with other users in the queue. If we put this in as it is, we are
stating that this feature must not be turned on if any preemptable queue in the cluster is
multi-tenant. Is that what we want?

> Priority support for preemption in ProportionalCapacityPreemptionPolicy
> -----------------------------------------------------------------------
>
>                 Key: YARN-2009
>                 URL: https://issues.apache.org/jira/browse/YARN-2009
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler
>            Reporter: Devaraj K
>            Assignee: Sunil G
>         Attachments: YARN-2009.0001.patch, YARN-2009.0002.patch, YARN-2009.0003.patch,
YARN-2009.0004.patch, YARN-2009.0005.patch, YARN-2009.0006.patch, YARN-2009.0007.patch, YARN-2009.0008.patch,
YARN-2009.0009.patch, YARN-2009.0010.patch, YARN-2009.0011.patch, YARN-2009.0012.patch, YARN-2009.0013.patch,
YARN-2009.0014.patch, YARN-2009.0015.patch
>
>
> While preempting containers based on the queue ideal assignment, we may need to consider
preempting the low priority application containers first.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message