hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yufei Gu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7859) New feature: add queue scheduling deadLine in fairScheduler.
Date Thu, 01 Feb 2018 05:10:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348019#comment-16348019
] 

Yufei Gu commented on YARN-7859:
--------------------------------

Hi [~wangwj], not sure I understand your propose. You seems introduce one more property(deadline)
for queues, which has the highest priority than others according to your code, e.g. fair share.
In that case, you can't really ensure any fairness of resource usage, which is core part of
fair scheduler.

> New feature: add queue scheduling deadLine in fairScheduler.
> ------------------------------------------------------------
>
>                 Key: YARN-7859
>                 URL: https://issues.apache.org/jira/browse/YARN-7859
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: fairscheduler
>    Affects Versions: 3.0.0
>            Reporter: wangwj
>            Priority: Major
>              Labels: fairscheduler, features, patch
>             Fix For: 3.0.0
>
>         Attachments: YARN-7859-v1.patch, log, screenshot-1.png, screenshot-3.png
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
>      As everyone knows.In FairScheduler the phenomenon of queue scheduling starvation
often occurs when the number of cluster jobs is large.The App in one or more queue are pending.So
I have thought a way to solve this problem.Add queue scheduling deadLine in fairScheduler.When
a queue is not scheduled for FairScheduler within a specified time.We mandatory scheduler
it!
>     Now the way of community solves queue scheduling to starvation is preempt container.But
this way may increases the failure rate of the job.
>     On the basis of the above, I propose this issue...



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message