hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-3054) Preempt policy in FairScheduler may cause mapreduce job never finish
Date Wed, 31 Aug 2016 03:42:20 GMT

     [ https://issues.apache.org/jira/browse/YARN-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Karthik Kambatla updated YARN-3054:
-----------------------------------
    Issue Type: Bug  (was: Sub-task)
        Parent:     (was: YARN-4752)

> Preempt policy in FairScheduler may cause mapreduce job never finish
> --------------------------------------------------------------------
>
>                 Key: YARN-3054
>                 URL: https://issues.apache.org/jira/browse/YARN-3054
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler
>    Affects Versions: 2.6.0
>            Reporter: Peng Zhang
>              Labels: fs-preemption-bugs
>
> Preemption policy is related with schedule policy now. Using comparator of schedule policy
to find preemption candidate cannot guarantee a subset of containers never be preempted. And
this may cause tasks to be preempted periodically before they finish. So job cannot make any
progress. 
> I think preemption in YARN should got below assurance:
> 1. Mapreduce jobs can get additional resources when others are idle;
> 2. Mapreduce jobs for one user in one queue can still progress with its min share when
others preempt resources back.
> Maybe always preempt the latest app and container can get this? 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message