hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peng Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-3054) Preempt policy in FairScheduler may cause mapreduce job never finish
Date Tue, 13 Jan 2015 08:01:34 GMT
Peng Zhang created YARN-3054:
--------------------------------

             Summary: Preempt policy in FairScheduler may cause mapreduce job never finish
                 Key: YARN-3054
                 URL: https://issues.apache.org/jira/browse/YARN-3054
             Project: Hadoop YARN
          Issue Type: Bug
          Components: fairscheduler
    Affects Versions: 2.6.0
            Reporter: Peng Zhang


Preemption policy is related with schedule policy now. Using comparator of schedule policy
to find preemption candidate cannot guarantee a subset of containers never be preempted. And
this may cause tasks to be preempted periodically before they finish. So job cannot make any
progress. 

I think preemption in YARN should got below assurance:
1. Mapreduce jobs can get additional resources when others are idle;
2. Mapreduce jobs for one user in one queue can still progress with its min share when others
preempt resources back.

Maybe always preempt the latest app and container can get this? 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message