hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Scott Chen (JIRA)" <j...@apache.org>
Subject [jira] Created: (MAPREDUCE-1129) Assign multiple Map and Reduce tasks in Fairscheduler
Date Wed, 21 Oct 2009 22:20:59 GMT
Assign multiple Map and Reduce tasks in Fairscheduler
-----------------------------------------------------

                 Key: MAPREDUCE-1129
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1129
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: contrib/fair-share
    Affects Versions: 0.20.1
            Reporter: Scott Chen


In Hadoop-0.20, the period of heartbeat becomes much longer. 
Fairscheduler assigns at most one Map and one Reduce task per heartbeat.
This makes the cluster become very inefficient. Often time only half of the slots are used.

One idea is that we make Fairscheduler detect this situation (cluster under used) and scheduler
more tasks in a heartbeat.
Thoughts?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message