hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-3210) Support delay scheduling for node locality in MR2's capacity scheduler
Date Wed, 19 Oct 2011 07:17:10 GMT
Support delay scheduling for node locality in MR2's capacity scheduler
----------------------------------------------------------------------

                 Key: MAPREDUCE-3210
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3210
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: mrv2
    Affects Versions: 0.23.0
            Reporter: Todd Lipcon
            Assignee: Todd Lipcon


The capacity scheduler in MR2 doesn't support delay scheduling for achieving node-level locality.
So, jobs exhibit poor data locality even if they have good rack locality. Especially on clusters
where disk throughput is much better than network capacity, this hurts overall job performance.
We should optionally support node-level delay scheduling heuristics similar to what the fair
scheduler implements in MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message