hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandy Ryza (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (MAPREDUCE-3951) Tasks are not evenly spread throughout cluster in MR2
Date Tue, 02 Apr 2013 22:11:17 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-3951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Sandy Ryza resolved MAPREDUCE-3951.

    Resolution: Not A Problem
> Tasks are not evenly spread throughout cluster in MR2
> -----------------------------------------------------
>                 Key: MAPREDUCE-3951
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3951
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: scheduler
>    Affects Versions: 0.23.0, 0.24.0
>            Reporter: Todd Lipcon
> In MR1 (at least with the fair and fifo schedulers), if you submit a job that needs fewer
resources than the cluster can provide, the tasks are spread relatively evenly across the
node. For example, submitting a 100-map job to a 50-node cluster, each with 10 slots, results
in 2 tasks on each machine. In MR2, however, the tasks would pile up on the first 10 nodes
of the cluster, leaving the other nodes unused. This is highly suboptimal for many use cases.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message