hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nathan Marz (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-5160) Hadoop reduce scheduler sometimes leaves machines idle
Date Tue, 03 Feb 2009 03:23:59 GMT
Hadoop reduce scheduler sometimes leaves machines idle

                 Key: HADOOP-5160
                 URL: https://issues.apache.org/jira/browse/HADOOP-5160
             Project: Hadoop Core
          Issue Type: Bug
          Components: mapred
            Reporter: Nathan Marz

I have a MapReduce application with number of reducers equal to the number of machines in
the cluster (and with speculative execution turned off). However, Hadoop schedules multiple
reduces to run on single machines and leaves other machines idle. This causes contention and
seriously slows down the job. Hadoop should employ the simple heuristic of utilizing as many
machines as possible when scheduling reduces.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message