hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (Resolved) (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (MAPREDUCE-68) Hadoop reduce scheduler sometimes leaves machines idle
Date Sat, 31 Dec 2011 09:54:31 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-68?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Harsh J resolved MAPREDUCE-68.
------------------------------

    Resolution: Not A Problem

Not a problem with current release schedulers.
                
> Hadoop reduce scheduler sometimes leaves machines idle
> ------------------------------------------------------
>
>                 Key: MAPREDUCE-68
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-68
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Nathan Marz
>
> I have a MapReduce application with number of reducers equal to the number of machines
in the cluster (and with speculative execution turned off). However, Hadoop schedules multiple
reduces to run on single machines and leaves other machines idle. This causes contention and
seriously slows down the job. Hadoop should employ the simple heuristic of utilizing as many
machines as possible when scheduling reduces.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message