hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Resolved: (HADOOP-484) Additional splilts for last reduces?
Date Tue, 29 Aug 2006 04:45:24 GMT
     [ http://issues.apache.org/jira/browse/HADOOP-484?page=all ]

Owen O'Malley resolved HADOOP-484.

    Resolution: Duplicate

Splitting a reduce would be difficult and error-prone while it was in progress. The standard
approach for this problem is to use speculative execution to shorten the tail. It seems to
be very effective.

> Additional splilts for last reduces?
> ------------------------------------
>                 Key: HADOOP-484
>                 URL: http://issues.apache.org/jira/browse/HADOOP-484
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: arkady borkovsky
> Often last few reduces take very long.  
> Would it make sense, if hardware is available, to resplit their inputs into smaller chunks
and to run multiple task instead?

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message