hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jerry Chen (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-4891) Pluggable merge at reduce side
Date Wed, 19 Dec 2012 03:06:12 GMT
Jerry Chen created MAPREDUCE-4891:

             Summary: Pluggable merge at reduce side
                 Key: MAPREDUCE-4891
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4891
             Project: Hadoop Map/Reduce
          Issue Type: New Feature
    Affects Versions: 3.0.0
            Reporter: Jerry Chen

The current implementation of sort in MapReduce is cooperated by Map side sort and Reduce
side merge.  MAPREDUCE-2454 provided pluggable sort at the Map side currently and pluggable
shuffle at Reduce side, while no pluggable merger provided.

Considering a general need of hash grouping and join, we may need to replace both the Map
Sort and Reduce merge with a more light weight hash grouping alorithm. A general pluggable
merge would help support this need.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message