hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4891) Pluggable merge at reduce side
Date Wed, 19 Dec 2012 03:36:12 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13535618#comment-13535618

Alejandro Abdelnur commented on MAPREDUCE-4891:

Jerry, unless I'm missing something from your description, this is a duplicate of MAPREDUCE-4808.
Would you please check if that is the case and close this this one as dup of MAPREDUCE-4808
is so?. Thx
> Pluggable merge at reduce side
> ------------------------------
>                 Key: MAPREDUCE-4891
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4891
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>    Affects Versions: 3.0.0
>            Reporter: Jerry Chen
>   Original Estimate: 504h
>  Remaining Estimate: 504h
> The current implementation of sort in MapReduce is cooperated by Map side sort and Reduce
side merge.  MAPREDUCE-2454 provided pluggable sort at the Map side currently and pluggable
shuffle at Reduce side, while no pluggable merger provided.
> Considering a general need of hash grouping and join, we may need to replace both the
Map Sort and Reduce merge with a more light weight hash grouping alorithm. A general pluggable
merge would help support this need.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message