hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mariappan Asokan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5294) Shuffle#MergeManager should support org.apache.hadoop.mapreduce.Reducer
Date Thu, 06 Jun 2013 19:19:20 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13677396#comment-13677396

Mariappan Asokan commented on MAPREDUCE-5294:

Hi Tsuyoshi,
  With the patch you submitted, the parameter {{combinerClass}} to {{MergeManagerImpl}} is
unused.  Can we get rid of it?


-- Asokan
> Shuffle#MergeManager should support org.apache.hadoop.mapreduce.Reducer
> -----------------------------------------------------------------------
>                 Key: MAPREDUCE-5294
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5294
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: mrv2
>    Affects Versions: trunk, 2.1.0-beta, 2.0.5-alpha
>            Reporter: Tsuyoshi OZAWA
>            Assignee: Tsuyoshi OZAWA
>         Attachments: MAPREDUCE-5294.1.patch, MAPREDUCE-5294.2.patch
> Shuffle#MergeManager only accepts org.apache.hadoop.mapred.Reducer currently. Because
of this, Reduce-side Combiner is not used when using the new API, and just ignored. By supporting
it and using the feature from ReduceTask, Reduce-side combiner can be enabled with new API.
Please see MAPREDUCE-5221 for more detail.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message