hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsuyoshi OZAWA (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-5294) Shuffle#MergeManager should support org.apache.hadoop.mapreduce.Reducer
Date Fri, 07 Jun 2013 06:39:22 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-5294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Tsuyoshi OZAWA updated MAPREDUCE-5294:

    Attachment: MAPREDUCE-5294.3.patch

Hi [~masokan],
Thanks you for review!

b.q With the patch you submitted, the parameter combinerClass to MergeManagerImpl is unused.
Can we get rid of it?

Yes, you're right. I've overlooked it until you pointed out. This new patch removed combinerClass
from MergeManagerImpl's constructor.
> Shuffle#MergeManager should support org.apache.hadoop.mapreduce.Reducer
> -----------------------------------------------------------------------
>                 Key: MAPREDUCE-5294
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5294
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: mrv2
>    Affects Versions: trunk, 2.1.0-beta, 2.0.5-alpha
>            Reporter: Tsuyoshi OZAWA
>            Assignee: Tsuyoshi OZAWA
>         Attachments: MAPREDUCE-5294.1.patch, MAPREDUCE-5294.2.patch, MAPREDUCE-5294.3.patch
> Shuffle#MergeManager only accepts org.apache.hadoop.mapred.Reducer currently. Because
of this, Reduce-side Combiner is not used when using the new API, and just ignored. By supporting
it and using the feature from ReduceTask, Reduce-side combiner can be enabled with new API.
Please see MAPREDUCE-5221 for more detail.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message