hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsuyoshi OZAWA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5221) Reduce side Combiner is not used when using the new API
Date Fri, 07 Jun 2013 09:20:21 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13677914#comment-13677914
] 

Tsuyoshi OZAWA commented on MAPREDUCE-5221:
-------------------------------------------

[~masokan]: I thought combinerClass can be removed from ReduceTask.java too. However,  combinerClass
in ReduceTask.java is used to decide whether the instance of CombineOutputCollector is created
now. If it need to avoid being instantiated, the combinerClass must remain. The instance seems
to be not so much expensive for me, so I think it can be removed.  Do you have any opinion
about this?
                
> Reduce side Combiner is not used when using the new API
> -------------------------------------------------------
>
>                 Key: MAPREDUCE-5221
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5221
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 2.0.4-alpha
>            Reporter: Siddharth Seth
>            Assignee: Tsuyoshi OZAWA
>         Attachments: MAPREDUCE-5221.1.patch, MAPREDUCE-5221.2.patch, MAPREDUCE-5221.3.patch
>
>
> If a combiner is specified using o.a.h.mapreduce.Job.setCombinerClass - this will silently
ignored on the reduce side since the reduce side usage is only aware of the old api combiner.
> This doesn't fail the job - since the new combiner key does not deprecate the old key.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message