hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Combiner in Secondary Sort
Date Thu, 21 Mar 2013 00:50:12 GMT
You're probably running into
https://issues.apache.org/jira/browse/MAPREDUCE-3310. There was also a
discussion on this IIRC but I cannot find the archive now. One aside
mention on this from Chris is at
http://search-hadoop.com/m/RH5AP11ob2o1.

On Wed, Mar 20, 2013 at 6:10 PM, Fatih Haltas <fatih.haltas@nyu.edu> wrote:
> Hi Everyone,
>
> I am trying to implement Secondary Sort Algorithm on mydata. But I am having
> a trouble with my Combiner.
>
> When I donot use Combiner, grouping is done well, I mean one reduce task is
> running for every pair, sharing the same first element.
>
> However, when I set Combiner as Reducer class itself, grouping in the
> Combiner is not done according to my custom GroupingComparator class though.
>
> How can i overrride Combiner class's grouping?
> Or, is there any diagram, showing the sequence of workflow(both in map
> phase, calls of mapper classes and calls of combiner classes and so on
> sequentially) in more detailed?
>
> Thanks very much.



--
Harsh J

Mime
View raw message