hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fatih Haltas <fatih.hal...@nyu.edu>
Subject Re: Combiner in Secondary Sort
Date Thu, 21 Mar 2013 07:06:56 GMT
Thanks very much Harsh


On Thu, Mar 21, 2013 at 4:50 AM, Harsh J <harsh@cloudera.com> wrote:

> You're probably running into
> https://issues.apache.org/jira/browse/MAPREDUCE-3310. There was also a
> discussion on this IIRC but I cannot find the archive now. One aside
> mention on this from Chris is at
> http://search-hadoop.com/m/RH5AP11ob2o1.
>
> On Wed, Mar 20, 2013 at 6:10 PM, Fatih Haltas <fatih.haltas@nyu.edu>
> wrote:
> > Hi Everyone,
> >
> > I am trying to implement Secondary Sort Algorithm on mydata. But I am
> having
> > a trouble with my Combiner.
> >
> > When I donot use Combiner, grouping is done well, I mean one reduce task
> is
> > running for every pair, sharing the same first element.
> >
> > However, when I set Combiner as Reducer class itself, grouping in the
> > Combiner is not done according to my custom GroupingComparator class
> though.
> >
> > How can i overrride Combiner class's grouping?
> > Or, is there any diagram, showing the sequence of workflow(both in map
> > phase, calls of mapper classes and calls of combiner classes and so on
> > sequentially) in more detailed?
> >
> > Thanks very much.
>
>
>
> --
> Harsh J
>

Mime
View raw message