hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jim Twensky <jim.twen...@gmail.com>
Subject Question on GroupingComparatorClass
Date Tue, 26 Jan 2010 07:27:12 GMT

I'm using a custom grouping comparator class to simulate a secondary
sort on values, and I set it via Job.setGroupingComparatorClass (using
Hadoop 0.20.x) inside my driver. I'm wondering if this class is also
used when grouping the records in the combiner.

Using a combiner greatly improves the performance in my case, but for
the combiners, I want to use the default comparator, not the custom
one that I use before  the actual reduce.

Is there a way to just set the custom grouping comparator for the
reduce and bypass it during the combine stage?


View raw message