hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eugene Morozov <emoro...@griddynamics.com>
Subject Partitioner vs GroupComparator
Date Fri, 23 Aug 2013 15:19:17 GMT

I have two different types of keys emerged from Map and processed by
Reduce. These keys have some part in common. And I'd like to have similar
keys in one reducer. For that purpose I used Partitioner and partition
everything gets in by this common part. It seems to be fine, but MRUnit
seems doesn't know anything about Partitioners. So, here is where
GroupComparator comes into play. It seems that MRUnit well aware of the
guy, but it surprises me: it looks like Partitioner and GroupComparator are
actually doing exactly same - they both somehow group keys to have them in
one reducer.
Could you shed some light on it, please.

View raw message