hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew John <tmatthewjohn1...@gmail.com>
Subject Reduce groups
Date Tue, 19 Oct 2010 11:47:35 GMT
Hi all,

The number of Reducer groups in my MapReduce is always the same as the
number of records output by the MapReduce. So what I understand is every
record from the Shuffle/Sort is going to different Reducer.reduce. How can I
change this? My key is BytesWritable and I tried writing my own Comparator
and set it in setOutputValueGroupingClass but still not more than one record
is entering the same reduce group. Someone please tell me the mechanism
behind this so that I can fix this problem . I am not caring about
Partitioner since I am using a single reducer.

Thanks,

Matthew

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message