hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thamizhannal Paramasivam <thamizhanna...@gmail.com>
Subject Re: Set<IntWritable> on reducer issue
Date Thu, 09 May 2013 09:43:01 GMT
Thanks a lot Paul. It sorted out my issue.
Because of this issue, I was enforced to use Set<Integer> instead of
Set<IntWritable>.


On Wed, May 8, 2013 at 8:59 PM, Paul Wilkinson
<paul.m.wilkinson@gmail.com>wrote:

> See
> http://cornercases.wordpress.com/2011/08/18/hadoop-object-reuse-pitfall-all-my-reducer-values-are-the-same/for
an explanation of what's going on.
>
> Paul
>
>
> On 8 May 2013 16:10, Thamizhannal Paramasivam <thamizhannal.p@gmail.com>wrote:
>
>> Hi,
>> I require clarification on using IntWritable in Set type in Java.
>> Below is my sample code. When I execute reducer with E.x. key="abcd"
>> values=12,3,4, then categorySet have any one value from values.
>>
>> My hadoop version is 0.20.2. Any help would be appreciated.
>>
>>         public void reduce(Text key, Iterable<IntWritable> values,
>>                 Context context) throws IOException, InterruptedException
>> {
>>
>>              Set<IntWritable> categorySet = new TreeSet<IntWritable>();
>>              for (IntWritable val : values) {
>>                  categorySet.add(val);
>>              }
>>             ----------------
>>             ----------------
>>     }
>>
>> Thanks,
>> Thamizh
>>
>
>

Mime
View raw message