hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Meagher" <john.meag...@gmail.com>
Subject Re: Problem in Map/Reduce
Date Fri, 29 Aug 2008 11:14:48 GMT
Did you override the equals and hashcode methods?  These are the
methods usually used in a map to determine equality for put/get
operations.  The comparator is probably only used for sorting, not
equality checks.



On Fri, Aug 29, 2008 at 2:55 AM, P.ILAYARAJA <ilayaraja@rediff.co.in> wrote:
>
> Hello:
>
> I wrote a simple Map/Reduce program. The ouput key of the Map function is a user defined
datum(class)
> with two member strings. The OutputKeyComparatorClass is set to this datum class and
the class
> overrides the "compareTo" function.
>
> The problem is the final MapOutput from reduce has the same "key" occuring in more than
one record.
> Any thougts on why this could happen?
>
> Also I see that the compareTo function never gets as input the pair of "keys" that are
same for comparison.....
>
> Regards,
> Ilay

Mime
View raw message