hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harshit Mathur <mathursh...@gmail.com>
Subject Re: Reducer called twice for same key
Date Mon, 29 Jun 2015 03:52:00 GMT
As per Map Reduce, it is not possible that two different reducers will get
same keys.
I think you have created some custom key type? If that is the case then
there should be some issue with the comparator.

On Mon, Jun 29, 2015 at 12:40 AM, Ravikant Dindokar <ravikant.iisc@gmail.com
> wrote:

> Hi Hadoop user,
>
> I have two map classes processing two different input files. Both map
> functions have same key,value format to emit.
>
> But Reducer called twice for same key , one for value from first map while
> one for value from other map.
>
> I am printing (key ,value) pairs in reducer  :
> ./container_1435326857837_0036_01_000102/stdout:Reduce:391:-1#11
>
> ./container_1435326857837_0036_01_000102/stdout:Reduce:391:3278620528725786624:5352454#-1
>
> ./container_1435326857837_0036_01_000102/stdout:Reduce:591:3278620528725852160:4194699#-1
> ./container_1435326857837_0036_01_000102/stdout:Reduce:591:-1#13
> ./container_1435326857837_0036_01_000102/stdout:Reduce:2391:-1#19
>
> ./container_1435326857837_0036_01_000102/stdout:Reduce:2391:3278620528725917696:5283986#-1
>
> ./container_1435326857837_0036_01_000102/stdout:Reduce:3291:3278620528725983232:4973087#-1
>
> both maps emit Longwritable key and Text value.
>
>
> Any idea why this is happening?
> Is there any way to get hash values generated by hadoop for keys emitted
> by mapper?
>
> Thanks
> Ravikant
>



-- 
Harshit Mathur

Mime
View raw message