hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Seigel <ja...@tynt.com>
Subject Re: the same key in different reducers
Date Wed, 09 Jun 2010 21:40:10 GMT
Oleg,

Are you wanting to have them in different reducers?  If so then you can write a Comparable
object to make that happen.

If you want them to be on the same reducer, then that is what hadoop will do.

:)


On 2010-06-09, at 3:06 PM, Ted Yu wrote:

> Can you disclose more about how K3 is generated.
> From your description below, it is possible.
> 
> On Wed, Jun 9, 2010 at 1:17 AM, Oleg Ruchovets <oruchovets@gmail.com> wrote:
> 
>> Hi ,
>> My hadoop job writes results of map/reduce to HBase.
>> I have 3 reducers.
>> 
>> Here is a sequence of input and output parameters for Mapper , Combiner and
>> Reducer
>>   *input: InputFormat<K1,V1>
>>   mapper: Mapper<K1,V1,K2,V2>
>>   combiner: Reducer<K2,V2,K2,V2>
>>   reducer: Reducer<K2,V2,K3,V3>
>>   output: RecordWriter<K3,V3>
>> 
>> *My question:
>> Is it possible that more than one reducer has the same output key K3.
>> Meaning in case I have 3 reducers is it possible that
>> reducer1    K3 -* 1* , V3 [1,2,3]
>> reducer2    K3 - 2 , V3 [5,6,9]
>> reducer3    K3 - *1* , V3 [10,15,22]
>> 
>> As you can see reducer1 has K3 - 1 and reducer3 has K3 - 1.
>> So is that case possible or every and every reducer has unique output key?
>> 
>> Thanks in advance
>> Oleg.
>> 


Mime
View raw message