hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Ruchovets <oruchov...@gmail.com>
Subject the same key in different reducers
Date Wed, 09 Jun 2010 08:17:42 GMT
Hi ,
My hadoop job writes results of map/reduce to HBase.
I have 3 reducers.

Here is a sequence of input and output parameters for Mapper , Combiner and
Reducer
    *input: InputFormat<K1,V1>
    mapper: Mapper<K1,V1,K2,V2>
    combiner: Reducer<K2,V2,K2,V2>
    reducer: Reducer<K2,V2,K3,V3>
    output: RecordWriter<K3,V3>

*My question:
  Is it possible that more than one reducer has the same output key K3.
Meaning in case I have 3 reducers is it possible that
reducer1    K3 -* 1* , V3 [1,2,3]
reducer2    K3 - 2 , V3 [5,6,9]
reducer3    K3 - *1* , V3 [10,15,22]

As you can see reducer1 has K3 - 1 and reducer3 has K3 - 1.
So is that case possible or every and every reducer has unique output key?

Thanks in advance
Oleg.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message