hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Spreitzer <mspre...@us.ibm.com>
Subject Re: Stupid questions about combiners in ...hadoop.mapreduce
Date Mon, 23 May 2011 19:14:25 GMT
Question 1 remains: What happens if one invocation of a combiner outputs 
more than one value?

My main interest in question 2 was about instances not classes, so let me 
rephrase question 2 this way: What happens if an output key object is not 
equal to the input key object (even though both are of the same class)?

Even for question 3, I did not exactly see an answer to "what happens" --- 
only a statement that I should not exercise that case.

Thanks,
Mike



From:   Ted Yu <yuzhihong@gmail.com>
To:     mapreduce-user@hadoop.apache.org
Date:   05/23/2011 03:04 PM
Subject:        Re: Stupid questions about combiners in 
...hadoop.mapreduce



Questions 2 and 3 can be answered relatively easily:
Remember, the output of the combiner is going to be consumed by the 
reducer.
So the output key/vlaue classes of the combiner have to align with the 
input key/vlaue classes of the reducer.

On Mon, May 23, 2011 at 11:32 AM, Mike Spreitzer <mspreitz@us.ibm.com> 
wrote:
In general, the Java interfaces say that one invocation of a combiner 
(technically, a Class<? extends Reducer>) can output multiple (key,value) 
pairs.  So: 

What happens if one invocation of a combiner outputs more than one value? 

What happens if an output key is different from the input key? 

What happens if an output value is of a different class than the class of 
the input values? 

Thanks, 
Mike Spreitzer


Mime
View raw message