hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@yahoo-inc.com>
Subject Re: Combiner is optional though it is specified?
Date Tue, 01 Jul 2008 15:09:43 GMT

On Jul 1, 2008, at 4:04 AM, novice user wrote:

> Hi all,
>    I have a query regarding the functionality of combiner.
> Is it possible to ignore combiner code for some of the outputs of  
> mapper and
> directly being sent to reducer though combiner is specified in job
> configuration?
> Because, I figured out that, when I am running on large amounts of  
> data,
> some of the mapper output is directly reached reducer. I am  
> wondering how
> can this be possible when I have specified combiner in the job
> configuration. Can any one please  let me know if this thing happens?

Can you elaborate on how you reached the conclusion that the output  
of some maps isn't going through the combiner?

Also, what version of hadoop are you using? hadoop-0.18 onwards there  
aren't guarantees on the number of times a combiner is run...


> -- 
> View this message in context: http://www.nabble.com/Combiner-is- 
> optional-though-it-is-specified--tp18213887p18213887.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.

View raw message