hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@yahoo-inc.com>
Subject Re: Combiner is optional though it is specified?
Date Tue, 01 Jul 2008 15:09:43 GMT

On Jul 1, 2008, at 4:04 AM, novice user wrote:

>
> Hi all,
>    I have a query regarding the functionality of combiner.
> Is it possible to ignore combiner code for some of the outputs of  
> mapper and
> directly being sent to reducer though combiner is specified in job
> configuration?
> Because, I figured out that, when I am running on large amounts of  
> data,
> some of the mapper output is directly reached reducer. I am  
> wondering how
> can this be possible when I have specified combiner in the job
> configuration. Can any one please  let me know if this thing happens?
>

Can you elaborate on how you reached the conclusion that the output  
of some maps isn't going through the combiner?

Also, what version of hadoop are you using? hadoop-0.18 onwards there  
aren't guarantees on the number of times a combiner is run...

Arun

>
>
> -- 
> View this message in context: http://www.nabble.com/Combiner-is- 
> optional-though-it-is-specified--tp18213887p18213887.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>


Mime
View raw message