hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Dunning" <ted.dunn...@gmail.com>
Subject Re: OK to remove conf.setCombinerClass(myReduce.class)?
Date Mon, 18 Aug 2008 20:57:06 GMT
This doesn't sound right.

There are two reasons.

First, the combiner is critical for good performance on word count
applications.

Secondly, having a combiner should not prevent having a different kind of
collector.  The combiner should just look like the reducer.

Why do you think that having the combiner is causing your problem?

On Mon, Aug 18, 2008 at 1:32 PM, Kunsheng Chen <hadoopchan@yahoo.com> wrote:

> Hello, everyone.
>
> I am doing  example  WordCount and have to remove
> conf.setCombinerClass(myReduce.class) in main method in order to have maper
> and reduce run with different types of output collector.
>
> I am not sure whether it cause the problem for the perfomance.  Please let
> me know if it does.   Any idea is appreciated.
>
>
>
>




-- 
ted

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message