hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: Combiner Problem
Date Sat, 04 Jul 2009 17:20:32 GMT
This question would be more appropriate on the mapreduce-users at 
hadoop.apache.org mailing list.

Unfortunately Hadoop's website is out of date right now and does not 
describe the currently correct mailing lists.  Is someone working to fix 
this?  We do not want general@ to become a place for end-user questions.

Doug

乔木 wrote:
> Hi, everyone
> 
> I've been learning hadoop recently and I'm confused about the combiner
> mechanism.
> 
> There is a property min.num.spills.for.combine specifying the minimum number
> of spills to run combiner when merging. The default value is 3. Why there is
> such a restriction? Should it be better that run the combiner no matter how
> many spills there are?
> 
> The second question is why the combiner could be run at the reduce side.
> Can't the reduce function take place of that?
> 
> Thanks very much.

Mime
View raw message