hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravi Prakash <ravi...@ymail.com>
Subject Re: Reduce Merge Memtomem Parameter
Date Mon, 28 Oct 2013 14:02:05 GMT
Hi!

This parameter triggers a sort of fetched map outputs on the reducer node when the number
of in memory map outputs > memToMemMergeOutputsThreshold . It is disabled by default. I
am guessing this was put in on the premise that it might be faster to 
sort fewer number of streams even in memory. And also we can sort 
perhaps while waiting to fetch.

HTH
Ravi.




On Friday, October 25, 2013 3:22 PM, - <commodore65@ymail.com> wrote:
 
Hi All,

What do "mapreduce.reduce.merge.memtomem.threshold" and "mapreduce.reduce.merge.memtomem.enabled"
do and by default is it enabled? There's no documentation for them but I noticed that they
exist in hadoop 2.2.0.

Thanks!
Mime
View raw message