Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 951DB104FD for ; Mon, 28 Oct 2013 14:02:46 +0000 (UTC) Received: (qmail 56117 invoked by uid 500); 28 Oct 2013 14:02:38 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 55966 invoked by uid 500); 28 Oct 2013 14:02:36 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 55958 invoked by uid 99); 28 Oct 2013 14:02:35 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 28 Oct 2013 14:02:35 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy includes SPF record at spf.trusted-forwarder.org) Received: from [72.30.239.216] (HELO nm40-vm8.bullet.mail.bf1.yahoo.com) (72.30.239.216) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 28 Oct 2013 14:02:28 +0000 Received: from [98.139.215.143] by nm40.bullet.mail.bf1.yahoo.com with NNFMP; 28 Oct 2013 14:02:07 -0000 Received: from [98.139.212.196] by tm14.bullet.mail.bf1.yahoo.com with NNFMP; 28 Oct 2013 14:02:07 -0000 Received: from [127.0.0.1] by omp1005.mail.bf1.yahoo.com with NNFMP; 28 Oct 2013 14:02:07 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 82394.49487.bm@omp1005.mail.bf1.yahoo.com Received: (qmail 85552 invoked by uid 60001); 28 Oct 2013 14:02:06 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ymail.com; s=s1024; t=1382968926; bh=VbFDpTJe8NhwyJwjn0GVXdyysD6g7//zB2mQTsTeFs0=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=NL4odeIS3bD2oRIqKvPokQLnJ0bhTo8qG77TBMVv8TLDM6+XKZE/xgS6X93CRXneHJy0ncJPxsu3pQOmRpb5sCpJZSkmOTOoO4Qoxn/kia16elW0Czsx145h/SaqpKJGgnEX5JXMMd8J+smy2miYjb+h14bPjXlqM2TR/YeqCGU= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=ymail.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=YsffP3NILpLeLY1ATQdKde8X+5fsJyFA1KFNe0w30v6eOnFTQTiK4i94HPGUJSGKU7KN9B/Wd8yRNGEGM55efJTTgFPgvGbHZRnxbsWY/Ii0Il5kAlsSFAdx2JFpVtJfrW2W4+kreaoQtGC8UstJ08vxFoU2GJ7KYr1lPR8cNzI=; X-YMail-OSG: ZHl8sOwVM1l3jJwdz5CkP1oUpyIf1uMTAI4owRE7F7I.Nrw ZUK9NDoMquPUmmh1HdTxE8Qw2IXm9N5sLmRzDvIqx5Yei2noH6x40NT_O6Zl 01NOeqwlaNJT0JAdkbU267Yof5A5rSYw1Kx77WqfEKLFD5aUZ0SEWnmAaTjh okPyxIryD2iwlGphEV6XTJP4IJoFdjlmiaL.x37MgKVjam8Yy41ppoDl77.k VyrLIP54VgjF8uKr4yuRKIWjOT49pV6dtKkkRIk1FkTU75xURoth9hzQssO7 QVzH9RxtdSx6hujJyVI_djVckBAsgRcAtyIgr9vkEb1SucPV0uuWfO93S5kA 0mqDqAjiQcM.NcWUquXRTle_L6hwjUakaMu3sWsGSOU7JLTPTNcWcBNdgnKE tLu5RkMJjFihxioVoTHKC4IbwQ4TrdLcfZ.oTaky_Unb3jQ9vHPa54uD3mQO LFw9kFr7rtqnp7dxALTAMO5ybNSQOJqVZfdNFLPw8A5i2SR82UK3ApWghzxp B5NtP3m4YAkFyNYYFDNlqBjTrfkwH89z9FOSHTW9gN.kbY40h7QoWUZWUSIZ dMm_bQP0- Received: from [216.145.54.50] by web141203.mail.bf1.yahoo.com via HTTP; Mon, 28 Oct 2013 07:02:05 PDT X-Rocket-MIMEInfo: 002.001,SGkhCgpUaGlzIHBhcmFtZXRlciB0cmlnZ2VycyBhIHNvcnQgb2YgZmV0Y2hlZCBtYXAgb3V0cHV0cyBvbiB0aGUgcmVkdWNlciBub2RlIHdoZW4gdGhlIG51bWJlciBvZiBpbiBtZW1vcnkgbWFwIG91dHB1dHMgPiBtZW1Ub01lbU1lcmdlT3V0cHV0c1RocmVzaG9sZCAuIEl0IGlzIGRpc2FibGVkIGJ5IGRlZmF1bHQuIEkgYW0gZ3Vlc3NpbmcgdGhpcyB3YXMgcHV0IGluIG9uIHRoZSBwcmVtaXNlIHRoYXQgaXQgbWlnaHQgYmUgZmFzdGVyIHRvIApzb3J0IGZld2VyIG51bWJlciBvZiBzdHJlYW1zIGV2ZW4gaW4BMAEBAQE- X-Mailer: YahooMailWebService/0.8.160.587 References: <1382732430.80700.YahooMailNeo@web121501.mail.ne1.yahoo.com> Message-ID: <1382968925.84966.YahooMailNeo@web141203.mail.bf1.yahoo.com> Date: Mon, 28 Oct 2013 07:02:05 -0700 (PDT) From: Ravi Prakash Reply-To: Ravi Prakash Subject: Re: Reduce Merge Memtomem Parameter To: "user@hadoop.apache.org" , - In-Reply-To: <1382732430.80700.YahooMailNeo@web121501.mail.ne1.yahoo.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="1123101620-1536591234-1382968925=:84966" X-Virus-Checked: Checked by ClamAV on apache.org --1123101620-1536591234-1382968925=:84966 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Hi!=0A=0AThis parameter triggers a sort of fetched map outputs on the reduc= er node when the number of in memory map outputs > memToMemMergeOutputsThre= shold . It is disabled by default. I am guessing this was put in on the pre= mise that it might be faster to =0Asort fewer number of streams even in mem= ory. And also we can sort =0Aperhaps while waiting to fetch.=0A=0AHTH=0ARav= i.=0A=0A=0A=0A=0AOn Friday, October 25, 2013 3:22 PM, - wrote:=0A =0AHi All,=0A=0AWhat do "mapreduce.reduce.merge.memtomem.thr= eshold" and=A0"mapreduce.reduce.merge.memtomem.enabled" do and by default i= s it enabled? There's no documentation for them but I noticed that they exi= st in hadoop 2.2.0.=0A=0AThanks! --1123101620-1536591234-1382968925=:84966 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
Hi!

This parameter triggers a sort of fetched map outputs on the red= ucer node when the number of in memory map outputs > memToMemMergeOutput= sThreshold . It is disabled by default. I am guessing this was put in on th= e premise that it might be faster to =0Asort fewer number of streams even i= n memory. And also we can sort =0Aperhaps while waiting to fetch.

HTH
Ravi.<= br>


<= div style=3D"font-family: HelveticaNeue, Helvetica Neue, Helvetica, Arial, = Lucida Grande, sans-serif; font-size: 10pt;">
On Friday, October 25, 2013 3:22 PM, - <comm= odore65@ymail.com> wrote:
Hi All,

What do "mapreduce.reduce.merge.memtomem.= threshold" and "mapreduce.reduce.merge.memtomem.enabled"= do and by default is it enabled? There's no=0A documentation for them but = I noticed that they exist in hadoop 2.2.0.
=
Thanks!


--1123101620-1536591234-1382968925=:84966--