Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id AB32A100FC for ; Mon, 1 Jul 2013 23:03:58 +0000 (UTC) Received: (qmail 54537 invoked by uid 500); 1 Jul 2013 23:03:53 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 54433 invoked by uid 500); 1 Jul 2013 23:03:53 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 54426 invoked by uid 99); 1 Jul 2013 23:03:53 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Jul 2013 23:03:53 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of dontariq@gmail.com designates 209.85.220.175 as permitted sender) Received: from [209.85.220.175] (HELO mail-vc0-f175.google.com) (209.85.220.175) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Jul 2013 23:03:49 +0000 Received: by mail-vc0-f175.google.com with SMTP id hr11so2341379vcb.20 for ; Mon, 01 Jul 2013 16:03:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=o5Hq+qx4tU2h3Hqf47GzyAr1dmO3e5SzluFx5cJGe8s=; b=apnj5E69K/laDqyzZnccLAeAKYGpFr6cdI7hdgt81mtdUDBKIcAB3pwDO/RoeV+glN 3af+YWkEcBXoHIkFPpCECEoBmwJqS/4ebAPPQgGc86sLrO/vrdhQcBR4tJefjPnyZ26d AficktnjT22qqp9FjN2QL7PsxRRbVLzENMDNE9eq6MmsFIRaLU1nGgCiBum+Ei4grXvf 8n2dMPctR+k1T8XWbVamAInSBqTUF4xIYPjQB6901C/GmeGyeGaQc3AtKt+I8T79AQOC 10VfGbmiLI2k5fzxyaVQDpOse29lF/DvurGGnQ29MsRAr9TJ57KErt6vMN32EQLBPHjk 64yg== X-Received: by 10.220.181.69 with SMTP id bx5mr10263392vcb.71.1372719808996; Mon, 01 Jul 2013 16:03:28 -0700 (PDT) MIME-Version: 1.0 Received: by 10.58.180.8 with HTTP; Mon, 1 Jul 2013 16:02:48 -0700 (PDT) In-Reply-To: <869970D71E26D7498BDAC4E1CA92226B658D5912@MBX021-E3-NJ-2.exch021.domain.local> References: <869970D71E26D7498BDAC4E1CA92226B658D5912@MBX021-E3-NJ-2.exch021.domain.local> From: Mohammad Tariq Date: Tue, 2 Jul 2013 04:32:48 +0530 Message-ID: Subject: Re: intermediate results files To: "user@hadoop.apache.org" Content-Type: multipart/alternative; boundary=001a1132f30c976fba04e07b3cfa X-Virus-Checked: Checked by ClamAV on apache.org --001a1132f30c976fba04e07b3cfa Content-Type: text/plain; charset=ISO-8859-1 Hello John, IMHO, it doesn't matter. Your job will write the result just once. Replica creation is handled at the HDFS layer so it has nothing to with your job. Your job will still be writing at the same speed. Warm Regards, Tariq cloudfront.blogspot.com On Tue, Jul 2, 2013 at 4:16 AM, John Lilley wrote: > If my reducers are going to create results that are temporary in nature > (consumed by the next processing stage) is it recommended to use a > replication factor <3 to improve performance? **** > > Thanks**** > > john**** > > ** ** > --001a1132f30c976fba04e07b3cfa Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hello John,

=A0 =A0 =A0 IMHO, it = doesn't matter. Your job will write the result just once. Replica creat= ion is handled at the HDFS layer so it has nothing to with your job. Your j= ob will still be writing at the same speed.

Wa= rm Regards,
Tariq


On Tue, Jul 2, 2013 at 4:16 AM, John Lil= ley <john.lilley@redpoint.net> wrote:

If my reducers are going to create results that are = temporary in nature (consumed by the next processing stage) is it recommend= ed to use a replication factor <3 to improve performance?=A0

Thanks

john

=A0


--001a1132f30c976fba04e07b3cfa--