Return-Path: 
 <common-user-return-29081-apmail-hadoop-common-user-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-common-user-archive@www.apache.org
Delivered-To: apmail-hadoop-common-user-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 01D754882
	for <apmail-hadoop-common-user-archive@www.apache.org>;
 Mon, 11 Jul 2011 21:13:53 +0000 (UTC)
Received: (qmail 43708 invoked by uid 500); 11 Jul 2011 21:13:51 -0000
Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org
Received: (qmail 43642 invoked by uid 500); 11 Jul 2011 21:13:50 -0000
Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:common-user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:common-user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:common-user@hadoop.apache.org>
List-Id: <common-user.hadoop.apache.org>
Reply-To: common-user@hadoop.apache.org
Delivered-To: mailing list common-user@hadoop.apache.org
Received: (qmail 43634 invoked by uid 99); 11 Jul 2011 21:13:50 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 11 Jul 2011 21:13:50 +0000
X-ASF-Spam-Status: No, hits=1.5 required=5.0
	tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL
X-Spam-Check-By: apache.org
Received-SPF: pass (athena.apache.org: domain of gordoslocos@gmail.com
 designates 74.125.82.48 as permitted sender)
Received: from [74.125.82.48] (HELO mail-ww0-f48.google.com) (74.125.82.48)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 11 Jul 2011 21:13:43 +0000
Received: by wwi14 with SMTP id 14so4544120wwi.29
        for <common-user@hadoop.apache.org>;
 Mon, 11 Jul 2011 14:13:22 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        bh=PpP+r2etG3rY9nB9ciW6z1XQr+qr/iqIviuyJKwewJo=;
        b=DFzUL9QoJkJ1muarOJcCofnhBEYQvw+Mh/tN90vEbroev6i5rC1q5kq0wL7dCE3iQg
         XptZE3mpJ+AbPWU4P3EJNL9Yd7junRpYqql5rt4Od6t15BL8xJQ+3YSsPAtT6aiOtdNj
         PSp4Y5AZ1DZAxKa91V9aiZNUC4EpDa3+QcUaQ=
MIME-Version: 1.0
Received: by 10.216.122.10 with SMTP id s10mr3500600weh.34.1310418802302; Mon,
 11 Jul 2011 14:13:22 -0700 (PDT)
Received: by 10.216.91.17 with HTTP; Mon, 11 Jul 2011 14:13:22 -0700 (PDT)
In-Reply-To: <F82635E4-3555-4265-B723-B955C6A19E67@apache.org>
References: 
 <CAD8ZxGqPVNvAb3Yt8eMmdvg1j=Ef7dA3ggPA4GNhx6HD+VwNhg@mail.gmail.com>
	<CALPvCiDngQi4-ysXRVw0VZ+rjmp7hPchpdYrmddGCYy8+Lr0Bg@mail.gmail.com>
	<F2BBF283-17DC-4F95-995F-2E26864491CF@gmail.com>
	<CAD8ZxGrWBQWg-Y1pECPY9jpgJS1jC=yC7d9OOdX3SpAgyikovQ@mail.gmail.com>
	<CAD8ZxGqYXiXSLek6Ljbvc9Lur3A=Xc9tiDWdN-eDGpMW7G+XJw@mail.gmail.com>
	<CAEGKEUPRSsMi8T10tzy_DRKs16D3nS9TusOnXi1Y7J0tbP1FUA@mail.gmail.com>
	<1310152879.95818.YahooMailNeo@web110702.mail.gq1.yahoo.com>
	<CAD8ZxGqTG9K7xbUtu6-tYLJDq5VHrutoP5uudvBTbPF3=1aUVg@mail.gmail.com>
	<F82635E4-3555-4265-B723-B955C6A19E67@apache.org>
Date: Mon, 11 Jul 2011 18:13:22 -0300
Message-ID: 
 <CAD8ZxGp_36Ozct=zSFB-EPyy=Q9vvXrePx4VWe7Edtt1Vo1Wuw@mail.gmail.com>
Subject: Re: Cluster Tuning
From: "Juan P." <gordoslocos@gmail.com>
To: common-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=00261887757e37de5a04a7d1a726

--00261887757e37de5a04a7d1a726
Content-Type: text/plain; charset=ISO-8859-1

Allen,
Say I were to bring the property back to the default of -Xmx200m, which
buffers do you think I should adjust? io.sort.mb? io.sort.factor? How would
you adjust them?

Thanks for your help!
Pony

On Mon, Jul 11, 2011 at 4:41 PM, Allen Wittenauer <aw@apache.org> wrote:

>
> On Jul 11, 2011, at 9:28 AM, Juan P. wrote:
> >
> > *  <property>*
> > *    <name>mapred.child.java.opts</name>*
> > *    <value>-Xmx400m</value>*
> > *  </property>*
>
> "Single core machines with 600MB of RAM."
>
>                 2x400m = 800m just for the heap of the map and reduce
> phases, not counting the other memory that the jvm will need.  io buffer
> sizes aren't adjusted downward either, so you're likely looking at a
> swapping + spills = death scenario.  slowstart set to 1 is going to be
> pretty much required.

--00261887757e37de5a04a7d1a726--