Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 573FF10794 for ; Tue, 15 Oct 2013 02:00:44 +0000 (UTC) Received: (qmail 91680 invoked by uid 500); 15 Oct 2013 02:00:36 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 91583 invoked by uid 500); 15 Oct 2013 02:00:36 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 91576 invoked by uid 99); 15 Oct 2013 02:00:35 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 15 Oct 2013 02:00:35 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of acm@hortonworks.com designates 209.85.160.54 as permitted sender) Received: from [209.85.160.54] (HELO mail-pb0-f54.google.com) (209.85.160.54) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 15 Oct 2013 02:00:29 +0000 Received: by mail-pb0-f54.google.com with SMTP id ro12so8103236pbb.41 for ; Mon, 14 Oct 2013 19:00:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:message-id:mime-version:subject:date :references:to:in-reply-to:content-type; bh=WeBbAmT6lDQrgUk6xqkNfMqRqDCbKeHYDKjOWsHxpCM=; b=PnuqyIAO0oRx64M4C+YqBzRnq9Ws1kiOe7T7Vpct9XMQJA+kF4V8cUHmPwJqah4KM8 lDGTEilLEh6qHK/lmOhheaGOsUKLRSGIhU+esRUmZzPWrByjgHHJhbdIQ0iZQGAxB1PK f5qg7Qg+YpLSU8Wvk3oWHz1IxuAj6c+RDkkW1Pg/L7jbIv43pox6mjeEIUwbsSWu4Qxf 9ejW+6r9MTQEVppvI3AN4CgRCNHJSqUmjNfXiNfnUjP5TtxLXNuCG2wl2hH6EY0JvhWM hl5/yy5fQjuOKiX9Ay7FFJDaRgr3TbYv7pTYXZIqyo9oyCrpuzFoXNygiEa2laOhz3xL Qv5Q== X-Gm-Message-State: ALoCoQlxwV4R2no2/RgZY9wo+zNFcQocE59AaE6PD2FGr82AGwLJlLh1bFtIzwSdZEc3CtjTOz660RhAUtoCZUeedTO3HFdzPNHkWsooxQnUJKr/DJs7Q/M= X-Received: by 10.66.26.112 with SMTP id k16mr40074251pag.65.1381802401736; Mon, 14 Oct 2013 19:00:01 -0700 (PDT) Received: from [10.0.1.25] (c-98-234-189-94.hsd1.ca.comcast.net. [98.234.189.94]) by mx.google.com with ESMTPSA id qz9sm80926724pbc.3.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 14 Oct 2013 18:59:57 -0700 (PDT) From: Arun C Murthy Message-Id: Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) Subject: Re: Hadoop Jobtracker cluster summary of heap size and OOME Date: Mon, 14 Oct 2013 18:59:58 -0700 References: To: user@hadoop.apache.org In-Reply-To: X-Mailer: Apple Mail (2.1510) Content-Type: multipart/alternative; boundary="Apple-Mail=_EDCED742-4421-49AC-9BF8-B3F3E30D203A" X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_EDCED742-4421-49AC-9BF8-B3F3E30D203A Content-Type: text/plain; charset=ISO-8859-1 Please don't cross-post. HADOOP_HEAPSIZE of 1024 is too low. You might want to bump it up to 16G or more, depending on: * #jobs * Scheduler you use. Arun On Oct 11, 2013, at 9:58 AM, Viswanathan J wrote: > Hi, > > I'm running a 14 nodes Hadoop cluster with tasktrackers running in all nodes. > > Have set the jobtracker default memory size in hadoop-env.sh > > HADOOP_HEAPSIZE="1024" > > Have set the mapred.child.java.opts value in mapred-site.xml as, > > > mapred.child.java.opts > -Xmx2048m > > > -- > Regards, > Viswa.J -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. --Apple-Mail=_EDCED742-4421-49AC-9BF8-B3F3E30D203A Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 Please don't cross-post= .

HADOOP_HEAPSIZE of 1024 is too low. You might want to = bump it up to 16G or more, depending on:
* #jobs
* Sche= duler you use.

Arun

On Oct= 11, 2013, at 9:58 AM, Viswanathan J <jayamviswanathan@gmail.com> wrote:

Hi,
=
I'm running a 14 nodes Hadoop cluster with tasktrackers running in= all nodes.

Have set the jobtracker default me= mory size in hadoop-env.sh

HADOOP_HEAPSIZE=3D"1024"

Ha= ve set the mapred.child.java.opts value in mapred-site.xml as,
<property>
  <name>mapred.chil= d.java.opts</name>
<value>-Xmx2048m</value>


--
Regards,
Viswa.J

--
Arun C. Murthy


CONFIDENTIALITY NOTICE
NOTICE: This message is = intended for the use of the individual or entity to which it is addressed a= nd may contain information that is confidential, privileged and exempt from= disclosure under applicable law. If the reader of this message is not the = intended recipient, you are hereby notified that any printing, copying, dis= semination, distribution, disclosure or forwarding of this communication is= strictly prohibited. If you have received this communication in error, ple= ase contact the sender immediately and delete it from your system. Thank Yo= u. --Apple-Mail=_EDCED742-4421-49AC-9BF8-B3F3E30D203A--