Return-Path: 
 <user-return-11863-apmail-hadoop-user-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 8B039105D1
	for <apmail-hadoop-user-archive@minotaur.apache.org>;
 Fri, 15 Nov 2013 13:24:38 +0000 (UTC)
Received: (qmail 29188 invoked by uid 500); 15 Nov 2013 13:24:29 -0000
Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org
Received: (qmail 29050 invoked by uid 500); 15 Nov 2013 13:24:28 -0000
Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:user@hadoop.apache.org>
List-Id: <user.hadoop.apache.org>
Reply-To: user@hadoop.apache.org
Delivered-To: mailing list user@hadoop.apache.org
Received: (qmail 29043 invoked by uid 99); 15 Nov 2013 13:24:28 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 15 Nov 2013 13:24:28 +0000
X-ASF-Spam-Status: No, hits=2.7 required=5.0
	tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (athena.apache.org: domain of tmp5330@gmail.com designates
 209.85.215.44 as permitted sender)
Received: from [209.85.215.44] (HELO mail-la0-f44.google.com) (209.85.215.44)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 15 Nov 2013 13:24:23 +0000
Received: by mail-la0-f44.google.com with SMTP id ep20so2796192lab.31
        for <user@hadoop.apache.org>; Fri, 15 Nov 2013 05:24:02 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        bh=/or4Mqtn7/6/oCHuOcxBtwNW6Z81tEZbWcRDPGwwFvE=;
        b=U+R2fJkU96mEr7z+xOOLUFGT1Rknc4FYityKWbEdmCoONZMGYzDN9YI6JXQksPq+Db
         q4Z1izZ17aIeMvAamwYEF5zVkRmrLELxwiArv4HjGTIDdLbwy62CrdPDTzlYvdVAC5cV
         OZTzTVE0PA0bMb3ebwyZUEP2kZj242p3a2wR7hLXpgkdPR148GZl3OgxzeRPT1zU20F2
         mI50MF+saxS2Fbpbj1UdIbLjH1lMTfvvd+BO35OLgBh/yMouCaWiFaVblPm+heuw54/M
         koVGi56qlQOJkiLJkHv2V7H33PcOWmfmhCiKgbIQGd2/ge4DfDHV9euuaXtc4GpmX9+Z
         EHdg==
MIME-Version: 1.0
X-Received: by 10.112.159.231 with SMTP id xf7mr4067323lbb.18.1384521842231;
 Fri, 15 Nov 2013 05:24:02 -0800 (PST)
Received: by 10.114.18.14 with HTTP; Fri, 15 Nov 2013 05:24:02 -0800 (PST)
In-Reply-To: 
 <CAFwH+a_UF43f_TSx4uRpTUxqOBR8ombHT8DCYDfmTQ3hprn4uQ@mail.gmail.com>
References: 
 <CACp0qUEX1SYbgTDp_dj-k8bko6=fGxB3imCsnfz8V7uy=R8pjg@mail.gmail.com>
	<CACp0qUHhyyWcavprV8xoobYqTKn1q_cYbXbXhn1_goSZfBMCjQ@mail.gmail.com>
	<CACp0qUF-Rs9GqppqHBtUksHuJX+kavOtcqMF7qO08Na7N+H4LQ@mail.gmail.com>
	<CAFwH+a_UF43f_TSx4uRpTUxqOBR8ombHT8DCYDfmTQ3hprn4uQ@mail.gmail.com>
Date: Fri, 15 Nov 2013 14:24:02 +0100
Message-ID: 
 <CAFwH+a8eKxBaiJ8xDYWJwCQo_wDxhEy+yCNgaOA5mNY4wQaDbg@mail.gmail.com>
Subject: Re: Java Heap space Exception
From: Rob Blah <tmp5330@gmail.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=001a11c3c4fe97115404eb371c0d
X-Virus-Checked: Checked by ClamAV on apache.org

--001a11c3c4fe97115404eb371c0d
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

MapReduce
http://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/
"We=92ll thus assign 4 GB for Map task Containers, and 8 GB for Reduce task=
s
Containers."

You can try editing configuration file:
<name>mapreduce.map.memory.mb</name>
<value>?</value>

<name>mapreduce.reduce.memory.mb</name>
 <value>?</value>

Or set these in a driver.

If you have custom AM, try this:
Driver:
capability.setMemory(amMemory);
appContext.setResource(capability);

AM:
Resource capability =3D Records.newRecord(Resource.class);
capability.setMemory(containerMemory);
AMRMClient.ContainerRequest request =3D new
AMRMClient.ContainerRequest(capability, null, null, pri);
OR AM (use java command to set heap size):
vargs.add("-Xmx" + containerMemory + "m");






2013/11/15 Rob Blah <tmp5330@gmail.com>

> Question have you tried giving more memory to the containers?
>
>
> 2013/11/15 unmesha sreeveni <unmeshabiju@gmail.com>
>
>> ys found
>> tried -D mapred.child.java.opts=3D-Xmx4096M on the command line:
>>
>>
>> On Fri, Nov 15, 2013 at 3:01 PM, unmesha sreeveni <unmeshabiju@gmail.com=
>wrote:
>>
>>> hadoop jar /home/my/hadoop2.jar /user/unmesha/inputdata /user/unmesha
>>> /out
>>>
>>>
>>>
>>> On Fri, Nov 15, 2013 at 2:53 PM, unmesha sreeveni <unmeshabiju@gmail.co=
m
>>> > wrote:
>>>
>>>> When i tried to execute my program with 100 MB file i am getting
>>>> JavaHeapSpace Exception
>>>>
>>>>
>>>> >hadoop jar /home/my/hadoop2.jar /user/unmesha/inputdata
>>>> /user/inputdata/out
>>>>
>>>> How to increase the heap size through commandline
>>>>
>>>>
>>>> --
>>>> *Thanks & Regards*
>>>>
>>>> Unmesha Sreeveni U.B
>>>>
>>>> *Junior Developer*
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> *Thanks & Regards*
>>>
>>> Unmesha Sreeveni U.B
>>>
>>> *Junior Developer*
>>>
>>>
>>>
>>
>>
>> --
>> *Thanks & Regards*
>>
>> Unmesha Sreeveni U.B
>>
>> *Junior Developer*
>>
>>
>>
>

--001a11c3c4fe97115404eb371c0d
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><div><div><div><div>MapReduce<br><a href=3D"http://ho=
rtonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/">http://horto=
nworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/</a><br>&quot;We=
=92ll thus assign 4 GB for Map task Containers, and 8 GB for Reduce tasks C=
ontainers.&quot;<br>
<br>You can try editing configuration file:<br>&lt;name&gt;mapreduce.map.me=
mory.mb&lt;/name&gt; <br>&lt;value&gt;?&lt;/value&gt; <br><br>&lt;name&gt;m=
apreduce.reduce.memory.mb&lt;/name&gt;<br>=A0&lt;value&gt;?&lt;/value&gt;<b=
r>
<br></div>Or set these in a driver.<br><br></div>If you have custom AM, try=
 this:<br></div>Driver:<br>capability.setMemory(amMemory);<br>appContext.se=
tResource(capability);<br></div><br>AM:<br>Resource capability =3D Records.=
newRecord(Resource.class);<br>
capability.setMemory(containerMemory);<br>AMRMClient.ContainerRequest reque=
st =3D new AMRMClient.ContainerRequest(capability, null, null, pri);<br></d=
iv>OR AM (use java command to set heap size):<br>vargs.add(&quot;-Xmx&quot;=
 + containerMemory + &quot;m&quot;);<br>
<br><div><div><div><div><div><br><br><br></div></div></div></div></div></di=
v><div class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">2013/11/15 =
Rob Blah <span dir=3D"ltr">&lt;<a href=3D"mailto:tmp5330@gmail.com" target=
=3D"_blank">tmp5330@gmail.com</a>&gt;</span><br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"ltr">Question have you tried giv=
ing more memory to the containers?<br></div><div class=3D"HOEnZb"><div clas=
s=3D"h5">
<div class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">2013/11/15 un=
mesha sreeveni <span dir=3D"ltr">&lt;<a href=3D"mailto:unmeshabiju@gmail.co=
m" target=3D"_blank">unmeshabiju@gmail.com</a>&gt;</span><br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div class=3D"gmail_default=
" style=3D"font-family:verdana,sans-serif">ys found=A0</div><div class=3D"g=
mail_default" style=3D"font-family:verdana,sans-serif">

tried<span style=3D"white-space:pre-wrap;font-family:arial"> -D mapred.chil=
d.java.opts=3D-Xmx4096M on the command line:</span></div>
</div><div><div><div class=3D"gmail_extra"><br><br><div class=3D"gmail_quot=
e">On Fri, Nov 15, 2013 at 3:01 PM, unmesha sreeveni <span dir=3D"ltr">&lt;=
<a href=3D"mailto:unmeshabiju@gmail.com" target=3D"_blank">unmeshabiju@gmai=
l.com</a>&gt;</span> wrote:<br>


<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div class=3D"gmail_default=
" style=3D"font-family:verdana,sans-serif"><div class=3D"gmail_default" sty=
le=3D"font-family:arial,sans-serif;font-size:13px">


<font face=3D"verdana, sans-serif">hadoop jar /home/my/hadoop2.jar /user/un=
mesha/inputdata /user/</font><span style=3D"font-family:verdana,sans-serif"=
>unmesha</span><font face=3D"verdana, sans-serif">/out</font></div>
<div><font face=3D"verdana, sans-serif"><br></font></div></div></div><div><=
div><div class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">On Fri, N=
ov 15, 2013 at 2:53 PM, unmesha sreeveni <span dir=3D"ltr">&lt;<a href=3D"m=
ailto:unmeshabiju@gmail.com" target=3D"_blank">unmeshabiju@gmail.com</a>&gt=
;</span> wrote:<br>



<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div class=3D"gmail_default=
" style=3D"font-family:verdana,sans-serif">When i tried to execute my progr=
am with 100 MB file i am getting JavaHeapSpace Exception</div>



<div class=3D"gmail_default" style=3D"font-family:verdana,sans-serif">
<br></div><div class=3D"gmail_default" style=3D"font-family:verdana,sans-se=
rif"><br></div><div class=3D"gmail_default"><span style=3D"font-family:verd=
ana,sans-serif">&gt;</span><font face=3D"verdana, sans-serif">hadoop jar /h=
ome/my/hadoop2.jar /user/unmesha/inputdata /user/inputdata/out</font></div>




<div class=3D"gmail_default"><font face=3D"verdana, sans-serif"><br></font>=
</div><div class=3D"gmail_default"><font face=3D"verdana, sans-serif">How t=
o=A0increase=A0the heap size through commandline</font></div><span><font co=
lor=3D"#888888"><div>



<br></div><div><br>
</div>-- <br><div dir=3D"ltr"><i style=3D"color:rgb(102,0,0)">Thanks &amp; =
Regards</i>
<div style=3D"color:rgb(102,0,0)"><i><br></i></div><div style=3D"color:rgb(=
102,0,0)">Unmesha Sreeveni U.B<i><br></i></div><div style=3D"color:rgb(102,=
0,0)"><i>Junior Developer<br></i></div><div style=3D"color:rgb(102,0,0)"><b=
r></div>




<i><span><br></span></i></div>
</font></span></div>
</blockquote></div><br><br clear=3D"all"><div><br></div>-- <br><div dir=3D"=
ltr"><i style=3D"color:rgb(102,0,0)">Thanks &amp; Regards</i>
<div style=3D"color:rgb(102,0,0)"><i><br></i></div><div style=3D"color:rgb(=
102,0,0)">Unmesha Sreeveni U.B<i><br></i></div><div style=3D"color:rgb(102,=
0,0)"><i>Junior Developer<br></i></div><div style=3D"color:rgb(102,0,0)"><b=
r></div>



<i><span><br></span></i></div>
</div>
</div></div></blockquote></div><br><br clear=3D"all"><div><br></div>-- <br>=
<div dir=3D"ltr"><i style=3D"color:rgb(102,0,0)">Thanks &amp; Regards</i>
<div style=3D"color:rgb(102,0,0)"><i><br></i></div><div style=3D"color:rgb(=
102,0,0)">Unmesha Sreeveni U.B<i><br></i></div><div style=3D"color:rgb(102,=
0,0)"><i>Junior Developer<br></i></div><div style=3D"color:rgb(102,0,0)"><b=
r></div>


<i><span><br></span></i></div>
</div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>

--001a11c3c4fe97115404eb371c0d--