hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rob Blah <tmp5...@gmail.com>
Subject Re: Java Heap space Exception
Date Fri, 15 Nov 2013 13:24:02 GMT
MapReduce
http://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/
"We’ll thus assign 4 GB for Map task Containers, and 8 GB for Reduce tasks
Containers."

You can try editing configuration file:
<name>mapreduce.map.memory.mb</name>
<value>?</value>

<name>mapreduce.reduce.memory.mb</name>
 <value>?</value>

Or set these in a driver.

If you have custom AM, try this:
Driver:
capability.setMemory(amMemory);
appContext.setResource(capability);

AM:
Resource capability = Records.newRecord(Resource.class);
capability.setMemory(containerMemory);
AMRMClient.ContainerRequest request = new
AMRMClient.ContainerRequest(capability, null, null, pri);
OR AM (use java command to set heap size):
vargs.add("-Xmx" + containerMemory + "m");






2013/11/15 Rob Blah <tmp5330@gmail.com>

> Question have you tried giving more memory to the containers?
>
>
> 2013/11/15 unmesha sreeveni <unmeshabiju@gmail.com>
>
>> ys found
>> tried -D mapred.child.java.opts=-Xmx4096M on the command line:
>>
>>
>> On Fri, Nov 15, 2013 at 3:01 PM, unmesha sreeveni <unmeshabiju@gmail.com>wrote:
>>
>>> hadoop jar /home/my/hadoop2.jar /user/unmesha/inputdata /user/unmesha
>>> /out
>>>
>>>
>>>
>>> On Fri, Nov 15, 2013 at 2:53 PM, unmesha sreeveni <unmeshabiju@gmail.com
>>> > wrote:
>>>
>>>> When i tried to execute my program with 100 MB file i am getting
>>>> JavaHeapSpace Exception
>>>>
>>>>
>>>> >hadoop jar /home/my/hadoop2.jar /user/unmesha/inputdata
>>>> /user/inputdata/out
>>>>
>>>> How to increase the heap size through commandline
>>>>
>>>>
>>>> --
>>>> *Thanks & Regards*
>>>>
>>>> Unmesha Sreeveni U.B
>>>>
>>>> *Junior Developer*
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> *Thanks & Regards*
>>>
>>> Unmesha Sreeveni U.B
>>>
>>> *Junior Developer*
>>>
>>>
>>>
>>
>>
>> --
>> *Thanks & Regards*
>>
>> Unmesha Sreeveni U.B
>>
>> *Junior Developer*
>>
>>
>>
>

Mime
View raw message