hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andy Srine <andy.sr...@gmail.com>
Subject Re: JVM option
Date Fri, 18 Apr 2014 19:20:52 GMT
I guess then the question is, how do I set the (system?) property and what
property exactly is this?

"export HADOOP_HEAPSIZE=4096" - Works from the environment.

And none of the following seem to work from the command line.
-Xmx4G

-Dmapred.child.java.opts=-Xmx4G

-DHADOOP_HEAPSIZE=4096

-DHADOOP_OPTS=-Xmx4G


I tried this from Java code as well and no luck.

System.setProperty("HADOOP_HEAPSIZE", "4096");


What am I missing?


Thanks,

Andy




On Fri, Apr 18, 2014 at 11:28 AM, Shahab Yunus <shahab.yunus@gmail.com>wrote:

> You can pass hadoop conf properties through the -D option. Have you seen
> this?
>
> http://stackoverflow.com/questions/15490090/how-to-specify-system-property-in-hadoop-except-modify-hadoop-env-sh
>
> This is not for system properties. The assumption is that you want to
> specify hadoop conf property through the command line.
>
> Regards,
> Shahab
>
>
> On Fri, Apr 18, 2014 at 2:12 PM, Andy Srine <andy.srine@gmail.com> wrote:
>
>> Thanks for the tip. May be it's a different setting I need. Let me
>> explain the problem.
>>
>> I get an heap error on the command line - "Exception in thread "main"
>> java.lang.OutOfMemoryError: Java heap space". It woks when I set "export
>> HADOOP_HEAPSIZE=4096" in my env. But how can I set it via the hadoop
>> command line (hadoop jar ...)? Next choice would be to do it
>> programmatically in Java using hadoop conf. I can't really change the XML
>> files across the cluster.
>>
>> Regards,
>> Andy
>>
>>
>> On Thu, Apr 17, 2014 at 6:39 PM, Azuryy Yu <azuryyyu@gmail.com> wrote:
>>
>>> Do you want add "-Xmx4g" to your MR tasks?  if so, just add it as
>>> "mapred.child.java.opts" in the mapred-site.xml
>>>
>>>
>>> On Fri, Apr 18, 2014 at 9:35 AM, Andy Srine <andy.srine@gmail.com>wrote:
>>>
>>>> Quick question. How would I pass the following JVM option to the Hadoop
>>>> command line?
>>>>
>>>> "-Xmx4G"
>>>>
>>>> hadoop jar <jar file> <class_name> <args>
>>>>
>>>> Thanks,
>>>> Andy
>>>>
>>>>
>>>
>>
>>
>> --
>> Thanks,
>> Andy
>>
>>
>


-- 
Thanks,
Andy

Mime
View raw message