hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <qwertyman...@gmail.com>
Subject Re: jobtracker heap
Date Thu, 17 Mar 2011 18:17:59 GMT
Nope, AFAIK 0.20.x does not have properties that can apply JVM options
separately to map and reduce tasks. "mapred.child.java.opts", which
applies to both, is the only go [mapred-site.xml with a final, or set
it specifically in jobs].

On Thu, Mar 17, 2011 at 10:59 PM, Lior Schachter <liors@infolinks.com> wrote:
> we are using hbase hadoop version 0.20.3
>
> maybe I can do it using the jobconf (separate jvm opts for map/reduce) ?
>
> On Thu, Mar 17, 2011 at 7:23 PM, Harsh J <qwertymaniac@gmail.com> wrote:
>>
>> What version of Hadoop are you using?
>>
>> 1 is a new property in 0.21, and is the right one if you're using that.
>> 2 is the property to use in 0.20, and applies to both Maps and
>> Reduces. In 0.20 there is no distinction available for setting JVM
>> opts to types of tasks.
>>
>> On Thu, Mar 17, 2011 at 10:37 PM, Lior Schachter <liors@infolinks.com>
>> wrote:
>> > sure. my mistake. thanks.
>> >
>> > I'm also trying to increase the reducer heap size (keeping the map with
>> > the
>> > default 200m). I found 2 related properties:
>> > 1. mapreduce.reduce.java.opts
>> > 2. mapred.child.java.opts
>> >
>> > Which should be configured and whats the purpose of the second property.
>> >
>> > thanks,
>> > Lior
>> >
>> > On Thu, Mar 17, 2011 at 6:41 PM, Harsh J <qwertymaniac@gmail.com> wrote:
>> >>
>> >> Hello,
>> >>
>> >> Remove the "=" from "-Xmx=2048m".
>> >>
>> >> P.s. If you're looking to change the JobTracker opts, it is
>> >> HADOOP_JOBTRACKER_OPTS, not TASKTRACKER :)
>> >>
>> >> On Thu, Mar 17, 2011 at 10:08 PM, Lior Schachter <liors@infolinks.com>
>> >> wrote:
>> >> > Hi,
>> >> > I'm trying to increase the job-tracker heap by adding Xmx definition
>> >> > to
>> >> > hadoop-env.sh (adding it to HADOOP_TASKTRACKER_OPTS).
>> >> > I keep getting this error while starting hadoop:
>> >> > Invalid maximum heap size: -Xmx=2048m
>> >> > Could not create the Java virtual machine.
>> >> >
>> >> > Any idea what am I doing wrong ?
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Harsh J
>> >> http://harshj.com
>> >
>> >
>>
>>
>>
>> --
>> Harsh J
>> http://harshj.com
>
>



-- 
Harsh J
http://harshj.com

Mime
View raw message