hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John <johnnyenglish...@gmail.com>
Subject Re: only one map or reduce job per time on one node
Date Wed, 06 Nov 2013 10:09:23 GMT
Does no one have an idea if it is possible? In my example the heapsize of
the tasktracker is 2gb. The max map heap size is 1gb and the mac heap size
of the reduce process 1,5gb. If there are running 2 maps or just the reduce
job everything is fine. But if there are running 2 maps + 1 reduce process
I got a "Error: java.lang.OutOfMemoryError: Java heap space at" message.

kind regards

2013/11/5 John <johnnyenglish739@gmail.com>

> Because my node swaps the memory if the 2 map slots + 1 reduce is occupied
> with my job. Sure I can minimize the max memory for the map/reduce process.
> I tried this already, but I got a out of memory exception if set the max
> heap size for the map/reduce process to low for my mr job.
> kind regards
> 2013/11/5 Vinod Kumar Vavilapalli <vinodkv@hortonworks.com>
>> Why do you want to do this?
>> +Vinod
>> On Nov 5, 2013, at 9:17 AM, John wrote:
>> Is it possible to force the jobtracker executing only 2 map jobs or 1
>> reduce job per time?
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.

View raw message