hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amogh Vasekar <am...@yahoo-inc.com>
Subject Re: Problem with mapred.job.reuse.jvm.num.tasks
Date Mon, 30 Nov 2009 13:10:50 GMT
Hi,
Task slots reuse JVM over the course of entire job right? Specifically, would like to point
to :
http://issues.apache.org/jira/browse/MAPREDUCE-453?focusedCommentId=12619492&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12619492

Thanks,
Amogh

On 11/30/09 5:44 PM, "Chandraprakash Bhagtani" <cpbhagtani@gmail.com> wrote:

Hi,

In one of my hadoop jobs I had set mapred.job.reuse.jvm.num.tasks = -1. The
job was such that each map/reduce
task occupied around 2GB RAM. So by setting mapred.job.reuse.jvm.num.tasks =
-1, even when all the maps had
finished, JVMs were still alive. Though the JVMs were not using CPU, but
occupied RAM. Thus reducing overall available
RAM for reducer JVMs. All the JVMs were killed when the Job completed.

Is this a bug in hadoop? If yes, should I open a JIRA for that?

Please enlighten, Thanks in advance.

--
Thanks & Regards,
Chandra Prakash Bhagtani


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message