hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chandraprakash Bhagtani <cpbhagt...@gmail.com>
Subject Re: Problem with mapred.job.reuse.jvm.num.tasks
Date Mon, 30 Nov 2009 13:45:58 GMT
Thanks for the reply Amogh.

Yes, my issue is similar to the pointer you have given. I just want to say
that as soon as the map tasks finish,
all map task JVMs should be killed since they are no more required.

On Mon, Nov 30, 2009 at 6:40 PM, Amogh Vasekar <amogh@yahoo-inc.com> wrote:

> Hi,
> Task slots reuse JVM over the course of entire job right? Specifically,
> would like to point to :
>
> http://issues.apache.org/jira/browse/MAPREDUCE-453?focusedCommentId=12619492&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12619492
>
> Thanks,
> Amogh
>
> On 11/30/09 5:44 PM, "Chandraprakash Bhagtani" <cpbhagtani@gmail.com>
> wrote:
>
> Hi,
>
> In one of my hadoop jobs I had set mapred.job.reuse.jvm.num.tasks = -1. The
> job was such that each map/reduce
> task occupied around 2GB RAM. So by setting mapred.job.reuse.jvm.num.tasks
> =
> -1, even when all the maps had
> finished, JVMs were still alive. Though the JVMs were not using CPU, but
> occupied RAM. Thus reducing overall available
> RAM for reducer JVMs. All the JVMs were killed when the Job completed.
>
> Is this a bug in hadoop? If yes, should I open a JIRA for that?
>
> Please enlighten, Thanks in advance.
>
> --
> Thanks & Regards,
> Chandra Prakash Bhagtani
>
>


-- 
Thanks & Regards,
Chandra Prakash Bhagtani,

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message