hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Robertson <timrobertson...@gmail.com>
Subject Re: JVM Reuse for parallel execution
Date Tue, 11 May 2010 14:23:14 GMT
http://hadoop.apache.org/common/docs/current/mapred_tutorial.html#Task+JVM+Reuse?

(Just so you know - there is a mapreduce users mailing list for MR specific
questions)

Hope this helps,
Tim

On Tue, May 11, 2010 at 4:02 PM, Rohan Rai <rohan.rai@inmobi.com> wrote:

> Forgot to add, we are using the following configuration.
>
> mapred.tasktracker.map.tasks.maximum as 4
> and mapred.tasktracker.reduce.tasks.maximum as 3.
>
> Regards
> Rohan
>
>
>
> Rohan Rai wrote:
>
>> Hi
>>
>> Is there a way of executing multiple tasks in parallel in a single JVM.
>>
>> For a particular job we see a child process being spawned for each
>> task, despite
>> mapred.job.reuse.jvm.num.tasks being set to -1
>>
>> In order to avoid the overhead of starting a JVM for each task and yet
>> achieve
>> parallel execution of tasks, wanted to know sharing the JVM is
>> possible, for
>> parallel task execution.
>>
>> Regards
>> Rohan
>>
>>
>
> The information contained in this communication is intended solely for the
> use of the individual or entity to whom it is addressed and others
> authorized to receive it. It may contain confidential or legally privileged
> information. If you are not the intended recipient you are hereby notified
> that any disclosure, copying, distribution or taking any action in reliance
> on the contents of this information is strictly prohibited and may be
> unlawful. If you have received this communication in error, please notify us
> immediately by responding to this email and then delete it from your system.
> The firm is neither liable for the proper and complete transmission of the
> information contained in this communication nor for any delay in its
> receipt.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message