hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bejoy KS" <bejoy...@yahoo.com>
Subject Re: about jvm reuse
Date Fri, 28 Sep 2012 20:09:17 GMT
Hi

Jvm reuse doesn't give you too much of a performance improvement. When you enable jvm reuse,
n number of tasks in the same task tracker use the same jvm rather than spinning up its own.
So the time savings is on avoiding the multiple jvm creations which would be just a few milli
seconds or seconds.

If a task somehow messes up with the jvm and crashes it, all other tasks on the same jvm fails,
that is the negative side.

But jvm reuse and free slots not utilized is not related I guess. If you are using any schedulers
you need to work in that direction to investigate why empty slots are not being used.

Regards
Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: researcher qiao <qiaoresearcher@gmail.com>
Date: Fri, 28 Sep 2012 14:48:51 
To: <user@hive.apache.org>
Reply-To: user@hive.apache.org
Subject: about jvm reuse

deal all,

i was running hive on hadoop. we noticed that there were maps slots
available even though you had many things queued up. so in the hive
interface, before run some query, i input:

set mapred.job.reuse.jvm.num.tasks = 50

i was hoping that this will improve the performance, but it doesnot and
there are many fails, any thoughts?


thanks,

Mime
View raw message