hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jothi Padmanabhan <joth...@yahoo-inc.com>
Subject Re: question about when shuffle/sort start working
Date Tue, 02 Jun 2009 07:58:11 GMT
JVM reuse can be enabled by

conf.setLong("mapred.job.reuse.jvm.num.tasks", -1);

However, I do not think you can reuse variables/objects from the previous
task that ran in the JVM.


On 6/2/09 12:01 PM, "Jianmin Woo" <jianmin_woo@yahoo.com> wrote:

> It's cool to address the job overhead in JVM reuse way. You mentioned that the
> code(and variables?) are kept in JIT cache, so can we refer the variables in
> the last iteration in some way? I am really caurious how this works in coding
> level. Could you please help to point out some material/resources I can start
> to get to know the usage of this feature? Is this implemented in the current
> 0.20 version? I am pretty fresh to haoop. :-)

View raw message