hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chandraprakash Bhagtani <cpbhagt...@gmail.com>
Subject Problem with mapred.job.reuse.jvm.num.tasks
Date Mon, 30 Nov 2009 12:14:06 GMT
Hi,

In one of my hadoop jobs I had set mapred.job.reuse.jvm.num.tasks = -1. The
job was such that each map/reduce
task occupied around 2GB RAM. So by setting mapred.job.reuse.jvm.num.tasks =
-1, even when all the maps had
finished, JVMs were still alive. Though the JVMs were not using CPU, but
occupied RAM. Thus reducing overall available
RAM for reducer JVMs. All the JVMs were killed when the Job completed.

Is this a bug in hadoop? If yes, should I open a JIRA for that?

Please enlighten, Thanks in advance.

-- 
Thanks & Regards,
Chandra Prakash Bhagtani

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message