hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: capacity - resource based scheduling
Date Wed, 18 Apr 2012 06:03:39 GMT
Corbin,

Have you got JVM reuse on? Those extra JVMs may just be from use of
that (there've been some issues lately with JVM reuse that may be the
cause).

CS does not launch up JVMs to fill up/reserve "slots".

On Wed, Apr 18, 2012 at 9:43 AM, Corbin Hoenes <corbin@tynt.com> wrote:
> I have a streaming job that uses a lot of memory.  Capacity scheduler
> lets me set the mapred.job.map.memory.mb property to something high
> like 2560.  The job then takes 5 slots (512mb slot) for every map
> task.  I have noticed that it appears to actually start many java
> processes that look like tasks (I've seen as many as 12) on my slaves
> to service my map tasks.  What are all the extra java processes for
> (are these considered slots?) they use memory but no cpu, are they
> just place holders?



-- 
Harsh J

Mime
View raw message