hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Milind Bhandarkar <mili...@yahoo-inc.com>
Subject Re: Running tasks in the TaskTracker VM
Date Mon, 19 Mar 2007 18:20:12 GMT
> Yes, but the issue remains present if you have to deal with a high
> number of map tasks to distribute the load on many machines.  
> Launching a
> JVM is costly, let's say it costs 1 second (i'm optimistic) , if you
> have to do 2000 map, there will be 2000 seconds lost in launching  
> JVMs...

Executing users' code in system daemons is a security risk. In my  
experience, security always wins in when pitted against performance.  
IMHO, there is a happy middle ground, i.e. to maintain a pool of  
running JVMs that are launched when the tasktracker starts up. Even  
then, care has to be taken against memory leaks etc.

- Milind

Milind Bhandarkar
(phone: 408-349-2136 W)

View raw message