hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Venner <ja...@attributor.com>
Subject Re: Why is the task run in a child JVM?
Date Mon, 14 Jul 2008 13:37:31 GMT
One benefit is that if your map or reduce behaves badly it can't take 
down the task tracker.

In our case we have some poorly behaved external native libraries we 
use, and we have to forcibly ensure that the child vms are killed when 
the child main finishes (often by kill -9), so the fact the child (task) 
is a separate jvm process is very helpful.

The downside is the jvm start time. Has anyone experimented with the jar 
freezing for more than the standard boot class path jars to speed up 

Shengkai Zhu wrote:
> What's the benefits from such design compared to multi-thread?
Jason Venner
Attributor - Program the Web <http://www.attributor.com/>
Attributor is hiring Hadoop Wranglers and coding wizards, contact if 

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message