hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Allen Wittenauer ...@apache.org>
Subject Re: Cluster Tuning
Date Mon, 11 Jul 2011 19:41:39 GMT

On Jul 11, 2011, at 9:28 AM, Juan P. wrote:
> 
> *  <property>*
> *    <name>mapred.child.java.opts</name>*
> *    <value>-Xmx400m</value>*
> *  </property>*

"Single core machines with 600MB of RAM."

		2x400m = 800m just for the heap of the map and reduce phases, not counting the other memory
that the jvm will need.  io buffer sizes aren't adjusted downward either, so you're likely
looking at a swapping + spills = death scenario.  slowstart set to 1 is going to be pretty
much required.
Mime
View raw message