hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From t...@apache.org
Subject svn commit: r1079605 - in /hadoop/common/trunk: CHANGES.txt bin/hadoop-config.sh
Date Tue, 08 Mar 2011 23:51:23 GMT
Author: todd
Date: Tue Mar  8 23:51:23 2011
New Revision: 1079605

URL: http://svn.apache.org/viewvc?rev=1079605&view=rev
Log:
HADOOP-7154. Should set MALLOC_ARENA_MAX in hadoop-env.sh. Contributed by Todd Lipcon.

Modified:
    hadoop/common/trunk/CHANGES.txt
    hadoop/common/trunk/bin/hadoop-config.sh

Modified: hadoop/common/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/common/trunk/CHANGES.txt?rev=1079605&r1=1079604&r2=1079605&view=diff
==============================================================================
--- hadoop/common/trunk/CHANGES.txt (original)
+++ hadoop/common/trunk/CHANGES.txt Tue Mar  8 23:51:23 2011
@@ -326,6 +326,8 @@ Release 0.22.0 - Unreleased
 
     HADOOP-6970. SecurityAuth.audit should be generated under /build. (boryas)
 
+    HADOOP-7154. Should set MALLOC_ARENA_MAX in hadoop-env.sh (todd)
+
   OPTIMIZATIONS
 
     HADOOP-6884. Add LOG.isDebugEnabled() guard for each LOG.debug(..).

Modified: hadoop/common/trunk/bin/hadoop-config.sh
URL: http://svn.apache.org/viewvc/hadoop/common/trunk/bin/hadoop-config.sh?rev=1079605&r1=1079604&r2=1079605&view=diff
==============================================================================
--- hadoop/common/trunk/bin/hadoop-config.sh (original)
+++ hadoop/common/trunk/bin/hadoop-config.sh Tue Mar  8 23:51:23 2011
@@ -78,6 +78,11 @@ then
   exit 1
 fi
 
+# Newer versions of glibc use an arena memory allocator that causes virtual
+# memory usage to explode. This interacts badly with the many threads that
+# we use in Hadoop. Tune the variable down to prevent vmem explosion.
+export MALLOC_ARENA_MAX=${MALLOC_ARENA_MAX:-4}
+
 # some Java parameters
 if [ "$JAVA_HOME" != "" ]; then
   #echo "run java in $JAVA_HOME"



Mime
View raw message