Author: mattf
Date: Tue Jul 24 07:51:27 2012
New Revision: 1364929
URL: http://svn.apache.org/viewvc?rev=1364929&view=rev
Log:
HADOOP-7154. Set MALLOC_ARENA_MAX in hadoop-env.sh. Contributed by Todd Lipcon.
Modified:
hadoop/common/branches/branch-1.0/CHANGES.txt
hadoop/common/branches/branch-1.0/bin/hadoop-config.sh
Modified: hadoop/common/branches/branch-1.0/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-1.0/CHANGES.txt?rev=1364929&r1=1364928&r2=1364929&view=diff
==============================================================================
--- hadoop/common/branches/branch-1.0/CHANGES.txt (original)
+++ hadoop/common/branches/branch-1.0/CHANGES.txt Tue Jul 24 07:51:27 2012
@@ -6,6 +6,9 @@ Release 1.0.4 - Unreleased
IMPROVEMENTS
+ HADOOP-7154. Should set MALLOC_ARENA_MAX in hadoop-env.sh
+ (todd via mattf)
+
BUG FIXES
HDFS-3652. FSEditLog failure removes the wrong edit stream when storage
Modified: hadoop/common/branches/branch-1.0/bin/hadoop-config.sh
URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-1.0/bin/hadoop-config.sh?rev=1364929&r1=1364928&r2=1364929&view=diff
==============================================================================
--- hadoop/common/branches/branch-1.0/bin/hadoop-config.sh (original)
+++ hadoop/common/branches/branch-1.0/bin/hadoop-config.sh Tue Jul 24 07:51:27 2012
@@ -75,6 +75,11 @@ if [ "$HADOOP_HOME_WARN_SUPPRESS" = "" ]
echo 1>&2
fi
+# Newer versions of glibc use an arena memory allocator that causes virtual
+# memory usage to explode. This interacts badly with the many threads that
+# we use in Hadoop. Tune the variable down to prevent vmem explosion.
+export MALLOC_ARENA_MAX=${MALLOC_ARENA_MAX:-4}
+
export HADOOP_HOME=${HADOOP_PREFIX}
export HADOOP_HOME_WARN_SUPPRESS=1
|