hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anoop Sam John <anoo...@huawei.com>
Subject RE: Region server Memory Use is double the -Xmx setting
Date Thu, 24 Jan 2013 03:21:47 GMT
Are  you using compression for HFiles?

Yes we are using  "MaxDirectMemorySize" and we dont use off-heap cache.

-Anoop-
________________________________________
From: Buckley,Ron [buckleyr@oclc.org]
Sent: Wednesday, January 23, 2013 8:49 PM
To: user@hbase.apache.org
Subject: RE: Region server Memory Use is double the -Xmx setting

Liang,

Thanks.  I wasn’t really aware that the direct memory could get that large. (Full disclosure,
we did switch from jdk1.6.0_25 to jdk1.6.0_31 the last time we restarted HBase.)

I've only seen explicit setting of -XX:MaxDirectMemorySize for regionservers associated with
the experimental off-heap cache.

Is anyone else running their region servers with -XX:MaxDirectMemorySize (not using the off-heap
cache)?

Ron

-----Original Message-----
From: 谢良 [mailto:xieliang@xiaomi.com]
Sent: Tuesday, January 22, 2013 9:20 PM
To: user@hbase.apache.org
Subject: 答复: Region server Memory Use is double the -Xmx setting

Please set "-XX:MaxDirectMemorySize" explicitly,  else the default is taking the value like
-Xmx in currenty JDK6, at least for jdk1.6.30+

Best Regards,
Liang
________________________________________
发件人: Buckley,Ron [buckleyr@oclc.org]
发送时间: 2013年1月23日 5:17
收件人: user@hbase.apache.org
主题: Region server Memory Use is double the -Xmx setting

We have a 50 node cluster replicating to a 6 node cluster. Both clusters
are running CDH4.1.2 and HBase 0.94.2.



Today we noticed that the region servers at our replica site are using
10GB more memory than the '-Xmx12288m' we have defined in hbase-env.sh



These region servers have been up since January 9, 2013.



Does anyone have suggestions about tracking down this additional memory
use?



I'm not necessarily expecting the Region Server to stay right at the
12GB that we allocated, but having it running at 24GB is starting to
cause the servers to swap.






PID    USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND


28544  prodcon   18   0 24.1g  23g  18m S 20.0 74.0   9071:34 java




28544:   /usr/java/jdk1.6.0_31/bin/java -XX:OnOutOfMemoryError=kill -9
%p -Xmx12288m -Dcom.sun.management.jmxr

emote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.port=9021 -ea

-server -XX:+HeapDumpOnOutOfMemoryError -Xmn256m -XX:+UseParNewGC
-XX:+UseConcMarkSweepGC -XX:CMSInitiatingOc

cupancyFraction=70 -verbose:gc -XX:+PrintGCDetails
-XX:+PrintGCDateStamps -Xloggc:/drive1/hadoop/2.0/isoft/../

hbase/logs/gc-hbase.log -ea -server -XX:+HeapDumpOnOutOfMemoryError
-Xmn256m -XX





----------------------------------------------------------------------

Ron Buckley

x6365

http://intranet-wiki.oclc.org/wiki/XWC/XServe/SRU



http://intranet-wiki.oclc.org/wiki/FIND



http://intranet-wiki.oclc.org/wiki/Higgins


http://intranet-wiki.oclc.org/wiki/Firefly
Mime
View raw message