cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "MaHaiyang (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-3537) JVM heap is full and GC take no effect ,so many NonBlockingHashMap objects in heap
Date Wed, 07 Dec 2011 03:39:40 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-3537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164112#comment-13164112
] 

MaHaiyang commented on CASSANDRA-3537:
--------------------------------------

Thanks very much . Helps a lot .
                
> JVM heap is full and  GC take no effect ,so many NonBlockingHashMap objects in heap 
> ------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-3537
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3537
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 1.0.0
>         Environment: 当前堆大小: 
> 5,815,955 Kb
> 堆大小的最大值: 
> 6,045,696 Kb
> 分配的内存: 
> 6,045,696 Kb
> 暂挂结束操作: 
> 0 个对象
> 垃圾收集器: 
> Name = 'ParNew', Collections = 3,294, Total time spent = 2 minutes
> 垃圾收集器: 
> Name = 'ConcurrentMarkSweep', Collections = 5,909, Total time spent = 2 hours 17 minutes
>  
> 操作系统: 
> Linux 2.6.32.12-0.7-default
> 体系结构: 
> amd64
> 处理器的数目: 
> 16
> 分配的虚拟内存: 
> 42,748,416 Kb
> 物理内存总量: 
> 24,568,836 Kb
> 可用物理内存: 
>  7,136,380 Kb
> 交换空间总量: 
>  2,104,472 Kb
> 可用交换空间: 
>  1,970,800 Kb
>  
> VM 参数: 
> -ea -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms6G -Xmx6G -Xmn2400M -XX:+HeapDumpOnOutOfMemoryError
-Xss128k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8
-XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly
-Djava.net.preferIPv4Stack=true -Dcom.sun.management.jmxremote.port=9000 -Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false -Dpasswd.properties=/opt/obs/cassandra/conf/passwd.properties
-Dpasswd.mode=MD5 -Dlog4j.configuration=log4j-server.properties -Dlog4j.defaultInitOverride=true

> 类路径: 
> /opt/obs/cassandra/conf:/opt/obs/cassandra/build/classes/main:/opt/obs/cassandra/build/classes/thrift:/opt/obs/cassandra/lib/antlr-3.2.jar:/opt/obs/cassandra/lib/apache-cassandra-1.0.0.jar:/opt/obs/cassandra/lib/apache-cassandra-clientutil-1.0.0.jar:/opt/obs/cassandra/lib/apache-cassandra-thrift-1.0.0.jar:/opt/obs/cassandra/lib/avro-1.4.0-fixes.jar:/opt/obs/cassandra/lib/avro-1.4.0-sources-fixes.jar:/opt/obs/cassandra/lib/cassandra_simple_authentication.jar:/opt/obs/cassandra/lib/commons-cli-1.1.jar:/opt/obs/cassandra/lib/commons-codec-1.2.jar:/opt/obs/cassandra/lib/commons-lang-2.4.jar:/opt/obs/cassandra/lib/compress-lzf-0.8.4.jar:/opt/obs/cassandra/lib/concurrentlinkedhashmap-lru-1.2.jar:/opt/obs/cassandra/lib/guava-r08.jar:/opt/obs/cassandra/lib/high-scale-lib-1.1.2.jar:/opt/obs/cassandra/lib/jackson-core-asl-1.4.0.jar:/opt/obs/cassandra/lib/jackson-mapper-asl-1.4.0.jar:/opt/obs/cassandra/lib/jamm-0.2.5.jar:/opt/obs/cassandra/lib/jline-0.9.94.jar:/opt/obs/cassandra/lib/json-simple-1.1.jar:/opt/obs/cassandra/lib/libthrift-0.6.jar:/opt/obs/cassandra/lib/log4j-1.2.16.jar:/opt/obs/cassandra/lib/servlet-api-2.5-20081211.jar:/opt/obs/cassandra/lib/slf4j-api-1.6.1.jar:/opt/obs/cassandra/lib/slf4j-log4j12-1.6.1.jar:/opt/obs/cassandra/lib/snakeyaml-1.6.jar:/opt/obs/cassandra/lib/snappy-java-1.0.3.jar
>            Reporter: MaHaiyang
>
> I have 4 cassandra nodes ,and I put about 30G data to db for every nodes . It's just
4 days before I start the cluster ,but now every 4 nodes have the same problem ,JVM heap is
full  ,and  GC take no effect ,There must be some memory leak . Jmap the memory as follow:
> Object Histogram:
> num 	  #instances	#bytes	Class description
> --------------------------------------------------------------------------
> 1:		15793606	758093088	java.nio.HeapByteBuffer
> 2:		2153811	320138208	java.lang.Object[]
> 3:		6163192	197222144	org.apache.cassandra.db.Column
> 4:		2543836	175890256	int[]
> 5:		2168816	155397192	long[]
> 6:		2078123	116374888	org.cliffc.high_scale_lib.ConcurrentAutoTable$CAT
> 7:		1847111	73884440	java.math.BigInteger
> 8:		1234243	59243664	java.util.Hashtable
> 9:		1770829	58233000	char[]
> 10:		1770627	56660064	java.lang.String
> 11:		1665886	39981264	org.apache.cassandra.db.DecoratedKey
> 12:		692706	38791536	org.cliffc.high_scale_lib.NonBlockingHashMap$CHM
> 13:		1234274	37172088	java.util.Hashtable$Entry[]
> 14:		1133541	36273312	java.net.Inet4Address
> 15:		738528	35449344	org.apache.cassandra.service.ReadCallback
> 16:		2078118	33249888	org.cliffc.high_scale_lib.Counter
> 17:		1373886	32973264	org.apache.cassandra.db.ReadResponse
> 18:		1234023	29616552	org.apache.cassandra.net.Message
> 19:		1234019	29616456	org.apache.cassandra.net.Header
> 20:		1846185	29538960	org.apache.cassandra.dht.BigIntegerToken
> 21:		891378	28524096	org.apache.cassandra.utils.ExpiringMap$CacheableObject
> 22:		692706	27708240	org.cliffc.high_scale_lib.NonBlockingHashMap
> 23:		1148252	27558048	java.util.Collections$SynchronizedSet
> 24:		541977	26014896	org.apache.cassandra.db.SliceFromReadCommand
> 25:		998001	23952024	java.util.concurrent.ConcurrentSkipListMap$Node
> 26:		928792	22291008	java.util.ArrayList
> 27:		692715	22166880	java.util.concurrent.atomic.AtomicReferenceFieldUpdater$AtomicReferenceFieldUpdaterImpl
> 28:		891378	21393072	org.apache.cassandra.net.CallbackInfo
> 29:		1148247	18371952	java.util.Hashtable$KeySet
> 30:		731859	17564616	org.apache.cassandra.db.Row
> 31:		529991	16959712	org.apache.cassandra.db.ArrayBackedSortedColumns
> 32:		691425	16594200	org.apache.cassandra.db.AbstractColumnContainer$DeletionInfo
> 33:		648580	15565920	org.apache.cassandra.db.filter.QueryPath
> 34:		648338	15560112	org.apache.cassandra.service.RowDigestResolver
> 35:		971376	15542016	java.util.concurrent.atomic.AtomicInteger
> 36:		837418	13398688	org.apache.cassandra.utils.SimpleCondition
> 37:		535614	12854736	org.apache.cassandra.db.ColumnFamily
> 38:		725634	11610144	java.util.concurrent.atomic.AtomicReference
> 39:		195117	9365616	org.apache.cassandra.db.ThreadSafeSortedColumns
> 40:		281921	9021472	java.util.concurrent.ConcurrentSkipListMap$HeadIndex
> 41:		277679	8885728	java.util.concurrent.locks.ReentrantLock$NonfairSync
> 42:		314424	7546176	java.util.concurrent.ConcurrentSkipListMap$Index
> 43:		275186	6604464	java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject
> 44:		270280	6486720	java.util.concurrent.LinkedBlockingQueue$Node
> 45:		219553	5269272	org.apache.cassandra.io.sstable.IndexSummary$KeyPosition
> 46:		106436	5108928	java.util.TreeMap
> 47:		122185	4887400	org.apache.cassandra.db.ExpiringColumn
> 48:		189968	4559232	org.apache.cassandra.db.SuperColumn
> 49:		275659	4410544	java.util.concurrent.locks.ReentrantLock
> 50:		90213	4330224	java.util.concurrent.LinkedBlockingQueue
> 51:		107026	4281040	java.util.TreeMap$Entry
> 52:		30501	4222056	* ConstMethodKlass

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

Mime
View raw message