incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Huming Wu <huming...@gmail.com>
Subject Cassandra memory footprint
Date Tue, 11 Aug 2009 00:20:36 GMT
I am currently doing some test on cassandra (0.3.0-final). two nodes
with each node
has 8G ram and 8 core cpus. And here are some setting from my
storage-conf.xml:

<ReplicationFactor>2</ReplicationFactor>
<ColumnIndexSizeInKB>256</ColumnIndexSizeInKB>
<MemtableSizeInMB>1024</MemtableSizeInMB>
<MemtableObjectCountInMillions>2</MemtableObjectCountInMillions>

The test data I have has about 880K unique keys and my test program
simply inserts the same 5 columns into Table1.Standard1 using
thrift.batch_insert. For each key, the record size ranges from 21
bytes to 5K with the average being 40 bytes. The program calls
batch_insert repeatedly - 4 million times with 50 concurrent thrift
connections (about 220MB data excluding keys is sent to cassandra).
What I see was basically the JAVA resident memory grows to the GC
limit (6G) and everything just halts after that. If I restarted
cassandra I can see the footprint is around 1.9G and I can do insert
again but the memory keeps growing and so on. Here is my JVM setting:

-Xmx6000m -Xms6000m -XX:+HeapDumpOnOutOfMemoryError -XX:NewSize=1000m
-XX:MaxNewSize=1000m -XX:SurvivorRatio=8 -XX:+UseConcMarkSweepGC
-XX:+CMSIncrementalMode -verbose:gc -XX:+PrintHeapAtGC
-XX:+PrintGCDetails -Xloggc:gc.log

Here is the jmap output (top 10 objects):

num   #instances    #bytes  class name
--------------------------------------

 1:   1436005   658220304  [Ljava.lang.Object;
 2:  12100491   484019640  java.lang.String
 3:   9904511   437577600  [C
 4:   2709812   322398784  [I
 5:   5607988   224319520  java.util.concurrent.ConcurrentSkipListMap$Node
 6:   4469810   214550880  org.apache.cassandra.db.Column
 7:   3339219   213710016  org.cliffc.high_scale_lib.ConcurrentAutoTable$CAT
 8:   3339230   191142200  [J
 9:   4506140   179147648  [B
 10:   2220024   106561152  java.util.concurrent.ConcurrentSkipListMap$HeadIndex

Does anyone have any idea why cassandra uses so much memory? From
gc.log I do see gc has kicked in many times (but not major compaction
though). I'd expect that with this small data set everything would
just work fine with the avail. memory (I mean the test should just go
on for weeks).

Any suggestion?

Thanks,
Huming

Mime
View raw message