when i stop cassandra and start it again, this is what is printed. it takes just a couple of seconds for this to run.
and after that it becomes really fast.


Listening for transport dt_socket at address: 8888
DEBUG - Loading settings from ./../conf/storage-conf.xml
DEBUG - adding Super1 as 0
DEBUG - adding Standard2 as 1
DEBUG - adding Standard1 as 2
DEBUG - adding StandardByTime1 as 3
DEBUG - adding LocationInfo as 4
DEBUG - adding HintsColumnFamily as 5
DEBUG - INDEX LOAD TIME for /home/mark/local/var/cassandra/data/Table1-Super1-9-Data.db: 400 ms.
DEBUG - INDEX LOAD TIME for /home/mark/local/var/cassandra/data/Table1-Super1-52-Data.db: 300 ms.
DEBUG - INDEX LOAD TIME for /home/mark/local/var/cassandra/data/Table1-Super1-92-Data.db: 300 ms.
DEBUG - INDEX LOAD TIME for /home/mark/local/var/cassandra/data/Table1-Super1-138-Data.db: 751 ms.
DEBUG - INDEX LOAD TIME for /home/mark/local/var/cassandra/data/Table1-Super1-150-Data.db: 100 ms.
DEBUG - INDEX LOAD TIME for /home/mark/local/var/cassandra/data/Table1-Super1-152-Data.db: 50 ms.
DEBUG - INDEX LOAD TIME for /home/mark/local/var/cassandra/data/Table1-Super1-154-Data.db: 100 ms.
INFO - Compacting [/home/mark/local/var/cassandra/data/Table1-Super1-138-Data.db,/home/mark/local/var/cassandra/data/Table1-Super1-150-Data.db,/home/mark/local/var/cassandra/data/Table1-Super1-152-Data.db,/home/mark/local/var/cassandra/data/Table1-Super1-154-Data.db]
DEBUG - INDEX LOAD TIME for /home/mark/local/var/cassandra/data/Table1-Standard1-2-Data.db: 0 ms.
DEBUG - INDEX LOAD TIME for /home/mark/local/var/cassandra/data/Table1-Standard1-4-Data.db: 50 ms.
DEBUG - INDEX LOAD TIME for /home/mark/local/var/cassandra/data/Table1-Standard1-6-Data.db: 0 ms.
INFO - Replaying /home/mark/local/var/cassandra/commitlog/CommitLog-1247454203796.log
DEBUG - index size for bloom filter calc for file  : /home/mark/local/var/cassandra/data/Table1-Super1-138-Data.db   : 73600
DEBUG - index size for bloom filter calc for file  : /home/mark/local/var/cassandra/data/Table1-Super1-150-Data.db   : 84224
DEBUG - index size for bloom filter calc for file  : /home/mark/local/var/cassandra/data/Table1-Super1-152-Data.db   : 94848
DEBUG - index size for bloom filter calc for file  : /home/mark/local/var/cassandra/data/Table1-Super1-154-Data.db   : 105472
DEBUG - Expected bloom filter size : 105472
INFO - Compacted to /home/mark/local/var/cassandra/data/Table1-Super1-139-Data.db.  0/28831084 bytes for 104856/104860 keys read/written.  Time: 8119ms.
INFO - Flushing Memtable(Super1)@552364977
DEBUG - Submitting Super1 for compaction
INFO - Completed flushing Memtable(Super1)@552364977
INFO - Flushing Memtable(Standard1)@1290243769
DEBUG - Submitting Standard1 for compaction
INFO - Completed flushing Memtable(Standard1)@1290243769
INFO - Compacting [/home/mark/local/var/cassandra/data/Table1-Standard1-2-Data.db,/home/mark/local/var/cassandra/data/Table1-Standard1-4-Data.db,/home/mark/local/var/cassandra/data/Table1-Standard1-6-Data.db,/home/mark/local/var/cassandra/data/Table1-Standard1-8-Data.db]
DEBUG - index size for bloom filter calc for file  : /home/mark/local/var/cassandra/data/Table1-Standard1-2-Data.db   : 256
DEBUG - index size for bloom filter calc for file  : /home/mark/local/var/cassandra/data/Table1-Standard1-4-Data.db   : 512
DEBUG - index size for bloom filter calc for file  : /home/mark/local/var/cassandra/data/Table1-Standard1-6-Data.db   : 768
DEBUG - index size for bloom filter calc for file  : /home/mark/local/var/cassandra/data/Table1-Standard1-8-Data.db   : 1024
DEBUG - Expected bloom filter size : 1024
INFO - Compacted to /home/mark/local/var/cassandra/data/Table1-Standard1-3-Data.db.  0/210 bytes for 0/1 keys read/written.  Time: 301ms.
DEBUG - Starting to listen on 127.0.0.1:7001
INFO - Cassandra starting up...



On Mon, Jul 13, 2009 at 6:06 AM, rkmr.em@gmail.com <rkmr.em@gmail.com> wrote:

how do i find out if JVM is GCing?

On Sun, Jul 12, 2009 at 10:37 PM, Sandeep Tata <sandeep.tata@gmail.com> wrote:
What hardware are you running one?

 dual quadcore intel xeon 2.0 ghz, 32GB ram, and hardware raid config
operating system is fedora core 9


How long does the slowdown last ?

i stopped inserting data after slowdown starts, and it is still slow now after over 10 hours.
however if i stop cassandra and start it, it becomes super fast immediately. till i insert another 100k or so rows when it becomes really slow again.

 

 There are a few reasons for temporary slowdowns ... perhaps the JVM
started GCing?
how do i find out if this is the cause?

 
Cassandra spends time cleaning up the on-disk SSTables
in a process called compaction. This could cause the client to observe
a slowdown.

Things you could try --
Reduce the Memtable size in the config files. (If GCing was the problem)
Increasing the number of SSTables written before compaction kicks in.

can you tell me what numbers i should use?

thanks!