incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andras Szerdahelyi <andras.szerdahe...@ignitionone.com>
Subject Re: Cassandra freezes
Date Wed, 20 Mar 2013 12:29:57 GMT

I'd say GC. Please fill in form CASS-FREEZE-001 below and get back to us
:-) ( sorry )

How big is your JVM heap ? How many CPUs ?
Garbage collection taking long ? ( look for log lines from GCInspector)
Running out of heap ? ( "heap is .. full" log lines )
Any tasks backing up / being dropped ? ( nodetool tpstats and ".. dropped
in last .. ms" log lines )
Are writes really slow? ( nodetool cfhistograms Keyspace ColumnFamily )

How much is lots of data? Wide or skinny rows? Mutations/sec ?
Which Compaction Strategy are you using? Output of show schema (
cassandra-cli ) for the relevant Keyspace/CF might help as well

What consistency are you doing your writes with ? I assume ONE or ANY if
you have a single node.

What are the values for these settings in cassandra.yaml

memtable_total_space_in_mb:
memtable_flush_writers:
memtable_flush_queue_size:
compaction_throughput_mb_per_sec:

concurrent_writes:



Which version of Cassandra?



Regards,
Andras

From:  Joel Samuelsson <samuelsson.joel@gmail.com>
Reply-To:  "user@cassandra.apache.org" <user@cassandra.apache.org>
Date:  Wednesday 20 March 2013 13:06
To:  "user@cassandra.apache.org" <user@cassandra.apache.org>
Subject:  Cassandra freezes


Hello,

I've been trying to load test a one node cassandra cluster. When I add
lots of data, the Cassandra node freezes for 4-5 minutes during which
neither reads nor writes are served.
During this time, Cassandra takes 100% of a single CPU core.
My initial thought was that this was Cassandra flushing memtables to the
disk, however, the disk i/o is very low during this time.
Any idea what my problem could be?
I'm running in a virtual environment in which I have no control of drives.
So commit log and data directory is (probably) on the same drive.

Best regards,
Joel Samuelsson


Mime
View raw message