cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mcasandra <mohitanch...@gmail.com>
Subject Re: Timeout during stress test
Date Mon, 11 Apr 2011 21:56:44 GMT
I see this occurring often when all cassandra nodes all of a sudden show CPU
spike. All reads fail for about 2 mts. GC.log and system.log doesn't reveal
much.

Only think I notice is that when I restart nodes there are tons of files
that gets deleted. cfstats from one of the nodes looks like this:

nodetool -h `hostname` tpstats
Pool Name                    Active   Pending      Completed
ReadStage                        27        27          21491
RequestResponseStage              0         0         201641
MutationStage                     0         0         236513
ReadRepairStage                   0         0           7222
GossipStage                       0         0          31498
AntiEntropyStage                  0         0              0
MigrationStage                    0         0              0
MemtablePostFlusher               0         0            324
StreamStage                       0         0              0
FlushWriter                       0         0            324
FILEUTILS-DELETE-POOL             0         0           1220
MiscStage                         0         0              0
FlushSorter                       0         0              0
InternalResponseStage             0         0              0
HintedHandoff                     1         3              9

--

----------------
Keyspace: StressKeyspace
        Read Count: 21957
        Read Latency: 46.91765058978913 ms.
        Write Count: 222104
        Write Latency: 0.008302124230090408 ms.
        Pending Tasks: 0
                Column Family: StressStandard
                SSTable count: 286
                Space used (live): 377916657941
                Space used (total): 377916657941
                Memtable Columns Count: 362
                Memtable Data Size: 164403613
                Memtable Switch Count: 326
                Read Count: 21958
                Read Latency: 631.464 ms.
                Write Count: 222104
                Write Latency: 0.007 ms.
                Pending Tasks: 0
                Key cache capacity: 1000000
                Key cache size: 22007
                Key cache hit rate: 0.002453626459907744
                Row cache: disabled
                Compacted row minimum size: 87
                Compacted row maximum size: 5839588
                Compacted row mean size: 552698

----------------


--
View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Timeout-during-stress-test-tp6262430p6263087.html
Sent from the cassandra-user@incubator.apache.org mailing list archive at Nabble.com.

Mime
View raw message