cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcelo Valle (BLOOMBERG/ LONDON)" <mvallemil...@bloomberg.net>
Subject Re:Cassandra Read Timeout
Date Tue, 24 Feb 2015 16:19:54 GMT
Yulian,

Maybe other people have other clues, but I think if you could monitor the behavior in tpstats
after activity "Seeking to partition beginning in data file" it could help to find the problem.
Which type of thread is getting stuck? Do you see any number increasing continuously during
the request?

Best regards,
Marcelo.

From: user@cassandra.apache.org 
Subject: Re:Cassandra Read Timeout

Hello to all 
I have single node cassandra on amazon ec2.
Currently i am having a read timeout problem on single CF , single raw.

Raw size is aroung 190MB.There are bigger raws with similar structure ( its index raws , which
actually stores keys ) and everything is working fine on them, everything is working also
fine on this cf but on other raw.

Tables data from CFStats ( First table has bigger raws but works fine , where second has timeout
) :
Column Family: pendindexes
SSTable count: 5
Space used (live): 462298352
Space used (total): 462306752
SSTable Compression Ratio: 0.3511107495795905
Number of Keys (estimate): 640
Memtable Columns Count: 63339
Memtable Data Size: 12328802
Memtable Switch Count: 78
Read Count: 10
Read Latency: NaN ms.
Write Count: 1530113
Write Latency: 0.022 ms.
Pending Tasks: 0
Bloom Filter False Positives: 0
Bloom Filter False Ratio: 0.00000
Bloom Filter Space Used: 3920
Compacted row minimum size: 73
Compacted row maximum size: 223875792
Compacted row mean size: 42694982
Average live cells per slice (last five minutes): 21.0
Average tombstones per slice (last five minutes): 0.0

Column Family: statuspindexes
SSTable count: 1
Space used (live): 99602136
Space used (total): 99609360
SSTable Compression Ratio: 0.34278775390997873
Number of Keys (estimate): 128
Memtable Columns Count: 6250
Memtable Data Size: 6061097
Memtable Switch Count: 65
Read Count: 1000
Read Latency: NaN ms.
Write Count: 1193142
Write Latency: 3.616 ms.
Pending Tasks: 0
Bloom Filter False Positives: 0
Bloom Filter False Ratio: 0.00000
Bloom Filter Space Used: 656
Compacted row minimum size: 180
Compacted row maximum size: 186563160
Compacted row mean size: 63225562
Average live cells per slice (last five minutes): 0.0
Average tombstones per slice (last five minutes): 0.0

I have tried to debug it with cql , thats what i get:

 activity                                                                                
       | timestamp    | source       | source_elapsed
-------------------------------------------------------------------------------------------------+--------------+--------------+----------------
                                                                              execute_cql3_query
| 15:39:53,120 | 172.31.6.173 |              0
                                               Parsing Select * from statuspindexes LIMIT
10000; | 15:39:53,120 | 172.31.6.173 |            875
                                                                             Preparing statement
| 15:39:53,121 | 172.31.6.173 |           1643
                                                                   Determining replicas to
query | 15:39:53,121 | 172.31.6.173 |           1740
 Executing seq scan across 1 sstables for [min(-9223372036854775808), min(-9223372036854775808)]
| 15:39:53,122 | 172.31.6.173 |           2581
                                                     Seeking to partition beginning in data
file | 15:39:53,123 | 172.31.6.173 |           3118
                                           Timed out; received 0 of 1 responses for range
2 of 2 | 15:40:03,121 | 172.31.6.173 |       10001370
                                                                                Request complete
| 15:40:03,121 | 172.31.6.173 |       10001513

I have executed compaction on that cf.
What could lead to that problem?
Best regards
Yulian Oifa


Mime
View raw message