incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dathan Pattishall <datha...@gmail.com>
Subject Re: what causes a cassandra to block and throw a null exception
Date Wed, 21 Jul 2010 04:09:28 GMT
Just sent one of the nodes back.

Pool Name                    Active   Pending      Completed
STREAM-STAGE                      0         0              0
RESPONSE-STAGE                    0         0         151071
ROW-READ-STAGE                    0         0         100398
LB-OPERATIONS                     0         0              0
MESSAGE-DESERIALIZER-POOL         0         0         281268
GMFD                              0         0            935
LB-TARGET                         0         0              0
CONSISTENCY-MANAGER               0         0          59545
ROW-MUTATION-STAGE                0         0          71453
MESSAGE-STREAMING-POOL            0         0              0
LOAD-BALANCER-STAGE               0         0              0
FLUSH-SORTER-POOL                 0         0              0
MEMTABLE-POST-FLUSHER             0         0              1
FLUSH-WRITER-POOL                 0         0              1
AE-SERVICE-STAGE                  0         0              0
HINTED-HANDOFF-POOL               0         0              3


On Tue, Jul 20, 2010 at 9:03 PM, Chris Goffinet <cg@chrisgoffinet.com>
wrote:
> Can you provide the output from `nodetool tpstats`.
>
> -Chris
>
> On Jul 20, 2010, at 8:59 PM, Dathan Pattishall wrote:
>
>> Type 'help' or '?' for help. Type 'quit' or 'exit' to quit.
>> cassandra> connect cass01/9160
>> cassandra> get TimeFrameClicks.Standard2['test_cassandra_alive']
>> Exception null
>>
>> The data exists and I can grab the data after I restart all the nodes,
>> but once the cluster runs for a few minutes I cannot grab this
>> specific key or random other keys. It takes about  3 seconds until the
>> Exception null message. My storage-conf.xml is very simple:
>>
>> <Keyspace Name="TimeFrameClicks">
>>  <ColumnFamily Name="Standard2" CompareWith="UTF8Type" />
>> ....
>>
>> Now my data is very small like 20 GB across 4 servers. Writes
>> consistently remain fast, yet reads fail like crazy. I hope its
>> something that I am doing wrong because
>>
>> nodetool cfstats
>>
>> says that the read latency for the keyspace and this specific column
>> family is less then 0.3 ms which means that something is lying to me.
>>
>> To head off some questions:
>>
>> CPU utilization is very little.
>> There is hardly any I/O on the box
>> The servers are all the same class enterprise boxes
>> There is 12 GB of ram per server
>> Each Server uses a local RAID.
>> Nothing in any of the system logs that indicates there any problem.
>>
>> Additionally is there a stat or series of stats that I can lookup to
>> determine the health of read performance.
>
>

Mime
View raw message