hum..no, it wasn't swapping. cassandra was the only thing running on that server
and i was querying the same keys over and over

i restarted Cassandra and doing the same thing, io is now down to zero while cpu is up which dosen't surprise me as much.

I'll report if it happens again.

Le 5 juin 2011 16:55, "Jonathan Ellis" <jbellis@gmail.com> a écrit :
> You may be swapping.
>
> http://spyced.blogspot.com/2010/01/linux-performance-basics.html
> explains how to check this as well as how to see what threads are busy
> in the Java process.
>
> On Sat, Jun 4, 2011 at 5:34 PM, Philippe <watcherfr@gmail.com> wrote:
>> Hello,
>> I am evaluating using cassandra and I'm running into some strange IO
>> behavior that I can't explain, I'd like some help/ideas to troubleshoot it.
>> I am running a 1 node cluster with a keyspace consisting of two columns
>> families, one of which has dozens of supercolumns itself containing dozens
>> of columns.
>> All in all, this is a couple gigabytes of data, 12GB on the hard drive.
>> The hardware is pretty good : 16GB memory + RAID-0 SSD drives with LVM and
>> an i5 processor (4 cores).
>> Keyspace: xxxxxxxxxxxxxxxxxxx
>>         Read Count: 460754852
>>         Read Latency: 1.108205793092766 ms.
>>         Write Count: 30620665
>>         Write Latency: 0.01411020877567486 ms.
>>         Pending Tasks: 0
>>                 Column Family: xxxxxxxxxxxxxxxxxxxxxxxxxx
>>                 SSTable count: 5
>>                 Space used (live): 548700725
>>                 Space used (total): 548700725
>>                 Memtable Columns Count: 0
>>                 Memtable Data Size: 0
>>                 Memtable Switch Count: 11
>>                 Read Count: 2891192
>>                 Read Latency: NaN ms.
>>                 Write Count: 3157547
>>                 Write Latency: NaN ms.
>>                 Pending Tasks: 0
>>                 Key cache capacity: 367396
>>                 Key cache size: 367396
>>                 Key cache hit rate: NaN
>>                 Row cache capacity: 112683
>>                 Row cache size: 112683
>>                 Row cache hit rate: NaN
>>                 Compacted row minimum size: 125
>>                 Compacted row maximum size: 924
>>                 Compacted row mean size: 172
>>                 Column Family: yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
>>                 SSTable count: 7
>>                 Space used (live): 8707538781
>>                 Space used (total): 8707538781
>>                 Memtable Columns Count: 0
>>                 Memtable Data Size: 0
>>                 Memtable Switch Count: 30
>>                 Read Count: 457863660
>>                 Read Latency: 2.381 ms.
>>                 Write Count: 27463118
>>                 Write Latency: NaN ms.
>>                 Pending Tasks: 0
>>                 Key cache capacity: 4518387
>>                 Key cache size: 4518387
>>                 Key cache hit rate: 0.9247881700850826
>>                 Row cache capacity: 1349682
>>                 Row cache size: 1349682
>>                 Row cache hit rate: 0.39400533823415573
>>                 Compacted row minimum size: 125
>>                 Compacted row maximum size: 6866
>>                 Compacted row mean size: 165
>> My app makes a bunch of requests using a MultigetSuperSliceQuery for a set
>> of keys, typically a couple dozen at most. It also selects a subset of the
>> supercolumns. I am running 8 requests in parallel at most.
>>
>> Two days, I ran a 1.5 hour process that basically read every key. The server
>> had no IOwaits and everything was humming along. However, right at the end
>> of the process, there was a huge spike in IOs. I didn't think much of it.
>> Today, after two days of inactivity, any query I run raises the IOs to 80%
>> utilization of the SSD drives even though I'm running the same query over
>> and over (no cache??)
>> Any ideas on how to troubleshoot this, or better, how to solve this ?
>> thanks
>> Philippe
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com