cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "R. Verlangen" <ro...@us2.nl>
Subject Re: Writes slower then reads
Date Thu, 05 Jan 2012 15:34:17 GMT
The write and read load is very minimal the moment. Roughly 10 writes + 10
reads / second. So 20 operations per second. Don't think that overloads my
cluster, does it?

2012/1/5 Philippe <watcherfr@gmail.com>

> You may be overloading the cluster though...
>
> My hypothesis is that your traffic is being spread across your node and
> that one slow node is slowing down the fraction of traffic that goes to
> that node (when it's acting as coordinator).
> So what I would do is reduce the read load a lot to make sure I don't
> overload the cluster and measure if I see a 1/RF improvement in response
> time which would validate my hypothesis.
>
>
> 2012/1/5 R. Verlangen <robin@us2.nl>
>
> It does not appear to affect the response time, certainly not in a
>> positive way.
>>
>>
>> 2012/1/5 Philippe <watcherfr@gmail.com>
>>
>>> What if you shutdown the cassandra service on the slow node, does that
>>> improve your read performance ?
>>> If it does then that sole node is responsible for the slow down because
>>> it can't act as a coordinator fast enough.
>>>
>>> 2012/1/5 R. Verlangen <robin@us2.nl>
>>>
>>> I'm also reading with CL = ONE
>>>>
>>>>
>>>> 2012/1/5 Philippe <watcherfr@gmail.com>
>>>>
>>>>> Depending on the CL you're reading at it will yes : if the CL requires
>>>>> that the "slow" node create a digest of the data and send it to the
>>>>> coordinator then it might explain the poor performance on reads. What
is
>>>>> your read CL ?
>>>>>
>>>>> 2012/1/5 R. Verlangen <robin@us2.nl>
>>>>>
>>>>> As I posted this I noticed that the other node's CPU is running high
>>>>>> on some other cronjobs (every couple of minutes to 60% usage). Is
the lack
>>>>>> of more CPU cycles a problem in this case?
>>>>>>
>>>>>> Robin
>>>>>>
>>>>>> 2012/1/5 R. Verlangen <robin@us2.nl>
>>>>>>
>>>>>> CPU is idle (< 10% usage). Disk reads occasionally blocks over
>>>>>>> 32/64K. Writes around 0-5MB per second. Network traffic 0.1 /
0.1 MB/s (in
>>>>>>> / out). Paging 0. System int ~ 1300, csw ~ 2500.
>>>>>>>
>>>>>>>
>>>>>>> 2012/1/5 Philippe <watcherfr@gmail.com>
>>>>>>>
>>>>>>>> What can you see in vmstat/dstat ?
>>>>>>>> Le 5 janv. 2012 11:58, "R. Verlangen" <robin@us2.nl>
a écrit :
>>>>>>>>
>>>>>>>> Hi there,
>>>>>>>>>
>>>>>>>>> I'm running a cassandra 0.8.6 cluster with 2 nodes (in
2 DC's), RF
>>>>>>>>> = 2. Actual data on the nodes is only 1GB. Disk latency
< 1ms. Disk
>>>>>>>>> throughput ~ 0.4MB/s. OS load always below 1 (on a 8
core machine with 16GB
>>>>>>>>> ram).
>>>>>>>>>
>>>>>>>>> When I'm running my writes against the cluster with cl
= ONE all
>>>>>>>>> reads appear to be faster then the writes.
>>>>>>>>>
>>>>>>>>> Average write speed = 1600us/operation
>>>>>>>>> Average read speed = 200us/operation
>>>>>>>>>
>>>>>>>>> I'm really wondering why this is the case. Anyone got
a clue?
>>>>>>>>>
>>>>>>>>> With kind regards,
>>>>>>>>> Robin
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message