cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Philippe <watche...@gmail.com>
Subject Re: Writes slower then reads
Date Thu, 05 Jan 2012 15:02:28 GMT
You may be overloading the cluster though...

My hypothesis is that your traffic is being spread across your node and
that one slow node is slowing down the fraction of traffic that goes to
that node (when it's acting as coordinator).
So what I would do is reduce the read load a lot to make sure I don't
overload the cluster and measure if I see a 1/RF improvement in response
time which would validate my hypothesis.


2012/1/5 R. Verlangen <robin@us2.nl>

> It does not appear to affect the response time, certainly not in a
> positive way.
>
>
> 2012/1/5 Philippe <watcherfr@gmail.com>
>
>> What if you shutdown the cassandra service on the slow node, does that
>> improve your read performance ?
>> If it does then that sole node is responsible for the slow down because
>> it can't act as a coordinator fast enough.
>>
>> 2012/1/5 R. Verlangen <robin@us2.nl>
>>
>> I'm also reading with CL = ONE
>>>
>>>
>>> 2012/1/5 Philippe <watcherfr@gmail.com>
>>>
>>>> Depending on the CL you're reading at it will yes : if the CL requires
>>>> that the "slow" node create a digest of the data and send it to the
>>>> coordinator then it might explain the poor performance on reads. What is
>>>> your read CL ?
>>>>
>>>> 2012/1/5 R. Verlangen <robin@us2.nl>
>>>>
>>>> As I posted this I noticed that the other node's CPU is running high on
>>>>> some other cronjobs (every couple of minutes to 60% usage). Is the lack
of
>>>>> more CPU cycles a problem in this case?
>>>>>
>>>>> Robin
>>>>>
>>>>> 2012/1/5 R. Verlangen <robin@us2.nl>
>>>>>
>>>>> CPU is idle (< 10% usage). Disk reads occasionally blocks over 32/64K.
>>>>>> Writes around 0-5MB per second. Network traffic 0.1 / 0.1 MB/s (in
/ out).
>>>>>> Paging 0. System int ~ 1300, csw ~ 2500.
>>>>>>
>>>>>>
>>>>>> 2012/1/5 Philippe <watcherfr@gmail.com>
>>>>>>
>>>>>>> What can you see in vmstat/dstat ?
>>>>>>> Le 5 janv. 2012 11:58, "R. Verlangen" <robin@us2.nl> a
écrit :
>>>>>>>
>>>>>>> Hi there,
>>>>>>>>
>>>>>>>> I'm running a cassandra 0.8.6 cluster with 2 nodes (in 2
DC's), RF
>>>>>>>> = 2. Actual data on the nodes is only 1GB. Disk latency <
1ms. Disk
>>>>>>>> throughput ~ 0.4MB/s. OS load always below 1 (on a 8 core
machine with 16GB
>>>>>>>> ram).
>>>>>>>>
>>>>>>>> When I'm running my writes against the cluster with cl =
ONE all
>>>>>>>> reads appear to be faster then the writes.
>>>>>>>>
>>>>>>>> Average write speed = 1600us/operation
>>>>>>>> Average read speed = 200us/operation
>>>>>>>>
>>>>>>>> I'm really wondering why this is the case. Anyone got a clue?
>>>>>>>>
>>>>>>>> With kind regards,
>>>>>>>> Robin
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message