cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ben Coverston <ben.covers...@datastax.com>
Subject Re: Local read/write vs write request
Date Thu, 12 Jul 2012 15:33:00 GMT
Do you have three rows (or a small number) that are written to, or read
from more frequently than the others?

Remember your data is distributed based on the key, so if you have a very
active index on a particular, or small set of, row(s) you could see strange
modalities like this.

Ben

On Thu, Jul 12, 2012 at 2:31 AM, Olivier Mallassi <omallassi@octo.com>wrote:

> yep, all nodes have a similar token range ownership and we use
> randompartitionner.
>
> After the initial load, all my nodes have the same amount of data (aroud
> 2Go each)
> In fact, my guesses is that these load tests were only using a small part
> of our data set (and that data was routed to this couple of nodes and
> replicas)
> Running other load tests that imply the complete data set, I do not see
> the same symptoms. Local Read/Writes are much more balanced across all
> nodes.
>
> Anyway, if further tests confirmed my data is unbalanced (or not very well
> balanced), I guess my only way to resolve this is to change the token
> assignment to split more the data on nodes that are the most used. Am i
> wrong?
>
>
>
> On Wed, Jul 11, 2012 at 5:39 PM, rohit bhatia <rohit2412@gmail.com> wrote:
>
>> The requests are clearly balanced.. But the data is not
>>
>> Do all nodes have the similar token range ownership?
>> Are you using a randomPartitioner or an OrderPreserving partitioner?
>>
>>
>> On Wed, Jul 11, 2012 at 7:29 PM, Olivier Mallassi <omallassi@octo.com>wrote:
>>
>>> Hi all
>>>
>>> During our load tests, we observe that a couple of servers are much more
>>> used than others (in terms of CPU). In front of the cassandra cluster we
>>> have a IPVS (ipvsadm configured in rr)
>>>
>>> in opscenter we have the following graph :
>>> Write / read Request : almost evenly distributed ...
>>> [image: Inline image 1]
>>>
>>> Local Writes / Read Request : clearly not evenly distributed and as you
>>> can imagine, the high loaded nodes (in CPU) are the ones with more TPS....
>>> [image: Inline image 2]
>>>
>>>
>>> Based on that, my best idea is that load tests we are doing use data
>>> that is physically located on a couple of server (bad luck....).
>>>
>>> We tried using different data (in our load tests), so different key but
>>> the behavior seem identical (we will repeat the tests).
>>>
>>> Do you have any ideas of what can be wrong?
>>>
>>> thx a lot.
>>>
>>> --
>>> ............................................................
>>> Olivier Mallassi
>>> OCTO Technology
>>> ............................................................
>>> 50, Avenue des Champs-Elysées
>>> 75008 Paris
>>>
>>> Mobile: (33) 6 28 70 26 61
>>> Tél: (33) 1 58 56 10 00
>>> Fax: (33) 1 58 56 10 01
>>>
>>> http://www.octo.com
>>> Octo Talks! http://blog.octo.com
>>>
>>>
>>>
>>
>
>
> --
> ............................................................
> Olivier Mallassi
> OCTO Technology
> ............................................................
> 50, Avenue des Champs-Elysées
> 75008 Paris
>
> Mobile: (33) 6 28 70 26 61
> Tél: (33) 1 58 56 10 00
> Fax: (33) 1 58 56 10 01
>
> http://www.octo.com
> Octo Talks! http://blog.octo.com
>
>
>


-- 
Ben Coverston
DataStax -- The Apache Cassandra Company

Mime
View raw message