cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "B. Todd Burruss" <bto...@gmail.com>
Subject Re: unbalanced ring
Date Wed, 10 Oct 2012 19:56:00 GMT
it should not have any other impact except increased usage of system
resources.

and i suppose, cleanup would not have an affect (over normal compaction) if
all nodes contain the same data

On Wed, Oct 10, 2012 at 12:12 PM, Tamar Fraenkel <tamar@tok-media.com>wrote:

> Hi!
> Apart from being heavy load (the compact), will it have other effects?
> Also, will cleanup help if I have replication factor = number of nodes?
> Thanks
>
> *Tamar Fraenkel *
> Senior Software Engineer, TOK Media
>
> [image: Inline image 1]
>
> tamar@tok-media.com
> Tel:   +972 2 6409736
> Mob:  +972 54 8356490
> Fax:   +972 2 5612956
>
>
>
>
>
> On Wed, Oct 10, 2012 at 6:12 PM, B. Todd Burruss <btoddb@gmail.com> wrote:
>
>> major compaction in production is fine, however it is a heavy operation
>> on the node and will take I/O and some CPU.
>>
>> the only time i have seen this happen is when i have changed the tokens
>> in the ring, like "nodetool movetoken".  cassandra does not auto-delete
>> data that it doesn't use anymore just in case you want to move the tokens
>> again or otherwise "undo".
>>
>> try "nodetool cleanup"
>>
>>
>> On Wed, Oct 10, 2012 at 2:01 AM, Alain RODRIGUEZ <arodrime@gmail.com>wrote:
>>
>>> Hi,
>>>
>>> Same thing here:
>>>
>>> 2 nodes, RF = 2. RCL = 1, WCL = 1.
>>> Like Tamar I never ran a major compaction and repair once a week each
>>> node.
>>>
>>> 10.59.21.241    eu-west     1b          Up     Normal  133.02 GB
>>> 50.00%              0
>>> 10.58.83.109    eu-west     1b          Up     Normal  98.12 GB
>>>  50.00%              85070591730234615865843651857942052864
>>>
>>> What phenomena could explain the result above ?
>>>
>>> By the way, I have copy the data and import it in a one node dev
>>> cluster. There I have run a major compaction and the size of my data has
>>> been significantly reduced (to about 32 GB instead of 133 GB).
>>>
>>> How is that possible ?
>>> Do you think that if I run major compaction in both nodes it will
>>> balance the load evenly ?
>>> Should I run major compaction in production ?
>>>
>>> 2012/10/10 Tamar Fraenkel <tamar@tok-media.com>
>>>
>>>> Hi!
>>>> I am re-posting this, now that I have more data and still *unbalanced
>>>> ring*:
>>>>
>>>> 3 nodes,
>>>> RF=3, RCL=WCL=QUORUM
>>>>
>>>>
>>>> Address         DC          Rack        Status State   Load
>>>> Owns    Token
>>>>
>>>> 113427455640312821154458202477256070485
>>>> x.x.x.x    us-east     1c          Up     Normal  24.02 GB
>>>> 33.33%  0
>>>> y.y.y.y     us-east     1c          Up     Normal  33.45 GB
>>>> 33.33%  56713727820156410577229101238628035242
>>>> z.z.z.z    us-east     1c          Up     Normal  29.85 GB
>>>> 33.33%  113427455640312821154458202477256070485
>>>>
>>>> repair runs weekly.
>>>> I don't run nodetool compact as I read that this may cause the minor
>>>> regular compactions not to run and then I will have to run compact
>>>> manually. Is that right?
>>>>
>>>> Any idea if this means something wrong, and if so, how to solve?
>>>>
>>>>
>>>> Thanks,
>>>> *
>>>> Tamar Fraenkel *
>>>> Senior Software Engineer, TOK Media
>>>>
>>>> [image: Inline image 1]
>>>>
>>>> tamar@tok-media.com
>>>> Tel:   +972 2 6409736
>>>> Mob:  +972 54 8356490
>>>> Fax:   +972 2 5612956
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Mar 27, 2012 at 9:12 AM, Tamar Fraenkel <tamar@tok-media.com>wrote:
>>>>
>>>>> Thanks, I will wait and see as data accumulates.
>>>>> Thanks,
>>>>>
>>>>> *Tamar Fraenkel *
>>>>> Senior Software Engineer, TOK Media
>>>>>
>>>>> [image: Inline image 1]
>>>>>
>>>>> tamar@tok-media.com
>>>>> Tel:   +972 2 6409736
>>>>> Mob:  +972 54 8356490
>>>>> Fax:   +972 2 5612956
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Mar 27, 2012 at 9:00 AM, R. Verlangen <robin@us2.nl> wrote:
>>>>>
>>>>>> Cassandra is built to store tons and tons of data. In my opinion
>>>>>> roughly ~ 6MB per node is not enough data to allow it to become a
fully
>>>>>> balanced cluster.
>>>>>>
>>>>>>
>>>>>> 2012/3/27 Tamar Fraenkel <tamar@tok-media.com>
>>>>>>
>>>>>>> This morning I have
>>>>>>>  nodetool ring -h localhost
>>>>>>> Address         DC          Rack        Status State   Load
>>>>>>>    Owns    Token
>>>>>>>
>>>>>>>            113427455640312821154458202477256070485
>>>>>>> 10.34.158.33    us-east     1c          Up     Normal  5.78 MB
>>>>>>>   33.33%  0
>>>>>>> 10.38.175.131   us-east     1c          Up     Normal  7.23 MB
>>>>>>>   33.33%  56713727820156410577229101238628035242
>>>>>>>  10.116.83.10    us-east     1c          Up     Normal  5.02
MB
>>>>>>>     33.33%  113427455640312821154458202477256070485
>>>>>>>
>>>>>>> Version is 1.0.8.
>>>>>>>
>>>>>>>
>>>>>>>  *Tamar Fraenkel *
>>>>>>> Senior Software Engineer, TOK Media
>>>>>>>
>>>>>>> [image: Inline image 1]
>>>>>>>
>>>>>>> tamar@tok-media.com
>>>>>>> Tel:   +972 2 6409736
>>>>>>> Mob:  +972 54 8356490
>>>>>>> Fax:   +972 2 5612956
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Mar 27, 2012 at 4:05 AM, Maki Watanabe <
>>>>>>> watanabe.maki@gmail.com> wrote:
>>>>>>>
>>>>>>>> What version are you using?
>>>>>>>> Anyway try nodetool repair & compact.
>>>>>>>>
>>>>>>>> maki
>>>>>>>>
>>>>>>>>
>>>>>>>> 2012/3/26 Tamar Fraenkel <tamar@tok-media.com>
>>>>>>>>
>>>>>>>>> Hi!
>>>>>>>>> I created Amazon ring using datastax image and started
filling the
>>>>>>>>> db.
>>>>>>>>> The cluster seems un-balanced.
>>>>>>>>>
>>>>>>>>> nodetool ring returns:
>>>>>>>>> Address         DC          Rack        Status State
  Load
>>>>>>>>>      Owns    Token
>>>>>>>>>
>>>>>>>>>              113427455640312821154458202477256070485
>>>>>>>>> 10.34.158.33    us-east     1c          Up     Normal
 514.29 KB
>>>>>>>>>     33.33%  0
>>>>>>>>> 10.38.175.131   us-east     1c          Up     Normal
 1.5 MB
>>>>>>>>>      33.33%  56713727820156410577229101238628035242
>>>>>>>>> 10.116.83.10    us-east     1c          Up     Normal
 1.5 MB
>>>>>>>>>      33.33%  113427455640312821154458202477256070485
>>>>>>>>>
>>>>>>>>> [default@tok] describe;
>>>>>>>>> Keyspace: tok:
>>>>>>>>>   Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
>>>>>>>>>   Durable Writes: true
>>>>>>>>>     Options: [replication_factor:2]
>>>>>>>>>
>>>>>>>>> [default@tok] describe cluster;
>>>>>>>>> Cluster Information:
>>>>>>>>>    Snitch: org.apache.cassandra.locator.Ec2Snitch
>>>>>>>>>    Partitioner: org.apache.cassandra.dht.RandomPartitioner
>>>>>>>>>    Schema versions:
>>>>>>>>>         4687d620-7664-11e1-0000-1bcb936807ff: [10.38.175.131,
>>>>>>>>> 10.34.158.33, 10.116.83.10]
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Any idea what is the cause?
>>>>>>>>> I am running similar code on local ring and it is balanced.
>>>>>>>>>
>>>>>>>>> How can I fix this?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> *Tamar Fraenkel *
>>>>>>>>> Senior Software Engineer, TOK Media
>>>>>>>>>
>>>>>>>>> [image: Inline image 1]
>>>>>>>>>
>>>>>>>>> tamar@tok-media.com
>>>>>>>>> Tel:   +972 2 6409736
>>>>>>>>> Mob:  +972 54 8356490
>>>>>>>>> Fax:   +972 2 5612956
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> With kind regards,
>>>>>>
>>>>>> Robin Verlangen
>>>>>> www.robinverlangen.nl
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message