incubator-cassandra-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edward Capriolo <edlinuxg...@gmail.com>
Subject Re: RFC: Cassandra Virtual Nodes
Date Wed, 21 Mar 2012 20:17:03 GMT
On Wed, Mar 21, 2012 at 3:24 PM, Tom Wilkie <tom@acunu.com> wrote:
> Hi Edward
>
>> 1) No more raid 0. If a machine is responsible for 4 vnodes they
>> should correspond to for JBOD.
>
> So each vnode corresponds to a disk?  I suppose we could have a
> separate data directory per disk, but I think this should be a
> separate, subsequent change.
>
I think having more micro-ranges makes the process much easier.
Image a token ring 1-30

Node1 | major range 0-10  | disk 1    0-2 , disk2 3-4, disk 3 5-7, disk 4 8-10
Node2 | major range 11-20| disk 1 11-12 , disk2 13-14, disk 3 15-17,
disk 4 18-20
Node3 | major range 21-30| disk 1 21-22 , disk2 23-24, disk 3 25-27,
disk 4 28-30

Adding a 4th node is easy:
If you are at the data center, just take disk 4 out of each node and
place it in new server :)
Software wise it is the same deal. Each node streams off only disk 4
to the new node.

Now at this point disk 4 is idle and each machine should re balance
its own data across its 4 disks.

> However, do note that making the vnode ~size of a disk (and only have
> 4-8 per machine) would make any non-hotswap rebuilds slower.  To get
> the fast distributed rebuilds, you need to have at least as many
> vnodes per node as you do nodes in the cluster.  And you would still
> need the distributed rebuilds to deal with disk failure.
>
>> 2) Vnodes should be able to be hot pluged. My normal cassandra chassis
>> would be a 2U with 6 drive bays. Imagine I have 10 nodes. Now if my
>> chassis dies I should be able to take the disks out and physically
>> plug them into another chassis. Then in cassandra I should be able to
>> run a command like.
>> nodetool attach '/mnt/disk6'. disk6 should contain all data an it's
>> vnode information.
>>
>> Now this would be awesome for upgrades/migrations/etc.
>
> You know, your not the first person I've spoke to who has asked for
> this!  I do wonder whether it is optimising for the right thing though
> - in my experience, disks fail more often than machines.
>
> Thanks
>
> Tom

Mime
View raw message