cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alain RODRIGUEZ <>
Subject Re: question about the gain of increasing the number of vnode
Date Mon, 21 Jan 2019 12:53:39 GMT
There have been some discussion on this topic in this mailing list,
including a paper from Netflix with the impact of vnodes. I could not find
it quickly, but I invite you to check.

To share some ideas:

More vnodes:
+ Better balance between nodes
+ maximize the streaming throughput for operations as all nodes share a
small bit of the data of all the other nodes (according to the topology).
- When the cluster fails, there is more chance to lose availability as we
256 vnodes for example, 2 nodes down in distinct racks would for sure make
data partially unavailable.
- Overheads / Operational issues (in practice, using 256 vnodes have been a
nightmare for multiple reasons, see below)

Less vnodes
- Imbalances can be big before C* 3.0. After, using
allocate_tokens_for_keyspace -->,
you can mitigate this issue. With this and some technics*, you can have
good results in terms of balances.
* Off the top of my head: this involve bootstrapping the seeds first,
picking the tokens to use, create your keyspace then adding nodes with the
option above. You can test it quite easily. Then with "nodetool status
<keyspace" you can check the ownership balance.
- The streaming throughput is generally limited by the receiving host when
using vnodes, thus 16 vnodes is probably not worse than 256 in terms of
+ The other way around, the overhead of having 256 vnodes makes operations
such as repair almost impossible, or at least way longer and complex.
Repairing tables almost empty can take up to minutes and repairing big
dataset might never end.
+ In Netflix paper about this topic (very interesting, I recommend
reading), it is explained that reducing the number of vnodes reduces the
chances of an outage.
+ There was a discussion in the dev mailing list. I believe the community
agreed on the need to reduce the number of vnodes by default. Here again,
you can have a quick look at the archive, Jira, github/trunk.

I think that commonly accepted values would be 16/32. Values as low as 4
are considered to improve availability, reduce overheads induced by vnodes.
I would suggest you test it and see if low values you still manage to keep
the balance between nodes.

Also using "physical" nodes (initial_token, no vnodes) gives the
possibility to reason about token distribution. You can perform
advanced operations
where you bootstrap 1/3 of the cluster at once. This is very good
especially for big clusters, I would say. While with many vnodes you'll
have to go add a node at the time as each node is actually the 'neighbor'
of all the others (according to the topology again - ie racks/data

I would stay away from the default in this case (256 vnodes). I think this
value is way too high by default.

Also, keep in mind that to change the number of vnodes cannot be changed in
a running cluster. The best way to change it is to add a new data center I

Alain Rodriguez -
France / Spain

The Last Pickle - Apache Cassandra Consulting

Le lun. 21 janv. 2019 à 11:08, VICTOR IBARRA <> a écrit :

> Good morning every one,
> I would like have a contact with the cassandra community for the questions
> of cluster configuration
> Today i have many questions and differents projets about the configuration
> of cluster cassandra and with the general problems of configuration
> migration and for the use of vnodes.
> and the principal question is what about the gain to use 256 vnodes vs 16
> vnodes for example
> Best regards
> --
>  L'integrité de ce message n'étant pas assurée sur internet, VICTOR IBARRA
> ne peut être tenue responsable de son contenu en ce compris les pièces
> jointes. Toute utilisation ou diffusion non autorisée est interdite. Si
> vous n'êtes pas destinataire de ce message, merci de le  détruire et
> d'avertir l'expéditeur.
>  The integrity of this message cannot be guaranteed on the Internet.
> VICTOR IBARRA can not therefore be considered liable for the  contents
> including its attachments. Any unauthorized use or dissemination is
> prohibited. If you are not the intended recipient of  this message, then
> please delete it and notify the sender.

View raw message