I think he actually meant *increase*, for this reason &quo= t;For small T, = a random choice of initial tokens will in most cases give a poor distributi= on of data. =A0The larger T is, the closer to uniform the distribution will= be, with increasing probability."

Al= ain

2013/6/11 Theo Hultberg
thanks, that makes sense, but I assume in your last senten= ce you mean decrease it for large clusters, not increase it?

T#
<= /div>

=
On Mon, Jun 10, 2013 at 11:02 PM, Richard Low <richard@wentnet.com&g= t; wrote:
Hi Theo,

The number (let's call it = T and the number of nodes N) 256 was chosen to give good load balancing for= random token assignments for most cluster sizes. =A0For small T, a random = choice of initial tokens will in most cases give a poor distribution of dat= a. =A0The larger T is, the closer to uniform the distribution will be, with= increasing probability.

Also, for small T, when a new node is added, it won'= ;t have many ranges to split so won't be able to take an even slice of = the data.

For this reason T should be large. =A0Bu= t if it is too large, there are too many slices to keep track of as you say= . =A0The function to find which keys live where becomes more expensive and = operations that deal with individual vnodes e.g. repair become slow. =A0(An= extreme example is SELECT * LIMIT 1, which when there is no data has to sc= an each vnode in turn in search of a single row. =A0This is O(NT) and for e= ven quite small T takes seconds to complete.)

So 256 was chosen to be a reasonable balance. =A0I don&= #39;t think most users will find it too slow; users with extremely large cl= usters may need to increase it.

Richard.

=
On 10 June 2013 18:55, Theo Hultberg wrote:
I'm not sure I follow w= hat you mean, or if I've misunderstood what Cassandra is telling me. Ea= ch node has 256 vnodes (or tokens, as the prefered name seems to be). When = I run `nodetool status` each node is reported as having 256 vnodes, regardl= ess of how many nodes are in the cluster. A single node cluster has 256 vno= des on the single node, a six node cluster has 256 nodes on each machine, m= aking 1590 vnodes in total. When I run `SELECT tokens FROM system.peers` or= `nodetool ring` each node lists 256 tokens.

This is different from how it works in Riak and Voldemort, i= f I'm not mistaken, and that is the source of my confusion.
=

T#

On Mon, Jun 10, 2013 at 4:54 PM, Milind Parikh wrote:

There are n vnodes regardless of the size of the physical cl= uster.
Regards
Milind

On Jun 10, 2013 7:48 AM, "Theo Hultberg&quo= t; <theo@iconara.n= et> wrote:
Hi,

The default number of vnodes = is 256, is there any significance in this number? Since Cassandra's vno= des don't work like for example Riak's, where there is a fixed numb= er of vnodes distributed evenly over the nodes, why so many? Even with a mo= derately sized cluster you get thousands of slices. Does this matter? If yo= ur cluster grows to over thirty machines and you start looking at ten thous= and slices, would that be a problem? I guess trat traversing a list of a th= ousand or ten thousand slices to find where a token lives isn't a huge = problem, but are there any other up or downsides to having a small or large= number of vnodes per node?

I understand the benefits for splitting up the ring into pie= ces, for example to be able to stream data from more nodes when bootstrappi= ng a new one, but that works even if each node only has say 32 vnodes (unle= ss your cluster is truly huge).

yours,
Theo

--001a11c373600323c904dedb2305--