incubator-cassandra-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sam Overton <>
Subject Update: Cassandra Virtual Nodes
Date Fri, 01 Jun 2012 16:50:15 GMT
Hello cassandra-dev,

This is an update on the previous discussion on virtual nodes - for
anyone who missed that, an archive is here:

Work on our proposed design is being tracked on JIRA:

We feel that our current implementation has sufficient functionality
to be useful in its own right, but is still a fairly self-contained
set of changes that can be incrementally improved - something which
was expressed as desirable in the last discussion. This is a good
point for a first round of review and to reach out to anyone
interested in testing or contributing.

== Obtaining the source ==

The easiest way to test these changes is to clone our github repo and
switch to the topic branch representing the top patch:

> git clone git://
> cd cassandra
> git checkout --track remotes/origin/p/4127/01_migration_path

Now just build with ant. To create a cluster with virtual-nodes
support just un-comment/edit the following parameter in
> num_tokens: 256

Make sure you have left initial_token blank. The node should
automatically assign itself this many tokens. You can view the token
assignment as usual with "nodetool ring" but this becomes fairly
useless with any large number of hosts/tokens, which is why we have
added "nodetool clusterinfo" which shows ring ownership without
cluttering the output with the token assignment.

If you want to test with a specific token assignment, initial_token
now supports a comma-separated list of tokens which will override the
num_tokens setting. This is just for convenience of testing - users
should no longer need to manually specify tokens as balancing is

== Patches ==

The tickets which are in scope for this round of review are:

Each of these tickets has links to the corresponding patches. The
order they should be applied is just increasing numerical order of
ticket number.

These patches are based off another patch (currently pending review)
for CASSANDRA-3881, which was an existing issue blocking virtual

Links to the individual patches can also be found all in one place on
the github wiki:

The current patches in the series are as follows:

* p/4121/01_support_multiple_tokens_per_host
support multiple tokens per host in TokenMetadata (CASSANDRA-4121)
* p/4122/01_bootstrap_decommission
support bootstrapping a node into the ring with multiple tokens and
minor changes for decommission (CASSANDRA-4122)
* p/4122/02_remove_tokens
minor changes to support remove-token with multiple tokens (CASSANDRA-4122)
* p/4125/01_admin_tools
nodetool support (addition of nodetool clusterinfo and changes to
nodetool ring) (CASSANDRA-4125)
* p/4127/01_migration_path
support for migration from 1-token to multi-token per node (CASSANDRA-4127)

If you wish to contribute or work with a clone from github then it
would be advisable to familiarise yourself with TopGit, the tool we
have been using for branch-based patch queue management. We've written
up a tutorial here:

== What's left? ==

We haven't included any patches for tickets CASSANDRA-4123 and
CASSANDRA-4124 which relate to the replication strategy and repair.
Currently replication and repair "just work" with the current patches
without any additional changes required.

CASSANDRA-4126 relates to testing. We're running virtual nodes builds
through our own test suites but we will also be writing new tests in

I look forward to your questions and comments!

Sam Overton
Acunu | | @acunu

View raw message