incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Clint Kelly <>
Subject Re: Meaning of "token" column in system.peers and system.local
Date Mon, 31 Mar 2014 02:06:54 GMT
BTW one other thing that I have not been able to debug today that maybe
someone can help me with:

I am using a three-node Cassandra cluster with Vagrant.  The nodes in my
cluster are,, and

If I use cqlsh to connect to, I see unique sets of tokens
when I run the following three commands:

select tokens from system.local
select tokens from system.peers where peer=
select tokens from system.peers where peer=

This is what I expect.  However, when I tried making an application with
the Java driver that does the following:

   - Create a Session by connecting to
   - From that session, "select tokens from system.local"
   - From that session, "select tokens, peer from system.peers"

Now I get the exact-same set of tokens from system.local and from the row
in system.peers in which peer=

Anyone have any idea why this would happen?  I'm not sure how to debug
this.  I see the following log from the Java driver:

14/03/30 19:05:24 DEBUG com.datastax.driver.core.Cluster: Starting new
cluster with contact points [/]
14/03/30 19:05:24 INFO com.datastax.driver.core.Cluster: New Cassandra host
/ added
14/03/30 19:05:24 INFO com.datastax.driver.core.Cluster: New Cassandra host
/ added

I'm running Cassandra 2.0.6 in the virtual machine and I built my
application with version 2.0.1 of the driver.

Best regards,

On Sun, Mar 30, 2014 at 4:51 PM, Clint Kelly <> wrote:

> Hi all,
> I am working on a Hadoop InputFormat implementation that uses only the
> native protocol Java driver and not the Thrift API.  I am currently trying
> to replicate some of the behavior of
> *Cassandra.client.describe_ring(myKeyspace)* from the Thrift API.  I
> would like to do the following:
>    - Get a list of all of the token ranges for a cluster
>    - For every token range, determine the replica nodes on which the data
>    in the token range resides
>    - Estimate the number of rows for every range of tokens
>    - Groups ranges of tokens on common replica nodes such that we can
>    create a set of input splits for Hadoop with total estimated line counts
>    that are reasonably close to the requested split size
> Last week I received some much-appreciated help on this list that pointed
> me to using the system.peers table to get the list of token ranges for the
> cluster and the corresponding hosts.  Today I created a three-node C*
> cluster in Vagrant ( and
> tried inspecting some of the system tables.  I have a couple of questions
> now:
> 1. *How many total unique tokens should I expect to see in my cluster?*
> If I have three nodes, and each node has a cassandra.yaml with num_tokens =
> 256, then should I expect a total of 256*3 = 768 distinct vnodes?
> 2. *How does the creation of vnodes and their assignment to nodes relate
> to the replication factor for a given keyspace?*  I never thought about
> this until today, and I tried to reread the documentation on virtual nodes,
> replication in Cassandra, etc., and now I am sadly still confused.  Here is
> what I think I understand.  :)
>    - Given a row with a partition key, any client request for an
>    operation on that row will go to a coordinator node in the cluster.
>    - The coordinator node will compute the token value for the row and
>    from that determine a set of replica nodes for that token.
>       - One of the replica nodes I assume is the node that "owns" the
>       vnode with the token range that encompasses the token
>       - The identity of the "owner" of this virtual node is a
>       cross-keyspace property
>       - And the other replicas were originally chosen based on the
>       replica-placement strategy
>       - And therefore the other replicas will be different for each
>       keyspace (because replication factors and replica-placement strategy are
>       properties of a keyspace)
> 3. What do the values in the "token" column in system.peers and
> system.local refer to then?
>    - Since these tables appear to be global, and not per-keyspace
>    properties, I assume that they don't have any information about replication
>    in them, is that correct?
>    - If I have three nodes in my cluster, 256 vnodes per node, and I'm
>    using the Murmur3 partitioner, should I then expect to see the values of
>    "tokens" in system.peers and system.local be 768 evenly-distributed values
>    between -2^63 and 2^63?
> 4. Is there any other way, without using Thift, to get as much information
> as possible about what nodes contain replicas of data for all of the token
> ranges in a given cluster?
> I really appreciate any help, thanks!
> Best regards,
> Clint

View raw message