incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Clint Kelly <clint.ke...@gmail.com>
Subject Meaning of "token" column in system.peers and system.local
Date Sun, 30 Mar 2014 23:51:10 GMT
Hi all,


I am working on a Hadoop InputFormat implementation that uses only the
native protocol Java driver and not the Thrift API.  I am currently trying
to replicate some of the behavior of
*Cassandra.client.describe_ring(myKeyspace)* from the Thrift API.  I would
like to do the following:

   - Get a list of all of the token ranges for a cluster
   - For every token range, determine the replica nodes on which the data
   in the token range resides
   - Estimate the number of rows for every range of tokens
   - Groups ranges of tokens on common replica nodes such that we can
   create a set of input splits for Hadoop with total estimated line counts
   that are reasonably close to the requested split size

Last week I received some much-appreciated help on this list that pointed
me to using the system.peers table to get the list of token ranges for the
cluster and the corresponding hosts.  Today I created a three-node C*
cluster in Vagrant (https://github.com/dholbrook/vagrant-cassandra) and
tried inspecting some of the system tables.  I have a couple of questions
now:

1. *How many total unique tokens should I expect to see in my cluster?*  If
I have three nodes, and each node has a cassandra.yaml with num_tokens =
256, then should I expect a total of 256*3 = 768 distinct vnodes?

2. *How does the creation of vnodes and their assignment to nodes relate to
the replication factor for a given keyspace?*  I never thought about this
until today, and I tried to reread the documentation on virtual nodes,
replication in Cassandra, etc., and now I am sadly still confused.  Here is
what I think I understand.  :)

   - Given a row with a partition key, any client request for an operation
   on that row will go to a coordinator node in the cluster.
   - The coordinator node will compute the token value for the row and from
   that determine a set of replica nodes for that token.
      - One of the replica nodes I assume is the node that "owns" the vnode
      with the token range that encompasses the token
      - The identity of the "owner" of this virtual node is a
      cross-keyspace property
      - And the other replicas were originally chosen based on the
      replica-placement strategy
      - And therefore the other replicas will be different for each
      keyspace (because replication factors and replica-placement strategy are
      properties of a keyspace)

3. What do the values in the "token" column in system.peers and
system.local refer to then?

   - Since these tables appear to be global, and not per-keyspace
   properties, I assume that they don't have any information about replication
   in them, is that correct?
   - If I have three nodes in my cluster, 256 vnodes per node, and I'm
   using the Murmur3 partitioner, should I then expect to see the values of
   "tokens" in system.peers and system.local be 768 evenly-distributed values
   between -2^63 and 2^63?

4. Is there any other way, without using Thift, to get as much information
as possible about what nodes contain replicas of data for all of the token
ranges in a given cluster?

I really appreciate any help, thanks!

Best regards,
Clint

Mime
View raw message