incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rene Kochen <rene.koc...@schange.com>
Subject Re: Cluster processing not balanced
Date Fri, 28 Jun 2013 15:14:53 GMT
I found the issue.

Stupid mistake.

Although I have 1000000 customers in the cluster, I do a get_range_slices
to get the first 10000 customers. I use these customers as a seed for my
test program. I random select customers from that set and fire requests at
the cluster. Those 10000 customers all reside on the first node (actually
the first three nodes because of the replication factor of three).

Thanks,

Rene


2013/6/28 Rene Kochen <rene.kochen@schange.com>

> Hi All,
>
> I have the following situation:
>
> - Cassandra 1.0.11
> - A 6 node cluster
> - Random partitioner
> - Tokens are balanced (according to node-tool)
> - Data-load is balanced (according to node-tool)
>
> I have a customers column-family with 1000000 customers. I also have a
> test client which requests random customers from the cluster in a
> round-robin fashion, i.e. each request is targeted to a different machine
> and each request contains a random customer-id (a number in the range of 0
> - 1000000).
>
> What I see is that one node (node 4) is only proxying. Node 4 does receive
> requests. I can see that in the storage-proxy counters. However, I see in
> the Customers column-family a zero in the ReadCount counter (local reads).
> Node 4 should also serve reads because I have 1000000 customers in the
> cluster, the cluster is balanced and I do requests with random
> customer-ids. In fact, two of the four nodes have this behavior. The result
> is that the processing load within the cluster is not balanced.
>
> I have never witnessed this behavior before.
>
> Am I missing something?
>
> Thanks,
>
> Rene
>

Mime
View raw message