incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suruchi Deodhar <suruchi.deod...@generalsentiment.com>
Subject Re: Cassandra 1.2.9 cluster with vnodes is heavily unbalanced.
Date Tue, 24 Sep 2013 22:12:18 GMT
As an update to this thread, we conducted several tests with
Cassandra-1.2.9, varying parameters such as partitioner
(Murmur3Partitioner/RandomParttioner), using NetworkToplogyStrategy (with
Ec2Snitch) / SimpleStrategy (with SimpleSnitch) across 2 Availability zones
and 1 AZ. We also tested the configurations separately with vnodes and
without vnodes.

Every time before each test, we wiped the cassandra cluster data and
commitlog folders and restarted with an empty cassandra db. However, in all
the cases using 1.2.9 we continued to see very heavy imbalance across the
nodes as reported in this thread.

We then tested the same exports with cassandra 1.2.5 version that we had
been testing previously (without vnodes across 2 AZs) and the data was
balanced across the nodes of the cluster. The output from bin/nodetool
status is attached.

Was there some change from 1.2.5 to 1.2.9 that could be responsible for the
imbalance or is there some parameter setting that we may have completely
missed in our configuration wrt 1.2.9? Has anyone else experienced such an
imbalance issue?

Also,  we were contemplating on using vnodes with NetworkTopologyStrategy
(We want to replicate data across 2 AZs)
We came across the following links that mention that vnodes with
NetworkToplogyStrategy may create hotspots and the issue is marked as Open.
Does that mean using vnodes with NetworkToplogyStrategy is a bad idea?

[ https://issues.apache.org/jira/browse/CASSANDRA-4658 ,
https://issues.apache.org/jira/browse/CASSANDRA-3810 ,
https://issues.apache.org/jira/browse/CASSANDRA-4123 ] .

Thanks again for all your replies.

Suruchi





On Fri, Sep 20, 2013 at 7:04 PM, Robert Coli <rcoli@eventbrite.com> wrote:

> On Fri, Sep 20, 2013 at 3:42 PM, Suruchi Deodhar <
> suruchi.deodhar@generalsentiment.com> wrote:
>
>> Using the nodes in the same availability zone(us-east-1b), we still get a
>> highly imbalanced cluster. The nodetool status and ring output is attached.
>> Even after running repairs, the cluster does not seem to balance.
>>
>
> If your cluster doesn't experience exceptions when loading and/or store a
> lot of hints, repair is almost certainly just wasting your and your CPU's
> time.
>
> =Rob
>

Mime
View raw message