incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Ellis <jbel...@gmail.com>
Subject Re: ConsistencyLevel greater ONE + node failure = non-responsive Cassandra 0.6.5 cluster
Date Mon, 21 Mar 2011 17:54:26 GMT
I suggest upgrading to either 0.6.12 or 0.7.4 and re-testing.

On Mon, Mar 21, 2011 at 12:52 PM, Markus Klems <markusklems@gmail.com> wrote:
> Hi guys,
>
> we are currently benchmarking various configurations of an EC2-based
> Cassandra cluster. This is our current setup:
>
> 1) 8 nodes where each node is an m1.xlarge EC2 instance
> 2) Cassandra version 0.6.5
> 3) Replication Factor = 3
> 4) this delivers ~7K to 10K ops/sec with 50% GET and 50% INSERT
> depending on the consistency level
>
> We have been benchmarking the cluster with YCSB, while altering the
> consistency levels ONE, QUORUM, and ALL, ceteris paribus. This works
> fine if all nodes are alive. Then, we wanted to benchmark the cluster
> performance behavior when one node goes down. So, we killed one node
> and tested the cluster with consistency level ONE, which delivered
> reasonable throughput of multiple thousand ops/sec. Then, we wanted to
> test QUORUM and ALL. However, when one node is down, the cluster
> throughput sharply drops to a few operations and then stops responding
> to the YCSB client if the consistency level of operations in the
> benchmark is set to QUORUM or ALL. For ALL, this behavior would (kind
> of) make sense for read requests but we are puzzled that even QUORUM
> won't work. And for 100% write operations in consistency level ALL it
> won't work either.
>
> Any ideas why the cluster stops responding for QUORUM and ALL?
>
> Thanks,
>
> Markus
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com

Mime
View raw message