cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Ellis <>
Subject Re: Node failback scenario: How can clients be sure hinted handoff is completed ?
Date Fri, 16 Jul 2010 17:48:05 GMT
If you can't accept out of date data you shouldn't be reading at
CL.ONE.  Making HH more complex is not the answer.

On Fri, Jul 16, 2010 at 7:52 AM, Oleg Anastasjev <> wrote:
> Hello all,
> I currently working on testing of various HA scenarios on small cassandra
> cluster of 8 nodes, RF=3. I have a test environment with thrift clients doing
> double writes of all operations to cassandra cluster and reliable storage and
> cross checking read results. Reads are performed with CL=ONE due to latency
> requirements. I tested how fail over and fail back is working.
> I found, that on failback, a lot of data mismatches between reliable storage and
> cassandra was discovered as soon as failed back node started to accept reads.
> Later, as soon as hinted handoff was completed, no more mismatches was ever
> reported.
> So, the idea is, that even with CL=ONE we could have almost no inconsistencies
> on node failback, if cassandra node started to accept reads after hinted handoff
> is completed.
> So, the question is: is it possible for thrift client to know the current status
> of hinted handoff of just failed back node ?
> This way clients could wait for HH to complete to not query just failed back
> node and reroute queries to other endpoints, while failing back node
> synchronizes itself with cluster.

Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support

View raw message