incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Morton <aa...@thelastpickle.com>
Subject Re: How does Cassandra handle failure during synchronous writes
Date Wed, 23 Feb 2011 19:43:35 GMT
At CL levels high than ANY hinted handoff will be used if enabled. It does not contribute to
the number of replicas considered written by the coordinator though. E.g. If you ask for quorum,
and this is 3 nodes, and only 2 are up the write will fail without starting. In this case
the HH is included in the message sent to one of the up nodes.

At CL any HH is accepted as a viable replica. Even if all the natural endpoints are down the
coordinator node will store the HH.

aaron
On 24/02/2011, at 3:28 AM, Javier Canillas <javier.canillas@gmail.com> wrote:
> There is something call Hinted Handoff. Suppose that you WRITE something with ConsistencyLevel.ONE
on a cluster defined by 4 nodes. Then, the write is done on the corresponding node and it
is returned an OK to the client, even if the ReplicationFactor over the destination Keyspace
is set to a higher value.
> 
> If in that write, one of the replicated nodes is down, then the coordinator node (the
one that will hold value if first place) will mark that replication message as not sent and
will retry eventually, making the replication happens.
> 
> Please, if I have explained it wrongly correct me. 
> 
> On Wed, Feb 23, 2011 at 5:45 AM, Aaron Morton <aaron@thelastpickle.com> wrote:
> In the case described below if less than CL nodes respond in rpc_timeout (from conf yaml)
the client will get a timeout error. I think most higher level clients will automatically
retry in this case.
> 
> If there are not enough nodes to start the request you will get an Unavailable exception.
Again the client can retry safely.
> 
> Aaron
> 
> 
> On 23/02/2011, at 8:07 PM, Dave Revell <dave@meebo-inc.com> wrote:
> 
>> Ritesh,
>> 
>> There is no commit protocol. Writes may be persisted on some replicas even though
the quorum fails. Here's a sequence of events that shows the "problem:"
>> 
>> 1. Some replica R fails, but recently, so its failure has not yet been detected
>> 2. A client writes with consistency > 1
>> 3. The write goes to all replicas, all replicas except R persist the write to disk
>> 4. Replica R never responds
>> 5. Failure is returned to the client, but the new value is still in the cluster,
on all replicas except R.
>> 
>> Something very similar could happen for CL QUORUM.
>> 
>> This is a conscious design decision because a commit protocol would constitute tight
coupling between nodes, which goes against the Cassandra philosophy. But unfortunately you
do have to write your app with this case in mind.
>> 
>> Best,
>> Dave
>> 
>> On Tue, Feb 22, 2011 at 8:22 PM, tijoriwala.ritesh <tijoriwala.ritesh@gmail.com>
wrote:
>> 
>> Hi,
>> I wanted to get details on how does cassandra do synchronous writes to W
>> replicas (out of N)? Does it do a 2PC? If not, how does it deal with
>> failures of of nodes before it gets to write to W replicas? If the
>> orchestrating node cannot write to W nodes successfully, I guess it will
>> fail the write operation but what happens to the completed writes on X (W >
>> X) nodes?
>> 
>> Thanks,
>> Ritesh
>> --
>> View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/How-does-Cassandra-handle-failure-during-synchronous-writes-tp6055152p6055152.html
>> Sent from the cassandra-user@incubator.apache.org mailing list archive at Nabble.com.
>> 
> 

Mime
View raw message