cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aaron morton <aa...@thelastpickle.com>
Subject Re: sync commitlog in batch mode lose data
Date Tue, 31 May 2011 10:15:14 GMT
In your step 5 which node are you connected to ? 

There is a possibility that during the time node A was turning off, mutations which had been
started were completed on node B. In which case the client would have gotten a TimedOutExceptions.
Meaning the operation did not complete at the required CL, but was started and probably written
to node B. 

In the scenario you describe it's possible for a key to be on node B and not node A. What
happens you do you the read in step 5 at QUORUM?  

AFAIK configuring the commit log as you have will result in every write blocking on the commit
log sync'ing.
 
Hope that helps. 

-----------------
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com

On 31 May 2011, at 20:46, Preston Chang wrote:

> My RF is 2.
> 
> When the node A is down, the commit log should be fsynced to disk in my test scene, so
there should be no NOTFOUND key, but there are some NOTFOUND keys. I am puzzled.
> 
> 2011/5/31 Maki Watanabe <watanabe.maki@gmail.com>
> How much replication factor did you set for the keyspace?
> If the RF is 2, your data should be replicated to both of nodes. If
> the RF is 1, you will lose the half of data when the node A is down.
> 
> maki
> 
> 
> 2011/5/31 Preston Chang <zhangyf2007@gmail.com>:
> > Hi,
> > I have a cluster with two nodes (node A and node B) and make a test as
> > follows:
> > 1). set commitlog sync in batch mode and the sync batch window in 0 ms
> > 2). one client wrote random keys in infinite loop with consistency level
> > QUORUM and record the keys in file after the insert() method return normally
> > 3). unplug one server (node A) power cord
> > 4). restart the server and cassandra service
> > 5). read the key list generated in step 2) with consistency level ONE
> > I thought the result of test is all the key in list can be read normally,
> > but actually there are some NotFound keys.
> > My question is why there are NotFound keys. In my opinion server would not
> > ack the client before finishing syncing the commitlog if I set commitlog
> > sync in batch mode and the sync batch window in 0 ms. So if the insert()
> > method return normally it means the mutation had been written in commitlog
> > and the commitlog had been synced to the disk. Am I right?
> > My Cassandra version is 0.7.3.
> > Thanks for your help very much.
> > --
> > by Preston Chang
> >
> >
> 
> 
> 
> --
> w3m
> 
> 
> 
> -- 
> by Preston Chang
> 


Mime
View raw message