If you are reading at QUOURM there is no problem, this is how eventual consistency works in Cassandra.

The coordinator will resolve the differences between and the column with the higher timestamp will win. 

If the delete was applied to less then CL nodes the client should have received a TimedOutException.

 
Cheers
 
-----------------
Aaron Morton
Freelance Developer
@aaronmorton

On 28/06/2012, at 7:41 PM, 黄荣桢 wrote:

Background: My application is running on a cluster system(which have 4 nodes), and system time of these four nodes are synchronizing by NTP. I use Write.QUORUM and Read.QUORUM strategy. The probability of this problem is not very high. Cassandra version is 1.0.3, I have tried Cassandra 1.1.1, this problem is still exist.

Problem: I deleted a column, but after 6 seconds, Cassandra can still get the old record which "isMarkedForDelete" is still false.

Is anybody meet the same problem? And how to solve it?

Detail: See the log below:

Node 3(Local node):
[pool-2-thread-42] 2012-06-27 14:49:23,732 SliceQueryFilter.java (line 123) collecting 0 of 2147483647: SuperColumn(667072 [......7fffffffffffffff000001382ca96c8b636b698a:false:36@1340779097312016,......)

[pool-2-thread-44] 2012-06-27 14:51:21,367 StorageProxy.java (line 172) Mutations/ConsistencyLevel are [RowMutation(keyspace='drc', key='3332', modifications=[ColumnFamily(fpr_index [SuperColumn(667072 [7fffffffffffffff000001382ca96c8b636b698a:true:4@1340779881338000,]),])])]/QUORUM

-- I delete this record at 14:51:21,367

[pool-2-thread-37] 2012-06-27 14:51:27,400 SliceQueryFilter.java (line 123) collecting 0 of 2147483647: SuperColumn(667072 [......,7fffffffffffffff000001382ca96c8b636b698a:false:36@1340779097312016,......)

-- But I can still get the old record at 14:51:27,400

Node2:
[MutationStage:118] 2012-06-27 14:51:21,373 RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='drc', key='3332', modifications=[ColumnFamily(fpr_index [SuperColumn(667072 [7fffffffffffffff000001382ca96c8b636b698a:true:4@1340779881338000,]),])])

[MutationStage:118] 2012-06-27 14:51:21,374 RowMutationVerbHandler.java (line 60) RowMutation(keyspace='drc', key='3332', modifications=[ColumnFamily(fpr_index [SuperColumn(667072 [7fffffffffffffff000001382ca96c8b636b698a:true:4@1340779881338000,]),])]) applied. Sending response to 6692098@/192.168.0.3

[MutationStage:123] 2012-06-27 14:51:27,405 RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='drc', key='3332', modifications=[ColumnFamily(fpr_index [SuperColumn(667072 [......,7fffffffffffffff000001382ca96c8b636b698a:false:36@1340779097312016,......])

[MutationStage:123] 2012-06-27 14:51:27,405 RowMutationVerbHandler.java (line 60) RowMutation(keyspace='drc', key='3332', modifications=[ColumnFamily(fpr_index [SuperColumn(667072 [......,7fffffffffffffff000001382ca96c8b636b698a:false:36@1340779097312016,.......]),])]) applied. Sending response to 6698516@/192.168.0.3

Node1:
[MutationStage:98] 2012-06-27 14:51:24,661 RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='drc', key='3332', modifications=[ColumnFamily(fpr_index [SuperColumn(667072 [7fffffffffffffff000001382ca96c8b636b698a:true:4@1340779881338000,]),])])

[MutationStage:98] 2012-06-27 14:51:24,675 RowMutationVerbHandler.java (line 60) RowMutation(keyspace='drc', key='3332', modifications=[ColumnFamily(fpr_index [SuperColumn(667072 [7fffffffffffffff000001382ca96c8b636b698atrue :4@1340779881338000,]),])]) applied. Sending response to 6692099@/192.168.0.3

[MutationStage:93] 2012-06-27 14:51:40,932 RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='drc', key='3332', modifications=[ColumnFamily(fpr_index [SuperColumn(667072 [7fffffffffffffff000001382ca96c8b636b698a:true:4@1340779900915004,]),])])

DEBUG [MutationStage:93] 2012-06-27 14:51:40,933 RowMutationVerbHandler.java (line 60) RowMutation(keyspace='drc', key='3332', modifications=[ColumnFamily(fpr_index [SuperColumn(667072 [7fffffffffffffff000001382ca96c8b636b698atrue :4@1340779900915004,]),])]) applied. Sending response to 6706555@/192.168.0.3

[ReadStage:55] 2012-06-27 14:51:43,074 SliceQueryFilter.java (line 123) collecting 0 of 5000:7fffffffffffffff000001382ca96c8b636b698a:true:4@1340779900915004

Node 4:

There is no log about this record on Node 4.