incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Vanderfeesten <feest...@gmail.com>
Subject failing reconcilliation during counter increment?
Date Fri, 30 Nov 2012 01:44:27 GMT
Hi

We are running  a Cassandra 1.0.7, 2 node setup  on EC2 using ephemeral
disks for storage.
Debugging inconsistent dara between 2 nodes we saw logging  below.
Following the counter CF design, I assume this means the READ step
reconciliating the memtable value with the SSTables on the replica
receiving the increment is failing.
To my understanding client will not be notified because this read step is
asynchronously. Also the other replica cannot be updated properly
(replicate-on-write stage is backing up).
Question is:
- Will the counter value eventually become correct?
- Read repair is turned off for the moment, but would this (not
reconciliated) memtable  value be merged correctly if read repair would be
enabled?
- What with the final value after running nodetool repairs or compaction
process...

Load shouldn't be that high on this trial setup but we need to investigate
more before concluding.

Txs,

David


INFO [ScheduledTasks:1] 2012-11-28 16:58:28,924 MessagingService.java (line
607) 1175 READ messages dropped in last 5000ms
 INFO [ScheduledTasks:1] 2012-11-28 16:58:28,924 StatusLogger.java (line
50) Pool Name Active Pending Blocked
 INFO [ScheduledTasks:1] 2012-11-28 16:58:28,925 StatusLogger.java (line
65) ReadStage 0 0 0
 INFO [ScheduledTasks:1] 2012-11-28 16:58:28,925 StatusLogger.java (line
65) RequestResponseStage 0 0 0
 INFO [ScheduledTasks:1] 2012-11-28 16:58:28,925 StatusLogger.java (line
65) ReadRepairStage 0 0 0
 INFO [ScheduledTasks:1] 2012-11-28 16:58:28,926 StatusLogger.java (line
65) MutationStage 0 0 0
 INFO [ScheduledTasks:1] 2012-11-28 16:58:28,926 StatusLogger.java (line
65) ReplicateOnWriteStage 32 106 0
 INFO [ScheduledTasks:1] 2012-11-28 16:58:28,926 StatusLogger.java (line
65) GossipStage 0 0 0
 INFO [ScheduledTasks:1] 2012-11-28 16:58:28,926 StatusLogger.java (line
65) AntiEntropyStage 0 0 0
 INFO [ScheduledTasks:1] 2012-11-28 16:58:28,927 StatusLogger.java (line
65) MigrationStage 0 0 0
 INFO [ScheduledTasks:1] 2012-11-28 16:58:28,927 StatusLogger.java (line
65) StreamStage 0 0 0
 INFO [ScheduledTasks:1] 2012-11-28 16:58:28,927 StatusLogger.java (line
65) MemtablePostFlusher 0 0 0
 INFO [ScheduledTasks:1] 2012-11-28 16:58:28,928 StatusLogger.java (line
65) FlushWriter 0 0 0
 INFO [ScheduledTasks:1] 2012-11-28 16:58:28,928 StatusLogger.java (line
65) MiscStage 0 0 0
 INFO [ScheduledTasks:1] 2012-11-28 16:58:28,928 StatusLogger.java (line
65) InternalResponseStage 0 0 0
 INFO [ScheduledTasks:1] 2012-11-28 16:58:28,928 StatusLogger.java (line
65) AntiEntropySessions 0 0 0
 INFO [ScheduledTasks:1] 2012-11-28 16:58:28,929 StatusLogger.java (line
65) HintedHandoff 0 0 0
 INFO [ScheduledTasks:1] 2012-11-28 16:58:28,929 StatusLogger.java (line
69) CompactionManager n/a 2
 INFO [ScheduledTasks:1] 2012-11-28 16:58:28,929 StatusLogger.java (line
81) MessagingService n/a 0,0

Mime
View raw message