Return-Path: X-Original-To: apmail-cassandra-commits-archive@www.apache.org Delivered-To: apmail-cassandra-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0986C1068A for ; Thu, 6 Mar 2014 20:54:03 +0000 (UTC) Received: (qmail 72371 invoked by uid 500); 6 Mar 2014 20:53:54 -0000 Delivered-To: apmail-cassandra-commits-archive@cassandra.apache.org Received: (qmail 72282 invoked by uid 500); 6 Mar 2014 20:53:53 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 72111 invoked by uid 99); 6 Mar 2014 20:53:50 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 06 Mar 2014 20:53:50 +0000 Date: Thu, 6 Mar 2014 20:53:50 +0000 (UTC) From: "Michael Shuler (JIRA)" To: commits@cassandra.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (CASSANDRA-6285) 2.0 HSHA server introduces corrupt data MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CASSANDRA-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13923043#comment-13923043 ] Michael Shuler commented on CASSANDRA-6285: ------------------------------------------- With jna enabled, yes, on a single node, after running the attack jar and restarting c*, I get: {noformat} INFO [main] 2014-03-06 14:46:51,272 ColumnFamilyStore.java (line 254) Initializing tmp.CF INFO [main] 2014-03-06 14:46:51,277 ColumnFamilyStore.java (line 254) Initializing system_traces.sessions INFO [main] 2014-03-06 14:46:51,280 ColumnFamilyStore.java (line 254) Initializing system_traces.events INFO [main] 2014-03-06 14:46:51,281 CassandraDaemon.java (line 291) completed pre-loading (5 keys) key cache. INFO [main] 2014-03-06 14:46:51,288 CommitLog.java (line 130) Replaying /var/lib/cassandra/commitlog/CommitLog-3-1394138577628.log, /var/lib/ cassandra/commitlog/CommitLog-3-1394138577629.log INFO [main] 2014-03-06 14:46:51,311 CommitLogReplayer.java (line 184) Replaying /var/lib/cassandra/commitlog/CommitLog-3-1394138577628.log (C L version 3, messaging version 7) ERROR [main] 2014-03-06 14:46:51,432 CommitLogReplayer.java (line 306) Unexpected error deserializing mutation; saved to /tmp/mutation77387084 28696995512dat and ignored. This may be caused by replaying a mutation against a table with the same name but incompatible schema. Exception follows: org.apache.cassandra.serializers.MarshalException: Invalid version for TimeUUID type. at org.apache.cassandra.serializers.TimeUUIDSerializer.validate(TimeUUIDSerializer.java:39) at org.apache.cassandra.db.marshal.AbstractType.validate(AbstractType.java:172) at org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:276) at org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:97) at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:151) at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:131) at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:312) at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:471) at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:560) {noformat} I'll double-check a 3 node cluster, then patch and see where I get. > 2.0 HSHA server introduces corrupt data > --------------------------------------- > > Key: CASSANDRA-6285 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6285 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: 4 nodes, shortly updated from 1.2.11 to 2.0.2 > Reporter: David Sauer > Assignee: Pavel Yaskevich > Priority: Critical > Fix For: 2.0.6 > > Attachments: 6285_testnotes1.txt, CASSANDRA-6285-disruptor-heap.patch, compaction_test.py > > > After altering everything to LCS the table OpsCenter.rollups60 amd one other none OpsCenter-Table got stuck with everything hanging around in L0. > The compaction started and ran until the logs showed this: > ERROR [CompactionExecutor:111] 2013-11-01 19:14:53,865 CassandraDaemon.java (line 187) Exception in thread Thread[CompactionExecutor:111,1,RMI Runtime] > java.lang.RuntimeException: Last written key DecoratedKey(1326283851463420237, 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574426c6f6f6d46696c746572537061636555736564) >= current key DecoratedKey(954210699457429663, 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574546f74616c4469736b5370616365557365640b0f) writing into /var/lib/cassandra/data/OpsCenter/rollups60/OpsCenter-rollups60-tmp-jb-58656-Data.db > at org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:141) > at org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:164) > at org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:160) > at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48) > at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60) > at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) > at org.apache.cassandra.db.compaction.CompactionManager$6.runMayThrow(CompactionManager.java:296) > at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:724) > Moving back to STC worked to keep the compactions running. > Especialy my own Table i would like to move to LCS. > After a major compaction with STC the move to LCS fails with the same Exception. -- This message was sent by Atlassian JIRA (v6.2#6252)