Return-Path: X-Original-To: apmail-cassandra-commits-archive@www.apache.org Delivered-To: apmail-cassandra-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 46D2A10663 for ; Tue, 30 Jul 2013 20:11:49 +0000 (UTC) Received: (qmail 87177 invoked by uid 500); 30 Jul 2013 20:11:48 -0000 Delivered-To: apmail-cassandra-commits-archive@cassandra.apache.org Received: (qmail 87155 invoked by uid 500); 30 Jul 2013 20:11:48 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 87146 invoked by uid 99); 30 Jul 2013 20:11:48 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 30 Jul 2013 20:11:48 +0000 Date: Tue, 30 Jul 2013 20:11:48 +0000 (UTC) From: "Jonathan Ellis (JIRA)" To: commits@cassandra.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (CASSANDRA-5761) Issue with secondary index sstable. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CASSANDRA-5761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13724343#comment-13724343 ] Jonathan Ellis commented on CASSANDRA-5761: ------------------------------------------- bq. We usually used 28 chars key - first part of the key is event_id and the second is generated string numbers - but the length was 28. The problem started when for some event_id length has changed so the key length became 38. When we found who changed the event_id length and fixed that the problem has gone. But what if in the future we will change the length again? Sounds to me like your new keys were overlapping with your old ones unexpectedly. If you use a composite key, you won't have to worry about this happening. > Issue with secondary index sstable. > ----------------------------------- > > Key: CASSANDRA-5761 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5761 > Project: Cassandra > Issue Type: Bug > Components: Core > Affects Versions: 1.2.5 > Reporter: Andriy Yevsyukov > Priority: Critical > > With Cassandra 1.2.5 having issue very similar to [CASSANDRA-5225|https://issues.apache.org/jira/browse/CASSANDRA-5225] but for secondary index sstable. Every query that uses this index fails in Hector with ConnectionTimeout but cassandra log says that reason is: > {noformat} > ERROR [ReadStage:55803] 2013-07-15 12:11:35,392 CassandraDaemon.java (line 175) Exception in thread Thread[ReadStage:55803,5,main] > java.lang.RuntimeException: org.apache.cassandra.io.sstable.CorruptSSTableException: org.apache.cassandra.db.ColumnSerializer$CorruptColumnException: invalid column name length 0 (/data/cassandra/data/betting/events/betting-events.events_sport_type_idx-ic-1-Data.db, 19658 bytes remaining) > at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1582) > at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > Caused by: org.apache.cassandra.io.sstable.CorruptSSTableException: org.apache.cassandra.db.ColumnSerializer$CorruptColumnException: invalid column name length 0 (/data/cassandra/data/betting/events/betting-events.events_sport_type_idx-ic-1-Data.db, 19658 bytes remaining) > at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:108) > at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:39) > at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) > at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) > at org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:90) > at org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:171) > at org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:154) > at org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:143) > at org.apache.cassandra.utils.MergeIterator$ManyToOne.(MergeIterator.java:86) > at org.apache.cassandra.utils.MergeIterator.get(MergeIterator.java:45) > at org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:134) > at org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:84) > at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:293) > at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:65) > at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1357) > at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1214) > at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1126) > at org.apache.cassandra.db.index.keys.KeysSearcher$1.computeNext(KeysSearcher.java:140) > at org.apache.cassandra.db.index.keys.KeysSearcher$1.computeNext(KeysSearcher.java:109) > at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) > at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) > at org.apache.cassandra.db.ColumnFamilyStore.filter(ColumnFamilyStore.java:1466) > at org.apache.cassandra.db.index.keys.KeysSearcher.search(KeysSearcher.java:82) > at org.apache.cassandra.db.index.SecondaryIndexManager.search(SecondaryIndexManager.java:548) > at org.apache.cassandra.db.ColumnFamilyStore.search(ColumnFamilyStore.java:1454) > at org.apache.cassandra.service.RangeSliceVerbHandler.executeLocally(RangeSliceVerbHandler.java:44) > at org.apache.cassandra.service.StorageProxy$LocalRangeSliceRunnable.runMayThrow(StorageProxy.java:1076) > at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1578) > ... 3 more > Caused by: org.apache.cassandra.db.ColumnSerializer$CorruptColumnException: invalid column name length 0 (/data/cassandra/data/betting/events/betting-events.events_sport_type_idx-ic-1-Data.db, 19658 bytes remaining) > at org.apache.cassandra.db.ColumnSerializer$CorruptColumnException.create(ColumnSerializer.java:148) > at org.apache.cassandra.db.RangeTombstone$Serializer.deserializeBody(RangeTombstone.java:279) > at org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:90) > at org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:73) > at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:104) > ... 30 more > {noformat} > We cleared CF but we have a lot of insertion and in few hours the issue happened again. Are there any advices? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira