cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sylvain Lebresne (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-5761) Issue with secondary index sstable.
Date Wed, 24 Jul 2013 12:35:50 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-5761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13718314#comment-13718314
] 

Sylvain Lebresne commented on CASSANDRA-5761:
---------------------------------------------

I've added an assertion in commit 9ae960a1a4f57e3c9ec018f3cbb32fd3312d7a6e to validate at
the time of the index update that we don't write an empty column name. But what you describe
should not happen, so short of being able to reproduce, not sure what else we can do. Are
you sure you don't have broken disk?
                
> Issue with secondary index sstable.
> -----------------------------------
>
>                 Key: CASSANDRA-5761
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5761
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.2.5
>            Reporter: Andriy Yevsyukov
>            Priority: Critical
>
> With Cassandra 1.2.5 having issue very similar to [CASSANDRA-5225|https://issues.apache.org/jira/browse/CASSANDRA-5225]
but for secondary index sstable. Every query that uses this index fails in Hector with ConnectionTimeout
but cassandra log says that reason is:
> {noformat}
> ERROR [ReadStage:55803] 2013-07-15 12:11:35,392 CassandraDaemon.java (line 175) Exception
in thread Thread[ReadStage:55803,5,main]
> java.lang.RuntimeException: org.apache.cassandra.io.sstable.CorruptSSTableException:
org.apache.cassandra.db.ColumnSerializer$CorruptColumnException: invalid column name length
0 (/data/cassandra/data/betting/events/betting-events.events_sport_type_idx-ic-1-Data.db,
19658 bytes remaining)
> 	at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1582)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> 	at java.lang.Thread.run(Thread.java:662)
> Caused by: org.apache.cassandra.io.sstable.CorruptSSTableException: org.apache.cassandra.db.ColumnSerializer$CorruptColumnException:
invalid column name length 0 (/data/cassandra/data/betting/events/betting-events.events_sport_type_idx-ic-1-Data.db,
19658 bytes remaining)
> 	at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:108)
> 	at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:39)
> 	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> 	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> 	at org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:90)
> 	at org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:171)
> 	at org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:154)
> 	at org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:143)
> 	at org.apache.cassandra.utils.MergeIterator$ManyToOne.<init>(MergeIterator.java:86)
> 	at org.apache.cassandra.utils.MergeIterator.get(MergeIterator.java:45)
> 	at org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:134)
> 	at org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:84)
> 	at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:293)
> 	at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:65)
> 	at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1357)
> 	at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1214)
> 	at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1126)
> 	at org.apache.cassandra.db.index.keys.KeysSearcher$1.computeNext(KeysSearcher.java:140)
> 	at org.apache.cassandra.db.index.keys.KeysSearcher$1.computeNext(KeysSearcher.java:109)
> 	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> 	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> 	at org.apache.cassandra.db.ColumnFamilyStore.filter(ColumnFamilyStore.java:1466)
> 	at org.apache.cassandra.db.index.keys.KeysSearcher.search(KeysSearcher.java:82)
> 	at org.apache.cassandra.db.index.SecondaryIndexManager.search(SecondaryIndexManager.java:548)
> 	at org.apache.cassandra.db.ColumnFamilyStore.search(ColumnFamilyStore.java:1454)
> 	at org.apache.cassandra.service.RangeSliceVerbHandler.executeLocally(RangeSliceVerbHandler.java:44)
> 	at org.apache.cassandra.service.StorageProxy$LocalRangeSliceRunnable.runMayThrow(StorageProxy.java:1076)
> 	at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1578)
> 	... 3 more
> Caused by: org.apache.cassandra.db.ColumnSerializer$CorruptColumnException: invalid column
name length 0 (/data/cassandra/data/betting/events/betting-events.events_sport_type_idx-ic-1-Data.db,
19658 bytes remaining)
> 	at org.apache.cassandra.db.ColumnSerializer$CorruptColumnException.create(ColumnSerializer.java:148)
> 	at org.apache.cassandra.db.RangeTombstone$Serializer.deserializeBody(RangeTombstone.java:279)
> 	at org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:90)
> 	at org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:73)
> 	at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:104)
> 	... 30 more
> {noformat}
> We cleared CF but we have a lot of insertion and in few hours the issue happened again.
Are there any advices?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message