cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Francois Richard (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-7368) Compaction stops after org.apache.cassandra.io.sstable.CorruptSSTableException
Date Tue, 10 Jun 2014 19:17:01 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-7368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14026867#comment-14026867
] 

Francois Richard commented on CASSANDRA-7368:
---------------------------------------------

The scrub of the sstables completed and the problems does not seem to happen again.  Coming
back to the original problem, why did compaction totally stop on this node even if we have
the following setting:

concurrent_compactors:  26.

Should there be more resilience at the compaction level to keep on going when some SSTABLE
are corrupted? 


> Compaction stops after org.apache.cassandra.io.sstable.CorruptSSTableException
> ------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-7368
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7368
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: OS: RHEL 6.5
> Cassandra version: 1.2.16
>            Reporter: Francois Richard
>
> Hi,
> We are getting a case where compaction stops totally on a node after an exception related
to: org.apache.cassandra.io.sstable.CorruptSSTableException.
> nodetool compactionstats remains at the same level for hours:
> {code}
> pending tasks: 1451
>           compaction type        keyspace   column family       completed           total
     unit  progress
>                Compaction        SyncCoreContactPrefixBytesIndex       257799931    
  376785179     bytes    68.42%
> Active compaction remaining time :        n/a
> {code}
> Here is the exception log:
> {code}
> ERROR [Deserialize SSTableReader(path='/home/y/var/cassandra/data/SyncCore/ContactPrefixBytesIndex/SyncCore-ContactPrefixBytesIndex-ic-116118-Data.db')]
2014-06-09 06:39:37,570 CassandraDaemon.java (line 191) Exception in thread Thread[Deserialize
SSTableReader(path='/home/y/var/cassandra/data/SyncCore/ContactPrefixBytesIndex/SyncCore-ContactPrefixBytesIndex-ic-116118-Data.db'),1,main]
> org.apache.cassandra.io.sstable.CorruptSSTableException: java.io.IOException: dataSize
of 7421941880990663551 starting at 257836699 would be larger than file /home/y/var/cassandra/data/SyncCore/ContactPrefixBytesIndex/SyncCore-ContactPrefixBytesIndex-ic-116118-Data.db
length 376785179
> 	at org.apache.cassandra.io.sstable.SSTableIdentityIterator.<init>(SSTableIdentityIterator.java:167)
> 	at org.apache.cassandra.io.sstable.SSTableIdentityIterator.<init>(SSTableIdentityIterator.java:83)
> 	at org.apache.cassandra.io.sstable.SSTableIdentityIterator.<init>(SSTableIdentityIterator.java:69)
> 	at org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.next(SSTableScanner.java:180)
> 	at org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.next(SSTableScanner.java:155)
> 	at org.apache.cassandra.io.sstable.SSTableScanner.next(SSTableScanner.java:142)
> 	at org.apache.cassandra.io.sstable.SSTableScanner.next(SSTableScanner.java:38)
> 	at org.apache.cassandra.db.compaction.LeveledCompactionStrategy$LeveledScanner.computeNext(LeveledCompactionStrategy.java:238)
> 	at org.apache.cassandra.db.compaction.LeveledCompactionStrategy$LeveledScanner.computeNext(LeveledCompactionStrategy.java:207)
> 	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> 	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> --
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message