cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Griffith (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-7408) System hints corruption - dataSize ... would be larger than file
Date Wed, 11 Nov 2015 19:37:11 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-7408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15000950#comment-15000950
] 

Jeff Griffith commented on CASSANDRA-7408:
------------------------------------------

no problem [~iamaleksey]  i seem to recall this being related to an issue i reported separately
where a short integer was overflowing. pretty sure it's all good now.

> System hints corruption - dataSize ... would be larger than file
> ----------------------------------------------------------------
>
>                 Key: CASSANDRA-7408
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7408
>             Project: Cassandra
>          Issue Type: Bug
>         Environment: RHEL 6.5
> Cassandra 1.2.16
> RF=3
> Thrift
>            Reporter: Jeff Griffith
>
> I've found several unresolved JIRA tickets related to SSTable corruption but not sure
if they apply to the case we are seeing in system/hints. We see periodic exceptions such as:
> {noformat}
> dataSize of 144115248479299639 starting at 17209 would be larger than file /home/y/var/cassandra/data/system/hints/system-hints-ic-219-Data.db
length 35542
> {noformat}
> Is there something we could possibly be doing from the application to cause this sort
of corruption? We also see it on some of our own column families also some *negative* lengths
which are presumably a similar corruption.
> {noformat}
> ERROR [HintedHandoff:57] 2014-06-17 17:08:04,690 CassandraDaemon.java (line 191) Exception
in thread Thread[HintedHandoff:57,1,main]
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: org.apache.cassandra.io.sstable.CorruptSSTableException:
java.io.IOException: dataSize of 144115248479299639 starting at 17209 would be larger than
file /home/y/var/cassandra/data/system/hints/system-hints-ic-219-Data.db length 35542
>         at org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:441)
>         at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:282)
>         at org.apache.cassandra.db.HintedHandOffManager.access$300(HintedHandOffManager.java:90)
>         at org.apache.cassandra.db.HintedHandOffManager$4.run(HintedHandOffManager.java:508)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException: org.apache.cassandra.io.sstable.CorruptSSTableException:
java.io.IOException: dataSize of 144115248479299639 starting at 17209 would be larger than
file /home/y/var/cassandra/data/system/hints/system-hints-ic-219-Data.db length 35542
>         at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:188)
>         at org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:437)
>         ... 6 more
> Caused by: org.apache.cassandra.io.sstable.CorruptSSTableException: java.io.IOException:
dataSize of 144115248479299639 starting at 17209 would be larger than file /home/y/var/cassandra/data/system/hints/system-hints-ic-219-Data.db
length 35542
>         at org.apache.cassandra.io.sstable.SSTableIdentityIterator.<init>(SSTableIdentityIterator.java:167)
>         at org.apache.cassandra.io.sstable.SSTableIdentityIterator.<init>(SSTableIdentityIterator.java:83)
>         at org.apache.cassandra.io.sstable.SSTableIdentityIterator.<init>(SSTableIdentityIterator.java:69)
>         at org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.next(SSTableScanner.java:180)
>         at org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.next(SSTableScanner.java:155)
>         at org.apache.cassandra.io.sstable.SSTableScanner.next(SSTableScanner.java:142)
>         at org.apache.cassandra.io.sstable.SSTableScanner.next(SSTableScanner.java:38)
>         at org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:145)
>         at org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:122)
>         at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:96)
>         at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>         at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
>         at org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:145)
>         at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>         at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:58)
>         at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
>         at org.apache.cassandra.db.compaction.CompactionManager$7.runMayThrow(CompactionManager.java:442)
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>         ... 3 more
> Caused by: java.io.IOException: dataSize of 144115248479299639 starting at 17209 would
be larger than file /home/y/var/cassandra/data/system/hints/system-hints-ic-219-Data.db length
35542
>         at org.apache.cassandra.io.sstable.SSTableIdentityIterator.<init>(SSTableIdentityIterator.java:123)
>         ... 23 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message