cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sammy Yu (JIRA)" <j...@apache.org>
Subject [jira] Updated: (CASSANDRA-452) Corrupt SSTable
Date Tue, 22 Sep 2009 22:08:17 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sammy Yu updated CASSANDRA-452:
-------------------------------

    Description: 
We noticed on one of our node the number of SStables is growing.  The compaction thread is
alive and running.  We can see that it is constantly trying to compact the same set of sstables.
 However, it is failing because one of the sstable is corrupt:

ERROR [ROW-READ-STAGE:475] 2009-09-21 00:29:17,068 DebuggableThreadPoolExecutor.java (line
125) Error in ThreadPoolExecutor
java.lang.RuntimeException: java.io.EOFException
        at org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:110)
        at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:44)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.EOFException
        at java.io.RandomAccessFile.readFully(RandomAccessFile.java:383)
        at java.io.RandomAccessFile.readFully(RandomAccessFile.java:361)
        at org.apache.cassandra.utils.FBUtilities.readByteArray(FBUtilities.java:390)
        at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:64)
        at org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:349)
        at org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:309)
        at org.apache.cassandra.db.filter.SSTableNamesIterator.<init>(SSTableNamesIterator.java:102)
        at org.apache.cassandra.db.filter.NamesQueryFilter.getSSTableColumnIterator(NamesQueryFilter.java:69)
        at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1467)
        at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1401)
        at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1420)
        at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1401)
        at org.apache.cassandra.db.Table.getRow(Table.java:589)
        at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:65)
        at org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:78)
        ... 4 more

  was:
We noticed on one of our node the number of SStables is growing.  It appears that compaction
thread is still alive.  However, compaction is failing because one of the sstable is corrupt:

ERROR [ROW-READ-STAGE:475] 2009-09-21 00:29:17,068 DebuggableThreadPoolExecutor.java (line
125) Error in ThreadPoolExecutor
java.lang.RuntimeException: java.io.EOFException
        at org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:110)
        at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:44)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.EOFException
        at java.io.RandomAccessFile.readFully(RandomAccessFile.java:383)
        at java.io.RandomAccessFile.readFully(RandomAccessFile.java:361)
        at org.apache.cassandra.utils.FBUtilities.readByteArray(FBUtilities.java:390)
        at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:64)
        at org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:349)
        at org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:309)
        at org.apache.cassandra.db.filter.SSTableNamesIterator.<init>(SSTableNamesIterator.java:102)
        at org.apache.cassandra.db.filter.NamesQueryFilter.getSSTableColumnIterator(NamesQueryFilter.java:69)
        at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1467)
        at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1401)
        at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1420)
        at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1401)
        at org.apache.cassandra.db.Table.getRow(Table.java:589)
        at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:65)
        at org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:78)
        ... 4 more


> Corrupt SSTable
> ---------------
>
>                 Key: CASSANDRA-452
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-452
>             Project: Cassandra
>          Issue Type: Bug
>         Environment: Pre 0.4 based on r805615 on trunk w/ #370, #392, #394, #405, #406,
#418
>            Reporter: Sammy Yu
>
> We noticed on one of our node the number of SStables is growing.  The compaction thread
is alive and running.  We can see that it is constantly trying to compact the same set of
sstables.  However, it is failing because one of the sstable is corrupt:
> ERROR [ROW-READ-STAGE:475] 2009-09-21 00:29:17,068 DebuggableThreadPoolExecutor.java
(line 125) Error in ThreadPoolExecutor
> java.lang.RuntimeException: java.io.EOFException
>         at org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:110)
>         at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:44)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.EOFException
>         at java.io.RandomAccessFile.readFully(RandomAccessFile.java:383)
>         at java.io.RandomAccessFile.readFully(RandomAccessFile.java:361)
>         at org.apache.cassandra.utils.FBUtilities.readByteArray(FBUtilities.java:390)
>         at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:64)
>         at org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:349)
>         at org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:309)
>         at org.apache.cassandra.db.filter.SSTableNamesIterator.<init>(SSTableNamesIterator.java:102)
>         at org.apache.cassandra.db.filter.NamesQueryFilter.getSSTableColumnIterator(NamesQueryFilter.java:69)
>         at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1467)
>         at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1401)
>         at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1420)
>         at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1401)
>         at org.apache.cassandra.db.Table.getRow(Table.java:589)
>         at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:65)
>         at org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:78)
>         ... 4 more

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message