cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David King (JIRA)" <j...@apache.org>
Subject [jira] Created: (CASSANDRA-1248) ArrayIndexOutOfBoundsException thrown when reading partially-written sstable
Date Mon, 05 Jul 2010 19:02:49 GMT
ArrayIndexOutOfBoundsException thrown when reading partially-written sstable
----------------------------------------------------------------------------

                 Key: CASSANDRA-1248
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1248
             Project: Cassandra
          Issue Type: Bug
          Components: Core
    Affects Versions: 0.6.3
         Environment: Cassandra 0.6.3
java version "1.6.0_0"
IcedTea6 1.3.1 (6b12-0ubuntu6.6) Runtime Environment (build 1.6.0_0-b12)
OpenJDK 64-Bit Server VM (build 1.6.0_0-b12, mixed mode)
Ubuntu 8.10
6 node cluster
RF==3
            Reporter: David King


A node died by running out of disk space. After growing the volume, trying to execute reads
dies with:

{quote}
ERROR 11:02:51,957 Uncaught exception in thread Thread[ROW-READ-STAGE:10,5,main]
java.lang.ArrayIndexOutOfBoundsException
        at org.apache.cassandra.io.util.BufferedRandomAccessFile.read(BufferedRandomAccessFile.java:326)
        at java.io.RandomAccessFile.readFully(RandomAccessFile.java:414)
        at java.io.DataInputStream.readUTF(DataInputStream.java:609)
        at java.io.RandomAccessFile.readUTF(RandomAccessFile.java:938)
        at org.apache.cassandra.db.filter.SSTableSliceIterator$ColumnGroupReader.<init>(SSTableSliceIterator.java:125)
        at org.apache.cassandra.db.filter.SSTableSliceIterator.<init>(SSTableSliceIterator.java:59)
        at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:63)
        at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:859)
        at org.apache.cassandra.db.ColumnFamilyStore.cacheRow(ColumnFamilyStore.java:747)
        at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:772)
        at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:739)
        at org.apache.cassandra.db.Table.getRow(Table.java:381)
        at org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCommand.java:56)
        at org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:70)
        at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:40)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:636)
{quote}

The clients then die with TimeoutExceptions.

It might be better to return an exception to the client immediately instead of letting them
die, and it would be helpful if this exception were accompanied with a message indicating
*which* SStable is malformed. 

This command does *not* turn up the failed table (nor does the same with sstablekeys):

{quote}
ri@pmc08:/cassandra/data$ for x in $(ls -t */*-Data.db); do ls -lh $x; /usr/local/cassandra/bin/sstable2json
$x > /dev/null; done
{quote}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message