cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David King (JIRA)" <>
Subject [jira] Created: (CASSANDRA-1248) ArrayIndexOutOfBoundsException thrown when reading partially-written sstable
Date Mon, 05 Jul 2010 19:02:49 GMT
ArrayIndexOutOfBoundsException thrown when reading partially-written sstable

                 Key: CASSANDRA-1248
             Project: Cassandra
          Issue Type: Bug
          Components: Core
    Affects Versions: 0.6.3
         Environment: Cassandra 0.6.3
java version "1.6.0_0"
IcedTea6 1.3.1 (6b12-0ubuntu6.6) Runtime Environment (build 1.6.0_0-b12)
OpenJDK 64-Bit Server VM (build 1.6.0_0-b12, mixed mode)
Ubuntu 8.10
6 node cluster
            Reporter: David King

A node died by running out of disk space. After growing the volume, trying to execute reads
dies with:

ERROR 11:02:51,957 Uncaught exception in thread Thread[ROW-READ-STAGE:10,5,main]
        at org.apache.cassandra.db.filter.SSTableSliceIterator$ColumnGroupReader.<init>(
        at org.apache.cassandra.db.filter.SSTableSliceIterator.<init>(
        at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(
        at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(
        at org.apache.cassandra.db.ColumnFamilyStore.cacheRow(
        at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(
        at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(
        at org.apache.cassandra.db.Table.getRow(
        at org.apache.cassandra.db.SliceByNamesReadCommand.getRow(
        at org.apache.cassandra.db.ReadVerbHandler.doVerb(
        at java.util.concurrent.ThreadPoolExecutor.runWorker(
        at java.util.concurrent.ThreadPoolExecutor$

The clients then die with TimeoutExceptions.

It might be better to return an exception to the client immediately instead of letting them
die, and it would be helpful if this exception were accompanied with a message indicating
*which* SStable is malformed. 

This command does *not* turn up the failed table (nor does the same with sstablekeys):

ri@pmc08:/cassandra/data$ for x in $(ls -t */*-Data.db); do ls -lh $x; /usr/local/cassandra/bin/sstable2json
$x > /dev/null; done

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message