incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Ellis <jbel...@gmail.com>
Subject Re: file corruption since 0.6.2
Date Sat, 12 Jun 2010 12:26:44 GMT
You're getting errors from BitSetSerializer... that hasn't changed at
all from 0.6.1 to 0.6.2.  (or even from 0.5 to 0.6.2...)  Sounds more
like bad RAM to me.

2010/6/11 Lu Ming <xluke@live.com>:
> Many files are corrupted when our cassandra is update to 0.6.2
> COMPACTION-POOL is down caused by the following error.
> and some nodes can NOT startup because of this error.
>
> Is it caused by the issue CASSANDRA-1169? The node got the wrong or
> corrupted stream file?
>
> ERROR [COMPACTION-POOL:1] 2010-06-12 11:28:40,279
> DebuggableThreadPoolExecutor.java (line 94) Error in executor futuretask
> java.util.concurrent.ExecutionException:
> java.lang.StringIndexOutOfBoundsException: String index out of range: -1
>         at
> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>         at
> org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.afterExecute(DebuggableThreadPoolExecutor.java:86)
>         at
> org.apache.cassandra.db.CompactionManager$CompactionExecutor.afterExecute(CompactionManager.java:582)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:888)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:619)
> Caused by: java.lang.StringIndexOutOfBoundsException: String index out of
> range: -1
>         at java.lang.String.substring(String.java:1937)
>         at
> org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:50)
>         at org.apache.cassandra.io.IteratingRow.<init>(IteratingRow.java:48)
>         at
> org.apache.cassandra.io.SSTableScanner$KeyScanningIterator.next(SSTableScanner.java:136)
>         at
> org.apache.cassandra.io.SSTableScanner$KeyScanningIterator.next(SSTableScanner.java:113)
>         at
> org.apache.cassandra.io.SSTableScanner.next(SSTableScanner.java:105)
>         at
> org.apache.cassandra.io.SSTableScanner.next(SSTableScanner.java:34)
>         at
> org.apache.commons.collections.iterators.CollatingIterator.set(CollatingIterator.java:284)
>         at
> org.apache.commons.collections.iterators.CollatingIterator.least(CollatingIterator.java:326)
>         at
> org.apache.commons.collections.iterators.CollatingIterator.next(CollatingIterator.java:230)
>         at
> org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:68)
>         at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:135)
>         at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:130)
>         at
> org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
>         at
> org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
>         at
> org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:284)
>         at
> org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:102)
>         at
> org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:83)
>         at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
>
>
>  INFO [main] 2010-06-12 10:38:43,764 SSTableReader.java (line 124) Sampling
> index for /data4/cassandra2/data/GroupChat/GroupTopTopic-839-Data.db
> ERROR [main] 2010-06-12 10:38:43,777 CassandraDaemon.java (line 195)
> Exception encountered during startup.
> java.lang.StringIndexOutOfBoundsException: String index out of range: -1
>         at java.lang.String.substring(String.java:1937)
>         at
> org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:50)
>         at
> org.apache.cassandra.io.SSTableReader.loadIndexFile(SSTableReader.java:261)
>         at
> org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:125)
>         at
> org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:114)
>         at
> org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:178)
>         at
> org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:248)
>         at org.apache.cassandra.db.Table.<init>(Table.java:338)
>         at org.apache.cassandra.db.Table.open(Table.java:199)
>         at
> org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:91)
>         at
> org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:177)
>
>
>  INFO [main] 2010-06-12 10:45:12,507 SSTableReader.java (line 124) Sampling
> index for /data4/cassandra2/data/GroupChat/Topic-1236-Data.db
> ERROR [main] 2010-06-12 10:45:12,528 ColumnFamilyStore.java (line 182)
> Corrupt file /data4/cassandra2/data/GroupChat/Topic-1236-Data.db; skipped
> java.io.StreamCorruptedException: invalid stream header: 32393031
>         at
> java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:783)
>         at java.io.ObjectInputStream.<init>(ObjectInputStream.java:280)
>         at
> org.apache.cassandra.utils.BitSetSerializer.deserialize(BitSetSerializer.java:41)
>         at
> org.apache.cassandra.utils.BloomFilterSerializer.deserialize(BloomFilter.java:207)
>         at
> org.apache.cassandra.utils.BloomFilterSerializer.deserialize(BloomFilter.java:195)
>         at
> org.apache.cassandra.io.SSTableReader.loadBloomFilter(SSTableReader.java:236)
>         at
> org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:126)
>         at
> org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:114)
>         at
> org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:178)
>         at
> org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:248)
>         at org.apache.cassandra.db.Table.<init>(Table.java:338)
>         at org.apache.cassandra.db.Table.open(Table.java:199)
>         at
> org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:91)
>         at
> org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:177)
>
> ERROR [ROW-READ-STAGE:87] 2010-06-12 10:08:16,334
> DebuggableThreadPoolExecutor.java (line 101) Error in ThreadPoolExecutor
> ERROR [ROW-READ-STAGE:87] 2010-06-12 10:08:16,335 CassandraDaemon.java (line
> 78) Fatal exception in thread Thread[ROW-READ-STAGE:87,5,main]
> ERROR [ROW-READ-STAGE:79] 2010-06-12 10:09:06,606
> DebuggableThreadPoolExecutor.java (line 101) Error in ThreadPoolExecutor
> ERROR [ROW-READ-STAGE:79] 2010-06-12 10:09:06,607 CassandraDaemon.java (line
> 78) Fatal exception in thread Thread[ROW-READ-STAGE:79,5,main]
> ERROR [ROW-READ-STAGE:54] 2010-06-12 10:22:42,623
> DebuggableThreadPoolExecutor.java (line 101) Error in ThreadPoolExecutor
> ERROR [ROW-READ-STAGE:54] 2010-06-12 10:22:42,623 CassandraDaemon.java (line
> 78) Fatal exception in thread Thread[ROW-READ-STAGE:54,5,main]
> ERROR [ROW-READ-STAGE:86] 2010-06-12 10:22:42,873
> DebuggableThreadPoolExecutor.java (line 101) Error in ThreadPoolExecutor
> ERROR [ROW-READ-STAGE:86] 2010-06-12 10:22:42,874 CassandraDaemon.java (line
> 78) Fatal exception in thread Thread[ROW-READ-STAGE:86,5,main]
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com

Mime
View raw message