Can someone with a better understanding of CASSANDRA-1992 jump in ? 

Aaron


On 11 Feb, 2011,at 02:51 AM, Attila Babo <babo.online@gmail.com> wrote:

The same problem here, even with
apache-cassandra-2011-02-10_06-30-00-bin.tar.gz from hudson. I'm happy
to share the full log if needed or run tests to identify the core
problem which looks like an overflow for me. Database was upgraded
from 0.6.8, there were no problems with it before.

/Attila

---------------

This is the first error line:

ERROR [CompactionExecutor:1] 2011-02-10 13:25:18,508
AbstractCassandraDaemon.java (line 114) Fatal exception in thread
Thread[CompactionExecutor:1,1,main]
java.lang.NegativeArraySizeException
at org.apache.cassandrautils.obs.OpenBitSet.<init>(OpenBitSet.java:85)
at org.apache.cassandra.utils.BloomFilter.bucketsFor(BloomFilter.java:65)
at org.apache.cassandra.utils.BloomFilter.getFilter(BloomFilter.java:82)
at org.apache.cassandra.db.ColumnIndexer.serializeInternal(ColumnIndexer.java:62)
at org.apache.cassandra.db.ColumnIndexer.serialize(ColumnIndexer.java:50)
at org.apache.cassandra.io.LazilyCompactedRow.<init>(LazilyCompactedRow.java:88)
at org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:137)
at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:108)
at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:43)
at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
at org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:416)
at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:123)
at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:93)

After a few errors of the above comes a NullPointerException:

ERROR [CompactionExecutor:1] 2011-02-10 13:25:53,807
AbstractCassandraDaemon.java (line 114) Fatal exception in thread
Thread[CompactionExecutor:1,1,main]
java.lang.NullPointerException
at org.apache.cassandra.io.LazilyCompactedRow.<init>(LazilyCompactedRow.java:90)
at org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:137)
at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:108)
at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:43)
at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
at org.apache.cassandra.db.CompactionManagerdoCompaction(CompactionManager.java:427)
at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:123)
at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:93)

Then comes an EOFException:

ERROR [CompactionExecutor:1] 2011-02-10 13:25:55,365
AbstractCassandraDaemon.java (line 114) Fatal exception in thread
Thread[CompactionExecutor:1,1,main]
java.io.IOError: java.io.EOFException
at org.apache.cassandra.io.util.ColumnIterator.deserializeNext(ColumnSortedMap.java:246)
at org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:262)
at org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:223)
at java.util.concurrent.ConcurrentSkipListMap.buildFromSorted(ConcurrentSkipListMap.java:1521)
at java.util.concurrent.ConcurrentSkipListMap.<init>(ConcurrentSkipListMap.java:1471)
at org.apache.cassandra.dbSuperColumnSerializer.deserialize(SuperColumn.java:366)
at org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:314)
at org.apache.cassandra.io.sstable.SSTableIdentityIterator.next(SSTableIdentityIterator.java:101)
at org.apache.cassandra.io.sstable.SSTableIdentityIterator.next(SSTableIdentityIterator.java:34)
at org.apache.commons.collections.iterators.CollatingIterator.set(CollatingIterator.java:284)
at org.apache.commons.collections.iterators.CollatingIterator.least(CollatingIterator.java:326)
at org.apache.commons.collections.iterators.CollatingIterator.next(CollatingIterator.java:230)
at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:68)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at com.google.commoncollect.Iterators$7.computeNext(Iterators.java:604)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at org.apache.cassandra.db.ColumnIndexer.serializeInternal(ColumnIndexer.java:76)
at org.apache.cassandra.db.ColumnIndexer.serialize(ColumnIndexer.java:50)
at org.apache.cassandra.io.LazilyCompactedRow.<init>(LazilyCompactedRow.java:88)
at org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:137)
at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:108)
at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:43)
at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
at org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:427)
at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:123)
at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:93)