incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mateusz Korniak <mateusz-li...@ant.gliwice.pl>
Subject [0.7.2] Compacting exception
Date Fri, 18 Mar 2011 11:44:49 GMT
Hi !
I have run with Cassandra 0.7.2 out of disc space and after moving to bigger 
partition I experience compaction failures[1].

1) I suspect one of SSTables is broken. If I am right how can I find which one 
exactly ?

2) Knowing which one is broken is it safe to stop Cassandra, remove 
-Data.db -Filter.db -Index.db -Statistics.db of broken SSTable and restart ? 
Sure I will loose data but it's one of RF=3 nodes so it's not big problem.

Any suggestions, hints ?
Thanks in advance, regards,

[1]
INFO [CompactionExecutor:1] 2011-03-18 12:27:17,103 CompactionManager.java 
(line 458) Compacted to /var/lib/cassandra/data/foo/bar-tmp-f-707-Data.db.  
42,276,185 to 42,103,628 (~99% of original) bytes for 33,869 keys.  Time: 
25,924ms.
 INFO [CompactionExecutor:1] 2011-03-18 12:27:17,133 CompactionManager.java 
(line 373) Compacting [org.apache.cassandra.io.sstable.SSTableReader(path=
'/var/lib/cassandra/data/foo/bar-f-238-Data.db'), 
org.apache.cassandra.io.sstable.SSTableReader( 
path='/var/lib/cassandra/data/foo/bar-f-243-Data.db'), 
org.apache.cassandra.io.sstable.SSTableReader( 
path='/var/lib/cassandra/data/foo/bar-f-361-Data.db'), 
org.apache.cassandra.io.sstable.SSTableReader( 
path='/var/lib/cassandra/data/foo/bar-f-635-Data.db'), 
org.apache.cassandra.io.sstable.SSTableReader( 
path='/var/lib/cassandra/data/foo/bar-f-643-Data.db'), 
org.apache.cassandra.io.sstable.SSTableReader( 
path='/var/lib/cassandra/data/foo/bar-f-680-Data.db'), 
org.apache.cassandra.io.sstable.SSTableReader( 
path='/var/lib/cassandra/data/foo/bar-f-684-Data.db'), 
org.apache.cassandra.io.sstable.SSTableReader( 
path='/var/lib/cassandra/data/foo/bar-f-685-Data.db'), 
org.apache.cassandra.io.sstable.SSTableReader( 
path='/var/lib/cassandra/data/foo/bar-f-687-Data.db')]
ERROR [CompactionExecutor:1] 2011-03-18 12:27:18,241 
AbstractCassandraDaemon.java (line 114) Fatal exception in thread 
Thread[CompactionExecutor:1,1,main]
java.io.IOError: java.io.EOFException
        at 
org.apache.cassandra.io.sstable.SSTableIdentityIterator.<init>(SSTableIdentityIterator.java:78)
        at 
org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.next(SSTableScanner.java:179)
        at 
org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.next(SSTableScanner.java:144)
        at 
org.apache.cassandra.io.sstable.SSTableScanner.next(SSTableScanner.java:136)
        at 
org.apache.cassandra.io.sstable.SSTableScanner.next(SSTableScanner.java:39)
        at 
org.apache.commons.collections.iterators.CollatingIterator.set(CollatingIterator.java:284)
        at 
org.apache.commons.collections.iterators.CollatingIterator.least(CollatingIterator.java:326)
        at 
org.apache.commons.collections.iterators.CollatingIterator.next(CollatingIterator.java:230)
        at 
org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:68)
        at 
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
        at 
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
        at 
org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
        at 
org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
        at 
org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:427)
        at 
org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:123)
        at 
org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:93)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.EOFException
        at 
org.apache.cassandra.io.sstable.IndexHelper.skipIndex(IndexHelper.java:65)
        at 
org.apache.cassandra.io.sstable.SSTableIdentityIterator.<init>(SSTableIdentityIterator.java:70)
        ... 20 more


-- 
Mateusz Korniak

Mime
View raw message