incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Colby <jonathan.co...@gmail.com>
Subject recurring EOFException exception in 0.7.4
Date Fri, 15 Apr 2011 19:40:54 GMT
I've been struggling with these kinds of exceptions for some time now.  I thought it might
have been a one-time thing, so on the 2 nodes where I saw this problem I pulled in fresh data
with a repair on an empty data directory.

Unfortunately, this problem is now coming up on a new node that has, up until now, not had
this problem.

What could be causing this?  Could it be related to encoding?   Why are these rows not readable?
  

This exception prevents cassandra from doing repairs, and even minor compactions.  It also
messes up memtable management (with a normal load of 25GB,  disk goes to almost 100% full
on a 500 GB hd).

This is incredibly frustrating.  This is the only pain-point I have had with cassandra so
far.   By the way, this node was never upgraded - it was 0.7.4 from the start, so that eliminates
format compatibility problems.

ERROR [CompactionExecutor:1] 2011-04-15 21:31:23,479 PrecompactedRow.java (line 82) Skipping
row DecoratedKey(105452551814086725777389040553659117532, 4d657373616765456e726963686d656e743a313032343937)
in /var/lib/cassandra/data/DFS/main-f-91-Data.db
java.io.EOFException
	at java.io.RandomAccessFile.readFully(RandomAccessFile.java:383)
	at java.io.RandomAccessFile.readFully(RandomAccessFile.java:361)
	at org.apache.cassandra.io.util.BufferedRandomAccessFile.readBytes(BufferedRandomAccessFile.java:270)
	at org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:315)
	at org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:272)
	at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:94)
	at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:35)
	at org.apache.cassandra.db.ColumnFamilySerializer.deserializeColumns(ColumnFamilySerializer.java:129)
	at org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:176)
	at org.apache.cassandra.io.PrecompactedRow.<init>(PrecompactedRow.java:78)
	at org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:147)
	at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:108)
	at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:43)
	at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
	at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
	at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
	at org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:449)
	at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:124)
	at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:94)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:662)


Mime
View raw message