incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erik Onnen <eon...@gmail.com>
Subject SSTable Corruption
Date Wed, 23 Mar 2011 21:59:59 GMT
After an upgrade from 0.7.3 to 0.7.4, we're seeing the following on
several data files:

ERROR [main] 2011-03-23 18:58:33,137 ColumnFamilyStore.java (line 235)
Corrupt sstable
/mnt/services/cassandra/var/data/0.7.4/data/Helium/dp_idx-f-4844=[Index.db,
Statistics.db, Data.db, Filter.db]; skipped
java.io.EOFException
        at java.io.DataInputStream.readInt(DataInputStream.java:375)
        at org.apache.cassandra.utils.EstimatedHistogram$EstimatedHistogramSerializer.deserialize(EstimatedHistogram.java:207)
        at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:166)
        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:226)
        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:472)
        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:453)
        at org.apache.cassandra.db.Table.initCf(Table.java:317)
        at org.apache.cassandra.db.Table.<init>(Table.java:254)
        at org.apache.cassandra.db.Table.open(Table.java:110)
        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:160)
        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:314)
        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)

Prior to the upgrade, there weren't problems opening these data files
with the exception that we were not able to bootstrap new nodes into
the ring due to what we believe was caused by CASSANDRA-2283. In
trying to fix 2283, we did run a scrub on this node.

With an RF of 3 in our ring, what are the implications of this? Specifically,

1) If the node is a natural endpoint for rows in the corrupt sstable,
will it simply not respond to requests for that row or will there be
errors?
2) Should a repair fix the broken sstable? This was my expectation
based on similar threads on the mailing list.
3) If #2 is correct and the repair did not fix the corrupt sstables
(which it did not), how should we proceed to repair the tables?

Thanks,
-erik

Mime
View raw message