incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Doubleday <daniel.double...@gmx.net>
Subject Strange corrupt sstable
Date Thu, 28 Apr 2011 16:58:26 GMT
Hi all

on one of our dev machines we ran into this:

INFO [CompactionExecutor:1] 2011-04-28 15:07:35,174 SSTableWriter.java (line 108) Last written
key : DecoratedKey(12707736894140473154801792860916528374, 74657374)
 INFO [CompactionExecutor:1] 2011-04-28 15:07:35,174 SSTableWriter.java (line 109) Current
key : DecoratedKey(9639328335678452015784116654185025244, 61c080c080c080c080c080c0803a60)
 INFO [CompactionExecutor:1] 2011-04-28 15:07:35,175 SSTableWriter.java (line 110) Writing
into file /var/lib/cassandra/data/Smeet/Tracker-tmp-f-752-Data.db
ERROR [CompactionExecutor:1] 2011-04-28 15:07:35,179 AbstractCassandraDaemon.java (line 113)
Fatal exception in thread Thread[CompactionExecutor:1,1,main]
java.io.IOException: Keys must be written in ascending order.
        at org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:111)
        at org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:128)
        at org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:451)
        at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:124)
        at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:94)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:636)


After looking into this we found that one SSTable was corrupt:

INFO 18:10:23,767 checking sstable /Users/dd/Temp/cassandra/data/Smeet/Tracker-f-21-Data.db
 INFO 18:10:23,773 DecoratedKey(12707736894140473154801792860916528374, 74657374)
 INFO 18:10:23,777 DecoratedKey(9639328335678452015784116654185025244, 61c080c080c080c080c080c0803a60)
 INFO 18:10:23,781 DecoratedKey(14745077953608664589374317792467471221, 7265706f2d333631343039)
 INFO 18:10:23,783 DecoratedKey(20959981347795136420050187639971803838, 6576656e746c6f672d333631383838)
 INFO 18:10:23,785 DecoratedKey(21549594967496343960898734735584772470, 6576656e746c6f672d333631383837)
 INFO 18:10:23,787 DecoratedKey(21604488955263763122920767208831708343, 6576656e746c6f672d333631383835)
 INFO 18:10:23,790 DecoratedKey(50100329769829417461462475380721306089, 61c080c080c080c080c080c0803a5f)
 INFO 18:10:23,792 DecoratedKey(46089976107344533463199626515706634206, 7265706f2d333631303436)
 INFO 18:10:23,794 DecoratedKey(50825113246224326722027014732583502298, 6c6f674576656e742d333631383431)
 INFO 18:10:23,797 DecoratedKey(146509745693074257726394585872920772980, 61c080c080c080c080c080c0803a59)
 INFO 18:10:23,801 DecoratedKey(63662277073629697060427801149510996510, 6c6f674576656e742d333631383432)
 INFO 18:10:23,803 DecoratedKey(79706531464725069255477768601476202854, 6576656e746c6f672d333631383834)
 INFO 18:10:23,808 DecoratedKey(99662740527281958972877086277408797556, 6c6f674576656e742d333631383430)
 INFO 18:10:23,810 DecoratedKey(24216004522287050570821311008920284627, 61c080c080c080c080c080c0803a5a)
 INFO 18:10:23,816 DecoratedKey(130819691046705791898531759480356609485, 61c080c080c080c080c080c0803a5e)
 INFO 18:10:24,121 DecoratedKey(107312182597869061361893705497084751672, 61c080c080c080c080c080c0803a58)
 INFO 18:10:24,123 DecoratedKey(142873550623537979564886292314205248041, 7265706f2d333631363731)
 INFO 18:10:24,124 DecoratedKey(147476468396567670371575440897348610391, 6576656e746c6f672d333631383836)

I couldn't figure out how SSTableWriter could possible write such a file so ... Just wanted
to report. Maybe it rings a bell somewhere.

It seems really strange that some data corruption would leave the file readable for the SSTableReader.

History of this server is 0.6.5 -> 0.7.4. But this file was obviously written by 0.7.4



Cheers,
Daniel



Mime
View raw message