Our app currently moves from hibernate to cassandra+solr combination (we use our own storage abstraction for this). We have a lot of junit integration tests that use database. Now we've decided to use these tests to test our cassandra+solr storage implementation (given that on hibernate implementation these tests run fine). For that purpose we've installed cassandra and solr on a dedicated vm and we've launched the tests.
Now there are two things that should be mentioned about these tests and how we use cassandra.
1. We have 1 keyspace and about 200-300 column families.
2. Tests are written in such a manner that at the beginning of each test we drop the keyspace and then recreate it from scratch (together with all column families).
3. On client side we use hector.
The problem is that each time we launch the tests they start failing randomly at some point. In cassandra logs we see a lot of CorruptSSTableException-s in system keyspace caused by CorruptBlockException-s. An example of full stack trace of such exception can be found here . The tests start to fail due to thrift transport exceptions. At that point we are also unable to connect to cassandra using cassandra-cli. Then, after some time cassandra goes back to normal state all by itself.
P.S. I can provide full system.log and output.log from clean start (all data folders erased) till the errors and later when the system is ok once again. Although I'm not sure which file sharing service to utilise.
Thanks in advance,