Sorry, I sent the mail too early.

This is the stack trace:

2014-02-28 10:56:03.205+0100 [SSTableBatchOpen:1] [ERROR] DebuggableThreadPoolExecutor.java(218) org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor: Error in ThreadPoolExecutor
org.apache.cassandra.io.sstable.CorruptSSTableException: java.io.EOFException
       at org.apache.cassandra.io.compress.CompressionMetadata.<init>(CompressionMetadata.java:108)
       at org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:63)
       at org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:42)
       at org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:407)
       at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:198)
       at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:157)
       at org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:262)
       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
       at java.util.concurrent.FutureTask.run(FutureTask.java:262)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
       at java.lang.Thread.run(Thread.java:744)
Caused by: java.io.EOFException
       at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:340)
       at java.io.DataInputStream.readUTF(DataInputStream.java:589)
       at java.io.DataInputStream.readUTF(DataInputStream.java:564)
       at org.apache.cassandra.io.compress.CompressionMetadata.<init>(CompressionMetadata.java:83)
       ... 11 more

Snappy is used for compression on this table.

ondrej c.


On Fri, Feb 28, 2014 at 11:09 AM, Ondřej Černoš <cernoso@gmail.com> wrote:
Hello,

we are trying to add authentication to our Cassandra cluster. We add our authenticated users during puppet deployment using the default user, which is then disabled.

We have the following issues:

- we see CorruptSSTableException in system_auth.users table
- we are not able to add users after delete, which can be explained by the following statement found in the source code: "INSERT INTO %s.%s (username, salted_hash) VALUES ('%s', '%s') USING TIMESTAMP 0" (see the 0 - is this really correct?)

nodetool scrub didn't help, compactation didn't help - tombstones were still there, as well as the exception.

Has anybody else seen this?

It's cassandra 1.2.11 with vnodes on.

regards,
ondrej cernos