cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcus Eriksson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-11468) Reading from early opened, and compressed, sstable throws CorruptSSTableException
Date Fri, 29 Apr 2016 11:33:12 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-11468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15263926#comment-15263926
] 

Marcus Eriksson commented on CASSANDRA-11468:
---------------------------------------------

If I'm reading the code correctly it is because during normal operation we only flush when
the buffer gets full - with compression we flush 64k of uncompressed data into a chunk on
disk. This means that for the first chunk, we guarantee that positions 0 -> 64k [are in
the first chunk|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java#L230].
If we for example have force flushed a 10k first chunk and ask for position 15k, we will read
the first chunk on disk, but the actual data is in the next.

Since this cannot happen during standard operation I doubt we should fix it as it would probably
force us to rewrite the compression chunk handling (unless anyone has a brilliant idea how
to fix it safely)

> Reading from early opened, and compressed, sstable throws CorruptSSTableException
> ---------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-11468
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-11468
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Blake Eggleston
>            Assignee: Marcus Eriksson
>             Fix For: 3.0.x
>
>
> When a compressed sstable is early opened during compaction, reading from it can fail
with the following exception:
> {code}
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: build/test/cassandra/data/ks_1459378971131/early_open_test-893eb3d0f6cb11e59e1b4f343b985d3e/ma-3-big-Data.db
> 	at org.apache.cassandra.db.columniterator.AbstractSSTableIterator.<init>(AbstractSSTableIterator.java:130)
> 	at org.apache.cassandra.db.columniterator.SSTableIterator.<init>(SSTableIterator.java:46)
> 	at org.apache.cassandra.db.columniterator.SSTableIterator.<init>(SSTableIterator.java:36)
> 	at org.apache.cassandra.io.sstable.format.big.BigTableReader.iterator(BigTableReader.java:62)
> 	at org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndSSTablesInTimestampOrder(SinglePartitionReadCommand.java:715)
> 	at org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDiskInternal(SinglePartitionReadCommand.java:482)
> 	at org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDisk(SinglePartitionReadCommand.java:459)
> 	at org.apache.cassandra.db.SinglePartitionReadCommand.queryStorage(SinglePartitionReadCommand.java:325)
> 	at org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:363)
> 	at org.apache.cassandra.db.ReadCommand.executeInternal(ReadCommand.java:393)
> 	at org.apache.cassandra.db.SinglePartitionReadCommand$Group.executeInternal(SinglePartitionReadCommand.java:950)
> 	at org.apache.cassandra.cql3.statements.SelectStatement.executeInternal(SelectStatement.java:397)
> 	at org.apache.cassandra.cql3.statements.SelectStatement.executeInternal(SelectStatement.java:76)
> 	at org.apache.cassandra.cql3.QueryProcessor.executeInternal(QueryProcessor.java:295)
> 	at org.apache.cassandra.io.sstable.EarlyOpenTest.earlyOpen(EarlyOpenTest.java:160)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:44)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
> 	at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
> 	at org.junit.runner.JUnitCore.run(JUnitCore.java:159)
> 	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78)
> 	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212)
> 	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
> Caused by: java.io.EOFException
> 	at org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:146)
> 	at org.apache.cassandra.io.util.RebufferingInputStream.readPrimitiveSlowly(RebufferingInputStream.java:108)
> 	at org.apache.cassandra.io.util.RebufferingInputStream.readShort(RebufferingInputStream.java:164)
> 	at org.apache.cassandra.io.util.RebufferingInputStream.readUnsignedShort(RebufferingInputStream.java:170)
> 	at org.apache.cassandra.utils.ByteBufferUtil.readShortLength(ByteBufferUtil.java:366)
> 	at org.apache.cassandra.utils.ByteBufferUtil.skipShortLength(ByteBufferUtil.java:392)
> 	at org.apache.cassandra.db.columniterator.AbstractSSTableIterator.<init>(AbstractSSTableIterator.java:97)
> 	... 41 more
> {code}
> In my testing, it's been the second read that fails, and is because the buffer the sstable
iterator is trying to read from has it's position set to it's length. Uncompressed sstables
work fine.
> I've pushed up a branch with a reproducing unit test [here|https://github.com/bdeggleston/cassandra/tree/11468]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message