cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Horia Mocioi (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-14673) Removing user defined type column results in ReadFailure due to CorruptSSTableException or IllegalStateException
Date Thu, 30 Aug 2018 09:57:00 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-14673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16597174#comment-16597174
] 

Horia Mocioi edited comment on CASSANDRA-14673 at 8/30/18 9:56 AM:
-------------------------------------------------------------------

[~iamaleksey]-: could you be more specific, please?-

-Which operation is forbidden? Creating the a column of a user defined type or dropping it?-

Later edit: never mind.

I tested it on trunk. The operation that is forbidden is dropping a non-frozen column.


was (Author: umcanl):
[~iamaleksey]: could you be more specific, please?

Which operation is forbidden? Creating the a column of a user defined type or dropping it?

> Removing user defined type column results in ReadFailure due to CorruptSSTableException
or IllegalStateException
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-14673
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-14673
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Local Write-Read Paths
>            Reporter: Horia Mocioi
>            Priority: Minor
>         Attachments: script
>
>
> Steps to reproduce:
>  # create keyspace
>  # create user defined type
>  # create a table that would use the udt
>  # insert some data
>  # drop the udt column
>  # restart cassandra
>  # query the table
> See the attached script for steps 1-5.
> When querying the table in cqlsh I got the following error:
> {code:java}
> ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read] message="Operation
failed - received 0 responses and 1 failures" info={'failures': 1, 'received_responses': 0,
'required_responses': 1, 'consistency': 'ONE'}{code}
> On system.log I get errors. The errors are different, meaning that some times I get:
> {code:java}
> WARN [ReadStage-1] 2018-08-29 10:22:30,229 AbstractLocalAwareExecutorService.java:167
- Uncaught exception on thread Thread[ReadStage-1,10,main]: {}
> java.lang.RuntimeException: org.apache.cassandra.io.sstable.CorruptSSTableException:
Corrupted: /.ccm/3113/node1/data0/my_ks/my_table-90f73700ab6411e8a72e45b164590124/mc-1-big-Data.db
>  at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2601)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_131]
>  at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:134)
[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) [apache-cassandra-3.11.3.jar:3.11.3]
>  at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
> Caused by: org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: /.ccm/3113/node1/data0/my_ks/my_table-90f73700ab6411e8a72e45b164590124/mc-1-big-Data.db
>  at org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:391)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:258)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) ~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) ~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133) ~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133) ~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:136)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:92)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:79)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:308)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:187)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.db.ReadResponse$LocalDataResponse.<init>(ReadResponse.java:180)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.db.ReadResponse$LocalDataResponse.<init>(ReadResponse.java:176)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:76) ~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:352) ~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1889)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2597)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  ... 5 common frames omitted
> Caused by: java.io.EOFException: null
>  at org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:146)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.utils.vint.VIntCoding.readUnsignedVInt(VIntCoding.java:65) ~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.io.util.RebufferingInputStream.readUnsignedVInt(RebufferingInputStream.java:209)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.db.Columns$Serializer.deserializeSubset(Columns.java:513) ~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.db.rows.UnfilteredSerializer.deserializeRowBody(UnfilteredSerializer.java:599)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.db.UnfilteredDeserializer$CurrentDeserializer.readNext(UnfilteredDeserializer.java:209)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.computeNext(SSTableIterator.java:153)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.hasNextInternal(SSTableIterator.java:182)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:378)
~[apache-cassandra-3.11.3.jar:3.11.3]
>  ... 25 common frames omitted
> {code}
> and with some other occasions I get:
> {code:java}
> WARN  [ReadStage-4] 2018-08-28 16:47:22,734 AbstractLocalAwareExecutorService.java:167
- Uncaught exception on thread Thread[ReadStage-4,5,main]: {}
> java.lang.RuntimeException: java.lang.IllegalStateException: [mt] is not a subset of
[]
> 	at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2591)
~[apache-cassandra-3.10.jar:3.10]
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_131]
> 	at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
~[apache-cassandra-3.10.jar:3.10]
> 	at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:134)
[apache-cassandra-3.10.jar:3.10]
> 	at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) [apache-cassandra-3.10.jar:3.10]
> 	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
> Caused by: java.lang.IllegalStateException: [mt] is not a subset of []
> 	at org.apache.cassandra.db.Columns$Serializer.encodeBitmap(Columns.java:543) ~[apache-cassandra-3.10.jar:3.10]
> 	at org.apache.cassandra.db.Columns$Serializer.serializeSubset(Columns.java:477) ~[apache-cassandra-3.10.jar:3.10]
> 	at org.apache.cassandra.db.rows.UnfilteredSerializer.serializeRowBody(UnfilteredSerializer.java:225)
~[apache-cassandra-3.10.jar:3.10]
> 	at org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:200)
~[apache-cassandra-3.10.jar:3.10]
> 	at org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:132)
~[apache-cassandra-3.10.jar:3.10]
> 	at org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:120)
~[apache-cassandra-3.10.jar:3.10]
> 	at org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:137)
~[apache-cassandra-3.10.jar:3.10]
> 	at org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:92)
~[apache-cassandra-3.10.jar:3.10]
> 	at org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:79)
~[apache-cassandra-3.10.jar:3.10]
> 	at org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:315)
~[apache-cassandra-3.10.jar:3.10]
> 	at org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:145)
~[apache-cassandra-3.10.jar:3.10]
> 	at org.apache.cassandra.db.ReadResponse$LocalDataResponse.<init>(ReadResponse.java:138)
~[apache-cassandra-3.10.jar:3.10]
> 	at org.apache.cassandra.db.ReadResponse$LocalDataResponse.<init>(ReadResponse.java:134)
~[apache-cassandra-3.10.jar:3.10]
> 	at org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:76) ~[apache-cassandra-3.10.jar:3.10]
> 	at org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:333) ~[apache-cassandra-3.10.jar:3.10]
> 	at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1884)
~[apache-cassandra-3.10.jar:3.10]
> 	at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2587)
~[apache-cassandra-3.10.jar:3.10]
> 	... 5 common frames omitted
> {code}
> I was not able to figure out a scenario when the first or second error would appear.
> I tested it on ccm cluster with 1 node, with RF=1 and 3 node cluster with RF=3.
> I was not able to reproduce it if instead of a udt column the table contains a normal
text column.
> The issue appeared initially [here|https://stackoverflow.com/questions/52025963/cassandra-corruption-due-to-udt-column-drop].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org


Mime
View raw message