cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Brown (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-7465) DecoratedKey assertion error on reads
Date Fri, 27 Jun 2014 21:43:24 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-7465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14046430#comment-14046430
] 

Jason Brown commented on CASSANDRA-7465:
----------------------------------------

similar, but perhaps completely unrelated to, CASSANDRA-7245.

[~benedict] and I suspect there might be something funky with the KeyCache index

> DecoratedKey assertion error on reads
> -------------------------------------
>
>                 Key: CASSANDRA-7465
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7465
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: 3 nodes
> Oracle Linux Server 6.3 
> kernel ver 2.6.39
>            Reporter: Jason Brown
>            Priority: Blocker
>             Fix For: 2.1.0
>
>
> Getting the following exception when running read stress:
> {code}WARN  [SharedPool-Worker-31] 2014-06-27 21:25:51,391 AbstractTracingAwareExecutorService.java:166
- Uncaught exception on thread Thread[SharedPool-Worker-31,10,main]: {}
> java.lang.AssertionError: DecoratedKey(-5397116645141815707, 30303031393143364639) !=
DecoratedKey(-5397116645141815707, 30303031343439443233) in /u/sdd/cassandra-jasobrown/data/Keyspace1/Standard1-6ab9bd90fe3b11e385edff96c2ef2fd6/Keyspace1-Standard1-ka-73-Data.db
> 	at org.apache.cassandra.db.columniterator.SSTableNamesIterator.read(SSTableNamesIterator.java:111)
~[apache-cassandra-2.1.0-rc2-SNAPSHOT.jar:2.1.0-rc2-SNAPSHOT]
> 	at org.apache.cassandra.db.columniterator.SSTableNamesIterator.<init>(SSTableNamesIterator.java:59)
~[apache-cassandra-2.1.0-rc2-SNAPSHOT.jar:2.1.0-rc2-SNAPSHOT]
> 	at org.apache.cassandra.db.filter.NamesQueryFilter.getSSTableColumnIterator(NamesQueryFilter.java:89)
~[apache-cassandra-2.1.0-rc2-SNAPSHOT.jar:2.1.0-rc2-SNAPSHOT]
> 	at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:62)
~[apache-cassandra-2.1.0-rc2-SNAPSHOT.jar:2.1.0-rc2-SNAPSHOT]
> 	at org.apache.cassandra.db.CollationController.collectTimeOrderedData(CollationController.java:125)
~[apache-cassandra-2.1.0-rc2-SNAPSHOT.jar:2.1.0-rc2-SNAPSHOT]
> 	at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:59)
~[apache-cassandra-2.1.0-rc2-SNAPSHOT.jar:2.1.0-rc2-SNAPSHOT]
> 	at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1873)
~[apache-cassandra-2.1.0-rc2-SNAPSHOT.jar:2.1.0-rc2-SNAPSHOT]
> 	at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1681)
~[apache-cassandra-2.1.0-rc2-SNAPSHOT.jar:2.1.0-rc2-SNAPSHOT]
> 	at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:341) ~[apache-cassandra-2.1.0-rc2-SNAPSHOT.jar:2.1.0-rc2-SNAPSHOT]
> 	at org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCommand.java:55)
~[apache-cassandra-2.1.0-rc2-SNAPSHOT.jar:2.1.0-rc2-SNAPSHOT]
> 	at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1393)
~[apache-cassandra-2.1.0-rc2-SNAPSHOT.jar:2.1.0-rc2-SNAPSHOT]
> 	at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2060)
~[apache-cassandra-2.1.0-rc2-SNAPSHOT.jar:2.1.0-rc2-SNAPSHOT]
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_13]
> 	at org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:162)
~[apache-cassandra-2.1.0-rc2-SNAPSHOT.jar:2.1.0-rc2-SNAPSHOT]
> 	at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:103) [apache-cassandra-2.1.0-rc2-SNAPSHOT.jar:2.1.0-rc2-SNAPSHOT]
> 	at java.lang.Thread.run(Thread.java:722) [na:1.7.0_13]
> {code}
> I have a three node cluster, which I populate with the following stress command:
> {code}
> cassandra-stress write n=60000000 -schema replication\(factor\=2\) -key populate=1..60000000
-rate threads=42  -mode native prepared cql3 -port native=9043 thrift=9161  -node athena06-a,athena06-b,athena06-c
-col n=fixed\(21\) size=exp\(11..42\)
> {code}
> Then I run the read stress:
> {code}
> cassandra-stress read n=100000000  -key dist=extr\(1..600000000,2\)  -mode native prepared
cql3 -port native=9043 thrift=9161  -node athena06-b,athena06-c,athena06-a -col n=fixed\(21\)
-rate threads=64
> {code}
> The above exception occurs semi-frequently (several to ~50 times a minute, but seems
to depend on amount of data in cluster - anecdotal evidence only).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message