cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gábor Auth (JIRA) <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-12259) Weird select issue on counters
Date Thu, 21 Jul 2016 15:12:20 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387870#comment-15387870
] 

Gábor Auth commented on CASSANDRA-12259:
----------------------------------------

I've tried this 'toolset' before submit this issue:
{code}
/home/cassandra/datastax-ddc-3.6.0/bin/nodetool upgradesstables
/home/cassandra/datastax-ddc-3.6.0/bin/nodetool compact
/home/cassandra/datastax-ddc-3.6.0/bin/nodetool scrub
/home/cassandra/datastax-ddc-3.6.0/bin/nodetool cleanup
/home/cassandra/datastax-ddc-3.6.0/bin/nodetool clearsnapshot
/home/cassandra/datastax-ddc-3.6.0/bin/nodetool truncatehints
/home/cassandra/datastax-ddc-3.6.0/bin/nodetool repair -full -pr
{code}


> Weird select issue on counters
> ------------------------------
>
>                 Key: CASSANDRA-12259
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-12259
>             Project: Cassandra
>          Issue Type: Bug
>         Environment: datastax-3.6.0, CentOS 7, x86_64
>            Reporter: Gábor Auth
>
> Tried to select value from a counter table:
> {code}
> cassandra@cqlsh:keyspace> select id,science,treasury,tutorial from counters where
id='38cab4b0-4e8e-11e6-a443-55982d8a0e8d';
>  id                                   | science | treasury | tutorial
> --------------------------------------+---------+----------+----------
>  38cab4b0-4e8e-11e6-a443-55982d8a0e8d | 2379459 |  1000000 |        5
> (1 rows)
> cassandra@cqlsh:keyspace> select * from counters where id='38cab4b0-4e8e-11e6-a443-55982d8a0e8d';
>  id                                   | science | treasury | tutorial
> --------------------------------------+---------+----------+----------
>  38cab4b0-4e8e-11e6-a443-55982d8a0e8d | 2379459 |  1000000 |        5
> (1 rows)
> cassandra@cqlsh:keyspace> select tutorial from counters where id='38cab4b0-4e8e-11e6-a443-55982d8a0e8d';
> ReadTimeout: code=1200 [Coordinator node timed out waiting for replica nodes' responses]
message="Operation timed out - received only 0 responses." info={'received_responses': 0,
'required_responses': 1, 'consistency': 'ONE'}
> {code}
> Meanwhile in the log of the node:
> {code}
> WARN  [SharedPool-Worker-2] 2016-07-21 11:21:09,055 AbstractLocalAwareExecutorService.java:169
- Uncaught exception on thread Thread[SharedPool-Worker-2,5,main]: {}
> java.lang.RuntimeException: java.lang.IndexOutOfBoundsException
>         at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2470)
~[apache-cassandra-3.6.0.jar:3.6.0]
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_91]
>         at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
~[apache-cassandra-3.6.0.jar:3.6.0]
>         at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
[apache-cassandra-3.6.0.jar:3.6.0]
>         at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) [apache-cassandra-3.6.0.jar:3.6.0]
>         at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> Caused by: java.lang.IndexOutOfBoundsException: null
>         at java.nio.Buffer.checkIndex(Buffer.java:546) ~[na:1.8.0_91]
>         at java.nio.HeapByteBuffer.getShort(HeapByteBuffer.java:314) ~[na:1.8.0_91]
>         at org.apache.cassandra.db.context.CounterContext.headerLength(CounterContext.java:141)
~[apache-cassandra-3.6.0.jar:3.6.0]
>         at org.apache.cassandra.db.context.CounterContext.access$100(CounterContext.java:76)
~[apache-cassandra-3.6.0.jar:3.6.0]
>         at org.apache.cassandra.db.context.CounterContext$ContextState.<init>(CounterContext.java:758)
~[apache-cassandra-3.6.0.jar:3.6.0]
>         at org.apache.cassandra.db.context.CounterContext$ContextState.wrap(CounterContext.java:765)
~[apache-cassandra-3.6.0.jar:3.6.0]                                                      
                                                                                
>         at org.apache.cassandra.db.context.CounterContext.merge(CounterContext.java:272)
~[apache-cassandra-3.6.0.jar:3.6.0]                                                      
                                                                                         
  
>         at org.apache.cassandra.db.Conflicts.mergeCounterValues(Conflicts.java:76) ~[apache-cassandra-3.6.0.jar:3.6.0]
                                                                                         
                                                              
>         at org.apache.cassandra.db.rows.Cells.reconcile(Cells.java:143) ~[apache-cassandra-3.6.0.jar:3.6.0]
                                                                                         
                                                                         
>         at org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:591)
~[apache-cassandra-3.6.0.jar:3.6.0]                                                      
                                                                                       
>         at org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:549)
~[apache-cassandra-3.6.0.jar:3.6.0]                                                      
                                                                                       
>         at org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217)
~[apache-cassandra-3.6.0.jar:3.6.0]                                                      
                                                                                       
>         at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
~[apache-cassandra-3.6.0.jar:3.6.0]                                                      
                                                                                   
>         at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
~[apache-cassandra-3.6.0.jar:3.6.0]                                                      
                                                                                         
  
>         at org.apache.cassandra.db.rows.Row$Merger.merge(Row.java:526) ~[apache-cassandra-3.6.0.jar:3.6.0]
                                                                                         
                                                                          
>         at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:473)
~[apache-cassandra-3.6.0.jar:3.6.0]                                                      
                                  
>         at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:437)
~[apache-cassandra-3.6.0.jar:3.6.0]                                                      
                                  
>         at org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217)
~[apache-cassandra-3.6.0.jar:3.6.0]                                                      
                                                                                       
>         at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
~[apache-cassandra-3.6.0.jar:3.6.0]                                                      
                                                                                   
>         at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
~[apache-cassandra-3.6.0.jar:3.6.0]                                                      
                                                                                         
  
>         at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:419)
~[apache-cassandra-3.6.0.jar:3.6.0]                                                      
                                              
>         at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:279)
~[apache-cassandra-3.6.0.jar:3.6.0]                                                      
                                              
>         at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
~[apache-cassandra-3.6.0.jar:3.6.0]                                                      
                                                                                         
  
>         at org.apache.cassandra.db.rows.UnfilteredRowIterator.isEmpty(UnfilteredRowIterator.java:70)
~[apache-cassandra-3.6.0.jar:3.6.0]                                                      
                                                                                
>         at org.apache.cassandra.db.SinglePartitionReadCommand.withSSTablesIterated(SinglePartitionReadCommand.java:637)
~[apache-cassandra-3.6.0.jar:3.6.0]                                                      
                                                             
>         at org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDiskInternal(SinglePartitionReadCommand.java:586)
~[apache-cassandra-3.6.0.jar:3.6.0]                                                      
                                                     
>         at org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDisk(SinglePartitionReadCommand.java:463)
~[apache-cassandra-3.6.0.jar:3.6.0]                                                      
                                                             
>         at org.apache.cassandra.db.SinglePartitionReadCommand.queryStorage(SinglePartitionReadCommand.java:325)
~[apache-cassandra-3.6.0.jar:3.6.0]                                                      
                                                                     
>         at org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:366) ~[apache-cassandra-3.6.0.jar:3.6.0]
                                                                                         
                                                             
>         at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1797)
~[apache-cassandra-3.6.0.jar:3.6.0]                                                      
                                                                          
>         at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2466)
~[apache-cassandra-3.6.0.jar:3.6.0]                                                      
                                                                                  
>         ... 5 common frames omitted                                                 
                                                                                         
                                                                                         
      {code}
> I think, the stored files of the counters are corrupted but the repair not fixed the
issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message