cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keith Wright (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CASSANDRA-6948) Native & ReplicateOnWriteStage AssertionError: Callback already exists
Date Fri, 28 Mar 2014 14:47:15 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-6948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Keith Wright updated CASSANDRA-6948:
------------------------------------

    Attachment: system.log.1.gz
                system.log.gz

Attached for one of the nodes showing the issue. 

> Native & ReplicateOnWriteStage AssertionError: Callback already exists
> ----------------------------------------------------------------------
>
>                 Key: CASSANDRA-6948
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6948
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>            Reporter: Keith Wright
>         Attachments: cassandra.yaml, system.log.1.gz, system.log.gz
>
>
> We are using the datastax driver and have seen the following exceptions appear periodically
in our cluster in "groups".  The client will gradually start sending less write requests to
the nodes where this occurs the most often.  This may be related to CASSANDRA-6476.  Exception
from logs:
> ERROR [Native-Transport-Requests:7636] 2014-03-28 13:32:10,638 ErrorMessage.java (line
222) Unexpected exception during request
> java.lang.AssertionError: Callback already exists for id -1665979622! (CallbackInfo(target=/10.106.160.84,
callback=org.apache.cassandra.service.WriteResponseHandler@5d36d8ea, serializer=org.apache.cassandra.db.WriteResponse$WriteResponseSerializer@6ed37f0b))
> 	at org.apache.cassandra.net.MessagingService.addCallback(MessagingService.java:549)
> 	at org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:601)
> 	at org.apache.cassandra.service.StorageProxy.mutateCounter(StorageProxy.java:984)
> 	at org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:449)
> 	at org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:524)
> 	at org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:521)
> 	at org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:505)
> 	at org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:188)
> 	at org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:358)
> 	at org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:131)
> 	at org.apache.cassandra.transport.Message$Dispatcher.messageReceived(Message.java:304)
> 	at org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43)
> 	at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:744)
> ERROR [ReplicateOnWriteStage:102766] 2014-03-28 13:32:10,638 CassandraDaemon.java (line
196) Exception in thread Thread[ReplicateOnWriteStage:102766,5,main]
> java.lang.AssertionError: Callback already exists for id -1665979620! (CallbackInfo(target=/10.106.160.84,
callback=org.apache.cassandra.service.WriteResponseHandler@3bdb1a75, serializer=org.apache.cassandra.db.WriteResponse$WriteResponseSerializer@6ed37f0b))
> 	at org.apache.cassandra.net.MessagingService.addCallback(MessagingService.java:549)
> 	at org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:601)
> 	at org.apache.cassandra.service.StorageProxy.sendToHintedEndpoints(StorageProxy.java:806)
> 	at org.apache.cassandra.service.StorageProxy$8$1.runMayThrow(StorageProxy.java:1074)
> 	at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1896)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:744)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message