incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohica Jasha <mohica.ja...@gmail.com>
Subject Re: error in cassandra log file under stress
Date Mon, 05 May 2014 21:03:27 GMT
forgot to say which Cassandra version we are using.

Our prod deployment:

C* 2.0.5, DC1:3, DC2:3




On Mon, May 5, 2014 at 5:01 PM, Mohica Jasha <mohica.jasha@gmail.com> wrote:

> one query in our prod system under heavy load failed.
> I cant tell which query it was. But I don't think we were using an invalid
> consistency level since our configuration work all the time except when it
> goes under heavy load.
> If I can reproduce this I will pass more information.
>
> I wonder if this is due to a race condition in Paxos implementation.
> I found the following in the cassandra's log file:
>
>
> ERROR [Native-Transport-Requests:457009] 2014-05-02 13:49:09,794
>>> QueryMessage.java (line 131) Unexpected error during query
>>
>> java.lang.UnsupportedOperationException: Invalid consistency level:
>>> LOCAL_SERIAL
>>
>>         at
>>> org.apache.cassandra.db.ConsistencyLevel.blockFor(ConsistencyLevel.java:137)
>>
>>         at
>>> org.apache.cassandra.service.StorageProxy.beginAndRepairPaxos(StorageProxy.java:416)
>>
>>         at
>>> org.apache.cassandra.service.StorageProxy.cas(StorageProxy.java:228)
>>
>>         at
>>> org.apache.cassandra.cql3.statements.ModificationStatement.executeWithCondition(ModificationStatement.java:423)
>>
>>         at
>>> org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:380)
>>
>>         at
>>> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:188)
>>
>>         at
>>> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:222)
>>
>>         at
>>> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:119)
>>
>>         at
>>> org.apache.cassandra.transport.Message$Dispatcher.messageReceived(Message.java:304)
>>
>>         at
>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
>>
>>         at
>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
>>
>>         at
>>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
>>
>>         at
>>> org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43)
>>
>>         at
>>> org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67)
>>
>>         at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>
>>         at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>
>>         at java.lang.Thread.run(Thread.java:744)
>>
>> ERROR [Native-Transport-Requests:457009] 2014-05-02 13:49:09,794
>>> ErrorMessage.java (line 222) Unexpected exception during request
>>
>> java.lang.UnsupportedOperationException: Invalid consistency level:
>>> LOCAL_SERIAL
>>
>>         at
>>> org.apache.cassandra.db.ConsistencyLevel.blockFor(ConsistencyLevel.java:137)
>>
>>         at
>>> org.apache.cassandra.service.StorageProxy.beginAndRepairPaxos(StorageProxy.java:416)
>>
>>         at
>>> org.apache.cassandra.service.StorageProxy.cas(StorageProxy.java:228)
>>
>>         at
>>> org.apache.cassandra.cql3.statements.ModificationStatement.executeWithCondition(ModificationStatement.java:423)
>>
>>         at
>>> org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:380)
>>
>>         at
>>> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:188)
>>
>>         at
>>> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:222)
>>
>>         at
>>> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:119)
>>
>>         at
>>> org.apache.cassandra.transport.Message$Dispatcher.messageReceived(Message.java:304)
>>
>>         at
>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
>>
>>         at
>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
>>
>>         at
>>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
>>
>>         at
>>> org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43)
>>
>>         at
>>> org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67)
>>
>>         at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>
>>         at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>
>>         at java.lang.Thread.run(Thread.java:744)
>>
>>

Mime
View raw message