Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Wed, 21 Nov 2012 17:27:58 +0000 (UTC)
From: "Tyler Patterson (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <2071394447.13108.1353518878798.JavaMail.jiratomcat@arcas>
In-Reply-To: <1005784227.13103.1353518759973.JavaMail.jiratomcat@arcas>
Subject: [jira] [Updated] (CASSANDRA-4981) Error when starting a node with
 vnodes while counter-add operations underway
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/CASSANDRA-4981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tyler Patterson updated CASSANDRA-4981:
---------------------------------------

    Description: 
Start both nodes, start stress on one node like this: "cassandra-stress --replication-factor=2 --operation=COUNTER_ADD"

While that is running: On the other node, kill cassandra, wait for "nodetool status" to show the node as down, and restart cassandra. I sometimes have to kill and restart cassandra several times to get the problem to happen.

I get this error several times in the log:

{code}
ERROR 15:39:33,198 Exception in thread Thread[MutationStage:16,5,main]
java.lang.AssertionError
	at org.apache.cassandra.locator.TokenMetadata.firstTokenIndex(TokenMetadata.java:748)
	at org.apache.cassandra.locator.TokenMetadata.firstToken(TokenMetadata.java:762)
	at org.apache.cassandra.locator.AbstractReplicationStrategy.getNaturalEndpoints(AbstractReplicationStrategy.java:95)
	at org.apache.cassandra.service.StorageService.getNaturalEndpoints(StorageService.java:2426)
	at org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:396)
	at org.apache.cassandra.service.StorageProxy.applyCounterMutationOnLeader(StorageProxy.java:755)
	at org.apache.cassandra.db.CounterMutationVerbHandler.doVerb(CounterMutationVerbHandler.java:53)
	at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:662)
{code}

  was:
Start both nodes, start stress on one node like this: "cassandra-stress --replication-factor=2 --operation=COUNTER_ADD"

While that is running: On the other node, kill cassandra, wait for "nodetool status" to show the node as down, and restart cassandra. I sometimes have to kill and restart cassandra several times to get the problem to happen.

{code}
ERROR 15:39:33,198 Exception in thread Thread[MutationStage:16,5,main]
java.lang.AssertionError
	at org.apache.cassandra.locator.TokenMetadata.firstTokenIndex(TokenMetadata.java:748)
	at org.apache.cassandra.locator.TokenMetadata.firstToken(TokenMetadata.java:762)
	at org.apache.cassandra.locator.AbstractReplicationStrategy.getNaturalEndpoints(AbstractReplicationStrategy.java:95)
	at org.apache.cassandra.service.StorageService.getNaturalEndpoints(StorageService.java:2426)
	at org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:396)
	at org.apache.cassandra.service.StorageProxy.applyCounterMutationOnLeader(StorageProxy.java:755)
	at org.apache.cassandra.db.CounterMutationVerbHandler.doVerb(CounterMutationVerbHandler.java:53)
	at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:662)
{code}

    
> Error when starting a node with vnodes while counter-add operations underway
> ----------------------------------------------------------------------------
>
>                 Key: CASSANDRA-4981
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4981
>             Project: Cassandra
>          Issue Type: Bug
>         Environment: 2-node cluster on ec2, ubuntu, cassandra-1.2.0 commit a32eb9f7d2f2868e8154d178e96e045859e1d855
>            Reporter: Tyler Patterson
>
> Start both nodes, start stress on one node like this: "cassandra-stress --replication-factor=2 --operation=COUNTER_ADD"
> While that is running: On the other node, kill cassandra, wait for "nodetool status" to show the node as down, and restart cassandra. I sometimes have to kill and restart cassandra several times to get the problem to happen.
> I get this error several times in the log:
> {code}
> ERROR 15:39:33,198 Exception in thread Thread[MutationStage:16,5,main]
> java.lang.AssertionError
> 	at org.apache.cassandra.locator.TokenMetadata.firstTokenIndex(TokenMetadata.java:748)
> 	at org.apache.cassandra.locator.TokenMetadata.firstToken(TokenMetadata.java:762)
> 	at org.apache.cassandra.locator.AbstractReplicationStrategy.getNaturalEndpoints(AbstractReplicationStrategy.java:95)
> 	at org.apache.cassandra.service.StorageService.getNaturalEndpoints(StorageService.java:2426)
> 	at org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:396)
> 	at org.apache.cassandra.service.StorageProxy.applyCounterMutationOnLeader(StorageProxy.java:755)
> 	at org.apache.cassandra.db.CounterMutationVerbHandler.doVerb(CounterMutationVerbHandler.java:53)
> 	at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> 	at java.lang.Thread.run(Thread.java:662)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira