cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brandon Williams (JIRA)" <j...@apache.org>
Subject [jira] Commented: (CASSANDRA-2073) Streaming occasionally makes gossip back up
Date Fri, 28 Jan 2011 22:10:43 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-2073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12988272#action_12988272
] 

Brandon Williams commented on CASSANDRA-2073:
---------------------------------------------

It looks like in my case, both nodes are stuck in the same spot:

{noformat}

"GossipStage:1" prio=10 tid=0x0000000041644000 nid=0xece waiting on condition [0x00007fdbe861f000]
   java.lang.Thread.State: WAITING (parking)
    at sun.misc.Unsafe.park(Native Method)
    - parking to wait for  <0x00007fdc5ecf8828> (a java.util.concurrent.FutureTask$Sync)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281)
    at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
    at java.util.concurrent.FutureTask.get(FutureTask.java:83)
    at org.apache.cassandra.db.HintedHandOffManager.deleteHintsForEndPoint(HintedHandOffManager.java:156)
    at org.apache.cassandra.service.StorageService.excise(StorageService.java:831)
    at org.apache.cassandra.service.StorageService.handleStateLeft(StorageService.java:787)
    at org.apache.cassandra.service.StorageService.onChange(StorageService.java:645)
    at org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:742)
    at org.apache.cassandra.gms.Gossiper.applyApplicationStateLocally(Gossiper.java:732)
    at org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:649)
    at org.apache.cassandra.gms.GossipDigestAckVerbHandler.doVerb(GossipDigestAckVerbHandler.java:68)
    at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:70)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
{noformat}

They are waiting on deleting hints, and then likely for the hints CF to compact.

> Streaming occasionally makes gossip back up
> -------------------------------------------
>
>                 Key: CASSANDRA-2073
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2073
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.0
>            Reporter: Brandon Williams
>            Priority: Minor
>
> Streaming occasionally makes gossip back up, causing nodes to mark each other as down
even though the network is ok.  This appears to happen just after streaming has finished.
 I noticed this in the course of working on CASSANDRA-2072, so decommission is one way to
reproduce.  It seems to happen maybe one of fifteen or twenty tries, so it's fairly rare.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message