Return-Path: X-Original-To: apmail-cassandra-commits-archive@www.apache.org Delivered-To: apmail-cassandra-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 434A19D73 for ; Thu, 9 Feb 2012 10:39:35 +0000 (UTC) Received: (qmail 35587 invoked by uid 500); 9 Feb 2012 10:39:33 -0000 Delivered-To: apmail-cassandra-commits-archive@cassandra.apache.org Received: (qmail 35469 invoked by uid 500); 9 Feb 2012 10:39:25 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 35450 invoked by uid 99); 9 Feb 2012 10:39:24 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 09 Feb 2012 10:39:23 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED,T_RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.116] (HELO hel.zones.apache.org) (140.211.11.116) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 09 Feb 2012 10:39:20 +0000 Received: from hel.zones.apache.org (hel.zones.apache.org [140.211.11.116]) by hel.zones.apache.org (Postfix) with ESMTP id 8F91E1AC40B for ; Thu, 9 Feb 2012 10:38:59 +0000 (UTC) Date: Thu, 9 Feb 2012 10:38:59 +0000 (UTC) From: "Peter Schuller (Commented) (JIRA)" To: commits@cassandra.apache.org Message-ID: <896061855.18893.1328783939589.JavaMail.tomcat@hel.zones.apache.org> In-Reply-To: <1104023920.312.1328139953495.JavaMail.tomcat@hel.zones.apache.org> Subject: [jira] [Commented] (CASSANDRA-3832) gossip stage backed up due to migration manager future de-ref MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/CASSANDRA-3832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204433#comment-13204433 ] Peter Schuller commented on CASSANDRA-3832: ------------------------------------------- The remaining issue is still causing problems for bootstrap, though not quite as sever as the original problem. Follow-up work filed in CASSANDRA-3882. > gossip stage backed up due to migration manager future de-ref > -------------------------------------------------------------- > > Key: CASSANDRA-3832 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3832 > Project: Cassandra > Issue Type: Bug > Components: Core > Affects Versions: 1.1 > Reporter: Peter Schuller > Assignee: Peter Schuller > Priority: Blocker > Fix For: 1.1 > > Attachments: CASSANDRA-3832-trunk-dontwaitonfuture.txt > > > This is just bootstrapping a ~ 180 trunk cluster. After a while, a > node I was on was stuck with thinking all nodes are down, because > gossip stage was backed up, because it was spending a long time > (multiple seconds or more, I suppose RPC timeout maybe) doing the > following. Cluster-wide restart -> back to normal. I have not > investigated further. > {code} > "GossipStage:1" daemon prio=10 tid=0x00007f9d5847a800 nid=0xa6fc waiting on condition [0x000000004345f000] > java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x00000005029ad1c0> (a java.util.concurrent.FutureTask$Sync) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) > at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969) > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281) > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218) > at java.util.concurrent.FutureTask.get(FutureTask.java:83) > at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:364) > at org.apache.cassandra.service.MigrationManager.rectifySchema(MigrationManager.java:132) > at org.apache.cassandra.service.MigrationManager.onAlive(MigrationManager.java:75) > at org.apache.cassandra.gms.Gossiper.markAlive(Gossiper.java:802) > at org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:918) > at org.apache.cassandra.gms.GossipDigestAckVerbHandler.doVerb(GossipDigestAckVerbHandler.java:68) > {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira