Return-Path: X-Original-To: apmail-cassandra-commits-archive@www.apache.org Delivered-To: apmail-cassandra-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id AF2D07431 for ; Fri, 25 Nov 2011 15:36:01 +0000 (UTC) Received: (qmail 82076 invoked by uid 500); 25 Nov 2011 15:36:01 -0000 Delivered-To: apmail-cassandra-commits-archive@cassandra.apache.org Received: (qmail 82049 invoked by uid 500); 25 Nov 2011 15:36:01 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 82040 invoked by uid 99); 25 Nov 2011 15:36:01 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 25 Nov 2011 15:36:01 +0000 X-ASF-Spam-Status: No, hits=-2001.2 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.116] (HELO hel.zones.apache.org) (140.211.11.116) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 25 Nov 2011 15:36:00 +0000 Received: from hel.zones.apache.org (hel.zones.apache.org [140.211.11.116]) by hel.zones.apache.org (Postfix) with ESMTP id 474E69FF32 for ; Fri, 25 Nov 2011 15:35:40 +0000 (UTC) Date: Fri, 25 Nov 2011 15:35:40 +0000 (UTC) From: "Sylvain Lebresne (Commented) (JIRA)" To: commits@cassandra.apache.org Message-ID: <397744129.12066.1322235340293.JavaMail.tomcat@hel.zones.apache.org> In-Reply-To: <1407753269.10783.1322179960274.JavaMail.tomcat@hel.zones.apache.org> Subject: [jira] [Commented] (CASSANDRA-3530) Header class not thread safe, but mutated by multiple threads MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CASSANDRA-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13157196#comment-13157196 ] Sylvain Lebresne commented on CASSANDRA-3530: --------------------------------------------- There is no StorageProxy thread per-se, is there? Why wouldn't 0.8 be subject to the same concurrency bug as reported on that issue? > Header class not thread safe, but mutated by multiple threads > ------------------------------------------------------------- > > Key: CASSANDRA-3530 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3530 > Project: Cassandra > Issue Type: Bug > Affects Versions: 0.8.8, 1.0.3 > Reporter: Sean Bridges > Assignee: Jonathan Ellis > Fix For: 0.8.8, 1.0.4 > > Attachments: 3530-0.8.txt, 3530-v2.txt, CASSANDRA-3530.patch > > > With Cassandra 1.0.3 we are getting exceptions like, > Fatal exception in thread Thread[WRITE-/xx.xx.xx.xx,5,main]java.util.ConcurrentModificationException > at java.util.Hashtable$Enumerator.next(Unknown Source) > at org.apache.cassandra.net.Header.serializedSize(Header.java:97) > at org.apache.cassandra.net.OutboundTcpConnection.messageLength(OutboundTcpConnection.java:164) > at org.apache.cassandra.net.OutboundTcpConnection.write(OutboundTcpConnection.java:154) > at org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:115) > at org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:94) > and, > ERROR [WRITE-/xx.xx.xx.xx] 2011-11-24 22:08:28,981 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[WRITE-/10.30.12.79,5,main]java.lang.NullPointerException > at org.apache.cassandra.net.Header.serializedSize(Header.java:101) > at org.apache.cassandra.net.OutboundTcpConnection.messageLength(OutboundTcpConnection.java:164) > at org.apache.cassandra.net.OutboundTcpConnection.write(OutboundTcpConnection.java:154) > at org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:115) > at org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:94) > It looks like Header is not thread safe, but the same header instance is modified concurrently while being sent to several threads in StorageProxy.sendMessages. > This bug eventually causes the node to OOM, as it kills the OutboundTcpConnection thread, which means nothing is dequeing from queue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira