From user-return-35077-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Fri Jul 5 15:16:26 2013 Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id F19DEF744 for ; Fri, 5 Jul 2013 15:16:26 +0000 (UTC) Received: (qmail 60921 invoked by uid 500); 5 Jul 2013 15:16:24 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 60897 invoked by uid 500); 5 Jul 2013 15:16:24 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 60883 invoked by uid 99); 5 Jul 2013 15:16:20 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 05 Jul 2013 15:16:20 +0000 X-ASF-Spam-Status: No, hits=-0.1 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_MED,UNPARSEABLE_RELAY X-Spam-Check-By: apache.org Received-SPF: error (athena.apache.org: local policy) Received: from [193.109.254.118] (HELO mail1.bemta14.messagelabs.com) (193.109.254.118) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 05 Jul 2013 15:16:15 +0000 Received: from [193.109.255.147:55294] by server-14.bemta-14.messagelabs.com id F7/8D-23280-513E6D15; Fri, 05 Jul 2013 15:15:33 +0000 X-Env-Sender: gdavies@omnifone.com X-Msg-Ref: server-11.tower-72.messagelabs.com!1373037332!9807237!1 X-Originating-IP: [217.156.157.31] X-StarScan-Received: X-StarScan-Version: 6.9.9; banners=omnifone.com,-,- X-VirusChecked: Checked Received: (qmail 32379 invoked from network); 5 Jul 2013 15:15:32 -0000 Received: from unknown (HELO EXFE1IS02.omnifone.com) (217.156.157.31) by server-11.tower-72.messagelabs.com with AES128-SHA encrypted SMTP; 5 Jul 2013 15:15:32 -0000 Received: from EXBE1IS02.omnifone.com ([::1]) by EXFE1IS02.omnifone.com ([::1]) with mapi id 14.01.0438.000; Fri, 5 Jul 2013 16:15:32 +0100 From: Glyn Davies To: "user@cassandra.apache.org" Subject: CassandraDaemon - recent unsafe memory access operation in compiled Java code Thread-Topic: CassandraDaemon - recent unsafe memory access operation in compiled Java code Thread-Index: AQHOeZJ9eQvZybPCfE+Z8K28sg7RBQ== Date: Fri, 5 Jul 2013 15:15:31 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.3.1.130117 x-originating-ip: [10.4.1.15] Content-Type: multipart/alternative; boundary="_000_CDFCA1A11B5E3gdaviesomnifonecom_" MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org --_000_CDFCA1A11B5E3gdaviesomnifonecom_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Hi, Just starting to experiment with Cassandra, and have hit an early snag. I'm using 1.2.6 on Ubuntu AWS m1.xlarge instances with the Datastax Commun= ity package and have tried using Java versions jdk1.7.0_25 jre1.6.0_45 Also testing with and without libjna-java However, something has triggered a bug in the CassandraDaemon: ERROR [COMMIT-LOG-ALLOCATOR] 2013-07-05 15:00:51,663 CassandraDaemon.java = (line 192) Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main] java.lang.InternalError: a fault occurred in a recent unsafe memory access= operation in compiled Java code at org.apache.cassandra.db.commitlog.CommitLogSegment.(Commi= tLogSegment.java:126) at org.apache.cassandra.db.commitlog.CommitLogSegment.freshSegment= (CommitLogSegment.java:81) at org.apache.cassandra.db.commitlog.CommitLogAllocator.createFres= hSegment(CommitLogAllocator.java:250) at org.apache.cassandra.db.commitlog.CommitLogAllocator.access$500= (CommitLogAllocator.java:48) at org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayTh= row(CommitLogAllocator.java:104) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.= java:28) at java.lang.Thread.run(Unknown Source) This brought two nodes down out of a three node cluster =96 using QUORUM w= rite with 3 replicas. Restarting the node replays this error, so I have the system in a 'stable'= unstable state =96 which is probably a good place for trouble shooting. Presumably something a client wrote triggered this situation, and the othe= r third node was to be the final replication point =96 and is thus still u= p. Any suggestions on next steps? I've had a good google for the error combinations, but didn't find any hit= s. Thanks, Glyn ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. For more information please visit http://www.symanteccloud.com ______________________________________________________________________ --_000_CDFCA1A11B5E3gdaviesomnifonecom_ Content-Type: text/html; charset="Windows-1252" Content-ID: Content-Transfer-Encoding: quoted-printable


Hi,

Just starting to experiment with Cassandra, and have hit an early sna= g.

I'm using 1.2.6 on Ubuntu AWS m1.xlarge instances with the Datastax C= ommunity package and have tried using Java versions jdk1.7.0_25  jre1= .6.0_45
Also testing with and without libjna-java

However, something has triggered a bug in the CassandraDaemon:

ERROR [COMMIT-LOG-ALLOCATOR] 2013-07-05 15:00:51,663 CassandraDaemon.= java (line 192) Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main]
java.lang.InternalError: a fault occurred in a recent unsafe memory a= ccess operation in compiled Java code
        at org.apache.cassandra.db.commitlog.Comm= itLogSegment.<init>(CommitLogSegment.java:126)
        at org.apache.cassandra.db.commitlog.Comm= itLogSegment.freshSegment(CommitLogSegment.java:81)
        at org.apache.cassandra.db.commitlog.Comm= itLogAllocator.createFreshSegment(CommitLogAllocator.java:250)
        at org.apache.cassandra.db.commitlog.Comm= itLogAllocator.access$500(CommitLogAllocator.java:48)
        at org.apache.cassandra.db.commitlog.Comm= itLogAllocator$1.runMayThrow(CommitLogAllocator.java:104)
        at org.apache.cassandra.utils.WrappedRunn= able.run(WrappedRunnable.java:28)
        at java.lang.Thread.run(Unknown Source)

This brought two nodes down out of a three node cluster =96 using QUO= RUM write with 3 replicas.
Restarting the node replays this error, so I have the system in a 'st= able' unstable state =96 which is probably a good place for trouble shooti= ng.

Presumably something a client wrote triggered this situation, and the= other third node was to be the final replication point =96 and is thus st= ill up.

Any suggestions on next steps?
I've had a good google for the error combinations, but didn't find an= y hits.

Thanks,

Glyn

______________________________________________________________________
= This email has been scanned by the Symantec Email Security.cloud service.<= BR> For more information please visit http://www.symanteccloud.com
______________________________________________________________________
= --_000_CDFCA1A11B5E3gdaviesomnifonecom_--