Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 29051 invoked from network); 10 Aug 2010 00:19:23 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 10 Aug 2010 00:19:23 -0000 Received: (qmail 87540 invoked by uid 500); 10 Aug 2010 00:19:22 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 87492 invoked by uid 500); 10 Aug 2010 00:19:21 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 87484 invoked by uid 99); 10 Aug 2010 00:19:21 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 10 Aug 2010 00:19:21 +0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests=FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of jbellis@gmail.com designates 209.85.161.44 as permitted sender) Received: from [209.85.161.44] (HELO mail-fx0-f44.google.com) (209.85.161.44) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 10 Aug 2010 00:19:15 +0000 Received: by fxm18 with SMTP id 18so400874fxm.31 for ; Mon, 09 Aug 2010 17:18:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:in-reply-to :references:from:date:message-id:subject:to:content-type :content-transfer-encoding; bh=w1gue6zA7FCDBmMJeJLLbP/4mr0a8X8tZK6MWmEovjg=; b=wdIcm/O3ijlxCuYk2cLxOz/iJzoesDyGXfN4v+oBQbSrCtg/3BlxqOtDiuXh1OraPO 2Kc3/1wtNbl/6RZ6jq/yjUcFhBM9gvz1bvSMWDKNBjI1stW7c00dkMcZ/9zJL9Xh8YFO a2HcIowruP2dLQrjJNuhEAIMfA5RCJ2nuqpAw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type:content-transfer-encoding; b=Ly5Tjak7gt9mcvlnFptIxS/JfBGf4Yaxp0U34aT3MUCC9xh+P+PeiJ4A5hRn9i/omc K5s+oJ3iuw3CJmX6e9NFGtV6FKXqO80/8cZBn2k/YExQ/JD4ExKOTNI3WWdSxi7z4Ni/ ASmL7rP7xTF0tw/I6szD/Uom2Mtyjc5gzmVCI= Received: by 10.223.104.139 with SMTP id p11mr17243046fao.106.1281399535149; Mon, 09 Aug 2010 17:18:55 -0700 (PDT) MIME-Version: 1.0 Received: by 10.223.38.92 with HTTP; Mon, 9 Aug 2010 17:18:35 -0700 (PDT) In-Reply-To: <7612794.13.1281382252611.JavaMail.arya@aryanet> References: <15575896.11.1281382238884.JavaMail.arya@aryanet> <7612794.13.1281382252611.JavaMail.arya@aryanet> From: Jonathan Ellis Date: Mon, 9 Aug 2010 20:18:35 -0400 Message-ID: Subject: Re: COMMIT-LOG_WRITER Assertion Error To: user@cassandra.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org Sounds like you upgraded to trunk from 0.6 without draining your commitlog first? On Mon, Aug 9, 2010 at 3:30 PM, Arya Goudarzi wr= ote: > Just throwing this out there as it could be a concern. I had a cluster of= 3 nodes running. Over the weekend I updated to trunc (Aug 9th @ 2pm). Toda= y, I came to run my daily tests and my client kept giving me TSocket timeou= ts. Checking the error log of Cassandra servers, all 3 nodes had this and t= hey all became unresponsive! Not sure how to reproduce this but a restart o= f all 3 nodes fixed the issue: > > ERROR [COMMIT-LOG-WRITER] 2010-08-09 11:30:27,722 CassandraDaemon.java (l= ine 82) Uncaught exception in thread Thread[COMMIT-LOG-WRITER,5,main] > java.lang.AssertionError > =A0 =A0 =A0 =A0at org.apache.cassandra.db.commitlog.CommitLogHeader$Commi= tLogHeaderSerializer.serialize(CommitLogHeader.java:157) > =A0 =A0 =A0 =A0at org.apache.cassandra.db.commitlog.CommitLogHeader.write= CommitLogHeader(CommitLogHeader.java:124) > =A0 =A0 =A0 =A0at org.apache.cassandra.db.commitlog.CommitLogSegment.writ= eHeader(CommitLogSegment.java:70) > =A0 =A0 =A0 =A0at org.apache.cassandra.db.commitlog.CommitLogSegment.writ= e(CommitLogSegment.java:103) > =A0 =A0 =A0 =A0at org.apache.cassandra.db.commitlog.CommitLog$LogRecordAd= der.run(CommitLog.java:521) > =A0 =A0 =A0 =A0at org.apache.cassandra.db.commitlog.PeriodicCommitLogExec= utorService$1.runMayThrow(PeriodicCommitLogExecutorService.java:52) > =A0 =A0 =A0 =A0at org.apache.cassandra.utils.WrappedRunnable.run(WrappedR= unnable.java:30) > =A0 =A0 =A0 =A0at java.lang.Thread.run(Thread.java:636) > > > -Arya > --=20 Jonathan Ellis Project Chair, Apache Cassandra co-founder of Riptano, the source for professional Cassandra support http://riptano.com