Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 74251 invoked from network); 10 Feb 2011 13:52:23 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 10 Feb 2011 13:52:23 -0000 Received: (qmail 50948 invoked by uid 500); 10 Feb 2011 13:52:21 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 50718 invoked by uid 500); 10 Feb 2011 13:52:17 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 50710 invoked by uid 99); 10 Feb 2011 13:52:16 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Feb 2011 13:52:16 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of babo.online@gmail.com designates 209.85.214.172 as permitted sender) Received: from [209.85.214.172] (HELO mail-iw0-f172.google.com) (209.85.214.172) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Feb 2011 13:52:08 +0000 Received: by iwc10 with SMTP id 10so1400520iwc.31 for ; Thu, 10 Feb 2011 05:51:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:x-google-sender-delegation :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type; bh=g2NCpFRny4+4vWdHHvfqEJBnXM/H1qddmjhiJ177QGE=; b=XCnw9feToWj2US7uv3Q3S7WTD9L3r1OfvZMFmarqIqKvLvfwI3k/siH7k0BtHNSgYF iVddZ385TwoPpR1g1n2GzaWrOBVpIyt5NwEQzq1n2kGeWHu8tmT83qIxl3s30qa9octw XTOSP+kESdFG9982vDoDlirtIjFwgw1W67umU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:x-google-sender-delegation:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to :content-type; b=sDmWZsUcb+JOB8JmPkECwfDa+JnMpHVqGM2wE2tyY16hlGkoSGQyhdmi5XZ3jTvrOW ke9nxX7psy3cuOdMYsAJuTdErZaj/qHRVHN/auZ/lUCXIrgSbESxKgZupSeVkV9zwtqy NMbu1/DLeNY80iHdghmF3RsGHApAjX9IiAZAE= MIME-Version: 1.0 Received: by 10.42.217.202 with SMTP id hn10mr8854625icb.168.1297345907155; Thu, 10 Feb 2011 05:51:47 -0800 (PST) Sender: attila.babo@gmail.com X-Google-Sender-Delegation: attila.babo@gmail.com Received: by 10.42.29.198 with HTTP; Thu, 10 Feb 2011 05:51:47 -0800 (PST) In-Reply-To: References: <4d533466.914ee50a.1bc4.2150@mx.google.com> <275F7950-06D5-416B-AF1F-17D9570E2EB4@thelastpickle.com> <79C33A81-875E-437A-8A02-6B6DC9072BEA@thelastpickle.com> Date: Thu, 10 Feb 2011 14:51:47 +0100 X-Google-Sender-Auth: 68NB49m-wXw81shS_J4tHRW0RSs Message-ID: Subject: Re: Exceptions on 0.7.0 From: Attila Babo To: user@cassandra.apache.org Content-Type: text/plain; charset=ISO-8859-1 X-Virus-Checked: Checked by ClamAV on apache.org The same problem here, even with apache-cassandra-2011-02-10_06-30-00-bin.tar.gz from hudson. I'm happy to share the full log if needed or run tests to identify the core problem which looks like an overflow for me. Database was upgraded from 0.6.8, there were no problems with it before. /Attila --------------- This is the first error line: ERROR [CompactionExecutor:1] 2011-02-10 13:25:18,508 AbstractCassandraDaemon.java (line 114) Fatal exception in thread Thread[CompactionExecutor:1,1,main] java.lang.NegativeArraySizeException at org.apache.cassandra.utils.obs.OpenBitSet.(OpenBitSet.java:85) at org.apache.cassandra.utils.BloomFilter.bucketsFor(BloomFilter.java:65) at org.apache.cassandra.utils.BloomFilter.getFilter(BloomFilter.java:82) at org.apache.cassandra.db.ColumnIndexer.serializeInternal(ColumnIndexer.java:62) at org.apache.cassandra.db.ColumnIndexer.serialize(ColumnIndexer.java:50) at org.apache.cassandra.io.LazilyCompactedRow.(LazilyCompactedRow.java:88) at org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:137) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:108) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:43) at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131) at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183) at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94) at org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:416) at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:123) at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:93) After a few errors of the above comes a NullPointerException: ERROR [CompactionExecutor:1] 2011-02-10 13:25:53,807 AbstractCassandraDaemon.java (line 114) Fatal exception in thread Thread[CompactionExecutor:1,1,main] java.lang.NullPointerException at org.apache.cassandra.io.LazilyCompactedRow.(LazilyCompactedRow.java:90) at org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:137) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:108) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:43) at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131) at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183) at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94) at org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:427) at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:123) at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:93) Then comes an EOFException: ERROR [CompactionExecutor:1] 2011-02-10 13:25:55,365 AbstractCassandraDaemon.java (line 114) Fatal exception in thread Thread[CompactionExecutor:1,1,main] java.io.IOError: java.io.EOFException at org.apache.cassandra.io.util.ColumnIterator.deserializeNext(ColumnSortedMap.java:246) at org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:262) at org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:223) at java.util.concurrent.ConcurrentSkipListMap.buildFromSorted(ConcurrentSkipListMap.java:1521) at java.util.concurrent.ConcurrentSkipListMap.(ConcurrentSkipListMap.java:1471) at org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:366) at org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:314) at org.apache.cassandra.io.sstable.SSTableIdentityIterator.next(SSTableIdentityIterator.java:101) at org.apache.cassandra.io.sstable.SSTableIdentityIterator.next(SSTableIdentityIterator.java:34) at org.apache.commons.collections.iterators.CollatingIterator.set(CollatingIterator.java:284) at org.apache.commons.collections.iterators.CollatingIterator.least(CollatingIterator.java:326) at org.apache.commons.collections.iterators.CollatingIterator.next(CollatingIterator.java:230) at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:68) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131) at com.google.common.collect.Iterators$7.computeNext(Iterators.java:604) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131) at org.apache.cassandra.db.ColumnIndexer.serializeInternal(ColumnIndexer.java:76) at org.apache.cassandra.db.ColumnIndexer.serialize(ColumnIndexer.java:50) at org.apache.cassandra.io.LazilyCompactedRow.(LazilyCompactedRow.java:88) at org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:137) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:108) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:43) at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131) at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183) at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94) at org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:427) at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:123) at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:93)