Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 35410 invoked from network); 15 Feb 2011 19:00:57 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 15 Feb 2011 19:00:57 -0000 Received: (qmail 65151 invoked by uid 500); 15 Feb 2011 19:00:54 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 64948 invoked by uid 500); 15 Feb 2011 19:00:51 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 64931 invoked by uid 99); 15 Feb 2011 19:00:50 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 15 Feb 2011 19:00:50 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of dan.hendry.junk@gmail.com designates 209.85.216.179 as permitted sender) Received: from [209.85.216.179] (HELO mail-qy0-f179.google.com) (209.85.216.179) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 15 Feb 2011 19:00:46 +0000 Received: by qyj19 with SMTP id 19so398881qyj.10 for ; Tue, 15 Feb 2011 11:00:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:from:to:references:in-reply-to:subject:date :message-id:mime-version:content-type:x-mailer:thread-index :content-language; bh=exEjEfTe2OBQoKtmwd/Wd3LOcbs+882mYlYprNXMZT4=; b=gv5ABBWzYT/qbFvx/kuOSCQ32IIUMu3/z8jFO5mfaqTHe1xS/tZ22pDK/fA+SRw2gO E+Lu39Z4yH5dT1d2/0aHeOVFaAmxVbl8jz1As1O6m6IcEcJcHI5A5PA6Tm5CB32ohsHp p058CwvoeJ7v23h/aSRvxX3XvAjKD3So+J5EA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:references:in-reply-to:subject:date:message-id:mime-version :content-type:x-mailer:thread-index:content-language; b=LnTUs2CbqrR2n2OM8krbzzNPP+Zb08XGLawiYKmbGp6ztGs0oIWEV0UydZCU8V6mU/ krAQideXLOfAp5JyXLsB4HHt4bYMZCwKB3t/8uvNYoYpIfzDCGXvnww2oBMjCTqtzevo 6OLIdibwJVFgvgpacZojj7ARCIzPNJ5VhVkV0= Received: by 10.229.83.201 with SMTP id g9mr4425649qcl.33.1297796424732; Tue, 15 Feb 2011 11:00:24 -0800 (PST) Received: from DHTABLET (kik.client.acceleratorcentre.net [38.121.79.182]) by mx.google.com with ESMTPS id l17sm2857174qck.20.2011.02.15.11.00.22 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 15 Feb 2011 11:00:23 -0800 (PST) From: "Dan Hendry" To: References: <4D5AA409.8070207@trioptima.com> In-Reply-To: Subject: RE: Possible EOFException regression in 0.7.1 Date: Tue, 15 Feb 2011 14:00:19 -0500 Message-ID: <4d5acd47.914ee50a.6241.ffffdacd@mx.google.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_006D_01CBCD18.AF04AEE0" X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcvNPGZn8iN4SKJjT4a3JNSnSfTHZwABDjTQ Content-Language: en-ca This is a multi-part message in MIME format. ------=_NextPart_000_006D_01CBCD18.AF04AEE0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable I have been having plenty of problems (on 0.7.0, http://www.mail-archive.com/user@cassandra.apache.org/msg09341.html, http://www.mail-archive.com/user@cassandra.apache.org/msg09230.html, http://www.mail-archive.com/user@cassandra.apache.org/msg09122.html, http://www.mail-archive.com/dev@cassandra.apache.org/msg01746.html, and = from others: http://www.mail-archive.com/user@cassandra.apache.org/msg09838.html,) = which are very similar to what was reported and apparently fixed for this = case. In my instance, I have not been able to find a reproducible case but its = not all that feasible to log what is going into my nodes. Could this bug = have existed in 0.7.0 in another form or could this problem occur elsewhere = in the code? =20 Dan =20 From: Sylvain Lebresne [mailto:sylvain@datastax.com]=20 Sent: February-15-11 13:15 To: user@cassandra.apache.org Subject: Re: Possible EOFException regression in 0.7.1 =20 On Tue, Feb 15, 2011 at 7:10 PM, ruslan usifov wrote: It will be great if patch appear very quick =20 patch attached here: = https://issues.apache.org/jira/browse/CASSANDRA-2165 =20 Hoping this is quick enough. =20 =20 2011/2/15 Jonathan Ellis =20 I can reproduce with your script. Thanks! 2011/2/15 Jonas Borgstr=F6m : > Hi all, > > While testing the new 0.7.1 release I got the following exception: > > ERROR [ReadStage:11] 2011-02-15 16:39:18,105 > DebuggableThreadPoolExecutor.java (line 103) Error in = ThreadPoolExecutor > java.io.IOError: java.io.EOFException > at > org.apache.cassandra.db.columniterator.SSTableNamesIterator.(SSTabl= eNa mesIterator.java:75) > at > org.apache.cassandra.db.filter.NamesQueryFilter.getSSTableColumnIterator(= Nam esQueryFilter.java:59) > at > org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(Query= Fil ter.java:80) > at > org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamily= Sto re.java:1274) > at > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilySto= re. java:1166) > at > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilySto= re. java:1095) > at org.apache.cassandra.db.Table.getRow(Table.java:384) > at > org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCo= mma nd.java:60) > at > org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(S= tor ageProxy.java:473) > at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java= :11 10) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav= a:6 03) > at java.lang.Thread.run(Thread.java:636) > Caused by: java.io.EOFException > at java.io.DataInputStream.readInt(DataInputStream.java:392) > at > org.apache.cassandra.utils.BloomFilterSerializer.deserialize(BloomFilterS= eri alizer.java:48) > at > org.apache.cassandra.utils.BloomFilterSerializer.deserialize(BloomFilterS= eri alizer.java:30) > at > org.apache.cassandra.io.sstable.IndexHelper.defreezeBloomFilter(IndexHelp= er. java:108) > at > org.apache.cassandra.db.columniterator.SSTableNamesIterator.read(SSTableN= ame sIterator.java:106) > at > org.apache.cassandra.db.columniterator.SSTableNamesIterator.(SSTabl= eNa mesIterator.java:71) > ... 12 more > > I'm able reliably reproduce this using the following one node cluster: > - apache-cassandra-0.7.1-bin.tar.gz > - Fedora 14 > - java version "1.6.0_20". > OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode) > - Default cassandra.yaml > - cassandra-env.sh: MAX_HEAP_SIZE=3D"1G"; HEAP_NEWSIZE=3D"200M" > > cassandra-cli initialization: > - create keyspace foo; > - use foo; > - create column family datasets; > > $ python dataset_check.py (attached) > Inserting row 0 of 10 > Inserting row 1 of 10 > Inserting row 2 of 10 > Inserting row 3 of 10 > Inserting row 4 of 10 > Inserting row 5 of 10 > Inserting row 6 of 10 > Inserting row 7 of 10 > Inserting row 8 of 10 > Inserting row 9 of 10 > Attempting to fetch key 0 > Traceback (most recent call last): > ... > pycassa.pool.MaximumRetryException: Retried 6 times > > After this I have 6 EOFExceptions in system.log. > Running "get datasets[0]['name'];" using cassandra-cli also triggers = the > same exception. > I've not been able to reproduce this with cassandra 0.7.0. > > Regards, > Jonas > > > -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com =20 =20 No virus found in this incoming message. Checked by AVG - www.avg.com Version: 9.0.872 / Virus Database: 271.1.1/3443 - Release Date: 02/15/11 02:34:00 ------=_NextPart_000_006D_01CBCD18.AF04AEE0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

I have been having plenty of problems (on 0.7.0, http://www.mail-archive.com/user@cassandra.apache.org/msg09341.html, http://www.mail-archive.com/user@cassandra.apache.org/msg09230.html, http://www.mail-archive.com/user@cassandra.apache.org/msg09122.html, http://www.mail-archive.com/dev@cassandra.apache.org/msg01746.html= , and from others: http://www.mail-archive.com/user@cassandra.apache.org/msg09838.html,) which are very similar to what was reported and apparently fixed = for this case. In my instance, I have not been able to find a = reproducible case but its not all that feasible to log what is going = into my nodes. Could this bug have existed in 0.7.0 in another form or = could this problem occur elsewhere in the code?

 

Dan

 

 

On Tue, = Feb 15, 2011 at 7:10 PM, ruslan usifov <ruslan.usifov@gmail.com> = wrote:

It will be great if patch = appear very quick

 

 

Hoping this is quick = enough.

 

 

2011/2/15 Jonathan Ellis <jbellis@gmail.com>

 

I can = reproduce with your script.  Thanks!

2011/2/15 Jonas = Borgstr=F6m <jonas.borgstrom@trioptima.com>:

> Hi = all,
>
> While testing the new 0.7.1 release I got the = following exception:
>
> ERROR [ReadStage:11] 2011-02-15 = 16:39:18,105
> DebuggableThreadPoolExecutor.java (line 103) Error = in ThreadPoolExecutor
> java.io.IOError: = java.io.EOFException
>        at
> = org.apache.cassandra.db.columniterator.SSTableNamesIterator.<init>(= SSTableNamesIterator.java:75)
>       =  at
> = org.apache.cassandra.db.filter.NamesQueryFilter.getSSTableColumnIterator(= NamesQueryFilter.java:59)
>        at
> = org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(Query= Filter.java:80)
>        at
> = org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamily= Store.java:1274)
>        at
> = org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilySto= re.java:1166)
>        at
> = org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilySto= re.java:1095)
>        at = org.apache.cassandra.db.Table.getRow(Table.java:384)
>   =      at
> = org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCo= mmand.java:60)
>        at
> = org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(S= torageProxy.java:473)
>        at = org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)>        at
> = java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java= :1110)
>        at
> = java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav= a:603)
>        at = java.lang.Thread.run(Thread.java:636)
> Caused by: = java.io.EOFException
>        at = java.io.DataInputStream.readInt(DataInputStream.java:392)
>   =      at
> = org.apache.cassandra.utils.BloomFilterSerializer.deserialize(BloomFilterS= erializer.java:48)
>        at
> = org.apache.cassandra.utils.BloomFilterSerializer.deserialize(BloomFilterS= erializer.java:30)
>        at
> = org.apache.cassandra.io.sstable.IndexHelper.defreezeBloomFilter(IndexHelp= er.java:108)
>        at
> = org.apache.cassandra.db.columniterator.SSTableNamesIterator.read(SSTableN= amesIterator.java:106)
>        at
> = org.apache.cassandra.db.columniterator.SSTableNamesIterator.<init>(= SSTableNamesIterator.java:71)
>        ... 12 = more
>
> I'm able reliably reproduce this using the = following one node cluster:
> - = apache-cassandra-0.7.1-bin.tar.gz
> - Fedora 14
> - java = version "1.6.0_20".
>  OpenJDK 64-Bit Server VM = (build 19.0-b09, mixed mode)
> - Default cassandra.yaml
> - = cassandra-env.sh: MAX_HEAP_SIZE=3D"1G"; = HEAP_NEWSIZE=3D"200M"
>
> cassandra-cli = initialization:
> - create keyspace foo;
> - use = foo;
> - create column family datasets;
>
> $ python = dataset_check.py (attached)
> Inserting row 0 of 10
> = Inserting row 1 of 10
> Inserting row 2 of 10
> Inserting = row 3 of 10
> Inserting row 4 of 10
> Inserting row 5 of = 10
> Inserting row 6 of 10
> Inserting row 7 of 10
> = Inserting row 8 of 10
> Inserting row 9 of 10
> Attempting = to fetch key 0
> Traceback (most recent call last):
> = ...
> pycassa.pool.MaximumRetryException: Retried 6 = times
>
> After this I have 6 EOFExceptions in = system.log.
> Running "get datasets[0]['name'];" using = cassandra-cli also triggers the
> same exception.
> I've not = been able to reproduce this with cassandra 0.7.0.
>
> = Regards,
> = Jonas
>
>
>


--
Jonathan = Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the = source for professional Cassandra support
http://www.datastax.com

 

 

No virus = found in this incoming message.
Checked by AVG - = www.avg.com
Version: 9.0.872 / Virus Database: 271.1.1/3443 - Release = Date: 02/15/11 02:34:00

------=_NextPart_000_006D_01CBCD18.AF04AEE0--