Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1EC31D2BA for ; Mon, 1 Oct 2012 00:13:44 +0000 (UTC) Received: (qmail 81527 invoked by uid 500); 1 Oct 2012 00:13:41 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 81487 invoked by uid 500); 1 Oct 2012 00:13:41 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 81477 invoked by uid 99); 1 Oct 2012 00:13:41 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Oct 2012 00:13:41 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [208.113.200.5] (HELO homiemail-a57.g.dreamhost.com) (208.113.200.5) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Oct 2012 00:13:35 +0000 Received: from homiemail-a57.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a57.g.dreamhost.com (Postfix) with ESMTP id 39233208060 for ; Sun, 30 Sep 2012 17:13:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=thelastpickle.com; h=from :content-type:message-id:mime-version:subject:date:references:to :in-reply-to; s=thelastpickle.com; bh=570DDPsxo0+B1R1ynRrUVvztsO 8=; b=u57BAFB1MCoPgVyUzpvbgGRDdOS5s9ozMr6CmolWrrsDKS7kOFkMn4FPY/ 4o0EH4e4bZLVrGpLwQGtCi3vPJVB14iASE0wL6PUjYuVkkjBNHcardT40GUcEoLh votUkOfJOVHTqWlqrOXF0CJSmHToYFekJv3mBmHmyEZLGe8/c= Received: from [192.168.2.77] (unknown [116.90.132.105]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: aaron@thelastpickle.com) by homiemail-a57.g.dreamhost.com (Postfix) with ESMTPSA id 8D376208058 for ; Sun, 30 Sep 2012 17:13:14 -0700 (PDT) From: aaron morton Content-Type: multipart/alternative; boundary="Apple-Mail=_E9E4C548-D102-4F1C-B6EA-3B21B643CCC7" Message-Id: <7AE4975C-67BD-4159-9FD7-522580275766@thelastpickle.com> Mime-Version: 1.0 (Mac OS X Mail 6.1 \(1498\)) Subject: Re: Truncate causing subsequent timeout on KeyIterator? Date: Mon, 1 Oct 2012 13:13:14 +1300 References: To: user@cassandra.apache.org In-Reply-To: X-Mailer: Apple Mail (2.1498) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_E9E4C548-D102-4F1C-B6EA-3B21B643CCC7 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 Nothing jumps out.=20 Turn the cassandra logging up to DEBUG and take a look at the cassandra = logs. Your read will be logged in a [ReadStage-*] thread.=20 Hope that helps.=20 ----------------- Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 27/09/2012, at 6:13 AM, Conan Cook wrote: > Hi, >=20 > I'm running a bunch of integration tests using an embedded cassandra = instance via the Cassandra Maven Plugin v1.0.0-1, using Hector v1.0-5. = I've got an issue where one of the tests is using a StringKeyIterator to = iterate over all the keys in a CF, but it gets TimedOutExceptions every = time when trying to communicate with Cassandra; all the other tests = using the same (Spring-wired) keyspace behave fine (stack trace below). = A previous test is calling a cluster.truncate() to ensure an empty CF = before each test, and it's this that seems to cause the problem - at = least, commenting it out causes the other test to run fine. =20 >=20 > Any ideas on what could be causing this? Both tests are using the = same instance of Keyspace, autowired via Spring, and the same instance = of Cluster in the same way. No exceptions are being thrown by the = truncate operation - it completes successfully and does its job. >=20 > Thanks, >=20 >=20 > Conan >=20 > Stack trace: >=20 > [2012-09-26 18:59:53,002] [WARN ] [main] [m.p.c.c.HConnectionManager] = Could not fullfill request on this host = CassandraClient > [2012-09-26 18:59:53,003] [WARN ] [main] [m.p.c.c.HConnectionManager] = Exception: > me.prettyprint.hector.api.exceptions.HTimedOutException: = TimedOutException() > at = me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(Except= ionsTranslatorImpl.java:35) ~[hector-core-1.0-5.jar:na] > at = me.prettyprint.cassandra.service.KeyspaceServiceImpl$3.execute(KeyspaceSer= viceImpl.java:163) ~[hector-core-1.0-5.jar:na] > at = me.prettyprint.cassandra.service.KeyspaceServiceImpl$3.execute(KeyspaceSer= viceImpl.java:145) ~[hector-core-1.0-5.jar:na] > at = me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.j= ava:103) ~[hector-core-1.0-5.jar:na] > at = me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover= (HConnectionManager.java:258) ~[hector-core-1.0-5.jar:na] > at = me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(K= eyspaceServiceImpl.java:131) [hector-core-1.0-5.jar:na] > at = me.prettyprint.cassandra.service.KeyspaceServiceImpl.getRangeSlices(Keyspa= ceServiceImpl.java:167) [hector-core-1.0-5.jar:na] > at = me.prettyprint.cassandra.model.thrift.ThriftRangeSlicesQuery$1.doInKeyspac= e(ThriftRangeSlicesQuery.java:66) [hector-core-1.0-5.jar:na] > at = me.prettyprint.cassandra.model.thrift.ThriftRangeSlicesQuery$1.doInKeyspac= e(ThriftRangeSlicesQuery.java:62) [hector-core-1.0-5.jar:na] > at = me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMe= asure(KeyspaceOperationCallback.java:20) [hector-core-1.0-5.jar:na] > at = me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspa= ce.java:85) [hector-core-1.0-5.jar:na] > at = me.prettyprint.cassandra.model.thrift.ThriftRangeSlicesQuery.execute(Thrif= tRangeSlicesQuery.java:61) [hector-core-1.0-5.jar:na] > at = me.prettyprint.cassandra.service.KeyIterator.runQuery(KeyIterator.java:102= ) [hector-core-1.0-5.jar:na] >=20 > ...... >=20 > Caused by: org.apache.cassandra.thrift.TimedOutException: null > at = org.apache.cassandra.thrift.Cassandra$get_range_slices_result.read(Cassand= ra.java:12270) ~[cassandra-thrift-1.1.0.jar:1.1.0] > at = org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) = ~[libthrift-0.7.0.jar:0.7.0] > at = org.apache.cassandra.thrift.Cassandra$Client.recv_get_range_slices(Cassand= ra.java:683) ~[cassandra-thrift-1.1.0.jar:1.1.0] > at = org.apache.cassandra.thrift.Cassandra$Client.get_range_slices(Cassandra.ja= va:667) ~[cassandra-thrift-1.1.0.jar:1.1.0] > at = me.prettyprint.cassandra.service.KeyspaceServiceImpl$3.execute(KeyspaceSer= viceImpl.java:151) ~[hector-core-1.0-5.jar:na] --Apple-Mail=_E9E4C548-D102-4F1C-B6EA-3B21B643CCC7 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=iso-8859-1
http://www.thelastpickle.com
On 27/09/2012, at 6:13 AM, Conan Cook <conan.cook@amee.com> = wrote:

Hi,

I'm running a bunch of integration = tests using an embedded cassandra instance via the Cassandra Maven = Plugin v1.0.0-1, using Hector v1.0-5.  I've got an issue where one = of the tests is using a StringKeyIterator to iterate over all the keys = in a CF, but it gets TimedOutExceptions every time when trying to = communicate with Cassandra; all the other tests using the same = (Spring-wired) keyspace behave fine (stack trace below).  A = previous test is calling a cluster.truncate() to ensure an empty CF = before each test, and it's this that seems to cause the problem - at = least, commenting it out causes the other test to run fine.  

Any ideas on what could be causing this?  Both = tests are using the same instance of Keyspace, autowired via Spring, and = the same instance of Cluster in the same way.  No exceptions are = being thrown by the truncate operation - it completes successfully and = does its job.
=

Thanks,


Conan<= /div>

Stack = trace:

[2012-09-26 18:59:53,002] [WARN ] = [main] [m.p.c.c.HConnectionManager] Could not fullfill request on this = host CassandraClient<localhost:9170-195>
[2012-09-26 18:59:53,003] [WARN ] [main] = [m.p.c.c.HConnectionManager] = Exception:
me.prettyprint.hector.api.exceptions.HTimedOutExcepti= on: TimedOutException()
        at = me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(Except= ionsTranslatorImpl.java:35) ~[hector-core-1.0-5.jar:na]
        at = me.prettyprint.cassandra.service.KeyspaceServiceImpl$3.execute(KeyspaceSer= viceImpl.java:163) ~[hector-core-1.0-5.jar:na]
    =     at = me.prettyprint.cassandra.service.KeyspaceServiceImpl$3.execute(KeyspaceSer= viceImpl.java:145) ~[hector-core-1.0-5.jar:na]
        at = me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.j= ava:103) ~[hector-core-1.0-5.jar:na]
      =   at = me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover= (HConnectionManager.java:258) ~[hector-core-1.0-5.jar:na]
        at = me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(K= eyspaceServiceImpl.java:131) [hector-core-1.0-5.jar:na]
  =       at = me.prettyprint.cassandra.service.KeyspaceServiceImpl.getRangeSlices(Keyspa= ceServiceImpl.java:167) [hector-core-1.0-5.jar:na]
        at = me.prettyprint.cassandra.model.thrift.ThriftRangeSlicesQuery$1.doInKeyspac= e(ThriftRangeSlicesQuery.java:66) = [hector-core-1.0-5.jar:na]
        at = me.prettyprint.cassandra.model.thrift.ThriftRangeSlicesQuery$1.doInKeyspac= e(ThriftRangeSlicesQuery.java:62) [hector-core-1.0-5.jar:na]
        at = me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMe= asure(KeyspaceOperationCallback.java:20) = [hector-core-1.0-5.jar:na]
        at = me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspa= ce.java:85) [hector-core-1.0-5.jar:na]
        at = me.prettyprint.cassandra.model.thrift.ThriftRangeSlicesQuery.execute(Thrif= tRangeSlicesQuery.java:61) [hector-core-1.0-5.jar:na]
  =       at = me.prettyprint.cassandra.service.KeyIterator.runQuery(KeyIterator.java:102= ) [hector-core-1.0-5.jar:na]
=

...<snip>...

Caused by: org.apache.cassandra.thrift.TimedOutException: = null
        at = org.apache.cassandra.thrift.Cassandra$get_range_slices_result.read(Cassand= ra.java:12270) ~[cassandra-thrift-1.1.0.jar:1.1.0]
        at = org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) = ~[libthrift-0.7.0.jar:0.7.0]
        at = org.apache.cassandra.thrift.Cassandra$Client.recv_get_range_slices(Cassand= ra.java:683) ~[cassandra-thrift-1.1.0.jar:1.1.0]
        at = org.apache.cassandra.thrift.Cassandra$Client.get_range_slices(Cassandra.ja= va:667) ~[cassandra-thrift-1.1.0.jar:1.1.0]
    =     at = me.prettyprint.cassandra.service.KeyspaceServiceImpl$3.execute(KeyspaceSer= viceImpl.java:151) ~[hector-core-1.0-5.jar:na]

= --Apple-Mail=_E9E4C548-D102-4F1C-B6EA-3B21B643CCC7--