Nothing jumps out. 

Turn the cassandra logging up to DEBUG and take a look at the cassandra logs. Your read will be logged in a [ReadStage-*] thread. 

Hope that helps. 


-----------------
Aaron Morton
Freelance Developer
@aaronmorton

On 27/09/2012, at 6:13 AM, Conan Cook <conan.cook@amee.com> wrote:

Hi,

I'm running a bunch of integration tests using an embedded cassandra instance via the Cassandra Maven Plugin v1.0.0-1, using Hector v1.0-5.  I've got an issue where one of the tests is using a StringKeyIterator to iterate over all the keys in a CF, but it gets TimedOutExceptions every time when trying to communicate with Cassandra; all the other tests using the same (Spring-wired) keyspace behave fine (stack trace below).  A previous test is calling a cluster.truncate() to ensure an empty CF before each test, and it's this that seems to cause the problem - at least, commenting it out causes the other test to run fine.  

Any ideas on what could be causing this?  Both tests are using the same instance of Keyspace, autowired via Spring, and the same instance of Cluster in the same way.  No exceptions are being thrown by the truncate operation - it completes successfully and does its job.

Thanks,


Conan

Stack trace:

[2012-09-26 18:59:53,002] [WARN ] [main] [m.p.c.c.HConnectionManager] Could not fullfill request on this host CassandraClient<localhost:9170-195>
[2012-09-26 18:59:53,003] [WARN ] [main] [m.p.c.c.HConnectionManager] Exception:
me.prettyprint.hector.api.exceptions.HTimedOutException: TimedOutException()
        at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:35) ~[hector-core-1.0-5.jar:na]
        at me.prettyprint.cassandra.service.KeyspaceServiceImpl$3.execute(KeyspaceServiceImpl.java:163) ~[hector-core-1.0-5.jar:na]
        at me.prettyprint.cassandra.service.KeyspaceServiceImpl$3.execute(KeyspaceServiceImpl.java:145) ~[hector-core-1.0-5.jar:na]
        at me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103) ~[hector-core-1.0-5.jar:na]
        at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:258) ~[hector-core-1.0-5.jar:na]
        at me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:131) [hector-core-1.0-5.jar:na]
        at me.prettyprint.cassandra.service.KeyspaceServiceImpl.getRangeSlices(KeyspaceServiceImpl.java:167) [hector-core-1.0-5.jar:na]
        at me.prettyprint.cassandra.model.thrift.ThriftRangeSlicesQuery$1.doInKeyspace(ThriftRangeSlicesQuery.java:66) [hector-core-1.0-5.jar:na]
        at me.prettyprint.cassandra.model.thrift.ThriftRangeSlicesQuery$1.doInKeyspace(ThriftRangeSlicesQuery.java:62) [hector-core-1.0-5.jar:na]
        at me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20) [hector-core-1.0-5.jar:na]
        at me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:85) [hector-core-1.0-5.jar:na]
        at me.prettyprint.cassandra.model.thrift.ThriftRangeSlicesQuery.execute(ThriftRangeSlicesQuery.java:61) [hector-core-1.0-5.jar:na]
        at me.prettyprint.cassandra.service.KeyIterator.runQuery(KeyIterator.java:102) [hector-core-1.0-5.jar:na]

...<snip>...

Caused by: org.apache.cassandra.thrift.TimedOutException: null
        at org.apache.cassandra.thrift.Cassandra$get_range_slices_result.read(Cassandra.java:12270) ~[cassandra-thrift-1.1.0.jar:1.1.0]
        at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) ~[libthrift-0.7.0.jar:0.7.0]
        at org.apache.cassandra.thrift.Cassandra$Client.recv_get_range_slices(Cassandra.java:683) ~[cassandra-thrift-1.1.0.jar:1.1.0]
        at org.apache.cassandra.thrift.Cassandra$Client.get_range_slices(Cassandra.java:667) ~[cassandra-thrift-1.1.0.jar:1.1.0]
        at me.prettyprint.cassandra.service.KeyspaceServiceImpl$3.execute(KeyspaceServiceImpl.java:151) ~[hector-core-1.0-5.jar:na]