incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Ellis <jbel...@gmail.com>
Subject Re: Cassandra restart
Date Wed, 14 Oct 2009 13:46:14 GMT
Turn on debug logging and see what nodes it is sending requests to,
and what happens on that node to that request.

Maybe there is an exception on the data node.

2009/10/14 Teodor Sigaev <teodor@sigaev.ru>:
>> Why is it timing out?
>
> Sorry, I can't say why, I just see on client side
> {
>                 'code' => 0,
>                 'message' => 'TSocket: timed out reading 4 bytes from
> 10.3.2.25:9160'
>               }, 'Thrift::TException'
>
> and it's repeatable for this key, for another key it could be ok.
> And on server side:
>
>>> ERROR [pool-1-thread-46] 2009-10-14 17:06:40,804 Cassandra.java (line
>>> 679)
>>> Internal error processing get_slice
>>> java.lang.RuntimeException: error reading key 00000000000028344851
>>>       at
>>>
>>> org.apache.cassandra.service.StorageProxy.weakReadRemote(StorageProxy.java:265)
>>>       at
>>>
>>> org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:312)
>>>       at
>>>
>>> org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:100)
>>>       at
>>>
>>> org.apache.cassandra.service.CassandraServer.getSlice(CassandraServer.java:182)
>>>       at
>>>
>>> org.apache.cassandra.service.CassandraServer.multigetSliceInternal(CassandraServer.java:251)
>>>       at
>>>
>>> org.apache.cassandra.service.CassandraServer.get_slice(CassandraServer.java:220)
>>>       at
>>>
>>> org.apache.cassandra.service.Cassandra$Processor$get_slice.process(Cassandra.java:671)
>>>       at
>>>
>>> org.apache.cassandra.service.Cassandra$Processor.process(Cassandra.java:627)
>>>       at
>>>
>>> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:253)
>>>       at
>>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>>>       at
>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>>>       at java.lang.Thread.run(Thread.java:636)
>>> Caused by: java.util.concurrent.TimeoutException: Operation timed out.
>>>       at org.apache.cassandra.net.AsyncResult.get(AsyncResult.java:97)
>>>       at
>>>
>>> org.apache.cassandra.service.StorageProxy.weakReadRemote(StorageProxy.java:261)
>>>       ... 11 more
>
> --
> Teodor Sigaev                                   E-mail: teodor@sigaev.ru
>                                                   WWW: http://www.sigaev.ru/
>

Mime
View raw message