Mailing-List: contact cassandra-commits-help@incubator.apache.org;
 run by ezmlm
Precedence: bulk
Reply-To: cassandra-dev@incubator.apache.org
Message-ID: <175816867.1250809694799.JavaMail.jira@brutus>
Date: Thu, 20 Aug 2009 16:08:14 -0700 (PDT)
From: "Phillip Michalak (JIRA)" <jira@apache.org>
To: cassandra-commits@incubator.apache.org
Subject: [jira] Updated: (CASSANDRA-381) Quorum read timeout stemming from
 TcpConnectionManager initialization
In-Reply-To: <698920548.1250719994839.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/CASSANDRA-381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Phillip Michalak updated CASSANDRA-381:
---------------------------------------


I can no longer re-produce this after cleaning the data/ directories for each node and re-starting the cluster. I suspect that I got to an inconsistent state by incrementally adding column-families to the keyspace.

> Quorum read timeout stemming from TcpConnectionManager initialization
> ---------------------------------------------------------------------
>
>                 Key: CASSANDRA-381
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-381
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.4
>         Environment: ubuntu linux (#42-Ubuntu SMP Fri Apr 17 02:48:10 UTC 2009) running in VMWare Fusion virtual machine, Cassandra 0.4-beta1
>            Reporter: Phillip Michalak
>
> A quorum read from the Cassandra-remote tool times out on certain queries, depending on the key and node at which the query is directed. An example of a failed query and its output:
> vadmin@vadmin:~/cassandra$ interface/gen-py/cassandra/Cassandra-remote -h 192.168.133.130:9160 get 'MockElementLibrary' '0401318uuuuruepwdcznr' "ColumnPath('strings', None, 'id')" 2
> /usr/local/lib/python2.6/dist-packages/thrift/Thrift.py:58: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
>   self.message = message
> /usr/local/lib/python2.6/dist-packages/thrift/Thrift.py:99: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
>   self.message = iprot.readString();
> Traceback (most recent call last):
>   File "interface/gen-py/cassandra/Cassandra-remote", line 93, in <module>
>     pp.pprint(client.get(args[0],args[1],eval(args[2]),eval(args[3]),))
>   File "/home/vadmin/cassandra-0.4.0-beta1/interface/gen-py/cassandra/Cassandra.py", line 182, in get
>     return self.recv_get()
>   File "/home/vadmin/cassandra-0.4.0-beta1/interface/gen-py/cassandra/Cassandra.py", line 201, in recv_get
>     raise x
> thrift.Thrift.TApplicationException/usr/local/lib/python2.6/dist-packages/thrift/Thrift.py:76: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
>   if self.message:
> /usr/local/lib/python2.6/dist-packages/thrift/Thrift.py:77: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
>   return self.message
> : Internal error processing get
> The relevant portion of the log file generated from this request:
> DEBUG [pool-1-thread-22] 2009-08-19 16:54:57,618 CassandraServer.java (line 221) get
> DEBUG [pool-1-thread-22] 2009-08-19 16:54:57,618 StorageProxy.java (line 420) strongread reading data for SliceByNamesReadCommand(table='MockElementLibrary', key='0401318uuuuruepwdcznr', columnParent='QueryPath(columnFamilyName='strings', superColumnName='null', columnName='null')', columns=[id,]) from 38184@null
> DEBUG [pool-1-thread-22] 2009-08-19 16:54:57,619 StorageProxy.java (line 427) strongread reading digest for SliceByNamesReadCommand(table='MockElementLibrary', key='0401318uuuuruepwdcznr', columnParent='QueryPath(columnFamilyName='strings', superColumnName='null', columnName='null')', columns=[id,]) from 38185@192.168.133.129:7000
>  WARN [MESSAGE-SERIALIZER-POOL:4] 2009-08-19 16:54:57,619 MessageSerializationTask.java (line 81) Exception was generated at : 08/19/2009 16:54:57 on thread MESSAGE-SERIALIZER-POOL:4
> java.lang.NullPointerException
>         at org.apache.cassandra.net.TcpConnection.<init>(TcpConnection.java:83)
>         at org.apache.cassandra.net.TcpConnectionManager.getConnection(TcpConnectionManager.java:64)
>         at org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:306)
>         at org.apache.cassandra.net.MessageSerializationTask.run(MessageSerializationTask.java:66)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:619)
> DEBUG [RESPONSE-STAGE:4] 2009-08-19 16:54:57,622 ResponseVerbHandler.java (line 38) Processing response on a callback from 65B1E352-A0A3-1A7F-138B-9BEA3E1D787F@192.168.133.129:7000
> ERROR [pool-1-thread-22] 2009-08-19 16:55:02,619 Cassandra.java (line 608) Internal error processing get
> java.lang.RuntimeException: java.util.concurrent.TimeoutException: Operation timed out - received only 1 responses from 192.168.133.129:7000 .
>         at org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:100)
>         at org.apache.cassandra.service.CassandraServer.get(CassandraServer.java:226)
>         at org.apache.cassandra.service.Cassandra$Processor$get.process(Cassandra.java:602)
>         at org.apache.cassandra.service.Cassandra$Processor.process(Cassandra.java:560)
>         at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:252)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:619)
> Caused by: java.util.concurrent.TimeoutException: Operation timed out - received only 1 responses from 192.168.133.129:7000 .
>         at org.apache.cassandra.service.QuorumResponseHandler.get(QuorumResponseHandler.java:86)
>         at org.apache.cassandra.service.StorageProxy.strongRead(StorageProxy.java:435)
>         at org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:330)
>         at org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:92)
>         ... 7 more

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.