Mailing-List: contact cassandra-commits-help@incubator.apache.org;
 run by ezmlm
Precedence: bulk
Reply-To: cassandra-dev@incubator.apache.org
Message-ID: <698920548.1250719994839.JavaMail.jira@brutus>
Date: Wed, 19 Aug 2009 15:13:14 -0700 (PDT)
From: "Phillip Michalak (JIRA)" <jira@apache.org>
To: cassandra-commits@incubator.apache.org
Subject: [jira] Created: (CASSANDRA-381) Quorum read timeout stemming from
 TcpConnectionManager initialization
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Quorum read timeout stemming from TcpConnectionManager initialization
---------------------------------------------------------------------

                 Key: CASSANDRA-381
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-381
             Project: Cassandra
          Issue Type: Bug
          Components: Core
    Affects Versions: 0.4
         Environment: ubuntu linux (#42-Ubuntu SMP Fri Apr 17 02:48:10 UTC 2009) running in VMWare Fusion virtual machine.
            Reporter: Phillip Michalak


A quorum read from the Cassandra-remote tool times out on certain queries, depending on the key and node at which the query is directed. An example of a failed query and its output:

vadmin@vadmin:~/cassandra$ interface/gen-py/cassandra/Cassandra-remote -h 192.168.133.130:9160 get 'MockElementLibrary' '0401318uuuuruepwdcznr' "ColumnPath('strings', None, 'id')" 2
/usr/local/lib/python2.6/dist-packages/thrift/Thrift.py:58: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
  self.message = message
/usr/local/lib/python2.6/dist-packages/thrift/Thrift.py:99: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
  self.message = iprot.readString();
Traceback (most recent call last):
  File "interface/gen-py/cassandra/Cassandra-remote", line 93, in <module>
    pp.pprint(client.get(args[0],args[1],eval(args[2]),eval(args[3]),))
  File "/home/vadmin/cassandra-0.4.0-beta1/interface/gen-py/cassandra/Cassandra.py", line 182, in get
    return self.recv_get()
  File "/home/vadmin/cassandra-0.4.0-beta1/interface/gen-py/cassandra/Cassandra.py", line 201, in recv_get
    raise x
thrift.Thrift.TApplicationException/usr/local/lib/python2.6/dist-packages/thrift/Thrift.py:76: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
  if self.message:
/usr/local/lib/python2.6/dist-packages/thrift/Thrift.py:77: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
  return self.message
: Internal error processing get

The relevant portion of the log file generated from this request:

DEBUG [pool-1-thread-22] 2009-08-19 16:54:57,618 CassandraServer.java (line 221) get
DEBUG [pool-1-thread-22] 2009-08-19 16:54:57,618 StorageProxy.java (line 420) strongread reading data for SliceByNamesReadCommand(table='MockElementLibrary', key='0401318uuuuruepwdcznr', columnParent='QueryPath(columnFamilyName='strings', superColumnName='null', columnName='null')', columns=[id,]) from 38184@null
DEBUG [pool-1-thread-22] 2009-08-19 16:54:57,619 StorageProxy.java (line 427) strongread reading digest for SliceByNamesReadCommand(table='MockElementLibrary', key='0401318uuuuruepwdcznr', columnParent='QueryPath(columnFamilyName='strings', superColumnName='null', columnName='null')', columns=[id,]) from 38185@192.168.133.129:7000
 WARN [MESSAGE-SERIALIZER-POOL:4] 2009-08-19 16:54:57,619 MessageSerializationTask.java (line 81) Exception was generated at : 08/19/2009 16:54:57 on thread MESSAGE-SERIALIZER-POOL:4

java.lang.NullPointerException
        at org.apache.cassandra.net.TcpConnection.<init>(TcpConnection.java:83)
        at org.apache.cassandra.net.TcpConnectionManager.getConnection(TcpConnectionManager.java:64)
        at org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:306)
        at org.apache.cassandra.net.MessageSerializationTask.run(MessageSerializationTask.java:66)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)

DEBUG [RESPONSE-STAGE:4] 2009-08-19 16:54:57,622 ResponseVerbHandler.java (line 38) Processing response on a callback from 65B1E352-A0A3-1A7F-138B-9BEA3E1D787F@192.168.133.129:7000
ERROR [pool-1-thread-22] 2009-08-19 16:55:02,619 Cassandra.java (line 608) Internal error processing get
java.lang.RuntimeException: java.util.concurrent.TimeoutException: Operation timed out - received only 1 responses from 192.168.133.129:7000 .
        at org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:100)
        at org.apache.cassandra.service.CassandraServer.get(CassandraServer.java:226)
        at org.apache.cassandra.service.Cassandra$Processor$get.process(Cassandra.java:602)
        at org.apache.cassandra.service.Cassandra$Processor.process(Cassandra.java:560)
        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:252)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)
Caused by: java.util.concurrent.TimeoutException: Operation timed out - received only 1 responses from 192.168.133.129:7000 .
        at org.apache.cassandra.service.QuorumResponseHandler.get(QuorumResponseHandler.java:86)
        at org.apache.cassandra.service.StorageProxy.strongRead(StorageProxy.java:435)
        at org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:330)
        at org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:92)
        ... 7 more

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.