Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Mon, 10 Oct 2011 08:40:30 +0000 (UTC)
From: "Ivo Ladage-van Doorn (Created) (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: 
 <1687965929.14023.1318236030048.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Created] (CASSANDRA-3339) Invalid queries in
 Cassandra.Client causes subsequent, valid, queries to fail
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Invalid queries in Cassandra.Client causes subsequent, valid, queries to fail
-----------------------------------------------------------------------------

                 Key: CASSANDRA-3339
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3339
             Project: Cassandra
          Issue Type: Bug
          Components: API
    Affects Versions: 0.8.6
         Environment: Windows
            Reporter: Ivo Ladage-van Doorn


First of all; I'm quite new to Cassandra, so I hope that my analysis is correct. 
I am using the Hector client to perform queries on Cassandra. The problem is that once I invoked an invalid slice query with a null rowKey, subsequent queries also fail with roughly the same error. So the first time I invoke the invalid query, I get this exception:

org.apache.thrift.protocol.TProtocolException: Required field 'key' was not present! 
Struct: get_slice_args(key:null, column_parent:ColumnParent(column_family:AmdatuToken), predicate:SlicePredicate(slice_range:SliceRange(start:, finish:, reversed:false, count:1000000)), consistency_level:ONE)
       at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:56)
       at me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:285)
       at me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:268)
       ...

which is expected behavior. However, after invoking this invalid query, subsequent valid calls also fail with roughly the same error:

me.prettyprint.hector.api.exceptions.HCassandraInternalException: Cassandra encountered an internal error processing this request: TApplicationError type: 7 message:Required field 'key' was not present! Struct: get_slice_args(key:null, column_parent:null, predicate:null, consistency_level:ONE) 
org.apache.felix.log.LogException: me.prettyprint.hector.api.exceptions.HCassandraInternalException: 
Cassandra encountered an internal error processing this request: TApplicationError type: 7 message:Required field 'key' was not present! Struct: get_slice_args(key:null, column_parent:null, predicate:null, consistency_level:ONE)
        at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:29)
        at me.prettyprint.cassandra.service.KeyspaceServiceImpl$2.execute(KeyspaceServiceImpl.java:121)
        at me.prettyprint.cassandra.service.KeyspaceServiceImpl$2.execute(KeyspaceServiceImpl.java:114)
        ...

In the case of Hector it goes downhill from there, ending in socket write errors and marking the Cassandra host as being down.

Now this is what I think happens:
The Hector client uses the org.apache.cassandra.thrift.Cassandra.Client class to execute the queries on Cassandra.
When I perform a Thrift slice query from Hector, it invokes the get_slice method in the Cassandra.Client, which in its turn invokes send_get_slice. In my case a bug in my own software caused an invocation of this method with a rowKey that equals null. Now although the rowKey is invalid, the method call continues all the way to send_get_slice. This send_get_slice method looks like this:

(from org.apache.cassandra.thrift.Cassandra)
public void send_get_slice(ByteBuffer key, ColumnParent column_parent, SlicePredicate predicate, ConsistencyLevel consistency_level) throws org.apache.thrift.TException
{
  oprot_.writeMessageBegin(new org.apache.thrift.protocol.TMessage("get_slice", org.apache.thrift.protocol.TMessageType.CALL, ++seqid_));
  get_slice_args args = new get_slice_args();
  args.setKey(key);
  args.setColumn_parent(column_parent);
  args.setPredicate(predicate);
  args.setConsistency_level(consistency_level);
  args.write(oprot_);
  oprot_.writeMessageEnd();
  oprot_.getTransport().flush();
}

The problem is that the TMessage is written to the output protocol at the first line. When subsequently the arguments are written in args.write(oprot_), it first calls validate(). The validate() method detects the null rowKey and throws an exception:

public void validate() throws org.apache.thrift.TException {
  // check for required fields
  if (key == null) {
    throw new org.apache.thrift.protocol.TProtocolException("Required field 'key' was not present! Struct: " + toString());
  }
  ...
}
    
Now Hector finally catches the exception and returns the Cassandra client to its connection pool. When that Cassandra client is drawn from the pool and reused for another query, it still has the TMessage written to its oprot, resulting in another failed call. Call reusing this client from the connection pool keep getting such errors, and in the end result in socket write errors.

The problem is not only restricted to this particular method by the way, the same construct is used in many more methods in this class. Also passing a null value for columnsPath or ConsistencyLevel seems to cause the same issue.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira