cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sylvain Lebresne (JIRA)" <>
Subject [jira] [Updated] (CASSANDRA-2478) Custom CQL protocol/transport
Date Wed, 16 May 2012 15:23:13 GMT


Sylvain Lebresne updated CASSANDRA-2478:

    Attachment: cql_binary_protocol

Attaching cql_binary_protocol as a draft for such a custom binary protocol (CQL3 only). The
protocol follows more or less Rick's outline above. It is frame based, each frame has a small
header indicating amongst other the opcode for the message that defines what the body of said
message must contain.

A typical communication start by a small handshake/authentication phase, after which queries
(and preparation/execution) can be performed. The protocol support a form of 'cursor' api
for queries. Given a select query, the client can ask the server to return only a handful
of rows first, and then the client can fetch more rows at his own rate using a NEXT message.

Outside of the cursor thing, the protocol as described here pretty much expose the same things
than the thrift transport (as far as CQL is concerned) but not much more (a small exception
is that CASSANDRA-3707 is included). I plan on experimenting next with a few additional features,
like allowing clients to register to events like 'a new node joined' and be notified when
such event happen, but I'll leave that to follow up tickets.

I've push at an initial implementation of
this protocol (using netty). Almost all of the protocol is implemented except for the NEXT
message, but I'll get to that.  There is currently mainly 4 patches:
* the first one is the bulk of the new server
* the second one is a simple client, along with a small "debug" console (whose code is ugly),
that allow to send message to test the server. This does not necessarily have to make it in
the final commit, but it is useful for testing.
* the third one replace the use of CqlResult in the CQL3 code to directly build the new messages.
And for the thrift interface, it simply translate those message to CqlResult. I've done it
that way (instead of generating CqlResult and convert that to messages of the native protocol)
because I think that is the direction we want to go. However currently that means that there
is a few info that don't make it anymore in the CqlResult, namely the timestamp of the columns.
Anywa, imo CASSANDRA-4217 is a much better way to access the timestamp and I'm not sure existing
client were exposing the timestamp, but if there is complaints, that can be fixe is a much
better way to access the timestamp and I'm not sure existing client were exposing the timestamp,
but if there is complaints, that can be fixed.
* the last one changes our CassandraDaemon business so that we can run a server with both
the thrift and native protocol server running cleanly.

Other than that, I have really benchmarked this (but that should be done). I meant to update
stress to use the new server but realized that stress doesn't work for CQL3 at all, so that
will be a separate ticket probably.

> Custom CQL protocol/transport
> -----------------------------
>                 Key: CASSANDRA-2478
>                 URL:
>             Project: Cassandra
>          Issue Type: New Feature
>          Components: API, Core
>            Reporter: Eric Evans
>            Assignee: Sylvain Lebresne
>            Priority: Minor
>              Labels: cql
>         Attachments: cql_binary_protocol
> A custom wire protocol would give us the flexibility to optimize for our specific use-cases,
and eliminate a troublesome dependency (I'm referring to Thrift, but none of the others would
be significantly better).  Additionally, RPC is bad fit here, and we'd do better to move in
the direction of something that natively supports streaming.
> I don't think this is as daunting as it might seem initially.  Utilizing an existing
server framework like Netty, combined with some copy-and-paste of bits from other FLOSS projects
would probably get us 80% of the way there.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message