incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandeep Kalidindi at" <>
Subject Re: non blocking Cassandra with Tornado
Date Tue, 27 Jul 2010 13:45:46 GMT
@aaron - thanks a lot. i will test it. This is very much needed.


On Tue, Jul 27, 2010 at 6:03 PM, aaron morton <>wrote:

> Today I worked out how to make non blocking calls to Cassandra inside of
> the non blocking Tornado web server ( using
> Python. I thought I'd share it here and see if anyone thinks I'm abusing
> Thrift too much and inviting trouble.
> It's a bit mucky and I have not tested it for things like timeouts and
> errors. But here goes...
> The idea is rather than calling a cassandra client function like
> get_slice(), call the send_get_slice() then have a non blocking wait on the
> socket thrift is using, then call recv_get_slice().
> So the steps in Tornado are:
> 1.  Web handler creates an object from the model, calls a function on it
> like start_read() to populate it.
> 2. model.start_read() needs to call get_slice() on the thrift generated
> Cassandra client. Instead it calls send_get_slice() and returns to the
> calling web handler.
> 3  Web Handler then asks Tornado to epoll wait for any activity on the
> thrift socket. It gets access to the socket file descriptor by following
> this chain from the thrift generated Cassandra client
> _iprot.trans.__TTransportBase__trans.handle.fileno()
> 4,  Web handler function called in 1 above returns, Tornado keeps the http
> connection alive and the web handler instance alive. Later when the socket
> has activity Tornado will call back into the web handler.
> 5. To get the result of the call to cassandra the Web Handler calls a
> function on the model such as finish_read(). finish_read() wants to get the
> results of the get_slice() and do something, so it calls recv_get_slice on
> the thrift Cassandra client. Processes the result and returns to the web
> handler.
> This looks like the same process the transport in the thrift
> package is using. Except it's not using the nasty reference to get to the
> raw socket.
> I'm not sure about any adverse affects on the Cassandra server from the
> client not servicing the socket immediately when it starts sending data
> back. I'm guessing there are some buffers there, but not sure. Could I be
> accidentally blocking / hurting the cassandra server ?
> Thanks
> Aaron

View raw message